Re: Filesystem limits

[der Mouse <mouse%Rodents-Montreal.ORG@localhost>]

>>> Is there a portable way of finding out the limits for the maximum
>>> number of files and directories that can be created under a give
>>> directory?
>> I doubt it.  Traditional Unix has no such limits,

Doh.  Your example points out one I didn't think of, perhaps because I
was thinking of cases where files and directories are equivalent.
There is indeed a relatively small limit on the number of
subdirectories a given directory may have in FFS (and close relatives
such as ext2fs and lfs).

> I can't create more than 32766 subdirectories in a given directory.
> If I run a test program, which creates subdirectories (1, 2, 3...N)
> in a loop, after 32766 it exits with:

> p3smp$ ./a.out
> mkdir() error: : Too many links
> i = 32766

> Where is this "Too many links" coming from?  Is it a filesystem
> limit, or a dynamically adjustable sysctl limit?

You are probably using FFS (or a close relative).  Assuming that's so,
this is an architectural limit in the filesystem.  You can create files
or other non-directory entities all you like,  but subdirectories
increase the link count of the parent directory, and there is a 16-bit
limit on the link count - a signed 16-bit limit, it would appear (which
surprises me; I would have expected it to be unsigned).  Your test fell
over with #32766 rather than #32768 because . and .. mean the base link
count is 2.

> Is there a way to increase this limit, or maybe it will substantially
> decrease performance when doing directory lookups?

Again, assuming FFS, increasing that limit means changing the on-disk
inode format (well, increasing it substantially; merely converting it
from signed 16-bit to unsigned 16-bit would probably be fairly easy,
but would gain you only about a factor of 2).  There is spare space in
the on-disk inode structure to make di_nlink larger, but it would be a
flag day - the code after the change would be unable to handle
filesystems created before it, and vice versa.  This means it is, umm,
unlikely to happen in the main tree, but I suspect it would be fairly
easy to do as a local change (turn the current di_nlink into a padding
field and steal one of the spare fields for a wider di_nlink - based on
a few quick greps and eyeball scans, it'd be tedious but not
particularly difficult to find and fix all the dependencies).

Arguably it wokuld be better to clone the necessary fractions of ffs
and create a variant, hlcffs let us say, so that you could continue to
use existing filesystems.  It would, however, be more work, be of
relatively limited utility (few people need that many subdirs), and I
suspect would thus also be unlikely to go in.

Addressing your original desire, there is <sys/syslimits.h>, which
includes a LINK_MAX #define.  I saw this used in the FFS code several
places, but (because the data types are signed 16-bit and LINK_MAX is
already 32767)), you can't just raise LINK_MAX and rebuild to get a
higher limit.

Surprisingly (to me), even ufs2 seems to have stuck with 16 bits for
di_nlink.

And this is rapidly veering away from the tech-userlevel viewpoint.

/~\ The ASCII                             Mouse
\ / Ribbon Campaign
 X  Against HTML                mouse%rodents-montreal.org@localhost
/ \ Email!           7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B