Subject: Re: "too many files" in an FFS dir?
To: None <tech-kern@netbsd.org>
From: Paul A Vixie <vixie@mfnx.net>
List: tech-kern
Date: 12/16/2000 11:01:20
> Is there any good reason why I should not plan to store all of the
> [tens of thousands of] files in one directory?

used to be, yes.  at this point it's merely an urban legend.

> What I know about berkeley ffs suggests not, but it worries me that
> squid (the http proxy) goes to a great deal of trouble to store small
> (less than 1000 entries) numbers of cached documents in one directory.

squid has to work well on systems without softdep.  since MAPS and most
other "ticket systems" need to store a large number of arbitrary-sized
text objects and then access them randomly, i did some measurements.

http://www.vix.com/~vixie/fbsd-flat.png shows a freebsd 4.2 system putting
~75K files (the MAPS RSS if you must know) into a single directory and
then accessing them in a different order (to prevent implicit double
buffering).

http://www.vix.com/~vixie/bsdi-flat.png is exactly the same test run on a
bsd/os 3.1 system.  as you can see, there's a remarkable mode difference.

the difference comes down to soft dependencies.  if someone would like to
run these tests on netbsd, i'll make my test tools available.