Subject: Re: "too many files" in an FFS dir?
To: Paul A Vixie <vixie@mfnx.net>
From: Andrew Brown <atatat@atatdot.net>
List: tech-kern
Date: 12/19/2000 01:03:42
>> Is there any good reason why I should not plan to store all of the
>> [tens of thousands of] files in one directory?
>
>used to be, yes.  at this point it's merely an urban legend.

well...i'd have to prove that to myself to be certain.

>> What I know about berkeley ffs suggests not, but it worries me that
>> squid (the http proxy) goes to a great deal of trouble to store small
>> (less than 1000 entries) numbers of cached documents in one directory.
>
>squid has to work well on systems without softdep.  since MAPS and most
>other "ticket systems" need to store a large number of arbitrary-sized
>text objects and then access them randomly, i did some measurements.
>
>http://www.vix.com/~vixie/fbsd-flat.png shows a freebsd 4.2 system putting
>~75K files (the MAPS RSS if you must know) into a single directory and
>then accessing them in a different order (to prevent implicit double
>buffering).
>
>http://www.vix.com/~vixie/bsdi-flat.png is exactly the same test run on a
>bsd/os 3.1 system.  as you can see, there's a remarkable mode difference.
>
>the difference comes down to soft dependencies.  if someone would like to
>run these tests on netbsd, i'll make my test tools available.

i'm just curious...would it be trivial to modify these tools to use a
hierarchical arrangement of files a directories as well (eg, instead
of opening file "1234567", one would open "12/34/567")?  i'd be
curious to see the difference there, since that was the usual attack
against the problem.

-- 
|-----< "CODE WARRIOR" >-----|
codewarrior@daemon.org             * "ah!  i see you have the internet
twofsonet@graffiti.com (Andrew Brown)                that goes *ping*!"
andrew@crossbar.com       * "information is power -- share the wealth."