Subject: Re: Limitations of current buffer cache on 32-bit ports
To: Thor Lancelot Simon <tls@rek.tjls.com>
From: Chuck Silvers <chuq@chuq.com>
List: tech-kern
Date: 07/23/2002 00:03:57
hi,

yes, there's more work to be done with file system caching.

I've been talking about moving directory and symlink data to the page
cache instead of the buffer cache for quite a while now, but I haven't
had a chance to work on it yet.  I haven't thought much about exactly
how to do this, but the interactions with the softdep code will no doubt
be interesting.

another change we should make is to redesign the buffer cache (which
will still be needed for things like indirect blocks and FFS cylinder
groups) to not need so much KVA.  the simplest and least intrusive way
of doing this is probably to have the buffer cache not use a fixed
amount of virtual space for every buffer, but instead use only as much
virtual space as each buffer really needs.  if done right, this could
make the first change much easier, now that I think about it.  we'll
likely want a better allocator than we have now for managing the buffer
cache virtual space.

the BSD 4.4 buffer cache is the only one I know of that has a fixed size
for all buffers, that's why this isn't so much of a problem in other
systems.

-Chuck


On Tue, Jul 16, 2002 at 12:13:53PM -0400, Thor Lancelot Simon wrote:
> While building the new AnonCVS server, we noticed a few minor nits which
> impact 32-bit machines with very large memories (e.g. our 3.5GB x86 anoncvs
> box).  Probably the most significant was that with a very large page cache
> and a sufficient number of vnode cache entries, we still do some entirely
> unnecessary disk I/O because of limitations of the traditional buffer cache,
> which is used for filesystem metadata (e.g. directories).
...
> Basically, ISTM there are two problems.  1) The use of fixed-size buffers
> allocated in kernel space for all metadata.  2) Filesystem limitations
> that force that fixed size to be quite large.
> 
> How do other systems which retain a "traditional" buffer cache for
> metadata deal with this issue?