Subject: buffer cache tuning - running out of kernel virtual space
To: None <tech-kern@netbsd.org>
From: Chuck Silvers <chuq@chuq.com>
List: tech-kern
Date: 12/10/2005 16:50:06
while yamt and I were looking into problems with the project autobuild
machines, he noticed that VM_MAX_KERNEL_BUF no longer does anything,
and the default tuning of the buffer cache is much higher than the limit
that (eg.) the x86 code wants to set.  in configurations with a lot more
physical memory than kernel virtual space, this can result in the buffer
cache exhausting kernel's virtual space and the system hanging (which is
what was happening on the autobuild machines).

I see that a mechanism already exists to limit virtual space usage by
the buffer cache, buf_setvalimit(), which is used by alpha, mips, sparc
and vax.  however, it seems to me that the usage for all but sparc is wrong
since it sets the limit to whatever the limit would have been anyway:

	bufsz = buf_memcalc();
	buf_setvalimit(bufsz);

or maybe this is just supposed to keep people from tuning the limit higher
than its original value.  I'm not sure why that would be desirable though.
does anyone know why these are here?  if not, I'll remove them.  (the sparc
instance actually imposes a smaller limit, so I'll leave that one.)

I think that it would be good to support a tunable similar to "BUFCACHE"
(which specifies a percentage of physical memory to be used for buffer cache),
but using a percentage of kernel virtual space instead.  this could have
a default of maybe 40, could be overridable by the kernel config file.
we probably need to allow for a port-specific default, so that this general
mechanism can replace the sparc-specific thing I mentioned above.

does anyone have any other suggestions on this issue?

this fix is needed for 3.0, since the project's own servers have already
had trouble with the issue.  I'll post some diffs probably tomorrow.

-Chuck