Subject: Re: observations on a large-memory system.
To: Thor Lancelot Simon <tls@rek.tjls.com>
From: Chuck Silvers <chuq@chuq.com>
List: tech-kern
Date: 06/18/2001 23:45:32
On Mon, Jun 18, 2001 at 04:57:21PM -0400, Thor Lancelot Simon wrote:
> On Mon, Jun 18, 2001 at 10:00:18AM -0700, Chuck Silvers wrote:
> > On Fri, Jun 08, 2001 at 10:45:54AM -0400, Bill Sommerfeld wrote:
> > > The following is with -current..
> > > 
> > > 1) ubc needs many vnodes to effectively use physmem for vnode page
> > > cache.  For the NetBSD source tree, you need roughly 67000 vnodes to
> > > cache roughly 670k of data.
> > > 
> > > A 2G system autoconfigures with roughly 40000 vnodes, where the above
> > > ratio suggests that somewhere around 200000 vnodes might be a more
> > > appropriate size.
> > 
> > yes, we definitely need to adjust the tuning curves for various resources.
> > I've been increasing the number of vnodes via /etc/sysctl.conf on my machines
> > for a while now, but it would be nice to get more reasonable settings
> > by default.
> 
> Has the memory consumption per vnode dropped with UBC, or are the vnode
> structures themselves no longer in wired kernel memory?  200,000 vnodes
> used to be entirely impractical because it consumed a massive amount of
> wired KVA space.


vnodes are still in wired kernel memory.  200000 vnodes used to be
impractical more because configuring even 1 buffer for each of 200000
vnodes would require 12.8 GB of kernel virtual space because of how
buffers use 64KB of virtual space each regardless of the amount of
physical memory per buffer (usually 8KB).

with UBC, the amount of virtual space used to access cached file data is
decoupled from the amount of physical memory use to hold the file data,
so you can easily use 1.5 GB of the 2GB in bill's machine to store
file data without running out of kernel virtual space.

-Chuck