Hi!

On Mon, 15 Nov 2004, Thor Lancelot Simon wrote:

> I am curious as to _how_ bufpages got to be so high.  Do you have,
> perhaps, a huge number of directories on a very-large-block filesystem?
> Or are you accessing the buffer cache through a block device node (this
> is a pretty bad idea)?

Could dump(8) be doing this? I've ruled out log rotation at
midnight (bufmem grows sometime between 00:05 and 08:00 local time,
and about the only thing taking place at that time is Amanda's backup
run, which would cause lots of fs accesses). I've already set
monitoring to see if the growth correlates with dump runs.
(No, squid's disk cache is not backed up).

> The buffer cache growth algorithm is _extremely_ conservative.  Once it
> gets to bufmem_hiwater, it should _always_ recycle an existing buffer
> rather than allocating a new one.  The algorithm is from an old lecture

Statistically a rather limited observation (based on less than a dozen
samples): vm.bufmem seems to exceed hiwater mark by roughly 100%, ie.
if hiwater mark is set at X, then highest vm.bufmem I've seen has been
a bit over 2*X.

Is there a better way to monitor bufmem growth than maybe adding
a couple of counters into uvmexp structure, and then incrementing those
(and adding them to systat or some such for monitoring)? Ideally,
I would like to know each buffer allocation (size) and each buffer
resize (size change) with time stamps, but I doubt if I could get those
into any file from within kern/vfs_bio.c ? Before I start messing with
my kernel, I'd like to know that the way I try to get data out from it
actually works without causing too much overhead.

It would be nice to know whether the bufmem growth is a result of a few
alloc/resize, or a result of several resizes, and the statistics involved
(in my case). It would also be nice to know who makes the calls that
cause the biggest growth (and why, if not explained by 'who').


Artsi
-- 
#######======------  http://www.selonen.org/arto/  --------========########
Everstinkuja 5 B 35                               Don't mind doing it.
FIN-02600 Espoo        arto@selonen.org         Don't mind not doing it.
Finland              tel +358 50 560 4826     Don't know anything about it.