Subject: Re: increasing NMBCLUSTERS
To: Chuck Silvers <chuq@chuq.com>
From: Manuel Bouyer <bouyer@antioche.eu.org>
List: tech-net
Date: 09/29/2005 16:45:42
On Thu, Sep 29, 2005 at 07:25:44AM -0700, Chuck Silvers wrote:
> On Wed, Sep 28, 2005 at 11:43:16PM +0200, Manuel Bouyer wrote:
> > On Wed, Sep 28, 2005 at 02:16:12PM -0600, Herb Peyerl wrote:
> > > [...]
> > > 714 mbufs in use:
> > >         146 mbufs allocated to data
> > >         567 mbufs allocated to packet headers
> > >         1 mbufs allocated to socket names and addresses
> > > 25619 calls to protocol drain routines
> > > 
> > > [...]
> > > Memory resource pool statistics
> > > Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg  
> > > Maxpg Idle
> > > mclpl       2048   183151 22639  117629 40643  7882 32761 32761     4  
> > > 32768    0
> > 
> > Hum
> > 183151 - 117629 = 65522, so this really makes 32761 pages.
> > Looks like a leak somewhere, because it seems these clusters are not
> > attached to mbufs (there isn't 65522 mbufs allocated).
> 
> I was just looking at a similar problem on john klos's cobalt box last night:
> 
> Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
> mbpl         256  2030774    0  2013827  2924  1691  1233  1321     1   inf    0
> mclpl       2048   429969 5987311 413585 40542 32346 8196  8196     4  8192    4
> 
> # netstat -m
> 16540 mbufs in use:
>         16538 mbufs allocated to data
>         2 mbufs allocated to packet headers
> 6001641 calls to protocol drain routines

This time the number of mclpl used matches the number of mbufs:
429969 - 413585 = 16384 (this more or less matches the 16540 mbufs in use).

> 
> definitely a leak somewhere.  this case was on 2.1_RC3.
> we have a dump if anyone wants to look at it.

Can you do a netstat -a on this dump, and look at the sockets send and receive
queues ? It's also possibles that these mbufs are used for socket queues
that have problems to drain. I've seen this happens on web servers talking
with clients behind a broken DSL setup (pmtu blackhole).

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--