Subject: Re: increasing NMBCLUSTERS
To: None <tech-net@netbsd.org>
From: Jan Schaumann <jschauma@netmeister.org>
List: tech-net
Date: 11/12/2005 13:24:41
--RE3pQJLXZi4fr8Xo
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Jan Schaumann <jschauma@netmeister.org> wrote:
> Manuel Bouyer <bouyer@antioche.eu.org> wrote:
> > On Wed, Sep 28, 2005 at 01:10:31PM -0400, Jan Schaumann wrote:
> =20
> > > WARNING: mclpool limit reached; increase NMBCLUSTERS
> > >=20
> > > I've already bumped NMBCLUSTERS twice, up to 65536 now.  options(4) is
> > > not very exhaustive of what the side effects (if any) may be of
> > > increasing this number.  How far can I bump this number?  What is
> > > affected by it?
> >=20
> > Did you see what could be using that much clusters ?
>=20
> Well, right now, after a fresh reboot, netstat -m gives me:
>=20
> 803 mbufs in use:
>         772 mbufs allocated to data
>         16 mbufs allocated to packet headers
>         15 mbufs allocated to socket names and addresses
> 0 calls to protocol drain routines

I still am seeing this problem, and my machine locks up after a couple
of days even after continuing to increase NMBCLUSTERS.  Looking at the
output of vmstat and netstat, I notice:

$ vmstat -m | egrep "(Name|mbpl|mclpl)"
Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg=
 Idle
mbpl         256     2550    0        0   161     0   161   161     1 inf  =
  1
mclpl       2048     2523    0        0  1266     0  1266  1266     4 13107=
2 4
$ netstat -m                              =20
810 mbufs in use:
        809 mbufs allocated to data
        1 mbufs allocated to packet headers
0 calls to protocol drain routines


It seems odd to me that I don't have any Releases, 0 calls to protocol
drain routines.  However, if I perform NFS I/O, the output of 'netstat
-m' shows more mbufs in use, but those will go down again if I stop I/O.
So it's not a continually increasing usage of mbufs.

I added MBUFTRACE to the kernel, and looking at the output of 'netstat
-mssv' I notice that 'nfs' and 'unknown data' seem to claim more than
they release (see below).

What else can I do to find out where I'm leaking mbufs (which I
understand to be the problem)?

-Jan

$ netstat -mssv
815 mbufs in use:
        814 mbufs allocated to data
        1 mbufs allocated to packet headers
0 calls to protocol drain routines
                                             small        ext    cluster
           vlan0 rx            inuse             0          0          0
                               claims         1893       1893       1893
                               releases       1893       1893       1893
           vlan0 tx            inuse             0          0          0
                               claims         2468        164        164
                               releases       2468        164        164
            unix               inuse             0          0          0
                               claims          391          3          3
                               releases        391          3          3
             tcp               inuse             1          0          0
                               claims           63          0          0
                               releases         62          0          0
             tcp rx            inuse             0          0          0
                               claims         1301       1301       1301
                               releases       1301       1301       1301
             tcp tx            inuse             1          0          0
                               claims         2949        299        299
                               releases       2948        299        299
             udp               inuse             0          0          0
                               claims          930          0          0
                               releases        930          0          0
             udp rx            inuse             0          0          0
                               claims       819979     762979     762979
                               releases     819979     762979     762979
                                             small        ext    cluster
             udp tx            inuse             0          0          0
                               claims        57316      35304      35304
                               releases      57316      35304      35304
        internet rx            inuse             0          0          0
                               claims      1389943    1389932    1389932
                               releases    1389943    1389932    1389932
        internet tx            inuse             0          0          0
                               claims        67080         14         14
                               releases      67080         14         14
        internet               inuse             0          0          0
                               claims           13          6          6
                               releases         13          6          6
             arp               inuse             0          0          0
                               claims          385        332        332
                               releases        385        332        332
           route               inuse             0          0          0
                               claims            7          0          0
                               releases          7          0          0
             lo0               inuse             0          0          0
                               claims           11          0          0
                               releases         11          0          0
             wm1 rx            inuse             0          0          0
                               claims       742129     742129     742129
                               releases     742129     742129     742129
                                             small        ext    cluster
             wm1 tx            inuse             0          0          0
                               claims       109090      44151      35337
                               releases     109090      44151      35337
             wm0 rx            inuse             0          0          0
                               claims            4          4          4
                               releases          4          4          4
             wm0 tx            inuse             0          0          0
                               claims            1          0          0
                               releases          1          0          0
             nfs               inuse            45         45         45
                               claims        97203      75942      67128
                               releases      97158      75897      67083
         unknown data          inuse           768        768        768
                               claims      1584064     742901     742901
                               releases    1583296     742133     742133
         unknown header        inuse             0          0          0
                               claims        12600          0          0
                               releases      12600          0          0
         unknown soname        inuse             0          0          0
                               claims        58062          0          0
                               releases      58062          0          0
         unknown soopts        inuse             0          0          0
                               claims          105          0          0
                                             small        ext    cluster
         unknown control       inuse             0          0          0
                               claims           12          0          0
                               releases         12          0          0

--=20
chown -R us:enemy your_base

--RE3pQJLXZi4fr8Xo
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (NetBSD)

iD8DBQFDdjNofFtkr68iakwRAshaAKCrOnUBc8WtC4oZosEMtOkBb2GRYwCgjwFy
+w3p1r65MFT2aVZxw6DVnQA=
=5B1l
-----END PGP SIGNATURE-----

--RE3pQJLXZi4fr8Xo--