Subject: Re: increasing NMBCLUSTERS
To: None <tech-net@netbsd.org>
From: Jan Schaumann <jschauma@netmeister.org>
List: tech-net
Date: 11/12/2005 13:24:41
--RE3pQJLXZi4fr8Xo
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Jan Schaumann <jschauma@netmeister.org> wrote:
> Manuel Bouyer <bouyer@antioche.eu.org> wrote:
> > On Wed, Sep 28, 2005 at 01:10:31PM -0400, Jan Schaumann wrote:
> =20
> > > WARNING: mclpool limit reached; increase NMBCLUSTERS
> > >=20
> > > I've already bumped NMBCLUSTERS twice, up to 65536 now. options(4) is
> > > not very exhaustive of what the side effects (if any) may be of
> > > increasing this number. How far can I bump this number? What is
> > > affected by it?
> >=20
> > Did you see what could be using that much clusters ?
>=20
> Well, right now, after a fresh reboot, netstat -m gives me:
>=20
> 803 mbufs in use:
> 772 mbufs allocated to data
> 16 mbufs allocated to packet headers
> 15 mbufs allocated to socket names and addresses
> 0 calls to protocol drain routines
I still am seeing this problem, and my machine locks up after a couple
of days even after continuing to increase NMBCLUSTERS. Looking at the
output of vmstat and netstat, I notice:
$ vmstat -m | egrep "(Name|mbpl|mclpl)"
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg=
Idle
mbpl 256 2550 0 0 161 0 161 161 1 inf =
1
mclpl 2048 2523 0 0 1266 0 1266 1266 4 13107=
2 4
$ netstat -m =20
810 mbufs in use:
809 mbufs allocated to data
1 mbufs allocated to packet headers
0 calls to protocol drain routines
It seems odd to me that I don't have any Releases, 0 calls to protocol
drain routines. However, if I perform NFS I/O, the output of 'netstat
-m' shows more mbufs in use, but those will go down again if I stop I/O.
So it's not a continually increasing usage of mbufs.
I added MBUFTRACE to the kernel, and looking at the output of 'netstat
-mssv' I notice that 'nfs' and 'unknown data' seem to claim more than
they release (see below).
What else can I do to find out where I'm leaking mbufs (which I
understand to be the problem)?
-Jan
$ netstat -mssv
815 mbufs in use:
814 mbufs allocated to data
1 mbufs allocated to packet headers
0 calls to protocol drain routines
small ext cluster
vlan0 rx inuse 0 0 0
claims 1893 1893 1893
releases 1893 1893 1893
vlan0 tx inuse 0 0 0
claims 2468 164 164
releases 2468 164 164
unix inuse 0 0 0
claims 391 3 3
releases 391 3 3
tcp inuse 1 0 0
claims 63 0 0
releases 62 0 0
tcp rx inuse 0 0 0
claims 1301 1301 1301
releases 1301 1301 1301
tcp tx inuse 1 0 0
claims 2949 299 299
releases 2948 299 299
udp inuse 0 0 0
claims 930 0 0
releases 930 0 0
udp rx inuse 0 0 0
claims 819979 762979 762979
releases 819979 762979 762979
small ext cluster
udp tx inuse 0 0 0
claims 57316 35304 35304
releases 57316 35304 35304
internet rx inuse 0 0 0
claims 1389943 1389932 1389932
releases 1389943 1389932 1389932
internet tx inuse 0 0 0
claims 67080 14 14
releases 67080 14 14
internet inuse 0 0 0
claims 13 6 6
releases 13 6 6
arp inuse 0 0 0
claims 385 332 332
releases 385 332 332
route inuse 0 0 0
claims 7 0 0
releases 7 0 0
lo0 inuse 0 0 0
claims 11 0 0
releases 11 0 0
wm1 rx inuse 0 0 0
claims 742129 742129 742129
releases 742129 742129 742129
small ext cluster
wm1 tx inuse 0 0 0
claims 109090 44151 35337
releases 109090 44151 35337
wm0 rx inuse 0 0 0
claims 4 4 4
releases 4 4 4
wm0 tx inuse 0 0 0
claims 1 0 0
releases 1 0 0
nfs inuse 45 45 45
claims 97203 75942 67128
releases 97158 75897 67083
unknown data inuse 768 768 768
claims 1584064 742901 742901
releases 1583296 742133 742133
unknown header inuse 0 0 0
claims 12600 0 0
releases 12600 0 0
unknown soname inuse 0 0 0
claims 58062 0 0
releases 58062 0 0
unknown soopts inuse 0 0 0
claims 105 0 0
small ext cluster
unknown control inuse 0 0 0
claims 12 0 0
releases 12 0 0
--=20
chown -R us:enemy your_base
--RE3pQJLXZi4fr8Xo
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (NetBSD)
iD8DBQFDdjNofFtkr68iakwRAshaAKCrOnUBc8WtC4oZosEMtOkBb2GRYwCgjwFy
+w3p1r65MFT2aVZxw6DVnQA=
=5B1l
-----END PGP SIGNATURE-----
--RE3pQJLXZi4fr8Xo--