Subject: Re: #32287 Processes hang in "mclpl" - feedback
To: Christos Zoulas <christos@astron.com>
From: Jorgen Lundman <lundman@lundman.net>
List: netbsd-users
Date: 08/28/2006 10:54:03
>
> So we see from the data that you have the same problem like I do. We are
> leaking mbufs in the tcp recv. I've been reading the code, but I have not
> found a problem...
>
They finally managed to make it happen:
Aug 26 14:28:16 mirror /netbsd: WARNING: mclpool limit reached; increase NMBCLUS
TERS
Aug 26 14:30:11 mirror /netbsd: WARNING: mclpool limit reached; increase NMBCLUS
TERS
Aug 26 14:31:14 mirror /netbsd: WARNING: mclpool limit reached; increase NMBCLUS
TERS
Aug 26 14:37:55 mirror last message repeated 5 times
Aug 26 15:03:28 mirror /netbsd: WARNING: mclpool limit reached; increase NMBCLUS
TERS
Aug 26 15:07:53 mirror last message repeated 3 times
Aug 26 15:23:00 mirror last message repeated 8 times
Aug 26 15:31:48 mirror last message repeated 5 times
Aug 26 15:47:12 mirror /netbsd: WARNING: mclpool limit reached; increase NMBCLUS
TERS
Crontab runs netstat every 5 minutes, which show just before:
Sat Aug 26 14:20:00 CEST 2006
33001 mbufs in use:
32967 mbufs allocated to data
34 mbufs allocated to packet headers
14666 calls to protocol drain routines
small ext cluster
route inuse 0 0 0
claims 19 0 0
releases 19 0 0
arp inuse 0 0 0
claims 604 443 443
releases 604 443 443
unix inuse 0 0 0
claims 151316 14061 5504
releases 151316 14061 5504
tcp inuse 34 0 0
claims 541276 0 0
releases 541242 0 0
tcp rx inuse 0 0 0
claims 454399042 454302211 454302211
releases 454399042 454302211 454302211
tcp tx inuse 32838 17142 17142
claims 3235280646 1585116978 1585116792
releases 3235247808 1585099836 1585099650
udp inuse 0 0 0
claims 11362 0 0
releases 11362 0 0
udp rx inuse 0 0 0
claims 25640 16844 16844
releases 25640 16844 16844
small ext cluster
udp tx inuse 0 0 0
claims 10356 0 0
releases 10356 0 0
internet rx inuse 0 0 0
claims 454403644 454310455 454310455
releases 454403644 454310455 454310455
internet tx inuse 0 0 0
claims 693325148 3 3
releases 693325148 3 3
lo0 inuse 0 0 0
claims 93207 0 0
releases 93207 0 0
ex0 rx inuse 0 0 0
claims 454310898 454310898 454310898
releases 454310898 454310898 454310898
ex0 tx inuse 0 0 0
claims 2188204597 1076832568 1076832397
releases 2188204597 1076832568 1076832397
unknown data inuse 128 128 128
claims 2996472228 454311282 454311282
releases 2996472100 454311154 454311154
unknown header inuse 0 0 0
claims 693331839 0 0
releases 693331839 0 0
small ext cluster
unknown soname inuse 0 0 0
claims 497540 0 0
releases 497540 0 0
unknown soopts inuse 0 0 0
claims 99586 0 0
releases 99586 0 0
unknown control inuse 0 0 0
claims 4 0 0
releases 4 0 0
And this mornings reading is:
Mon Aug 28 03:25:01 CEST 2006
53478 mbufs in use:
53429 mbufs allocated to data
49 mbufs allocated to packet headers
286534 calls to protocol drain routines
small ext cluster
route inuse 0 0 0
claims 19 0 0
releases 19 0 0
arp inuse 0 0 0
claims 884 647 647
releases 884 647 647
unix inuse 0 0 0
claims 228266 20680 8191
releases 228266 20680 8191
tcp inuse 49 0 0
claims 736300 0 0
releases 736251 0 0
tcp rx inuse 0 0 0
claims 617399674 617271075 617271075
releases 617399674 617271075 617271075
tcp tx inuse 53301 27885 27885
claims 154404677 2183697636 2183141342
releases 154351376 2183669751 2183113457
udp inuse 0 0 0
claims 14544 0 0
releases 14544 0 0
udp rx inuse 0 0 0
claims 32344 21205 21205
releases 32344 21205 21205
small ext cluster
udp tx inuse 0 0 0
claims 13095 0 0
releases 13095 0 0
internet rx inuse 0 0 0
claims 617440932 617317548 617317548
releases 617440932 617317548 617317548
internet tx inuse 0 0 0
claims 943883790 36582 36582
releases 943883790 36582 36582
internet inuse 0 0 0
claims 148 35 35
releases 148 35 35
lo0 inuse 0 0 0
claims 123414 0 0
releases 123414 0 0
ex0 rx inuse 0 0 0
claims 617318195 617318195 617318195
releases 617318195 617318195 617318195
ex0 tx inuse 0 0 0
claims 3002895211 1481989909 1481482587
releases 3002895211 1481989909 1481482587
unknown data inuse 128 128 128
claims 4123143040 617318579 617318579
releases 4123142912 617318451 617318451
small ext cluster
unknown header inuse 0 0 0
claims 943861192 0 0
releases 943861192 0 0
unknown soname inuse 0 0 0
claims 674196 0 0
releases 674196 0 0
unknown soopts inuse 0 0 0
claims 140343 0 0
releases 140343 0 0
unknown control inuse 0 0 0
claims 12 0 0
releases 12 0 0
tcp tx inuse 53301 27885 27885
claims 154404677 2183697636 2183141342
releases 154351376 2183669751 2183113457
tcp rx:
154351376 + 53301 = 154404677
2183669751 + 27885 = 2183697636
2183113457 + 27885 = 2183141342
Is it really leaking?
Lund
--
Jorgen Lundman | <lundman@lundman.net>
Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell)
Japan | +81 (0)3 -3375-1767 (home)