Subject: Re: #32287 Processes hang in "mclpl" - feedback
To: None <netbsd-users@netbsd.org>
From: Christos Zoulas <christos@astron.com>
List: netbsd-users
Date: 08/22/2006 12:39:04
In article <44EAEE0A.8010005@lundman.net>,
Jorgen Lundman  <lundman@lundman.net> wrote:

Compile a kernel with options MBUFTRACE and then use netstat to check
where the leak is.

christos

>  netstat -m
>63830 mbufs in use:
>         63768 mbufs allocated to data
>         62 mbufs allocated to packet headers
>66079 calls to protocol drain routines
>
>
>tcp:
>         40470144 packets sent
>                 29243662 data packets (41070844734 bytes)
>                 29358 data packets (32997535 bytes) retransmitted
>                 6515097 ack-only packets (9159013 delayed)
>                 0 URG only packets
>                 113 window probe packets
>                 4677532 window update packets
>                 4785 control packets
>                 256 send attempts resulted in self-quench
>         30363819 packets received
>                 13588596 acks (for 40948780734 bytes)
>                 1017732 duplicate acks
>                 0 acks for unsent data
>                 15108894 packets (21799017730 bytes) received in-sequence
>                 8868 completely duplicate packets (12231715 bytes)
>                 0 old duplicate packets
>                 64 packets with some dup. data (37458 bytes duped)
>                 119844 out-of-order packets (139931632 bytes)
>                 0 packets (0 bytes) of data after window
>                 0 window probes
>                 126854 window update packets
>                 1042 packets received after close
>                 286 discarded for bad checksums
>                 0 discarded for bad header offset fields
>                 0 discarded because packet too short
>         2211 connection requests
>         4441 connection accepts
>         6639 connections established (including accepts)
>         11113 connections closed (including 161 drops)
>         7 embryonic connections dropped
>         0 delayed frees of tcpcb
>         2775125 segments updated rtt (of 3042007 attempts)
>         5935 retransmit timeouts
>                 72 connections dropped by rexmit timeout
>         117 persist timeouts (resulting in 0 dropped connections)
>         4 keepalive timeouts
>                 0 keepalive probes sent
>                 0 connections dropped by keepalive
>         4882954 correct ACK header predictions
>         15094287 correct data packet header predictions
>         11500 PCB hash misses
>         1270 dropped due to no socket
>         240 connections drained due to memory shortage
>         148 PMTUD blackholes detected
>         2 bad connection attempts
>         4443 SYN cache entries added
>                 0 hash collisions
>                 4441 completed
>                 0 aborted (no space to build PCB)
>                 1 timed out
>                 0 dropped due to overflow
>                 0 dropped due to bucket overflow
>                 1 dropped due to RST
>                 0 dropped due to ICMP unreachable
>                 0 delayed free of SYN cache entries
>         11 SYN,ACKs retransmitted
>         6 duplicate SYNs received for entries already in the cache
>         34 SYNs dropped (no route or no space)
>         0 packets with bad signature
>         0 packets with good signature
>
>Pavel Cahyna wrote:
>> On Tue, Aug 22, 2006 at 04:59:01PM +0900, Jorgen Lundman wrote:
>> 
>>>Actually still triggers with NMBCLUSTERS at 65536, after roughly 1h,
>10m, 38s. 
>>>The network hang is for about 7 minutes.
>>>
>>>WARNING: mclpool limit reached; increase NMBCLUSTERS
>>>WARNING: mclpool limit reached; increase NMBCLUSTERS
>> 
>> 
>> what does netstat -m show? are the numbers increasing since boot?
>> 
>> Pavel
>> 
>
>-- 
>Jorgen Lundman       | <lundman@lundman.net>
>Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
>Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
>Japan                | +81 (0)3 -3375-1767          (home)
>