Subject: Re: Are tlp[01] weak for big traffic?
To: None <port-cobalt@netbsd.org>
From: NAKAJI Hiroyuki <nakaji@jp.freebsd.org>
List: port-cobalt
Date: 12/21/2006 07:29:21
I started another download of about 1GB in total and went to bed.
When I have got up, all IPv4 connections are totally lost. Serial
console is functional. Another log of one Windows application shows
that the connection is lost at 2:46 am JST (about 3 hours later after
the start of this download).

>>>>> In <061220005911.M0115302@mirage.ceres.dti.ne.jp> 
>>>>>	Izumi Tsutsui <tsutsui@ceres.dti.ne.jp> wrote:
> nakaji@jp.freebsd.org wrote:

> > $ netstat -in

> What does "netstat -in 1" show?

When IPv4 is alive, it shows no errs nor colls. And now,

bash-3.2# netstat -in -I tlp0 1
 tlp0 in       tlp0 out              total in      total out            
 packets  errs  packets  errs colls   packets  errs  packets  errs colls
 1660378     0   977657     0     0   2651602     0  2601825     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
^C
bash-3.2# netstat -in -I tlp1 1
 tlp1 in       tlp1 out              total in      total out            
 packets  errs  packets  errs colls   packets  errs  packets  errs colls
  915078     0  1548022     0     0   2651602     0  2601825     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
       0     0        0     0     0         0     0        0     0     0
^C

> > I was running "tcpdump -n -i tlp0", "tcpdump -n -i tlp1" and "vmstat
> > -i -w 5" while downloading.

> Hmm, how about "tcpdump -env"?

I recorded with -w option. What is to be checked especially?

> > The maximum rate of "soft net" was 29.

> What about int 1 and int 2 (for tlp0 and tlp1)?

When all tlp connections are lost, "vmstat -i" shows

bash-3.2# vmstat -i
interrupt                                     total     rate
soft serial                                  245440        2
soft net                                    1094919       13
soft clock                                   108236        1
cpu int 3                                    247481        2
cpu int 1                                   2237276       26
icu irq 14                                   211981        2
cpu int 2                                   2317763       27
mips int 5 (clock)                          8356124      100
Total                                      14819220      177

> Does ping(8) (and other TCP programs) via your cobalt
> work at that time?

Ping to/from cobalt does not work: all packets are lost. Ping to
cobalt from cobalt itself neither. And "ifconfig -a" shows tlp[01] are
active.

bash-3.2# ifconfig -a
tlp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        address: 00:10:e0:00:6f:33
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 60.32.13.193 netmask 0xfffffff8 broadcast 60.32.13.199
        inet6 fe80::210:e0ff:fe00:6f33%tlp0 prefixlen 64 scopeid 0x1
tlp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        address: 00:10:e0:00:6f:32
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::210:e0ff:fe00:6f32%tlp1 prefixlen 64 scopeid 0x2
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33192
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3

Ping losts 100% of packets to localhost also.

bash-3.2# ping 127.0.0.1
PING localhost (127.0.0.1): 56 data bytes
ping: sendto: No buffer space available
ping: sendto: No buffer space available
ping: sendto: No buffer space available
ping: sendto: No buffer space available
^C
----localhost PING Statistics----
4 packets transmitted, 0 packets received, 100.0% packet loss

Ping to 60.32.13.193 and 192.168.1.1 does not start, no output.

bash-3.2# ping 192.168.1.1
^C
bash-3.2# ping 60.32.13.193
^C

> > And, tcpdump shows only SNMP, NTP, and SMB now.

> Hmm, if any packets are handled in upper layer,
> it shouldn't be a driver problem (nor cobalt specific)
> but some issue around ipf or other settings, I guess.

I'm confused very much. I just copied the settings from NetBSD/i386
(former nat box). What is the next step?

Thanks.
-- 
NAKAJI Hiroyuki