Re: wm + vlan tso4 problem with 10_BETA

[Masanobu SAITOH <msaitoh%execsw.org@localhost>]

Hi, Thor.

I'm very sorry that I've not noticed this mail until now.

On 2023/10/15 3:09, Thor Lancelot Simon wrote:
> Since upgrading my home gateway from an older version of NetBSD 9 to
> NetBSD 10.0_BETA, I've been having some throughput issues - forwarding
> performance seems to have dropped by between 30% and 50%, in a way
> that is highly sensitive to client TCP stack behavior.  More perhaps on
> that later.
> 
> However this has resulted in my testing every link in my internal network,
> to establish a baseline.  In the course of this I discovered that if TSO4
> is enabled on a vlan interface which is on wm, transmitting bulk data over
> TCP from the NetBSD gateway to a local client (on the same vlan) very
> quickly stalls.
> 
> Has anyone else seen anything like this recently?

No. I've never used TSO[46] on vlan interface...

> This configuration will stall almost immediately when sending to a local
> peer with ttcp -s -t -b $((192 * 1024)) -l 65536:
> 
> hotpoint# ifconfig vlan2
> vlan2: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> 	capabilities=0x7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
> 	capabilities=0x7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
> 	capabilities=0x7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
> 	enabled=0xf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
> 	vlan: 2 parent: wm1
> 	address: 00:e0:67:10:75:91
> 	status: active
> 	inet6 fe80::2e0:67ff:fe10:7591%vlan2/64 flags 0 scopeid 0x9
> 	inet6 2001:470:89fa:1::1/64 flags 0
> 	inet 192.168.100.1/24 broadcast 192.168.100.255 flags 0
> 
> But "ifconfig vlan2 -tso4" will make the problem go away.
> 
> Packets seem to be stuck somewhere in the offload path, because if I
> interrupt the sender while things are "stuck", the receiver never gets FIN,
> until I "ifconfig vlan2 -tso4" on the sender at which point the receiver
> immediately gets a FIN.

One of possibility is that the offset of the IP header is miscalculated...


> Here's some dmesg output:
> 
> NetBSD 10.0_BETA (GENERIC) #0: Mon Oct  2 17:39:38 UTC 2023
>         mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC
> total memory = 3987 MB
> avail memory = 3830 MB
> timecounter: Timecounters tick every 10.000 msec
> Kernelized RAIDframe activated
> i915drmkms* disabled
> timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
> mainbus0 (root)
> [...]
> wm1 at pci2 dev 0 function 0, 64-bit DMA: Intel i82583V (rev. 0x00)
> wm1: interrupting at msi3 vec 0
> wm1: PCI-Express bus
> wm1: 64 words (8 address bits) SPI EEPROM, version 1.10.0
> wm1: ASPM L0s and L1 are disabled to workaround the errata.
> wm1: Ethernet address 00:e0:67:10:75:91
> wm1: 0x2a4040<SPI,PCIE,ASF_FIRM,AMT,WOL>
> makphy1 at wm1 phy 1: Marvell 88E1149 Gigabit PHY, rev. 1
> makphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
> 1000baseT-FDX, auto
> 

-- 
-----------------------------------------------
                SAITOH Masanobu (msaitoh%execsw.org@localhost
                                 msaitoh%netbsd.org@localhost)