Re: Surprising observations on the NetBSD TCP stack

[mlelstv%serpens.de@localhost (Michael van Elst)]

kardel%netbsd.org@localhost (Frank Kardel) writes:

>I just got my Fiber 1GBit/sec internet connection (don't ask).
>When testing the throughput with breitbandmessung.de I got a meager
>88Mbit/sec download rate.

There are lots of things that do have an effect.

Download speeds are mostly affected by local buffering.
Upload speeds are mostly affected by congestion control.


>Testing local connectivity on the gigbit interfaces found:
> Â Â Â Â Â Â Â  gateway->notebook xfer rate: 61201.87 KB/sec !!! should be more 
>like 113000 KB/sec
> Â Â Â Â Â Â Â  notebook->gateway xfer rate: 113958.41 KB/sec
>This begs the question what is limiting the transfer rate asymmetrically?

Usually different network interfaces on either side, bad or nonexistent
interrupt mitigation, bad use of hardware queues, etc..

This is two NetBSD machines (Xeon+wm vs Ryzen+wm) on the local network.

iperf
[  7]   0.00-10.00  sec  1.06 GBytes   914 Mbits/sec    0             sender
[  7]   0.00-10.04  sec  1.06 GBytes   909 Mbits/sec                  receiver

iperf -R
[  7]   0.00-10.02  sec  1.07 GBytes   920 Mbits/sec    0             sender
[  7]   0.00-10.00  sec  1.07 GBytes   919 Mbits/sec                  receiver


>The NetBSD performance is not competative at all - barely able to keep 
>up with ADSL modems, though
>the send rate is ok.

I don't have a fibre connection, so testing on the internet gets
difficult.

Instead I put an RPI running Linux as a "WAN simulator" between
two NetBSD hosts to analyze the issues. I now have a couple of
changes in the TCP stack and workaround in the wm driver to
get almost 1Gbit/s over a simulated WAN with 200ms TTL.

This also helped a bit with internet upload speed, but at
40Mbit/s (VDSL) the effect is small.


> net.inet.tcp.recvbuf_max = 262144
> net.inet.tcp.sendbuf_max = 262144

You may need larger buffers (and larger increments).