NetBSD-Users archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Low capped TCP network speeds over the Internet
Hello,
I'm new to the BSD world and I've been *really* enjoying using NetBSD so
far. I run mostly Linux on my servers but I'd love to migrate at least
one to NetBSD. Sadly I'm hitting a big wall and I need your help.
In my local network NetBSD (running on Proxmox/KVM) saturates the 1 Gbps
link no problem, but the issue arises over the Internet. On a Proxmox
node located abroad (209ms RTT) I get these speeds:
Alpine Linux:
- iperf3 UDP: 609 Mbit/s
- iperf3 TCP: 90 Mbit/s
- HTTP: 9.70 MB/s
FreeBSD:
- iperf3 UDP: 607 Mbit/s
- iperf3 TCP: 91 Mbit/s
- HTTP: 9.65 MB/s
OpenBSD:
- iperf3 UDP: 619 Mbit/s
- iperf3 TCP: 62 Mbit/s
- HTTP: 9.07 MB/s
NetBSD:
- iperf3 UDP: 612 Mbit/s
- iperf3 TCP: 9.10 Mbit/s
- HTTP: 986 KB/s (hard capped and unstable)
UDP speeds are great, but NetBSD specifically has an issue with
TCP/HTTP. The speed seems to be capped to exactly 986 KB/s and it's very
unstable (congestion handling issues I assume).
I tried tuning the following:
net.inet.tcp.recvbuf_auto=1
net.inet.tcp.sendbuf_auto=1
net.bpf.jit=1
net.bpf.maxbufsize=4194304
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216
net.inet.ip.ifq.maxlen=4096
net.inet.tcp.mssdflt=1072
This fixed the unstability, but it's still seemingly hard capped to 986
KB/s.
Since post-tuning it's very stable I don't think it's about congestion
anymore - but I'm not sure why I'm hitting this hard cap, even after
increasing the buffer sizes. Could it be that NetBSD is not increasing
the window size?
I wrote about it in detail (and made some graphs + uploaded tcpdump
output) here:
https://omaera.org/wlog/tech/bsd_network
I saw someone last year around here mention that they got around 17 MB/s
with default settings over the Internet in a VPS, so I'm lost about
what's going on. I also tried moving the VM to Germany and Japan, but
similar result (different speed cap, all below 1 MB/s).
Since I only serve HTTP content I don't need huge speeds, just something
good enough. I'm not knowledgeable about networking so any pointers are
really helpful.
Home |
Main Index |
Thread Index |
Old Index