NetBSD-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
port-arm32/61006: Network issues with 32bit BeagleBone Black
>Number: 61006
>Category: port-arm32
>Synopsis: Network issues with 32bit BeagleBone Black
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: port-arm32-maintainer
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Nov 14 04:00:01 +0000 2026
>Originator: Glen Johnsen
>Release: 11
>Organization:
Private
>Environment:
System: NetBSD beaglebone 11.0 GENERIC evbarm earmv7hf
(stock 11.0 release kernel; the installation has since been
overwritten, so exact uname/dmesg output is no longer available)
Hardware: BeagleBone Black (TI AM335x, single-core Cortex-A8, 512 MiB RAM)
Board revision: Rev C
Boot media: eMMC
Power: USB
Interface: cpsw0, autoselect, negotiates 100baseTX full-duplex
Driver: sys/arch/arm/ti/if_cpsw.c rev 1.17 (same code as netbsd-10 rev 1.16.4.1)
dmesg.boot was not saved before the installation was overwritten.
I can reinstall and reproduce on request.
>Description:
On a lightly loaded LAN, cpsw0 repeatedly stops passing traffic in
both directions. Each episode ends when the driver watchdog fires
("cpsw0: device timeout"), which re-runs cpsw_init(). Afterwards
the IPv4 address goes through DAD again, which adds about 8 s of
unreachability (ARP probes from 0.0.0.0 and then announcements
were observed in tcpdump before the host began replying again).
The link stays "status: active, 100baseTX full-duplex" throughout.
The same board, cable, and switch running Debian 8 (Linux
4.4.30-ti-r64) show no outages. Under Linux, iperf3 3.21 UDP from
the board reached 80 Mbit/s with 0 loss, including 71-packet bursts
(-b 80M/71).
Observations under NetBSD 11.0:
1. Every Oerrs on cpsw0 is a watchdog reset.
`dmesg | grep -c 'cpsw0: device timeout'` = 23 while
`netstat -I cpsw0` showed Oerrs = 23 and Ierrs = 0.
2. Frequency. Kernel timestamps of "device timeout" during one boot
(seconds since boot):
1424.6, 1502.6, 1709.6, 1733.6, 1832.6, 1860.6, 2029.6,
2053.6, 2087.7
Intervals: 78, 207, 24, 99, 28, 169, 24, 34 s. The workstation
was sending no test traffic during most of this time.
3. Both RX and TX interrupts stop together. On the AM335x, cpsw RX
is irq 41 and TX is irq 42. I polled `vmstat -i` and
`netstat -n -I cpsw0` every 2 s (times are UTC):
23:19:07 last echo reply seen by remote host
23:19:10 irq41=305 irq42=370 (RX count stops)
23:19:12 irq41=305 irq42=377 (TX count stops)
23:19:12 .. 23:19:22 both counts frozen
23:19:27 irq41=305 irq42=381 (TX resumes)
23:19:29 irq41=308 irq42=386 (RX resumes)
23:19:32 remote host gets replies again
During the freeze, Ipkts and Opkts each rose by 7. This appears
to be IFF_SIMPLEX loopback of the host's own broadcast ARP
requests, not hardware traffic. irq 40 (RX threshold) and irq 43
(misc, which includes CPDMA host error) never appeared in
`vmstat -i`, i.e. their counts stayed at zero. I have not checked
whether the driver enables the host-error interrupt.
4. While stalled, local sends fail with "ping: sendto: Host is down"
(ARP for the gateway cannot complete). The remote side sees ARP
INCOMPLETE for the board.
5. Outages get worse with uptime. A remote host pinged the board
every 2 s after a fresh boot:
first 15 min: 8.9% loss, outages of 25-38 s
next 30 min: 35% loss, outages of 14-51 s, then 198 s and
258 s (23:49:56-23:53:14 and
23:55:07-23:59:25 UTC)
Possibly relevant code observation (not verified as the cause):
cpsw_start() sets ifp->if_timer = 5 whenever it queues new
descriptors, even if no earlier transmit has completed. So as long
as the stack keeps queueing packets (ARP retries, etc.) at least
every 5 s, the watchdog may be pushed back repeatedly. This could
explain outages much longer than 5 s before "device timeout"
appears. The underlying stall, with RX and TX interrupts stopping
together and no misc interrupt, is not explained by this.
Separately, and possibly related: before this investigation, UDP
sent from the board with iperf3 3.21 (-u -l 1400) lost 22-34% at
40-80 Mbit/s, including over a direct cable. The datagrams were
visible in a tcpdump capture on the board but missing at the
receiver. sndq drops stayed at 0. Linux on the same hardware showed
no loss. I can file this separately if preferred.
>How-To-Repeat:
Boot NetBSD 11.0 GENERIC (evbarm earmv7hf) on a BeagleBone Black
connected to a 100 Mbit/s or gigabit switch. Leave it on the LAN
with normal background traffic, optionally pinged every 1-2 s
from another host. Within a few minutes, "cpsw0: device timeout"
appears on the console, and the host is unreachable for tens of
seconds at a time. To watch interrupts during a stall:
while :; do echo "== $(date +%T) $(vmstat -i | \
awk '/irq 4[0-3] /{printf "irq%s=%s ", $4, $5}')"; \
netstat -n -I cpsw0 | sed -n 2p; sleep 2; done
>Fix:
Unknown. Workarounds: none found. The watchdog reset restores
traffic until the next stall. `ifconfig cpsw0 down; ifconfig cpsw0
up` also restored traffic in one test, but a watchdog reset
happened at the same time, so that result is ambiguous.
Home |
Main Index |
Thread Index |
Old Index