misc/57509: Automated test case ndp_stray_entries fails intermittently on some architectures

[jspath55%gmail.com@localhost]

>Number:         57509
>Category:       misc
>Synopsis:       Automated test case ndp_stray_entries fails intermittently on some architectures
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 06 20:25:00 +0000 2023
>Originator:     Jim Spath
>Release:        10.0_BETA
>Organization:
>Environment:
NetBSD etc.etc 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Sun Feb 12 12:39:37 UTC 2023  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64 x86_64 AMD 686-class NetBSD

>Description:
This test case fails intermittently on some architectures I tested but not all.
Several tests were chosen from the full test suite after observing inconsistent pass/fail results over many runs.
Tests were run periodically from cron, timed to avoid overlapping runs while completing as many attempts as feasible based on the system speed.

The set being reported in July 2023 includes failure rates in the one-percent ranges, to be followed by other tests that fail more frequently.

The results for systems used:

System Passes Fails

amd64 3983 21
Pi3 817 8
Pi4 2360 9
Pi0W 2615 80
Pi02W 4020 15
i386 (Intel Atom) 710 0



Example failure message (snippet):

[...]
tc-so:Executing command [ rump.ndp -n fc00::2 ]
tc-so:Executing command [ rump.ndp -n fc00::2 ]
tc-so:Executing command [ rump.ndp -c ]
tc-so:Executing command [ rump.ndp -n fc00::2 ]
tc-se:Fail: incorrect exit status: 0, expected: anything else
tc-se:stdout:
tc-se:Neighbor                                Linklayer Address  Netif Expire    S Fl
tc-se:fc00::2                                 ..:..:..:..:..:.. shmif0 5s        D 
tc-se:
tc-se:stderr:
tc-se:
tc-end: 1685955020.447826, ndp_stray_entries, failed, atf-check failed; see the output of the test for details



Example pass message (snippet):

tc-so:Executing command [ env LD_PRELOAD=/usr/lib/librumphijack.so RUMPHIJACK=path=/rump,socket
=all:nolocal,sysctl=yes ifmcstat ]
tc-so:Executing command [ rump.ifconfig shmif1 destroy ]
tc-so:Executing command [ rump.ifconfig ]
tc-so:Executing command [ rump.ifconfig shmif0 destroy ]
tc-so:Executing command [ rump.ifconfig ]
tc-so:Executing command [ rump.ifconfig shmif0 destroy ]
tc-so:Executing command [ rump.ifconfig ]
tc-end: 1685958693.599555, ndp_stray_entries, passed
tp-end: 1685958693.648237, net/ndp/t_ndp


>How-To-Repeat:
$ atf-run net/ndp/t_ndp:ndp_stray_entries

>Fix:
Unknown.