misc/57506: Automated test case ping_of_death fails intermittently on different architectures

[jspath55%gmail.com@localhost]

>Number:         57506
>Category:       misc
>Synopsis:       Automated test case ping_of_death fails intermittently on different architectures
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 06 19:00:01 +0000 2023
>Originator:     Jim Spath
>Release:        10.0_BETA
>Organization:
>Environment:
NetBSD etc.etc 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Sun Feb 12 12:39:37 UTC 2023  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64 x86_64 AMD 686-class NetBSD
>Description:
This test case fails intermittently on each architecture I tested.
Several examples were chosen from the full test suite after observing inconsistent pass/fail on specific tests over many runs.
Tests were run periodically from cron, tuned to avoid overlapping runs while completing as many attempts as feasible based on the system speed.

The set being reported in July 2023 includes failure rates in the one-percent ranges, to be followed by other tests that fail more frequently.

The results for systems used:

System Passes Fails

amd64 3958 47
Pi3 814 11
Pi4 2357 12
Pi0W 2685 10
Pi02W 3964 71
i386 (Intel Atom) 707 3


Example failure message:

tps-count: 1
tp-start: 1685496361.190626, net/icmp/t_ping, 1
tc-start: 1685496361.190682, ping_of_death
tc-end: 1685496381.218400, ping_of_death, failed, Test case timed out after 20 seconds
tp-end: 1685496381.219241, net/icmp/t_ping
info: time.end, Wed May 31 01:26:21 UTC 2023

All observed failures contain the same 20 second timeout message.

Example pass message:

tps-count: 1
tp-start: 1685378232.675280, net/icmp/t_ping, 1
tc-start: 1685378232.675307, ping_of_death
tc-so:[   1.0000000] entropy: ready
tc-end: 1685378232.824926, ping_of_death, passed
tp-end: 1685378232.825618, net/icmp/t_ping
info: time.end, Mon May 29 16:37:12 UTC 2023


>How-To-Repeat:
$ atf-run net/icmp/t_ping:ping_of_death


>Fix:
Increase the 20-second limit?