Re: Automated report: NetBSD-current/i386 test failure

[Andreas Gustafsson <gson%gson.org@localhost>]

The NetBSD Test Fixture wrote:
> This is an automatically generated notice of a new failure of the
> NetBSD test suite.
> 
> The newly failing test case is:
> 
>     net/ipsec/t_ipsec_tunnel_odd:ipsec_tunnel_v4v6_esp_cast128cbc
> 
> The above test failed in each of the last 4 test runs, and passed in
> at least 26 consecutive runs before that.
> 
> The following commits were made between the last successful test and
> the first failed test:
> 
>     2023.10.12.17.18.38 riastradh src/crypto/external/bsd/heimdal/Makefile.inc 1.11
(etc)

Sorry about the multitude of automated reports, some of which (such as
the one quoted above) pointed at the wrong commit.

What happened is that a commit by ad@ caused a large number of tests
to start failing randomly, and because the failures are random and
numerous and today is Friday the 13th, some of the tests happened to
fail for the first time only after some later commit, and then
also happened to fail several times in a row.

This resulted in a pattern of a long stretch of successes followed by
several failures in a row.  This looks very much the same as a
deterministic failure introduced by the later commit, and the testbed
mistook it as such.

Even though some of the test failures were reported as an excessive
number of separate emails and attributed to the wrong commit, the
failures themselves are real.
-- 
Andreas Gustafsson, gson%gson.org@localhost