misc/59389: releng testbed running tests from wrong build?

[campbell+netbsd%mumble.net@localhost]

>Number:         59389
>Category:       misc
>Synopsis:       releng testbed running tests from wrong build?
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat May 03 13:10:00 +0000 2025
>Originator:     Taylor R Campbell
>Release:        
>Organization:
The TimewarpBSD Testification
>Environment:
>Description:
At https://releng.netbsd.org/b5reports/amd64/commits-2025.05.html#build-2025.05.02.22.30.44, it shows:

build: OK with 739094 lines of log, install: OK, tests: 10504 passed, 613 skipped, 120 expected_failure, 1 failed, ATF output: raw, xml, html
new failures: net/if_wg/t_misc/wg_rekey

and the linked test output at https://releng.netbsd.org/b5reports/amd64/2025/2025.05.02.22.30.44/test.html#xfail-tcs-summary shows xfails for lib/csu/t_hello:relr and libexec/ld.elf_so/t_r_rel:run_relative_pack -- that's what I expect.

Shortly after that, at CVS date 2025.05.02.23.03.16, I committed a fix and removed the xfail marker for libexec/ld.elf_so/t_r_rel:run_relative_pack:

https://cvsweb.netbsd.org/bsdweb.cgi/src/tests/libexec/ld.elf_so/t_r_rel.sh#rev1.2
https://mail-index.netbsd.org/source-changes/2025/05/02/msg156742.html

--- src/tests/libexec/ld.elf_so/t_r_rel.sh:1.1	Fri May  2 03:26:26 2025
+++ src/tests/libexec/ld.elf_so/t_r_rel.sh	Fri May  2 23:03:16 2025
...
@@ -91,8 +91,6 @@ run_relative_pack_body()
 {
 	case `uname -p` in
 	i386|powerpc64*|x86_64)
-		atf_expect_fail "PR bin/59360: ld.elf_so(8):"
-		    " missing RELR support"
 		;;
 	*)	# Missing GNU binutils ld(1) support to generate RELR
 		# sections, so the program should run just fine because


Similarly, at CVS date 2025.05.02.23.04.06, I committed a fix and removed the xfail marker for lib/csu/t_hello:relr:

https://cvsweb.netbsd.org/bsdweb.cgi/src/tests/lib/csu/t_hello.sh#rev1.3
https://mail-index.netbsd.org/source-changes/2025/05/02/msg156743.html

--- src/tests/lib/csu/t_hello.sh:1.2	Fri May  2 22:30:29 2025
+++ src/tests/lib/csu/t_hello.sh	Fri May  2 23:04:06 2025
...
@@ -93,9 +93,6 @@ relr_body()
 	checksupport h_hello_relr
 	case `uname -p` in
 	i386|x86_64)
-		# Actually csu missing RELR support for this, not ld.elf_so.
-		atf_expect_fail "PR bin/59360: ld.elf_so(8):" \
-		    " missing RELR support"
 		;;
 	*)	atf_expect_fail "PR lib/59359: static pies are broken"
 		;;

The next build at https://releng.netbsd.org/b5reports/amd64/commits-2025.05.html#build-2025.05.03.02.00.57 _after_ these fix, though, still shows 120 expected failures, even though these were the only changes to xfails in the time since the last build before it:

build: OK with 738720 lines of log, install: OK, tests: 10505 passed, 613 skipped, 120 expected_failure, 0 failed, ATF output: raw, xml, html
no longer failing: net/if_wg/t_misc/wg_rekey

And the build output shows the tests xfailing in the same way:

https://releng.netbsd.org/b5reports/amd64/2025/2025.05.03.02.00.57/test.html#xfail-tcs-summary

The _following_ build -- after unrelated changes to lib/libutil/strpct.c and tests/lib/libutil/t_strpct.c -- shows the csu and ld.elf_so changes passing:

https://releng.netbsd.org/b5reports/amd64/2025/2025.05.03.08.18.33/test.html

What happened with the middle build?  It must have run a stale build of ld.elf_so and the tests.
>How-To-Repeat:
unsure
>Fix:
Yes, please!