Re: misc/59389 (releng testbed running tests from wrong build?)

["Andreas Gustafsson via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR misc/59389; it has been noted by GNATS.

From: Andreas Gustafsson <gson%NetBSD.org@localhost>
To: riastradh%NetBSD.org@localhost, gnats-bugs%netbsd.org@localhost
Cc: campbell+netbsd%mumble.net@localhost
Subject: Re: misc/59389 (releng testbed running tests from wrong build?)
Date: Sat, 3 May 2025 18:21:40 +0300

 Taylor,
 
 I think this was not testing the wrong build, but building with the
 wrong version of some of the source files.
 
 The testbeds work by mirroring the CVS repository from anoncvs
 using rsync, and it looks like some of your commits, for example
 src/tests/lib/csu/t_hello.sh 1.3, did not appear on anoncvs until
 several hours after other, later commits, for example
 src/sys/arch/mips/mips/mips_emul.c 1.32.
 
 Sadly, as far as I know, the NetBSD project does not have a method of
 public read access to the repository that offers any consistency
 guarantees.  Different files in the copy on anoncvs can be out of date
 by arbitrary and different amounts of time, and there is no way to
 tell when this happens.  This issue is not specific to rsync, but also
 affects normal CVS checkouts from anoncvs.
 
 The testbeds attempt to alleviate the problem by only checking out
 versions that were already at least 10 minutes old at the time when
 the rsync process started, to allow for up to 10 minutes of time skew
 between the different parts of the repository on anoncvs, but this
 will of course not help if the actual skew is measured in hours.
 
 I'm attaching some log output to illustrate.  This was extracted from
 the logs on babylon4.netbsd.org by running the following script and
 manually trimming the output for size:
 
   cd /bracket/i386/log/repo-update/2025
   for f in rsync-2025050[23]*
   do
       echo $f
       grep ",v" $f | sed 's/^/  /'
   done
 
 It shows that a rsync run on 2025-05-03 02:09 UTC picked up a bunch
 of mips changes, then rsync was run 10 times without picking up any
 changes, until finally the run on 2025-05-03 04:35 UTC picked up
 a bunch of other, older commits:
 
 rsync-20250503-0209.log
   src/sys/arch/mips/include/cache.h,v
   src/sys/arch/mips/include/mips_param.h,v
   src/sys/arch/mips/mips/fp.S,v
   src/sys/arch/mips/mips/mips_emul.c,v
   src/sys/arch/mips/mips/mips_fixup.c,v
   src/sys/arch/mips/mips/mips_fputrap.c,v
   src/sys/arch/mips/mips/process_machdep.c,v
   src/sys/arch/mips/rmi/rmixlreg.h,v
 rsync-20250503-0221.log
 rsync-20250503-0242.log
 rsync-20250503-0312.log
 rsync-20250503-0324.log
 rsync-20250503-0336.log
 rsync-20250503-0347.log
 rsync-20250503-0359.log
 rsync-20250503-0411.log
 rsync-20250503-0423.log
 rsync-20250503-0435.log
   src/lib/csu/common/crt0-common.c,v
   src/lib/libc/include/__sysctl.h,v
   src/lib/libc/misc/stack_protector.c,v
   src/libexec/ld.elf_so/Makefile,v
   src/libexec/ld.elf_so/expand.c,v
   src/libexec/ld.elf_so/headers.c,v
   src/libexec/ld.elf_so/paths.c,v
   src/libexec/ld.elf_so/reloc.c,v
   src/libexec/ld.elf_so/rtld.c,v
   src/libexec/ld.elf_so/rtld.h,v
   src/libexec/ld.elf_so/symbol.c,v
   src/sys/fs/exfatfs/Attic/exfatfs_dirent.h,v
   src/sys/fs/exfatfs/Attic/exfatfs_extern.c,v
   src/sys/fs/exfatfs/Attic/exfatfs_inode.h,v
   src/sys/sys/exec_elf.h,v
   src/tests/lib/csu/t_hello.sh,v
   src/tests/libexec/ld.elf_so/t_r_rel.sh,v
   src/usr.sbin/dumpexfatfs/Attic/dumpexfatfs.c,v
 rsync-20250503-0457.log
 
 -- 
 Andreas Gustafsson, gson%NetBSD.org@localhost