[[ I'm not sure where to direct this, so I'm CCing current-users... ]]
I've been running sysbuild daily on a stock NetBSD/i386-10.0 system
since 2024/08/09. There have been very few failures since then and none
without obvious related changes coming from CVS, and they've all
resolved themselves with subsequent updates.
Yesterday (2024/10/06) I finally did an upgrade of the system itself
from yesterday's build, and this morning I find the following failure in
the latest log:
# link binstall/xinstall
cc -O -I/home/sysbuild/i386/obj/home/sysbuild/src/tools/compat -I/home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/include -I/home/sysbuild/src/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -I/home/sysbuild/src/tools/binstall/../compat/sys -DTARGET_STRIP=\"/home/sysbuild/i386/tools/bin/i486--netbsdelf-strip\" -I/home/sysbuild/src/usr.sbin/mtree -o xinstall xinstall.lo getid.lo -L/home/sysbuild/i386/obj/home/sysbuild/src/tools/compat -lnbcompat -lrt -lz
ld: /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/libnbcompat.a(sha2.lo): in function `SHA256_Transform':
sha2.c:(.text+0x11b): undefined reference to `be32dec'
ld: /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/libnbcompat.a(sha2.lo): in function `SHA224_256_Final':
sha2.c:(.text+0x4fe): undefined reference to `be32enc'
ld: /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/libnbcompat.a(sha2.lo): in function `SHA512_Transform':
sha2.c:(.text+0xaf9): undefined reference to `be64dec'
ld: /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/libnbcompat.a(sha2.lo): in function `SHA512_Final':
sha2.c:(.text+0x1665): undefined reference to `be64enc'
ld: /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/libnbcompat.a(sha2.lo): in function `SHA384_Final':
sha2.c:(.text+0x1824): undefined reference to `be64enc'
*** Failed target: xinstall
*** Failed commands:
${_MKTARGET_LINK}
=> @echo '# ' " link " binstall/xinstall
${HOST_LINK.c} ${HOST_LDSTATIC} -o ${.TARGET} ${OBJS} ${LDADD}
=> cc -O -I/home/sysbuild/i386/obj/home/sysbuild/src/tools/compat -I/home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/include -I/home/sysbuild/src/tools/compat -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -I/home/sysbuild/src/tools/binstall/../compat/sys -DTARGET_STRIP=\"/home/sysbuild/i386/tools/bin/i486--netbsdelf-strip\" -I/home/sysbuild/src/usr.sbin/mtree -o xinstall xinstall.lo getid.lo -L/home/sysbuild/i386/obj/home/sysbuild/src/tools/compat -lnbcompat -lrt -lz
*** [xinstall] Error code 1
nbmake[7]: stopped in /home/sysbuild/src/tools/binstall
1 error
There were no related source changes in the daily "cvs update", but this
is part of the tools build, so I assume it's something that changed
during the upgrade.
My libnbcompat.a is not stale -- it was rebuilt today. Indeed the
object causing the failure was also rebuilt today:
$ ll /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/sha2.lo
24 -rw-r--r-- 1 sysbuild sysbuild 10500 Oct 7 00:20 /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/sha2.lo
Those beNNXXc() functions should be inlined, I think -- they are defined
in <sys/endian.h> via a set of macros for any compiler since gcc-2.95.
However they do not get inlined:
$ nm /home/sysbuild/i386/obj/home/sysbuild/src/tools/compat/sha2.lo
00000340 r K256
00000080 r K512
000002d5 t SHA224_256_Final
00000914 T SHA224_Final
00000834 T SHA224_Init
000008f8 T SHA224_Transform
000008d5 T SHA224_Update
0000081f T SHA256_Final
00000000 T SHA256_Init
000000a1 T SHA256_Transform
0000056e T SHA256_Update
000017ef T SHA384_Final
000016bd T SHA384_Init
000017d3 T SHA384_Transform
000017b0 T SHA384_Update
00001630 T SHA512_Final
00000929 T SHA512_Init
00000fec t SHA512_Last
00000a1c T SHA512_Transform
000012ad T SHA512_Update
U be32dec
U be32enc
U be64dec
U be64enc
00000320 r sha224_initial_hash_value
00000300 r sha256_initial_hash_value
00000040 r sha384_initial_hash_value
00000000 r sha512_initial_hash_value
From a little earlier in the build log I see that indeed the inline
definitions were not used:
# compile compat/sha2.lo
cc -O -I. -I./include -I/home/sysbuild/src/tools/compat -I/home/sysbuild/src/tools/compat/sys -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -I. -I./include -I/home/sysbuild/src/tools/compat -I/home/sysbuild/src/tools/compat/sys -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D__DBINTERFACE_PRIVATE -c -o sha2.lo.o /home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c: In function 'SHA256_Transform':
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c:388:13: warning: implicit declaration of function 'be32dec' [-Wimplicit-function-declaration]
388 | W256[j] = be32dec(data);
| ^~~~~~~
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c: In function 'SHA224_256_Final':
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c:564:4: warning: implicit declaration of function 'be32enc' [-Wimplicit-function-declaration]
564 | be32enc(digest + 4 * i, context->state[i]);
| ^~~~~~~
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c: In function 'SHA512_Transform':
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c:729:13: warning: implicit declaration of function 'be64dec' [-Wimplicit-function-declaration]
729 | W512[j] = be64dec(data);
| ^~~~~~~
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c: In function 'SHA512_Final':
/home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c:912:4: warning: implicit declaration of function 'be64enc' [-Wimplicit-function-declaration]
912 | be64enc(digest + 8 * i, context->state[i]);
| ^~~~~~~
So this appears to be due to fallout from the following change:
revision 1.31.4.2
date: 2025-04-12 05:56:47 -0700; author: martin; state: Exp; lines: +16 -16; commitid: yyI3PgWxFB7QjLQF;
Pull up following revision(s) (requested by rin in ticket #1095):
sys/sys/endian.h: revision 1.36
sys/sys/endian.h: revision 1.37
endian.h: Consistently use __CAST() where appropriate, NFC
In preparation for PR lib/57478
endian.h: Cast to suitable type when byte order conversion is NOP
Otherwise, e.g., following examples do not work as expected on
big-endian machines:
- printf("%x\n", ntohl(0x01234567890abcdefULL);
- uint64_t u64 = ntohl(0x01234567890abcdefULL);
- etc.
Fix PR lib/57478
I've been tested this patch both on big-/little-endian and
LP64/ILP32 environments over a year without troubles.
But I don't understand why! It's looking more like a compiler bug!
When I look at the post-CPP output of the compiler for sha2.c I find
that it drops out of including <machine/bswap.h> and suddenly instead of
continuing in <sys/endian.h> it is jumping into <i386/endian.h> and then
on to <sys/types.h>!!!:
# 12 "/usr/include/i386/bswap.h" 2 3 4
# 110 "/usr/include/sys/endian.h" 2 3 4
# 4 "/usr/include/i386/endian.h" 2 3 4
# 99 "/usr/include/sys/types.h" 2 3 4
There is obviously no include of <machine/endian.h> or <i386/endian.h>
after line 100 of <sys/endian.h>! In fact there are no more #include
lines in <sys/endian.h> after the include of <machine/bswap.h>. Where
did the rest of <sys/endian.h> go?!?!??
That's from:
cc -E -I. -I./include -I/home/sysbuild/src/tools/compat -I/home/sysbuild/src/tools/compat/sys -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -I. -I./include -I/home/sysbuild/src/tools/compat -I/home/sysbuild/src/tools/compat/sys -DHAVE_NBTOOL_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D__DBINTERFACE_PRIVATE /home/sysbuild/src/tools/compat/../../common/lib/libc/hash/sha2/sha2.c 2>&1 | less
I'm at a loss.
--
Greg A. Woods <gwoods%acm.org@localhost>
Kelowna, BC +1 250 762-7675 RoboHack <woods%robohack.ca@localhost>
Planix, Inc. <woods%planix.com@localhost> Avoncote Farms <woods%avoncote.ca@localhost>
Attachment:
pgpGZgx3A3DCh.pgp
Description: OpenPGP Digital Signature