toolchain/42495: Corrupt db files created by cross-built wd_mkdb and cap_mkdb

[mcl%gmx.us@localhost]

>Number:         42495
>Category:       toolchain
>Synopsis:       Corrupt db files created by cross-built wd_mkdb and cap_mkdb
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    toolchain-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Dec 22 15:40:00 +0000 2009
>Originator:     Mark Lang
>Release:        5.0.1
>Organization:
>Environment:
Linux 2.6.26-2-powerpc64 #1 SMP ppc64 GNU/Linux
>Description:
I am cross-compiling a NetBSD 5.0.1 system.  The target arch is i386.  The 
build host arch is Debian Linux powerpc64 ('lenny').  Everything built without 
errors:

 ./build.sh -U -j2 tools
 ./build.sh -u -U -j 2 kernel=SOEKRIS_4801
 ./build.sh -u -U -j 2 build
 ./build.sh -u -U distribution
 ./build.sh -U -u sets

I un-tarred the sets into an empty directory on the build host which is the 
NFS-mounted rootfs for the netbooting target machine (a Soekris 4801).  After 
booting the just-built system on the Soekris, I observed the following problems:

/etc/pwd.db is 260MB.  Any operations on it emit an error message.
/etc/spwd.db is likewise 260MB

These anomalies are in the $DESTDIR files, and consequently the sets:

$ tar tvzf release/i386/binary/sets/etc.tgz | grep pwd
-rw-r--r-- root/wheel 268712332 2009-12-20 15:08 ./etc/pwd.db
-rw-r--r-- root/wheel 268703884 2009-12-20 15:08 ./etc/spwd.db

Being more interested in getting my Soekris system working than figuring out 
what went wrong, I worked around this by replacing those two files with proper 
ones from another i386 NetBSD 5.0 box, and then updating them with pwd_mkdb.

However, further investigation revealed additional corrupt db files in the 
target filesystem (and in the $DESTDIR):

$ ls -l rootfs/usr/share/misc/ | fgrep db
-r--r--r-- 1 root root 269115652 2009-12-20 10:26 termcap.db
-r--r--r-- 1 root root 268599556 2009-12-20 10:19 vgrindefs.db

As an experiment, on the Debian powerpc machine, I then manually ran the 
toolchain nbcap_mkdb using essentially the same command as is found in 
src/usr.bin/vgrind/Makefile:

$ toolchain/bin/nbcap_mkdb -l -f vgrindefs.src src/usr.bin/vgrind/vgrindefs.src

And see that it creates a 260MB db file:

$ ls -l vgrindefs.src.db
-rw-r--r-- 1 mcl mcl 268599556 2009-12-22 08:48 vgrindefs.src.db

And which is corrupt.  On the target machine:

soekris# db hash /usr/share/misc/vgrindefs.db
db: Opening database `/usr/share/misc/vgrindefs.db': Inappropriate file type or 
format

This all seems to indicate that the nbcap_mkdb and nbpwd_mkdb executables have 
bugs when they are cross-compiled (at least, when the build machine is Linux 
powerpc).

$ file toolchain/bin/nbcap_mkdb
toolchain/bin/nbcap_mkdb: ELF 32-bit MSB executable, PowerPC or cisco 4500, 
version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, 
with unknown capability 0x41000000 = 0xf676e75, not stripped

$ ldd toolchain/bin/nbcap_mkdb
        linux-vdso32.so.1 =>  (0x00100000)
        libz.so.1 => /usr/lib/libz.so.1 (0x0ffcb000)
        libc.so.6 => /lib/libc.so.6 (0x0fe45000)
        /lib/ld.so.1 (0xf7fce000)

At the end of the day: it is not possible to successfully cross-build an 
error-free complete NetBSD 5.0.1 i386 system using a Linux powerpc system as 
the build host.
>How-To-Repeat:

>Fix: