toolchain/57584: Running build.sh with -N 3 or -N 4 causes debug logging to be included in the MKVARS shell variable in distrib/sets/sets.subr

[palle%lyckegaard.dk@localhost]

>Number:         57584
>Category:       toolchain
>Synopsis:       Running build.sh with -N 3 or -N 4 causes debug logging to be included in the MKVARS shell variable in distrib/sets/sets.subr
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    toolchain-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Aug 13 18:00:00 +0000 2023
>Originator:     Palle Lyckegaard
>Release:        current
>Organization:
NetBSD
>Environment:
SunOS instance-20230726-2011 5.11 11.4.42.111.0 i86pc i386 i86pc kvm

>Description:
Doing a 'release' build on a Solaris 11.4host (x86):

The folowing error occurs:

#    create  set lists
cd /export/home/opc/git/netbsd_src/distrib/sets &&  DESTDIR=/export/home/opc/data/netbsd/build/sparc64/destdir  MACHINE=sparc64  MACHINE_ARCH=sparc64  AWK=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbawk  CKSUM=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbcksum  DB=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbdb  EGREP=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbgrep\ -E  HOST_SH=/usr/bin/bash  MAKE=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbmake  MKTEMP=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbmktemp  MTREE=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbmtree  PAX=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbpax  COMPRESS_PROGRAM=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbxz  GZIP=-n  XZ_OPT=-9  TAR_SUFF=tar.xz  PKG_CREATE=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbpkg_create  SED=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbsed  TSORT=/export/
 home/opc/data/netbsd/build/sparc64/tooldir/bin/nbtsort\ -q  /usr/bin/bash ./maketars -S -d /export/home/opc/data/netbsd/build/sparc64/destdir   -M /export/home/opc/data/netbsd/build/sparc64/destdir/METALOG.sanitised  -N /export/home/opc/git/netbsd_src/etc -L base -t /export/home/opc/data/netbsd/build/sparc64/reldir/sparc64/binary/sets
Creating flist.base
/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbawk: non-terminated string echo MKARG... at source line 21
 context is
        MKOBJDIRS echo MKPCC MKPCC echo MKPICINSTALL MKPICINSTALL echo MKPIGZGZIP MKPIGZGZIP echo MKRADEONFIRMWARE MKRADEONFIRMWARE echo MKREPRO MKREPRO echo MKSLJIT MKSLJIT echo MKSOFTFLOAT MKSOFTFLOAT echo MKSTRIPIDENT MKSTRIPIDENT echo MKTEGRAFIRMWARE MKTEGRAFIRMWARE echo MKTPM MKTPM echo MKUNPRIVED >>> 
 <<< MKUNPRIVED echo MKUPDATE MKUPDATE echo MKX11 MKX11 echo MKX11MOTIF MKX11MOTIF echo MKXORG_SERVER MKXORG_SERVER echo MKZFS MKZFS echo ARCH64 ARCH64 echo EABI EABI echo HAVE_ACPI HAVE_ACPI echo HAVE_BINUTILS HAVE_BINUTILS echo HAVE_GCC HAVE_GCC echo HAVE_GDB HAVE_GDB echo HAVE_LIBGCC_EH HAVE_LIBGCC_EH echo HAVE_MESA_VER HAVE_MESA_VER echo HAVE_NVMM HAVE_NVMM echo HAVE_OPENSSL HAVE_OPENSSL echo HAVE_SSP HAVE_SSP echo HAVE_UEFI HAVE_UEFI echo HAVE_XORG_GLAMOR HAVE_XORG_GLAMOR echo HAVE_XORG_SERVER_VER HAVE_XORG_SERVER_VER echo KERNEL_DIR KERNEL_DIR echo MACHINE MACHINE echo MACHINE_ARCH MACHINE_ARCH echo MACHINE_CPU MACHINE_CPU echo MAKEVERBOSE MAKEVERBOSE echo MKCOMPAT MKCOMPAT echo MKCOMPATMODULES MKCOMPATMODULES echo MKMANPAGES MKMANPAGES echo MKSTATICPIE MKSTATICPIE echo MKXORG MKXORG echo NETBSDSRCDIR NETBSDSRCDIR echo OBJECT_FMT OBJECT_FMT echo TARGET_ENDIANNESS TARGET_ENDIANNESS echo TOOLCHAIN_MISSING TOOLCHAIN_MISSING echo USE_INET6 USE_INET6 echo USE_KERBEROS USE_KERBEROS e
 cho USE_LDAP USE_LDAP echo USE_YP USE_YP echo COMPATARCHDIRS COMPATARCHDIRS echo KMODARCHDIRS KMODARCHDIRS echo MKSOLARIS MKSOLARIS", needvars)
/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbawk: non-terminated string echo MKARG... at source line 22
/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbawk: non-terminated string echo MKARG... at source line 23
/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbawk: giving up
 source line number 235
xargs: Child killed with signal 13
makeflist output is empty for base

*** Failed target: makesetfiles
*** Failed commands:
        ${_MKMSG_CREATE} "set lists"
        => @echo '#  ' " create " "set lists"
        ${SETSCMD} ./maketars -S -d ${DESTDIR:S,^$,/,} ${MAKETARS_FLAGS}  ${METALOG.unpriv}  -N ${NETBSDSRCDIR}/etc ${MAKEFLIST_FLAGS} -t ${TARDIR}
        => cd /export/home/opc/git/netbsd_src/distrib/sets &&  DESTDIR=/export/home/opc/data/netbsd/build/sparc64/destdir  MACHINE=sparc64  MACHINE_ARCH=sparc64  AWK=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbawk  CKSUM=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbcksum  DB=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbdb  EGREP=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbgrep\ -E  HOST_SH=/usr/bin/bash  MAKE=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbmake  MKTEMP=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbmktemp  MTREE=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbmtree  PAX=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbpax  COMPRESS_PROGRAM=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbxz  GZIP=-n  XZ_OPT=-9  TAR_SUFF=tar.xz  PKG_CREATE=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbpkg_create  SED=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbsed  TSO
 RT=/export/home/opc/data/netbsd/build/sparc64/tooldir/bin/nbtsort\ -q  /usr/bin/bash ./maketars -S -d /export/home/opc/data/netbsd/build/sparc64/destdir   -M /export/home/opc/data/netbsd/build/sparc64/destdir/METALOG.sanitised  -N /export/home/opc/git/netbsd_src/etc -L base -t /export/home/opc/data/netbsd/build/sparc64/reldir/sparc64/binary/sets
*** [makesetfiles] Error code 1

nbmake[3]: stopped in /export/home/opc/git/netbsd_src/distrib/sets
1 error

nbmake[3]: stopped in /export/home/opc/git/netbsd_src/distrib/sets

nbmake[2]: stopped in /export/home/opc/git/netbsd_src/etc

nbmake[1]: stopped in /export/home/opc/git/netbsd_src

nbmake: stopped in /export/home/opc/git/netbsd_src

ERROR: Failed to make release

*** BUILD ABORTED ***



It looks like the MKVARS shell variable is polluted with the debug 'echo' commands found in distrib/sets/mkvars.mk (mkvarsyesno target).

Doing a similar build on a NetBSD host shows same issue, but NetBSD is apparently able to cope with it, whereas Solaris isn't (don't know why).

 

>How-To-Repeat:
Issue a build.sh release on a Solaris 11.4 host with N set to 3 or 4.
When N is 0, 1 or 2 the build is ok.




>Fix:
I'm not sure how to fix the issue properly.

I'm considering a fix to the MKVARS assignment in sets.subr so the SED command replaces the extra lines with 'echo' with nothing effectively removing the lines.
But I'm unsure if this is the proper fix.