Re: header installation not make-jobs safe?

[Thomas Klausner <wiz%NetBSD.org@localhost>]

On Sat, Oct 12, 2024 at 10:30:55AM +0200, Thomas Klausner wrote:
> On Thu, Apr 11, 2024 at 05:19:07PM +0200, Thomas Klausner wrote:
> > Hi!
> > 
> > I had an interesting build failure today when using
> > 
> > build.sh -j 32 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -T /usr/obj/tools.gcc -m amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.gcc.20240411 -R /usr/obj/amd64.gcc.20240411.release distribution
> > 
> > The build stopped quite early with:
> > 
> > --- /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h ---
> > *** Failed target: /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h
> > *** Failed commands:
> >         @cmp -s ${.ALLSRC} ${.TARGET} > /dev/null 2>&1 ||  (${_MKSHMSG_INSTALL} ${.TARGET};  ${_MKSHECHO} "${INSTALL_FILE} -o ${BINOWN} -g ${BINGRP}  -m ${NONBINMODE} ${.ALLSRC} ${.TARGET}" &&  ${INSTALL_FILE} -o ${BINOWN} -g ${BINGRP}  -m ${NONBINMODE} ${.ALLSRC} ${.TARGET})
> >         => @cmp -s krb5_asn1.h /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h > /dev/null 2>&1 ||  (echo '#  ' "install " /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h;  echo "/usr/obj/tools.gcc/bin/x86_64--netbsd-install  -N /disk/storage-202004/archive/foreign/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h" &&  /usr/obj/tools.gcc/bin/x86_64--netbsd-install  -N /disk/storage-202004/archive/foreign/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h)
> > *** [/usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h] Error code 1
> > nbmake[4]: stopped in /disk/storage-202004/archive/foreign/src/crypto/external/bsd/heimdal/lib/libasn1
> > 
> > A second try of the same command on the same machine with the same
> > sources succeeded.
> > 
> > Full log available on request.
> 
> This bug is still there, just saw it again. I have a new log.

Reading 'make -n includes' in src/crypto/external/bsd/heimdal/lib/libasn1, I see:

/usr/obj/tools/bin/nbasn1_compile                        --one-code-file                 --option-file=/disk/storage-202004/archive/foreign/src/crypto/external/bsd/heimdal/dist/lib/asn1/krb5.opt       /disk/storage-202004/archive/foreign/src/crypto/external/bsd/heimdal/dist/lib/asn1/krb5.asn1 krb5_asn1
/usr/obj/tools/bin/nbsed -E                                              -e 's,#include <(.*)_asn1\.h>,#include <krb5/\1_asn1.h>,'       2> /dev/null < krb5_asn1.hx > krb5_asn1.h
...
cmp -s krb5_asn1.h /usr/include/krb5/krb5_asn1.h > /dev/null 2>&1 ||  (echo '#  ' "install " /usr/include/krb5/krb5_asn1.h;  echo "/usr/obj/tools/bin/x86_64--netbsd-install  -N /disk/storage-202004/archive/foreign/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/include/krb5/krb5_asn1.h" &&  /usr/obj/tools/bin/x86_64--netbsd-install  -N /disk/storage-202004/archive/foreign/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/include/krb5/krb5_asn1.h)


I tried reading src/crypto/external/bsd/heimdal/Makefile.rules.inc but
I couldn't find the dependency that states that krb5_asn1.h is a file
that is generated by the (sed after the) nbasn1_compile command.

AFAICT, this is the block that's generating the header file:

.if exists(${src:.asn1=.opt})
${src:.asn1=.ts}: ${src} ${src:.asn1=.opt} ${TOOL_ASN1_COMPILE}
        @touch $@
        ${TOOL_ASN1_COMPILE}                    \
                ${ASN1_OPTS.${src}}             \
                --option-file=${.ALLSRC:[2]}    \
                ${.ALLSRC:[1]} ${src:.asn1=_asn1}
        @${TOOL_SED} -E                                                 \
            -e 's,#include <(.*)_asn1\.h>,#include <krb5/\1_asn1.h>,'   \
            2> /dev/null < ${src:.asn1=_asn1.hx} > ${src:.asn1=_asn1.h}
        @cmp -s ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}    \
            2> /dev/null ||                                             \
            cp ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}
.else
${src:.asn1=.ts}: ${src} ${TOOL_ASN1_COMPILE}
        @touch $@
        ${TOOL_ASN1_COMPILE}                    \
                ${ASN1_OPTS.${src}}             \
                ${.ALLSRC:[1]} ${src:.asn1=_asn1}
        @${TOOL_SED} -E                                                 \
            -e 's,#include <(.*)_asn1\.h>,#include <krb5/\1_asn1.h>,'   \
            2> /dev/null < ${src:.asn1=_asn1.hx} > ${src:.asn1=_asn1.h}
        @cmp -s ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}    \
            2> /dev/null ||                                             \
            cp ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}
.endif

and it only has the .ts file on the left hand side of the dependency.

Can someone please fix this?

Thanks,
 Thomas