This may water down to my error.  I was flipping the '-o' off and on as a
way to get around waiting for the obj dirs to be tree'd.  It appears that I
didn't watch carefully for what was updated via cvs and new source came down
and the first build didn't create/require the obj dirs.

Since this I've been patiently waiting for the tree to get [re]built and
things have worked much better.

I did try to mount_union the src/xsrc tree's read-only (some 16 mounts) and
build that way, but my -current (2.99.10) system would hang every so often
for no apparent reason.  That's another PR for another day.

Sorry for the noise, thanks for the help,

peter

On 12/1/04 6:28 PM, "James Chacon" <jmc@NetBSD.org> wrote:

> On Wed, Nov 24, 2004 at 06:57:01PM +0000, peter@boku.net wrote:
>>> Number:         28414
>>> Category:       toolchain
>>> Synopsis:       -current Makefiles don't honor MAKEOBJDIRPREFIX
>>> Confidential:   no
>>> Severity:       non-critical
>>> Priority:       low
>>> Responsible:    toolchain-manager
>>> State:          open
>>> Class:          sw-bug
>>> Submitter-Id:   net
>>> Arrival-Date:   Wed Nov 24 18:57:01 +0000 2004
>>> Originator:     Peter Eisch
>>> Release:        Current
>>> Organization:
>>> Environment:
>> NetBSD gvlsite 2.0G NetBSD 2.0G (PETER-FW) #4: Sat Sep  4 14:42:02 CDT 2004
>> peter@gvlsite:/builds/current/i386/obj/builds/current/src/sys/arch/i386/compi
>> le/PETER-FW i386
>> 
>>> Description:
>> When cross-building -current on i386, after building i386 I proceed to build
>> sparc64.  This build fails with:
>> 
>> ===> build.sh command: /builds/current/src/build.sh -j 4 -u -o -U -x -M
>> /builds/current/sparc64/obj -m sparc64 -D /
>> builds/current/sparc64/dest -R /builds/current/sparc64/release -N 1 -T
>> /builds/current/sparc64/tools kernel=PETER-F
>> W releasekernel=PETER-FW release
> 
> How did you build your i386 build? If you didn't use objdirs there, it won't
> matter if you use -M on sparc64 as .o's in the source directory always have
> priority over any others. i.e. .o's sprinklked through your source tree and
> you will lose.
> 
> James
> 
>> ===> build.sh started: Tue Nov 23 23:06:25 CST 2004
>> ===> MACHINE:          sparc64
>> ===> MACHINE_ARCH:     sparc64
>> ===> TOOLDIR path:     /builds/current/sparc64/tools
>> ===> DESTDIR path:     /builds/current/sparc64/dest
>> ===> RELEASEDIR path:  /builds/current/sparc64/release
>> ===> makewrapper:      /builds/current/sparc64/tools/bin/nbmake-sparc64
>> ===> Updated /builds/current/sparc64/tools/bin/nbmake-sparc64
>> ===> Building kernel without building new tools
>> ===> Building kernel:  PETER-FW
>> ===> Build directory:
>> /builds/current/sparc64/obj/builds/current/src/sys/arch/sparc64/compile/PETE
>> R-FW
>> Build directory is
>> /builds/current/sparc64/obj/builds/current/src/sys/arch/sparc64/compile/PETE
>> R-FW
>> Don't forget to run "make depend"
>> depending the kern library objects
>> depending the compat library objects
>> ...
>> (Kernel builds fine)
>> ...
>> --- getent ---
>>        link  getent/getent
>> --- dependall-gzip ---
>> dependall ===> usr.bin/gzip
>> --- dependall-getent ---
>> /builds/current/sparc64/tools/lib/gcc-lib/sparc64--netbsd/3.3.3/../../../../
>> sparc64--netbsd/bin/ld: getent.o: Reloc
>> ations in generic ELF (EM: 3)
>> getent.o: could not read symbols: File in wrong format
>> collect2: ld returned 1 exit status
>> *** [getent] Error code 1
>> 1 error
>> 
>> nbmake: stopped in /builds/current/src/usr.bin/getent
>> *** [dependall] Error code 2
>> 1 error
>>        
>> nbmake: stopped in /builds/current/src/usr.bin/getent
>> *** [dependall-getent] Error code 2
>> --- dependall-head ---
>> dependall ===> usr.bin/head
>> --- dependall-fstat ---
>> --- fstat ---
>>        link  fstat/fstat
>> --- dependall-head ---
>> --- dependall ---
>> --- head --- 
>>        link  head/head
>> --- dependall-gzip ---
>> --- dependall ---
>> --- gzip ---
>>        link  gzip/gzip
>> 1 error
>> 
>> nbmake: stopped in /builds/current/src/usr.bin
>> 
>> *** Failed target:  dependall-usr.bin
>> *** Failed command: _makedirtarget() { dir="$1"; shift; target="$1"; shift;
>> case "${dir}" in /*) this="${dir}/"; re al="${dir}" ;; .) this="";
>> real="/builds/current/src" ;; *) this="${dir}/";
>> real="/builds/current/src/${dir}" ;; es
>> ac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+ (with: $@)}"; cd
>> "${real}" && /builds/current/sparc64/tool
>> s/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget usr.bin
>> dependall
>> *** Error code 2
>> 
>> Stop.
>> nbmake: stopped in /builds/current/src
>> *** [do-build] Error code 1
>> 1 error
>>        
>> nbmake: stopped in /builds/current/src
>> 
>> *** Failed target:  build
>> *** Failed command: _makedirtarget() { dir="$1"; shift; target="$1"; shift;
>> case "${dir}" in /*) this="${dir}/"; re
>> al="${dir}" ;; .) this=""; real="/builds/current/src" ;; *) this="${dir}/";
>> real="/builds/current/src/${dir}" ;; es
>> ac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+ (with: $@)}"; cd
>> "${real}" && /builds/current/sparc64/tool
>> s/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget .
>> do-build
>> *** Error code 2
>> 
>> Stop.
>> nbmake: stopped in /builds/current/src
>> *** [release] Error code 1
>> 
>> nbmake: stopped in /builds/current/src
>> 
>> 
>> -------
>> 
>> Looking in src I find:
>> 
>> gvlsite: {3} find . -name '*.o'
>> ./gnu/usr.sbin/sendmail/editmap/editmap.o
>> ./gnu/usr.sbin/sendmail/libmilter/comm.o
>> ./gnu/usr.sbin/sendmail/libmilter/engine.o
>> ./gnu/usr.sbin/sendmail/libmilter/errstring.o
>> ./gnu/usr.sbin/sendmail/libmilter/handler.o
>> ./gnu/usr.sbin/sendmail/libmilter/listener.o
>> ./gnu/usr.sbin/sendmail/libmilter/main.o
>> ./gnu/usr.sbin/sendmail/libmilter/signal.o
>> ./gnu/usr.sbin/sendmail/libmilter/sm_gethost.o
>> ./gnu/usr.sbin/sendmail/libmilter/smfi.o
>> ./gnu/usr.sbin/sendmail/libmilter/strl.o
>> ./gnu/usr.sbin/sendmail/libsm/strcasecmp.o
>> ./gnu/usr.sbin/sendmail/libsm/assert.o
>> ./gnu/usr.sbin/sendmail/libsm/cf.o
>> ./gnu/usr.sbin/sendmail/libsm/clock.o
>> ./gnu/usr.sbin/sendmail/libsm/clrerr.o
>> ./gnu/usr.sbin/sendmail/libsm/config.o
>> ./gnu/usr.sbin/sendmail/libsm/debug.o
>> ./gnu/usr.sbin/sendmail/libsm/errstring.o
>> ./gnu/usr.sbin/sendmail/libsm/exc.o
>> ./gnu/usr.sbin/sendmail/libsm/fclose.o
>> ./gnu/usr.sbin/sendmail/libsm/feof.o
>> ./gnu/usr.sbin/sendmail/libsm/ferror.o
>> ./gnu/usr.sbin/sendmail/libsm/fflush.o
>> ./gnu/usr.sbin/sendmail/libsm/fget.o
>> ./gnu/usr.sbin/sendmail/libsm/findfp.o
>> ./gnu/usr.sbin/sendmail/libsm/flags.o
>> ./gnu/usr.sbin/sendmail/libsm/fopen.o
>> ./gnu/usr.sbin/sendmail/libsm/fpos.o
>> ./gnu/usr.sbin/sendmail/libsm/fprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/fpurge.o
>> ./gnu/usr.sbin/sendmail/libsm/fput.o
>> ./gnu/usr.sbin/sendmail/libsm/fread.o
>> ./gnu/usr.sbin/sendmail/libsm/fscanf.o
>> ./gnu/usr.sbin/sendmail/libsm/fseek.o
>> ./gnu/usr.sbin/sendmail/libsm/fvwrite.o
>> ./gnu/usr.sbin/sendmail/libsm/fwalk.o
>> ./gnu/usr.sbin/sendmail/libsm/fwrite.o
>> ./gnu/usr.sbin/sendmail/libsm/get.o
>> ./gnu/usr.sbin/sendmail/libsm/heap.o
>> ./gnu/usr.sbin/sendmail/libsm/ldap.o
>> ./gnu/usr.sbin/sendmail/libsm/makebuf.o
>> ./gnu/usr.sbin/sendmail/libsm/match.o
>> ./gnu/usr.sbin/sendmail/libsm/mbdb.o
>> ./gnu/usr.sbin/sendmail/libsm/mpeix.o
>> ./gnu/usr.sbin/sendmail/libsm/niprop.o
>> ./gnu/usr.sbin/sendmail/libsm/path.o
>> ./gnu/usr.sbin/sendmail/libsm/put.o
>> ./gnu/usr.sbin/sendmail/libsm/refill.o
>> ./gnu/usr.sbin/sendmail/libsm/rewind.o
>> ./gnu/usr.sbin/sendmail/libsm/rpool.o
>> ./gnu/usr.sbin/sendmail/libsm/setvbuf.o
>> ./gnu/usr.sbin/sendmail/libsm/shm.o
>> ./gnu/usr.sbin/sendmail/libsm/signal.o
>> ./gnu/usr.sbin/sendmail/libsm/smstdio.o
>> ./gnu/usr.sbin/sendmail/libsm/snprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/sscanf.o
>> ./gnu/usr.sbin/sendmail/libsm/stdio.o
>> ./gnu/usr.sbin/sendmail/libsm/strdup.o
>> ./gnu/usr.sbin/sendmail/libsm/strerror.o
>> ./gnu/usr.sbin/sendmail/libsm/strexit.o
>> ./gnu/usr.sbin/sendmail/libsm/string.o
>> ./gnu/usr.sbin/sendmail/libsm/stringf.o
>> ./gnu/usr.sbin/sendmail/libsm/strio.o
>> ./gnu/usr.sbin/sendmail/libsm/strl.o
>> ./gnu/usr.sbin/sendmail/libsm/strrevcmp.o
>> ./gnu/usr.sbin/sendmail/libsm/strto.o
>> ./gnu/usr.sbin/sendmail/libsm/syslogio.o
>> ./gnu/usr.sbin/sendmail/libsm/test.o
>> ./gnu/usr.sbin/sendmail/libsm/ungetc.o
>> ./gnu/usr.sbin/sendmail/libsm/vasprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/vfprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/vfscanf.o
>> ./gnu/usr.sbin/sendmail/libsm/vprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/wbuf.o
>> ./gnu/usr.sbin/sendmail/libsm/vsnprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/vsprintf.o
>> ./gnu/usr.sbin/sendmail/libsm/vsscanf.o
>> ./gnu/usr.sbin/sendmail/libsm/wsetup.o
>> ./gnu/usr.sbin/sendmail/libsm/xtrap.o
>> ./gnu/usr.sbin/sendmail/libsmdb/smndbm.o
>> ./gnu/usr.sbin/sendmail/libsmdb/smdb.o
>> ./gnu/usr.sbin/sendmail/libsmdb/smdb1.o
>> ./gnu/usr.sbin/sendmail/libsmdb/smdb2.o
>> ./gnu/usr.sbin/sendmail/libsmutil/lockfile.o
>> ./gnu/usr.sbin/sendmail/libsmutil/cf.o
>> ./gnu/usr.sbin/sendmail/libsmutil/debug.o
>> ./gnu/usr.sbin/sendmail/libsmutil/err.o
>> ./gnu/usr.sbin/sendmail/libsmutil/safefile.o
>> ./gnu/usr.sbin/sendmail/libsmutil/snprintf.o
>> ./gnu/usr.sbin/sendmail/mailstats/mailstats.o
>> ./gnu/usr.sbin/sendmail/makemap/makemap.o
>> ./gnu/usr.sbin/sendmail/praliases/praliases.o
>> ./gnu/usr.sbin/sendmail/sendmail/alias.o
>> ./gnu/usr.sbin/sendmail/sendmail/arpadate.o
>> ./gnu/usr.sbin/sendmail/sendmail/bf.o
>> ./gnu/usr.sbin/sendmail/sendmail/collect.o
>> ./gnu/usr.sbin/sendmail/sendmail/conf.o
>> ./gnu/usr.sbin/sendmail/sendmail/control.o
>> ./gnu/usr.sbin/sendmail/sendmail/convtime.o
>> ./gnu/usr.sbin/sendmail/sendmail/daemon.o
>> ./gnu/usr.sbin/sendmail/sendmail/deliver.o
>> ./gnu/usr.sbin/sendmail/sendmail/domain.o
>> ./gnu/usr.sbin/sendmail/sendmail/envelope.o
>> ./gnu/usr.sbin/sendmail/sendmail/err.o
>> ./gnu/usr.sbin/sendmail/sendmail/headers.o
>> ./gnu/usr.sbin/sendmail/sendmail/macro.o
>> ./gnu/usr.sbin/sendmail/sendmail/main.o
>> ./gnu/usr.sbin/sendmail/sendmail/map.o
>> ./gnu/usr.sbin/sendmail/sendmail/mci.o
>> ./gnu/usr.sbin/sendmail/sendmail/milter.o
>> ./gnu/usr.sbin/sendmail/sendmail/mime.o
>> ./gnu/usr.sbin/sendmail/sendmail/parseaddr.o
>> ./gnu/usr.sbin/sendmail/sendmail/queue.o
>> ./gnu/usr.sbin/sendmail/sendmail/readcf.o
>> ./gnu/usr.sbin/sendmail/sendmail/recipient.o
>> ./gnu/usr.sbin/sendmail/sendmail/sasl.o
>> ./gnu/usr.sbin/sendmail/sendmail/savemail.o
>> ./gnu/usr.sbin/sendmail/sendmail/sfsasl.o
>> ./gnu/usr.sbin/sendmail/sendmail/shmticklib.o
>> ./gnu/usr.sbin/sendmail/sendmail/sm_resolve.o
>> ./gnu/usr.sbin/sendmail/sendmail/srvrsmtp.o
>> ./gnu/usr.sbin/sendmail/sendmail/stab.o
>> ./gnu/usr.sbin/sendmail/sendmail/stats.o
>> ./gnu/usr.sbin/sendmail/sendmail/sysexits.o
>> ./gnu/usr.sbin/sendmail/sendmail/timers.o
>> ./gnu/usr.sbin/sendmail/sendmail/tls.o
>> ./gnu/usr.sbin/sendmail/sendmail/trace.o
>> ./gnu/usr.sbin/sendmail/sendmail/usersmtp.o
>> ./gnu/usr.sbin/sendmail/sendmail/util.o
>> ./gnu/usr.sbin/sendmail/sendmail/version.o
>> ./gnu/usr.sbin/sendmail/smrsh/smrsh.o
>> ./lib/csu/i386_elf/crtend.o
>> ./lib/csu/i386_elf/crtendS.o
>> ./lib/csu/i386_elf/crtbeginS.o
>> ./lib/csu/i386_elf/crtbegin.o
>> ./regress/lib/libutil/sockaddr_snprintf/sockaddr_snprintf.o
>> ./usr.bin/kdump/ioctl.o
>> ./usr.bin/getent/getent.o
>> gvlsite: {4} 
>> 
>> 
>> Which likely explains the issue.
>>> How-To-Repeat:
>> Use build args like:
>> 
>> ===> build.sh command: /builds/current/src/build.sh -j 4 -u -o -U -x -M
>> /builds/current/sparc64/obj -m sparc64 -D /builds/current/sparc64/dest -R
>> /builds/current/sparc64/release -N 1 -T /builds/current/sparc64/tools
>> kernel=PETER-FW releasekernel=PETER-FW release
>> 
>> 
>> (the kernel args are inconsequential, of course)
>> 
>>> Fix:
>> Workaround:
>> 
>> Prior to doing build, run a command to remove the lingering .o's:
>> 
>> /bin/rm -f `find /builds/current/src -name '*.o'`
>> 
>> 
>> 
>