Subject: Re: Simultaneous builds
To: Alistair Crooks <agc@pkgsrc.org>
From: Greg A. Woods <woods@weird.com>
List: tech-pkg
Date: 01/15/2002 13:18:45
[ On Tuesday, January 15, 2002 at 15:35:00 (+0100), Alistair Crooks wrote: ]
> Subject: Re: Simultaneous builds
>
> You may have noticed that I made some modifications to bsd.pkg.mk
> to append the first component of the hostname to a WRKDIR, if
> OBJHOSTNAME is defined in /etc/mk.conf. I've also committed a
> package for the shlock utility, which can be used on Operating
> Systems which don't have it as standard (and here I've guessed
> at Solaris and Linux - unfortunately I can't connect to the
> Darwin host right now to check).

There's no need to add such complication to bsd.pkg.mk.

I talked to David about this the other day by phone, but I didn't
realise at the time the combination of shlock and the mechanism I use
would allow both simultaneous builds on one host, as well as
simultaneous builds on many hosts, all from one source tree, and all the
while never requiring shlock to know about any other host.

All you need to do (I think -- I have many changes in my bsd.pkg.mk, but
IIRC this is the only relevant one) is make the following change:

***************
*** 2506,2514 ****
                fi;                                                     \
        fi
  .  ifdef WRKOBJDIR
!       -${_PKG_SILENT}${_PKG_DEBUG}                                    \
!       ${RMDIR} ${BUILD_DIR} 2>/dev/null;                              \
!       ${RM} -f ${WRKDIR_BASENAME}
  .  endif
  .endif
  
--- 2597,2603 ----
                fi;                                                     \
        fi
  .  ifdef WRKOBJDIR
!       -${_PKG_SILENT}${_PKG_DEBUG}${RMDIR} ${BUILD_DIR} 2>/dev/null;
  .  endif
  .endif
  

I.e. never, normally, remove the WRKOBJDIR symlink in "make clean".

Then by setting WRKOBJDIR to a host specific location you're done:

	# if you want to build on a (fast(er)) local disk...
	#
	# This can be a symlink.
	#
	WRKOBJDIR=              /var/package-obj


Note that you DO NOT EVER want to use the most broken concept of
OBJMACHINE either (not with pkgsrc, nor with /usr/src).  You really do
only want one and only one "work" ("obj") symlink in the source tree.

For the same reason you really never want to use your new OBJHOSTNAME
feature.  It's a totally unnecessary complication.

Flexibility is best done with one simple "work" ("obj") symlink and then
optionally further host-specific redirection with another unique symlink
wherever that points to, for eg. I have /var/package-obj -> obj/package
so that I can have one /var/obj partition on my build machine and yet
share the definition of WRKOBJDIR for my shared /usr/pkgsrc.

Complete removal of OBJMACHINE from all of the build stuff for both
/usr/pkgsrc and /usr/src would simplify things greatly and may even
improve performance.  You sure as heck don't want to have to ship a
/usr/src CD-ROM with 50-odd obj.${MACHINE} symlinks!  USR_OBJMACHINE
does all that's necessary, even for cross-builds on the same host
(USR_OBJMACHINE support might be needed in pkgsrc if pkgsrc were to ever
support cross-builds).

I've been using the above patch for many months now and am very happy
with it in my environment where I NFS share /usr/pkgsrc amongst many
machines of different architectures.  I've never wanted to do
simultaneous builds of packages on one host as David has, but with my
scheme all that's necessary is a plain dumb "shlock" hook to keep things
from colliding.

One could in theory write a "make wrkobjdir" target for pkgsrc, but
that'd only be necessary if you wanted to include the symlinks in a
read-only copy (eg. CD-ROM, "mount -t nfs -r", etc.) of the tree.  I've
not bothered to do that yet myself.  Similarly a "make clean-wrkobjdir"
(or whatever "make cleandir"-like) target could be written to remove the
WRKOBJDIR links (in case you ever have to change them, or whatever).

-- 
								Greg A. Woods

+1 416 218-0098;  <gwoods@acm.org>;  <g.a.woods@ieee.org>;  <woods@robohack.ca>
Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>