Subject: Re: Package building with make -j 2
To: Eric Delcamp <e.delcamp@wanadoo.fr>
From: Simon J. Gerraty <sjg@crufty.net>
List: current-users
Date: 07/24/2003 00:29:36
>I have tried to build some packages with include a MAKEFLAGS "-j 2" in my env
>variables. Things really go bad, because make dont sync the 2 threads ;) (by
>example, it try to patch the source before creating the "work" directory).
>Is there a feature inside Makefile to sync theses actions ? Or should I just
>forget the -j 2 arg when building packages ?

The following is to help you decide ... ;-)

Correct building with -j requires that all the dependencies be excplicitly 
stated.  When make runs in compat mode, dependencies are built in the
order listed, that is not guaranteed in jobs mode (-jN).  So given:

all: one two three

with compat mode, one will be built before two which will be followed 
by three.
with -j2 though, one and two will be started in parallel and as soon 
as one of them completes three will start.

If there is in fact a dependency b/w two and three it either needs to 
be explicitly stated in the makefile, or the sort-cut .WAIT put between 
them.  As in:

all: one two .WAIT three

Simply using -j1 will typically show up a lot of dependency issues,
without the the confusing output that you get from -j2 or more.

You can achieve a good dose of parallelism by starting off with
make -j4 say, but running sub-makes with -j1 -B, which might help
you build 4 dependent packages in parallel, but each one would be
built in compat mode.  A bit of handwaving to be sure ;-) but I've
used that techique to good effect on multi-cpu boxes (not building
pkgsrc).

--sjg