Subject: Re: Package building with make -j 2
To: Eric Delcamp <e.delcamp@wanadoo.fr>
From: Simon J. Gerraty <sjg@crufty.net>
List: current-users
Date: 07/24/2003 00:29:36
>I have tried to build some packages with include a MAKEFLAGS "-j 2" in my env
>variables. Things really go bad, because make dont sync the 2 threads ;) (by
>example, it try to patch the source before creating the "work" directory).
>Is there a feature inside Makefile to sync theses actions ? Or should I just
>forget the -j 2 arg when building packages ?
The following is to help you decide ... ;-)
Correct building with -j requires that all the dependencies be excplicitly
stated. When make runs in compat mode, dependencies are built in the
order listed, that is not guaranteed in jobs mode (-jN). So given:
all: one two three
with compat mode, one will be built before two which will be followed
by three.
with -j2 though, one and two will be started in parallel and as soon
as one of them completes three will start.
If there is in fact a dependency b/w two and three it either needs to
be explicitly stated in the makefile, or the sort-cut .WAIT put between
them. As in:
all: one two .WAIT three
Simply using -j1 will typically show up a lot of dependency issues,
without the the confusing output that you get from -j2 or more.
You can achieve a good dose of parallelism by starting off with
make -j4 say, but running sub-makes with -j1 -B, which might help
you build 4 dependent packages in parallel, but each one would be
built in compat mode. A bit of handwaving to be sure ;-) but I've
used that techique to good effect on multi-cpu boxes (not building
pkgsrc).
--sjg