Subject: Re: pkgsrc on SMP machines
To: Hubert Feyrer <feyrer@cs.stevens.edu>
From: Dieter Baron <dillo@danbala.tuwien.ac.at>
List: tech-pkg
Date: 12/15/2005 12:20:20
In article <Pine.LNX.4.61.0512150434360.12122@m24s24.vlinux.de> Hubert wrote:
: On Thu, 15 Dec 2005 segv@netctl.net wrote:
: > Hi, can anyone suggest a good strategy for utilising the most of SMP hardware
: > when building software from pkgsrc??
: ...
: > Fixing all Makefiles to support -j > 1 seems quite unreasonable, I guess the
: > best solution would be to have a multithreaded compiler, which distributes
: > workload to a specified number of threads, i.e.:
: How about compiling the pkgs themselves (i.e. whatever gets extracted from
: sources) with 'make -j', e.g. as the patch below does. At least that's how
: Unix is supposed to work...
Not all packages' build system support this, many probably fail (or
become unreliable) when make is invoked with -j.
: Of course that's a crude hack, and a real solution would involving a
: user-settable variable plus a per-package maximum, and the real value
: would be the somewhere in between there.
If we want to support this, we would need a variable in package
Makefiles that tell wether this package can use -j, defaulting to no
(a working build is more important than performance improvements). We
could then test individual packages (beginning with popular big ones)
and enable it there.
: (There is that other idea to build pkgsrc packages in parallel which is
: stuck to some people's head, but it grossly violates the KISS principle
: and involves all sort of nasty things like locking and whatnot; I still
: suggest not going there)
We need locking even if we do not build dependencies in parallel:
Users can invoke multiple makes simultaneously in the same pkgsrc
tree.
yours,
dillo