Subject: Re: pkgsrc on SMP machines
To: Alan Post <email@example.com>
From: Lars Nordlund <firstname.lastname@example.org>
Date: 12/16/2005 19:44:53
On Thu, 15 Dec 2005 03:16:55 +0000 (UTC)
Alan Post <email@example.com> wrote:
> In article <firstname.lastname@example.org>, email@example.com wrote:
> > Hi, can anyone suggest a good strategy for utilising the most of SMP
> > hardware when building software from pkgsrc??
> > I have Sun E450 with 4X 250MHz UltraSparc II CPUs, most of the time
> > the system is 70% idle, because pkgsrc tends to use mainly 1 CPU.
> > Fixing all Makefiles to support -j > 1 seems quite unreasonable, I
> > guess the best solution would be to have a multithreaded compiler,
> > which distributes workload to a specified number of threads, i.e
> Alternatively, if you're building a bunch of packages, pkgsrc could
> run multiple builds concurrently, since it has all the dependency
> information. I seem to recall reading on this list about an
> implementation of this in the past.
Yes, this is the way to do it. Reason has been posted to the list
before. Hint: take a look at how the files work/.wrapper/tmp/cache-*
are handled during the build of any package. If multiple gcc/g+
+/libtool/etc. wrappers are reading/changing these files at the same
time (as would happen with MAKE_FLAG+=-j X), the build will fail.
So, how can it work in portage? Well, portage do not have the wrapper
framework that pkgsrc has. I suppose portage requires that the whole
machine is correctly configured/installed during all package builds.
pkgsrc does not have this requirement since it wraps all calls to the
compiler/linker and changes all paths to point into the work/.buildlink