Subject: Re: pkgsrc on SMP machines
To: Alan Post <apost@recalcitrant.org>
From: Lars Nordlund <lars.nordlund@hem.utfors.se>
List: tech-pkg
Date: 12/16/2005 19:44:53
On Thu, 15 Dec 2005 03:16:55 +0000 (UTC)
Alan Post <apost@recalcitrant.org> wrote:
> In article <20051215025259.57449f5c.segv@netctl.net>, segv@netctl.net wrote:
> > Hi, can anyone suggest a good strategy for utilising the most of SMP
> > hardware when building software from pkgsrc??
> > 
> > I have Sun E450 with 4X 250MHz UltraSparc II CPUs, most of the time
> > the system is 70% idle, because pkgsrc tends to use mainly 1 CPU.
> > 
> > Fixing all Makefiles to support -j > 1 seems quite unreasonable, I
> > guess the best solution would be to have a multithreaded compiler,
> > which distributes workload to a specified number of threads, i.e
> 
> Alternatively, if you're building a bunch of packages, pkgsrc could
> run multiple builds concurrently, since it has all the dependency
> information.  I seem to recall reading on this list about an
> implementation of this in the past.

Yes, this is the way to do it. Reason has been posted to the list
before. Hint: take a look at how the files work/.wrapper/tmp/cache-*
are handled during the build of any package. If multiple gcc/g+
+/libtool/etc. wrappers are reading/changing these files at the same
time (as would happen with MAKE_FLAG+=-j X), the build will fail.

So, how can it work in portage? Well, portage do not have the wrapper
framework that pkgsrc has. I suppose portage requires that the whole
machine is correctly configured/installed during all package builds.
pkgsrc does not have this requirement since it wraps all calls to the
compiler/linker and changes all paths to point into the work/.buildlink
directories.


Best regards,
	Lars Nordlund