Subject: Re: pkgsrc on SMP machines
To: None <segv@netctl.net>
From: Lars Nordlund <lars.nordlund@hem.utfors.se>
List: tech-pkg
Date: 12/16/2005 19:45:02
On Thu, 15 Dec 2005 03:54:33 +0000
segv@netctl.net wrote:
> > Alternatively, if you're building a bunch of packages, pkgsrc could
> > run multiple builds concurrently, since it has all the dependency
> > information.  I seem to recall reading on this list about an
> > implementation of this in the past.
> > 
> >   Alan
> > 
> > 
> 
> The downside of this apporach is: if you have a large package like www/firefox,
> it won't be able to take advantage for multiple CPUs

A fair tradeoff, given that the alternative is no SMP support at all.


Well, hmm.. I suppose it could be possible to change the buildlink
wrappers to not update the cache files during the build, and instead
everything from scratch at every invocation. However, rumours say that
it costs an awful amount of CPU to do that. Anyone care to comment on
this?

Ok, found a variable which seems to control the cache behaviour;
WRAPPER_UPDATE_CACHE. I set it to 'no' in mk.conf and did some tests on
archivers/zoo:

With cache:    12 sec build time
Without cache: 16 sec build time

There goes the (already not so) quick build times..

I do not have access to an SMP machine at the moment to do serious
testing on, but I suppose 'MAKE_FLAGS+=-j N' have a much bigger hit
rate with the cache turned off. And of course all packages must be
tried and marked SMP safe like portage has done. And it will be really
hard to guarantee 100% multi-job safety.


Best regards,
	Lars Nordlund