Re: Conceptual talk about distributed pbulk on exotic port

[Jonathan Perkin <jperkin%joyent.com@localhost>]

* On 2018-07-08 at 09:42 BST, Joerg Sonnenberger wrote:

> On Sun, Jul 08, 2018 at 08:57:01AM +0100, Jonathan Perkin wrote:
> > You will always get better throughput with multiple build processes
> > running single threaded than a single build process with a high
> > MAKE_JOBS, assuming you're building more than a couple of packages.
> 
> I don't completely agree with this, it is misguiding. It is desirable to
> have a medium high MAKE_JOBS, since that will benefit some of the very
> heavy packages like webkit. Rule of thumb is number of instances *
> MAKE_JOBS should be no higher than RAM in GB * 2 or so.

We simply keep a list of packages for where MAKE_JOBS > 1 is highly
beneficial, and just increase those based on PKGPATH:

  https://github.com/joyent/pkgbuild/blob/master/conf/2018Q2-x86_64/mk.conf#L62..L165

Those values aren't exactly what we use ourselves, we keep a log of
how long each package took and adjust as necessary.

But other than that small list of packages, everything else uses
MAKE_JOBS=1.

-- 
Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com