Re: Issues with larger numbers with -j

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

On Wed, Jun 14, 2017 at 01:03:44AM +0000, John Klos wrote:
 > I've noticed a number of instances when compiling pkgsrc packages with a
 > large value for -j failures, where compiling with a small number or -j 1
 > compiles just fine. For instance, devel/readline and lang/python36 have
 > failed in the last few minutes with -j 40.
 > 
 > This is on a GNU/Linux Ubuntu system running on Power which can have up to
 > 160 simultaneous threads. I don't have any NetBSD systems that run more
 > than 16 simultaneous threads, so I can't say whether the same issues exist
 > elsewhere.
 > 
 > Has anyone seen this?

It is common for upstream makefiles to be wrong, especially (IME) ones
generated by automake.

Until relatively recently these tended to get smoked out and tagged
(or fixed) by people doing pbulk runs, but for various reasons the
trend for pbulk setups now is to run a bunch of chroots at once and
relatively low values of MAKE_JOBS.

So as upstream makefiles regress they don't necessarily get caught any
more.

(Also, I remember at one point there was some notion of having a
MAKE_JOBS_LIMIT in addition to MAKE_JOBS_SAFE, as it's a pretty big
hammer to build a large project with only one job at a time just
because its makefiles blow up for 16+ jobs. Then again, I guess if the
problem with the makefiles is understood well enough to be sure of
that mostly it can just be fixed instead...)

-- 
David A. Holland
dholland%netbsd.org@localhost