Re: MAKE_JOBS_SAFE=[number]

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

On Mon, Sep 17, 2012 at 04:01:29PM -0400, Julio Merino wrote:
 > >> Is it possible to allow MAKE_JOBS_SAFE=[number] in addition to YES/NO?
 > >
 > > I don't think this makes sense. If -j16 fails, it is just a question of
 > > scheduling whether -j8 fails as well.
 > 
 > Agreed.  If the software fails to build with any degree of
 > parallelization greater than 1, the Makefiles are bogus.  As with
 > threading bugs, it's just a matter of time before the problem appears.

It's not that simple. It's not like thread scheduling in C code, where
the timeslices and variances caused by the scheduler are much larger
than individual actions or blocks of code.

In a build, the amount of time each target takes may scale with your
processor speed (or not) but is mostly going to be roughly the same
each time and (more importantly) roughly the same each time with
respect to how long other targets take. Also, make dispatches the
targets in the same or nearly the same order each time; it is not
random.

Suppose you have a target X, which is the first thing in the build,
and another target Y that requires X to have been made but doesn't
enforce this in make, and 10 other targets T0..T9 in between them. If
you build with -j12, then X, T*, and Y will all be started at once,
and the build will fail. If you build with -j11, then X and T* will be
started at once, and the build will fail if and only if one of the T*
targets finishes before X. If you build with -j10, two of them have
to, and so on. If all the targets are roughly the same length
(e.g. they're all ordinary compiles) or X is shorter than average, as
is common with this kind of target (yacc runs, header generation
scripts, etc.) then building with -j8 will pretty much always succeed.

For this example, with only twelve targets to build, it doesn't matter
much, but there are probably large packages that have this property
such that being able to do -j8 vs. only one compile at a time is
highly desirable.

I don't think this usage should be encouraged, but I think it might be
worth supporting.

A better solution is to fix the offending makefiles, of course. But
this isn't always easy, especially for the more regrettable kind of
automake spew that depends heavily on gmake.

-- 
David A. Holland
dholland%netbsd.org@localhost