Subject: Re: bin/1734: New parallel make behaves incorrectly and non-intuitively.
To: Christos Zoulas <christos@deshaw.com>
From: Jason Thorpe <thorpej@nas.nasa.gov>
List: netbsd-bugs
Date: 11/06/1995 22:46:00
On Tue, 7 Nov 1995 04:32:35 GMT 
 christos@deshaw.com (Christos Zoulas) wrote:

 > I don't have the answer to that...
 > 
 > Consider:
 > 
 > foo: foo.o bar.o baz.o
 > 	cc -o ...
 > 
 > 
 > Now if what you are saying is to be done, then the parallel make should make
 > first foo.o wait for that to finish, then bar.o wait for that to finish,
 > and then finally baz.o... Well, there is not much parallelism in there is
 > there?

No, but if you didn't specify a "max jobs" (thereby defaulting to 1), 
there's not much point in processing dependencies in parallel.

 > I don't know what is the best solution to solve the problem you are describing.
 > I've been using pmake for many years now, and I've seen the broken behavior
 > you mention. Most Makefiles do work correcly without -B though [I would say
 > close to 95% of the Makefiles that one tries]

It's been eluded to by others (in private converstations previous to my 
filing this PS) that pmake only behaved in this fashion when invoked as 
"pmake", and worked like the traditional make when invoked as "make".

 >     make -j 4 obj && make -j 4 depend && make -j 4 all
 > 
 > works like you expect it. 
 > 
 > Maybe the default -j 1 should imply -B?
 > 
 > Note that I did explain all that in detail when I committed the changes...

The other parallel make utility I have experience with is Sequent's.  To 
me, its behavior was much more intuitive, and much closer to being 
correct.  If the parallel feature was not explicitly enabled, it behaved 
like a traditional, non-parallel make.  If I remember correctly, it also 
processed targets on the command line in order regardless of wether or 
not the parallel feature was enbled, thus eliminating the need to use the 
shell to achieve something that has always, to my knowledge, been 
possible with make.

Granted, Sequent's make enabled the parallel feature somewhat 
differently; a single `&' before a list of dependencies caused those (and 
only those) dependencies to be processed in parallel.  There was also a 
command line option (-P N) to limit the number of concurrent jobs (like 
our -j N).  Thus:

all: & foo bar zap

foo, bar, and zap would be processed in parallel.

all: foo & bar zap

foo would be processed, then bar and zap in parallel.

There was a limitation on this.  Only the last command of a multiple 
command sequece for a target was elidgeable for concurrent execution.

In any case, at the risk of getting into a religious war about the 
semantics of make, I respectfully submit that "parallelism with max jobs 
of 1" should not be default, but rather "backwards compat mode" should 
be.

However, the ability to specificy parallelism for individual 
targets/dependencies rather than parallelism applying globally to the 
entire Makefile is arguably a worthy goal, and possibly something to work 
towards.

--------------------------------------------------------------------------
Jason R. Thorpe                                       thorpej@nas.nasa.gov
NASA Ames Research Center                               Home: 408.866.1912
NAS: M/S 258-6                                          Work: 415.604.0935
Moffett Field, CA 94035                                Pager: 415.428.6939