Re: MAKE_JOBS.devel/cmake ineffective

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

On Mon, Oct 11, 2021 at 08:35:32AM +1100, Paul Ripke wrote:
 > > > At least for bmake it makes a huge difference and not necessarily in the
 > > > right direction. -j1 and no -j are quite different modes.
 > > 
 > > Understood, thanks.
 > > 
 > > I've updated the diff to revert back to the previous behaviour, and
 > > also added some more documentation and _VARS support.
 > 
 > Did this go anywhere? I see I still have my little patch applied for
 > cmake on the 2021Q3 branch.

Apparently not.

It also appears that there's a lot of ad hoc per-package setting of
job limits in conjunction with various tools, and that should really
all be in the pkgsrc infrastructure. There is no such infrastructure
for most build tools, so I propose that we create tool.mk files for
build tools that don't have anything else (e.g. ninja) that both take
care of setting TOOL_DEPENDS and also set up the parallelism
arguments. (And also, anything else that should be shared rather than
open-coded in every package.) Then packages that use these tools to
build can include the tool.mk file instead of trying to do everything
by hand.

Also there seem to be at least two packages that call make directly
and therefore need not the number of jobs but the make -j argument
(that is possibly entirely empty).

There's also maybe an argument that it's not just make any more and
possibly it should be called BUILD_JOBS -- changing the user-facing
variables is a big deal but possibly the new one we're about to
introduce should be PKG_BUILD_JOBS rather than PKG_MAKE_JOBS.

I think we should do the following:


1. Take jperkin's patch to build.mk and adjust as follows:
   - leave _MAKE_JOBS_N in place for now;
   - provide PKG_BUILD_JOBS instead of PKG_MAKE_JOBS if that's the consensus;
   - also provide PKG_MAKE_JOBS_FLAGS that contains either -jN or "" for make.
Commit that much. This makes it possible for packages to DTRT.

2. Set up a tool.mk for ninja and get ninja-using packages to use it.
Take care to do this right so that when people copy it for other tools
they don't make a mess.

3. Hunt down any further references in packages to _MAKE_JOBS_N, not
necessarily all at once, and ultimately remove it. Also hunt down any
direct uses of MAKE_JOBS. There are some not in jperkin's patch, e.g.
ocaml.mk.

(Also if anyone else is doing this note that a couple of the changes
in jperkin's current patch change the argument spacing, -jN vs. -j N,
and at least one or two are in places where Python code is likely to
die on it.)

4. Proceed on other tool.mk files as time and interest allows.


I'll do at least (1) if nobody thinks this is totally the wrong
direction.

Oh, also, might be a good idea to have pkglint suggest changing
MAKE_JOBS to PKG_BUILD_JOBS (or PKG_MAKE_JOBS, as we decide) if it
appears in packages.

-- 
David A. Holland
dholland%netbsd.org@localhost