Re: patch: parallelized pkg_chk

[Greg Troxel <gdt%lexort.com@localhost>]

Tobias Nygren <tnn%NetBSD.org@localhost> writes:

> "pkg_chk -uqB" as used by pkg_rolling-replace is quite slow on systems
> with many packages installed because it needs to run bmake on in each
> package directory to extract PKGNAME and optionally other things.
> The attached patch parallelizes this phase up to MAKE_JOBS.
>
> On my 10c/20t system with 804 packages installed the time to run
> "pkg_chk -uq" with this patch drops from 10m to 1m24s. Still not fast
> but much better. Comments?

I think we are at a crossroads where using all cores is becoming the
normal case vs a special operation.   I do like it that make by default
only runs one command at once, so I can run a make overnight with low
fan noise, and only run parallel jobs if I ask for them.

However, this is using MAKE_JOBS, so it respects that notion, and thus I
think it's exactly right.

We do have to be careful about MAKE_JOBS=n and running n commands each
of which think they can use n cores; rust is the poster child for this.
But I don't see pkg_rr/pkg_chk as running into this for now.