Re: patch: parallelized pkg_chk

[Joerg Sonnenberger <joerg%bec.de@localhost>]

On Thu, Mar 12, 2020 at 10:30:28PM +0100, Tobias Nygren wrote:
> On Thu, 12 Mar 2020 21:51:16 +0100
> Joerg Sonnenberger <joerg%bec.de@localhost> wrote:
> 
> > On Thu, Mar 12, 2020 at 09:17:39PM +0100, Tobias Nygren wrote:
> > > "pkg_chk -uqB" as used by pkg_rolling-replace is quite slow on systems
> > > with many packages installed because it needs to run bmake on in each
> > > package directory to extract PKGNAME and optionally other things.
> > > The attached patch parallelizes this phase up to MAKE_JOBS.
> > 
> > I am generally in favor of the idea, but I don't think this should be
> > tied to MAKE_JOBS and I'd prefer to depend on xargs instead of open
> > coding the job queue logic.
> 
> Using xargs was my first idea but unfortunately xargs can't directly
> call shell functions, so pkg_chk.sh needs some custom entry points for
> that to work.

Correct, something like list mode where it just extracts the output for
one location with some prefix.

> What do you propose instead of MAKE_JOBS? In an earlier revision I did:
> 
> numjobs() {
>   case "$(uname -s)" in
>     NetBSD) /sbin/sysctl hw.nprocsonline ;;
>     *) echo 1 ;;
>   esac
> }
> 
> ... but I removed that for similar reasons as cited by Greg. We can do
> command line -j option as well, or a combination of sysctl and/or
> MAKE_JOBS with command line override.

-jX and without argument, default to one of
   getconf _NPROCESSORS_ONLN
   getconf NPROCESSORS_ONLN
   sysctl hw.nproc
   1

Joerg