pkg/56957: Broken behavior with PKGSRC_LOCKTYPE=sleep

[cryintothebluesky%gmail.com@localhost]

>Number:         56957
>Category:       pkg
>Synopsis:       Broken behavior with PKGSRC_LOCKTYPE=sleep
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Aug 07 11:15:00 +0000 2022
>Originator:     Sad Clouds
>Release:        pkgsrc-2021Q3
>Organization:
>Environment:
NetBSD rp3 9.2_STABLE NetBSD 9.2_STABLE (GENERIC64)
>Description:
Pkgsrc mk.conf has these settings:

PKGSRC_LOCKTYPE=sleep
PKGSRC_SLEEPSECS=5
OBJHOSTNAME=yes
DEPENDS_TARGET=package-install


I'm building multiple packages in parallel - net/chrony and mail/dovecot2

Both of these packages depend on devel/pkgconf, however they both interfere with one another due to race conditions and fail to build correctly. There seems to be issues with pkgsrc design and the way package locks are used.

Chrony tries to build pkgconf but has to wait on a lock. The lock is released at the wrong time and Chrony tries to build pkgconf and fails:

=> Bootstrap dependency digest>=20010302: found digest-20190127
=> Checksum SHA1 OK for chrony-4.1.tar.gz
=> Checksum RMD160 OK for chrony-4.1.tar.gz
=> Checksum SHA512 OK for chrony-4.1.tar.gz
===> Installing dependencies for chrony-4.1
=> Tool dependency gmake>=3.81: found gmake-4.3nb2
=> Tool dependency pkgconf-[0-9]*: NOT found
=> Verifying package-install for ../../devel/pkgconf
=> Bootstrap dependency digest>=20010302: found digest-20190127
=> Checksum SHA1 OK for pkgconf-1.8.0.tar.gz
=> Checksum RMD160 OK for pkgconf-1.8.0.tar.gz
=> Checksum SHA512 OK for pkgconf-1.8.0.tar.gz
===> Skipping vulnerability checks.
WARNING: No /usr/pkg/db/pkg/pkg-vulnerabilities file found.
WARNING: To fix run: `/usr/sbin/pkg_admin -K /usr/pkg/db/pkg fetch-pkg-vulnerabilities'.
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
=> Lock acquire-tools-lock is held by pid 28498
===> Overriding tools for pkgconf-1.8.0
*** Error code 1

Stop.
make[2]: stopped in /opt/netbsd/pkgsrc-2021Q3/devel/pkgconf
*** Error code 1

Stop.
make[1]: stopped in /opt/netbsd/pkgsrc-2021Q3/devel/pkgconf
*** Error code 1

Stop.
make: stopped in /opt/netbsd/pkgsrc-2021Q3/net/chrony



Dovecot builds pkgconf and then tries to create binary package, but during this time Chrony locks pkgconf as it is trying to build it. Both packages eventually fail:

=> Bootstrap dependency digest>=20010302: found digest-20190127
=> Checksum SHA1 OK for dovecot-2.3.16.tar.gz
=> Checksum RMD160 OK for dovecot-2.3.16.tar.gz
=> Checksum SHA512 OK for dovecot-2.3.16.tar.gz
===> Installing dependencies for dovecot-2.3.16nb1
=> Tool dependency libtool-base>=2.4.2nb9: found libtool-base-2.4.6nb3
=> Tool dependency gmake>=3.81: found gmake-4.3nb2
=> Tool dependency pkgconf-[0-9]*: NOT found
=> Verifying package-install for ../../devel/pkgconf
=> Bootstrap dependency digest>=20010302: found digest-20190127
=> Checksum SHA1 OK for pkgconf-1.8.0.tar.gz
=> Checksum RMD160 OK for pkgconf-1.8.0.tar.gz
=> Checksum SHA512 OK for pkgconf-1.8.0.tar.gz
===> Installing dependencies for pkgconf-1.8.0
=> Tool dependency libtool-base>=2.4.2nb9: found libtool-base-2.4.6nb3
=> Build dependency cwrappers>=20150314: found cwrappers-20180325
===> Skipping vulnerability checks.
WARNING: No /usr/pkg/db/pkg/pkg-vulnerabilities file found.
WARNING: To fix run: `/usr/sbin/pkg_admin -K /usr/pkg/db/pkg fetch-pkg-vulnerabilities'.
===> Overriding tools for pkgconf-1.8.0
===> Extracting for pkgconf-1.8.0
===> Patching for pkgconf-1.8.0
===> Creating toolchain wrappers for pkgconf-1.8.0
===> Configuring for pkgconf-1.8.0
...
===> Installing for pkgconf-1.8.0
...
=> Automatic manual page handling
=> Generating post-install file lists
=> Checking file-check results for pkgconf-1.8.0
=> Creating binary package /opt/netbsd/pkg.objects/devel/pkgconf/work.rp3/.packages/pkgconf-1.8.0.tgz
=> Lock acquire-package-lock is held by pid 21160
===> Building binary package for pkgconf-1.8.0
*** Error code 1

Stop.
make[2]: stopped in /opt/netbsd/pkgsrc-2021Q3/devel/pkgconf
*** Error code 1

Stop.
make[1]: stopped in /opt/netbsd/pkgsrc-2021Q3/devel/pkgconf
*** Error code 1

Stop.
make: stopped in /opt/netbsd/pkgsrc-2021Q3/mail/dovecot2



>How-To-Repeat:

>Fix: