pkgsrc-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: wip/pkgchkxx: a complete reimplementation of pkg_chk and pkg_rolling-replace



On 7/22/23 02:54, PHO wrote:
Hi all,

I just created a new package wip/pkgchkxx. This is a complete
reimplementation of pkgtools/pkg_chk and pkgtools/pkg_rolling_replace in
C++17.

This implementation achieves better performance by using a faster
language, better algorithms, and making use of many CPUs whenever
possible. The latter is the primary reason why a complete rewrite was
needed: doing it in POSIX shell was simply not feasible.

When you update your pkgsrc tree on a running system and you have like
200 outdated packages to update, the chance of pkg_rr -u completing
without any failures is somewhere between 5% and 0%. You usually have to
fix broken things here and there, re-run pkg_rr repeatedly, and waste
your time staring at "Checking for xxx packages (xxx=YES)" for several
seconds each time you restart pkg_rr. I wanted to change that.

* pkgchkxx(8) takes options fully compatible with pkg_chk(8).
* pkgrrxx(8) takes options fully compatible with pkg_rolling-replace(8).
* Unlike pkg_rr, you can run pkgrrxx as a non-root user. It makes use of
${ROOT_CMD} whenever it needs a root access.
* "pkgrrxx -u" runs roughly N times faster than "pkg_rolling-replace -u"
where N is the number of CPUs you have.
* "pkgrrxx -sn" runs 14.8x faster than "pkg_rolling-replace -sn".
* "pkgchkxx -aur -b" runs 11x faster than "pkg_chk -aur -b" when
pkg_summary file is available.
* "pkgchkxx -aur -s" runs roughly N times faster than "pkg_chk -aur -s"
on a system with N CPUs.
* "pkgchkxx -l" runs 185x faster than "pkg_chk -l" when pkg_summary file
is available, and runs 24.8x faster when it's unavailable (and needs to
scan archives).
* "pkgchkxx -p" runs 1.3x faster than "pkg_chk -p".
* "pkgchkxx -g" runs 1.6x faster than "pkg_chk -g".
* "pkgchkxx -N" runs 22x faster than "pkg_chk -N".

Please consider the package as experimental. Do not forget to backup
your /usr/pkg (I recommend sysutils/bup for that) before using it in
case it messes up your system.

Also do not question the value of faster pkg_rr when the most
time-consuming part of its job is to actually rebuild packages!

Thanks for working on this!

It would be good, at least for now, if you would eliminate the need for

USE_CXX_FEATURES=filesystem

This adds a GCC dependency, which adds hours to the build time every
time there's a minor update to the gcc package.


Home | Main Index | Thread Index | Old Index