Re: wip/pkgchkxx: a complete reimplementation of pkg_chk and pkg_rolling-replace

[Greg Troxel <gdt%lexort.com@localhost>]

PHO <pho%cielonegro.org@localhost> writes:

> In the beginning of my work I tried hard to stick with C. Yes, plain
> old C. But I quickly got tired of its low-level nature. I am a
> Haskeller who really doesn't like to code something complicated in a
> language just a little bit convenient than ASM. So I decided to go
> with C++11, which is still not as non-portable as Haskell,
> but... C++11 without Boost is really a nightmare, and of course I
> didn't want the tool to depend on Boost! So I switched to C++14 and
> realized that std::shared_ptr and std::filesystem didn't exist in that
> version. This is how I ended up on C++17. I knew people would hate me
> for requiring C++17 for the tool but I thought it's okay as long as it
> stayed in wip :D

I can see wanting Boost, even if it imposes a requirement that is
slightly troublesome.

>>> * pkgchkxx(8) takes options fully compatible with pkg_chk(8).
>> You didn't say, and probably this is future, but the biggest problem
>> right now in the pkg_rr/pkg_chk world is failing to deal with
>> multiversion packages like pyNN-foo, where it basically errors if NN is
>> not PYTHON_VERSION_DEFAULT.  That requires pkg_chk to output both
>> PKGPATH and some version kv objects, and carry that around as the thing
>> that needs doing, rather than just PKGPATH.
>
> My implementation does something better than the original code, but I
> can't remember what changes I made, which is why I didn't say anything
> about it.

It would be really nice for the pkgrrxx docs to document that.

>>> * "pkgchkxx -aur -b" runs 11x faster than "pkg_chk -aur -b" when
>>>    pkg_summary file is available.
>> I don't follow this.  I didn't think pkg_-chk used pkg_summary, and
>> pkg_summary is generally collected metadata for a bunch of binary
>> packages.  But this is about comparing installed packages to the source
>> tree.
>
> It's about comparing installed packages to the set of available binary
> packages. When pkg_summary is available it just needs to read the
> summary, otherwise it has to decompress binary packages and see what
> are in them.

OK - would be good to be clear about that, but maybe it's just me.  I
use pkg_chk from pkg_rr when the thing to compare against for "are you
up to date" is the sources.  To me, once you have a binary package set
and a summary file, there is no need to use any of this and you should
just be using pkgin.

>> The other thing that is wrong with pkg_rr (and always has been, but it
>> has been increasingly painful) is that many packages have lots of
>> build-only dependencies.  pkg_rr goes over the installed set and
>> extracts the dependencies (which are recorded) but doesn't extract the
>> build deps (which need a make invocatino) until building each one.
>> However it would probably be better to get the TOOL_DEPENDS and
>> BUILD_DEPENDS dependencies up front, and this can be parallized.
>
> My implementation parallelizes it. I mean it doesn't extract build
> dependencies up front, but it at least invokes make(1) parallelly in
> the "checking if xxx has new depends" phase.

I have been dimly pondering having pkg_rr do this for each package up
front.

>> Also in the brave new world of cross, it would be nice to be able to
>> somehow DTRT about TOOL/BUILD when running pkg_rr for a cross build.  I
>> have so far not contemplated or tried.
>
> I haven't considered that. Don't know what to do in that case.

I'm not sure either, but the big point is that there are two trees, the
host tree and the target tree (avoiding Canadian cross notation), and if
you run pkg_rr in the host, that's normal, and if you run it in the
target, I guess it just assumes the host tools are ok and that's that.
Someone with actual experience updating a cross world will have to chime
in and maybe we'll need to improve something.