Re: Script pkg_tsort

[Greg Troxel <gdt%lexort.com@localhost>]

tlaronde%polynum.com@localhost writes:

> Hello,
>
> On Mon, Apr 15, 2019 at 03:51:53PM -0400, Greg Troxel wrote:
>> tlaronde%polynum.com@localhost writes:
>> 
>> > Here is a simple Bourne shell script that takes what is currently
>> > installed on a node, takes the build dependencies and the running
>> > dependencies and produce a series of records in topological order
>> > with the following fields:
>> 
>> pkg_rolling-replace does a bit of this.  If you end up packaging it, it
>> might make sense to depend on it instead.

What I meant is that if your script, which figures out all the
dependencies, could perhaps be called by pkg_rr instead of what pkg_rr
is doing.  Basically decomposing it into simpler tools as you suggest.

> But pkg_rolling-replace ignores +PRESERVE.

It does not pay attention to +PRESERVE, but it doesn't call pkg_delete.
It simply calls make replace, which is not documented to fail on
+PRESERVE, and I don't think should fail in that case.

Are you intending to avoid doing a replace of things like pkg_install?

> I don't know either if pkg_rolling-replace takes pkg-vulnerabilities
> into account (when not doing things but telling what it would have
> done).

It does not.  Are you thinking about the situation where an installed
package is not vulnerable but the new one is?  Or both the old and new?
Or just old?  I have more or less viewed updating to a newer version as
always neutral or positive, although I realize there could be a
pathological case.

What is your proposed course of action (for the human who interprets the
pkg_tsort output) relative to a list of things to replace and
pkg-vulnerabilities?

> I have tried it but this above and the fact that all is mixed: finding
> dependencies and updating, is not to my taste.
>
> I specifically wanted to have an utility to "sort" things and, based on
> this listing, one can do whatever one sees fit with very simple
> scripting (or edit the listing for ad hoc fixing).

Sure - I tried to refrain from saying "you should use pkg_rr", and was
merely trying to point it out as having similar goals and therefore
useful to look at, as well as to understand what you wanted different.

>> With the -n option, it doesn't do the replaces, but will more or less
>> produce a list of what it would have udpatd.
>> 
>> It looks like your script finds the run and build dependencies of the
>> installed version.  pkg_rr finds the run and build dependencies of the
>> new version, and now recently the tool dependencies also.   I don't know
>> how you'd do that without running make show-depends three times.
>
> You have probably to call make since there is no simple way to parse the
> Makefile, with all the inclusions and the variable expansions---and 
> this is what takes a huge amount of time---I don't know for the pkgsrc
> version of the package if I could simply take the distinfo file without
> bothering with make(1) and the Makefile.
>
> But for the dependencies, one can perhaps avoid calling make by sed'ing 
> the Makefile for the buildlink inclusions and then looking for the 
> inclusions if this has not already be done for the packages (example:
> one starts with a list of packages; the package itself is PROVIDE(d);
> the dependencies are REQUIRE(d); after the first pass, sort(1) and
> uniq(1) the REQUIRE(d); cat with PROVIDE(d), sort on name field and
> if the pkg has no PROVIDE line, iterate with the list of not provided
> ---should be faster than using awk(1) to compare names even in an array).

I prefer to call make, rather than try to reimplement part of it, for
reliability.  The time to run is slightly annoying, but is not
problematic for me.

If the tsort output is used to guide an order for make replace, then I
think you need to know the dependencies of the package that will be
built even if they are different than the dependencies of the package
that is installed.

If your goals are other than a make replace plan, then I'd be interested
in hearing about them.

> As I said, the script is relatively simple now and should be expanded
> to list, when one gives a specific package or a list of specific
> packages, what packages have to be updated and what packages not
> listed will have to be updated too.

There's an interesting question of updating only if mismatched, and
updating based on unsafe_depends.  With out-of-order replacing, I think
one needs unsafe_depends to be sure to end up in a sound state.  I
realize what you posted about only tosrts, but I am curious about your
approach with the output.

> But the principle of splitting between providing a list of records that
> can be script processed, and treating the packages is, IMHO, the sound
> one.

Sure, no argument in principle about composing simpler tools.

What is your approach to the situation of starting with one package out
of date, and no packages marked unsafe_depends?  Once that is replaced,
things depending on it probably need to be too.