Re: Script pkg_tsort

[Greg Troxel <gdt%lexort.com@localhost>]

tlaronde%polynum.com@localhost writes:

>> It does not pay attention to +PRESERVE, but it doesn't call pkg_delete.
>> It simply calls make replace, which is not documented to fail on
>> +PRESERVE, and I don't think should fail in that case.
>> 
>> Are you intending to avoid doing a replace of things like pkg_install?
>> 
>
> No. The tsort does take into account +PRESERVE the goal being that the
> listing finally produced can be taken as is for further action and there
> will be no surprise with a package being replaced that was not intended
> to be. For example, with a database manager (PostgreSQL to name one),
> updating to an incompatible major version can render your data
> inaccessible. You need to plan the dumping of the bases before switching
> to a new version. It should not be updated, involontarily, as a side
> effect.

I understand the concern about pgsql and similar, but I am really not
following you.

+PRESERVE is documented to require -f on pkg_delete.  As I understand
it, no more and no less.  It is not supposed to affect make replace.
The semantics are that the package is somehow critical and the system
should not be without it; there is no implication that updates are bad
or dangerous.  The semantics for postgresql versions (and surely many
others) are different, which is that a major version update requires
extra work to be safe.

So I think we need another package annotation, maybe a +MANUAL_UPDATE,
or maybe a variable, that tells package tools (all of them, including
pkg_rr, pkgin, and what you are building) not to do the update without
an explicit request from the admin.   I have had this thought for a
long time, but never did anything about it.

> In the output as for now, there are the two fields: status of installed
> (OK or VULNERABLE), status of the pkgsrc version (OK, VULNERABLE or
> UNKNOWN when there is no such package in the pkgsrc version---then the
> new "version" is UNAVAILABLE; I had the surprise to discover that this
> is the case for some of my installed packages...).

So people that write tools to parse your output and do things can make
choices; that seems sensible.  I'm still not sure what people would want
to do if the current package is vulnerable - I'd still want to update
(unless I decide to remove).

>> If the tsort output is used to guide an order for make replace, then I
>> think you need to know the dependencies of the package that will be
>> built even if they are different than the dependencies of the package
>> that is installed.
>> 
>
> Yes. The main goal is updating that is a make replace. So I will have to
> add the dependencies of the new package in case they are not simple
> perhaps updated versions of the installed ones.
>
> But the program could be useful too for a not already installed package
> to see if adding it will not end in trying to upgrade various already
> installed packages breaking a bunch of installed packages (this happens
> too much with huge multimedia packages).

I generally take the view that one should always have installed packages
match the pkgsrc tree one is using.  Basically that selective updating
is trouble.  If you can figure out circumstances where it's safe, that's
interesting.

>> What is your approach to the situation of starting with one package out
>> of date, and no packages marked unsafe_depends?  Once that is replaced,
>> things depending on it probably need to be too.
>
> Yes, this is one of the goal: to know in advance, as far as possible,
> the implications of trying to upgrade a package (or add a new one). I'm
> sure I'm not the only one who tried to update "one" package because
> there were bugs or crashes and seeing half of the packages being
> updated with a significant number of packages broken, whether because of
> forcing a replacement that broke remaining packages, or stopping
> upgrading with the package replaced being definitively broken.

That does happen.  I find it's fairly infrequent.  And, if there aren't
ABI changes, the intermediate state is often ok.

I guess you can produce a list of what would need to be replaced, but I
don't see how to verify any particular build since it will need a
not-yet-produced state.

This is an argument for basically setting up pbulk and building
everything you need and then doing a pkgin fug from that.

> The question arising for me know being if it would not be a better place
> to insert all this in the mk infrastructure: one start with a copy
> read-only of the pkg database and the pkgsrc infrastructure adding fake
> entries to the database for packages that checks for vulnerabilities and
> dependencies (this can be done with the unionfs on a NetBSD for example)
> without downloading, configuring or compiling---it is not 100% safe
> since there can still be problems with these three; but I will not be able
> to make better than that.

I don't quite follow union/read-only.  For things that can be computed,
like vulnerabilities and dependencies (tools, and those of the new
package), they can either be computed as needed or computed and then
stored in a memoization kind of way.  Then there is the hard question of
cache invalidation.

> Other solution: using also unionfs for a fake copy of the system and
> playing with PREFIX in order to compile in the subdirectory populated
> and trying to upgrade the list of packages as given by pkg_tsort.

It sounds like pbulk would be much easier!

> I'm planning to play a little with the latter this week to see if it
> can work (avoiding the need to install a complete image of the system to
> chroot).

Hard to say what's best to avoid - but I'll be interested to hear how it
goes.