Re: CVS commit: pkgsrc/mk/flavor/pkg

[Joerg Sonnenberger <joerg%britannica.bec.de@localhost>]

On Sat, Jun 12, 2010 at 08:13:43PM -0400, Greg Troxel wrote:
> >> Again, it seems to me that you are working towards a world where extra
> >> pkgsrc installs and/or chroots are mandatory, and I think that this is
> >> seriously misguided.
> >
> > If you care about keeping your systems running, anything else is
> > misguided. I certainly don't want to start compiling any non-trivial
> > package during a maintainance window.
> 
> So, if you think that the problems of make replace/pkg_rr are so serious
> that they outweight the convenience, that's fine, and a reasonable
> position.  But there's no basis to say that replace/pkg_rr is so broken
> that anyone who wants to use it is confused and should just be told not
> to.  After all, the tsorted rolling replace fixes all the broken
> dependencies, so we're just arguing about the time frame of the breakage
> (missing package or broken dependency) and the nature of it, and we're
> evaluating the odds of various situations in terms of total harm vs
> effort - not something with a black and white answer.

I am saying that forcefully overwriting consistency checks based on the
meta data of packages is broken. Non-forward compatible dependencies are
one class of such problems. Files moving nearer to the root in the
dependency chain is another issue and broken with the current change.
The currently discussed situation was artificially created and pre-existing
cases differ in so far as "ignore the issue" doesn't work for
incremental changes.

The situation will get worse at point where pkgsrc does handle ABI
compatiblity correctly. Correct handling of that makes incremental
updates work correctly for the most interest change -- no ABI dumps.
Subpackages can reduce the impact of the remaining cases by making it
easier to keep the old libraries around. But the current approach of
just closing the eyes and hoping for the best will not provide value
because the good case has been eliminated already.

>   people who want to choose liveness properties like "my packages remain
>   in place while I update them one at a time so that even if the update
>   has trouble in the middle almost all of them are there, even if that
>   means sometimes dependencies are broken" should be able to do so.
>   (This is make replace and pkg_rr.)

This is what I disagree on, see above. The dependencies are not broken,
the packages are. The dependencies are just a sign for the actual
trouble, if they are specified correctly. IMO the correct approach for
something like pkg_rr is:

(1) Run make package USE_DESTDIR=yes
(2) Try pkg_add -U foo.tgz, if it successful, do or do not do the unsafe
depends dance and continue as needed.
(3) If it fails, fallback to "make update" like behavior.

Note that step (3) doesn't necessarily have to backout all packages,
just offending ones. E.g. glib2 itself doesn't have to be removed for an
inplace perl update over major versions to work. Bonus points for
continuing to rebuild intermediate leaf nodes after failure, e.g. with

A -- B -- C
  \- D

as dependencies, even if the build of B fails, D can and should be
rebuild/reinstalled. One side effect of the destdir support is that it
is relatively safe to do so without leaving a garbled system behind.

>   people who want to choose properties like "I need safe dependencies,
>   and minimal downtime, and I'm willing to spend the time to prebuild
>   all packages I want in a chroot and manage them with pkg_chk" can do
>   so.  (This is pkg_comp or pbulk and pkg_chk
>   remove-mismatched/add/update.)

I'd say bulk build + pkgin as tools, but that's just the choice of
tools.

>   1) shlib changes don't cause dependency failures (dependency "foo>=12"
>   is basically incorrect).  We all agree this would be good to fix, I
>   think, and no one's done the work yet.

I hope to find the time to address this over the summer.

>   2) make replace fails due to pinned dependencies and/or the override
>   to make it work is overly broad

See above. I think the cure is worse than the problem.

Joerg