Subject: Package views, make replace, and history [was Re: make update hell]
To: Pavel Cahyna <pavel.cahyna@st.ms.mff.cuni.cz>
From: Alistair Crooks <agc@pkgsrc.org>
List: tech-pkg
Date: 10/07/2004 13:25:46
On Thu, Oct 07, 2004 at 08:31:17AM +0200, Pavel Cahyna wrote:
> > One solution (but has not been done yet) is to take advantage of the
> > information already recorded by pkgsrc. We have the list of libraries
> > provided and required in the +BUILD_INFO files (see PROVIDES= and
> > REQUIRES=). Maybe we can add some rules that check this information on
> > installs and deinstalls and upgrades.

Firstly, some history.

I proposed package views in a paper presented to EuroBSDcon 2002 in
Amsterdam. To summarise, package views is a mechanism by which many
packages, which would normally conflict with each other, can be
installed simultaneously on a single machine.

	ftp://ftp.netbsd.org/pub/NetBSD/misc/agc/pkgviews.pdf

We are in the process of moving pkgsrc to take advantage of package
views, and a number of branches in pkgsrc have been made, including
the most recent one by Johnny Lam.  Johnny subsequently merged his
branch onto the head, and so right now some packages can take
advantages of a package views infrastructure.

It's been obvious for a long while that a better mechanism for
updating and replacing packages was needed within pkgsrc.  "make
update" is there, but is relatively heavyweight.  As package views
involves some infrastructure change, *as an interim measure*, I added
"make replace" to bsd.pkg.mk.  The "replace" target aims to make a
binary package of a package which is already installed on a machine,
and replace it in place by the current version, at the same time
modifying all pointers to the old package to point to the new one.  If
the replace failed, you can rollback by using the "undo-replace"
target, which will delete the current version, and reinstall the old
version from the binary package, modifying the links again.

> The intent is to use this infomation to tell if a "make replace" is safe
> or not? It would be a step forward, but it still would not enable to have
> multiple versions of a library installed in parallel, no? So if "make
> replace" decided that it would be unsafe, nothing would break, but you
> still would have no way to proceed except rebuilding all the depending
> packages. Is this correct?

The replace target displays a warning message that it is an
experimental target.  This is because, if the old package has shared
libraries with a different ABI from those in the new package, then
applications which link in those shared libraries at run time may not
DTRT.  This ABI change is often, but not always, accompanied by a
major number bump in the shared lib version number.  Rather than rely
on an imperfect mechanism, that's the reason for the warning message. 
In practice, I've found that it's hardly ever a problem, and, when it
has happened, a "make replace" in the applications that link with the
new shared libs does the trick.
 
I then added the PROVIDES and REQUIRES information in order to make it
more obvious what was happening when updating.  In pkgsrc, we
sometimes have to think ahead - people use packages that can be quite
old, and so getting this information into the metadata of the package
is necessary before we can use it.  Over time, as people update their
systems, more and more of the installed packages will include the
PROVIDES and REQUIRES information, and have it available through
pkg_info -B.  Recording the OBJECT_FMT information in the binary
package metadata, prior to switching some of the platforms from a.out
to ELF, was another instance where we had to do this exercise ahead of
time.

So although the definitions are in most of the binary packages by now,
we don't make use of it yet.

In addition, we are aiming to deploy package views completely across
pkgsrc, but (a) package views installations will not be mandatory, and
(b) we're waiting for some infrastructure modifications to support
package views better, and so it won't be until Q1 or Q2 2005 before
we have something like that.
 
Regards,
Alistair