Subject: Re: Removing pkgviews?
To: Johnny Lam <email@example.com>
From: Peter Schuller <firstname.lastname@example.org>
Date: 04/12/2006 21:27:42
> I'd be happy to keep discussing this further, as we still don't have a
> decent way to manage multiple installations of the same package in
> pkgsrc, and package views is really the only proposal put forth to
> address this in a general way.
Your post was mostly centered around implementation issues rather than design,
but it still prompted me to think of some things I had previously been
considering when trying to figure out what I ideally wanted out of a
packaging system. So here goes my thoughts, relevant or not...
Ideally, I would like a package system to:
(1) Allow for source based install for maximum flexibility and configurabilty.
(2) Have full support for binary packages in a way that makes mixing of binary
and source installs seamless.
(3) Allow for multiple versions of the same software to be installed at the
same time without affecting each other.
(4) Allow for fully automatic upgrading, even of packages with "state" (e.g.
PostgreSQL), subject to sysadmin configuration.
(5) Allow for atomic upgrading.
The two points I would like to address here are (1)-(3). I am not sure in
which end to start in order to best explain what I mean, so I will begin by
defining a few terms (partially overlapping with current pkgsrc terminology).
* "origin" refer to the location in pkgsrc and its contends ("stuff/foobar in
a certain revision of the pkgsrc tree")
* "realization" (sorry, cant think of a better word) refer to a specific
combination of an origin, package configuration and (resucrively) the same
for all dependencies (e.g. "foobar 1.2.0 with options SSL IPv6 and
dependencies (....), (....)").
* "installation" refer to an actual installation of a realization on a system
* "marshalled installation" refer to what is now called a binary package; i.e.
a packaging of an installation.
* "package" refer to the general package concept ("foobar 1.2.0").
* "view" refer to a user-visible set of realizations
Assuming a typical end-user system with preference to binary packages where
source installations are not needed, performing a
"cd /usr/pkgsrc/stuff/foobar && make install" will essentially result in the
system working out the realization the user is asking for, based on any
package options that applies to stuff/foobar, and its dependencies. This
realization might be
"foobar 1.2.0 IMAP,IPv6 (mail/libimap 0.5.4 SSL)"
The system first checks whether there is an installation that satisfies the
"mail/libimap 0.5.4 SSL" realization. If there is not, it checks whether
there is a marshalled installation that does so. If there is, the realization
is satisfied by unmarshallaing the marshalled installation into an actual
installatino satisfying the realization (say that fast ten times...). If
there is no suitable marshalled installation, it goes to the origin and
builds from source (optionally also creating a marshalled installation for
Once there exists an appropriate installation of mail/libimap, foobar is to be
installed. The system goes through the same procedure here. Note that in
order for a marshalled installation to satisfy the realization, it must be
match exactly - including all dependencies with appropriate options.
Once stuff/foobar is installed, the system registeres the fact that the user
"wants" that particular realization (that is, "foobar 1.2.0 IMAP,IPv6
(mail/libimap 0.5.4 SSL)").
At this time there is only *one* logical "wanted" realization identified
uniquely by the realization descriptor. The user asks the system to
incorporate this realization in the default view.
Now suppose a week passes and a new version of foobar is commited. This
version has a built-in IMAP implementation, and now supports POP - but
through a library. The user again asks the system to install stuff/foobar.
This time, the realization is:
"foobar 1.3.0 IMAP,IPv6,POP (mail/libpop 1.5.1)"
The system proceeds by installation mail/libpop as previously described and,
when that dependency is satisfied, foobar 1.3.0. At this time the new version
of foobar is again registered as a "wanted" realization.
At this point the old version of foobar is still the one in the default view.
The user clones the default view but changes the realization of the
stuff/foobar origin to the new one. The user tests the software. Once
satisfied, the user tells the system to modify the default view to match that
of the test view. The new version of foobar is now available in the default
The user proceeds by asking the system to remove any "want" registrations that
are not referenced by a view. The result is that the first realization of
foobar 1.2.0 is removed from the want list.
After this, the system is free to 'garbage collect' the installations of the
mail/libimap and old stuff/foobar realization; possibly keeping marshalled
The key features I want, and which I tried to exemplify above, is the
separation of the different aspects of the packaging systems. In terms of
installations, there is some central "cache" of available installations, each
being dependent on other installations as needed. When a realization is
brought into a view, those installations are brought into that view (by some
method, be it symlinks or some advanced VFS acrobats or whatever). Any
installation can be satisfied by source or a marshalled installation. One can
freely mix installed-from-source packages with
installed-from-marshalled-installations packages. The system will
automatically build a package from source if there is no pre-built version
that satisfies all desired configuration options.
A global system upgrade is just a matter of telling some tool to upgrade a
given view. The tool looks at all "wanted" realizations in that view, and
constructs a new view with the latest version of the corresponding
realizations (by going to the origins). Some packages might be upgraded
without building from source because the upstream provider of marshalled
installations have provided an up-to-date installation, some may be built
from source because of non-standard options. Those realizations that have not
changed between the two resulting views need not be re-installed; only
reflected into the new view.
The end-result I hope would be the best of the source and binary worlds, in
combination with multiple views.
I hope this made at least some sense...
/ Peter Schuller, InfiDyne Technologies HB
PGP userID: 0xE9758B7D or 'Peter Schuller <email@example.com>'
Key retrieval: Send an E-Mail to firstname.lastname@example.org
E-Mail: email@example.com Web: http://www.scode.org