> I'd be happy to keep discussing this further, as we still don't have a
> decent way to manage multiple installations of the same package in
> pkgsrc, and package views is really the only proposal put forth to
> address this in a general way.

Your post was mostly centered around implementation issues rather than design, 
but it still prompted me to think of some things I had previously been 
considering when trying to figure out what I ideally wanted out of a 
packaging system. So here goes my thoughts, relevant or not...

Ideally, I would like a package system to:

(1) Allow for source based install for maximum flexibility and configurabilty.

(2) Have full support for binary packages in a way that makes mixing of binary 
and source installs seamless.

(3) Allow for multiple versions of the same software to be installed at the 
same time without affecting each other.

(4) Allow for fully automatic upgrading, even of packages with "state" (e.g. 
PostgreSQL), subject to sysadmin configuration.

(5) Allow for atomic upgrading.

The two points I would like to address here are (1)-(3). I am not sure in 
which end to start in order to best explain what I mean, so I will begin by 
defining a few terms (partially overlapping with current pkgsrc terminology). 
Let

* "origin" refer to the location in pkgsrc and its contends ("stuff/foobar in 
a certain revision of the pkgsrc tree")

* "realization" (sorry, cant think of a better word) refer to a specific 
combination of an origin, package configuration and (resucrively) the same 
for all dependencies (e.g. "foobar 1.2.0 with options SSL IPv6 and 
dependencies (....), (....)").

* "installation" refer to an actual installation of a realization on a system

* "marshalled installation" refer to what is now called a binary package; i.e. 
a packaging of an installation.

* "package" refer to the general package concept ("foobar 1.2.0").

* "view" refer to a user-visible set of realizations

Assuming a typical end-user system with preference to binary packages where 
source installations are not needed, performing a 
"cd /usr/pkgsrc/stuff/foobar && make install" will essentially result in the 
system working out the realization the user is asking for, based on any 
package options that applies to stuff/foobar, and its dependencies. This 
realization might be 

   "foobar 1.2.0 IMAP,IPv6 (mail/libimap 0.5.4 SSL)"

The system first checks whether there is an installation that satisfies the 
"mail/libimap 0.5.4 SSL" realization. If there is not, it checks whether 
there is a marshalled installation that does so. If there is, the realization 
is satisfied by unmarshallaing the marshalled installation into an actual 
installatino satisfying the realization (say that fast ten times...). If 
there is no suitable marshalled installation, it goes to the origin and 
builds from source (optionally also creating a marshalled installation for 
future use).

Once there exists an appropriate installation of mail/libimap, foobar is to be 
installed. The system goes through the same procedure here. Note that in 
order for a marshalled installation to satisfy the realization, it must be 
match exactly - including all dependencies with appropriate options.

Once stuff/foobar is installed, the system registeres the fact that the user 
"wants" that particular realization (that is, "foobar 1.2.0 IMAP,IPv6 
(mail/libimap 0.5.4 SSL)").

At this time there is only *one* logical "wanted" realization identified 
uniquely by the realization descriptor. The user asks the system to 
incorporate this realization in the default view.

Now suppose a week passes and a new version of foobar is commited. This 
version has a built-in IMAP implementation, and now supports POP - but 
through a library. The user again asks the system to install stuff/foobar. 
This time, the realization is:

   "foobar 1.3.0 IMAP,IPv6,POP (mail/libpop 1.5.1)"

The system proceeds by installation mail/libpop as previously described and, 
when that dependency is satisfied, foobar 1.3.0. At this time the new version 
of foobar is again registered as a "wanted" realization.

At this point the old version of foobar is still the one in the default view. 
The user clones the default view but changes the realization of the 
stuff/foobar origin to the new one. The user tests the software. Once 
satisfied, the user tells the system to modify the default view to match that 
of the test view. The new version of foobar is now available in the default 
view.

The user proceeds by asking the system to remove any "want" registrations that 
are not referenced by a view. The result is that the first realization of 
foobar 1.2.0 is removed from the want list.

After this, the system is free to 'garbage collect' the installations of the 
mail/libimap and old stuff/foobar realization; possibly keeping marshalled 
installations intact.

The key features I want, and which I tried to exemplify above, is the 
separation of the different aspects of the packaging systems. In terms of 
installations, there is some central "cache" of available installations, each 
being dependent on other installations as needed. When a realization is 
brought into a view, those installations are brought into that view (by some 
method, be it symlinks or some advanced VFS acrobats or whatever). Any 
installation can be satisfied by source or a marshalled installation. One can 
freely mix installed-from-source packages with 
installed-from-marshalled-installations packages. The system will 
automatically build a package from source if there is no pre-built version 
that satisfies all desired configuration options.

A global system upgrade is just a matter of telling some tool to upgrade a 
given view. The tool looks at all "wanted" realizations in that view, and 
constructs a new view with the latest version of the corresponding 
realizations (by going to the origins). Some packages might be upgraded 
without building from source because the upstream provider of marshalled 
installations have provided an up-to-date installation, some may be built 
from source because of non-standard options. Those realizations that have not 
changed between the two resulting views need not be re-installed; only 
reflected into the new view.

The end-result I hope would be the best of the source and binary worlds, in 
combination with multiple views.

I hope this made at least some sense...

-- 
/ Peter Schuller, InfiDyne Technologies HB

PGP userID: 0xE9758B7D or 'Peter Schuller <peter.schuller@infidyne.com>'
Key retrieval: Send an E-Mail to getpgpkey@scode.org
E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org