Subject: Refactoring "install" and "package" phases
To: None <tech-pkg@NetBSD.org>
From: Johnny C. Lam <jlam@pkgsrc.org>
List: tech-pkg
Date: 05/21/2006 14:42:36
I'm about to embark on a project I had mentioned briefly to some folks
while at pkgsrcCon this year -- I am going to split out the variables
and targets associated with the "install" and "package" phases from
bsd.pkg.mk into their own modules, e.g. bsd.pkg.{install,package}.mk.

My main reason for doing this has always been for better code clarity
by separating logically distinct modules into their own files.  After
these modules have been disentangled from bsd.pkg.mk, it is far, far
simpler to re-implement, maintain, and extend these modules while
retaining the same "API".  As examples, I've done this successfully
now for "extract", "patch", "pkginstall" and "plist".  I was part of
a lengthy discussion at pkgsrcCon about doing staged installations,
and I believe it will be simpler to implement this idea after "install"
and "package" have been split from bsd.pkg.mk.

What I've mentioned above will be done over the next week or two, but
there is another part of this project which is somewhat controversial,
and I'd like to get some feedback on it.  In my mind, pkgsrc does two
separate and distinct things, even though we just call the whole thing
"pkgsrc" -- pkgsrc is a system to build software, and pkgsrc is a
system to manage software.  I would like to leverage our strength in
making software build on lots of different platforms, and then allow
installing and packaging software in the platform's "native" package
format.  For example, this would be "SVR4 package format" on Solaris,
or RPM on Red Hat and SuSE, and of course, "pkgsrc format" will always
be supported by default for all platforms.  I note that there is a
pkgtools/gensolpkg package written by Alistair Crooks which points
the way do this for SVR4 packages on Solaris, so this project is
clearly doable.

The following are what I see as benefits of this proposal:

  * This lets pkgsrc build the software, which I feel is indisputably
    one of our main strengths, while the native package tools are used
    to manage the software, which is an area which I feel pkgsrc still
    needs improvement.

  * It would be possible to create repositories of binary packages in
    the platform's native format which can be installed without needing
    to install a bootstrap kit.

  * A system administrator who is already familiar with his platform's
    native package tools would be able to manage software in the way
    most comfortable to him.

The following are what I see as negatives of this proposal:

  * It's a disincentive to improve pkgsrc's package management tools.
    We'll certainly keep improving them for the sake of NetBSD and
    DragonFly, but we may lose testing from Linux and Solaris users.

Through my investigations within pkgsrc/mk, I believe that the relevant
parts of the package workflow which would be affected by this are
"install-depends", "install", and "package".  As part of my project to
split out "install" and "package" into separate modules, I'd like to
re-implement them with the idea of extending them to initially support
"pkgsrc" and "SVR4 package" formats.

Comments?

	Cheers,

	-- Johnny Lam <jlam@pkgsrc.org>