Subject: RFC: Subpackages from a single package Makefile
To: None <tech-pkg@NetBSD.org>
From: Johnny C. Lam <jlam@NetBSD.org>
List: tech-pkg
Date: 05/30/2004 19:39:08
--2oS5YaxWCcQjTEyO
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

I've attached a proposal to allow generating multiple subpackages from
a single package directory.  I would appreciate comments on:

   (1) why this is a bad idea in pkgsrc;
   (2) how to improve the implementation outlined in the document.

I have one week of free time ending on Friday of this week to work on
pkgsrc, and I'd be grateful if all the comments could be received by
Tuesday.  I'm pursuing this project as step toward the complete
integration of pkgviews into pkgsrc.

Please cc: me on all replies.

	Thanks,

	-- Johnny Lam <jlam@NetBSD.org>

--2oS5YaxWCcQjTEyO
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="RFC.subpackages"

$NetBSD$

 0 Subpackages
 =============

The idea is to allow a single package Makefile skeleton to produce
multiple subpackages.  This is good for several reasons:

  (1) No need to separate the package builds into multiple packages
      manually, e.g. freetype-{lib,utils}.  The overhead of maintaining
      such packages is quite high for the developer, and simplifying
      this would vastly improve the package _updating_ experience,
      hopefully leading to more timely updates of packages.

  (2) For package views users, it becomes easier to install bits of a
      package into different views without dragging in the whole package
      into every view.  This becomes important with "extensible"
      packages that need to be added to the depot directory of the
      extended package, e.g. add just the Perl bits of cyrus-imapd-2.2.4
      into the perl-5.8.4 depot directory.

  (3) You can later remove parts of packages that you don't want but
      preserve the parts that you do (although this depends on the
      granularity of the package, which depends on the foresight of
      the package maintainer), e.g. delete tiff-3.5.4, but leave
      tiff-lib-3.5.4 installed (this does require that packages that
      depend on "tiff" actually depend on the "tiff-lib" subpackage).


 1 Constraints
 =============

There are some constraints that I make to simplify the implementation.

  (1) Every subpackage has the same revision number as the main package.
  (2) Every subpackage has all of the "real" dependencies of the main
      package.
  (3) All subpackages are always built and installed if building from
      source.

These two constraints are a byproduct of generating all of the subpackages
from a single Makefile.  While it would be possible to allow granularity
even to this scale, I don't believe the extra complexity would be
worthwhile.

  (4) Every main package depends on all of its subpackages.

This is just common sense since a package presumably needs all of its
parts to be considred "complete".  This also allows binary package
upgrades to work correctly, e.g. tiff-3.6.0 will satisfy "tiff>=3.5.4",
even if tiff-3.6.0 doesn't contain libtiff.so, because it will depend
on tiff-lib-3.6.0 which will still force the libraries to be present.


 2 Implementation
 ================

There are various Makefile variables and files associated with the
subpackage concept.  In general, variables and files affecting the
subpackage build are the usual variables and files, but suffixed with
the subpackage label.

 2.1 SUBPACKAGES
 ===============

This variable lists the various subpackages that are built by the
Makefile, e.g.,

	DISTNAME=	foo-1.0
	PKGREVISION=	2
	SUBPACKAGES=	server client lib

The subpackages would be foo-{server,client,lib}-1.0nb2, and they all
would be dependencies for the main package foo-1.0nb2.

 2.2 SUBDEPENDS.*
 ================

These variables, if present, describe the dependency structure within
the subpackages.  For example, if foo-server-1.0nb2 should depend on
both foo-client-1.0nb2 and foo-lib-1.0nb2, then the following setting
should be added to the package Makefile:

	SUBDEPENDS.server=	client lib

There is no provision nor a need for noting a "build dependency"
structure between the subpackages because all of the packages are built
at once during the package build phase.

 2.3 PLIST.*, PLIST_SRC.*
 ========================

If PLIST_SRC.<subpkg> isn't defined in the package Makefile, then the
file PLIST.<subpkg> must exist in the package directory.  PLIST.<subpkg>
lists the files belonging to that subpackage.  There should be no
common file or directory entries amongst the PLIST.* files.  If there
are ${OPSYS}-specific files for the subpackages, then they can be added
to PLIST.<subpkg>.${OPSYS} files.

 2.4 bsd.pkg.install.mk
 ======================

USE_PKGINSTALL is changed to either be "yes" to denote that the main
package uses bsd.pkg.install.mk, or is a list of the subpackages that
need to use bsd.pkg.install.mk, e.g.

	USE_PKGINSTALL=		yes
or
	USE_PKGINSTALL=		server

The following variables have subpackage variants to associate particular
bits of the install-script process with particular subpackages:

	{DE,}INSTALL_TMPL
	{DE,}INSTALL_SRC
	PKG_{USERS,GROUPS}
	SPECIAL_PERMS
	CONF_FILES
	SUPPORT_FILES
	RCD_SCRIPTS
	MAKE_DIRS{,_PERMS}
	OWN_DIRS{,_PERMS}


 2.5 fake-pkg, real-su-package
 =============================

These bsd.pkg.mk targets are modified so that "make install" will
register each subpackage and "make package" will build binary packages
for each subpackage listed in the package Makefile.

--2oS5YaxWCcQjTEyO--