Subject: Re: (Incomplete) List of pkgsrc Improvements
To: Dieter Baron <dillo@danbala.ifoer.tuwien.ac.at>
From: Marc Espie <espie@nerim.net>
List: tech-pkg
Date: 08/02/2007 12:11:15
On Tue, Jul 31, 2007 at 12:17:46PM +0200, Dieter Baron wrote:
> 2.3 sub-packages
> 
>   GOAL: Build multiple binary packages from one source package, each
> containing part of the package (e.g. server and client).  Currently,
> we have no framework to deal with this, and each packages is manually
> split into multiple packages, often resulting in parts being done
> multiple times (at least the extraction and patching, sometimes even
> compiling).  This new framework should also support for building a
> subset of the supported packages (so source users don't incur the full
> build and dependency impact of parts they are not interested in).
> 
>   As I see it, the following would be needed:
> 
>   - the user can select which sub-packages should be built/installed
>   - the package lists which files belong to which sub-package
>     (possibly via patterns)
>   - the package can optionally disable building the unwanted parts (if
>     the build infrastructure allows it)
>   - per sub-package dependencies
>   - one sub-package can depend on another sub-package
> 
>   RATIONALE: This would allow us to split big packages with a big
> dependency tree (like gst-plugins) into multiple binary packages with
> fine-grained dependencies, with less maintenance overhead than we have
> now.

Again, we have this in OpenBSD. This evolved a bit over the times.
Initially, we have MULTI_PACKAGES for specific ports, with a main
package, and sub packages. It turned out to be very useful, but
a bit difficult to understand, so that I ended up fixing a lot of
subtle bugs in some ports.

The current situation is much simpler: each port is a MULTI_PACKAGES
ports. By default, the main subpackage is called -main.
If you have more than one, you just set MULTI_PACKAGES.

Straight away, you run into the issue that subpackage names participate
in dependencies. We fixed that one along with `options' (which we call
flavors) in a very systematic way: in the ports tree, we can encode a
full PKGPATH to a dependency, which is the usual path, followed by a
list of separated options/subpackages (options never start with a -,
subpackages always do). And package names follow similar rules.
- multi-packages produce completely distinct package names.
- flavors are appended to the end of a package name, and a dependency
can specify that some flavors are required, or are not usable.


A few variables do depend on the SUBPACKAGE part: they have a default
value, which has the usual name, and if you want to get a specific value,
you do postpend the value with the SUBPACKAGE name, e.g.,

.for v in PREFIX PKG_ARCH DESCR PLIST
  .for s in ${MULTI_PACKAGES}
$v$s ?= $$v
.  endfor
.endfor

Dependencies are tricky. For MULTI_PACKAGES to work well, you have to
separate build dependencies and run dependencies, because a build can
produce packages with inter-dependencies.

If you don't want to, you have to check dependencies once during build
(just remove all the stuff that mention the same pkgpath), and a second
time before install.

You also have to decide what targets like install will mean: install
all subpackages, or just the main one  (especially with a DESTDIR framework)