Subject: Re: Ruminations on pkg_chk...
To: Bruce J.A. Nourish <bjan+tech-pkg@bjan.net>
From: Frederick Bruckman <fredb@immanent.net>
List: tech-pkg
Date: 12/30/2003 21:43:34
On Tue, 30 Dec 2003, Bruce J.A. Nourish wrote:

> Several people (including myself) have complained about pkg_chk's rather
> poor handling of complex dependency graphs, that in practice results in
> packages bieng repeatedly rebuilt during an update.
>
> I spent the afternoon trying to hack pkg_chk to tsort the update/install
> list. The result looks so ugly, I don't even want to use it myself.
>
> The main problem is the way pkg_chk represents the packages it is
> going to install/upgrade: a list of 2-tuples of (name, dir). It is
> impossible to tsort the list in that form. The obvious solution,
> to make an associative array with one data as key, and the other
> as value is not practially possible with any of the base system shells.
>
> If anyone has any bright ideas on this, perhaps they'd care to share
> them with me?

Here's a utility I wrote to help with upgrades:

    ftp://ftp.netbsd.org/pub/NetBSD/misc/fredb/pkgorder.tar.gz

It first parses pkgsrc to build a handful of databases in
"~/.pkgorder", which is the most time-consuming part. With the
databases made, you can quickly pull lists of packages in many
different ways. (If you utilize the option to limit to installed
packages, though, that takes longer.) It's not perfect; I usually
have to tweak the lists in some way after making them.

Once you have the list of packages, of course, it's easy to operate
on them. E.g, assuming "pkgsrc" is in ${CDPATH}, the following would
download the distfiles needed to update "audio/libmad" and all
installed packages which require it:

  # cd /var/tmp
  # pkgorder -dIr audio/libmad > PKGLIST	# tweak as needed
  # for p in $(cat PKGLIST); do pd $p; done
  # while [ ! -f PKGLIST ]; do make fetch-list|sh && popd || break; done

You could start the fetches in one shell; in another, "deinstall"
(going the other way), then again "package", going forward.

TECHNICAL DETAILS:

Realize that a directive of the form. "A requires B, C, and D", is
equivalent to the directives "A requires B", "A requires C", and "A
requires D". The later form can be fed to "tsort".

I did some expermenting with shell array's, and with awk arrays, but
the best performance and reliability comes from creating files that
awk can process in one pass.

Frederick