Script pkg_tsort

[tlaronde%polynum.com@localhost]

Hello,

Here is a simple Bourne shell script that takes what is currently
installed on a node, takes the build dependencies and the running
dependencies and produce a series of records in topological order
with the following fields:

pkg_path installed_version vuln_status\
	src_version vuln_src_status\
	os os_release os_proc_arch

By default, the packages with PRESERVE set (i.e. +PRESERVE file in the
pkg database) are not listed and their _running_ dependencies are
"frozen" i.e. not put in the final listing.

The main use for me is a help when upgrading in order to be able to
launch updating having an idea about what has to be done and what is
about to succeed or not.

The topological sorting is done with NetBSD rcorder(1) that gives as a
bonus dependencies that are not present (typically build dependencies
that one has suppressed).

The vulnerability is tested both for the current packages and for the
source version (i.e. the version in /usr/pkgsrc) so that if one wants to
automate the updating, one knows that one has to set
ALLOW_VULNERABLE_PACKAGES for the updated version that has
vulnerabilities.

I have tried initially to set the version of the packages in /usr/pkgsrc
simply by sed'ing the Makefile. But since there are, in some
packages, a bunch of variables replacements, I had to fall back to
calling "make show-vars":  this is what takes the huge majority of
time...

It is a simple script and its purpose is to give informations, upon
which one can make decisions. It does not update anything (Unix
philosophy: pipe programs. It is simple, after that, to select the
packages based on the values in the different fields (it is not magic:
some things are so tangled that a package will refuse to build with an
old frozen version required by a PRESERVE(d) package etc.; but at least
one can sort the things before).

I didn't find the equivalent in existing so it might be of some use for
others.

It does what I intended it to do, but it has been done rapidly and it
could be improved (if the "improvment" is not to try to add "actions"
to it; this would NOT be an improvment).

Cheers,
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

#!/bin/sh -e
#$Id: tsort,v 1.1 2019/04/15 14:59:23 tlaronde Exp $
# Gives in topological order informations about all or some installed
# packages for further processing of the records.
#
# C) 2019 Thierry Laronde <tlaronde%polynum.com@localhost>
# BSD 2 clause license.
#
# Requires: NetBSD rcorder(1) and pkg_admin(1).

: ${TMPDIR:=/tmp}
: ${PKGSRCDIR:=/usr/pkgsrc}

usage="$0 [-h] [-c] [pkg...]
	Print in topological order the tree of installed packages as
	registered in PKG_DBDIR (value retrieved via pkg_admin(1)) with
	the build and running dependencies, giving the details of installed
	version, status for vulnerabilities, source (PKGSRCDIR retrieved in
	environment) version, status of source for vulnerabilities,
	installed OS version, release and processor architecture.

	Options:
	-h	print this Help
	-c	Complete packages. By default, preserved packages are NOT
		listed.
"

#========== SUBR

#========== CHECKS
do_complete=NO
pkg_list=
while test $# -gt 0; do
	case "$1" in
		-h) echo "$usage"; exit 0;;
		-c) do_complete=YES;;
		*) pkg_list="$pkg_list $1";;
	esac
	shift
done

# Required.
#
test $(command -v pkg_admin)\
	|| { echo "pkg_admin(1) is required!"; exit 2; }

PKG_DBDIR=$(pkg_admin config-var PKG_DBDIR)  

tmpdir="$TMPDIR/$$"

#========== PROCESSING: stores
#
mkdir "$tmpdir"
list="$tmpdir/list"

# Take all at first.
#
find "$PKG_DBDIR" -type d -maxdepth 1\
	| sed -e "\\!$PKG_DBDIR\$!d" -e "s!$PKG_DBDIR/!!"  >"$list"

# Select using regexp if pkg are specified on cmd line.
#
if test "$pkg_list"; then
	rm -f "$tmpdir/junk"
	for pkg in $pkg_list; do
		grep "$pkg" "$list" >>"$tmpdir/junk"
	done
	mv "$tmpdir/junk" "$list"
fi

mkdir "$tmpdir/pkg"

# If we exclude preserved packages, we need to exclude from update
# their _running_ dependencies.
#
if test $do_complete = NO; then
	echo "Looking for PRESERVE(d) dependencies..." >&2
	pkg_info | while read pkg junk; do
		test -f "$PKG_DBDIR/$pkg/+PRESERVE" || continue
		sed -n 's/^@pkgdep \(.*\)-[0-9.]*$/\1/p' "$PKG_DBDIR/$pkg/+CONTENTS" >>"$tmpdir/junk"
	done
	sort "$tmpdir/junk" | uniq >"$tmpdir/frozen"
	echo "Excluding PRESERVE(d) dependencies: " >&2
	cat "$tmpdir/frozen" >&2
fi

echo "I will call make to retrieve current variables, so be patient..." >&2
while read pkg junk; do
	test do_complete = YES || test ! -f "$PKG_DBDIR/$pkg/+PRESERVE"\
		|| continue
	name=${pkg%-*}
	version=${pkg##*-}
	if pkg_admin audit-pkg $pkg >/dev/null 2>&1; then
		status=OK
	else
		status=VULNERABLE
	fi
	if test -f "$tmpdir/frozen"; then
		! fgrep "$name" "$tmpdir/frozen" || continue
	fi

	pkgpath=$(sed -n 's/^PKGPATH=//p' "$PKG_DBDIR/$pkg/+BUILD_INFO")
	uname_s=$(sed -n 's/^OPSYS=//p' "$PKG_DBDIR/$pkg/+BUILD_INFO")
	uname_r=$(sed -n 's/^OS_VERSION=//p' "$PKG_DBDIR/$pkg/+BUILD_INFO")
	uname_p=$(sed -n 's/^MACHINE_ARCH=//p' "$PKG_DBDIR/$pkg/+BUILD_INFO")
	if test -f "$PKGSRCDIR/$pkgpath/Makefile"; then
		srcversion=$(cd "$PKGSRCDIR/$pkgpath"; make show-vars VARNAMES=PKGVERSION)
		if pkg_admin audit-pkg $name-$srcversion >/dev/null 2>&1; then
			srcstatus=OK
		else
			scrstatus=VULNERABLE
		fi
	else
		srcversion=UNAVAILABLE
		srcstatus=UNKNOWN
	fi

	sed -n 's/^@[a-z]*dep \(.*\)-[0-9.]*[nb0-9]*$/# REQUIRE: \1/p' "$PKG_DBDIR/$pkg/+CONTENTS" >"$tmpdir/junk"

	# Fix for xorgproto bunch.
	#
	if grep "# REQUIRE: .*proto" "$tmpdir/junk" >/dev/null 2>&1; then
		ed -s "$tmpdir/junk" <<EOT
g/^# REQUIRE: .*proto/d
a
# REQUIRE: xorgproto
.
w
q
EOT
	fi

	sort "$tmpdir/junk" | uniq > "$tmpdir/pkg/$name"
	echo "# PROVIDE: $name" >>"$tmpdir/pkg/$name"
	echo "##PKGINFO: $pkgpath $version $status $srcversion $srcstatus $uname_s $uname_r $uname_p" >>"$tmpdir/pkg/$name"
done <"$list"

# The list is given as is.
# One can use "pkg_info -R pkg" to see what depends upon pkg.
#

cd "$tmpdir/pkg/"

echo "##pkgpath cur_version cur_status src_version src_status cur_OS cur_OS_release cur_OS_procarch"
rcorder * | while read name; do
	sed -n 's/^##PKGINFO: //p' "$tmpdir/pkg/$name"
done

rm -fr "$tmpdir"

exit 0