Subject: Re: Smarter make update / pkg_chk algo
To: None <tech-pkg@netbsd.org>
From: Frederick Bruckman <fredb@immanent.net>
List: tech-pkg
Date: 05/05/2005 03:38:34
In article <20050504081129.GA14546@circe.entropie.net>,
	"Martin S. Weber" <Ephaeton@gmx.net> writes:
> Well then...
> 
> I think I made it obvious that the current choice of algorithm for updating
> several packages at once (i.e. pkg_chk + make update) is sub-optimal, and that
> one can easily plug in something more suitable to the task of upgrading
> several packages at once.

I'm sorry I'm not familiar with "pkg_chk" or "portupgrade" at
all.  This is what I've been using for the last few cycles:

  ftp://ftp.netbsd.org/pub/NetBSD/misc/fredb/pkgorder.tar.bz2

Given a package, or packages, or the option for all installed
packages, (or dependents) it produces a list of packages.  For
example,

  pkgorder -d -r -i

on the workstation I'm at now gives a list of 665 packages, with
"meta-pkgs/gnome-base" at the top, and "archivers/bunzip" at the
bottom.  The list produced by the "-i" option gets more packages
than are installed because of string matches.  (This is done
deliberately, to help deal with packages that have been renamed.)
The list can be pruned by hand, then filled out again (without
the matches), with

  pkgorder -d -r $(< _old_list_) > _new_list_

Once you have your list, it's easy to fetch all the packages with

  for p in $(< _list_); do pd $p; done
  while [ ! -f _list_ ];
  do make fetch-list | sh && popd || break;
  done

The build is done the same way:

  for p in $(< _list_); do pd $p; done
  while [ ! -f _list_ ];
  do (make package && make clean) && popd || break;
  done

The shell is "/bin/ksh" plus the contributed shell functions
"pushd" and "popd", which are in the sources, but not installed
by default.  The full path to "pkgsrc" needs to be in ${CDPATH}.
All that should probably work with "bash", too.

The beauty of it is, when the build breaks -- something always
breaks -- you're in the directory you need to be in to fix it.

If you'd rather use a "read...while" loop on the list, or a "for"
loop, leave off the "-r".

The last time I brought this up here, I got some good suggestions,
particularly to make use of the "pkgdep" database, but I haven't
gotten around to implementing that yet. I have some more ideas for
grouping and batching, so it's not quite finished yet, but you may
find it reasonably useful as it is.

-- 
Frederick