Subject: Re: using pkg_comp to rebuild out-of-date packages?
To: Steven M. Bellovin <smb@cs.columbia.edu>
From: segv <segv@netctl.net>
List: tech-pkg
Date: 08/15/2005 21:40:28
On Mon, 15 Aug 2005 14:29:09 -0400
"Steven M. Bellovin" <smb@cs.columbia.edu> wrote:

> Is there a good way to use pkg_comp to rebuild just the out-of-date
> packages?
> 
> I know how to use pkg_comp to rebuild everything: use pkg_chk -g
> to produce a list, and put that into the pkg_comp file.  I can also 
> delete/rebuild just the out-of-date stuff by using lintpkgsrc -i
> and pkgdepgraph -D/-R.  I'd like to do the latter in a chroot'd 
> partition without having to recompile -- or even pkg_tarup plus install 
> under chroot -- all of the dependencies that aren't out of date.  Is 
> there any good way to do that?  (If I had to, I'd settle for something 
> that automatically built tarballs of the dependencies, for automatic 
> preinstallation in the pkg_comp area, but I don't even know a good way 
> to do that.)
> 

Not sure about pkg_comp. I developed my own scripts to automate package
upgrades and NetBSD kernel+userland compilation. I'll give you a quick tutorial
on how to use my scripts, the best thing is to also look at the scripts and see
how they work.

The scripts are tailored to my system, where I keep my pkgsrc/src/xsrc
under /opt. If you want to change that, you need to edit 'dirdef' file, but I
suggest you simply create /opt and follow my instructions.

If you have enough disk space, backup your existing packages:

cp -Rp /usr/pkg /usr/pkg.orig
cp -Rp /var/db/pkg /var/db/pkg.orig

Create /opt directory that will contain NetBSD sources and object files

mkdir /opt /opt/src /opt/xsrc

Move your pkgsrc directory to /opt
/opt/src and /opt/xsrc must exist, but can be empty if you're interested in
building NetBSD kernel

Create a chroot jail and a directory inside chroot jail that contains my
scripts and distfiles

mkdir /opt/sandbox /opt/sandbox/scripts
mkdir -p /opt/sandbox/opt/distfiles

Move your distfiles to /opt/sandbox/opt/distfiles/ directory

Download my scripts from http://www.netctl.net/scripts/
into /opt/sandbox/scripts I assume you're building on i386, so
edit /opt/sandbox/scripts/mk.i386.conf file and customise optimisation flags to
suit your architecture. Mine are set for Pentium 3

Now make sure your have pkgtools/pkglint and pkgtools/pkgdepgraph installed
before you do anything else. You now need to run /opt/sandbox/scripts/pkgupdate
to generate lists of packages to update/delete/install etc. You need to run
that script in the following way:

cd /opt/sandbox/scripts
./pkgupdate outdated_list
./pkgupdate delete_list
./pkgupdate rebuild_list
./pkgupdate install_list

Now you need to build your chroot jail and chroot into it. Make sure there are
no errors when you run the scripts. Running chroot.mkdirs needs to be done only
once, when you're creating a new chroot jail. When it's done, you can simply
chroot into it by running chroot.enter.

cd /opt/sandbox/scripts
./chroot.mkdirs
./chroot.enter arch=i386

What this does is creates system directories inside /opt/sandbox, then
'mount_null -o rdonly' your system directories on the directories
inside /opt/sandbox and then enters chroot jail.

Once you're inside chroot jail, make sure your can ping google.com in case you
need to fetch distfiles from the internet.

To start building a list of new packages, do this:

cd /scripts
./pkgupdate rebuild_outdated

Once all the packages have been built and there are no errors, you can exit
chroot jail and install them. All new packages will be created
under /opt/sandbox/opt/packages.i386

exit
./chroot.umount
./pkgupdate delete_outdated
cd /opt/sandbox/opt/packages.i386/All
cat /opt/sandbox/scripts/pkglog_install | xargs pkg_add


Any questions, feel free to ask me :-)