Re: PKG_DBDIR migration script

[Simon Burge <simonb%NetBSD.org@localhost>]

Hi Greg,

Greg Troxel wrote:

> I just committed a script to deal with PKG_DBDIR migration; it lives in
> the top-level of /usr/pkgsrc as _NetBSD-pkgdb.
>
> [ ... ]
>
> This script can "check", "fix", or "migrate".  I've tested it on a
> number of my systems but I'm running out of computers with pkgsrc.
>
> Using "check" can be done as non-root and should be entirely safe.  It
> simply figures out where your PKG_DBDIR is, checks if that's set in
> pkg_install.conf (/etc and /usr/pkg/etc) and in the mk.conf world, and
> checks your refcount dir for referencs to the other PKG_DBDIR location.
>
> The "fix" operation will add PKG_DBDIR= lines to config files, and
> patch up refcount files.
>
> The "migrate" operation will do all the above, and actually move the
> database.

I have NetBSD 9.0 and pkgsrc-2020Q4. I usually do a build of new
packages in a clean chroot made with pkgtools/mksandbox and use
"pkg_chk -ausk" to build packages.  The only path related option
in my /etc/mk.conf is:
    
    PKG_SYSCONFBASE?=       /etc/pkg

Looking at http://pkgsrc.org/pkgdb-change/ there appeared to be no
special instructions I should have followed before starting a build.

After "pkg_chk -ausk" finished, I see:

    thoreau 1158> ls -l /usr/sbin/pkg_info /usr/pkg/sbin/pkg_info
    ls: /usr/pkg/sbin/pkg_info: No such file or directory
    -r-xr-xr-x  1 root  wheel  107680 Jan 10  2020 /usr/sbin/pkg_info*
    thoreau 1159> pkg_info
    thoreau 1160> pkg_info -K /usr/pkg/pkgdb | wc
         839    6109   55315
    thoreau 1161> pkg_info -V
    20191008
    thoreau 1162>

    thoreau 1210> ./_NetBSD-pkgdb check
    Found PKG_DBDIR as /usr/pkg/pkgdb.
    WARNING: /etc/pkg_install.conf does not exist
    WARNING: /etc/pkg_install.conf does not set PKG_DBDIR to /usr/pkg/pkgdb
    WARNING: /usr/pkg/etc/pkg_install.conf does not exist
    WARNING: /usr/pkg/etc/pkg_install.conf does not set PKG_DBDIR to /usr/pkg/pkgdb
    NB: Did not find /usr/pkgsrc -- did not check mk.conf
        If your system does not have sources, this is ok.  If it does, symlink
        /usr/pkgsrc to the sources and rerun the check.
    TOTAL PROBLEMS FOUND: 2
    Re-run script until there are no problems!

A "fix" or a "migrate" fixes these:

    thoreau 1222> priv ./_NetBSD-pkgdb migrate
    Found PKG_DBDIR as /usr/pkg/pkgdb.
    WARNING: THIS SCRIPT IS NOT YET ADEQUATELY TESTED.
    WARNING: Hit ^C NOW if you do not have a good backup.
    WARNING: Proceeding in 10 seconds...
    WARNING: migrate requested when already done
    WARNING: /etc/pkg_install.conf does not exist
    WARNING: /etc/pkg_install.conf does not set PKG_DBDIR to /usr/pkg/pkgdb
 >> sed: /etc/pkg_install.conf: No such file or directory
    WARNING: /usr/pkg/etc/pkg_install.conf does not exist
    WARNING: /usr/pkg/etc/pkg_install.conf does not set PKG_DBDIR to /usr/pkg/pkgdb
 >> sed: /usr/pkg/etc/pkg_install.conf: No such file or directory
    NB: Did not find /usr/pkgsrc -- did not check mk.conf
        If your system does not have sources, this is ok.  If it does, symlink
        /usr/pkgsrc to the sources and rerun the check.
    TOTAL PROBLEMS FOUND: 2
    Re-run script until there are no problems!
    thoreau 1223> pkg_info | wc
         840    6118   55388
    thoreau 1224> 

You could maybe test for file existing before sed'ing it purely for
cosmetic reasons?  But otherwise all good.

Going back to the start, if I make a fresh sandbox on NetBSD 9.0 I see
this:

    # cd /NetBSD/pkgsrc
    # ./_NetBSD-pkgdb check
    FATAL: /usr/pkg/bin is not a directory

Should "check" handle the case where we have a clean install of NetBSD
9.0 or NetBSD 9.1?  Or do we want a separate "init" for this?  I'm not
sure how, but http://pkgsrc.org/pkgdb-change/ should try to cover this
case - maybe a section for clean install?

One nit:

    thoreau 1218> ./_NetBSD-pkgdb fix
    Found PKG_DBDIR as /usr/pkg/pkgdb.
    WARNING: THIS SCRIPT IS NOT YET ADEQUATELY TESTED.
    WARNING: Hit ^C NOW if you do not have a good backup.
    WARNING: Proceeding in 10 seconds...
    	[ waiting ... ]
    FATAL: Not root: will not be able to fix or migrate

Do the not-root check before the wait?


Can I also suggest a -n option to "fix" and "migrate" to see the
proposed mv's, sed's and echo's?  Just a bit of extra confidence!


Hope all this helps, and thanks for your the research and the script.

Cheers,
Simon.