Re: pkg.refcount database as a flat file?

["Johnny C. Lam" <jlam%NetBSD.org@localhost>]

On Sun, Feb 12, 2017 at 04:17:59PM +0000, Taylor R Campbell wrote:
> > Date: Sat, 11 Feb 2017 17:31:11 +0000
> > From: "Johnny C. Lam" <jlam%NetBSD.org@localhost>
> > 
> > Over the years, has there been any discussion about changing the
> > refcount database from a directory tree under /var/db/pkg.refcount
> > to a text file, perhaps in structured format like JSON or a Lua
> > table?
> 
> I'd be more inclined to use a sqlite3 database -- for this and the
> rest of the pkgdb.
> 
> Atomic backup:  echo .backup /path/to/backup | sqlite3 /var/db/pkg.refcount
> Diff:  diff -u <(echo .dump | sqlite3 old) <(echo .dump | sqlite3 new)
> More advanced manipulation:  any SQL queries under the sun.
> 
> Updates become transactional, &c.  Can process with awk, e.g.:
> 
> echo 'select * from pkgrefcount' | sqlite3 | awk -F'|' ...
> 
> (This is subject to some constraints about the character `|', of
> course; can get quoted CSV with `.mode csv'.)

I've abstracted the refcount manipulations into shell functions to be
used by the install scripts so it will be easy to switch out refcount
implementations later.

> There is, of course, a problem of gracefully transitioning any changes
> to the file format in existing installations.

The transition would be as follows:

1. Update pkgtools/pkg_install to use the new implementation and then
   be able to generate on-the-fly the package-specific parts of
   /var/db/pkg.refcount for the package(s) given on the command line.

2. Change the INSTALL scripts generated in pkgsrc-current to the new
   implementation and bump PKGTOOLS_REQD in pkgsrc-current to the
   latest version.

3. Wait through two quarterly tags of pkgsrc.

4. Remove the old implementation from pkgtools/pkg_install. There is
   no need to bump PKGTOOLS_REQD in pkgsrc-current.

I plan on figuring out what refcount implementation to switch to
after my changes to mk/pkginstall are in pkgsrc-current.

-- Johnny C. Lam