Subject: Re: problems with choosing a Berkeley DB
To: Todd Vierling <tv@duh.org>
From: Johnny C. Lam <jlam@NetBSD.org>
List: tech-pkg
Date: 01/05/2005 21:39:20
On Wed, Jan 05, 2005 at 02:07:48PM -0500, Todd Vierling wrote:
> On Tue, 4 Jan 2005, Jeremy C. Reed wrote:
> 
> > I'd like to suggest that we pick a DB and stay with it.
> >
> > Also, maybe some packages maybe should use the lightest weight db.
> 
> > One idea would be to phase out db3 (except for rare package that requires
> > it). Use db2 for everything needing a light-weight db (if that is true
> > that db2 is quicker and/or smaller). And then use db4 for rest.
> 
> The policy of pkgsrc seems to be slanted toward consistency, so perhaps db4
> should be the default globally, nuking db2/db3, and bumping PKGREVISIONs on
> db4's soname bump.  I can't speak for db2 vs. db4 resource consumption,
> however.

The goal should be to allow pkgsrc to automatically build packages
and not have to worry about packages that are linked simulaenously to
multiple versions of -ldb[234].  The main problem here is essentially
the one reported in PR pkg/21040: suppose we build cyrus-sasl against
db[23] and then try to build cyrus-imapd against db4 -- then imapd will
be linked against both libdb4 and libdb[23], which breaks at either
link time or run time.

> One thing I definitely don't want to see disappear is db1 support.  Though
> db4 could be the default, and the trigger for bumping PKGREVISIONs, there is
> still a pretty sizable chunk of software that is perfectly happy with
> db1.8x.  So BDB_DEFAULT should still be settable by the user to "db1" to
> make use of db1 in packages that support it.

Judging from my examination of the db4 sources, I believe that there
is no conflict between libdb1 and libdb4, meaning that it's safe for
an application to link against both -ldb1[*] and -ldb4.  Therefore, I
think we should do the following:

1. If a package required a db-1.85 interface, then the package should
   mark itself as either USE_DB185=yes, or list the "db1" in BDB_ACCEPTED.

2. BDB_DEFAULT is set in /etc/mk.conf for the life of pkgsrc and
   represents the package we use when either (1) a db-1.85 interface
   isn't required, or (2) a db-1.85 interface *is* required, but it's
   not builtin.  It defaults to "db4" but can be any of db[234].

3. BDB185_DEFAULT is set in /etc/mk.conf for the life of pkgsrc and
   represents the package we use when a db-1.85 interface is required.
   It defaults to "db1" if it's builtin, or to ${BDB_DEFAULT}
   otherwise.[**]

4. If we want db1, then we use the package named in BDB185_DEFAULT.
   If we don't want db1, then we use the package named in BDB_DEFAULT.

5. If ${BDB_DEFAULT} isn't listed in BDB_ACCEPTED for a package, then
   that package will be marked with PKG_FAIL_REASON as "${BDB_DEFAULT}
   not a supported Berkeley DB type for ${PKGNAME}."

Using the above scheme, on NetBSD, the default of BDB_DEFAULT=db4 and
BDB185_DEFAULT=db1 will cause db1 to be used whenever we want a db-1.85
interface and db4 to be used for all other packages.  If we want to
force using db4 for everything, then we set BDB185_DEFAULT=${BDB_DEFAULT}.

This scheme will avoid the problem of package dependencies depending
on the order of the packages built, and still allows enough flexibility
to allow us to select which Berkeley DB package we wish to use, e.g.
set BDB_DEFAULT=db3 for backwards-compatibility reasons, and also
allows us to not absolutely require a pkgsrc Berkeley DB package when
the builtin one will do.

Comments?

	Cheers,

	-- Johnny Lam <jlam@NetBSD.org>


[*] I say "-ldb1" but on NetBSD, this is simply the db-1.85 interface in
    libc.  I may be importing a db1 package at some point in the future.

[**] If databases/db1 is ever imported, BDB185_DEFAULT will always
     default to "db1".  In that case, the value of PREFER_{NATIVE,PKGSRC}
     will be used to determine whether we use the pkgsrc db1 or not.