Re: Moving mk.conf

[David Holland <dholland-pkgtech%netbsd.org@localhost>]

On Tue, Dec 14, 2010 at 08:28:15PM +0100, Alistair Crooks wrote:
 > > > This is mainly because I haven't the faintest clue what you're trying
 > > > to achieve, and, when I think I do find something, it's overlaid with
 > > > dogma and religion about assumptions that things must be done in a
 > > > certain way.  pkgsrc was designed to be flexible, to be able to
 > > > co-exist in a number of different instances, and installed
 > > > hierarchies, which is why I'm having problems here.
 > > > 
 > > > So help me out here, please - what exactly are you trying to achieve?
 > > 
 > > asau and abs and I thrashed this out earlier this evening (or last
 > > night, depending on $TZ) on irc and I'll summarize when I'm awake.
 > 
 > That's great, but it would be good to discuss it here on the mailing
 > list, or let people know when the thrashing on irc is going to happen,
 > so that others can have their say too.

Nah, I was unclear. What we thrashed out was "what exactly are you
trying to achieve?" and a large part of the answer turned out to be
"not breaking everything".

Currently, pkgsrc finds its configuration by reading <bsd.own.mk> out
of make's include path. On a nonbootstrapped NetBSD system this reads
/etc/mk.conf. On a bootstrapped pkgsrc install, make's include path
has the proper pkgsrc prefix compiled into it, and a copy of
bsd.own.mk is installed in $(PREFIX)/share/mk that has had the proper
sysconfigdir baked in so it can include mk.conf from there.

The basic thrust of the disagreement seems to have been that changing
whether mk.conf is the config file requires rearranging this, and
rearranging it incautiously will break it.

The current setup is less than ideal on three counts:

 - The NetBSD base system's build configuration is mixed with pkgsrc
build configuration; there have been conflicts and mixups in the past
and there doubtless will be again.

 - pkgsrc should not really be including bsd.own.mk, because it's part
of the base system's build infrastructure and much of what it defines
is not (or no longer) relevant to pkgsrc.

 - It's not really desirable to bake the pkgsrc directory
configuration into installed files, and where it can be avoided it
should be.

There's also some disagreement over whether the pkgsrc configuration
should be (by default) per-system, per-distinct-$(PREFIX), or
per-distinct-pkgsrc-tree. Some people use multiple pkgsrc trees with
the same prefix; others use multiple prefixes with the same pkgsrc
tree. This means any choice for its default location will be wrong for
somebody.

We also want to not break random third-party code that uses the
bsd.*.mk make library. This means two things though: (1) it should not
be exposed to the pkgsrc build configuration, in case that conflicts
with it, and (2) the make library that pkgsrc needs to be able to
configure *its* directory locations safely without getting entangled
in pkgsrc definitions.


We came up with three proposals for fixing the above issues and moving
forward. These assume that the base system config remains "mk.conf"
and that we'll call the pkgsrc config "pkgsrc.mk.conf" - obviously the
mechanisms are independent of the exact names.

(1)

 - Take the definitions from bsd.own.mk that are relevant to/wanted in
pkgsrc (which isn't too many AFAICT) and move them to bsd.prefs.mk or
some other file in pkgsrc/mk.

 - Have bootstrap install a different file in $(PREFIX)/share/mk,
perhaps pkgsrc.boot.mk or something like that, that contains either
nothing but the configured definition of sysconfdir, or alternatively
all the pkgsrc path configuration definitions that can't be safely
redefined without rebootstrapping.

 - Also have pkgsrc.boot.mk set MAKECONF=$(SYSCONFDIR)/pkgsrc.mk.conf.
Then have bsd.prefs.mk establish the path defaults and do
   MAKECONF=/etc/pkgsrc.mk.conf
   .-include <pkgsrc.boot.mk>
   .-include "${MAKECONF}"

This will DTRT for both bootstrapped and unbootstrapped installs: a
bootstrapped install will use the paths provided in pkgsrc.boot.mk
and/or the bootstrap-chosen pkgsrc.mk.conf file, and a native
unbootstrapped install will find no pkgsrc.boot.mk and use /etc.

 - We can also then avoid baking paths into bootstrap-mk-files,
because they can use pkgsrc.boot.mk to get the path configuration.
However, the bsd.own.mk installed by pkgsrc should probably include
$(SYSCONFDIR)/mk.conf rather pkgsrc.mk.conf.

(2)

 - Do the same, but instead of pkgsrc.conf.mk, have bootstrap compile
a __MAKECONF__ variable into make.

 - Then have bsd.prefs.mk check if it's defined; if so set MAKECONF
from it, and if not use the file from /etc.

This is both more and less tidy; on the one hand, it avoids installing
the extra file, but on the other hand, compiling stuff into make is
ugly. However, it means that bootstrap-mk-files still needs to bake in
directory config, which is less desirable.

(3)

 - Put pkgsrc.mk.conf in the root of the pkgsrc tree, or in
pkgsrc/conf or some such place. Then it can always be found without
needing any path configuration whatsoever. For bootstrapped installs
it can safely contain the complete path configuration.

The downside is that it doesn't allow avoiding baking paths into
bootstrap-mk files. Also, it complicates the lives of people who use
one pkgsrc tree with multiple separate $(PREFIX)es.


All of these should make it possible to avoid needing
bootstrap-mk-files at all except for specific packages that use the
NetBSD make library. This seems like a positive development to me.
(We'd need to change things around so sys.mk is handled by devel/bmake
instead of bootstrap-mk-files; but that should be done sometime
anyway, as sys.mk is part of make in a way that the NetBSD make
library isn't.)

Of these I think the best approach is some mixture of #1 and #3; that
is, bootstrap should proceed as per #1, but I think explicitly
supporting having config in the build tree as well is desirable.

The other choice, of course, is doing nothing, as the extant problems
aren't really that serious.

(I'd also like to make a token protest in favor of having config files
that aren't Turing-complete, but this currently isn't realistic.)

-- 
David A. Holland
dholland%netbsd.org@localhost