On Wed, 7 Apr 1999, Chris G. Demetriou wrote:

> I just noticed -- by accident -- that make install and make and make
> includes at the top level seem to do _nothing_ with domestic.  This is
> broken.

Hi Chris,

Well, here's the issue.  Descending into the domestic directory while
handling the includes target, particularly, is in and of itself is broken.
There are at least two trivial ways to provoke lossage:

- Start with a just-installed NetBSD release, sans secr set, and then try a
`make build'.  (You with be at this point if you've just finished an
exportable build for release, too.)  Notice that you can't generate domestic
includes because compile_et and mk_cmds are required, but neither program is
installed or built before the `includes' target.

- Fix the previous problem by compiling compile_et and mk_cmds before the
`includes' step.  Then, do a build to a non-root DESTDIR, and notice that
this fails because you don't have $DESTDIR/usr/include populated, yet.
We've been here before, by the way.

There are also somewhat more subtle problems that complicated the old
`build' target.  The new build target is considerably more robust due to the
fact that the cycles in the dependency graph have been eliminated.  It also
more or less guarantees that the right bits will be built correctly the
first time, rather than relying on an evil hack like `do "make -k includes"
twice'.  The price of this is that you need root access to $DESTDIR and your
object tree.

> Currently, to build domestic bits, you have to make build, or do it by
> hand.

Well, yes, but that's exactly the case with any build, right?  (Are there
any other choices besides `make build' or `do it by hand'?  Note: the
release and snapshot targets don't count. ;-)

> When doing a minor update of the system, i.e. between two 'near'
> versions of the source, previously you could do:
>
>  make -k includes
>  <wait a little bit>
>  su then make -k includes
>  <wait a little bit>
>  make -k
>  <go away for a long time>
>  su then make -k install
>
> used to do the right thing with no pain.  Yes, it might not generate
> perfectly correct results w.r.t. domestic, but if you were doing a
> rebuild soon after a previous build, it was OK.

In fact, in some common cases it can't possibly generate any results, as
noted above.  If you have a domestic tree, you are required to have a
specific environment for this to work.  To wit, (a) the secr set must be
installed and (b) you must either set DESTDIR=/ (or not set DESTDIR at all).

> In order to do the 'equivalent' now, you need to run twice the number
> of commands at least, and if you want to fire off each 'stage' of the
> build one at a time you have to spend a lot more time hand-holding
> and/or watching the build.

If your system really is `close enough' to being up to date, you could do:

make -k includes && cd domestic && make -k includes
<wait a little bit>
su then make -k includes && cd domestic && make -k includes
<wait a little bit>
make -k && cd domestic && make -k
<go away for a long time>
su then make -k install && cd domestic && make -k install

Yup, it's more commands.  I'm not sure that's necessarily a bad thing, but
I'll grant you that it's less convenient.

> Until we can actually do a complete build without privilege (which
> should be the goal; the need to install things is broken), in my
> opinion, even though they do the potentially-wrong thing, the
> top-level build and install targets _should_ descend into domestic, or
> warn loudly about it in a 'break the build unless you define a special
> option indicating you know what's going on' kind of way.

There was real evil going on here, not potential evil.  On the other hand, I
know you're asking for rope, and I don't really care to tell you that you're
not allowed to hang yourself.  The price of providing that rope in the past
has been that the build target is incredibly fragile.  There have been
numerous attempts to `fix' it, and each attempt has caused some loss of
functionality.  (Indeed, I can't recall a time in the last 4 years where it
actually worked properly in all of my various and several environments.)

In addition to robustness, another benefit of the current build target is
that the domestic tree no longer depends on being intertwined the exportable
build.  It all happens after the exportable build is complete, thereby
ensuring that all of the prerequisites are in place before we compile
compile_et and mk_cmds.  Also, by pulling the necessary ordering logic into
the domestic Makefile's build target, we can replace the domestic tree
relatively easily.

Finally, if you already have an exportable release installed and you wish to
upgrade it (using the term loosely) with the domestic bits, you can simply
go to the domestic directory and type `make build'.  This in itself is a
major win both for distribution builds and for the end user wishing to
experiment.

> Failing that, i'd settle for some flag that you could set in mk.conf
> to reenable those targets.

Having said all of the above:  I'll do this, with a big warning sign on it
that says "don't do this unless you Know What You're Doing."  Also, the
automated build target will have to make sure this flag isn't set (which is
easy enough to enforce).

> _BUILD is almost the right thing, but it's
> obviously meant for internal consumption only, and it has undesirable
> side effects (e.g. disabling whatis.db generation.

Actually, if you look carefully, the _BUILD flag does _not_ disable
whatis.db generation in the domestic Makefile.  The procedure I outlined
above should DTRT for you.

--scott
(... who is still trying to allocate time to review PR 7089)