Subject: Re: Toolchain Update (27-Nov-2001)
To: Todd Vierling <tv@wasabisystems.com>
From: Frederick Bruckman <fredb@immanent.net>
List: tech-toolchain
Date: 11/29/2001 09:49:23
On Thu, 29 Nov 2001, Todd Vierling wrote:

> On Thu, 29 Nov 2001, Frederick Bruckman wrote:
>
> : How about, the top-level Makefile builds and installs a whole toolchain
> : in place (after updating "tools", and using "tools" to build it), only
> : if DESTDIR is not set?  Then, the bulk of the build, as well as any
> : single directory builds, can use the installed toolchain (if DESTDIR is
> : not set). For the "DESTDIR not set" case, this would build the compiler
> : one more time than it probably needs to be, but on the other hand, it
> : would lift the temptation/necessity to do the whole build twice.
>
> Installing src/tools into /usr would wreak all kinds of havoc; src/tools and
> the src/{usr.bin,gnu} tools have very different audiences.  The separate
> $TOOLDIR configuration ensures that the system-build toolchain is kept
> *separate* from the installed toolchain.

No. I don't mean to install the (renamed) "tools", into /usr, I mean to
do another pass -- that is, to update and install the in-tree compiler,
make, etc., then fork off another "make" and build with that. If you're
not using "DESTDIR", you're going to update all that stuff anyway, so
why not use it?

> : Folks who are tracking the release branch and updating their systems
> : selectively would get the effect they're accustomed to (i.e. no
> : "tools"). Only builds from the top-level would get the full treatment.
>
> Toolchain components have been updated on release branches in the past
> (rpcgen comes to mind).  Release branches should be no exception to the
> default rules.
>
> It's configurable, anyway.  "USETOOLS=never" exists for this purpose.

OK, so what happens if someone tries to "cvs update" only
"gnu/usr.sbin/sendmail" and "gnu/dist/sendmail"? If they try that, they
must have some reasonable expectation that it'll work. If it's doomed to
fail because of "rpcgen" or whatever, it's gonna build a whole toolchain
(the wrong one), and still fail. So then you must say, "No, sorry, you
have to update foo, bar, baz, first -- please just "cvs update" and
build the whole thing from the top", then they'll have to do that, and
that'll work. There's never a need for the user to monkey with the
USETOOLS setting.

> : Anyone doing a cross-build, or anyone who didn't want his installed
> : system messed up, would necessarily have to have DESTDIR set, so he'd
> : always build with "tools", just as now.
>
> DESTDIR isn't *really* relevant.  The default of USETOOLS=yes for full
> source trees exists because full source trees imply updating the system
> incrementally from some base point (whether a release branch or -current).

But "make release" or "make build", would always get the full treatment.
I suggest that if someone types "make includes dependall install" in
"sendmail", they never, ever really want to rebuild the full toolchain.
If they want/need to build a new toolchain, they'll take it from the top.

The thing about using DESTDIR as the hook is: the arguments for never
overwriting anything in the base system break down completely when you
fail to have DESTDIR set.

> In incremental updates, only a separate toolchain pass can give us some
> reasonable hope of automating the update process.  The way we've handled
> UPDATING in the past has been too cumbersome and tricky to get right.

Acknowledged. Here's the point: I say if one pass is good, TWO IS
BETTER. The benefit is, after the "tools" are built, and the "toolchain"
is built, it looks and acts much like the old system.

> It's reasonable for bsd.pkg.mk to include <bsd.own.mk> to get some intimate
> knowledge of the installed system.  It's therefore primarily the
> responsibility of bsd.pkg.mk/bsd.prefs.mk to ensure that functionality added
> to the base system doesn't interfere with development of pkgsrc.

No argument there. I wouldn't like to see work-arounds for pkgsrc
problems in "bsd.own.mk", not at all. My concern is that some portable
software, somewhere, will actually meet the apparent requirements for
being in the NetBSD source tree ("build.sh" is not an unusual name).
Granted, someone setting "DESTDIR" on such a package would still have a
problem, but "DESTDIR" setting is an advanced thing, and even though
it's supported by the popular build systems, it doesn't necessarily work
in the general case, so tough luck.

Frederick