Subject: Re: Building -current from 1.4.1?
To: der Mouse <mouse@Rodents.Montreal.QC.CA>
From: Chuck McManis <cmcmanis@mcmanis.com>
List: port-vax
Date: 03/13/2000 15:34:16
Sir Mouse, et al.

Its a RANT ok? A rant expresses frustration and angst and a wish for change
that is not percieved to be possible. Ok? So take it in that vein.

>It sounds to me as though you'd be a whole lot happier if you'd just
>forget -current exists at all.

A bit of background. I've been building UNIX systems for a long time now. I
started in the kernel group at Sun Microsystems when the "current" release
was SunOS 3.5 and lived through the hell that was involved with getting
Solaris to build/run on multiple platforms (aka Solaris 2.x). I then got
tired of working on UNIX for a while and did security stuff on the Java
development team (which also had an interesting build experience). I've
been around the block a few times, fair enough?

In vax-land a lot of critical stuff has happened in -current and not in
1.4.x, specifically on-board ethernet support and SCSI support for the
M3100's (and 4000's). I'd rather build with one of those than the 3800
sitting here.

> There's no guarantee *any* environment exists under which any particular
> instant's -current builds.

The above statement is of course true, my comment though was that (IMHO)
the methodology for doing a build should have some guarantees around it.
Things like "make build" will always actually attempt to build the system,
the environment variable set (X, Y, and Z) should be set to (this, that,
and the_other) etc. The benefit I see in this sort of "dictum" if you will
is that it makes building the system an automatic process without human
intervention (except when the code breaks of course)

>-current exposes the Project's internal working tree.  If this
>constitutes "lack of discipline" to you, well, as I said, you'd
>probably be happier pretending we didn't make it accessible.

Resync Mouse, the "exposure of the tree" does not constitute a lack of
discipline, the makefiles in "crypto-us" constitute a lack of discipline.
Using this _very specific_ example, consider the new "Makefile.frag" stuff
that is being tossed about as a way to "plug-in" crypto. The file
	../src/crypto-us/lib/Makefile.frag
Starts with the statement:
	USLIBDIR:=${.PARSEDIR}

This works fine WITH THE NEW MAKE, but it croaks with the version of make
that ships in 1.4.x leaving USLIBDIR set to the null string. Had the
developer 'aidan' who changed it from CRYPTOPATH been a bit more creative,
he/she should could have continued to use CRYPTOPATH and made it compatible
with existing make (I've back ported it and the build is running as I
type). However, instead, he/she chose to use a new feature (in some but not
all!) of the crypto makefiles and broke compatibility with the stable build
tools. 

That is a trade-off choice, that IMHO again, was poorly made and exhibited
what I choose to call a "lack of discipline." The only reason given for
this change in the CVS logs is "... for adding SUBDIR variables in
*/Makefile.frag" which had they simply put
	USLIBDIR:=${CRYPTOPATH}/lib
it would have worked equally well for "adding SUBDIR variables" and it
doesn't break old make.

Please don't construe this as a personal attack on anyone, it isn't. It is
an illustrative example of what I mean when I say "poor discipline." 

>Point 1: why is *your* opinion of a "properly managed development
>environment" necessarily the correct one for the Project?

Lighten up, its a RANT ok? Everyone has the correct opinion in a rant. I
hold this opinion because I've spent a lot of time building systems and
know the ones I like to use and the ones I don't like to use. I can hack
_anything_ to work, but I don't necessarily enjoy it. :-) People who have
used different build systems will have different opinions, so what? 

>Point 2: the paradigm you propose would require vastly more releng
>cycles, because it would mean we would be forced to cut a release each
>time our build tools change.  I see barely enough brains to handle
>releng for what we're doing.  Where do you propose we get the human
>time to support your proposed paradigm?  Are you volunteering?

I would (volunteer that is) but I suspect that more of the younger
generation shares your views Mouse. Which is "it works for me today, what's
your problem?" And it is a volunteer organization so normally I don't
complain. I suspect its like telling my daughter her new boyfriend is a
flake "But Daaaddy, we're in LOVE!" Then six weeks later we're not in love
anymore. I needed to blow off some steam after working several hours on
getting the "obvious" to work. Whatever you do, don't put me in charge of
the build process.

>Remember, this is a volunteer project.  We're making progress with the
>paradigm we've got.  With yours, I see that slowing down drastically,
>perhaps even stopping as people get fed up and switch to a
>faster-moving system.

Those guys are switching to Linux already, it moves faster, is more
chaotic, and it is currently "in vogue." So I doubt putting some structure
or rules around changing the build would scare any more folks off. 

>Each release builds under itself - or it's critically broken.  I don't
>think we've ever promised that any release builds under anything but
>itself, and I see no reason to try.

Yup. I was trying to show you a reason to try, but it isn't critical to my
happiness that you see it. Everyone on the project has done a tremendous
job in getting stuff done, and it is a marvel to be using a wonderful OS on
some "antiquated" hardware as well as newer hardware. 

>As far as I can see, this has nothing to do with -current, and I don't
>see why you're trying to argue it does.

The subject of the rant was that "building the system" and "the system"
should not be so intertwined. In an ideal world, the two halves would be
completely disjoint. Were that true, one could build the "build tools" on a
PC and then use the "build tools" to build the "vax flavor" of the OS.
While it is inevitable that limitations in the build system will have to be
dealt with to facilitate more complex builds, my rant is that the decision
to rely on freshly introduced features into the build tools is taken too
lightly by this community. 

>> Now this is important, if you don't "lose" anything by using the
>> stable tools to build the new release and new tools and you gain a
>> tremendous amount in terms of buildability.
>
>If.  That's an awfully big if.  We would lose, big, by forcing
>ourselves to stick with each release's build tools until the next
>release; it would either force much more frequent releases (which we
>don't have the releng time to support AFAICT) or it would severely slow
>down feature uptake (which is slow enough as it is), because each new
>"need to build the tool to build the tool" iteration would have to
>block waiting for a release.

I must admit I found this response rather unsupported. What aspect of the
release has _required_ the use of new features in the tools? Has anyone
looked at the choice made and said, "can I do this using the existing
feature set?", or "if I use this, could I provide a means of using the new
feature in the stable release?" I don't think these questions ever get
asked. Perhaps they do and I'm just "out of touch" but in an undisciplined
world they tend to not even come up. Its more like "cool if I do this, then
I can do this other thing, if anyone complains we'll just tell them to sync
up with the latest stuff!" 

I'm guessing that if someone actually packaged "the build tools" as a
distinct component (which the gnu toolchain very nearly is, BTW) it would
be possible to "release" the build tools on a relatively frequent schedule
without disrupting the OS releases. And it would make it a lot easier to
know one was correctly building the system as well. 

The important thing to remember here is that "development" is a much
smaller part of the OS delivery mechanism than "change control",
"building", and "testing" are. By putting leverage into the build tools you
can get much more time to do development.

Enough for now, I've got to check on my build...
--Chuck