Subject: Re: Why did NetBSD and FreeBSD diverge?
To: None <opentrax@email.com>
From: Terry Lambert <tlambert@primenet.com>
List: netbsd-advocacy
Date: 01/18/2001 06:26:15
Minor nits:
> > So at this point, I am trying to figure out why NetBSD and FreeBSD didn't
> > pool resource early. I know why OpenBSD exists so that is not a question
> > (though great quotes are appreciated:).
>
> Again, the start was 386BSD, then the "Unoffical Patch Kit".
> The UPK was written by Terry Lambert, then run by Dave Burgess
> and later Rodney Grimes. Jordan Hubbart was also a main person into.
> Jordan was one of the founders of FreeBSD.
Rodney Grimes, then Nate Williams and Jordan Hubbard. Dave
Burgess took over the "Unofficial FAQ" from me before I handed
off the patchkit.
As a rule of thumb, unless I'm passionate about the subject (and
so can't trust it to someone else), I have this tendency to get
things that I think are neat or need to work to the point where
they work, and then find someone to hand them to so I can go on
to the next thing on my "important things that won't otherwise
get done" list.
> NetBSD pulled out early from the 386BSD effort. Their direction
> was based on BSD tradition; make run on everything. Again, they
> left mostly because of tensions between the authors of 386BSD
> and the UPK. That is, people were making fixes to 386BSD, but
> the only way to incorporate them for more that 1 1/2 year was
> the UPK.
>
> The UPK had many problems it was a disaster (Sorry Terry).
> The UPK was never intended to run for more than a few months,
> but one (1) year later it was the only to get things to work.
It was a very basic version control system, which relied on a
human being, rather than software, to ensure that order of
operation was maintained.
Contrary to your repeated opinion of my intent in writing the
scripts which created, managed, and installed the patches, the
software itself was intended to last a very long time. It is
still in use today at at least 6 commercial organizations, none
of which, incidently, I ever worked for: the code was adopted
on its merits.
The patchkit had several attributes:
o It ensured patch application would not fail due to
conflicting authors changes to the same file.
o It was the only realistic method of integrating lots
of Usenet-posted patches, without retarding progress
by setting up a control hierarchy, like the one in
effect in all longer-lived open source projects today.
NB: 386BSD lived a very long time -- from the
establishment of the first patchkit release,
through to the establishment of the FreeBSD
0.1 (quickly, 1.0) source tree, all public
work on 386BSD was done in the context of
Usenet postings and/or patchkit patches --
quite fine with the patchkit.
o It worked around the "damage" of the actual source
code control system being unavailable to all but a
few people.
NB: Linux _still_ uses a limited availability
("keys to the kingdom") model; this works
for them because Linux, being only a kernel,
is orders of magnitude less code than any
BSD system. For all its faults, Linus is
sincerely wrong about his arguments against
source control -- though there are valid
ones, as the next point shows, Linus never,
to my knowledge, actually points to them,
since they are points against having a
"Linus" figure, as well.
o It required that patch conflict domains be well-known
on a per file basis, so that application of patches
could be serialized based on the topology of their
affect.
NB: CVS has this attribute; use of CVS, like use
of the patchkit, constrains BSD growth in
many ways, including acting as a brake on
not only the rate of growth, but the rate of
growth of the rate of growth. If I had been
aware of "mutual security" games at the time
I created it, I would have picked a different
approach, which did not have a centralized
control constraint as an emergent property,
and there would probably not be a "core team"
structure today, nor would having "commit
priviledges" be such a big deal (or stumbling
block, depending on your point of view).
o Developement under its auspices significantly
outstripped the ability of Bill Jolitz to keep up
with the work, as anything other than an editor,
a role for which he was unprepared.
NB: This is a good thing: all works should outlive
their authors utility; we have an Internet
today, despite the regrettable deaths of John
Postel and Richard Stevens, precisely because they
built things to last, instead of for their own
aggrandization. This process is called "monument
building", and engineers who do other than
likewise are diddling themselves.
o It had no sense of history: there was no modification
history, other than ordering, and there was no real
accreditation.
NB: This is good; it keeps away people who are in
it for ego. Unfortunately, it also attracts
those same people, since it provided a chokepoint
that was insufficiently decentralized to survive
an egomaniac. Like Linux, FreeBSD has been very
lucky, but not as lucky as it could have been,
lacking such a chokepoint in the incarnations of
its organization.
It's also bad, if you want to protect public
projects from intentional disruption, by Luddites
or power-seekers who look to wield power for its
own sake. Again, luck has played a role here,
since the organizational incarnations we've seen
haven't required perfect altruism in order to
continue to at least function.
o It was still to slow, for some people.
NB: Another bad point; I mostly blame this on the
implicit serialization of operations, which means
CVS has the same problem: FreeBSD, under CVS, has
occasionally been too slow for people, including
myself on several occasions.
Larry McVoy's source code control system, and
Perforce's system, both overcome this problem, by
offering the capability for multiple lines of
developement (often abbreviated on Larry's mailing
lists as "LODs"). Unfortunately, both systems
attach unacceptable economic restrictions on the
use of their software for overcoming this problem
in the BSDs, since they add cost to commercial use,
but don't incent the projects themselves, which
have already adjusted to the concept, and are
unlikely to change without incentive (no BSD
project, so far as I have been able to tell, seems
to see increased rate of growth or rate of increase
in rate of growth as incentive; mostly they view
both as a threat. Of course, that's what they've
been trained to do, by their initial choice of
tools).
So the patchkit was not a failure, it was an experiment, and it
had perhaps too much success, particularly as a braking system
where people wanted a "downhill racer", and a multiplier of
effort, where other people wanted an "atomic pile" instead of
a "nuke".
--
As an aside:
Actually, I view the whole thing as a useful applied sociology
experiment, from which much useful information derived. If I
had the academic credentials as a social scientist to be seriously
published in the field, or wasn't busy with other things to the
point of being unable to waste time acquiring them, I'd write
several papers on the topic. As it is, I've shared the models and
information with others (including the Sante Fe Institute and
the Foresight Institute) who had an interest and appeared to
understand them, so I'm not a single chokepoint myself.
I've since used the information gained to successfully identify
the minimum amount of effort required to trigger four still-viable
Open Source software projects, and some of the people whom I
shared information with have triggered no less than six others.
--
The commentary by "Jesse M" in response to these questions failed
to answer them...
> > Why did Jolitz pull support from 386BSD?
Bill Jolitz supported Lynne Jolitz after a Usenet tirade that,
while understandable, under the circumstances of the time, a
lot of people took personally. One of the things he did as
part of this support of his wife was to revoke the right of the
patchkit people to use the "386BSD" name for a "386BSD 0.5
interim release". Rather than waste the work, which was quite
substantial, they followed the NetBSD example and committed the
code to a CVS tree, and released FreeBSD instead.
In retrospect, my public advice to Bill Jolitz to trademark the
name "386BSD", which everyone assumed he had done, when he was
revoking rights to use the name, was probably a bad move.
> > And what was BSDi doing at the time?
BSDI was pursuing a commercial version of BSD, as well as
fending off a lawsuit brought on by their waking up the USL
lawyers with their "1-800-ITS-UNIX" phone number, which the
USL lawyers hated, since lawyers don't understand the concept
of namespaces (see ".COM" for other examples of the trademark
namespace being applied to orthogonal namespaces).
Once awake, the USL lawyers refused to go back to sleep,
even after the phone number "problem" was fixed, since the
law is used as a business weapon, as well as making trademarks
into a defensive obligation.
I can't speak to the agendas involved; I have no idea why UCB
did not rip USL a new one, since MIT offered to bankroll them,
including putting their patent portfolio behind the effort
(anyone with a patent portfolio large enough can find infringement
by anyone with a technology dependent business; patents were being
granted despite obviousness even back then, and the problem has
only gotten worse since then, with the patenting of algorithms as
if they were processes).
[ ... 386BSD ... ]
> With the 0.1 release, at least people could work and move
> forward, but many drivers and the VM had problems - hence
> the UPK.
Actually, the patchkit solved a _lot_ of problems people had
with the 386BSD 0.1 distribution. Only a few of these were
related to something other than the distribution structure,
which was the biggest problem most people had with the 0.1
release
Yeah, I know the VM system was a problem: I did the first
public patch for it with my first FAQ release (it was in the
top three original reasons for the FAQ). But most people had
a problem with having to wait for the promised "0.2 release",
which was supposed to incorporate most of the Usenet patches,
but which never materialized. The promises of "soon now" and
the faith in Bill frimly taking the steering wheel of the bus
(eventually) was the reason I labelled both the FAQ and the
patchkit as "unofficial": I did so with the full expectation
that "official" replacements would be forthcoming.
NB: Actually, I'm proud of doing that: if it had been
successful, we'd probably have an organization that
would be much more helpful to people with problems,
and much less likely to say things like "you want it
fixed, where's the code, you whiney moron?". I'm
not that happy with the clique-ish nature of the
community that's developed, where everyone thinks
it's the order of the universe that "newbies must
pay their dues". The BSD community has grown to
resemble a college fraternity, with its own set of
"hazing" rules, which, thankfully, Linux and other
Open Source software projects seem to have sucessfully
avoided.
[ ... ]
> As time went on, but well before 1.0, a Newsgroup formed
> and Chris Demetrious became the Moderator. The group
> was form as a support mechanisum(sp?) for Bill.
The newsgroup was a side-note, and brought on mostly by a
hypersensitive attitude of legal political correctness,
which haunts its name to this day. The comp.unix.bsd group
was perfectly adequate to the job, and was used for it for
quite a long time.
> However, individuals (no longer at BSDi) continously
> sent messages to cause insurrection and undermined
> trust in the community. Eventually, NetBSD was formed
> because of the reasons I stated earlier.
NetBSD people were merely less patient with Bill's 0.2
release promises, and went off on their own much earlier.
> FreeBSD form later but many of the original FreeBSD
> people were upset at the NetBSD people becuase
> they still wanted to support Bill.
FreeBSD and NetBSD pretty much evolved independently, at
nearby times.
FreeBSD people were not upset at the NetBSD people, per se;
they merely didn't "rally to the flag", once a new banner
was declared. Much of that had to do with the work being
carried out by a small group, in nominal seclusion, to get
out from under what appeared to be the yoke of promises
which would never materialize, in their opinions. In fact,
they turned out to be right, but through no fault of their
own. There was only a minimal amount of friction, mostly
caused by people who didn't understand the necessity of the
serialization of the production of patchkit patches; this
went away for everyone but a few "stamp collector"
personalities (people who hold unreasonably strong grudges
forever) when the FreeBSD/386BSD 0.5 split occurred, which
was very shortly after the original (0.8) NetBSD release.
> In a sense, alot of the bad blood is Bill's fault, but
> other people (including myself) must share the blame.
> I could have done more at the time to mend fences, but
> I knew that the community could not move forward
> without a commone enemy. The eventually found one.
> It was Bill and Lynn Jolitz, the original authors
> of 386BSD.
I disagree. The "bad blood", what there is of it, is the
result of a fringe of volatile personalities, which have
mostly been purged from the natural chokepoints of the
various BSD-derived projects.
I really don't buy the "common enemy" theory for most
events in the Open Source community; the only place that
really applies is in a project "split", and that generally
only happens as a result of very strong ideological reasons.
One of the reasons I constantly caution against splits,
even though I clash as much as anyone, on ideological
grounds, is that the new project will _inevitably_ attract
the most volatile elements to itself; that's what has
historically happened in _any_ social schism, throughout
human history. You don't have to run an experiment too
many times before you can predict the outcome which will
result from running it yet again.
Without strong ideological reasons, coupled with the power
in the system being embedded in a much smaller group of
people with a conflicting ideology, you'll always end up
with a rabble. This is why the U.S. electoral system, as
strange as it looks, has been so successful, and why the
relatively recent "core team" reforms in FreeBSD have had
the effect of making it even more unlikely that there will
be a true "FreeBSD schism" in the near future.
> So, today myself and other people you would not expect
> are trying to get the community back together.
> I can mention Rick Moen of the Cabal, and Ernest
> Prabhakar Appple's Open Source Project Manager.
> Together they and other people I should mention are
> working hard to get the groups back together, but
> as I've said, there is too much bad blood out there.
Again, I must disagree. The U.S., Australia, and the U.K.
are the best of friends these days, but they hardly want
to become a single country. You don't need to point at
putative "bad blood" to explain why there are three
seperate and distinct groups.
I think the reason the "openports" thing hasn't really
gotten anywhere yet in displacing the ports trees of the
various projects, is that there is not demonstrable benefit
for the majority of the people doing the actualy work: the
people with the power to "officially" adopt it in place of
the existing systems currently in use by the various
projects. Like source code control systems, the people
involved have been "trained", by weeding out all of the
people who clash with the system, through self selection.
Just providing a replacement system is not sufficient
incentive to cause a change-over: the existing system is
metastable, and won't "tunnel" of its own accord, even though
it would mean moving to a state of _net_ lower energy, since
they are measuring their energy only in their own realm, and
the net value in all BSD realms is irrelevent to them.
Continuing with this example (once all subelements have been
integrated, there's really no difference between the projects,
and one will "fade away", should we ever get to that point),
you would need either buy-in from the principals in each of
the groups, or you would need to provide an _additional and
compelling benefit_ to overcome social inertia. I think it's
that simple: no "bad blood" need apply, as an explanation.
NB: If you're interested, the status quo here is called
a "Richardson Non-Linear Mutual Security Game"; an
analytical mechanics buff would recognize it as a
"damped driven harmonic oscillator", where the damping
force exceeds the driving force. If you understood
that, then it's also probably obvious to you now how
you could preterb two of the systems sufficiently to
force your new paradigm to be adopted naturally; it's
also probable that you'll recognize it involves at
least some work on your part.
Terry Lambert
terry@lambert.org
---
Any opinions in this posting are my own and not those of my present
or previous employers.