Subject: Re: Soft update code integrated into the main tree
To: Laine Stump <lainestump@rcn.com>
From: Miles Nordin <carton@Ivy.NET>
List: netbsd-advocacy
Date: 11/15/1999 18:02:53
I'm moving this to -advocacy, so that people trying to get work done won't
have to read another of my rants.

On Mon, 15 Nov 1999, Laine Stump wrote:

> Who else already has this? (ie, are we behind the curve, in front of the
> curve, or somewhere in the middle).
> 
> Little bits of info like this are useful in "OS Choice Wars", and I'm about
> to go into a fairly substantial battle ;-)

Solaris, IBM, SGI, HP, Novell, NetBSD and Digital have variants on the
motivation of soft updates--various log-structured or transaction-logged
filesystems that improve write performance--and they've had it for a long
time, with the possible exception of NetBSD lfs which has only been
working for, what, half a year or so?  Windows NT and Linux have nothing
but a bunch of unsupported vaporware add-on announcements.  FreeBSD has
apparently had something very similar to this for a while, but i'm
guessing probably not as long as the vendors.  i know that's all rather
hazy--so much for my knowledge of history.

I haven't gotten into any really challenging arguments of this sort
recently, so i'm not sure how well this suggestion will work, but
generally when people bring up features that NetBSD is missing or was slow
to add, i point out that NetBSD has fewer but rediculously competent
developers compared to most OS's (Linux, or NT, whatever), and then
attempt to refocus the discussion.

Rather than arguing about the past, you can try to establish evidence of
future potential.  It's a lot more difficult than dredging up facts from
the past, but it is _the_ important thing, especially if you're a
developer.  The industry abounds with code that has become poorly
architected and eventually collapsed under its own weight--Netscape,
PageMaker, any Microsoft product, Oracle perhaps?  Danny Thomas recently
posted a reference to BIND 9:

http://www.sendmail.net/?CssUID=&CssServer=&SessionName=&feed=interview000lisa01

in which Vixie seems to state that BIND was rewritten, from scratch,
_twice_, because the old code was too difficult to maintain.  The common
strategy of:

  o let the code grow to rediculous size by accepting contributions from 
    an army of inexperienced monkeys-with-keyboards
  o beat the code into submission with mass testing (or (Microsoft) not)

is a repeatedly-proven failure.  It is the motivation for OO (force
developers to write reusable code) and Java (the former, and also make
certain types of sloppiness irrelevant).  It is a BFD.  It is, arguably,
_the_ deal in the software industry today:  how can we make programs
bigger, or perhaps merely more complicated, without collapsing.

Linux is showing signs of strain already.  RedHat 6.0 was the most
bug-ridden RedHat release since Slackware and Kernel 0.99.14--it's by now
quite notorious, and i wouldn't reccomend it to anyone.

One quick example:  Linux was never able to get swapping over NFS to work
(believe me, i watched them try), and invented the ``Network Block
Device'' driver to evade the issue.  BSD takes tricky things like this for
granted.

Linux sound drivers were a godawful mess from the beginning--one of Cox's
first jobs at RedHat was to pound Slovansen's old code into some remote
form of submission.  NetBSD recently added AC97 support in record time,
thanks to highly reusable BSD sound drivers that could be imported, and
sane maintainable code.

On a similar note, I for one was astounded by how quickly Bill Sommerfeld
managed to provide the IETF attendees with a working Bay Networks 802.11
driver, and doubt that a driver could have been developed so quickly for
Linux (or, by a developer trained through working on Linux).

Linux and FreeBSD both claim SMP support, but they have collectively made
free UNIX notorious for substandard SMP because there is very little
parallelism in the kernel:  if you have a collection of processes that
spends 80% of it's time in ``system'' according to ps or top, Linux and
FreeBSD will extract little to no benefit from multiple CPU's, while
Solaris or AIX (or SunOS?) will do a lot better about splitting filesystem
and driver time among CPU's. You'll notice, perhaps even recall, Linux in
particular eating a lot of mud for this in the press.  Those magazines
that don't want to insult anyone and say, ``well, every OS has it's
strengths and weaknesses,'' usually incorporate mentioning substandard SMP
into their weak little uncritical articles, so I suspect we've all read a
lot about it.

What i'm trying to establish, is that while NetBSD doesn't claim SMP
support at all, NetBSD SMP has much greater potential to develop into
truly useful vendor-quality academically interesting SMP than the trivial
SMP in Linux and FreeBSD.  Restated, NetBSD, with no SMP support at all,
is likely to develop real SMP _before_ Linux and FreeBSD.  

Linux and FreeBSD's SMP pretty much topped off after adding a few
scheduler hacks to distribute that 20% of ``user'' CPU time.  but the SMP
they provide isn't actually _useful_ unless you're attacking some gigantic
mathematical problem that never calls the kernel, a situation in which you
can get better scalability by writing distributed code that runs on a lot
of loosely-connected single-CPU network nodes. they missed the whole
relevance of SMP out of a desire to heat up the second gigantic Pentium
``processor''-card on their brand new FuguTech motherboard. ``Look!  the
chip is hot.  It's done--we have SMP,'' they said, ``and now we can
finally compete with Windows NT in today's multi-CPU Wintel Server
Market.'' yee-haw.

Real SMP is a very big deal (outside the Wintel Server Market, at least),
with languages, API's, algorithms that presume mt, new CPU designs with
multiple cores on a single chip, and the recent relevance of real-time
scheduling and low-latency distributed computing.  The BeBox didn't have
two CPU's in it just because they wanted to make it ``faster.'' It has two
CPU's because Be developers were reading journals while Linux hackers were
reading white papers.  well, and staring at the empty space on their
motherboard--a motherboard designed by some desperate low-margin wintel
junk vendor that could care less about the future of the Industry.

NetBSD is the most meticulously-architected general-purpose operating
system in the world, and this gives it the greatest potential for future
development.  This is why groups routinely start dauntinigly complex
projects on NetBSD and expect to finish with something that works:  UVM,
real SMP, HFSC queueing--whatever.  Never mind a driver that surprisingly
manages to dodge the most recent bugs in the latest Adaptec
chipset--simply buy a different equally-cheap card that _works_!  These
little driverlets are irrelevant.  NetBSD is in the best position to
integrate truly interesting developments in the field of computer science.

It's not necessary to be in this position if you want to compete with
Microsoft--you can tolerate all the hard-and-fast limits Micosoft has
imposed upon themselves and their archaic architecture.  but, if you want
to compete with Sun, with video game consoles, or god forbid actually
invent something new rather than simply trying to catch up with others,
you ought to be using NetBSD.

k.  hope that helps with your impending battle.

-- 
Miles Nordin / v:1-888-857-2723 fax:+1 530 579-8680
555 Bryant Street PMB 182 / Palo Alto, CA 94301-1700 / US