Are you running system with UBC?  I'm "really" newby in OS'es
but I've read sort of related things about solaris (I'm
talking here about bad performance under high IO load, not
stability).

Many observed bad perf. on solaris boxen under heavy IO.  The
problem has been diagnosed back to bad combination of UBC +
swapper algorithm.  The reason is that UBC was introduced back
in SunOS 4 release (or so ;)) and algorithm page-swapper used
was fairly natural straightforward two-handed clock algorithm
(one hand went ahead and reset "accessed" bit, second hand
checked the bit and swapped/freed "unaccessed" pages).

Swapper kicked in when free core dropped beyond certain point
(lotsfree) and paid no attention whose pages it frees: fs
cache's or processes's.

HW those days just wasn't capable of enormous IO speeds
available today.  Therefore filesystem could use core pages
for cacheing stuff and not cause serious contention.  Scales
fs-cache vs. processes were sort of balanced.

Todays systems OTOH are easily producing say 30MB/s.  Given
that "hands" are separated by say 4 seconds (actually, it
depends, at Sun they control swapper's speeds VERY nicely but
that's beyond the point and would take too much time to type
in :)) to few hours (! depending on the system, load,
configuration) and thus, have at the very minimum 30MB/s *
4-6secs = 120-180MB "very-recently used pages".  What happens
than?  Swapper steels memory from processeses and even some
kernel subsystems, which grinds the whole box to the halt.
Interestingly enough, they reported that "it almost didn't
matter how much RAM one had: it only defined number of
seconds/minutes it took to stall the system".

How they fixed it?  They caught it at the very end of the
release cycle for Solaris 7 (that's why it wasn't default
paging algo till Solaris 8) and introduced "priority_paging".
The heart of new page sweeping algorithm said that UBC is not
so "U" after all.  They introduced new param (cachefree,
typically = lotsfree * 2) and made swapper to recognize pages
that where used for fs-cache.  New algorithm kicked in when
free core dropped below cachefree and freed only fs-cache
occupied pages.  Such slight page non-uniformity proved to be
a very good compromise.

That's in a nutshell their story.  NetBSD might have
absolutely different design/implementation of these things, or
might even not have them at all, but it just too many people
repeat the same damn thing: under have IO system grinds and
performs worse than it did years before.  It sounded a bit
alike to the story above, and I thought I'll toss it around.

I heard NetBSD implemented UBC on it's own but have no idea
how.  If someone knowledgable could point the rest of us to
the right pages to read I'll appreciate it very much.  Thanks,

-Evgeni