"Erik E. Fair" <fair@clock.org> writes:
> Is there some reason why "crashme" isn't a standard regression test for all
> NetBSD releases?

uh, if it's a time-limited run, and starts with a fixed seed, that
seems reasonable.  however, using a random seed kinda defeats the
whole point.

Really, NetBSD needs a Much Better regression test setup.  the
existing 'regress' tree doesn't scale particularly well, nor does it
produce output in a particularly meaningful/useful form.  It's also
annoyingly hard to create additional test cases.

I'm not keen on trying to make people throw additional effort into the
existing framework/tests when, really, they kinda bite.


In my opinion, the right 'first' place to improve 'quality' of NetBSD
is making sure that -current builds all the time, and making sure that
people who break it catch heat for it.  (I.e. change the age-old
belief that not compiling or working for weeks at a time is 'ok'.)  A
tool like tinderbox could be great for this, but you'd need to fix the
build system so that you can really build bits from scratch in one go
without privs (so that the builds could repeat infinitely without
bootstrapping issues).

After you get something like that going, you could use diskless
clients to do automated regression testing of the built bits.
(i.e. finish a build, then install the whole bloody thing in a
diskless client area, boot the client, and run regression tests.)



cgd
-- 
Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html
Disclaimer: Not speaking for NetBSD, just expressing my own opinion.