Subject: Re: NetBSD kernel "underbelly" testing
To: Matthew Orgass <>
From: Bill Studenmund <>
List: tech-kern
Date: 05/22/2004 17:22:19
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, May 21, 2004 at 06:36:36PM -0400, Matthew Orgass wrote:
> On 2004-05-21 wrote:
> > I've been giving thought to testing the kernel using a virtualisation
> > system like Xen.  If we presuppose that NetBSD, being so portable, has
> > a minimum amount of machine dependent code, we can get a good
> > approximation for all NetBSD ports by testing NetBSD/xen on a Xen
> > system.
>   That depends on what exactly is being tested and what all xen can do.
> The major factors that I can think of for MI code are 32/64-bit pointers,
> alignment, and endien.  Being able to cycle through all of those would be
> helpful (at least endien testing is likely to be impossible on xen).

I don't think that's what Ben meant. I don't think he's talking about=20
testing NetBSD across a spectrum of hardware/architectures, for which=20
ILP32 vs LP64, alignment, and endianness make sense. I think he's instead=
implying that since a lot of NetBSD is MI, that one simulator can cover a=
lot of bases. Ben, please correct me if I'm wrong!

I think cross-arch simulators would be cool, too, but that that's a=20
different proposal. I think there's a lot we can get just from this=20

> However, MD code is also quite important and can affect every aspect of
> the system in unexpected ways, so in general a single test platform is not
> sufficient.  Interrupt code is particularly sensitive to MD variations.
> > This would involve modifying Xen so that the behaviour of the
> > underlying virtual hardware could be manipulated by a scriptable
> > interface.
> >
> > Tests could be scripted such that a variety of hardware events could
> > be triggered and the kernel's response to these tested (eg. disk
> > errors, bus timeouts, etc).
>  As long as the test script side of this general enough to be attached to
> any emulator, I think this is a good idea.  However, I don't think this is
> the best way to improve reliability.  More non-virtualized test code could
> be added (I can list a bunch of stuff if you want).  Documented interfaces
> of sufficient detail and accuracy to verify the overall correctness of the
> kernel design and individually verify particular usage would have a much
> greater impact on reliability.  Even without doing this, there are plenty
> of known problems that could be fixed (observe the size of the PR
> database) and many more that could be discovered with a bit of code
> reading.  I also think it could be helpful to designate a core subset of
> best supported hardware and kernel systems for closer examination.
> Matthew Orgass

Content-Type: application/pgp-signature
Content-Disposition: inline

Version: GnuPG v1.2.3 (NetBSD)