Subject: Re: NetBSD kernel "underbelly" testing
To: Matthew Orgass <firstname.lastname@example.org>
From: Bill Studenmund <email@example.com>
Date: 05/22/2004 17:22:19
Content-Type: text/plain; charset=us-ascii
On Fri, May 21, 2004 at 06:36:36PM -0400, Matthew Orgass wrote:
> On 2004-05-21 firstname.lastname@example.org wrote:
> > I've been giving thought to testing the kernel using a virtualisation
> > system like Xen. If we presuppose that NetBSD, being so portable, has
> > a minimum amount of machine dependent code, we can get a good
> > approximation for all NetBSD ports by testing NetBSD/xen on a Xen
> > system.
> That depends on what exactly is being tested and what all xen can do.
> The major factors that I can think of for MI code are 32/64-bit pointers,
> alignment, and endien. Being able to cycle through all of those would be
> helpful (at least endien testing is likely to be impossible on xen).
I don't think that's what Ben meant. I don't think he's talking about=20
testing NetBSD across a spectrum of hardware/architectures, for which=20
ILP32 vs LP64, alignment, and endianness make sense. I think he's instead=
implying that since a lot of NetBSD is MI, that one simulator can cover a=
lot of bases. Ben, please correct me if I'm wrong!
I think cross-arch simulators would be cool, too, but that that's a=20
different proposal. I think there's a lot we can get just from this=20
> However, MD code is also quite important and can affect every aspect of
> the system in unexpected ways, so in general a single test platform is not
> sufficient. Interrupt code is particularly sensitive to MD variations.
> > This would involve modifying Xen so that the behaviour of the
> > underlying virtual hardware could be manipulated by a scriptable
> > interface.
> > Tests could be scripted such that a variety of hardware events could
> > be triggered and the kernel's response to these tested (eg. disk
> > errors, bus timeouts, etc).
> As long as the test script side of this general enough to be attached to
> any emulator, I think this is a good idea. However, I don't think this is
> the best way to improve reliability. More non-virtualized test code could
> be added (I can list a bunch of stuff if you want). Documented interfaces
> of sufficient detail and accuracy to verify the overall correctness of the
> kernel design and individually verify particular usage would have a much
> greater impact on reliability. Even without doing this, there are plenty
> of known problems that could be fixed (observe the size of the PR
> database) and many more that could be discovered with a bit of code
> reading. I also think it could be helpful to designate a core subset of
> best supported hardware and kernel systems for closer examination.
> Matthew Orgass
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (NetBSD)
-----END PGP SIGNATURE-----