On 2004-05-21 bje+dated+1085566922.83af9c@air.net.au wrote:

> I've been giving thought to testing the kernel using a virtualisation
> system like Xen.  If we presuppose that NetBSD, being so portable, has
> a minimum amount of machine dependent code, we can get a good
> approximation for all NetBSD ports by testing NetBSD/xen on a Xen
> system.

  That depends on what exactly is being tested and what all xen can do.
The major factors that I can think of for MI code are 32/64-bit pointers,
alignment, and endien.  Being able to cycle through all of those would be
helpful (at least endien testing is likely to be impossible on xen).
However, MD code is also quite important and can affect every aspect of
the system in unexpected ways, so in general a single test platform is not
sufficient.  Interrupt code is particularly sensitive to MD variations.

> This would involve modifying Xen so that the behaviour of the
> underlying virtual hardware could be manipulated by a scriptable
> interface.
>
> Tests could be scripted such that a variety of hardware events could
> be triggered and the kernel's response to these tested (eg. disk
> errors, bus timeouts, etc).

 As long as the test script side of this general enough to be attached to
any emulator, I think this is a good idea.  However, I don't think this is
the best way to improve reliability.  More non-virtualized test code could
be added (I can list a bunch of stuff if you want).  Documented interfaces
of sufficient detail and accuracy to verify the overall correctness of the
kernel design and individually verify particular usage would have a much
greater impact on reliability.  Even without doing this, there are plenty
of known problems that could be fixed (observe the size of the PR
database) and many more that could be discovered with a bit of code
reading.  I also think it could be helpful to designate a core subset of
best supported hardware and kernel systems for closer examination.

Matthew Orgass
darkstar@city-net.com