The HyperSPARC has a "feature" ... whenever the MMU detects an
error, it saves away the fault type.  If a second fault occurs
you get an "overwrite" fault, with the fault information being
"locked" to match the first fault.  (So far, this is more or less
just what any sun4m does.)  But --

There is an ASI for "look up the page table entry for a virtual
address", i.e., start the MMU translation, but stop as soon as the
translation finishes or fails, and return the PTE or 0 on failure.
On every sun4m, including the Hypersparc, this never causes a
fault trap.  On every sun4m *except* the Hypersparc, this also
never saves away a fault type.  On the Hypersparc, it does.

This means that every MMU probe leaves a bogus "fault address"
lying around, so that subsequent *real* faults fail to record the
correct fault address.  This was the biggest problem I had getting
BSD/OS running on Hypersparc sun4m boxes; it took me something like
two weeks of tracing just to figure out what was happening.

The fix is to be sure to clear out fault status after a failing
probe (read the SFSR; this is safe even on non-Hyper sun4m boxes).
This may (or may not :-) ) be what you need to get going.

I am still seeing occasional weird NMIs that (this is just a
slightly-cleaned-up version of Aaron's original code) shows up as:
"got NMI with sfsr=0x0, sfar=0x0, afsr=0x0, afar=0x119ff8. Retrying..."
(=> valid AFAR, as if for an I/O fault, but no AFSR).

Chris