Ramblings about sparc64 pmap locking

[Andrew Doran <ad%NetBSD.org@localhost>]

1) Someone asked why pmap_lock is IPL_NONE on sparc64.

Generally the only activity in the pmap that occurs at interrupt level is
pmap_kenter_pa and pmap_kremove.  These deal wih unmanaged mappings so
there are typically no PV lists or anything like that to maintain.
Additionally these are always kernel mappings, and the kernel is always
self-consistent.  So it can be guaranteed that nothing will concurrently
modify pages in the same VA range.  There can be activity around the
same PA range though, either managed or unmanaged mappings (how this
is dealt with is of course up to the pmap).

Pretty much everything else can involve concurrent access to overlapping
sets of pages, and through the P->V interfaces (pmap_page_*) you can get
concurrent access around a VA range while you are working on the
same range via the V->P interfaces (pmap_enter etc).

Concurrent access purely via activity on the V->P side is unlikely to
happen due to the vm_map lock being held write locked during these types
of operations (however I'm not 100% sure on that *).  Anyway all these 
activities all deal with managed mappings so there are data structures to
manage in the pmap, and interference between P->V and V->P activity so
the pmap_lock is taken for most of the interfaces.  This stuff will only
ever happen with process context (even kthread, proc0) but never from
interrupt level or a softint.  So this is why pmap_lock is at IPL_NONE.

* pmap_extract() is likely to be a renegade

2) I'm told there were deadlocks in the sparc64 pmap due to pmap_lock being
taking in pmap_activate/pmap_deactivate.

The short answer is "don't do that".  From pmap_activate() one could end
up recursing or introducing lock-order-reversal as this is called from
mi_switch() which is a very common routine ("common to everything and
everywhere" as it were).  For pmap_deactivate() it's more clear cut as
that's called during LWP exit, at which point we can't context switch.

3) Looking at sparc64 pmap_activate(), it seems the lock is needed for
context management.  I'm told the management of said is per-CPU, so is
there any reason we need a lock at all?