re: Ramblings about sparc64 pmap locking

[matthew green <mrg%eterna.com.au@localhost>]

   Concurrent access purely via activity on the V->P side is unlikely to
   happen due to the vm_map lock being held write locked during these types
   of operations (however I'm not 100% sure on that *).  Anyway all these 
   activities all deal with managed mappings so there are data structures to
   manage in the pmap, and interference between P->V and V->P activity so
   the pmap_lock is taken for most of the interfaces.  This stuff will only
   ever happen with process context (even kthread, proc0) but never from
   interrupt level or a softint.  So this is why pmap_lock is at IPL_NONE.
   
   * pmap_extract() is likely to be a renegade

can you expand on this point at all?  i'm trying to understand what
pmap_extract() does that needs a lock.  right now (except for DEBUG)
all it does is a single pseg_get(), and that should run without a lock
(and currently is locked via the pseg_lock i'm still not entirely sure
why helps avoiding "impossible" pseg_set() failures.)

my testing so far has shown that it doesn't need to take pmap_lock,
but your comments above give me pause.
   
   2) I'm told there were deadlocks in the sparc64 pmap due to pmap_lock being
   taking in pmap_activate/pmap_deactivate.
   
   The short answer is "don't do that".  From pmap_activate() one could end
   up recursing or introducing lock-order-reversal as this is called from
   mi_switch() which is a very common routine ("common to everything and
   everywhere" as it were).  For pmap_deactivate() it's more clear cut as
   that's called during LWP exit, at which point we can't context switch.

right.  this is why we were hanging on quad cpu boxes.
   
   3) Looking at sparc64 pmap_activate(), it seems the lock is needed for
   context management.  I'm told the management of said is per-CPU, so is
   there any reason we need a lock at all?  

it's not 100% per-cpu.  allocate is, but free's can come from any
where.  i've replaced these usages of pmap_lock with a new
pmap_ctx_lock at IPL_VM, and reverted pmap_lock back to IPL_NONE.

that seems to work just great.


thanks for your post, it cleared up some stuff for me!


.mrg.