Re: Merge of rmind-uvmplock branch

[Andrew Doran <ad%netbsd.org@localhost>]

Mindaugas, thank you very much for your hard work!!

On Tue, Jun 07, 2011 at 03:16:06AM +0000, YAMAMOTO Takashi wrote:

> the idea is to protect pv chains with object lock, right?

That was the initial driver for me anyway.

Now that this work is in, there are some changes that can be built upon it
to realise the full value -- and hopefully make exit() etc. very cheap.
Looking at this from an x86 perspective but some of the ideas will apply
to other ports:

- My initial idea was to kill the global PV hash and associated locks.  To
  replace this we would embed a list head in each uvm_map_entry (or wherever
  else a mapping is managed).  This would be supplied by the caller on each
  relevant pmap call - pmap_enter() and so on.  PV entries would be added to
  and removed from this structure by the pmap module.  An initial
  implementation could get away with a dumb linked list I think.

- So then PV entries tracked with the mapping instead of globally. Once
  pmap_remove_all() has been called pmap_remove() could switch to a
  "shortcut" mode and become very quick.  From memory I believe all it would
  need to do is tear down the software PV entries, and not touch any page or
  pmap state.  Tearing down pmap state would be deferred to pmap_destroy(). 
  At that point we can then clear out the pmap's uvm_objects and free all
  pmap pages in bulk.  This would avoid potentially thousands of expensive
  scans and atomic updates to the hardware paging structures, which account
  for the bulk of expense during exit() etc.  If the system is short on
  memory we might want a mechanism to switch CPUs away from this pmap if
  they are hanging onto it as pmap_kernel - i.e. preventing pmap_destroy()
  from being called.

- In the x86 pmap, we have a number retryable sequences when we operate in
  the P->V direction.  pmap_page_remove() for instance.  I think these can
  now be simplified as there are fewer concurrency issues, no need for
  retryable sequences.  Yamamoto-san can you confirm?