Re: Changes to reduce pressure on uvm_pageqlock

[Andrew Doran <ad%netbsd.org@localhost>]

On Sun, Dec 08, 2019 at 07:00:43AM -0800, Jason Thorpe wrote:

> > On Dec 8, 2019, at 6:00 AM, Mindaugas Rasiukevicius <rmind%netbsd.org@localhost> wrote:
> > 
> > Just a small note: since vm_page::wire_count and vm_page::loan_count are
> > kind of mutually exclusive, I think suggestion by yamt@ (long time ago) to
> > merge them (positive value meaning wired count and negative -- loan) is a
> > useful one.  It would also free up those 16-bits, since Andrew seems to be
> > in need of some.

Thank you for the suggestion.  I have added stuff into vm_page which I don't
particularly like doing, so I'll look into the change you mentioned, maybe
not this week though and after I deliver this set of changes.

I do want to add more fields.  It turns out the first two fit into the
unused lower bits of pg->phys_addr without too much inconvenience for
anybody:

- freelist: 'cos uvm_page_lookup_freelist() turns up prominently in traces
- bucket: which cpu->ci_package_id I am from, for L2/L3 locality or NUMA

The third looks like it'll fit in the bottom bits of pg->offset but I need
to investigate what dificulties it might cause:

- waiter count for PG_BUSY, replacing PG_WANTED, to avoid thundering herd

> That seems like a good change to make, but omg please wrap the
> manipulation in macros / inlines because it seems like otherwise it could
> be quite error-prone.

Agreed.

Andrew