Re: aarch64 pmap tweaks for review

[Andrew Doran <ad%netbsd.org@localhost>]

On Wed, Jun 03, 2020 at 12:19:42AM +0000, Andrew Doran wrote:

> On Tue, Jun 02, 2020 at 11:15:17AM +0100, Nick Hudson wrote:
> > On 01/06/2020 23:42, Andrew Doran wrote:
> > > Hi,
> > > 
> > > I made some tweaks to the aarch64 pmap based on lessons learned in the x86
> > > pmap recently.  They reduce memory consumption and speed up things like
> > > fork/exec/exit/UBC a little:
> > > 
> > > 	http://www.netbsd.org/~ad/2020/aarch64.diff
> > > 
> > > Approximate times for kernel build on RK3399 with all 6 cores running at
> > > 600MHz:
> > > 
> > > 	before	1354.07s real  6092.55s user  1591.35s system
> > > 	after	1307.90s real  6026.60s user  1432.83s system
> > 
> > Good job.
> > 
> > From a first pass I'm curious why you don't use the low 12 bits of
> > pv_va for flags and avoid the "interesting" mutex hack?
> 
> It seemed liked the lesser of two evils but on reflection putting that into
> pv_va may be better just because it's clearer what's going on.  I'll make
> the change and come back with another patch when I have time unless there
> are objections.
>  
> > From memory the pp_flags in ryo's pmap can be removed fairly easily as
> > well if required.
> 
> x86 needs them even after a page is unmapped, because UVM does:
> 
> 	pmap_page_protect(pg, VM_PROT_NONE);
> 	modified = pmap_is_modified(pg);
> 
> I don't recall seeing a way in ryo's pmap but I was mainly looking at PV
> list stuff + locking.  I was thinking for NetBSD 11 of trying out a change
> to make everything that touches pg->flags use atomics, and if that doesn't
> cause too many problems, then maybe a couple of flag bits could be made
> available to MD code.
> 
> I was also thinking of trying to have UVM track which pmaps *might* have a
> given set of pages mapped and then the PV entry stuff could go away.  For
> example pmap_page_remove() could take (pmap, va, pg) arguments and mean "pg
> might be mapped at va in this pmap, remove if it is".  Page loaning might
> make that too difficult, I'm not sure yet.

After writing that I realised that kind of scheme won't work very well where
PV entries are needed to deal the peculiarities of virtual caches but I
think it's still worth trying for systems that don't need that.

Andrew