Re: aarch64 pmap tweaks for review

[Andrew Doran <ad%netbsd.org@localhost>]

On Tue, Jun 02, 2020 at 11:15:17AM +0100, Nick Hudson wrote:
> On 01/06/2020 23:42, Andrew Doran wrote:
> > Hi,
> > 
> > I made some tweaks to the aarch64 pmap based on lessons learned in the x86
> > pmap recently.  They reduce memory consumption and speed up things like
> > fork/exec/exit/UBC a little:
> > 
> > 	http://www.netbsd.org/~ad/2020/aarch64.diff
> > 
> > Approximate times for kernel build on RK3399 with all 6 cores running at
> > 600MHz:
> > 
> > 	before	1354.07s real  6092.55s user  1591.35s system
> > 	after	1307.90s real  6026.60s user  1432.83s system
> 
> Good job.
> 
> From a first pass I'm curious why you don't use the low 12 bits of
> pv_va for flags and avoid the "interesting" mutex hack?

It seemed liked the lesser of two evils but on reflection putting that into
pv_va may be better just because it's clearer what's going on.  I'll make
the change and come back with another patch when I have time unless there
are objections.
 
> From memory the pp_flags in ryo's pmap can be removed fairly easily as
> well if required.

x86 needs them even after a page is unmapped, because UVM does:

	pmap_page_protect(pg, VM_PROT_NONE);
	modified = pmap_is_modified(pg);

I don't recall seeing a way in ryo's pmap but I was mainly looking at PV
list stuff + locking.  I was thinking for NetBSD 11 of trying out a change
to make everything that touches pg->flags use atomics, and if that doesn't
cause too many problems, then maybe a couple of flag bits could be made
available to MD code.

I was also thinking of trying to have UVM track which pmaps *might* have a
given set of pages mapped and then the PV entry stuff could go away.  For
example pmap_page_remove() could take (pmap, va, pg) arguments and mean "pg
might be mapped at va in this pmap, remove if it is".  Page loaning might
make that too difficult, I'm not sure yet.

Andrew