Re: Some questions about x86/xen pmap

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, May 07, 2009 at 04:15:32PM +0200, Jean-Yves Migeon wrote:
> Manuel Bouyer wrote:
>> On Thu, May 07, 2009 at 01:03:19AM +0200, Jean-Yves Migeon wrote:
>>> Alright, then I am adding Manuel to the loop, as I would like to know 
>>>  why removing the APDP would be a problem comparing to temporarily   
>>> switching pmaps. Is it a performance or a design issue, 
>>> NetBSD-specific  or anything else?
>>
>> on xen/amd64 there is a separate pmap for kernel and user processes; the
>> kernel never runs in the context of a user process pmap (and user process
>> pmaps have no kernel mappings).
>
> So avoiding the use of APDP under Xen/amd64 would need some re-thinking  
> inside pmap (and more work to get it efficient).

Yes, a completely different way of accessing page table.
That could be the openbsd way, i.e a mapping of all the physical RAM in
the kernel VM space (there's enough VM space to do it)

>
>>> [...]
>>> The initial goal was to expand my locking to include protection for 
>>> the  pmap_map_ptes/pmap_unmap_ptes calls, so that APDP entries are 
>>> cleared by  pmap_unmap_ptes(). That way, I could avoid the pmap 
>>> iterations during  save, as I could guarantee that they all APDP 
>>> entries are left unmapped.
>>
>> note that mapping/unmapping APDP on each pmap_map_ptes/pmap_unmap_ptes call
>> is extremely expensive on amd64. We want to avoid it as much as possible
>> (i.e. change APDP only when another pmap needs to be mapped)
>
> Due to the MMU hypercall?

yes.

>
> If we ever support MP with Xen, modifying mappings found in APDP will  
> require a full TLB flush on all CPUs that could use these mappings,  
> which will also be very expensive.

This is a different issue. While porting NetBSD/xen to amd64 I found that
a single 10s or even 100s of distinct APDP operations, each requiring
a map/unmap of ADPD. batching APDP operations would fix most of the 
performances issues, but I think this needs changes in UVM.

-- 
Manuel Bouyer, LIP6, Universite Paris VI.           
Manuel.Bouyer%lip6.fr@localhost
     NetBSD: 26 ans d'experience feront toujours la difference
--