Re: [PATCH v2] port/xen: map memory directly in privcmd PRIVCMD_MMAPBATCH

[Roger Pau Monne <roger.pau%citrix.com@localhost>]

Cherry G.Mathew wrote:
> > > > > > "Roger" == Roger Pau Monne<roger.pau%citrix.com@localhost>  writes:
>
>      Roger>  Cherry G.Mathew wrote:
>      >>>>>>>  "Roger" == Roger Pau Monne<roger.pau%citrix.com@localhost>  
> writes:
>      >>  I'm still missing something here, but I think something is not
>      >>  right here. I *think* qemu or NetBSD is mixing up the guest's
>      >>  gmfns (pfns) and dom0 mfns here.
>      >>
>      >>  - qemu queries the hypervisor for the guest pfn (aka gmfn) via:
>      >>  [1] xc_get_hvm_param(xen_xc, xen_domid,
>      >>  HVM_PARAM_IOREQ_PFN,&ioreq_pfn);
>
>      Roger>  With this call we get a set of gfns of the guest.
>
> what are "gfns" ? Are they mfns (ie; the fns go into the
> page tables) or gmfns (what the guest *thinks* goes into page tables) ?

They are gmfns.

>      >>  - It then maps it into the dom0 qemu pmap via: [2]
>      state->  shared_page = xc_map_foreign_range(xen_xc, xen_domid,
>      >>  XC_PAGE_SIZE, PROT_READ|PROT_WRITE, ioreq_pfn);
>
>      Roger>  Then we map those gfn pages to the virtual address space of
>      Roger>  Qemu.
>
>
>
>      >>  Note that NetBSD is assuming that ioreq_pfn is a dom0 mfn at this
>      >>  point!
>
>      Roger>  NetBSD only has it's own p2m table. This is just mapping a
>      Roger>  region of the guest memory to the Qemu address space, but all
>      Roger>  the mappings are done by the hypervisor, since the mfns of
>      Roger>  the guest are not part of the mfns of the NetBSD kernel
>      Roger>  (dom0).
>
> That's irrelevant. The fact that you can map mfns into the VA of the
> qemu process is what matters.

You are not mapping mfns, Qemu only knows the gmfns.

>      >>  At this point, as if by magic, the dom0 mfns point to the correct
>      >>  hvm ioreq_pfns. I'm guessing this is because the dom0 p2m table
>      >>  is identity mapped ? hmmmmmmmmmmmmmmmm....... not good!
>
>      Roger>  No, this is outside of the NetBSD p2m table. We don't have
>      Roger>  the guest p2m table, we only have a common memory region
>      Roger>  mapped to both domains (guest and dom0), but this underlying
>      Roger>  mfns are not part of the dom0 mfns, they are handled by the
>      Roger>  hypervisor.
>
> That's irrelevant. My fear here is that dom0 may be mapping in the wrong
> mfns into the qemu VA.
>
>
>      >>  - next qemu updates the *guest* p2m by calling: [3] rc =
>      >>  xc_domain_add_to_physmap(xen_xc, xen_domid, XENMAPSPACE_gmfn,
>      >>  idx, gpfn);
>      >>
>      >>  I'm not sure how this mapping is working at all. I think NetBSD's
>      >>  privcmd.c needs to be updating its own p2m during the foreign
>      >>  domain page mappings by considering them to be gmfns (ie; guest
>      >>  pfns).
>
>      Roger>  I don't think so, the NetBSD p2m table is related to the
>      Roger>  NetBSD kernel only (dom0), it doesn't contain entries from
>      Roger>  other domains, and this mappings are not using dom0 memory.
>
> My main problem here can be summarised into two questions:
>
> i) What exactly is qemu mapping in via NetBSDs privcmd.c ? (machine
> frames or pseudo physical addresses ?)

pseudo physical addresses (gmfns), belonging to the guest, which are 
translated to mfns by the Xen hypervisor.

> ii) If machine frames, why does the guest P->M mapping change affect
> qemu's mapping at all ?

This can affect Qemu if the mapping is not done straight away, since the 
"P" (gmfn) address that we store in privcmd_object is no longer pointing 
to the same M (mfn) address that it was when we did the call to 
xc_map_foreign*, because the call to xc_domain_add_to_physmap changed 
that P->M mapping before we could map the mfn (M) to Qemu address space.

So if we try to perform a pmap_enter_ma on the pagefault handler after 
the call to xc_domain_add_to_physmap with the stored gmfns, we get an 
error from the Hypervisor.

> As I said there's something missing here, either in my understanding, or
> what's going on in code.