Re: [suspend/resume] Initiating grant table

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Sat, Apr 26, 2008 at 06:27:38PM +0200, Jean-Yves Migeon wrote:
> Hi list,
> 
> Some questions regarding the manipulation and initiation of grant tables 
> (arch/xen/xen/xengnt.c), when called during hypervisor attach upon 
> autoconf part.
> 
> Most of it deals with the setup of the grant table and grant table 
> entries, but I would like to see if I am not mistaken.
> 
> My biggest problem (as of now :o ) being the way Xen and NetBSD both 
> cooperate (NetBSD being a PV guest, without using HVM instructions) for 
> memory management, especially for shared memory. I did not find any 
> "clear" definition on do's and don'ts for a suspend/resume. This may get 
> tricky for following cases:
> 
> - some parts of a domU kernel which needs, or uses, real physical 
> addresses, which will obviously change upon a resume (or migration).
> 
> For NetBSD, which parts of the kernel requires such real physical 
> addresses, in the situation of a PV guest (with the added level of 
> indirection by Xen, m2p and p2m)?

There should be very few of them in the NetBSD kernel, and all should
be in arch/xen. uvm istelf only knows about the pseudo-physical 
addresses.
You can look at parts using pmap_enter_ma(), pmap_extract_ma(),
xpmap_ptom() and the like.

> 
> - shared memory mappings, grant tables. If I understand correctly, upon 
> setting up the grant table, hypervisor maps the pages in caller's 
> address space, which is why NetBSD issues later a call to 
> pmap_kenter_ma(), in arch/xen/xen/xengnt.c:154. As the name suggests 
> (please correct if I am wrong), NetBSD maps the "grant_table" virtual 
> address to the machine one, obtained earlier with the setup hypercall.

Yes

> 
> However, the man page for pmap(9) does not contain documentation for 
> such a fonction. The closest one being pmap_kenter_pa(), for unmanaged 
> mapping between virtual and physical addresses. Should I consider the 
> pmap_kenter_ma() to work the same way as pmap_kenter_pa(), except that 
> it deals with "guest machine" addresses and not real physical ones?

That's the opposite. pmap_kenter_pa() deals with guest machine adresses
(called "pseudo-physical addresses" in xen world, and often refered
to as just "physical address"). This is part of the pmap interface to MI
systems, and MI systems only knows about pseudo-physical addresses.
pmap_kenter_ma() is the same as pmap_kenter_pa() but for real physical
(which are called "machine addresses" in Xen world).

> 
> Other than that, one little more question:
> - arch/xen/xen/xengnt.c:132, pages variable is malloc()ed to contain the 
> list of frame numbers for the grant table. But it is never free()d 
> later. Shouldn't be the variable free()d, as we are only intersted by 
> the values (frame_list) it points to?

You're right, it should be freed. That's not a big leak though, this
function being called only a few times in the life of a guest.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--