Re: port-xen/58395: panic: HYPERVISOR_mmu_update failed, ret: -22

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Jul 04, 2024 at 11:53:29AM +0000, Taylor R Campbell wrote:
> Some more details:
> 
> 1. hypervisor0 at mainbus0: Xen version 4.14.0.88.g1d1d1f53
> 
> 2. At the time of xengnt_init, GNTTABOP_query_size returns:
>    - nr_frames=32
>    - max_nr_frames=64
> 
> 3. The pfns returned by GNTTABOP_setup_table look reasonable at first,
>    e.g.:
> 
>    xengnt_more_entries: pages[0]@0xffffd380025b9c00=54d87
>    xengnt_more_entries: pages[1]@0xffffd380025b9c08=54d86
>    xengnt_more_entries: pages[2]@0xffffd380025b9c10=54d85
>    xengnt_more_entries: pages[3]@0xffffd380025b9c18=54d84
>    xengnt_more_entries: pages[4]@0xffffd380025b9c20=54df3
>    ...
>    xengnt_more_entries: pages[27]@0xffffd380025c04d8=54d94
>    xengnt_more_entries: pages[28]@0xffffd380025c04e0=54d93
>    xengnt_more_entries: pages[29]@0xffffd380025c04e8=54d92
>    xengnt_more_entries: pages[30]@0xffffd380025c04f0=54d91
>    xengnt_more_entries: pages[31]@0xffffd380025c04f8=54d90
> 
> 4. Both (2) and (3) remain true until we call GNTTABOP_setup_table
>    with nr_frames=33, at which point:
>    - GNTTABOP_setup_table returns a pfn of -1 (i.e., ffffffffffffffff,
>      all bits set), but only for frame 32, and it still returns zero
>      and sets op.status = GNTST_okay indicating success
>    - GNTTABOP_query_size returns nr_frames=33 as expected
> 
>    xengnt_more_entries: GNTTABOP_query_size before: rc=0 nr_frames=32 max_nr_frames=64 status=0
>    xengnt_more_entries: pages=0xffffd3800297c5c0 n=33
>    xengnt_more_entries: pages[28]@0xffffd3800297c6a0=54d93
>    xengnt_more_entries: pages[29]@0xffffd3800297c6a8=54d92
>    xengnt_more_entries: pages[30]@0xffffd3800297c6b0=54d91
>    xengnt_more_entries: pages[31]@0xffffd3800297c6b8=54d90
>    xengnt_more_entries: pages[32]@0xffffd3800297c6c0=ffffffffffffffff
>    xengnt_more_entries: GNTTABOP_query_size after: rc=0 nr_frames=33 max_nr_frames=64 status=0
> 
> xengnt_more_entries then passes pfn=-1 into pmap_kenter_ma which
> passes it through to HYPERVISOR_mmu_update which fails with EINVAL
> (22) presumably because pfn=-1 is invalid.
> 
> If I patch xengnt_init to do
> 
> -		gnt_max_grant_frames = query.max_nr_frames;
> +		gnt_max_grant_frames = MIN(32, query.max_nr_frames);
> 
> then the kernel boots just fine in this environment.
> 
> I'm guessing that setting max_grant_frames=32 in the domU's xl.conf
> would also work but I don't have control over that.

This looks like a bug in this hypervisor version. Looking at the linux code it
seems to assume that if GNTTABOP_setup_table returned OK then all the entries
are valid. But I don't think linux tries to allocate max_nr_frames
at startup as we do (and I think this was changed between -9 and -10)
so maybe it doens't run into this in common usage.

A workaround could be to fail the request when the last entry of pages[] is
-1, but we'd still need to pmap_kenter_ma() the previous ones.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--