Re: port-xen/58395: panic: HYPERVISOR_mmu_update failed, ret: -22

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

The following reply was made to PR port-xen/58395; it has been noted by GNATS.

From: Manuel Bouyer <bouyer%antioche.eu.org@localhost>
To: Taylor R Campbell <riastradh%NetBSD.org@localhost>
Cc: gnats-bugs%NetBSD.org@localhost, netbsd-bugs%NetBSD.org@localhost
Subject: Re: port-xen/58395: panic: HYPERVISOR_mmu_update failed, ret: -22
Date: Thu, 4 Jul 2024 18:16:51 +0200

 On Thu, Jul 04, 2024 at 06:10:16PM +0200, Manuel Bouyer wrote:
 > On Thu, Jul 04, 2024 at 11:53:29AM +0000, Taylor R Campbell wrote:
 > > Some more details:
 > > 
 > > 1. hypervisor0 at mainbus0: Xen version 4.14.0.88.g1d1d1f53
 > > 
 > > 2. At the time of xengnt_init, GNTTABOP_query_size returns:
 > >    - nr_frames=32
 > >    - max_nr_frames=64
 > > 
 > > 3. The pfns returned by GNTTABOP_setup_table look reasonable at first,
 > >    e.g.:
 > > 
 > >    xengnt_more_entries: pages[0]@0xffffd380025b9c00=54d87
 > >    xengnt_more_entries: pages[1]@0xffffd380025b9c08=54d86
 > >    xengnt_more_entries: pages[2]@0xffffd380025b9c10=54d85
 > >    xengnt_more_entries: pages[3]@0xffffd380025b9c18=54d84
 > >    xengnt_more_entries: pages[4]@0xffffd380025b9c20=54df3
 > >    ...
 > >    xengnt_more_entries: pages[27]@0xffffd380025c04d8=54d94
 > >    xengnt_more_entries: pages[28]@0xffffd380025c04e0=54d93
 > >    xengnt_more_entries: pages[29]@0xffffd380025c04e8=54d92
 > >    xengnt_more_entries: pages[30]@0xffffd380025c04f0=54d91
 > >    xengnt_more_entries: pages[31]@0xffffd380025c04f8=54d90
 > > 
 > > 4. Both (2) and (3) remain true until we call GNTTABOP_setup_table
 > >    with nr_frames=33, at which point:
 > >    - GNTTABOP_setup_table returns a pfn of -1 (i.e., ffffffffffffffff,
 > >      all bits set), but only for frame 32, and it still returns zero
 > >      and sets op.status = GNTST_okay indicating success
 > >    - GNTTABOP_query_size returns nr_frames=33 as expected
 > > 
 > >    xengnt_more_entries: GNTTABOP_query_size before: rc=0 nr_frames=32 max_nr_frames=64 status=0
 > >    xengnt_more_entries: pages=0xffffd3800297c5c0 n=33
 > >    xengnt_more_entries: pages[28]@0xffffd3800297c6a0=54d93
 > >    xengnt_more_entries: pages[29]@0xffffd3800297c6a8=54d92
 > >    xengnt_more_entries: pages[30]@0xffffd3800297c6b0=54d91
 > >    xengnt_more_entries: pages[31]@0xffffd3800297c6b8=54d90
 > >    xengnt_more_entries: pages[32]@0xffffd3800297c6c0=ffffffffffffffff
 > >    xengnt_more_entries: GNTTABOP_query_size after: rc=0 nr_frames=33 max_nr_frames=64 status=0
 > > 
 > > xengnt_more_entries then passes pfn=-1 into pmap_kenter_ma which
 > > passes it through to HYPERVISOR_mmu_update which fails with EINVAL
 > > (22) presumably because pfn=-1 is invalid.
 > > 
 > > If I patch xengnt_init to do
 > > 
 > > -		gnt_max_grant_frames = query.max_nr_frames;
 > > +		gnt_max_grant_frames = MIN(32, query.max_nr_frames);
 > > 
 > > then the kernel boots just fine in this environment.
 > > 
 > > I'm guessing that setting max_grant_frames=32 in the domU's xl.conf
 > > would also work but I don't have control over that.
 > 
 > This looks like a bug in this hypervisor version. Looking at the linux code it
 > seems to assume that if GNTTABOP_setup_table returned OK then all the entries
 > are valid. But I don't think linux tries to allocate max_nr_frames
 > at startup as we do (and I think this was changed between -9 and -10)
 > so maybe it doens't run into this in common usage.
 > 
 > A workaround could be to fail the request when the last entry of pages[] is
 > -1, but we'd still need to pmap_kenter_ma() the previous ones.
 
 But it's not that simple, given how the code is actually structured.
 We can do this on boot but not on resume.
 
 -- 
 Manuel Bouyer <bouyer%antioche.eu.org@localhost>
      NetBSD: 26 ans d'experience feront toujours la difference
 --