Re: port-xen/58395: panic: HYPERVISOR_mmu_update failed, ret: -22

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Jul 04, 2024 at 06:10:16PM +0200, Manuel Bouyer wrote:
> On Thu, Jul 04, 2024 at 11:53:29AM +0000, Taylor R Campbell wrote:
> > Some more details:
> > 
> > 1. hypervisor0 at mainbus0: Xen version 4.14.0.88.g1d1d1f53
> > 
> > 2. At the time of xengnt_init, GNTTABOP_query_size returns:
> >    - nr_frames=32
> >    - max_nr_frames=64
> > 
> > 3. The pfns returned by GNTTABOP_setup_table look reasonable at first,
> >    e.g.:
> > 
> >    xengnt_more_entries: pages[0]@0xffffd380025b9c00=54d87
> >    xengnt_more_entries: pages[1]@0xffffd380025b9c08=54d86
> >    xengnt_more_entries: pages[2]@0xffffd380025b9c10=54d85
> >    xengnt_more_entries: pages[3]@0xffffd380025b9c18=54d84
> >    xengnt_more_entries: pages[4]@0xffffd380025b9c20=54df3
> >    ...
> >    xengnt_more_entries: pages[27]@0xffffd380025c04d8=54d94
> >    xengnt_more_entries: pages[28]@0xffffd380025c04e0=54d93
> >    xengnt_more_entries: pages[29]@0xffffd380025c04e8=54d92
> >    xengnt_more_entries: pages[30]@0xffffd380025c04f0=54d91
> >    xengnt_more_entries: pages[31]@0xffffd380025c04f8=54d90
> > 
> > 4. Both (2) and (3) remain true until we call GNTTABOP_setup_table
> >    with nr_frames=33, at which point:
> >    - GNTTABOP_setup_table returns a pfn of -1 (i.e., ffffffffffffffff,
> >      all bits set), but only for frame 32, and it still returns zero
> >      and sets op.status = GNTST_okay indicating success
> >    - GNTTABOP_query_size returns nr_frames=33 as expected
> > 
> >    xengnt_more_entries: GNTTABOP_query_size before: rc=0 nr_frames=32 max_nr_frames=64 status=0
> >    xengnt_more_entries: pages=0xffffd3800297c5c0 n=33
> >    xengnt_more_entries: pages[28]@0xffffd3800297c6a0=54d93
> >    xengnt_more_entries: pages[29]@0xffffd3800297c6a8=54d92
> >    xengnt_more_entries: pages[30]@0xffffd3800297c6b0=54d91
> >    xengnt_more_entries: pages[31]@0xffffd3800297c6b8=54d90
> >    xengnt_more_entries: pages[32]@0xffffd3800297c6c0=ffffffffffffffff
> >    xengnt_more_entries: GNTTABOP_query_size after: rc=0 nr_frames=33 max_nr_frames=64 status=0
> > 
> > xengnt_more_entries then passes pfn=-1 into pmap_kenter_ma which
> > passes it through to HYPERVISOR_mmu_update which fails with EINVAL
> > (22) presumably because pfn=-1 is invalid.
> > 
> > If I patch xengnt_init to do
> > 
> > -		gnt_max_grant_frames = query.max_nr_frames;
> > +		gnt_max_grant_frames = MIN(32, query.max_nr_frames);
> > 
> > then the kernel boots just fine in this environment.
> > 
> > I'm guessing that setting max_grant_frames=32 in the domU's xl.conf
> > would also work but I don't have control over that.
> 
> This looks like a bug in this hypervisor version. Looking at the linux code it
> seems to assume that if GNTTABOP_setup_table returned OK then all the entries
> are valid. But I don't think linux tries to allocate max_nr_frames
> at startup as we do (and I think this was changed between -9 and -10)
> so maybe it doens't run into this in common usage.
> 
> A workaround could be to fail the request when the last entry of pages[] is
> -1, but we'd still need to pmap_kenter_ma() the previous ones.

But it's not that simple, given how the code is actually structured.
We can do this on boot but not on resume.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--