Re: port-xen/58395: panic: HYPERVISOR_mmu_update failed, ret: -22

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

The following reply was made to PR port-xen/58395; it has been noted by GNATS.

From: Manuel Bouyer <bouyer%antioche.eu.org@localhost>
To: Taylor R Campbell <riastradh%NetBSD.org@localhost>
Cc: gnats-bugs%NetBSD.org@localhost, netbsd-bugs%NetBSD.org@localhost
Subject: Re: port-xen/58395: panic: HYPERVISOR_mmu_update failed, ret: -22
Date: Thu, 4 Jul 2024 18:10:16 +0200

 On Thu, Jul 04, 2024 at 11:53:29AM +0000, Taylor R Campbell wrote:
 > Some more details:
 > 
 > 1. hypervisor0 at mainbus0: Xen version 4.14.0.88.g1d1d1f53
 > 
 > 2. At the time of xengnt_init, GNTTABOP_query_size returns:
 >    - nr_frames=32
 >    - max_nr_frames=64
 > 
 > 3. The pfns returned by GNTTABOP_setup_table look reasonable at first,
 >    e.g.:
 > 
 >    xengnt_more_entries: pages[0]@0xffffd380025b9c00=54d87
 >    xengnt_more_entries: pages[1]@0xffffd380025b9c08=54d86
 >    xengnt_more_entries: pages[2]@0xffffd380025b9c10=54d85
 >    xengnt_more_entries: pages[3]@0xffffd380025b9c18=54d84
 >    xengnt_more_entries: pages[4]@0xffffd380025b9c20=54df3
 >    ...
 >    xengnt_more_entries: pages[27]@0xffffd380025c04d8=54d94
 >    xengnt_more_entries: pages[28]@0xffffd380025c04e0=54d93
 >    xengnt_more_entries: pages[29]@0xffffd380025c04e8=54d92
 >    xengnt_more_entries: pages[30]@0xffffd380025c04f0=54d91
 >    xengnt_more_entries: pages[31]@0xffffd380025c04f8=54d90
 > 
 > 4. Both (2) and (3) remain true until we call GNTTABOP_setup_table
 >    with nr_frames=33, at which point:
 >    - GNTTABOP_setup_table returns a pfn of -1 (i.e., ffffffffffffffff,
 >      all bits set), but only for frame 32, and it still returns zero
 >      and sets op.status = GNTST_okay indicating success
 >    - GNTTABOP_query_size returns nr_frames=33 as expected
 > 
 >    xengnt_more_entries: GNTTABOP_query_size before: rc=0 nr_frames=32 max_nr_frames=64 status=0
 >    xengnt_more_entries: pages=0xffffd3800297c5c0 n=33
 >    xengnt_more_entries: pages[28]@0xffffd3800297c6a0=54d93
 >    xengnt_more_entries: pages[29]@0xffffd3800297c6a8=54d92
 >    xengnt_more_entries: pages[30]@0xffffd3800297c6b0=54d91
 >    xengnt_more_entries: pages[31]@0xffffd3800297c6b8=54d90
 >    xengnt_more_entries: pages[32]@0xffffd3800297c6c0=ffffffffffffffff
 >    xengnt_more_entries: GNTTABOP_query_size after: rc=0 nr_frames=33 max_nr_frames=64 status=0
 > 
 > xengnt_more_entries then passes pfn=-1 into pmap_kenter_ma which
 > passes it through to HYPERVISOR_mmu_update which fails with EINVAL
 > (22) presumably because pfn=-1 is invalid.
 > 
 > If I patch xengnt_init to do
 > 
 > -		gnt_max_grant_frames = query.max_nr_frames;
 > +		gnt_max_grant_frames = MIN(32, query.max_nr_frames);
 > 
 > then the kernel boots just fine in this environment.
 > 
 > I'm guessing that setting max_grant_frames=32 in the domU's xl.conf
 > would also work but I don't have control over that.
 
 This looks like a bug in this hypervisor version. Looking at the linux code it
 seems to assume that if GNTTABOP_setup_table returned OK then all the entries
 are valid. But I don't think linux tries to allocate max_nr_frames
 at startup as we do (and I think this was changed between -9 and -10)
 so maybe it doens't run into this in common usage.
 
 A workaround could be to fail the request when the last entry of pages[] is
 -1, but we'd still need to pmap_kenter_ma() the previous ones.
 
 -- 
 Manuel Bouyer <bouyer%antioche.eu.org@localhost>
      NetBSD: 26 ans d'experience feront toujours la difference
 --