Re: XenServer and viridian

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Mar 27, 2025 at 12:46:34PM +0000, Stephen Borrill wrote:
> On Thu, 27 Mar 2025, Stephen Borrill wrote:
> > On Thu, 27 Mar 2025, Manuel Bouyer wrote:
> > > On Thu, Mar 27, 2025 at 08:53:01AM +0000, Stephen Borrill wrote:
> > > > On Wed, 26 Mar 2025, Manuel Bouyer wrote:
> > > > > On Wed, Mar 26, 2025 at 05:24:39PM +0000, Stephen Borrill wrote:
> > > > > > On XenServer/XCP-ng VMs can have 'Viridian
> > > > > > enlightenments' enabled. This is
> > > > > > used by Windows VMs to get some features that would otherwise require
> > > > > > Hyper-V, for instance, VM generation counter as used by
> > > > > > Active Directory to
> > > > > > spot it if is has been rolled back.
> > > > > > 
> > > > > > These shouldn't affect any non-Windows VMs, but NetBSD
> > > > > > mistakenly thinks it
> > > > > > is running on Hyper-V rather than Xen, so the PVH code is not enabled.
> > > > > > 
> > > > > > I tried the following naive patch:
> > > > > > 
> > > > > > --- sys/arch/x86/x86/identcpu.c 2 Feb 2025 14:51:59
> > > > > > -0000 1.123.4.2
> > > > > > +++ sys/arch/x86/x86/identcpu.c 26 Mar 2025 17:20:44 -0000
> > > > > > @@ -1132,8 +1132,22 @@
> > > > > >                                 vm_guest = VM_GUEST_VMWARE;
> > > > > >                         else if (memcmp(hv_vendor,
> > > > > > "Microsoft Hv", 12) == 0) {
> > > > > >                                 vm_guest = VM_GUEST_HV;
> > > > > > +                               p = pmf_get_platform("system-vendor");
> > > > > > +                               if (p != NULL) {
> > > > > > +                                       if (strncmp(p, "Xen", 3) == 0) {
> > > > > > +                                               /*
> > > > > > +                                                *
> > > > > > XenServer VMs may have
> > > > > > +                                                *
> > > > > > Viridian enlightenments
> > > > > > +                                                *
> > > > > > enabled which may make
> > > > > > +                                                *
> > > > > > Hyper-V be falsely matched
> > > > > > +                                                */
> > > > > > +                                               vm_guest
> > > > > > = VM_GUEST_XENHVM;
> > > > > > +                                       }
> > > > > > +                               }
> > > > > >  #if NHYPERV > 0
> > > > > > -                               hyperv_early_init();
> > > > > > +                               if (vm_guest == VM_GUEST_HV) {
> > > > > > +                                       hyperv_early_init();
> > > > > > +                               }
> > > > > >  #endif
> > > > > >                         } else if (memcmp(hv_vendor,
> > > > > > "KVMKVMKVM\0\0\0", 12) == 0)
> > > > > >                                 vm_guest = VM_GUEST_KVM;
> > > > > > 
> > > > > > 
> > > > > > It correctly "Identified Guest XEN in HVM mode" instead of Hyper-C, but
> > > > > > immediately panicked in xen_hvm_init().
> > > > > > 
> > > > > > Is there a less naive patch I could try? :-)
> > > > > 
> > > > > Can you give more details about the panic ?
> > > > 
> > > > [   1.0000000] NetBSD 10.1_STABLE (GENERIC) #6: Wed Mar 26
> > > > 17:13:37 GMT 2025
> > > > [   1.0000000]
> > > > root@builder10:/usr/work/obj/10/amd64/sys/arch/amd64/compile/GENERIC
> > > > [   1.0000000] total memory = 1019 MB
> > > > [   1.0000000] avail memory = 957 MB
> > > > [   1.0000040] efi: systbl at pa 3f5ee018
> > > > [   1.0000040] mainbus0 (root)
> > > > [   1.0000040] Identified Guest XEN in HVM mode.
> > > > [   1.0000040] fatal protection fault in supervisor mode
> > > > [   1.0000040] trap type 4 code 0 rip 0xffffffff802499cd cs 0x8
> > > > rflags 0x246 cr2 0 ilevel 0x8 rsp 0xffffffff81d40cf0
> > > > [   1.0000040] curlwp 0xffffffff8188b000 pid 0.0 lowest kstack
> > > > 0xffffffff81d3b2c0
> > > > kernel: protection fault trap, code=0
> > > > Stopped in pid 0.0 (system) at  netbsd:xen_hvm_init+0xad:       wrmsr
> > > > xen_hvm_init() at netbsd:xen_hvm_init+0xad
> > > 
> > > So my guess is that it's failing in xen_init_hypercall_page(),
> > > when doing:
> > >        wrmsr(descs[1], (uintptr_t)&hypercall_page - KERNBASE);
> > > 
> > > this means that when booting in this mode, the Xen interfaces are not
> > > available to the guest.
> > > 
> > > When booting in this mode without the hack, do you see some unattached Xen
> > > PCI devices in the dmesg ?
> > 
> > Yes:
> > # dmesg | grep -i xen
> > ACPI: RSDP 0x000000003F5FA014 000024 (v02 Xen   )
> > ACPI: XSDT 0x000000003F5F90E8 00004C (v01 Xen    HVM      00000000
> > 01000013)
> > ACPI: FACP 0x000000003F5F8000 0000F4 (v04 Xen    HVM      00000000 HVML
> > 00000000)
> > ACPI: DSDT 0x000000003E50C000 0092A3 (v02 Xen    HVM      00000000 INTL
> > 20160527)
> > ACPI: APIC 0x000000003F5F7000 000260 (v02 Xen    HVM      00000000 HVML
> > 00000000)
> > ACPI: HPET 0x000000003F5F6000 000038 (v01 Xen    HVM      00000000 HVML
> > 00000000)
> > ACPI: WAET 0x000000003F5F5000 000028 (v01 Xen    HVM      00000000 HVML
> > 00000000)
> > acpi0: X/RSDT: OemId <   Xen,     HVM,00000000>, AslId <    ,01000013>
> > XenSource, Inc. Xen Platform Device (undefined, subclass 0x80, revision
> > 0x01) at pci0 dev 3 function 0 not configured
> 
> I also tried FreeBSD 14.2 with exactly the same config and they get it
> right.

OK, so in this case the interface to use is proably though the
"Xen Platform Device" PCI device, as the native Xen interface is not available.

More code needed ...

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--