Re: port-xen/55285: XEN3_DOM0 panics with GSI fail

[Jaromír Doleček <jaromir.dolecek%gmail.com@localhost>]

The following reply was made to PR port-xen/55285; it has been noted by GNATS.

From: =?UTF-8?B?SmFyb23DrXIgRG9sZcSNZWs=?= <jaromir.dolecek%gmail.com@localhost>
To: "gnats-bugs%NetBSD.org@localhost" <gnats-bugs%netbsd.org@localhost>
Cc: Frank Kardel <kardel%kardel.name@localhost>
Subject: Re: port-xen/55285: XEN3_DOM0 panics with GSI fail
Date: Wed, 15 Jul 2020 13:09:25 +0200

 Good, can you send me output from cpuctl identify so that I can form
 up a proper conditional?
 
 Can you also send me the whole dmesg including the Xen message? I want
 to check which particular device virtualization options are enabled.
 
 The sluggishness might be due to the interrupts being delivered to a
 different processor than cpu0 and only being picked up by timer
 interrupt. Can you try if forcing dom0 to use only one cpu changes the
 sluggishness?
 
 Jaromir
 
 Le mer. 15 juil. 2020 =C3=A0 12:10, Frank Kardel <kardel%netbsd.org@localhost> a =C3=
 =A9crit :
 >
 > The following reply was made to PR port-xen/55285; it has been noted by G=
 NATS.
 >
 > From: Frank Kardel <kardel%netbsd.org@localhost>
 > To: =3D?UTF-8?B?SmFyb23DrXIgRG9sZcSNZWs=3D?=3D <jaromir.dolecek%gmail.com@localhost=
 >,
 >  "gnats-bugs%NetBSD.org@localhost" <gnats-bugs%netbsd.org@localhost>
 > Cc:
 > Subject: Re: port-xen/55285: XEN3_DOM0 panics with GSI fail
 > Date: Wed, 15 Jul 2020 12:08:25 +0200
 >
 >  Good catch - enabling VT made it boot with xen, but the system isvery
 >  slow/sluggish.
 >
 >  Reading via SATA from SSD is around 180MB/sec, reading from NVMe is
 >  below 2 MB/sec.
 >
 >  Using the NO_PCI_MSI_MSIX kernel reads alse 180MB/sec from SATA and 556
 >  MB/sec from NVMe
 >
 >  So, while running a non NO_PCI_MSI_MSIX DOM0 on Xen is possible it is
 >  barely usable.
 >
 >  Frank
 >
 >  On 07/14/20 18:26, Jarom=C3=ADr Dole=C4=8Dek wrote:
 >  > Frank,
 >  >
 >  > I see this in the xen boot:
 >  >
 >  > (XEN) CPU0: VMX disabled by BIOS.
 >  > (XEN) VMX: failed to initialise.
 >  >
 >  > Can you please check your BIOS and see if you can enable anything
 >  > related to VT-x or VT-d or virtualization in general, and see if it
 >  > makes any difference?
 >  >
 >  > Jaromir
 >  >
 >  > Le lun. 13 juil. 2020 =C3=A0 09:51, Frank Kardel <kardel%netbsd.org@localhost> a=
  =C3=A9crit :
 >  >> Yes it still panics with PCPU_IDT patch installed and NO_PCI_MSI_MSIX
 >  >> commented out regardless of PCPU_IDT option being present or not.
 >  >>
 >  >> NO_PCI_MSI_MSIX makes it boot successfully.
 >  >>
 >  >> Frank
 >  >>
 >  >> [   1.0000030] vendor 8086 product a1ed (undefined, subclass 0x00,
 >  >> revision 0x09) at pci0 dev 17 function 1 not configured
 >  >> [   1.0000030] ahcisata0 at pci0 dev 17 function 5: vendor 8086 produ=
 ct
 >  >> a1d2 (rev. 0x09)
 >  >> [   1.0000030] ahcisata0: AHCI revision 1.31, 2 ports, 32 slots, CAP
 >  >> 0xeb34ff41<EMS,PSC,SSC,PMD,SAM,ISS=3D0x3=3DGen3,SCLO,SAL,SSS,SSNTF,SN=
 CQ,S64A>
 >  >> [   1.0000030] panic: physdev_op(PHYSDEVOP_map_pirq) MSI fail - 5:
 >  >> vendor 8086 product a1d2 (rev. 0x09)
 >  >> [   1.0000030] ahcisata0: AHCI revision 1.31, 2 ports, 32 slots, CAP
 >  >> 0xeb34ff41<EMS,PSC,SSC,PMD,SAM,ISS=3D0x3=3DGen3,SCLO,SAL,SSS,SSNTF,SN=
 CQ,S64A>
 >  >> [   1.0000030] panic: physdev_op(PHYSDEVOP_map_pirq) MSI fail -19
 >  >> [   1.0000030] cpu0: Begin traceback...
 >  >> [   1.0000030] vpanic() at netbsd:vpanic+0x146
 >  >> [   1.0000030] snprintf() at netbsd:snprintf
 >  >> [   1.0000030] xen_pic_to_gsi() at netbsd:xen_pic_to_gsi+0x32c
 >  >> [   1.0000030] intr_establish_xname() at netbsd:intr_establish_xname+=
 0x8e
 >  >> [   1.0000030] x86_pci_msi_establish() at netbsd:x86_pci_msi_establis=
 h+0x7c
 >  >> [   1.0000030] ahci_pci_intr_establish() at
 >  >> netbsd:ahci_pci_intr_establish+0x1ac
 >  >> [   1.0000030] ahci_attach() at netbsd:ahci_attach+0x2fc
 >  >> [   1.0000030] ahci_pci_attach() at netbsd:ahci_pci_attach+0x21a
 >  >> [   1.0000030] config_attach_loc() at netbsd:config_attach_loc+0x17a
 >  >> [   1.0000030] pci_probe_device() at netbsd:pci_probe_device+0x571
 >  >> [   1.0000030] pci_enumerate_bus() at netbsd:pci_enumerate_bus+0x1b7
 >  >> [   1.0000030] pcirescan() at netbsd:pcirescan+0x4e
 >  >> [   1.0000030] pciattach() at netbsd:pciattach+0x186
 >  >> [   1.0000030] config_attach_loc() at netbsd:config_attach_loc+0x17a
 >  >> [   1.0000030] mp_pci_scan() at netbsd:mp_pci_scan+0xa4
 >  >> [   1.0000030] hypervisor_attach() at netbsd:hypervisor_attach+0x3df
 >  >> [   1.0000030] config_attach_loc() at netbsd:config_attach_loc+0x17a
 >  >> [   1.0000030] xen_mainbus_attach() at netbsd:xen_mainbus_attach+0x51
 >  >> [   1.0000030] mainbupci_scan+0xa4
 >  >> [   1.0000030] hypervisor_attach() at netbsd:hypervisor_attach+0x3df
 >  >> [   1.0000030] config_attach_loc() at netbsd:config_attach_loc+0x17a
 >  >> [   1.0000030] xen_mainbus_attach() at netbsd:xen_mainbus_attach+0x51
 >  >> [   1.0000030] mainbus_attach() at netbsd:mainbus_attach+0x49
 >  >> [   1.0000030] config_attach_loc() at netbsd:config_attach_loc+0x17a
 >  >> [   1.0000030] cpu_configure() at netbsd:cpu_configure+0x25
 >  >> [   1.0000030] main() at netbsd:main+0x2ec
 >  >> [   1.0000030] cpu0: End traces_attach() at netbsd:mainbus_attach+0x4=
 9
 >  >> [   1.0000030] config_attach_loc() at netbsd:config_attach_loc+0x17a
 >  >> [   1.0000030] cpu_configure() at netbsd:cpu_configure+0x25
 >  >> [   1.0000030] main() at netbsd:main+0x2ec
 >  >> [   1.0000030] cpu0: End traceback...
 >  >> [   1.0000030] fatal breakpoint trap in supervisor mode
 >  >> [   1.0000030] trap type 1 code 0 rip 0xffffffff8023e93d cs 0xe030
 >  >> rflags 0x202 cr2 0 ilevel 0x8 rsp 0xffffffff819933f0
 >  >> [   1.0000030] curlwp 0xffffffff80e6c700 pid 0.0 lowest kstacback...
 >  >>
 >  >>
 >  >> On 07/11/20 17:16, Jarom=C3=ADr Dole=C4=8Dek wrote:
 >  >>> Hi,
 >  >>> can you confirm whether the Xen Dom0 panic still happens when using
 >  >>> the IDT patch? I don't think it would affect it, but it's worth
 >  >>> checking.
 >  >>>
 >  >>> You'd need to compile a XEN3_DOM0 kernel with the option
 >  >>> NO_PCI_MSI_MSIX commented out.
 >  >>>
 >  >>> Jaromir
 >