Re: kern/55667 (regression: XEN3_DOM0 fails to boot on)

[Frank Kardel <kardel%netbsd.org@localhost>]

In 9.99.55 ixg(4) works with ioapic (I am running that now) from 
/var/run/dmesg.boot:

ixg0 at pci16 dev 0 function 0: Intel(R) PRO/10GbE PCI-Express Network 
Driver, Version - 4.0.1-k
ixg0: clearing prefetchable bit
ixg0: device X550
ixg0: NVM Image Version 1.93, PHY FW Revision 2.0b ID 0xb, NVM Map 
version 1.93, OEM NVM Image version 0.04, ETrackID 80000aee
ixg0: interrupting at ioapic3 pin 31
ixg0: Ethernet address ac:1f:6b:f5:bc:cc
ixg0: PHY OUI 0x00aa00, model 0x0022, rev. 0
ixg0: PCI Express Bus: Speed 8.0GT/s Width x4
ixg0: feature cap 0x17a0<TEMP_SENSOR,LEGACY_TX,FDIR,MSI,MSIX,LEGACY_IRQ>
ixg0: feature ena 0x1020<TEMP_SENSOR,LEGACY_IRQ>
ixg1 at pci16 dev 0 function 1: Intel(R) PRO/10GbE PCI-Express Network 
Driver, Version - 4.0.1-k
ixg1: clearing prefetchable bit
ixg1: device X550
ixg1: NVM Image Version 1.93, PHY FW Revision 2.0b ID 0xb, NVM Map 
version 1.93, OEM NVM Image version 0.04, ETrackID 80000aee
ixg1: interrupting at ioapic3 pin 30
ixg1: Ethernet address ac:1f:6b:f5:bc:cd
ixg1: PHY OUI 0x00aa00, model 0x0022, rev. 0
ixg1: PCI Express Bus: Speed 8.0GT/s Width x4
ixg1: feature cap 0x17a0<TEMP_SENSOR,LEGACY_TX,FDIR,MSI,MSIX,LEGACY_IRQ>
ixg1: feature ena 0x1020<TEMP_SENSOR,LEGACY_IRQ>

Looks like something broke there. It is certainly not a hardware 
limitation but rather a
driver/interrut subsystem bug.

Frank


On 05/23/22 14:40, Manuel Bouyer wrote:
> The following reply was made to PR kern/55667; it has been noted by GNATS.
>
> From: Manuel Bouyer <bouyer%antioche.eu.org@localhost>
> To: gnats-bugs%netbsd.org@localhost
> Cc: jdolecek%netbsd.org@localhost, gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost,
>          kardel%netbsd.org@localhost
> Subject: Re: kern/55667 (regression: XEN3_DOM0 fails to boot on)
> Date: Mon, 23 May 2022 14:34:47 +0200
>
>   On Mon, May 23, 2022 at 09:00:02AM +0000, Frank Kardel wrote:
>   > The following reply was made to PR kern/55667; it has been noted by GNATS.
>   >
>   > From: Frank Kardel <kardel%netbsd.org@localhost>
>   > To: gnats-bugs%netbsd.org@localhost, jdolecek%netbsd.org@localhost, gnats-admin%netbsd.org@localhost,
>   >  netbsd-bugs%netbsd.org@localhost
>   > Cc:
>   > Subject: Re: kern/55667 (regression: XEN3_DOM0 fails to boot on)
>   > Date: Mon, 23 May 2022 10:58:11 +0200
>   >
>   >   From the partial dmesg output captures (I need to bump MSGSIZE) I see:
>   >
>   >  [     1.893878] ixg1 at pci16 dev 0 function 1: Intel(R) PRO/10GbE
>   >  PCI-Express Network Driver, Version - 4.0.1-k
>   >  [     1.894043] ixg1: clearing prefetchable bit
>   >  [     1.894382] ixg1: device X550
>   >  [     1.163206] ixg1: NVM Image Version 1.93, PHY FW Revision 2.0b ID
>   >  0xb, NVM Map version 1.93, OEM NVM Image version 0.04, ETrackID 80000aee
>   >  [     1.187495] ixg1: PBA number 020C08-0F8
>   >  [>>> 1.187536] ixg1: autoconfiguration error: failed to allocate MSI-X
>   >  interrupt
>   >  [     1.188286] ixg1: interrupting at ioapic3 pin 30
>   >  [     1.208223] ixg1: Ethernet address ac:1f:6b:f5:bc:cd
>   >  [     1.210360] ixg1: PHY: OUI 0x00aa00 model 0x0022, rev. 0
>   >  [     1.214695] ixg1: PCI Express Bus: Speed 8.0GT/s Width x4
>   >  [     1.216804] ixg1: feature cap
>   >  0x17a0<TEMP_SENSOR,LEGACY_TX,FDIR,MSI,MSIX,LEGACY_IRQ>
>   >  [     1.216806] ixg1: feature ena 0x1020<TEMP_SENSOR,LEGACY_IRQ>
>   
>   ixg(4) doens't work in ioapic mode in GENERIC either, it looks like a hardware
>   limitation (or maybe a driver bug). I tried to disable only MSI-X but this
>   cause a kernel fault with GENERIC, in the idle loop of a secondary CPU
>   (so no usefull informations).
>   
>   I have MSI working with Xen but it needs cleanup before commit.
>   MSI-X is still not working yet (and ixg is not working with MSI either - all
>   other devices do work).
>   
>   --
>   Manuel Bouyer <bouyer%antioche.eu.org@localhost>
>        NetBSD: 26 ans d'experience feront toujours la difference
>   --
>