re: kern/57404: Can't see NVMe drives on ASUS Rampage VI mb in DIMM slot

[matthew green <mrg%eterna.com.au@localhost>]

The following reply was made to PR kern/57404; it has been noted by GNATS.

From: matthew green <mrg%eterna.com.au@localhost>
To: gnats-bugs%netbsd.org@localhost, maxim%synrc.com@localhost
Cc: kern-bug-people%netbsd.org@localhost, gnats-admin%netbsd.org@localhost,
    netbsd-bugs%netbsd.org@localhost
Subject: re: kern/57404: Can't see NVMe drives on ASUS Rampage VI mb in DIMM slot
Date: Sun, 14 May 2023 16:55:27 +1000

 > NetBSD localhost 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 20=
 22  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd=
 64
 > >Description:
 > [     1.042790] nvme3 at pci15 dev 0 function 0: vendor 15b7 product 501=
 1 (rev. 0x01)
 > [     1.042790] nvme3: NVMe 1.4
 > [     1.042790] nvme3: for admin queue interrupting at msix11 vec 0
 > [     1.042790] nvme3: WDS100T1X0E-00AFY0, firmware 614600WD, serial 213=
 6HR449906
 > [     1.042790] nvme3: autoconfiguration error: unable to establish nvme=
 3 ioq1 interrupt
 > [     1.042790] nvme3: autoconfiguration error: unable to create io queu=
 e
 >
 > [     1.042790] nvme4 at pci17 dev 0 function 0: vendor 144d product a80=
 8 (rev. 0x00)
 > [     1.042790] nvme4: NVMe 1.3
 > [     1.042790] nvme4: for admin queue interrupting at msix11 vec 0
 > [     1.042790] nvme4: Samsung SSD 970 EVO Plus 1TB, firmware 2B2QEXM7, =
 serial S4EWNX0R946108Y
 > [     1.042790] nvme4: autoconfiguration error: unable to establish nvme=
 4 ioq1 interrupt
 > [     1.042790] nvme4: autoconfiguration error: unable to create io queu=
 e
 >
 > [     1.042790] nvme5 at pci18 dev 0 function 0: vendor 144d product a80=
 8 (rev. 0x00)
 > [     1.042790] nvme5: NVMe 1.3
 > [     1.042790] nvme5: for admin queue interrupting at msix11 vec 0
 > [     1.042790] nvme5: Samsung SSD 970 EVO Plus 1TB, firmware 2B2QEXM7, =
 serial S4EWNX0R946133P
 > [     1.042790] nvme5: autoconfiguration error: unable to establish nvme=
 5 ioq1 interrupt
 > [     1.042790] nvme5: autoconfiguration error: unable to create io queu=
 e
 > >How-To-Repeat:
 > You need the ASUS Rampage VI motherboard I can provide access to.
 
 can you show the full dmesg?  or at least, the cpus, and all the
 nvme lines?
 
 there's a problem with many cpus and several nvme devices in netbsd-9
 that is partly solved in netbsd-10, but i'm not sure that 6 devices
 will work, nor that it's exactly the same problem, but it certainly
 fails to attach all the per-cpu interrupts due to running out.  one
 method to work around this would be to either on on "force_intx" or
 turn off "mq" settings in the kernel (unfortunately, requires a
 kernel build or early ddb to modify these variables):
 
 sys/dev/pci/nvme_pci.c:67:int nvme_pci_force_intx =3D 0;
 sys/dev/pci/nvme_pci.c:69:int nvme_pci_mq =3D 1;          /* INTx: ioq=3D1=
 , MSI/MSI-X: ioq=3Dncpu */
 
 
 .mrg.