RE: kern/57404: Can't see NVMe drives on ASUS Rampage VI mb in DIMM slot

[Namdak Tonpa <maxim%synrc.com@localhost>]

Sure, here is my dmesg: https://gist.github.com/5HT/d00e14b9fbf73f3fcb332d101a64feb9

 

From: matthew green
Sent: Sunday, May 14, 2023 9:55 AM
To: gnats-bugs%netbsd.org@localhost; maxim%synrc.com@localhost
Cc: kern-bug-people%netbsd.org@localhost; gnats-admin%netbsd.org@localhost; netbsd-bugs%netbsd.org@localhost
Subject: re: kern/57404: Can't see NVMe drives on ASUS Rampage VI mb in DIMM slot

 

> NetBSD localhost 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 2022  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64
> >Description:
> [     1.042790] nvme3 at pci15 dev 0 function 0: vendor 15b7 product 5011 (rev. 0x01)
> [     1.042790] nvme3: NVMe 1.4
> [     1.042790] nvme3: for admin queue interrupting at msix11 vec 0
> [     1.042790] nvme3: WDS100T1X0E-00AFY0, firmware 614600WD, serial 2136HR449906
> [     1.042790] nvme3: autoconfiguration error: unable to establish nvme3 ioq1 interrupt
> [     1.042790] nvme3: autoconfiguration error: unable to create io queue
>
> [     1.042790] nvme4 at pci17 dev 0 function 0: vendor 144d product a808 (rev. 0x00)
> [     1.042790] nvme4: NVMe 1.3
> [     1.042790] nvme4: for admin queue interrupting at msix11 vec 0
> [     1.042790] nvme4: Samsung SSD 970 EVO Plus 1TB, firmware 2B2QEXM7, serial S4EWNX0R946108Y
> [     1.042790] nvme4: autoconfiguration error: unable to establish nvme4 ioq1 interrupt
> [     1.042790] nvme4: autoconfiguration error: unable to create io queue
>
> [     1.042790] nvme5 at pci18 dev 0 function 0: vendor 144d product a808 (rev. 0x00)
> [     1.042790] nvme5: NVMe 1.3
> [     1.042790] nvme5: for admin queue interrupting at msix11 vec 0
> [     1.042790] nvme5: Samsung SSD 970 EVO Plus 1TB, firmware 2B2QEXM7, serial S4EWNX0R946133P
> [     1.042790] nvme5: autoconfiguration error: unable to establish nvme5 ioq1 interrupt
> [     1.042790] nvme5: autoconfiguration error: unable to create io queue
> >How-To-Repeat:
> You need the ASUS Rampage VI motherboard I can provide access to.

can you show the full dmesg?  or at least, the cpus, and all the
nvme lines?

there's a problem with many cpus and several nvme devices in netbsd-9
that is partly solved in netbsd-10, but i'm not sure that 6 devices
will work, nor that it's exactly the same problem, but it certainly
fails to attach all the per-cpu interrupts due to running out.  one
method to work around this would be to either on on "force_intx" or
turn off "mq" settings in the kernel (unfortunately, requires a
kernel build or early ddb to modify these variables):

sys/dev/pci/nvme_pci.c:67:int nvme_pci_force_intx = 0;
sys/dev/pci/nvme_pci.c:69:int nvme_pci_mq = 1;          /* INTx: ioq=1, MSI/MSI-X: ioq=ncpu */


.mrg.