kern/56669: crash at MegaRAID SAS 9341-8i

[6bone%6bone.informatik.uni-leipzig.de@localhost]

>Number:         56669
>Category:       kern
>Synopsis:       crash at MegaRAID SAS 9341-8i
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jan 25 15:15:00 +0000 2022
>Originator:     Uwe
>Release:        NetBSD 9.99.93
>Organization:
University of Leipzig
>Environment:
NetBSD ipv6worker.rz.uni-leipzig.de 9.99.93 NetBSD 9.99.93 (MYCONF10.gdb) #1: Tue Jan 25 08:52:14 CET 2022  root%ipv6worker.rz.uni-leipzig.de@localhost:/usr/obj/sys/arch/amd64/compile/MYCONF10.gdb amd64
>Description:
Kernel crash when using a MegaRAID SAS 9341-8i controller.

dmesg -M netbsd.0.core -N netbsd.0
....
[    31.621971] wsdisplay0: screen 1 added (80x25, vt100 emulation)
[    31.621971] wsdisplay0: screen 2 added (80x25, vt100 emulation)
[    31.621971] wsdisplay0: screen 3 added (80x25, vt100 emulation)
[    31.631973] wsdisplay0: screen 4 added (80x25, vt100 emulation)
[   346.108259] mfii0: cmd timeout ccb 0xffff8900fb45e2a0
[   346.108259] mfii0: cmd timeout ccb 0xffff8900fb461640
[   346.108259] mfii0: cmd timeout ccb 0xffff8900fb4620c0
[   346.108259] mfii0: cmd timeout ccb 0xffff8900fb464740
[   346.108259] mfii0: cmd timeout ccb 0xffff8900fb45a100
....
[   346.248263] mfii0: cmd timeout ccb 0xffff8900fb45f500
[   346.248263] mfii0: cmd timeout ccb 0xffff8900fb45c5c0
[   346.248263] uvm_fault(0xffffffff81973da0, 0x0, 2) -> e
[   346.248263] mfii0: cmd timeout ccb 0xffff8900fb45f180
[   346.248263] fatal page fault in supervisor mode
[   346.248263] trap type 6 code 0x2 rip 0xffffffff8029dc75 cs 0x8 rflags 0x10246 cr2 0xa8 ilevel 0 rsp 0xffff8909104d4f58
[   346.248263] curlwp 0xfffffde0880e7280 pid 0.487 lowest kstack 0xffff8909104d02c0
[   346.248263] mfii0: cmd timeout ccb 0xffff8900fb45efc0
[   346.248263] panic: trap
[   346.248263] cpu9: Begin traceback...
[   346.248263] mfii0: cmd timeout ccb 0xffff8900fb45d820
[   346.248263] vpanic() at netbsd:vpanic+0x156
[   346.248263] panic() at netbsd:panic+0x3c
[   346.248263] trap() at netbsd:trap+0xb25
[   346.248263] --- trap (number 6) ---
[   346.248263] mfii_scrub_ccb() at netbsd:mfii_scrub_ccb+0x3
[   346.258260] workqueue_worker() at netbsd:workqueue_worker+0xd7
[   346.258260] cpu9: End traceback...

[   346.258260] dumping to dev 168,0 (offset=232740548, size=33422197):
[   346.258260] dump mfii0: cmd timeout ccb 0xffff8900fb45aaa0
[   346.278260] mfii0: cmd timeout ccb 0xffff8900fb45ce80
[   346.318262] mfii0: cmd timeout ccb 0xffff8900fb45bfa0
[   346.318262] mfii0: cmd timeout ccb 0xffff8900fb459a00
...
[   346.448266] mfii0: cmd timeout ccb 0xffff8900fb461100
[   346.448266] mfii0: cmd timeout ccb 0xffff8900fb465a80
Kernel lock error: _kernel_lock,239: spinout

[   362.918583] lock address : 0xffffffff818a5080 type     :               spin
[   362.918583] initialized  : 0xffffffff80f8b0f0
[   362.918583] shared holds :                  0 exclusive:                  1
[   362.918583] shares wanted:                  0 exclusive:                  2
[   362.918583] relevant cpu :                  0 last held:                  0
[   362.918583] relevant lwp : 0xfffffdf0043e6080 last held: 0xfffffdf0043e6080
[   362.918583] last locked* : 0xffffffff80dcc631 unlocked : 0xffffffff80e6db36
[   362.918583] curcpu holds :                  0 wanted by: 0xfffffdf0043e6080


Reading symbols from /netbsd...
(gdb) target kvm netbsd.0.core
0xffffffff80226145 in cpu_reboot (howto=howto@entry=260,
    bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:720
720                     dumpsys();
(gdb) bt
#0  0xffffffff80226145 in cpu_reboot (howto=howto@entry=260,
    bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:720
#1  0xffffffff80dad9a7 in kern_reboot (howto=howto@entry=260,
    bootstr=bootstr@entry=0x0) at /usr/src/sys/kern/kern_reboot.c:73
#2  0xffffffff80df3172 in vpanic (fmt=fmt@entry=0xffffffff81391160 "trap",
    ap=ap@entry=0xffff8909104d4d18) at /usr/src/sys/kern/subr_prf.c:290
#3  0xffffffff80df3237 in panic (fmt=fmt@entry=0xffffffff81391160 "trap")
    at /usr/src/sys/kern/subr_prf.c:209
#4  0xffffffff80228f77 in trap (frame=0xffff8909104d4e60)
    at /usr/src/sys/arch/amd64/amd64/trap.c:326
#5  0xffffffff80221023 in alltraps ()
#6  0x0000000000000000 in ?? ()
(gdb)

(gdb) list *(0xffffffff80dcc631)
0xffffffff80dcc631 is in callout_softclock (/usr/src/sys/kern/kern_timeout.c:782).
777
778                     mutex_spin_exit(cc->cc_lock);
779                     KASSERT(func != NULL);
780                     if (__predict_false(!mpsafe)) {
781                             KERNEL_LOCK(1, NULL);
782                             (*func)(arg);
783                             KERNEL_UNLOCK_ONE(NULL);
784                     } else
785                             (*func)(arg);
786                     mutex_spin_enter(cc->cc_lock);

>How-To-Repeat:
Normal use of the raid device. Please let me know if you need more information.
>Fix: