Re: kern/60029: panic: cpu0: softints stuck for 16 seconds

["Thomas Klausner via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR kern/60029; it has been noted by GNATS.

From: Thomas Klausner <wiz%netbsd.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: kern/60029: panic: cpu0: softints stuck for 16 seconds
Date: Sun, 22 Feb 2026 14:20:21 +0100

 And here's the bt/a output:
 
 crash> bt/a ffff8fb6ba506000
 trace: pid 0 lid 26 at 0xffffa72469fa40e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa72469fa40f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 1ece83ef168e83eb:
 crash> bt/a ffff8fb6ba506400
 trace: pid 0 lid 27 at 0xffffa72469faafa0
 uvm_aio_aiodone() at uvm_aio_aiodone+0xd4
 crash> bt/a ffff8fb6ba506800
 trace: pid 0 lid 28 at 0xffffa72495732bc0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 ffffa72495732db8:
 crash> bt/a ffff8fb6ba506c00
 trace: pid 0 lid 29 at 0xffffa724695da000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fc5faead1c0
 crash> bt/a ffff8fb6ba562800
 trace: pid 0 lid 97 at 0xffffa72469fdf0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa72469fdf0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 a95c7635a51c7631:
 crash> bt/a ffff8fb6ba562c00
 trace: pid 0 lid 98 at 0xffffa72469fe60e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa72469fe60f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 331934af3f5934ab:
 crash> bt/a ffff8fb6ba5cf000
 trace: pid 0 lid 99 at 0xffffa7248ed64bc0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 ffffa7248ed64db8:
 crash> bt/a ffff8fb6ba5cf400
 trace: pid 0 lid 100 at 0xffffa72469fc2000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba518640
 crash> bt/a ffff8fb6ba61c000
 trace: pid 0 lid 103 at 0xffffa7246a01a0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a01a0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 52437fb05e037fb4:
 crash> bt/a ffff8fb6ba61c400
 trace: pid 0 lid 104 at 0xffffa7246a0210e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0210f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 cb28bad7c768bad3:
 crash> bt/a ffff8fb6ba61c800
 trace: pid 0 lid 105 at 0xffffa7246a027da0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 crash> bt/a ffff8fb6ba61cc00
 trace: pid 0 lid 106 at 0xffffa72469ffd000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba518f40
 crash> bt/a ffff8fb6ba6d8800
 trace: pid 0 lid 109 at 0xffffa7246a0550e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0550f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 dcc1e310d085e314:
 crash> bt/a ffff8fb6ba6d8c00
 trace: pid 0 lid 110 at 0xffffa7246a05c0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a05c0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 78ad9ddf74ed9ddb:
 crash> bt/a ffff8fb6ba725000
 trace: pid 0 lid 111 at 0xffffa7246a062bd0
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 psref_release() at psref_release+0xbc
 ip_output() at ip_output+0xa40
 tcp_output() at tcp_output+0x1049
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at db8501c183c8440f
 crash: _kvm_kvatop(ffffffffb8cbbc17)
 crash: kvm_read(0xffffffffb8cbbc17, 8): invalid translation (invalid level 2 PDE)
 crash> bt/a ffff8fb6ba725400
 trace: pid 0 lid 112 at 0xffffa7246a06a0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a06a0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 ad7052a0a13056ec:
 crash> bt/a ffff8fb6ba7b4000
 trace: pid 0 lid 115 at 0xffffa7246a0920e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0920f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 a4c1696aa985796e:
 crash> bt/a ffff8fb6ba7b4400
 trace: pid 0 lid 116 at 0xffffa7246a0990e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0990f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 4e5ba5ac421ba1a8:
 crash> bt/a ffff8fb6ba7b4800
 trace: pid 0 lid 117 at 0xffffa7246a09fdb0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 crash> bt/a ffff8fb6ba7b4c00
 trace: pid 0 lid 118 at 0xffffa7246a0a70e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0a70f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 cb3f6901cf7f6901:
 crash> bt/a ffff8fb6ba7d0800
 trace: pid 0 lid 121 at 0xffffa7246a0cf0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0cf0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 36433bd33a033bd7:
 crash> bt/a ffff8fb6ba7d0c00
 trace: pid 0 lid 122 at 0xffffa7246a0d60e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a0d60f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 86ff48f48abf48f0:
 crash> bt/a ffff8fb6ba84e000
 trace: pid 0 lid 193 at 0xffffa7246a10c0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a10c0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 84de823040de827:
 crash> bt/a ffff8fb6ba84e400
 trace: pid 0 lid 194 at 0xffffa7246a1130e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1130f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 921f796b9e5f796f:
 crash> bt/a ffff8fb6ba84e800
 trace: pid 0 lid 195 at 0xffffa7246a119bd0
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 psref_release() at psref_release+0xbc
 ip_output() at ip_output+0xa40
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffffa7246a119fa0
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at db8501c183c8440f
 crash: _kvm_kvatop(ffffffffb8cbbc17)
 crash: kvm_read(0xffffffffb8cbbc17, 8): invalid translation (invalid level 2 PDE)
 crash> bt/a ffff8fb6ba84ec00
 trace: pid 0 lid 196 at 0xffffa7246a0ef000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba8b84c0
 crash> bt/a ffff8fb6ba8bf000
 trace: pid 0 lid 123 at 0xffffa7246a0dcfb0
 __kernel_end() at 0
 crash> bt/a ffff8fb6ba8bf400
 trace: pid 0 lid 124 at 0xffffa7246a0b2000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba742b80
 crash> bt/a ffff8fb6ba96a800
 trace: pid 0 lid 199 at 0xffffa7246a1490e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1490f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 1cadddbc10edd9b8:
 crash> bt/a ffff8fb6ba96ac00
 trace: pid 0 lid 200 at 0xffffa7246a1500e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1500f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 c2df9683ce9f9687:
 crash> bt/a ffff8fb6baa19000
 trace: pid 0 lid 201 at 0xffffa7246a156c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 arprequest() at arprequest+0x1e0
 arp_llinfo_output() at arp_llinfo_output+0x1bf
 nd_timer() at nd_timer+0x1f6
 callout_softclock() at callout_softclock+0x103
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1570f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 722e35e57e6e35e3:
 crash> bt/a ffff8fb6baa19400
 trace: pid 0 lid 202 at 0xffffa7246a12c000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba668880
 crash> bt/a ffff8fb6baa44800
 trace: pid 0 lid 211 at 0xffffa7246a1c30e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1c30f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 25ec7d6c29ac7d69:
 crash> bt/a ffff8fb6baa44c00
 trace: pid 0 lid 212 at 0xffffa7246a1ca0e0
 softint_dispatch() at softint_dispatch+0x492
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1ca0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 26bb16a62afb16a2:
 crash> bt/a ffff8fb6baa68000
 trace: pid 0 lid 205 at 0xffffa7246a1860e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a1860f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 753b4157797b4153:
 crash> bt/a ffff8fb6baa68400
 trace: pid 0 lid 206 at 0xffffa7246a18d0e0
 softint_dispatch() at softint_dispatch+0x492
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a18d0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 d665a48fda25a48b:
 crash> bt/a ffff8fb6baa68800
 trace: pid 0 lid 207 at 0xffffa7246a193c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 psref_release() at psref_release+0xbc
 ip_output() at ip_output+0xa40
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6c43f1d60
 crash> bt/a ffff8fb6baa68c00
 trace: pid 0 lid 208 at 0xffffa7246a169000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba9fe640
 crash> bt/a ffff8fb6bab33000
 trace: pid 0 lid 213 at 0xffffa72495594c10
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 __kernel_end() at 0
 uvm_fault_internal() at uvm_fault_internal+0xd5a
 trap() at trap+0x46b
 --- trap (number 6) ---
 ff3dda:
 crash> bt/a ffff8fb6bab33400
 trace: pid 0 lid 214 at 0xffffa7246a1a6000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6ba9fef40
 crash> bt/a ffff8fb6baba2000
 trace: pid 0 lid 217 at 0xffffa7246a2000e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2000f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 f0058a6dfc458a69:
 crash> bt/a ffff8fb6baba2400
 trace: pid 0 lid 218 at 0xffffa7246a2070e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2070f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 81ea3c468daa3c42:
 crash> bt/a ffff8fb6baba2800
 trace: pid 0 lid 219 at 0x181c
 lwp_locked() at lwp_locked+0x16
 crash: _kvm_kvatop(1824)
 crash: kvm_read(0x1824, 8): invalid translation (invalid level 4 PDE)
 crash> bt/a ffff8fb6baba2c00
 trace: pid 0 lid 220 at 0xffffa7246a1e3000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6baad0880
 crash> bt/a ffff8fb6babed000
 trace: pid 0 lid 161 at 0xffffa7246a24adb0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 crash> bt/a ffff8fb6babed400
 trace: pid 0 lid 162 at 0xffffa7246a220000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bac1f1c0
 crash> bt/a ffff8fb6bac3e800
 trace: pid 0 lid 95 at 0xffffa7246a23d0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a23d0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 25ba633929fa633d:
 crash> bt/a ffff8fb6bac3ec00
 trace: pid 0 lid 160 at 0xffffa7246a2440e0
 softint_dispatch() at softint_dispatch+0x492
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2440f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 bfff21a3b3bf21a7:
 crash> bt/a ffff8fb6bacdc000
 trace: pid 0 lid 165 at 0xffffa7246a27a0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a27a0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 d993f1abd5d3f1af:
 crash> bt/a ffff8fb6bacdc400
 trace: pid 0 lid 166 at 0xffffa7246a2810e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2810f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 4cf31e90c8f396d:
 crash> bt/a ffff8fb6bacdc800
 trace: pid 0 lid 167 at 0xffffa7246a287bd0
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 psref_release() at psref_release+0xbc
 _kernel_lock() at _kernel_lock+0x27
 crash> bt/a ffff8fb6bacdcc00
 trace: pid 0 lid 168 at 0xffffa7246a25d000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bac1fac0
 crash> bt/a ffff8fb6bad67000
 trace: pid 0 lid 173 at 0x181c
 lwp_locked() at lwp_locked+0x16
 crash: _kvm_kvatop(1824)
 crash: kvm_read(0x1824, 8): invalid translation (invalid level 4 PDE)
 crash> bt/a ffff8fb6bad67400
 trace: pid 0 lid 174 at 0xffffa7246a29a000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bad2a400
 crash> bt/a ffff8fb6badb8800
 trace: pid 0 lid 171 at 0xffffa7246a2b70e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2b70f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 3ec2db4b7282db4f:
 crash> bt/a ffff8fb6badb8c00
 trace: pid 0 lid 172 at 0xffffa7246a2be0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2be0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 bc8ba5cdb0cbe5c9:
 crash> bt/a ffff8fb6badf6000
 trace: pid 0 lid 177 at 0xffffa7246a2f40e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2f40f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 c91230e8c55230ec:
 crash> bt/a ffff8fb6badf6400
 trace: pid 0 lid 178 at 0xffffa7246a2fb0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a2fb0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 e844851de4048519:
 crash> bt/a ffff8fb6badf6800
 trace: pid 0 lid 179 at 0xffffa7246a3020e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a3020f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 99e6e37795a6e373:
 crash> bt/a ffff8fb6badf6c00
 trace: pid 0 lid 180 at 0xffffa7246a3090e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a3090f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 72782bb17e382bb5:
 crash> bt/a ffff8fb6baeb2800
 trace: pid 0 lid 183 at 0xffffa7246a3310e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a3310f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 1403286b1843286f:
 crash> bt/a ffff8fb6baeb2c00
 trace: pid 0 lid 184 at 0xffffa7246a314000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6baec8640
 crash> bt/a ffff8fb6baf4c800
 trace: pid 0 lid 391 at 0xffffa7246a3ab0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a3ab0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 5155d9fd5d15d9f9:
 crash> bt/a ffff8fb6baf4cc00
 trace: pid 0 lid 392 at 0xffffa7246a38e000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bafdb880
 crash> bt/a ffff8fb6baf82000
 trace: pid 0 lid 185 at 0xffffa7246a33edb0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 crash> bt/a ffff8fb6baf82400
 trace: pid 0 lid 186 at 0xffffa7246a314000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6baec8640
 crash> bt/a ffff8fb6baf90000
 trace: pid 0 lid 385 at 0xffffa7246a36e0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a36e0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 6d59a1826119a186:
 crash> bt/a ffff8fb6baf90400
 trace: pid 0 lid 386 at 0xffffa7246a351000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6baec8f40
 crash> bt/a ffff8fb6baf90800
 trace: pid 0 lid 387 at 0xffffa7246a37bd8c
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at 4280f86188
 crash> bt/a ffff8fb6baf90c00
 trace: pid 0 lid 388 at 0xffffa7246a3830e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a3830f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 1ecda230dacda27:
 crash> bt/a ffff8fb6bb0db000
 trace: pid 0 lid 393 at 0xffffa7246a3b8bd0
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 psref_release() at psref_release+0xbc
 _kernel_lock() at _kernel_lock+0x27
 crash> bt/a ffff8fb6bb0db400
 trace: pid 0 lid 394 at 0xffffa7246a38e000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bafdb880
 crash> bt/a ffff8fb6bb10a000
 trace: pid 0 lid 397 at 0xffffa7246a3e80e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a3e80f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 238f7bf92fc33bfd:
 crash> bt/a ffff8fb6bb10a400
 trace: pid 0 lid 398 at 0xffffa7246a3cb000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb0891c0
 crash> bt/a ffff8fb6bb10a800
 trace: pid 0 lid 399 at 0xffffa7246a3f5db0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 crash> bt/a ffff8fb6bb10ac00
 trace: pid 0 lid 400 at 0xffffa7246a3cb000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb0891c0
 crash> bt/a ffff8fb6bb1a6800
 trace: pid 0 lid 403 at 0xffffa7246a4250e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a4250f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 ea9c2d01e65c2d05:
 crash> bt/a ffff8fb6bb1a6c00
 trace: pid 0 lid 404 at 0xffffa7246a408000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb089ac0
 crash> bt/a ffff8fb6bb1d5000
 trace: pid 0 lid 405 at 0xffff8fb6bac8d008
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bac8d008
 Bad user frame pointer: 0xffff8fb6bac8d000
 crash> bt/a ffff8fb6bb1d5400
 trace: pid 0 lid 406 at 0xffffa7246a408000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb089ac0
 crash> bt/a ffff8fb6bb1e4000
 trace: pid 0 lid 409 at 0xffffa7246a445000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb1d6400
 crash> bt/a ffff8fb6bb1e4400
 trace: pid 0 lid 410 at 0xffffa7246a445000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb1d6400
 crash> bt/a ffff8fb6bb1e4800
 trace: pid 0 lid 411 at 0xffffa7246a46fc70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 arprequest() at arprequest+0x1e0
 arp_llinfo_output() at arp_llinfo_output+0x1bf
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6be2d3400
 crash> bt/a ffff8fb6bb1e4c00
 trace: pid 0 lid 412 at 0xffffa7246a445000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb1d6400
 crash> bt/a ffff8fb6bb280800
 trace: pid 0 lid 159 at 0xffffa7246a49f0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a49f0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 e78ad240038ad20:
 crash> bt/a ffff8fb6bb280c00
 trace: pid 0 lid 288 at 0xffffa7246a482000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb1d6d00
 crash> bt/a ffff8fb6bb390000
 trace: pid 0 lid 289 at 0xffffa7246a4acc70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 arprequest() at arprequest+0x1e0
 arp_llinfo_output() at arp_llinfo_output+0x1bf
 nd_timer() at nd_timer+0x1f6
 crash> bt/a ffff8fb6bb390400
 trace: pid 0 lid 290 at 0xffffa7246a482000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb1d6d00
 crash> bt/a ffff8fb6bb3de000
 trace: pid 0 lid 293 at 0xffffa7246a4dc0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a4dc0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 db915a2cd7d15a20:
 crash> bt/a ffff8fb6bb3de400
 trace: pid 0 lid 294 at 0xffffa7246a4bf000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb353640
 crash> bt/a ffff8fb6bb3de800
 trace: pid 0 lid 295 at 0xffffa7246a4e9db0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 crash> bt/a ffff8fb6bb3dec00
 trace: pid 0 lid 296 at 0xffffa7246a4bf000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb353640
 crash> bt/a ffff8fb6bb43a800
 trace: pid 0 lid 299 at 0xffffa7246a4fc000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb353f40
 crash> bt/a ffff8fb6bb43ac00
 trace: pid 0 lid 300 at 0xffffa7246a4fc000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb353f40
 crash> bt/a ffff8fb6bb449000
 trace: pid 0 lid 301 at 0xffffa7246a526c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 pmap_tlb_intr() at pmap_tlb_intr+0x129
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffffa72473ead000
 crash> bt/a ffff8fb6bb449400
 trace: pid 0 lid 302 at 0xffffa7246a4fc000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb353f40
 crash> bt/a ffff8fb6bb578000
 trace: pid 0 lid 305 at 0xffffa7246a5560e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a5560f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 47bafef449fafef0:
 crash> bt/a ffff8fb6bb578400
 trace: pid 0 lid 306 at 0xffffa7246a539000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb4a6880
 crash> bt/a ffff8fb6bb578800
 trace: pid 0 lid 307 at 0xffffa7246a563c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 arprequest() at arprequest+0x1e0
 arp_llinfo_output() at arp_llinfo_output+0x1bf
 crash> bt/a ffff8fb6bb578c00
 trace: pid 0 lid 308 at 0xffffa7246a539000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb4a6880
 crash> bt/a ffff8fb6bb58e800
 trace: pid 0 lid 323 at 0xffffa7246a5f0000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb680400
 crash> bt/a ffff8fb6bb58ec00
 trace: pid 0 lid 324 at 0xffffa7246a5f0000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb680400
 crash> bt/a ffff8fb6bb5b4800
 trace: pid 0 lid 311 at 0xffffa7246a576000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb5351c0
 crash> bt/a ffff8fb6bb5b4c00
 trace: pid 0 lid 312 at 0xffffa7246a576000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb5351c0
 crash> bt/a ffff8fb6bb5e3000
 trace: pid 0 lid 313 at 0xffffa7246a5a0c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 arprequest() at arprequest+0x1e0
 arp_llinfo_output() at arp_llinfo_output+0x1bf
 Bad frame pointer: 0xffffa72449a45000
 crash> bt/a ffff8fb6bb5e3400
 trace: pid 0 lid 314 at 0xffffa7246a5a80e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a5a80f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 691603e2655603e6:
 crash> bt/a ffff8fb6bb672000
 trace: pid 0 lid 445 at 0xffffa7246a5d00e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a5d00f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 2594318529d43181:
 crash> bt/a ffff8fb6bb672400
 trace: pid 0 lid 446 at 0xffffa7246a5b3000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb535ac0
 crash> bt/a ffff8fb6bb672800
 trace: pid 0 lid 191 at 0xffffa7246a5ddd8c
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at 4280f40a9c
 crash> bt/a ffff8fb6bb672c00
 trace: pid 0 lid 320 at 0xffffa7246a5b3000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb535ac0
 crash> bt/a ffff8fb6bb6cc000
 trace: pid 0 lid 329 at 0xffffa7246a62d000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb680d00
 crash> bt/a ffff8fb6bb6cc400
 trace: pid 0 lid 330 at 0xffffa7246a62d000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb680d00
 crash> bt/a ffff8fb6bb6cc800
 trace: pid 0 lid 331 at 0xffffa7246a657c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffffa7246c85b000
 crash> bt/a ffff8fb6bb6ccc00
 trace: pid 0 lid 332 at 0xffffa7246a65f0e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a65f0f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 e5dfe48de99fe488:
 crash> bt/a ffff8fb6bb79d000
 trace: pid 0 lid 325 at 0xffffa7246a61ac70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 arprequest() at arprequest+0x1e0
 arp_llinfo_output() at arp_llinfo_output+0x1bf
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffffa7247a123000
 crash> bt/a ffff8fb6bb79d400
 trace: pid 0 lid 326 at 0xffffa7246a5f0000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb680400
 crash> bt/a ffff8fb6bb888800
 trace: pid 0 lid 335 at 0xffffa7246a6870e0
 softint_dispatch() at softint_dispatch+0x1d2
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa7246a6870f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 42baf21b4efaf21f:
 crash> bt/a ffff8fb6bb888c00
 trace: pid 0 lid 336 at 0xffffa7246a66a000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb83e640
 crash> bt/a ffff8fb6bb897000
 trace: pid 0 lid 337 at 0xffffa7246a694c70
 bridge_output() at bridge_output+0x3ab
 ether_output() at ether_output+0x548
 x86_send_ipi() at x86_send_ipi+0x6f
 __kernel_end() at 0
 _kernel_unlock() at _kernel_unlock+0x81
 __kernel_end() at 0
 crash> bt/a ffff8fb6bb897400
 trace: pid 0 lid 338 at 0xffffa7246a66a000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb83e640
 crash> bt/a ffff8fb6bb966000
 trace: pid 0 lid 341 at 0xffffa7246a6a7000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb83ef40
 crash> bt/a ffff8fb6bb966400
 trace: pid 0 lid 342 at 0xffffa7246a6a7000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb83ef40
 crash> bt/a ffff8fb6bb966800
 trace: pid 0 lid 343 at 0xffffa7246a6d1db0
 rt_clonedmsg() at rt_clonedmsg+0x1cc
 pmap_tlb_intr() at pmap_tlb_intr+0x129
 crash> bt/a ffff8fb6bb966c00
 trace: pid 0 lid 344 at 0xffffa7246a6a7000
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fb6bb83ef40
 crash> bt/a ffff8fc60b222000
 trace: pid 0 lid 6 at 0xffffffff81a3ce40
 _KERNEL_OPT_PMS_DISABLE_POWERHOOK() at ffff8fc60f2c2ac0
 crash> bt/a ffff8fc60b25f400
 trace: pid 0 lid 3 at 0xffffa72469e83ed0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 Bad frame pointer: 0xffffa700b537bf80
 dkiodone+0xa8:
 crash> bt/a ffff8fc60b25f800
 trace: pid 0 lid 4 at 0xffffa72469e83fa0
 uvm_aio_aiodone() at uvm_aio_aiodone+0xbe
 dkiodone() at dkiodone+0xa8
 lddone() at lddone+0xf
 nvme_q_complete() at nvme_q_complete+0xf2
 softint_dispatch() at softint_dispatch+0x112
 DDB lost frame for Xsoftintr+0x4c, trying 0xffffa72469e840f0
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 d1eb424017eb420:
 crash> bt/a ffff8fc60b25fc00
 trace: pid 0 lid 5 at 0xffffa7247baabe80
 Xsoftintr() at Xsoftintr+0x4c
 --- interrupt ---
 ffffa700a3011980:
 
  Thomas