Re: Help diagnosing panic: pr_phinpage_check

[joelp%sdf.org@localhost]

> Every few days I'm getting a kernel panic on a RockPro64, 4GB; drive info:
> ld0 at sdmmc0: 64gb eMMC as /
> ld4 at nvme0 nsid 1: 500 gb NVMe M.2 2280
>
> Kernel: GENERIC64 9.3 #0
>
> It seemed to start happening after I increased disk i/o on the M.2 storage
> on ld4. Can you help me diagnose?
>
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5765256] panic:
> pr_phinpage_check: [bufpl] item 0xffff00007f719d80 not part of pool
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5765256] cpu3: Begin traceback...
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5765256] trace fp
> ffffffc062457820
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5865282] fp ffffffc062457840
> vpanic() at ffffffc00042b4d0 netbsd:vpanic+0x198
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5865282] fp ffffffc0624578a0
> panic() at ffffffc00042b5a0 netbsd:panic+0x48
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5865282] fp ffffffc062457930
> pool_cache_put_paddr() at ffffffc000429410
> netbsd:pool_cache_put_paddr+0x128
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5865282] fp ffffffc062457960
> buf_trim() at ffffffc00046498c netbsd:buf_trim+0x5c
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5965294] fp ffffffc062457990
> allocbuf() at ffffffc000464e0c netbsd:allocbuf+0x1fc
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5965294] fp ffffffc062457a00
> getblk() at ffffffc00046536c netbsd:getblk+0x164
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.5965294] fp ffffffc062457a50
> ffs_getblk() at ffffffc0003a6fd4 netbsd:ffs_getblk+0x2c
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6065305] fp ffffffc062457a80
> ffs_balloc() at ffffffc0003a05b4 netbsd:ffs_balloc+0x11e4
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6065305] fp ffffffc062457bf0
> ufs_gop_alloc() at ffffffc0003b3bc4 netbsd:ufs_gop_alloc+0xb4
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6065305] fp ffffffc062457c60
> ffs_write() at ffffffc0003ab214 netbsd:ffs_write+0x374
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6065305] fp ffffffc062457d00
> VOP_WRITE() at ffffffc000481268 netbsd:VOP_WRITE+0x40
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6165320] fp ffffffc062457d50
> vn_write() at ffffffc00047b3d0 netbsd:vn_write+0x110
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6165320] fp ffffffc062457da0
> dofilewrite() at ffffffc0004350f0 netbsd:dofilewrite+0x78
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6165320] fp ffffffc062457e20
> syscall() at ffffffc000074a34 netbsd:syscall+0x19c
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6265329] tf ffffffc062457ed0
> el0_trap() at ffffffc000073148 netbsd:el0_trap
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6265329] ---- trapframe
> 0xffffffc062457ed0 (304 bytes) ----
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6265329]     pc=0000f9d1374685a8,
>   spsr=0000000040000000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6265329]    esr=0000000056000004,
>    far=ffffffc061253000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6265329]     x0=0000000000000016,
>     x1=0000f9d12f9de3e0
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6265329]     x2=0000000000010000,
>     x3=0000000000000000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]     x4=0000f9d137c95d68,
>     x5=0000000000000000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]     x6=0000ffffffa905d8,
>     x7=0000000000000000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]     x8=0000f9d1373305c8,
>     x9=0000f9d1373305b0
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x10=0000000000000001,
>    x11=0000000000010000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x12=0000f9d12f9de3e0,
>    x13=0000000000000000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x14=00000000ffffffff,
>    x15=0000f9d137c9d1d5
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x16=0000f9d1378b3908,
>    x17=0000f9d1374685a4
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x18=000000000000001a,
>    x19=0000f9d1378b3000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x20=0000f9d1378fd800,
>    x21=0000f9d1373eb000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x22=0000000000010000,
>    x23=0000f9d12f9de3e0
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6365361]    x24=0000000000000016,
>    x25=0000f9d137c61000
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6465381]    x26=0000000000010000,
>    x27=0000f9d130033ce0
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6465381]    x28=0000000000000001,
> fp=x29=0000ffffffa90540
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6465381] lr=x30=0000f9d137898d58,
>     sp=0000ffffffa90540
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6465381]
> ------------------------------------------------
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6465381] cpu3: End traceback...
> Mar 19 11:52:30 rock64 /netbsd:
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.6465381] dump to dev 92,33 not
> possible
> Mar 19 11:52:30 rock64 /netbsd: [ 422844.7165395] rebooting...
>
>
>

More info on this:

I'm seeing evidence this panic (and 2 others, below) are related to use of
NFS. I was copying about 1.5 Gb of images per day over NFS. Once I
switched to using SSH (SCP) I've gone from 1-2 panics per day to none.

The other 2 kernel panics before switching to SCP from NFS:

[ 68350.008353] panic: Trap: Data Abort (EL1): Translation Fault L0 with
read access for 000000007780a5a4: pc ffffffc000478f28: opcode b9402413:
ldr w19, [x0,#36]

[ 68350.008353] cpu0: Begin traceback...
[ 68350.008353] trace fp ffffffc0606977f0
[ 68350.008353] fp ffffffc060697810 vpanic() at ffffffc00042b4d0
netbsd:vpanic+0x198
[ 68350.008353] fp ffffffc060697870 panic() at ffffffc00042b5a0
netbsd:panic+0x48
[ 68350.018353] fp ffffffc060697900 data_abort_handler() at
ffffffc000075d2c netbsd:data_abort_handler+0x4c4
[ 68350.018353] tf ffffffc060697980 el1_trap() at ffffffc0000730dc
netbsd:el1_trap
...

[ 173637.965616] panic: trap_el0_error
[ 173637.965616] cpu4: Begin traceback...
[ 173637.965616] trace fp ffffffc0661f7db0
[ 173637.975617] fp ffffffc0661f7dd0 vpanic() at ffffffc00042b4d0
netbsd:vpanic+0x198
[ 173637.975617] fp ffffffc0661f7e30 panic() at ffffffc00042b5a0
netbsd:panic+0x48
[ 173637.975617] fp ffffffc0661f7ec0 trap_el0_error() at ffffffc000075094
netbsd:trap_el0_error+0x1c
[ 173637.975617] tf ffffffc0661f7ed0 el0_trap() at ffffffc000073148
netbsd:el0_trap
[ 173637.975617] ---- trapframe 0xffffffc0661f7ed0 (304 bytes) ----
...