Re: kern/58776: RAIDframe panic on I/O error during reconstruction

["Emmanuel Dreyfus via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR kern/58776; it has been noted by GNATS.

From: Emmanuel Dreyfus <manu%netbsd.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: kern/58776: RAIDframe panic on I/O error during reconstruction
Date: Mon, 28 Oct 2024 13:28:27 +0000

 On Sun, Oct 27, 2024 at 04:00:03PM +0000, Greg Oster wrote:
 >  Did you catch anything else printed from the kernel?  
 
 This time it was wd2 write error 
 
 [ 144555.7191324] raid1: initiating in-place reconstruction on column 1
 [ 191191.9781226] wd2d: device timeout writing fsbn 15953755106 of 15953755106-15953755137 (wd2 bn 15953755106; cn 15827138 tn 0 sn 2), xfer d68, retry 0
 [ 191191.9914893] wd2d: device timeout writing fsbn 15953755138 of 15953755138-15953755169 (wd2 bn 15953755138; cn 15827138 tn 0 sn 34), xfer ef8, retry 0
 [ 191192.0050090] wd2d: device timeout writing fsbn 15953755170 of 15953755170-15953755201 (wd2 bn 15953755170; cn 15827138 tn 1 sn 3), xfer e94, retry 0
 [ 191192.0184433] wd2d: device timeout writing fsbn 15953755202 of 15953755202-15953755233 (wd2 bn 15953755202; cn 15827138 tn 1 sn 35), xfer b74, retry 0
 [ 191192.7281226] wd2: soft error (corrected) xfer d68
 [ 191192.7396135] wd2: soft error (corrected) xfer b74
 [ 191192.7396135] wd2: soft error (corrected) xfer e94
 [ 191192.7493198] wd2: soft error (corrected) xfer ef8
 [ 191241.6781226] wd2d: device timeout writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44), xfer f5c, retry 0
 [ 191241.6916686] wd2d: device timeout writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13), xfer bd8, retry 0
 [ 191241.7052746] wd2d: device timeout writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45), xfer e94, retry 0
 [ 191241.7188813] wd2d: device timeout writing fsbn 15966378050 of 15966378050-15966378081 (wd2 bn 15966378050; cn 15839660 tn 12 sn 14), xfer d04, retry 0
 [ 191252.2181226] wd2d: device timeout writing fsbn 15966378050 of 15966378050-15966378081 (wd2 bn 15966378050; cn 15839660 tn 12 sn 14), xfer d04, retry 1
 [ 191252.2316606] wd2d: device timeout writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45), xfer e94, retry 1
 [ 191252.2452678] wd2d: device timeout writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13), xfer bd8, retry 1
 [ 191252.2588739] wd2d: device timeout writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44), xfer f5c, retry 1
 [ 191262.7581228] wd2d: device timeout writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44), xfer f5c, retry 2
 [ 191262.7716625] wd2d: device timeout writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13), xfer bd8, retry 2
 [ 191262.7852692] wd2d: device timeout writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45), xfer e94, retry 2
 [ 191262.7988751] wd2d: device timeout writing fsbn 15966378050 of 15966378050-15966378081 (wd2 bn 15966378050; cn 15839660 tn 12 sn 14), xfer d04, retry 2
 [ 191273.2981226] wd2d: device timeout writing fsbn 15966378050 of 15966378050-15966378081 (wd2 bn 15966378050; cn 15839660 tn 12 sn 14), xfer d04, retry 3
 [ 191273.3116644] wd2d: device timeout writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45), xfer e94, retry 3
 [ 191273.3252700] wd2d: device timeout writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13), xfer bd8, retry 3
 [ 191273.3388764] wd2d: device timeout writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44), xfer f5c, retry 3
 [ 191680.6601668] wd2d: device timeout writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44), xfer f5c, retry 4
 [ 191690.6751717] wd2d: device timeout writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13), xfer bd8, retry 4
 [ 191700.6901767] wd2d: device timeout writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45), xfer e94, retry 4
 [ 191710.7051816] wd2d: device timeout writing fsbn 15966378050 of 15966378050-15966378081 (wd2 bn 15966378050; cn 15839660 tn 12 sn 14), xfer d04, retry 4
 [ 191991.0752951] wd2d: device timeout writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44)
 [ 191991.0871980] wd2d: error writing fsbn 15966377954 of 15966377954-15966377985 (wd2 bn 15966377954; cn 15839660 tn 10 sn 44)
 [ 191991.0970825] raid1: Recon write failed (status 5(0x5))!
 [ 191991.0970825] raid1: reconstruction failed.
 [ 192001.1003051] wd2d: device timeout writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13)
 [ 192001.1122002] wd2d: error writing fsbn 15966377986 of 15966377986-15966378017 (wd2 bn 15966377986; cn 15839660 tn 11 sn 13)
 [ 192001.1220854] raid1: Recon write failed (status 5(0x5))!
 [ 192001.1220854] raid1: 566502314 recon event waits, 11 recon delays
 [ 192001.1303189] raid1: 2821808363 max exec ticks
 [ 192011.1253150] wd2d: device timeout writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45)
 [ 192011.1372114] wd2d: error writing fsbn 15966378018 of 15966378018-15966378049 (wd2 bn 15966378018; cn 15839660 tn 11 sn 45)
 [ 192011.1470958] raid1: Recon write failed (status 5(0x5))!
 [ 192011.1470958] uvm_fault(0xc420f140, 0xfffff000, 1) -> 0xe
 [ 192011.1592364] fatal page fault in supervisor mode
 [ 192011.1592364] trap type 6 code 0 eip 0xc0900a08 cs 0x8 eflags 0x210206 cr2 0xfffffff0 ilevel 0 esp 0
 [ 192011.1731966] curlwp 0xc6dde3c0 pid 0 lid 234 lowest kstack 0xdf0f72c0
 [ 192011.1797854] panic: trap
 [ 192011.1797854] cpu0: Begin traceback...
 [ 192011.1862893] vpanic(c0d6708c,df0f8dd0,df0f8e8c,c012fd98,c0d6708c,df0f8e98,df0f8e98,ea,df0f72c0,210206) at netbsd:vpanic+0x196
 [ 192011.1957062] panic(c0d6708c,df0f8e98,df0f8e98,ea,df0f72c0,210206,fffffff0,0,0,c42716a0) at netbsd:panic+0x18
 [ 192011.2055306] trap() at netbsd:trap+0xd51
 [ 192011.2055306] --- trap (number 6) ---
 [ 192011.2173051] mutex_vector_enter(c75b507c,1,1,c6dd7f44,c6dd7f44,c670916c,c6dd8308,df0f8f78,c078dc03,c670916c) at netbsd:mutex_vector_enter+0x91
 [ 192011.2259154] rf_CauseReconEvent(c670916c,1,c6dd7f44,9,c6709244,df0f8f9c,c077e310,c6dd7f44,5,5) at netbsd:rf_CauseReconEvent+0x58
 [ 192011.2354273] ReconWriteDoneProc(c6dd7f44,5,5,c6709248,c670916c,c077e271,c6dde3c0,0,c0102011,c670916c) at netbsd:ReconWriteDoneProc+0x79
 [ 192011.2557962] rf_RaidIOThread(c670916c,43e6000,43fb000,0,c01005a8,0,0,0,0,0) at netbsd:rf_RaidIOThread+0x9f
 [ 192011.2661329] cpu0: End traceback...
 
 -- 
 Emmanuel Dreyfus
 manu%netbsd.org@localhost