kern/52769: Deadlock in ffs?

[martin%NetBSD.org@localhost]

>Number:         52769
>Category:       kern
>Synopsis:       Deadlock in ffs?
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Nov 28 15:00:00 +0000 2017
>Originator:     Martin Husemann
>Release:        NetBSD 8.99.7
>Organization:
The NetBSD Foundation, Inc.
>Environment:
System: NetBSD seven-days-to-the-wolves.aprisoft.de 8.99.7 NetBSD 8.99.7 (GENERIC) #177: Thu Nov 23 09:03:25 CET 2017 martin%seven-days-to-the-wolves.aprisoft.de@localhost:/work/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:

I got two cvs update process hanging hard (unkillable) in this state:

Memory: 7098M Act, 385M Inact, 11M Wired, 12M Exec, 7439M File, 6607M Free
Swap: 

  PID USERNAME PRI NICE   SIZE   RES STATE      TIME   WCPU    CPU COMMAND
    0 root       0    0     0K   25M CPU/5      1:23  0.00%  0.00% [system]
22820 martin    95    0    47M 8952K bioloc/5   0:15  0.00%  0.00% cvs
18588 martin   127    0    48M 9124K bioloc/2   0:08  0.00%  0.00% cvs

Tried to reboot, which didn't work.

So I dropped into ddb and got:

Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
db{0}> ps
PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
29181    1 3   0         0   ffffe403e1d3c200             reboot tstile
3481     1 3   0        80   ffffe40368d5e080               tcsh ttyraw
18588    1 3   2   1000000   ffffe4040092db80                cvs biolock
14652    1 3   3        80   ffffe403be513640                 sh wait
13756    1 3   1        80   ffffe40397f5f460          ssh-agent select
7734     1 3   0        80   ffffe4038e491460               tcsh pause
22820    1 3   5   1000000   ffffe403ccf032c0                cvs biolock
db{0}> bt /a 0xffffe4040092db80
trace: pid 18588 lid 1 at 0xffffe4011e02b920
sleepq_block() at netbsd:sleepq_block+0x97
cv_timedwait() at netbsd:cv_timedwait+0xa9
bbusy() at netbsd:bbusy+0x7b
getblk() at netbsd:getblk+0x5a
bio_doread() at netbsd:bio_doread+0x1d
bread() at netbsd:bread+0x1a
ffs_update.part.3() at netbsd:ffs_update.part.3+0x112
ffs_truncate() at netbsd:ffs_truncate+0xd91
ufs_truncate_retry() at netbsd:ufs_truncate_retry+0x79
ufs_inactive() at netbsd:ufs_inactive+0x1ac
VOP_INACTIVE() at netbsd:VOP_INACTIVE+0x33
vrelel() at netbsd:vrelel+0x168
ufs_remove() at netbsd:ufs_remove+0xab
VOP_REMOVE() at netbsd:VOP_REMOVE+0x37
do_sys_unlinkat.isra.5() at netbsd:do_sys_unlinkat.isra.5+0x1eb
syscall() at netbsd:syscall+0x1bc
--- syscall (number 10) ---
7a93d4312a7a:

and:

db{0}> bt /a ffffe403ccf032c0
trace: pid 22820 lid 1 at 0xffffe4011de34820
sleepq_block() at netbsd:sleepq_block+0x97
cv_timedwait() at netbsd:cv_timedwait+0xa9
bbusy() at netbsd:bbusy+0x7b
getblk() at netbsd:getblk+0x5a
bio_doread() at netbsd:bio_doread+0x1d
bread() at netbsd:bread+0x1a
ffs_alloccg() at netbsd:ffs_alloccg+0xcc
ffs_hashalloc() at netbsd:ffs_hashalloc+0x28
ffs_alloc() at netbsd:ffs_alloc+0x11e
ffs_balloc() at netbsd:ffs_balloc+0x570
ufs_mkdir() at netbsd:ufs_mkdir+0x22b
VOP_MKDIR() at netbsd:VOP_MKDIR+0x3b
do_sys_mkdirat.isra.7.constprop.12() at netbsd:do_sys_mkdirat.isra.7.constprop.1
2+0x167
syscall() at netbsd:syscall+0x1bc
--- syscall (number 136) ---
78c2024a686a:


The cvses where updating different trees on the same file system.

Got a crash dump and netbsd.gdb, if anyone is interestd (internal
note: seven/netbsd.0.*)


>How-To-Repeat:
n/a

>Fix:
n/a