Re: Substantial COMPAT_LINUX changes in netbsd-5?

[Havard Eidnes <he%nordu.net@localhost>]

Hi,

another update: I wanted to add a KASSERT() at the top of
uvn_findpages(), and while rebooting discovered that reboot also
got stuck in tstile.  Anyway, broke into ddb, and found:

db{0}> ps      
PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
20756    1 3   1         4           e80c0d40             reboot tstile
6695     1 3   2   9020004           e4807580             bpbkar tstile
3952     1 3   2   9020004           e46bd280             bpbkar tstile
3081     1 3   2   9020004           e9408d20                 df tstile
2519     1 3   1   9020004           d89250c0                 df tstile
3006     1 3   1   9020004           d8898ca0                 df tstile
17026    1 3   2   9020004           e4807800             bpbkar nfsrcv
5421     1 3   1   9020004           d89a07a0             bpbkar uvn_fp2
1        1 3   2   8020084           ce3bc840               init wait
0       73 3   0       204           e94080a0             ktrace ktrwait
...
db{0}> trace/t 0t5421
trace: pid 5421 lid 1 at 0xd89c43cc
sleepq_block(0,0,c0aaba51,c0b27c80,0,c150a9ac,9,c2580910,da4a13a0,0) at netbsd:s
leepq_block+0xeb
mtsleep(c2580910,204,c0aaba51,0,da4a13a0,da4a13a0,10,6,0,0) at netbsd:mtsleep+0x
12d
uvn_findpage(d89c45ac,0,d89c44ac,c05343fa,0,0,2,0,994000,d89c45cc) at netbsd:uvn
_findpage+0x92
uvn_findpages(da4a13a0,24e60000,2,d89c45ec,d89c45ac,0,994000,20,2,0) at netbsd:u
vn_findpages+0x73
genfs_getpages(d89c46b0,0,0,0,0,24ed0000,0,0,2,d89c465c) at netbsd:genfs_getpage
s+0x743
nfs_getpages(d89c46b0,4,24e62000,2,0,10000,24ee0000,c089d600,da4a13a0,24e60000) 
a
t netbsd:nfs_getpages+0xbb
VOP_GETPAGES(da4a13a0,24e60000,2,d89c4750,d89c47c8,0,1,0,1802,0) at netbsd:VOP_G
ETPAGES+0x65
uvn_get(da4a13a0,24e60000,2,d89c4750,d89c47c8,0,1,0,1802,d89a07a0) at netbsd:uvn
_get+0x117
ubc_fault(d89c48e0,d3981000,d89c48a0,1,0,1,42,246,8,c0bc8d04) at netbsd:ubc_faul
t+0x170
uvm_fault_internal(c0bc21c0,d3981000,1,0,c262cfca,c0000,0,c05a6cfa,6,6) at netbs
d:uvm_fault_internal+0x3a9
trap() at netbsd:trap+0x797
--- trap (number 6) ---
copyout(d87906c0,d3981000,8249438,2000,d87906c0,0,d3981000,24e60000,2,d3981000) 
a
t netbsd:copyout+0x33
uiomove(d3981000,2000,d89c4c8c,d89c4adc,0,101,deaddead,0,1829b58,0) at netbsd:ui
omove+0x62
ubc_uiomove(da4a13a0,d89c4c8c,10000,0,101,7c356d21,d89c4b2c,c085d206,da4945c0,da
4a1440) at netbsd:ubc_uiomove+0xeb
nfs_bioread(da4a13a0,d89c4c8c,0,ce3a6f00,0,da4a13a0,d89c4c2c,c053d6f4,d89c4c14,d
a4a13a0) at netbsd:nfs_bioread+0x312
nfs_read(d89c4c14,da4a13a0,c089d3c0,da4a13a0,1,20001,d89c4c2c,c0534d58,c089ce80,
da4a13a0) at netbsd:nfs_read+0x43
VOP_READ(da4a13a0,d89c4c8c,0,ce3a6f00,d40a1040,0,9c4c6c,16,10000,8249438) at net
bsd:VOP_READ+0x44
vn_read(d8c4d940,d8c4d940,d89c4c8c,ce3a6f00,1,0,0,0,d89a07a0,d89c4d48) at netbsd
:vn_read+0x93
dofileread(9,d8c4d940,8249438,10000,d8c4d940,1,d89c4d28,d89c4d48,d89c4d48,d89a07
a0) at netbsd:dofileread+0x75
sys_read(d89a07a0,d89c4d10,d89c4d28,9c4d20,96,10,c0b4a744,9,8249438,10000) at ne
tbsd:sys_read+0x6f
linux_syscall(d89c4d48,2b,2b,2b,2b,610,8259338,bfbeec08,9,10000) at netbsd:linux
_syscall+0x9b
db{0}>
db{0}>
db{0}>
db{0}>
db{0}> show lock 0xda4a13a0
lock address : 0x00000000da4a13a0 type     :     sleep/adaptive
initialized  : 0x00000000c052b9c6
shared holds :                  0 exclusive:                  0
shares wanted:                  0 exclusive:                  0
current cpu  :                  0 last held:                  1
current lwp  : 0x00000000ce3a7c80 last held: 000000000000000000
last locked  : 0x00000000c03d3f4c unlocked : 0x00000000c03d403b
owner field  : 000000000000000000 wait/spin:                0/0

Turnstile chain at 0xc150ba80.
=> No active turnstile for this lock.
db{0}> 
db{0}> x/i 0x00000000c03d3f4c
netbsd:nfs_sync+0x7c:   cmpl    $0x3,0xc(%ebp)
db{0}> x/i 0x00000000c03d403b
netbsd:nfs_sync+0x16b:  jmp     netbsd:nfs_sync+0x44
db{0}> 

and some disassembly of nfs_sync() reveals that the mutex in
question must be mntvnode_lock -- a global lock (not per file
system) for inspecting or manipulating the list of vnodes
associated with any file system.

Now, if I read the above correctly, that lock is currently not
held.  However, the bpbkar process is still waiting on it,
ref. the 5th argument to mtsleep() in the stack backtrace, and
this might explain why the bpbkar process is not making any
progress at this point.

Now, where to look next?

Regards,

- Håvard