Re: NFS experts: stale vnode pointer in ufs_fhtovp ?

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Sun, Aug 30, 2009 at 10:47:38PM +0200, Manuel Bouyer wrote:
> > I can't think of any reason a valid UFS vnode should have a null inode
> > pointer. Maybe someone else can though?
> > 
> > My first guess would be that you got back some other fs's vnode and
> > that something upstream, maybe the vnode cache, is/was all screwed
> > up. My second guess would be that you've hit a(nother) bug in vnode
> > reclaim/recycle, and you got what was a valid ufs vnode when it was
> > fetched from the cache but has since been cleaned on you.
> > 
> > I don't know why either of these would be happening, thoughI got the same 
> > panic again, this time I got a core dump:
> #23 0xc010cc37 in calltrap ()
> #24 0xc02ccc89 in ufs_fhtovp (mp=0xccf7b000, ufhp=0xcd585918, vpp=0xcd585b40)
>     at /home/bouyer/src-5/src/sys/ufs/ufs/ufs_vfsops.c:191
> #25 0xc02a5425 in ffs_fhtovp (mp=0xccf7b000, fhp=0xcd585ae0, vpp=0xcd585b40)
>     at /home/bouyer/src-5/src/sys/ufs/ffs/ffs_vfsops.c:1992
> #26 0xc039a6cc in VFS_FHTOVP (mp=0xccf7b000, a=0xcd585ae0, b=0xcd585b40)
>     at /home/bouyer/src-5/src/sys/kern/vfs_subr.c:3034
> #27 0xc026b518 in nfsrv_fhtovp (nsfh=0xcd585ad4, lockflag=1, vpp=0xcd585b40, 
>     cred=0xcd7ff000, slp=0xcdb8feb0, nam=0xc2c0e800, rdonlyp=0xcd585b4c, 
>     kerbflag=0, pubflag=0) at /home/bouyer/src-5/src/sys/nfs/nfs_subs.c:2515
> #28 0xc0260d38 in nfsrv_write (nfsd=0xcd5ece10, slp=0xcdb8feb0, 
>     lwp=0xccee92a0, mrq=0xcd585bd0)
>     at /home/bouyer/src-5/src/sys/nfs/nfs_serv.c:872
> #29 0xc0270134 in nfssvc_nfsd (nsd=0xcd585c38, argp=0xbb5ffc28, l=0xccee92a0)
>     at /home/bouyer/src-5/src/sys/nfs/nfs_syscalls.c:685
> #30 0xc0270de2 in sys_nfssvc (l=0xccee92a0, uap=0xcd585d00, retval=0xcd585d28)
>     at /home/bouyer/src-5/src/sys/nfs/nfs_syscalls.c:349
> #31 0xc03f6ec8 in syscall (frame=0xcd585d48)
>     at /home/bouyer/src-5/src/sys/sys/syscallvar.h:49
> #32 0xc010056e in syscall1 ()
> 
> c89 in ufs_fhtovp (mp=0xccf7b000, ufhp=0xcd585918, vpp=0xcd585b40)
>     at /home/bouyer/src-5/src/sys/ufs/ufs/ufs_vfsops.c:191
> 191             ip = VTOI(nvp);
> (gdb) print nvp
> $1 = (struct vnode *) 0xd2e542e8
> (gdb) print *nvp
> $2 = {v_uobj = {vmobjlock = {u = {mtxa_owner = 4}}, pgops = 0xc04e7aa0,
>     memq = {tqh_first = 0x0, tqh_last = 0xd2e542f0}, uo_npages = 0,
>     uo_refs = 1, rb_tree = {rbt_root = 0x0, rbt_ops = 0xc04e79bc,
>       rbt_minmax = {0xd2e54300, 0xd2e54300}}}, v_cv = {cv_opaque = {0x0,
>       0xd2e54310}, cv_wmesg = 0xc05f036b "vnode"}, v_size = 0,
>   v_writesize = 0, v_iflag = 524288, v_vflag = 16, v_uflag = 0,
>   v_numoutput = 0, v_writecount = 0, v_holdcnt = 0, v_synclist_slot = 4,
>   v_mount = 0xccf7b000, v_op = 0xc1ac0e00, v_freelist = {
>     tqe_next = 0xd212e964, tqe_prev = 0xcf06c40c}, v_freelisthd = 0x0,
>   v_mntvnodes = {tqe_next = 0xd2e4945c, tqe_prev = 0xcd3e2a90},
>   v_cleanblkhd = {lh_first = 0x0}, v_dirtyblkhd = {lh_first = 0x0},
>   v_synclist = {tqe_next = 0xd212e964, tqe_prev = 0xce765364}, v_dnclist = {
>     lh_first = 0x0}, v_nclist = {lh_first = 0x0}, v_un = {
>     vu_mountedhere = 0x0, vu_socket = 0x0, vu_specnode = 0x0,
>     vu_fifoinfo = 0x0, vu_ractx = 0x0}, v_type = VREG, v_tag = VT_NON,
>   v_lock = {vl_lock = {rw_owner = 3438187180}, vl_canrecurse = 0,
>     vl_recursecnt = 0}, v_vnlock = 0xd2e54388, v_data = 0x0, v_klist = {
>     slh_first = 0x0}}
> 
> The v_mount pointer matches what was passed to ufs_fhtovp, but
> the VI_CLEAN flag is set ...


Actually I can't see what prevents the vget() in ufs_ihashget()
from getting a VI_CLEAN vnode. in vget(), vn_lock() will drop the v_interlock
before trying to get the LK_EXCLUSIVE (in our case) lock via VOP_LOCK().
AKAIK it can sleep or get preempted in while doing this. While it's
sleeping vlean could run (for example as part of a VOP_REMOVE()), and
when vget() returns the vnode has been cleaned, and the associated inode
removed from the ihashtbl[]. vrelel() and vclean() will clean the vnode
even if the refcount is > 1, VOP_INACTIVE() returned recycle == true.

what I am missing ?

-- 
Manuel Bouyer, LIP6, Universite Paris VI.           
Manuel.Bouyer%lip6.fr@localhost
     NetBSD: 26 ans d'experience feront toujours la difference
--