Right now the way we do vnode locking looses for stacked filesystems, like
nullfs. I'm going to propose a change in how we do things to help them
along. I'll discuss everything in terms of nullfs, but all the other fs's
should be about the same (union will be a bit tricky). This email is to
solicit comments & suggestions.

The problem is that in a stacked filesystem, each layer has its own vnode
and assosciated locks(*). As locks are used both to lock struct vnode
fields and to also limit access to the file (say during a read or write),
locking the vnode on the upper layer really needs to also imply locking
the node on the lower layer.

(*) nullfs at present doesn't lock its vnodes - no wonder we have so many
problems!

Right now the locking procedure in response to the VOP_LOCK call is to
(kinda, see (*) above) lock the upper vnode, and then lock the underlying
vnode. The problem is that this locking order is opposite the order used
in response to a VOP which returns a locked vnode, such as VOP_LOOKUP. In
the latter case, the returned vnode will be locked on the lower node
before the upper node is able to lock it. If two processes are trying to
access the same vnode, one via VOP_LOCK and the other via VOP_LOOKUP, we
get a race condition. If each one gets the first of the two nodes locked,
we will deadlock when they go to lock the other.

Rather than just change the VOP_LOCK protocol to lock bottom up, I'd
rather we change the vnode lock interface so that all vnodes point to the
relevant struct lock for the node. For stacked fs's, upper vnodes would
point to the struct lock for the lower vnode. Thus if we have multiple
layers stacked on each other, whenever any one node gets locked, they all
are locked.

To do this, I'll add a struct lock to struct vnode, and have a genfs_lock,
genfs_unlock, and genfs_islocked which would actually use the struct lock
* in the vnode. vget would make sure that the struct lock * pointed to
that vnode's struct lock when it gets pulled off of the free list.

nullfs & umapfs would then just point the struct lock * in the upper node
to the lock in the lower node. I think for unionfs, the union lock should
point to the upper vndoe if present, and the lower vnode otherwise.

Thoughts?

Take care,

Bill