Re: Vnode API change: add global vnode cache

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

   Date: Wed, 9 Apr 2014 11:10:37 +0200
   From: "J. Hannken-Illjes" <hannken%eis.cs.tu-bs.de@localhost>

   There is no need to do this VI_CHANGING etc. here.  Primary goal of
   the vnode cache is to always initialise vnode / fs node pairs before
   they become visible or appear on any lists.

But the vnode/fsnode pair effectively already does appear on a list,
namely in the vcache -- it's just that you've added a new mechanism
for waiting for the vnode's state to change so that *fs_vget now has
to have two cv_wait loops, one in vcache and one in vget.

Either way, once someone has decided to create a vnode for an inode,
and before the inode has been loaded from disk, everyone else wanting
a vnode for that inode must wait for its state to change.  I don't see
a compelling reason to have more than one mechanism to wait for vnode
state transitions.

   There is no need to make the vcache API look like getnewvnode().  But
   getnewvnode() should change to a two-pass allocator that initialises
   in the first pass and finalises, adds to mount list in the second pass.

Splitting it into two parts sounds good to me.  I don't think it
matters much which pass puts it in the mount list, though.  I propose
we do the following:

1. Add vready(vp).  After getnewvnode, you must call either
ungetnewvnode or vready.  Anyone trying to vget a newly created vnode
blocks until ungetnewvnode or vready.  (We can argue about names
separately -- this is morally equivalent to a two-pass allocator.)

I have a patch to do this in all file systems, and to break the
getnewvnode API so existing calls fail to compile.  It passes all
tests.

2. Change VFS_ROOT, VFS_VGET, and VFS_FHTOVP to return the vnode
unlocked, and mechanically convert all callers to lock on success and
implementations to unlock on exit so that this patch is easy to audit.

I have a patch to do this in all file systems, and to break the
signatures so existing calls fail to compile.  It is mostly debugged
-- only puffs and zfs tests are failing, probably a trivial bug.

3. Add vcache(9) to reduce the copypasta and variance in structure of
the *fs_vget or *fs_fhtovp routines.  This can be done incrementally
per-file-system and requires changes only in *fs_vget or *fs_fhtovp.

4. Make *fs_root, *fs_vget, and *fs_fhtovp never lock the vnodes.
This is not a major API change because it really matters only for
fs-internal callers, so it can be done incrementally.

5. Omit needless lock/unlock pairs that are left as a result of (2),
and omit the needless unlock/vget ../relock dance now that (4) is
done.  This is a clean-up step that doesn't need to be mixed with the
API changes or new functionality, and can also be done bit by bit.