Re: Vnode API change: add global vnode cache

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

   Date: Mon, 14 Apr 2014 15:58:28 +0200
   From: "J. Hannken-Illjes" <hannken%eis.cs.tu-bs.de@localhost>

   Also replaced the cv_wait/cv_broadcast with kpause as insert collisions
   are very rare and moved the common vnode initialisation to vnalloc.

   New diff at http://www.netbsd.org/~hannken/vnode-pass6-3.diff

   Plan to commit early wednesday ...

I still don't think this approach is right.  It makes a long-term copy
of logic in getnewvnode (because this vcache(9) will not be applicable
everywhere), there's no reason to use kpause or any new condvars when
we already have one inside each vnode which we'll be using anyway in
vget, and it still increases the overhead of every vnode using it.

I don't object to the principle of VFS_LOAD_NODE, or VOP_LOAD, but at
the moment I think it will cause greater divergence between file
systems and maintenance headaches as a result.

As an alternative, I propose the set of patches at
<https://www.NetBSD.org/~riastradh/tmp/20140415/>, to do the
following:

1. Add vready(vp), which callers of getnewvnode are required to call
before anyone else can vget vp.  This doesn't really add any new
states to vnodes -- it just changes the wait-for-initialization
mechanism from abusing the vnode lock to waiting on the vnode's
condvar.

2. Make VFS_VGET, VFS_FHTOVP, and VFS_ROOT return unlocked (but
referenced) vnodes.

3. Import a vcache-like abstraction I wrote a couple months ago,
vinotab (`vfs inode table'), which adds no memory overhead to vnodes
versus ufs_ihash.

4. Convert some file systems to use it (more to come later), and kill
various lock dances.

This changes all file systems at once to use vready and to change the
contract of VFS_VGET, VFS_FHTOVP, and VFS_ROOT in steps (1) and (2).
The remaining changes are incremental and entail no changes to the
VFS/vnode API itself -- notably, vinotab sits outside vfs_vnode.c and
only captures a pattern that file systems could use directly.