Re: virtualized nfsd (Re: virtual kernels, syscall routing, etc.)

[Antti Kantee <pooka%cs.hut.fi@localhost>]

On Fri Oct 16 2009 at 05:08:54 -0400, der Mouse wrote:
> >> Good news everyone!
> 
> >> I've made the kernel nfs service (nfsd) run in userspace.
> 
> > Ok, I've worked on this a little more.  [...]
> 
> Okay, this is relatively impressive, extracting what is normally a
> piece of the kernel into a userland process - two pieces, in fact, the
> NFS server and the underlying networking stack.

Actually three, since the NFS server requires something to serve files
off of, i.e. a file system driver.

> But...why?  I had an NFS daemon that ran in userland as an ordinary
> userland process, no "unholy cocktails" or "virtual kernels" or even
> hacked-up librpcsvc needed, decades ago.  I can't have been the only
> one.  Is this just an exercise in recreating a kernel code environment
> in userspace, or is there something else I'm missing?

You are asking two separate questions.

  1) why do we want to virtualize nfsd+networking with a process?

It's convenient, lightweight, and pluggable.  For instance, on a network
you can move your nfs services freely between any physical host (as long
as you can move the backend file system image efficiently).  With the file
system driver in the bundle, that host doesn't necessarily even have to be
NetBSD, but you can still serve files from an image containing our ffs.
And if you want many, say >1000, isolated virtual services, you don't
waste a gigaton of memory like you do in full-OS style virtualization
such as Xen or Usermode $OS.

Someone mentioned they had a real life use case for this.  Maybe they
would care to comment more?

  2) why do we want to do it by reusing kernel code?

Do you have your decades-old nfsd code still working?  Does it support
all the features our kernel nfs service supports?  If you modify one to
fix a bug or add a feature, does the other one get automatically modified?

Part of my point is to show that userspace reimplementations of kernel
services are pointless.  Kernel code can be just as usable, maintainable
and portable, and assuming you already have it, there is no need to
rewrite the wheel.


And a side-benefit, as Matthias mentioned, is kernel development.
Want to run valgrind on kernel code?  Well now you can (only if there
were a fully functional valgrind for NetBSD.  Good thing it works on
Linux ...  Install Arnaud's excellent misc/rump package from pkgsrc on
Linux and give it a try).

Besides, the current cocktail is already very martini-like (extra dry)
and not at all evil or unholy.

Finally, let's take a closer look at what "hacked-up librpcsvc"
involves.  It involves replacing calls like "socket(a,b,c)"
with "rump_sys_socket(a,b,c)" and "sendto(params)" with
"rump_sys_sendto(params)".  That's all.  In other words, it's quite
straightforward.  Eventually I'd like to support both from the same
libc.so.  Maybe for now I'll just make it a compile-time decision for
whoever includes libc/rpc/Makefile.inc.