virtualized nfsd (Re: virtual kernels, syscall routing, etc.)

[Antti Kantee <pooka%cs.hut.fi@localhost>]

On Thu Nov 27 2008 at 20:32:15 +0200, Antti Kantee wrote:
> Good news everyone!
> 
> I've made the kernel nfs service (nfsd) run in userspace.

Ok, I've worked on this a little more.  Now it's possible to run a fully
selfcontained nfsd with a virtualized TCP/IP stack and hence a dedicated
IP address (the previous solution used host IP and rpcbind in a very
unholy cocktail).

What rump nfsd does now is it mounts a file system image to be served
inside the virtual kernel (although you still could serve files from the
host too using a null mount) starts rpcbind, mountd and nfsd as pthreads
and serves away.

So how does it work now?  Just like a normal nfs service.  I've configured
mine to have the address 10.181.181.11, so after running the service
and checking it responds to ping, I can do what I normally could with
nfs, e.g.

pain-rustique:9:~> showmount -e 10.181.181.11
Exports list on 10.181.181.11:
/daexport                          10.181.181.55 10.181.181.195 
pain-rustique:10:~> rump_nfs -p 10.181.181.11:/daexport /puffs
pain-rustique:11:~> df -h /puffs
Filesystem                   Size       Used      Avail %Cap Mounted on
10.181.181.11:/daexport      496M        46M       426M   9% /puffs
pain-rustique:12:~> rump_nfs -p 10.181.181.11:/fail /puffs2
rump_nfs: can't access /fail: Permission denied

etcetc. (yes, you could use mount_nfs instead of rump_nfs, but I don't
trust the nfs client to not crash and burn especially if the server
happens to hiccup a bit).

The bad news is that this currently requires a hacked version of the libc
rpc client.  Without syscall routing mentioned in my first email on the
subject, we cannot route the syscalls libc makes to the right kernel.
The good news is that the modifications are selfcontained and I've put
up a tarball.

So, how to use it?

0) make sure you have latest rump libraries from about 5s ago
0) compile and install sys/rump/fs/lib/libnfsserver (not built by default)
1) get ftp://ftp.netbsd.org/pub/NetBSD/misc/pooka/rumpnfssvc-virtual.tar.gz
2) make
3) run rumpnfs (simply ./rumpnfsd will do)
4) play

Between 1 and 2 you might want to look at ifconf.c to see how the address
is configured.  You also might want to look at rumpnfsd.c to see how
the exported file system is configured.  And between steps 2 and 3 you
might want to modify /etc/exports (yes, it uses the host /etc/exports.
let's not pretend it's not a 30s job to make it use something else).

Now, if someone has any uberbright ideas how to integrate this with
stock rpcbind/mountd/nfsd/libc, I'm pretty much all eyes.

  - antti