Re: Lightweight virtualization - the rump approach

[Jean-Yves Migeon <jeanyves.migeon%free.fr@localhost>]

On 05/15/10 16:16, Antti Kantee wrote:
> On Sat May 15 2010 at 12:44:11 +0200, Jean-Yves Migeon wrote:
> Hmm, I think there is some confusion as to how rump works.  The best
> way to find out is probably by running some programs and seeing what
> they do and what routines in what files they call (easy now thanks to
> Paul Koning fixing our forever-broken gdb).

Will do.

> When you get above rumpkern (and some code in the faction libs, like
> vm support support routines in rumpvfs), the only code not from the
> regular kernel is (off the top of my head):
> a) devices specific to rump (like virtif already mentioned in the
>     previous email)
> b) some module improvements (supporting dynamic attach where the kernel
>     code is not yet module-ready, e.g. most usb drivers)
> c) things dealing with the lack of devfs.  a regular system can assume
>     MAKEDEV has been run.  a rump kernel cannot, so drivers create
>     necessary device nodes when they attach (if the device node files
>     cannot be encoded with my recent conf/majors stuff)
>
> "b" and "c" will go away when NetBSD is fixed.  So there's really nothing
> rump-specific left.
>
> So to answer your original question, for tcpct it's probably a matter of
> adding a source file to SRCS.  OTOH, I already have most of the support
> done to write rump Makefiles in terms of config, e.g. ffs looks like this:
>
> === snip ===
> include "conf/files"
>
> file-system FFS
>
> options         WAPBL
> options         APPLE_UFS
> options         UFS_DIRHASH
> options         FFS_EI
> options         QUOTA
> === snip ===
> (that can and should be used for the kernel module too, of course.
> full exploitation requires some of the stuff uebs was writing about in
> his tech-kern marathon 2 months ago.)

Ok, thanks for the explanations.

> For USB it depends on what USB3 demands from the host controller.
> You might need to modify ugenhc.  Notably, that doesn't have anything to
> do with running the USB stack in a rump kernel, "only" with accessing the
> devices.  ... I guess the usb stack without devices is a little boring.
> Another more generic but maybe less flexible (in terms of runtime) way
> would be to use IOMMUs and run the hardware host controller driver in
> a rump kernel.
>
> > On a more general perspective, consider that a specific driver, file
> > system, network protocol... is ported within NetBSD. Would that make it
> > easy to port to another OS via rump? Does the code has to be written
> > following specific guidelines, so it can take advantage from the
> > "faction" layer?
>
> I'm not really sure what you mean, but I think "port" is the wrong word.
>
> The portability of rump and the drivers within is defined in terms
> of rumpuser.

Let's take an example; suppose that $somefs_support is integrated in 
NetBSD (like ZFS, NILFS, ...), what kind of additional work is needed to 
go from mount_somefs to rump_somefs (thinking about all _KERNEL 
functions that are missing).

ZFS is known for being invasive, and reproduce functionalities you could 
find in other layers, like raid(4). Do such things affect rump?

> > Well, the dom0 should be minimal, for security reasons. In a "perfect"
> > world, all devices would be exported to "semi privileged domUs" and
> > protected from others with different mechanisms, like IOMMU. The dom0
> > would just be an OS running the XenStore, and some simple backends.
>
> Well, maybe.  That does make some sense if you already are capable
> of hosting domU's.  I never found Xen very convenient for my use case
> (occasional testing) because it needs a special dom0 kernel.  Why isn't
> that stuff in GENERIC again?  IIRC there were some device support issues
> years ago, but do they still remain?

At least for x86, key parts of MD code are handled differently, 
especially pmap(9), locore's boot stuff, and some bus_space code. So you 
have #ifdef/inline thingies, which are, as you have noticed, not that 
well modular-friendly.

Making this MD part dynamic would need some clean up in x86+xen, and 
probably introduce function pointers to make the thing more dynamic and 
"stable", like Linux and the big paravirt_ops.

dom0 vs domU is another story, it is basically a reduced version of 
dom0, cleaned from all drivers and code that are not required for a 
domU. May become unnecessary when kernel becomes more and more modular:
-rwxr-xr-x  1 root  wheel   11M Apr 28 09:21 netbsd_XEN3PAE_DOM0
-rwxr-xr-x  1 root  wheel  4,1M Apr 28 09:21 netbsd_XEN3PAE_DOMU

> > > I'm no longer really sure why someone would want to use a usermode
> > > operating system on top of POSIX at all, since scheduling in them seems
> > > quite inefficient in my experience.  But if someone does, the overlap is
> > > that lot of the drivers written for rump can be used, for example ugenhc
> > > which accesses host USB devices or virtif which shovels network traffic
> > > to/from /dev/tap.  rumpfs could probably be extended into a fullblown
> > > hostfs, etc ...
> >
> > Wild guess: from an admin PoV (not a developer), compared to other
> > approaches like capabilities, or rump, the containers/zones approach are
> > familiar; you "deprivilege" a full blown kernel in an
> > implementation-defined namespace protected by the host kernel; then, you
> > configure the guest in the same way as you would do it with a
> > traditional general OS sitting above.
>
> Oh I wasn't arguing against full virtualization, that is definitely
> useful.  Just that the usermode way of doing it doesn't seem to fit
> any bill.  It's not particularly fast (vs. e.g. Xen) and it's not
> particularly convenient due to limited hostability (vs. e.g. qemu).
> For these reasons, I don't find it better than the alternatives.  Plus,
> it has the full virtualization admin drag where you need to setup and
> maintain an entire system regardless of how little you are actually
> interested in virtualizing.  Sure, it's "familiar" maintenance, but
> maintenance nonetheless.

Point taken.

Cheers,

--
Jean-Yves Migeon
jeanyves.migeon%free.fr@localhost