tech-kern archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Lightweight virtualization - the rump approach

On 05/15/10 16:16, Antti Kantee wrote:
On Sat May 15 2010 at 12:44:11 +0200, Jean-Yves Migeon wrote:
Hmm, I think there is some confusion as to how rump works.  The best
way to find out is probably by running some programs and seeing what
they do and what routines in what files they call (easy now thanks to
Paul Koning fixing our forever-broken gdb).

Will do.

When you get above rumpkern (and some code in the faction libs, like
vm support support routines in rumpvfs), the only code not from the
regular kernel is (off the top of my head):
a) devices specific to rump (like virtif already mentioned in the
    previous email)
b) some module improvements (supporting dynamic attach where the kernel
    code is not yet module-ready, e.g. most usb drivers)
c) things dealing with the lack of devfs.  a regular system can assume
    MAKEDEV has been run.  a rump kernel cannot, so drivers create
    necessary device nodes when they attach (if the device node files
    cannot be encoded with my recent conf/majors stuff)

"b" and "c" will go away when NetBSD is fixed.  So there's really nothing
rump-specific left.

So to answer your original question, for tcpct it's probably a matter of
adding a source file to SRCS.  OTOH, I already have most of the support
done to write rump Makefiles in terms of config, e.g. ffs looks like this:

=== snip ===
include "conf/files"

file-system FFS

options         WAPBL
options         APPLE_UFS
options         UFS_DIRHASH
options         FFS_EI
options         QUOTA
=== snip ===
(that can and should be used for the kernel module too, of course.
full exploitation requires some of the stuff uebs was writing about in
his tech-kern marathon 2 months ago.)

Ok, thanks for the explanations.

For USB it depends on what USB3 demands from the host controller.
You might need to modify ugenhc.  Notably, that doesn't have anything to
do with running the USB stack in a rump kernel, "only" with accessing the
devices.  ... I guess the usb stack without devices is a little boring.
Another more generic but maybe less flexible (in terms of runtime) way
would be to use IOMMUs and run the hardware host controller driver in
a rump kernel.

On a more general perspective, consider that a specific driver, file
system, network protocol... is ported within NetBSD. Would that make it
easy to port to another OS via rump? Does the code has to be written
following specific guidelines, so it can take advantage from the
"faction" layer?

I'm not really sure what you mean, but I think "port" is the wrong word.

The portability of rump and the drivers within is defined in terms
of rumpuser.

Let's take an example; suppose that $somefs_support is integrated in NetBSD (like ZFS, NILFS, ...), what kind of additional work is needed to go from mount_somefs to rump_somefs (thinking about all _KERNEL functions that are missing).

ZFS is known for being invasive, and reproduce functionalities you could find in other layers, like raid(4). Do such things affect rump?

Well, the dom0 should be minimal, for security reasons. In a "perfect"
world, all devices would be exported to "semi privileged domUs" and
protected from others with different mechanisms, like IOMMU. The dom0
would just be an OS running the XenStore, and some simple backends.

Well, maybe.  That does make some sense if you already are capable
of hosting domU's.  I never found Xen very convenient for my use case
(occasional testing) because it needs a special dom0 kernel.  Why isn't
that stuff in GENERIC again?  IIRC there were some device support issues
years ago, but do they still remain?

At least for x86, key parts of MD code are handled differently, especially pmap(9), locore's boot stuff, and some bus_space code. So you have #ifdef/inline thingies, which are, as you have noticed, not that well modular-friendly.

Making this MD part dynamic would need some clean up in x86+xen, and probably introduce function pointers to make the thing more dynamic and "stable", like Linux and the big paravirt_ops.

dom0 vs domU is another story, it is basically a reduced version of dom0, cleaned from all drivers and code that are not required for a domU. May become unnecessary when kernel becomes more and more modular:
-rwxr-xr-x  1 root  wheel   11M Apr 28 09:21 netbsd_XEN3PAE_DOM0
-rwxr-xr-x  1 root  wheel  4,1M Apr 28 09:21 netbsd_XEN3PAE_DOMU

I'm no longer really sure why someone would want to use a usermode
operating system on top of POSIX at all, since scheduling in them seems
quite inefficient in my experience.  But if someone does, the overlap is
that lot of the drivers written for rump can be used, for example ugenhc
which accesses host USB devices or virtif which shovels network traffic
to/from /dev/tap.  rumpfs could probably be extended into a fullblown
hostfs, etc ...

Wild guess: from an admin PoV (not a developer), compared to other
approaches like capabilities, or rump, the containers/zones approach are
familiar; you "deprivilege" a full blown kernel in an
implementation-defined namespace protected by the host kernel; then, you
configure the guest in the same way as you would do it with a
traditional general OS sitting above.

Oh I wasn't arguing against full virtualization, that is definitely
useful.  Just that the usermode way of doing it doesn't seem to fit
any bill.  It's not particularly fast (vs. e.g. Xen) and it's not
particularly convenient due to limited hostability (vs. e.g. qemu).
For these reasons, I don't find it better than the alternatives.  Plus,
it has the full virtualization admin drag where you need to setup and
maintain an entire system regardless of how little you are actually
interested in virtualizing.  Sure, it's "familiar" maintenance, but
maintenance nonetheless.

Point taken.


Jean-Yves Migeon

Home | Main Index | Thread Index | Old Index