Subject: Re: NetBSD and Xen 2.0
To: Thor Lancelot Simon <tls@rek.tjls.com>
From: Manuel Bouyer <bouyer@antioche.eu.org>
List: port-xen
Date: 12/12/2004 18:17:48
Hi,


On Thu, Dec 09, 2004 at 10:08:21AM -0500, Thor Lancelot Simon wrote:
> > So Xen is not backward-compatible? OS for Xen 1.2 can't be run on 2.0 in
> > unprivileged domains?
> 
> The interface between the guest OS and the "hypervisor" (the Xen kernel
> that controls certain privileged operations) changed in 2.0, even for
> unprivileged domains.  It looks like there is some effort for backwards
> compatibility but not a complete one, so the answer is "you need a
> different NetBSD kernel that supports the 2.0 hypervisor interface". 
> 
> In theory, we could maintain two sets of architecture-dependent code
> to allow a NetBSD kernel to run with xen1 or xen2, but that would be
> pretty ugly and painful.

I looked a bit at this, it woud mean a lot of duplicated code.

> A single NetBSD "port" that could do either
> hypervisor interface would be...less ugly, but more painful, I think.

Xen 2.01 includes changes for NetBSD 2.0, but I don't know if they're working;
I'll setup a box tomorow a work to play with this. I looked at the diffs
between what's distributed with Xen, and the stock 2.0. I don't think it would
be that hard to do. But I'm not sure it's really worth it. I don't think
Xen 1.2 is maintained any more, and diverging from the sources distributed
with Xen 2.0 will make maintenance harder.

> 
> The xen2 hypervisor interface offers significant advantages and we should,
> I think, transition the code that is in the NetBSD CVS repository to work
> with it ASAP.
> 
> The only problem is that we will lose domain0 ability.  With that, we
> will lose the ability to switch back and forth between NetBSD/i386 and
> NetBSD/xen from the same root filesystem.  That will be a real pain.
> 
> Someone needs to step up and do the work to make NetBSD a block and
> network provider so it can run as domain0 with the xen2 hypervisor.
> That someone will probably not be Christian, since AFAIK he is very
> very busy; and it will certainly not be me, among other reasons because
> I would like to pass my current courses and I have an even heavier
> courseload coming up next semester.
> 
> However, I am sure that if anyone *does* step up to do the work, everyone
> who can will help out will help as much as they are able.  The work looks
> rather mechanical and time-consuming but not very hard.

I'm *very* interested in this, and have time to work on it.
After reading the Xen interface manual here is a summary of what I think needs
to be done (but I didn't look at any code yet):
- get a pseudo ethernet device in tree (maybe cube's ethfoo device, modified
  so that it can be compiled in a static kernel), similar to tun(4) but with
  ethernet semantics. This is interesting for other reasons, for example to
  build a virtual lan across an internet link, in a ssh, ipsec or other
  kind of securised transport tool. It would also be good for softwares
  such as simh: simh currently use /dev/bpf to talk to network, but this has
  some limitation, and one annoying bug (simh can't talk to his host though
  network). Using such a device instead would allow to build a vitual ethernet
  network between simh and its host, and then bridge, NAT, or route it with the
  real network device. I guess vmware could use it too.
- add support for PCI devices. The harder part will probably be that we'll
  need custom versions of bus_space and bus_dma. I may need help with this,
  as I'm not familiar with the details of virtual memory management.
- once we have PCI support, we should be able to run as domain0, but
  other guests won't have access to disk or network.
- once we can run as domain0, add support to provide block and network devices
  to other OSes.

Did I forget some steps, or missed some important details ?

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--