Port-xen archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Utility of PV on non-obsolete hardware

I have a few Xen servers with NetBSD dom0, and have been running
primarily NetBSD domUs, with one production FreeBSD dom0.

I started with, and am still running, Xen 4.5.0nb1 kernel with
xentools-4.5.0nb1 on two machines, while the third has Xen 4.8.

The first FreeBSD was 11.0 I think, and it was installed from an ISO
boot under Xen.  It's now upgraded to 11.2, still on Xen 4.8.

I now have FreeBSD-12.0 installed from their new "raw" VM image and
running as shown below.

Both FreeBSD 11 and 12 boot a GENERIC kernel and detect Xen and
configure their PV drivers for HVM mode.  I.e. PV-on-HVM or PVHVM.
There is one minor annoyance -- they have the PV xencons driver in the
kernel, but I've been unable to convince the kernel to use it as the
console, or indeed to even attach it at all.  As a result if you set up
a 'console="comconsole"' then you end up with two PTYs dedicated to
every FreeBSD VM.  Hopefully full PVH mode will fix this.  In fact I'm
going to try it this week once I upgrade a machine to Xen-4.11 -- I
think FreeBSD-12.0 is ready for full PVH in GENERIC, though I've yet to
confirm this anywhere, other than what this Xen wiki says:


I have not yet done any performance studies, even trivial, but
everything I read says PVH will be the best of both/all Xen worlds.

The future is PVH domUs!

# xl.cfg(5) - startup for a PV-on-HVM FreeBSD guest
name = "freebsd"
uuid = "46118b1d-53fb-11e9-ba99-d4ae528b9511"
# n.b.:  in xentools-4.11 this will change to 'type'
builder = "hvm"
memory = 1000
maxmem = 4000
vcpus = 4
boot = "c"
vif = [ 'bridge=bridge0' ]
disk = [
# lvm lvcreate -L 40G -n lv40 vg0
# xzcat /images/FreeBSD-12.0-RELEASE-amd64.raw.xz > /dev/mapper/vg0-lv40
# boot single-user and run /etc/rc.d/growfs forcestart
        'format=raw, vdev=hda, access=rw, target=/dev/mapper/vg0-freebsd.0',
# lvm lvcreate -L 8G -n lv41 vg0
        'format=raw, vdev=hdb, access=rw, target=/dev/mapper/vg0-freebsd.1',
# or if you want to install (and change 'boot = "d"' to boot from emulated CD-ROM)
#       'format=raw, vdev=hdd, access=ro, devtype=cdrom, target=/images/FreeBSD-11.0-RELEASE-amd64-bootonly.iso'
# XXX none of this is necessary -- vnc is the default
#vnc = 1
#vnclisten = ''
#vncdisplay = 1
# xxx this could be useful...
#vncpassword = 'blah'
#usbdevice = "tablet"
# XXX to make serial console actually work, boot with vncviewer, then:
# echo 'console="comconsole"' >> /boot/loader.conf
serial = "pty"

My more detailed notes are as follows:

(notes mentioning conserver expect my "mk-conserver.sh" script is
installed with Xen to auto-configure conserver from xenstore)

Steps to a FreeBSD domU:

  - Must have the line 'builder="hvm"' in the domU config

    The FreeBSD Handbook says:  "Support for para-virtualized (PV)
    domains has been removed from FreeBSD 11 in favor of hardware
    virtualized (HVM) domains, which provides better performance."

    Which is only true because they now support "PV on HVM" in their
    GENERIC kernel, i.e. full hardware CPU virtualization (HVM) with
    (some) para-virtualized (PV) device drivers (e.g. network and disk)
    for superior I/O performance.

  - initial configuration with 'serial="pty"' can be specified, but it
    uses VNC anyway and a serial console won't work until
    /boot/loader.conf in the domU is updated as shown below (unless the
    initial filesystem image is already configured that way)

  - disks are named "hda", "hdb", "hdd", etc.

	lvm lvcreate -L 40G -n ${domu_name}.0 vg0
	xzcat FreeBSD-12.0-RELEASE-amd64.raw.xz > /dev/mapper/vg0-${domu_name}.0

	'format=raw, vdev=hda, access=rw, target=/dev/mapper/vg0-${domu_name}.0'

  - swap may not be necessary?

	lvm lvcreate -L 8G -n ${domu_name}.1 vg0

  - with the raw disk image you can start with 'boot="c"' (i.e. boot
    with an emulated hard disk.

  - boot using vncviewer, i.e.  "xl create -V ${domu_name}", and then
    login as root and run these commands:

  	echo 'console="comconsole"' >> /boot/loader.conf
  	echo 'comconsole_speed="115200"' >> /boot/loader.conf
	# possibly:  vi /etc/rc.conf # to change hostname
	echo 'growfs_enable=YES' >> /etc/rc.conf
	echo 'sshd_enable=YES' >> /etc/rc.conf
	halt -pq

    to reboot the domU, then "xl create /etc/xen/${domu_name}" in dom0
    again to enable the new console setting (and maybe run
    "/etc/rc.d/conserver reload" too to configure its console port(s))

  - WARNING:  FreeBSD PVHVM abuses QEMU for UART emulation for the
    serial console instead of using the PV Xen console support, thus
    you'll end up with two PTYs and two conserver connections dedicated
    to every FreeBSD PHHVM domU.  The one with the "-cons" suffix would
    be the Xen console one, but it will be dead and unattached on the
    FreeBSD side.  (Rumour has it that upcoming "PVH" support will
    return to using the Xen console.  [PVH requires an Intel chip with
    EPT support, and Xen >4.10] https://wiki.xen.org/wiki/FreeBSD_PVH
    and https://wiki.xenproject.org/wiki/PVH_(v2)_Domu)

  - you'll want to enable "pkg" support (run "pkg"), etc., install
    things like "avahi" (for MDNS), "rsync", etc.  Watch out for things
    using gtk3 though, such as emacs26 -- they now install wayland too!

					Greg A. Woods <gwoods%acm.org@localhost>

+1 250 762-7675                           RoboHack <woods%robohack.ca@localhost>
Planix, Inc. <woods%planix.com@localhost>     Avoncote Farms <woods%avoncote.ca@localhost>

Attachment: pgpMzg42uBnHV.pgp
Description: OpenPGP Digital Signature

Home | Main Index | Thread Index | Old Index