Re: How to control booting via BIOS vs booting via EFI?

[Lars-Johan Liman <liman%cafax.se@localhost>]

Manuel (others?),

Johan and I are old friends and also both NetBSD afficionados. I took a
stern look at Johan's system and actually managed to get it to boot ...
albeit by cheating. By using the EFI boot loader (which is configurable)
from Debian's Xen package, I managed to make the NetBSD Xen kernel
"survive" and multiboot a NetBSD Dom0. The curious part is that the
machine obviously boots by virtue of the EFI boot process, but the
NetBSD kernel nevertheless reports

dom0# sysctl machdep.bootmethod
machdep.bootmethod = BIOS

which puzzles me.

Also, the NetBSD kernel VGA console logging ("the green text") scrolls
really slowly (as if it was an old serial port at 9600 baud?), but it
works. Apart from that, the machine seems happy.

I append a MarkDown file with my process for getting it to work - for
you or anyone else who would like to try. I leve no guarantees of
success, and, as always, "your computer and its disk contents is your
responsibility, not mine". Don't blame me if it doesn't work. It worked
for me.

Ideally, using a Debian boot loader wouldn't be necessary. ;-)

				Cheers,
				  /Liman

#-------------------------------------------------------------------------
# Lars-Johan Liman, M.Sc.		 ! E-mail: liman%cafax.se@localhost
# Cafax AB				 ! HTTP  : //www.cafax.se/
# Computer Consultants, Sweden		 ! Voice : +46 8 - 564 702 30
#-------------------------------------------------------------------------


johani%johani.org@localhost 2025-10-14 20:47 [+0200]:
> Hi Manuel,

>  On 14 Oct 2025, at 09:27, Manuel Bouyer <bouyer%antioche.eu.org@localhost> wrote:

>  On 12 Oct 2025, at 18:36, Martin Husemann <martin%duskware.de@localhost> wrote:

>  I would be suprised if NetBSD/xen could not boot via UEFI but actually all
>  my xen machines still use BIOS boot (mostly for historical reasons).

>  We have multiboot2 support, so technically it is unclear to me why xen
>  would fail.

>  While I would very much like that to be true, I simply do not understand enough of the
>  boot magic to have an opinion other that ?it seems not to work?.

>  I have a Xen system booting via EFI (this machine doesn't support BIOS boot):
>  menu=Boot Xen PV:load /netbsd console=pc root=dk1; multiboot /xen-debug.gz
>  dom0_mem=8192M console=com1,vga com1=115200,8n1 conring_size=4096k
>  sync_console

> I tried this, using Xen 4.20 and NetBSD 10.1. Unfortunately the hypervisor crashed instantly. My
> guess is that this is the same crash as before (although at that time I used Xen 4.18). However,
> while it was very fast I took a movie and I’ve attached the last useful frame from that movie. I’m
> sorry about the glare. Does that picture provide any useful clues?

> Regards,
> Johan

> IMG_2367.jpeg

# NOTE! THAT THIS COMES WITH NO GUARANTEE OF SUCCESS. THIS IS FOR TESTING ONLY. EXPECT FAILURES AND DATA LOSS.

# How To Install a NetBSD Xen Dom0 under EFI
Time-stamp: <2025-11-19 11:25:41 UTC liman>

## Install the Necessary NetBSD Xen Pieces

Install the 4.20 versions of the Xen kernel and tools.

```
dom0# pkgin install xenkernel420 xentools420
```

Make sure you have the NetBSD Dom0 kernel "at hand".

```
dom0# ls -l /netbsd-XEN3_DOM0.gz
-rw-r--r--  1 root  wheel  7030577 Nov 15 10:23 /netbsd-XEN3_DOM0.gz
dom0#
```

## Set Up The EFI Boot System 

1. Download the EFI bootloader for Debian/Linux, unpack it, and unpack
   it.

   ```
   dom0# cd /tmp
   dom0# curl -O -L 'http://ftp.se.debian.org/debian/pool/main/x/xen/xen-hypervisor-4.20-amd64_4.20.0+68-g35cb38b222-1_amd64.deb'
   dom0# mkdir debian
   dom0# cd debian
   dom0# ar x ../xen-hypervisor-4.20-amd64_4.20.0+68-g35cb38b222-1_amd64.deb
   dom0# tar Jxf data.tar.xz
   ```

1. Mount the EFI boot partition as writable. (The partition ID `dk0`
   may vary, but is likely to be dk0.)

   ```
   dom0# cd /
   dom0# mkdir -p /efi
   dom0# mount -t msdos /dev/dk0 /efi
   ```

1. Carefully move the NetBSD EFI boot loader out of place, and replace
   it with the EFI boot loader from Debian, which is renamed to
   `bootx64.efi` in the process. Using the correct target filename is
   important.

   ```
   dom0# cd /efi/efi/
   dom0# mkdir netbsd
   dom0# cd boot
   dom0# mv boot*.efi ../netbsd
   dom0# cp /tmp/debian/boot/xen-4.20-amd64.efi bootx64.efi
   dom0# chmod 0755 bootx64.efi
   ```

1. Copy the NetBSD Xen kernel and the NetBSD Dom0 kernel to the EFI
   boot directory, and uncompress them. I'm not sure uncompressing is
   essential, but it works. I haven't tried booting compressed
   kernels. Give them short filenames, since the MS-DOS file system is
   not happy with long filenames. The filenames play a role in a
   config file below.

   ```
   dom0# cp /netbsd-XEN3_DOM0.gz netbsd.gz
   dom0# gunzip netbsd.gz
   dom0# cp /usr/pkg/xen420-kernel/xen.gz .
   dom0# gunzip xen.gz
   ```

1. Configure the Debian EFI boot loader to load the NetBSD Xen kernel
   you just installed, and tell the Xen kernel to boot the NetBSD Dom0
   kernel. The filename of the config file is important.
   
   ```
   dom0# ${EDITOR} bootx64.cfg
   ```

   Remove any spurious backup files the editor might have left behind
   (e.g., `~` files generated by `emacs` or `mg`).

1. The content of the `bootx64.cfg` file should look as follows
   (except for intended modifications). **NOTE!!** that it's necessary
   to specify to the NetBSD kernel which partiton (wedge) to use as
   the file system root for the running system (`root=dk1`) and onto
   which partition (wedge) to put crash dump files, should the need
   arise (`dump=dk2` = normally the swap partition). **You may need to
   adjust these! Truly bad things may happen if they are wrong!**

   ```
   [global]
   default=xen

   [xen]
   # CAREFULLY CHECK THE EXACT WEGES FOR ROOT AND DUMP (dk1, dk2 below)!
   # The console specifications need to be exact.
   # Maybe add more options? dom0_max_vcpus=1 dom0_vcpus_pin
   options=console=vga loglvl=all noreboot dom0_mem=2048M
   kernel=netbsd console=pc root=dk1 dump=dk2
   ```

## Set Up The Dom0 Xen Environment

Don't forget to enable Xen-tools. If `xencommons` is not enabled, you
won't be able to shutdown or reboot the machine. Enable the packages
by putting the following in `/etc/rc.conf`. Enabling `xendomains` is
optional.

```
xenwatchdog=YES
xencommons=YES
# xendomains_config="/data/xen/config/%s"
# xendomains="domU"
```

## Reboot

Your machine should now be ready to boot. May the force be with you.

```
dom0# sync
dom0# shutdown -r now
```

## Backing Out

If your machine fails to boot, use a USB stick or equivalent to boot
the NetBSD install media. Choose "Utilities" from the installation
menu, and start a /bin/sh shell. From there you can mount the EFI
partition, and swap the newly installed Linux EFI boot loader with the
NetBSD one you put away above, and try to reboot from the harddrive.
There is reasonable hope that you'll get a working "plain" ( = no Xen)
NetBSD back.

## Upgrading

**Note!** that the Xen and NetBSD kernels being booted are the ones
located in the `/efi/efi/boot` directory on the EFI partition
(normally `dk0`) and not the ones in the NetBSD `/` directory. When
upgrading NetBSD it is important to repeat the step above where the
kernels are copied to `/efi/efi/boot` and uncompressed.

## Last Comments

I've noted that, for some reason, the NetBSD console log from the
kernel ("the green text") scrolls really slow. I have no idea why. The
machine seems to do its job, though.

Another note is that the machine still reports

```
dom0# sysctl machdep.bootmethod
machdep.bootmethod = BIOS
```

I have no idea about that either ...

– END –