qemu tricks and tips for working with VM?

[Jeff Rizzo <riz%tastylime.net@localhost>]

This is a continuation of a previous thread ("Boot VM from a zvol using 
qemu and nvmm?"), but as I've actually solved the basic problem in that 
thread - ie, the kernel in the image is successfully booting - I figured 
I'd start a new thread.


The gist is this: I have a virtual machine (running NetBSD 10, now) 
running on a FreeBSD hypervisor that I'd like to migrate to a NetBSD one 
- and as the current image is backed by a zvol, the easiest way for me 
to test the full migration is by copying that zvol to the netbsd host, 
and booting it there.  Given that the virtualization is different 
between the hosts (bhyve vs. qemu+nvmm), there's some trickiness 
involved.  From what I can tell, on the FreeBSD system, the VM boots 
using uefi;  on the NetBSD system, I had to do some additional 
installboot(8) stuff to the ffs partition to get things as far as they are.

*Where* they are is that it's reading a NetBSD boot block from disk, and 
booting the kernel that's on there.  However, the kernel is not finding 
init, which is a problem.  :)  There's also some other weirdness, so 
I'll take things in order.


I'm using -nographic under qemu, but I don't seem to be able to interact 
with the bootloader - it counts down, but no keypresses register.  This 
is what the boot looks like:

NetBSD/x86 ffsv2 Primary Bootstrap
:-(0

>> NetBSD/x86 BIOS Boot, Revision 5.11 (Mon Jul 20 18:35:18 UTC 2020) 
(from NetBSD 9.0_STABLE)
>> Memory: 639/1047404 k

     1. Boot normally
     2. Boot single user
     3. Drop to boot prompt

Choose an option; RETURN for default; SPACE to stop countdown.
Option 1 will be chosen in 10 seconds.
0 seconds.
command(s): rndseed /var/db/entropy-file;boot
default boot twice, skipping...
21288680+782888+1314264 [939258+1396248+1023001]=0x1b34a30
WARNING: couldn't open /var/db/entropy-file
WARNING: 1 module failed to load

... it's clearly loading from disk.  It also appears to be reading 
/boot.cfg, as that's where I set the 10 second timeout. As I said, I 
can't stop the countdown with the space bar, so it proceeds to boot the 
NetBSD-10 GENERIC kernel.  (the keen-eyed will notice the bootblock was 
from NetBSD 9.0_STABLE - I just upgraded the VM to 10 "just in case" 
recently, without apparent effect)

I won't paste the entire boot message here (unless asked), but here's 
the interesting bit:

disks are found:

[   1.6300183] dk0 at ld0: "b03a8e82-74d8-4c24-be1d-8566c46b6770", 
262144 blocks at 64, type: msdos
[   1.6300183] dk1 at ld0: "74b6c1b2-fd39-4226-aa20-94f2824c9c48", 
20185021 blocks at 262272, type: ffs
[   1.6300183] dk2 at ld0: "acfebf87-5861-4fc7-8a95-e4204ed63182", 
524191 blocks at 20447296, type: swap


but it can't open /dev/console (or /sbin/init):

[   7.5252308] boot device: ld0
[   7.5252308] root on dk1 dumps on dk2
[   7.5252308] root file system type: ffs
[   7.5353943] kern.module.path=/stand/amd64/10.0/modules
[   7.5384322] warning: lookup /dev/console: error 22
[   7.5384322] exec /sbin/init: error 22


...errno 22 is EINVAL, but I'm not familiar enough to know what I'm 
hitting, here.  Or why the console seems one-way.  Thoughts?

+j

--
Jeff Rizzo
riz%tastylime.net@localhost
+1 415 823 1847