Re: Xen boot strangeness (Was: Re: [SOLVED] Re: Xen 4.18.5_20250521nb0 not ELF binary (Was: Re: EFI and Xen))

[Chuck Zmudzinski <frchuckz%gmail.com@localhost>]

On 5/29/2025 4:02 PM, Greg A. Woods wrote:
> At Thu, 29 May 2025 15:01:50 -0400, Chuck Zmudzinski <frchuckz%gmail.com@localhost> wrote:
> Subject: Xen boot strangeness (Was: Re: [SOLVED] Re: Xen 4.18.5_20250521nb0 not ELF binary (Was: Re: EFI and Xen))
>>
>> On 5/27/2025 7:37 PM, Greg A. Woods wrote:
>> > However it doesn't look like you're using an old-fashioned "standard"
>> > COM port.  According to the "Xen Serial Console" notes you should
>> > probably be telling Xen to use "com1", not "com2":
>> >
>> > 	Xen com1= option for non-standard serial ports (IPMI SOL, Intel AMT, PCI serial)
>> >
>> > 	Note that even if your SOL device is, for example, COM3, you
>> > 	still need to specify "com1=<foo> console=com1" options for Xen.
>> > 	If you specify "com3=" the serial console won't work!  Remember
>> > 	to list the correct (actual) serial port IOport and IRQ in the
>> > 	Xen "com1=" parameters!
>> >
>> > 	https://wiki.xenproject.org/wiki/Xen_Serial_Console
>>
>> I saw that but...
>>
>> As I explained earlier, I think Xen sees an AMT com device as com1 (NetBSD
>> sees it as com0), but I cannot access it because my BIOS does not have the
>> AMT feature. Therefore I have no choice but to use com2 or com3 which are
>> connected to the PCI serial card.
> 
> Try reading that wiki excerpt again!  (substituting /com3/com2/)
> 
> 	"If you specify 'com2=' the serial console won't work!"
> 
> You still need to tell Xen that it is using "com1" -- just give it the
> parameters for the device you actually want it to use:
> 
> 	com1=9600,8n1,0x40c0,16,1:0.0 console=com1
> 
> I don't know what Xen will do by default, but if you tell it exactly
> what to do then that's all that matters.

Ok, let's see what happens. I use this boot command line:

menu=Boot normally with Xen:dev hd2d:;load /netbsd-XEN3_DOM0.gz -c console=xencons bootdev=wd1;multiboot /xen.gz dom0_mem=2G dom0_max_vcpus=4 com1=9600,8n1,0x40c0,16,1:0.0 console=com1 cet=no-ibt pv-l1tf=false

Only difference from the one that works is that I do a s/com2/com1/g

Still, DOM0 sees the serial port that Xen is using as com2, not com0, so
with this configuration it still crashes unless I disable com* using userconf.

So it behaves exactly the same as when I told Xen the console is com2 instead of com1.

> 
> 
>> When I pass bootdev=dk12 in boot.cfg, the bootloader strangely tries dk1 as root
>> (which is wrong) and correctly detects dk11 as the dump device. But it never
>> gives me the chance to enter the correct root device and instead tries to load
>> init which of course it cannot find the NetBSD init on dk1 because dk1 is not
>> the correct NetBSD root device. In fact on this box a Linux distro is installed
>> on dk1, as evidenced by the filesystem type detected on dk1: ext2fs.
> 
> Ah, I think that's a bug related to some bizarre/old hacks to find the
> "booted_partion" for non-GPT disks:
> 
> 		if (strncmp(xcp.xcp_bootdev, devname, strlen(devname)))
> 			continue;
> 
> 		if (is_disk && strlen(xcp.xcp_bootdev) > strlen(devname)) {
> 			/* XXX check device_cfdata as in x86_autoconf.c? */
> 			booted_partition = toupper(
> 				xcp.xcp_bootdev[strlen(devname)]) - 'A';
> 			DPRINTF(("%s: booted_partition: %d\n", __func__, booted_partition));
> 		}
> 
> It looks like if the first "devname" that's tested is "dk1", then you
> get a mess when you're looking for "dk12".
> 
> The code for all this root-finding stuff is spaghetti at best!  Tons of
> old assumptions held together by magic.
> 
> Do you get similar results if you try to boot GENERIC from that same
> device?  The code in arch/x86/x86/x86_autoconf.c looks like it tries a
> bit harder, but maybe it fails too?

I boot GENERIC without Xen with this:

menu=Boot normally:dev hd2d:;rndseed /var/db/entropy-file;boot

Very simple and no searching for dk* devices needed, apparently. It
can find root at boot because I set 'dev hd2d:' which is the fourth
GPT partition on what Xen also sees as dk12.

Note I also set dev hd2d: when booting Xen, but this information does not
get passed to NetBSD dom0, so with Xen, we need to go through the search
of all the dk* devices.

> 
> The very fact there are multiple implementations of all this mess is
> maddening.
> 
> 
>> > What devices does it suggest if you type a "?" at the "root device"
>> > prompt?
>>
>> [   5.1706012] root device (default wd1a): ?
>>
>> [  44.8006002] use one of: dk0 dk1 dk2 dk3 dk4 dk5 dk6 dk7 dk8 dk9 dk10 dk11 dk12 rge0 athn0 wd0[a-p] wd1[a-p] ld0[a-p] cd0[a-p] wedge:<UUID or Label> ... ddb halt reboot
> 
> Yeah, "dk1" is there, and appears before "dk11".

Perhaps a patch will be forthcoming to improve this?

> 
>> Quoting from the Xen Howto Wiki:
>>
>> "bootdev" (or the earlier form "root") is also in general required, because
>> the boot device from /boot is not passed via Xen to the dom0 kernel.
>>
>> End of Quote from Wiki.
> 
> Ah, yes, that's where that was.  Should be in a manual page!!!
> 
> --
> 					Greg A. Woods <gwoods%acm.org@localhost>
> 
> Kelowna, BC     +1 250 762-7675           RoboHack <woods%robohack.ca@localhost>
> Planix, Inc. <woods%planix.com@localhost>     Avoncote Farms <woods%avoncote.ca@localhost>