Re: Problems booting NetBSD/Xen on Intel NUC 12 Extreme w/ i9-12900

[Johan Stenstam <johani%johani.org@localhost>]

Hi agin,

> On 11 May 2023, at 08:54, Manuel Bouyer <bouyer%antioche.eu.org@localhost> wrote:
> 
> On Wed, May 10, 2023 at 10:20:49PM +0200, Johan Stenstam wrote:
>> 
>>> On Wed, May 10, 2023 at 03:04:37PM -0400, Greg Troxel wrote:
>>> 
>>> AFAIK NetBSD/xen still need separate modules (and I don't think it's going
>>> to change). But as I don't use modules I don't know how they're supposed
>>> to be built.
>> 
>> Well, I have them (copied from /stand/amd64/ to /stand/amd64-xen/), so either way this should not be the issue.
> 
> It is, because modules from /stand/amd64/ and not compatible with
> /stand/amd64-xen/ (the ABI is not the same)

Yes, I’ve found that out now. However, note that one of the last messages from the successful boot of XEN3_DOM0 using the serial console was:

kern.module.path=/stand/amd64/10.0/modules

So it seems that there is some confusion in the XEN3_DOM0 kernel about the correct path for locating modules.

>> I have no idea what has caused NetBSD/Xen to think that the boot device is dk0, but that?s clearly where everything goes south.
>> 
>> Any suggestions?
> 
> try
> root=dk1 
> as argument to NetBSD ?

Sure, I did that last night, but the results are… a bit confusing. Using your nice config to get a the serial console displayed via Xen + the root=dk1 argument I eventually get the box to boot XEN3_DOM0 via the following line in boot.cfg (this is from memory, but I’m sure it’s accurate):

menu=FOO:load /netbsd.XEN3_DOM0.10_BETA root=dk1 console=com0;multiboot /xen415.gz dom0_mem=1024M vga=current,keep

It takes about 15 minutes, due to the extremely slow console, but the result is a booted system:

ssh -A root@172.16.1.214
Last login: Thu May 11 05:29:43 2023 from 172.16.1.25
NetBSD 10.0_BETA (XEN3_DOM0) #0: Thu May  4 18:57:28 UTC 2023
bash#

So that’s obviously great. 

However, if I keep everything the same except that I change the console config:

menu=BAR:load /netbsd.XEN3_DOM0.10_BETA root=dk1 console=pc;multiboot /xen415.gz dom0_mem=1024M

…the result is that when Xen hands over the console to the DOM0 the screen goes dark (that’s expected) and the box never comes up (in the sense of responding to ping and being able to log in over ssh.

So my interpretation of this is that there seem to be multiple issues at hand:

1. There is some sort of problem with handing over the console from Xen to the DOM0.

2. There is some other sort of problem with the DOM0 not booting successfully due to problems with the console.

It is possible that these two things are actually the same, eg. the handover problem cause the DOM0 to crash, but I see no easy way to tell whether the DOM0 crashes immediately or if it becomes unhappy later on. 

What would have been nice would be an alternative like “console=none”, as in “I know there is no working console, just send the output to the log so that I can examine it later”, but I cannot find one.

Things I will try this evening when I get back to where the box is:

1.Try to speed up the serial console output a la: “...console=com0,115200” to get away from the ultra sluggishness of the current serial console, which is likely 9600 baud.

2. Right now Xen puts the console in some sort of high resolution mode (I have at least 100 rows of very tiny characters) while NetBSD/GENERIC runs the console as a more normal lower resolution text terminal. I’ll see if it is possible to get Xen to use the same to make the handover easier. 

Regards,
Johan

PS. Separate from the console and booting problems it would be truly great if someone (Greg?) has a binary pkg for xentools415 that they would be willing to share with me.