NetBSD-Users archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: Xen 4.18.5_20250521nb0 not ELF binary (Was: Re: EFI and Xen)
On 5/24/2025 7:38 AM, Manuel Bouyer wrote:
> On Sat, May 24, 2025 at 07:00:55AM -0400, Chuck Zmudzinski wrote:
>> On 5/24/2025 3:40 AM, Manuel Bouyer wrote:
>> > On Fri, May 23, 2025 at 10:06:05PM -0400, Chuck Zmudzinski wrote:
>> >> On 5/23/2025 5:18 PM, Chuck Zmudzinski wrote:
>> >> > On 5/23/2025 4:22 PM, Manuel Bouyer wrote:
>> >> >> On Fri, May 23, 2025 at 04:19:46PM -0400, Chuck Zmudzinski wrote:
>> >> >>> On 5/23/2025 4:15 PM, Manuel Bouyer wrote:
>> >> >>> > On Fri, May 23, 2025 at 04:13:01PM -0400, Chuck Zmudzinski wrote:
>> >> >>> >> [...]
>> >> >>> >> I needed to acquire a PCI serial card to get some useful messages because the motherboard does not have a built-in serial port.
>> >> >>> >>
>> >> >>> >> It appears Xen 4.18.4_20241221nb0 from pkgsrc crashes on my box before the NetBSD dom0 starts:
>> >> >>> >>
>> >> >>> >> -------- snip ------
>> >> >>> >> (XEN) Xen call trace:
>> >> >>> >> (XEN) [<ffff82d040261a00>] R intel_iommu_enable_eim+0/0xe0
>> >> >>> >> (XEN) [<ffff82d040268ac4>] S iommu_enable_x2apic+0x24/0x70
>> >> >>> >> (XEN) [<ffff82d0403f579d>] S x2apic_bsp_setup+0x12d/0x220
>> >> >>> >> (XEN) [<ffff82d0403ff0e4>] S __start_xen+0x1af4/0x2620
>> >> >>> >> (XEN) [<ffff82d040204434>] S __high_start+0x94/0xa0
>> >> >>> >> (XEN)
>> >> >>> >> (XEN)
>> >> >>> >> (XEN) ****************************************
>> >> >>> >> (XEN) Panic on CPU 0:
>> >> >>> >> (XEN) CONTROL-FLOW PROTECTION FAULT: #CP[0003] endbranch
>> >> >>> >> (XEN) ****************************************
>> >> >>> >> (XEN)
>> >> >>> >> (XEN) Reboot in five seconds...
>> >> >>> >> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>> >> >>> >> -------- end messages ------
>> >> >>> >
>> >> >>> > Do you have a way to disable x2apic in the BIOS ?
>> >> >>>
>> >> >>> I can check later this evening but I have an appointment now.
>> >> >>>
>> >> >>> >
>> >> >>> >>
>> >> >>> >> Please note only the pkgsrc version of Xen crashes. I can boot Xen with no problems using Xen packaged by Fedora, AlmaLinux, etc. on this box.
>> >> >>> >
>> >> >>> > Is it the same Xen version ?
>> >> >>>
>> >> >>> More or less - I know Xen 4.18.4 worked, now Fedora is at Xen 4.19.
>> >> >>
>> >> >> strange; maybe they had a patch.
>> >> >> Note that I just updated xen in pkgsrc to 4.18.5
>> >> >>
>> >> >
>> >> > Actually, I do remember that Xen 4.19.1 from Fedora did not work on my box but a patch that was added in 4.19.2 fixed it. I see the patch I needed on the Xen 4.19 branch was not added in Xen 4.18.4 but it is added in Xen 4.18.5, so maybe it will be fixed in Xen 4.18.5. I will try it when I can download Xen 4.18.5.
>> >> >
>> >> > Thanks.
>> >>
>> >> I rebuilt the latest Xen from pkgsrc but no help. Then I searched
>> >> Google for information about the error I am getting which has to
>> >> do with Control-flow Enforcement Technology (cet) and tried experimenting
>> >> with the Xen cet= options.
>> >>
>> >> Now I am not getting the crash, but just when it is ready to build the
>> >> PV Dom0, it complains that it does not have an ELF binary.
>> >>
>> >> Note the magic incantation to get past the crash is to add the option
>> >> cet=no-ibt to the Xen command line.
>> >>
>> >> With this configuration, I get:
>> >>
>> >> (XEN) Xen version 4.18.5_20250521nb0 (chuckz@) (gcc (nb3 20231008) 10.5.0) debug=n Fri May 23 18:59:50 EDT 2025
>> >> (XEN) Latest ChangeSet:
>> >> (XEN) build-id: 86afac8b52e2d124001208bb6261d17088a2f26f
>> >> (XEN) Bootloader: NetBSD/x86 EFI Boot (x64), Revision 1.2 (Mon Dec 16 13:08:11 UTC 2024) (from NetBSD 10.1)
>> >> (XEN) Command line: dom0_mem=2G com2=9600,8n1,0x40c0,16,1:0.0 console=com2 cet=no-ibt pv-l1tf=false
>> >> ... snip many messages
>> >> (XEN) First cpu on runqueue, activating
>> >> (XEN) Adding cpu 17 to runqueue 2
>> >> (XEN) Adding cpu 18 to runqueue 2
>> >> (XEN) Adding cpu 19 to runqueue 2
>> >> (XEN) mcheck_poll: Machine check polling timer started.
>> >> (XEN) NX (Execute Disable) protection active
>> >> (XEN) d0 has maximum 4160 PIRQs
>> >> (XEN) *** Building a PV Dom0 ***
>> >> (XEN) ELF: not an ELF binary
>> >> (XEN)
>> >> (XEN) ****************************************
>> >> (XEN) Panic on CPU 0:
>> >> (XEN) Could not construct domain 0
>> >> (XEN) ****************************************
>> >> (XEN)
>> >> (XEN) Reboot in five seconds...
>> >> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>> >>
>> >> I tried uncompressing (with gunzip) the netbsd-XEN3_DOM0.gz file, and I
>> >> tried omitting the dom0_mem setting, but neither change helped.
>> >
>> > This one I don't know; what boot loader command are you using ?
>> >
>>
>> menu=Boot normally with Xen:dev hd2d:;rndseed /var/db/entropy-file;load /netbsd-XEN3_DOM0 console=pc;multiboot /xen.gz dom0_mem=2G com2=9600,8n1,0x40c0,16,1:0.0 console=com2 cet=no-ibt pv-l1tf=false
>
> I don't have the rndseed entry in my settings; maybe that's the problem
>
> BTW, as you're using a serial console you probably want
> console=com0
> for NetBSD
>
This is better without rndseed and with console=com0 for dom0 (or no console setting for dom0), now it starts to boot but it hangs:
snip ...
(XEN) d0 has maximum 4160 PIRQs
(XEN) *** Building a PV Dom0 ***
(XEN) Xen kernel: 64-bit, lsb
(XEN) Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff8121c000
(XEN) Dom0 symbol map 0xffffffff8121c000 -> 0xffffffff813b5218
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000876000000->0000000878000000 (516096 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80200000->ffffffff813b5218
(XEN) Phys-Mach map: ffffffff813b6000->ffffffff817b6000
(XEN) Start info: ffffffff817b6000->ffffffff817b64b8
(XEN) Page tables: ffffffff817b7000->ffffffff817c8000
(XEN) Boot stack: ffffffff817c8000->ffffffff817c9000
(XEN) TOTAL: ffffffff80000000->ffffffff81c00000
(XEN) ENTRY ADDRESS: ffffffff80234000
(XEN) Dom0 has maximum 20 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: Errors, warnings and info
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 668kB init memory
[ 1.0000000] cpu_rng: rdrand/rdseed
[ 1.0000000] entropy: ready
[ 1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
[ 1.0000000] 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
[ 1.0000000] 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
[ 1.0000000] 2024
[ 1.0000000] The NetBSD Foundation, Inc. All rights reserved.
[ 1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[ 1.0000000] The Regents of the University of California. All rights reserved.
[ 1.0000000] NetBSD 10.1 (XEN3_DOM0) #0: Mon Dec 16 13:08:11 UTC 2024
[ 1.0000000] mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/xen/compile/XEN3_DOM0
[ 1.0000000] total memory = 2048 MB
[ 1.0000000] avail memory = 1957 MB
[ 1.0000000] mainbus0 (root)
[ 1.0000000] ACPI: RSDP 0x0000000064C2C014 000024 (v02 ALASKA)
[ 1.0000000] ACPI: XSDT 0x0000000064C2B728 0000EC (v01 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: FACP 0x0000000064BA1000 000114 (v06 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: DSDT 0x0000000064B18000 088B1F (v02 ALASKA A M I 00000001 INTL 20200717)
[ 1.0000000] ACPI: FACS 0x0000000064CCF000 000040
[ 1.0000000] ACPI: FIDT 0x0000000064B17000 00009C (v01 ALASKA A M I 00000001 AMI 00010013)
[ 1.0000000] ACPI: SSDT 0x0000000064BA3000 00038C (v02 PmaxDv Pmax_Dev 00000001 INTL 20200717)
[ 1.0000000] ACPI: SSDT 0x0000000064B11000 005D34 (v02 CpuRef CpuSsdt 00003000 INTL 20200717)
[ 1.0000000] ACPI: AAFT 0x0000000064BA2000 000306 (v01 ALASKA OEMAAFT 00000001 MSFT 00000097)
[ 1.0000000] ACPI: SSDT 0x0000000064B0E000 002A81 (v02 SaSsdt SaSsdt 00003000 INTL 20200717)
[ 1.0000000] ACPI: SSDT 0x0000000064B0A000 00334F (v02 INTEL IgfxSsdt 00003000 INTL 20200717)
[ 1.0000000] ACPI: HPET 0x0000000064B09000 000038 (v01 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: APIC 0x0000000064B08000 0001DC (v05 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: MCFG 0x0000000064B07000 00003C (v01 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: NHLT 0x0000000064B06000 00002D (v00 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: LPIT 0x0000000064B05000 0000CC (v01 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: SSDT 0x0000000064AFA000 0092F4 (v02 ALASKA TbtTypeC 00000000 INTL 20200717)
[ 1.0000000] ACPI: DBGP 0x0000000064AF9000 000034 (v01 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: DBG2 0x0000000064AF8000 000054 (v00 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: SSDT 0x0000000064AF6000 00190A (v02 ALASKA UsbCTabl 00001000 INTL 20200717)
[ 1.0000000] ACPI: RMAD 0x0000000064AF5000 000088 (v01 INTEL EDK2 00000002 01000013)
[ 1.0000000] ACPI: FPDT 0x0000000064AF4000 000044 (v01 ALASKA A M I 01072009 AMI 01000013)
[ 1.0000000] ACPI: SSDT 0x0000000064AF2000 001480 (v02 INTEL xh_rps14 00000000 INTL 20200717)
[ 1.0000000] ACPI: SSDT 0x0000000064AEE000 003AEA (v02 SocGpe SocGpe 00003000 INTL 20200717)
[ 1.0000000] ACPI: SSDT 0x0000000064AEA000 0039DA (v02 SocCmn SocCmn 00003000 INTL 20200717)
[ 1.0000000] ACPI: TPM2 0x0000000064AE8000 00004C (v04 ALASKA A M I 00000001 AMI 00000000)
[ 1.0000000] Firmware Warning (ACPI): Incorrect checksum in table [BGRT] - 0x6E, should be 0xFFFFFFEA (20221020/utcksum-100)
[ 1.0000000] ACPI: BGRT 0x0000000064AE9000 000038 (v01 ALASKA A M I 00000001 AMI 00010013)
[ 1.0000000] ACPI: PHAT 0x0000000064AE7000 00083C (v01 ALASKA A M I 00000005 MSFT 0100000D)
[ 1.0000000] ACPI: WSMT 0x0000000064B04000 000028 (v01 ALASKA A M I 00000001 AMI 00010013)
[ 1.0000000] ACPI: 10 ACPI AML tables successfully acquired and loaded
[ 1.0000000] ioapic0 at mainbus0 apid 2
(XEN) allocated vector b8 for irq 17
(XEN) allocated vector c0 for irq 18
(XEN) allocated vector c8 for irq 19
(XEN) allocated vector d0 for irq 20
(XEN) allocated vector d8 for irq 21
(XEN) allocated vector e0 for irq 22
(XEN) allocated vector e8 for irq 23
(XEN) allocated vector 31 for irq 24
(XEN) allocated vector 39 for irq 25
(XEN) allocated vector 41 for irq 26
(XEN) allocated vector 49 for irq 27
(XEN) allocated vector 51 for irq 28
(XEN) allocated vector 59 for irq 29
(XEN) allocated vector 61 for irq 30
(XEN) allocated vector 69 for irq 31
(XEN) allocated vector 71 for irq 32
(XEN) allocated vector 79 for irq 33
(XEN) allocated vector 81 for irq 34
(XEN) allocated vector 89 for irq 35
(XEN) allocated vector 91 for irq 36
(XEN) allocated vector 99 for irq 37
(XEN) allocated vector a1 for irq 38
(XEN) allocated vector a9 for irq 39
(XEN) allocated vector b1 for irq 40
(XEN) allocated vector b9 for irq 41
(XEN) allocated vector c1 for irq 42
(XEN) allocated vector c9 for irq 43
(XEN) allocated vector d1 for irq 44
(XEN) allocated vector d9 for irq 45
(XEN) allocated vector e1 for irq 46
(XEN) allocated vector e9 for irq 47
(XEN) allocated vector 32 for irq 48
(XEN) allocated vector 3a for irq 49
(XEN) allocated vector 42 for irq 50
(XEN) allocated vector 4a for irq 51
(XEN) allocated vector 52 for irq 52
(XEN) allocated vector 5a for irq 53
(XEN) allocated vector 62 for irq 54
(XEN) allocated vector 6a for irq 55
(XEN) allocated vector 72 for irq 56
(XEN) allocated vector 7a for irq 57
(XEN) allocated vector 82 for irq 58
(XEN) allocated vector 8a for irq 59
(XEN) allocated vector 92 for irq 60
(XEN) allocated vector 9a for irq 61
(XEN) allocated vector a2 for irq 62
(XEN) allocated vector aa for irq 63
(XEN) allocated vector b2 for irq 64
(XEN) allocated vector ba for irq 65
(XEN) allocated vector c2 for irq 66
(XEN) allocated vector ca for irq 67
(XEN) allocated vector d2 for irq 68
(XEN) allocated vector da for irq 69
(XEN) allocated vector e2 for irq 70
(XEN) allocated vector ea for irq 71
(XEN) allocated vector 33 for irq 72
(XEN) allocated vector 3b for irq 73
(XEN) allocated vector 43 for irq 74
(XEN) allocated vector 4b for irq 75
(XEN) allocated vector 53 for irq 76
(XEN) allocated vector 5b for irq 77
(XEN) allocated vector 63 for irq 78
(XEN) allocated vector 6b for irq 79
(XEN) allocated vector 73 for irq 80
(XEN) allocated vector 7b for irq 81
(XEN) allocated vector 83 for irq 82
(XEN) allocated vector 8b for irq 83
(XEN) allocated vector 93 for irq 84
(XEN) allocated vector 9b for irq 85
(XEN) allocated vector a3 for irq 86
(XEN) allocated vector ab for irq 87
(XEN) allocated vector b3 for irq 88
(XEN) allocated vector bb for irq 89
(XEN) allocated vector c3 for irq 90
(XEN) allocated vector cb for irq 91
(XEN) allocated vector d3 for irq 92
(XEN) allocated vector db for irq 93
(XEN) allocated vector e3 for irq 94
(XEN) allocated vector eb for irq 95
(XEN) allocated vector 34 for irq 96
(XEN) allocated vector 3c for irq 97
(XEN) allocated vector 44 for irq 98
(XEN) allocated vector 4c for irq 99
(XEN) allocated vector 54 for irq 100
(XEN) allocated vector 5c for irq 101
(XEN) allocated vector 64 for irq 102
(XEN) allocated vector 6c for irq 103
(XEN) allocated vector 74 for irq 104
(XEN) allocated vector 7c for irq 105
(XEN) allocated vector 84 for irq 106
(XEN) allocated vector 8c for irq 107
(XEN) allocated vector 94 for irq 108
(XEN) allocated vector 9c for irq 109
(XEN) allocated vector a4 for irq 110
(XEN) allocated vector ac for irq 111
(XEN) allocated vector b4 for irq 112
(XEN) allocated vector bc for irq 113
(XEN) allocated vector c4 for irq 114
(XEN) allocated vector cc for irq 115
(XEN) allocated vector d4 for irq 116
(XEN) allocated vector dc for irq 117
(XEN) allocated vector e4 for irq 118
(XEN) allocated vector ec for irq 119
[ 1.0000000] cpu0 at mainbus0 apid 0
[ 1.0000000] cpu1 at mainbus0 apid 1
[ 1.0000000] cpu2 at mainbus0 apid 8
[ 1.0000000] cpu3 at mainbus0 apid 9
[ 1.0000000] cpu4 at mainbus0 apid 16
[ 1.0000000] cpu5 at mainbus0 apid 17
[ 1.0000000] cpu6 at mainbus0 apid 24
[ 1.0000000] cpu7 at mainbus0 apid 25
[ 1.0000000] cpu8 at mainbus0 apid 32
[ 1.0000000] cpu9 at mainbus0 apid 33
[ 1.0000000] cpu10 at mainbus0 apid 40
[ 1.0000000] cpu11 at mainbus0 apid 41
[ 1.0000000] cpu12 at mainbus0 apid 48
[ 1.0000000] cpu13 at mainbus0 apid 50
[ 1.0000000] cpu14 at mainbus0 apid 52
[ 1.0000000] cpu15 at mainbus0 apid 54
[ 1.0000000] cpu16 at mainbus0 apid 56
[ 1.0000000] cpu17 at mainbus0 apid 58
[ 1.0000000] cpu18 at mainbus0 apid 60
[ 1.0000000] cpu19 at mainbus0 apid 62
[ 1.0000000] hypervisor0 at mainbus0: Xen version 4.18.5_20250521nb0
[ 1.0000000] vcpu0 at hypervisor0
[ 1.0000000] vcpu0: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu0: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu1 at hypervisor0
[ 1.0000000] vcpu1: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu1: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu2 at hypervisor0
[ 1.0000000] vcpu2: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu2: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu3 at hypervisor0
[ 1.0000000] vcpu3: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu3: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu4 at hypervisor0
[ 1.0000000] vcpu4: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu4: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu5 at hypervisor0
[ 1.0000000] vcpu5: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu5: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu6 at hypervisor0
[ 1.0000000] vcpu6: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu6: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu7 at hypervisor0
[ 1.0000000] vcpu7: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu7: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu8 at hypervisor0
[ 1.0000000] vcpu8: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu8: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu9 at hypervisor0
[ 1.0000000] vcpu9: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu9: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu10 at hypervisor0
[ 1.0000000] vcpu10: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu10: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu11 at hypervisor0
[ 1.0000000] vcpu11: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu11: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu12 at hypervisor0
[ 1.0000000] vcpu12: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu12: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu13 at hypervisor0
[ 1.0000000] vcpu13: Intel(R) Core(TM) i5-14500, id 0xb06f2
[ 1.0000000] vcpu13: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu14 at hypervisor0
[ 1.0000000] vcpu14: Intel(R) Core(TM) i
N.B. My cpu has 14 cores and 20 threads because there are 6 cores that have hyperthreading - so it hangs where the hyperthreading starts. Last thing to try is with smt=false...
Home |
Main Index |
Thread Index |
Old Index