I had a Xen server reboot (not sure why -- no console log on it and it
is remote), and one of the domUs locked up on boot with the last message
being "xenbus0: holding up boot". It was stuck for quite a while as you
can see before I was able to access the console. I sent a "break" to
see what was going on:
[ 1.0000000] pool redzone disabled for 'pvpage'
[ 1.0000000] pool redzone disabled for 'kmem-02048'
[ 1.0000000] pool redzone disabled for 'kmem-04096'
[ 1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[ 1.0000000] 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[ 1.0000000] 2018, 2019, 2020, 2021 The NetBSD Foundation, Inc. All rights reserved.
[ 1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[ 1.0000000] The Regents of the University of California. All rights reserved.
[ 1.0000000] NetBSD 9.99.81 (XEN3_DOMU) #0: Mon Jan 1 16:53:41 PST 2024
[ 1.0000000] woods@b2:/build/woods/b2/current-amd64-amd64-obj/work/woods/m-NetBSD-current/sys/arch/amd64/compile/XEN3_DOMU
[ 1.0000000] total memory = 4000 MB
[ 1.0000000] avail memory = 3855 MB
[ 1.0000000] No module info at boot
[ 1.0000000] pool redzone disabled for 'buf32k'
[ 1.0000000] pool redzone disabled for 'buf64k'
[ 1.0000000] entropy: seed attached as an entropy source (collecting)
[ 1.0000000] entropy: no seed from bootloader
[ 1.0000000] pool redzone disabled for 'sigacts'
[ 1.0000000] pool redzone disabled for 'mclpl'
[ 1.0000000] entropy: autoconf attached as an entropy source (collecting without estimation)
[ 1.0000000] timecounter: Timecounters tick every 1.000 msec
[ 1.0000000] entropy: system-power attached as an entropy source (collecting)
[ 1.0000000] No bootinfo commands at boot
[ 1.0000000] mainbus0 (root)
[ 1.0000000] hypervisor0 at mainbus0: Xen version 4.18.0_20231116nb0
[ 1.0000000] hypervisor0: Xen Domain-0: xentral
[ 1.0000000] hypervisor0: features: mmu_pt_update_preserve_ad highmem_assist gnttab_map_avail_bits
[ 1.0000000] VIRQ_DEBUG interrupt using event channel 3
[ 1.0000000] vcpu0 at hypervisor0
[ 1.0000000] vcpu0: 8 page colors
[ 1.0000000] event_set_handler: id:<NULL>, xname:vcpu0 ipi, handler:(*0xffffffff80202281)(), evtch 4, level 8
[ 1.0000000] event_set_handler: id:xenev0 chan 4, xname:vcpu0 ipi, attaching evtch 4 to VCPU vcpu0 0
[ 1.0000000] vcpu0: Intel(R) Xeon(R) CPU E5645 @ 2.40GHz, id 0x206c2
[ 1.0000000] vcpu0: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu1 at hypervisor0
[ 1.0000000] vcpu1: 2 page colors
[ 1.0000000] vcpu1: starting
[ 1.0000000] vcpu1: is started.
[ 1.0000000] vcpu1: Intel(R) Xeon(R) CPU E5645 @ 2.40GHz, id 0x206c2
[ 1.0000000] vcpu1: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu2 at hypervisor0
[ 1.0000000] vcpu2: 2 page colors
[ 1.0000000] vcpu2: starting
[ 1.0000000] vcpu2: is started.
[ 1.0000000] vcpu2: Intel(R) Xeon(R) CPU E5645 @ 2.40GHz, id 0x206c2
[ 1.0000000] vcpu2: node 0, package 0, core 0, smt 0
[ 1.0000000] vcpu3 at hypervisor0
[ 1.0000000] vcpu3: 2 page colors
[ 1.0000000] vcpu3: starting
[ 1.0000000] vcpu3: is started.
[ 1.0000000] vcpu3: Intel(R) Xeon(R) CPU E5645 @ 2.40GHz, id 0x206c2
[ 1.0000000] vcpu3: node 0, package 0, core 0, smt 0
[ 1.0000000] xenbus0 at hypervisor0: Xen Virtual Bus Interface
[ 1.0000000] xencons0 at hypervisor0: Xen Virtual Console Driver
[ 1.0000000] xencons0: console major 143, unit 0
[ 1.0000000] event_set_handler: id:xenev0 chan 2, xname:xencons0, handler:(*0xffffffff8020f69c)(), evtch 2, level 6
[ 1.0000000] event_set_handler: id:xenev0 chan 2, xname:xencons0, attaching and binding evtch 2 to current VCPU vcpu0 0
[ 1.0000000] xencons0: using event channel 2
[ 1.0000000] entropy: WARNING: extracting entropy too early
[ 1.0000000] Initializing SSP: 87c6ce013bdaa738 998f5961c4d3fa8e b9dacf067d416d0b 6f6ebd447e20b17 93a30bc01334d323 a813ac8edfabfbf8 164a33b490b67f8d c515cad3ae8e714d
[ 1.0000000] timecounter: Timecounter "xen_system_time" frequency 1000000000 Hz quality 10000
[ 1.0000030] event_set_handler: id:<NULL>, xname:vcpu0 clock, handler:(*0xffffffff80209b24)(), evtch 5, level 7
[ 1.0000030] event_set_handler: id:xenev0 chan 5, xname:vcpu0 clock, attaching evtch 5 to VCPU vcpu0 0
[ 1.0000030] Xen vcpu0 clock: using event channel 5
[ 1.0000030] timecounter: Timecounter "clockinterrupt" frequency 1000 Hz quality 0
[ 1.0000030] entropy: hardclock attached as an entropy source (collecting)
[ 1.0000030] entropy: cpu0 attached as an entropy source (off)
[ 1.0000030] entropy: cpu1 attached as an entropy source (off)
[ 1.0000030] entropy: cpu2 attached as an entropy source (off)
[ 1.0000030] entropy: cpu3 attached as an entropy source (off)
[ 1.1021072] UVM: using package allocation scheme, 1 package(s) per bucket
[ 1.1021072] event_set_handler: id:<NULL>, xname:vcpu1 ipi, handler:(*0xffffffff80202281)(), evtch 6, level 8
[ 1.1021072] event_set_handler: id:xenev0 chan 6, xname:vcpu1 ipi, attaching evtch 6 to VCPU vcpu1 1
[ 1.1021072] event_set_handler: id:<NULL>, xname:vcpu1 clock, handler:(*0xffffffff80209b24)(), evtch 7, level 7
[ 1.1021072] event_set_handler: id:xenev0 chan 7, xname:vcpu1 clock, attaching evtch 7 to VCPU vcpu1 1
[ 1.1021072] Xen vcpu1 clock: using event channel 7
[ 1.1021072] event_set_handler: id:<NULL>, xname:vcpu2 ipi, handler:(*0xffffffff80202281)(), evtch 8, level 8
[ 1.1021072] event_set_handler: id:xenev0 chan 8, xname:vcpu2 ipi, attaching evtch 8 to VCPU vcpu2 2
[ 1.1021072] event_set_handler: id:<NULL>, xname:vcpu2 clock, handler:(*0xffffffff80209b24)(), evtch 9, level 7
[ 1.1021072] event_set_handler: id:xenev0 chan 9, xname:vcpu2 clock, attaching evtch 9 to VCPU vcpu2 2
[ 1.1021072] Xen vcpu2 clock: using event channel 9
[ 1.1021072] event_set_handler: id:<NULL>, xname:vcpu3 ipi, handler:(*0xffffffff80202281)(), evtch 10, level 8
[ 1.1021072] event_set_handler: id:xenev0 chan 10, xname:vcpu3 ipi, attaching evtch 10 to VCPU vcpu3 3
[ 1.1021072] event_set_handler: id:<NULL>, xname:vcpu3 clock, handler:(*0xffffffff80209b24)(), evtch 11, level 7
[ 1.1021072] event_set_handler: id:xenev0 chan 11, xname:vcpu3 clock, attaching evtch 11 to VCPU vcpu3 3
[ 1.1021072] Xen vcpu3 clock: using event channel 11
[ 1.9991062] vcpu1: running
[ 1.9991062] vcpu3: running
[ 1.9991062] vcpu2: running
[ 2.0001074] event_set_handler: id:xenev0 chan 1, xname:xenbus0, handler:(*0xffffffff8020bd72)(), evtch 1, level 6
[ 2.0001074] event_set_handler: id:xenev0 chan 1, xname:xenbus0, attaching and binding evtch 1 to current VCPU vcpu0 0
[ 2.0001074] xenbus0: using event channel 1
[ 2.0091137] aes: Intel AES-NI
[ 2.0091137] chacha: x86 SSE2 ChaCha
[ 2.0091137] adiantum: self-test passed
[ 2.0091137] aes_ccm: self-test passed
[ 2.0101131] blake2s: self-test passed
[ 2.0101131] xenbus0: holding up boot
[ 51815.5491248] fatal breakpoint trap in supervisor mode
[ 51815.5491248] trap type 1 code 0 rip 0xffffffff8022d93d cs 0xe030 rflags 0x202 cr2 0 ilevel 0x6 rsp 0xffffc980ce4cdc08
[ 51815.5491248] curlwp 0xffffc98007901040 pid 0.2 lowest kstack 0xffffc980ce4c92c0
Stopped in pid 0.2 (system) at netbsd:breakpoint+0x5: leave
breakpoint() at netbsd:breakpoint+0x5
xencons_tty_input.isra.0() at netbsd:xencons_tty_input.isra.0+0xb0
xencons_handler() at netbsd:xencons_handler+0x69
xen_intr_biglock_wrapper() at netbsd:xen_intr_biglock_wrapper+0x1b
evtchn_do_event() at netbsd:evtchn_do_event+0x10d
do_hypervisor_callback() at netbsd:do_hypervisor_callback+0x186
Xhandle_hypervisor_callback() at netbsd:Xhandle_hypervisor_callback+0x19
--- interrupt ---
hypercall_page() at netbsd:hypercall_page+0x3aa
idle_loop() at netbsd:idle_loop+0x11f
ds 0
es 3e20
fs 9ca7
gs dbd8
rdi 2b
rsi ffffffff810bd017
rbp ffffc980ce4cdc08
rbx ffffffff810bd017
rdx 2b
rcx 15
rax 1
r8 ffffc98007b20e80
r9 63932c08
r10 ff
r11 246
r12 ffffc980078f66a8
r13 ffffffff810bd018
r14 ffffc98007b3e100
r15 400
rip ffffffff8022d93d breakpoint+0x5
cs e030
rflags 202
rsp ffffc980ce4cdc08
ss e02b
netbsd:breakpoint+0x5: leave
db{0}>
At that point I destroyed the domU and restarted it and this time it
rebooted A-OK.
--
Greg A. Woods <gwoods%acm.org@localhost>
Kelowna, BC +1 250 762-7675 RoboHack <woods%robohack.ca@localhost>
Planix, Inc. <woods%planix.com@localhost> Avoncote Farms <woods%avoncote.ca@localhost>
Attachment:
pgp9vDVp8Bldt.pgp
Description: OpenPGP Digital Signature