kern/45913: some Xen DomUs hang during boot since kmem changes

[riz%NetBSD.org@localhost]

>Number:         45913
>Category:       kern
>Synopsis:       some Xen DomUs hang during boot since kmem changes
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Feb 02 19:05:00 +0000 2012
>Originator:     Jeff Rizzo
>Release:        NetBSD 5.99.63
>Organization:
        
>Environment:
System: NetBSD breadfruit.tastylime.net 5.99.63 NetBSD 5.99.63 (XEN3_DOMU) #1: 
Wed Feb 1 11:58:08 PST 2012 
riz%breadfruit.tastylime.net@localhost:/home/riz/obj/sys/arch/amd64/compile/XEN3_DOMU
 amd64
Architecture: x86_64
Machine: amd64
>Description:

The "changes" referred to here are this commit:

http://mail-index.netbsd.org/source-changes/2012/01/27/msg031054.html

        Since the the changes to kmem, Xen DOMUs on one (but not the other)
of my Dom0s hangs during mounting root:

Using config file "/usr/pkg/etc/xen/xendev".
Started domain xendev (id=1)
                            Loaded initial symtab at 0xffffffff807fa204, strtab 
at 0xffffffff8084fdf0, # entries 14532
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 5.99.63 (XEN3_DOMU) #1: Wed Feb  1 11:58:08 PST 2012
        
riz%breadfruit.tastylime.net@localhost:/home/riz/obj/sys/arch/amd64/compile/XEN3_DOMU
total memory = 512 MB
avail memory = 486 MB
cprng kernel: WARNING insufficient entropy at creation.
mainbus0 (root)
hypervisor0 at mainbus0: Xen version 4.1
vcpu0 at hypervisor0: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz, id 0x206a7
vcpu1 at hypervisor0pmap_kenter_pa: mapping already present
vcpu1: feature mismatch: cpu_feature[0] is 0xbeebcbf7, but CPU reported 
0xbfebfbff
: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz, id 0x206a7
vcpu2 at hypervisor0pmap_kenter_pa: mapping already present
vcpu2: feature mismatch: cpu_feature[0] is 0xbeebcbf7, but CPU reported 
0xbfebfbff
: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz, id 0x206a7
vcpu3 at hypervisor0pmap_kenter_pa: mapping already present
vcpu3: feature mismatch: cpu_feature[0] is 0xbeebcbf7, but CPU reported 
0xbfebfbff
: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz, id 0x206a7
xenbus0 at hypervisor0: Xen Virtual Bus Interface
xencons0 at hypervisor0: Xen Virtual Console Driver
xbd0 at xenbus0 id 1: Xen Virtual Block Device Interface
xbd1 at xenbus0 id 2: Xen Virtual Block Device Interface
xbd2 at xenbus0 id 3: Xen Virtual Block Device Interface
xennet0 at xenbus0 id 0: Xen Virtual Network Interface
xennet0: MAC address 02:02:02:11:11:11
balloon0 at xenbus0 id 0: Xen Balloon driver
balloon0: current reservation: 524288 KiB
xennet0: using RX copy mode
balloon0: current reservation: 131072 pages => target: 131072 pages
pmap_kenter_pa: mapping already present
cprng sysctl: WARNING insufficient entropy at creation.
boot device: xbd0
root on xbd0a dumps on xbd0b
mountroot: trying smbfs...
mountroot: trying ntfs...
mountroot: trying nfs...
mountroot: trying msdos...
mountroot: trying lfs...
mountroot: trying ext2fs...
mountroot: trying ffs...
root file system type: ffs
init: copying out path `/sbin/init' 11
pmap_kenter_pa: mapping already present
pmap_kenter_pa: mapping already present
pmap_kenter_pa: mapping already present
pmap_kenter_pa: mapping already present
pmap_kenter_pa: mapping already present
pmap_kenter_pa: mapping already present
Thu Feb  2 10:28:13 PST 2012
pmap_kenter_pa: mapping already present
Starting root file system check:
pmap_kenter_pa: mapping already present
/dev/rxbd0a: file system is clean; not checking
pmap_kenter_pa: mapping already present


... it hangs here indefinitely.  SIGINFO shows:
load: 0.00  cmd: mount_ffs 23 [biowait] 0.00u 0.00s 0% 944k


and breaking into DDB shows:

++++fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff801345c5 cs e030 rflags 202 cr2  7f7ff730f948 
cpl 8 rsp ffffa00000ff5968
Stopped in pid 0.2 (system) at  netbsd:breakpoint+0x5:  leave
breakpoint() at netbsd:breakpoint+0x5
xencons_tty_input() at netbsd:xencons_tty_input+0xc9
xencons_handler() at netbsd:xencons_handler+0x79
intr_biglock_wrapper() at netbsd:intr_biglock_wrapper+0x1d
evtchn_do_event() at netbsd:evtchn_do_event+0x175
call_evtchn_do_event() at netbsd:call_evtchn_do_event+0xd
hypervisor_callback() at netbsd:hypervisor_callback+0xa2
idle_loop() at netbsd:idle_loop+0x18c
ds          d2a7
es          ca00
fs          a9c2
gs          b0c0
rdi         1
rsi         ffffffff80a19005
rbp         ffffa00000ff5968
rbx         ffffffff80a19005
rdx         2b
rcx         2b
rax         7f
r8          0
r9          0
r10         1
r11         0
r12         ffffa000012e51e0
r13         ffffffff80a19005
r14         ffffa000012f5c80
r15         1
rip         ffffffff801345c5    breakpoint+0x5
cs          e030
rflags      202
rsp         ffffa00000ff5968
ss          e02b
netbsd:breakpoint+0x5:  leave
db{0}>    

>How-To-Repeat:
        Build a XEN3_DOMU kernel since the kmem changes.  Boot on certain
dom0 boxes.

>Fix:
        None Given.