NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

port-xen/45975: panic: HYPERVISOR_mmu_update failed, ret: -22 on -current MP domU (amd64)



>Number:         45975
>Category:       port-xen
>Synopsis:       panic: HYPERVISOR_mmu_update failed, ret: -22 during heavy 
>activity
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    port-xen-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Feb 10 18:35:00 +0000 2012
>Originator:     Jeff Rizzo
>Release:        NetBSD 5.99.64
>Organization:
        
>Environment:
        
        
System: NetBSD breadfruit.tastylime.net 5.99.64 NetBSD 5.99.64 (XEN3_DOMU) #0: 
Fri Feb 10 09:10:33 PST 2012  
riz%breadfruit.tastylime.net@localhost:/home/riz/obj/sys/arch/amd64/compile/XEN3_DOMU
 amd64
Architecture: x86_64
Machine: amd64
>Description:

I've been discussing this particular problem with cherry@ for a while,
and figured it was time to file a PR.

During heavy load (such as a build.sh -j16), my amd64 MP domUs sometimes
panic like so:


evtchn_do_event: handler 0xffffffff80121b77 didn't lower ipl 8 7
xpq_flush_queue: 1 entries (0 successful)
0x00000000d9fe9de0: 0x000000011beb9007
panic: HYPERVISOR_mmu_update failed, ret: -22

fatal breakpoint trap in supervisor mode
trap type 1 code 0 rip ffffffff801345c5 cs e030 rflags 246 cr2  7f7ff780027c 
cWl R6NIrNsGp:  fSPfLf aN0O0T0 bL9O9W0EfR7E1D0 
N TRAP EXIT St6o p0p                        O
d in pid 1833.1 (sh) at   netbsd:breakpoint+0x5:  leave
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x1f2
printf_nolog() at netbsd:printf_nolog
xpq_queue_machphys_update() at netbsd:xpq_queue_machphys_update
pmap_enter_ma() at netbsd:pmap_enter_ma+0xb74
pmap_enter() at netbsd:pmap_enter+0x35
uvm_fault_internal() at netbsd:uvm_fault_internal+0xf17
trap() at netbsd:trap+0x5f5
--- trap (number 7632997) ---
7374757066007469:
ds          ffea
es          f750
fs          100
gs          b180
rdi         0
rsi         d
rbp         ffffa000b990f710
rbx         104
rdx         0
rcx         8
rax         1
r8          ffffa00008978000
r9          1
r10         0
r11         e033
r12         ffffffff804b4a10    copyright+0x3ea10
r13         ffffa000b990f750
r14         ffffffea
r15         2
rip         ffffffff801345c5    breakpoint+0x5
cs          e030
rflags      246
rsp         ffffa000b990f710
ss          e02b
netbsd:breakpoint+0x5:  leave
db{3}> 

Here's the dom0's 'xm dmesg' (some may not be relevant):

durian:riz  ~> sudo xm dmesg
aps.c:2432:d164 Domain attempted WRMSR 0000000000000277 from 0x0000050100070406 
to 0x0007010600070106.
(XEN) domain.c:652:d164 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d164 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d164 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d164 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d165 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d165 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d165 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d165 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d165 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d165 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1f4443 (pfn 5d67)
(XEN) mm.c:915:d165 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d166 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d166 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d166 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d166 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d166 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d166 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1f689f (pfn 1970c)
(XEN) mm.c:915:d166 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d167 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d167 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d167 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d167 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d167 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d168 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d168 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d168 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d168 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d168 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d169 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d169 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d169 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d169 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d169 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d170 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d170 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d170 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d170 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d170 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d170 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1e6705 (pfn 6edb)
(XEN) mm.c:915:d170 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d171 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d171 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d171 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d171 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d171 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d172 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d172 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d172 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d172 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d172 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d173 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d173 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d173 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d173 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d173 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d173 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d173 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d174 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d174 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d174 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d174 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d174 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d175 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d175 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d175 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d175 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d175 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d175 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d175 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d174 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1e51ff (pfn 1edaf)
(XEN) mm.c:915:d174 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d176 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d176 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d176 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d176 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d176 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d177 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d177 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d177 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d177 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d177 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d177 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1c496c (pfn de83)
(XEN) mm.c:915:d177 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d178 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d178 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d178 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d178 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d178 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d179 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d179 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d179 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d179 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d179 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d179 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d179 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d178 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1c0e0e (pfn 141c4)
(XEN) mm.c:915:d178 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d180 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d180 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d180 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d180 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d180 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d181 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d181 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d181 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d181 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d181 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d182 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d182 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d182 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d182 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d182 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d183 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d183 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d183 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d183 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d183 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d183 Bad type (saw 7400000000000001 != exp 3000000000000000) 
for mfn 1c47ef (pfn 65cb)
(XEN) mm.c:982:d183 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d184 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d184 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d184 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d184 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d184 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d185 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d185 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d185 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d185 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d185 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d186 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d186 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d186 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d186 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d186 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d186 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1f4ff6 (pfn b804)
(XEN) mm.c:915:d186 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d187 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d187 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d187 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d187 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d187 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d187 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 1e48ca (pfn 196f5)
(XEN) mm.c:915:d187 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d188 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d188 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d188 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d188 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d188 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d179 Bad type (saw 7400000000000001 != exp 3000000000000000) 
for mfn 1b224c (pfn 723a3)
(XEN) mm.c:982:d179 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d190 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d190 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d190 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d190 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d190 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d190 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d190 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) traps.c:2432:d191 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d191 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d191 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d191 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d191 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d191 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d191 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) mm.c:2424:d191 Bad type (saw 7400000000000001 != exp 1000000000000000) 
for mfn 11beb9 (pfn c8753)
(XEN) mm.c:915:d191 Attempt to create linear p.t. with write perms
(XEN) traps.c:2432:d192 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d192 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d192 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d192 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d192 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.
(XEN) domain.c:652:d192 Attempt to change CR4 flags 00002660 -> 00000620
(XEN) traps.c:2432:d192 Domain attempted WRMSR 0000000000000277 from 
0x0000050100070406 to 0x0007010600070106.

        
>How-To-Repeat:
        build.sh -j16 on an MP amd64 domU;  sometimes the problem takes
        a while to manifest; not usually more than 3 or 4 builds, and
        often during the first.
>Fix:
        None given. 

>Unformatted:
        
        


Home | Main Index | Thread Index | Old Index