Re: Dom0 ballooning: crash on guest start

[Christoph Egger <Christoph_Egger%gmx.de@localhost>]

On 04/29/11 13:44, Jean-Yves Migeon wrote:
> On Fri, 29 Apr 2011 11:49:33 +0200, Christoph Egger wrote:
> > On 04/29/11 11:49, Jean-Yves Migeon wrote:
> > > On Fri, 29 Apr 2011 11:14:54 +0200, Christoph Egger wrote:
> > > > Then when I start a HVM guest the balloon driver crashes:
> > > >
> > > > balloon0: current reservation: 2039476 pages => target: 2039476 pages
> > > > balloon0: current reservation: 2039476 pages => target: 981172 pages
> > > > uvm_fault(0xffffffff80c6b220, 0xffffffff81400000, 1) -> e
> > > > fatal page fault in supervisor mode
> > > > trap type 6 code 0 rip ffffffff8054584d cs e030 rflags 10216 cr2
> > > > ffffffff81400000 cpl 0 rsp ffffa00057ca8a50
> > > > kernel: page fault trap, code=0
> > > > Stopped in pid 0.58 (system) at netbsd:pmap_zero_page+0x41: movq
> > > > 0(%rax,%
> > > > rbx,8),%rbx
> > > > pmap_zero_page() at netbsd:pmap_zero_page+0x41
> > > > uvm_pagealloc_strat() at netbsd:uvm_pagealloc_strat+0x22d
> > > > bpge_ctor() at netbsd:bpge_ctor+0x22
> > >
> > > Hmm, a fault while zeroing the page. Race, TLB out of sync, or maybe
> > > developer's (my) mistake :o
> > >
> > > Is this kernel compiled with DIAGNOSTIC?
> >
> > It is compiled with DEBUG + DIAGNOSTIC + LOCKDEBUG
>
> Okay, I'll try reproducing the issue. In the meanwhile, could you
> trigger the issue again with BALLOONDEBUG set to 1 (in xen/balloon.c),
> and post the last dmesg output before the fault? I'd like to see if it
> happens under high memory pressure or right from the start.
>
> Thanks for testing!

Ok, I retried with BALLOONDEBUG set to 1.

When starting xenstored, xenconsoled and xenbackendd
the kernel prints:

balloon0 at xenbus0 id 0: Xen Balloon driver
balloon0: current reservation: 8157904 KiB
balloon0: min reservation: 2048 KiB
balloon0: max reservation: 17179869180 KiB
balloon0: WARNING: power management not supported
balloon0: error, couldn't read xenbus target node


Then when starting a HVM guest, the kernel floods the serial console with

balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512


for about 20 seconds until the crash happens:

balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
balloon0: inflate 512 => inflated by 512
uvm_fault(0xffffffff80c6b220, 0xffffffff81400000, 1) -> e
fatal page fault in supervisor mode
trap type 6 code 0 rip ffffffff8054593d cs e030 rflags 10216 cr2 
ffffffff81400000 cpl 0 rsp f
fffa00057ce3a40
kernel: page fault trap, code=0
Stopped in pid 0.58 (system) at netbsd:pmap_zero_page+0x41:     movq 
0(%rax,%
rbx,8),%rbx
pmap_zero_page() at netbsd:pmap_zero_page+0x41
uvm_pagealloc_strat() at netbsd:uvm_pagealloc_strat+0x22d
bpge_ctor() at netbsd:bpge_ctor+0x22
pool_cache_get_slow() at netbsd:pool_cache_get_slow+0x1df
pool_cache_get_paddr() at netbsd:pool_cache_get_paddr+0x1f7
balloon_thread() at netbsd:balloon_thread+0x3bc
ds          0
es          0x1122
fs          0x9aa0
gs          0
rdi         0xbbe00000
rsi         0xffffffff80cec1c0  ld_prime+0x1860
rbp         0xffffa00057ce3a70
rbx         0xbbe00
rdx         0xffffffff80bd0048  cpu_info_primary+0x188
rcx         0
rax         0xffffffff80e21000
r8          0xffffa0000f215f00
r9          0xffffa000055838c0
r10         0
r11         0
r12         0
r13         0x7fd000000010
r14         0
r15         0
rip         0xffffffff8054593d  pmap_zero_page+0x41
cs          0xe030
rflags      0x10216
rsp         0xffffa00057ce3a40
ss          0xe02b
db>


BTW: The HVM guest gets 4GB RAM assigned and Xen uses
0x200 4KB pages, 0x3fb 2MB pages and 0x1 1GB pages for it.

Christoph