Re: NetBSD-6-0 i386 PAE of Nov 18th hangs

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Fri, Nov 30, 2012 at 04:35:18PM -0600, Cherry G. Mathew wrote:
> On 25 November 2012 09:01, S.P.Zeidler <spz%serpens.de@localhost> wrote:
> > Hi,
> >
> > there's a NetBSD-6-0 i386 PAE build of Nov 18th source that should
> > do pbulk pkgsrc builds, but has a tendency to hang after a while.
> >
> > Doing the 'many +' from Xen console during a stuck state gives:
> >
> >  fatal breakpoint trap in supervisor mode
> >  trap type 1 code 0 eip c011c4d4 cs 9 eflags 282 cr2 bb78727c ilevel 6
> >  Stopped in pid 0.7 (system) at  netbsd:breakpoint+0x4:  popl    %ebp
> >  breakpoint(0,c2c35cc0,da384f7c,c01cbb5b,0,0,da384f9c,2a,1,2b) at 
> > netbsd:breakpoint+0x4
> >  
> > xencons_tty_input(c2b993c0,c074902a,1,c2c3a000,1,0,da384fcc,6,c046ee00,da384fe8)
> >  at netbsd:xencons_tty_input+0xb8
> >  
> > xencons_handler(c2b993c0,0,0,0,da527bb0,c2b9e0c0,c3467000,c0101230,c2b9e0c0,da527bcc)
> >  at netbsd:xencons_handler+0x78
> >  
> > intr_biglock_wrapper(c2b9e0c0,da527bcc,c2c2c2c2,c2c2c2c2,c2c2c2c2,c2c2c2c2,c2c2c2c2,c2c2c2c2,c2c2c2c2,c2c2c2c2)
> >  at netbsd:intr_biglock_wrapper+0x1f
> >  --- switch to interrupt stack ---
> >  Xresume_xenev6() at netbsd:Xresume_xenev6+0x40
> >  --- interrupt ---
> >  Xspllower(0,c2c48000,0,c2b80011,c0758880,c2c48000,da527c9c,0,5c10,0) at 
> > netbsd:Xspllower+0xe
> >  
> > mi_switch(c2c3a000,3,1,c2c3a000,c2c3a070,c03440f0,da527cac,0,c03440f0,c2c3a000)
> >  at netbsd:mi_switch+0x1eb
> >  
> > sleepq_block(0,0,c0338ca0,c034d770,0,c2c48000,c2c37d40,c2c36d80,c011c356,9) 
> > at netbsd:sleepq_block+0xad
> >  
> > cv_wait(c03440f0,c0354180,c2c3a000,c046f9e0,c2c3a000,da527d28,0,c03440f0,c2c2c2c2,da527da0)
> >  at netbsd:cv_wait+0xb1
> >  xc_thread(c2c3a000,757000,c0498200,0,c010006d,0,0,0,0,0) at 
> > netbsd:xc_thread+0xb9
> 
> ....
> 
> I have a suspicion this may be priority inversion via running
> spllower() at IPL_HIGH *always* in the event path. I'd fixed it in
> -current, are you able to test -current under similar circumstances ?

I've just debugged this, and it looks like it's the pagedaemon which is
spinning. So far I found that uvm_pageout() would spin because
uvm_km_va_starved_p() always return true, because kmem_arena is almost
full. decerasing drastically kern.maxvnodes stopped the pagedaemon
spinning. xc_thread() appears in the stack trace because
uvm_pageout() would do xcalls via pool_drain_end() in its loop.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--