Re: Problems with hangs under NetBSD-5.x

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Tue, Aug 04, 2009 at 11:14:22PM -0700, Brian Buhrow wrote:
>       Hello.  I've been able to track down my problem a little further.
> Here  is what I know:
> 
> 1.  Server: I386
> 
> 2.  Version NetBSD-5.0-stable as of July 11, 2009
> 
> 3.  FFS options: softdep, quota.

FFS is suspect. I also fixed an issue in quota:
http://releng.netbsd.org/cgi-bin/req-5.cgi?show=898
this has not been pulled up to netbsd-5 yet. I'm not sure if it
can explain you problem.

> 
> 4.  The machine doesn't hang, but any processes which need to write to the
> filesystem with quotas enabled get stuck in tstile after a while.  The
> hangs don't happen immediately, but they can be triggered if one runs a
> bunch of processes which create and write to files and then delete them
> over and over again.
> The filesystem without quotas is perfectly usable and happy.
> ps -laxwww with processes which are in this state are shown below.

that's not the same issue as mine. In my case the system was dead exept
I could enter ddb from serial console. 

> 
> 5.  If someone could tell me what commands from ddb would be useful when I
> get the machine into this state, I'm happy to post output.  Unfortunately,
> I forgot to turn off the watchdog timer, so when I dropped into ddb to
> begin this process with the stuff shown below, the alarm fired and rebooted
> the machine.  No worries, though, I can bring it to its knees again. :)

From ddb:
ps /l

this will give you the 'struct lwp *' for each thread.
Then you can
tr /a <struct lwp *>
for each LWP. Interesting ones are those stuck on tstile, this will
give the place where they're stuck.
Getting the trace for each other blocked LWP can be interesting too, as
it should allow to find which thread is sleeping with the lock held.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--