Subject: Pseudo-disk trouble
To: None <tech-kern@netbsd.org>
From: der Mouse <mouse@Rodents.Montreal.QC.CA>
List: tech-kern
Date: 10/31/2001 18:36:51
My pseudo-disk driver hung on me today.  I found and fixed some bugs
and got it to the point where I could label, newfs, fsck, mount, and
start to use it.  But then I started compiling some programs in a
little ten-meg pseudo-disk, and the silly thing locked up on me.

Breaking into ddb and doing a ps, I noticed that cpp was blocked in
"mreq", which is a wchan string used for the pseudo-disk IPC mechanism.
This is semi-normal, though normally it stays there only long enough
for the control process to accept the request and reply.

But the control process was also blocked in mreq, which is shouldn't've
been - even when it's in the driver, it never goes through the
codepaths that use that wait string.

Once I figured out how to get sparc ddb to print a stack trace of an
arbitrary process by PID, it was obvious what was going on: the driver
process was trying to write to a file, which needed a buffer, which
flushed a dirty buffer, which happened to be on the pdisk device.  I
don't know whether this happened before or after cpp got into the code,
but one of them is blocked waiting for the driver process to come back
and the other is blocked waiting for the first, and neither will
unblock.

I was able to bring it back to life by killing the driver process from
ddb, which causes all the I/O to complete with errors (the control
device close routine twiddles flag bits that the IPC codepath notices).
The machine crashed shortly after I unmounted the now-unbacked
filesystem; I haven't yet looked at why.

Any thoughts on preventing this?  Is there some standard technique for
it?  For my purposes, it would be good enough to have it speak to a TCP
socket rather than a special device for its control data stream, and
just depend on humans to never make that a loopback TCP connection.
But if there's a proper way of fixing it, I'd much rather do that.

I could, I suppose, buffer the data and mark the I/O as done before
speaking with the control process, queueing that for later, but that's
rather ugly and potentially needs effectively unlimited buffer
space.  But it ought to at least work...maybe I should just think of it
as "first make it work, then make it better".

Anyone have any better ideas?

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       mouse@rodents.montreal.qc.ca
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B