Subject: Re: Mutex question
To: Garrett D'Amore <email@example.com>
From: Bill Studenmund <firstname.lastname@example.org>
Date: 03/16/2006 15:05:52
Content-Type: text/plain; charset=us-ascii
On Thu, Mar 16, 2006 at 10:07:14AM -0800, Garrett D'Amore wrote:
> Eric Haszlakiewicz wrote:
> > On Thu, Mar 16, 2006 at 11:49:47AM -0500, Steven M. Bellovin wrote:
> >> On Thu, 16 Mar 2006 09:16:57 -0500 (EST), Matthew Orgass
> >> <email@example.com> wrote:
> >>> On 2006-03-16 firstname.lastname@example.org wrote:
> >> The question is this: what do you do in the interrupt handler or
> >> timeout if you can't get the mutex? You can't sleep; there's no
> >> process context.
Answer to Steve: you have to spin. For now.
> > I think you have to spin. Since the splfoo() in non-interrupt code
> > ensures that the current processor can't have been in the middle of the
> > mutex protected code, you know that it must be some other processor that
> > holds the lock. Eventually, that processor will release the mutex and
> > the interrupt handler will get it. Of course, if the non-interrupt code
> > holds that mutex for long the interrupt latency will suck.
> > I don't know what you'd do in the callout. Maybe re-register the
> > callout and try again later?
> I think this is a deficiency in NetBSD's design right now. If we have
> prioritized interrupts, and the interrupt handler runs at a SPL lower
> than the clock, then it should be possible to sleep.
Kinda. That's hard to get right.
Right now our interrupt handling is still under biglock. So right now this=
isn't an issue, as you have to wait for the other CPU to release it.
I think we're going to try for spinlocking first. It's a good intermediate=
step, and can let us have multiple CPUs in kernel code simultaneously.
And once we get spinlocks working, migrating to mutexes isn't too hard.
> Granted, I suspect not all platforms have reasonably prioritized
> interrupts where the clock interrupt gets serviced at a priority higher
> than device interrupts.
See a post from me back in I think it was December. It's worse than that.=
You can NOT list a common hierarchy of spl levels for MI code. Sometimes=20
spltty < splnet, sometimes spltty > splnet....
> However, for the platforms I am familiar with (PCs, MIPS, SPARC), the
> clock does get prioritized treatment over at least *most* interrupts.=20
> (The exceptions I know about are serial ports and PCMCIA status change
> Another OS with which I'm familiar calls this difference out by having a
> way to identify an interrupt as being "high priority" or not. You can
> do things like sleep with a lock initialized with a cookie corresponding
> to a regular priority interrupt. You cannot sleep with such a lock if
> the interrupt is a high priority interrupt.
I believe that said OS has a lot more $$, people, and history thrown at=20
this problem. We are where we are now.
> This other OS also runs interrupt handlers on their own threads, so that
> sleeping doesn't prevent other interrupts from being serviced. :-)=20
> This probably has all kinds of performance implications for devices that
> require low interrupt latency. (Usually com ports, which is why on the
> hardware this os runs on, com ports are generally a high-priority
> interrupt.) There is also stuff in this OS to handle priority
> inversion, so that a regular thread holding such a lock gets CPU time so
> that the interrupt handler doesn't have to wait too long.
> Device drivers are responsible for not sleeping if they are holding a
> high-priority lock. (This is easily tested for by adding an assertion
> test against the SPL in the sleep routines.)
As above, I think the idea is to go spinlocks first. But I don't think=20
anyone is fundamentally opposed to doing this better.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (NetBSD)
-----END PGP SIGNATURE-----