Subject: Re: Mutex question
To: Eric Haszlakiewicz <erh@nimenees.com>
From: Garrett D'Amore <garrett_damore@tadpole.com>
List: tech-kern
Date: 03/16/2006 10:07:14
Eric Haszlakiewicz wrote:
> On Thu, Mar 16, 2006 at 11:49:47AM -0500, Steven M. Bellovin wrote:
>   
>> On Thu, 16 Mar 2006 09:16:57 -0500 (EST), Matthew Orgass
>> <darkstar@city-net.com> wrote:
>>
>>     
>>> On 2006-03-16 hselasky@c2i.net wrote:
>>>
>>>       
>>>> If I want to aquire a mutex from:
>>>>
>>>> 1) Interrupt handler
>>>> 2) Callout (timeout)
>>>>
>>>> How do I do that? Do I have to use a soft interrupt handler?
>>>>         
>>>   No, you just need to use the right spl calls to block the hard interrupt
>>> in the callout (placed outside the lock).
>>>
>>>       
>> The question is this: what do you do in the interrupt handler or
>> timeout if you can't get the mutex?  You can't sleep; there's no
>> process context.
>>     
>
> 	I think you have to spin.  Since the splfoo() in non-interrupt code
> ensures that the current processor can't have been in the middle of the
> mutex protected code, you know that it must be some other processor that
> holds the lock.  Eventually, that processor will release the mutex and
> the interrupt handler will get it.  Of course, if the non-interrupt code
> holds that mutex for long the interrupt latency will suck.
>
> 	I don't know what you'd do in the callout.  Maybe re-register the
> callout and try again later?
>   

I think this is a deficiency in NetBSD's design right now.  If we have
prioritized interrupts, and the interrupt handler runs at a SPL lower
than the clock, then it should be possible to sleep.

Granted, I suspect not all platforms have reasonably prioritized
interrupts where the clock interrupt gets serviced at a priority higher
than device interrupts.

However, for the platforms I am familiar with (PCs, MIPS, SPARC), the
clock does get prioritized treatment over at least *most* interrupts. 
(The exceptions I know about are serial ports and PCMCIA status change
interrupts.)

Another OS with which I'm familiar calls this difference out by having a
way to identify an interrupt as being "high priority" or not.  You can
do things like sleep with a lock initialized with a cookie corresponding
to a regular priority interrupt.  You cannot sleep with such a lock if
the interrupt is a high priority interrupt.

This other OS also runs interrupt handlers on their own threads, so that
sleeping doesn't prevent other interrupts from being serviced.  :-) 
This probably has all kinds of performance implications for devices that
require low interrupt latency.  (Usually com ports, which is why on the
hardware this os runs on, com ports are generally a high-priority
interrupt.)   There is also stuff in this OS to handle priority
inversion, so that a regular thread holding such a lock gets CPU time so
that the interrupt handler doesn't have to wait too long.

Device drivers are responsible for not sleeping if they are holding a
high-priority lock.  (This is easily tested for by adding an assertion
test against the SPL in the sleep routines.)

-- 
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134  Fax: 951 325-2191