Re: kthread with kpause or callout

[Andrew Doran <ad%NetBSD.org@localhost>]

On Mon, Feb 22, 2010 at 09:06:13PM +0100, Frank Wille wrote:
> Andrew Doran wrote:
> 
> >> I'm just unsure about using mutexes during the callout. I have an
> >> IPL_NONE-kmutex which locks register access (my chip supports several
> >> register banks, so I need to make sure they are not switched). May I
> >> acquire this mutex during a callout (which is a softint, as I
> >> understand)? Will the softint sleep or busy-wait?
> >
> > Yes, you can take an IPL_NONE (adaptive) mutex from a softint /
> > callout. Whether it sleeps or spins is a decision that the kernel makes
> > at runtime. It tries to spin. If that would deadlock the system, it
> > sleeps. This is the only time a callout/softint can sleep - for a
> > mutex/rwlock.
> 
> Thanks. Good to know.
> 
> In the meantime I had to upgrade to an IPL_VM mutex, because the chip
> registers will have to be accessed from a VM-level interrupt too. I already
> discussed that with Martin in private and I guess it will have the effect
> that a process or softint holding the mutex will lock out any IPL_VM
> interrupt during this time?

In short, yes.  There is a tradeoff to be made between:

- how long you block interrupts
- how much thread concurrency you want to support
- the expense of adding additional concurrency controls like locks

Note that the IPL_VM lock will block interrupts at that level, only on the
CPU where the lock is held.  The interrupt masking bit will not affect other
CPUs (although of course, it's still a spinlock so another CPU will have to
wait while its held).

Depending on what you're doing, a IPL_VM lock alone may be fine, or it
may make sense to have a two level setup for concurrency where you protect
your hardware state and things touched from the hard interrupt handler with
an IPL_VM lock, and then have an IPL_NONE lock to protect everything else.
Or some other scheme.  It's an "it depends".