Subject: Re: Interrupt, interrupt threads, continuations, and kernel lwps
To: None <tech-kern@netbsd.org>
From: Andrew Doran <ad@netbsd.org>
List: tech-kern
Date: 06/20/2007 22:54:14
On Wed, Jun 20, 2007 at 08:43:38PM +0100, David Laight wrote:

> On Wed, Jun 20, 2007 at 11:07:26AM +0100, Andrew Doran wrote:
> > Hi,
> > 
> > Something else occured to me. Since callouts will be able to block on locks,
> > then the interfaces used to manage the callouts have to be able to block
> > also. That means it won't be possible to schedule a callout directly from a
> > hardware interrupt handler.
> 
> Do you mean 'run the callout function' ?
> Isn't that done from the soft timer interrupt anyway?

The callout will run in thread context from the softclock interrupt, yup.
What I mean is that from a hardware ISR that does not have thread context
(like at IPL_NET) it wouldn't be possible to call callout_reset() or similar
functions. That would need to be deferred to a soft interrupt or a kernel
thread.

> > The reason being, the callout routines (like callout_stop()) shouldn't
> > return until the callout has stopped executing. Currently we spin if a
> > callout is running on another CPU.
> 
> Yes SVR4 did that, and it was a right PITA.
> If (and I assume it is) callout_stop() can be called with a driver mutex
> held, then having a try_callout_stop() would help.

That may well make sense. There is the potential to cause a deadlock by
maniuplating a callout when holding a lock that the callout routine wants
to grab:

resetit()
{
	mutex_enter(&lock);
	callout_reset(&ch, 1, mycallout, myarg);
		^ blocks waiting for mycallout()
	mutex_exit(&lock);
}

mycallout()
{

	mutex_enter(&lock);
	/* do stuff */
	mutex_enter(&lock);
}

> The other issue came with periodic timers (which I don't think NetBSD has)
> where the callout function couldn't cancel its own timer.

There would need to be a check so that the callout can control when it's to
be next fired. Since the softclock interrupt isn't re-entrant we don't need
to block waiting for the callout to finish:

void
callout_stop(callout_t *c)
{
	if (c->c_runninglwp == curlwp) {
		/* don't have to wait, we're it */
	} else {
		/* sleep until it has completed */
	}
}
  
Andrew