Re: MP callouts

[Andrew Doran <ad%netbsd.org@localhost>]

On Sat, Apr 12, 2008 at 10:30:26AM -0700, Matt Thomas wrote:

> On Apr 12, 2008, at 9:57 AM, Andrew Doran wrote:
> >Hi,
> >
> >At the moment, callouts only fire on the boot CPU. The below diff  
> >allows for
> >callout processing on all CPUs, and for callouts to be bound to CPUs.
> >
> >Each CPU has its own lock and call wheel. Non-MPSAFE callouts are  
> >always
> >run on the boot CPU, just in case there is an odd sync issue lurking
> >somewhere.
> >
> >     http://www.netbsd.org/~ad/callout.diff
> >
> >Any comments?
> 
> What are the implications of this?  Will each CPU have its own timer  
> interrupt
> or can the timer interrupt servicing be spread among all the CPUs?

Currently we require the hardclock to fire on all CPUs. There is still one
callout soft interrupt. Since the softint stuff is CPU-bound, triggering the
softint on a given CPU means it will fire there, so nothing special is
required at that level.
 
The main change in the patch is to give each CPU its own call wheel and todo
list. There are no special tricks to it, other than in schedule_locked. If
the callout isn't bound and the trylock works, then it gets scheduled on the
current CPU. Otherwise it gets scheduled on the CPU it was last associated
with. By and large that kills most of the lock contention due to callout
processing.

> Can we add a void cpu_nextcallout(unsigned int ticks); so that ports  
> that can
> schedule a variable clock can avoid scheduling clock intrs that won't  
> actually
> do anything.  It would be nice if hardclock could take a 2nd argument  
> which is
> the number of [additional] ticks that this invocation encompasses.

I can't see a reason why that would be a problem, but moving to a tickless
kernel is large project in its own right and not what I want to accomplish
with this patch.

Thanks,
Andrew