Subject: Re: SMP re-entrancy in kernel drivers/"bottom half?"
To: Reinoud Zandijk <reinoud@netbsd.org>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-kern
Date: 02/22/2005 05:47:26
In message <20050222130254.GA287@rangerover.13thmonkey.org>,
Reinoud Zandijk writes:

>On Tue, Dec 30, 2003 at 01:34:53PM +0900, YAMAMOTO Takashi wrote:
>> > >for your purpose, i think what you need are:
>> > >- change lowlevel intr handler (Xintr_* in the case of i386) not to
>> > >  take kernel_lock if the driver is marked as mp-safe.
>> > >- make the driver mp-safe.
>> > >	- change the driver to acquire per-device lock if needed.
>> > >	- change the driver to acquire kernel_lock
>> > >	  (or other fine-grained locks) when needed.
>> 
>> for someone interested..
>...
>
>yamt, is this patch a proposal for implementing mp-safe networking? or an 
>expiriment containing it?

[...]

I think the best answer to that is: "Yes", at least for receive-side
processing.

Today now, NICs are getting faster (or 10GbE is riding down the price
curve) more quickly than individual CPUs are getting faster. That trend
seems set to continue for the next few years, as CPU vendors move to
multi-core CPUs rather than exponentially faster single-core CPUs.

So, expect SMP support in the networking stack, for multiple CPUs to
drive fast NICs, to become a hot topic.

I'm a little blurry-eyed from a plane trip right now, but I brought up
Yamaoto-san's patch in various places late Jan/early February. I plan
to write a more detailed message sometime this week, for action in
March and beyond.

The big open question with the approach Yamamoto-san's patch uses (and
blame me for that, as I was the one pestering about it about a year
ago!) is that I don't know of any recent hard data on how much time
our TCP spends handling hardware interrupts, versus the soft-interrupt
(receive-side IP and TCP). It used to be a lot, but interrupt
mitigation may change that.  And then, maybe transmit-side quickly
becomes another bottleneck, which isn't helped much by this patch...

Anyway, watch this space..