Subject: Re: SMP re-entrancy in kernel drivers/"bottom half?"
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-kern
Date: 02/22/2005 15:55:10
In message <1109109930.613567.1628.nullmailer@yamt.dyndns.org>,
YAMAMOTO Takashi 
writes
>hi,
>
>> If I had my 'druthers, I'd like to see our TCP restructured along the
>> same lines as the TCP in IRIX. Ten years ago, I understand IRIX could
>> handle 10 gigabits (GSN, basically HIPPI on steroids) with one CPU
>> handling interrupts and packet demux , a second running the IP/TCP
>> code, and a third doing the bcopy()s to userspace.
>
>i'm not sure if it's a good idea for today's cpus
>as passing mbufs among cpus increases cache misses.

I do see your point. However, the run-to-completion model will either
be _very_ limited in supporting a single monster flow, or will incur
even more lock contention, as adjacent packets from a single very
large flow are handled on different CPUs.

My instinct (based on private observations by many colleagus, but on
machines that are now rather long in the tooth) is that splitting
different layers or sublayers across CPUs will be the best approach.
It'd be great if we could measure the approaches and decide which is
best, without having to implement them both before deciding which to pick :-/.

Anyway...  we're touching on most of the points I was planning to
bring up. Any way you slice it, I think the bottom line is that we
need an SMP-capable network stack in the later-this-year timeframe.