In message <E1D3k3n-0001Bv-00@smeg.dsg.stanford.edu>,
Jonathan Stone writes:


>In message <20050222235717.GL4454@snowdrop.l8s.co.uk>,
>David Laight writes:
>
>>My 'gut feel' also is that passing data between cpus is a cost hit.

But for a single large flow, it's either a performance bottleneck
again , or an even bigger cost in locking throughout the entire stack
so you can process packet K from a flwo on one CPU and packet K+1 from
the same flow on another.

Not that you'd want to strictly alternate packets between CPUs. I'd
want to alternate per interrupt, or something.  But if you process
some packets on one CPU and others on another, you have to do the
locking *always*. Not to mention introducing out-of-order packet
processing: horrible. (Think about the apperarent dupacks, ugh!)


>>Doing all the RX processing on a single cpu probably isn't a problem,
>>well not until we are talking about 8+ way systems that are doing nothing
>>else.

Perhaps I should clarify that I have several dual-cpu-capable systems
with four to six gigabit ports. (I could install up to 12 ports using
dual-port NICs at under $100 per port. I have spare cards on my desk
as I type.)  I would like to fill all those wires, in both directions.

Equally, I have applications where (if I could), I would fill two
10GbE NICs, ideally in both directions. Filling one 10GbE wire would
do for now, but even that is about 3x what I can get from the fastest
single-CPU system I can find.  To be fair  PCI-X bandwidth is limited
to about 2x that fastest-single-CPU TCP throughput, but you get the idea.

This particular performance point may not be important to you, but it
is very important to some of us. (I know who I'd finger, but I'll let
them speak for themselves.)