In message <20070222021434.GJ19732@canolog.ninthwonder.com>,
Allen Briggs writes:

>[ trimmed cc ]
>
>On Wed, Feb 21, 2007 at 05:57:41PM -0800, jonathan@dsg.stanford.edu wrote:
>> Is the above roughly right? If it is, then (to steal a comment I made
>> to Sam Leffler early in FAST_IPSEC development): 300,000 context
>> switches/sec are _not_ your friend.
>
>Your modest 50,000 packets/sec would presumably be on hardware that
>can do interrupt coalescing, right?  

Yes, but I was assuming modest load, and thus little interrupt
mitigation.  With interrupt-mitigation in effect, that hardware can
handle more like 400,000 *non*-IPsec packets.  I don't have saturation
IPsec numbers at my fingertips; the accelerator I was then using was
the bottleneck.

>I would hope that you're handling more than a single packet in each
>interrupt / context switch, which should reduce the number of
>context switches somewhat.

If (dim) memory serves, the numbers I tossed out were from a setup
with a couple of Windows hosts with 100Mbit NICs using NIC-onboard
IPsec crypto offload.  I don't have access to any of that hardware
anymore, but (again) if memory serves, packets:interrupt ratil was
around 2:1.

If so, interrupt coalescing gets you only a factor of about 2;
and 150,000 context-switches a second are _still_ not your friend.

Heck, even 15,000 are painful, from where we're sitting now.

Sigh, I _really_ need to dig out a redistributable version of that
non-redistributable OCF userland test tool. (OTOH, it needs something
like a bcm5723 to hit the rates I'm tossing around...)

--Jonathan