Subject: Re: Interrupt, interrupt threads, continuations, and kernel lwps
To: Allen Briggs <email@example.com>
From: None <firstname.lastname@example.org>
Date: 02/21/2007 18:39:39
In message <20070222021434.GJ19732@canolog.ninthwonder.com>,
Allen Briggs writes:
>[ trimmed cc ]
>On Wed, Feb 21, 2007 at 05:57:41PM -0800, email@example.com wrote:
>> Is the above roughly right? If it is, then (to steal a comment I made
>> to Sam Leffler early in FAST_IPSEC development): 300,000 context
>> switches/sec are _not_ your friend.
>Your modest 50,000 packets/sec would presumably be on hardware that
>can do interrupt coalescing, right?
Yes, but I was assuming modest load, and thus little interrupt
mitigation. With interrupt-mitigation in effect, that hardware can
handle more like 400,000 *non*-IPsec packets. I don't have saturation
IPsec numbers at my fingertips; the accelerator I was then using was
>I would hope that you're handling more than a single packet in each
>interrupt / context switch, which should reduce the number of
>context switches somewhat.
If (dim) memory serves, the numbers I tossed out were from a setup
with a couple of Windows hosts with 100Mbit NICs using NIC-onboard
IPsec crypto offload. I don't have access to any of that hardware
anymore, but (again) if memory serves, packets:interrupt ratil was
If so, interrupt coalescing gets you only a factor of about 2;
and 150,000 context-switches a second are _still_ not your friend.
Heck, even 15,000 are painful, from where we're sitting now.
Sigh, I _really_ need to dig out a redistributable version of that
non-redistributable OCF userland test tool. (OTOH, it needs something
like a bcm5723 to hit the rates I'm tossing around...)