Subject: Re: sk(4) interrupt moderation timing fix, and sysctl support
To: Jeff Rizzo <riz@NetBSD.org>
From: Jonathan Stone <jonathan@Pescadero.dsg.stanford.edu>
List: tech-net
Date: 11/27/2005 12:29:01
In message <20051127040020.GA23282@boogers.sf.ca.us>Jeff Rizzo writes

>On Thu, Nov 24, 2005 at 09:49:39AM -0800, Jeff Rizzo wrote:
>> Jason Thorpe wrote:
>>
>> >
>> > On Nov 23, 2005, at 8:10 PM, Jeff Rizzo wrote:
>> >
>> >> 1. Should I bother supporting a per-board value for the interrupt
>> >> moderation
>> >> timer?  In this patch, it's global, and only takes effect when
>> >> sk_init()
>> >> is called. (ifconfig down/up is a good way to trigger this)
>> >
>> >
>> > Seems like it should be per-instance rather than global, especially
>> > if different revisions of the chip need to have different values.

I have to disagree here. My eventual intent was to add support for
dynamic auto-tuning of interrupt mitigation. For that, one really
needs to apply *global* moderation, for stability reasons: either all
devices stay at their current moderation level, or all go up a little,
or all go down a little.

The nonzero bge values were intended to be

	(1 + log_2 (Rx packets per interrupt)

but that didn't quite work out with newer chips.

For manual tweaking, I can sort-of see Jason's point, but I submit
that, for systems with multiple NICs, having drivers map global
"levels" with cross-device maning levels into chip-specific constants
is a generally more-useful hook than per-device hooks; though the
latter can express policies (e.g., one interface with fixed low
latency, others with high throughput) that the former cannot.

Nevertheless, I still advocate a global FSM which computes one of
three states, "too hot/too cold/just right", as a function of idle
CPU; and if the FSM computes "too hot" or "too cold" K times in a row,
the FSM increases or decreases interrupt mitigation globally, just a
tad per device, to match.

(Once I put that way, it should be immediately apparent that having
each driver do its own computation in isolation will yield an system
that is prone to oscillations, if not worse.)


>Here's a new patch - if this looks OK to folks, I'll commit it.

Can you keep support for global hooks which go up or down one notch,
even if they're not exposed to userland?