Subject: Re: Melting down your network [Subject changed]
To: Bill Studenmund <wrstuden@netbsd.org>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-kern
Date: 03/28/2005 14:18:56
In message <20050328045603.GB10444@netbsd.org>,
Bill Studenmund writes:

>
>On Sun, Mar 27, 2005 at 08:37:38AM +0200, Emmanuel Dreyfus wrote:
>> Jonathan Stone <jonathan@dsg.stanford.edu> wrote:
>>
>> > I think you'd be better off to fix your application, instead of
>> > pursuing a (well-meant, but deeply misguided) attempt to "fix" the
>> > BSD networking code.
>>
>> About fixing the app to workaround the current bhavior of send(2): I
>> don't see any workaround for the problem at the application level. I'm
>> interested if you have one.=20
>
>Throttle your application. You need to try to send data at a specific
>rate, which is chosen to suit your clients.

I say again (since some ill-informed person indicated in private email
that they thougt it was a strawman):

Deploying any application designed to pump out wire-speed, non-rate-
adaptive, non-congestion-responsive, is an evil act.  It may lead to
your ISP removing service. It may lead to criminal or civil prosecution.

Just don't do it.  It's not just ignorant or stupid; it's *evil*.



>Here's my suggestion:
>
>1) Switch to kqueue
>2) Add a kevent filter of type EVFILT_TIMER set for one ms.
>3) Create a filter processor. This filter processor will get told
>	how many ms have passed since its last call (in the 'data'
>	field of the event).
>
>A) Send X packets (about 12k bytes in this case) for each tick.
>
>B) Keep a count of "bytes to send." For each tick, add X bytes. Then keep
>sending packets as long as you have bytes (in the "count") to send. 
>Exactly what you do when your count isn't a full packet is a policy
>decision (do you send until you don't have enough to send, or do you send
>until you get a negative count), though it won't really matter after the
>first tick.
>
>This way, you will send data at the desired rate regardless of queue
>status and without changing the kernel.


That's a reasonable first stab, but its not enough.  You must also
reduce the send rate in response to NACKs.  Do enough of this, and
pretty soon you're doing as much as SRM or TCP-SMO are doing.

As I think I commented earlier: classic mistake #1 of a student who's
passed an intro networking course, is to decide TCP is "too heavyweight"
for their immediate purpose, and try to invent something more lightweight.
Pretty soon, they discover cases where their "lightweight" design falls down.

Now, anyone who takes this approach is quite clearly implicitly saying
that that the TCP designers, and everyone who looked at it since, are
too inept to see how to design a more-lightweight protocol.

After a few instanceses (let alone a few dozen), responses to this
particular approach become rather short --- even in cases when the
person in question didn't start out their explanation of their poor
design choices by repeatedly whining that something in the OS is "broken".