Re: Why is my gigabit ethernet so slow?

[Steven Bellovin <smb%cs.columbia.edu@localhost>]

On Jan 27, 2010, at 6:59 PM, Fredrik Pettai wrote:

> On Jan 27, 2010, at 7:48 PM, Thor Lancelot Simon wrote:
>> On Wed, Jan 27, 2010 at 02:44:29PM +0200, Martti Kuparinen wrote:
>>> 
>>> I found http://proj.sunet.se/E2E/tcptune.html and played a little with
>>> the receiver (NetBSD 5.99.24), below 3 runs with both settings. The
>>> conclusion is that I'm happy now as I'm able to route 850+ Mbps through
>>> my router...
>> 
>> Changing the initial window is highly antisocial.  It's certainly not
>> good general-purpose advice for tuning TCP stacks and it's probably not
>> necessary to get the throughput you're looking for.
> 
> That was done a long time a ago with the goal and sole purpose to break the 
> Internet land speed record (http://proj.sunet.se/LSR2/index.html).
> Note, SUNET used iNTEL 10GbE prototype cards for this, and those card doesn't 
> perform very well with NetBSD today.
> 
> However, while testing those old prototype cards and some other NICs for 
> speed with NetBSD 5.0 RC...  we found that turning of sack would increase 
> throughput performance.
> 
> # Disable SACK, buggy with dge(4) 10GbE-LR?
> sysctl -w net.inet.tcp.sack.enable=0
> sysctl -w net.inet6.tcp6.sack.enable=0
> 
> I don't think anyone investigated it further to conclude if it where the NIC 
> or the NetBSD code that was buggy.
> You might give it a try and see what happens...

Sack is mostly of benefit on LFNs (Long, Fat, Networks).  On a LAN or pair of 
LANs, it won't matter much, I suspect.  SACK is "selective acknowledgment" -- 
it can resend just the missing segments, rather than everything from the 
missing segment on forwards.  But that's only of interest if there are packet 
drops, and with few routers, little congestion, and steady-state sending, those 
more or less should not happen very often.

Still, it won't hurt to try.

                --Steve Bellovin, http://www.cs.columbia.edu/~smb