Re: X520-DA2 10Gbps

[David Young <dyoung%pobox.com@localhost>]

On Fri, Jun 14, 2013 at 09:20:32AM -0300, Wayne Marsh wrote:
> Dell R410 with Intel X520-DA2 -> Force10 switch -> Dell R810 with Intel 
> X520-DA2
> R410 has NetBSD 6.0 RC 2
> R810 has CentOS Linux 6.4
> The Force10 model escapes by but has 48 10Gbps ports + 4x 40Gbps ports
> We're trying all benchmarks using iperf.
> 
> When I first modified the driver for this card, I enabled Jumbo
> Packets on the 410 by replacing the FreeBSD predefined constant with
> (1024 * 9), which is what the definition in FreeBSD was.  My second
> modification to the driver was changing the number of descriptors,
> replacing the FreeBSD predefined constant again when setting the
> number of descriptors from 1 to 4 - again replacing MJUM9BYTES with
> (1024 * 9) - which raised the maximum TCP window size from 128K to
> 512K.  This seems to not affect speed at all.

I don't see the connection between the driver constants that you are
modifying and the TCP window size.

Have you enabled any offload features such as ip4csum?  Those make a
difference.

Have you tuned up TCP and the maximum socket buffers?  For example,
I use these settings on my *1*-gigabit hosts:

        net.inet.tcp.sendspace = 262144
        net.inet.tcp.recvspace = 262144
        kern.sbmax = 16777216

How have you invoked iperf on each end?  Typically you have to tune it
up, too.

Dave

-- 
David Young
dyoung%pobox.com@localhost    Urbana, IL    (217) 721-9981