Subject: Re: sk(4) jumbo frames support - patch
To: None <tls@rek.tjls.com>
From: Jeff Rizzo <riz@tastylime.net>
List: tech-net
Date: 11/27/2005 14:01:22
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigB473A2208E3B6F08D8A64521
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Thor Lancelot Simon wrote:

>On Sun, Nov 27, 2005 at 08:58:19AM -0800, Jeff Rizzo wrote:
>  
>
>>With this patch applied, I was able to use 9000 byte frames with a DLink
>>DGE-530T board, and combined with lowering the interrupt moderation timer,
>>I was able to improve tcp throughput of a simple test by about 15% over
>>1500-byte frames.
>>    
>>
>
>Two notes here:
>
>1) You might want to look at increasing the mbuf cluster size and see
>   what kind of performance you get (or does this driver use a private
>   allocator for jumbograms?)
>
>  
>

This does nearly verbatim what the bge(4) driver does - uses its own
allocator & bus_dmamem_alloc().  Plus, I've been having some issues with
messing around with mbuf cluster size - MCLSHIFT of 13 on my macppc box
creates a non-booting kernel.  :(  Plus, one of my wm(4) cards panics
the kernel unless MCLBYTES is 2048.  (which I don't understand yet, and
haven't had time to investigate - thus far, I've had much better success
in general with intel's "em" driver for those boards)

>2) It's been widely noted that MTU 9000 is harmful: it causes an extra
>   page to be allocated which is left almost empty, for an extra 50% VM
>   system overhead for a tiny decrease in encapsulation overhead.  You
>   might want to reduce the MTU to 8K minus the header length and see
>   if performance goes _up_ -- it did for me in my test when I turned on
>   jumbo support in the wm driver.
>
>  
>
I hadn't thought about this issue, but I think you're right - I'd
actually been using an MTU of around 8000, because my bge(4) board seems
to have trouble with anything past 8998 bytes anyway (not sure whether
it's a hardware or a driver issue), so I'd stopped it down for the
throughput testing.  This may be where a lot of the performance boost
came from.

You note testing jumbos with wm(4) - I haven't had any success there
yet.  I have a couple of i82540EM boards which only seem to work up to
about 2k bytes with wm(4), but work fine with em(4), and an i82542 which
does the same.  What boards have you worked with jumbos using the wm driver?

Also, I got a private mail noting that my jumbo support patch has a
memory leak - I'm investigating, and will have a new patch soon.

+j



--------------enigB473A2208E3B6F08D8A64521
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQCVAwUBQ4ostrOuUtxCgar5AQMt/wQAmp+RqwiAkF8QxsVJoxGu0iM/s+LDYv+6
GkXiEp+PUDPWB+MF3Fokkwgkn4Bp8oxw9p9cHfYvZyAAvZOEdK0EIrZQghLpyYK2
blVY+8tMxjWBtZIo1KbwQ3/iC+2tyYIPVbt4OSPEQcD2kBIPbpuMeRBXfzca9bYC
jN4tYUx7Mqo=
=7jxD
-----END PGP SIGNATURE-----

--------------enigB473A2208E3B6F08D8A64521--