Subject: Re: State of "jumbo frames" support?
To: Jeff Rizzo <riz@tastylime.net>
From: Jonathan Stone <jonathan@Pescadero.dsg.stanford.edu>
List: tech-net
Date: 11/25/2005 16:34:34
In message <4387A5BC.3000403@tastylime.net>Jeff Rizzo writes,

>Before I beat my head against this too much more, I thought I should ask
>here -
>
>what's the word on "jumbo frames" support in NetBSD?

Jumbo frames are a non-standard. Ergo, support for "jumbo frames"
depends on the vagaries of individual hardware chips; sometimes even
on variants within a family. Therefore it doesn' tmake sense to ask
about "jumbo frame" support in NetBSD, other than on a
device-by-device (or chip-variant by chip-variant) level.


> I was interested
>in adding it to the sk(4) driver, but wanted to get a look at what
>working support was like, first... i've been unsuccessful at getting it
>to work with a couple of re(4) boards (the man page says the chips don't
>seem to support past an MTU of 7500, but I wasn't able to get 4500 to
>work), or a couple wm(4) boards  (ping -s 2002 works, ping -s 2003 does
>not).

I don't recall if I wrote that, or if Wiz added it, or if it predates
the gigabit/cardbus chips; but (if memory serves) that's exaclty what
the 8169/8110S datahseets say.  From memory, the 8169/8110S can
receive up to 9000-odd bytes, but (due to a Tx FIFO limit) can
transmit at most 7500 bytes. (Or maybe I'm mixing up re(4) with that
absurd Netgear GA-620T...)


>I _have_ been able to get jumbo frames working with a bge(4) board, and
>with a pair of ti(4) boards I have.  (No great amount of testing, but
>with an MTU of 9000, "ping -s 8972" fits everything into a single frame,
>and gets a successful echo reply).

Hmm. Was it me who added jumbo support to our bge(4) driver? If it
was, I don't recall if it was ever disabled it for the non-"server"
PCI-e chips which don't support jumbo frames (or for the 5721, which
according to marketing datasheets can do jumbo frames, but requires
special massaging not, AFAIK, present in the Linux drivers or in any
*BSD bge(4).

>Any other reports?  Things I should watch out for?

Other than crashing non-jumbo-capable switches or other
non-jumbo-capable hardware?

Be careful when you mix jumbo frames with VLAN tags. Sometimes the
VLAN tag comes out of the jumbo limit.  (in other words, if you want
jumbo frames and VLAN-tagged packets, you must sometimes deduct the
VLAN tag size from your software-specified MTU). Quoting yet again
from memory, the jumbo-capable BGE devices are one case in point.

Also be aware that, since "jumbo" frames are a non-standard, there's
no standard agreement on jumbo-frame length. the bge(4) devices
support 9000 bytes. Other hardware aims 9128 bytes, presumably
insipred by the ATM LAN-emulation (LANE) MTU.  And then there's the
Intel cards, where some variatns supported jumbo frames up to 16k. 
(Good luck finding a switch which can handle _those_.)

Then again, if you've gotten as far as you say, quite possibly you
know all of the above already.