Subject: Re: freebsd em driver
To: Frank van der Linden <fvdl@wasabisystems.com>
From: Jonathan Stone <jonathan@DSG.Stanford.EDU>
List: tech-net
Date: 06/17/2002 13:14:25
In message <20020617204936.A13044@vaasje.org>Frank van der Linden writes
>Wire speed is Hard to obtain still. 

Nope.

>Numbers vary greatly, especially
>with gig-e over copper. If you're doing 50-60 MB/sec over IP, userspace
>to userspace, and using normal framesizes, you're doing alright.

Huh? Dunno about you, in my world, that's doing rather poorly.


>That's still only half the wire speed. If you want to go higher,
>you'll need jumbo frames, 

No, you *don't* need jumbo frames.  (I'm sure you mean well,
but there's no need to spread disinformation.)

ttcp (with the larger windows/send-sizes I've been personally using
for a decade) with an Intel gig-e card and the FreeBSD drivers, can
reproducibly get on the very close order of 650 Mbit/sec (I measured
637Mbit/sec, iirc), against a Broadcom in a similar FreeBSD box.

Two Broadcom cards in similar hardware, with FreeBSD 4.5, gets
somewhere in excess of 850 Mbit/sec. (my lab book says 895 mbit/sec;
I've seen claims of 900-950 Mbit/sec out-of-the-box from FreeBSD users.)

Let's be generous and assume that Mbyte/sec is M = 1024*1024.
That means 50 Mbyte/sec, userspace to userspace, is

	8 * 50 * 1024 * 1024 / (1000 * 100)

or some 419 Mbit/sec, as reported by ttcp. Which is slightly more than
a factor of 2 off the competition.  That's not doing well in my book,
not well at all.


> and a faster/wider PCI bus. 

Uh, faster than 64/66?  That's commonplace now; either faster or wider
(or both) is pretty much a pre-requisite for getting sensible numbers
with gig-e.  Preferably both, when you consider that PCI-X is already
here.


>> (I'm about ready to port Bill Paul's bge driver.)

>I have already done this, but I want to clean up a few things
>before committing it, which should happen later this week. 

Uh, ``me too''. i've been playing with it since the last time drivers
for the Intel chips came up. my code is nowhere near ready to go
intree (even if it were, I imagine yours has been tested on non-i386
hardware.)


> If you want to test it before then, I can send you a tarball.

That'd be interesting, esp. if I can collar enough machines to run
some head-to-head tests.

Also, there are some bug workarounds we should get into the driver.
Do you have access to broadcom docs? Or workarounds for the various
hardware bugs (checksum offload, 5701 DMA bugs, or the patch for 5701
firmware, as uploaded by various Linux drivers?)