Subject: Re: Hardware checksums slow down max throughput on ti(4) board?
To: None <current-users@netbsd.org>
From: David Laight <David.Laight@btinternet.com>
List: current-users
Date: 12/17/2001 21:10:08
>  > Using ttcp for some basic throughput testing, I find that without
>  > hardware checksumming (ip4csum, etc) enabled, I can get a max
>  > throughput of about 280Mbps on a TCP connection, and close to
>  > 800Mbps for raw UDP (with drops).  *With* hardware checksums enabled,
>  > max throughput is about 227Mbps for a TCP connection, and 175Mbps for
>  > raw UDP. (!)
> 
> This is a known bug in the Tigon-1 Gig-E chip.  The Tigon-1 doesn't
> have the horsepower to do normal Gig-E stuff + checksumming (it's a
> MIPS R4000 core).  The Tigon-2 has 2 cores in the chip, which made up
> for a lot of it.
> 
> Plus, the stock Alteon firmware is known to suck in a lot of ways; SGI,
> in their driver for that chip, heavily hacked the firmware to make it
> not suck.

It is also, to some extent, true that by the time you have worked out
how to tell the hardware how to calculate the checksum, you might have
well have done it yourself.

Similar problems arise trying to 'hand off' other parts of the protocol
stack to an 'intelligent' IO card.  You have to be VERY careful how
you do the kernel-card interface to get reasonable performance.
Latency almost certainly suffers, throughput and max connections get
limited by the card (so you can't just throw cpu and memory at them).
You end up having to (effectively) support separate protocol stacks
on multiple io cards.

(Did intel ever sell that PCI ethernet server card with an i860 on it?
They were touting around some enourmous infrastructure for io cards at
the time.
I had two samples filling my cupboard once.)

    David