Subject: wm(4) with i82546GB on an AlphaServer ES40
To: NetBSD/alpha Discussion List <port-alpha@NetBSD.ORG>
From: Greg A. Woods <woods@weird.com>
List: tech-net
Date: 03/11/2005 21:13:50
I've now got a dual-port optical i82546GB card installed in the ES40,
and testing with ~yesterday's -current (and if_wm.c rev. 1.99), _and_
with '#define _PCI_HAVE_DMA64' added to the alpha's pci_machdep.h, the
wm(4) driver still won't go very fast, even with "big" writes (and with
all the csum offloading enabled):

[console]<@> 2.99.16 # ./nttcp -T -l 16384 -n 10000 insecurity  
     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s   CPU-C/s
l163840000    4.74    4.11    276.3485    319.0750   10000   2108.37    2434.3
1163840000    4.74    0.54    276.3206   2427.2593   36157   7622.47   66957.4

However the good news though is that turning on _PCI_HAVE_DMA64 seems to
have no ill effect (though it doesn't seem to work any faster than
without, though I suppose with such simplistic tests that's not surprising).

Also good news is that I can pull data off the two FibreChannel RAID
arrays (each on its own isp(4) HBA), simultaneously, and push data out
the wm(4) interface, all without error and without too much degradation
in speed (about 20% less TCP throughput, but one isp(4) shares a
33MHz/64-bit PCI bus with the i82546GB card).  (and that's without
fiddling with any PCI latency timers :-)

Still pretty pitiful compared to the same i82546GB card in an Opteron
box with FreeBSD's Intel-authored driver.... (which can seemingly easily
fly along at full wire, er, laser, speed with 1500-byte packets) ;-)

Anyhow, I'm going to go into production on this machine next week with
1.6.2_STABLE, which at least does better than any 100baseT/FDX interface:

[console]<@> 1.6.2_STABLE # ./nttcp -T -l 8192 -n 10000 insecurity
     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s   CPU-C/s
l 81920000    3.96    1.81    165.5008    362.7088   10000   2525.34    5534.5
1 81920000    3.96    0.24    165.4743   2730.6667   24959   6301.99  103995.8

The throughput degradation with even one FiberChannel port going full
bore is much more drastic than with -current though....

At least I'll have a good guarantee of better, if still not stellar,
performance when we're ready to upgrade the OS again!  ;-)

-- 
						Greg A. Woods

H:+1 416 218-0098  W:+1 416 489-5852 x122  VE3TCP  RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>          Secrets of the Weird <woods@weird.com>