On Thu, Feb 24, 2005 at 12:14:35PM -0500, Jonathan Kay wrote:
>  
> the wm card is onboard, so can't really yank it, but if I were creative
> with a soldering iron.. :-)
> 
> looking up whats actually in the 600sc, it has 1 pci bus--64/bit 33mhz.
> oops.  though fwiw, the dell PE 1600sc (which has 3 pci buses: 1 PCI-X,
> 1 @ 64/66mhz and 1 and 32/33mhz) has an onboard wm0 which claims to be
> runninat @ 32/33mhz too.

Maximum throughput for a 64-bit, 33Mhz PCI bus is 264MB/sec.  In the real
world, with reasonable latency you will probably see more like 160MB/sec
throughput.  But when you put 32-bit devices into the mix, you have to
remember that every cycle they're on the bus, a 64-bit device can't
transfer.  So the throughput will be somewhere between 80 and 100 MB/sec
but it's hard to say where exactly.

Let's pretend that every transfer goes from the mpt card, into main
memory, and then out to the wm.  So, every other transfer will be
32-bit, so we can split the difference and land around 120MB/sec
expected maximum performance.  If you're moving anywhere near that
much data on or off the Xserve RAID I would not find the problem you're
having at all surprising -- though the wm driver should recover much
more gracefully from the error in question so there's clearly a bug
there that needs to be investigated.

You should still win by adding a 64-bit wm on a card to the system (it
will let you use more of the available PCI bandwidth by halving the
cost of each transfer to/from the ethernet).  Also, it would be a
good idea to yank the 3ware controller if you aren't using it; maybe
something is not going quite right with the PCI bus latency calculations
and removing devices from the mix cannot hurt.

Thor