[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: NetBSD 5.1 TCP performance issue (lots of ACK)
On Wed, Oct 26, 2011 at 08:15:44PM -0400, Greg Troxel wrote:
> The speed
> with glusterfs: seems to be a combination of 74 MB/s and pauses
> with ttcp: seems to be 112 MB/s burst (for 0.2s) and some smaller
> pauses, which gut checks with 95 MB/s as reported by ttcp.
> So are you seeing high 40s MB/s out of glusterfs?
Yes, between 40 and 50MB/s
> What is between these two devices? Is this just a gigabit switch, or
> anything more complicated?
they're all (the 2 NetBSD and the linux host) connected to a cisco 3750
gigabit switch. I also tested with a single crossover cable, this doens't
change anything .
> We are seeing reordering which I would not
> expect on an ethernet. I wonder if the tso4 option is causing that.
> What if you turn off the offload options? (I realize it may slow down,
> but if both are then equal, that's a clue.)
that's easy. And yes, I get better performances: 77MB/s instead of < 50.
So it looks like we have something wrong with TSO.
The traces are still at ftp://asim.lip6.fr/outgoing/bouyer/
Did you see the reordering in the ttcp trace too ?
But, that still doesn't explain why I get good performances when one
of the host is linux. NetBSD used tso as well, and it didn't seem to cause
problems for linux ...
BTW, how is TSO working ? does the adapter get a single data block of
a full window size ? if so, maybe the transmit ring just isn't big
> I wonder if the very large buffers get full and that causes cache
> thrashing. What happens if you change gluster to have smaller buffers
> (I don't understand why it's ok to have the FS change the tcp socket
> buffer options from system default)?
Because it knows the size of its packets, or its internal receive buffers ?
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
NetBSD: 26 ans d'experience feront toujours la difference
Main Index |
Thread Index |