Re: NetBSD 5.1 TCP performance issue (lots of ACK)

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Oct 27, 2011 at 04:02:12PM +0200, Manuel Bouyer wrote:
> > Do you have actual problems if gluster doesn't force the buffer to be
> > large?
> 
> that's interesting: I now have 78MB/s with tso4, and 48MB/s without
> tso4. Just as if the setsockopt would turn tso4 off.

Even more interesting: without changes on the linux side, a linux client
gets only 25MB out of the NetBSD server without setsockopt (it gets 95MB/s
when the netbsd server sets the snd/rcv buf size),
and the NetBSD client without setsockopt gets only 73MB/s out of the linux
server (it gets 95MB/s)

I'll sumarize this in the table below (all hosts are using tso4 and
no large receive offload)

server / client         NetBSD          NetBSD          Linux
                        no SND/RVCBUF   with SND/RVCBUF with SND/RVCBUF
NetBSD no SND/RVCBUF    78MB/s          49MB/s          25MB/s
NetBSD with SND/RVCBUF  52MB/s          49MB/s          95MB/s
Linux with SND/RVCBUF   73MB/s          99MB/s

I wonder if it could have something to do with the handling of PUSH in tcp.
With large socket buffers, data may be delayed for longer before being made
available to userland process.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--