Subject: Re: Very slow pipe/TCP connection in 1.6_BETA4
To: Havard Eidnes <he@netbsd.org>
From: Manuel Bouyer <bouyer@antioche.lip6.fr>
List: current-users
Date: 07/29/2002 12:35:36
Hi, sorry for the delay

On Tue, Jul 16, 2002 at 11:19:07AM +0200, Havard Eidnes wrote:
> > I am running a Amanda amrecover for a single file on a i386/1.6_BETA4
> > system and the througput through /dev/nrst0 >> amrestore >pipe> gzip
> > -d >TCP> /sbin/restore manages less than 10Kbytes/sec.
> >
> > A tcpdump shows that the ACK is sent 0.2 seconds after the packet it
> > acknowledges:
> 
> This indicates that you are only getting ACKs on timeouts for this TCP
> connection.
> 
> > netstat shows the Send-Q of the gzip end of the connection
> > always at 16384 byes, the Recv-Q of the restore side always
> > at 0 bytes.
> 
> This tends to indicate that the TCP window sizes are 16KB, at least on
> the sender side.  Now, I wonder how this interacts with the lo0 MTU of
> 33220 bytes.  Badly, is my best guess...  If your TCP window size is
> 16384 bytes, no TCP segment sent across this interface will ever be
> MSS-sized.  If you have Nagle's algorithm enabled on the connection it
> will interfere (it'll tend to hold on to segments), as will probably
> the ack strategy, as the general rule for sending ACKs is, I think, one
> ACK per 2 MSS or on timeout, and you're seeing the latter.
> 
> This could be some extreme form of the TCP performance anomalies
> earlier described in a paper by Kjersti Moldeklev (a fellow Norwegian),
> a pointer can be found on
> 
>   http://citeseer.nj.nec.com/update/224127
> 
> These problems are triggered when there is a mismatch between the MTU
> and the TCP window sizes.
> 
> However, simplistic testing on an i386/1.6B SMP system with ttcp, 16KB
> receive/send space with a connection across lo0 does not appear to
> trigger the problem, so I'm slightly puzzled.
> 
> If my theory holds some water, a possible short-term workaround could
> be to tweak /sys/net/if_loop.c to define LOMTU to something a little
> less than 1/3 of 16KB, e.g. (4096 + MHLEN + MLEN).  (Apparently the MTU
> of the loopback interface can not be reconfigured with ifconfig on the
> fly.)

Yes, this fixes the problem for me.

--
Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
--