Subject: Re: IP-in-TCP?
To: Seth Kurtzberg <>
From: Daniel Carosone <>
List: tech-net
Date: 02/02/2005 18:51:17
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 02, 2005 at 12:16:29AM -0700, Seth Kurtzberg wrote:
> I think if you get into a situation with packet disassembly/reassembly,=
> you would have all the problems you described plus a mess at that=20
> level.  So choosing a packet size that guarantees that this never=20
> happens is a good idea. =20

Sure, as with any other tunnel, you will want to make the MTU of the
tun(4) interface small enough that a single tun packet, plus outer TCP
headers and whatever other framing you end up adding yourself, if any,
fits across the tunnel path without further fragmentation, of course.
Normal PMTUD then applies and exposes this narrow link to the endpoint
TCPs, no special work required from the tunnel daemons.

> Of course you care about both the maximum burst rate and the maximum=20
> steady state rate.

Under load, the multi-tunnel winds up looking like a link segment with
no loss, possibly high latency, and large buffers.  Using smaller
socket buffers for each tunnel stream might well be a good idea. You
could get fancier, and use nonblocking socket I/O and send(), or
select/poll and SO_SNDLOWAT, on the tunnel sockets, with small
buffers, to try and make sure that tunnel streams don't have multiple
outstanding packets.

All of which is about trying to recreate the independent ordering of
tunnelled packets over the TCP tunnels.  You have to rely on the
end-end TCP for congestion control, making sure to give it useful
information by dropping packets at the tunnel if buffers are full,
like any other router.


Content-Type: application/pgp-signature
Content-Disposition: inline

Version: GnuPG v1.4.0 (NetBSD)