Subject: ipsec, path mtu, and fragmentation
To: None <tech-net@netbsd.org>
From: Daniel Carosone <dan@geek.com.au>
List: tech-net
Date: 02/06/2004 19:36:22
--QKdGvSO+nmPlgiQ/
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

What is presently the accepted/recommended way to avoid fragmentation
of ESP tunnel mode packets when the traffic being tunnelled is tcp?

I've just traced really lousy wireless performance after a recent
upgrade to this, because I protect my wireless by using ipsec.  I only
see the problem in on direction/case, which is why it took me a while
to even realise there was an issue.

The relevant portion of my setup is like this:

  {I}
     \
      [G]--[AP] /\/\/ [W]
     /
  [L]

Host L on a wired LAN (with other boxes)
Host W in wireless, via an AP
Host G an ipf/ipsec gateway/router
Host I on the general Internet cloud

I have IPsec tunnel mode between G-W for traffic both to/from the LAN with
L, and for traffic to/from the internet.  I use IPsec transport mode
for traffic between hosts on the wireless lan, including W and G.

All boxes (except some I's, alas :) are NetBSD-current, I recently
upgraded G from a few-months-old -current.

Now, when L or I sends W large packets, they get encapsulated by G and
then the ESP packets get fragmented, and TCP throughput is truly
abysmal.  When W sends large packets, they are not fragmented as far
as I have seen; certainly the throughput is as it was before G was
upgraded.

I'm not sure really what changed - the performance impact of
fragmentation, or the fragmentation itself, or both.  For the sake of
experiment, while a large file copy from L->W was running, I changed
the MTU of L'd ethernet interface from 1500 -> 1430 to avoid the
fragmentation at G, and suddenly the transfer speed shot up massively
- not just to where it used to be before G was upgraded, but even
higher (~500K/s where previously I'd get ~350K/s). I was getting
~15K/s with fragmentation.

Fiddling with net.inet.ipsec.dfbit on G seems to have no effect, not
that I really expected it to (since that would be for other gateways
with narrow MTU's between G and W).

I also have another setup where I effectively use gif tunnels
encapsulated in ESP transport mode. There are a number of reasons to
prefer that setup, and now I have another: I could set the MTU of
gif0, and I would expect that to be properly exposed to
PMTU-discovery.  But that is not this setup, and does have some other
restrictions, especially since gif no longer has multipoint.

So, for the straight ESP-tunnel setup, what is the "official"
configuration to avoid the fragmentation at all, by exposing the
narrower tunnel-MTU to TCP and PMTU-discovery?=20

(If there's a new performance problem with ESP fragmentation, that
also needs to be looked into).

Is the FAST_IPSEC code any different in this behaviour? I haven't yet
tried it.

--
Dan.
--QKdGvSO+nmPlgiQ/
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (NetBSD)

iD8DBQFAI1IFEAVxvV4N66cRAuzOAJ0VY/MngXJsLaO89yNqxBqgoLDkTQCeMS5h
fJF2Uuchudtx+It71aly0Rk=
=FuHI
-----END PGP SIGNATURE-----

--QKdGvSO+nmPlgiQ/--