Subject: problem with ipsec tunnel fragmentation
To: None <current-users@netbsd.org>
From: Brett Lymn <blymn@baesystems.com.au>
List: current-users
Date: 10/15/2007 21:25:18
Folks,

I am building a VPN solution for a client based on a NetBSD 4.0rc1
server with (sorry) Windows XP clients.  I have succeeded in building
everything and have it working the way my client wants - the ipsec
tunnels work fine for the most part, they come up and traffic flows as
it should

The problem I have is that if a packet gets larger than the esp
fragment setting in racoon.conf (esp_frag) then I start seeing
problems.  I have done a packet capture on the Windows client when it
is connecting to a machine on the other side of the vpn server - I can
see the packets flow between the client and the other machine fine
until the other machine decides to send a packet that is larger than
than esp_frag.  At this point I see some tcp fragments that originate
from the external facing address of the vpn server directed to the vpn
client tunnel end point.  These packets are rejected because they are
out of state.  Of course, at this point, the connection through to the
other machine stalls and eventually dies.

Both client and server machines are behind NAT devices and the vpn
server performs a NAT of the networks behind it to the address of it's
externally facing interface (this could be muddying where the frags
come from - I didn't check if they had been nat'ed but I can do so) I
also perform a mss clamp on the packets going out.  I tried playing
with net.inet.ipsec.dfbit - it was set at 2, setting it to 1 seemed to
have no effect.

I have bumped esp_frag up high enough that it is currently not causing
an issue for my client but I would like to find a more correct fix for
the problem.

-- 
Brett Lymn
"Warning:
The information contained in this email and any attached files is
confidential to BAE Systems Australia. If you are not the intended
recipient, any use, disclosure or copying of this email or any
attachments is expressly prohibited.  If you have received this email
in error, please notify us immediately. VIRUS: Every care has been
taken to ensure this email and its attachments are virus free,
however, any loss or damage incurred in using this email is not the
sender's responsibility.  It is your responsibility to ensure virus
checks are completed before installing any data sent in this email to
your computer."