Interactions of NAT, gre(4) and IPsec [was: Re: reverse processing order: NAT, IPsec ?]

[Hubert Feyrer <hubert%feyrer.de@localhost>]

Short: Does anyone use a gre-with-nat-over-ipsec setup that they'd like to
        share the config?

Long:

I've made another stab at this, and run into some unexpected effects.
The setup I have (full version appended below):


   home-LAN -> RouterA -> Internet -> RouterB -> office-LAN
                    \                  /
                     \________________/
                         gre Tunnel


I have a gre(4) tunnel between the home-LAN and the office-LAN, using 
inside addresses of RouterA and RouterB as the gre(4) endpoints.
This follows the "create link between two private networks" example in 
gre(4).

Adding NAT on RouterA's gre interface went fine: Changes that I made are 
using the NATted address for RouterA's endpoint in both gre configs, 
setting a bimap rule in IPfilter, and adding a route to RouterB to route 
the NAT network (not the home network) to RouterA. With that, I see NATed 
packets arrive at RouterB, and routing to the office-LAN works like a 
breeze in both directions.

After adding NAT was surprisingly painless, I expected that securing the 
transport for the GRE tunnel with IPsec would be too: adding 
point-to-point encryption between the external addresses of the routers. 
For starters I chose static keys via setkey(8), no racoon (yet). As the 
gif(4) interfaces aren't involved in encrypting the point-to-point 
connection between the routers, I didn't expect any problems.
Wrong assumption!

What I see is sort of a routing loop between the routers where RouterB
sends an answer to RouterA, but RouterA doesn't know what to do with
the packet, so sends it back to RouterB. Which thinks it's for Router A, 
which loops a number of times.

Playing with the link1 flag on the gre tunnels, adding more routes and NAT 
rules on RouterA didn't result in any useful results (yet), and again I 
wonder if this is some funny interaction between NAT and IPsec?

Does anyone use a gre-with-nat-over-ipsec setup that they'd like to share 
the config?


 - Hubert


P.S.:

Here is the detailled config that I use for NAT & GRE without IPsec
(I'm doing this inside a VMware Team environment for now, hence the
funny "Internet" IPs):


          Home-LAN           Internet            Office-LAN
Home-PC  ---------- RouterA ----------- RouterB ------------ Office-PC
192.168.11.3  Home: 192.168.11.2      Office: 10.22.22.1     10.22.22.3
              Internet: 10.11.11.2    Internet: 10.11.11.1
                      \                  /
                       \________________/
           gre0: 10.253.253.2      gre0: 10.22.22.1
           NAT: 192.168.11.0/24
             -> 10.253.253.0/24


Config Home-LAN: 192.168.11.0/24, NATted to 10.253.253.0/24
Config Office-LAN: 10.22.22.0/24
Internet (Transport network): 10.11.11.0/24

Config Home-PC:
 * rc.conf:
        ifconfig_pcn0="192.168.11.3/24"
        defaultroute="192.168.11.2"

Config RouterA:
 * rc.conf:
        ipfilter=yes
        ipnat=yes
        auto_ifconfig=no
        net_interfaces="pcn0 pcn1 gre0"
        ifconfig_pcn0="192.168.11.2/24"
        ifconfig_pcn1="10.11.11.2/24"
        ifconfig_gre0="10.253.253.2/24 10.22.22.1 tunnel 10.11.11.2 10.11.11.1 
link1"

 * route.conf:
        default 10.22.22.1

 * ipf.conf:
        pass in from any to any
        pass out from any to any

 * ipnat.conf:
        bimap gre0 192.168.11.0/24 -> 10.253.253.0/24

Config RouterB:
 * rc.conf:
        auto_ifconfig=no
        net_interfaces="pcn0 pcn1 gre0"
        ifconfig_pcn0="10.22.22.1/24"
        ifconfig_pcn1="10.11.11.1/24"
        ifconfig_gre0="10.22.22.1/24 10.253.253.2 tunnel 10.11.11.1 10.11.11.2 
link1"

 * route.conf:
        net 10.253.253.0/24 10.253.253.2

Config Office-PC:
 * rc.conf:
        ifconfig_pcn0="10.22.22.3/24"
        defaultrouter="10.22.22.1"

 * route.conf:
        net default 10.22.22.1

Test:
 * On Home-PC: ping 10.22.22.3
 * On Office-PC: ping 10.253.253.3
 * Both testcases work with the above configuration, but fail
   with a routing loop where RouterA sends back packets to RouterB
   instead of performing NAT and forwarding them to the home-LAN when
   adding SAs and SPDs for encryption between 10.11.11.1 and 10.11.11.2.