kern/42592: PFkey interface causing problems with recent racoon

[d.zebralla%ape-net.com@localhost]

>Number:         42592
>Category:       kern
>Synopsis:       PFkey interface causing problems with recent racoon
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jan 08 14:05:00 +0000 2010
>Originator:     Daniel Zebralla
>Release:        NetBSD-5.0-release
>Organization:
A.P.E. GmbH
>Environment:
NetBSD GW-B 5.0_STABLE NetBSD 5.0_STABLE (GW-B) #11: Mon Jan  4 11:50:37 CET 
2010  
zebralla@sirrug:/home/zebralla/GW-B/obj.build/sys/arch/i386/compile/GW_B_NB5 
i386

>Description:
We've tried to work with racoon-current (as from 2009-12-14) in combination 
with NetBSD-5.0-release Kernel and encountered problems in establishing an 
IPsec-connection in tunnel mode between two GWs (both with racoon-current and 
NetBSD-5.0-release Kernel). 

When the initiator (GW-A) tried to establish an IPsec-tunnel, racoon on the 
responder-side (GW-B) stated that both, Phase1 (ISAKMP-SAs) and Phase2 
(IPsec-SAs) were established successfully, and the resulting SAD- and 
SPD-entries looked like expected. However, trying to ping a machine inside 
GW-B's LAN out of GW-A's LAN, we saw that racoon on GW-B always stated 
"initiate new Phase2 negotiation ..." and that new Phase2-SAs for the same 
connection were established. This behavior repeated itself every ~20 seconds 
without a established tunnel being ever usable.

We saw a mailing list entry by Brett Lymn on ipsec-tools-devel stating the same 
issue but with NetBSD-current and racoon-current as of July 2009 and a slightly 
different config (Checkpoint FW-1 FW as initiator instead of NetBSD/racoon and 
NAT-T activated) [1]. 
Judging by Yvans' reply, the problem seems to be inside the kernels 
PFkey-interface [2] and also affects NetBSD-current: apparently racoon was 
changed to require some changes in the PFkey interface.
While this change was made on Linux and FreeBSD, it was never done on NetBSD.

In effect, NetBSD-current's raccoon is probably non-working today due to the 
lack of this PFkey change.

Even a PR (#41757) was opened back then which is still open [3].

We've also posted this issue onto tech-net mailing list recently [4].

[1] 
http://sourceforge.net/mailarchive/message.php?msg_id=20090725215420.GA3745%40internode.on.net
[2] 
http://sourceforge.net/mailarchive/message.php?msg_name=20090729195853.GA5366%40zeninc.net
[3] http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=41757
[4] http://mail-index.netbsd.org/tech-net/2009/12/18/msg001803.html

>How-To-Repeat:
Use NetBSD-5.0-release kernel and racoon-current (as from 2009-12-14) on two 
GWs and let them establish an IPSec-connection in tunnel mode.
>Fix:
The problem can be worked around by using racoon from NetBSD-5.0-release but 
the newer racoon features that we wanted to use (having dedicated remote 
sections In racoon.conf for separate peers identified by something different 
than the peers' IP addresses) are not available there.