I've been using GSSAPI (Kerberos 5) with racoon successfully for some  
years, now. But I haven't been able to get it to work with the new,  
ipsec-tools version at all. Using the new default config (/usr/share/ 
examples/racoon/racoon.conf.sample-gssapi), I get startup messages  
like this (these are on a 3.0_STABLE machine, as it happens):

Apr 30 00:00:15 aten racoon: INFO: @(#)ipsec-tools 0.6.3-20051204  
(http://ipsec-tools.sourceforge.net)
Apr 30 00:00:15 aten racoon: INFO: @(#)This product linked OpenSSL  
0.9.7d 17 Mar 2004 (http://www.openssl.org/)
Apr 30 00:00:15 aten racoon: INFO: fe80::1%lo0[500] used as isakmp  
port (fd=9)
Apr 30 00:00:15 aten racoon: INFO: ::1[500] used as isakmp port (fd=10)
Apr 30 00:00:15 aten racoon: INFO: 127.0.0.1[500] used as isakmp port  
(fd=11)
Apr 30 00:00:15 aten racoon: WARNING: setsockopt 
(UDP_ENCAP_ESPINUDP_NON_IKE): Invalid argument
Apr 30 00:00:15 aten racoon: INFO: 2001:408:1002:a000::80[500] used  
as isakmp port (fd=12)
Apr 30 00:00:15 aten racoon: INFO: 2001:470:1f01:309::80[500] used as  
isakmp port (fd=13)
Apr 30 00:00:15 aten racoon: INFO: fe80::a00:20ff:fec1:ed14%hme0[500]  
used as isakmp port (fd=14)
Apr 30 00:00:15 aten racoon: INFO: 69.33.244.72[500] used as isakmp  
port (fd=15)
Apr 30 00:00:15 aten racoon: WARNING: setsockopt 
(UDP_ENCAP_ESPINUDP_NON_IKE): Invalid argument

OK, so far so good. The setsockopt warning is reportedly fixed on the  
trunk (although a racoon compiled from the trunk still shows that  
warning), but I don't think lack of NAT-T is causing me any grief  
here, since I don't care to do that, anyway. (As I understand it, I'm  
getting the warning because I don't have NAT-T configured in the  
kernel.)

Moving on from there, if I actually try to use any of my SPs, I see  
messages like this, on both sides:

Apr 30 00:09:03 aten racoon: INFO: IPsec-SA request for 69.33.244.71  
queued due to no phase1 found.
Apr 30 00:09:03 aten racoon: INFO: initiate new phase 1 negotiation:  
69.33.244.72[500]<=>69.33.244.71[500]
Apr 30 00:09:03 aten racoon: INFO: begin Identity Protection mode.
Apr 30 00:09:03 aten racoon: INFO: received Vendor ID: DPD
Apr 30 00:09:03 aten racoon: INFO: ISAKMP-SA established 69.33.244.72 
[500]-69.33.244.71[500] spi:74b92783a2a260cb:0527369e0d54d439
Apr 30 00:09:03 aten racoon: ERROR: Hybrid auth negotiated but peer  
did not announced as Xauth capable
Apr 30 00:09:03 aten racoon: ERROR: Attempt to start phase 2 whereas  
Xauth failed

Well, that's not so promising. I didn't ask for Xauth anywhere. I'd  
suspect that I didn't ask properly for gssapi in phase 2, but phase 2  
shouldn't care about gssapi, right? That's just used to set up phase  
1. My phase 2 configuration looks like this, unmodified from the  
sample config file:

sainfo anonymous {
         lifetime time 2 hour;

         encryption_algorithm rijndael, 3des;
         authentication_algorithm hmac_sha1, hmac_md5;
         compression_algorithm deflate;
}

What's going wrong?

There seem to be other problems, too. I've also discovered that the  
new ipsec-tools racoon goes into some sort of an infinite loop,  
filling /var/log/messages with many, many, messages, if you allow it  
to converse with an older, KAME racoon. Youch. Looks like I need to  
upgrade all my machines at once. And, of course, I need to get the  
ipsec-tools racoon working at all.

And finally, if you kill racoon on one machine, racoon on another  
machine notices (if there was a phase 1 association) and removes its  
SAs:

Apr 30 00:28:09 aten racoon: INFO: purging ISAKMP-SA  
spi=74b92783a2a260cb:0527369e0d54d439.
Apr 30 00:28:09 aten racoon: INFO: purged ISAKMP-SA  
spi=74b92783a2a260cb:0527369e0d54d439.
Apr 30 00:28:10 aten racoon: INFO: ISAKMP-SA deleted 69.33.244.72 
[500]-69.33.244.71[500] spi: 74b92783a2a260cb:0527369e0d54d439

Nice enough. That ought to prevent a lot of problems I've had in the  
past with stale SAs. But sadly, if the two machines try to  
communicate again within a short time (a few seconds), the racoon  
that wasn't restarted becomes unhappy. You just get this:

Apr 30 00:28:12 aten racoon: INFO: IPsec-SA request for 69.33.244.71  
queued due to no phase1 found.
Apr 30 00:28:12 aten racoon: INFO: initiate new phase 1 negotiation:  
69.33.244.72[500]<=>69.33.244.71[500]
Apr 30 00:28:12 aten racoon: INFO: begin Identity Protection mode.
Apr 30 00:28:12 aten racoon: ERROR: failed to allocate my sa buffer
Apr 30 00:28:12 aten racoon: ERROR: failed to begin ipsec sa  
negotication.
Apr 30 00:28:12 aten racoon: INFO: respond new phase 1 negotiation:  
69.33.244.72[500]<=>69.33.244.71[500]
Apr 30 00:28:12 aten racoon: INFO: begin Identity Protection mode.
Apr 30 00:28:12 aten racoon: INFO: received Vendor ID: DPD
Apr 30 00:28:12 aten racoon: ERROR: failed to allocate my sa buffer

And racoon has crashed, leaving a core file in /. If the machines  
wait a minute after the SA purge before trying to communicate again,  
then racoon is able to recover, and it doesn't crash.

But we still have the other, basic GSSAPI problem. So, what am I  
missing, here? Has anyone else been successful getting Kerberos key  
distribution to work with the new, ipsec-tools racoon?

Thanks,
- Geoff