Now I have installed 1.5.2 on my VAX I have tried again to establish
IPsec communications. Earlier, with 1.5.0 they failed and it seemed it
was because the VAX side didn't decrypt properly while racoon was trying
to establish keys.

Now my results are slightly different.

With my first, -O2 kernel which is the default, racoons on both sides
complained about hash mismatches. Keys were not established.

With a later -O1 kernel, the racoons appeared to manage to establish
agreement, and I started to see ESP packets on the wire when I was
pinging. However, they were still not accepted by the VAX. But it was
generating them apparently correctly, since I saw responses to them from
the alpha. On the alpha, the positive counters from netstat -s -p ipsec
were going up (but more slowly I think than the number of packets I
saw). On the VAX, counters for netstat -s -p ipsec were all 0 except AH
and ESP input histograms which showed many apparently random values for
#nnn and very large counters for them.

The spi that is used for the alpha-> vax direction is not known to the
vax (according to setkey -D). The VAX-side syslog shows the message
"racoon: ERROR: pfkey.c:217:pfkey_handler(): pfkey UPDATE failed: Invalid
argument" which I recall as the apparent showstopper from earlier 1.5
times.

I am now compiling a -O0 kernel to see if it makes more difference.

-Olaf.
-- 
___ Olaf 'Rhialto' Seibert - rhialto@polder --They that can give up essential 
\X/ land.nl --liberty to purchase a little temporary safety
--------------deserve neither liberty or safety. - Benjamin Franklin, 1759