Currently my racoon.conf stands as:

path pre_shared_key "/etc/racoon/psk.txt";

remote anonymous {
        exchange_mode main;
        lifetime time 12 hour;
        initial_contact off;
        proposal_check strict;
        passive on;
        proposal {
                encryption_algorithm rijndael 256;
                hash_algorithm sha1;
                authentication_method pre_shared_key;
                dh_group modp2048;
        }
}

sainfo anonymous {
        pfs_group modp2048;
        lifetime time 6 hour;
        encryption_algorithm rijndael 256;
        authentication_algorithm hmac_sha1;
        compression_algorithm deflate ;
}

EOF

psk.txt contains the right IP for the other endpoint and a 256-bit key
preceeded by 0x - is this fed in exactly to the phase 1 negotiation, or
is it treated as a string? I've always been curious about that.

Everything works, I was just curious about ways to maximise the security
of the key negotiation. Well, besides requring huge D-H sizes and
reducing the lifetime to mere seconds. For instance: how much more
secure would certificate-based exchanges be than PSKs? My assumption on
the theory of PSKs is that a random key used to encrypt another random
key is perfectly secret, since the attacker has no indication whether
they caught the right 'plaintext' in a brute force session - since any
key is just as probable as any other. So by that token, in terms of
protecting the keys being exchanged, as long as your PSK isn't known, it
will never be cracked and so won't the keys - unless they are cracked by
investigating the actual ESP packets until it is found that way.

Is this correct, or am I oversimplifying things? Would using
pubkeys/certificates resolve any security issues there, or is it really
only more useful for authenticating your clients?

And is there any larger bit length for Rijndael than 256? 512 was called
out-of-range on my attempt.

Is there a way NOT to use compression? 'none' didn't work, and I really
don't see a need to add more overhead to this link.

Any more secure hashing algorithm than sha1 for hmac? I can use
ripemd160 in NetBSD but Linux doesn't support it, and my laptop client
is Linux (for hardware support reasons).

Any algorithm recommended more than Rijndael, that both NetBSD and Linux
2.6.11 support? I've always trusted my life to Rijndael, but if anyone
has found any even theoretical attacks against it I might reconsider.

What do the D-H calculations actually govern? The effectiveness of the
keys or something else? It wasn't clearly documented in racoon.conf(5)
and Google didn't get me anything useful quickly, but I trust people on
this mailing list.

This is really more for educational value than anything: right now it's
securing a rarely-used wireless link that doesn't do much more than surf
a bit. But I'm hoping to learn these vital things so I can move on to
bigger tasks that really do demand trustworthy security.

Thanks in advance to any answers received,
Dmitri Nikulin

PS: For reference, the install is a NetBSD 3-beta, so it has the new
ipsec-tools.