I was toying with running ipsec in a way that would allow me to not
run ipf.  Basically the idea was to have a few specific "pass" rules
for a few udp and tcp ports and the rest of the connections would
default to requiring ipsec if they were local and would be plain if
external.

The setup was similar to my IPF where the specific rules would come
first and the more general ones would be a backdrop for the packets to
hit if they didn't get claimed upstream.  Clearly the ordering of the
rules is very important.

I was lead to believe that I could count on the spd rules to be
applied *in order*.  What I'm seeing is that spd rules don't really
work that way, at least in netbsd/kame.  Doing a "setkey -F; setkey
-FP" doesn't clean the slate.  Quite a large number of spd's are still
left laying around (perhaps because they still have a non-zero
refcount).  These old spd's are left at the head of the spd list and
any new spd's are appended to the end.  The upshot is a general rule
will invariably find its way to the top of the spd list claim all the
packets and non of the specific "bypass" type rules will ever match
again.  The only way to reliably clear the old spd's is to reboot.

Is this a bug?  Should I PR it?  Or is the effective re-ordering of
spd's simply something that is allowed.

-wolfgang

In case anyone cares, here is a slightly simplified example of what I
was trying to do:

# Out rules: 
#
spdadd 127.0.0.1 127.0.0.1        any -P out none;  # loopback
spdadd 0.0.0.0/0 0.0.0.0/0[22]    tcp -P out none;  # ssh
spdadd 0.0.0.0/0 192.83.197.0/24  any -P out ipsec esp/transport//use     ah/transport//use;      # all wsrcc local ether
spdadd 0.0.0.0/0 0.0.0.0/0        any -P out none;  # default: no encryption

# In rules: 
#
spdadd 127.0.0.1 127.0.0.1        any -P in  none;  # loopback - XXX: spoofable
spdadd 0.0.0.0/0 0.0.0.0/0[22]    tcp -P in  none;  # ssh
spdadd 0.0.0.0/0 192.83.197.0/24  any -P in  ipsec esp/transport//use     ah/transport//use;      # all wsrcc local ether

# this is what we want to say, but the ipsec interface doesn't let us.
#
# spdadd 0.0.0.0/0 0.0.0.0/0[49000-65535] tcp -P in  none;  # for our clients  # allow "client" range to pass
# spdadd 0.0.0.0/0 0.0.0.0/0        any -P in  ipsec esp/transport//use     ah/transport//require;  # default: require ipsec 

# this is how we have to say it.
# innumerate all the services that need ipsec:
spdadd 0.0.0.0/0 0.0.0.0/0[111]   any -P in  ipsec esp/transport//use     ah/transport//require;  # sunrpc
...
# and finally, the catch-all, allow any packet that isn't yet matched to pass
spdadd 0.0.0.0/0 0.0.0.0/0        any -P in  none;  # default: no encryption

-- 
Wolfgang S. Rupprecht 		     http://www.wsrcc.com/wolfgang/