IPFilter and policy routing?

[Urban Boquist <urban%boquist.net@localhost>]

I have a machine that needs to be reachable on two public IPs
(connected to two different ISPs) during a transition period. So I
need a way to force some packets out a certain interface rather than
to take the default route. After some googling it seems that IPFilter
should be able to do this, with rules like:

pass out quick on fxp0 to tlp0 from 172.16.1.1/32 to any

As far as I understand this should make a package that is about to go
out on fxp0 and matching the address, instead be re-routed to tlp0?

Problem is I can't get this to work...

The following experiment is between two NetBSD-4.0 hosts, trying to
mimic the situation with two IPs on the "amanda" host.

Host gw:

gw# ifconfig hme0 192.168.1.1
gw# route add -net 172.16.1 -interface 192.168.1.1

Host amanda:

amanda# ifconfig fxp0 192.168.1.27
amanda# route add default 192.168.1.1
amanda# ifconfig tlp0 172.16.1.1

amanda# ifconfig fxp0 
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        address: 00:d0:b7:ac:11:94
        media: Ethernet autoselect (100baseTX 
full-duplex,flowcontrol,rxpause,txpause)
        status: active
        inet 192.168.1.27 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::2d0:b7ff:feac:1194%fxp0 prefixlen 64 scopeid 0x1
amanda# ifconfig tlp0
tlp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        address: 00:80:ad:71:85:82
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 172.16.1.1 netmask 0xffff0000 broadcast 172.16.255.255
        inet6 fe80::280:adff:fe71:8582%tlp0 prefixlen 64 scopeid 0x2

Default route is on fxp0. But now I want the reply to packets coming
in on tlp0 to also go out via tlp0, so I setup IPFilter like this, the
test service is daytime:

amanda# ipfstat -io
pass in quick proto tcp from any to any port = daytime keep state
pass out quick on fxp0 to tlp0 from 172.16.1.1/32 to any

The experiment now becomes: gw# telnet 172.16.1.1 daytime

This "works" and gives the expected output on gw. But the problem is
that the reply packets from amanda still goes out via the fxp0
interface (see fxp0 ether address below). In the real world scenario I
would expect the ISP to drop these outgoing packets, since they are
coming from an incorrect source address (looks spoofed).

gw# tcpdump -e -i hme0 host 172.16.1.1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on hme0, link-type EN10MB (Ethernet), capture size 96 bytes
10:10:39.353052 08:00:20:a7:12:85 > 00:80:ad:71:85:82, ethertype IPv4 (0x0800), 
length 78: IP gw.boquist.net.65033 > 172.16.1.1.daytime: S 
398595490:398595490(0) win 32768 <mss 1460,nop,wscale 
0,sackOK,nop,nop,nop,nop,timestamp 0 0>
10:10:39.353348 00:d0:b7:ac:11:94 > 08:00:20:a7:12:85, ethertype IPv4 (0x0800), 
length 78: IP 172.16.1.1.daytime > gw.boquist.net.65033: S 
2671644190:2671644190(0) ack 398595491 win 32768 <mss 1460,nop,wscale 
0,nop,nop,timestamp 0 0,sackOK,nop,nop>
10:10:39.353483 08:00:20:a7:12:85 > 00:80:ad:71:85:82, ethertype IPv4 (0x0800), 
length 66: IP gw.boquist.net.65033 > 172.16.1.1.daytime: . ack 1 win 33580 
<nop,nop,timestamp 0 0>
10:10:39.353809 00:d0:b7:ac:11:94 > 08:00:20:a7:12:85, ethertype IPv4 (0x0800), 
length 92: IP 172.16.1.1.daytime > gw.boquist.net.65033: P 1:27(26) ack 1 win 
33580 <nop,nop,timestamp 0 0>
10:10:39.353835 00:d0:b7:ac:11:94 > 08:00:20:a7:12:85, ethertype IPv4 (0x0800), 
length 66: IP 172.16.1.1.daytime > gw.boquist.net.65033: F 27:27(0) ack 1 win 
33580 <nop,nop,timestamp 0 0>
10:10:39.353962 08:00:20:a7:12:85 > 00:80:ad:71:85:82, ethertype IPv4 (0x0800), 
length 66: IP gw.boquist.net.65033 > 172.16.1.1.daytime: . ack 28 win 33554 
<nop,nop,timestamp 0 0>
10:10:39.361408 08:00:20:a7:12:85 > 00:80:ad:71:85:82, ethertype IPv4 (0x0800), 
length 66: IP gw.boquist.net.65033 > 172.16.1.1.daytime: F 1:1(0) ack 28 win 
33580 <nop,nop,timestamp 0 0>
10:10:39.361523 00:d0:b7:ac:11:94 > 08:00:20:a7:12:85, ethertype IPv4 (0x0800), 
length 66: IP 172.16.1.1.daytime > gw.boquist.net.65033: . ack 2 win 33580 
<nop,nop,timestamp 0 0>

I did a quick experiment with PF too and it seems that PF does exactly
what I want with this rule:

pass in quick on tlp0 reply-to { tlp0 } from any to any keep state

But still, it would be interesting to know if this is possible or not
with IPFilter.

Thanks in advance,

    -- Urban