Re: carp and routing

[Ryota Ozaki <ozaki-r%netbsd.org@localhost>]

Hi,

I fixed an issue of CARP in -current, which is a regression between
-7 and -current, but I'm not sure the fix solves your problem
completely. Could you try the latest source code and report how the
fix changes the situation (or not).

Thanks,
  ozaki-r

On Wed, Mar 15, 2017 at 4:15 AM, Stephen Borrill
<netbsd%precedence.co.uk@localhost> wrote:
> I'm trying to set up redundant firewalls using carp(4) as detailed in
> section 28.5 here:
> https://www.netbsd.org/docs/guide/en/chap-carp.html
>
> The examples ignore routing, especially setting a default gateway.
>
> Machine 1:
> carp0:  flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
>         enabled=0
>         carp: MASTER carpdev xennet0 vhid 1 advbase 1 advskew 0
>         address: 00:00:5e:00:01:01
>         inet 192.168.1.88 netmask 0xffffff00 broadcast 192.168.1.255 carp1:
> flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
>         enabled=0
>         carp: MASTER carpdev xennet1 vhid 2 advbase 1 advskew 0
>         address: 00:00:5e:00:01:02
>         inet 80.x.y.20 netmask 0xffffffc0 broadcast 80.71.28.63
>
> Machine 2:
> carp0:  flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
>         enabled=0
>         carp: BACKUP carpdev xennet0 vhid 1 advbase 1 advskew 100
>         address: 00:00:5e:00:01:01
>         inet 192.168.1.88 netmask 0xffffff00 broadcast 192.168.1.255
> carp1:  flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
>         enabled=0
>         carp: BACKUP carpdev xennet1 vhid 2 advbase 1 advskew 100
>         address: 00:00:5e:00:01:02
>         inet 80.x.y.20 netmask 0xffffffc0 broadcast 80.71.28.63
>
> My first attempt just set the default gateway in /etc/mygate with just
> mahcine 1 running
>
> The routes looked OK the face of it:
>
> Internet:
> Destination        Gateway            Flags    Refs      Use    Mtu
> Interface
> default            80.x.y.62          UGS         -        -      -  carp1
> 80.x.y.0/26        link#5             UC          -        -      -  carp1
> 80.x.y.62          c4:71:fe:65:53:61  UHLc        -        -      -  carp1
> 127/8              127.0.0.1          UGRS        -        -  33648  lo0
> 127.0.0.1          127.0.0.1          UH          -        -  33648  lo0
> 192.168.1/24       link#4             UC          -        -      -  carp0
>
> But it didn't work:
> # ping -n 8.8.8.8
> PING 8.8.8.8 (8.8.8.8): 56 data bytes
> ping: sendto: No route to host
> ping: sendto: No route to host
> ^C
> ----8.8.8.8 PING Statistics----
> 2 packets transmitted, 0 packets received, 100.0% packet loss
>
> Guessing at some sort of race condition, between setting up carp and the
> route, I added the "route add default" command to /etc/rc.local after a
> sleep 5. This fixes it with a single machine. The routing table in both
> cases looks identical.
>
> I then started up the second machine and looked its routing table:
> Internet:
> Destination        Gateway            Flags    Refs      Use    Mtu
> Interface
> default            80.x.y.62          UGS         -        -      -  carp1
> 80.x.y.0/26        80.x.y.20          U           -        -      -  carp1
> 127/8              127.0.0.1          UGRS        -        -  33648  lo0
> 127.0.0.1          127.0.0.1          UH          -        -  33648  lo0
> 192.168.1/24       192.168.1.88       U           -        -      -  carp0
>
> If I forced machine 1 down (ifconfig carp0 down;ifconfig carp1 down),
> machine 2 shows its interfaces as MASTER, but again, no route to hosts even
> though MAC address of the router does appear in the routing table after a
> while:
>
> Internet:
> Destination        Gateway            Flags    Refs      Use    Mtu
> Interface
> default            80.x.y.62          UGS         -        -      -  carp1
> 80.x.y.0/26        link#5             UC          -        -      -  carp1
> 80.x.y.62          c4:71:fe:65:53:61  UHLc        -        -      -  carp1
> 127/8              127.0.0.1          UGRS        -        -  33648  lo0
> 127.0.0.1          127.0.0.1          UH          -        -  33648  lo0
> 192.168.1/24       link#4             UC          -        -      -  carp0
> # ping -c1 80.x.y.62
> PING 80.x.y.62 (80.x.y.62): 56 data bytes
> 64 bytes from 80.x.y.62: icmp_seq=0 ttl=255 time=0.875988 ms
>
> ----80.x.y.62 PING Statistics----
> 1 packets transmitted, 1 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 0.875988/0.875988/0.875988/0.000000 ms
> # ping -c1 8.8.8.8
> PING google-public-dns-a.google.com (8.8.8.8): 56 data bytes
> ping: sendto: No route to host
> ^C
> ----google-public-dns-a.google.com PING Statistics----
> 1 packets transmitted, 0 packets received, 100.0% packet loss
>
> A similar problem happens at failback to the master. FreeBSD and OpenBSD
> have similar problems reported too, but with no clear answers.
>
> --
> Stephen
>