Re: IPv6 "is unreachable" route flapping

[Paul Ripke <stix%stix.id.au@localhost>]

On Wed, Aug 21, 2019 at 07:44:02AM -0700, Andy Ruhl wrote:
> On Wed, Aug 21, 2019 at 6:15 AM Paul Ripke <stix%stix.id.au@localhost> wrote:
> >
> > I must be doing something silly, but I can't figure out what.
> > Setup is a NetBSD 8.0 box running as router, dhcp6c getting a prefix
> > over pppoe0, assigning it to alc0. This works fine, and IPv6 on this
> > box is rock-solid. It also runs rtadvd on alc0.
> >
> > Client machines vary, but for example an Orange PI with netbsd-8.99.51
> > and ip6mode autohost with dhcpcd gets intermittent IPv6 connectivity,
> > and logs reachability flaps to the default router:
> >
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: fe80::52e5:49ff:fea6:f5d8 is unreachable, expiring it
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: deleting route to 2001:44b8:31a3:5d00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: deleting route to 2001:44b8:3158:7e00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: deleting default route via fe80::52e5:49ff:fea6:f5d8
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: fe80::52e5:49ff:fea6:f5d8 is reachable again
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: adding route to 2001:44b8:3158:7e00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: adding route to 2001:44b8:31a3:5d00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: adding default route via fe80::52e5:49ff:fea6:f5d8
> > Aug 21 00:52:10 armv7 dhcpcd[279]: emac0: adding route to 192.168.0.0/24
> > Aug 21 00:52:10 armv7 dhcpcd[279]: emac0: pid 279 deleted route to 192.168.0.0/24
> >
> > IPv4 is rock-solid, there's no issues there.
> >
> > On the router, rtadvd logs very little, just a steady stream of:
> >
> > Aug 21 20:56:48 slave rtadvd[765]: main: reloading config on SIGHUP
> > Aug 21 20:56:48 slave rtadvd[765]: <getconfig> ioctl:SIOCSIFINFO_IN6 at alc0: Operation not permitted
> > Aug 21 20:56:53 slave rtadvd[765]: rs_input: RS received on reconfiguring advertising interface(alc0)
> > Aug 21 20:56:53 slave rtadvd[765]: rs_input: RS received on reconfiguring advertising interface(alc0)
> > Aug 21 21:26:50 slave rtadvd[765]: main: reloading config on SIGHUP
> > Aug 21 21:26:50 slave rtadvd[765]: <getconfig> ioctl:SIOCSIFINFO_IN6 at alc0: Operation not permitted
> > Aug 21 21:26:57 slave rtadvd[765]: rs_input: RS received on reconfiguring advertising interface(alc0)
> > Aug 21 21:26:57 slave rtadvd[765]: rs_input: RS received on reconfiguring advertising interface(alc0)
> >
> > I ran a tcpdump for icmp6 on the PI, overlapping with the reachability flap
> > above (timezone mismatch), but nothing stands out as obviously wrong:
> >
> > 10:46:15.350437 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::a00:20ff:fe7b:f607 > ff02::1: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is 2001:44b8:31a3:5d00:a00:20ff:fe7b:f607, Flags [override]
> >           destination link-address option (2), length 8 (1): 08:00:20:7b:f6:07
> >             0x0000:  0800 207b f607
> > 10:48:57.811937 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) 2001:44b8:3158:7e00:e9d4:45c8:5678:effe > ff02::1:ffa6:f5d8: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fe80::52e5:49ff:fea6:f5d8
> >           source link-address option (1), length 8 (1): 02:42:db:e7:b6:a4
> >             0x0000:  0242 dbe7 b6a4
> > 10:48:57.812062 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) 2001:44b8:3158:7e00:52e5:49ff:fea6:f5d8 > 2001:44b8:3158:7e00:e9d4:45c8:5678:effe: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fe80::52e5:49ff:fea6:f5d8, Flags [router, solicited, override]
> >           destination link-address option (2), length 8 (1): 50:e5:49:a6:f5:d8
> >             0x0000:  50e5 49a6 f5d8
> > 10:49:02.019159 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 88) fe80::52e5:49ff:fea6:f5d8 > ff02::1: [icmp6 sum ok] ICMP6, router advertisement, length 88
> >         hop limit 64, Flags [none], pref medium, router lifetime 3600s, reachable time 0s, retrans time 0s
> >           source link-address option (1), length 8 (1): 50:e5:49:a6:f5:d8
> >             0x0000:  50e5 49a6 f5d8
> >           prefix info option (3), length 32 (4): 2001:44b8:3158:7e00::/64, Flags [onlink, auto], valid time 2592000s, pref. time 604800s
> >             0x0000:  40c0 0027 8d00 0009 3a80 0000 0000 2001
> >             0x0010:  44b8 3158 7e00 0000 0000 0000 0000
> >           prefix info option (3), length 32 (4): 2001:44b8:31a3:5d00::/64, Flags [onlink, auto], valid time 2592000s, pref. time 604800s
> >             0x0000:  40c0 0027 8d00 0009 3a80 0000 0000 2001
> >             0x0010:  44b8 31a3 5d00 0000 0000 0000 0000
> >
> > Ideas? Config is pretty simple, I have tried setting maxinterval=300 &
> > rltime=3600 for rtadvd, with no noticable change.
> 
> Simplest stuff first, are you really losing connectivity to
> fe80::a00:20ff:fe7b:f607? If so why? If not, why does the client think
> it's losing connectivity?
> 
> There's all kinds of reasons you might drop traffic in between. I'm a
> network guy though, so I always start from that angle.
> 
> Andy

Yeah, understood. In this case, though, the Orange PI was directly
connected via a switch, IPv4 reachability is fine, IPv6 fails.
Oh, and I've also seen this via qemu via a bridged local tap interface,
I wouldn't expect too many lost packets in that situation.

-- 
Paul Ripke
"Great minds discuss ideas, average minds discuss events, small minds
 discuss people."
-- Disputed: Often attributed to Eleanor Roosevelt. 1948.