Re: IPv6 "is unreachable" route flapping

[Paul Ripke <stix%stix.id.au@localhost>]

On Wed, Aug 21, 2019 at 04:12:30PM +0100, Roy Marples wrote:
> On 21/08/2019 14:14, Paul Ripke wrote:
> > I must be doing something silly, but I can't figure out what.
> > Setup is a NetBSD 8.0 box running as router, dhcp6c getting a prefix
> > over pppoe0, assigning it to alc0. This works fine, and IPv6 on this
> > box is rock-solid. It also runs rtadvd on alc0.
> > 
> > Client machines vary, but for example an Orange PI with netbsd-8.99.51
> > and ip6mode autohost with dhcpcd gets intermittent IPv6 connectivity,
> > and logs reachability flaps to the default router:
> > 
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: fe80::52e5:49ff:fea6:f5d8 is unreachable, expiring it
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: deleting route to 2001:44b8:31a3:5d00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: deleting route to 2001:44b8:3158:7e00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: deleting default route via fe80::52e5:49ff:fea6:f5d8
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: fe80::52e5:49ff:fea6:f5d8 is reachable again
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: adding route to 2001:44b8:3158:7e00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: adding route to 2001:44b8:31a3:5d00::/64
> > Aug 21 00:48:57 armv7 dhcpcd[279]: emac0: adding default route via fe80::52e5:49ff:fea6:f5d8
> > Aug 21 00:52:10 armv7 dhcpcd[279]: emac0: adding route to 192.168.0.0/24
> > Aug 21 00:52:10 armv7 dhcpcd[279]: emac0: pid 279 deleted route to 192.168.0.0/24
> 
> dhcpcd listens for RTM_ADD/RTM_DELETE messages.
> If the destination matches the router address and it's RTM_DELETE or the
> gateway address is not AF_LINK or sdl_alen of the gateway address is zero
> then the kernel has lost reachability to the router and dhcpcd will expire
> it accordingly or it will mark the router as reachable.

Ok, so these are routing updates from the kernel, not necessarily triggered
by network activity (router advertisements and such).

> You should be able to see this in `route monitor` output.

I've attached output from the Orange Pi - I do see RTM_DELETE, but as for why,
I know not.

> Does this attached patch help any?

This is for the client (dhcpcd running) side? And/or the server (rtadvd) side?
I'll test it on a few machines and see what I see (I need to figure out a nice
way of upgrading the Pi).

> Roy

> Index: sys/netinet6/nd6.c
> ===================================================================
> RCS file: /cvsroot/src/sys/netinet6/nd6.c,v
> retrieving revision 1.257
> diff -u -p -r1.257 nd6.c
> --- sys/netinet6/nd6.c	14 Aug 2019 08:34:44 -0000	1.257
> +++ sys/netinet6/nd6.c	21 Aug 2019 15:05:59 -0000
> @@ -2225,11 +2225,17 @@ nd6_cache_lladdr(
>  		break;
>  	}
>  
> -#if 0
> -	/* XXX should we send rtmsg as it used to be? */
> -	if (do_update)
> -		rt_newmsg(RTM_CHANGE, rt);  /* tell user process */
> -#endif
> +	if (do_update) {
> +		struct sockaddr_in6 sin6;
> +		struct rtentry *rt;
> +
> +		sockaddr_in6_init(&sin6, from, 0, 0, 0);
> +		rt = rtalloc1(sin6tosa(&sin6), 0);
> +		if (rt != NULL) {
> +			rt_newmsg(RTM_CHANGE, rt);  /* tell user process */
> +			rt_unref(rt);
> +		}
> +	}
>  
>  	if (ln != NULL) {
>  		router = ln->ln_router;
> Index: sys/netinet6/nd6_nbr.c
> ===================================================================
> RCS file: /cvsroot/src/sys/netinet6/nd6_nbr.c,v
> retrieving revision 1.166
> diff -u -p -r1.166 nd6_nbr.c
> --- sys/netinet6/nd6_nbr.c	29 Apr 2019 16:12:30 -0000	1.166
> +++ sys/netinet6/nd6_nbr.c	21 Aug 2019 15:05:59 -0000
> @@ -883,11 +883,18 @@ nd6_na_input(struct mbuf *m, int off, in
>  	 */
>  	ln->ln_asked = 0;
>  	nd6_llinfo_release_pkts(ln, ifp);
> -	/* FIXME */
> -#if 0
> -	if (rt_announce) /* tell user process about any new lladdr */
> -		rt_newmsg(RTM_CHANGE, rt);
> -#endif
> +
> +	if (rt_announce) {
> +		struct sockaddr_in6 sin6;
> +		struct rtentry *rt;
> +
> +		sockaddr_in6_init(&sin6, &taddr6, 0, 0, 0);
> +		rt = rtalloc1(sin6tosa(&sin6), 0);
> +		if (rt != NULL) {
> +			rt_newmsg(RTM_CHANGE, rt);  /* tell user process */
> +			rt_unref(rt);
> +		}
> +	}
>  
>   freeit:
>  	if (ln != NULL)


-- 
Paul Ripke
"Great minds discuss ideas, average minds discuss events, small minds
 discuss people."
-- Disputed: Often attributed to Eleanor Roosevelt. 1948.

got message of size 176 on Thu Aug 22 09:30:19 2019
RTM_ADD: Add Route: len 176, pid 345, seq 0, errno 0, flags: 0x2445<UP,HOST,DONE,LLINFO,CLONED>
locks: 0x0 inits: 0x0
sockaddrs: 0x3<DST,GATEWAY>
 fe80::52e5:49ff:fea6:f5d8%emac0 link#1
got message of size 192 on Thu Aug 22 09:30:19 2019
RTM_DELETE: Delete Route: len 192, pid 279, seq 23189, errno 0, flags: 0x940<DONE,CONNECTED,STATIC>
locks: 0x0 inits: 0x0
sockaddrs: 0x7<DST,GATEWAY,NETMASK>
 2001-44b8-31a3-5d00-0000-0000-0000-0000.static.ipv6.internode.on.net link#1 ffff:ffff:ffff:ffff::
got message of size 192 on Thu Aug 22 09:30:19 2019
RTM_DELETE: Delete Route: len 192, pid 279, seq 23190, errno 0, flags: 0x940<DONE,CONNECTED,STATIC>
locks: 0x0 inits: 0x0
sockaddrs: 0x7<DST,GATEWAY,NETMASK>
 2001-44b8-3158-7e00-0000-0000-0000-0000.static.ipv6.internode.on.net link#1 ffff:ffff:ffff:ffff::
got message of size 192 on Thu Aug 22 09:30:19 2019
RTM_DELETE: Delete Route: len 192, pid 279, seq 23191, errno 0, flags: 0x842<GATEWAY,DONE,STATIC>
locks: 0x0 inits: 0x0
sockaddrs: 0x7<DST,GATEWAY,NETMASK>
 :: fe80::52e5:49ff:fea6:f5d8%emac0 default
got message of size 264 on Thu Aug 22 09:30:19 2019
RTM_ADD: Add Route: len 264, pid 279, seq 23192, errno 0, flags: 0x941<UP,DONE,CONNECTED,STATIC>
locks: 0x0 inits: 0x0
sockaddrs: 0x37<DST,GATEWAY,NETMASK,IFP,IFA>
 2001-44b8-3158-7e00-0000-0000-0000-0000.static.ipv6.internode.on.net link#1 ffff:ffff:ffff:ffff:: link#1 2001-44b8-3158-7e00-e9d4-45c8-5678-effe.static.ipv6.internode.on.net
got message of size 264 on Thu Aug 22 09:30:19 2019
RTM_ADD: Add Route: len 264, pid 279, seq 23193, errno 0, flags: 0x941<UP,DONE,CONNECTED,STATIC>
locks: 0x0 inits: 0x0
sockaddrs: 0x37<DST,GATEWAY,NETMASK,IFP,IFA>
 2001-44b8-31a3-5d00-0000-0000-0000-0000.static.ipv6.internode.on.net link#1 ffff:ffff:ffff:ffff:: link#1 2001-44b8-31a3-5d00-d635-0546-09bc-f0e7.static.ipv6.internode.on.net
got message of size 240 on Thu Aug 22 09:30:19 2019
RTM_ADD: Add Route: len 240, pid 279, seq 23194, errno 0, flags: 0x843<UP,GATEWAY,DONE,STATIC>
locks: 0x0 inits: 0x0
sockaddrs: 0x17<DST,GATEWAY,NETMASK,IFP>
 :: fe80::52e5:49ff:fea6:f5d8%emac0 :: link#1