Subject: gif(4) and gre(4) interact poorly with dynamic routing
To: None <tech-net@netbsd.org>
From: Miles Nordin <carton@Ivy.NET>
List: tech-net
Date: 09/25/2004 12:14:27
I'm trying to use gif or gre with quagga.  

The point was to implement rfc1772 section A.2.3, to carry transit
traffic among IBGP peers with a full view of the default-free zone,
even when that traffic must pass through routers without full BGP
tables.  To make the implementation reasonable, the path tunneled
packets take between exterior gateways must be adjustable by the
interior gateway protocol, OSPF, so that it's resilient to failures
inside my autonomous system.  

In the example below I've made the tunnel's outer source and
destination are aliases on the loopback interface.  OSPF will set up
routes to these addresses over any path it can find between the two
tunnel endpoints.  This is a common technique, and works fine, at
first.

The problem is, when one of these loopback aliases I've used for the
tunnel outer destination becomes unreachable and then comes back, the
tunnel stops working and then stays broken.  Here's a typescript from
a gif tunnel that broke when I rebooted its endpoint.  gif seems to be
the less quirky of gif and gre, in that I can get the tunnel working
by changing the outer destination to a different address and change it
back, on the endpoint that was not rebooted:

Script started on Sat Sep 25 15:43:18 2004
$ ifconfig gif2
gif2: flags=a051<UP,POINTOPOINT,RUNNING,LINK1,MULTICAST> mtu 1280
        tunnel inet 192.168.168.4 --> 192.168.168.2
        inet 192.168.169.4 -> 192.168.169.2 netmask 0xffffffff
        inet6 fe80::203:baff:fe0f:aa45%gif2 ->  prefixlen 64 scopeid 0x10
$ route get 192.168.169.2       # <-- valid routes to outer and inner address
   route to: 192.168.169.2
destination: 192.168.169.2
 local addr: 192.168.169.4
  interface: gif2
      flags: <UP,HOST,DONE>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0         0         0 
$ route -n get 192.168.168.2
   route to: 192.168.168.2
destination: 192.168.168.2
    gateway: 192.168.3.102
 local addr: 192.168.3.1
  interface: tlp3
      flags: <UP,GATEWAY,HOST,DONE,PROTO1> # <-- PROTO1 means from quagga/OSPF
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0        20         0         0 
$ ping 192.168.169.2      # <-- outer address reachable, inner address not
PING 192.168.169.2 (192.168.169.2): 56 data bytes
^C
----192.168.169.2 PING Statistics----
3 packets transmitted, 0 packets received, 100.0% packet loss
$ ping 192.168.168.2
PING 192.168.168.2 (192.168.168.2): 56 data bytes
64 bytes from 192.168.168.2: icmp_seq=0 ttl=255 time=0.873 ms
64 bytes from 192.168.168.2: icmp_seq=1 ttl=255 time=0.863 ms
64 bytes from 192.168.168.2: icmp_seq=2 ttl=255 time=0.826 ms
^C
----192.168.168.2 PING Statistics----
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.826/0.854/0.873/0.025 ms

$ sudo ifconfig gif2 down up     # <--- doesn't help
$ ping 192.168.169.2
PING 192.168.169.2 (192.168.169.2): 56 data bytes
^C
----192.168.169.2 PING Statistics----
3 packets transmitted, 0 packets received, 100.0% packet loss

$ sudo ifconfig gif2 inet tunnel 192.168.168.4 192.168.168.2 # <-- no good
$ ping 192.168.169.2
PING 192.168.169.2 (192.168.169.2): 56 data bytes
^C
----192.168.169.2 PING Statistics----
2 packets transmitted, 0 packets received, 100.0% packet loss

$ sudo ifconfig gif2 inet tunnel 192.168.168.4 192.168.168.5 # <-- works
$ sudo ifconfig gif2 inet tunnel 192.168.168.4 192.168.168.2
$ ping 192.168.169.2
PING 192.168.169.2 (192.168.169.2): 56 data bytes
64 bytes from 192.168.169.2: icmp_seq=0 ttl=255 time=1.458 ms
64 bytes from 192.168.169.2: icmp_seq=1 ttl=255 time=1.226 ms
64 bytes from 192.168.169.2: icmp_seq=2 ttl=255 time=1.045 ms
^C
----192.168.169.2 PING Statistics----
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.045/1.243/1.458/0.207 ms
$ 
Script done on Sat Sep 25 15:45:14 2004

In the last step, you can see the tunnel came back, even though there
was no change in the routing table.

This is bad, because it means tunnels will be unreliable for anyone
using any dynamic routing, not just my odd application.  And IPv6 is
all about using a routing protocol.

Is there an explanation or a way around this, or should I file a PR?

-- 
Le fascisme est la dictature ouverte de la bourgeoisie.
		-- Georg Dimitrov