tech-net archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: carp and routing

On Thu, 18 May 2017, Ryota Ozaki wrote:

On Thu, May 18, 2017 at 8:44 PM, Stephen Borrill
<netbsd%precedence.co.uk@localhost> wrote:

On Thu, 18 May 2017, Ryota Ozaki wrote:


I fixed an issue of CARP in -current, which is a regression between
-7 and -current, but I'm not sure the fix solves your problem
completely. Could you try the latest source code and report how the
fix changes the situation (or not).



Prior to your changes I found that with -current, I could not send any
packets at all even on the LAN (a regression from -7). With your changes I
find:
1) When first booted, machine 1 (Master) can send packets on LAN and via
default route (regression fixed)
2) Upon failover to machine 2, machine 2 can now send packets via default
route without having to do a "route change default" (fixed problem)


Good.


3) Upon failback to machine 1, machine 1 can no longer send packets
(regression still present)
I noticed that pinging after failback to machine 1 only failed if I had pinged on machine 2. So I reasoned that the problem was because switches, etc. hadn't noticed the change. I proved this by using arping to send a gratuitous arp reply: 
arping -c 1 -A -I carp0 192.168.1.88
arping -c 1 -A -I carp1 80.x.y.20
If the above commands are run after the interface becomes a master, then it works. Are we missing out on sending a gratuitous arp after becomg master? I notice OpenBSD send two, the second after a small delay: 
http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/netinet/ip_carp.c.diff?r1=1.127&r2=1.128&f=h


Could you show me the results of the following command on machine 1?
- ifconfig carp0
- ifconfig xennet0
- netstat -nr -f inet
- arp -na


I'll skip these because of the above arp tests.


I tried to apply your if_arp.c patch to -7. but the code is just too
different.


The patch can be applicable to -current and is even unnecessary to -7
because it fixes a regression introduced recently.
I don't think it is as simple as that. My email below (and elaborated on here: http://mail-index.netbsd.org/tech-net/2017/05/15/msg006331.html ) describes a problem with -7 here you cannot route via the default gateway without doing a "route change default" after becoming master. It appears that with the correct gratuitous arps, -current works OK. 


On Wed, Mar 15, 2017 at 4:15 AM, Stephen Borrill
<netbsd%precedence.co.uk@localhost> wrote:


I'm trying to set up redundant firewalls using carp(4) as detailed in
section 28.5 here:
https://www.netbsd.org/docs/guide/en/chap-carp.html

The examples ignore routing, especially setting a default gateway.

Machine 1:
carp0:  flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
        enabled=0
        carp: MASTER carpdev xennet0 vhid 1 advbase 1 advskew 0
        address: 00:00:5e:00:01:01
        inet 192.168.1.88 netmask 0xffffff00 broadcast 192.168.1.255
carp1:
flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
        enabled=0
        carp: MASTER carpdev xennet1 vhid 2 advbase 1 advskew 0
        address: 00:00:5e:00:01:02
        inet 80.x.y.20 netmask 0xffffffc0 broadcast 80.71.28.63

Machine 2:
carp0:  flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
        enabled=0
        carp: BACKUP carpdev xennet0 vhid 1 advbase 1 advskew 100
        address: 00:00:5e:00:01:01
        inet 192.168.1.88 netmask 0xffffff00 broadcast 192.168.1.255
carp1:  flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=2800<TCP4CSUM_Tx,UDP4CSUM_Tx>
        enabled=0
        carp: BACKUP carpdev xennet1 vhid 2 advbase 1 advskew 100
        address: 00:00:5e:00:01:02
        inet 80.x.y.20 netmask 0xffffffc0 broadcast 80.71.28.63

My first attempt just set the default gateway in /etc/mygate with just
mahcine 1 running

The routes looked OK the face of it:

Internet:
Destination        Gateway            Flags    Refs      Use    Mtu
Interface
default            80.x.y.62          UGS         -        -      -
carp1
80.x.y.0/26        link#5             UC          -        -      -
carp1
80.x.y.62          c4:71:fe:65:53:61  UHLc        -        -      -
carp1
127/8              127.0.0.1          UGRS        -        -  33648  lo0
127.0.0.1          127.0.0.1          UH          -        -  33648  lo0
192.168.1/24       link#4             UC          -        -      -
carp0

But it didn't work:
# ping -n 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: No route to host
ping: sendto: No route to host
^C
----8.8.8.8 PING Statistics----
2 packets transmitted, 0 packets received, 100.0% packet loss

Guessing at some sort of race condition, between setting up carp and the
route, I added the "route add default" command to /etc/rc.local after a
sleep 5. This fixes it with a single machine. The routing table in both
cases looks identical.

I then started up the second machine and looked its routing table:
Internet:
Destination        Gateway            Flags    Refs      Use    Mtu
Interface
default            80.x.y.62          UGS         -        -      -
carp1
80.x.y.0/26        80.x.y.20          U           -        -      -
carp1
127/8              127.0.0.1          UGRS        -        -  33648  lo0
127.0.0.1          127.0.0.1          UH          -        -  33648  lo0
192.168.1/24       192.168.1.88       U           -        -      -
carp0

If I forced machine 1 down (ifconfig carp0 down;ifconfig carp1 down),
machine 2 shows its interfaces as MASTER, but again, no route to hosts
even
though MAC address of the router does appear in the routing table after a
while:

Internet:
Destination        Gateway            Flags    Refs      Use    Mtu
Interface
default            80.x.y.62          UGS         -        -      -
carp1
80.x.y.0/26        link#5             UC          -        -      -
carp1
80.x.y.62          c4:71:fe:65:53:61  UHLc        -        -      -
carp1
127/8              127.0.0.1          UGRS        -        -  33648  lo0
127.0.0.1          127.0.0.1          UH          -        -  33648  lo0
192.168.1/24       link#4             UC          -        -      -
carp0
# ping -c1 80.x.y.62
PING 80.x.y.62 (80.x.y.62): 56 data bytes
64 bytes from 80.x.y.62: icmp_seq=0 ttl=255 time=0.875988 ms

----80.x.y.62 PING Statistics----
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.875988/0.875988/0.875988/0.000000 ms
# ping -c1 8.8.8.8
PING google-public-dns-a.google.com (8.8.8.8): 56 data bytes
ping: sendto: No route to host
^C
----google-public-dns-a.google.com PING Statistics----
1 packets transmitted, 0 packets received, 100.0% packet loss

A similar problem happens at failback to the master. FreeBSD and OpenBSD
have similar problems reported too, but with no clear answers.

--
Stephen
Home | Main Index | Thread Index | Old Index