tech-kern: RE: question about CARP

Subject: RE: question about CARP
To: 'Liam J. Foy' <liamfoy@sepulcrum.org>
From: George Chen <georgechen2101@msn.com>
List: tech-kern
Date: 05/11/2006 12:07:18
Hi Liam,

The command on fw1 is
ifconfig spi4 up
ifconfig spi7 up
ifconfig carp0 create
ifconfig carp0 vhid 1
ifconfig carp0 carpdev spi4
ifconfig carp0 advskew 100
ifconfig carp0 11.10.4.100
ifconfig carp1 create
ifconfig carp1 vhid 2
ifconfig carp1 carpdev spi7
ifconfig carp1 advskew 100
ifconfig carp1 12.10.4.100
sysctl -w net.inet.carp.preempt=1

The result of 'ifconfig -a' is as below,
wm0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        capabilities=87<IP4CSUM,TCP4CSUM,UDP4CSUM,TSO4>
        enabled=0
        address: 00:30:64:03:49:50
        media: Ethernet autoselect
        status: no carrier
wm1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        capabilities=87<IP4CSUM,TCP4CSUM,UDP4CSUM,TSO4>
        enabled=0
        address: 00:30:64:03:49:51
        media: Ethernet autoselect
        status: no carrier
wm2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=87<IP4CSUM,TCP4CSUM,UDP4CSUM,TSO4>
        enabled=0
        address: 00:30:64:03:49:52
        media: Ethernet 10baseT
        status: active
        inet 192.168.0.22 netmask 0xffffff00 broadcast 192.168.0.255
sgsync0: flags=0 mtu 1500
        sgsync: syncdev none
lo0: flags=8009<UP,LOOPBACK,MULTICAST> mtu 33192
        inet 127.0.0.1 netmask 0xff000000
spi0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:96:38
        media: Ethernet autoselect (none)
        status: no carrier
spi1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:da:2a
        media: Ethernet autoselect (none)
        status: no carrier
spi2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:e5:2f
        media: Ethernet autoselect (none)
        status: no carrier
spi3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:71:e4
        media: Ethernet autoselect (none)
        status: no carrier
spi4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        capabilities=100
        enabled=100
        address: 00:e2:4c:b1:ba:bf
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
spi5: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:b5:bf
        media: Ethernet autoselect (none)
        status: no carrier
spi6: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:2a:d6
        media: Ethernet autoselect (none)
        status: no carrier
spi7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:60:e6
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        carp: BACKUP carpdev spi4 vhid 1 advbase 100 advskew 100
        address: 00:00:5e:00:01:01
        inet 11.10.4.100 netmask 0xff000000 broadcast 11.255.255.255
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        carp: BACKUP carpdev spi7 vhid 2 advbase 100 advskew 100
        address: 00:00:5e:00:01:02
        inet 12.10.4.100 netmask 0xff000000 broadcast 12.255.255.255

The command on fw2 is
ifconfig spi4 up
ifconfig spi7 up
ifconfig carp0 create
ifconfig carp0 vhid 1
ifconfig carp0 carpdev spi4
ifconfig carp0 11.10.4.100
ifconfig carp1 create
ifconfig carp1 vhid 2
ifconfig carp1 carpdev spi7
ifconfig carp1 12.10.4.100
sysctl -w net.inet.carp.preempt=1

The result of 'ifconfig -a' is as below,wm0:
flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        capabilities=87<IP4CSUM,TCP4CSUM,UDP4CSUM,TSO4>
        enabled=0
        address: 00:30:64:03:a3:08
        media: Ethernet autoselect
        status: no carrier
wm1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        capabilities=87<IP4CSUM,TCP4CSUM,UDP4CSUM,TSO4>
        enabled=0
        address: 00:30:64:03:a3:09
        media: Ethernet autoselect
        status: no carrier
wm2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=87<IP4CSUM,TCP4CSUM,UDP4CSUM,TSO4>
        enabled=0
        address: 00:30:64:03:a3:0a
        media: Ethernet 10baseT
        status: active
        inet 192.168.0.37 netmask 0xffffff00 broadcast 192.168.0.255
sgsync0: flags=0 mtu 1500
        sgsync: syncdev none
lo0: flags=8009<UP,LOOPBACK,MULTICAST> mtu 33192
        inet 127.0.0.1 netmask 0xff000000
spi0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:26:81
        media: Ethernet autoselect (none)
        status: no carrier
spi1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:22:9f
        media: Ethernet autoselect (none)
        status: no carrier
spi2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:c8:cb
        media: Ethernet autoselect (none)
        status: no carrier
spi3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:1c:40
        media: Ethernet autoselect (none)
        status: no carrier
spi4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        capabilities=100
        enabled=100
        address: 00:e2:4c:b1:20:f3
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
spi5: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:9f:fa
        media: Ethernet autoselect (none)
        status: no carrier
spi6: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:d7:f7
        media: Ethernet autoselect (none)
        status: no carrier
spi7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e2:4c:b1:fb:01
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        carp: MASTER carpdev spi4 vhid 1 advbase 100 advskew 0
        address: 00:00:5e:00:01:01
        inet 11.10.4.100 netmask 0xff000000 broadcast 11.255.255.255
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        carp: MASTER carpdev spi7 vhid 2 advbase 100 advskew 0
        address: 00:00:5e:00:01:02
        inet 12.10.4.100 netmask 0xff000000 broadcast 12.255.255.255

Sorry, there are so many lines.:)
When I bring down carp0 on fw2 by 'ifconfig carp0 down', carp0 on fw1
becomes MASTER, but carp1 on both sides remains unchanged.

Sorry I didn't sync to head version, didn't use your patch neither. Instead
I checked your patch. My opinion is that this is a design bug. The reason is
that there is no mechanism to transfer the message of one carp going from
backup to master to all other carps on the same firewall. Without the
message, how can other carp react to it?

What is my modification? There are bugs in arp balance, which also exists in
your patch. Additionally I'm making carp support for layer 2 HA and bridge
balance.

Cheers!

George Chen

-----Original Message-----
From: tech-kern-owner@NetBSD.org [mailto:tech-kern-owner@NetBSD.org] On
Behalf Of Liam J. Foy
Sent: 2006 05 10 15:21
To: George Chen
Cc: tech-kern@netbsd.org
Subject: Re: question about CARP


On 10 May 2006, at 02:54, George Chen wrote:

> Hi Liam,
>
> I have a same script on both firewalls.
> ifconfig spi4 up
> ifconfig spi7 up
> ifconfig carp0 create
> ifconfig carp0 vhid 1
> ifconfig carp0 carpdev spi4
> ifconfig carp0 advskew 150
> ifconfig carp0 11.10.4.100
> ifconfig carp1 create
> ifconfig carp1 vhid 2
> ifconfig carp1 carpdev spi7
> ifconfig carp1 advskew 150
> ifconfig carp1 12.10.4.100
> sysctl -w net.inet.carp.preempt=1
>
> When setup, carp0 and carp1 are MASTER on fw1. I bring carp0 down with
> command
> Ifconfig carp0 down.
>
> I expect carp1 on fw1 will be down automatically. But it doesn't  
> happen. The
> result is that the traffic is blocked.

On fw1, try not setting the advskew. Only do it on fw2. So, use that
script on fw2 but remove the 'ifconfig carp* advskew 150' for fw1.

>
> Actually I have made some modification to the code from
> ftp://packages.stura.uni-rostock.de/patches/carp.diff

This is my patch. The latest can be seen at:

http://www.netbsd.org/~liamjfoy/new-carp-patch.diff

(this patch includes MANY cleanups.)

>
> I wonder if this problem came from the original code or from my
> modification.

What modifications have you made. I suggest trying to achieve the
above by using my patch (and not setting the advskew on fw1, assume
this is your prefered master). Also, once you've set up both  
firewalls could
you send me the output of 'ifconfig -a' please?

>
> I didn't know your patch. Would you share it with me?
>
> Thanks
>
> George Chen
>
> On 9 May 2006, at 05:34, George Chen wrote:
>
>> Hi,
>>
>> I have a question about CARP.
>>
>> I have two firewalls named fw1 and fw2. eth0 and eth0 are two
>> interfaces on
>> both fw1 and fw2. It works well when fw1 serves as MASTER, which  
>> means
>> fw1.eth0 and fw1.eth1 are all MASTER. The problem is, when I down
>> fw1.eth0
>> and therefore fw1.eth0 becomes BACKUP while fw2.eth0 becomes
>> MASTER, will
>> fw1.eth1 failovers to fw2.eth1? I didn't see that fw1.eth1 becomes
>> BACKUP,
>> which leads to the traffic fails.
>>
>> I don't know if CARPs on different interfaces but one same
>> appliance are
>> associated. If not, CARP can't help if individual interface fails.
>> Am I
>> right? If yes, how does it implemented?
>>
>> Thanks for all your time,
>>
>> Regards,
>> George Chen

		---
		Liam J. Foy
		<liamjfoy@netbsd.org>
		<liamfoy@sepulcrum.org>
		BSDPortal.org