kern/39084: carp preempt does not failover other interfaces when one fails.

[spribyl%peel.com@localhost]

>Number:         39084
>Category:       kern
>Synopsis:       carp preempt does not failover other interfaces when one fails.
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Jul 02 15:35:00 +0000 2008
>Originator:     Steve Pribyl
>Release:        4.0
>Organization:
>Environment:
NetBSD <snip> 4.0 NetBSD 4.0 (PEEL) #0: Wed May  7 15:43:00 CDT 2008  
<snip>@<snip>:/home/<snip>/src/sys/arch/i386/compile/PEEL i386

>Description:
I have two nodes with identical OS levels and 3 carp nic pairs on three 
separate networks.

When I pull the plug on one nic on the master node only that on interface fails 
over.  
On the master node the failed interface remains MASTER as to all other 
interfaces.
On the backup node the failed interface goes to MASTER and starts working. All 
other interfaces remain in BACKUP.

ips and passwords have been changed to protect the guilty
I can build kernels and add debugging code if that is needed.

MASTER NODE
from /etc/pf.ctl
pass quick proto carp all keep state

#ifconcig sysctl -a | grep arp
net.inet.carp.preempt = 1
net.inet.carp.arpbalance = 1
net.inet.carp.allow = 1
net.inet.carp.log = 0

/etc/ifconfig.carp0
create
vhid 1 pass notreal1 10.0.0.1 netmask 255.255.255.0 carpdev fxp0

/etc/ifconfig.carp1
create
vhid 2 pass notreal2 192.168.115.139 netmask 255.255.255.248 carpdev ex0
inet 192.168.115.140 netmask 255.255.255.255 alias
inet 192.168.115.141 netmask 255.255.255.255 alias

/etc/ifconfig.carp2
create
vhid 3 pass notreal3 192.168.114.107 netmask 255.255.255.248 carpdev ex1
inet 192.168.114.108 netmask 255.255.255.255 alias
inet 192.168.114.109 netmask 255.255.255.255 alias

/etc/ifconfig.ex0
inet 192.168.115.138 netmask 255.255.255.248 broadcast 192.168.115.143

/etc/ifconfig.ex0
inet 192.168.114.106 netmask 255.255.255.248 broadcast 192.168.114.111

/etc/ifconfig.fxp0
inet 10.0.0.4 netmask 255.255.255.0


BACKUP NODE
from /etc/pf.ctl
pass quick proto carp all keep state

# sysctl -a | grep carp
net.inet.carp.preempt = 1
net.inet.carp.arpbalance = 1
net.inet.carp.allow = 1
net.inet.carp.log = 0

/etc/ifconfig.carp0
create
vhid 1 advskew 100 pass notreal1 10.0.0.1 netmask 255.255.255.0 carpdev fxp0

/etc/ifconfig.carp1
create
vhid 2  advskew 100 pass notreal2 192.168.115.139 netmask 255.255.255.248 
carpdev tlp0
inet 192.168.115.140 netmask 255.255.255.255 alias
inet 192.168.115.141 netmask 255.255.255.255 alias

/etc/ifconfig.carp2
create
vhid 3 advskew 100 pass notreal3 192.168.114.107 netmask 255.255.255.248 
carpdev tlp1
inet 192.168.114.108 netmask 255.255.255.255 alias
inet 192.168.114.109 netmask 255.255.255.255 alias

/etc/ifconfig.fxp0
inet 10.0.0.5 netmask 255.255.255.0

/etc/ifconfig.tlp0
inet 192.168.115.137 netmask 255.255.255.248 broadcast 192.168.115.143

/etc/ifconfig.tlp1
inet 192.168.114.105 netmask 255.255.255.248 broadcast 192.168.114.111 
>How-To-Repeat:
Set up two carp interfaces and pull the plug on one of them.
I have duplicated this on two different sets of systems.
>Fix: