I have to setup, no matter what, a cluster of 3 active OpenBSD 5.5 with CARP on 3 separate ESXi 5.5 hosts.
I think I'm close to the end but I have this message in /var/log/messages : carp0: replay or network loop detected
Here is my configuration :
Standard Vswitchs, no VDS Promiscuous mode, mac changing and packet forge ON on every vswitch net.reversePathFwsCheckPromisc=1
/etc/hostname.em0
inet 172.0.0.19 255.255.0.0 NONE
/etc/hostname.carp0
inet 172.0.0.16 255.255.0.0 172.0.255.255 balancing ip-unicast carpnodes 1:0,2:100,3:50 pass "password" carpdev em0
/etc/sysctl.conf
net.inet.carp.allow=1
net.inet.carp.preempt=1
net.inet.carp.log=2
/etc/pf.conf
pass quick on em0 inet proto carp from any to any
ifconfig carp0
carp0: flags=8843 mtu 1500
lladdr 00:00:5e:00:01:01
priority: 0
carp: carpdev em0 advbase 1 balancing ip-unicast
state MASTER vhid 1 advskew 0
state BACKUP vhid 2 advskew 100
state BACKUP vhid 3 advskew 50
groups: carp
status: master
inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x4
inet 172.0.0.16 netmask 0xffff0000 broadcast 172.0.255.255
/etc/hostname.em0
inet 172.0.0.20 255.255.0.0 NONE
/etc/hostname.carp0
inet 172.0.0.16 255.255.0.0 172.0.255.255 balancing ip-unicast carpnodes 1:50,2:0,3:100 pass "password" carpdev em0
/etc/sysctl.conf
net.inet.carp.allow=1
net.inet.carp.preempt=1
net.inet.carp.log=2
/etc/pf.conf
pass quick on em0 inet proto carp from any to any
ifconfig carp0
carp0: flags=8843 mtu 1500
lladdr 00:00:5e:00:01:01
priority: 0
carp: carpdev em0 advbase 1 balancing ip-unicast
state BACKUP vhid 1 advskew 50
state MASTER vhid 2 advskew 0
state BACKUP vhid 3 advskew 100
groups: carp
status: backup
inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x4
inet 172.0.0.16 netmask 0xffff0000 broadcast 172.0.255.255
/etc/hostname.em0
inet 172.0.0.21 255.255.0.0 NONE
/etc/hostname.carp0
inet 172.0.0.16 255.255.0.0 172.0.255.255 balancing ip-unicast carpnodes 1:100,2:50,3:0 pass "password" carpdev em0
/etc/sysctl.conf
net.inet.carp.allow=1
net.inet.carp.preempt=1
net.inet.carp.log=2
/etc/pf.conf
pass quick on em0 inet proto carp from any to any
ifconfig carp0
carp0: flags=8843 mtu 1500
lladdr 00:00:5e:00:01:01
priority: 0
carp: carpdev em0 advbase 1 balancing ip-unicast
state BACKUP vhid 1 advskew 100
state BACKUP vhid 2 advskew 50
state MASTER vhid 3 advskew 0
groups: carp
status: backup
inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x4
inet 172.0.0.16 netmask 0xffff0000 broadcast 172.0.255.255
Firewall disabled
tcpdump | grep CARP
12:08:37.098390 CARPv2-advertise 36: vhid=3 advbase=1 advskew=50 demote=0 (DF) [tos 0x10]
12:08:37.098533 CARPv2-advertise 36: vhid=3 advbase=1 advskew=50 demote=0 (DF) [tos 0x10]
12:08:37.271021 CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:37.271028 CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:37.948303 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:37.948415 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:38.280916 CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:38.280921 CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:38.308264 CARPv2-advertise 36: vhid=3 advbase=1 advskew=50 demote=0 (DF) [tos 0x10]
12:08:38.308395 CARPv2-advertise 36: vhid=3 advbase=1 advskew=50 demote=0 (DF) [tos 0x10]
12:08:38.958201 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:38.958305 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:39.290790 CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
12:08:39.290808 CARPv2-advertise 36: vhid=2 advbase=1 advskew=0 demote=0 (DF) [tos 0x10]
Thank you for your help.
Use the following filter on tcpdump to check the source address for CARP requests: tcpdump -pni em0 vrrp
After you enable net.reversePathFwsCheckPromisc flag, you have to disable and reenable the promiscuous mode to activate.
Had the same issue. The strange thing is that I have some ESXi servers on 1Gbit and some on 10Gbit linked to different switches. Whenever I move any OpenBSD (5.8) system on a a host with 10Gbit network got these errors in the logs. Following this article https://kb.vmware.com/s/article/59235 I did:
esxcli system settings advanced set -o /Net/ReversePathFwdCheckPromisc -i 1
Then move your interfaces off and back on promiscuous mode and worked just fine.
User contributions licensed under CC BY-SA 3.0