Server cluster generates UnsolicitedARPReplies in log (Gratuitous/GARP)
Last modified on 19 Feb, 2021. Revision 9Up to date for | 12.00.xx / 13.00.xx |
Supported since | 8 |
Status | OK |
This article applies to:Clavister Security Gateway 8.x, 9.x and 10.x
Note changes as of 8.30.01 for Linux Heartbeat clusters and Microsoft Cluster Server
Note changes as of 8.50.02 for Microsoft Network Load Balancing
Background
Some high-availability server clusters work by sharing an IP address, where only one server “owns” this address at a time. If the server currently owning the IP address goes down, the backup server will start serving requests on that IP address.
A basic problem here is getting surrounding network equipment (routers, Security Gateways) to understand that the IP address should be routed a new hardware address. This is often done through gratitious ARP responses, whereby the new server sends responses to all hosts that need to know about the change, even though these units have not sent ARP queries.
The problem
Clavister Secturity Gateway will not listen to ARP responses that it has not sent out queries for. This is to make “ARP spoofing” (IP address spoofing on the local network through bogus ARPs) harder.
This means that the ARP cache of the Security Gateway will not be updated by the gratuitous ARPs sent by the new server. Rather, it will take up to 15 minutes (using default settings) for the Security Gateway to start routing traffic to the new server.
You will likely also see entries like these in your logs (syslog example):
Jun 6 11:38:13 mygw EFW: ARP: rule=UnsolicitedARPReplies action=drop reason=already_exists recvif=int hwsender=0090:0b02:5c70 hwdest=0001:020d:fb16 arp=reply srcenet=0090:0b02:5c70 srcip=192.168.16.136 destenet=0001:020d:fb16 destip=192.168.16.1
The solution
Clavister Secturity Gateway can be made to accept gratuitous ARPs by turning off the extra checks and making it fully compliant with the ARP specification: RFC 826.
Make the following modifications to Advanced Settings -> ARP:
Note: In newer cOS Core versions (10+) the ARP settings can be found under Network->ARP->Advanced Settings.
- UnsolicitedARPReplies: Accept or AcceptLog (default: DropLog)
This is necessary if the server cluster transmits gratitious requests. - ARPRequests: Accept (default: Drop)
This may be necessary if the server cluster transmits queries rather than requests. - ARPChanges: Accept or AcceptLog (default: AcceptLog)
This is necessary, since the hardware address will change when failover occurs. - ARPMatchEnetSender: Ignore or Log (default: DropLog)
Some server cluster implementations build somewhat ugly ARP packets, where the sender address in the MAC header does not match the address supplied in the ARP data. If that is the case, this setting cannot be Drop or DropLog. - ARPMulticast: Accept, Log, Drop or DropLog (default: DropLog)
Some cluster implementations like Windows 2003 servers with IIS requires that this option be set to Accept or Log.
Note that Clavister Security Gateway before 8.30.01 only listens to ARP packets destined to the Security Gateway itself. This presents a problem for the Linux Heartbeat cluster system and MS Cluster Server, which does not send targeted ARP queries when it attempts to update the ARP caches of nearby units.
As of v8.30.01, the Security Gateway will listen to any ARP queries if ARPRequests is set to Accept, and any ARP responses if UnsolicitedARPReplies is set to Accept or AcceptLog.
Note that Clavister Security Gateway before 8.50.02 always sent ARP responses to the MAC address found in the ethernet header of the query. Microsoft NLB apparently does rely on the response to be sent to the source MAC address in the ARP data.
As of v8.50.02 the core sends ARP responses to the source MAC address in the ARP data.
Related articles
11 Jan, 2023 ipsec core vpn
22 Sep, 2021 core ftps sftp
23 Aug, 2022 core ipreputation
21 Oct, 2022 core arp routing
1 Feb, 2023 core wizard hardware migration netwall
25 Nov, 2022 core hyperv azure
9 Dec, 2022 arp core
14 Nov, 2022 incontrol cli core webui
14 Dec, 2022 core
10 Jan, 2023 arm x86 core
23 Aug, 2022 core certificate oneconnect ipsec vpn
23 Aug, 2022 sslvpn openconnect oneconnect android core
17 Nov, 2022 core license updates idp antivirus wcf ipreputation applicationcontrol
19 Oct, 2022 core wizard setup
8 Sep, 2020 snmp core wireshark
30 Mar, 2022 core ethernet netwall coscore
9 Dec, 2022 core stateless connections
2 Feb, 2021 core sslvpn macos certificate
31 Jan, 2023 ldap core authentication radius
26 Jan, 2023 core rules transpose
23 Nov, 2022 core ipsec
7 Sep, 2022 core cli pcap netwall pcapdump
23 Aug, 2022 core ha cluster
17 Oct, 2022 core license
22 Mar, 2021 core ipsec routing
23 Aug, 2022 vmware log ha rarp arp core
24 Nov, 2022 core snmp
22 Jan, 2021 core ipreputation
15 Apr, 2021 core brokenlink cluster
17 Jun, 2021 core ipsec routing
23 Aug, 2022 core arp garp
9 Mar, 2021 core ping connections
5 Mar, 2021 sslvpn openconnect oneconnect linux core
8 Apr, 2021 core sslvpn oneconnect interfaces arp
30 Nov, 2022 core routing
1 Jun, 2022 core routing management
8 Sep, 2020 core ipreputation blacklist threatprevention
23 Jun, 2021 core connections
31 Mar, 2022 incontrol core netcon netwall ha cluster coscore
25 May, 2022 howto core authenticator authentication webauth captive
25 Nov, 2022 core routing bgp
13 May, 2022 core license
24 May, 2021 core howto mdns multicast transparentmode airprint igmp dns
21 Apr, 2021 core cluster ha
14 Apr, 2021 core license ipsec
8 Sep, 2020 core ipsec rules access
26 May, 2021 kvm core arm x86
27 Jan, 2023 ipsec core windows vpn l2tp
25 Jan, 2022 core ethernet settings
15 Nov, 2022 tcp log core
28 Nov, 2022 core loopback license
6 Jul, 2021 core stream tcpsequence sequence stateless
15 Nov, 2022 core cli
2 Nov, 2022 core threshold
11 Apr, 2021 core sfp gbic hardware
23 Aug, 2022 core oneconnect
28 Nov, 2022 core configuration oneconnect
17 Oct, 2022 core wcf
1 Dec, 2022 ipsec core
28 Oct, 2020 core howto ethernet packetloss cpu
14 Dec, 2022 ha core idp cli cluster antivirus configuration
24 Nov, 2021 core arm kvm
20 Jan, 2023 core log webui memlog
25 Nov, 2022 core configuration sslvpn management
7 Dec, 2022 pcapdump log cli core logsnoop
29 Jun, 2021 core oneconnect
7 Dec, 2022 ipsec ike troubleshoot core
23 May, 2022 core log logreceiver
14 Dec, 2022 core ipsec
23 Aug, 2022 howto core pbr routing netwall isp
20 May, 2021 kvm core arm coscore netwall
1 Dec, 2022 applicationcontrol core
23 Aug, 2022 core behaviour icmp ping traceroute
15 Dec, 2022 core routing ospf
6 Dec, 2022 core dns
7 Nov, 2022 core cpu troubleshoot
3 Feb, 2023 core trafficshaping pipes tcp
23 Aug, 2022 core ipsec license memory
7 Nov, 2022 core arp log routing
18 Nov, 2022 core cluster
30 Nov, 2020 howto core cloud-init dhcp
15 Dec, 2021 core idp ipreputation log4j
28 Nov, 2022 core stream
24 Mar, 2021 core connections
21 Jun, 2022 core ha hacluster netwall coscore slb
7 May, 2021 core ethernet vlan arp garp
2 Dec, 2022 netwall ikev2 windows certificate vpn core
10 Oct, 2022 core mtu netwall mtudiscovery
23 Aug, 2022 core connections ipsec memory
27 Jan, 2021 core stateless routing brokenlink
23 Aug, 2022 sslvpn openconnect oneconnect macos windows linux core
8 Jul, 2021 incontrol domains core
21 Nov, 2022 radius ldap authentication core
2 Dec, 2022 dhcp ipsec core