Device initiated InControl management of NetWall HA clusters with a single public IP

This how-to applies to:

Clavister cOS Core 15.00.07 and later, managed by InControl. cOS Core 15.00.08 and later need InControl 4.02.00 or later.

Problem:
You want InControl to manage a NetWall HA cluster over the Internet, but the cluster has only one public IP, for example one IP from DHCP on a backup WAN. Each node connects to InControl (Netcon) from its own private node IP, and the Internet drops it. So InControl never sees the nodes.

Solution:
Use Device Initiated management and send Netcon through a small loop. Each node sends Netcon out on an extra interface (ge2) that is cabled to the LAN switch. The active node receives it on its LAN interface (gesw) and NATs it out on the WAN, like normal LAN traffic. The firewall routes between ge2 and gesw, so this is not a Layer 2 loop. InControl sees both nodes behind the one public IP, also after an HA failover and a WAN change.

Used an earlier revision of this article without success? Add step 4 (the Access rule), and check step 5 and the Remote Management IDs. The old step that made gesw a member of the main table only is not needed, but you can keep it.

Before you start

• Netcon is set to Device Initiated, with the InControl Server IP, port 998 and a PSK. Each node has its own Remote Management ID, for example Master and Slave. HA does not synchronize the ID, so set it on each node. In InControl, the cluster has one Device Initiated member per node, with the ID of that node and the PSK.
• InControl accepts TCP 998 from the public IP of the cluster. If this IP can change, accept TCP 998 from any source.
• One free interface on each node (the same on both), and three free IPs in the LAN network, outside any DHCP pool.
• SSH to each node on its gesw node IP, for step 6 and in case Netcon goes down. Back up the configuration first.

The screenshots are from InControl, where gesw is If2, ge2 is If6, sh_If2 is the gesw shared IP and ic_pub is the InControl IP. The Web Interface has the same settings.

How to set this up

Steps 1 to 5 build the loop, step 6 checks it, and only then step 7 moves Netcon. In InControl, deploy steps 1 to 5 together and step 7 as a second deploy. In the Web Interface of the active node, activate after step 5 and after step 7. HA copies the changes to the other node.

Use a maintenance window: each activation reloads the nodes and can cause a failover (see Avoiding cOS Core HA interruptions during configuration deployment). Before step 6, wait until ha shows HA cluster peer is ALIVE.

Step 1 - Create the routing table

Add a routing table named incontrol with Ordering set to Only. With Default, a route in the main table can win, and Netcon does not use the loop.

Step 2 - Configure ge2 and cable it

• Virtual Routing tab, first: Make interface a member of a specific routing table, table incontrol.
• General tab: IP Address 10.10.10.5. Network is the LAN network, the same as on gesw. No default gateway (0.0.0.0). DHCP client off.
• Advanced tab: keep Automatically add a route for this interface on, turn Automatically add a default route off. Under High Availability, Private IP Address is an IP4 HA Address object with 10.10.10.6 (master) and 10.10.10.7 (slave).
• Connect ge2 on both nodes to the LAN switch, in the gesw VLAN.

Step 3 - Add the default route for the loop

In the incontrol table, add a route: Interface ge2, Network all-nets, Gateway the gesw shared IP. Any metric is fine. The second route in the table comes from the interface.

Step 4 - Add an Access rule on ge2

Required. Without it, no node connects. Add an Access rule: Action Accept, Interface ge2, Network the gesw shared IP, first in the list. Web Interface: Threat Prevention, Access Rules. InControl: Threat Prevention, General, Access Rules.

Why: ARP packets from the gesw shared IP arrive on ge2. The firewall sees this IP as its own in the incontrol table too, so the default anti-spoofing check drops them.

Step 5 - Add the NAT policy

Add an IP policy above your other rules:

• Source: interface gesw, network an address group with the ge2 shared IP and both ge2 node IPs. The active node sends the test in step 6 from the shared IP.
• Destination: an interface group with your WAN interfaces (one is fine), network the InControl IP.
• Service TCP 998, action Allow, source translation NAT with the IP of the outgoing interface.

Step 6 - Check the loop

SSH to each node on its gesw node IP and run:

ping <InControl IP> -pbr=incontrol -tcp -port=998 -count=3

After about 8 seconds the last line must start with TCP Ping Results: Sent: 3, RST/ACKs Received:3, Loss: 0%. If not, run it once more. If it fails again, stop here and see If it does not work. Netcon still uses its old path.

Also check that a LAN PC still reaches the Internet.

Step 7 - Move Netcon to the loop

In the Netcon object, set Outgoing Routing Table to incontrol and change nothing else. Set it once, HA copies it. Web Interface: System, Device, Remote Management. InControl: System, Device, Device Settings, Remote Management, NetconMgmt, field Outbound Routing Table.

Both nodes connect again within about 30 seconds. If the change breaks Netcon after a deploy from InControl, the firewall goes back by itself after the Validation Timeout (30 seconds by default), but InControl keeps the change: set the field back before the next deploy. In the Web Interface, do not count on this: run netcon on each node, and set the field back if a node shows no ge2 line.

Check that it works

• InControl shows the public IP of the cluster for both members in the Connection column, not the node IPs.
• On each node, netcon shows a line with ge2, the InControl IP, port 998 and (reverse).
• In a maintenance window, run ha -deactivate on the active node. After about 30 seconds both nodes show the ge2 line again. With a backup WAN, also cut the primary WAN before the firewall (for example switch off the ISP router, not the WAN cable of one node).

If it does not work

To watch the log in the CLI: logsnoop -on -event=invalid_arp_sender_ip_address -num=5, wait 10 seconds, then logsnoop -off.

More about Device Initiated management: InControl Administration Guide. To reach the Web Interface of the active node through the same public IP, see Manage NetWall HA cluster with a Single Public IP Address.

Related articles