Troubleshooting cOS Core rules/routes with ping simulation

Last modified on 17 Mar, 2023. Revision 18
This article will describe using the CLI "ping" command's packet simulation option to get output from tests for troubleshooting issues with IP rule set and routing table lookups.
Up to date for
cOS Core 14.00.09
Status OK
Author
Peter Nilsson

Using ping packet simulation

A useful tool when trying to troubleshoot problems with the IP rule sets and/or routing tables in a NetWall firewall configuration is the packet simulation option of the “ping” command. This option injects a ping packet on a chosen interface with a simulated (faked) sender address, sending it to a desired destination address. This simulates a packet entering the firewall on one interface, passing through cOS Core processing and exiting on the interface decided by the associated routing table lookup. The route chosen and the IP rule set entries that trigger can be viewed. This can show if the expected rule or route triggered.

Note that access to the CLI to issue the ping command could be via SSH over a network, directly via the local console, or through a CLI console in the InControl client if the firewall is under InControl centralized management.

Command syntax

The command syntax for ping simulation is as follows (the text in brackets is chosen by the user):

ping <destination IP> -srcif=<source_interface> -srcip=<source_ip> -verbose

Type “help ping” in the console for a detailed explanation of the syntax.

An example scenario

  • 192.168.200.0/24 is the private internal network.
  • 192.168.98.33 is the external IP of the firewall.
  • 192.168.98.1 is the external gateway of the firewall.
  • We have an IP rule set entry that allows ping traffic, as shown below:

An example of a successful attempt

VSG-31:/> ping 192.36.125.18 -srcif=lan -srcip=192.168.200.50 -verbose

This will send a ping packet (ICMP):

  • To the host 192.36.125.18, located at Sunet (the Swedish University Network)
  • Simulated that the packet arrived from the Lan interface
  • Simulated that the packets source IP addressed came from 192.168.200.50
  • The output is in verbose mode which provides additional output.


Since the IP 192.168.200.50 is routed on the Lan interface, this will be a successful simulation and the output will look like this.

VSG-31:/> ping 192.36.125.18 -srcif=Lan -srcip=192.168.200.50 -verbose
Rule and routing information for ping:
allowed by rule "Lan_To_Internet_All"

Sending 1 4-byte ICMP ping to 192.36.125.18 from 192.168.98.33
sent via route "0.0.0.0/0 via Wan, gw 192.168.98.1" in PBR table "main"
ICMP Reply from 192.36.125.18 seq=0 time= 10 ms TTL=53

Ping Results: Sent: 1, Received:1, Avg RTT: 10.0 ms


We will see which rule triggered for the traffic as well as what route the traffic will follow (what destination interface the traffic is sent out on). This can be of great help when troubleshooting. The source IP will be 192.168.98.33 instead of 192.168.200.50 since the IP policy is performing NAT and will use the IP address set on the Wan interface as the sending IP towards the internet (or in this case, the next jump/router).

Unsuccessful example 1 - Routing

So what would happen if we change the <src_ip> to an IP address that is not routed on the Lan interface?

VSG-31:/> ping 192.36.125.18 -srcif=lan -srcip=192.168.100.50 -v
Rule and routing information for ping:
DROPPED by rule "Default_Access_Rule"

The traffic is dropped by Default_Access_Rule, which means that the source network is NOT routed on the Lan interface when the firewall performs the route lookup, which is correct in this case as we have 192.168.200.0/24 on the Lan interface and not 192.168.100.0/24. This is also known as ingress routing, where we check that the incoming packet’s source address is valid on the receiving interface.

We have concluded that it is a routing problem we need to look for. The message “Default_Access_Rule” is ALWAYS a routing problem, it ALWAYS means that the packet received on the receiving interface is not routed there based on the firewalls route lookup.

Unsuccessful example 2 - Rules

To take another example, what if we change the simulation to try use another port/protocol that is not allowed by the IP rule set.

VSG-31:/> ping 192.36.125.18 -srcif=Lan -srcip=192.168.200.50 -verbose -tcp -port=8080
Rule and routing information for ping:
TCP: 192.168.200.50:25135 -> 192.36.125.18:8080 DROPPED by rule "Default_Rule"

The above output will print out a “Default_Rule” drop. It is quite similar to “Default_Access_Rule” this does not indicate a routing problem but instead a rule problem. It means that no matching rule/policy was found in the entire rule set and the traffic was then dropped by a “catch all” rule drop called “Default_Rule”.

In order to make the above ping simulation work, we need to allow/open TCP port 8080 towards the internet.

A note regarding using ping simulations through an ALG (Application Layer Gateway)

Using ping simulation will always result in 100% packet loss if the rule/policy triggering is using an ALG (with features such as Anti-Virus, Web Content Filtering, File control and more). The simulation can still give valuable output such indicating which rule triggers. However, the route lookup may look as if the target route is routed on the Core interface and that is due to the ALG being involved. This is a known limitation in the ping simulation.

Further explanation and examples

Apart from the cOS Core Administration Guide, the ping simulation feature is also explained in greater detail in the cOS Core VPN Cookbook (Appendix-A) along with more examples. The VPN cookbook can be freely downloaded as a PDF from here : https://www.clavister.com/services/resources/configuration-cookbooks/


Related articles

Using PCAP packet capture in cOS Core
7 Sep, 2022 core cli pcap netwall pcapdump
Avoiding cOS Core HA interruptions during configuration deployment
20 Feb, 2023 ha core idp cli cluster antivirus configuration
Automatically stop active PCAPdump or Logsnoop in the CLI
7 Dec, 2022 pcapdump log cli core logsnoop