Setup of a Layer-3 bridge over IPsec in cOS Core

Last modified on 12 Apr, 2023. Revision 8
This guide will describe how to configure cOS Core to act as a layer-3 bridge over IPsec. The most common scenario is when a central site and a remote site have the same network and we want to "assign" part of this network to the remote site without the need to make any major changes on either side.
Up to date for
cOS Core 14.00.09
Supported since
cOS Core 9.30.xx
Status OK
Author
Peter Nilsson

Introduction and scenario

In a typical networking scenario, each site is assigned a unique IPv4 network address to ensure that devices within the network can communicate with each other without conflicts. However, in some cases, a central site and a remote site may have been assigned the same IPv4 network address. This could occur, for example, if the sites were set up by different organizations with overlapping IP address ranges or if the sites were merged without proper network planning.

This situation can create communication problems, as devices on one network may be unable to communicate with devices on the other network due to conflicting IP addresses.

This guide will go through one such scenario and how it can be solved using static routing and ProxyARP in cOS Core. In the above picture the hosts on the left would like to communicate with hosts on the right.

Limitations

There are two limitations that exists for this setup that is good to be aware of:

  1. Not possible to forward global broadcasts between HQ and Office-1.
    1. Details: Ethernet interfaces on a physical network are designed to transmit and receive packets only within the same broadcast domain. When a broadcast packet is sent on a network, it is transmitted by the source device to all devices on the same network segment. However, Ethernet interfaces are designed to only forward packets to devices that are on the same network segment as the source. This means that broadcast packets cannot be forwarded to devices on different network segments, even if those segments are part of the same physical network. Most routers/firewalls do not forward broadcast packets, because doing so could cause network congestion and security issues.
    2. Note: Directed Broadcasts are possible to forward as unlike global broadcasts, directed broadcasts can be forwarded between different network segments by routers, as long as the router/firewall has been configured to allow directed broadcast forwarding. For example, if the IP address of a network segment is 192.168.10.0/24, then the directed broadcast address for that segment would be 192.168.10.255. 
  2. Manually configure hosts.
    1. Details: Each single host (or range) must be configured manually on each site in order to tell the firewalls where the hosts are if they are not located in the local network. More details in the solution section.
    2. Note: The DHCP relay can be used to offset this problem as it has the option to automatically add route to incoming leases and have ProxyARP functionality. Basically the DHCP relay can be used to keep track of host located behind a VPN interface, assuming a DHCP server is located behind e.h. HQ and then configure the relayer on Office-1. We will not go into details about this workaround in this guide.

The solution

There are several ways to solve this particular problem but not all solutions is possible depending on how the network is designed, uptime requirements, integration with other systems etc. some of the possible solutions involve:

  • Changing the network on either side.
    • This is the preferred solution, but it is not always possible.
  • Creating a "fake network" that only exists inside the IPsec tunnel between HQ and Office-1.
  • Setup a small L3 bridge between HQ and Office-1 with statically configured routes.
    • This is the solution we will discuss in this KB.

How to configure

The general idea is to tell the firewalls at HQ and Office-1 were hosts not part of the local network is located behind an IPsec interface. This is possible to do due to the routing principle “smallest route first”, this means that a single-host e.g. 192.168.10.100 is smaller than 192.168.10.0/24 and will be consulted first, it is this principle we will use to solve the problem.

Note: We will not go into details about all settings, only the settings that are relevant to the scenario/configuration.

HQ configuration

Address book

Host-100 192.168.10.100

Host-101 192.168.10.101

Create an IPv4 group, then add the above hosts into the group

Group name = Office-1-Hosts

IPsec tunnel

IPsec tunnel name=VPN_To_Office-1

Local Network=192.168.10.0/24

Remote Network=192.168.10.0/24

Remove the setting for static routing as we want to configure the routes manually. The local / remote network are only definitions and will determine what kind of network/IP’s we want to send through the tunnel using static routing and IP policies. The screenshot below from cOS Core shows the setting.

Routing

Route VPN_To_Office-1 Office-1-hosts ProxyARP=LAN

IP policies

Allow Lan Lannet VPN_To_Office-1 Office-1-hosts service=All_Services
Allow VPN_To_Office-1 Office-1-hosts Lan Lannet service=All_Services

Office-1 configuration

Address book

Host-15 192.168.10.15

Host-16 192.168.10.16

Create a group, then add the above hosts into the group

Group name = HQ-Hosts

IPsec tunnel

IPsec tunnel name=VPN_To_HQ

Local Network=192.168.10.0/24

Remote Network=192.168.10.0/24

Remove the setting for static routing as we want to configure the routes manually. The local / remote network are only definitions and will determine what kind of network/IP’s we want to send through the tunnel using static routing and IP policies.

Routing

Route VPN_To_HQ HQ-hosts ProxyARP=LAN

IP policies

Allow Lan Lannet VPN_To_HQ HQ-hosts service=All_Services
Allow VPN_To_HQ HQ-hosts Lan Lannet service=All_Services

Summary

So what does all this do? What we have done here is to use the routing principle “smallest route first” to tell the firewall on HQ and Office-1 that some of the hosts that would normally be routed on the LAN interface are, instead, routed behind an IPsec tunnel. If we look at how the routing looks at HQ, it would look like the list below in the firewall’s routing table (for the routes related to this guide):

Route LAN 192.168.10.0/24
Route VPN_To_Office-1 192.168.10.100 ProxyARP=LAN
Route VPN_To_Office-1 192.168.10.101 ProxyARP=LAN

The above routing table entries would tell the firewall that all hosts in the 192.168.10.0/24 network are located behind the LAN interface, except the host 192.168.10.100 and 192.168.10.101 which would be routed behind an IPsec tunnel (also the exception would be 192.168.10.1 which would be interface IP of the LAN interface). ProxyARP is required as a host behind LAN asking for 192.168.10.100 or 192.168.10.101 would believe the target machine belongs to it’s own layer-2 network and would send an ARP query to locate it, and then the firewall must reply to that pretending to be 192.168.10.100 and 192.168.10.101 in terms of ARP. Then the requesting client is happy regarding ARP and would continue to send packets to the host which the firewall then forwards into the IPsec tunnel to the remote side.

Two IP policies are needed in case hosts from HQ to Office-1 should be allowed to initiate connections for either side. If only one side is allowed to initiate connections, only one IP policy would be needed.

Troubleshooting

This scenario would mainly have potential problems in 3 areas, IPsec tunnel, routing and rules. For IPsec related problems this KB article could be helpful: https://kb.clavister.com/324736401/troubleshooting-ipsec-tunnels-ikev1-

For rules and routing problems, using the ping simulation would be the best tool to use for this type of problem, more information about ping simulations can be found in the following KB: https://kb.clavister.com/324735909/troubleshooting-cos-core-rules-routes-with-ping-simulation

For ARP related problems, the CLI command “arpsnoop -verbose <interface>” can be very useful to get output from ARP queries to/from the selected interface(s).


Related articles

cOS Core IKEv2 split tunneling with Windows and local user database.
28 Mar, 2023 ikev2 windows vpn routing splittunneling
Problem with auto-created Core routes
22 Mar, 2021 core ipsec routing
Setting up OSPF with IPsec in cOS Core
16 Apr, 2024 core routing ospf ipsec
Using /31 network masks in cOS Core (RFC-3021)
1 Jun, 2022 core routing management
The meaning of the Default_Access_Rule log entry
7 Nov, 2022 core arp log routing
Troubleshooting cOS Core rules/routes with ping simulation
17 Mar, 2023 core routing rules ping icmp cli
Is Statless (FwdFast) faster than a normal IP policy?
27 Jan, 2021 core stateless routing brokenlink
Route failover with IPsec tunnels in cOS Core
13 Feb, 2023 ipsec core routing failover
Public network transparency using cOS Core Proxy ARP instead of subnetting
18 Apr, 2023 core routing transparentmode proxyarp