Problem:

There are several situations that can occur when the Firewall is running low on available memory.

Note: What defines as low can also vary a bit depending on which functions/features that are used in the Firewall, but overall having at least 100 MB of free RAM is recommended.

• Many functions/features are activated in the Firewall that causes the currently installed RAM to be insufficient. A few examples would be, Anti-Virus, IDP, ALG's, IPsec tunnels and more. The more features that are activated, the higher the "cost" of memory and CPU.
  • This can also happen for appliance models.
• The traffic pattern requires the adjustments on some advanced settings that cause the memory usage to increase. A few examples would be increasing the RingSize on the Ethernet interfaces, increasing the number of rules for IPsec, increasing HighBuffers and more.
• Newer cOS Core versions uses more memory than the previous version.
  • This is quite normal. Clavister is constantly adding new features/functions that over time would make cOS Core allocate more memory, newer graphics, scripts, features and more as the product is enhanced.
• Memory leak
  • This happens from time to time due to bugs and other problems. Having more available RAM helps in lessen the impact and time it would take before memory becomes critical, making the impact less urgent until such time as Clavister has found and solved the problem.
• Memory fragmentation
  • Although unusual, there are some features such as Anti-Virus that are sensitive to memory fragmentation. The more memory the less chance/need of memory fragmentation. This usually has the effect that the Firewall sometimes needs to be restarted to clear up the fragmentation.

Solution:

There are several areas that can be adjusted to free up more memory, but the two biggest memory pre-allocations are done by Connections and IPsec tunnels. Connections and IPsec tunnels also interact with each other, meaning that memory allocations for IPsec tunnels would be higher the more connections you have in configured (default is based on the license unless adjusted, see below).

Adjusting connections and IPsec tunnel max values

Lets say our license supports 512 000 connections and 500 IPsec tunnels. We conclude that we would never need more than 128 000 connections and 100 IPsec tunnels. By default cOS Core looks at the license and allocates memory based on that. These settings can be overridden and a manual value can be entered.

• Connection limit can be configured under "Advanced Settings->State Settings->Max Connections".
  • Uncheck the checkbox for "dynamic" and input a manual value
  • Calculation on memory per connection can be viewed if you run the "memory" CLI command. Currently (tested version 13.00.08) cOS Core allocates 412 bytes per connection. So 512 000 connections would require about 210 MB of RAM while 128 000 uses about 50. So we can save at least 150 MB of RAM by going from 512 000 connections down to 128 000.
• For IPsec the option is under "Interfaces->IPsec->Advanced Settings->IPsec Max Tunnels".
  • Default value is zero, which means "based on license". Input a manual value.

By adjusting the above settings (based on preference) we free up more available memory. We can then activate and use some of the more memory consuming features without having to add more memory to the unit (if a Virtual Firewall). Or if a memory leak or other problem is encountered that is related to memory; by freeing up more memory we lessen the time (and chance) that the problem triggers. For instance, in case of a memory leak, the need to reboot the Firewall once a week would instead be e.g. once a month.

Important note 1: Making changes to Max Connections or IPsec Max Tunnels requires a system restart to take effect. The reason for this is because memory allocations for these functions/features are performed at system boot.

Important note 2: Making changes to Max Connections will cause all currently opened connections to be torn down. This would cause disruptions in the network, it is recommended that this change be done out of office hours or during a planned maintenance window.

Related articles