Introduction

When managing a NetWall firewall, there may be situations where cOS Core is blocking a TCP connection that seems suspicious. This can be due to the way cOS Core handles connections where the it has been told to close the connection by receiving a FIN notification from either the client or server side, but if the client (or server) tries to re-open or create a new connection that looks identical to the old connection, cOS Core would, by default, not accept that and drop the packet. An example of a log message generated by this situation is:

LogStateViolations;Warning;TCP_FLAG;unexpected_tcp_flags;drop;WAN:192.0.2.50:57909;203.0.113.77:443;"IP: TCP";"Unexpected tcp flags SYN from originator during state FIN_RCVD. Dropping"

The FIN_RCVD state

When a normal TCP connection is opened, a 3-way handshake occurs where the client/server agrees to open a connection from A to B on port X and Y. Then they can start sending data back and forth as needed, and when the connection is no longer needed, one of the sides sends a FIN, which tells all parties involved that the connection can now be closed.

But cOS Core does not close the connection directly at this stage. Instead, it places the connection in a “FIN_RCVD” (FIN Received) state where the specific connection is not allowed to be used for the next 80 seconds. We can think of it as a type of queue where the connection will be removed after 80 seconds and after those 80 seconds the exact connection IP & Port combination can be re-used again.

This is a type of security mechanism where cOS Core does not allow clients to attempt to reuse the same connection again within that 80 timeout. When a new connection is created, such a connection should reasonably use at least a new source port, and cOS Core then handles it as a new connection, and there are no problems.

A connection example

Here is an example of an HTTP connection to 203.0.113.77, as seen in cOS Core CLI output from the connections command:

TCP_OPEN TCP Wan:192.0.2.50:5555 core:203.0.113.77:443 262142

Where 5555 is a random source port and 443 is the destination port. As new connections are created, it would look something like this if we have multiple connections to the same destination IP.

Device:/> connections
State Proto Source Destination Tmout
-------- ------- --------------------------- --------------------------- ------
TCP_OPEN TCP Wan:192.0.2.50:5555 core:203.0.113.77:443 262142
TCP_OPEN TCP Wan:192.0.2.50:8215 core:203.0.113.77:443 175443
TCP_OPEN TCP Wan:192.0.2.50:17312 core:203.0.113.77:443 201441

An example of how to trigger the log message

For each new connection, there is (normally) a new source port generated. But in our example, the following happens

1. A new connection is created with the following IP/port combination:

   TCP_OPEN TCP Wan:192.0.2.50:5555 core:203.0.113.77:443 262142

2. cOS Core receive a FIN on the above connection combination telling cOS Core that this connection should be terminated/closed, cOS Core then places the above connection in the FIN_RCVD state (80 seconds) where it awaits closure:

   FIN_RCVD TCP Wan:192.0.2.50:5555 core:203.0.113.77:443 80

3. A new connection attempts to be created before the 80 second timeout has reached zero by sending a TCP SYN with the following IP port combination:

   SYN 192.0.2.50:5555 203.0.113.77:443

This is then the same connection/port combination that is in the FIN_RCVD state, and cOS Core find this a bit suspicious and drop the packet due to “Unexpected TCP Flags”.

After 80 seconds, this IP/Port combination is allowed to be used again.

Allowing the behavior

The behavior is a bit unusual but there have been reports of systems opening and closing the same TCP connection rapidly, in one example the time between FIN and a new SYN for the same IP/Port combination was less than 1 second.

To stop cOS Core from blocking/dropping the traffic when a connection is in the FIN_RCVD state there are a few options

1. Allow the TCP connection to reopen when it is in the FIN_RCVD state. This option can be found under System->Advanced settings->TCP->"Allow TCP Reopen."
   1. Note: This changes the behavior globally for all TCP connections
2. Change the global timeout value related to the FIN_RCVD state, this option can be found under System→Advanced Settings->TCP->TCP FIN Idle Lifetime.
   1. Note: This changes the behavior globally for all TCP connections
3. Create a custom timeout for the service that has this problem (then we also avoid changing the global timeout values). We can do this under Objects->Services->YourObject->Custom Timeouts. The value for "TCP Closing Idle Lifetime" is what we want to set to a lower value on as this is related to the FIN_RCVD state.
   1. Note-1: In older versions, the lowest value was 1, but it is possible to set it to 0 in newer versions (14.00.06 and above). A zero value means that the connection will be removed immediately once a FIN has been received.
   2. Note-2: There is no need to set a value for all fields, those left blank will use the global default values.

The recommended method is #3 as it can then be customized for only a specific service/port and then used in IP policies for a limited amount of devices without affecting the firewall as a whole.

Related articles