Why are two hash tuples stored for each connection in the connection tracking system?

Why are two hash tuples stored for each connection in the connection tracking system?

[Will Sewell]

Hi,

The connection tracking system stores one hash tuple of layer 3/4
information for each packet's original direction, and one for the
reply direction. The are embedded in the conntrack struct which stores
the actual connection state.

I'm curious why this is. Is it so connection state can be looked up
with only partial information, e.g. only the reply direction
information? If that's true, then in which cases do we only have
partial information? My assumption would be that information on both
direction would be available in the packet.

I'm also curious why these hash tuples are stored in a doubly linked
list, rather than a singly linked list. Is it just for more efficient
deletion?

Thank you,
Will

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 27/09/2017 à 09:52, Will Sewell a écrit :
> 
> The connection tracking system stores one hash tuple of layer 3/4
> information for each packet's original direction, and one for the
> reply direction. The are embedded in the conntrack struct which stores
> the actual connection state.
> 
> I'm curious why this is.

Because conntrack has to process packets in both directions, and the 
packet header fields in the reply direction may not be obtained just by 
swapping the fields in the original direction.

A few examples :
- DNAT : original destination address/port != reply source address/port
- SNAT : original source address/port != reply destination address/port
- ICMP echo/reply : original type/code 8/0 != reply type/code 8/0

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 27/09/2017 à 10:20, Pascal Hambourg a écrit :
> Le 27/09/2017 à 09:52, Will Sewell a écrit >>
>> The connection tracking system stores one hash tuple of layer 3/4
>> information for each packet's original direction, and one for the
>> reply direction. The are embedded in the conntrack struct which stores
>> the actual connection state.
>>
>> I'm curious why this is.
> 
> Because conntrack has to process packets in both directions, and the 
> packet header fields in the reply direction may not be obtained just by 
> swapping the fields in the original direction.
> 
> A few examples :
> - DNAT : original destination address/port != reply source address/port
> - SNAT : original source address/port != reply destination address/port
> - ICMP echo/reply : original type/code 8/0 != reply type/code 8/0

Oops, I meant to wrote :
- ICMP echo request/reply : original type/code 8/0 != reply type/code 0/0

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Will Sewell]

> Because conntrack has to process packets in both directions, and the
> packet header fields in the reply direction may not be obtained just by
> swapping the fields in the original direction.

Thanks. That makes sense. I still have some confusion however: in the
case where the original/reply connection information is not equal, how
does the connection tracking system ensure that both hash tuples are
embedded in the same conntrack struct (nf_conn)? Let's say an original
packet arrives with some new original direction connection
information. I'm assuming conntrack will allocate a new nf_conn struct
with just the original hash tuple embedded. When a packet arrives from
the reply direction, how does conntrack find the correct nf_conn to
embed it in? From what I can tell the orifginal connection information
is required to look this up.

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 28/09/2017 à 17:09, Will Sewell a écrit :
>> Because conntrack has to process packets in both directions, and the
>> packet header fields in the reply direction may not be obtained just by
>> swapping the fields in the original direction.
> 
> Thanks. That makes sense. I still have some confusion however: in the
> case where the original/reply connection information is not equal, how
> does the connection tracking system ensure that both hash tuples are
> embedded in the same conntrack struct (nf_conn)? Let's say an original
> packet arrives with some new original direction connection
> information. I'm assuming conntrack will allocate a new nf_conn struct
> with just the original hash tuple embedded. When a packet arrives from
> the reply direction, how does conntrack find the correct nf_conn to
> embed it in? From what I can tell the orifginal connection information
> is required to look this up.

I don't know the detail of conntrack data structures, but conntrack 
watches packets in two places :
- when the packet enters netfilter (PREROUTING or OUTPUT)
- when the packet leaves netfilter (POSTROUTING or INPUT)

When the first original packet enters netfilter, its characteristics are 
used to define the original tuple.

When the packet leaves netfilter after being transformed, its 
characteristics are used to define the reply tuple matching expected 
reply packets.

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Will Sewell]

> I don't know the detail of conntrack data structures, but conntrack watches
> packets in two places :
> - when the packet enters netfilter (PREROUTING or OUTPUT)
> - when the packet leaves netfilter (POSTROUTING or INPUT)

Ah, I think my confusion came from the meaning of the "original" and
"reply" directions. I had assumed original packets were those packets
entering Netfilter from the outside network, and reply packets
entering Netfilter from the local machine and which are sent
externally. It sounds like you're saying original packets are what
what enters Netfiler, and reply packets are those which leave
Netfilter (after potentially being transformed) and are propagated up
the networking stack. In which case my previous goes away because
conntrack can hold a reference to the connection to the conntrack
struct while the packet is passing through Netfilter.

Thank you,
Will

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 02/10/2017 à 14:07, Will Sewell a écrit :
>> I don't know the detail of conntrack data structures, but conntrack watches
>> packets in two places :
>> - when the packet enters netfilter (PREROUTING or OUTPUT)
>> - when the packet leaves netfilter (POSTROUTING or INPUT)
> 
> Ah, I think my confusion came from the meaning of the "original" and
> "reply" directions. I had assumed original packets were those packets
> entering Netfilter from the outside network, and reply packets
> entering Netfilter from the local machine and which are sent
> externally.

This is only 25% true, when the connection comes from the outside and is 
directed at the local host. It is the opposite when the connection 
originates from the local host and is directed at a remote host.
It is wrong for local to local connections (no packet is going outside) 
and forwarded connections (no packet is sent or received locally).

> It sounds like you're saying original packets are what
> what enters Netfiler, and reply packets are those which leave
> Netfilter (after potentially being transformed) and are propagated up
> the networking stack.

You totally misunderstood me.
All packets in both direction enter netfilter, and all packets leave 
netfilter unless they are discarded or queued.

The "direction" (original or reply) is not defined by the path into 
netfilter and the IP stack. It is defined by the first packet of the 
connection from the client to the server, the one which creates the 
conntrack entry. This packet and subsequent packets with similar 
features are in the original direction. Packets from the server to the 
client are in the reply direction. This is true regardless of whether 
the local host is the client, the server or an intermediate router.

Let me illustrate this with an example :

A TCP connection from 192.0.2.6:6666 to 192.0.2.7:7777
Apply DNAT to destination 192.0.2.8:8888
Apply SNAT to source 192.0.2.9:9999

Packet in the original direction when entering netfilter :
source address 192.192.0.2.6
source port 6666
destination address 192.192.0.2.7
destination port 7777

Packet in the original direction when leaving netfilter :
source address 192.192.0.2.9
source port 9999
destination address 192.192.0.2.8
destination port 8888

Packet in the reply direction when entering netfilter :
source address 192.192.0.2.8
source port 8888
destination address 192.192.0.2.9
destination port 9999

Packet in the reply direction when leaving netfilter :
source address 192.192.0.2.7
source port 7777
destination address 192.192.0.2.6
destination port 6666

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Will Sewell]

> You totally misunderstood me.
> All packets in both direction enter netfilter, and all packets leave
> netfilter unless they are discarded or queued.

Thanks for the example. It looks like my original intuition for what
the "original" and "reply" directions meant was correct (even though I
described it quite imprecisely).

This makes me still unclear on how conntrack figures out that the
original and reply packets are related. e.g. from the information in
the packet entering netfilter in the reply direction, how can it
determine that it is part of the same connection as the packet
entering netfilter in the original direction. I'll try and figure this
out and reply to this thread if I do.

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 03/10/2017 à 14:32, Will Sewell a écrit :
> 
> This makes me still unclear on how conntrack figures out that the
> original and reply packets are related. e.g. from the information in
> the packet entering netfilter in the reply direction, how can it
> determine that it is part of the same connection as the packet
> entering netfilter in the original direction. I'll try and figure this
> out and reply to this thread if I do.

Quite simple. After conntrack has seen the first original packet leave, 
it knows what a reply packet should look like when it enters.
Roughly, a packet entering netfilter in the reply direction is the 
reverse of a packet leaving netfilter in the original direction.

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Will Sewell]

> Quite simple. After conntrack has seen the first original packet leave, it
> knows what a reply packet should look like when it enters.
> Roughly, a packet entering netfilter in the reply direction is the reverse
> of a packet leaving netfilter in the original direction.

Ah, so are you saying conntrack can simply switch the source
information with the destination information of a original direction
packet leaving netfilter to get the source/destination information of
a packet that is entering netfilter from the reply direction? That
makes sense if so!

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 03/10/2017 à 15:23, Will Sewell a écrit :
> 
> Ah, so are you saying conntrack can simply switch the source
> information with the destination information of a original direction
> packet leaving netfilter to get the source/destination information of
> a packet that is entering netfilter from the reply direction?

Roughly, yes.
There are variations for handling of "special" protocols such as ICMP 
which use header fields other than source/destination ports to identify 
connections.

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Pascal Hambourg]

Le 03/10/2017 à 16:03, Payam Chychi a écrit :
>
> The concept of an original packet that is then tracked in table would be a
> tcp syn flag. syn bit is set to instruct that this is the 1st packet and
> its intention is to creat a connection/open socket.
> 
> reply traffic relys on the 3way handshake for TCP (other methods used for
> UDP) which includes seq/ack values to track the progress as well as other
> attributes

These are transport protocol-specific details. Netfilter connection 
tracking is more generic than this. TCP connection tracking does not 
even require by default that the first packet has the SYN flag.

Re: Why are two hash tuples stored for each connection in the connection tracking system?

[Will Sewell]

> There are variations for handling of "special" protocols such as ICMP which
> use header fields other than source/destination ports to identify
> connections.

Makes sense. Thanks again for your help!