Handle a packet by netfilter after traversing a veth pair

Handle a packet by netfilter after traversing a veth pair

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 2190 bytes --]

Hello,

I have a use case when I may want to DNAT different addresses to a potentially
_same_ destination address that belongs to _different_ (virtual) hosts that are
connected to different VRFs. To achieve that, I plan to route the original
packet to a veth interface leading to the "right" VRF based on the packet's
original destination address. As the packet exists the peer of the veth pair,
(that is enslaved in the VRF), I want to apply DNAT rule in the prerouting hook.
"Dnated" destination address may be anything, and may happen to be the same in
different VRFs. This is how it looks:


+-------------+        +-------------+
|   GuestA    |        |   GuestB    |
| 192.168.1.2 |        | 192.168.1.2 |
+------|------+        +------|------+
       | tapA                 | tapB
+------|------+        +------|------+
| 192.168.1.1 |        | 192.168.1.1 |
|    VRF-A    |        |    VRF-B    |
| 100.64.10.2 |        | 100.64.20.2 |
+------|------+        +------|------+
       |                      |
     vethA                  vethB

    route:                 route:
100.64.10.2/32         100.64.20.2/32
  dev vethA              dev vethB
            \           /
             \         /

        default routing space


The problem that I see is that after the packet has traversed the veth, it
apparently is not handled by any netfilter hooks at all! I am assuming that a
veth pair ought to behave like two physical NICs connected by a crossover
Ethernet cable. That is, as the packet sent to one NIC enters from the other,
the system has no knowledge that the packet has visited before, treat it as a
fresh new packet of unknown origin, and let it pass through the whole stack of
routing and filtering. But I observe different behaviour.

I cannot apply DNAT to the packet _before_ it enters the VRF, because then
information needed to chose the right VRF will be lost.

My question is - how can I overcome this problem? Is there a way to ensure that
after traversing the veth the packet is treated as a new packet? Are there other
tricks, maybe, that would allow configuration as described?

Thank you,

Eugene


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Handle a packet by netfilter after traversing a veth pair

[Eugene Crosser]

[-- Attachment #1.1: Type: text/plain, Size: 2928 bytes --]

Answering my own question.

This is not a peculiarity of the veth; physical "loop" presumably would behave
the same way. The problem that I faced was caused by the fact that some(?) of
the hooks are apparently not executed when conntrack match was found. In my
scenario, the packet has created a conntrack entry on its first pass through the
system, and matched it (presumably) after its reentry via the veth pair.
Requesting `notrack` early in the first path solved the problem, `dnat` rule
started to work.

I only wish it were documented in some discoverable place...

On 13/07/2021 21:14, Eugene Crosser wrote:
> Hello,
> 
> I have a use case when I may want to DNAT different addresses to a potentially
> _same_ destination address that belongs to _different_ (virtual) hosts that are
> connected to different VRFs. To achieve that, I plan to route the original
> packet to a veth interface leading to the "right" VRF based on the packet's
> original destination address. As the packet exists the peer of the veth pair,
> (that is enslaved in the VRF), I want to apply DNAT rule in the prerouting hook.
> "Dnated" destination address may be anything, and may happen to be the same in
> different VRFs. This is how it looks:
> 
> 
> +-------------+        +-------------+
> |   GuestA    |        |   GuestB    |
> | 192.168.1.2 |        | 192.168.1.2 |
> +------|------+        +------|------+
>        | tapA                 | tapB
> +------|------+        +------|------+
> | 192.168.1.1 |        | 192.168.1.1 |
> |    VRF-A    |        |    VRF-B    |
> | 100.64.10.2 |        | 100.64.20.2 |
> +------|------+        +------|------+
>        |                      |
>      vethA                  vethB
> 
>     route:                 route:
> 100.64.10.2/32         100.64.20.2/32
>   dev vethA              dev vethB
>             \           /
>              \         /
> 
>         default routing space
> 
> 
> The problem that I see is that after the packet has traversed the veth, it
> apparently is not handled by any netfilter hooks at all! I am assuming that a
> veth pair ought to behave like two physical NICs connected by a crossover
> Ethernet cable. That is, as the packet sent to one NIC enters from the other,
> the system has no knowledge that the packet has visited before, treat it as a
> fresh new packet of unknown origin, and let it pass through the whole stack of
> routing and filtering. But I observe different behaviour.
> 
> I cannot apply DNAT to the packet _before_ it enters the VRF, because then
> information needed to chose the right VRF will be lost.
> 
> My question is - how can I overcome this problem? Is there a way to ensure that
> after traversing the veth the packet is treated as a new packet? Are there other
> tricks, maybe, that would allow configuration as described?
> 
> Thank you,
> 
> Eugene
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]