Linux IPV6 TCP egress path device passed for LOCAL_OUT hook is incorrect

Linux IPV6 TCP egress path device passed for LOCAL_OUT hook is incorrect

[Preethi Ramachandra]

Hi David Ahren,

In TCP egress path for ipv6 the device passed to NF_INET_LOCAL_OUT hook should be skb_dst(skb)->dev instead of dst->dev.

https://elixir.bootlin.com/linux/latest/source/net/ipv6/ip6_output.c#L202
struct dst_entry *dst = skb_dst(skb); >>> This may return slave device.

In this code path the DST Dev and SKB DST Dev will be set to VRF master device.
ip6_xmit->l3mdev_ip6_out->vrf_l3_out->vrf_ip6_out (This will set SKB DST Dev to vrf0)

However, once the control passes back to ip6_xmit, https://elixir.bootlin.com/linux/latest/source/net/ipv6/ip6_output.c#L280
Slave device is passed to LOCAL_OUT nf_hook instead of skb_dst(skb)->dev.

return NF_HOOK(NFPROTO_IPV6, NF_INET_LOCAL_OUT,
       net, (struct sock *)sk, skb, NULL, dst->dev, <<<< Should be skb_dst(skb)->dev
       dst_output);

This will cause a bug in firewall filters. nf_hook->ip6table_mangle_hook->ip6table_filter_hook (Device passed is slave device). If the firewall filter rule is configured with device as VRF it will fail to apply the filter. As per Linux documentation filters should work for VRF devices.

Firewall Rule:

ip6tables -A OUTPUT -o vrf0 -m comment --comment F-outer6_T-B-lo0_I-1001 -j outer6 (or -A OUTPUT -o vrf0 -m mark --mark 0x1000000 -m comment --comment F-outer6_T-B-lo0_I-1001 -j outer6 (Rule fails to apply for TCP packets)

Linux firewall doc:

https://www.kernel.org/doc/Documentation/networking/vrf.txt

[2] Iptables on ingress supports PREROUTING with skb->dev set to the real
    ingress device and both INPUT and PREROUTING rules with skb->dev set to
    the VRF device. For egress POSTROUTING and OUTPUT rules can be written
    using either the VRF device or real egress device.

Comparison on device passed to LOCALOUT hook for IPV4 TCP and IPV6 raw/udp scenarios.

TCP IPV4 egress : In __ip_local_out to NF_INET_LOCAL_OUT skb_dst(skb)->dev is passed.

https://elixir.bootlin.com/linux/latest/source/net/ipv4/ip_output.c#L115
return nf_hook(NFPROTO_IPV4, NF_INET_LOCAL_OUT,
                      net, sk, skb, NULL, skb_dst(skb)->dev, << will be set to vrf0
                      dst_output);
}

Raw/UDP V6 egress path: skb_dst(skb)->dev is passed onto NF_INET_LOCAL_OUT

rawv6_sendmsg-> ip6_send_skb-> ip6_local_out-> __ip6_local_out (LOCAL_OUT hook, device passed is SKB DST which is VRF)->nf_hook-> ip6table_mangle_hook->ip6table_filter_hook

https://elixir.bootlin.com/linux/latest/source/net/ipv6/output_core.c#L167

return nf_hook(NFPROTO_IPV6, NF_INET_LOCAL_OUT,
       net, sk, skb, NULL, skb_dst(skb)->dev,
       dst_output);

Thanks,
Preethi


Juniper Business Use Only

Re: Linux IPV6 TCP egress path device passed for LOCAL_OUT hook is incorrect

[David Ahern]

On 12/2/20 4:31 AM, Preethi Ramachandra wrote:
> Hi David Ahren,
> 
> In TCP egress path for ipv6 the device passed to NF_INET_LOCAL_OUT hook should be skb_dst(skb)->dev instead of dst->dev.
> 
> https://elixir.bootlin.com/linux/latest/source/net/ipv6/ip6_output.c#L202
> struct dst_entry *dst = skb_dst(skb); >>> This may return slave device.
> 
> In this code path the DST Dev and SKB DST Dev will be set to VRF master device.
> ip6_xmit->l3mdev_ip6_out->vrf_l3_out->vrf_ip6_out (This will set SKB DST Dev to vrf0)
> 
> However, once the control passes back to ip6_xmit, https://elixir.bootlin.com/linux/latest/source/net/ipv6/ip6_output.c#L280
> Slave device is passed to LOCAL_OUT nf_hook instead of skb_dst(skb)->dev.
> 
> return NF_HOOK(NFPROTO_IPV6, NF_INET_LOCAL_OUT,
>        net, (struct sock *)sk, skb, NULL, dst->dev, <<<< Should be skb_dst(skb)->dev
>        dst_output);

The vrf device version of that call is managed by the vrf driver. See
vrf_ip6_out_direct():

       err = nf_hook(NFPROTO_IPV6, NF_INET_LOCAL_OUT, net, sk,
                      skb, NULL, vrf_dev, vrf_ip6_out_direct_finish);

Re: Linux IPV6 TCP egress path device passed for LOCAL_OUT hook is incorrect

[Preethi Ramachandra]

Thanks David for the response.
Our local kernel doesn’t have vrf_ip6_out_direct patch. I will pick this change from upstream to fix the firewall issue.

Thanks,
Preethi

On 03/12/20, 5:42 AM, "David Ahern" <dsahern@gmail.com> wrote:

    [External Email. Be cautious of content]


    On 12/2/20 4:31 AM, Preethi Ramachandra wrote:
    > Hi David Ahren,
    >
    > In TCP egress path for ipv6 the device passed to NF_INET_LOCAL_OUT hook should be skb_dst(skb)->dev instead of dst->dev.
    >
    > https://urldefense.com/v3/__https://elixir.bootlin.com/linux/latest/source/net/ipv6/ip6_output.c*L202__;Iw!!NEt6yMaO-gk!TTEMIr6Y_qiOz_QY3_Fh4QnRglxHEfu1mwoSo_Sve_Q6rdpmCMaf3l8ZCBNAnN4W$
    > struct dst_entry *dst = skb_dst(skb); >>> This may return slave device.
    >
    > In this code path the DST Dev and SKB DST Dev will be set to VRF master device.
    > ip6_xmit->l3mdev_ip6_out->vrf_l3_out->vrf_ip6_out (This will set SKB DST Dev to vrf0)
    >
    > However, once the control passes back to ip6_xmit, https://urldefense.com/v3/__https://elixir.bootlin.com/linux/latest/source/net/ipv6/ip6_output.c*L280__;Iw!!NEt6yMaO-gk!TTEMIr6Y_qiOz_QY3_Fh4QnRglxHEfu1mwoSo_Sve_Q6rdpmCMaf3l8ZCDcZbfyI$
    > Slave device is passed to LOCAL_OUT nf_hook instead of skb_dst(skb)->dev.
    >
    > return NF_HOOK(NFPROTO_IPV6, NF_INET_LOCAL_OUT,
    >        net, (struct sock *)sk, skb, NULL, dst->dev, <<<< Should be skb_dst(skb)->dev
    >        dst_output);

    The vrf device version of that call is managed by the vrf driver. See
    vrf_ip6_out_direct():

           err = nf_hook(NFPROTO_IPV6, NF_INET_LOCAL_OUT, net, sk,
                          skb, NULL, vrf_dev, vrf_ip6_out_direct_finish);


Juniper Business Use Only