Flow label persistence

Flow label persistence

[Tom Herbert]

HI,

A potential issue came up on v6ops list in IETF that Linux stack
changes the flow label for a connection at every RTO, this is the
feature where we change the txhash on a failing connection to try to
find a route (the flow label is derived from the txhash). The problem
with changing the flow label for a connection is that it may cause
problems when stateful middleboxes are in the path, for instance if a
flow label change causes packets for a connection to take a different
route they might be forwarded to a different stateful firewall that
didn't see the 3WHS so don't have any flow state and hence drop the
packets.

I was under the assumption that we had a sysctl that would enable
changing the txhash for a connection and the default was off so that
flow labels would be persistent for the life of the connection.
Looking at the code now, I don't see that safety net, it looks like
the defauly behavior allows changing the hash. Am I missing something?

Thanks,
Tom

Re: Flow label persistence

[Eric Dumazet]

On 11/19/20 4:49 PM, Tom Herbert wrote:
> HI,
> 
> A potential issue came up on v6ops list in IETF that Linux stack
> changes the flow label for a connection at every RTO, this is the
> feature where we change the txhash on a failing connection to try to
> find a route (the flow label is derived from the txhash). The problem
> with changing the flow label for a connection is that it may cause
> problems when stateful middleboxes are in the path, for instance if a
> flow label change causes packets for a connection to take a different
> route they might be forwarded to a different stateful firewall that
> didn't see the 3WHS so don't have any flow state and hence drop the
> packets.
> 
> I was under the assumption that we had a sysctl that would enable
> changing the txhash for a connection and the default was off so that
> flow labels would be persistent for the life of the connection.
> Looking at the code now, I don't see that safety net, it looks like
> the defauly behavior allows changing the hash. Am I missing something?
> 
> Thanks,
> Tom
> 


"Stateful middleboxes" using flowlabels to identify a flow instead of the
standard 4-tuple are broken.

No sysctl addition/change can possibly help, since it wont appear magically
on billions of linux hosts.

Your question is a bit ironic, since historically you wrote the
first change for this stuff, without a sysctl.

commit 265f94ff54d62503663d9c788ba1f082e448f8b8
Author: Tom Herbert <tom@herbertland.com>
Date:   Tue Jul 28 16:02:06 2015 -0700

    net: Recompute sk_txhash on negative routing advice
    
    When a connection is failing a transport protocol calls
    dst_negative_advice to try to get a better route. This patch includes
    changing the sk_txhash in that function. This provides a rudimentary
    method to try to find a different path in the network since sk_txhash
    affects ECMP on the local host and through the network (via flow labels
    or UDP source port in encapsulation).
    
    Signed-off-by: Tom Herbert <tom@herbertland.com>
    Signed-off-by: David S. Miller <davem@davemloft.net>

Re: Flow label persistence

[Willem de Bruijn]

On Thu, Nov 19, 2020 at 1:00 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
>
>
> On 11/19/20 4:49 PM, Tom Herbert wrote:
> > HI,
> >
> > A potential issue came up on v6ops list in IETF that Linux stack
> > changes the flow label for a connection at every RTO, this is the
> > feature where we change the txhash on a failing connection to try to
> > find a route (the flow label is derived from the txhash). The problem
> > with changing the flow label for a connection is that it may cause
> > problems when stateful middleboxes are in the path, for instance if a
> > flow label change causes packets for a connection to take a different
> > route they might be forwarded to a different stateful firewall that
> > didn't see the 3WHS so don't have any flow state and hence drop the
> > packets.
> >
> > I was under the assumption that we had a sysctl that would enable
> > changing the txhash for a connection and the default was off so that
> > flow labels would be persistent for the life of the connection.
> > Looking at the code now, I don't see that safety net, it looks like
> > the defauly behavior allows changing the hash. Am I missing something?

Were you thinking of the net.ipv6.auto_flowlabels sysctl that can turn
off the entire feature (but not manually reserved flowlabels):

        if (flowlabel ||
            net->ipv6.sysctl.auto_flowlabels == IP6_AUTO_FLOW_LABEL_OFF ||
            (!autolabel &&
             net->ipv6.sysctl.auto_flowlabels != IP6_AUTO_FLOW_LABEL_FORCED))
                return flowlabel;

        hash = skb_get_hash_flowi6(skb, fl6);