* Re: [PATCH nf] netfilter: conntrack: always store window size un-scaled
2019-07-11 22:29 [PATCH nf] netfilter: conntrack: always store window size un-scaled Florian Westphal
@ 2019-07-12 10:50 ` Jozsef Kadlecsik
2019-07-16 11:23 ` Pablo Neira Ayuso
2019-07-27 12:15 ` Reindl Harald
2019-08-13 8:47 ` Thomas Jarosch
2 siblings, 1 reply; 5+ messages in thread
From: Jozsef Kadlecsik @ 2019-07-12 10:50 UTC (permalink / raw)
To: Florian Westphal; +Cc: netfilter-devel, Jakub Jankowski
On Fri, 12 Jul 2019, Florian Westphal wrote:
> Jakub Jankowski reported following oddity:
>
> After 3 way handshake completes, timeout of new connection is set to
> max_retrans (300s) instead of established (5 days).
>
> shortened excerpt from pcap provided:
> 25.070622 IP (flags [DF], proto TCP (6), length 52)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
> 26.070462 IP (flags [DF], proto TCP (6), length 48)
> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
> 27.070449 IP (flags [DF], proto TCP (6), length 40)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
>
> Turns out the last_win is of u16 type, but we store the scaled value:
> 512 << 8 (== 0x20000) becomes 0 window.
>
> The Fixes tag is not correct, as the bug has existed forever, but
> without that change all that this causes might cause is to mistake a
> window update (to-nonzero-from-zero) for a retransmit.
>
> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
> Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
It was a nice report and catch!
Best regards,
Jozsef
> ---
> net/netfilter/nf_conntrack_proto_tcp.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
> index 7ba01d8ee165..9fe1d5e46249 100644
> --- a/net/netfilter/nf_conntrack_proto_tcp.c
> +++ b/net/netfilter/nf_conntrack_proto_tcp.c
> @@ -475,6 +475,7 @@ static bool tcp_in_window(const struct nf_conn *ct,
> struct ip_ct_tcp_state *receiver = &state->seen[!dir];
> const struct nf_conntrack_tuple *tuple = &ct->tuplehash[dir].tuple;
> __u32 seq, ack, sack, end, win, swin;
> + u16 win_raw;
> s32 receiver_offset;
> bool res, in_recv_win;
>
> @@ -483,7 +484,8 @@ static bool tcp_in_window(const struct nf_conn *ct,
> */
> seq = ntohl(tcph->seq);
> ack = sack = ntohl(tcph->ack_seq);
> - win = ntohs(tcph->window);
> + win_raw = ntohs(tcph->window);
> + win = win_raw;
> end = segment_seq_plus_len(seq, skb->len, dataoff, tcph);
>
> if (receiver->flags & IP_CT_TCP_FLAG_SACK_PERM)
> @@ -658,14 +660,14 @@ static bool tcp_in_window(const struct nf_conn *ct,
> && state->last_seq == seq
> && state->last_ack == ack
> && state->last_end == end
> - && state->last_win == win)
> + && state->last_win == win_raw)
> state->retrans++;
> else {
> state->last_dir = dir;
> state->last_seq = seq;
> state->last_ack = ack;
> state->last_end = end;
> - state->last_win = win;
> + state->last_win = win_raw;
> state->retrans = 0;
> }
> }
> --
> 2.21.0
>
>
-
E-mail : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.mta.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics, Hungarian Academy of Sciences
H-1525 Budapest 114, POB. 49, Hungary
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH nf] netfilter: conntrack: always store window size un-scaled
2019-07-12 10:50 ` Jozsef Kadlecsik
@ 2019-07-16 11:23 ` Pablo Neira Ayuso
0 siblings, 0 replies; 5+ messages in thread
From: Pablo Neira Ayuso @ 2019-07-16 11:23 UTC (permalink / raw)
To: Jozsef Kadlecsik; +Cc: Florian Westphal, netfilter-devel, Jakub Jankowski
On Fri, Jul 12, 2019 at 12:50:35PM +0200, Jozsef Kadlecsik wrote:
> On Fri, 12 Jul 2019, Florian Westphal wrote:
>
> > Jakub Jankowski reported following oddity:
> >
> > After 3 way handshake completes, timeout of new connection is set to
> > max_retrans (300s) instead of established (5 days).
> >
> > shortened excerpt from pcap provided:
> > 25.070622 IP (flags [DF], proto TCP (6), length 52)
> > 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
> > 26.070462 IP (flags [DF], proto TCP (6), length 48)
> > 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
> > 27.070449 IP (flags [DF], proto TCP (6), length 40)
> > 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
> >
> > Turns out the last_win is of u16 type, but we store the scaled value:
> > 512 << 8 (== 0x20000) becomes 0 window.
> >
> > The Fixes tag is not correct, as the bug has existed forever, but
> > without that change all that this causes might cause is to mistake a
> > window update (to-nonzero-from-zero) for a retransmit.
> >
> > Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
> > Reported-by: Jakub Jankowski <shasta@toxcorp.com>
> > Tested-by: Jakub Jankowski <shasta@toxcorp.com>
> > Signed-off-by: Florian Westphal <fw@strlen.de>
>
> Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
Applied, thanks for reviewing Jozsef.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH nf] netfilter: conntrack: always store window size un-scaled
2019-07-11 22:29 [PATCH nf] netfilter: conntrack: always store window size un-scaled Florian Westphal
2019-07-12 10:50 ` Jozsef Kadlecsik
@ 2019-07-27 12:15 ` Reindl Harald
2019-08-13 8:47 ` Thomas Jarosch
2 siblings, 0 replies; 5+ messages in thread
From: Reindl Harald @ 2019-07-27 12:15 UTC (permalink / raw)
To: Florian Westphal, netfilter-devel; +Cc: Jakub Jankowski
this seemed to be fixed in 5.1.19 but not announced in
https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.1.19 and after
update to 5.1.20 last night the freezes of a ssh-tunneled vnc session
are starting again without exclude "lo" while the session was stable 3 days
see topic "Connection timeouts due to INVALID state rule"
Am 12.07.19 um 00:29 schrieb Florian Westphal:
> Jakub Jankowski reported following oddity:
>
> After 3 way handshake completes, timeout of new connection is set to
> max_retrans (300s) instead of established (5 days).
>
> shortened excerpt from pcap provided:
> 25.070622 IP (flags [DF], proto TCP (6), length 52)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
> 26.070462 IP (flags [DF], proto TCP (6), length 48)
> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
> 27.070449 IP (flags [DF], proto TCP (6), length 40)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
>
> Turns out the last_win is of u16 type, but we store the scaled value:
> 512 << 8 (== 0x20000) becomes 0 window.
>
> The Fixes tag is not correct, as the bug has existed forever, but
> without that change all that this causes might cause is to mistake a
> window update (to-nonzero-from-zero) for a retransmit.
>
> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
> Signed-off-by: Florian Westphal <fw@strlen.de>
> ---
> net/netfilter/nf_conntrack_proto_tcp.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
> index 7ba01d8ee165..9fe1d5e46249 100644
> --- a/net/netfilter/nf_conntrack_proto_tcp.c
> +++ b/net/netfilter/nf_conntrack_proto_tcp.c
> @@ -475,6 +475,7 @@ static bool tcp_in_window(const struct nf_conn *ct,
> struct ip_ct_tcp_state *receiver = &state->seen[!dir];
> const struct nf_conntrack_tuple *tuple = &ct->tuplehash[dir].tuple;
> __u32 seq, ack, sack, end, win, swin;
> + u16 win_raw;
> s32 receiver_offset;
> bool res, in_recv_win;
>
> @@ -483,7 +484,8 @@ static bool tcp_in_window(const struct nf_conn *ct,
> */
> seq = ntohl(tcph->seq);
> ack = sack = ntohl(tcph->ack_seq);
> - win = ntohs(tcph->window);
> + win_raw = ntohs(tcph->window);
> + win = win_raw;
> end = segment_seq_plus_len(seq, skb->len, dataoff, tcph);
>
> if (receiver->flags & IP_CT_TCP_FLAG_SACK_PERM)
> @@ -658,14 +660,14 @@ static bool tcp_in_window(const struct nf_conn *ct,
> && state->last_seq == seq
> && state->last_ack == ack
> && state->last_end == end
> - && state->last_win == win)
> + && state->last_win == win_raw)
> state->retrans++;
> else {
> state->last_dir = dir;
> state->last_seq = seq;
> state->last_ack = ack;
> state->last_end = end;
> - state->last_win = win;
> + state->last_win = win_raw;
> state->retrans = 0;
> }
> }
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH nf] netfilter: conntrack: always store window size un-scaled
2019-07-11 22:29 [PATCH nf] netfilter: conntrack: always store window size un-scaled Florian Westphal
2019-07-12 10:50 ` Jozsef Kadlecsik
2019-07-27 12:15 ` Reindl Harald
@ 2019-08-13 8:47 ` Thomas Jarosch
2 siblings, 0 replies; 5+ messages in thread
From: Thomas Jarosch @ 2019-08-13 8:47 UTC (permalink / raw)
To: Florian Westphal; +Cc: netfilter-devel, Jakub Jankowski
Hi Florian,
You wrote on Fri, Jul 12, 2019 at 12:29:05AM +0200:
> Jakub Jankowski reported following oddity:
>
> After 3 way handshake completes, timeout of new connection is set to
> max_retrans (300s) instead of established (5 days).
>
> shortened excerpt from pcap provided:
> 25.070622 IP (flags [DF], proto TCP (6), length 52)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
> 26.070462 IP (flags [DF], proto TCP (6), length 48)
> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
> 27.070449 IP (flags [DF], proto TCP (6), length 40)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
>
> Turns out the last_win is of u16 type, but we store the scaled value:
> 512 << 8 (== 0x20000) becomes 0 window.
>
> The Fixes tag is not correct, as the bug has existed forever, but
> without that change all that this causes might cause is to mistake a
> window update (to-nonzero-from-zero) for a retransmit.
>
> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
> Signed-off-by: Florian Westphal <fw@strlen.de>
it seems the patch fixed a kernel bugzilla entry, too:
https://bugzilla.kernel.org/show_bug.cgi?id=202287
I had a feeling it could be related since the reporter bisected it
down to changes in the TCP window scaling defaults.
Cheers,
Thomas
^ permalink raw reply [flat|nested] 5+ messages in thread