netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module
@ 2019-08-02 13:22 Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 03/42] netfilter: Fix rpfilter dropping vrf packets by mistake Sasha Levin
                   ` (4 more replies)
  0 siblings, 5 replies; 10+ messages in thread
From: Sasha Levin @ 2019-08-02 13:22 UTC (permalink / raw)
  To: linux-kernel, stable
  Cc: Florian Westphal, Thomas Jarosch, Juliana Rodrigueiro,
	Pablo Neira Ayuso, Sasha Levin, netfilter-devel, coreteam,
	netdev

From: Florian Westphal <fw@strlen.de>

[ Upstream commit 1b0890cd60829bd51455dc5ad689ed58c4408227 ]

Thomas and Juliana report a deadlock when running:

(rmmod nf_conntrack_netlink/xfrm_user)

  conntrack -e NEW -E &
  modprobe -v xfrm_user

They provided following analysis:

conntrack -e NEW -E
    netlink_bind()
        netlink_lock_table() -> increases "nl_table_users"
            nfnetlink_bind()
            # does not unlock the table as it's locked by netlink_bind()
                __request_module()
                    call_usermodehelper_exec()

This triggers "modprobe nf_conntrack_netlink" from kernel, netlink_bind()
won't return until modprobe process is done.

"modprobe xfrm_user":
    xfrm_user_init()
        register_pernet_subsys()
            -> grab pernet_ops_rwsem
                ..
                netlink_table_grab()
                    calls schedule() as "nl_table_users" is non-zero

so modprobe is blocked because netlink_bind() increased
nl_table_users while also holding pernet_ops_rwsem.

"modprobe nf_conntrack_netlink" runs and inits nf_conntrack_netlink:
    ctnetlink_init()
        register_pernet_subsys()
            -> blocks on "pernet_ops_rwsem" thanks to xfrm_user module

both modprobe processes wait on one another -- neither can make
progress.

Switch netlink_bind() to "nowait" modprobe -- this releases the netlink
table lock, which then allows both modprobe instances to complete.

Reported-by: Thomas Jarosch <thomas.jarosch@intra2net.com>
Reported-by: Juliana Rodrigueiro <juliana.rodrigueiro@intra2net.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 net/netfilter/nfnetlink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/netfilter/nfnetlink.c b/net/netfilter/nfnetlink.c
index 916913454624f..7f2c1915763f8 100644
--- a/net/netfilter/nfnetlink.c
+++ b/net/netfilter/nfnetlink.c
@@ -575,7 +575,7 @@ static int nfnetlink_bind(struct net *net, int group)
 	ss = nfnetlink_get_subsys(type << 8);
 	rcu_read_unlock();
 	if (!ss)
-		request_module("nfnetlink-subsys-%d", type);
+		request_module_nowait("nfnetlink-subsys-%d", type);
 	return 0;
 }
 #endif
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH AUTOSEL 4.19 03/42] netfilter: Fix rpfilter dropping vrf packets by mistake
  2019-08-02 13:22 [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Sasha Levin
@ 2019-08-02 13:22 ` Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled Sasha Levin
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 10+ messages in thread
From: Sasha Levin @ 2019-08-02 13:22 UTC (permalink / raw)
  To: linux-kernel, stable
  Cc: Miaohe Lin, Pablo Neira Ayuso, Sasha Levin, netfilter-devel,
	coreteam, netdev

From: Miaohe Lin <linmiaohe@huawei.com>

[ Upstream commit b575b24b8eee37f10484e951b62ce2a31c579775 ]

When firewalld is enabled with ipv4/ipv6 rpfilter, vrf
ipv4/ipv6 packets will be dropped. Vrf device will pass
through netfilter hook twice. One with enslaved device
and another one with l3 master device. So in device may
dismatch witch out device because out device is always
enslaved device.So failed with the check of the rpfilter
and drop the packets by mistake.

Signed-off-by: Miaohe Lin <linmiaohe@huawei.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 net/ipv4/netfilter/ipt_rpfilter.c  | 1 +
 net/ipv6/netfilter/ip6t_rpfilter.c | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/net/ipv4/netfilter/ipt_rpfilter.c b/net/ipv4/netfilter/ipt_rpfilter.c
index 12843c9ef1421..74b19a5c572e9 100644
--- a/net/ipv4/netfilter/ipt_rpfilter.c
+++ b/net/ipv4/netfilter/ipt_rpfilter.c
@@ -96,6 +96,7 @@ static bool rpfilter_mt(const struct sk_buff *skb, struct xt_action_param *par)
 	flow.flowi4_mark = info->flags & XT_RPFILTER_VALID_MARK ? skb->mark : 0;
 	flow.flowi4_tos = RT_TOS(iph->tos);
 	flow.flowi4_scope = RT_SCOPE_UNIVERSE;
+	flow.flowi4_oif = l3mdev_master_ifindex_rcu(xt_in(par));
 
 	return rpfilter_lookup_reverse(xt_net(par), &flow, xt_in(par), info->flags) ^ invert;
 }
diff --git a/net/ipv6/netfilter/ip6t_rpfilter.c b/net/ipv6/netfilter/ip6t_rpfilter.c
index c3c6b09acdc4f..0f3407f2851ed 100644
--- a/net/ipv6/netfilter/ip6t_rpfilter.c
+++ b/net/ipv6/netfilter/ip6t_rpfilter.c
@@ -58,7 +58,9 @@ static bool rpfilter_lookup_reverse6(struct net *net, const struct sk_buff *skb,
 	if (rpfilter_addr_linklocal(&iph->saddr)) {
 		lookup_flags |= RT6_LOOKUP_F_IFACE;
 		fl6.flowi6_oif = dev->ifindex;
-	} else if ((flags & XT_RPFILTER_LOOSE) == 0)
+	/* Set flowi6_oif for vrf devices to lookup route in l3mdev domain. */
+	} else if (netif_is_l3_master(dev) || netif_is_l3_slave(dev) ||
+		  (flags & XT_RPFILTER_LOOSE) == 0)
 		fl6.flowi6_oif = dev->ifindex;
 
 	rt = (void *)ip6_route_lookup(net, &fl6, skb, lookup_flags);
@@ -73,7 +75,9 @@ static bool rpfilter_lookup_reverse6(struct net *net, const struct sk_buff *skb,
 		goto out;
 	}
 
-	if (rt->rt6i_idev->dev == dev || (flags & XT_RPFILTER_LOOSE))
+	if (rt->rt6i_idev->dev == dev ||
+	    l3mdev_master_ifindex_rcu(rt->rt6i_idev->dev) == dev->ifindex ||
+	    (flags & XT_RPFILTER_LOOSE))
 		ret = true;
  out:
 	ip6_rt_put(rt);
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled
  2019-08-02 13:22 [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 03/42] netfilter: Fix rpfilter dropping vrf packets by mistake Sasha Levin
@ 2019-08-02 13:22 ` Sasha Levin
  2019-08-08  9:02   ` Thomas Jarosch
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 05/42] netfilter: nft_hash: fix symhash with modulus one Sasha Levin
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 10+ messages in thread
From: Sasha Levin @ 2019-08-02 13:22 UTC (permalink / raw)
  To: linux-kernel, stable
  Cc: Florian Westphal, Jakub Jankowski, Jozsef Kadlecsik,
	Pablo Neira Ayuso, Sasha Levin, netfilter-devel, coreteam,
	netdev

From: Florian Westphal <fw@strlen.de>

[ Upstream commit 959b69ef57db00cb33e9c4777400ae7183ebddd3 ]

Jakub Jankowski reported following oddity:

After 3 way handshake completes, timeout of new connection is set to
max_retrans (300s) instead of established (5 days).

shortened excerpt from pcap provided:
25.070622 IP (flags [DF], proto TCP (6), length 52)
10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
26.070462 IP (flags [DF], proto TCP (6), length 48)
10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
27.070449 IP (flags [DF], proto TCP (6), length 40)
10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0

Turns out the last_win is of u16 type, but we store the scaled value:
512 << 8 (== 0x20000) becomes 0 window.

The Fixes tag is not correct, as the bug has existed forever, but
without that change all that this causes might cause is to mistake a
window update (to-nonzero-from-zero) for a retransmit.

Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
Reported-by: Jakub Jankowski <shasta@toxcorp.com>
Tested-by: Jakub Jankowski <shasta@toxcorp.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 net/netfilter/nf_conntrack_proto_tcp.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
index 842f3f86fb2e7..7011ab27c4371 100644
--- a/net/netfilter/nf_conntrack_proto_tcp.c
+++ b/net/netfilter/nf_conntrack_proto_tcp.c
@@ -480,6 +480,7 @@ static bool tcp_in_window(const struct nf_conn *ct,
 	struct ip_ct_tcp_state *receiver = &state->seen[!dir];
 	const struct nf_conntrack_tuple *tuple = &ct->tuplehash[dir].tuple;
 	__u32 seq, ack, sack, end, win, swin;
+	u16 win_raw;
 	s32 receiver_offset;
 	bool res, in_recv_win;
 
@@ -488,7 +489,8 @@ static bool tcp_in_window(const struct nf_conn *ct,
 	 */
 	seq = ntohl(tcph->seq);
 	ack = sack = ntohl(tcph->ack_seq);
-	win = ntohs(tcph->window);
+	win_raw = ntohs(tcph->window);
+	win = win_raw;
 	end = segment_seq_plus_len(seq, skb->len, dataoff, tcph);
 
 	if (receiver->flags & IP_CT_TCP_FLAG_SACK_PERM)
@@ -663,14 +665,14 @@ static bool tcp_in_window(const struct nf_conn *ct,
 			    && state->last_seq == seq
 			    && state->last_ack == ack
 			    && state->last_end == end
-			    && state->last_win == win)
+			    && state->last_win == win_raw)
 				state->retrans++;
 			else {
 				state->last_dir = dir;
 				state->last_seq = seq;
 				state->last_ack = ack;
 				state->last_end = end;
-				state->last_win = win;
+				state->last_win = win_raw;
 				state->retrans = 0;
 			}
 		}
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH AUTOSEL 4.19 05/42] netfilter: nft_hash: fix symhash with modulus one
  2019-08-02 13:22 [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 03/42] netfilter: Fix rpfilter dropping vrf packets by mistake Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled Sasha Levin
@ 2019-08-02 13:22 ` Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 14/42] mac80211: don't warn about CW params when not using them Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 15/42] allocate_flower_entry: should check for null deref Sasha Levin
  4 siblings, 0 replies; 10+ messages in thread
From: Sasha Levin @ 2019-08-02 13:22 UTC (permalink / raw)
  To: linux-kernel, stable
  Cc: Laura Garcia Liebana, Pablo Neira Ayuso, Sasha Levin,
	netfilter-devel, coreteam, netdev

From: Laura Garcia Liebana <nevola@gmail.com>

[ Upstream commit 28b1d6ef53e3303b90ca8924bb78f31fa527cafb ]

The rule below doesn't work as the kernel raises -ERANGE.

nft add rule netdev nftlb lb01 ip daddr set \
	symhash mod 1 map { 0 : 192.168.0.10 } fwd to "eth0"

This patch allows to use the symhash modulus with one
element, in the same way that the other types of hashes and
algorithms that uses the modulus parameter.

Signed-off-by: Laura Garcia Liebana <nevola@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 net/netfilter/nft_hash.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/netfilter/nft_hash.c b/net/netfilter/nft_hash.c
index c2d237144f747..b8f23f75aea6c 100644
--- a/net/netfilter/nft_hash.c
+++ b/net/netfilter/nft_hash.c
@@ -196,7 +196,7 @@ static int nft_symhash_init(const struct nft_ctx *ctx,
 	priv->dreg = nft_parse_register(tb[NFTA_HASH_DREG]);
 
 	priv->modulus = ntohl(nla_get_be32(tb[NFTA_HASH_MODULUS]));
-	if (priv->modulus <= 1)
+	if (priv->modulus < 1)
 		return -ERANGE;
 
 	if (priv->offset + priv->modulus - 1 < priv->offset)
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH AUTOSEL 4.19 14/42] mac80211: don't warn about CW params when not using them
  2019-08-02 13:22 [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Sasha Levin
                   ` (2 preceding siblings ...)
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 05/42] netfilter: nft_hash: fix symhash with modulus one Sasha Levin
@ 2019-08-02 13:22 ` Sasha Levin
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 15/42] allocate_flower_entry: should check for null deref Sasha Levin
  4 siblings, 0 replies; 10+ messages in thread
From: Sasha Levin @ 2019-08-02 13:22 UTC (permalink / raw)
  To: linux-kernel, stable
  Cc: Brian Norris, Johannes Berg, Sasha Levin, linux-wireless, netdev

From: Brian Norris <briannorris@chromium.org>

[ Upstream commit d2b3fe42bc629c2d4002f652b3abdfb2e72991c7 ]

ieee80211_set_wmm_default() normally sets up the initial CW min/max for
each queue, except that it skips doing this if the driver doesn't
support ->conf_tx. We still end up calling drv_conf_tx() in some cases
(e.g., ieee80211_reconfig()), which also still won't do anything
useful...except it complains here about the invalid CW parameters.

Let's just skip the WARN if we weren't going to do anything useful with
the parameters.

Signed-off-by: Brian Norris <briannorris@chromium.org>
Link: https://lore.kernel.org/r/20190718015712.197499-1-briannorris@chromium.org
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 net/mac80211/driver-ops.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/net/mac80211/driver-ops.c b/net/mac80211/driver-ops.c
index bb886e7db47f1..f783d1377d9a8 100644
--- a/net/mac80211/driver-ops.c
+++ b/net/mac80211/driver-ops.c
@@ -169,11 +169,16 @@ int drv_conf_tx(struct ieee80211_local *local,
 	if (!check_sdata_in_driver(sdata))
 		return -EIO;
 
-	if (WARN_ONCE(params->cw_min == 0 ||
-		      params->cw_min > params->cw_max,
-		      "%s: invalid CW_min/CW_max: %d/%d\n",
-		      sdata->name, params->cw_min, params->cw_max))
+	if (params->cw_min == 0 || params->cw_min > params->cw_max) {
+		/*
+		 * If we can't configure hardware anyway, don't warn. We may
+		 * never have initialized the CW parameters.
+		 */
+		WARN_ONCE(local->ops->conf_tx,
+			  "%s: invalid CW_min/CW_max: %d/%d\n",
+			  sdata->name, params->cw_min, params->cw_max);
 		return -EINVAL;
+	}
 
 	trace_drv_conf_tx(local, sdata, ac, params);
 	if (local->ops->conf_tx)
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH AUTOSEL 4.19 15/42] allocate_flower_entry: should check for null deref
  2019-08-02 13:22 [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Sasha Levin
                   ` (3 preceding siblings ...)
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 14/42] mac80211: don't warn about CW params when not using them Sasha Levin
@ 2019-08-02 13:22 ` Sasha Levin
  4 siblings, 0 replies; 10+ messages in thread
From: Sasha Levin @ 2019-08-02 13:22 UTC (permalink / raw)
  To: linux-kernel, stable
  Cc: Navid Emamdoost, David S . Miller, Sasha Levin, netdev

From: Navid Emamdoost <navid.emamdoost@gmail.com>

[ Upstream commit bb1320834b8a80c6ac2697ab418d066981ea08ba ]

allocate_flower_entry does not check for allocation success, but tries
to deref the result. I only moved the spin_lock under null check, because
 the caller is checking allocation's status at line 652.

Signed-off-by: Navid Emamdoost <navid.emamdoost@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c
index f2aba5b160c2d..d45c435a599d6 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_flower.c
@@ -67,7 +67,8 @@ static struct ch_tc_pedit_fields pedits[] = {
 static struct ch_tc_flower_entry *allocate_flower_entry(void)
 {
 	struct ch_tc_flower_entry *new = kzalloc(sizeof(*new), GFP_KERNEL);
-	spin_lock_init(&new->lock);
+	if (new)
+		spin_lock_init(&new->lock);
 	return new;
 }
 
-- 
2.20.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled
  2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled Sasha Levin
@ 2019-08-08  9:02   ` Thomas Jarosch
  2019-08-14 10:19     ` Reindl Harald
  0 siblings, 1 reply; 10+ messages in thread
From: Thomas Jarosch @ 2019-08-08  9:02 UTC (permalink / raw)
  To: Sasha Levin
  Cc: linux-kernel, stable, Florian Westphal, Jakub Jankowski,
	Jozsef Kadlecsik, Pablo Neira Ayuso, netfilter-devel, coreteam,
	netdev

Hello together,

You wrote on Fri, Aug 02, 2019 at 09:22:24AM -0400:
> From: Florian Westphal <fw@strlen.de>
> 
> [ Upstream commit 959b69ef57db00cb33e9c4777400ae7183ebddd3 ]
> 
> Jakub Jankowski reported following oddity:
> 
> After 3 way handshake completes, timeout of new connection is set to
> max_retrans (300s) instead of established (5 days).
> 
> shortened excerpt from pcap provided:
> 25.070622 IP (flags [DF], proto TCP (6), length 52)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
> 26.070462 IP (flags [DF], proto TCP (6), length 48)
> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
> 27.070449 IP (flags [DF], proto TCP (6), length 40)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
> 
> Turns out the last_win is of u16 type, but we store the scaled value:
> 512 << 8 (== 0x20000) becomes 0 window.
> 
> The Fixes tag is not correct, as the bug has existed forever, but
> without that change all that this causes might cause is to mistake a
> window update (to-nonzero-from-zero) for a retransmit.
> 
> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
> Signed-off-by: Florian Westphal <fw@strlen.de>
> Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> Signed-off-by: Sasha Levin <sashal@kernel.org>

Also:
Tested-by: Thomas Jarosch <thomas.jarosch@intra2net.com>

;)

We've hit the issue with the wrong conntrack timeout at two different sites,
long-lived connections to a SAP server over IPSec VPN were constantly dropping.

For us this was a regression after updating from kernel 3.14 to 4.19.
Yesterday I've applied the patch to kernel 4.19.57 and the problem is fixed.

The issue was extra hard to debug as we could just boot the new kernel
for twenty minutes in the evening on these productive systems.

The stable kernel patch from last Friday came right on time. I was just
about the replay the TCP connection with tcpreplay, so this saved
me from another week of debugging. Thanks everyone!

Best regards,
Thomas Jarosch

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled
  2019-08-08  9:02   ` Thomas Jarosch
@ 2019-08-14 10:19     ` Reindl Harald
  2019-08-14 11:17       ` Jakub Jankowski
  0 siblings, 1 reply; 10+ messages in thread
From: Reindl Harald @ 2019-08-14 10:19 UTC (permalink / raw)
  To: Thomas Jarosch, Sasha Levin
  Cc: linux-kernel, stable, Florian Westphal, Jakub Jankowski,
	Jozsef Kadlecsik, Pablo Neira Ayuso, netfilter-devel, coreteam,
	netdev

that's still not in 5.2.8

without the exception and "nf_conntrack_tcp_timeout_max_retrans = 60" a
vnc-over-ssh session having the VNC view in the background freezes
within 60 secods

-----------------------------------------------------------------------------------------------
IPV4 TABLE MANGLE (STATEFUL PRE-NAT/FILTER)
-----------------------------------------------------------------------------------------------
Chain PREROUTING (policy ACCEPT 100 packets, 9437 bytes)
num   pkts bytes target     prot opt in     out     source
 destination
1     6526 3892K ACCEPT     all  --  *      *       0.0.0.0/0
 0.0.0.0/0            ctstate RELATED,ESTABLISHED
2      125  6264 ACCEPT     all  --  lo     *       0.0.0.0/0
 0.0.0.0/0
3       64  4952 ACCEPT     all  --  vmnet8 *       0.0.0.0/0
 0.0.0.0/0
4        1    40 DROP       all  --  *      *       0.0.0.0/0
 0.0.0.0/0            ctstate INVALID

-------- Weitergeleitete Nachricht --------
Betreff: [PATCH AUTOSEL 5.2 07/76] netfilter: conntrack: always store
window size un-scaled

Am 08.08.19 um 11:02 schrieb Thomas Jarosch:
> Hello together,
> 
> You wrote on Fri, Aug 02, 2019 at 09:22:24AM -0400:
>> From: Florian Westphal <fw@strlen.de>
>>
>> [ Upstream commit 959b69ef57db00cb33e9c4777400ae7183ebddd3 ]
>>
>> Jakub Jankowski reported following oddity:
>>
>> After 3 way handshake completes, timeout of new connection is set to
>> max_retrans (300s) instead of established (5 days).
>>
>> shortened excerpt from pcap provided:
>> 25.070622 IP (flags [DF], proto TCP (6), length 52)
>> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
>> 26.070462 IP (flags [DF], proto TCP (6), length 48)
>> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
>> 27.070449 IP (flags [DF], proto TCP (6), length 40)
>> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
>>
>> Turns out the last_win is of u16 type, but we store the scaled value:
>> 512 << 8 (== 0x20000) becomes 0 window.
>>
>> The Fixes tag is not correct, as the bug has existed forever, but
>> without that change all that this causes might cause is to mistake a
>> window update (to-nonzero-from-zero) for a retransmit.
>>
>> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
>> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
>> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
>> Signed-off-by: Florian Westphal <fw@strlen.de>
>> Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
>> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
>> Signed-off-by: Sasha Levin <sashal@kernel.org>
> 
> Also:
> Tested-by: Thomas Jarosch <thomas.jarosch@intra2net.com>
> 
> ;)
> 
> We've hit the issue with the wrong conntrack timeout at two different sites,
> long-lived connections to a SAP server over IPSec VPN were constantly dropping.
> 
> For us this was a regression after updating from kernel 3.14 to 4.19.
> Yesterday I've applied the patch to kernel 4.19.57 and the problem is fixed.
> 
> The issue was extra hard to debug as we could just boot the new kernel
> for twenty minutes in the evening on these productive systems.
> 
> The stable kernel patch from last Friday came right on time. I was just
> about the replay the TCP connection with tcpreplay, so this saved
> me from another week of debugging. Thanks everyone!

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled
  2019-08-14 10:19     ` Reindl Harald
@ 2019-08-14 11:17       ` Jakub Jankowski
  2019-08-14 17:01         ` Sasha Levin
  0 siblings, 1 reply; 10+ messages in thread
From: Jakub Jankowski @ 2019-08-14 11:17 UTC (permalink / raw)
  To: Reindl Harald
  Cc: Thomas Jarosch, Sasha Levin, linux-kernel, stable,
	Florian Westphal, Jozsef Kadlecsik, Pablo Neira Ayuso,
	netfilter-devel, coreteam, netdev

On 2019-08-14, Reindl Harald wrote:

> that's still not in 5.2.8

It will make its way into next 5.2.x release, as it is now in the pending 
queue: 
https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/queue-5.2

Regards,
  Jakub.


>
> without the exception and "nf_conntrack_tcp_timeout_max_retrans = 60" a
> vnc-over-ssh session having the VNC view in the background freezes
> within 60 secods
>
> -----------------------------------------------------------------------------------------------
> IPV4 TABLE MANGLE (STATEFUL PRE-NAT/FILTER)
> -----------------------------------------------------------------------------------------------
> Chain PREROUTING (policy ACCEPT 100 packets, 9437 bytes)
> num   pkts bytes target     prot opt in     out     source
> destination
> 1     6526 3892K ACCEPT     all  --  *      *       0.0.0.0/0
> 0.0.0.0/0            ctstate RELATED,ESTABLISHED
> 2      125  6264 ACCEPT     all  --  lo     *       0.0.0.0/0
> 0.0.0.0/0
> 3       64  4952 ACCEPT     all  --  vmnet8 *       0.0.0.0/0
> 0.0.0.0/0
> 4        1    40 DROP       all  --  *      *       0.0.0.0/0
> 0.0.0.0/0            ctstate INVALID
>
> -------- Weitergeleitete Nachricht --------
> Betreff: [PATCH AUTOSEL 5.2 07/76] netfilter: conntrack: always store
> window size un-scaled
>
> Am 08.08.19 um 11:02 schrieb Thomas Jarosch:
>> Hello together,
>>
>> You wrote on Fri, Aug 02, 2019 at 09:22:24AM -0400:
>>> From: Florian Westphal <fw@strlen.de>
>>>
>>> [ Upstream commit 959b69ef57db00cb33e9c4777400ae7183ebddd3 ]
>>>
>>> Jakub Jankowski reported following oddity:
>>>
>>> After 3 way handshake completes, timeout of new connection is set to
>>> max_retrans (300s) instead of established (5 days).
>>>
>>> shortened excerpt from pcap provided:
>>> 25.070622 IP (flags [DF], proto TCP (6), length 52)
>>> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
>>> 26.070462 IP (flags [DF], proto TCP (6), length 48)
>>> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
>>> 27.070449 IP (flags [DF], proto TCP (6), length 40)
>>> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
>>>
>>> Turns out the last_win is of u16 type, but we store the scaled value:
>>> 512 << 8 (== 0x20000) becomes 0 window.
>>>
>>> The Fixes tag is not correct, as the bug has existed forever, but
>>> without that change all that this causes might cause is to mistake a
>>> window update (to-nonzero-from-zero) for a retransmit.
>>>
>>> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
>>> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
>>> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
>>> Signed-off-by: Florian Westphal <fw@strlen.de>
>>> Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
>>> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
>>> Signed-off-by: Sasha Levin <sashal@kernel.org>
>>
>> Also:
>> Tested-by: Thomas Jarosch <thomas.jarosch@intra2net.com>
>>
>> ;)
>>
>> We've hit the issue with the wrong conntrack timeout at two different sites,
>> long-lived connections to a SAP server over IPSec VPN were constantly dropping.
>>
>> For us this was a regression after updating from kernel 3.14 to 4.19.
>> Yesterday I've applied the patch to kernel 4.19.57 and the problem is fixed.
>>
>> The issue was extra hard to debug as we could just boot the new kernel
>> for twenty minutes in the evening on these productive systems.
>>
>> The stable kernel patch from last Friday came right on time. I was just
>> about the replay the TCP connection with tcpreplay, so this saved
>> me from another week of debugging. Thanks everyone!
>

-- 
Jakub Jankowski|shasta@toxcorp.com|https://toxcorp.com/

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled
  2019-08-14 11:17       ` Jakub Jankowski
@ 2019-08-14 17:01         ` Sasha Levin
  0 siblings, 0 replies; 10+ messages in thread
From: Sasha Levin @ 2019-08-14 17:01 UTC (permalink / raw)
  To: Jakub Jankowski
  Cc: Reindl Harald, Thomas Jarosch, linux-kernel, stable,
	Florian Westphal, Jozsef Kadlecsik, Pablo Neira Ayuso,
	netfilter-devel, coreteam, netdev

On Wed, Aug 14, 2019 at 01:17:30PM +0200, Jakub Jankowski wrote:
>On 2019-08-14, Reindl Harald wrote:
>
>>that's still not in 5.2.8
>
>It will make its way into next 5.2.x release, as it is now in the 
>pending queue: https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/queue-5.2

In general, AUTOSEL stuff soak for much longer before they make it to
the queue.

If there's an urgent need for a fix to go in, please make it explicit.

--
Thanks,
Sasha

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2019-08-14 17:01 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-02 13:22 [PATCH AUTOSEL 4.19 01/42] netfilter: nfnetlink: avoid deadlock due to synchronous request_module Sasha Levin
2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 03/42] netfilter: Fix rpfilter dropping vrf packets by mistake Sasha Levin
2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled Sasha Levin
2019-08-08  9:02   ` Thomas Jarosch
2019-08-14 10:19     ` Reindl Harald
2019-08-14 11:17       ` Jakub Jankowski
2019-08-14 17:01         ` Sasha Levin
2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 05/42] netfilter: nft_hash: fix symhash with modulus one Sasha Levin
2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 14/42] mac80211: don't warn about CW params when not using them Sasha Levin
2019-08-02 13:22 ` [PATCH AUTOSEL 4.19 15/42] allocate_flower_entry: should check for null deref Sasha Levin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).