[PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[Jason A. Donenfeld]

The below patch has made it into Linus' tree, but should be backported
quite far. From testing, I know this applies to 4.1, 4.2, and 4.3. I
haven't looked further back, but I imagine it's relevant there too.

Upstream commit:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=b4fe85f9c9146f60457e9512fb6055e69e6a7a65

=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
>From b4fe85f9c9146f60457e9512fb6055e69e6a7a65 Mon Sep 17 00:00:00 2001
From: "Jason A. Donenfeld" <Jason@zx2c4.com>
Date: Thu, 12 Nov 2015 17:35:58 +0100
Subject: ip_tunnel: disable preemption when updating per-cpu tstats

Drivers like vxlan use the recently introduced
udp_tunnel_xmit_skb/udp_tunnel6_xmit_skb APIs. udp_tunnel6_xmit_skb
makes use of ip6tunnel_xmit, and ip6tunnel_xmit, after sending the
packet, updates the struct stats using the usual
u64_stats_update_begin/end calls on this_cpu_ptr(dev->tstats).
udp_tunnel_xmit_skb makes use of iptunnel_xmit, which doesn't touch
tstats, so drivers like vxlan, immediately after, call
iptunnel_xmit_stats, which does the same thing - calls
u64_stats_update_begin/end on this_cpu_ptr(dev->tstats).

While vxlan is probably fine (I don't know?), calling a similar function
from, say, an unbound workqueue, on a fully preemptable kernel causes
real issues:

[  188.434537] BUG: using smp_processor_id() in preemptible [00000000]
code: kworker/u8:0/6
[  188.435579] caller is debug_smp_processor_id+0x17/0x20
[  188.435583] CPU: 0 PID: 6 Comm: kworker/u8:0 Not tainted 4.2.6 #2
[  188.435607] Call Trace:
[  188.435611]  [<ffffffff8234e936>] dump_stack+0x4f/0x7b
[  188.435615]  [<ffffffff81915f3d>] check_preemption_disabled+0x19d/0x1c0
[  188.435619]  [<ffffffff81915f77>] debug_smp_processor_id+0x17/0x20

The solution would be to protect the whole
this_cpu_ptr(dev->tstats)/u64_stats_update_begin/end blocks with
disabling preemption and then reenabling it.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Acked-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
---
 include/net/ip6_tunnel.h | 3 ++-
 include/net/ip_tunnels.h | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/include/net/ip6_tunnel.h b/include/net/ip6_tunnel.h
index aaee6fa..ff788b6 100644
--- a/include/net/ip6_tunnel.h
+++ b/include/net/ip6_tunnel.h
@@ -90,11 +90,12 @@ static inline void ip6tunnel_xmit(struct sock *sk,
struct sk_buff *skb,
  err = ip6_local_out(dev_net(skb_dst(skb)->dev), sk, skb);

  if (net_xmit_eval(err) == 0) {
- struct pcpu_sw_netstats *tstats = this_cpu_ptr(dev->tstats);
+ struct pcpu_sw_netstats *tstats = get_cpu_ptr(dev->tstats);
  u64_stats_update_begin(&tstats->syncp);
  tstats->tx_bytes += pkt_len;
  tstats->tx_packets++;
  u64_stats_update_end(&tstats->syncp);
+ put_cpu_ptr(tstats);
  } else {
  stats->tx_errors++;
  stats->tx_aborted_errors++;
diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h
index f6dafec..62a750a 100644
--- a/include/net/ip_tunnels.h
+++ b/include/net/ip_tunnels.h
@@ -287,12 +287,13 @@ static inline void iptunnel_xmit_stats(int err,
        struct pcpu_sw_netstats __percpu *stats)
 {
  if (err > 0) {
- struct pcpu_sw_netstats *tstats = this_cpu_ptr(stats);
+ struct pcpu_sw_netstats *tstats = get_cpu_ptr(stats);

  u64_stats_update_begin(&tstats->syncp);
  tstats->tx_bytes += err;
  tstats->tx_packets++;
  u64_stats_update_end(&tstats->syncp);
+ put_cpu_ptr(tstats);
  } else if (err < 0) {
  err_stats->tx_errors++;
  err_stats->tx_aborted_errors++;
--

Re: [PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[Greg KH]

On Wed, Nov 18, 2015 at 09:22:19PM +0100, Jason A. Donenfeld wrote:
> The below patch has made it into Linus' tree, but should be backported
> quite far. From testing, I know this applies to 4.1, 4.2, and 4.3. I
> haven't looked further back, but I imagine it's relevant there too.
> 
> Upstream commit:
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=b4fe85f9c9146f60457e9512fb6055e69e6a7a65
> 
> =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
> >From b4fe85f9c9146f60457e9512fb6055e69e6a7a65 Mon Sep 17 00:00:00 2001
> From: "Jason A. Donenfeld" <Jason@zx2c4.com>
> Date: Thu, 12 Nov 2015 17:35:58 +0100
> Subject: ip_tunnel: disable preemption when updating per-cpu tstats

For networking patches, please email the netdev mailing list and ask for
a patch to be applied to the stable tree, the maintainer there will
handle it for you, he doesn't want people to send patches directly to
stable@ like this so that he can properly test them.

thanks,

greg k-h

Re: [PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[Jason A. Donenfeld]

By the way, in case anybody is interested, I've done a little bit of
historical digging work. The functions in question date back to
aa0010f8 from 2012. Before that commit, statistics structures would be
incremented after each tunnel's driver itself dereferenced the per-cpu
variable. When this got factored out into iptunnel_xmit_stats, the
author of aa0010f8 simply took the code from the tunnel drivers, which
included the use of "this_cpu_ptr" instead of "get_cpu_ptr", because
presumably each driver was able to ensure preemption was disabled in
its codepath. With the generalization of this functionality into the
globally useful iptunnel_xmit_stats, we'll need to be using
"get_cpu_ptr" instead.

Re: [PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[Jason A. Donenfeld]

On Thu, Nov 12, 2015 at 5:25 PM, Hannes Frederic Sowa
<hannes@stressinduktion.org> wrote:
> The canonical way is get_cpu_ptr(stats) / put_cpu_ptr.

Thanks for the pointer. Fixed in v2.

Re: [PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[Hannes Frederic Sowa]

On Thu, Nov 12, 2015, at 16:30, Jason A. Donenfeld wrote:
>  	if (err > 0) {
> -               struct pcpu_sw_netstats *tstats = this_cpu_ptr(stats);
> -
> +               struct pcpu_sw_netstats *tstats;
> +               preempt_disable();
> +               tstats = this_cpu_ptr(stats);

The canonical way is get_cpu_ptr(stats) / put_cpu_ptr.

Bye,
Hannes

[PATCH] ip_tunnel: disable preemption when updating per-cpu tstats

[Jason A. Donenfeld]

Drivers like vxlan use the recently introduced
udp_tunnel_xmit_skb/udp_tunnel6_xmit_skb APIs. udp_tunnel6_xmit_skb
makes use of ip6tunnel_xmit, and ip6tunnel_xmit, after sending the
packet, updates the struct stats using the usual
u64_stats_update_begin/end calls on this_cpu_ptr(dev->tstats).
udp_tunnel_xmit_skb makes use of iptunnel_xmit, which doesn't touch
tstats, so drivers like vxlan, immediately after, call
iptunnel_xmit_stats, which does the same thing - calls
u64_stats_update_begin/end on this_cpu_ptr(dev->tstats).

While vxlan is probably fine (I don't know?), calling a similar function
from, say, an unbound workqueue, on a fully preemptable kernel causes
real issues:

[  188.434537] BUG: using smp_processor_id() in preemptible [00000000] code: kworker/u8:0/6
[  188.435579] caller is debug_smp_processor_id+0x17/0x20
[  188.435583] CPU: 0 PID: 6 Comm: kworker/u8:0 Not tainted 4.2.6 #2
[  188.435607] Call Trace:
[  188.435611]  [<ffffffff8234e936>] dump_stack+0x4f/0x7b
[  188.435615]  [<ffffffff81915f3d>] check_preemption_disabled+0x19d/0x1c0
[  188.435619]  [<ffffffff81915f77>] debug_smp_processor_id+0x17/0x20

The solution would be to protect the whole
this_cpu_ptr(dev->tstats)/u64_stats_update_begin/end blocks with
disabling preemption and then reenabling it.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
 include/net/ip6_tunnel.h | 5 ++++-
 include/net/ip_tunnels.h | 6 ++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/include/net/ip6_tunnel.h b/include/net/ip6_tunnel.h
index fa915fa..67dc00d 100644
--- a/include/net/ip6_tunnel.h
+++ b/include/net/ip6_tunnel.h
@@ -90,11 +90,14 @@ static inline void ip6tunnel_xmit(struct sock *sk, struct sk_buff *skb,
 	err = ip6_local_out_sk(sk, skb);
 
 	if (net_xmit_eval(err) == 0) {
-		struct pcpu_sw_netstats *tstats = this_cpu_ptr(dev->tstats);
+		struct pcpu_sw_netstats *tstats;
+		preempt_disable();
+		tstats = this_cpu_ptr(dev->tstats);
 		u64_stats_update_begin(&tstats->syncp);
 		tstats->tx_bytes += pkt_len;
 		tstats->tx_packets++;
 		u64_stats_update_end(&tstats->syncp);
+		preempt_enable();
 	} else {
 		stats->tx_errors++;
 		stats->tx_aborted_errors++;
diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h
index f6dafec..6544955 100644
--- a/include/net/ip_tunnels.h
+++ b/include/net/ip_tunnels.h
@@ -287,12 +287,14 @@ static inline void iptunnel_xmit_stats(int err,
 				       struct pcpu_sw_netstats __percpu *stats)
 {
 	if (err > 0) {
-		struct pcpu_sw_netstats *tstats = this_cpu_ptr(stats);
-
+		struct pcpu_sw_netstats *tstats;
+		preempt_disable();
+		tstats = this_cpu_ptr(stats);
 		u64_stats_update_begin(&tstats->syncp);
 		tstats->tx_bytes += err;
 		tstats->tx_packets++;
 		u64_stats_update_end(&tstats->syncp);
+		preempt_enable();
 	} else if (err < 0) {
 		err_stats->tx_errors++;
 		err_stats->tx_aborted_errors++;
-- 
2.6.2