[raw v1 1/4] Use raw_cpu ops for determining current NUMA node

[raw v1 1/4] Use raw_cpu ops for determining current NUMA node

[Christoph Lameter]

With the preempt check logic we will get false positives from
these locations. Before the use of __this_cpu ops there were
no checks for preemption present either and smp_raw_processor_id()
was used. See http://www.spinics.net/lists/linux-numa/msg00641.html

Use raw_cpu_read() to avoid preemption messages.

Note that this issue has been discussed in prior years.
If the process changes nodes after retrieving the current numa node then
that is acceptable since most uses of numa_node etc are for optimization
and not for correctness.

CC: linux-mm@kvack.org
Cc: Alex Shi <alex.shi@intel.com>
Signed-off-by: Christoph Lameter <cl@linux.com>

Index: linux/include/linux/topology.h
===================================================================
--- linux.orig/include/linux/topology.h	2013-09-24 11:29:51.000000000 -0500
+++ linux/include/linux/topology.h	2013-09-24 11:30:18.893831971 -0500
@@ -182,7 +182,7 @@ DECLARE_PER_CPU(int, numa_node);
 /* Returns the number of the current Node. */
 static inline int numa_node_id(void)
 {
-	return __this_cpu_read(numa_node);
+	return raw_cpu_read(numa_node);
 }
 #endif
 
@@ -239,7 +239,7 @@ static inline void set_numa_mem(int node
 /* Returns the number of the nearest Node with memory */
 static inline int numa_mem_id(void)
 {
-	return __this_cpu_read(_numa_mem_);
+	return raw_cpu_read(_numa_mem_);
 }
 #endif
 

[raw v1 2/4] [NET] Use raw_cpu ops for SNMP stats

[Christoph Lameter]

SNMP stats are not protected by preemption but by bh handling.
Since protection is provided outside of preemption raw_cpu_ops
need to be used to avoid false positives.

Cc: Eric Dumazet <edumazet@google.com>
CC: netdev@vger.kernel.org
Signed-off-by: Christoph Lameter <cl@linux.com>

Index: linux/include/net/snmp.h
===================================================================
--- linux.orig/include/net/snmp.h	2013-10-07 09:16:07.595206864 -0500
+++ linux/include/net/snmp.h	2013-10-07 09:16:07.591206909 -0500
@@ -126,7 +126,7 @@ struct linux_xfrm_mib {
 	extern __typeof__(type) __percpu *name[SNMP_ARRAY_SZ]
 
 #define SNMP_INC_STATS_BH(mib, field)	\
-			__this_cpu_inc(mib[0]->mibs[field])
+			raw_cpu_inc(mib[0]->mibs[field])
 
 #define SNMP_INC_STATS_USER(mib, field)	\
 			this_cpu_inc(mib[0]->mibs[field])
@@ -141,7 +141,7 @@ struct linux_xfrm_mib {
 			this_cpu_dec(mib[0]->mibs[field])
 
 #define SNMP_ADD_STATS_BH(mib, field, addend)	\
-			__this_cpu_add(mib[0]->mibs[field], addend)
+			raw_cpu_add(mib[0]->mibs[field], addend)
 
 #define SNMP_ADD_STATS_USER(mib, field, addend)	\
 			this_cpu_add(mib[0]->mibs[field], addend)

[raw v1 4/4] net: __this_cpu_inc in route.c

[Christoph Lameter]

I do not see any synchronization however commit
dbd2915ce87e811165da0717f8e159276ebb803e by Andrew justifies
the use of raw_smp_processor_id() here because "we do not care"

So lets use raw_cpu ops here and hope for the best. The use of
__this_cpu op improves the situation already from what commit
dbd2915ce87e811165da0717f8e159276ebb803e did since the single instruction
emitted on x86 does not allow the race to occur anymore. However,
non x86 platforms could still experience a race here.

Cc: Eric Dumazet <edumazet@google.com>
Signed-off-by: Christoph Lameter <cl@linux.com>

Index: linux/net/ipv4/route.c
===================================================================
--- linux.orig/net/ipv4/route.c	2013-10-07 10:27:25.000000000 -0500
+++ linux/net/ipv4/route.c	2013-10-07 12:41:23.993823743 -0500
@@ -197,7 +197,7 @@ const __u8 ip_tos2prio[16] = {
 EXPORT_SYMBOL(ip_tos2prio);
 
 static DEFINE_PER_CPU(struct rt_cache_stat, rt_cache_stat);
-#define RT_CACHE_STAT_INC(field) __this_cpu_inc(rt_cache_stat.field)
+#define RT_CACHE_STAT_INC(field) raw_cpu_inc(rt_cache_stat.field)
 
 #ifdef CONFIG_PROC_FS
 static void *rt_cache_seq_start(struct seq_file *seq, loff_t *pos)

[raw v1 3/4] Use raw_cpu_write for initialization of per cpu refcount.

[Christoph Lameter]

The initialization of a structure is not subject to synchronization.
So simply disable the check.

Cc: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Christoph Lameter <cl@linux.com>

Index: linux/kernel/module.c
===================================================================
--- linux.orig/kernel/module.c	2013-09-05 13:43:30.557687773 -0500
+++ linux/kernel/module.c	2013-10-07 12:33:43.732059759 -0500
@@ -643,7 +643,7 @@ static int module_unload_init(struct mod
 	INIT_LIST_HEAD(&mod->target_list);
 
 	/* Hold reference count during initialization. */
-	__this_cpu_write(mod->refptr->incs, 1);
+	raw_cpu_write(mod->refptr->incs, 1);
 	/* Backwards compatibility macros put refcount during init. */
 	mod->waiter = current;
 

Re: [raw v1 2/4] [NET] Use raw_cpu ops for SNMP stats

[Ingo Molnar]

* Christoph Lameter <cl@linux.com> wrote:

> SNMP stats are not protected by preemption but by bh handling.

Most forms of bh exclusion work via the preemption count though, and 
softirq contexts themselves are generally not preemptible [to other CPUs] 
either.

So the warnings should, in most cases, not trigger.

> Since protection is provided outside of preemption raw_cpu_ops
> need to be used to avoid false positives.

Could you quote the warning that pops up? It might be better to annotate 
the specific safe usages than to potentially hide bugs:

> --- linux.orig/include/net/snmp.h	2013-10-07 09:16:07.595206864 -0500
> +++ linux/include/net/snmp.h	2013-10-07 09:16:07.591206909 -0500
> @@ -126,7 +126,7 @@ struct linux_xfrm_mib {
>  	extern __typeof__(type) __percpu *name[SNMP_ARRAY_SZ]
>  
>  #define SNMP_INC_STATS_BH(mib, field)	\
> -			__this_cpu_inc(mib[0]->mibs[field])
> +			raw_cpu_inc(mib[0]->mibs[field])
>  
>  #define SNMP_INC_STATS_USER(mib, field)	\
>  			this_cpu_inc(mib[0]->mibs[field])
> @@ -141,7 +141,7 @@ struct linux_xfrm_mib {
>  			this_cpu_dec(mib[0]->mibs[field])
>  
>  #define SNMP_ADD_STATS_BH(mib, field, addend)	\
> -			__this_cpu_add(mib[0]->mibs[field], addend)
> +			raw_cpu_add(mib[0]->mibs[field], addend)
>  
>  #define SNMP_ADD_STATS_USER(mib, field, addend)	\
>  			this_cpu_add(mib[0]->mibs[field], addend)

Doing this will hide any true bugs if any of these primitives is used in 
an unsafe context.

Thanks,

	Ingo

Re: [raw v1 2/4] [NET] Use raw_cpu ops for SNMP stats

[Peter Zijlstra]

On Tue, Oct 08, 2013 at 09:21:14AM +0200, Ingo Molnar wrote:
> 
> * Christoph Lameter <cl@linux.com> wrote:
> 
> > SNMP stats are not protected by preemption but by bh handling.
> 
> Most forms of bh exclusion work via the preemption count though, and 
> softirq contexts themselves are generally not preemptible [to other CPUs] 
> either.
> 
> So the warnings should, in most cases, not trigger.

Right, so softirqs run either in the irq tail at which point
preempt_count += SOFTIRQ_OFFSET and thus preemption is disabled, or it
runs in ksoftirqd which has strict cpu affinity which also disables the
warning, and it also increments preempt_count with SOFTIRQ_OFFSET to
exclude the softirq from interrupts while its running, also disabling
the warning.

So it should very much not trigger.. if it does you want to know about
it.

Re: [raw v1 2/4] [NET] Use raw_cpu ops for SNMP stats

[Ingo Molnar]

* Peter Zijlstra <peterz@infradead.org> wrote:

> On Tue, Oct 08, 2013 at 09:21:14AM +0200, Ingo Molnar wrote:
> > 
> > * Christoph Lameter <cl@linux.com> wrote:
> > 
> > > SNMP stats are not protected by preemption but by bh handling.
> > 
> > Most forms of bh exclusion work via the preemption count though, and 
> > softirq contexts themselves are generally not preemptible [to other CPUs] 
> > either.
> > 
> > So the warnings should, in most cases, not trigger.
> 
> Right, so softirqs run either in the irq tail at which point 
> preempt_count += SOFTIRQ_OFFSET and thus preemption is disabled, or it 
> runs in ksoftirqd which has strict cpu affinity which also disables the 
> warning, and it also increments preempt_count with SOFTIRQ_OFFSET to 
> exclude the softirq from interrupts while its running, also disabling 
> the warning.

A third context would be syscall-level code that runs with 
local_bh_disable()/enable() - but that too ought to have the preempt count 
elevated.

> So it should very much not trigger.. if it does you want to know about 
> it.

Yes. If nothing else then for the education value.

Thanks,

	Ingo