[PATCH v2 0/6] RCU fixes for rcu_assign_pointer() usage

[PATCH v2 0/6] RCU fixes for rcu_assign_pointer() usage

[Joel Fernandes (Google)]

These patches fix various sparse errors found as a result of the recent check
to add rcu_check_sparse() to rcu_assign_pointer().  The errors in some cases
seem to either missing API usage, or missing annotations. The annotations added
in the series can also help avoid future incorrect usages and bugs so it is a
good idea to do in any case.

RFC v1 -> Patch v2:
Made changes to various scheduler patches (Peter Zijlstra)

Joel Fernandes (Google) (6):
net: rtnetlink: Fix incorrect RCU API usage
ixgbe: Fix incorrect RCU API usage
sched/cpufreq: Annotate cpufreq_update_util_data pointer with __rcu
sched_domain: Annotate RCU pointers properly
rcuwait: Annotate task_struct with __rcu
sched: Annotate perf_domain pointer with __rcu

drivers/net/ethernet/intel/ixgbe/ixgbe.h      |  4 ++--
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 ++++++++++-----
include/linux/rcuwait.h                       |  2 +-
include/linux/sched/topology.h                |  4 ++--
kernel/sched/cpufreq.c                        |  2 +-
kernel/sched/sched.h                          | 18 +++++++++---------
kernel/sched/topology.c                       | 10 +++++-----
net/core/rtnetlink.c                          |  4 ++--
8 files changed, 32 insertions(+), 27 deletions(-)

--
2.21.0.rc0.258.g878e2cd30e-goog

[PATCH v2 1/6] net: rtnetlink: Fix incorrect RCU API usage

[Joel Fernandes (Google)]

rtnl_register_internal() and rtnl_unregister_all tries to directly
dereference an RCU protected pointed outside RCU read side section.
While this is Ok to do since a lock is held, let us use the correct
API to avoid programmer bugs in the future.

This also fixes sparse warnings arising from not using RCU API.

net/core/rtnetlink.c:332:13: warning: incorrect type in assignment
(different address spaces) net/core/rtnetlink.c:332:13:    expected
struct rtnl_link **tab net/core/rtnetlink.c:332:13:    got struct
rtnl_link *[noderef] <asn:4>*<noident>

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 net/core/rtnetlink.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 5ea1bed08ede..98be4b4818a9 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -188,7 +188,7 @@ static int rtnl_register_internal(struct module *owner,
 	msgindex = rtm_msgindex(msgtype);
 
 	rtnl_lock();
-	tab = rtnl_msg_handlers[protocol];
+	tab = rtnl_dereference(rtnl_msg_handlers[protocol]);
 	if (tab == NULL) {
 		tab = kcalloc(RTM_NR_MSGTYPES, sizeof(void *), GFP_KERNEL);
 		if (!tab)
@@ -329,7 +329,7 @@ void rtnl_unregister_all(int protocol)
 	BUG_ON(protocol < 0 || protocol > RTNL_FAMILY_MAX);
 
 	rtnl_lock();
-	tab = rtnl_msg_handlers[protocol];
+	tab = rtnl_dereference(rtnl_msg_handlers[protocol]);
 	if (!tab) {
 		rtnl_unlock();
 		return;
-- 
2.21.0.rc0.258.g878e2cd30e-goog

[PATCH v2 2/6] ixgbe: Fix incorrect RCU API usage

[Joel Fernandes (Google)]

Recently, I added an RCU annotation check in rcu_assign_pointer. This
caused a sparse error to be reported by the ixgbe driver.

Further looking, it seems the adapter->xdp_prog pointer is not annotated
with __rcu. Annonating it fixed the error, but caused a bunch of other
warnings.

This patch tries to fix all warnings by using RCU API properly. This
makes sense to do because not using RCU properly can result in various
hard to find bugs. This is a best effort fix and is only build tested.
The sparse errors and warnings go away with the change. I request
maintainers / developers in this area to review / test it properly.

The sparse error fixed is:
ixgbe_main.c:10256:25: error: incompatible types in comparison expression

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h      |  4 ++--
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 ++++++++++-----
 2 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
index 08d85e336bd4..3b14daf27516 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
@@ -311,7 +311,7 @@ struct ixgbe_ring {
 	struct ixgbe_ring *next;	/* pointer to next ring in q_vector */
 	struct ixgbe_q_vector *q_vector; /* backpointer to host q_vector */
 	struct net_device *netdev;	/* netdev ring belongs to */
-	struct bpf_prog *xdp_prog;
+	struct bpf_prog __rcu *xdp_prog;
 	struct device *dev;		/* device for DMA mapping */
 	void *desc;			/* descriptor ring memory */
 	union {
@@ -560,7 +560,7 @@ struct ixgbe_adapter {
 	unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
 	/* OS defined structs */
 	struct net_device *netdev;
-	struct bpf_prog *xdp_prog;
+	struct bpf_prog __rcu *xdp_prog;
 	struct pci_dev *pdev;
 	struct mii_bus *mii_bus;
 
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index daff8183534b..408a312aa6ba 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -2199,7 +2199,7 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter,
 	u32 act;
 
 	rcu_read_lock();
-	xdp_prog = READ_ONCE(rx_ring->xdp_prog);
+	xdp_prog = rcu_dereference(rx_ring->xdp_prog);
 
 	if (!xdp_prog)
 		goto xdp_out;
@@ -6547,7 +6547,7 @@ int ixgbe_setup_rx_resources(struct ixgbe_adapter *adapter,
 			     rx_ring->queue_index) < 0)
 		goto err;
 
-	rx_ring->xdp_prog = adapter->xdp_prog;
+	rcu_assign_pointer(rx_ring->xdp_prog, adapter->xdp_prog);
 
 	return 0;
 err:
@@ -10246,7 +10246,8 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
 	if (nr_cpu_ids > MAX_XDP_QUEUES)
 		return -ENOMEM;
 
-	old_prog = xchg(&adapter->xdp_prog, prog);
+	old_prog = rcu_access_pointer(adapter->xdp_prog);
+	rcu_assign_pointer(adapter->xdp_prog, prog);
 
 	/* If transitioning XDP modes reconfigure rings */
 	if (!!prog != !!old_prog) {
@@ -10271,13 +10272,17 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
 static int ixgbe_xdp(struct net_device *dev, struct netdev_bpf *xdp)
 {
 	struct ixgbe_adapter *adapter = netdev_priv(dev);
+	struct bpf_prog *prog;
 
 	switch (xdp->command) {
 	case XDP_SETUP_PROG:
 		return ixgbe_xdp_setup(dev, xdp->prog);
 	case XDP_QUERY_PROG:
-		xdp->prog_id = adapter->xdp_prog ?
-			adapter->xdp_prog->aux->id : 0;
+		rcu_read_lock();
+		prog = rcu_dereference(adapter->xdp_prog);
+		xdp->prog_id = prog ? prog->aux->id : 0;
+		rcu_read_unlock();
+
 		return 0;
 	case XDP_QUERY_XSK_UMEM:
 		return ixgbe_xsk_umem_query(adapter, &xdp->xsk.umem,
-- 
2.21.0.rc0.258.g878e2cd30e-goog

[PATCH v2 3/6] sched/cpufreq: Annotate cpufreq_update_util_data pointer with __rcu

[Joel Fernandes (Google)]

Recently I added an RCU annotation check to rcu_assign_pointer(). All
pointers assigned to RCU protected data are to be annotated with __rcu
inorder to be able to use rcu_assign_pointer() similar to checks in
other RCU APIs.

This resulted in a sparse error: kernel//sched/cpufreq.c:41:9: sparse:
error: incompatible types in comparison expression (different address
spaces)

Fix this by annotating cpufreq_update_util_data pointer with __rcu. This
will also help sparse catch any future RCU misuage bugs.

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 kernel/sched/cpufreq.c | 2 +-
 kernel/sched/sched.h   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/cpufreq.c b/kernel/sched/cpufreq.c
index 22bd8980f32f..e316ee7bb2e5 100644
--- a/kernel/sched/cpufreq.c
+++ b/kernel/sched/cpufreq.c
@@ -7,7 +7,7 @@
  */
 #include "sched.h"
 
-DEFINE_PER_CPU(struct update_util_data *, cpufreq_update_util_data);
+DEFINE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
 
 /**
  * cpufreq_add_update_util_hook - Populate the CPU's update_util_data pointer.
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index d04530bf251f..2ab545d40381 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2166,7 +2166,7 @@ static inline u64 irq_time_read(int cpu)
 #endif /* CONFIG_IRQ_TIME_ACCOUNTING */
 
 #ifdef CONFIG_CPU_FREQ
-DECLARE_PER_CPU(struct update_util_data *, cpufreq_update_util_data);
+DECLARE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
 
 /**
  * cpufreq_update_util - Take a note about CPU utilization changes.
-- 
2.21.0.rc0.258.g878e2cd30e-goog

[PATCH v2 4/6] sched_domain: Annotate RCU pointers properly

[Joel Fernandes (Google)]

The scheduler uses RCU API in various places to access sched_domain
pointers. These cause sparse errors as below.

Many new errors show up because of an annotation check I added to
rcu_assign_pointer(). Let us annotate the pointers correctly which also
will help sparse catch any potential future bugs.

This fixes the following sparse errors:

rt.c:1681:9: error: incompatible types in comparison expression
deadline.c:1904:9: error: incompatible types in comparison expression
core.c:519:9: error: incompatible types in comparison expression
core.c:1634:17: error: incompatible types in comparison expression
fair.c:6193:14: error: incompatible types in comparison expression
fair.c:9883:22: error: incompatible types in comparison expression
fair.c:9897:9: error: incompatible types in comparison expression
sched.h:1287:9: error: incompatible types in comparison expression
topology.c:612:9: error: incompatible types in comparison expression
topology.c:615:9: error: incompatible types in comparison expression
sched.h:1300:9: error: incompatible types in comparison expression
topology.c:618:9: error: incompatible types in comparison expression
sched.h:1287:9: error: incompatible types in comparison expression
topology.c:621:9: error: incompatible types in comparison expression
sched.h:1300:9: error: incompatible types in comparison expression
topology.c:624:9: error: incompatible types in comparison expression
topology.c:671:9: error: incompatible types in comparison expression
stats.c:45:17: error: incompatible types in comparison expression
fair.c:5998:15: error: incompatible types in comparison expression
fair.c:5989:15: error: incompatible types in comparison expression
fair.c:5998:15: error: incompatible types in comparison expression
fair.c:5989:15: error: incompatible types in comparison expression
fair.c:6120:19: error: incompatible types in comparison expression
fair.c:6506:14: error: incompatible types in comparison expression
fair.c:6515:14: error: incompatible types in comparison expression
fair.c:6623:9: error: incompatible types in comparison expression
fair.c:5970:17: error: incompatible types in comparison expression
fair.c:8642:21: error: incompatible types in comparison expression
fair.c:9253:9: error: incompatible types in comparison expression
fair.c:9331:9: error: incompatible types in comparison expression
fair.c:9519:15: error: incompatible types in comparison expression
fair.c:9533:14: error: incompatible types in comparison expression
fair.c:9542:14: error: incompatible types in comparison expression
fair.c:9567:14: error: incompatible types in comparison expression
fair.c:9597:14: error: incompatible types in comparison expression
fair.c:9421:16: error: incompatible types in comparison expression
fair.c:9421:16: error: incompatible types in comparison expression

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 include/linux/sched/topology.h |  4 ++--
 kernel/sched/sched.h           | 14 +++++++-------
 kernel/sched/topology.c        | 10 +++++-----
 3 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index c31d3a47a47c..4819c9e01e42 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -76,8 +76,8 @@ struct sched_domain_shared {
 
 struct sched_domain {
 	/* These fields must be setup */
-	struct sched_domain *parent;	/* top domain must be null terminated */
-	struct sched_domain *child;	/* bottom domain must be null terminated */
+	struct sched_domain __rcu *parent;	/* top domain must be null terminated */
+	struct sched_domain __rcu *child;	/* bottom domain must be null terminated */
 	struct sched_group *groups;	/* the balancing groups of the domain */
 	unsigned long min_interval;	/* Minimum balance interval ms */
 	unsigned long max_interval;	/* Maximum balance interval ms */
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 2ab545d40381..ca6a79f57e7a 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -866,8 +866,8 @@ struct rq {
 	atomic_t		nr_iowait;
 
 #ifdef CONFIG_SMP
-	struct root_domain	*rd;
-	struct sched_domain	*sd;
+	struct root_domain		*rd;
+	struct sched_domain __rcu	*sd;
 
 	unsigned long		cpu_capacity;
 	unsigned long		cpu_capacity_orig;
@@ -1305,13 +1305,13 @@ static inline struct sched_domain *lowest_flag_domain(int cpu, int flag)
 	return sd;
 }
 
-DECLARE_PER_CPU(struct sched_domain *, sd_llc);
+DECLARE_PER_CPU(struct sched_domain __rcu *, sd_llc);
 DECLARE_PER_CPU(int, sd_llc_size);
 DECLARE_PER_CPU(int, sd_llc_id);
-DECLARE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
-DECLARE_PER_CPU(struct sched_domain *, sd_numa);
-DECLARE_PER_CPU(struct sched_domain *, sd_asym_packing);
-DECLARE_PER_CPU(struct sched_domain *, sd_asym_cpucapacity);
+DECLARE_PER_CPU(struct sched_domain_shared __rcu *, sd_llc_shared);
+DECLARE_PER_CPU(struct sched_domain __rcu *, sd_numa);
+DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
+DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
 extern struct static_key_false sched_asym_cpucapacity;
 
 struct sched_group_capacity {
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 3f35ba1d8fde..0844ee757dad 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -586,13 +586,13 @@ static void destroy_sched_domains(struct sched_domain *sd)
  * the cpumask of the domain), this allows us to quickly tell if
  * two CPUs are in the same cache domain, see cpus_share_cache().
  */
-DEFINE_PER_CPU(struct sched_domain *, sd_llc);
+DEFINE_PER_CPU(struct sched_domain __rcu *, sd_llc);
 DEFINE_PER_CPU(int, sd_llc_size);
 DEFINE_PER_CPU(int, sd_llc_id);
-DEFINE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
-DEFINE_PER_CPU(struct sched_domain *, sd_numa);
-DEFINE_PER_CPU(struct sched_domain *, sd_asym_packing);
-DEFINE_PER_CPU(struct sched_domain *, sd_asym_cpucapacity);
+DEFINE_PER_CPU(struct sched_domain_shared __rcu *, sd_llc_shared);
+DEFINE_PER_CPU(struct sched_domain __rcu *, sd_numa);
+DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
+DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
 DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
 
 static void update_top_cache_domain(int cpu)
-- 
2.21.0.rc0.258.g878e2cd30e-goog

[PATCH v2 5/6] rcuwait: Annotate task_struct with __rcu

[Joel Fernandes (Google)]

This suppresses sparse error generated due to the recently added
rcu_assign_pointer sparse check.

percpu-rwsem.c:162:9: sparse: error: incompatible types in comparison expression
exit.c:316:16: sparse: error: incompatible types in comparison expression

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 include/linux/rcuwait.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/rcuwait.h b/include/linux/rcuwait.h
index 90bfa3279a01..563290fc194f 100644
--- a/include/linux/rcuwait.h
+++ b/include/linux/rcuwait.h
@@ -18,7 +18,7 @@
  * awoken.
  */
 struct rcuwait {
-	struct task_struct *task;
+	struct task_struct __rcu *task;
 };
 
 #define __RCUWAIT_INITIALIZER(name)		\
-- 
2.21.0.rc0.258.g878e2cd30e-goog

[PATCH v2 6/6] sched: Annotate perf_domain pointer with __rcu

[Joel Fernandes (Google)]

This fixes the following sparse errors in sched/fair.c:

fair.c:6506:14: error: incompatible types in comparison expression
fair.c:8642:21: error: incompatible types in comparison expression

Using __rcu will also help sparse catch any future bugs.

Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
 kernel/sched/sched.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index ca6a79f57e7a..c8e6514433a9 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -780,7 +780,7 @@ struct root_domain {
 	 * NULL-terminated list of performance domains intersecting with the
 	 * CPUs of the rd. Protected by RCU.
 	 */
-	struct perf_domain	*pd;
+	struct perf_domain __rcu *pd;
 };
 
 extern struct root_domain def_root_domain;
-- 
2.21.0.rc0.258.g878e2cd30e-goog

Re: [PATCH v2 1/6] net: rtnetlink: Fix incorrect RCU API usage

[Paul E. McKenney]

On Sat, Feb 23, 2019 at 01:34:29AM -0500, Joel Fernandes (Google) wrote:
> rtnl_register_internal() and rtnl_unregister_all tries to directly
> dereference an RCU protected pointed outside RCU read side section.
> While this is Ok to do since a lock is held, let us use the correct
> API to avoid programmer bugs in the future.
> 
> This also fixes sparse warnings arising from not using RCU API.
> 
> net/core/rtnetlink.c:332:13: warning: incorrect type in assignment
> (different address spaces) net/core/rtnetlink.c:332:13:    expected
> struct rtnl_link **tab net/core/rtnetlink.c:332:13:    got struct
> rtnl_link *[noderef] <asn:4>*<noident>
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

>From an RCU perspective:

Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

> ---
>  net/core/rtnetlink.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
> index 5ea1bed08ede..98be4b4818a9 100644
> --- a/net/core/rtnetlink.c
> +++ b/net/core/rtnetlink.c
> @@ -188,7 +188,7 @@ static int rtnl_register_internal(struct module *owner,
>  	msgindex = rtm_msgindex(msgtype);
>  
>  	rtnl_lock();
> -	tab = rtnl_msg_handlers[protocol];
> +	tab = rtnl_dereference(rtnl_msg_handlers[protocol]);
>  	if (tab == NULL) {
>  		tab = kcalloc(RTM_NR_MSGTYPES, sizeof(void *), GFP_KERNEL);
>  		if (!tab)
> @@ -329,7 +329,7 @@ void rtnl_unregister_all(int protocol)
>  	BUG_ON(protocol < 0 || protocol > RTNL_FAMILY_MAX);
>  
>  	rtnl_lock();
> -	tab = rtnl_msg_handlers[protocol];
> +	tab = rtnl_dereference(rtnl_msg_handlers[protocol]);
>  	if (!tab) {
>  		rtnl_unlock();
>  		return;
> -- 
> 2.21.0.rc0.258.g878e2cd30e-goog
> 

Re: [PATCH v2 2/6] ixgbe: Fix incorrect RCU API usage

[Paul E. McKenney]

On Sat, Feb 23, 2019 at 01:34:30AM -0500, Joel Fernandes (Google) wrote:
> Recently, I added an RCU annotation check in rcu_assign_pointer. This
> caused a sparse error to be reported by the ixgbe driver.
> 
> Further looking, it seems the adapter->xdp_prog pointer is not annotated
> with __rcu. Annonating it fixed the error, but caused a bunch of other
> warnings.
> 
> This patch tries to fix all warnings by using RCU API properly. This
> makes sense to do because not using RCU properly can result in various
> hard to find bugs. This is a best effort fix and is only build tested.
> The sparse errors and warnings go away with the change. I request
> maintainers / developers in this area to review / test it properly.
> 
> The sparse error fixed is:
> ixgbe_main.c:10256:25: error: incompatible types in comparison expression
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

>From an RCU perspective:

Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

> ---
>  drivers/net/ethernet/intel/ixgbe/ixgbe.h      |  4 ++--
>  drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 ++++++++++-----
>  2 files changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
> index 08d85e336bd4..3b14daf27516 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
> @@ -311,7 +311,7 @@ struct ixgbe_ring {
>  	struct ixgbe_ring *next;	/* pointer to next ring in q_vector */
>  	struct ixgbe_q_vector *q_vector; /* backpointer to host q_vector */
>  	struct net_device *netdev;	/* netdev ring belongs to */
> -	struct bpf_prog *xdp_prog;
> +	struct bpf_prog __rcu *xdp_prog;
>  	struct device *dev;		/* device for DMA mapping */
>  	void *desc;			/* descriptor ring memory */
>  	union {
> @@ -560,7 +560,7 @@ struct ixgbe_adapter {
>  	unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
>  	/* OS defined structs */
>  	struct net_device *netdev;
> -	struct bpf_prog *xdp_prog;
> +	struct bpf_prog __rcu *xdp_prog;
>  	struct pci_dev *pdev;
>  	struct mii_bus *mii_bus;
>  
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> index daff8183534b..408a312aa6ba 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> @@ -2199,7 +2199,7 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter,
>  	u32 act;
>  
>  	rcu_read_lock();
> -	xdp_prog = READ_ONCE(rx_ring->xdp_prog);
> +	xdp_prog = rcu_dereference(rx_ring->xdp_prog);
>  
>  	if (!xdp_prog)
>  		goto xdp_out;
> @@ -6547,7 +6547,7 @@ int ixgbe_setup_rx_resources(struct ixgbe_adapter *adapter,
>  			     rx_ring->queue_index) < 0)
>  		goto err;
>  
> -	rx_ring->xdp_prog = adapter->xdp_prog;
> +	rcu_assign_pointer(rx_ring->xdp_prog, adapter->xdp_prog);
>  
>  	return 0;
>  err:
> @@ -10246,7 +10246,8 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
>  	if (nr_cpu_ids > MAX_XDP_QUEUES)
>  		return -ENOMEM;
>  
> -	old_prog = xchg(&adapter->xdp_prog, prog);
> +	old_prog = rcu_access_pointer(adapter->xdp_prog);
> +	rcu_assign_pointer(adapter->xdp_prog, prog);
>  
>  	/* If transitioning XDP modes reconfigure rings */
>  	if (!!prog != !!old_prog) {
> @@ -10271,13 +10272,17 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
>  static int ixgbe_xdp(struct net_device *dev, struct netdev_bpf *xdp)
>  {
>  	struct ixgbe_adapter *adapter = netdev_priv(dev);
> +	struct bpf_prog *prog;
>  
>  	switch (xdp->command) {
>  	case XDP_SETUP_PROG:
>  		return ixgbe_xdp_setup(dev, xdp->prog);
>  	case XDP_QUERY_PROG:
> -		xdp->prog_id = adapter->xdp_prog ?
> -			adapter->xdp_prog->aux->id : 0;
> +		rcu_read_lock();
> +		prog = rcu_dereference(adapter->xdp_prog);
> +		xdp->prog_id = prog ? prog->aux->id : 0;
> +		rcu_read_unlock();
> +
>  		return 0;
>  	case XDP_QUERY_XSK_UMEM:
>  		return ixgbe_xsk_umem_query(adapter, &xdp->xsk.umem,
> -- 
> 2.21.0.rc0.258.g878e2cd30e-goog
> 

Re: [PATCH v2 3/6] sched/cpufreq: Annotate cpufreq_update_util_data pointer with __rcu

[Paul E. McKenney]

On Sat, Feb 23, 2019 at 01:34:31AM -0500, Joel Fernandes (Google) wrote:
> Recently I added an RCU annotation check to rcu_assign_pointer(). All
> pointers assigned to RCU protected data are to be annotated with __rcu
> inorder to be able to use rcu_assign_pointer() similar to checks in
> other RCU APIs.
> 
> This resulted in a sparse error: kernel//sched/cpufreq.c:41:9: sparse:
> error: incompatible types in comparison expression (different address
> spaces)
> 
> Fix this by annotating cpufreq_update_util_data pointer with __rcu. This
> will also help sparse catch any future RCU misuage bugs.
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

>From an RCU perspective:

Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

> ---
>  kernel/sched/cpufreq.c | 2 +-
>  kernel/sched/sched.h   | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/cpufreq.c b/kernel/sched/cpufreq.c
> index 22bd8980f32f..e316ee7bb2e5 100644
> --- a/kernel/sched/cpufreq.c
> +++ b/kernel/sched/cpufreq.c
> @@ -7,7 +7,7 @@
>   */
>  #include "sched.h"
>  
> -DEFINE_PER_CPU(struct update_util_data *, cpufreq_update_util_data);
> +DEFINE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
>  
>  /**
>   * cpufreq_add_update_util_hook - Populate the CPU's update_util_data pointer.
> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index d04530bf251f..2ab545d40381 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -2166,7 +2166,7 @@ static inline u64 irq_time_read(int cpu)
>  #endif /* CONFIG_IRQ_TIME_ACCOUNTING */
>  
>  #ifdef CONFIG_CPU_FREQ
> -DECLARE_PER_CPU(struct update_util_data *, cpufreq_update_util_data);
> +DECLARE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
>  
>  /**
>   * cpufreq_update_util - Take a note about CPU utilization changes.
> -- 
> 2.21.0.rc0.258.g878e2cd30e-goog
> 

Re: [PATCH v2 4/6] sched_domain: Annotate RCU pointers properly

[Paul E. McKenney]

On Sat, Feb 23, 2019 at 01:34:32AM -0500, Joel Fernandes (Google) wrote:
> The scheduler uses RCU API in various places to access sched_domain
> pointers. These cause sparse errors as below.
> 
> Many new errors show up because of an annotation check I added to
> rcu_assign_pointer(). Let us annotate the pointers correctly which also
> will help sparse catch any potential future bugs.
> 
> This fixes the following sparse errors:
> 
> rt.c:1681:9: error: incompatible types in comparison expression
> deadline.c:1904:9: error: incompatible types in comparison expression
> core.c:519:9: error: incompatible types in comparison expression
> core.c:1634:17: error: incompatible types in comparison expression
> fair.c:6193:14: error: incompatible types in comparison expression
> fair.c:9883:22: error: incompatible types in comparison expression
> fair.c:9897:9: error: incompatible types in comparison expression
> sched.h:1287:9: error: incompatible types in comparison expression
> topology.c:612:9: error: incompatible types in comparison expression
> topology.c:615:9: error: incompatible types in comparison expression
> sched.h:1300:9: error: incompatible types in comparison expression
> topology.c:618:9: error: incompatible types in comparison expression
> sched.h:1287:9: error: incompatible types in comparison expression
> topology.c:621:9: error: incompatible types in comparison expression
> sched.h:1300:9: error: incompatible types in comparison expression
> topology.c:624:9: error: incompatible types in comparison expression
> topology.c:671:9: error: incompatible types in comparison expression
> stats.c:45:17: error: incompatible types in comparison expression
> fair.c:5998:15: error: incompatible types in comparison expression
> fair.c:5989:15: error: incompatible types in comparison expression
> fair.c:5998:15: error: incompatible types in comparison expression
> fair.c:5989:15: error: incompatible types in comparison expression
> fair.c:6120:19: error: incompatible types in comparison expression
> fair.c:6506:14: error: incompatible types in comparison expression
> fair.c:6515:14: error: incompatible types in comparison expression
> fair.c:6623:9: error: incompatible types in comparison expression
> fair.c:5970:17: error: incompatible types in comparison expression
> fair.c:8642:21: error: incompatible types in comparison expression
> fair.c:9253:9: error: incompatible types in comparison expression
> fair.c:9331:9: error: incompatible types in comparison expression
> fair.c:9519:15: error: incompatible types in comparison expression
> fair.c:9533:14: error: incompatible types in comparison expression
> fair.c:9542:14: error: incompatible types in comparison expression
> fair.c:9567:14: error: incompatible types in comparison expression
> fair.c:9597:14: error: incompatible types in comparison expression
> fair.c:9421:16: error: incompatible types in comparison expression
> fair.c:9421:16: error: incompatible types in comparison expression
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

>From an RCU perspective:

Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

> ---
>  include/linux/sched/topology.h |  4 ++--
>  kernel/sched/sched.h           | 14 +++++++-------
>  kernel/sched/topology.c        | 10 +++++-----
>  3 files changed, 14 insertions(+), 14 deletions(-)
> 
> diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
> index c31d3a47a47c..4819c9e01e42 100644
> --- a/include/linux/sched/topology.h
> +++ b/include/linux/sched/topology.h
> @@ -76,8 +76,8 @@ struct sched_domain_shared {
>  
>  struct sched_domain {
>  	/* These fields must be setup */
> -	struct sched_domain *parent;	/* top domain must be null terminated */
> -	struct sched_domain *child;	/* bottom domain must be null terminated */
> +	struct sched_domain __rcu *parent;	/* top domain must be null terminated */
> +	struct sched_domain __rcu *child;	/* bottom domain must be null terminated */
>  	struct sched_group *groups;	/* the balancing groups of the domain */
>  	unsigned long min_interval;	/* Minimum balance interval ms */
>  	unsigned long max_interval;	/* Maximum balance interval ms */
> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index 2ab545d40381..ca6a79f57e7a 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -866,8 +866,8 @@ struct rq {
>  	atomic_t		nr_iowait;
>  
>  #ifdef CONFIG_SMP
> -	struct root_domain	*rd;
> -	struct sched_domain	*sd;
> +	struct root_domain		*rd;
> +	struct sched_domain __rcu	*sd;
>  
>  	unsigned long		cpu_capacity;
>  	unsigned long		cpu_capacity_orig;
> @@ -1305,13 +1305,13 @@ static inline struct sched_domain *lowest_flag_domain(int cpu, int flag)
>  	return sd;
>  }
>  
> -DECLARE_PER_CPU(struct sched_domain *, sd_llc);
> +DECLARE_PER_CPU(struct sched_domain __rcu *, sd_llc);
>  DECLARE_PER_CPU(int, sd_llc_size);
>  DECLARE_PER_CPU(int, sd_llc_id);
> -DECLARE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
> -DECLARE_PER_CPU(struct sched_domain *, sd_numa);
> -DECLARE_PER_CPU(struct sched_domain *, sd_asym_packing);
> -DECLARE_PER_CPU(struct sched_domain *, sd_asym_cpucapacity);
> +DECLARE_PER_CPU(struct sched_domain_shared __rcu *, sd_llc_shared);
> +DECLARE_PER_CPU(struct sched_domain __rcu *, sd_numa);
> +DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
> +DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
>  extern struct static_key_false sched_asym_cpucapacity;
>  
>  struct sched_group_capacity {
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 3f35ba1d8fde..0844ee757dad 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -586,13 +586,13 @@ static void destroy_sched_domains(struct sched_domain *sd)
>   * the cpumask of the domain), this allows us to quickly tell if
>   * two CPUs are in the same cache domain, see cpus_share_cache().
>   */
> -DEFINE_PER_CPU(struct sched_domain *, sd_llc);
> +DEFINE_PER_CPU(struct sched_domain __rcu *, sd_llc);
>  DEFINE_PER_CPU(int, sd_llc_size);
>  DEFINE_PER_CPU(int, sd_llc_id);
> -DEFINE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
> -DEFINE_PER_CPU(struct sched_domain *, sd_numa);
> -DEFINE_PER_CPU(struct sched_domain *, sd_asym_packing);
> -DEFINE_PER_CPU(struct sched_domain *, sd_asym_cpucapacity);
> +DEFINE_PER_CPU(struct sched_domain_shared __rcu *, sd_llc_shared);
> +DEFINE_PER_CPU(struct sched_domain __rcu *, sd_numa);
> +DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
> +DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
>  DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
>  
>  static void update_top_cache_domain(int cpu)
> -- 
> 2.21.0.rc0.258.g878e2cd30e-goog
> 

Re: [PATCH v2 5/6] rcuwait: Annotate task_struct with __rcu

[Paul E. McKenney]

On Sat, Feb 23, 2019 at 01:34:33AM -0500, Joel Fernandes (Google) wrote:
> This suppresses sparse error generated due to the recently added
> rcu_assign_pointer sparse check.
> 
> percpu-rwsem.c:162:9: sparse: error: incompatible types in comparison expression
> exit.c:316:16: sparse: error: incompatible types in comparison expression
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

>From an RCU perspective:

Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

> ---
>  include/linux/rcuwait.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/rcuwait.h b/include/linux/rcuwait.h
> index 90bfa3279a01..563290fc194f 100644
> --- a/include/linux/rcuwait.h
> +++ b/include/linux/rcuwait.h
> @@ -18,7 +18,7 @@
>   * awoken.
>   */
>  struct rcuwait {
> -	struct task_struct *task;
> +	struct task_struct __rcu *task;
>  };
>  
>  #define __RCUWAIT_INITIALIZER(name)		\
> -- 
> 2.21.0.rc0.258.g878e2cd30e-goog
> 

Re: [PATCH v2 6/6] sched: Annotate perf_domain pointer with __rcu

[Paul E. McKenney]

On Sat, Feb 23, 2019 at 01:34:34AM -0500, Joel Fernandes (Google) wrote:
> This fixes the following sparse errors in sched/fair.c:
> 
> fair.c:6506:14: error: incompatible types in comparison expression
> fair.c:8642:21: error: incompatible types in comparison expression
> 
> Using __rcu will also help sparse catch any future bugs.
> 
> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>

>From an RCU perspective:

Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

> ---
>  kernel/sched/sched.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index ca6a79f57e7a..c8e6514433a9 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -780,7 +780,7 @@ struct root_domain {
>  	 * NULL-terminated list of performance domains intersecting with the
>  	 * CPUs of the rd. Protected by RCU.
>  	 */
> -	struct perf_domain	*pd;
> +	struct perf_domain __rcu *pd;
>  };
>  
>  extern struct root_domain def_root_domain;
> -- 
> 2.21.0.rc0.258.g878e2cd30e-goog
> 

Re: [PATCH v2 3/6] sched/cpufreq: Annotate cpufreq_update_util_data pointer with __rcu

[Joel Fernandes]

On Mon, Feb 25, 2019 at 01:10:26PM -0800, Paul E. McKenney wrote:
> On Sat, Feb 23, 2019 at 01:34:31AM -0500, Joel Fernandes (Google) wrote:
> > Recently I added an RCU annotation check to rcu_assign_pointer(). All
> > pointers assigned to RCU protected data are to be annotated with __rcu
> > inorder to be able to use rcu_assign_pointer() similar to checks in
> > other RCU APIs.
> > 
> > This resulted in a sparse error: kernel//sched/cpufreq.c:41:9: sparse:
> > error: incompatible types in comparison expression (different address
> > spaces)
> > 
> > Fix this by annotating cpufreq_update_util_data pointer with __rcu. This
> > will also help sparse catch any future RCU misuage bugs.
> > 
> > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> 
> From an RCU perspective:
> 
> Reviewed-by: Paul E. McKenney <paulmck@linux.ibm.com>

Thanks a lot Paul.

Peter, Rafael, Steve,
Are you Ok with the patches 3-6? It will be nice to quiet down sparse.

thanks,

 - Joel

> > ---
> >  kernel/sched/cpufreq.c | 2 +-
> >  kernel/sched/sched.h   | 2 +-
> >  2 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kernel/sched/cpufreq.c b/kernel/sched/cpufreq.c
> > index 22bd8980f32f..e316ee7bb2e5 100644
> > --- a/kernel/sched/cpufreq.c
> > +++ b/kernel/sched/cpufreq.c
> > @@ -7,7 +7,7 @@
> >   */
> >  #include "sched.h"
> >  
> > -DEFINE_PER_CPU(struct update_util_data *, cpufreq_update_util_data);
> > +DEFINE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
> >  
> >  /**
> >   * cpufreq_add_update_util_hook - Populate the CPU's update_util_data pointer.
> > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> > index d04530bf251f..2ab545d40381 100644
> > --- a/kernel/sched/sched.h
> > +++ b/kernel/sched/sched.h
> > @@ -2166,7 +2166,7 @@ static inline u64 irq_time_read(int cpu)
> >  #endif /* CONFIG_IRQ_TIME_ACCOUNTING */
> >  
> >  #ifdef CONFIG_CPU_FREQ
> > -DECLARE_PER_CPU(struct update_util_data *, cpufreq_update_util_data);
> > +DECLARE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
> >  
> >  /**
> >   * cpufreq_update_util - Take a note about CPU utilization changes.
> > -- 
> > 2.21.0.rc0.258.g878e2cd30e-goog
> > 
>