All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing
@ 2020-02-27 19:18 Mel Gorman
  2020-02-27 19:41 ` Paul E. McKenney
  2020-03-06 14:42 ` [tip: sched/core] " tip-bot2 for Mel Gorman
  0 siblings, 2 replies; 3+ messages in thread
From: Mel Gorman @ 2020-02-27 19:18 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Qian Cai, Peter Zijlstra, Juri Lelli, Steven Rostedt,
	Valentin Schneider, paulmck, linux-kernel

Qian Cai reported the following

  The linux-next commit ff7db0bf24db ("sched/numa: Prefer using an idle CPU as a
  migration target instead of comparing tasks") introduced a boot warning,

  [   86.520534][    T1] WARNING: suspicious RCU usage
  [   86.520540][    T1] 5.6.0-rc3-next-20200227 #7 Not tainted
  [   86.520545][    T1] -----------------------------
  [   86.520551][    T1] kernel/sched/fair.c:5914 suspicious rcu_dereference_check() usage!
  [   86.520555][    T1]
  [   86.520555][    T1] other info that might help us debug this:
  [   86.520555][    T1]
  [   86.520561][    T1]
  [   86.520561][    T1] rcu_scheduler_active = 2, debug_locks = 1
  [   86.520567][    T1] 1 lock held by systemd/1:
  [   86.520571][    T1]  #0: ffff8887f4b14848 (&mm->mmap_sem#2){++++}, at: do_page_fault+0x1d2/0x998
  [   86.520594][    T1]
  [   86.520594][    T1] stack backtrace:
  [   86.520602][    T1] CPU: 1 PID: 1 Comm: systemd Not tainted 5.6.0-rc3-next-20200227 #7

task_numa_migrate() checks for idle cores when updating NUMA-related statistics.
This relies on reading a RCU-protected structure in test_idle_cores() via this
call chain

task_numa_migrate
  -> update_numa_stats
    -> numa_idle_core
      -> test_idle_cores

While the locking could be fine-grained, it is more appropriate to acquire
the RCU lock for the entire scan of the domain. This patch removes the
warning triggered at boot time.

Fixes: ff7db0bf24db ("sched/numa: Prefer using an idle CPU as a migration target instead of comparing tasks")
Reported-by: Qian Cai <cai@lca.pw>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
 kernel/sched/fair.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 10f9e6729fcf..1592b6d26239 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1595,6 +1595,7 @@ static void update_numa_stats(struct task_numa_env *env,
 	memset(ns, 0, sizeof(*ns));
 	ns->idle_cpu = -1;
 
+	rcu_read_lock();
 	for_each_cpu(cpu, cpumask_of_node(nid)) {
 		struct rq *rq = cpu_rq(cpu);
 
@@ -1614,6 +1615,7 @@ static void update_numa_stats(struct task_numa_env *env,
 			idle_core = numa_idle_core(idle_core, cpu);
 		}
 	}
+	rcu_read_unlock();
 
 	ns->weight = cpumask_weight(cpumask_of_node(nid));
 

^ permalink raw reply related	[flat|nested] 3+ messages in thread
* Re: [PATCH] sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing
  2020-02-27 19:18 [PATCH] sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing Mel Gorman
@ 2020-02-27 19:41 ` Paul E. McKenney
  2020-03-06 14:42 ` [tip: sched/core] " tip-bot2 for Mel Gorman
  1 sibling, 0 replies; 3+ messages in thread
From: Paul E. McKenney @ 2020-02-27 19:41 UTC (permalink / raw)
  To: Mel Gorman
  Cc: Ingo Molnar, Qian Cai, Peter Zijlstra, Juri Lelli,
	Steven Rostedt, Valentin Schneider, linux-kernel

On Thu, Feb 27, 2020 at 07:18:04PM +0000, Mel Gorman wrote:
> Qian Cai reported the following
> 
>   The linux-next commit ff7db0bf24db ("sched/numa: Prefer using an idle CPU as a
>   migration target instead of comparing tasks") introduced a boot warning,
> 
>   [   86.520534][    T1] WARNING: suspicious RCU usage
>   [   86.520540][    T1] 5.6.0-rc3-next-20200227 #7 Not tainted
>   [   86.520545][    T1] -----------------------------
>   [   86.520551][    T1] kernel/sched/fair.c:5914 suspicious rcu_dereference_check() usage!
>   [   86.520555][    T1]
>   [   86.520555][    T1] other info that might help us debug this:
>   [   86.520555][    T1]
>   [   86.520561][    T1]
>   [   86.520561][    T1] rcu_scheduler_active = 2, debug_locks = 1
>   [   86.520567][    T1] 1 lock held by systemd/1:
>   [   86.520571][    T1]  #0: ffff8887f4b14848 (&mm->mmap_sem#2){++++}, at: do_page_fault+0x1d2/0x998
>   [   86.520594][    T1]
>   [   86.520594][    T1] stack backtrace:
>   [   86.520602][    T1] CPU: 1 PID: 1 Comm: systemd Not tainted 5.6.0-rc3-next-20200227 #7
> 
> task_numa_migrate() checks for idle cores when updating NUMA-related statistics.
> This relies on reading a RCU-protected structure in test_idle_cores() via this
> call chain
> 
> task_numa_migrate
>   -> update_numa_stats
>     -> numa_idle_core
>       -> test_idle_cores
> 
> While the locking could be fine-grained, it is more appropriate to acquire
> the RCU lock for the entire scan of the domain. This patch removes the
> warning triggered at boot time.
> 
> Fixes: ff7db0bf24db ("sched/numa: Prefer using an idle CPU as a migration target instead of comparing tasks")
> Reported-by: Qian Cai <cai@lca.pw>
> Signed-off-by: Mel Gorman <mgorman@techsingularity.net>

From an RCU viewpoint:

Reviewed-by: Paul E. McKenney <paulmck@kernel.org>

> ---
>  kernel/sched/fair.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 10f9e6729fcf..1592b6d26239 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -1595,6 +1595,7 @@ static void update_numa_stats(struct task_numa_env *env,
>  	memset(ns, 0, sizeof(*ns));
>  	ns->idle_cpu = -1;
>  
> +	rcu_read_lock();
>  	for_each_cpu(cpu, cpumask_of_node(nid)) {
>  		struct rq *rq = cpu_rq(cpu);
>  
> @@ -1614,6 +1615,7 @@ static void update_numa_stats(struct task_numa_env *env,
>  			idle_core = numa_idle_core(idle_core, cpu);
>  		}
>  	}
> +	rcu_read_unlock();
>  
>  	ns->weight = cpumask_weight(cpumask_of_node(nid));
>  

^ permalink raw reply	[flat|nested] 3+ messages in thread
* [tip: sched/core] sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing
  2020-02-27 19:18 [PATCH] sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing Mel Gorman
  2020-02-27 19:41 ` Paul E. McKenney
@ 2020-03-06 14:42 ` tip-bot2 for Mel Gorman
  1 sibling, 0 replies; 3+ messages in thread
From: tip-bot2 for Mel Gorman @ 2020-03-06 14:42 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Qian Cai, Paul E. McKenney, Mel Gorman, Peter Zijlstra (Intel),
	Ingo Molnar, x86, LKML

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     0621df315402dd7bc56f7272fae9778701289825
Gitweb:        https://git.kernel.org/tip/0621df315402dd7bc56f7272fae9778701289825
Author:        Mel Gorman <mgorman@techsingularity.net>
AuthorDate:    Thu, 27 Feb 2020 19:18:04 
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Fri, 06 Mar 2020 12:57:22 +01:00

sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing

Qian Cai reported the following bug:

  The linux-next commit ff7db0bf24db ("sched/numa: Prefer using an idle CPU as a
  migration target instead of comparing tasks") introduced a boot warning,

  [   86.520534][    T1] WARNING: suspicious RCU usage
  [   86.520540][    T1] 5.6.0-rc3-next-20200227 #7 Not tainted
  [   86.520545][    T1] -----------------------------
  [   86.520551][    T1] kernel/sched/fair.c:5914 suspicious rcu_dereference_check() usage!
  [   86.520555][    T1]
  [   86.520555][    T1] other info that might help us debug this:
  [   86.520555][    T1]
  [   86.520561][    T1]
  [   86.520561][    T1] rcu_scheduler_active = 2, debug_locks = 1
  [   86.520567][    T1] 1 lock held by systemd/1:
  [   86.520571][    T1]  #0: ffff8887f4b14848 (&mm->mmap_sem#2){++++}, at: do_page_fault+0x1d2/0x998
  [   86.520594][    T1]
  [   86.520594][    T1] stack backtrace:
  [   86.520602][    T1] CPU: 1 PID: 1 Comm: systemd Not tainted 5.6.0-rc3-next-20200227 #7

task_numa_migrate() checks for idle cores when updating NUMA-related statistics.
This relies on reading a RCU-protected structure in test_idle_cores() via this
call chain

task_numa_migrate
  -> update_numa_stats
    -> numa_idle_core
      -> test_idle_cores

While the locking could be fine-grained, it is more appropriate to acquire
the RCU lock for the entire scan of the domain. This patch removes the
warning triggered at boot time.

Reported-by: Qian Cai <cai@lca.pw>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Fixes: ff7db0bf24db ("sched/numa: Prefer using an idle CPU as a migration target instead of comparing tasks")
Link: https://lkml.kernel.org/r/20200227191804.GJ3818@techsingularity.net
---
 kernel/sched/fair.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index bba9452..3887b73 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1608,6 +1608,7 @@ static void update_numa_stats(struct task_numa_env *env,
 	memset(ns, 0, sizeof(*ns));
 	ns->idle_cpu = -1;
 
+	rcu_read_lock();
 	for_each_cpu(cpu, cpumask_of_node(nid)) {
 		struct rq *rq = cpu_rq(cpu);
 
@@ -1627,6 +1628,7 @@ static void update_numa_stats(struct task_numa_env *env,
 			idle_core = numa_idle_core(idle_core, cpu);
 		}
 	}
+	rcu_read_unlock();
 
 	ns->weight = cpumask_weight(cpumask_of_node(nid));
 

^ permalink raw reply related	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-03-06 14:42 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-27 19:18 [PATCH] sched/numa: Acquire RCU lock for checking idle cores during NUMA balancing Mel Gorman
2020-02-27 19:41 ` Paul E. McKenney
2020-03-06 14:42 ` [tip: sched/core] " tip-bot2 for Mel Gorman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.