All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH RFC] sched: Allow migrating kthreads into online but inactive CPUs
@ 2017-06-17 12:10 Tejun Heo
  2017-06-17 12:11 ` simple repro case Tejun Heo
  2017-07-25 16:58 ` [PATCH RFC] sched: Allow migrating kthreads into online but inactive CPUs Peter Zijlstra
  0 siblings, 2 replies; 8+ messages in thread
From: Tejun Heo @ 2017-06-17 12:10 UTC (permalink / raw)
  To: Ingo Molnar, Peter Zijlstra
  Cc: Paul E. McKenney, Steven Rostedt, linux-kernel, Lai Jiangshan,
	kernel-team

Per-cpu workqueues have been tripping CPU affinity sanity checks while
a CPU is being offlined.  A per-cpu kworker ends up running on a CPU
which isn't its target CPU while the CPU is online but inactive.

While the scheduler allows kthreads to wake up on an online but
inactive CPU, it doesn't allow a running kthread to be migrated to
such a CPU, which leads to an odd situation where setting affinity on
a sleeping and running kthread leads to different results.

Each mem-reclaim workqueue has one rescuer which guarantees forward
progress and the rescuer needs to bind itself to the CPU which needs
help in making forward progress; however, due to the above issue,
while set_cpus_allowed_ptr() succeeds, the rescuer doesn't end up on
the correct CPU if the CPU is in the process of going offline,
tripping the sanity check and executing the work item on the wrong
CPU.

This patch updates __migrate_task() so that kthreads can be migrated
into an inactive but online CPU.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Reported-by: Steven Rostedt <rostedt@goodmis.org>
---
 kernel/sched/core.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 803c3bc274c4..1500217ce4b4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -980,8 +980,13 @@ struct migration_arg {
 static struct rq *__migrate_task(struct rq *rq, struct rq_flags *rf,
 				 struct task_struct *p, int dest_cpu)
 {
-	if (unlikely(!cpu_active(dest_cpu)))
-		return rq;
+	if (p->flags & PF_KTHREAD) {
+		if (unlikely(!cpu_online(dest_cpu)))
+			return rq;
+	} else {
+		if (unlikely(!cpu_active(dest_cpu)))
+			return rq;
+	}
 
 	/* Affinity changed (again). */
 	if (!cpumask_test_cpu(dest_cpu, &p->cpus_allowed))

^ permalink raw reply related	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2018-05-31 12:28 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-17 12:10 [PATCH RFC] sched: Allow migrating kthreads into online but inactive CPUs Tejun Heo
2017-06-17 12:11 ` simple repro case Tejun Heo
2017-06-21 14:24   ` Steven Rostedt
2017-06-21 17:59     ` Tejun Heo
2017-07-25 16:58 ` [PATCH RFC] sched: Allow migrating kthreads into online but inactive CPUs Peter Zijlstra
2017-07-25 17:52   ` Paul E. McKenney
2017-07-26 12:57   ` Paul E. McKenney
2018-05-31 12:28   ` [tip:sched/urgent] sched/core: Fix rules for running on online && !active CPUs tip-bot for Peter Zijlstra

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.