From: "tip-bot2 for Peter Zijlstra" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: stable@kernel.org,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
Valentin Schneider <valentin.schneider@arm.com>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: sched/urgent] sched: Simplify migration_cpu_stop()
Date: Mon, 01 Mar 2021 10:16:17 -0000 [thread overview]
Message-ID: <161459377764.20312.14971720353963713012.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20210224131355.430014682@infradead.org>
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: 6430eb536a97036b1d529cbf383cfe36e41a2f97
Gitweb: https://git.kernel.org/tip/6430eb536a97036b1d529cbf383cfe36e41a2f97
Author: Peter Zijlstra <peterz@infradead.org>
AuthorDate: Wed, 24 Feb 2021 11:50:39 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Mon, 01 Mar 2021 11:02:13 +01:00
sched: Simplify migration_cpu_stop()
When affine_move_task() issues a migration_cpu_stop(), the purpose of
that function is to complete that @pending, not any random other
p->migration_pending that might have gotten installed since.
This realization much simplifies migration_cpu_stop() and allows
further necessary steps to fix all this as it provides the guarantee
that @pending's stopper will complete @pending (and not some random
other @pending).
Fixes: 6d337eab041d ("sched: Fix migrate_disable() vs set_cpus_allowed_ptr()")
Cc: stable@kernel.org
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Link: https://lkml.kernel.org/r/20210224131355.430014682@infradead.org
---
kernel/sched/core.c | 56 ++++++--------------------------------------
1 file changed, 8 insertions(+), 48 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 79ddba5..088e8f4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1898,8 +1898,8 @@ static struct rq *__migrate_task(struct rq *rq, struct rq_flags *rf,
*/
static int migration_cpu_stop(void *data)
{
- struct set_affinity_pending *pending;
struct migration_arg *arg = data;
+ struct set_affinity_pending *pending = arg->pending;
struct task_struct *p = arg->task;
int dest_cpu = arg->dest_cpu;
struct rq *rq = this_rq();
@@ -1921,25 +1921,6 @@ static int migration_cpu_stop(void *data)
raw_spin_lock(&p->pi_lock);
rq_lock(rq, &rf);
- pending = p->migration_pending;
- if (pending && !arg->pending) {
- /*
- * This happens from sched_exec() and migrate_task_to(),
- * neither of them care about pending and just want a task to
- * maybe move about.
- *
- * Even if there is a pending, we can ignore it, since
- * affine_move_task() will have it's own stop_work's in flight
- * which will manage the completion.
- *
- * Notably, pending doesn't need to match arg->pending. This can
- * happen when tripple concurrent affine_move_task() first sets
- * pending, then clears pending and eventually sets another
- * pending.
- */
- pending = NULL;
- }
-
/*
* If task_rq(p) != rq, it cannot be migrated here, because we're
* holding rq->lock, if p->on_rq == 0 it cannot get enqueued because
@@ -1950,31 +1931,20 @@ static int migration_cpu_stop(void *data)
goto out;
if (pending) {
- p->migration_pending = NULL;
+ if (p->migration_pending == pending)
+ p->migration_pending = NULL;
complete = true;
}
- /* migrate_enable() -- we must not race against SCA */
- if (dest_cpu < 0) {
- /*
- * When this was migrate_enable() but we no longer
- * have a @pending, a concurrent SCA 'fixed' things
- * and we should be valid again. Nothing to do.
- */
- if (!pending) {
- WARN_ON_ONCE(!cpumask_test_cpu(task_cpu(p), &p->cpus_mask));
- goto out;
- }
-
+ if (dest_cpu < 0)
dest_cpu = cpumask_any_distribute(&p->cpus_mask);
- }
if (task_on_rq_queued(p))
rq = __migrate_task(rq, &rf, p, dest_cpu);
else
p->wake_cpu = dest_cpu;
- } else if (dest_cpu < 0 || pending) {
+ } else if (pending) {
/*
* This happens when we get migrated between migrate_enable()'s
* preempt_enable() and scheduling the stopper task. At that
@@ -1989,23 +1959,14 @@ static int migration_cpu_stop(void *data)
* ->pi_lock, so the allowed mask is stable - if it got
* somewhere allowed, we're done.
*/
- if (pending && cpumask_test_cpu(task_cpu(p), p->cpus_ptr)) {
- p->migration_pending = NULL;
+ if (cpumask_test_cpu(task_cpu(p), p->cpus_ptr)) {
+ if (p->migration_pending == pending)
+ p->migration_pending = NULL;
complete = true;
goto out;
}
/*
- * When this was migrate_enable() but we no longer have an
- * @pending, a concurrent SCA 'fixed' things and we should be
- * valid again. Nothing to do.
- */
- if (!pending) {
- WARN_ON_ONCE(!cpumask_test_cpu(task_cpu(p), &p->cpus_mask));
- goto out;
- }
-
- /*
* When migrate_enable() hits a rq mis-match we can't reliably
* determine is_migration_disabled() and so have to chase after
* it.
@@ -2022,7 +1983,6 @@ out:
complete_all(&pending->done);
/* For pending->{arg,stop_work} */
- pending = arg->pending;
if (pending && refcount_dec_and_test(&pending->refs))
wake_up_var(&pending->refs);
next prev parent reply other threads:[~2021-03-01 10:19 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-24 12:24 [PATCH 0/6] sched: Fix affine_move_task() wreckage Peter Zijlstra
2021-02-24 12:24 ` [PATCH 1/6] sched: Fix migration_cpu_stop() requeueing Peter Zijlstra
2021-03-01 10:16 ` [tip: sched/urgent] " tip-bot2 for Peter Zijlstra
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-24 12:24 ` [PATCH 2/6] sched: Simplify migration_cpu_stop() Peter Zijlstra
2021-02-24 15:34 ` Valentin Schneider
2021-02-25 8:45 ` Peter Zijlstra
2021-02-25 11:10 ` Valentin Schneider
2021-03-01 10:16 ` tip-bot2 for Peter Zijlstra [this message]
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-24 12:24 ` [PATCH 3/6] sched: Collate affine_move_task() stoppers Peter Zijlstra
2021-03-01 10:16 ` [tip: sched/urgent] " tip-bot2 for Peter Zijlstra
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-24 12:24 ` [PATCH 4/6] sched: Optimize migration_cpu_stop() Peter Zijlstra
2021-03-01 10:16 ` [tip: sched/urgent] " tip-bot2 for Peter Zijlstra
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-24 12:24 ` [PATCH 5/6] sched: Fix affine_move_task() self-concurrency Peter Zijlstra
2021-03-01 10:16 ` [tip: sched/urgent] " tip-bot2 for Peter Zijlstra
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-02-24 12:24 ` [PATCH 6/6] sched: Simplify set_affinity_pending refcounts Peter Zijlstra
2021-02-24 15:34 ` Valentin Schneider
2021-02-24 15:34 ` Peter Zijlstra
2021-02-24 17:59 ` Valentin Schneider
2021-02-25 9:27 ` Peter Zijlstra
2021-02-25 11:11 ` Valentin Schneider
2021-03-01 10:16 ` [tip: sched/urgent] " tip-bot2 for Peter Zijlstra
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=161459377764.20312.14971720353963713012.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=stable@kernel.org \
--cc=valentin.schneider@arm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).