From: "tip-bot2 for Valentin Schneider" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Valentin Schneider <valentin.schneider@arm.com>,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: sched/core] sched: Comment affine_move_task()
Date: Wed, 11 Nov 2020 08:23:14 -0000 [thread overview]
Message-ID: <160508299477.11244.9464017467265290417.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20201013140116.26651-2-valentin.schneider@arm.com>
The following commit has been merged into the sched/core branch of tip:
Commit-ID: c777d847107e80df24dae87fc9cf4b4c0bf4dfed
Gitweb: https://git.kernel.org/tip/c777d847107e80df24dae87fc9cf4b4c0bf4dfed
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Tue, 13 Oct 2020 15:01:16 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Tue, 10 Nov 2020 18:39:02 +01:00
sched: Comment affine_move_task()
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20201013140116.26651-2-valentin.schneider@arm.com
---
kernel/sched/core.c | 81 ++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 79 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 88c6fcb..c6409f3 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2076,7 +2076,75 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask)
}
/*
- * This function is wildly self concurrent, consider at least 3 times.
+ * This function is wildly self concurrent; here be dragons.
+ *
+ *
+ * When given a valid mask, __set_cpus_allowed_ptr() must block until the
+ * designated task is enqueued on an allowed CPU. If that task is currently
+ * running, we have to kick it out using the CPU stopper.
+ *
+ * Migrate-Disable comes along and tramples all over our nice sandcastle.
+ * Consider:
+ *
+ * Initial conditions: P0->cpus_mask = [0, 1]
+ *
+ * P0@CPU0 P1
+ *
+ * migrate_disable();
+ * <preempted>
+ * set_cpus_allowed_ptr(P0, [1]);
+ *
+ * P1 *cannot* return from this set_cpus_allowed_ptr() call until P0 executes
+ * its outermost migrate_enable() (i.e. it exits its Migrate-Disable region).
+ * This means we need the following scheme:
+ *
+ * P0@CPU0 P1
+ *
+ * migrate_disable();
+ * <preempted>
+ * set_cpus_allowed_ptr(P0, [1]);
+ * <blocks>
+ * <resumes>
+ * migrate_enable();
+ * __set_cpus_allowed_ptr();
+ * <wakes local stopper>
+ * `--> <woken on migration completion>
+ *
+ * Now the fun stuff: there may be several P1-like tasks, i.e. multiple
+ * concurrent set_cpus_allowed_ptr(P0, [*]) calls. CPU affinity changes of any
+ * task p are serialized by p->pi_lock, which we can leverage: the one that
+ * should come into effect at the end of the Migrate-Disable region is the last
+ * one. This means we only need to track a single cpumask (i.e. p->cpus_mask),
+ * but we still need to properly signal those waiting tasks at the appropriate
+ * moment.
+ *
+ * This is implemented using struct set_affinity_pending. The first
+ * __set_cpus_allowed_ptr() caller within a given Migrate-Disable region will
+ * setup an instance of that struct and install it on the targeted task_struct.
+ * Any and all further callers will reuse that instance. Those then wait for
+ * a completion signaled at the tail of the CPU stopper callback (1), triggered
+ * on the end of the Migrate-Disable region (i.e. outermost migrate_enable()).
+ *
+ *
+ * (1) In the cases covered above. There is one more where the completion is
+ * signaled within affine_move_task() itself: when a subsequent affinity request
+ * cancels the need for an active migration. Consider:
+ *
+ * Initial conditions: P0->cpus_mask = [0, 1]
+ *
+ * P0@CPU0 P1 P2
+ *
+ * migrate_disable();
+ * <preempted>
+ * set_cpus_allowed_ptr(P0, [1]);
+ * <blocks>
+ * set_cpus_allowed_ptr(P0, [0, 1]);
+ * <signal completion>
+ * <awakes>
+ *
+ * Note that the above is safe vs a concurrent migrate_enable(), as any
+ * pending affinity completion is preceded by an uninstallation of
+ * p->migration_pending done with p->pi_lock held.
*/
static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
int dest_cpu, unsigned int flags)
@@ -2120,6 +2188,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
if (!(flags & SCA_MIGRATE_ENABLE)) {
/* serialized by p->pi_lock */
if (!p->migration_pending) {
+ /* Install the request */
refcount_set(&my_pending.refs, 1);
init_completion(&my_pending.done);
p->migration_pending = &my_pending;
@@ -2165,7 +2234,11 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
}
if (task_running(rq, p) || p->state == TASK_WAKING) {
-
+ /*
+ * Lessen races (and headaches) by delegating
+ * is_migration_disabled(p) checks to the stopper, which will
+ * run on the same CPU as said p.
+ */
task_rq_unlock(rq, p, rf);
stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg);
@@ -2190,6 +2263,10 @@ do_complete:
if (refcount_dec_and_test(&pending->refs))
wake_up_var(&pending->refs);
+ /*
+ * Block the original owner of &pending until all subsequent callers
+ * have seen the completion and decremented the refcount
+ */
wait_var_event(&my_pending.refs, !refcount_read(&my_pending.refs));
return 0;
prev parent reply other threads:[~2020-11-11 8:23 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-05 14:57 [PATCH -v2 00/17] sched: Migrate disable support Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 01/17] stop_machine: Add function and caller debug info Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 02/17] sched: Fix balance_callback() Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 03/17] sched/hotplug: Ensure only per-cpu kthreads run during hotplug Peter Zijlstra
2020-10-06 7:25 ` Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 04/17] sched/core: Wait for tasks being pushed away on hotplug Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 05/17] workqueue: Manually break affinity " Peter Zijlstra
2020-10-05 16:31 ` Tejun Heo
2020-10-05 14:57 ` [PATCH -v2 06/17] sched/hotplug: Consolidate task migration on CPU unplug Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 07/17] sched: Fix hotplug vs CPU bandwidth control Peter Zijlstra
2020-10-06 12:46 ` Vincent Guittot
2020-10-06 13:33 ` Peter Zijlstra
2020-10-09 20:41 ` Dietmar Eggemann
2020-10-12 12:52 ` Peter Zijlstra
2020-10-12 13:18 ` Peter Zijlstra
2020-10-12 14:14 ` Dietmar Eggemann
2020-10-05 14:57 ` [PATCH -v2 08/17] sched: Massage set_cpus_allowed() Peter Zijlstra
2020-10-06 11:20 ` Valentin Schneider
2020-10-05 14:57 ` [PATCH -v2 09/17] sched: Add migrate_disable() Peter Zijlstra
2020-10-06 11:20 ` Valentin Schneider
2020-10-05 14:57 ` [PATCH -v2 10/17] sched: Fix migrate_disable() vs set_cpus_allowed_ptr() Peter Zijlstra
[not found] ` <CH2PR14MB41833F828B4D3BA5A7B6CE7B9A0B0@CH2PR14MB4183.namprd14.prod.outlook.com>
2020-10-12 11:36 ` Peter Zijlstra
[not found] ` <CH2PR14MB4183BF26F02A4AD4F45CADA59A070@CH2PR14MB4183.namprd14.prod.outlook.com>
2020-10-13 13:20 ` Valentin Schneider
2020-10-05 14:57 ` [PATCH -v2 11/17] sched/core: Make migrate disable and CPU hotplug cooperative Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 12/17] sched,rt: Use cpumask_any*_distribute() Peter Zijlstra
2020-10-06 14:09 ` Juri Lelli
2020-10-06 14:20 ` Peter Zijlstra
2020-10-06 15:55 ` Qais Yousef
2020-10-07 7:22 ` Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 13/17] sched,rt: Use the full cpumask for balancing Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 14/17] sched, lockdep: Annotate ->pi_lock recursion Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 15/17] sched: Fix migrate_disable() vs rt/dl balancing Peter Zijlstra
2020-10-06 7:59 ` Peter Zijlstra
2020-10-06 13:44 ` Steven Rostedt
2020-10-06 13:50 ` Peter Zijlstra
2020-10-06 11:20 ` Valentin Schneider
2020-10-06 13:48 ` Peter Zijlstra
2020-10-06 14:37 ` Juri Lelli
2020-10-06 14:48 ` Peter Zijlstra
2020-10-07 5:29 ` Juri Lelli
2020-10-06 16:19 ` Valentin Schneider
2020-10-07 19:22 ` Valentin Schneider
2020-10-07 21:08 ` Peter Zijlstra
2020-10-07 22:32 ` Valentin Schneider
2020-10-08 10:48 ` Sebastian Andrzej Siewior
2020-10-12 9:56 ` Dietmar Eggemann
2020-10-12 11:28 ` Peter Zijlstra
2020-10-12 12:22 ` Dietmar Eggemann
2020-10-05 14:57 ` [PATCH -v2 16/17] sched/proc: Print accurate cpumask vs migrate_disable() Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 17/17] sched: Add migrate_disable() tracepoints Peter Zijlstra
2020-10-13 14:01 ` [PATCH 1/2] sched: Deny self-issued __set_cpus_allowed_ptr() when migrate_disable() Valentin Schneider
2020-10-13 14:15 ` Sebastian Andrzej Siewior
2020-10-13 14:21 ` Peter Zijlstra
2020-10-15 9:14 ` Valentin Schneider
2020-10-13 14:22 ` Valentin Schneider
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-10-13 14:01 ` [PATCH 2/2] sched: Comment affine_move_task() Valentin Schneider
2020-11-11 8:23 ` tip-bot2 for Valentin Schneider [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=160508299477.11244.9464017467265290417.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=valentin.schneider@arm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).