From: Valentin Schneider <valentin.schneider@arm.com>
To: Peter Zijlstra <peterz@infradead.org>, Qais Yousef <qais.yousef@arm.com>
Cc: tglx@linutronix.de, mingo@kernel.org,
linux-kernel@vger.kernel.org, bigeasy@linutronix.de,
swood@redhat.com, juri.lelli@redhat.com,
vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
bristot@redhat.com, vincent.donnefort@arm.com, tj@kernel.org,
ouwen210@hotmail.com
Subject: Re: [PATCH v4 15/19] sched: Fix migrate_disable() vs rt/dl balancing
Date: Fri, 05 Mar 2021 15:41:46 +0000 [thread overview]
Message-ID: <jhjblbx7glh.mognet@arm.com> (raw)
In-Reply-To: <YEJGq3aKM9lfYked@hirez.programming.kicks-ass.net>
On 05/03/21 15:56, Peter Zijlstra wrote:
> On Sat, Dec 26, 2020 at 01:54:45PM +0000, Qais Yousef wrote:
>>
>> > +static inline struct task_struct *get_push_task(struct rq *rq)
>> > +{
>> > + struct task_struct *p = rq->curr;
>>
>> Shouldn't we verify the class of the task here? The RT task in migration
>> disabled could have been preempted by a dl or stopper task. Similarly, the dl
>> task could have been preempted by a stopper task.
>>
>> I don't think an RT task should be allowed to push a dl task under any
>> circumstances?
>
> Hmm, quite. Fancy doing a patch?
Last time we talked about this, I looked into
push_rt_task() + find_lowest_rq()
IIRC, with how
find_lowest_rq() + cpupri_find_fitness()
currently work, find_lowest_rq() should return -1 in push_rt_task() if
rq->curr is DL (CPUPRI_INVALID). IOW, Migration-Disabled RT tasks shouldn't
actually interfere with DL tasks (unless a DL task gets scheduled after we
drop the rq lock and kick the stopper, but we have that problem everywhere
including CFS active balance).
Now, for some blabbering. Re SMP invariant; wouldn't we actually want this
to happen? Consider:
MD := Migration-Disabled.
rq
DL
RT3
RT2 (MD) RT1
current DL RT1 idle
CPU0 CPU1 CPU2
If we were to ignore MD, the best spread for this would be something
like:
rq
RT1
DL RT3 RT2
current DL RT3 RT2
CPU0 CPU1 CPU2
Now, with Migration-Disabled we can't move RT2 to CPU2 - it has to stay
on CPU0 for as long as it is Migration-Disabled. Thus, a possible spread
would be:
rq
RT1
RT2 (MD) DL RT3
current RT2 DL RT3
CPU0 CPU1 CPU
If you look closely, this is exactly the same as the previous spread
modulo CPU numbers. IOW, this is (again) a CPU renumbering exercise.
To respect the aforementioned scheduling invariant, we've had to move
that DL task, and while it does add interference, it's similar as to why we
push higher RT priority tasks to make room for lower RT priority, migration
disabled tasks. You get interference caused by a lower-priority entity for
the sake of your SMP scheduling invariant.
next prev parent reply other threads:[~2021-03-05 15:42 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-23 10:11 [PATCH v4 00/19] sched: Migrate disable support Peter Zijlstra
2020-10-23 10:11 ` [PATCH v4 01/19] stop_machine: Add function and caller debug info Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 02/19] sched: Fix balance_callback() Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-11-11 20:30 ` Paul Bolle
2020-11-11 20:45 ` Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 03/19] sched/hotplug: Ensure only per-cpu kthreads run during hotplug Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 04/19] sched/core: Wait for tasks being pushed away on hotplug Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Thomas Gleixner
2020-10-23 10:12 ` [PATCH v4 05/19] workqueue: Manually break affinity " Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 06/19] sched/hotplug: Consolidate task migration on CPU unplug Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Thomas Gleixner
2020-10-23 10:12 ` [PATCH v4 07/19] sched: Fix hotplug vs CPU bandwidth control Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 08/19] sched: Massage set_cpus_allowed() Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 09/19] sched: Add migrate_disable() Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 10/19] sched: Fix migrate_disable() vs set_cpus_allowed_ptr() Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-11-12 16:38 ` [PATCH v4 10/19] " Qian Cai
2020-11-12 17:26 ` Valentin Schneider
2020-11-12 18:01 ` Qian Cai
2020-11-12 19:31 ` Valentin Schneider
2020-11-12 19:41 ` Qian Cai
2020-11-12 20:37 ` Qian Cai
2020-11-12 21:26 ` Valentin Schneider
2020-11-13 10:27 ` Peter Zijlstra
2020-11-12 18:35 ` Qian Cai
2020-11-20 12:34 ` [tip: sched/core] sched/core: Add missing completion for affine_move_task() waiters tip-bot2 for Valentin Schneider
2020-10-23 10:12 ` [PATCH v4 11/19] sched/core: Make migrate disable and CPU hotplug cooperative Peter Zijlstra
2020-10-29 16:27 ` Valentin Schneider
2020-10-29 17:34 ` Peter Zijlstra
2020-10-29 17:55 ` Valentin Schneider
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Thomas Gleixner
2020-11-13 15:06 ` [PATCH v4 11/19] " Qian Cai
2020-11-17 19:28 ` Valentin Schneider
2020-11-18 14:44 ` Qian Cai
2020-11-23 18:13 ` Sebastian Andrzej Siewior
2020-12-02 21:59 ` Qian Cai
2020-12-03 12:31 ` Qian Cai
2020-12-04 0:23 ` Qian Cai
2020-12-04 21:19 ` Qian Cai
2020-12-05 18:37 ` Valentin Schneider
2020-12-06 1:17 ` Qian Cai
2020-12-07 19:27 ` Valentin Schneider
2020-12-08 13:46 ` Qian Cai
2020-12-09 19:16 ` Valentin Schneider
2020-10-23 10:12 ` [PATCH v4 12/19] sched,rt: Use cpumask_any*_distribute() Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 13/19] sched,rt: Use the full cpumask for balancing Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 14/19] sched, lockdep: Annotate ->pi_lock recursion Peter Zijlstra
2020-10-29 16:27 ` Valentin Schneider
2020-10-29 17:38 ` Peter Zijlstra
2020-10-29 18:09 ` Valentin Schneider
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 15/19] sched: Fix migrate_disable() vs rt/dl balancing Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-12-26 13:54 ` [PATCH v4 15/19] " Qais Yousef
2021-03-05 14:56 ` Peter Zijlstra
2021-03-05 15:41 ` Valentin Schneider [this message]
2021-03-05 17:11 ` Qais Yousef
2021-03-10 14:44 ` Qais Yousef
2021-03-05 16:48 ` Qais Yousef
2020-10-23 10:12 ` [PATCH v4 16/19] sched/proc: Print accurate cpumask vs migrate_disable() Peter Zijlstra
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 17/19] sched: Add migrate_disable() tracepoints Peter Zijlstra
2020-10-29 16:27 ` Valentin Schneider
2020-10-29 17:43 ` Peter Zijlstra
2020-10-29 17:56 ` Valentin Schneider
2020-10-29 17:59 ` Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 18/19] sched: Deny self-issued __set_cpus_allowed_ptr() when migrate_disable() Peter Zijlstra
2020-10-23 10:12 ` [PATCH v4 19/19] sched: Comment affine_move_task() Peter Zijlstra
2020-10-29 16:27 ` Valentin Schneider
2020-10-29 17:44 ` Peter Zijlstra
2020-10-29 19:03 ` [PATCH v4 00/19] sched: Migrate disable support Valentin Schneider
2020-11-09 16:39 ` Daniel Bristot de Oliveira
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=jhjblbx7glh.mognet@arm.com \
--to=valentin.schneider@arm.com \
--cc=bigeasy@linutronix.de \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=ouwen210@hotmail.com \
--cc=peterz@infradead.org \
--cc=qais.yousef@arm.com \
--cc=rostedt@goodmis.org \
--cc=swood@redhat.com \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=vincent.donnefort@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.