From: Aaron Lu <aaron.lu@linux.alibaba.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: mingo@kernel.org, tglx@linutronix.de, pjt@google.com,
tim.c.chen@linux.intel.com, torvalds@linux-foundation.org,
linux-kernel@vger.kernel.org, subhra.mazumdar@oracle.com,
fweisbec@gmail.com, keescook@chromium.org, kerrnel@google.com,
Aubrey Li <aubrey.intel@gmail.com>
Subject: Re: [RFC][PATCH 13/16] sched: Add core wide task selection and scheduling.
Date: Tue, 16 Apr 2019 21:43:51 +0800 [thread overview]
Message-ID: <20190416134350.GA66092@aaronlu> (raw)
In-Reply-To: <20190402082812.GJ12232@hirez.programming.kicks-ass.net>
On Tue, Apr 02, 2019 at 10:28:12AM +0200, Peter Zijlstra wrote:
> On Tue, Apr 02, 2019 at 02:46:13PM +0800, Aaron Lu wrote:
...
> > Perhaps we can test if max is on the same cpu as class_pick and then
> > use cpu_prio_less() or core_prio_less() accordingly here, or just
> > replace core_prio_less(max, p) with cpu_prio_less(max, p) in
> > pick_next_task(). The 2nd obviously breaks the comment of
> > core_prio_less() though: /* cannot compare vruntime across CPUs */.
>
> Right, so as the comment states, you cannot directly compare vruntime
> across CPUs, doing that is completely buggered.
>
> That also means that the cpu_prio_less(max, class_pick) in pick_task()
> is buggered, because there is no saying @max is on this CPU to begin
> with.
I find it difficult to decide which task of fair_sched_class having
higher priority when the two tasks belong to different CPUs.
Please see below.
> Another approach would be something like the below:
>
>
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -87,7 +87,7 @@ static inline int __task_prio(struct tas
> */
>
> /* real prio, less is less */
> -static inline bool __prio_less(struct task_struct *a, struct task_struct *b, bool runtime)
> +static inline bool __prio_less(struct task_struct *a, struct task_struct *b, u64 vruntime)
> {
> int pa = __task_prio(a), pb = __task_prio(b);
>
> @@ -104,21 +104,25 @@ static inline bool __prio_less(struct ta
> if (pa == -1) /* dl_prio() doesn't work because of stop_class above */
> return !dl_time_before(a->dl.deadline, b->dl.deadline);
>
> - if (pa == MAX_RT_PRIO + MAX_NICE && runtime) /* fair */
> - return !((s64)(a->se.vruntime - b->se.vruntime) < 0);
> + if (pa == MAX_RT_PRIO + MAX_NICE) /* fair */
> + return !((s64)(a->se.vruntime - vruntime) < 0);
>
> return false;
> }
>
> static inline bool cpu_prio_less(struct task_struct *a, struct task_struct *b)
> {
> - return __prio_less(a, b, true);
> + return __prio_less(a, b, b->se.vruntime);
> }
>
> static inline bool core_prio_less(struct task_struct *a, struct task_struct *b)
> {
> - /* cannot compare vruntime across CPUs */
> - return __prio_less(a, b, false);
> + u64 vruntime = b->se.vruntime;
> +
> + vruntime -= task_rq(b)->cfs.min_vruntime;
> + vruntime += task_rq(a)->cfs.min_vruntime
(I used task_cfs_rq() instead of task_rq() above.)
Consider the following scenario:
(assume cpu0 and cpu1 are siblings of core0)
1 a cpu-intensive task belonging to cgroupA running on cpu0;
2 launch 'ls' from a shell(bash) which belongs to cgroupB;
3 'ls' blocked for a long time(if not forever).
Per my limited understanding: the launch of 'ls' cause bash to fork,
then the newly forked process' vruntime will be 6ms(probably not
precise) ahead of its cfs_rq due to START_DEBIT. Since there is no other
running task on that cfs_rq, the cfs_rq's min_vruntime doesn't have a
chance to get updated and the newly forked process will always have a
distance of 6ms compared to its cfs_rq and it will always 'lose' to the
cpu-intensive task belonging to cgroupA by core_prio_less().
No idea how to solve this...
> +
> + return __prio_less(a, b, vruntime);
> }
>
> static inline bool __sched_core_less(struct task_struct *a, struct task_struct *b)
next prev parent reply other threads:[~2019-04-16 13:44 UTC|newest]
Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-18 16:56 [RFC][PATCH 00/16] sched: Core scheduling Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 01/16] stop_machine: Fix stop_cpus_in_progress ordering Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 02/16] sched: Fix kerneldoc comment for ia64_set_curr_task Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 03/16] sched: Wrap rq::lock access Peter Zijlstra
2019-02-19 16:13 ` Phil Auld
2019-02-19 16:22 ` Peter Zijlstra
2019-02-19 16:37 ` Phil Auld
2019-03-18 15:41 ` Julien Desfossez
2019-03-20 2:29 ` Subhra Mazumdar
2019-03-21 21:20 ` Julien Desfossez
2019-03-22 13:34 ` Peter Zijlstra
2019-03-22 20:59 ` Julien Desfossez
2019-03-23 0:06 ` Subhra Mazumdar
2019-03-27 1:02 ` Subhra Mazumdar
2019-03-29 13:35 ` Julien Desfossez
2019-03-29 22:23 ` Subhra Mazumdar
2019-04-01 21:35 ` Subhra Mazumdar
2019-04-03 20:16 ` Julien Desfossez
2019-04-05 1:30 ` Subhra Mazumdar
2019-04-02 7:42 ` Peter Zijlstra
2019-03-22 23:28 ` Tim Chen
2019-03-22 23:44 ` Tim Chen
2019-02-18 16:56 ` [RFC][PATCH 04/16] sched/{rt,deadline}: Fix set_next_task vs pick_next_task Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 05/16] sched: Add task_struct pointer to sched_class::set_curr_task Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 06/16] sched/fair: Export newidle_balance() Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 07/16] sched: Allow put_prev_task() to drop rq->lock Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 08/16] sched: Rework pick_next_task() slow-path Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 09/16] sched: Introduce sched_class::pick_task() Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 10/16] sched: Core-wide rq->lock Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 11/16] sched: Basic tracking of matching tasks Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 12/16] sched: A quick and dirty cgroup tagging interface Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 13/16] sched: Add core wide task selection and scheduling Peter Zijlstra
[not found] ` <20190402064612.GA46500@aaronlu>
2019-04-02 8:28 ` Peter Zijlstra
2019-04-02 13:20 ` Aaron Lu
2019-04-05 14:55 ` Aaron Lu
2019-04-09 18:09 ` Tim Chen
2019-04-10 4:36 ` Aaron Lu
2019-04-10 14:18 ` Aubrey Li
2019-04-11 2:11 ` Aaron Lu
2019-04-10 14:44 ` Peter Zijlstra
2019-04-11 3:05 ` Aaron Lu
2019-04-11 9:19 ` Peter Zijlstra
2019-04-10 8:06 ` Peter Zijlstra
2019-04-10 19:58 ` Vineeth Remanan Pillai
2019-04-15 16:59 ` Julien Desfossez
2019-04-16 13:43 ` Aaron Lu [this message]
2019-04-09 18:38 ` Julien Desfossez
2019-04-10 15:01 ` Peter Zijlstra
2019-04-11 0:11 ` Subhra Mazumdar
2019-04-19 8:40 ` Ingo Molnar
2019-04-19 23:16 ` Subhra Mazumdar
2019-02-18 16:56 ` [RFC][PATCH 14/16] sched/fair: Add a few assertions Peter Zijlstra
2019-02-18 16:56 ` [RFC][PATCH 15/16] sched: Trivial forced-newidle balancer Peter Zijlstra
2019-02-21 16:19 ` Valentin Schneider
2019-02-21 16:41 ` Peter Zijlstra
2019-02-21 16:47 ` Peter Zijlstra
2019-02-21 18:28 ` Valentin Schneider
2019-04-04 8:31 ` Aubrey Li
2019-04-06 1:36 ` Aubrey Li
2019-02-18 16:56 ` [RFC][PATCH 16/16] sched: Debug bits Peter Zijlstra
2019-02-18 17:49 ` [RFC][PATCH 00/16] sched: Core scheduling Linus Torvalds
2019-02-18 20:40 ` Peter Zijlstra
2019-02-19 0:29 ` Linus Torvalds
2019-02-19 15:15 ` Ingo Molnar
2019-02-22 12:17 ` Paolo Bonzini
2019-02-22 14:20 ` Peter Zijlstra
2019-02-22 19:26 ` Tim Chen
2019-02-26 8:26 ` Aubrey Li
2019-02-27 7:54 ` Aubrey Li
2019-02-21 2:53 ` Subhra Mazumdar
2019-02-21 14:03 ` Peter Zijlstra
2019-02-21 18:44 ` Subhra Mazumdar
2019-02-22 0:34 ` Subhra Mazumdar
2019-02-22 12:45 ` Mel Gorman
2019-02-22 16:10 ` Mel Gorman
2019-03-08 19:44 ` Subhra Mazumdar
2019-03-11 4:23 ` Aubrey Li
2019-03-11 18:34 ` Subhra Mazumdar
2019-03-11 23:33 ` Subhra Mazumdar
2019-03-12 0:20 ` Greg Kerr
2019-03-12 0:47 ` Subhra Mazumdar
2019-03-12 7:33 ` Aaron Lu
2019-03-12 7:45 ` Aubrey Li
2019-03-13 5:55 ` Aubrey Li
2019-03-14 0:35 ` Tim Chen
2019-03-14 5:30 ` Aubrey Li
2019-03-14 6:07 ` Li, Aubrey
2019-03-18 6:56 ` Aubrey Li
2019-03-12 19:07 ` Pawan Gupta
2019-03-26 7:32 ` Aaron Lu
2019-03-26 7:56 ` Aaron Lu
2019-02-19 22:07 ` Greg Kerr
2019-02-20 9:42 ` Peter Zijlstra
2019-02-20 18:33 ` Greg Kerr
2019-02-22 14:10 ` Peter Zijlstra
2019-03-07 22:06 ` Paolo Bonzini
2019-02-20 18:43 ` Subhra Mazumdar
2019-03-01 2:54 ` Subhra Mazumdar
2019-03-14 15:28 ` Julien Desfossez
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190416134350.GA66092@aaronlu \
--to=aaron.lu@linux.alibaba.com \
--cc=aubrey.intel@gmail.com \
--cc=fweisbec@gmail.com \
--cc=keescook@chromium.org \
--cc=kerrnel@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=subhra.mazumdar@oracle.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).