From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Qais Yousef <qais.yousef@arm.com>
Cc: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Steven Rostedt <rostedt@goodmis.org>,
Pavan Kondeti <pkondeti@codeaurora.org>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/3] sched/rt: cpupri_find: implement fallback mechanism for !fit case
Date: Tue, 18 Feb 2020 10:53:55 +0100 [thread overview]
Message-ID: <50eee4ae-a733-d8e4-9f57-ab05678545fc@arm.com> (raw)
In-Reply-To: <20200217234549.rpv3ns7bd7l6twqu@e107158-lin>
On 18/02/2020 00:45, Qais Yousef wrote:
> On 02/17/20 20:09, Dietmar Eggemann wrote:
>> On 14/02/2020 17:39, Qais Yousef wrote:
>>
>> [...]
>>
>>> /**
>>> * cpupri_find - find the best (lowest-pri) CPU in the system
>>> * @cp: The cpupri context
>>> @@ -62,80 +115,72 @@ int cpupri_find(struct cpupri *cp, struct task_struct *p,
>>> struct cpumask *lowest_mask,
>>> bool (*fitness_fn)(struct task_struct *p, int cpu))
>>> {
>>> - int idx = 0;
>>> int task_pri = convert_prio(p->prio);
>>> + int best_unfit_idx = -1;
>>> + int idx = 0, cpu;
>>>
>>> BUG_ON(task_pri >= CPUPRI_NR_PRIORITIES);
>>>
>>> for (idx = 0; idx < task_pri; idx++) {
>>> - struct cpupri_vec *vec = &cp->pri_to_cpu[idx];
>>> - int skip = 0;
>>>
>>> - if (!atomic_read(&(vec)->count))
>>> - skip = 1;
>>> - /*
>>> - * When looking at the vector, we need to read the counter,
>>> - * do a memory barrier, then read the mask.
>>> - *
>>> - * Note: This is still all racey, but we can deal with it.
>>> - * Ideally, we only want to look at masks that are set.
>>> - *
>>> - * If a mask is not set, then the only thing wrong is that we
>>> - * did a little more work than necessary.
>>> - *
>>> - * If we read a zero count but the mask is set, because of the
>>> - * memory barriers, that can only happen when the highest prio
>>> - * task for a run queue has left the run queue, in which case,
>>> - * it will be followed by a pull. If the task we are processing
>>> - * fails to find a proper place to go, that pull request will
>>> - * pull this task if the run queue is running at a lower
>>> - * priority.
>>> - */
>>> - smp_rmb();
>>> -
>>> - /* Need to do the rmb for every iteration */
>>> - if (skip)
>>> - continue;
>>> -
>>> - if (cpumask_any_and(p->cpus_ptr, vec->mask) >= nr_cpu_ids)
>>> + if (!__cpupri_find(cp, p, lowest_mask, idx))
>>> continue;
>>>
>>> - if (lowest_mask) {
>>> - int cpu;
>>
>> Shouldn't we add an extra condition here?
>>
>> + if (!static_branch_unlikely(&sched_asym_cpucapacity))
>> + return 1;
>> +
>>
>> Otherwise non-heterogeneous systems have to got through this
>> for_each_cpu(cpu, lowest_mask) further below for no good reason.
>
> Hmm below is the best solution I can think of at the moment. Works for you?
>
> It's independent of what this patch tries to fix, so I'll add as a separate
> patch to the series in the next update.
OK.
Since we can't set it as early as init_sched_rt_class()
root@juno:~# dmesg | grep "\*\*\*"
[ 0.501697] *** set sched_asym_cpucapacity <-- CPU cap asym by uArch
[ 0.505847] *** init_sched_rt_class()
[ 1.796706] *** set sched_asym_cpucapacity <-- CPUfreq kicked in
we probably have to do it either by bailing out of cpupri_find() early
with this extra condition (above) or by initializing the func pointer
dynamically (your example).
[...]
> @@ -1708,6 +1710,7 @@ static int find_lowest_rq(struct task_struct *task)
> struct cpumask *lowest_mask = this_cpu_cpumask_var_ptr(local_cpu_mask);
> int this_cpu = smp_processor_id();
> int cpu = task_cpu(task);
> + fitness_fn_t fitness_fn;
>
> /* Make sure the mask is initialized first */
> if (unlikely(!lowest_mask))
> @@ -1716,8 +1719,17 @@ static int find_lowest_rq(struct task_struct *task)
> if (task->nr_cpus_allowed == 1)
> return -1; /* No other targets possible */
>
> + /*
> + * Help cpupri_find avoid the cost of looking for a fitting CPU when
> + * not really needed.
> + */
In case the commend is really needed, for me it would work better
logically inverse.
[...]
next prev parent reply other threads:[~2020-02-18 9:54 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-14 16:39 [PATCH 0/3] RT Capacity Awareness Improvements Qais Yousef
2020-02-14 16:39 ` [PATCH 1/3] sched/rt: cpupri_find: implement fallback mechanism for !fit case Qais Yousef
2020-02-17 17:07 ` Valentin Schneider
2020-02-17 23:34 ` Qais Yousef
2020-02-18 10:01 ` Valentin Schneider
2020-02-17 19:09 ` Dietmar Eggemann
2020-02-17 23:45 ` Qais Yousef
2020-02-18 9:53 ` Dietmar Eggemann [this message]
2020-02-18 17:28 ` Qais Yousef
2020-02-18 16:46 ` Steven Rostedt
2020-02-18 17:27 ` Qais Yousef
2020-02-18 18:03 ` Steven Rostedt
2020-02-18 18:52 ` Qais Yousef
2020-02-14 16:39 ` [PATCH 2/3] sched/rt: allow pulling unfitting task Qais Yousef
2020-02-17 9:10 ` Pavan Kondeti
2020-02-17 11:20 ` Qais Yousef
2020-02-19 13:43 ` Qais Yousef
2020-02-21 8:07 ` Pavan Kondeti
2020-02-21 11:08 ` Qais Yousef
2020-02-14 16:39 ` [PATCH 3/3] sched/rt: fix pushing unfit tasks to a better CPU Qais Yousef
2020-02-17 9:23 ` Pavan Kondeti
2020-02-17 13:53 ` Qais Yousef
2020-02-18 4:16 ` Pavan Kondeti
2020-02-18 17:47 ` Qais Yousef
2020-02-19 2:46 ` Pavan Kondeti
2020-02-19 10:46 ` Qais Yousef
2020-02-19 14:02 ` Qais Yousef
2020-02-21 8:15 ` Pavan Kondeti
2020-02-21 11:12 ` Qais Yousef
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50eee4ae-a733-d8e4-9f57-ab05678545fc@arm.com \
--to=dietmar.eggemann@arm.com \
--cc=bsegall@google.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=pkondeti@codeaurora.org \
--cc=qais.yousef@arm.com \
--cc=rostedt@goodmis.org \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).