From: Vincent Guittot <vincent.guittot@linaro.org>
To: Valentin Schneider <valentin.schneider@arm.com>
Cc: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Juri Lelli <juri.lelli@redhat.com>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
linux-kernel <linux-kernel@vger.kernel.org>,
Morten Rasmussen <morten.rasmussen@arm.com>,
Tao Zhou <ouwen210@hotmail.com>
Subject: Re: [PATCH v2] sched/fair: prefer prev cpu in asymmetric wakeup path
Date: Thu, 29 Oct 2020 14:58:22 +0100 [thread overview]
Message-ID: <CAKfTPtDNzMa_vEr69eUXQBoc_5M978w=m+nykVG40gamz0YBBw@mail.gmail.com> (raw)
In-Reply-To: <jhjk0v9p9ic.mognet@arm.com>
On Thu, 29 Oct 2020 at 12:16, Valentin Schneider
<valentin.schneider@arm.com> wrote:
>
>
> Hi Vincent,
>
> On 28/10/20 17:44, Vincent Guittot wrote:
> > During fast wakeup path, scheduler always check whether local or prev cpus
> > are good candidates for the task before looking for other cpus in the
> > domain. With
> > commit b7a331615d25 ("sched/fair: Add asymmetric CPU capacity wakeup scan")
> > the heterogenous system gains a dedicated path but doesn't try to reuse
> > prev cpu whenever possible. If the previous cpu is idle and belong to the
> > LLC domain, we should check it 1st before looking for another cpu because
> > it stays one of the best candidate and this also stabilizes task placement
> > on the system.
> >
> > This change aligns asymmetric path behavior with symmetric one and reduces
> > cases where the task migrates across all cpus of the sd_asym_cpucapacity
> > domains at wakeup.
> >
> > This change does not impact normal EAS mode but only the overloaded case or
> > when EAS is not used.
> >
> > - On hikey960 with performance governor (EAS disable)
> >
> > ./perf bench sched pipe -T -l 50000
> > mainline w/ patch
> > # migrations 999364 0
> > ops/sec 149313(+/-0.28%) 182587(+/- 0.40) +22%
> >
> > - On hikey with performance governor
> >
> > ./perf bench sched pipe -T -l 50000
> > mainline w/ patch
> > # migrations 0 0
> > ops/sec 47721(+/-0.76%) 47899(+/- 0.56) +0.4%
> >
> > According to test on hikey, the patch doesn't impact symmetric system
> > compared to current implementation (only tested on arm64)
> >
> > Also read the uclamped value of task's utilization at most twice instead
> > instead each time we compare task's utilization with cpu's capacity.
> >
> > Fixes: b7a331615d25 ("sched/fair: Add asymmetric CPU capacity wakeup scan")
> > Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
>
> Other than the below, I quite like this!
>
> > ---
> > Changes in v2:
> > - merge asymmetric and symmetric path instead of duplicating tests on target,
> > prev and other special cases.
> >
> > - factorize call to uclamp_task_util(p) and use fits_capacity(). This could
> > explain part of the additionnal improvement compared to v1 (+22% instead of
> > +17% on v1).
> >
> > - Keep using LLC instead of asym domain for early check of target, prev and
> > recent_used_cpu to ensure cache sharing between the task. This doesn't
> > change anything for dynamiQ but will ensure same cache for legacy big.LITTLE
> > and also simply the changes.
> >
>
> On legacy big.LITTLE systems, sd_asym_cpucapacity spans all CPUs, so we
> would iterate over those in select_idle_capacity() anyway - the policy
> we've been going for is that capacity fitness trumps cache use.
I agree on that but I haven't been able to convince myself that adding
the complexity for the 3 shortcuts (target, prev and recent_used) will
give any benefit.
For example, the problem that I raised with perf bench sched pipe, is
solved on legacy bg.LITTLE with this version because the 2 threads
ends up on the same cache domain
>
> This does require the system to have a decent interconnect, cache snooping
> & co, but that is IMO a requirement of any sane asymmetric system.
>
> To put words into code, this is the kind of check I would see:
>
> if (static_branch_unlikely(&sched_asym_cpucapacity))
> return fits_capacity(task_util, capacity_of(cpu));
> else
> return cpus_share_cache(cpu, other);
>
> > - don't check capacity for the per-cpu kthread UC because the assumption is
> > that the wakee queued work for the per-cpu kthread that is now complete and
> > the task was already on this cpu.
> >
> > - On an asymmetric system where an exclusive cpuset defines a symmetric island,
> > task's load is synced and tested although it's not needed. But taking care of
> > this special case by testing if sd_asym_cpucapacity is not null impacts by
> > more than 4% the performance of default sched_asym_cpucapacity path.
> >
> > - The huge increase of the number of migration for hikey960 mainline comes from
> > teh fact that the ftrace buffer was overloaded by events in the tests done
> > with v1.
next prev parent reply other threads:[~2020-10-29 13:59 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-28 17:44 [PATCH v2] sched/fair: prefer prev cpu in asymmetric wakeup path Vincent Guittot
2020-10-29 11:16 ` Valentin Schneider
2020-10-29 13:58 ` Vincent Guittot [this message]
2020-10-29 14:19 ` Vincent Guittot
2020-10-29 14:30 ` Vincent Guittot
2020-10-29 14:33 ` Vincent Guittot
2020-10-29 14:45 ` Valentin Schneider
[not found] ` <BN8PR12MB2978D627EE0D6456DC2EEA6B9A140@BN8PR12MB2978.namprd12.prod.outlook.com>
2020-10-29 12:24 ` Valentin Schneider
2020-10-29 13:49 ` Vincent Guittot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAKfTPtDNzMa_vEr69eUXQBoc_5M978w=m+nykVG40gamz0YBBw@mail.gmail.com' \
--to=vincent.guittot@linaro.org \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=ouwen210@hotmail.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=valentin.schneider@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).