From: Patrick Bellasi <patrick.bellasi@arm.com>
To: Quentin Perret <quentin.perret@arm.com>
Cc: peterz@infradead.org, rjw@rjwysocki.net,
linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org,
gregkh@linuxfoundation.org, mingo@redhat.com,
dietmar.eggemann@arm.com, morten.rasmussen@arm.com,
chris.redpath@arm.com, valentin.schneider@arm.com,
vincent.guittot@linaro.org, thara.gopinath@linaro.org,
viresh.kumar@linaro.org, tkjos@google.com,
joel@joelfernandes.org, smuckle@google.com,
adharmap@codeaurora.org, skannan@codeaurora.org,
pkondeti@codeaurora.org, juri.lelli@redhat.com,
edubezval@gmail.com, srinivas.pandruvada@linux.intel.com,
currojerez@riseup.net, javi.merino@kernel.org
Subject: Re: [PATCH v6 07/14] sched/topology: Introduce sched_energy_present static key
Date: Thu, 30 Aug 2018 10:23:29 +0100 [thread overview]
Message-ID: <20180830092329.GS2960@e110439-lin> (raw)
In-Reply-To: <20180829172004.afbe2oukprvptqs2@queper01-lin>
On 29-Aug 18:20, Quentin Perret wrote:
> On Wednesday 29 Aug 2018 at 17:50:58 (+0100), Patrick Bellasi wrote:
> > > +/*
> > > + * The complexity of the Energy Model is defined as: nr_pd * (nr_cpus + nr_cs)
> > > + * with: 'nr_pd' the number of performance domains; 'nr_cpus' the number of
> > > + * CPUs; and 'nr_cs' the sum of the capacity states numbers of all performance
> > > + * domains.
> > > + *
> > > + * It is generally not a good idea to use such a model in the wake-up path on
> > > + * very complex platforms because of the associated scheduling overheads. The
> > > + * arbitrary constraint below prevents that. It makes EAS usable up to 16 CPUs
> > > + * with per-CPU DVFS and less than 8 capacity states each, for example.
> >
> > According to the formula above, that should give a "complexity value" of:
> >
> > 16 * (16 + 9) = 384
> >
> > while, 2K complexity seems more like a 40xCPUs system with 8 OPPs.
> >
> > Maybe we should update either the example or the constant below ?
>
> Hmm I guess the example isn't really clear. 'nr_cs' is the _sum_ of the
> number of OPPs of all perf. domains. So, in the example above, if you
> have 16 CPUs with per-CPU DVFS, and each DVFS island has 8 OPPs, then
> nr_cs = 16 * 8 = 128.
>
> So if you apply the formula you get C = 16 * (16 + 128) = 2304, which is
> more than EM_MAX_COMPLEXITY, so EAS cannot start.
>
> If the DVFS island had 7 OPPs instead of 8 (for example) you would get
> nr_cs = 112, C = 2048, and so EAS could start.
Right, I see now.
> I can try to re-work that comment to explain things a bit better ...
Yes, dunno if it's just me but perhaps a bit of rephrasing could help.
Alternatively, why not having this comment and check after patches
11 and 12 ?
> > > + */
> > > +#define EM_MAX_COMPLEXITY 2048
> > > +
> > > static void build_perf_domains(const struct cpumask *cpu_map)
> > > {
> > > + int i, nr_pd = 0, nr_cs = 0, nr_cpus = cpumask_weight(cpu_map);
> > > struct perf_domain *pd = NULL, *tmp;
> > > int cpu = cpumask_first(cpu_map);
> > > struct root_domain *rd = cpu_rq(cpu)->rd;
> > > - int i;
> > > +
> > > + /* EAS is enabled for asymmetric CPU capacity topologies. */
> > > + if (!per_cpu(sd_asym_cpucapacity, cpu)) {
> > > + if (sched_debug()) {
> > > + pr_info("rd %*pbl: CPUs do not have asymmetric capacities\n",
> > > + cpumask_pr_args(cpu_map));
> > > + }
> > > + goto free;
> > > + }
> > >
> > > for_each_cpu(i, cpu_map) {
> > > /* Skip already covered CPUs. */
> > > @@ -288,6 +318,21 @@ static void build_perf_domains(const struct cpumask *cpu_map)
> > > goto free;
> > > tmp->next = pd;
> > > pd = tmp;
> > > +
> > > + /*
> > > + * Count performance domains and capacity states for the
> > > + * complexity check.
> > > + */
> > > + nr_pd++;
> >
> > A special case where EAS is not going to be used is for systems where
> > nr_pd matches the number of online CPUs, isn't it ?
>
> Well, it depends. Say you have only 4 CPUs with 3 OPPs each. Even with
> per-CPU DVFS the complexity is low enough to start EAS. I don't really
> see a good reason for not doing so no ?
Right... I was totally confused by the idea that we don't
run EAS if we just have 1 CPU per PD... my bad!
Although on those systems, since we don't have idle costs, should not
be a spreading strategy always the best from an energy efficiency
standpoint ?
> > If that's the case, then, by caching this nr_pd you can probably check
> > this condition in the sched_energy_start() and bail out even faster by
> > avoiding to scan all the doms_new's pd ?
> >
> >
> > > + nr_cs += em_pd_nr_cap_states(pd->obj);
> > > + }
> > > +
> > > + /* Bail out if the Energy Model complexity is too high. */
> > > + if (nr_pd * (nr_cs + nr_cpus) > EM_MAX_COMPLEXITY) {
> > > + if (sched_debug())
> > > + pr_info("rd %*pbl: EM complexity is too high\n ",
> > > + cpumask_pr_args(cpu_map));
> > > + goto free;
> > > }
> > >
> > > perf_domain_debug(cpu_map, pd);
> > > @@ -307,6 +352,35 @@ static void build_perf_domains(const struct cpumask *cpu_map)
> > > if (tmp)
> > > call_rcu(&tmp->rcu, destroy_perf_domain_rcu);
> > > }
> > > +
> > > +static void sched_energy_start(int ndoms_new, cpumask_var_t doms_new[])
> > > +{
> > > + /*
> > > + * The conditions for EAS to start are checked during the creation of
> > > + * root domains. If one of them meets all conditions, it will have a
> > > + * non-null list of performance domains.
> > > + */
> > > + while (ndoms_new) {
> > > + if (cpu_rq(cpumask_first(doms_new[ndoms_new - 1]))->rd->pd)
> > > + goto enable;
> > > + ndoms_new--;
> > > + }
> > > +
> > > + if (static_branch_unlikely(&sched_energy_present)) {
> > ^^^^^^^^
> > Is this defined unlikely to reduce overheads on systems which never
> > satisfy all the conditions above while still rebuild SDs from time to
> > time ?
>
> Something like that. I just thought that the case where EAS needs to be
> disabled after being enabled isn't very common. I mean, the most typical
> use-case is, EAS is enabled at boot and stays enabled forever, or EAS
> never gets enabled.
Right, if we have EAS compiled in... we are likely to have it enabled.
> Enabling/disabling EAS because of hotplug (for example) can definitely
> happen, but that shouldn't be the case very often in practice, I think.
Would say yes on sane platform, i.e. where hotplug is not being used
for power/thermal management... but hopefully EAS should improve on
that side ;)
> So we can optimize things out a bit I suppose.
Right thanks!
--
#include <best/regards.h>
Patrick Bellasi
next prev parent reply other threads:[~2018-08-30 9:23 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-08-20 9:44 [PATCH v6 00/14] Energy Aware Scheduling Quentin Perret
2018-08-20 9:44 ` [PATCH v6 01/14] sched: Relocate arch_scale_cpu_capacity Quentin Perret
2018-08-20 9:44 ` [PATCH v6 02/14] sched/cpufreq: Factor out utilization to frequency mapping Quentin Perret
2018-09-10 9:29 ` Rafael J. Wysocki
2018-08-20 9:44 ` [PATCH v6 03/14] PM: Introduce an Energy Model management framework Quentin Perret
2018-08-29 10:04 ` Patrick Bellasi
2018-08-29 13:28 ` Quentin Perret
2018-08-31 9:04 ` Patrick Bellasi
2018-09-11 9:34 ` Andrea Parri
2018-09-11 12:32 ` Quentin Perret
2018-09-11 13:31 ` Andrea Parri
2018-09-10 9:44 ` Rafael J. Wysocki
2018-09-10 10:38 ` Quentin Perret
2018-09-10 10:40 ` Rafael J. Wysocki
2018-08-20 9:44 ` [PATCH v6 04/14] PM / EM: Expose the Energy Model in sysfs Quentin Perret
2018-09-06 6:56 ` Dietmar Eggemann
2018-09-06 14:09 ` Quentin Perret
2018-09-07 0:14 ` Dietmar Eggemann
2018-08-20 9:44 ` [PATCH v6 05/14] sched/topology: Reference the Energy Model of CPUs when available Quentin Perret
2018-08-29 16:22 ` Patrick Bellasi
2018-08-29 16:56 ` Quentin Perret
2018-08-30 10:00 ` Patrick Bellasi
2018-08-30 10:47 ` Quentin Perret
2018-08-30 12:50 ` Patrick Bellasi
2018-08-20 9:44 ` [PATCH v6 06/14] sched/topology: Lowest CPU asymmetry sched_domain level pointer Quentin Perret
2018-08-20 9:44 ` [PATCH v6 07/14] sched/topology: Introduce sched_energy_present static key Quentin Perret
2018-08-29 16:50 ` Patrick Bellasi
2018-08-29 17:20 ` Quentin Perret
2018-08-30 9:23 ` Patrick Bellasi [this message]
2018-08-30 9:57 ` Quentin Perret
2018-08-30 10:18 ` Patrick Bellasi
2018-09-06 6:06 ` Dietmar Eggemann
2018-09-06 9:29 ` Quentin Perret
2018-09-06 23:49 ` Dietmar Eggemann
2018-09-07 8:24 ` Quentin Perret
2018-08-20 9:44 ` [PATCH v6 08/14] sched/fair: Clean-up update_sg_lb_stats parameters Quentin Perret
2018-08-20 9:44 ` [PATCH v6 09/14] sched: Add over-utilization/tipping point indicator Quentin Perret
2018-08-20 9:44 ` [PATCH v6 10/14] sched/cpufreq: Refactor the utilization aggregation method Quentin Perret
2018-09-10 9:53 ` Rafael J. Wysocki
2018-09-10 10:07 ` Quentin Perret
2018-09-10 10:25 ` Rafael J. Wysocki
2018-08-20 9:44 ` [PATCH v6 11/14] sched/fair: Introduce an energy estimation helper function Quentin Perret
2018-08-20 9:44 ` [PATCH v6 12/14] sched/fair: Select an energy-efficient CPU on task wake-up Quentin Perret
2018-08-20 9:44 ` [PATCH v6 13/14] sched/topology: Make Energy Aware Scheduling depend on schedutil Quentin Perret
2018-09-04 10:59 ` Quentin Perret
2018-09-06 9:18 ` Rafael J. Wysocki
2018-09-06 14:38 ` Quentin Perret
2018-09-07 8:52 ` Rafael J. Wysocki
2018-09-07 8:56 ` Rafael J. Wysocki
2018-09-07 9:02 ` Quentin Perret
2018-09-07 15:29 ` Quentin Perret
2018-09-09 20:13 ` Rafael J. Wysocki
2018-09-10 8:24 ` Quentin Perret
2018-09-10 8:55 ` Rafael J. Wysocki
2018-09-10 9:43 ` Quentin Perret
2018-08-20 9:44 ` [PATCH v6 14/14] OPTIONAL: cpufreq: dt: Register an Energy Model Quentin Perret
2018-09-10 9:12 ` [PATCH v6 00/14] Energy Aware Scheduling Rafael J. Wysocki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180830092329.GS2960@e110439-lin \
--to=patrick.bellasi@arm.com \
--cc=adharmap@codeaurora.org \
--cc=chris.redpath@arm.com \
--cc=currojerez@riseup.net \
--cc=dietmar.eggemann@arm.com \
--cc=edubezval@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=javi.merino@kernel.org \
--cc=joel@joelfernandes.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=peterz@infradead.org \
--cc=pkondeti@codeaurora.org \
--cc=quentin.perret@arm.com \
--cc=rjw@rjwysocki.net \
--cc=skannan@codeaurora.org \
--cc=smuckle@google.com \
--cc=srinivas.pandruvada@linux.intel.com \
--cc=thara.gopinath@linaro.org \
--cc=tkjos@google.com \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
--cc=viresh.kumar@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).