From: Beata Michalska <beata.michalska@arm.com>
To: Valentin Schneider <valentin.schneider@arm.com>
Cc: linux-kernel@vger.kernel.org, peterz@infradead.org,
mingo@redhat.com, juri.lelli@redhat.com,
vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
corbet@lwn.net, rdunlap@infradead.org, linux-doc@vger.kernel.org
Subject: Re: [PATCH v5 2/3] sched/topology: Rework CPU capacity asymmetry detection
Date: Tue, 25 May 2021 00:19:44 +0100 [thread overview]
Message-ID: <20210524231944.GB14880@e120325.cambridge.arm.com> (raw)
In-Reply-To: <20210524225508.GA14880@e120325.cambridge.arm.com>
On Mon, May 24, 2021 at 11:55:08PM +0100, Beata Michalska wrote:
> On Mon, May 24, 2021 at 07:01:04PM +0100, Valentin Schneider wrote:
> > Hi Beata,
> >
> > On 24/05/21 11:16, Beata Michalska wrote:
> > > Currently the CPU capacity asymmetry detection, performed through
> > > asym_cpu_capacity_level, tries to identify the lowest topology level
> > > at which the highest CPU capacity is being observed, not necessarily
> > > finding the level at which all possible capacity values are visible
> > > to all CPUs, which might be bit problematic for some possible/valid
> > > asymmetric topologies i.e.:
> > >
> > > DIE [ ]
> > > MC [ ][ ]
> > >
> > > CPU [0] [1] [2] [3] [4] [5] [6] [7]
> > > Capacity |.....| |.....| |.....| |.....|
> > > L M B B
> > >
> > > Where:
> > > arch_scale_cpu_capacity(L) = 512
> > > arch_scale_cpu_capacity(M) = 871
> > > arch_scale_cpu_capacity(B) = 1024
> > >
> > > In this particular case, the asymmetric topology level will point
> > > at MC, as all possible CPU masks for that level do cover the CPU
> > > with the highest capacity. It will work just fine for the first
> > > cluster, not so much for the second one though (consider the
> > > find_energy_efficient_cpu which might end up attempting the energy
> > > aware wake-up for a domain that does not see any asymmetry at all)
> > >
> > > Rework the way the capacity asymmetry levels are being detected,
> > > allowing to point to the lowest topology level (for a given CPU), where
> > > full set of available CPU capacities is visible to all CPUs within given
> > > domain. As a result, the per-cpu sd_asym_cpucapacity might differ across
> > > the domains. This will have an impact on EAS wake-up placement in a way
> > > that it might see different rage of CPUs to be considered, depending on
> > > the given current and target CPUs.
> > >
> > > Additionally, those levels, where any range of asymmetry (not
> > > necessarily full) is being detected will get identified as well.
> > > The selected asymmetric topology level will be denoted by
> > > SD_ASYM_CPUCAPACITY_FULL sched domain flag whereas the 'sub-levels'
> > > would receive the already used SD_ASYM_CPUCAPACITY flag. This allows
> > > maintaining the current behaviour for asymmetric topologies, with
> > > misfit migration operating correctly on lower levels, if applicable,
> > > as any asymmetry is enough to trigger the misfit migration.
> > > The logic there relies on the SD_ASYM_CPUCAPACITY flag and does not
> > > relate to the full asymmetry level denoted by the sd_asym_cpucapacity
> > > pointer.
> > >
> > > Detecting the CPU capacity asymmetry is being based on a set of
> > > available CPU capacities for all possible CPUs. This data is being
> > > generated upon init and updated once CPU topology changes are being
> > > detected (through arch_update_cpu_topology). As such, any changes
> > > to identified CPU capacities (like initializing cpufreq) need to be
> > > explicitly advertised by corresponding archs to trigger rebuilding
> > > the data.
> > >
> > > This patch also removes the additional -dflags- parameter used when
> > ^^^^^^^^^^^^^^^^^^^^^^^
> > s/^/Also remove/
> I would kind of ... disagree.
> All the commit msg is constructed using passive structure, the suggestion
> would actually break that. And it does 'sound' bit imperative but I guess
> that is subjective. I'd rather stay with impersonal structure (which is
> applied through out the whole patchset).
> >
> > > building sched domains as the asymmetry flags are now being set
> > > directly in sd_init.
> > >
> >
> > Few nits below, but beyond that:
> >
> > Tested-by: Valentin Schneider <valentin.schneider@arm.com>
> > Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
> >
> Thanks a lot for the review and testing!
>
> > > +static inline int
> > > +asym_cpu_capacity_classify(struct sched_domain *sd,
> > > + const struct cpumask *cpu_map)
> > > +{
> > > + int sd_asym_flags = SD_ASYM_CPUCAPACITY | SD_ASYM_CPUCAPACITY_FULL;
> > > + struct asym_cap_data *entry;
> > > + int asym_cap_count = 0;
> > > +
> > > + if (list_is_singular(&asym_cap_list))
> > > + goto leave;
> > > +
> > > + list_for_each_entry(entry, &asym_cap_list, link) {
> > > + if (cpumask_intersects(sched_domain_span(sd), entry->cpu_mask)) {
> > > + ++asym_cap_count;
> > > + } else {
> > > + /*
> > > + * CPUs with given capacity might be offline
> > > + * so make sure this is not the case
> > > + */
> > > + if (cpumask_intersects(entry->cpu_mask, cpu_map)) {
> > > + sd_asym_flags &= ~SD_ASYM_CPUCAPACITY_FULL;
> > > + if (asym_cap_count > 1)
> > > + break;
> > > + }
> >
> > Readability nit: That could be made into an else if ().
> It could but then this way the -comment- gets more exposed.
> But that might be my personal perception so I can change that.
> >
> >
> > > + }
> > > + }
> > > + WARN_ON_ONCE(!asym_cap_count);
> > > +leave:
> > > + return asym_cap_count > 1 ? sd_asym_flags : 0;
> > > +}
> > > +
> >
> > > +static void asym_cpu_capacity_scan(void)
> > > +{
> > > + struct asym_cap_data *entry, *next;
> > > + int cpu;
> > > +
> > > + list_for_each_entry(entry, &asym_cap_list, link)
> > > + cpumask_clear(entry->cpu_mask);
> > > +
> > > + entry = list_first_entry_or_null(&asym_cap_list,
> > > + struct asym_cap_data, link);
> > > +
> > > + for_each_cpu_and(cpu, cpu_possible_mask,
> > > + housekeeping_cpumask(HK_FLAG_DOMAIN)) {
> > > + unsigned long capacity = arch_scale_cpu_capacity(cpu);
> > > +
> > > + if (!entry || capacity != entry->capacity)
> > > + entry = asym_cpu_capacity_get_data(capacity);
> > > + if (entry)
> > > + __cpumask_set_cpu(cpu, entry->cpu_mask);
> >
> > That 'if' is only there in case the alloc within the helper failed, which
> > is a bit of a shame.
> >
> > You could pass the CPU to that helper function and have it set the right
> > bit, or you could even forgo the capacity != entry->capacity check here and
> > let the helper function do it all.
> >
> > Yes, that means more asym_cap_list iterations, but that's
> > O(nr_cpus * nr_caps); a topology rebuild is along the lines of
> > O(nr_cpus² * nr_topology_levels), so not such a big deal comparatively.
> >
> I could drop that check and make the helper function update the CPUs mask
> (along with dropping the initial grabbing of the first entry)
> +
> switching to list_for_each_entry_reverse which would result in less
> iterations for most (if not all) of the use cases.
>
Ignore the 'reverse' idea - the items are already prepended so regular
iteration should pick the last item added.
---
BR
B.
>
> ---
> BR
> B
> > > + }
> > > +
> > > + list_for_each_entry_safe(entry, next, &asym_cap_list, link) {
> > > + if (cpumask_empty(entry->cpu_mask)) {
> > > + list_del(&entry->link);
> > > + kfree(entry);
> > > + }
> > > + }
> > > +}
> > > +
next prev parent reply other threads:[~2021-05-24 23:19 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-24 10:16 [PATCH v5 0/3] Rework CPU capacity asymmetry detection Beata Michalska
2021-05-24 10:16 ` [PATCH v5 1/3] sched/core: Introduce SD_ASYM_CPUCAPACITY_FULL sched_domain flag Beata Michalska
2021-05-24 10:16 ` [PATCH v5 2/3] sched/topology: Rework CPU capacity asymmetry detection Beata Michalska
2021-05-24 18:01 ` Valentin Schneider
2021-05-24 22:55 ` Beata Michalska
2021-05-24 23:19 ` Beata Michalska [this message]
2021-05-25 9:53 ` Valentin Schneider
2021-05-25 10:29 ` Beata Michalska
2021-05-26 9:52 ` Dietmar Eggemann
2021-05-26 12:15 ` Beata Michalska
2021-05-26 12:51 ` Beata Michalska
2021-05-26 18:17 ` Dietmar Eggemann
2021-05-26 21:40 ` Beata Michalska
2021-05-27 15:08 ` Dietmar Eggemann
2021-05-27 17:07 ` Beata Michalska
2021-06-02 17:17 ` Dietmar Eggemann
2021-06-02 19:48 ` Beata Michalska
2021-06-03 9:09 ` Dietmar Eggemann
2021-06-03 9:24 ` Beata Michalska
2021-05-26 18:17 ` Dietmar Eggemann
2021-05-26 21:43 ` Beata Michalska
2021-05-27 7:03 ` Peter Zijlstra
2021-05-27 12:22 ` Dietmar Eggemann
2021-05-27 12:32 ` Beata Michalska
2021-05-25 8:25 ` Dietmar Eggemann
2021-05-25 9:30 ` Beata Michalska
2021-05-25 11:59 ` Dietmar Eggemann
2021-05-25 14:04 ` Beata Michalska
2021-05-24 10:16 ` [PATCH v5 3/3] sched/doc: Update the CPU capacity asymmetry bits Beata Michalska
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210524231944.GB14880@e120325.cambridge.arm.com \
--to=beata.michalska@arm.com \
--cc=corbet@lwn.net \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).