From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
To: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nathan Lynch <nathanl@linux.ibm.com>,
Gautham R Shenoy <ego@linux.vnet.ibm.com>,
Michael Neuling <mikey@neuling.org>,
Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
Peter Zijlstra <peterz@infradead.org>,
LKML <linux-kernel@vger.kernel.org>,
Nicholas Piggin <npiggin@gmail.com>,
Valentin Schneider <valentin.schneider@arm.com>,
Qian Cai <cai@redhat.com>, Oliver O'Halloran <oohall@gmail.com>,
Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>,
linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
Ingo Molnar <mingo@kernel.org>
Subject: [PATCH v3 09/11] powerpc/smp: Optimize update_mask_by_l2
Date: Thu, 8 Oct 2020 00:07:58 +0530 [thread overview]
Message-ID: <20201007183800.27415-10-srikar@linux.vnet.ibm.com> (raw)
In-Reply-To: <20201007183800.27415-1-srikar@linux.vnet.ibm.com>
All threads of a SMT4 core can either be part of this CPU's l2-cache
mask or not related to this CPU l2-cache mask. Use this relation to
reduce the number of iterations needed to find all the CPUs that share
the same l2-cache.
Use a temporary mask to iterate through the CPUs that may share l2_cache
mask. Also instead of setting one CPU at a time into cpu_l2_cache_mask,
copy the SMT4/sub mask at one shot.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
Changelog v2->v3:
Use GFP_ATOMIC instead of GFP_KERNEL since allocations need to
atomic at the time of CPU HotPlug
Reported by Qian Cai <cai@redhat.com>
arch/powerpc/kernel/smp.c | 52 +++++++++++++++++++++++++++++++++++++++++------
1 file changed, 46 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 6f866e6b12f8..17e90c2414af 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -671,6 +671,28 @@ static void set_cpus_unrelated(int i, int j,
#endif
/*
+ * Extends set_cpus_related. Instead of setting one CPU at a time in
+ * dstmask, set srcmask at oneshot. dstmask should be super set of srcmask.
+ */
+static void or_cpumasks_related(int i, int j, struct cpumask *(*srcmask)(int),
+ struct cpumask *(*dstmask)(int))
+{
+ struct cpumask *mask;
+ int k;
+
+ mask = srcmask(j);
+ for_each_cpu(k, srcmask(i))
+ cpumask_or(dstmask(k), dstmask(k), mask);
+
+ if (i == j)
+ return;
+
+ mask = srcmask(i);
+ for_each_cpu(k, srcmask(j))
+ cpumask_or(dstmask(k), dstmask(k), mask);
+}
+
+/*
* parse_thread_groups: Parses the "ibm,thread-groups" device tree
* property for the CPU device node @dn and stores
* the parsed output in the thread_groups
@@ -1220,7 +1242,9 @@ static struct device_node *cpu_to_l2cache(int cpu)
static bool update_mask_by_l2(int cpu)
{
+ struct cpumask *(*submask_fn)(int) = cpu_sibling_mask;
struct device_node *l2_cache, *np;
+ cpumask_var_t mask;
int i;
l2_cache = cpu_to_l2cache(cpu);
@@ -1240,22 +1264,38 @@ static bool update_mask_by_l2(int cpu)
return false;
}
- cpumask_set_cpu(cpu, cpu_l2_cache_mask(cpu));
- for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
+ /* In CPU-hotplug path, hence use GFP_ATOMIC */
+ alloc_cpumask_var_node(&mask, GFP_ATOMIC, cpu_to_node(cpu));
+ cpumask_and(mask, cpu_online_mask, cpu_cpu_mask(cpu));
+
+ if (has_big_cores)
+ submask_fn = cpu_smallcore_mask;
+
+ /* Update l2-cache mask with all the CPUs that are part of submask */
+ or_cpumasks_related(cpu, cpu, submask_fn, cpu_l2_cache_mask);
+
+ /* Skip all CPUs already part of current CPU l2-cache mask */
+ cpumask_andnot(mask, mask, cpu_l2_cache_mask(cpu));
+
+ for_each_cpu(i, mask) {
/*
* when updating the marks the current CPU has not been marked
* online, but we need to update the cache masks
*/
np = cpu_to_l2cache(i);
- if (!np)
- continue;
- if (np == l2_cache)
- set_cpus_related(cpu, i, cpu_l2_cache_mask);
+ /* Skip all CPUs already part of current CPU l2-cache */
+ if (np == l2_cache) {
+ or_cpumasks_related(cpu, i, submask_fn, cpu_l2_cache_mask);
+ cpumask_andnot(mask, mask, submask_fn(i));
+ } else {
+ cpumask_andnot(mask, mask, cpu_l2_cache_mask(i));
+ }
of_node_put(np);
}
of_node_put(l2_cache);
+ free_cpumask_var(mask);
return true;
}
--
2.17.1
next prev parent reply other threads:[~2020-10-07 18:58 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 01/11] powerpc/topology: Update topology_core_cpumask Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 02/11] powerpc/smp: Stop updating cpu_core_mask Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 03/11] powerpc/smp: Remove get_physical_package_id Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 04/11] powerpc/smp: Optimize remove_cpu_from_masks Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 05/11] powerpc/smp: Limit CPUs traversed to within a node Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 06/11] powerpc/smp: Stop passing mask to update_mask_by_l2 Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 07/11] powerpc/smp: Depend on cpu_l1_cache_map when adding CPUs Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 08/11] powerpc/smp: Check for duplicate topologies and consolidate Srikar Dronamraju
2020-10-07 18:37 ` Srikar Dronamraju [this message]
2020-10-07 18:37 ` [PATCH v3 10/11] powerpc/smp: Move coregroup mask updation to a new function Srikar Dronamraju
2020-10-07 18:38 ` [PATCH v3 11/11] powerpc/smp: Optimize update_coregroup_mask Srikar Dronamraju
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201007183800.27415-10-srikar@linux.vnet.ibm.com \
--to=srikar@linux.vnet.ibm.com \
--cc=cai@redhat.com \
--cc=ego@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mikey@neuling.org \
--cc=mingo@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=nathanl@linux.ibm.com \
--cc=npiggin@gmail.com \
--cc=oohall@gmail.com \
--cc=peterz@infradead.org \
--cc=sathnaga@linux.vnet.ibm.com \
--cc=valentin.schneider@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).