* [PATCH v3 01/11] powerpc/topology: Update topology_core_cpumask
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 02/11] powerpc/smp: Stop updating cpu_core_mask Srikar Dronamraju
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, Satheesh Rajendran, LKML,
Nicholas Piggin, Anton Blanchard, Oliver O'Halloran,
Nathan Lynch, Michael Neuling, Gautham R Shenoy, Ingo Molnar,
Peter Zijlstra, Valentin Schneider, Qian Cai
On Power, cpu_core_mask and cpu_cpu_mask refer to the same set of CPUs.
cpu_cpu_mask is needed by scheduler, hence look at deprecating
cpu_core_mask. Before deleting the cpu_core_mask, ensure its only user
is moved to cpu_cpu_mask.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/include/asm/topology.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
index 6609174918ab..e0f232533c9d 100644
--- a/arch/powerpc/include/asm/topology.h
+++ b/arch/powerpc/include/asm/topology.h
@@ -122,7 +122,7 @@ int get_physical_package_id(int cpu);
#endif
#define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
-#define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu))
+#define topology_core_cpumask(cpu) (cpu_cpu_mask(cpu))
#define topology_core_id(cpu) (cpu_to_core_id(cpu))
#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 02/11] powerpc/smp: Stop updating cpu_core_mask
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 01/11] powerpc/topology: Update topology_core_cpumask Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 03/11] powerpc/smp: Remove get_physical_package_id Srikar Dronamraju
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, Satheesh Rajendran, LKML,
Nicholas Piggin, Anton Blanchard, Oliver O'Halloran,
Nathan Lynch, Michael Neuling, Gautham R Shenoy, Ingo Molnar,
Peter Zijlstra, Valentin Schneider, Qian Cai
Anton Blanchard reported that his 4096 vcpu KVM guest took around 30
minutes to boot. He also analyzed it to the time taken to iterate while
setting the cpu_core_mask.
Further analysis shows that cpu_core_mask and cpu_cpu_mask for any CPU
would be equal on Power. However updating cpu_core_mask took forever to
update as its a per cpu cpumask variable. Instead cpu_cpu_mask was a per
NODE /per DIE cpumask that was shared by all the respective CPUs.
Also cpu_cpu_mask is needed from a scheduler perspective. However
cpu_core_map is an exported symbol. Hence stop updating cpu_core_map
and make it point to cpu_cpu_mask.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/include/asm/smp.h | 5 -----
arch/powerpc/kernel/smp.c | 33 +++++++--------------------------
2 files changed, 7 insertions(+), 31 deletions(-)
diff --git a/arch/powerpc/include/asm/smp.h b/arch/powerpc/include/asm/smp.h
index 041f0b97c45b..40e121dd16af 100644
--- a/arch/powerpc/include/asm/smp.h
+++ b/arch/powerpc/include/asm/smp.h
@@ -119,11 +119,6 @@ static inline struct cpumask *cpu_sibling_mask(int cpu)
return per_cpu(cpu_sibling_map, cpu);
}
-static inline struct cpumask *cpu_core_mask(int cpu)
-{
- return per_cpu(cpu_core_map, cpu);
-}
-
static inline struct cpumask *cpu_l2_cache_mask(int cpu)
{
return per_cpu(cpu_l2_cache_map, cpu);
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 3d96752d6570..ec41491beca4 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -953,12 +953,17 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
local_memory_node(numa_cpu_lookup_table[cpu]));
}
#endif
+ /*
+ * cpu_core_map is now more updated and exists only since
+ * its been exported for long. It only will have a snapshot
+ * of cpu_cpu_mask.
+ */
+ cpumask_copy(per_cpu(cpu_core_map, cpu), cpu_cpu_mask(cpu));
}
/* Init the cpumasks so the boot CPU is related to itself */
cpumask_set_cpu(boot_cpuid, cpu_sibling_mask(boot_cpuid));
cpumask_set_cpu(boot_cpuid, cpu_l2_cache_mask(boot_cpuid));
- cpumask_set_cpu(boot_cpuid, cpu_core_mask(boot_cpuid));
if (has_coregroup_support())
cpumask_set_cpu(boot_cpuid, cpu_coregroup_mask(boot_cpuid));
@@ -1260,9 +1265,7 @@ static void remove_cpu_from_masks(int cpu)
{
int i;
- /* NB: cpu_core_mask is a superset of the others */
- for_each_cpu(i, cpu_core_mask(cpu)) {
- set_cpus_unrelated(cpu, i, cpu_core_mask);
+ for_each_cpu(i, cpu_cpu_mask(cpu)) {
set_cpus_unrelated(cpu, i, cpu_l2_cache_mask);
set_cpus_unrelated(cpu, i, cpu_sibling_mask);
if (has_big_cores)
@@ -1312,7 +1315,6 @@ EXPORT_SYMBOL_GPL(get_physical_package_id);
static void add_cpu_to_masks(int cpu)
{
int first_thread = cpu_first_thread_sibling(cpu);
- int pkg_id = get_physical_package_id(cpu);
int i;
/*
@@ -1320,7 +1322,6 @@ static void add_cpu_to_masks(int cpu)
* add it to it's own thread sibling mask.
*/
cpumask_set_cpu(cpu, cpu_sibling_mask(cpu));
- cpumask_set_cpu(cpu, cpu_core_mask(cpu));
for (i = first_thread; i < first_thread + threads_per_core; i++)
if (cpu_online(i))
@@ -1342,26 +1343,6 @@ static void add_cpu_to_masks(int cpu)
set_cpus_related(cpu, i, cpu_coregroup_mask);
}
}
-
- if (pkg_id == -1) {
- struct cpumask *(*mask)(int) = cpu_sibling_mask;
-
- /*
- * Copy the sibling mask into core sibling mask and
- * mark any CPUs on the same chip as this CPU.
- */
- if (shared_caches)
- mask = cpu_l2_cache_mask;
-
- for_each_cpu(i, mask(cpu))
- set_cpus_related(cpu, i, cpu_core_mask);
-
- return;
- }
-
- for_each_cpu(i, cpu_online_mask)
- if (get_physical_package_id(i) == pkg_id)
- set_cpus_related(cpu, i, cpu_core_mask);
}
/* Activate a secondary processor. */
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 03/11] powerpc/smp: Remove get_physical_package_id
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 01/11] powerpc/topology: Update topology_core_cpumask Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 02/11] powerpc/smp: Stop updating cpu_core_mask Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 04/11] powerpc/smp: Optimize remove_cpu_from_masks Srikar Dronamraju
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, Satheesh Rajendran, LKML,
Nicholas Piggin, Anton Blanchard, Oliver O'Halloran,
Nathan Lynch, Michael Neuling, Gautham R Shenoy, Ingo Molnar,
Peter Zijlstra, Valentin Schneider, Qian Cai
Now that cpu_core_mask has been removed and topology_core_cpumask has
been updated to use cpu_cpu_mask, we no more need
get_physical_package_id.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/include/asm/topology.h | 5 -----
arch/powerpc/kernel/smp.c | 20 --------------------
2 files changed, 25 deletions(-)
diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
index e0f232533c9d..e45219f74be0 100644
--- a/arch/powerpc/include/asm/topology.h
+++ b/arch/powerpc/include/asm/topology.h
@@ -114,12 +114,7 @@ static inline int cpu_to_coregroup_id(int cpu)
#ifdef CONFIG_PPC64
#include <asm/smp.h>
-#ifdef CONFIG_PPC_SPLPAR
-int get_physical_package_id(int cpu);
-#define topology_physical_package_id(cpu) (get_physical_package_id(cpu))
-#else
#define topology_physical_package_id(cpu) (cpu_to_chip_id(cpu))
-#endif
#define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
#define topology_core_cpumask(cpu) (cpu_cpu_mask(cpu))
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index ec41491beca4..8c095fe237b2 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1292,26 +1292,6 @@ static inline void add_cpu_to_smallcore_masks(int cpu)
}
}
-int get_physical_package_id(int cpu)
-{
- int pkg_id = cpu_to_chip_id(cpu);
-
- /*
- * If the platform is PowerNV or Guest on KVM, ibm,chip-id is
- * defined. Hence we would return the chip-id as the result of
- * get_physical_package_id.
- */
- if (pkg_id == -1 && firmware_has_feature(FW_FEATURE_LPAR) &&
- IS_ENABLED(CONFIG_PPC_SPLPAR)) {
- struct device_node *np = of_get_cpu_node(cpu, NULL);
- pkg_id = of_node_to_nid(np);
- of_node_put(np);
- }
-
- return pkg_id;
-}
-EXPORT_SYMBOL_GPL(get_physical_package_id);
-
static void add_cpu_to_masks(int cpu)
{
int first_thread = cpu_first_thread_sibling(cpu);
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 04/11] powerpc/smp: Optimize remove_cpu_from_masks
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (2 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 03/11] powerpc/smp: Remove get_physical_package_id Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 05/11] powerpc/smp: Limit CPUs traversed to within a node Srikar Dronamraju
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, Satheesh Rajendran, LKML,
Nicholas Piggin, Anton Blanchard, Oliver O'Halloran,
Nathan Lynch, Michael Neuling, Gautham R Shenoy, Ingo Molnar,
Peter Zijlstra, Valentin Schneider, Qian Cai
While offlining a CPU, system currently iterate through all the CPUs in
the DIE to clear sibling, l2_cache and smallcore maps. However if there
are more cores in a DIE, system can end up spending more time iterating
through CPUs which are completely unrelated.
Optimize this by only iterating through smaller but relevant cpumap.
If shared_cache is set, cpu_l2_cache_map should be relevant else
cpu_sibling_map would be relevant.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/kernel/smp.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 8c095fe237b2..2e61a81aad88 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1263,14 +1263,21 @@ static bool update_mask_by_l2(int cpu, struct cpumask *(*mask_fn)(int))
#ifdef CONFIG_HOTPLUG_CPU
static void remove_cpu_from_masks(int cpu)
{
+ struct cpumask *(*mask_fn)(int) = cpu_sibling_mask;
int i;
- for_each_cpu(i, cpu_cpu_mask(cpu)) {
+ if (shared_caches)
+ mask_fn = cpu_l2_cache_mask;
+
+ for_each_cpu(i, mask_fn(cpu)) {
set_cpus_unrelated(cpu, i, cpu_l2_cache_mask);
set_cpus_unrelated(cpu, i, cpu_sibling_mask);
if (has_big_cores)
set_cpus_unrelated(cpu, i, cpu_smallcore_mask);
- if (has_coregroup_support())
+ }
+
+ if (has_coregroup_support()) {
+ for_each_cpu(i, cpu_coregroup_mask(cpu))
set_cpus_unrelated(cpu, i, cpu_coregroup_mask);
}
}
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 05/11] powerpc/smp: Limit CPUs traversed to within a node.
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (3 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 04/11] powerpc/smp: Optimize remove_cpu_from_masks Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 06/11] powerpc/smp: Stop passing mask to update_mask_by_l2 Srikar Dronamraju
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, Satheesh Rajendran, LKML,
Nicholas Piggin, Anton Blanchard, Oliver O'Halloran,
Nathan Lynch, Michael Neuling, Gautham R Shenoy, Ingo Molnar,
Peter Zijlstra, Valentin Schneider, Qian Cai
All the arch specific topology cpumasks are within a node/DIE.
However when setting these per CPU cpumasks, system traverses through
all the online CPUs. This is redundant.
Reduce the traversal to only CPUs that are online in the node to which
the CPU belongs to.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/kernel/smp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 2e61a81aad88..c860c4950c9f 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1241,7 +1241,7 @@ static bool update_mask_by_l2(int cpu, struct cpumask *(*mask_fn)(int))
}
cpumask_set_cpu(cpu, mask_fn(cpu));
- for_each_cpu(i, cpu_online_mask) {
+ for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
/*
* when updating the marks the current CPU has not been marked
* online, but we need to update the cache masks
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 06/11] powerpc/smp: Stop passing mask to update_mask_by_l2
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (4 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 05/11] powerpc/smp: Limit CPUs traversed to within a node Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 07/11] powerpc/smp: Depend on cpu_l1_cache_map when adding CPUs Srikar Dronamraju
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, LKML, Nicholas Piggin,
Anton Blanchard, Oliver O'Halloran, Nathan Lynch,
Michael Neuling, Gautham R Shenoy, Satheesh Rajendran,
Ingo Molnar, Peter Zijlstra, Valentin Schneider, Qian Cai
update_mask_by_l2 is called only once. But it passes cpu_l2_cache_mask
as parameter. Instead of passing cpu_l2_cache_mask, use it directly in
update_mask_by_l2.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/kernel/smp.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index c860c4950c9f..441c9c64b1e3 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1218,7 +1218,7 @@ static struct device_node *cpu_to_l2cache(int cpu)
return cache;
}
-static bool update_mask_by_l2(int cpu, struct cpumask *(*mask_fn)(int))
+static bool update_mask_by_l2(int cpu)
{
struct device_node *l2_cache, *np;
int i;
@@ -1240,7 +1240,7 @@ static bool update_mask_by_l2(int cpu, struct cpumask *(*mask_fn)(int))
return false;
}
- cpumask_set_cpu(cpu, mask_fn(cpu));
+ cpumask_set_cpu(cpu, cpu_l2_cache_mask(cpu));
for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
/*
* when updating the marks the current CPU has not been marked
@@ -1251,7 +1251,7 @@ static bool update_mask_by_l2(int cpu, struct cpumask *(*mask_fn)(int))
continue;
if (np == l2_cache)
- set_cpus_related(cpu, i, mask_fn);
+ set_cpus_related(cpu, i, cpu_l2_cache_mask);
of_node_put(np);
}
@@ -1315,7 +1315,7 @@ static void add_cpu_to_masks(int cpu)
set_cpus_related(i, cpu, cpu_sibling_mask);
add_cpu_to_smallcore_masks(cpu);
- update_mask_by_l2(cpu, cpu_l2_cache_mask);
+ update_mask_by_l2(cpu);
if (has_coregroup_support()) {
int coregroup_id = cpu_to_coregroup_id(cpu);
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 07/11] powerpc/smp: Depend on cpu_l1_cache_map when adding CPUs
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (5 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 06/11] powerpc/smp: Stop passing mask to update_mask_by_l2 Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 08/11] powerpc/smp: Check for duplicate topologies and consolidate Srikar Dronamraju
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, Satheesh Rajendran, LKML,
Nicholas Piggin, Anton Blanchard, Oliver O'Halloran,
Nathan Lynch, Michael Neuling, Gautham R Shenoy, Ingo Molnar,
Peter Zijlstra, Valentin Schneider, Qian Cai
Currently on hotplug/hotunplug, CPU iterates through all the CPUs in
its core to find threads in its thread group. However this info is
already captured in cpu_l1_cache_map. Hence reduce iterations and
cleanup add_cpu_to_smallcore_masks function.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Tested-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/kernel/smp.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 441c9c64b1e3..aeb219a4bf7a 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1285,16 +1285,15 @@ static void remove_cpu_from_masks(int cpu)
static inline void add_cpu_to_smallcore_masks(int cpu)
{
- struct cpumask *this_l1_cache_map = per_cpu(cpu_l1_cache_map, cpu);
- int i, first_thread = cpu_first_thread_sibling(cpu);
+ int i;
if (!has_big_cores)
return;
cpumask_set_cpu(cpu, cpu_smallcore_mask(cpu));
- for (i = first_thread; i < first_thread + threads_per_core; i++) {
- if (cpu_online(i) && cpumask_test_cpu(i, this_l1_cache_map))
+ for_each_cpu(i, per_cpu(cpu_l1_cache_map, cpu)) {
+ if (cpu_online(i))
set_cpus_related(i, cpu, cpu_smallcore_mask);
}
}
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 08/11] powerpc/smp: Check for duplicate topologies and consolidate
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (6 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 07/11] powerpc/smp: Depend on cpu_l1_cache_map when adding CPUs Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 09/11] powerpc/smp: Optimize update_mask_by_l2 Srikar Dronamraju
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, LKML, Nicholas Piggin,
Anton Blanchard, Oliver O'Halloran, Nathan Lynch,
Michael Neuling, Gautham R Shenoy, Satheesh Rajendran,
Ingo Molnar, Peter Zijlstra, Valentin Schneider, Qian Cai
CACHE and COREGROUP domains are now part of default topology. However on
systems that don't support CACHE or COREGROUP, these domains will
eventually be degenerated. The degeneration happens per CPU. Do note the
current fixup_topology() logic ensures that mask of a domain that is not
supported on the current platform is set to the previous domain.
Instead of waiting for the scheduler to degenerated try to consolidate
based on their masks and sd_flags. This is done just before setting
the scheduler topology.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/kernel/smp.c | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index aeb219a4bf7a..6f866e6b12f8 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1401,6 +1401,8 @@ int setup_profiling_timer(unsigned int multiplier)
static void fixup_topology(void)
{
+ int i;
+
#ifdef CONFIG_SCHED_SMT
if (has_big_cores) {
pr_info("Big cores detected but using small core scheduling\n");
@@ -1410,6 +1412,30 @@ static void fixup_topology(void)
if (!has_coregroup_support())
powerpc_topology[mc_idx].mask = powerpc_topology[cache_idx].mask;
+
+ /*
+ * Try to consolidate topology levels here instead of
+ * allowing scheduler to degenerate.
+ * - Dont consolidate if masks are different.
+ * - Dont consolidate if sd_flags exists and are different.
+ */
+ for (i = 1; i <= die_idx; i++) {
+ if (powerpc_topology[i].mask != powerpc_topology[i - 1].mask)
+ continue;
+
+ if (powerpc_topology[i].sd_flags && powerpc_topology[i - 1].sd_flags &&
+ powerpc_topology[i].sd_flags != powerpc_topology[i - 1].sd_flags)
+ continue;
+
+ if (!powerpc_topology[i - 1].sd_flags)
+ powerpc_topology[i - 1].sd_flags = powerpc_topology[i].sd_flags;
+
+ powerpc_topology[i].mask = powerpc_topology[i + 1].mask;
+ powerpc_topology[i].sd_flags = powerpc_topology[i + 1].sd_flags;
+#ifdef CONFIG_SCHED_DEBUG
+ powerpc_topology[i].name = powerpc_topology[i + 1].name;
+#endif
+ }
}
void __init smp_cpus_done(unsigned int max_cpus)
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 09/11] powerpc/smp: Optimize update_mask_by_l2
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (7 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 08/11] powerpc/smp: Check for duplicate topologies and consolidate Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:37 ` [PATCH v3 10/11] powerpc/smp: Move coregroup mask updation to a new function Srikar Dronamraju
2020-10-07 18:38 ` [PATCH v3 11/11] powerpc/smp: Optimize update_coregroup_mask Srikar Dronamraju
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, LKML, Nicholas Piggin,
Anton Blanchard, Oliver O'Halloran, Nathan Lynch,
Michael Neuling, Gautham R Shenoy, Satheesh Rajendran,
Ingo Molnar, Peter Zijlstra, Valentin Schneider, Qian Cai
All threads of a SMT4 core can either be part of this CPU's l2-cache
mask or not related to this CPU l2-cache mask. Use this relation to
reduce the number of iterations needed to find all the CPUs that share
the same l2-cache.
Use a temporary mask to iterate through the CPUs that may share l2_cache
mask. Also instead of setting one CPU at a time into cpu_l2_cache_mask,
copy the SMT4/sub mask at one shot.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
Changelog v2->v3:
Use GFP_ATOMIC instead of GFP_KERNEL since allocations need to
atomic at the time of CPU HotPlug
Reported by Qian Cai <cai@redhat.com>
arch/powerpc/kernel/smp.c | 52 +++++++++++++++++++++++++++++++++++++++++------
1 file changed, 46 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 6f866e6b12f8..17e90c2414af 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -671,6 +671,28 @@ static void set_cpus_unrelated(int i, int j,
#endif
/*
+ * Extends set_cpus_related. Instead of setting one CPU at a time in
+ * dstmask, set srcmask at oneshot. dstmask should be super set of srcmask.
+ */
+static void or_cpumasks_related(int i, int j, struct cpumask *(*srcmask)(int),
+ struct cpumask *(*dstmask)(int))
+{
+ struct cpumask *mask;
+ int k;
+
+ mask = srcmask(j);
+ for_each_cpu(k, srcmask(i))
+ cpumask_or(dstmask(k), dstmask(k), mask);
+
+ if (i == j)
+ return;
+
+ mask = srcmask(i);
+ for_each_cpu(k, srcmask(j))
+ cpumask_or(dstmask(k), dstmask(k), mask);
+}
+
+/*
* parse_thread_groups: Parses the "ibm,thread-groups" device tree
* property for the CPU device node @dn and stores
* the parsed output in the thread_groups
@@ -1220,7 +1242,9 @@ static struct device_node *cpu_to_l2cache(int cpu)
static bool update_mask_by_l2(int cpu)
{
+ struct cpumask *(*submask_fn)(int) = cpu_sibling_mask;
struct device_node *l2_cache, *np;
+ cpumask_var_t mask;
int i;
l2_cache = cpu_to_l2cache(cpu);
@@ -1240,22 +1264,38 @@ static bool update_mask_by_l2(int cpu)
return false;
}
- cpumask_set_cpu(cpu, cpu_l2_cache_mask(cpu));
- for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
+ /* In CPU-hotplug path, hence use GFP_ATOMIC */
+ alloc_cpumask_var_node(&mask, GFP_ATOMIC, cpu_to_node(cpu));
+ cpumask_and(mask, cpu_online_mask, cpu_cpu_mask(cpu));
+
+ if (has_big_cores)
+ submask_fn = cpu_smallcore_mask;
+
+ /* Update l2-cache mask with all the CPUs that are part of submask */
+ or_cpumasks_related(cpu, cpu, submask_fn, cpu_l2_cache_mask);
+
+ /* Skip all CPUs already part of current CPU l2-cache mask */
+ cpumask_andnot(mask, mask, cpu_l2_cache_mask(cpu));
+
+ for_each_cpu(i, mask) {
/*
* when updating the marks the current CPU has not been marked
* online, but we need to update the cache masks
*/
np = cpu_to_l2cache(i);
- if (!np)
- continue;
- if (np == l2_cache)
- set_cpus_related(cpu, i, cpu_l2_cache_mask);
+ /* Skip all CPUs already part of current CPU l2-cache */
+ if (np == l2_cache) {
+ or_cpumasks_related(cpu, i, submask_fn, cpu_l2_cache_mask);
+ cpumask_andnot(mask, mask, submask_fn(i));
+ } else {
+ cpumask_andnot(mask, mask, cpu_l2_cache_mask(i));
+ }
of_node_put(np);
}
of_node_put(l2_cache);
+ free_cpumask_var(mask);
return true;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 10/11] powerpc/smp: Move coregroup mask updation to a new function
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (8 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 09/11] powerpc/smp: Optimize update_mask_by_l2 Srikar Dronamraju
@ 2020-10-07 18:37 ` Srikar Dronamraju
2020-10-07 18:38 ` [PATCH v3 11/11] powerpc/smp: Optimize update_coregroup_mask Srikar Dronamraju
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:37 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, LKML, Nicholas Piggin,
Anton Blanchard, Oliver O'Halloran, Nathan Lynch,
Michael Neuling, Gautham R Shenoy, Satheesh Rajendran,
Ingo Molnar, Peter Zijlstra, Valentin Schneider, Qian Cai
Move the logic for updating the coregroup mask of a CPU to its own
function. This will help in reworking the updation of coregroup mask in
subsequent patch.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Qian Cai <cai@redhat.com>
---
arch/powerpc/kernel/smp.c | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 17e90c2414af..b48ae4e306d3 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1338,6 +1338,23 @@ static inline void add_cpu_to_smallcore_masks(int cpu)
}
}
+static void update_coregroup_mask(int cpu)
+{
+ int first_thread = cpu_first_thread_sibling(cpu);
+ int coregroup_id = cpu_to_coregroup_id(cpu);
+ int i;
+
+ cpumask_set_cpu(cpu, cpu_coregroup_mask(cpu));
+ for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
+ int fcpu = cpu_first_thread_sibling(i);
+
+ if (fcpu == first_thread)
+ set_cpus_related(cpu, i, cpu_coregroup_mask);
+ else if (coregroup_id == cpu_to_coregroup_id(i))
+ set_cpus_related(cpu, i, cpu_coregroup_mask);
+ }
+}
+
static void add_cpu_to_masks(int cpu)
{
int first_thread = cpu_first_thread_sibling(cpu);
@@ -1356,19 +1373,8 @@ static void add_cpu_to_masks(int cpu)
add_cpu_to_smallcore_masks(cpu);
update_mask_by_l2(cpu);
- if (has_coregroup_support()) {
- int coregroup_id = cpu_to_coregroup_id(cpu);
-
- cpumask_set_cpu(cpu, cpu_coregroup_mask(cpu));
- for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
- int fcpu = cpu_first_thread_sibling(i);
-
- if (fcpu == first_thread)
- set_cpus_related(cpu, i, cpu_coregroup_mask);
- else if (coregroup_id == cpu_to_coregroup_id(i))
- set_cpus_related(cpu, i, cpu_coregroup_mask);
- }
- }
+ if (has_coregroup_support())
+ update_coregroup_mask(cpu);
}
/* Activate a secondary processor. */
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v3 11/11] powerpc/smp: Optimize update_coregroup_mask
2020-10-07 18:37 [PATCH v3 00/11] Optimization to improve CPU online/offline on Powerpc Srikar Dronamraju
` (9 preceding siblings ...)
2020-10-07 18:37 ` [PATCH v3 10/11] powerpc/smp: Move coregroup mask updation to a new function Srikar Dronamraju
@ 2020-10-07 18:38 ` Srikar Dronamraju
10 siblings, 0 replies; 12+ messages in thread
From: Srikar Dronamraju @ 2020-10-07 18:38 UTC (permalink / raw)
To: Michael Ellerman
Cc: linuxppc-dev, Srikar Dronamraju, LKML, Nicholas Piggin,
Anton Blanchard, Oliver O'Halloran, Nathan Lynch,
Michael Neuling, Gautham R Shenoy, Satheesh Rajendran,
Ingo Molnar, Peter Zijlstra, Valentin Schneider
All threads of a SMT4/SMT8 core can either be part of CPU's coregroup
mask or outside the coregroup. Use this relation to reduce the
number of iterations needed to find all the CPUs that share the same
coregroup
Use a temporary mask to iterate through the CPUs that may share
coregroup mask. Also instead of setting one CPU at a time into
cpu_coregroup_mask, copy the SMT4/SMT8/submask at one shot.
Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Oliver O'Halloran <oohall@gmail.com>
Cc: Nathan Lynch <nathanl@linux.ibm.com>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Gautham R Shenoy <ego@linux.vnet.ibm.com>
Cc: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
---
Changelog v2->v3:
Use GFP_ATOMIC instead of GFP_KERNEL since allocations need to
atomic at the time of CPU HotPlug
Reported by Qian Cai <cai@redhat.com>
arch/powerpc/kernel/smp.c | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index b48ae4e306d3..bbaea93dc558 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1340,19 +1340,34 @@ static inline void add_cpu_to_smallcore_masks(int cpu)
static void update_coregroup_mask(int cpu)
{
- int first_thread = cpu_first_thread_sibling(cpu);
+ struct cpumask *(*submask_fn)(int) = cpu_sibling_mask;
+ cpumask_var_t mask;
int coregroup_id = cpu_to_coregroup_id(cpu);
int i;
- cpumask_set_cpu(cpu, cpu_coregroup_mask(cpu));
- for_each_cpu_and(i, cpu_online_mask, cpu_cpu_mask(cpu)) {
- int fcpu = cpu_first_thread_sibling(i);
+ /* In CPU-hotplug path, hence use GFP_ATOMIC */
+ alloc_cpumask_var_node(&mask, GFP_ATOMIC, cpu_to_node(cpu));
+ cpumask_and(mask, cpu_online_mask, cpu_cpu_mask(cpu));
+
+ if (shared_caches)
+ submask_fn = cpu_l2_cache_mask;
- if (fcpu == first_thread)
- set_cpus_related(cpu, i, cpu_coregroup_mask);
- else if (coregroup_id == cpu_to_coregroup_id(i))
- set_cpus_related(cpu, i, cpu_coregroup_mask);
+ /* Update coregroup mask with all the CPUs that are part of submask */
+ or_cpumasks_related(cpu, cpu, submask_fn, cpu_coregroup_mask);
+
+ /* Skip all CPUs already part of coregroup mask */
+ cpumask_andnot(mask, mask, cpu_coregroup_mask(cpu));
+
+ for_each_cpu(i, mask) {
+ /* Skip all CPUs not part of this coregroup */
+ if (coregroup_id == cpu_to_coregroup_id(i)) {
+ or_cpumasks_related(cpu, i, submask_fn, cpu_coregroup_mask);
+ cpumask_andnot(mask, mask, submask_fn(i));
+ } else {
+ cpumask_andnot(mask, mask, cpu_coregroup_mask(i));
+ }
}
+ free_cpumask_var(mask);
}
static void add_cpu_to_masks(int cpu)
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread