* [PATCH] arch_topology: Limit threads to one specific cluster
@ 2022-05-24 8:12 ` Gavin Shan
0 siblings, 0 replies; 8+ messages in thread
From: Gavin Shan @ 2022-05-24 8:12 UTC (permalink / raw)
To: linux-kernel
Cc: linux-arm-kernel, sudeep.holla, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
The sibling information for one particular CPU is updated after ACPI
PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
the CPUs in same core. However, cluster isn't considered when it's
populated. In this case, multiple threads belonging to different
clusters can be put together through the sibling information. It
eventually leads to unexpected warning from sched subsystem.
For example, the following warning is observed in a VM where we have
2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
is populated as below.
CPU Socket-ID Cluster-ID Core-ID Thread-ID
----------------------------------------------
0 0 0 0 0
1 0 0 0 1
2 0 0 1 0
3 0 0 1 1
4 0 1 0 0
5 0 1 0 1
6 0 1 1 0
7 0 1 1 1
8 1 0 0 0
9 1 0 0 1
10 1 0 1 0
11 1 0 1 1
12 1 1 0 0
13 1 1 0 1
14 1 1 1 0
15 1 1 1 1
[ 0.592181] CPU: All CPU(s) started at EL1
[ 0.593766] alternatives: patching kernel code
[ 0.595890] BUG: arch topology borken
[ 0.597210] the SMT domain not a subset of the CLS domain
[ 0.599286] child=0-1,4-5 sd=0-3
# cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
0-3
# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0-1,4-5
This fixes the issue by limiting threads to one specific cluster.
With this applied, the unexpected warning disappears in the VM.
# cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
0-3
# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0-1
Signed-off-by: Gavin Shan <gshan@redhat.com>
---
drivers/base/arch_topology.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index f73b836047cf..25a63a89ae7d 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -710,7 +710,8 @@ void update_siblings_masks(unsigned int cpuid)
cpumask_set_cpu(cpuid, &cpu_topo->core_sibling);
cpumask_set_cpu(cpu, &cpuid_topo->core_sibling);
- if (cpuid_topo->core_id != cpu_topo->core_id)
+ if (cpuid_topo->cluster_id != cpu_topo->cluster_id ||
+ cpuid_topo->core_id != cpu_topo->core_id)
continue;
cpumask_set_cpu(cpuid, &cpu_topo->thread_sibling);
--
2.23.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH] arch_topology: Limit threads to one specific cluster
@ 2022-05-24 8:12 ` Gavin Shan
0 siblings, 0 replies; 8+ messages in thread
From: Gavin Shan @ 2022-05-24 8:12 UTC (permalink / raw)
To: linux-kernel
Cc: linux-arm-kernel, sudeep.holla, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
The sibling information for one particular CPU is updated after ACPI
PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
the CPUs in same core. However, cluster isn't considered when it's
populated. In this case, multiple threads belonging to different
clusters can be put together through the sibling information. It
eventually leads to unexpected warning from sched subsystem.
For example, the following warning is observed in a VM where we have
2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
is populated as below.
CPU Socket-ID Cluster-ID Core-ID Thread-ID
----------------------------------------------
0 0 0 0 0
1 0 0 0 1
2 0 0 1 0
3 0 0 1 1
4 0 1 0 0
5 0 1 0 1
6 0 1 1 0
7 0 1 1 1
8 1 0 0 0
9 1 0 0 1
10 1 0 1 0
11 1 0 1 1
12 1 1 0 0
13 1 1 0 1
14 1 1 1 0
15 1 1 1 1
[ 0.592181] CPU: All CPU(s) started at EL1
[ 0.593766] alternatives: patching kernel code
[ 0.595890] BUG: arch topology borken
[ 0.597210] the SMT domain not a subset of the CLS domain
[ 0.599286] child=0-1,4-5 sd=0-3
# cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
0-3
# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0-1,4-5
This fixes the issue by limiting threads to one specific cluster.
With this applied, the unexpected warning disappears in the VM.
# cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
0-3
# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0-1
Signed-off-by: Gavin Shan <gshan@redhat.com>
---
drivers/base/arch_topology.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index f73b836047cf..25a63a89ae7d 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -710,7 +710,8 @@ void update_siblings_masks(unsigned int cpuid)
cpumask_set_cpu(cpuid, &cpu_topo->core_sibling);
cpumask_set_cpu(cpu, &cpuid_topo->core_sibling);
- if (cpuid_topo->core_id != cpu_topo->core_id)
+ if (cpuid_topo->cluster_id != cpu_topo->cluster_id ||
+ cpuid_topo->core_id != cpu_topo->core_id)
continue;
cpumask_set_cpu(cpuid, &cpu_topo->thread_sibling);
--
2.23.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] arch_topology: Limit threads to one specific cluster
2022-05-24 8:12 ` Gavin Shan
@ 2022-05-24 8:51 ` Sudeep Holla
-1 siblings, 0 replies; 8+ messages in thread
From: Sudeep Holla @ 2022-05-24 8:51 UTC (permalink / raw)
To: Gavin Shan
Cc: linux-kernel, linux-arm-kernel, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
On Tue, May 24, 2022 at 04:12:12PM +0800, Gavin Shan wrote:
> The sibling information for one particular CPU is updated after ACPI
> PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
> the CPUs in same core. However, cluster isn't considered when it's
> populated. In this case, multiple threads belonging to different
> clusters can be put together through the sibling information. It
> eventually leads to unexpected warning from sched subsystem.
>
> For example, the following warning is observed in a VM where we have
> 2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
> is populated as below.
>
> CPU Socket-ID Cluster-ID Core-ID Thread-ID
> ----------------------------------------------
> 0 0 0 0 0
> 1 0 0 0 1
> 2 0 0 1 0
> 3 0 0 1 1
> 4 0 1 0 0
> 5 0 1 0 1
> 6 0 1 1 0
> 7 0 1 1 1
> 8 1 0 0 0
> 9 1 0 0 1
> 10 1 0 1 0
> 11 1 0 1 1
> 12 1 1 0 0
> 13 1 1 0 1
> 14 1 1 1 0
> 15 1 1 1 1
>
> [ 0.592181] CPU: All CPU(s) started at EL1
> [ 0.593766] alternatives: patching kernel code
> [ 0.595890] BUG: arch topology borken
> [ 0.597210] the SMT domain not a subset of the CLS domain
> [ 0.599286] child=0-1,4-5 sd=0-3
>
> # cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
> 0-3
> # cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
> 0-1,4-5
>
> This fixes the issue by limiting threads to one specific cluster.
> With this applied, the unexpected warning disappears in the VM.
>
I have similar fix but as part of bigger series[1] to get DT support in
line with ACPI.
--
Regards,
Sudeep
[1] https://lore.kernel.org/lkml/20220518093325.2070336-3-sudeep.holla@arm.com
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] arch_topology: Limit threads to one specific cluster
@ 2022-05-24 8:51 ` Sudeep Holla
0 siblings, 0 replies; 8+ messages in thread
From: Sudeep Holla @ 2022-05-24 8:51 UTC (permalink / raw)
To: Gavin Shan
Cc: linux-kernel, linux-arm-kernel, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
On Tue, May 24, 2022 at 04:12:12PM +0800, Gavin Shan wrote:
> The sibling information for one particular CPU is updated after ACPI
> PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
> the CPUs in same core. However, cluster isn't considered when it's
> populated. In this case, multiple threads belonging to different
> clusters can be put together through the sibling information. It
> eventually leads to unexpected warning from sched subsystem.
>
> For example, the following warning is observed in a VM where we have
> 2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
> is populated as below.
>
> CPU Socket-ID Cluster-ID Core-ID Thread-ID
> ----------------------------------------------
> 0 0 0 0 0
> 1 0 0 0 1
> 2 0 0 1 0
> 3 0 0 1 1
> 4 0 1 0 0
> 5 0 1 0 1
> 6 0 1 1 0
> 7 0 1 1 1
> 8 1 0 0 0
> 9 1 0 0 1
> 10 1 0 1 0
> 11 1 0 1 1
> 12 1 1 0 0
> 13 1 1 0 1
> 14 1 1 1 0
> 15 1 1 1 1
>
> [ 0.592181] CPU: All CPU(s) started at EL1
> [ 0.593766] alternatives: patching kernel code
> [ 0.595890] BUG: arch topology borken
> [ 0.597210] the SMT domain not a subset of the CLS domain
> [ 0.599286] child=0-1,4-5 sd=0-3
>
> # cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
> 0-3
> # cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
> 0-1,4-5
>
> This fixes the issue by limiting threads to one specific cluster.
> With this applied, the unexpected warning disappears in the VM.
>
I have similar fix but as part of bigger series[1] to get DT support in
line with ACPI.
--
Regards,
Sudeep
[1] https://lore.kernel.org/lkml/20220518093325.2070336-3-sudeep.holla@arm.com
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] arch_topology: Limit threads to one specific cluster
2022-05-24 8:51 ` Sudeep Holla
@ 2022-05-24 9:05 ` Gavin Shan
-1 siblings, 0 replies; 8+ messages in thread
From: Gavin Shan @ 2022-05-24 9:05 UTC (permalink / raw)
To: Sudeep Holla
Cc: linux-kernel, linux-arm-kernel, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
Hi Sudeep,
On 5/24/22 4:51 PM, Sudeep Holla wrote:
> On Tue, May 24, 2022 at 04:12:12PM +0800, Gavin Shan wrote:
>> The sibling information for one particular CPU is updated after ACPI
>> PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
>> the CPUs in same core. However, cluster isn't considered when it's
>> populated. In this case, multiple threads belonging to different
>> clusters can be put together through the sibling information. It
>> eventually leads to unexpected warning from sched subsystem.
>>
>> For example, the following warning is observed in a VM where we have
>> 2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
>> is populated as below.
>>
>> CPU Socket-ID Cluster-ID Core-ID Thread-ID
>> ----------------------------------------------
>> 0 0 0 0 0
>> 1 0 0 0 1
>> 2 0 0 1 0
>> 3 0 0 1 1
>> 4 0 1 0 0
>> 5 0 1 0 1
>> 6 0 1 1 0
>> 7 0 1 1 1
>> 8 1 0 0 0
>> 9 1 0 0 1
>> 10 1 0 1 0
>> 11 1 0 1 1
>> 12 1 1 0 0
>> 13 1 1 0 1
>> 14 1 1 1 0
>> 15 1 1 1 1
>>
>> [ 0.592181] CPU: All CPU(s) started at EL1
>> [ 0.593766] alternatives: patching kernel code
>> [ 0.595890] BUG: arch topology borken
>> [ 0.597210] the SMT domain not a subset of the CLS domain
>> [ 0.599286] child=0-1,4-5 sd=0-3
>>
>> # cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
>> 0-3
>> # cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
>> 0-1,4-5
>>
>> This fixes the issue by limiting threads to one specific cluster.
>> With this applied, the unexpected warning disappears in the VM.
>>
>
> I have similar fix but as part of bigger series[1] to get DT support in
> line with ACPI.
>
Your patch resolves the issue I have. So please ignore mine. Sorry
for the noise.
https://lore.kernel.org/lkml/20220518093325.2070336-3-sudeep.holla@arm.com
# cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
0-3
# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0-1
Thanks,
Gavin
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] arch_topology: Limit threads to one specific cluster
@ 2022-05-24 9:05 ` Gavin Shan
0 siblings, 0 replies; 8+ messages in thread
From: Gavin Shan @ 2022-05-24 9:05 UTC (permalink / raw)
To: Sudeep Holla
Cc: linux-kernel, linux-arm-kernel, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
Hi Sudeep,
On 5/24/22 4:51 PM, Sudeep Holla wrote:
> On Tue, May 24, 2022 at 04:12:12PM +0800, Gavin Shan wrote:
>> The sibling information for one particular CPU is updated after ACPI
>> PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
>> the CPUs in same core. However, cluster isn't considered when it's
>> populated. In this case, multiple threads belonging to different
>> clusters can be put together through the sibling information. It
>> eventually leads to unexpected warning from sched subsystem.
>>
>> For example, the following warning is observed in a VM where we have
>> 2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
>> is populated as below.
>>
>> CPU Socket-ID Cluster-ID Core-ID Thread-ID
>> ----------------------------------------------
>> 0 0 0 0 0
>> 1 0 0 0 1
>> 2 0 0 1 0
>> 3 0 0 1 1
>> 4 0 1 0 0
>> 5 0 1 0 1
>> 6 0 1 1 0
>> 7 0 1 1 1
>> 8 1 0 0 0
>> 9 1 0 0 1
>> 10 1 0 1 0
>> 11 1 0 1 1
>> 12 1 1 0 0
>> 13 1 1 0 1
>> 14 1 1 1 0
>> 15 1 1 1 1
>>
>> [ 0.592181] CPU: All CPU(s) started at EL1
>> [ 0.593766] alternatives: patching kernel code
>> [ 0.595890] BUG: arch topology borken
>> [ 0.597210] the SMT domain not a subset of the CLS domain
>> [ 0.599286] child=0-1,4-5 sd=0-3
>>
>> # cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
>> 0-3
>> # cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
>> 0-1,4-5
>>
>> This fixes the issue by limiting threads to one specific cluster.
>> With this applied, the unexpected warning disappears in the VM.
>>
>
> I have similar fix but as part of bigger series[1] to get DT support in
> line with ACPI.
>
Your patch resolves the issue I have. So please ignore mine. Sorry
for the noise.
https://lore.kernel.org/lkml/20220518093325.2070336-3-sudeep.holla@arm.com
# cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
0-3
# cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
0-1
Thanks,
Gavin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] arch_topology: Limit threads to one specific cluster
2022-05-24 9:05 ` Gavin Shan
@ 2022-05-24 10:59 ` Sudeep Holla
-1 siblings, 0 replies; 8+ messages in thread
From: Sudeep Holla @ 2022-05-24 10:59 UTC (permalink / raw)
To: Gavin Shan
Cc: linux-kernel, linux-arm-kernel, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
On Tue, May 24, 2022 at 05:05:22PM +0800, Gavin Shan wrote:
> Hi Sudeep,
>
> On 5/24/22 4:51 PM, Sudeep Holla wrote:
> > On Tue, May 24, 2022 at 04:12:12PM +0800, Gavin Shan wrote:
> > > The sibling information for one particular CPU is updated after ACPI
> > > PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
> > > the CPUs in same core. However, cluster isn't considered when it's
> > > populated. In this case, multiple threads belonging to different
> > > clusters can be put together through the sibling information. It
> > > eventually leads to unexpected warning from sched subsystem.
> > >
> > > For example, the following warning is observed in a VM where we have
> > > 2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
> > > is populated as below.
> > >
> > > CPU Socket-ID Cluster-ID Core-ID Thread-ID
> > > ----------------------------------------------
> > > 0 0 0 0 0
> > > 1 0 0 0 1
> > > 2 0 0 1 0
> > > 3 0 0 1 1
> > > 4 0 1 0 0
> > > 5 0 1 0 1
> > > 6 0 1 1 0
> > > 7 0 1 1 1
> > > 8 1 0 0 0
> > > 9 1 0 0 1
> > > 10 1 0 1 0
> > > 11 1 0 1 1
> > > 12 1 1 0 0
> > > 13 1 1 0 1
> > > 14 1 1 1 0
> > > 15 1 1 1 1
> > >
> > > [ 0.592181] CPU: All CPU(s) started at EL1
> > > [ 0.593766] alternatives: patching kernel code
> > > [ 0.595890] BUG: arch topology borken
> > > [ 0.597210] the SMT domain not a subset of the CLS domain
> > > [ 0.599286] child=0-1,4-5 sd=0-3
> > >
> > > # cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
> > > 0-3
> > > # cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
> > > 0-1,4-5
> > >
> > > This fixes the issue by limiting threads to one specific cluster.
> > > With this applied, the unexpected warning disappears in the VM.
> > >
> >
> > I have similar fix but as part of bigger series[1] to get DT support in
> > line with ACPI.
> >
>
> Your patch resolves the issue I have. So please ignore mine. Sorry
Thanks for that.
> for the noise.
>
No worries, definitely not noise.
--
Regards,
Sudeep
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] arch_topology: Limit threads to one specific cluster
@ 2022-05-24 10:59 ` Sudeep Holla
0 siblings, 0 replies; 8+ messages in thread
From: Sudeep Holla @ 2022-05-24 10:59 UTC (permalink / raw)
To: Gavin Shan
Cc: linux-kernel, linux-arm-kernel, rafael, Jonathan.Cameron,
drjones, zhenyzha, shan.gavin
On Tue, May 24, 2022 at 05:05:22PM +0800, Gavin Shan wrote:
> Hi Sudeep,
>
> On 5/24/22 4:51 PM, Sudeep Holla wrote:
> > On Tue, May 24, 2022 at 04:12:12PM +0800, Gavin Shan wrote:
> > > The sibling information for one particular CPU is updated after ACPI
> > > PPTT table is parsed. struct cpu_topology::thread_sibling tracks the
> > > the CPUs in same core. However, cluster isn't considered when it's
> > > populated. In this case, multiple threads belonging to different
> > > clusters can be put together through the sibling information. It
> > > eventually leads to unexpected warning from sched subsystem.
> > >
> > > For example, the following warning is observed in a VM where we have
> > > 2 sockets, 4 clusters, 8 cores and 16 threads and the CPU topology
> > > is populated as below.
> > >
> > > CPU Socket-ID Cluster-ID Core-ID Thread-ID
> > > ----------------------------------------------
> > > 0 0 0 0 0
> > > 1 0 0 0 1
> > > 2 0 0 1 0
> > > 3 0 0 1 1
> > > 4 0 1 0 0
> > > 5 0 1 0 1
> > > 6 0 1 1 0
> > > 7 0 1 1 1
> > > 8 1 0 0 0
> > > 9 1 0 0 1
> > > 10 1 0 1 0
> > > 11 1 0 1 1
> > > 12 1 1 0 0
> > > 13 1 1 0 1
> > > 14 1 1 1 0
> > > 15 1 1 1 1
> > >
> > > [ 0.592181] CPU: All CPU(s) started at EL1
> > > [ 0.593766] alternatives: patching kernel code
> > > [ 0.595890] BUG: arch topology borken
> > > [ 0.597210] the SMT domain not a subset of the CLS domain
> > > [ 0.599286] child=0-1,4-5 sd=0-3
> > >
> > > # cat /sys/devices/system/cpu/cpu0/topology/cluster_cpus_list
> > > 0-3
> > > # cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list
> > > 0-1,4-5
> > >
> > > This fixes the issue by limiting threads to one specific cluster.
> > > With this applied, the unexpected warning disappears in the VM.
> > >
> >
> > I have similar fix but as part of bigger series[1] to get DT support in
> > line with ACPI.
> >
>
> Your patch resolves the issue I have. So please ignore mine. Sorry
Thanks for that.
> for the noise.
>
No worries, definitely not noise.
--
Regards,
Sudeep
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-05-24 11:01 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-24 8:12 [PATCH] arch_topology: Limit threads to one specific cluster Gavin Shan
2022-05-24 8:12 ` Gavin Shan
2022-05-24 8:51 ` Sudeep Holla
2022-05-24 8:51 ` Sudeep Holla
2022-05-24 9:05 ` Gavin Shan
2022-05-24 9:05 ` Gavin Shan
2022-05-24 10:59 ` Sudeep Holla
2022-05-24 10:59 ` Sudeep Holla
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.