* [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
@ 2021-11-10 9:58 Wang ShaoBo
2021-11-11 6:25 ` Barry Song
` (3 more replies)
0 siblings, 4 replies; 12+ messages in thread
From: Wang ShaoBo @ 2021-11-10 9:58 UTC (permalink / raw)
To: linux-kernel
Cc: rafael, gregkh, sudeep.holla, peterz, cj.chengjian, huawei.libin,
weiyongjun1
When testing cpu online and offline, warning happened like this:
[ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
[ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
[ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
[ 146.751213] Workqueue: events cpuset_hotplug_workfn
[ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 146.752048] pc : build_sched_domains+0x81c/0x11b0
[ 146.752461] lr : build_sched_domains+0x414/0x11b0
[ 146.752860] sp : ffff800040a83a80
[ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
[ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
[ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
[ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
[ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
[ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
[ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
[ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
[ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
[ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
[ 146.757070] Call trace:
[ 146.757421] build_sched_domains+0x81c/0x11b0
[ 146.757771] partition_sched_domains_locked+0x57c/0x978
[ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
[ 146.758460] rebuild_sched_domains+0x2c/0x48
[ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
[ 146.759114] process_one_work+0x1f4/0x480
[ 146.759429] worker_thread+0x48/0x460
[ 146.759734] kthread+0x158/0x168
[ 146.760030] ret_from_fork+0x10/0x20
[ 146.760318] ---[ end trace 82c44aad6900e81a ]---
For some architectures like risc-v and arm64 which use common code
clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
is set, cluster_sibling in cpu_topology of each sibling adjacent
to CPUx is missed clearing, this causes checking failed in
topology_span_sane() and rebuilding topology failure at end when CPU online.
Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
(CPU 92, 93, 94, 95 are in one cluster):
Before revision:
CPU [92] [93] [94] [95]
cluster_sibling [92] [92-95] [92-95] [92-95]
After revision:
CPU [92] [93] [94] [95]
cluster_sibling [92] [93-95] [93-95] [93-95]
Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
---
drivers/base/arch_topology.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 981e72a3dafb..ff16a36a908b 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -677,6 +677,8 @@ void remove_cpu_topology(unsigned int cpu)
cpumask_clear_cpu(cpu, topology_core_cpumask(sibling));
for_each_cpu(sibling, topology_sibling_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
+ for_each_cpu(sibling, topology_cluster_cpumask(cpu))
+ cpumask_clear_cpu(cpu, topology_cluster_cpumask(sibling));
for_each_cpu(sibling, topology_llc_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_llc_cpumask(sibling));
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-10 9:58 [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology() Wang ShaoBo
@ 2021-11-11 6:25 ` Barry Song
[not found] ` <943fef84-3920-42bc-b83f-4feaa3ab79f3@huawei.com>
2021-11-11 12:22 ` [tip: sched/urgent] " tip-bot2 for Wang ShaoBo
` (2 subsequent siblings)
3 siblings, 1 reply; 12+ messages in thread
From: Barry Song @ 2021-11-11 6:25 UTC (permalink / raw)
To: Wang ShaoBo
Cc: LKML, Rafael J. Wysocki, Greg Kroah-Hartman, Sudeep Holla,
Peter Zijlstra, cj.chengjian, huawei.libin, weiyongjun1
On Wed, Nov 10, 2021 at 10:53 PM Wang ShaoBo <bobo.shaobowang@huawei.com> wrote:
>
> When testing cpu online and offline, warning happened like this:
>
> [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> [ 146.752860] sp : ffff800040a83a80
> [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> [ 146.757070] Call trace:
> [ 146.757421] build_sched_domains+0x81c/0x11b0
> [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> [ 146.758460] rebuild_sched_domains+0x2c/0x48
> [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> [ 146.759114] process_one_work+0x1f4/0x480
> [ 146.759429] worker_thread+0x48/0x460
> [ 146.759734] kthread+0x158/0x168
> [ 146.760030] ret_from_fork+0x10/0x20
> [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
>
> For some architectures like risc-v and arm64 which use common code
> clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> is set, cluster_sibling in cpu_topology of each sibling adjacent
> to CPUx is missed clearing, this causes checking failed in
> topology_span_sane() and rebuilding topology failure at end when CPU online.
>
> Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> (CPU 92, 93, 94, 95 are in one cluster):
>
> Before revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [92-95] [92-95] [92-95]
>
> After revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [93-95] [93-95] [93-95]
>
> Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
The patch looks correct. But how do you reproduce it?
it seems the below pattern can't reproduce it:
root@ubuntu:/sys/devices/system/cpu# dmesg -c
root@ubuntu:/sys/devices/system/cpu# echo 0 > cpu92/online
root@ubuntu:/sys/devices/system/cpu# echo 0 > cpu93/online
root@ubuntu:/sys/devices/system/cpu# echo 0 > cpu94/online
root@ubuntu:/sys/devices/system/cpu# echo 0 > cpu95/online
root@ubuntu:/sys/devices/system/cpu# echo 1 > cpu92/online
root@ubuntu:/sys/devices/system/cpu# echo 1 > cpu93/online
root@ubuntu:/sys/devices/system/cpu# echo 1 > cpu94/online
root@ubuntu:/sys/devices/system/cpu# echo 1 > cpu95/online
root@ubuntu:/sys/devices/system/cpu# dmesg
[297239.288736] psci: CPU92 killed (polled 0 ms)
[297242.060617] psci: CPU93 killed (polled 0 ms)
[297244.748477] psci: CPU94 killed (polled 0 ms)
[297248.480440] psci: CPU95 killed (polled 0 ms)
[297254.966108] Detected VIPT I-cache on CPU92
[297254.966163] GICv3: CPU92: found redistributor 2f0000 region
92:0x00002000ae800000
[297254.966256] CPU92: Booted secondary processor 0x00002f0000 [0x481fd010]
[297261.997920] Detected VIPT I-cache on CPU93
[297261.997965] GICv3: CPU93: found redistributor 2f0100 region
93:0x00002000ae840000
[297261.998052] CPU93: Booted secondary processor 0x00002f0100 [0x481fd010]
[297264.893849] Detected VIPT I-cache on CPU94
[297264.893895] GICv3: CPU94: found redistributor 2f0200 region
94:0x00002000ae880000
[297264.893985] CPU94: Booted secondary processor 0x00002f0200 [0x481fd010]
[297267.869685] Detected VIPT I-cache on CPU95
[297267.869731] GICv3: CPU95: found redistributor 2f0300 region
95:0x00002000ae8c0000
[297267.869817] CPU95: Booted secondary processor 0x00002f0300 [0x481fd010]
root@ubuntu:/sys/devices/system/cpu#
> ---
> drivers/base/arch_topology.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> index 981e72a3dafb..ff16a36a908b 100644
> --- a/drivers/base/arch_topology.c
> +++ b/drivers/base/arch_topology.c
> @@ -677,6 +677,8 @@ void remove_cpu_topology(unsigned int cpu)
> cpumask_clear_cpu(cpu, topology_core_cpumask(sibling));
> for_each_cpu(sibling, topology_sibling_cpumask(cpu))
> cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
> + for_each_cpu(sibling, topology_cluster_cpumask(cpu))
> + cpumask_clear_cpu(cpu, topology_cluster_cpumask(sibling));
> for_each_cpu(sibling, topology_llc_cpumask(cpu))
> cpumask_clear_cpu(cpu, topology_llc_cpumask(sibling));
>
> --
> 2.25.1
>
Thanks
Barry
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
[not found] ` <943fef84-3920-42bc-b83f-4feaa3ab79f3@huawei.com>
@ 2021-11-11 9:07 ` Barry Song
2021-11-11 9:08 ` Barry Song
0 siblings, 1 reply; 12+ messages in thread
From: Barry Song @ 2021-11-11 9:07 UTC (permalink / raw)
To: Wangshaobo (bobo)
Cc: LKML, Rafael J. Wysocki, Greg Kroah-Hartman, Sudeep Holla,
Peter Zijlstra, cj.chengjian, huawei.libin, weiyongjun1
On Thu, Nov 11, 2021 at 8:25 PM Wangshaobo (bobo)
<bobo.shaobowang@huawei.com> wrote:
>
>
> 在 2021/11/11 14:25, Barry Song 写道:
>
> On Wed, Nov 10, 2021 at 10:53 PM Wang ShaoBo <bobo.shaobowang@huawei.com> wrote:
>
> When testing cpu online and offline, warning happened like this:
>
> [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> [ 146.752860] sp : ffff800040a83a80
> [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> [ 146.757070] Call trace:
> [ 146.757421] build_sched_domains+0x81c/0x11b0
> [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> [ 146.758460] rebuild_sched_domains+0x2c/0x48
> [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> [ 146.759114] process_one_work+0x1f4/0x480
> [ 146.759429] worker_thread+0x48/0x460
> [ 146.759734] kthread+0x158/0x168
> [ 146.760030] ret_from_fork+0x10/0x20
> [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
>
> For some architectures like risc-v and arm64 which use common code
> clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> is set, cluster_sibling in cpu_topology of each sibling adjacent
> to CPUx is missed clearing, this causes checking failed in
> topology_span_sane() and rebuilding topology failure at end when CPU online.
>
> Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> (CPU 92, 93, 94, 95 are in one cluster):
>
> Before revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [92-95] [92-95] [92-95]
>
> After revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [93-95] [93-95] [93-95]
>
> Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
>
> The patch looks correct. But how do you reproduce it?
>
> Hi Barry,
>
> you can try this test case in kunpeng 920:
>
>
echo 0 > cpu92/online
echo 0 > cpu93/online
echo 1 > cpu92/online
Yes. I was making the whole cluster offline. this warning can only be
reproduced when
we disable a part of CPUs in one cluster, then enable one of the disabled CPUs.
Acked-by: Barry Song <song.bao.hua@hisilicon.com>
Might need some refine to explain how to reproduce in commit log.
>
> - Wang ShaoBo
Thanks
Barry
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-11 9:07 ` Barry Song
@ 2021-11-11 9:08 ` Barry Song
2021-11-11 12:04 ` Dietmar Eggemann
0 siblings, 1 reply; 12+ messages in thread
From: Barry Song @ 2021-11-11 9:08 UTC (permalink / raw)
To: Wangshaobo (bobo)
Cc: LKML, Rafael J. Wysocki, Greg Kroah-Hartman, Sudeep Holla,
Peter Zijlstra, cj.chengjian, huawei.libin, weiyongjun1
On Thu, Nov 11, 2021 at 10:07 PM Barry Song <21cnbao@gmail.com> wrote:
>
> On Thu, Nov 11, 2021 at 8:25 PM Wangshaobo (bobo)
> <bobo.shaobowang@huawei.com> wrote:
> >
> >
> > 在 2021/11/11 14:25, Barry Song 写道:
> >
> > On Wed, Nov 10, 2021 at 10:53 PM Wang ShaoBo <bobo.shaobowang@huawei.com> wrote:
> >
> > When testing cpu online and offline, warning happened like this:
> >
> > [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> > [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> > [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> > [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> > [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> > [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> > [ 146.752860] sp : ffff800040a83a80
> > [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> > [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> > [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> > [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> > [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> > [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> > [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> > [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> > [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> > [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> > [ 146.757070] Call trace:
> > [ 146.757421] build_sched_domains+0x81c/0x11b0
> > [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> > [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> > [ 146.758460] rebuild_sched_domains+0x2c/0x48
> > [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> > [ 146.759114] process_one_work+0x1f4/0x480
> > [ 146.759429] worker_thread+0x48/0x460
> > [ 146.759734] kthread+0x158/0x168
> > [ 146.760030] ret_from_fork+0x10/0x20
> > [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
> >
> > For some architectures like risc-v and arm64 which use common code
> > clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> > is set, cluster_sibling in cpu_topology of each sibling adjacent
> > to CPUx is missed clearing, this causes checking failed in
> > topology_span_sane() and rebuilding topology failure at end when CPU online.
> >
> > Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> > (CPU 92, 93, 94, 95 are in one cluster):
> >
> > Before revision:
> > CPU [92] [93] [94] [95]
> > cluster_sibling [92] [92-95] [92-95] [92-95]
> >
> > After revision:
> > CPU [92] [93] [94] [95]
> > cluster_sibling [92] [93-95] [93-95] [93-95]
> >
> > Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
> >
> > The patch looks correct. But how do you reproduce it?
> >
> > Hi Barry,
> >
> > you can try this test case in kunpeng 920:
> >
> >
> echo 0 > cpu92/online
> echo 0 > cpu93/online
> echo 1 > cpu92/online
>
> Yes. I was making the whole cluster offline. this warning can only be
> reproduced when
> we disable a part of CPUs in one cluster, then enable one of the disabled CPUs.
>
> Acked-by: Barry Song <song.bao.hua@hisilicon.com>
>
> Might need some refine to explain how to reproduce in commit log.
and also a fix tag.
>
> >
> > - Wang ShaoBo
>
> Thanks
> Barry
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-11 9:08 ` Barry Song
@ 2021-11-11 12:04 ` Dietmar Eggemann
0 siblings, 0 replies; 12+ messages in thread
From: Dietmar Eggemann @ 2021-11-11 12:04 UTC (permalink / raw)
To: Barry Song, Wangshaobo (bobo)
Cc: LKML, Rafael J. Wysocki, Greg Kroah-Hartman, Sudeep Holla,
Peter Zijlstra, cj.chengjian, huawei.libin, weiyongjun1
On 11/11/2021 10:08, Barry Song wrote:
> On Thu, Nov 11, 2021 at 10:07 PM Barry Song <21cnbao@gmail.com> wrote:
>>
>> On Thu, Nov 11, 2021 at 8:25 PM Wangshaobo (bobo)
>> <bobo.shaobowang@huawei.com> wrote:
>>>
>>>
>>> 在 2021/11/11 14:25, Barry Song 写道:
>>>
>>> On Wed, Nov 10, 2021 at 10:53 PM Wang ShaoBo <bobo.shaobowang@huawei.com> wrote:
>>>
>>> When testing cpu online and offline, warning happened like this:
>>>
>>> [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
>>> [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
>>> [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
>>> [ 146.751213] Workqueue: events cpuset_hotplug_workfn
>>> [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
>>> [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
>>> [ 146.752461] lr : build_sched_domains+0x414/0x11b0
>>> [ 146.752860] sp : ffff800040a83a80
>>> [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
>>> [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
>>> [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
>>> [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
>>> [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
>>> [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
>>> [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
>>> [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
>>> [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
>>> [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
>>> [ 146.757070] Call trace:
>>> [ 146.757421] build_sched_domains+0x81c/0x11b0
>>> [ 146.757771] partition_sched_domains_locked+0x57c/0x978
>>> [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
>>> [ 146.758460] rebuild_sched_domains+0x2c/0x48
>>> [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
>>> [ 146.759114] process_one_work+0x1f4/0x480
>>> [ 146.759429] worker_thread+0x48/0x460
>>> [ 146.759734] kthread+0x158/0x168
>>> [ 146.760030] ret_from_fork+0x10/0x20
>>> [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
>>>
>>> For some architectures like risc-v and arm64 which use common code
>>> clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
>>> is set, cluster_sibling in cpu_topology of each sibling adjacent
>>> to CPUx is missed clearing, this causes checking failed in
>>> topology_span_sane() and rebuilding topology failure at end when CPU online.
>>>
>>> Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
>>> (CPU 92, 93, 94, 95 are in one cluster):
>>>
>>> Before revision:
>>> CPU [92] [93] [94] [95]
>>> cluster_sibling [92] [92-95] [92-95] [92-95]
>>>
>>> After revision:
>>> CPU [92] [93] [94] [95]
>>> cluster_sibling [92] [93-95] [93-95] [93-95]
>>>
>>> Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
>>>
>>> The patch looks correct. But how do you reproduce it?
>>>
>>> Hi Barry,
>>>
>>> you can try this test case in kunpeng 920:
>>>
>>>
>> echo 0 > cpu92/online
>> echo 0 > cpu93/online
>> echo 1 > cpu92/online
>>
>> Yes. I was making the whole cluster offline. this warning can only be
>> reproduced when
>> we disable a part of CPUs in one cluster, then enable one of the disabled CPUs.
I was asking myself the same question. When does this warning
`WARN_ON(!topology_span_sane(tl, cpu_map, i)` in build_sched_domains()
actually barf? Agree with what's said above.
echo 1 > cpu92/online:
...
[ 113.841009] Detected VIPT I-cache on CPU92
[ 113.845292] GICv3: CPU92: found redistributor 3d0000 region 92:0x00002000aa600000
[ 113.853059] CPU92: Booted secondary processor 0x00003d0000 [0x481fd010]
[ 114.018654] CPU92 i=94 cpu_map=0-92,94-95 tl->mask(cpu)=92,94-95 tl->mask(i)=92-95 <- !!!
[ 114.026240] ------------[ cut here ]------------
[ 114.030841] WARNING: CPU: 92 PID: 590 at kernel/sched/topology.c:2220 build_sched_domains+0x934/0x12f8
...
>>
>> Acked-by: Barry Song <song.bao.hua@hisilicon.com>
>>
>> Might need some refine to explain how to reproduce in commit log.
+1.
> and also a fix tag.
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Tested-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [tip: sched/urgent] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-10 9:58 [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology() Wang ShaoBo
2021-11-11 6:25 ` Barry Song
@ 2021-11-11 12:22 ` tip-bot2 for Wang ShaoBo
2021-11-11 14:04 ` Wangshaobo (bobo)
2021-11-11 12:46 ` [PATCH] " Sudeep Holla
2021-11-26 16:28 ` Greg KH
3 siblings, 1 reply; 12+ messages in thread
From: tip-bot2 for Wang ShaoBo @ 2021-11-11 12:22 UTC (permalink / raw)
To: linux-tip-commits
Cc: Wang ShaoBo, Peter Zijlstra (Intel),
Dietmar Eggemann, Barry Song, x86, linux-kernel
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: 4cc4cc28ec4154c4f1395648ab67ac9fd3e71fdc
Gitweb: https://git.kernel.org/tip/4cc4cc28ec4154c4f1395648ab67ac9fd3e71fdc
Author: Wang ShaoBo <bobo.shaobowang@huawei.com>
AuthorDate: Wed, 10 Nov 2021 17:58:56 +08:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Thu, 11 Nov 2021 13:09:33 +01:00
arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
When testing cpu online and offline, warning happened like this:
[ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
[ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
[ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
[ 146.751213] Workqueue: events cpuset_hotplug_workfn
[ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 146.752048] pc : build_sched_domains+0x81c/0x11b0
[ 146.752461] lr : build_sched_domains+0x414/0x11b0
[ 146.752860] sp : ffff800040a83a80
[ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
[ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
[ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
[ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
[ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
[ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
[ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
[ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
[ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
[ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
[ 146.757070] Call trace:
[ 146.757421] build_sched_domains+0x81c/0x11b0
[ 146.757771] partition_sched_domains_locked+0x57c/0x978
[ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
[ 146.758460] rebuild_sched_domains+0x2c/0x48
[ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
[ 146.759114] process_one_work+0x1f4/0x480
[ 146.759429] worker_thread+0x48/0x460
[ 146.759734] kthread+0x158/0x168
[ 146.760030] ret_from_fork+0x10/0x20
[ 146.760318] ---[ end trace 82c44aad6900e81a ]---
For some architectures like risc-v and arm64 which use common code
clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
is set, cluster_sibling in cpu_topology of each sibling adjacent
to CPUx is missed clearing, this causes checking failed in
topology_span_sane() and rebuilding topology failure at end when CPU online.
Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
(CPU 92, 93, 94, 95 are in one cluster):
Before revision:
CPU [92] [93] [94] [95]
cluster_sibling [92] [92-95] [92-95] [92-95]
After revision:
CPU [92] [93] [94] [95]
cluster_sibling [92] [93-95] [93-95] [93-95]
Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Barry Song <song.bao.hua@hisilicon.com>
Tested-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Link: https://lore.kernel.org/r/20211110095856.469360-1-bobo.shaobowang@huawei.com
---
drivers/base/arch_topology.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 981e72a..ff16a36 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -677,6 +677,8 @@ void remove_cpu_topology(unsigned int cpu)
cpumask_clear_cpu(cpu, topology_core_cpumask(sibling));
for_each_cpu(sibling, topology_sibling_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
+ for_each_cpu(sibling, topology_cluster_cpumask(cpu))
+ cpumask_clear_cpu(cpu, topology_cluster_cpumask(sibling));
for_each_cpu(sibling, topology_llc_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_llc_cpumask(sibling));
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-10 9:58 [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology() Wang ShaoBo
2021-11-11 6:25 ` Barry Song
2021-11-11 12:22 ` [tip: sched/urgent] " tip-bot2 for Wang ShaoBo
@ 2021-11-11 12:46 ` Sudeep Holla
2021-11-26 16:28 ` Greg KH
3 siblings, 0 replies; 12+ messages in thread
From: Sudeep Holla @ 2021-11-11 12:46 UTC (permalink / raw)
To: Wang ShaoBo
Cc: linux-kernel, rafael, gregkh, peterz, cj.chengjian, huawei.libin,
weiyongjun1
On Wed, Nov 10, 2021 at 05:58:56PM +0800, Wang ShaoBo wrote:
> When testing cpu online and offline, warning happened like this:
>
> [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> [ 146.752860] sp : ffff800040a83a80
> [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> [ 146.757070] Call trace:
> [ 146.757421] build_sched_domains+0x81c/0x11b0
> [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> [ 146.758460] rebuild_sched_domains+0x2c/0x48
> [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> [ 146.759114] process_one_work+0x1f4/0x480
> [ 146.759429] worker_thread+0x48/0x460
> [ 146.759734] kthread+0x158/0x168
> [ 146.760030] ret_from_fork+0x10/0x20
> [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
>
> For some architectures like risc-v and arm64 which use common code
> clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> is set, cluster_sibling in cpu_topology of each sibling adjacent
> to CPUx is missed clearing, this causes checking failed in
> topology_span_sane() and rebuilding topology failure at end when CPU online.
>
> Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> (CPU 92, 93, 94, 95 are in one cluster):
>
> Before revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [92-95] [92-95] [92-95]
>
> After revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [93-95] [93-95] [93-95]
>
Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
--
Regards,
Sudeep
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [tip: sched/urgent] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-11 12:22 ` [tip: sched/urgent] " tip-bot2 for Wang ShaoBo
@ 2021-11-11 14:04 ` Wangshaobo (bobo)
0 siblings, 0 replies; 12+ messages in thread
From: Wangshaobo (bobo) @ 2021-11-11 14:04 UTC (permalink / raw)
To: Peter Zijlstra (Intel)
Cc: linux-kernel, linux-tip-commits, Dietmar Eggemann, Barry Song,
x86, chengjian (D), Libin (Huawei)
Hi Peter,
I have sent v2 with adding Fixes flag and reproduce method,
merging v2 version patch can be more appropriate.
Thanks.
- Wang ShaoBo
在 2021/11/11 20:22, tip-bot2 for Wang ShaoBo 写道:
> The following commit has been merged into the sched/urgent branch of tip:
>
> Commit-ID: 4cc4cc28ec4154c4f1395648ab67ac9fd3e71fdc
> Gitweb: https://git.kernel.org/tip/4cc4cc28ec4154c4f1395648ab67ac9fd3e71fdc
> Author: Wang ShaoBo <bobo.shaobowang@huawei.com>
> AuthorDate: Wed, 10 Nov 2021 17:58:56 +08:00
> Committer: Peter Zijlstra <peterz@infradead.org>
> CommitterDate: Thu, 11 Nov 2021 13:09:33 +01:00
>
> arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
>
> When testing cpu online and offline, warning happened like this:
>
> [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> [ 146.752860] sp : ffff800040a83a80
> [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> [ 146.757070] Call trace:
> [ 146.757421] build_sched_domains+0x81c/0x11b0
> [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> [ 146.758460] rebuild_sched_domains+0x2c/0x48
> [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> [ 146.759114] process_one_work+0x1f4/0x480
> [ 146.759429] worker_thread+0x48/0x460
> [ 146.759734] kthread+0x158/0x168
> [ 146.760030] ret_from_fork+0x10/0x20
> [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
>
> For some architectures like risc-v and arm64 which use common code
> clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> is set, cluster_sibling in cpu_topology of each sibling adjacent
> to CPUx is missed clearing, this causes checking failed in
> topology_span_sane() and rebuilding topology failure at end when CPU online.
>
> Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> (CPU 92, 93, 94, 95 are in one cluster):
>
> Before revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [92-95] [92-95] [92-95]
>
> After revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [93-95] [93-95] [93-95]
>
> Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Acked-by: Barry Song <song.bao.hua@hisilicon.com>
> Tested-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Link: https://lore.kernel.org/r/20211110095856.469360-1-bobo.shaobowang@huawei.com
> ---
> drivers/base/arch_topology.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> index 981e72a..ff16a36 100644
> --- a/drivers/base/arch_topology.c
> +++ b/drivers/base/arch_topology.c
> @@ -677,6 +677,8 @@ void remove_cpu_topology(unsigned int cpu)
> cpumask_clear_cpu(cpu, topology_core_cpumask(sibling));
> for_each_cpu(sibling, topology_sibling_cpumask(cpu))
> cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
> + for_each_cpu(sibling, topology_cluster_cpumask(cpu))
> + cpumask_clear_cpu(cpu, topology_cluster_cpumask(sibling));
> for_each_cpu(sibling, topology_llc_cpumask(cpu))
> cpumask_clear_cpu(cpu, topology_llc_cpumask(sibling));
>
> .
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-10 9:58 [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology() Wang ShaoBo
` (2 preceding siblings ...)
2021-11-11 12:46 ` [PATCH] " Sudeep Holla
@ 2021-11-26 16:28 ` Greg KH
2021-11-26 18:39 ` Sudeep Holla
3 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2021-11-26 16:28 UTC (permalink / raw)
To: Wang ShaoBo
Cc: linux-kernel, rafael, sudeep.holla, peterz, cj.chengjian,
huawei.libin, weiyongjun1
On Wed, Nov 10, 2021 at 05:58:56PM +0800, Wang ShaoBo wrote:
> When testing cpu online and offline, warning happened like this:
>
> [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> [ 146.752860] sp : ffff800040a83a80
> [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> [ 146.757070] Call trace:
> [ 146.757421] build_sched_domains+0x81c/0x11b0
> [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> [ 146.758460] rebuild_sched_domains+0x2c/0x48
> [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> [ 146.759114] process_one_work+0x1f4/0x480
> [ 146.759429] worker_thread+0x48/0x460
> [ 146.759734] kthread+0x158/0x168
> [ 146.760030] ret_from_fork+0x10/0x20
> [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
>
> For some architectures like risc-v and arm64 which use common code
> clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> is set, cluster_sibling in cpu_topology of each sibling adjacent
> to CPUx is missed clearing, this causes checking failed in
> topology_span_sane() and rebuilding topology failure at end when CPU online.
>
> Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> (CPU 92, 93, 94, 95 are in one cluster):
>
> Before revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [92-95] [92-95] [92-95]
>
> After revision:
> CPU [92] [93] [94] [95]
> cluster_sibling [92] [93-95] [93-95] [93-95]
>
> Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
> ---
> drivers/base/arch_topology.c | 2 ++
> 1 file changed, 2 insertions(+)
What commit id does this fix?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-26 16:28 ` Greg KH
@ 2021-11-26 18:39 ` Sudeep Holla
2021-11-27 9:07 ` Greg KH
0 siblings, 1 reply; 12+ messages in thread
From: Sudeep Holla @ 2021-11-26 18:39 UTC (permalink / raw)
To: Greg KH
Cc: Wang ShaoBo, linux-kernel, rafael, peterz, cj.chengjian,
huawei.libin, weiyongjun1, Sudeep Holla
On Fri, Nov 26, 2021 at 05:28:19PM +0100, Greg KH wrote:
> On Wed, Nov 10, 2021 at 05:58:56PM +0800, Wang ShaoBo wrote:
> > When testing cpu online and offline, warning happened like this:
> >
> > [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> > [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> > [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> > [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> > [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> > [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> > [ 146.752860] sp : ffff800040a83a80
> > [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> > [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> > [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> > [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> > [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> > [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> > [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> > [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> > [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> > [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> > [ 146.757070] Call trace:
> > [ 146.757421] build_sched_domains+0x81c/0x11b0
> > [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> > [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> > [ 146.758460] rebuild_sched_domains+0x2c/0x48
> > [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> > [ 146.759114] process_one_work+0x1f4/0x480
> > [ 146.759429] worker_thread+0x48/0x460
> > [ 146.759734] kthread+0x158/0x168
> > [ 146.760030] ret_from_fork+0x10/0x20
> > [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
> >
> > For some architectures like risc-v and arm64 which use common code
> > clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> > is set, cluster_sibling in cpu_topology of each sibling adjacent
> > to CPUx is missed clearing, this causes checking failed in
> > topology_span_sane() and rebuilding topology failure at end when CPU online.
> >
> > Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> > (CPU 92, 93, 94, 95 are in one cluster):
> >
> > Before revision:
> > CPU [92] [93] [94] [95]
> > cluster_sibling [92] [92-95] [92-95] [92-95]
> >
> > After revision:
> > CPU [92] [93] [94] [95]
> > cluster_sibling [92] [93-95] [93-95] [93-95]
> >
> > Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
> > ---
> > drivers/base/arch_topology.c | 2 ++
> > 1 file changed, 2 insertions(+)
>
> What commit id does this fix?
>
v2[1] has the information and all the tags IIUC.
--
Regards,
Sudeep
[1] https://lore.kernel.org/all/20211111140435.3209001-1-bobo.shaobowang@huawei.com/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-26 18:39 ` Sudeep Holla
@ 2021-11-27 9:07 ` Greg KH
2021-11-30 1:08 ` Barry Song
0 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2021-11-27 9:07 UTC (permalink / raw)
To: Sudeep Holla
Cc: Wang ShaoBo, linux-kernel, rafael, peterz, cj.chengjian,
huawei.libin, weiyongjun1
On Fri, Nov 26, 2021 at 06:39:54PM +0000, Sudeep Holla wrote:
> On Fri, Nov 26, 2021 at 05:28:19PM +0100, Greg KH wrote:
> > On Wed, Nov 10, 2021 at 05:58:56PM +0800, Wang ShaoBo wrote:
> > > When testing cpu online and offline, warning happened like this:
> > >
> > > [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> > > [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> > > [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> > > [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> > > [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > > [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> > > [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> > > [ 146.752860] sp : ffff800040a83a80
> > > [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> > > [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> > > [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> > > [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> > > [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> > > [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> > > [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> > > [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> > > [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> > > [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> > > [ 146.757070] Call trace:
> > > [ 146.757421] build_sched_domains+0x81c/0x11b0
> > > [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> > > [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> > > [ 146.758460] rebuild_sched_domains+0x2c/0x48
> > > [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> > > [ 146.759114] process_one_work+0x1f4/0x480
> > > [ 146.759429] worker_thread+0x48/0x460
> > > [ 146.759734] kthread+0x158/0x168
> > > [ 146.760030] ret_from_fork+0x10/0x20
> > > [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
> > >
> > > For some architectures like risc-v and arm64 which use common code
> > > clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> > > is set, cluster_sibling in cpu_topology of each sibling adjacent
> > > to CPUx is missed clearing, this causes checking failed in
> > > topology_span_sane() and rebuilding topology failure at end when CPU online.
> > >
> > > Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> > > (CPU 92, 93, 94, 95 are in one cluster):
> > >
> > > Before revision:
> > > CPU [92] [93] [94] [95]
> > > cluster_sibling [92] [92-95] [92-95] [92-95]
> > >
> > > After revision:
> > > CPU [92] [93] [94] [95]
> > > cluster_sibling [92] [93-95] [93-95] [93-95]
> > >
> > > Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
> > > ---
> > > drivers/base/arch_topology.c | 2 ++
> > > 1 file changed, 2 insertions(+)
> >
> > What commit id does this fix?
> >
>
> v2[1] has the information and all the tags IIUC.
Odd, I don't see that in my queue :(
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology()
2021-11-27 9:07 ` Greg KH
@ 2021-11-30 1:08 ` Barry Song
0 siblings, 0 replies; 12+ messages in thread
From: Barry Song @ 2021-11-30 1:08 UTC (permalink / raw)
To: Greg KH
Cc: Sudeep Holla, Wang ShaoBo, LKML, Rafael J. Wysocki,
Peter Zijlstra, cj.chengjian, huawei.libin, weiyongjun1
On Tue, Nov 30, 2021 at 8:21 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Fri, Nov 26, 2021 at 06:39:54PM +0000, Sudeep Holla wrote:
> > On Fri, Nov 26, 2021 at 05:28:19PM +0100, Greg KH wrote:
> > > On Wed, Nov 10, 2021 at 05:58:56PM +0800, Wang ShaoBo wrote:
> > > > When testing cpu online and offline, warning happened like this:
> > > >
> > > > [ 146.746743] WARNING: CPU: 92 PID: 974 at kernel/sched/topology.c:2215 build_sched_domains+0x81c/0x11b0
> > > > [ 146.749988] CPU: 92 PID: 974 Comm: kworker/92:2 Not tainted 5.15.0 #9
> > > > [ 146.750402] Hardware name: Huawei TaiShan 2280 V2/BC82AMDDA, BIOS 1.79 08/21/2021
> > > > [ 146.751213] Workqueue: events cpuset_hotplug_workfn
> > > > [ 146.751629] pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > > > [ 146.752048] pc : build_sched_domains+0x81c/0x11b0
> > > > [ 146.752461] lr : build_sched_domains+0x414/0x11b0
> > > > [ 146.752860] sp : ffff800040a83a80
> > > > [ 146.753247] x29: ffff800040a83a80 x28: ffff20801f13a980 x27: ffff20800448ae00
> > > > [ 146.753644] x26: ffff800012a858e8 x25: ffff800012ea48c0 x24: 0000000000000000
> > > > [ 146.754039] x23: ffff800010ab7d60 x22: ffff800012f03758 x21: 000000000000005f
> > > > [ 146.754427] x20: 000000000000005c x19: ffff004080012840 x18: ffffffffffffffff
> > > > [ 146.754814] x17: 3661613030303230 x16: 30303078303a3239 x15: ffff800011f92b48
> > > > [ 146.755197] x14: ffff20be3f95cef6 x13: 2e6e69616d6f642d x12: 6465686373204c4c
> > > > [ 146.755578] x11: ffff20bf7fc83a00 x10: 0000000000000040 x9 : 0000000000000000
> > > > [ 146.755957] x8 : 0000000000000002 x7 : ffffffffe0000000 x6 : 0000000000000002
> > > > [ 146.756334] x5 : 0000000090000000 x4 : 00000000f0000000 x3 : 0000000000000001
> > > > [ 146.756705] x2 : 0000000000000080 x1 : ffff800012f03860 x0 : 0000000000000001
> > > > [ 146.757070] Call trace:
> > > > [ 146.757421] build_sched_domains+0x81c/0x11b0
> > > > [ 146.757771] partition_sched_domains_locked+0x57c/0x978
> > > > [ 146.758118] rebuild_sched_domains_locked+0x44c/0x7f0
> > > > [ 146.758460] rebuild_sched_domains+0x2c/0x48
> > > > [ 146.758791] cpuset_hotplug_workfn+0x3fc/0x888
> > > > [ 146.759114] process_one_work+0x1f4/0x480
> > > > [ 146.759429] worker_thread+0x48/0x460
> > > > [ 146.759734] kthread+0x158/0x168
> > > > [ 146.760030] ret_from_fork+0x10/0x20
> > > > [ 146.760318] ---[ end trace 82c44aad6900e81a ]---
> > > >
> > > > For some architectures like risc-v and arm64 which use common code
> > > > clear_cpu_topology() in shutting down CPUx, When CONFIG_SCHED_CLUSTER
> > > > is set, cluster_sibling in cpu_topology of each sibling adjacent
> > > > to CPUx is missed clearing, this causes checking failed in
> > > > topology_span_sane() and rebuilding topology failure at end when CPU online.
> > > >
> > > > Different sibling's cluster_sibling in cpu_topology[] when CPU92 offline
> > > > (CPU 92, 93, 94, 95 are in one cluster):
> > > >
> > > > Before revision:
> > > > CPU [92] [93] [94] [95]
> > > > cluster_sibling [92] [92-95] [92-95] [92-95]
> > > >
> > > > After revision:
> > > > CPU [92] [93] [94] [95]
> > > > cluster_sibling [92] [93-95] [93-95] [93-95]
> > > >
> > > > Signed-off-by: Wang ShaoBo <bobo.shaobowang@huawei.com>
> > > > ---
> > > > drivers/base/arch_topology.c | 2 ++
> > > > 1 file changed, 2 insertions(+)
> > >
> > > What commit id does this fix?
> > >
> >
> > v2[1] has the information and all the tags IIUC.
>
> Odd, I don't see that in my queue :(
Greg, this commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cc4cc28ec4154c4f1395648ab67ac9fd3e71fdc
which is shaobo's v1 patch lacking fixes tag.
Thanks
Barry
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-11-30 1:08 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-10 9:58 [PATCH] arch_topology: Fix missing clear cluster_cpumask in remove_cpu_topology() Wang ShaoBo
2021-11-11 6:25 ` Barry Song
[not found] ` <943fef84-3920-42bc-b83f-4feaa3ab79f3@huawei.com>
2021-11-11 9:07 ` Barry Song
2021-11-11 9:08 ` Barry Song
2021-11-11 12:04 ` Dietmar Eggemann
2021-11-11 12:22 ` [tip: sched/urgent] " tip-bot2 for Wang ShaoBo
2021-11-11 14:04 ` Wangshaobo (bobo)
2021-11-11 12:46 ` [PATCH] " Sudeep Holla
2021-11-26 16:28 ` Greg KH
2021-11-26 18:39 ` Sudeep Holla
2021-11-27 9:07 ` Greg KH
2021-11-30 1:08 ` Barry Song
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).