* [PATCH 1/2] cpu/hotplug: Mute hotplug lockdep during init
2018-12-19 18:23 [PATCH 0/2] System-wide mute of hotplug lockdep during init Valentin Schneider
@ 2018-12-19 18:23 ` Valentin Schneider
2018-12-19 18:31 ` Marc Zyngier
2018-12-19 18:23 ` [PATCH 2/2] Revert "sched/core: Take the hotplug lock in sched_init_smp()" Valentin Schneider
2019-01-11 9:53 ` [PATCH 0/2] System-wide mute of hotplug lockdep during init Valentin Schneider
2 siblings, 1 reply; 6+ messages in thread
From: Valentin Schneider @ 2018-12-19 18:23 UTC (permalink / raw)
To: linux-kernel
Cc: mark.rutland, cai, marc.zyngier, daniel.lezcano,
dietmar.eggemann, peterz, longman, tglx, mingo, linux-arm-kernel
Since we've had:
commit cb538267ea1e ("jump_label/lockdep: Assert we hold the hotplug lock for _cpuslocked() operations")
we've been getting some lockdep warnings during init, such as on HiKey960:
[ 0.820495] WARNING: CPU: 4 PID: 0 at kernel/cpu.c:316 lockdep_assert_cpus_held+0x3c/0x48
[ 0.820498] Modules linked in:
[ 0.820509] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G S 4.20.0-rc5-00051-g4cae42a #34
[ 0.820511] Hardware name: HiKey960 (DT)
[ 0.820516] pstate: 600001c5 (nZCv dAIF -PAN -UAO)
[ 0.820520] pc : lockdep_assert_cpus_held+0x3c/0x48
[ 0.820523] lr : lockdep_assert_cpus_held+0x38/0x48
[ 0.820526] sp : ffff00000a9cbe50
[ 0.820528] x29: ffff00000a9cbe50 x28: 0000000000000000
[ 0.820533] x27: 00008000b69e5000 x26: ffff8000bff4cfe0
[ 0.820537] x25: ffff000008ba69e0 x24: 0000000000000001
[ 0.820541] x23: ffff000008fce000 x22: ffff000008ba70c8
[ 0.820545] x21: 0000000000000001 x20: 0000000000000003
[ 0.820548] x19: ffff00000a35d628 x18: ffffffffffffffff
[ 0.820552] x17: 0000000000000000 x16: 0000000000000000
[ 0.820556] x15: ffff00000958f848 x14: 455f3052464d4d34
[ 0.820559] x13: 00000000769dde98 x12: ffff8000bf3f65a8
[ 0.820564] x11: 0000000000000000 x10: ffff00000958f848
[ 0.820567] x9 : ffff000009592000 x8 : ffff00000958f848
[ 0.820571] x7 : ffff00000818ffa0 x6 : 0000000000000000
[ 0.820574] x5 : 0000000000000000 x4 : 0000000000000001
[ 0.820578] x3 : 0000000000000000 x2 : 0000000000000001
[ 0.820582] x1 : 00000000ffffffff x0 : 0000000000000000
[ 0.820587] Call trace:
[ 0.820591] lockdep_assert_cpus_held+0x3c/0x48
[ 0.820598] static_key_enable_cpuslocked+0x28/0xd0
[ 0.820606] arch_timer_check_ool_workaround+0xe8/0x228
[ 0.820610] arch_timer_starting_cpu+0xe4/0x2d8
[ 0.820615] cpuhp_invoke_callback+0xe8/0xd08
[ 0.820619] notify_cpu_starting+0x80/0xb8
[ 0.820625] secondary_start_kernel+0x118/0x1d0
We've also had a similar warning in sched_init_smp() for every
asymmetric system that would enable the sched_asym_cpucapacity static
key, although that was singled out in:
commit 40fa3780bac2 ("sched/core: Take the hotplug lock in sched_init_smp()")
Those warnings are actually harmless, since we cannot have hotplug
operations at the time they appear. Instead of starting to sprinkle
useless hotplug lock operations in the init codepaths, mute the
warnings until they start warning about real problems.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
FYI Thomas Gleixner suggested using SYSTEM_SCHEDULING instead of
SYSTEM_RUNNING, but that seems to still be too early - sched_init_smp()
(and kernel_init() actually) hasn't completed yet, so we'd still get
those warnings.
---
kernel/cpu.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 91d5c38..34e40ef 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -313,6 +313,15 @@ void cpus_write_unlock(void)
void lockdep_assert_cpus_held(void)
{
+ /*
+ * We can't have hotplug operations before userspace starts running,
+ * and some init codepaths will knowingly not take the hotplug lock.
+ * This is all valid, so mute lockdep until it makes sense to report
+ * unheld locks.
+ */
+ if (system_state < SYSTEM_RUNNING)
+ return;
+
percpu_rwsem_assert_held(&cpu_hotplug_lock);
}
--
2.7.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 1/2] cpu/hotplug: Mute hotplug lockdep during init
2018-12-19 18:23 ` [PATCH 1/2] cpu/hotplug: Mute " Valentin Schneider
@ 2018-12-19 18:31 ` Marc Zyngier
0 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2018-12-19 18:31 UTC (permalink / raw)
To: Valentin Schneider, linux-kernel
Cc: mark.rutland, cai, peterz, daniel.lezcano, dietmar.eggemann,
longman, tglx, mingo, linux-arm-kernel
On 19/12/2018 18:23, Valentin Schneider wrote:
> Since we've had:
>
> commit cb538267ea1e ("jump_label/lockdep: Assert we hold the hotplug lock for _cpuslocked() operations")
>
> we've been getting some lockdep warnings during init, such as on HiKey960:
>
> [ 0.820495] WARNING: CPU: 4 PID: 0 at kernel/cpu.c:316 lockdep_assert_cpus_held+0x3c/0x48
> [ 0.820498] Modules linked in:
> [ 0.820509] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G S 4.20.0-rc5-00051-g4cae42a #34
> [ 0.820511] Hardware name: HiKey960 (DT)
> [ 0.820516] pstate: 600001c5 (nZCv dAIF -PAN -UAO)
> [ 0.820520] pc : lockdep_assert_cpus_held+0x3c/0x48
> [ 0.820523] lr : lockdep_assert_cpus_held+0x38/0x48
> [ 0.820526] sp : ffff00000a9cbe50
> [ 0.820528] x29: ffff00000a9cbe50 x28: 0000000000000000
> [ 0.820533] x27: 00008000b69e5000 x26: ffff8000bff4cfe0
> [ 0.820537] x25: ffff000008ba69e0 x24: 0000000000000001
> [ 0.820541] x23: ffff000008fce000 x22: ffff000008ba70c8
> [ 0.820545] x21: 0000000000000001 x20: 0000000000000003
> [ 0.820548] x19: ffff00000a35d628 x18: ffffffffffffffff
> [ 0.820552] x17: 0000000000000000 x16: 0000000000000000
> [ 0.820556] x15: ffff00000958f848 x14: 455f3052464d4d34
> [ 0.820559] x13: 00000000769dde98 x12: ffff8000bf3f65a8
> [ 0.820564] x11: 0000000000000000 x10: ffff00000958f848
> [ 0.820567] x9 : ffff000009592000 x8 : ffff00000958f848
> [ 0.820571] x7 : ffff00000818ffa0 x6 : 0000000000000000
> [ 0.820574] x5 : 0000000000000000 x4 : 0000000000000001
> [ 0.820578] x3 : 0000000000000000 x2 : 0000000000000001
> [ 0.820582] x1 : 00000000ffffffff x0 : 0000000000000000
> [ 0.820587] Call trace:
> [ 0.820591] lockdep_assert_cpus_held+0x3c/0x48
> [ 0.820598] static_key_enable_cpuslocked+0x28/0xd0
> [ 0.820606] arch_timer_check_ool_workaround+0xe8/0x228
> [ 0.820610] arch_timer_starting_cpu+0xe4/0x2d8
> [ 0.820615] cpuhp_invoke_callback+0xe8/0xd08
> [ 0.820619] notify_cpu_starting+0x80/0xb8
> [ 0.820625] secondary_start_kernel+0x118/0x1d0
>
> We've also had a similar warning in sched_init_smp() for every
> asymmetric system that would enable the sched_asym_cpucapacity static
> key, although that was singled out in:
>
> commit 40fa3780bac2 ("sched/core: Take the hotplug lock in sched_init_smp()")
>
> Those warnings are actually harmless, since we cannot have hotplug
> operations at the time they appear. Instead of starting to sprinkle
> useless hotplug lock operations in the init codepaths, mute the
> warnings until they start warning about real problems.
>
> Suggested-by: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
> ---
> FYI Thomas Gleixner suggested using SYSTEM_SCHEDULING instead of
> SYSTEM_RUNNING, but that seems to still be too early - sched_init_smp()
> (and kernel_init() actually) hasn't completed yet, so we'd still get
> those warnings.
> ---
> kernel/cpu.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 91d5c38..34e40ef 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -313,6 +313,15 @@ void cpus_write_unlock(void)
>
> void lockdep_assert_cpus_held(void)
> {
> + /*
> + * We can't have hotplug operations before userspace starts running,
> + * and some init codepaths will knowingly not take the hotplug lock.
> + * This is all valid, so mute lockdep until it makes sense to report
> + * unheld locks.
> + */
> + if (system_state < SYSTEM_RUNNING)
> + return;
> +
> percpu_rwsem_assert_held(&cpu_hotplug_lock);
> }
>
> --
> 2.7.4
>
FWIW: Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Thanks for having kept an eye on it!
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] Revert "sched/core: Take the hotplug lock in sched_init_smp()"
2018-12-19 18:23 [PATCH 0/2] System-wide mute of hotplug lockdep during init Valentin Schneider
2018-12-19 18:23 ` [PATCH 1/2] cpu/hotplug: Mute " Valentin Schneider
@ 2018-12-19 18:23 ` Valentin Schneider
2019-01-11 9:53 ` [PATCH 0/2] System-wide mute of hotplug lockdep during init Valentin Schneider
2 siblings, 0 replies; 6+ messages in thread
From: Valentin Schneider @ 2018-12-19 18:23 UTC (permalink / raw)
To: linux-kernel
Cc: mark.rutland, cai, marc.zyngier, daniel.lezcano,
dietmar.eggemann, peterz, longman, tglx, mingo, linux-arm-kernel
This reverts commit 40fa3780bac2b654edf23f6b13f4e2dd550aea10.
Now that we have a system-wide muting of hotplug lockdep during init,
this is no longer needed.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
kernel/sched/core.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e4ca15d..144b778 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5854,14 +5854,11 @@ void __init sched_init_smp(void)
/*
* There's no userspace yet to cause hotplug operations; hence all the
* CPU masks are stable and all blatant races in the below code cannot
- * happen. The hotplug lock is nevertheless taken to satisfy lockdep,
- * but there won't be any contention on it.
+ * happen.
*/
- cpus_read_lock();
mutex_lock(&sched_domains_mutex);
sched_init_domains(cpu_active_mask);
mutex_unlock(&sched_domains_mutex);
- cpus_read_unlock();
/* Move init over to a non-isolated CPU */
if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_DOMAIN)) < 0)
--
2.7.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 0/2] System-wide mute of hotplug lockdep during init
2018-12-19 18:23 [PATCH 0/2] System-wide mute of hotplug lockdep during init Valentin Schneider
2018-12-19 18:23 ` [PATCH 1/2] cpu/hotplug: Mute " Valentin Schneider
2018-12-19 18:23 ` [PATCH 2/2] Revert "sched/core: Take the hotplug lock in sched_init_smp()" Valentin Schneider
@ 2019-01-11 9:53 ` Valentin Schneider
2019-01-11 13:01 ` Peter Zijlstra
2 siblings, 1 reply; 6+ messages in thread
From: Valentin Schneider @ 2019-01-11 9:53 UTC (permalink / raw)
To: linux-kernel
Cc: mark.rutland, cai, marc.zyngier, daniel.lezcano,
dietmar.eggemann, peterz, longman, tglx, mingo, linux-arm-kernel
Hi,
On 19/12/2018 18:23, Valentin Schneider wrote:
> In [1] Peter had a suggestion to broadly mute hotplug lockdep warnings during
> init instead of going after every single warning and wrapping them with
> cpus_read_{lock, unlock}(). This is implemented in [PATCH 1/2].
>
> [PATCH 2/2] is a logical follow-up as [PATCH 1/2] is the generalized version
> of the reverted patch.
>
> [1]: https://lore.kernel.org/lkml/1543877121-4098-1-git-send-email-cai@gmx.us/#t
>
Gentle ping - I suppose this flew under the holidays radar.
> Valentin Schneider (2):
> cpu/hotplug: Mute hotplug lockdep during init
> Revert "sched/core: Take the hotplug lock in sched_init_smp()"
>
> kernel/cpu.c | 9 +++++++++
> kernel/sched/core.c | 5 +----
> 2 files changed, 10 insertions(+), 4 deletions(-)
>
> --
> 2.7.4
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 0/2] System-wide mute of hotplug lockdep during init
2019-01-11 9:53 ` [PATCH 0/2] System-wide mute of hotplug lockdep during init Valentin Schneider
@ 2019-01-11 13:01 ` Peter Zijlstra
0 siblings, 0 replies; 6+ messages in thread
From: Peter Zijlstra @ 2019-01-11 13:01 UTC (permalink / raw)
To: Valentin Schneider
Cc: mark.rutland, cai, marc.zyngier, daniel.lezcano, linux-kernel,
dietmar.eggemann, longman, tglx, mingo, linux-arm-kernel
On Fri, Jan 11, 2019 at 09:53:58AM +0000, Valentin Schneider wrote:
> Hi,
>
> On 19/12/2018 18:23, Valentin Schneider wrote:
> > In [1] Peter had a suggestion to broadly mute hotplug lockdep warnings during
> > init instead of going after every single warning and wrapping them with
> > cpus_read_{lock, unlock}(). This is implemented in [PATCH 1/2].
> >
> > [PATCH 2/2] is a logical follow-up as [PATCH 1/2] is the generalized version
> > of the reverted patch.
> >
> > [1]: https://lore.kernel.org/lkml/1543877121-4098-1-git-send-email-cai@gmx.us/#t
> >
>
> Gentle ping - I suppose this flew under the holidays radar.
Indeed so... got them now. Thanks!
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 6+ messages in thread