* [PATCH] notifier: Make atomic_notifiers use raw_spinlock
@ 2020-11-22 20:19 Valentin Schneider
2020-11-23 14:14 ` Peter Zijlstra
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Valentin Schneider @ 2020-11-22 20:19 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: peterz, rostedt, mhiramat, bristot, jbaron, torvalds, tglx,
mingo, namit, hpa, luto, ard.biesheuvel, jpoimboe, pbonzini,
mathieu.desnoyers, linux, Rafael J. Wysocki,
Sebastian Andrzej Siewior, Alex Shi, Daniel Lezcano,
Vincenzo Frascino
Booting a recent PREEMPT_RT kernel (v5.10-rc3-rt7-rebase) on my arm64 Juno
leads to the idle task blocking on an RT sleeping spinlock down some
notifier path:
[ 1.809101] BUG: scheduling while atomic: swapper/5/0/0x00000002
[ 1.809116] Modules linked in:
[ 1.809123] Preemption disabled at:
[ 1.809125] secondary_start_kernel (arch/arm64/kernel/smp.c:227)
[ 1.809146] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G W 5.10.0-rc3-rt7 #168
[ 1.809153] Hardware name: ARM Juno development board (r0) (DT)
[ 1.809158] Call trace:
[ 1.809160] dump_backtrace (arch/arm64/kernel/stacktrace.c:100 (discriminator 1))
[ 1.809170] show_stack (arch/arm64/kernel/stacktrace.c:198)
[ 1.809178] dump_stack (lib/dump_stack.c:122)
[ 1.809188] __schedule_bug (kernel/sched/core.c:4886)
[ 1.809197] __schedule (./arch/arm64/include/asm/preempt.h:18 kernel/sched/core.c:4913 kernel/sched/core.c:5040)
[ 1.809204] preempt_schedule_lock (kernel/sched/core.c:5365 (discriminator 1))
[ 1.809210] rt_spin_lock_slowlock_locked (kernel/locking/rtmutex.c:1072)
[ 1.809217] rt_spin_lock_slowlock (kernel/locking/rtmutex.c:1110)
[ 1.809224] rt_spin_lock (./include/linux/rcupdate.h:647 kernel/locking/rtmutex.c:1139)
[ 1.809231] atomic_notifier_call_chain_robust (kernel/notifier.c:71 kernel/notifier.c:118 kernel/notifier.c:186)
[ 1.809240] cpu_pm_enter (kernel/cpu_pm.c:39 kernel/cpu_pm.c:93)
[ 1.809249] psci_enter_idle_state (drivers/cpuidle/cpuidle-psci.c:52 drivers/cpuidle/cpuidle-psci.c:129)
[ 1.809258] cpuidle_enter_state (drivers/cpuidle/cpuidle.c:238)
[ 1.809267] cpuidle_enter (drivers/cpuidle/cpuidle.c:353)
[ 1.809275] do_idle (kernel/sched/idle.c:132 kernel/sched/idle.c:213 kernel/sched/idle.c:273)
[ 1.809282] cpu_startup_entry (kernel/sched/idle.c:368 (discriminator 1))
[ 1.809288] secondary_start_kernel (arch/arm64/kernel/smp.c:273)
Two points worth noting:
1) That this is conceptually the same issue as pointed out in:
313c8c16ee62 ("PM / CPU: replace raw_notifier with atomic_notifier")
2) Only the _robust() variant of atomic_notifier callchains suffer from
this
AFAICT only the cpu_pm_notifier_chain really needs to be changed, but
singling it out would mean introducing a new (truly) non-blocking API. At
the same time, callers that are fine with any blocking within the call
chain should use blocking notifiers, so patching up all atomic_notifier's
doesn't seem *too* crazy to me.
Fixes: 70d932985757 ("notifier: Fix broken error handling pattern")
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/notifier.h | 6 +++---
kernel/notifier.c | 12 ++++++------
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/include/linux/notifier.h b/include/linux/notifier.h
index 2fb373a5c1ed..723bc2df6388 100644
--- a/include/linux/notifier.h
+++ b/include/linux/notifier.h
@@ -58,7 +58,7 @@ struct notifier_block {
};
struct atomic_notifier_head {
- spinlock_t lock;
+ raw_spinlock_t lock;
struct notifier_block __rcu *head;
};
@@ -78,7 +78,7 @@ struct srcu_notifier_head {
};
#define ATOMIC_INIT_NOTIFIER_HEAD(name) do { \
- spin_lock_init(&(name)->lock); \
+ raw_spin_lock_init(&(name)->lock); \
(name)->head = NULL; \
} while (0)
#define BLOCKING_INIT_NOTIFIER_HEAD(name) do { \
@@ -95,7 +95,7 @@ extern void srcu_init_notifier_head(struct srcu_notifier_head *nh);
cleanup_srcu_struct(&(name)->srcu);
#define ATOMIC_NOTIFIER_INIT(name) { \
- .lock = __SPIN_LOCK_UNLOCKED(name.lock), \
+ .lock = __RAW_SPIN_LOCK_UNLOCKED(name.lock), \
.head = NULL }
#define BLOCKING_NOTIFIER_INIT(name) { \
.rwsem = __RWSEM_INITIALIZER((name).rwsem), \
diff --git a/kernel/notifier.c b/kernel/notifier.c
index 1b019cbca594..c20782f07643 100644
--- a/kernel/notifier.c
+++ b/kernel/notifier.c
@@ -142,9 +142,9 @@ int atomic_notifier_chain_register(struct atomic_notifier_head *nh,
unsigned long flags;
int ret;
- spin_lock_irqsave(&nh->lock, flags);
+ raw_spin_lock_irqsave(&nh->lock, flags);
ret = notifier_chain_register(&nh->head, n);
- spin_unlock_irqrestore(&nh->lock, flags);
+ raw_spin_unlock_irqrestore(&nh->lock, flags);
return ret;
}
EXPORT_SYMBOL_GPL(atomic_notifier_chain_register);
@@ -164,9 +164,9 @@ int atomic_notifier_chain_unregister(struct atomic_notifier_head *nh,
unsigned long flags;
int ret;
- spin_lock_irqsave(&nh->lock, flags);
+ raw_spin_lock_irqsave(&nh->lock, flags);
ret = notifier_chain_unregister(&nh->head, n);
- spin_unlock_irqrestore(&nh->lock, flags);
+ raw_spin_unlock_irqrestore(&nh->lock, flags);
synchronize_rcu();
return ret;
}
@@ -182,9 +182,9 @@ int atomic_notifier_call_chain_robust(struct atomic_notifier_head *nh,
* Musn't use RCU; because then the notifier list can
* change between the up and down traversal.
*/
- spin_lock_irqsave(&nh->lock, flags);
+ raw_spin_lock_irqsave(&nh->lock, flags);
ret = notifier_call_chain_robust(&nh->head, val_up, val_down, v);
- spin_unlock_irqrestore(&nh->lock, flags);
+ raw_spin_unlock_irqrestore(&nh->lock, flags);
return ret;
}
--
2.27.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2020-11-22 20:19 [PATCH] notifier: Make atomic_notifiers use raw_spinlock Valentin Schneider
@ 2020-11-23 14:14 ` Peter Zijlstra
2020-11-23 14:52 ` Valentin Schneider
2020-11-30 10:09 ` Valentin Schneider
2020-11-30 13:55 ` Daniel Bristot de Oliveira
2 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2020-11-23 14:14 UTC (permalink / raw)
To: Valentin Schneider
Cc: linux-kernel, linux-rt-users, rostedt, mhiramat, bristot, jbaron,
torvalds, tglx, mingo, namit, hpa, luto, ard.biesheuvel,
jpoimboe, pbonzini, mathieu.desnoyers, linux, Rafael J. Wysocki,
Sebastian Andrzej Siewior, Alex Shi, Daniel Lezcano,
Vincenzo Frascino
On Sun, Nov 22, 2020 at 08:19:04PM +0000, Valentin Schneider wrote:
> Booting a recent PREEMPT_RT kernel (v5.10-rc3-rt7-rebase) on my arm64 Juno
> leads to the idle task blocking on an RT sleeping spinlock down some
> notifier path:
>
> [ 1.809101] BUG: scheduling while atomic: swapper/5/0/0x00000002
> [ 1.809116] Modules linked in:
> [ 1.809123] Preemption disabled at:
> [ 1.809125] secondary_start_kernel (arch/arm64/kernel/smp.c:227)
> [ 1.809146] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G W 5.10.0-rc3-rt7 #168
> [ 1.809153] Hardware name: ARM Juno development board (r0) (DT)
> [ 1.809158] Call trace:
> [ 1.809160] dump_backtrace (arch/arm64/kernel/stacktrace.c:100 (discriminator 1))
> [ 1.809170] show_stack (arch/arm64/kernel/stacktrace.c:198)
> [ 1.809178] dump_stack (lib/dump_stack.c:122)
> [ 1.809188] __schedule_bug (kernel/sched/core.c:4886)
> [ 1.809197] __schedule (./arch/arm64/include/asm/preempt.h:18 kernel/sched/core.c:4913 kernel/sched/core.c:5040)
> [ 1.809204] preempt_schedule_lock (kernel/sched/core.c:5365 (discriminator 1))
> [ 1.809210] rt_spin_lock_slowlock_locked (kernel/locking/rtmutex.c:1072)
> [ 1.809217] rt_spin_lock_slowlock (kernel/locking/rtmutex.c:1110)
> [ 1.809224] rt_spin_lock (./include/linux/rcupdate.h:647 kernel/locking/rtmutex.c:1139)
> [ 1.809231] atomic_notifier_call_chain_robust (kernel/notifier.c:71 kernel/notifier.c:118 kernel/notifier.c:186)
> [ 1.809240] cpu_pm_enter (kernel/cpu_pm.c:39 kernel/cpu_pm.c:93)
> [ 1.809249] psci_enter_idle_state (drivers/cpuidle/cpuidle-psci.c:52 drivers/cpuidle/cpuidle-psci.c:129)
> [ 1.809258] cpuidle_enter_state (drivers/cpuidle/cpuidle.c:238)
> [ 1.809267] cpuidle_enter (drivers/cpuidle/cpuidle.c:353)
> [ 1.809275] do_idle (kernel/sched/idle.c:132 kernel/sched/idle.c:213 kernel/sched/idle.c:273)
> [ 1.809282] cpu_startup_entry (kernel/sched/idle.c:368 (discriminator 1))
> [ 1.809288] secondary_start_kernel (arch/arm64/kernel/smp.c:273)
>
> Two points worth noting:
>
> 1) That this is conceptually the same issue as pointed out in:
> 313c8c16ee62 ("PM / CPU: replace raw_notifier with atomic_notifier")
> 2) Only the _robust() variant of atomic_notifier callchains suffer from
> this
>
> AFAICT only the cpu_pm_notifier_chain really needs to be changed, but
> singling it out would mean introducing a new (truly) non-blocking API. At
> the same time, callers that are fine with any blocking within the call
> chain should use blocking notifiers, so patching up all atomic_notifier's
> doesn't seem *too* crazy to me.
How long are these notifier chains?, and all this pcs_enter_idle_state()
is still horribly broken vs RCU, witness the RCU_NONIDLE() there and the
rcu_irq_enter_irqson() in the pm_notifier code.
That said, we're running these notifiers from the idle path with IRQs
disabled, so taking that spinlock isn't going to make it worse..
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2020-11-23 14:14 ` Peter Zijlstra
@ 2020-11-23 14:52 ` Valentin Schneider
0 siblings, 0 replies; 10+ messages in thread
From: Valentin Schneider @ 2020-11-23 14:52 UTC (permalink / raw)
To: Peter Zijlstra
Cc: linux-kernel, linux-rt-users, rostedt, mhiramat, bristot, jbaron,
torvalds, tglx, mingo, namit, hpa, luto, ard.biesheuvel,
jpoimboe, pbonzini, mathieu.desnoyers, linux, Rafael J. Wysocki,
Sebastian Andrzej Siewior, Alex Shi, Daniel Lezcano,
Vincenzo Frascino
On 23/11/20 14:14, Peter Zijlstra wrote:
> On Sun, Nov 22, 2020 at 08:19:04PM +0000, Valentin Schneider wrote:
[...]
>> Two points worth noting:
>>
>> 1) That this is conceptually the same issue as pointed out in:
>> 313c8c16ee62 ("PM / CPU: replace raw_notifier with atomic_notifier")
>> 2) Only the _robust() variant of atomic_notifier callchains suffer from
>> this
>>
>> AFAICT only the cpu_pm_notifier_chain really needs to be changed, but
>> singling it out would mean introducing a new (truly) non-blocking API. At
>> the same time, callers that are fine with any blocking within the call
>> chain should use blocking notifiers, so patching up all atomic_notifier's
>> doesn't seem *too* crazy to me.
>
> How long are these notifier chains?,
On said Juno I get:
gic_notifier()
arch_timer_cpu_pm_notify()
fpsimd_cpu_pm_notifier()
cpu_pm_pmu_notify() x2
hyp_init_cpu_pm_notifier()
(I would take a guess that there's one PMU cb per cluster due to big.LITTLE
faffery)
> and all this pcs_enter_idle_state()
> is still horribly broken vs RCU, witness the RCU_NONIDLE() there and the
> rcu_irq_enter_irqson() in the pm_notifier code.
>
Hadn't paid attention to that, that's indeed... Interesting.
> That said, we're running these notifiers from the idle path with IRQs
> disabled, so taking that spinlock isn't going to make it worse..
And it's already taken on !PREEMPT_RT.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2020-11-22 20:19 [PATCH] notifier: Make atomic_notifiers use raw_spinlock Valentin Schneider
2020-11-23 14:14 ` Peter Zijlstra
@ 2020-11-30 10:09 ` Valentin Schneider
2020-11-30 13:51 ` Sebastian Andrzej Siewior
2020-11-30 13:55 ` Daniel Bristot de Oliveira
2 siblings, 1 reply; 10+ messages in thread
From: Valentin Schneider @ 2020-11-30 10:09 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: peterz, rostedt, mhiramat, bristot, jbaron, torvalds, tglx,
mingo, namit, hpa, luto, ard.biesheuvel, jpoimboe, pbonzini,
mathieu.desnoyers, linux, Rafael J. Wysocki,
Sebastian Andrzej Siewior, Alex Shi, Daniel Lezcano,
Vincenzo Frascino
On 22/11/20 20:19, Valentin Schneider wrote:
> Booting a recent PREEMPT_RT kernel (v5.10-rc3-rt7-rebase) on my arm64 Juno
> leads to the idle task blocking on an RT sleeping spinlock down some
> notifier path:
>
> [ 1.809101] BUG: scheduling while atomic: swapper/5/0/0x00000002
> [ 1.809116] Modules linked in:
> [ 1.809123] Preemption disabled at:
> [ 1.809125] secondary_start_kernel (arch/arm64/kernel/smp.c:227)
> [ 1.809146] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G W 5.10.0-rc3-rt7 #168
> [ 1.809153] Hardware name: ARM Juno development board (r0) (DT)
> [ 1.809158] Call trace:
> [ 1.809160] dump_backtrace (arch/arm64/kernel/stacktrace.c:100 (discriminator 1))
> [ 1.809170] show_stack (arch/arm64/kernel/stacktrace.c:198)
> [ 1.809178] dump_stack (lib/dump_stack.c:122)
> [ 1.809188] __schedule_bug (kernel/sched/core.c:4886)
> [ 1.809197] __schedule (./arch/arm64/include/asm/preempt.h:18 kernel/sched/core.c:4913 kernel/sched/core.c:5040)
> [ 1.809204] preempt_schedule_lock (kernel/sched/core.c:5365 (discriminator 1))
> [ 1.809210] rt_spin_lock_slowlock_locked (kernel/locking/rtmutex.c:1072)
> [ 1.809217] rt_spin_lock_slowlock (kernel/locking/rtmutex.c:1110)
> [ 1.809224] rt_spin_lock (./include/linux/rcupdate.h:647 kernel/locking/rtmutex.c:1139)
> [ 1.809231] atomic_notifier_call_chain_robust (kernel/notifier.c:71 kernel/notifier.c:118 kernel/notifier.c:186)
> [ 1.809240] cpu_pm_enter (kernel/cpu_pm.c:39 kernel/cpu_pm.c:93)
> [ 1.809249] psci_enter_idle_state (drivers/cpuidle/cpuidle-psci.c:52 drivers/cpuidle/cpuidle-psci.c:129)
> [ 1.809258] cpuidle_enter_state (drivers/cpuidle/cpuidle.c:238)
> [ 1.809267] cpuidle_enter (drivers/cpuidle/cpuidle.c:353)
> [ 1.809275] do_idle (kernel/sched/idle.c:132 kernel/sched/idle.c:213 kernel/sched/idle.c:273)
> [ 1.809282] cpu_startup_entry (kernel/sched/idle.c:368 (discriminator 1))
> [ 1.809288] secondary_start_kernel (arch/arm64/kernel/smp.c:273)
>
FWIW, still squealing under v5.10-rc5-rt11.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2020-11-30 10:09 ` Valentin Schneider
@ 2020-11-30 13:51 ` Sebastian Andrzej Siewior
0 siblings, 0 replies; 10+ messages in thread
From: Sebastian Andrzej Siewior @ 2020-11-30 13:51 UTC (permalink / raw)
To: Valentin Schneider
Cc: linux-kernel, linux-rt-users, peterz, rostedt, mhiramat, bristot,
jbaron, torvalds, tglx, mingo, namit, hpa, luto, ard.biesheuvel,
jpoimboe, pbonzini, mathieu.desnoyers, linux, Rafael J. Wysocki,
Alex Shi, Daniel Lezcano, Vincenzo Frascino
On 2020-11-30 10:09:41 [+0000], Valentin Schneider wrote:
> FWIW, still squealing under v5.10-rc5-rt11.
I could apply this into my current RT but it would be nice to get this
applied upstream.
Sebastian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2020-11-22 20:19 [PATCH] notifier: Make atomic_notifiers use raw_spinlock Valentin Schneider
2020-11-23 14:14 ` Peter Zijlstra
2020-11-30 10:09 ` Valentin Schneider
@ 2020-11-30 13:55 ` Daniel Bristot de Oliveira
2 siblings, 0 replies; 10+ messages in thread
From: Daniel Bristot de Oliveira @ 2020-11-30 13:55 UTC (permalink / raw)
To: Valentin Schneider, linux-kernel, linux-rt-users
Cc: peterz, rostedt, mhiramat, jbaron, torvalds, tglx, mingo, namit,
hpa, luto, ard.biesheuvel, jpoimboe, pbonzini, mathieu.desnoyers,
linux, Rafael J. Wysocki, Sebastian Andrzej Siewior, Alex Shi,
Daniel Lezcano, Vincenzo Frascino
On 11/22/20 9:19 PM, Valentin Schneider wrote:
> Booting a recent PREEMPT_RT kernel (v5.10-rc3-rt7-rebase) on my arm64 Juno
> leads to the idle task blocking on an RT sleeping spinlock down some
> notifier path:
>
> [ 1.809101] BUG: scheduling while atomic: swapper/5/0/0x00000002
> [ 1.809116] Modules linked in:
> [ 1.809123] Preemption disabled at:
> [ 1.809125] secondary_start_kernel (arch/arm64/kernel/smp.c:227)
> [ 1.809146] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G W 5.10.0-rc3-rt7 #168
> [ 1.809153] Hardware name: ARM Juno development board (r0) (DT)
> [ 1.809158] Call trace:
> [ 1.809160] dump_backtrace (arch/arm64/kernel/stacktrace.c:100 (discriminator 1))
> [ 1.809170] show_stack (arch/arm64/kernel/stacktrace.c:198)
> [ 1.809178] dump_stack (lib/dump_stack.c:122)
> [ 1.809188] __schedule_bug (kernel/sched/core.c:4886)
> [ 1.809197] __schedule (./arch/arm64/include/asm/preempt.h:18 kernel/sched/core.c:4913 kernel/sched/core.c:5040)
> [ 1.809204] preempt_schedule_lock (kernel/sched/core.c:5365 (discriminator 1))
> [ 1.809210] rt_spin_lock_slowlock_locked (kernel/locking/rtmutex.c:1072)
> [ 1.809217] rt_spin_lock_slowlock (kernel/locking/rtmutex.c:1110)
> [ 1.809224] rt_spin_lock (./include/linux/rcupdate.h:647 kernel/locking/rtmutex.c:1139)
> [ 1.809231] atomic_notifier_call_chain_robust (kernel/notifier.c:71 kernel/notifier.c:118 kernel/notifier.c:186)
> [ 1.809240] cpu_pm_enter (kernel/cpu_pm.c:39 kernel/cpu_pm.c:93)
> [ 1.809249] psci_enter_idle_state (drivers/cpuidle/cpuidle-psci.c:52 drivers/cpuidle/cpuidle-psci.c:129)
> [ 1.809258] cpuidle_enter_state (drivers/cpuidle/cpuidle.c:238)
> [ 1.809267] cpuidle_enter (drivers/cpuidle/cpuidle.c:353)
> [ 1.809275] do_idle (kernel/sched/idle.c:132 kernel/sched/idle.c:213 kernel/sched/idle.c:273)
> [ 1.809282] cpu_startup_entry (kernel/sched/idle.c:368 (discriminator 1))
> [ 1.809288] secondary_start_kernel (arch/arm64/kernel/smp.c:273)
>
> Two points worth noting:
>
> 1) That this is conceptually the same issue as pointed out in:
> 313c8c16ee62 ("PM / CPU: replace raw_notifier with atomic_notifier")
> 2) Only the _robust() variant of atomic_notifier callchains suffer from
> this
>
> AFAICT only the cpu_pm_notifier_chain really needs to be changed, but
> singling it out would mean introducing a new (truly) non-blocking API. At
> the same time, callers that are fine with any blocking within the call
> chain should use blocking notifiers, so patching up all atomic_notifier's
> doesn't seem *too* crazy to me.
>
> Fixes: 70d932985757 ("notifier: Fix broken error handling pattern")
> Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Thanks!
-- Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2021-08-06 18:06 ` Sebastian Andrzej Siewior
@ 2021-08-06 18:20 ` Peter Zijlstra
0 siblings, 0 replies; 10+ messages in thread
From: Peter Zijlstra @ 2021-08-06 18:20 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-kernel, Thomas Gleixner, Daniel Bristot de Oliveira,
Valentin Schneider, Ingo Molnar, Rafael J. Wysocki
On Fri, Aug 06, 2021 at 08:06:53PM +0200, Sebastian Andrzej Siewior wrote:
> On 2021-08-06 20:02:42 [+0200], Peter Zijlstra wrote:
> > On Fri, Aug 06, 2021 at 04:07:18PM +0200, Sebastian Andrzej Siewior wrote:
> > > What do we do with this?
> > > Do we merge this as-is, add another "robust atomic notifier" using only
> > > raw_spinlock_t for registration and notification (for only
> > > cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all
> > > atomic notifier in -tree?
> >
> > Right, so the problem I see with this is that
> > notifier_chain_{,un}register() are O(n). Hardly something we should be
> > putting under raw_spin_lock :/
>
> Yup, pretty much. So we make one robust notifier for
> cpu_pm_notifier_chain?
Yeah, I suppose so :-( Ideally that whole pm notifier thing goes, but
that's *far* more work and I really don't want to be responsible for the
brain damange resulting from looking at all that 'special' idle code.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2021-08-06 18:02 ` Peter Zijlstra
@ 2021-08-06 18:06 ` Sebastian Andrzej Siewior
2021-08-06 18:20 ` Peter Zijlstra
0 siblings, 1 reply; 10+ messages in thread
From: Sebastian Andrzej Siewior @ 2021-08-06 18:06 UTC (permalink / raw)
To: Peter Zijlstra
Cc: linux-kernel, Thomas Gleixner, Daniel Bristot de Oliveira,
Valentin Schneider, Ingo Molnar, Rafael J. Wysocki
On 2021-08-06 20:02:42 [+0200], Peter Zijlstra wrote:
> On Fri, Aug 06, 2021 at 04:07:18PM +0200, Sebastian Andrzej Siewior wrote:
> > What do we do with this?
> > Do we merge this as-is, add another "robust atomic notifier" using only
> > raw_spinlock_t for registration and notification (for only
> > cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all
> > atomic notifier in -tree?
>
> Right, so the problem I see with this is that
> notifier_chain_{,un}register() are O(n). Hardly something we should be
> putting under raw_spin_lock :/
Yup, pretty much. So we make one robust notifier for
cpu_pm_notifier_chain?
Sebastian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] notifier: Make atomic_notifiers use raw_spinlock
2021-08-06 14:07 Sebastian Andrzej Siewior
@ 2021-08-06 18:02 ` Peter Zijlstra
2021-08-06 18:06 ` Sebastian Andrzej Siewior
0 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2021-08-06 18:02 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-kernel, Thomas Gleixner, Daniel Bristot de Oliveira,
Valentin Schneider, Ingo Molnar, Rafael J. Wysocki
On Fri, Aug 06, 2021 at 04:07:18PM +0200, Sebastian Andrzej Siewior wrote:
> What do we do with this?
> Do we merge this as-is, add another "robust atomic notifier" using only
> raw_spinlock_t for registration and notification (for only
> cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all
> atomic notifier in -tree?
Right, so the problem I see with this is that
notifier_chain_{,un}register() are O(n). Hardly something we should be
putting under raw_spin_lock :/
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH] notifier: Make atomic_notifiers use raw_spinlock
@ 2021-08-06 14:07 Sebastian Andrzej Siewior
2021-08-06 18:02 ` Peter Zijlstra
0 siblings, 1 reply; 10+ messages in thread
From: Sebastian Andrzej Siewior @ 2021-08-06 14:07 UTC (permalink / raw)
To: linux-kernel
Cc: Thomas Gleixner, Peter Zijlstra, Daniel Bristot de Oliveira,
Valentin Schneider, Ingo Molnar, Rafael J. Wysocki
From: Valentin Schneider <valentin.schneider@arm.com>
Date: Sun, 22 Nov 2020 20:19:04 +0000
Booting a recent PREEMPT_RT kernel (v5.10-rc3-rt7-rebase) on my arm64 Juno
leads to the idle task blocking on an RT sleeping spinlock down some
notifier path:
| BUG: scheduling while atomic: swapper/5/0/0x00000002
…
| atomic_notifier_call_chain_robust (kernel/notifier.c:71 kernel/notifier.c:118 kernel/notifier.c:186)
| cpu_pm_enter (kernel/cpu_pm.c:39 kernel/cpu_pm.c:93)
| psci_enter_idle_state (drivers/cpuidle/cpuidle-psci.c:52 drivers/cpuidle/cpuidle-psci.c:129)
| cpuidle_enter_state (drivers/cpuidle/cpuidle.c:238)
| cpuidle_enter (drivers/cpuidle/cpuidle.c:353)
| do_idle (kernel/sched/idle.c:132 kernel/sched/idle.c:213 kernel/sched/idle.c:273)
| cpu_startup_entry (kernel/sched/idle.c:368 (discriminator 1))
| secondary_start_kernel (arch/arm64/kernel/smp.c:273)
Two points worth noting:
1) That this is conceptually the same issue as pointed out in:
313c8c16ee62 ("PM / CPU: replace raw_notifier with atomic_notifier")
2) Only the _robust() variant of atomic_notifier callchains suffer from
this
AFAICT only the cpu_pm_notifier_chain really needs to be changed, but
singling it out would mean introducing a new (truly) non-blocking API. At
the same time, callers that are fine with any blocking within the call
chain should use blocking notifiers, so patching up all atomic_notifier's
doesn't seem *too* crazy to me.
Fixes: 70d932985757 ("notifier: Fix broken error handling pattern")
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Link: https://lkml.kernel.org/r/20201122201904.30940-1-valentin.schneider@arm.com
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
What do we do with this?
Do we merge this as-is, add another "robust atomic notifier" using only
raw_spinlock_t for registration and notification (for only
cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all
atomic notifier in -tree?
include/linux/notifier.h | 6 +++---
kernel/notifier.c | 12 ++++++------
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/include/linux/notifier.h b/include/linux/notifier.h
index 2fb373a5c1ede..723bc2df63882 100644
--- a/include/linux/notifier.h
+++ b/include/linux/notifier.h
@@ -58,7 +58,7 @@ struct notifier_block {
};
struct atomic_notifier_head {
- spinlock_t lock;
+ raw_spinlock_t lock;
struct notifier_block __rcu *head;
};
@@ -78,7 +78,7 @@ struct srcu_notifier_head {
};
#define ATOMIC_INIT_NOTIFIER_HEAD(name) do { \
- spin_lock_init(&(name)->lock); \
+ raw_spin_lock_init(&(name)->lock); \
(name)->head = NULL; \
} while (0)
#define BLOCKING_INIT_NOTIFIER_HEAD(name) do { \
@@ -95,7 +95,7 @@ extern void srcu_init_notifier_head(struct srcu_notifier_head *nh);
cleanup_srcu_struct(&(name)->srcu);
#define ATOMIC_NOTIFIER_INIT(name) { \
- .lock = __SPIN_LOCK_UNLOCKED(name.lock), \
+ .lock = __RAW_SPIN_LOCK_UNLOCKED(name.lock), \
.head = NULL }
#define BLOCKING_NOTIFIER_INIT(name) { \
.rwsem = __RWSEM_INITIALIZER((name).rwsem), \
diff --git a/kernel/notifier.c b/kernel/notifier.c
index 1b019cbca594a..c20782f076432 100644
--- a/kernel/notifier.c
+++ b/kernel/notifier.c
@@ -142,9 +142,9 @@ int atomic_notifier_chain_register(struct atomic_notifier_head *nh,
unsigned long flags;
int ret;
- spin_lock_irqsave(&nh->lock, flags);
+ raw_spin_lock_irqsave(&nh->lock, flags);
ret = notifier_chain_register(&nh->head, n);
- spin_unlock_irqrestore(&nh->lock, flags);
+ raw_spin_unlock_irqrestore(&nh->lock, flags);
return ret;
}
EXPORT_SYMBOL_GPL(atomic_notifier_chain_register);
@@ -164,9 +164,9 @@ int atomic_notifier_chain_unregister(struct atomic_notifier_head *nh,
unsigned long flags;
int ret;
- spin_lock_irqsave(&nh->lock, flags);
+ raw_spin_lock_irqsave(&nh->lock, flags);
ret = notifier_chain_unregister(&nh->head, n);
- spin_unlock_irqrestore(&nh->lock, flags);
+ raw_spin_unlock_irqrestore(&nh->lock, flags);
synchronize_rcu();
return ret;
}
@@ -182,9 +182,9 @@ int atomic_notifier_call_chain_robust(struct atomic_notifier_head *nh,
* Musn't use RCU; because then the notifier list can
* change between the up and down traversal.
*/
- spin_lock_irqsave(&nh->lock, flags);
+ raw_spin_lock_irqsave(&nh->lock, flags);
ret = notifier_call_chain_robust(&nh->head, val_up, val_down, v);
- spin_unlock_irqrestore(&nh->lock, flags);
+ raw_spin_unlock_irqrestore(&nh->lock, flags);
return ret;
}
--
2.32.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-08-06 18:21 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-22 20:19 [PATCH] notifier: Make atomic_notifiers use raw_spinlock Valentin Schneider
2020-11-23 14:14 ` Peter Zijlstra
2020-11-23 14:52 ` Valentin Schneider
2020-11-30 10:09 ` Valentin Schneider
2020-11-30 13:51 ` Sebastian Andrzej Siewior
2020-11-30 13:55 ` Daniel Bristot de Oliveira
2021-08-06 14:07 Sebastian Andrzej Siewior
2021-08-06 18:02 ` Peter Zijlstra
2021-08-06 18:06 ` Sebastian Andrzej Siewior
2021-08-06 18:20 ` Peter Zijlstra
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).