* [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check
@ 2017-10-20 0:55 Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 01/14] lockdep: Add IRQs disabled/enabled assertion Frederic Weisbecker
` (13 more replies)
0 siblings, 14 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:55 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Checking whether IRQs are disabled or enabled is often an important
sanity check. Now we can be sometimes reluctant to have it on fastpath
because reading the flags can be somewhat costly.
As suggested by Peter Zijlstra some time ago, this would be a nice
lockdep extension: it's the natural concurrency sanity checker and it
already tracks down the irqflags state.
So here it is.
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
lockdep/irq
HEAD: 1cb58314de50c73eaf0b0322615ecb7a8ee39592
Thanks,
Frederic
---
Frederic Weisbecker (14):
lockdep: Add IRQs disabled/enabled assertion
irq: Use lockdep to assert IRQs are disabled/enabled
workqueue: Use lockdep to assert IRQs are disabled/enabled
nohz: Use lockdep to assert IRQs are disabled/enabled
hrtimer: Use lockdep to assert IRQs are disabled/enabled
smp: Use lockdep to assert IRQs are disabled/enabled
x86: Use lockdep to assert IRQs are disabled/enabled
perf: Use lockdep to assert IRQs are disabled/enabled
irq/timings: Use lockdep to assert IRQs are disabled/enabled
irq_work: Use lockdep to assert IRQs are disabled/enabled
sched: Use lockdep to assert IRQs are disabled/enabled
posix-cpu-timers: Use lockdep to assert IRQs are disabled/enabled
netpoll: Use lockdep to assert IRQs are disabled/enabled
rcu: Use lockdep to assert IRQs are disabled/enabled
arch/x86/entry/common.c | 2 +-
arch/x86/kernel/smpboot.c | 2 +-
include/linux/lockdep.h | 13 +++++++++++++
kernel/events/core.c | 12 ++++++------
kernel/irq/timings.c | 2 +-
kernel/irq_work.c | 2 +-
kernel/rcu/tree.c | 16 ++++++++--------
kernel/rcu/tree_plugin.h | 10 +++++-----
kernel/sched/clock.c | 2 +-
kernel/sched/cputime.c | 3 +--
kernel/smp.c | 2 +-
kernel/softirq.c | 10 +++++-----
kernel/time/hrtimer.c | 4 +---
kernel/time/posix-cpu-timers.c | 6 +++---
kernel/time/tick-sched.c | 5 ++---
kernel/workqueue.c | 2 +-
net/core/netpoll.c | 2 +-
17 files changed, 52 insertions(+), 43 deletions(-)
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 01/14] lockdep: Add IRQs disabled/enabled assertion
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
@ 2017-10-20 0:55 ` Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 02/14] irq: Use lockdep to assert IRQs are disabled/enabled Frederic Weisbecker
` (12 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:55 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Checking whether IRQs are enabled or disabled is a very common sanity
check, however not free of overhead especially on fastpath where such
assertion is very common.
Lockdep is a good host for such concurrency correctness check and it
even already tracks down IRQs disablement state. Just reuse its
machinery. This will allow us to get rid of the flags pop and check
overhead from fast path when kernel is built for production.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
include/linux/lockdep.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h
index bfa8e0b..fbe0127 100644
--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -719,9 +719,22 @@ do { \
lock_acquire(&(lock)->dep_map, 0, 0, 1, 1, NULL, _THIS_IP_); \
lock_release(&(lock)->dep_map, 0, _THIS_IP_); \
} while (0)
+
+#define lockdep_assert_irqs_enabled() ({ \
+ !WARN_ONCE(debug_locks && !current->hardirqs_enabled, \
+ "IRQs not enabled as expected\n"); \
+ })
+
+#define lockdep_assert_irqs_disabled() ({ \
+ !WARN_ONCE(debug_locks && current->hardirqs_enabled, \
+ "IRQs not disabled as expected\n"); \
+ })
+
#else
# define might_lock(lock) do { } while (0)
# define might_lock_read(lock) do { } while (0)
+static inline int lockdep_assert_irqs_enabled(void) { return 1; }
+static inline int lockdep_assert_irqs_disabled(void) { return 1; }
#endif
#ifdef CONFIG_LOCKDEP
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 02/14] irq: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 01/14] lockdep: Add IRQs disabled/enabled assertion Frederic Weisbecker
@ 2017-10-20 0:55 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 03/14] workqueue: " Frederic Weisbecker
` (11 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:55 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/softirq.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 4e09821..662f7b1b 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -137,7 +137,7 @@ EXPORT_SYMBOL(__local_bh_disable_ip);
static void __local_bh_enable(unsigned int cnt)
{
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
if (softirq_count() == (cnt & SOFTIRQ_MASK))
trace_softirqs_on(_RET_IP_);
@@ -158,7 +158,8 @@ EXPORT_SYMBOL(_local_bh_enable);
void __local_bh_enable_ip(unsigned long ip, unsigned int cnt)
{
- WARN_ON_ONCE(in_irq() || irqs_disabled());
+ WARN_ON_ONCE(in_irq());
+ lockdep_assert_irqs_enabled();
#ifdef CONFIG_TRACE_IRQFLAGS
local_irq_disable();
#endif
@@ -396,9 +397,8 @@ void irq_exit(void)
#ifndef __ARCH_IRQ_EXIT_IRQS_DISABLED
local_irq_disable();
#else
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
#endif
-
account_irq_exit_time(current);
preempt_count_sub(HARDIRQ_OFFSET);
if (!in_interrupt() && local_softirq_pending())
@@ -488,7 +488,7 @@ EXPORT_SYMBOL(__tasklet_hi_schedule);
void __tasklet_hi_schedule_first(struct tasklet_struct *t)
{
- BUG_ON(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
t->next = __this_cpu_read(tasklet_hi_vec.head);
__this_cpu_write(tasklet_hi_vec.head, t);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 03/14] workqueue: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 01/14] lockdep: Add IRQs disabled/enabled assertion Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 02/14] irq: Use lockdep to assert IRQs are disabled/enabled Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-21 15:39 ` Tejun Heo
2017-10-20 0:56 ` [PATCH 04/14] nohz: " Frederic Weisbecker
` (10 subsequent siblings)
13 siblings, 1 reply; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
---
kernel/workqueue.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index c77fdf6..4c6809e 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -1375,7 +1375,7 @@ static void __queue_work(int cpu, struct workqueue_struct *wq,
* queued or lose PENDING. Grabbing PENDING and queueing should
* happen with IRQ disabled.
*/
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
debug_work_activate(work);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 04/14] nohz: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (2 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 03/14] workqueue: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 05/14] hrtimer: " Frederic Weisbecker
` (9 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/time/tick-sched.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 7b258c5..e179336 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -199,7 +199,7 @@ static bool check_tick_dependency(atomic_t *dep)
static bool can_stop_full_tick(int cpu, struct tick_sched *ts)
{
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
if (unlikely(!cpu_online(cpu)))
return false;
@@ -962,8 +962,7 @@ void tick_nohz_idle_enter(void)
{
struct tick_sched *ts;
- WARN_ON_ONCE(irqs_disabled());
-
+ lockdep_assert_irqs_enabled();
/*
* Update the idle state in the scheduler domain hierarchy
* when tick_nohz_stop_sched_tick() is called from the idle loop.
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 05/14] hrtimer: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (3 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 04/14] nohz: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 06/14] smp: " Frederic Weisbecker
` (8 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/time/hrtimer.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 88f75f9..d325208 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -758,9 +758,7 @@ void clock_was_set(void)
*/
void hrtimers_resume(void)
{
- WARN_ONCE(!irqs_disabled(),
- KERN_INFO "hrtimers_resume() called with IRQs enabled!");
-
+ lockdep_assert_irqs_disabled();
/* Retrigger on the local CPU */
retrigger_next_event(NULL);
/* And schedule a retrigger for all others */
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 06/14] smp: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (4 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 05/14] hrtimer: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 07/14] x86: " Frederic Weisbecker
` (7 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/smp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/smp.c b/kernel/smp.c
index c94dd85..084c8b3 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -213,7 +213,7 @@ static void flush_smp_call_function_queue(bool warn_cpu_offline)
call_single_data_t *csd, *csd_next;
static bool warned;
- WARN_ON(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
head = this_cpu_ptr(&call_single_queue);
entry = llist_del_all(head);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 07/14] x86: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (5 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 06/14] smp: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-22 9:20 ` Peter Zijlstra
2017-10-20 0:56 ` [PATCH 08/14] perf: " Frederic Weisbecker
` (6 subsequent siblings)
13 siblings, 1 reply; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
arch/x86/entry/common.c | 2 +-
arch/x86/kernel/smpboot.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 03505ff..b4f3a55 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -186,7 +186,7 @@ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs)
addr_limit_user_check();
- if (IS_ENABLED(CONFIG_PROVE_LOCKING) && WARN_ON(!irqs_disabled()))
+ if (!lockdep_assert_irqs_disabled())
local_irq_disable();
lockdep_sys_exit();
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index aed1460..aee08a4 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1095,7 +1095,7 @@ int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
unsigned long flags;
int err, ret = 0;
- WARN_ON(irqs_disabled());
+ lockdep_assert_irqs_enabled();
pr_debug("++++++++++++++++++++=_---CPU UP %u\n", cpu);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 08/14] perf: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (6 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 07/14] x86: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 09/14] irq/timings: " Frederic Weisbecker
` (5 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/events/core.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 04989fb..f57cde0e 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -209,7 +209,7 @@ static int event_function(void *info)
struct perf_event_context *task_ctx = cpuctx->task_ctx;
int ret = 0;
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
perf_ctx_lock(cpuctx, task_ctx);
/*
@@ -306,7 +306,7 @@ static void event_function_local(struct perf_event *event, event_f func, void *d
struct task_struct *task = READ_ONCE(ctx->task);
struct perf_event_context *task_ctx = NULL;
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
if (task) {
if (task == TASK_TOMBSTONE)
@@ -1004,7 +1004,7 @@ static enum hrtimer_restart perf_mux_hrtimer_handler(struct hrtimer *hr)
struct perf_cpu_context *cpuctx;
int rotations = 0;
- WARN_ON(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
cpuctx = container_of(hr, struct perf_cpu_context, hrtimer);
rotations = perf_rotate_context(cpuctx);
@@ -1091,7 +1091,7 @@ static void perf_event_ctx_activate(struct perf_event_context *ctx)
{
struct list_head *head = this_cpu_ptr(&active_ctx_list);
- WARN_ON(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
WARN_ON(!list_empty(&ctx->active_ctx_list));
@@ -1100,7 +1100,7 @@ static void perf_event_ctx_activate(struct perf_event_context *ctx)
static void perf_event_ctx_deactivate(struct perf_event_context *ctx)
{
- WARN_ON(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
WARN_ON(list_empty(&ctx->active_ctx_list));
@@ -3521,7 +3521,7 @@ void perf_event_task_tick(void)
struct perf_event_context *ctx, *tmp;
int throttled;
- WARN_ON(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
__this_cpu_inc(perf_throttled_seq);
throttled = __this_cpu_xchg(perf_throttled_count, 0);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 09/14] irq/timings: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (7 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 08/14] perf: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 10/14] irq_work: " Frederic Weisbecker
` (4 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/irq/timings.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/irq/timings.c b/kernel/irq/timings.c
index c8c1d07..e0923fa 100644
--- a/kernel/irq/timings.c
+++ b/kernel/irq/timings.c
@@ -264,7 +264,7 @@ u64 irq_timings_next_event(u64 now)
* order to prevent the timings circular buffer to be updated
* while we are reading it.
*/
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
/*
* Number of elements in the circular buffer: If it happens it
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 10/14] irq_work: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (8 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 09/14] irq/timings: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 11/14] sched: " Frederic Weisbecker
` (3 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/irq_work.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/irq_work.c b/kernel/irq_work.c
index bcf107c..8995796 100644
--- a/kernel/irq_work.c
+++ b/kernel/irq_work.c
@@ -188,7 +188,7 @@ void irq_work_tick(void)
*/
void irq_work_sync(struct irq_work *work)
{
- WARN_ON_ONCE(irqs_disabled());
+ lockdep_assert_irqs_enabled();
while (work->flags & IRQ_WORK_BUSY)
cpu_relax();
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 11/14] sched: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (9 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 10/14] irq_work: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 12/14] posix-cpu-timers: " Frederic Weisbecker
` (2 subsequent siblings)
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/sched/clock.c | 2 +-
kernel/sched/cputime.c | 3 +--
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c
index ca0f8fc..e086bab 100644
--- a/kernel/sched/clock.c
+++ b/kernel/sched/clock.c
@@ -388,7 +388,7 @@ void sched_clock_tick(void)
if (unlikely(!sched_clock_running))
return;
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
scd = this_scd();
__scd_stamp(scd);
diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c
index 14d2dbf..9be8b68 100644
--- a/kernel/sched/cputime.c
+++ b/kernel/sched/cputime.c
@@ -259,8 +259,7 @@ static inline u64 account_other_time(u64 max)
{
u64 accounted;
- /* Shall be converted to a lockdep-enabled lightweight check */
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
accounted = steal_account_process_time(max);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 12/14] posix-cpu-timers: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (10 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 11/14] sched: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 13/14] netpoll: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 14/14] rcu: " Frederic Weisbecker
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/time/posix-cpu-timers.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/time/posix-cpu-timers.c b/kernel/time/posix-cpu-timers.c
index 8585ad6..8fafb87 100644
--- a/kernel/time/posix-cpu-timers.c
+++ b/kernel/time/posix-cpu-timers.c
@@ -602,7 +602,7 @@ static int posix_cpu_timer_set(struct k_itimer *timer, int timer_flags,
/*
* Disarm any old timer after extracting its expiry time.
*/
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
ret = 0;
old_incr = timer->it.cpu.incr;
@@ -1033,7 +1033,7 @@ static void posix_cpu_timer_rearm(struct k_itimer *timer)
/*
* Now re-arm for the new expiry time.
*/
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
arm_timer(timer);
unlock:
unlock_task_sighand(p, &flags);
@@ -1124,7 +1124,7 @@ void run_posix_cpu_timers(struct task_struct *tsk)
struct k_itimer *timer, *next;
unsigned long flags;
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
/*
* The fast path checks that there are no expired thread or thread
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 13/14] netpoll: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (11 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 12/14] posix-cpu-timers: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 14/14] rcu: " Frederic Weisbecker
13 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: David S. Miller <davem@davemloft.net>
---
net/core/netpoll.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/core/netpoll.c b/net/core/netpoll.c
index 912731b..57557a6 100644
--- a/net/core/netpoll.c
+++ b/net/core/netpoll.c
@@ -334,7 +334,7 @@ void netpoll_send_skb_on_dev(struct netpoll *np, struct sk_buff *skb,
/* It is up to the caller to keep npinfo alive. */
struct netpoll_info *npinfo;
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
npinfo = rcu_dereference_bh(np->dev->npinfo);
if (!npinfo || !netif_running(dev) || !netif_device_present(dev)) {
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 14/14] rcu: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
` (12 preceding siblings ...)
2017-10-20 0:56 ` [PATCH 13/14] netpoll: " Frederic Weisbecker
@ 2017-10-20 0:56 ` Frederic Weisbecker
2017-10-21 21:36 ` Paul E. McKenney
13 siblings, 1 reply; 20+ messages in thread
From: Frederic Weisbecker @ 2017-10-20 0:56 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Tejun Heo, Peter Zijlstra, David S . Miller,
Lai Jiangshan, Thomas Gleixner, Paul E . McKenney, Ingo Molnar
Lockdep now has an integrated IRQs disabled/enabled sanity check. Just
use it instead of the ad-hoc RCU version.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/tree.c | 16 ++++++++--------
kernel/rcu/tree_plugin.h | 10 +++++-----
2 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index b0ad62b..7661887 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -734,7 +734,7 @@ static int rcu_future_needs_gp(struct rcu_state *rsp)
int idx = (READ_ONCE(rnp->completed) + 1) & 0x1;
int *fp = &rnp->need_future_gp[idx];
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_future_needs_gp() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
return READ_ONCE(*fp);
}
@@ -746,7 +746,7 @@ static int rcu_future_needs_gp(struct rcu_state *rsp)
static bool
cpu_needs_another_gp(struct rcu_state *rsp, struct rcu_data *rdp)
{
- RCU_LOCKDEP_WARN(!irqs_disabled(), "cpu_needs_another_gp() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
if (rcu_gp_in_progress(rsp))
return false; /* No, a grace period is already in progress. */
if (rcu_future_needs_gp(rsp))
@@ -773,7 +773,7 @@ static void rcu_eqs_enter_common(bool user)
struct rcu_data *rdp;
struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_eqs_enter_common() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
trace_rcu_dyntick(TPS("Start"), rdtp->dynticks_nesting, 0);
if (IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
!user && !is_idle_task(current)) {
@@ -840,7 +840,7 @@ static void rcu_eqs_enter(bool user)
*/
void rcu_idle_enter(void)
{
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_idle_enter() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
rcu_eqs_enter(false);
}
@@ -855,7 +855,7 @@ void rcu_idle_enter(void)
*/
void rcu_user_enter(void)
{
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_user_enter() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
rcu_eqs_enter(true);
}
#endif /* CONFIG_NO_HZ_FULL */
@@ -880,7 +880,7 @@ void rcu_irq_exit(void)
{
struct rcu_dynticks *rdtp;
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_irq_exit() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
rdtp = this_cpu_ptr(&rcu_dynticks);
/* Page faults can happen in NMI handlers, so check... */
@@ -947,7 +947,7 @@ static void rcu_eqs_exit(bool user)
struct rcu_dynticks *rdtp;
long long oldval;
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_eqs_exit() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
rdtp = this_cpu_ptr(&rcu_dynticks);
oldval = rdtp->dynticks_nesting;
WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0);
@@ -1018,7 +1018,7 @@ void rcu_irq_enter(void)
struct rcu_dynticks *rdtp;
long long oldval;
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_irq_enter() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
rdtp = this_cpu_ptr(&rcu_dynticks);
/* Page faults can happen in NMI handlers, so check... */
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index e012b9b..df08e5c 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -325,7 +325,7 @@ static void rcu_preempt_note_context_switch(bool preempt)
struct rcu_data *rdp;
struct rcu_node *rnp;
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_preempt_note_context_switch() invoked with interrupts enabled!!!\n");
+ lockdep_assert_irqs_disabled();
WARN_ON_ONCE(!preempt && t->rcu_read_lock_nesting > 0);
if (t->rcu_read_lock_nesting > 0 &&
!t->rcu_read_unlock_special.b.blocked) {
@@ -1421,7 +1421,7 @@ int rcu_needs_cpu(u64 basemono, u64 *nextevt)
struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
unsigned long dj;
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_needs_cpu() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
/* Snapshot to detect later posting of non-lazy callback. */
rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted;
@@ -1470,7 +1470,7 @@ static void rcu_prepare_for_idle(void)
struct rcu_state *rsp;
int tne;
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_prepare_for_idle() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
if (rcu_is_nocb_cpu(smp_processor_id()))
return;
@@ -1525,7 +1525,7 @@ static void rcu_prepare_for_idle(void)
*/
static void rcu_cleanup_after_idle(void)
{
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_cleanup_after_idle() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
if (rcu_is_nocb_cpu(smp_processor_id()))
return;
if (rcu_try_advance_all_cbs())
@@ -2012,7 +2012,7 @@ static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_data *my_rdp,
struct rcu_data *rdp,
unsigned long flags)
{
- RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_nocb_adopt_orphan_cbs() invoked with irqs enabled!!!");
+ lockdep_assert_irqs_disabled();
if (!rcu_is_nocb_cpu(smp_processor_id()))
return false; /* Not NOCBs CPU, caller must migrate CBs. */
__call_rcu_nocb_enqueue(my_rdp, rcu_segcblist_head(&rdp->cblist),
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 03/14] workqueue: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:56 ` [PATCH 03/14] workqueue: " Frederic Weisbecker
@ 2017-10-21 15:39 ` Tejun Heo
0 siblings, 0 replies; 20+ messages in thread
From: Tejun Heo @ 2017-10-21 15:39 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: LKML, Peter Zijlstra, David S . Miller, Lai Jiangshan,
Thomas Gleixner, Paul E . McKenney, Ingo Molnar
On Fri, Oct 20, 2017 at 02:56:00AM +0200, Frederic Weisbecker wrote:
> Use lockdep to check that IRQs are enabled or disabled as expected. This
> way the sanity check only shows overhead when concurrency correctness
> debug code is enabled.
>
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Please feel free to route with other patches.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 14/14] rcu: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:56 ` [PATCH 14/14] rcu: " Frederic Weisbecker
@ 2017-10-21 21:36 ` Paul E. McKenney
0 siblings, 0 replies; 20+ messages in thread
From: Paul E. McKenney @ 2017-10-21 21:36 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: LKML, Tejun Heo, Peter Zijlstra, David S . Miller, Lai Jiangshan,
Thomas Gleixner, Ingo Molnar
On Fri, Oct 20, 2017 at 02:56:11AM +0200, Frederic Weisbecker wrote:
> Lockdep now has an integrated IRQs disabled/enabled sanity check. Just
> use it instead of the ad-hoc RCU version.
>
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Looks good to me and passes light rcutorture testing. Feel free to
push this up with the rest of this series.
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> ---
> kernel/rcu/tree.c | 16 ++++++++--------
> kernel/rcu/tree_plugin.h | 10 +++++-----
> 2 files changed, 13 insertions(+), 13 deletions(-)
>
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index b0ad62b..7661887 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -734,7 +734,7 @@ static int rcu_future_needs_gp(struct rcu_state *rsp)
> int idx = (READ_ONCE(rnp->completed) + 1) & 0x1;
> int *fp = &rnp->need_future_gp[idx];
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_future_needs_gp() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> return READ_ONCE(*fp);
> }
>
> @@ -746,7 +746,7 @@ static int rcu_future_needs_gp(struct rcu_state *rsp)
> static bool
> cpu_needs_another_gp(struct rcu_state *rsp, struct rcu_data *rdp)
> {
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "cpu_needs_another_gp() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> if (rcu_gp_in_progress(rsp))
> return false; /* No, a grace period is already in progress. */
> if (rcu_future_needs_gp(rsp))
> @@ -773,7 +773,7 @@ static void rcu_eqs_enter_common(bool user)
> struct rcu_data *rdp;
> struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_eqs_enter_common() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> trace_rcu_dyntick(TPS("Start"), rdtp->dynticks_nesting, 0);
> if (IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&
> !user && !is_idle_task(current)) {
> @@ -840,7 +840,7 @@ static void rcu_eqs_enter(bool user)
> */
> void rcu_idle_enter(void)
> {
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_idle_enter() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> rcu_eqs_enter(false);
> }
>
> @@ -855,7 +855,7 @@ void rcu_idle_enter(void)
> */
> void rcu_user_enter(void)
> {
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_user_enter() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> rcu_eqs_enter(true);
> }
> #endif /* CONFIG_NO_HZ_FULL */
> @@ -880,7 +880,7 @@ void rcu_irq_exit(void)
> {
> struct rcu_dynticks *rdtp;
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_irq_exit() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> rdtp = this_cpu_ptr(&rcu_dynticks);
>
> /* Page faults can happen in NMI handlers, so check... */
> @@ -947,7 +947,7 @@ static void rcu_eqs_exit(bool user)
> struct rcu_dynticks *rdtp;
> long long oldval;
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_eqs_exit() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> rdtp = this_cpu_ptr(&rcu_dynticks);
> oldval = rdtp->dynticks_nesting;
> WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0);
> @@ -1018,7 +1018,7 @@ void rcu_irq_enter(void)
> struct rcu_dynticks *rdtp;
> long long oldval;
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_irq_enter() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> rdtp = this_cpu_ptr(&rcu_dynticks);
>
> /* Page faults can happen in NMI handlers, so check... */
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index e012b9b..df08e5c 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -325,7 +325,7 @@ static void rcu_preempt_note_context_switch(bool preempt)
> struct rcu_data *rdp;
> struct rcu_node *rnp;
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_preempt_note_context_switch() invoked with interrupts enabled!!!\n");
> + lockdep_assert_irqs_disabled();
> WARN_ON_ONCE(!preempt && t->rcu_read_lock_nesting > 0);
> if (t->rcu_read_lock_nesting > 0 &&
> !t->rcu_read_unlock_special.b.blocked) {
> @@ -1421,7 +1421,7 @@ int rcu_needs_cpu(u64 basemono, u64 *nextevt)
> struct rcu_dynticks *rdtp = this_cpu_ptr(&rcu_dynticks);
> unsigned long dj;
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_needs_cpu() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
>
> /* Snapshot to detect later posting of non-lazy callback. */
> rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted;
> @@ -1470,7 +1470,7 @@ static void rcu_prepare_for_idle(void)
> struct rcu_state *rsp;
> int tne;
>
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_prepare_for_idle() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> if (rcu_is_nocb_cpu(smp_processor_id()))
> return;
>
> @@ -1525,7 +1525,7 @@ static void rcu_prepare_for_idle(void)
> */
> static void rcu_cleanup_after_idle(void)
> {
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_cleanup_after_idle() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> if (rcu_is_nocb_cpu(smp_processor_id()))
> return;
> if (rcu_try_advance_all_cbs())
> @@ -2012,7 +2012,7 @@ static bool __maybe_unused rcu_nocb_adopt_orphan_cbs(struct rcu_data *my_rdp,
> struct rcu_data *rdp,
> unsigned long flags)
> {
> - RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_nocb_adopt_orphan_cbs() invoked with irqs enabled!!!");
> + lockdep_assert_irqs_disabled();
> if (!rcu_is_nocb_cpu(smp_processor_id()))
> return false; /* Not NOCBs CPU, caller must migrate CBs. */
> __call_rcu_nocb_enqueue(my_rdp, rcu_segcblist_head(&rdp->cblist),
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 07/14] x86: Use lockdep to assert IRQs are disabled/enabled
2017-10-20 0:56 ` [PATCH 07/14] x86: " Frederic Weisbecker
@ 2017-10-22 9:20 ` Peter Zijlstra
2017-11-03 14:25 ` Frederic Weisbecker
0 siblings, 1 reply; 20+ messages in thread
From: Peter Zijlstra @ 2017-10-22 9:20 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: LKML, Tejun Heo, David S . Miller, Lai Jiangshan,
Thomas Gleixner, Paul E . McKenney, Ingo Molnar
On Fri, Oct 20, 2017 at 02:56:04AM +0200, Frederic Weisbecker wrote:
> diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
> index 03505ff..b4f3a55 100644
> --- a/arch/x86/entry/common.c
> +++ b/arch/x86/entry/common.c
> @@ -186,7 +186,7 @@ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs)
>
> addr_limit_user_check();
>
> - if (IS_ENABLED(CONFIG_PROVE_LOCKING) && WARN_ON(!irqs_disabled()))
> + if (!lockdep_assert_irqs_disabled())
> local_irq_disable();
>
> lockdep_sys_exit();
So this is the only site that ever uses the return value; and for this
you've chosen the wrong value for !CONFIG_PROVE_LOCKING (namely 1),
resulting in an unconditional CLI here for !lockdep kernels.
How about we replace that whole thing with a simple:
lockdep_assert_irqs_disabled();
And leave it at that, allowing us to remove the return value thing
entirely.
The whole if !disabled, disable logic is uber paranoid programming, but
I don't think we've ever seen that WARN trigger, and if it does (and
then burns the kernel) we at least know wtf happend.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 07/14] x86: Use lockdep to assert IRQs are disabled/enabled
2017-10-22 9:20 ` Peter Zijlstra
@ 2017-11-03 14:25 ` Frederic Weisbecker
0 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-11-03 14:25 UTC (permalink / raw)
To: Peter Zijlstra
Cc: LKML, Tejun Heo, David S . Miller, Lai Jiangshan,
Thomas Gleixner, Paul E . McKenney, Ingo Molnar
2017-10-22 11:20 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
> On Fri, Oct 20, 2017 at 02:56:04AM +0200, Frederic Weisbecker wrote:
>> diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
>> index 03505ff..b4f3a55 100644
>> --- a/arch/x86/entry/common.c
>> +++ b/arch/x86/entry/common.c
>> @@ -186,7 +186,7 @@ __visible inline void prepare_exit_to_usermode(struct
>> pt_regs *regs)
>>
>> addr_limit_user_check();
>>
>> - if (IS_ENABLED(CONFIG_PROVE_LOCKING) && WARN_ON(!irqs_disabled()))
>> + if (!lockdep_assert_irqs_disabled())
>> local_irq_disable();
>>
>> lockdep_sys_exit();
>
> So this is the only site that ever uses the return value; and for this
> you've chosen the wrong value for !CONFIG_PROVE_LOCKING (namely 1),
> resulting in an unconditional CLI here for !lockdep kernels.
Hmm, it should be the opposite. The assertion maps to 1 for
!CONFIG_PROVE_LOCKING
but we are testing with "!".
> How about we replace that whole thing with a simple:
>
> lockdep_assert_irqs_disabled();
>
> And leave it at that, allowing us to remove the return value thing
> entirely.
>
> The whole if !disabled, disable logic is uber paranoid programming, but
> I don't think we've ever seen that WARN trigger, and if it does (and
> then burns the kernel) we at least know wtf happend.
I certainly don't mind personally, as long as we get the warning. Now
I've seen different
opinions on matters such as this. Some prefer to have the
auto-correction, some don't.
I'll try as you say and see if anybody gets angry :)
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 03/14] workqueue: Use lockdep to assert IRQs are disabled/enabled
2017-11-06 15:01 [GIT PULL] lockdep: Add lightweight IRQs disabled/enabled check v2 Frederic Weisbecker
@ 2017-11-06 15:01 ` Frederic Weisbecker
0 siblings, 0 replies; 20+ messages in thread
From: Frederic Weisbecker @ 2017-11-06 15:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Tejun Heo, Peter Zijlstra,
David S . Miller, Lai Jiangshan, Thomas Gleixner,
Paul E . McKenney
Use lockdep to check that IRQs are enabled or disabled as expected. This
way the sanity check only shows overhead when concurrency correctness
debug code is enabled.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Tejun Heo <tj@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
---
kernel/workqueue.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 160fdc6..fa96d29 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -1375,7 +1375,7 @@ static void __queue_work(int cpu, struct workqueue_struct *wq,
* queued or lose PENDING. Grabbing PENDING and queueing should
* happen with IRQ disabled.
*/
- WARN_ON_ONCE(!irqs_disabled());
+ lockdep_assert_irqs_disabled();
debug_work_activate(work);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
end of thread, other threads:[~2017-11-06 15:05 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-20 0:55 [PATCH 00/14] lockdep: Add lightweight IRQs disabled/enabled check Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 01/14] lockdep: Add IRQs disabled/enabled assertion Frederic Weisbecker
2017-10-20 0:55 ` [PATCH 02/14] irq: Use lockdep to assert IRQs are disabled/enabled Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 03/14] workqueue: " Frederic Weisbecker
2017-10-21 15:39 ` Tejun Heo
2017-10-20 0:56 ` [PATCH 04/14] nohz: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 05/14] hrtimer: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 06/14] smp: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 07/14] x86: " Frederic Weisbecker
2017-10-22 9:20 ` Peter Zijlstra
2017-11-03 14:25 ` Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 08/14] perf: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 09/14] irq/timings: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 10/14] irq_work: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 11/14] sched: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 12/14] posix-cpu-timers: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 13/14] netpoll: " Frederic Weisbecker
2017-10-20 0:56 ` [PATCH 14/14] rcu: " Frederic Weisbecker
2017-10-21 21:36 ` Paul E. McKenney
2017-11-06 15:01 [GIT PULL] lockdep: Add lightweight IRQs disabled/enabled check v2 Frederic Weisbecker
2017-11-06 15:01 ` [PATCH 03/14] workqueue: Use lockdep to assert IRQs are disabled/enabled Frederic Weisbecker
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.