From: Peter Zijlstra <peterz@infradead.org>
To: mingo@kernel.org, will@kernel.org, tglx@linutronix.de
Cc: x86@kernel.org, linux-kernel@vger.kernel.org,
a.darwish@linutronix.de, rostedt@goodmis.org,
bigeasy@linutronix.de, peterz@infradead.org, davem@davemloft.net,
sparclinux@vger.kernel.org, mpe@ellerman.id.au,
linuxppc-dev@lists.ozlabs.org, heiko.carstens@de.ibm.com,
linux-s390@vger.kernel.org, linux@armlinux.org.uk
Subject: [PATCH v4 1/8] lockdep: Prepare for NMI IRQ state tracking
Date: Tue, 23 Jun 2020 10:36:46 +0200 [thread overview]
Message-ID: <20200623083721.155449112@infradead.org> (raw)
In-Reply-To: 20200623083645.277342609@infradead.org
There is no reason not to always, accurately, track IRQ state.
This change also makes IRQ state tracking ignore lockdep_off().
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/locking/lockdep.c | 44 +++++++++++++++++++++++++++++++++++++++++---
1 file changed, 41 insertions(+), 3 deletions(-)
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -3646,7 +3646,16 @@ static void __trace_hardirqs_on_caller(v
*/
void lockdep_hardirqs_on_prepare(unsigned long ip)
{
- if (unlikely(!debug_locks || current->lockdep_recursion))
+ if (unlikely(!debug_locks))
+ return;
+
+ /*
+ * NMIs do not (and cannot) track lock dependencies, nothing to do.
+ */
+ if (unlikely(in_nmi()))
+ return;
+
+ if (unlikely(current->lockdep_recursion & LOCKDEP_RECURSION_MASK))
return;
if (unlikely(current->hardirqs_enabled)) {
@@ -3692,7 +3701,27 @@ void noinstr lockdep_hardirqs_on(unsigne
{
struct task_struct *curr = current;
- if (unlikely(!debug_locks || curr->lockdep_recursion))
+ if (unlikely(!debug_locks))
+ return;
+
+ /*
+ * NMIs can happen in the middle of local_irq_{en,dis}able() where the
+ * tracking state and hardware state are out of sync.
+ *
+ * NMIs must save lockdep_hardirqs_enabled() to restore IRQ state from,
+ * and not rely on hardware state like normal interrupts.
+ */
+ if (unlikely(in_nmi())) {
+ /*
+ * Skip:
+ * - recursion check, because NMI can hit lockdep;
+ * - hardware state check, because above;
+ * - chain_key check, see lockdep_hardirqs_on_prepare().
+ */
+ goto skip_checks;
+ }
+
+ if (unlikely(current->lockdep_recursion & LOCKDEP_RECURSION_MASK))
return;
if (curr->hardirqs_enabled) {
@@ -3720,6 +3749,7 @@ void noinstr lockdep_hardirqs_on(unsigne
DEBUG_LOCKS_WARN_ON(current->hardirq_chain_key !=
current->curr_chain_key);
+skip_checks:
/* we'll do an OFF -> ON transition: */
curr->hardirqs_enabled = 1;
curr->hardirq_enable_ip = ip;
@@ -3735,7 +3765,15 @@ void noinstr lockdep_hardirqs_off(unsign
{
struct task_struct *curr = current;
- if (unlikely(!debug_locks || curr->lockdep_recursion))
+ if (unlikely(!debug_locks))
+ return;
+
+ /*
+ * Matching lockdep_hardirqs_on(), allow NMIs in the middle of lockdep;
+ * they will restore the software state. This ensures the software
+ * state is consistent inside NMIs as well.
+ */
+ if (unlikely(!in_nmi() && (current->lockdep_recursion & LOCKDEP_RECURSION_MASK)))
return;
/*
next prev parent reply other threads:[~2020-06-23 8:39 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-23 8:36 [PATCH v4 0/8] lockdep: Change IRQ state tracking to use per-cpu variables Peter Zijlstra
2020-06-23 8:36 ` Peter Zijlstra [this message]
2020-07-11 10:09 ` [tip: locking/core] lockdep: Prepare for NMI IRQ state tracking tip-bot2 for Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 2/8] x86/entry: Fix NMI vs " Peter Zijlstra
2020-07-11 10:09 ` [tip: locking/core] " tip-bot2 for Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 3/8] sparc64: Fix asm/percpu.h build error Peter Zijlstra
2020-06-23 21:35 ` David Miller
2020-07-11 10:09 ` [tip: locking/core] " tip-bot2 for Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 4/8] powerpc64: Break asm/percpu.h vs spinlock_types.h dependency Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 5/8] s390: Break cyclic percpu include Peter Zijlstra
2020-07-11 10:09 ` [tip: locking/core] " tip-bot2 for Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 6/8] arm: " Peter Zijlstra
2020-06-23 9:02 ` Will Deacon
2020-06-24 17:53 ` Peter Zijlstra
2020-06-25 7:31 ` Will Deacon
2020-07-11 10:09 ` [tip: locking/core] " tip-bot2 for Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 7/8] lockdep: Change hardirq{s_enabled,_context} to per-cpu variables Peter Zijlstra
2020-06-23 15:00 ` Ahmed S. Darwish
2020-06-23 15:24 ` Peter Zijlstra
2020-06-23 16:13 ` Ahmed S. Darwish
2020-06-23 16:37 ` Peter Zijlstra
2020-06-23 17:59 ` Marco Elver
2020-06-23 18:12 ` Peter Zijlstra
2020-06-23 18:39 ` Marco Elver
2020-06-23 19:13 ` Marco Elver
2020-06-23 19:41 ` Peter Zijlstra
2020-06-23 20:08 ` Peter Zijlstra
2020-06-23 20:24 ` Peter Zijlstra
2020-06-23 20:33 ` Peter Zijlstra
2020-06-24 9:00 ` Peter Zijlstra
2020-06-24 10:17 ` Marco Elver
2020-06-24 12:31 ` Peter Zijlstra
2020-06-24 11:32 ` Marco Elver
2020-06-24 15:18 ` Peter Zijlstra
2020-07-11 10:09 ` [tip: locking/core] kcsan: Make KCSAN compatible with new IRQ state tracking tip-bot2 for Marco Elver
2020-06-30 5:59 ` [PATCH v4 7/8] lockdep: Change hardirq{s_enabled,_context} to per-cpu variables Ahmed S. Darwish
2020-06-30 9:40 ` Peter Zijlstra
2020-07-11 10:09 ` [tip: locking/core] " tip-bot2 for Peter Zijlstra
2020-06-23 8:36 ` [PATCH v4 8/8] lockdep: Remove lockdep_hardirq{s_enabled,_context}() argument Peter Zijlstra
2020-07-11 10:09 ` [tip: locking/core] " tip-bot2 for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200623083721.155449112@infradead.org \
--to=peterz@infradead.org \
--cc=a.darwish@linutronix.de \
--cc=bigeasy@linutronix.de \
--cc=davem@davemloft.net \
--cc=heiko.carstens@de.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mingo@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=rostedt@goodmis.org \
--cc=sparclinux@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).