From: Peter Zijlstra <peterz@infradead.org>
To: Nicholas Piggin <npiggin@gmail.com>
Cc: linux-arch@vger.kernel.org, Alexey Kardashevskiy <aik@ozlabs.ru>,
linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
Ingo Molnar <mingo@redhat.com>, Will Deacon <will@kernel.org>
Subject: Re: [PATCH 1/2] lockdep: improve current->(hard|soft)irqs_enabled synchronisation with actual irq state
Date: Sat, 25 Jul 2020 22:26:17 +0200 [thread overview]
Message-ID: <20200725202617.GI10769@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20200723105615.1268126-1-npiggin@gmail.com>
On Thu, Jul 23, 2020 at 08:56:14PM +1000, Nicholas Piggin wrote:
> diff --git a/arch/powerpc/include/asm/hw_irq.h b/arch/powerpc/include/asm/hw_irq.h
> index 3a0db7b0b46e..35060be09073 100644
> --- a/arch/powerpc/include/asm/hw_irq.h
> +++ b/arch/powerpc/include/asm/hw_irq.h
> @@ -200,17 +200,14 @@ static inline bool arch_irqs_disabled(void)
> #define powerpc_local_irq_pmu_save(flags) \
> do { \
> raw_local_irq_pmu_save(flags); \
> - trace_hardirqs_off(); \
> + if (!raw_irqs_disabled_flags(flags)) \
> + trace_hardirqs_off(); \
> } while(0)
So one problem with the above is something like this:
raw_local_irq_save();
<NMI>
powerpc_local_irq_pmu_save();
that would now no longer call into tracing/lockdep at all. As a
consequence, lockdep and tracing would show the NMI ran with IRQs
enabled, which is exceptionally weird..
Similar problem with:
raw_local_irq_disable();
local_irq_save()
Now, most architectures today seem to do what x86 also did:
<NMI>
trace_hardirqs_off()
...
if (irqs_unmasked(regs))
trace_hardirqs_on()
</NMI>
Which is 'funny' when it interleaves like:
local_irq_disable();
...
local_irq_enable()
trace_hardirqs_on();
<NMI/>
raw_local_irq_enable();
Because then it will undo the trace_hardirqs_on() we just did. With the
result that both tracing and lockdep will see a hardirqs-disable without
a matching enable, while the hardware state is enabled.
Which is exactly the state Alexey seems to have ran into.
Now, x86, and at least arm64 call nmi_enter() before
trace_hardirqs_off(), but AFAICT Power never did that, and that's part
of the problem. nmi_enter() does lockdep_off() and that _used_ to also
kill IRQ tracking.
Now, my patch changed that, it makes IRQ tracking not respect
lockdep_off(). And that exposed x86 (and everybody else :/) to the same
problem you have.
And this is why I made x86 look at software state in NMIs. Because then
it all works out. For bonus points, trace_hardirqs_*() also has some
do-it-once logic for tracing.
Anyway, it's Saturday evening, time for a beer. I'll stare at this more
later.
next prev parent reply other threads:[~2020-07-25 20:28 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-23 10:56 [PATCH 1/2] lockdep: improve current->(hard|soft)irqs_enabled synchronisation with actual irq state Nicholas Piggin
2020-07-23 10:56 ` [PATCH 2/2] lockdep: warn on redundant or incorrect irq state changes Nicholas Piggin
2020-07-24 2:57 ` kernel test robot
2020-08-04 10:00 ` Alexey Kardashevskiy
2020-07-23 11:40 ` [PATCH 1/2] lockdep: improve current->(hard|soft)irqs_enabled synchronisation with actual irq state Peter Zijlstra
2020-07-23 13:11 ` Nicholas Piggin
2020-07-23 14:59 ` Peter Zijlstra
2020-07-23 16:20 ` Nicholas Piggin
2020-07-24 4:16 ` Alexey Kardashevskiy
2020-07-24 5:59 ` Nicholas Piggin
2020-07-26 7:40 ` Alexey Kardashevskiy
2020-07-24 6:16 ` Athira Rajeev
2020-07-24 2:19 ` kernel test robot
2020-07-24 3:15 ` kernel test robot
2020-07-25 20:26 ` Peter Zijlstra [this message]
2020-07-26 4:14 ` Nicholas Piggin
2020-07-26 11:59 ` peterz
2020-07-26 12:11 ` peterz
2020-07-28 11:22 ` Nicholas Piggin
2020-08-07 11:11 ` peterz
2020-08-12 8:18 ` Nicholas Piggin
2020-08-12 10:35 ` peterz
2020-08-18 7:22 ` Nicholas Piggin
2020-08-18 15:41 ` peterz
2020-08-18 23:54 ` Nicholas Piggin
2020-08-19 10:39 ` Alexey Kardashevskiy
2020-08-19 15:32 ` peterz
2020-08-19 15:39 ` peterz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200725202617.GI10769@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=aik@ozlabs.ru \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mingo@redhat.com \
--cc=npiggin@gmail.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).