From: "Paul E. McKenney" <paulmck@kernel.org>
To: Mark Rutland <mark.rutland@arm.com>
Cc: Pingfan Liu <kernelfans@gmail.com>,
Thomas Gleixner <tglx@linutronix.de>,
linux-arm-kernel@lists.infradead.org,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>,
Joey Gouly <joey.gouly@arm.com>,
Sami Tolvanen <samitolvanen@google.com>,
Julien Thierry <julien.thierry@arm.com>,
Yuichi Ito <ito-yuichi@fujitsu.com>,
linux-kernel@vger.kernel.org, Sven Schnelle <svens@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>
Subject: Re: [PATCHv2 0/5] arm64/irqentry: remove duplicate housekeeping of
Date: Fri, 24 Sep 2021 15:59:54 -0700 [thread overview]
Message-ID: <20210924225954.GN880162@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <20210924173615.GA42068@C02TD0UTHF1T.local>
On Fri, Sep 24, 2021 at 06:36:15PM +0100, Mark Rutland wrote:
> [Adding Paul for RCU, s390 folk for entry code RCU semantics]
>
> On Fri, Sep 24, 2021 at 09:28:32PM +0800, Pingfan Liu wrote:
> > After introducing arm64/kernel/entry_common.c which is akin to
> > kernel/entry/common.c , the housekeeping of rcu/trace are done twice as
> > the following:
> > enter_from_kernel_mode()->rcu_irq_enter().
> > And
> > gic_handle_irq()->...->handle_domain_irq()->irq_enter()->rcu_irq_enter()
> >
> > Besides redundance, based on code analysis, the redundance also raise
> > some mistake, e.g. rcu_data->dynticks_nmi_nesting inc 2, which causes
> > rcu_is_cpu_rrupt_from_idle() unexpected.
>
> Hmmm...
>
> The fundamental questionss are:
>
> 1) Who is supposed to be responsible for doing the rcu entry/exit?
>
> 2) Is it supposed to matter if this happens multiple times?
>
> For (1), I'd generally expect that this is supposed to happen in the
> arch/common entry code, since that itself (or the irqchip driver) could
> depend on RCU, and if that's the case thatn handle_domain_irq()
> shouldn't need to call rcu_irq_enter(). That would be consistent with
> the way we handle all other exceptions.
>
> For (2) I don't know whether the level of nesting is suppoosed to
> matter. I was under the impression it wasn't meant to matter in general,
> so I'm a little surprised that rcu_is_cpu_rrupt_from_idle() depends on a
> specific level of nesting.
>
> >From a glance it looks like this would cause rcu_sched_clock_irq() to
> skip setting TIF_NEED_RESCHED, and to not call invoke_rcu_core(), which
> doesn't sound right, at least...
>
> Thomas, Paul, thoughts?
It is absolutely required that rcu_irq_enter() and rcu_irq_exit() calls
be balanced. Normally, this is taken care of by the fact that irq_enter()
invokes rcu_irq_enter() and irq_exit() invokes rcu_irq_exit(). Similarly,
nmi_enter() invokes rcu_nmi_enter() and nmi_exit() invokes rcu_nmi_exit().
But if you are doing some special-case exception where the handler needs
to use RCU readers, but where the rest of the work is not needed, then
the resulting calls to rcu_irq_enter() and rcu_irq_exit() must be in
the architecture-specific code and must be properly balanced.
So if exception entry invokes rcu_irq_enter() twice, then exception
exit also needs to invoke rcu_irq_exit() twice.
There are some constraints on where calls to these functions are place.
For example, any exception-entry code prior to the call to rcu_irq_enter()
must consist solely of functions marked noinstr, but Thomas can tell
you more.
Or am I missing the point of your question?
Thanx, Paul
> AFAICT, s390 will have a similar flow on its IRQ handling path, so if
> this is a real issue they'll be affected too.
>
> Thanks,
> Mark.
>
> > Nmi also faces duplicate accounts. This series aims to address these
> > duplicate issues.
> > [1-2/5]: address nmi account duplicate
> > [3-4/5]: address rcu housekeeping duplicate in irq
> > [5/5]: as a natural result of [3-4/5], address a history issue. [1]
> >
> >
> > History:
> > v1 -> v2:
> > change the subject as the motivation varies.
> > add the fix for nmi account duplicate
> >
> > The subject of v1 is "[PATCH 1/3] kernel/irq: __handle_domain_irq()
> > makes irq_enter/exit arch optional". [2] It is brought up to fix [1].
> >
> > There have been some tries to enable crash-stop-NMI on arm64, one by me,
> > the other by Yuichi's [4]. I hope after this series, they can advance,
> > as Marc said in [3] "No additional NMI patches will make it until we
> > have resolved the issues"
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/87lfewnmdz.fsf@nanos.tec.linutronix.de/
> > [2] https://lore.kernel.org/linux-arm-kernel/1607912752-12481-1-git-send-email-kernelfans@gmail.com
> > [3] https://lore.kernel.org/linux-arm-kernel/afd82be798cb55fd2f96940db7be78c0@kernel.org
> > [4] https://lore.kernel.org/linux-arm-kernel/20201104080539.3205889-1-ito-yuichi@fujitsu.com
> >
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > Cc: Will Deacon <will@kernel.org>
> > Cc: Mark Rutland <mark.rutland@arm.com>
> > Cc: Marc Zyngier <maz@kernel.org>
> > Cc: Joey Gouly <joey.gouly@arm.com>
> > Cc: Sami Tolvanen <samitolvanen@google.com>
> > Cc: Julien Thierry <julien.thierry@arm.com>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Yuichi Ito <ito-yuichi@fujitsu.com>
> > Cc: linux-kernel@vger.kernel.org
> > To: linux-arm-kernel@lists.infradead.org
> >
> >
> > Pingfan Liu (5):
> > arm64/entry-common: push the judgement of nmi ahead
> > irqchip/GICv3: expose handle_nmi() directly
> > kernel/irq: make irq_{enter,exit}() in handle_domain_irq() arch
> > optional
> > irqchip/GICv3: let gic_handle_irq() utilize irqentry on arm64
> > irqchip/GICv3: make reschedule-ipi light weight
> >
> > arch/arm64/Kconfig | 1 +
> > arch/arm64/include/asm/irq.h | 7 ++++
> > arch/arm64/kernel/entry-common.c | 45 +++++++++++++++-------
> > arch/arm64/kernel/irq.c | 29 ++++++++++++++
> > drivers/irqchip/irq-gic-v3.c | 66 ++++++++++++++++++++------------
> > kernel/irq/Kconfig | 3 ++
> > kernel/irq/irqdesc.c | 4 ++
> > 7 files changed, 116 insertions(+), 39 deletions(-)
> >
> > --
> > 2.31.1
> >
next prev parent reply other threads:[~2021-09-24 22:59 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-24 13:28 [PATCHv2 0/5] arm64/irqentry: remove duplicate housekeeping of Pingfan Liu
2021-09-24 13:28 ` [PATCHv2 1/5] arm64/entry-common: push the judgement of nmi ahead Pingfan Liu
2021-09-24 17:53 ` Mark Rutland
2021-09-25 15:39 ` Pingfan Liu
2021-09-30 13:32 ` Mark Rutland
2021-10-08 4:01 ` Pingfan Liu
2021-10-08 14:55 ` Pingfan Liu
2021-10-08 17:25 ` Mark Rutland
2021-10-09 3:49 ` Pingfan Liu
2021-10-08 15:45 ` Paul E. McKenney
2021-10-09 4:14 ` Pingfan Liu
2021-09-24 13:28 ` [PATCHv2 2/5] irqchip/GICv3: expose handle_nmi() directly Pingfan Liu
2021-09-24 13:28 ` [PATCHv2 3/5] kernel/irq: make irq_{enter,exit}() in handle_domain_irq() arch optional Pingfan Liu
2021-09-28 8:55 ` Mark Rutland
2021-09-29 3:15 ` Pingfan Liu
2021-09-24 13:28 ` [PATCHv2 4/5] irqchip/GICv3: let gic_handle_irq() utilize irqentry on arm64 Pingfan Liu
2021-09-28 9:10 ` Mark Rutland
2021-09-29 3:10 ` Pingfan Liu
2021-09-29 7:20 ` Marc Zyngier
2021-09-29 8:27 ` Pingfan Liu
2021-09-29 9:23 ` Mark Rutland
2021-09-29 11:40 ` Pingfan Liu
2021-09-29 14:29 ` Pingfan Liu
2021-09-29 17:41 ` Mark Rutland
2021-09-24 13:28 ` [PATCHv2 5/5] irqchip/GICv3: make reschedule-ipi light weight Pingfan Liu
2021-09-29 7:24 ` Marc Zyngier
2021-09-29 8:32 ` Pingfan Liu
2021-09-24 17:36 ` [PATCHv2 0/5] arm64/irqentry: remove duplicate housekeeping of Mark Rutland
2021-09-24 22:59 ` Paul E. McKenney [this message]
2021-09-27 9:23 ` Mark Rutland
2021-09-28 0:09 ` Paul E. McKenney
2021-09-28 8:32 ` Mark Rutland
2021-09-28 8:35 ` Mark Rutland
2021-09-28 9:52 ` Sven Schnelle
2021-09-28 10:26 ` Mark Rutland
2021-09-28 13:55 ` Paul E. McKenney
2021-09-25 15:12 ` Pingfan Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210924225954.GN880162@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=gor@linux.ibm.com \
--cc=ito-yuichi@fujitsu.com \
--cc=joey.gouly@arm.com \
--cc=julien.thierry@arm.com \
--cc=kernelfans@gmail.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=samitolvanen@google.com \
--cc=svens@linux.ibm.com \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).