From: Pingfan Liu <kernelfans@gmail.com> To: linux-arm-kernel@lists.infradead.org Cc: Mark Rutland <mark.rutland@arm.com>, Pingfan Liu <kernelfans@gmail.com>, Marc Zyngier <maz@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Joey Gouly <joey.gouly@arm.com>, Sami Tolvanen <samitolvanen@google.com>, Julien Thierry <julien.thierry@arm.com>, Thomas Gleixner <tglx@linutronix.de>, Yuichi Ito <ito-yuichi@fujitsu.com>, linux-kernel@vger.kernel.org Subject: [PATCHv4 2/3] arm64: entry: refactor EL1 interrupt entry logic Date: Fri, 1 Oct 2021 22:44:05 +0800 [thread overview] Message-ID: <20211001144406.7719-3-kernelfans@gmail.com> (raw) In-Reply-To: <20211001144406.7719-1-kernelfans@gmail.com> From: Mark Rutland <mark.rutland@arm.com> Currently we distinguish IRQ and definitely-PNMI at entry/exit time via the enter_el1_irq_or_nmi() and enter_el1_irq_or_nmi() helpers. In subsequent patches we'll need to handle the two cases more distinctly in the body of the exception handler. To make this possible, this patch refactors el1_interrupt to be a top-level dispatcher to separate handlers for the IRQ and PNMI cases, removing the need for the enter_el1_irq_or_nmi() and exit_el1_irq_or_nmi() helpers. Note that since arm64_enter_nmi() calls __nmi_enter(), which increments the preemt_count, we could never preempt when handling a PNMI. We now only check for preemption in the IRQ case, which makes this clearer. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Pingfan Liu <kernelfans@gmail.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Joey Gouly <joey.gouly@arm.com> Cc: Sami Tolvanen <samitolvanen@google.com> Cc: Julien Thierry <julien.thierry@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Yuichi Ito <ito-yuichi@fujitsu.com> Cc: linux-kernel@vger.kernel.org To: linux-arm-kernel@lists.infradead.org --- arch/arm64/kernel/entry-common.c | 43 ++++++++++++++++---------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 32f9796c4ffe..fecf046f0708 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -219,22 +219,6 @@ static void noinstr arm64_exit_el1_dbg(struct pt_regs *regs) lockdep_hardirqs_on(CALLER_ADDR0); } -static void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs) -{ - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) - arm64_enter_nmi(regs); - else - enter_from_kernel_mode(regs); -} - -static void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs) -{ - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) - arm64_exit_nmi(regs); - else - exit_to_kernel_mode(regs); -} - static void __sched arm64_preempt_schedule_irq(void) { lockdep_assert_irqs_disabled(); @@ -432,14 +416,19 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) } } -static void noinstr el1_interrupt(struct pt_regs *regs, - void (*handler)(struct pt_regs *)) +static __always_inline void +__el1_pnmi(struct pt_regs *regs, void (*handler)(struct pt_regs *)) { - write_sysreg(DAIF_PROCCTX_NOIRQ, daif); - - enter_el1_irq_or_nmi(regs); + arm64_enter_nmi(regs); do_interrupt_handler(regs, handler); + arm64_exit_nmi(regs); +} +static __always_inline void +__el1_interrupt(struct pt_regs *regs, void (*handler)(struct pt_regs *)) +{ + enter_from_kernel_mode(regs); + do_interrupt_handler(regs, handler); /* * Note: thread_info::preempt_count includes both thread_info::count * and thread_info::need_resched, and is not equivalent to @@ -448,8 +437,18 @@ static void noinstr el1_interrupt(struct pt_regs *regs, if (IS_ENABLED(CONFIG_PREEMPTION) && READ_ONCE(current_thread_info()->preempt_count) == 0) arm64_preempt_schedule_irq(); + exit_to_kernel_mode(regs); +} - exit_el1_irq_or_nmi(regs); +static void noinstr el1_interrupt(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) +{ + write_sysreg(DAIF_PROCCTX_NOIRQ, daif); + + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) + __el1_pnmi(regs, handler); + else + __el1_interrupt(regs, handler); } asmlinkage void noinstr el1h_64_irq_handler(struct pt_regs *regs) -- 2.31.1
WARNING: multiple messages have this Message-ID (diff)
From: Pingfan Liu <kernelfans@gmail.com> To: linux-arm-kernel@lists.infradead.org Cc: Mark Rutland <mark.rutland@arm.com>, Pingfan Liu <kernelfans@gmail.com>, Marc Zyngier <maz@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Joey Gouly <joey.gouly@arm.com>, Sami Tolvanen <samitolvanen@google.com>, Julien Thierry <julien.thierry@arm.com>, Thomas Gleixner <tglx@linutronix.de>, Yuichi Ito <ito-yuichi@fujitsu.com>, linux-kernel@vger.kernel.org Subject: [PATCHv4 2/3] arm64: entry: refactor EL1 interrupt entry logic Date: Fri, 1 Oct 2021 22:44:05 +0800 [thread overview] Message-ID: <20211001144406.7719-3-kernelfans@gmail.com> (raw) In-Reply-To: <20211001144406.7719-1-kernelfans@gmail.com> From: Mark Rutland <mark.rutland@arm.com> Currently we distinguish IRQ and definitely-PNMI at entry/exit time via the enter_el1_irq_or_nmi() and enter_el1_irq_or_nmi() helpers. In subsequent patches we'll need to handle the two cases more distinctly in the body of the exception handler. To make this possible, this patch refactors el1_interrupt to be a top-level dispatcher to separate handlers for the IRQ and PNMI cases, removing the need for the enter_el1_irq_or_nmi() and exit_el1_irq_or_nmi() helpers. Note that since arm64_enter_nmi() calls __nmi_enter(), which increments the preemt_count, we could never preempt when handling a PNMI. We now only check for preemption in the IRQ case, which makes this clearer. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Pingfan Liu <kernelfans@gmail.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Joey Gouly <joey.gouly@arm.com> Cc: Sami Tolvanen <samitolvanen@google.com> Cc: Julien Thierry <julien.thierry@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Yuichi Ito <ito-yuichi@fujitsu.com> Cc: linux-kernel@vger.kernel.org To: linux-arm-kernel@lists.infradead.org --- arch/arm64/kernel/entry-common.c | 43 ++++++++++++++++---------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 32f9796c4ffe..fecf046f0708 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -219,22 +219,6 @@ static void noinstr arm64_exit_el1_dbg(struct pt_regs *regs) lockdep_hardirqs_on(CALLER_ADDR0); } -static void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs) -{ - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) - arm64_enter_nmi(regs); - else - enter_from_kernel_mode(regs); -} - -static void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs) -{ - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) - arm64_exit_nmi(regs); - else - exit_to_kernel_mode(regs); -} - static void __sched arm64_preempt_schedule_irq(void) { lockdep_assert_irqs_disabled(); @@ -432,14 +416,19 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) } } -static void noinstr el1_interrupt(struct pt_regs *regs, - void (*handler)(struct pt_regs *)) +static __always_inline void +__el1_pnmi(struct pt_regs *regs, void (*handler)(struct pt_regs *)) { - write_sysreg(DAIF_PROCCTX_NOIRQ, daif); - - enter_el1_irq_or_nmi(regs); + arm64_enter_nmi(regs); do_interrupt_handler(regs, handler); + arm64_exit_nmi(regs); +} +static __always_inline void +__el1_interrupt(struct pt_regs *regs, void (*handler)(struct pt_regs *)) +{ + enter_from_kernel_mode(regs); + do_interrupt_handler(regs, handler); /* * Note: thread_info::preempt_count includes both thread_info::count * and thread_info::need_resched, and is not equivalent to @@ -448,8 +437,18 @@ static void noinstr el1_interrupt(struct pt_regs *regs, if (IS_ENABLED(CONFIG_PREEMPTION) && READ_ONCE(current_thread_info()->preempt_count) == 0) arm64_preempt_schedule_irq(); + exit_to_kernel_mode(regs); +} - exit_el1_irq_or_nmi(regs); +static void noinstr el1_interrupt(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) +{ + write_sysreg(DAIF_PROCCTX_NOIRQ, daif); + + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) + __el1_pnmi(regs, handler); + else + __el1_interrupt(regs, handler); } asmlinkage void noinstr el1h_64_irq_handler(struct pt_regs *regs) -- 2.31.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-10-01 14:44 UTC|newest] Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-10-01 14:44 [PATCHv4 0/3] arm64/irqentry: remove duplicate housekeeping of rcu Pingfan Liu 2021-10-01 14:44 ` Pingfan Liu 2021-10-01 14:44 ` [PATCHv4 1/3] kernel/irq: make irq_{enter,exit}() in handle_domain_irq() arch optional Pingfan Liu 2021-10-01 14:44 ` [PATCHv4 1/3] kernel/irq: make irq_{enter, exit}() " Pingfan Liu 2021-10-04 16:30 ` [PATCHv4 1/3] kernel/irq: make irq_{enter,exit}() " Marc Zyngier 2021-10-04 16:30 ` [PATCHv4 1/3] kernel/irq: make irq_{enter, exit}() " Marc Zyngier 2021-10-01 14:44 ` Pingfan Liu [this message] 2021-10-01 14:44 ` [PATCHv4 2/3] arm64: entry: refactor EL1 interrupt entry logic Pingfan Liu 2021-10-01 14:44 ` [PATCHv4 3/3] arm64: entry: avoid double-accounting IRQ RCU entry Pingfan Liu 2021-10-01 14:44 ` Pingfan Liu 2021-10-04 16:31 ` [PATCHv4 0/3] arm64/irqentry: remove duplicate housekeeping of rcu Marc Zyngier 2021-10-04 16:31 ` Marc Zyngier 2021-10-05 17:17 ` Catalin Marinas 2021-10-05 17:17 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211001144406.7719-3-kernelfans@gmail.com \ --to=kernelfans@gmail.com \ --cc=catalin.marinas@arm.com \ --cc=ito-yuichi@fujitsu.com \ --cc=joey.gouly@arm.com \ --cc=julien.thierry@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=maz@kernel.org \ --cc=samitolvanen@google.com \ --cc=tglx@linutronix.de \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.