From: Mark Rutland <mark.rutland@arm.com>
To: linux-arm-kernel@lists.infradead.org
Cc: catalin.marinas@arm.com, james.morse@arm.com, joey.gouly@arm.com,
mark.rutland@arm.com, maz@kernel.org, will@kernel.org
Subject: [PATCH v3 06/20] arm64: entry: add a call_on_irq_stack helper
Date: Tue, 25 May 2021 19:32:48 +0100 [thread overview]
Message-ID: <20210525183302.56293-7-mark.rutland@arm.com> (raw)
In-Reply-To: <20210525183302.56293-1-mark.rutland@arm.com>
When handling IRQ/FIQ exceptions the entry assembly may transition from
a task's stack to that CPU's IRQ stack (and IRQ shadow call stack).
In subsequent patches we want to migrate the IRQ/FIQ triage logic to C,
and as we want to perform some actions on the task stack (e.g. EL1
preemption), we need to switch stacks within the C handler. So that we
can do so, this patch adds a helper to call a function on a CPU's IRQ
stack (and shadow stack as appropriate).
Subsequent patches will make use of the new helper function.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Will Deacon <will@kernel.org>
---
arch/arm64/include/asm/exception.h | 2 ++
arch/arm64/kernel/entry.S | 36 ++++++++++++++++++++++++++++++++++++
2 files changed, 38 insertions(+)
diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h
index 3a859d4e8b59..c24b69c0c589 100644
--- a/arch/arm64/include/asm/exception.h
+++ b/arch/arm64/include/asm/exception.h
@@ -40,6 +40,8 @@ asmlinkage void el0_error_compat_handler(struct pt_regs *regs);
asmlinkage void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs);
asmlinkage void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs);
+asmlinkage void call_on_irq_stack(struct pt_regs *regs,
+ void (*func)(struct pt_regs *));
asmlinkage void enter_from_user_mode(void);
asmlinkage void exit_to_user_mode(void);
void arm64_enter_nmi(struct pt_regs *regs);
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 8c7ddd651756..3b8dca9c4f2b 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -979,6 +979,42 @@ SYM_CODE_START(ret_from_fork)
SYM_CODE_END(ret_from_fork)
NOKPROBE(ret_from_fork)
+/*
+ * void call_on_irq_stack(struct pt_regs *regs,
+ * void (*func)(struct pt_regs *));
+ *
+ * Calls func(regs) using this CPU's irq stack and shadow irq stack.
+ */
+SYM_FUNC_START(call_on_irq_stack)
+#ifdef CONFIG_SHADOW_CALL_STACK
+ stp scs_sp, xzr, [sp, #-16]!
+ ldr_this_cpu scs_sp, irq_shadow_call_stack_ptr, x17
+#endif
+ /* Create a frame record to save our LR and SP (implicit in FP) */
+ stp x29, x30, [sp, #-16]!
+ mov x29, sp
+
+ ldr_this_cpu x16, irq_stack_ptr, x17
+ mov x15, #IRQ_STACK_SIZE
+ add x16, x16, x15
+
+ /* Move to the new stack and call the function there */
+ mov sp, x16
+ blr x1
+
+ /*
+ * Restore the SP from the FP, and restore the FP and LR from the frame
+ * record.
+ */
+ mov sp, x29
+ ldp x29, x30, [sp], #16
+#ifdef CONFIG_SHADOW_CALL_STACK
+ ldp scs_sp, xzr, [sp], #16
+#endif
+ ret
+SYM_FUNC_END(call_on_irq_stack)
+NOKPROBE(call_on_irq_stack)
+
#ifdef CONFIG_ARM_SDE_INTERFACE
#include <asm/sdei.h>
--
2.11.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-05-25 18:39 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-25 18:32 [PATCH v3 00/20] arm64: entry: migrate more code to C Mark Rutland
2021-05-25 18:32 ` [PATCH v3 01/20] arm64: remove redundant local_daif_mask() in bad_mode() Mark Rutland
2021-05-27 12:34 ` Marc Zyngier
2021-05-25 18:32 ` [PATCH v3 02/20] arm64: entry: unmask IRQ+FIQ after EL0 handling Mark Rutland
2021-06-04 16:47 ` Will Deacon
2021-05-25 18:32 ` [PATCH v3 03/20] arm64: entry: convert SError handlers to C Mark Rutland
2021-05-25 18:32 ` [PATCH v3 04/20] arm64: entry: move arm64_preempt_schedule_irq to entry-common.c Mark Rutland
2021-05-25 18:32 ` [PATCH v3 05/20] arm64: entry: move preempt logic to C Mark Rutland
2021-06-04 15:43 ` Will Deacon
2021-06-04 16:54 ` Mark Rutland
2021-05-25 18:32 ` Mark Rutland [this message]
2021-05-25 18:32 ` [PATCH v3 07/20] arm64: entry: convert IRQ+FIQ handlers " Mark Rutland
2021-05-25 18:32 ` [PATCH v3 08/20] arm64: entry: organise entry handlers consistently Mark Rutland
2021-06-04 16:51 ` Will Deacon
2021-06-04 17:39 ` Mark Rutland
2021-06-04 17:44 ` Will Deacon
2021-06-04 18:01 ` Mark Rutland
2021-05-25 18:32 ` [PATCH v3 09/20] arm64: entry: organise entry vectors consistently Mark Rutland
2021-05-25 18:32 ` [PATCH v3 10/20] arm64: entry: consolidate EL1 exception returns Mark Rutland
2021-05-25 18:32 ` [PATCH v3 11/20] arm64: entry: move bad_mode() to entry-common.c Mark Rutland
2021-06-04 16:57 ` Will Deacon
2021-06-04 17:42 ` Mark Rutland
2021-06-04 17:43 ` Will Deacon
2021-05-25 18:32 ` [PATCH v3 12/20] arm64: entry: improve bad_mode() Mark Rutland
2021-05-25 18:32 ` [PATCH v3 13/20] arm64: entry: template the entry asm functions Mark Rutland
2021-05-25 18:32 ` [PATCH v3 14/20] arm64: entry: handle all vectors with C Mark Rutland
2021-06-04 17:13 ` Will Deacon
2021-05-25 18:32 ` [PATCH v3 15/20] arm64: entry: fold el1_inv() into el1h_64_sync_handler() Mark Rutland
2021-05-25 18:32 ` [PATCH v3 16/20] arm64: entry: split bad stack entry Mark Rutland
2021-05-25 18:32 ` [PATCH v3 17/20] arm64: entry: split SDEI entry Mark Rutland
2021-05-25 18:33 ` [PATCH v3 18/20] arm64: entry: make NMI entry/exit functions static Mark Rutland
2021-05-25 18:33 ` [PATCH v3 19/20] arm64: entry: don't instrument entry code with KCOV Mark Rutland
2021-06-04 17:16 ` Will Deacon
2021-06-04 17:49 ` Mark Rutland
2021-05-25 18:33 ` [PATCH v3 20/20] arm64: idle: don't instrument idle " Mark Rutland
2021-05-27 14:06 ` [PATCH v3 00/20] arm64: entry: migrate more code to C Catalin Marinas
2021-05-27 15:31 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210525183302.56293-7-mark.rutland@arm.com \
--to=mark.rutland@arm.com \
--cc=catalin.marinas@arm.com \
--cc=james.morse@arm.com \
--cc=joey.gouly@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=maz@kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).