* [PATCH 0/5] riscv: SCS support
@ 2023-08-11 23:35 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:35 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
Hi folks,
This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
uses compiler instrumentation to store return addresses in a
separate shadow stack to protect them against accidental or
malicious overwrites. More information about SCS can be found
here:
https://clang.llvm.org/docs/ShadowCallStack.html
Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
handling by adding support for accessing per-CPU variables
directly in assembly. The patch is included in this series to
make IRQ stack switching cleaner with SCS, and I've simply
rebased it. Patch 2 uses this functionality to clean up the stack
switching by moving duplicate code into a single function. On
RISC-V, the compiler uses the gp register for storing the current
shadow call stack pointer, which is incompatible with global
pointer relaxation. Patch 3 moves global pointer loading into a
macro that can be easily disabled with SCS. Patch 4 implements
SCS register loading and switching, and allows the feature to be
enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
when CONFIG_IRQ_STACKS is enabled.
Note that this series requires Clang 17. Earlier Clang versions
support SCS on RISC-V, but use the x18 register instead of gp,
which isn't ideal. gcc has SCS support for arm64, but I'm not
aware of plans to support RISC-V. Once the Zicfiss extension is
ratified, it's probably preferable to use hardware-backed shadow
stacks instead of SCS on hardware that supports the extension,
and we may want to consider implementing CONFIG_DYNAMIC_SCS to
patch between the implementation at runtime (similarly to the
arm64 implementation, which switches to SCS when hardware PAC
support isn't available).
Sami
Deepak Gupta (1):
riscv: VMAP_STACK overflow detection thread-safe
Sami Tolvanen (4):
riscv: Deduplicate IRQ stack switching
riscv: Move global pointer loading to a macro
riscv: Implement Shadow Call Stack
riscv: Use separate IRQ shadow call stacks
arch/riscv/Kconfig | 6 ++
arch/riscv/Makefile | 4 +
arch/riscv/include/asm/asm.h | 35 ++++++++
arch/riscv/include/asm/irq_stack.h | 3 +
arch/riscv/include/asm/scs.h | 54 ++++++++++++
arch/riscv/include/asm/thread_info.h | 16 +++-
arch/riscv/kernel/asm-offsets.c | 4 +
arch/riscv/kernel/entry.S | 126 +++++++++++++--------------
arch/riscv/kernel/head.S | 19 ++--
arch/riscv/kernel/irq.c | 53 ++++++-----
arch/riscv/kernel/suspend_entry.S | 5 +-
arch/riscv/kernel/traps.c | 65 ++------------
arch/riscv/kernel/vdso/Makefile | 2 +-
arch/riscv/purgatory/Makefile | 4 +
14 files changed, 228 insertions(+), 168 deletions(-)
create mode 100644 arch/riscv/include/asm/scs.h
base-commit: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
--
2.41.0.640.ga95def55d0-goog
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH 0/5] riscv: SCS support
@ 2023-08-11 23:35 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:35 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
Hi folks,
This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
uses compiler instrumentation to store return addresses in a
separate shadow stack to protect them against accidental or
malicious overwrites. More information about SCS can be found
here:
https://clang.llvm.org/docs/ShadowCallStack.html
Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
handling by adding support for accessing per-CPU variables
directly in assembly. The patch is included in this series to
make IRQ stack switching cleaner with SCS, and I've simply
rebased it. Patch 2 uses this functionality to clean up the stack
switching by moving duplicate code into a single function. On
RISC-V, the compiler uses the gp register for storing the current
shadow call stack pointer, which is incompatible with global
pointer relaxation. Patch 3 moves global pointer loading into a
macro that can be easily disabled with SCS. Patch 4 implements
SCS register loading and switching, and allows the feature to be
enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
when CONFIG_IRQ_STACKS is enabled.
Note that this series requires Clang 17. Earlier Clang versions
support SCS on RISC-V, but use the x18 register instead of gp,
which isn't ideal. gcc has SCS support for arm64, but I'm not
aware of plans to support RISC-V. Once the Zicfiss extension is
ratified, it's probably preferable to use hardware-backed shadow
stacks instead of SCS on hardware that supports the extension,
and we may want to consider implementing CONFIG_DYNAMIC_SCS to
patch between the implementation at runtime (similarly to the
arm64 implementation, which switches to SCS when hardware PAC
support isn't available).
Sami
Deepak Gupta (1):
riscv: VMAP_STACK overflow detection thread-safe
Sami Tolvanen (4):
riscv: Deduplicate IRQ stack switching
riscv: Move global pointer loading to a macro
riscv: Implement Shadow Call Stack
riscv: Use separate IRQ shadow call stacks
arch/riscv/Kconfig | 6 ++
arch/riscv/Makefile | 4 +
arch/riscv/include/asm/asm.h | 35 ++++++++
arch/riscv/include/asm/irq_stack.h | 3 +
arch/riscv/include/asm/scs.h | 54 ++++++++++++
arch/riscv/include/asm/thread_info.h | 16 +++-
arch/riscv/kernel/asm-offsets.c | 4 +
arch/riscv/kernel/entry.S | 126 +++++++++++++--------------
arch/riscv/kernel/head.S | 19 ++--
arch/riscv/kernel/irq.c | 53 ++++++-----
arch/riscv/kernel/suspend_entry.S | 5 +-
arch/riscv/kernel/traps.c | 65 ++------------
arch/riscv/kernel/vdso/Makefile | 2 +-
arch/riscv/purgatory/Makefile | 4 +
14 files changed, 228 insertions(+), 168 deletions(-)
create mode 100644 arch/riscv/include/asm/scs.h
base-commit: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
--
2.41.0.640.ga95def55d0-goog
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-11 23:35 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:35 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Jisheng Zhang
From: Deepak Gupta <debug@rivosinc.com>
commit 31da94c25aea ("riscv: add VMAP_STACK overflow detection") added
support for CONFIG_VMAP_STACK. If overflow is detected, CPU switches to
`shadow_stack` temporarily before switching finally to per-cpu
`overflow_stack`.
If two CPUs/harts are racing and end up in over flowing kernel stack, one
or both will end up corrupting each other state because `shadow_stack` is
not per-cpu. This patch optimizes per-cpu overflow stack switch by
directly picking per-cpu `overflow_stack` and gets rid of `shadow_stack`.
Following are the changes in this patch
- Defines an asm macro to obtain per-cpu symbols in destination
register.
- In entry.S, when overflow is detected, per-cpu overflow stack is
located using per-cpu asm macro. Computing per-cpu symbol requires
a temporary register. x31 is saved away into CSR_SCRATCH
(CSR_SCRATCH is anyways zero since we're in kernel).
Please see Links for additional relevant disccussion and alternative
solution.
Tested by `echo EXHAUST_STACK > /sys/kernel/debug/provoke-crash/DIRECT`
Kernel crash log below
Insufficient stack space to handle exception!/debug/provoke-crash/DIRECT
Task stack: [0xff20000010a98000..0xff20000010a9c000]
Overflow stack: [0xff600001f7d98370..0xff600001f7d99370]
CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34
Hardware name: riscv-virtio,qemu (DT)
epc : __memset+0x60/0xfc
ra : recursive_loop+0x48/0xc6 [lkdtm]
epc : ffffffff808de0e4 ra : ffffffff0163a752 sp : ff20000010a97e80
gp : ffffffff815c0330 tp : ff600000820ea280 t0 : ff20000010a97e88
t1 : 000000000000002e t2 : 3233206874706564 s0 : ff20000010a982b0
s1 : 0000000000000012 a0 : ff20000010a97e88 a1 : 0000000000000000
a2 : 0000000000000400 a3 : ff20000010a98288 a4 : 0000000000000000
a5 : 0000000000000000 a6 : fffffffffffe43f0 a7 : 00007fffffffffff
s2 : ff20000010a97e88 s3 : ffffffff01644680 s4 : ff20000010a9be90
s5 : ff600000842ba6c0 s6 : 00aaaaaac29e42b0 s7 : 00fffffff0aa3684
s8 : 00aaaaaac2978040 s9 : 0000000000000065 s10: 00ffffff8a7cad10
s11: 00ffffff8a76a4e0 t3 : ffffffff815dbaf4 t4 : ffffffff815dbaf4
t5 : ffffffff815dbab8 t6 : ff20000010a9bb48
status: 0000000200000120 badaddr: ff20000010a97e88 cause: 000000000000000f
Kernel panic - not syncing: Kernel stack overflow
CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34
Hardware name: riscv-virtio,qemu (DT)
Call Trace:
[<ffffffff80006754>] dump_backtrace+0x30/0x38
[<ffffffff808de798>] show_stack+0x40/0x4c
[<ffffffff808ea2a8>] dump_stack_lvl+0x44/0x5c
[<ffffffff808ea2d8>] dump_stack+0x18/0x20
[<ffffffff808dec06>] panic+0x126/0x2fe
[<ffffffff800065ea>] walk_stackframe+0x0/0xf0
[<ffffffff0163a752>] recursive_loop+0x48/0xc6 [lkdtm]
SMP: stopping secondary CPUs
---[ end Kernel panic - not syncing: Kernel stack overflow ]---
Cc: Guo Ren <guoren@kernel.org>
Cc: Jisheng Zhang <jszhang@kernel.org>
Link: https://lore.kernel.org/linux-riscv/Y347B0x4VUNOd6V7@xhacker/T/#t
Link: https://lore.kernel.org/lkml/20221124094845.1907443-1-debug@rivosinc.com/
Signed-off-by: Deepak Gupta <debug@rivosinc.com>
Acked-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/asm.h | 16 +++++++
arch/riscv/include/asm/thread_info.h | 3 --
arch/riscv/kernel/asm-offsets.c | 1 +
arch/riscv/kernel/entry.S | 70 ++++------------------------
arch/riscv/kernel/traps.c | 36 +-------------
5 files changed, 28 insertions(+), 98 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index 114bbadaef41..f403e46e04f2 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -82,6 +82,22 @@
.endr
.endm
+#ifdef CONFIG_32BIT
+#define PER_CPU_OFFSET_SHIFT 2
+#else
+#define PER_CPU_OFFSET_SHIFT 3
+#endif
+
+.macro asm_per_cpu dst sym tmp
+ REG_L \tmp, TASK_TI_CPU_NUM(tp)
+ slli \tmp, \tmp, PER_CPU_OFFSET_SHIFT
+ la \dst, __per_cpu_offset
+ add \dst, \dst, \tmp
+ REG_L \tmp, 0(\dst)
+ la \dst, \sym
+ add \dst, \dst, \tmp
+.endm
+
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
REG_S x6, PT_T1(sp)
diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
index 1833beb00489..d18ce0113ca1 100644
--- a/arch/riscv/include/asm/thread_info.h
+++ b/arch/riscv/include/asm/thread_info.h
@@ -34,9 +34,6 @@
#ifndef __ASSEMBLY__
-extern long shadow_stack[SHADOW_OVERFLOW_STACK_SIZE / sizeof(long)];
-extern unsigned long spin_shadow_stack;
-
#include <asm/processor.h>
#include <asm/csr.h>
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index d6a75aac1d27..9f535d5de33f 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -39,6 +39,7 @@ void asm_offsets(void)
OFFSET(TASK_TI_KERNEL_SP, task_struct, thread_info.kernel_sp);
OFFSET(TASK_TI_USER_SP, task_struct, thread_info.user_sp);
+ OFFSET(TASK_TI_CPU_NUM, task_struct, thread_info.cpu);
OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]);
OFFSET(TASK_THREAD_F1, task_struct, thread.fstate.f[1]);
OFFSET(TASK_THREAD_F2, task_struct, thread.fstate.f[2]);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 143a2bb3e697..3d11aa3af105 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -10,9 +10,11 @@
#include <asm/asm.h>
#include <asm/csr.h>
#include <asm/unistd.h>
+#include <asm/page.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
#include <asm/errata_list.h>
+#include <linux/sizes.h>
SYM_CODE_START(handle_exception)
/*
@@ -170,67 +172,15 @@ SYM_CODE_END(ret_from_exception)
#ifdef CONFIG_VMAP_STACK
SYM_CODE_START_LOCAL(handle_kernel_stack_overflow)
- /*
- * Takes the psuedo-spinlock for the shadow stack, in case multiple
- * harts are concurrently overflowing their kernel stacks. We could
- * store any value here, but since we're overflowing the kernel stack
- * already we only have SP to use as a scratch register. So we just
- * swap in the address of the spinlock, as that's definately non-zero.
- *
- * Pairs with a store_release in handle_bad_stack().
- */
-1: la sp, spin_shadow_stack
- REG_AMOSWAP_AQ sp, sp, (sp)
- bnez sp, 1b
-
- la sp, shadow_stack
- addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
-
- //save caller register to shadow stack
- addi sp, sp, -(PT_SIZE_ON_STACK)
- REG_S x1, PT_RA(sp)
- REG_S x5, PT_T0(sp)
- REG_S x6, PT_T1(sp)
- REG_S x7, PT_T2(sp)
- REG_S x10, PT_A0(sp)
- REG_S x11, PT_A1(sp)
- REG_S x12, PT_A2(sp)
- REG_S x13, PT_A3(sp)
- REG_S x14, PT_A4(sp)
- REG_S x15, PT_A5(sp)
- REG_S x16, PT_A6(sp)
- REG_S x17, PT_A7(sp)
- REG_S x28, PT_T3(sp)
- REG_S x29, PT_T4(sp)
- REG_S x30, PT_T5(sp)
- REG_S x31, PT_T6(sp)
-
- la ra, restore_caller_reg
- tail get_overflow_stack
-
-restore_caller_reg:
- //save per-cpu overflow stack
- REG_S a0, -8(sp)
- //restore caller register from shadow_stack
- REG_L x1, PT_RA(sp)
- REG_L x5, PT_T0(sp)
- REG_L x6, PT_T1(sp)
- REG_L x7, PT_T2(sp)
- REG_L x10, PT_A0(sp)
- REG_L x11, PT_A1(sp)
- REG_L x12, PT_A2(sp)
- REG_L x13, PT_A3(sp)
- REG_L x14, PT_A4(sp)
- REG_L x15, PT_A5(sp)
- REG_L x16, PT_A6(sp)
- REG_L x17, PT_A7(sp)
- REG_L x28, PT_T3(sp)
- REG_L x29, PT_T4(sp)
- REG_L x30, PT_T5(sp)
- REG_L x31, PT_T6(sp)
+ /* we reach here from kernel context, sscratch must be 0 */
+ csrrw x31, CSR_SCRATCH, x31
+ asm_per_cpu sp, overflow_stack, x31
+ li x31, OVERFLOW_STACK_SIZE
+ add sp, sp, x31
+ /* zero out x31 again and restore x31 */
+ xor x31, x31, x31
+ csrrw x31, CSR_SCRATCH, x31
- //load per-cpu overflow stack
- REG_L sp, -8(sp)
addi sp, sp, -(PT_SIZE_ON_STACK)
//save context to overflow stack
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index f910dfccbf5d..deb2144d9143 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -397,48 +397,14 @@ int is_valid_bugaddr(unsigned long pc)
#endif /* CONFIG_GENERIC_BUG */
#ifdef CONFIG_VMAP_STACK
-/*
- * Extra stack space that allows us to provide panic messages when the kernel
- * has overflowed its stack.
- */
-static DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],
+DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],
overflow_stack)__aligned(16);
-/*
- * A temporary stack for use by handle_kernel_stack_overflow. This is used so
- * we can call into C code to get the per-hart overflow stack. Usage of this
- * stack must be protected by spin_shadow_stack.
- */
-long shadow_stack[SHADOW_OVERFLOW_STACK_SIZE/sizeof(long)] __aligned(16);
-
-/*
- * A pseudo spinlock to protect the shadow stack from being used by multiple
- * harts concurrently. This isn't a real spinlock because the lock side must
- * be taken without a valid stack and only a single register, it's only taken
- * while in the process of panicing anyway so the performance and error
- * checking a proper spinlock gives us doesn't matter.
- */
-unsigned long spin_shadow_stack;
-
-asmlinkage unsigned long get_overflow_stack(void)
-{
- return (unsigned long)this_cpu_ptr(overflow_stack) +
- OVERFLOW_STACK_SIZE;
-}
asmlinkage void handle_bad_stack(struct pt_regs *regs)
{
unsigned long tsk_stk = (unsigned long)current->stack;
unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
- /*
- * We're done with the shadow stack by this point, as we're on the
- * overflow stack. Tell any other concurrent overflowing harts that
- * they can proceed with panicing by releasing the pseudo-spinlock.
- *
- * This pairs with an amoswap.aq in handle_kernel_stack_overflow.
- */
- smp_store_release(&spin_shadow_stack, 0);
-
console_verbose();
pr_emerg("Insufficient stack space to handle exception!\n");
--
2.41.0.640.ga95def55d0-goog
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
@ 2023-08-11 23:35 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:35 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Jisheng Zhang
From: Deepak Gupta <debug@rivosinc.com>
commit 31da94c25aea ("riscv: add VMAP_STACK overflow detection") added
support for CONFIG_VMAP_STACK. If overflow is detected, CPU switches to
`shadow_stack` temporarily before switching finally to per-cpu
`overflow_stack`.
If two CPUs/harts are racing and end up in over flowing kernel stack, one
or both will end up corrupting each other state because `shadow_stack` is
not per-cpu. This patch optimizes per-cpu overflow stack switch by
directly picking per-cpu `overflow_stack` and gets rid of `shadow_stack`.
Following are the changes in this patch
- Defines an asm macro to obtain per-cpu symbols in destination
register.
- In entry.S, when overflow is detected, per-cpu overflow stack is
located using per-cpu asm macro. Computing per-cpu symbol requires
a temporary register. x31 is saved away into CSR_SCRATCH
(CSR_SCRATCH is anyways zero since we're in kernel).
Please see Links for additional relevant disccussion and alternative
solution.
Tested by `echo EXHAUST_STACK > /sys/kernel/debug/provoke-crash/DIRECT`
Kernel crash log below
Insufficient stack space to handle exception!/debug/provoke-crash/DIRECT
Task stack: [0xff20000010a98000..0xff20000010a9c000]
Overflow stack: [0xff600001f7d98370..0xff600001f7d99370]
CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34
Hardware name: riscv-virtio,qemu (DT)
epc : __memset+0x60/0xfc
ra : recursive_loop+0x48/0xc6 [lkdtm]
epc : ffffffff808de0e4 ra : ffffffff0163a752 sp : ff20000010a97e80
gp : ffffffff815c0330 tp : ff600000820ea280 t0 : ff20000010a97e88
t1 : 000000000000002e t2 : 3233206874706564 s0 : ff20000010a982b0
s1 : 0000000000000012 a0 : ff20000010a97e88 a1 : 0000000000000000
a2 : 0000000000000400 a3 : ff20000010a98288 a4 : 0000000000000000
a5 : 0000000000000000 a6 : fffffffffffe43f0 a7 : 00007fffffffffff
s2 : ff20000010a97e88 s3 : ffffffff01644680 s4 : ff20000010a9be90
s5 : ff600000842ba6c0 s6 : 00aaaaaac29e42b0 s7 : 00fffffff0aa3684
s8 : 00aaaaaac2978040 s9 : 0000000000000065 s10: 00ffffff8a7cad10
s11: 00ffffff8a76a4e0 t3 : ffffffff815dbaf4 t4 : ffffffff815dbaf4
t5 : ffffffff815dbab8 t6 : ff20000010a9bb48
status: 0000000200000120 badaddr: ff20000010a97e88 cause: 000000000000000f
Kernel panic - not syncing: Kernel stack overflow
CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34
Hardware name: riscv-virtio,qemu (DT)
Call Trace:
[<ffffffff80006754>] dump_backtrace+0x30/0x38
[<ffffffff808de798>] show_stack+0x40/0x4c
[<ffffffff808ea2a8>] dump_stack_lvl+0x44/0x5c
[<ffffffff808ea2d8>] dump_stack+0x18/0x20
[<ffffffff808dec06>] panic+0x126/0x2fe
[<ffffffff800065ea>] walk_stackframe+0x0/0xf0
[<ffffffff0163a752>] recursive_loop+0x48/0xc6 [lkdtm]
SMP: stopping secondary CPUs
---[ end Kernel panic - not syncing: Kernel stack overflow ]---
Cc: Guo Ren <guoren@kernel.org>
Cc: Jisheng Zhang <jszhang@kernel.org>
Link: https://lore.kernel.org/linux-riscv/Y347B0x4VUNOd6V7@xhacker/T/#t
Link: https://lore.kernel.org/lkml/20221124094845.1907443-1-debug@rivosinc.com/
Signed-off-by: Deepak Gupta <debug@rivosinc.com>
Acked-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/asm.h | 16 +++++++
arch/riscv/include/asm/thread_info.h | 3 --
arch/riscv/kernel/asm-offsets.c | 1 +
arch/riscv/kernel/entry.S | 70 ++++------------------------
arch/riscv/kernel/traps.c | 36 +-------------
5 files changed, 28 insertions(+), 98 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index 114bbadaef41..f403e46e04f2 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -82,6 +82,22 @@
.endr
.endm
+#ifdef CONFIG_32BIT
+#define PER_CPU_OFFSET_SHIFT 2
+#else
+#define PER_CPU_OFFSET_SHIFT 3
+#endif
+
+.macro asm_per_cpu dst sym tmp
+ REG_L \tmp, TASK_TI_CPU_NUM(tp)
+ slli \tmp, \tmp, PER_CPU_OFFSET_SHIFT
+ la \dst, __per_cpu_offset
+ add \dst, \dst, \tmp
+ REG_L \tmp, 0(\dst)
+ la \dst, \sym
+ add \dst, \dst, \tmp
+.endm
+
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
REG_S x6, PT_T1(sp)
diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
index 1833beb00489..d18ce0113ca1 100644
--- a/arch/riscv/include/asm/thread_info.h
+++ b/arch/riscv/include/asm/thread_info.h
@@ -34,9 +34,6 @@
#ifndef __ASSEMBLY__
-extern long shadow_stack[SHADOW_OVERFLOW_STACK_SIZE / sizeof(long)];
-extern unsigned long spin_shadow_stack;
-
#include <asm/processor.h>
#include <asm/csr.h>
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index d6a75aac1d27..9f535d5de33f 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -39,6 +39,7 @@ void asm_offsets(void)
OFFSET(TASK_TI_KERNEL_SP, task_struct, thread_info.kernel_sp);
OFFSET(TASK_TI_USER_SP, task_struct, thread_info.user_sp);
+ OFFSET(TASK_TI_CPU_NUM, task_struct, thread_info.cpu);
OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]);
OFFSET(TASK_THREAD_F1, task_struct, thread.fstate.f[1]);
OFFSET(TASK_THREAD_F2, task_struct, thread.fstate.f[2]);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 143a2bb3e697..3d11aa3af105 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -10,9 +10,11 @@
#include <asm/asm.h>
#include <asm/csr.h>
#include <asm/unistd.h>
+#include <asm/page.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
#include <asm/errata_list.h>
+#include <linux/sizes.h>
SYM_CODE_START(handle_exception)
/*
@@ -170,67 +172,15 @@ SYM_CODE_END(ret_from_exception)
#ifdef CONFIG_VMAP_STACK
SYM_CODE_START_LOCAL(handle_kernel_stack_overflow)
- /*
- * Takes the psuedo-spinlock for the shadow stack, in case multiple
- * harts are concurrently overflowing their kernel stacks. We could
- * store any value here, but since we're overflowing the kernel stack
- * already we only have SP to use as a scratch register. So we just
- * swap in the address of the spinlock, as that's definately non-zero.
- *
- * Pairs with a store_release in handle_bad_stack().
- */
-1: la sp, spin_shadow_stack
- REG_AMOSWAP_AQ sp, sp, (sp)
- bnez sp, 1b
-
- la sp, shadow_stack
- addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE
-
- //save caller register to shadow stack
- addi sp, sp, -(PT_SIZE_ON_STACK)
- REG_S x1, PT_RA(sp)
- REG_S x5, PT_T0(sp)
- REG_S x6, PT_T1(sp)
- REG_S x7, PT_T2(sp)
- REG_S x10, PT_A0(sp)
- REG_S x11, PT_A1(sp)
- REG_S x12, PT_A2(sp)
- REG_S x13, PT_A3(sp)
- REG_S x14, PT_A4(sp)
- REG_S x15, PT_A5(sp)
- REG_S x16, PT_A6(sp)
- REG_S x17, PT_A7(sp)
- REG_S x28, PT_T3(sp)
- REG_S x29, PT_T4(sp)
- REG_S x30, PT_T5(sp)
- REG_S x31, PT_T6(sp)
-
- la ra, restore_caller_reg
- tail get_overflow_stack
-
-restore_caller_reg:
- //save per-cpu overflow stack
- REG_S a0, -8(sp)
- //restore caller register from shadow_stack
- REG_L x1, PT_RA(sp)
- REG_L x5, PT_T0(sp)
- REG_L x6, PT_T1(sp)
- REG_L x7, PT_T2(sp)
- REG_L x10, PT_A0(sp)
- REG_L x11, PT_A1(sp)
- REG_L x12, PT_A2(sp)
- REG_L x13, PT_A3(sp)
- REG_L x14, PT_A4(sp)
- REG_L x15, PT_A5(sp)
- REG_L x16, PT_A6(sp)
- REG_L x17, PT_A7(sp)
- REG_L x28, PT_T3(sp)
- REG_L x29, PT_T4(sp)
- REG_L x30, PT_T5(sp)
- REG_L x31, PT_T6(sp)
+ /* we reach here from kernel context, sscratch must be 0 */
+ csrrw x31, CSR_SCRATCH, x31
+ asm_per_cpu sp, overflow_stack, x31
+ li x31, OVERFLOW_STACK_SIZE
+ add sp, sp, x31
+ /* zero out x31 again and restore x31 */
+ xor x31, x31, x31
+ csrrw x31, CSR_SCRATCH, x31
- //load per-cpu overflow stack
- REG_L sp, -8(sp)
addi sp, sp, -(PT_SIZE_ON_STACK)
//save context to overflow stack
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index f910dfccbf5d..deb2144d9143 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -397,48 +397,14 @@ int is_valid_bugaddr(unsigned long pc)
#endif /* CONFIG_GENERIC_BUG */
#ifdef CONFIG_VMAP_STACK
-/*
- * Extra stack space that allows us to provide panic messages when the kernel
- * has overflowed its stack.
- */
-static DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],
+DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],
overflow_stack)__aligned(16);
-/*
- * A temporary stack for use by handle_kernel_stack_overflow. This is used so
- * we can call into C code to get the per-hart overflow stack. Usage of this
- * stack must be protected by spin_shadow_stack.
- */
-long shadow_stack[SHADOW_OVERFLOW_STACK_SIZE/sizeof(long)] __aligned(16);
-
-/*
- * A pseudo spinlock to protect the shadow stack from being used by multiple
- * harts concurrently. This isn't a real spinlock because the lock side must
- * be taken without a valid stack and only a single register, it's only taken
- * while in the process of panicing anyway so the performance and error
- * checking a proper spinlock gives us doesn't matter.
- */
-unsigned long spin_shadow_stack;
-
-asmlinkage unsigned long get_overflow_stack(void)
-{
- return (unsigned long)this_cpu_ptr(overflow_stack) +
- OVERFLOW_STACK_SIZE;
-}
asmlinkage void handle_bad_stack(struct pt_regs *regs)
{
unsigned long tsk_stk = (unsigned long)current->stack;
unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
- /*
- * We're done with the shadow stack by this point, as we're on the
- * overflow stack. Tell any other concurrent overflowing harts that
- * they can proceed with panicing by releasing the pseudo-spinlock.
- *
- * This pairs with an amoswap.aq in handle_kernel_stack_overflow.
- */
- smp_store_release(&spin_shadow_stack, 0);
-
console_verbose();
pr_emerg("Insufficient stack space to handle exception!\n");
--
2.41.0.640.ga95def55d0-goog
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 2/5] riscv: Deduplicate IRQ stack switching
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-11 23:35 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:35 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
With CONFIG_IRQ_STACKS, we switch to a separate per-CPU IRQ stack
before calling handle_riscv_irq or __do_softirq. We currently
have duplicate inline assembly snippets for stack switching in
both code paths. Now that we can access per-CPU variables in
assembly, implement call_on_irq_stack in assembly, and use that
instead of redudant inline assembly.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/include/asm/asm.h | 5 +++++
arch/riscv/include/asm/irq_stack.h | 3 +++
arch/riscv/kernel/entry.S | 32 ++++++++++++++++++++++++++++++
arch/riscv/kernel/irq.c | 32 ++++++++----------------------
arch/riscv/kernel/traps.c | 29 ++++-----------------------
5 files changed, 52 insertions(+), 49 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index f403e46e04f2..13815a7f907a 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -98,6 +98,11 @@
add \dst, \dst, \tmp
.endm
+.macro load_per_cpu dst ptr tmp
+ asm_per_cpu \dst \ptr \tmp
+ REG_L \dst, 0(\dst)
+.endm
+
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
REG_S x6, PT_T1(sp)
diff --git a/arch/riscv/include/asm/irq_stack.h b/arch/riscv/include/asm/irq_stack.h
index e4042d297580..6441ded3b0cf 100644
--- a/arch/riscv/include/asm/irq_stack.h
+++ b/arch/riscv/include/asm/irq_stack.h
@@ -12,6 +12,9 @@
DECLARE_PER_CPU(ulong *, irq_stack_ptr);
+asmlinkage void call_on_irq_stack(struct pt_regs *regs,
+ void (*func)(struct pt_regs *));
+
#ifdef CONFIG_VMAP_STACK
/*
* To ensure that VMAP'd stack overflow detection works correctly, all VMAP'd
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 3d11aa3af105..39875f5e08a6 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -218,6 +218,38 @@ SYM_CODE_START(ret_from_fork)
tail syscall_exit_to_user_mode
SYM_CODE_END(ret_from_fork)
+#ifdef CONFIG_IRQ_STACKS
+/*
+ * void call_on_irq_stack(struct pt_regs *regs,
+ * void (*func)(struct pt_regs *));
+ *
+ * Calls func(regs) using the per-CPU IRQ stack.
+ */
+SYM_FUNC_START(call_on_irq_stack)
+ /* Create a frame record to save ra and s0 (fp) */
+ addi sp, sp, -RISCV_SZPTR
+ REG_S ra, (sp)
+ addi sp, sp, -RISCV_SZPTR
+ REG_S s0, (sp)
+ addi s0, sp, 2*RISCV_SZPTR
+
+ /* Switch to the per-CPU IRQ stack and call the handler */
+ load_per_cpu t0, irq_stack_ptr, t1
+ li t1, IRQ_STACK_SIZE
+ add sp, t0, t1
+ jalr a1
+
+ /* Switch back to the thread stack and restore ra and s0 */
+ addi sp, s0, -2*RISCV_SZPTR
+ REG_L s0, (sp)
+ addi sp, sp, RISCV_SZPTR
+ REG_L ra, (sp)
+ addi sp, sp, RISCV_SZPTR
+
+ ret
+SYM_FUNC_END(call_on_irq_stack)
+#endif /* CONFIG_IRQ_STACKS */
+
/*
* Integer register context switch
* The callee-saved registers must be saved and restored.
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index d0577cc6a081..95dafdcbd135 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -61,32 +61,16 @@ static void init_irq_stacks(void)
#endif /* CONFIG_VMAP_STACK */
#ifdef CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK
+static void ___do_softirq(struct pt_regs *regs)
+{
+ __do_softirq();
+}
+
void do_softirq_own_stack(void)
{
-#ifdef CONFIG_IRQ_STACKS
- if (on_thread_stack()) {
- ulong *sp = per_cpu(irq_stack_ptr, smp_processor_id())
- + IRQ_STACK_SIZE/sizeof(ulong);
- __asm__ __volatile(
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" ra, (sp) \n"
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" s0, (sp) \n"
- "addi s0, sp, 2*"RISCV_SZPTR "\n"
- "move sp, %[sp] \n"
- "call __do_softirq \n"
- "addi sp, s0, -2*"RISCV_SZPTR"\n"
- REG_L" s0, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- REG_L" ra, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- :
- : [sp] "r" (sp)
- : "a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7",
- "t0", "t1", "t2", "t3", "t4", "t5", "t6",
- "memory");
- } else
-#endif
+ if (on_thread_stack())
+ call_on_irq_stack(NULL, ___do_softirq);
+ else
__do_softirq();
}
#endif /* CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK */
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index deb2144d9143..83319b6816da 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -350,31 +350,10 @@ static void noinstr handle_riscv_irq(struct pt_regs *regs)
asmlinkage void noinstr do_irq(struct pt_regs *regs)
{
irqentry_state_t state = irqentry_enter(regs);
-#ifdef CONFIG_IRQ_STACKS
- if (on_thread_stack()) {
- ulong *sp = per_cpu(irq_stack_ptr, smp_processor_id())
- + IRQ_STACK_SIZE/sizeof(ulong);
- __asm__ __volatile(
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" ra, (sp) \n"
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" s0, (sp) \n"
- "addi s0, sp, 2*"RISCV_SZPTR "\n"
- "move sp, %[sp] \n"
- "move a0, %[regs] \n"
- "call handle_riscv_irq \n"
- "addi sp, s0, -2*"RISCV_SZPTR"\n"
- REG_L" s0, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- REG_L" ra, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- :
- : [sp] "r" (sp), [regs] "r" (regs)
- : "a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7",
- "t0", "t1", "t2", "t3", "t4", "t5", "t6",
- "memory");
- } else
-#endif
+
+ if (IS_ENABLED(CONFIG_IRQ_STACKS) && on_thread_stack())
+ call_on_irq_stack(regs, handle_riscv_irq);
+ else
handle_riscv_irq(regs);
irqentry_exit(regs, state);
--
2.41.0.640.ga95def55d0-goog
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 2/5] riscv: Deduplicate IRQ stack switching
@ 2023-08-11 23:35 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:35 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
With CONFIG_IRQ_STACKS, we switch to a separate per-CPU IRQ stack
before calling handle_riscv_irq or __do_softirq. We currently
have duplicate inline assembly snippets for stack switching in
both code paths. Now that we can access per-CPU variables in
assembly, implement call_on_irq_stack in assembly, and use that
instead of redudant inline assembly.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/include/asm/asm.h | 5 +++++
arch/riscv/include/asm/irq_stack.h | 3 +++
arch/riscv/kernel/entry.S | 32 ++++++++++++++++++++++++++++++
arch/riscv/kernel/irq.c | 32 ++++++++----------------------
arch/riscv/kernel/traps.c | 29 ++++-----------------------
5 files changed, 52 insertions(+), 49 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index f403e46e04f2..13815a7f907a 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -98,6 +98,11 @@
add \dst, \dst, \tmp
.endm
+.macro load_per_cpu dst ptr tmp
+ asm_per_cpu \dst \ptr \tmp
+ REG_L \dst, 0(\dst)
+.endm
+
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
REG_S x6, PT_T1(sp)
diff --git a/arch/riscv/include/asm/irq_stack.h b/arch/riscv/include/asm/irq_stack.h
index e4042d297580..6441ded3b0cf 100644
--- a/arch/riscv/include/asm/irq_stack.h
+++ b/arch/riscv/include/asm/irq_stack.h
@@ -12,6 +12,9 @@
DECLARE_PER_CPU(ulong *, irq_stack_ptr);
+asmlinkage void call_on_irq_stack(struct pt_regs *regs,
+ void (*func)(struct pt_regs *));
+
#ifdef CONFIG_VMAP_STACK
/*
* To ensure that VMAP'd stack overflow detection works correctly, all VMAP'd
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 3d11aa3af105..39875f5e08a6 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -218,6 +218,38 @@ SYM_CODE_START(ret_from_fork)
tail syscall_exit_to_user_mode
SYM_CODE_END(ret_from_fork)
+#ifdef CONFIG_IRQ_STACKS
+/*
+ * void call_on_irq_stack(struct pt_regs *regs,
+ * void (*func)(struct pt_regs *));
+ *
+ * Calls func(regs) using the per-CPU IRQ stack.
+ */
+SYM_FUNC_START(call_on_irq_stack)
+ /* Create a frame record to save ra and s0 (fp) */
+ addi sp, sp, -RISCV_SZPTR
+ REG_S ra, (sp)
+ addi sp, sp, -RISCV_SZPTR
+ REG_S s0, (sp)
+ addi s0, sp, 2*RISCV_SZPTR
+
+ /* Switch to the per-CPU IRQ stack and call the handler */
+ load_per_cpu t0, irq_stack_ptr, t1
+ li t1, IRQ_STACK_SIZE
+ add sp, t0, t1
+ jalr a1
+
+ /* Switch back to the thread stack and restore ra and s0 */
+ addi sp, s0, -2*RISCV_SZPTR
+ REG_L s0, (sp)
+ addi sp, sp, RISCV_SZPTR
+ REG_L ra, (sp)
+ addi sp, sp, RISCV_SZPTR
+
+ ret
+SYM_FUNC_END(call_on_irq_stack)
+#endif /* CONFIG_IRQ_STACKS */
+
/*
* Integer register context switch
* The callee-saved registers must be saved and restored.
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index d0577cc6a081..95dafdcbd135 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -61,32 +61,16 @@ static void init_irq_stacks(void)
#endif /* CONFIG_VMAP_STACK */
#ifdef CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK
+static void ___do_softirq(struct pt_regs *regs)
+{
+ __do_softirq();
+}
+
void do_softirq_own_stack(void)
{
-#ifdef CONFIG_IRQ_STACKS
- if (on_thread_stack()) {
- ulong *sp = per_cpu(irq_stack_ptr, smp_processor_id())
- + IRQ_STACK_SIZE/sizeof(ulong);
- __asm__ __volatile(
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" ra, (sp) \n"
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" s0, (sp) \n"
- "addi s0, sp, 2*"RISCV_SZPTR "\n"
- "move sp, %[sp] \n"
- "call __do_softirq \n"
- "addi sp, s0, -2*"RISCV_SZPTR"\n"
- REG_L" s0, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- REG_L" ra, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- :
- : [sp] "r" (sp)
- : "a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7",
- "t0", "t1", "t2", "t3", "t4", "t5", "t6",
- "memory");
- } else
-#endif
+ if (on_thread_stack())
+ call_on_irq_stack(NULL, ___do_softirq);
+ else
__do_softirq();
}
#endif /* CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK */
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index deb2144d9143..83319b6816da 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -350,31 +350,10 @@ static void noinstr handle_riscv_irq(struct pt_regs *regs)
asmlinkage void noinstr do_irq(struct pt_regs *regs)
{
irqentry_state_t state = irqentry_enter(regs);
-#ifdef CONFIG_IRQ_STACKS
- if (on_thread_stack()) {
- ulong *sp = per_cpu(irq_stack_ptr, smp_processor_id())
- + IRQ_STACK_SIZE/sizeof(ulong);
- __asm__ __volatile(
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" ra, (sp) \n"
- "addi sp, sp, -"RISCV_SZPTR "\n"
- REG_S" s0, (sp) \n"
- "addi s0, sp, 2*"RISCV_SZPTR "\n"
- "move sp, %[sp] \n"
- "move a0, %[regs] \n"
- "call handle_riscv_irq \n"
- "addi sp, s0, -2*"RISCV_SZPTR"\n"
- REG_L" s0, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- REG_L" ra, (sp) \n"
- "addi sp, sp, "RISCV_SZPTR "\n"
- :
- : [sp] "r" (sp), [regs] "r" (regs)
- : "a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7",
- "t0", "t1", "t2", "t3", "t4", "t5", "t6",
- "memory");
- } else
-#endif
+
+ if (IS_ENABLED(CONFIG_IRQ_STACKS) && on_thread_stack())
+ call_on_irq_stack(regs, handle_riscv_irq);
+ else
handle_riscv_irq(regs);
irqentry_exit(regs, state);
--
2.41.0.640.ga95def55d0-goog
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 3/5] riscv: Move global pointer loading to a macro
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-11 23:36 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:36 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
In Clang 17, -fsanitize=shadow-call-stack uses the newly declared
platform register gp for storing shadow call stack pointers. As
this is obviously incompatible with gp relaxation, in preparation
for CONFIG_SHADOW_CALL_STACK support, move global pointer loading
to a single macro, which we can cleanly disable when SCS is used
instead.
Link: https://reviews.llvm.org/rGaa1d2693c256
Link: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/include/asm/asm.h | 8 ++++++++
arch/riscv/kernel/entry.S | 6 ++----
arch/riscv/kernel/head.S | 15 +++------------
arch/riscv/kernel/suspend_entry.S | 5 +----
4 files changed, 14 insertions(+), 20 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index 13815a7f907a..c53388f784a2 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -103,6 +103,14 @@
REG_L \dst, 0(\dst)
.endm
+/* load __global_pointer to gp */
+.macro load_global_pointer
+.option push
+.option norelax
+ la gp, __global_pointer$
+.option pop
+.endm
+
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
REG_S x6, PT_T1(sp)
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 39875f5e08a6..2b4248c6b0a9 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -75,10 +75,8 @@ _save_context:
csrw CSR_SCRATCH, x0
/* Load the global pointer */
-.option push
-.option norelax
- la gp, __global_pointer$
-.option pop
+ load_global_pointer
+
move a0, sp /* pt_regs */
la ra, ret_from_exception
diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 11c3b94c4534..79b5a863c782 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -110,10 +110,7 @@ relocate_enable_mmu:
csrw CSR_TVEC, a0
/* Reload the global pointer */
-.option push
-.option norelax
- la gp, __global_pointer$
-.option pop
+ load_global_pointer
/*
* Switch to kernel page tables. A full fence is necessary in order to
@@ -134,10 +131,7 @@ secondary_start_sbi:
csrw CSR_IP, zero
/* Load the global pointer */
- .option push
- .option norelax
- la gp, __global_pointer$
- .option pop
+ load_global_pointer
/*
* Disable FPU & VECTOR to detect illegal usage of
@@ -228,10 +222,7 @@ pmp_done:
#endif /* CONFIG_RISCV_M_MODE */
/* Load the global pointer */
-.option push
-.option norelax
- la gp, __global_pointer$
-.option pop
+ load_global_pointer
/*
* Disable FPU & VECTOR to detect illegal usage of
diff --git a/arch/riscv/kernel/suspend_entry.S b/arch/riscv/kernel/suspend_entry.S
index 12b52afe09a4..556a4b166d8c 100644
--- a/arch/riscv/kernel/suspend_entry.S
+++ b/arch/riscv/kernel/suspend_entry.S
@@ -60,10 +60,7 @@ END(__cpu_suspend_enter)
ENTRY(__cpu_resume_enter)
/* Load the global pointer */
- .option push
- .option norelax
- la gp, __global_pointer$
- .option pop
+ load_global_pointer
#ifdef CONFIG_MMU
/* Save A0 and A1 */
--
2.41.0.640.ga95def55d0-goog
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 3/5] riscv: Move global pointer loading to a macro
@ 2023-08-11 23:36 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:36 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
In Clang 17, -fsanitize=shadow-call-stack uses the newly declared
platform register gp for storing shadow call stack pointers. As
this is obviously incompatible with gp relaxation, in preparation
for CONFIG_SHADOW_CALL_STACK support, move global pointer loading
to a single macro, which we can cleanly disable when SCS is used
instead.
Link: https://reviews.llvm.org/rGaa1d2693c256
Link: https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/include/asm/asm.h | 8 ++++++++
arch/riscv/kernel/entry.S | 6 ++----
arch/riscv/kernel/head.S | 15 +++------------
arch/riscv/kernel/suspend_entry.S | 5 +----
4 files changed, 14 insertions(+), 20 deletions(-)
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index 13815a7f907a..c53388f784a2 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -103,6 +103,14 @@
REG_L \dst, 0(\dst)
.endm
+/* load __global_pointer to gp */
+.macro load_global_pointer
+.option push
+.option norelax
+ la gp, __global_pointer$
+.option pop
+.endm
+
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
REG_S x6, PT_T1(sp)
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 39875f5e08a6..2b4248c6b0a9 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -75,10 +75,8 @@ _save_context:
csrw CSR_SCRATCH, x0
/* Load the global pointer */
-.option push
-.option norelax
- la gp, __global_pointer$
-.option pop
+ load_global_pointer
+
move a0, sp /* pt_regs */
la ra, ret_from_exception
diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 11c3b94c4534..79b5a863c782 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -110,10 +110,7 @@ relocate_enable_mmu:
csrw CSR_TVEC, a0
/* Reload the global pointer */
-.option push
-.option norelax
- la gp, __global_pointer$
-.option pop
+ load_global_pointer
/*
* Switch to kernel page tables. A full fence is necessary in order to
@@ -134,10 +131,7 @@ secondary_start_sbi:
csrw CSR_IP, zero
/* Load the global pointer */
- .option push
- .option norelax
- la gp, __global_pointer$
- .option pop
+ load_global_pointer
/*
* Disable FPU & VECTOR to detect illegal usage of
@@ -228,10 +222,7 @@ pmp_done:
#endif /* CONFIG_RISCV_M_MODE */
/* Load the global pointer */
-.option push
-.option norelax
- la gp, __global_pointer$
-.option pop
+ load_global_pointer
/*
* Disable FPU & VECTOR to detect illegal usage of
diff --git a/arch/riscv/kernel/suspend_entry.S b/arch/riscv/kernel/suspend_entry.S
index 12b52afe09a4..556a4b166d8c 100644
--- a/arch/riscv/kernel/suspend_entry.S
+++ b/arch/riscv/kernel/suspend_entry.S
@@ -60,10 +60,7 @@ END(__cpu_suspend_enter)
ENTRY(__cpu_resume_enter)
/* Load the global pointer */
- .option push
- .option norelax
- la gp, __global_pointer$
- .option pop
+ load_global_pointer
#ifdef CONFIG_MMU
/* Save A0 and A1 */
--
2.41.0.640.ga95def55d0-goog
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 4/5] riscv: Implement Shadow Call Stack
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-11 23:36 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:36 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
Implement CONFIG_SHADOW_CALL_STACK for RISC-V. When enabled, the
compiler injects instructions to all non-leaf C functions to
store the return address to the shadow stack and unconditionally
load it again before returning, which makes it harder to corrupt
the return address through a stack overflow, for example.
The active shadow call stack pointer is stored in the gp
register, which makes SCS incompatible with gp relaxation. Use
--no-relax-gp to ensure gp relaxation is disabled and disable
global pointer loading. Add SCS pointers to struct thread_info,
implement SCS initialization, and task switching
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/Kconfig | 6 ++++
arch/riscv/Makefile | 4 +++
arch/riscv/include/asm/asm.h | 6 ++++
arch/riscv/include/asm/scs.h | 47 ++++++++++++++++++++++++++++
arch/riscv/include/asm/thread_info.h | 13 ++++++++
arch/riscv/kernel/asm-offsets.c | 3 ++
arch/riscv/kernel/entry.S | 11 +++++++
arch/riscv/kernel/head.S | 4 +++
arch/riscv/kernel/vdso/Makefile | 2 +-
arch/riscv/purgatory/Makefile | 4 +++
10 files changed, 99 insertions(+), 1 deletion(-)
create mode 100644 arch/riscv/include/asm/scs.h
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4c07b9189c86..8fe31ec59da4 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -46,6 +46,7 @@ config RISCV
select ARCH_SUPPORTS_HUGETLBFS if MMU
select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
+ select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK
select ARCH_USE_MEMTEST
select ARCH_USE_QUEUED_RWLOCKS
select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
@@ -169,6 +170,11 @@ config GCC_SUPPORTS_DYNAMIC_FTRACE
def_bool CC_IS_GCC
depends on $(cc-option,-fpatchable-function-entry=8)
+config HAVE_SHADOW_CALL_STACK
+ def_bool $(cc-option,-fsanitize=shadow-call-stack)
+ # https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769
+ depends on $(ld-option,--no-relax-gp)
+
config ARCH_MMAP_RND_BITS_MIN
default 18 if 64BIT
default 8
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 6ec6d52a4180..e518a74640fb 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -55,6 +55,10 @@ endif
endif
endif
+ifeq ($(CONFIG_SHADOW_CALL_STACK),y)
+ KBUILD_LDFLAGS += --no-relax-gp
+endif
+
# ISA string setting
riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima
riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index c53388f784a2..ac77738835ae 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -103,6 +103,11 @@
REG_L \dst, 0(\dst)
.endm
+#ifdef CONFIG_SHADOW_CALL_STACK
+/* gp is used as the shadow call stack pointer instead */
+.macro load_global_pointer
+.endm
+#else
/* load __global_pointer to gp */
.macro load_global_pointer
.option push
@@ -110,6 +115,7 @@
la gp, __global_pointer$
.option pop
.endm
+#endif /* CONFIG_SHADOW_CALL_STACK */
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
new file mode 100644
index 000000000000..94726ea773e3
--- /dev/null
+++ b/arch/riscv/include/asm/scs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_SCS_H
+#define _ASM_SCS_H
+
+#ifdef __ASSEMBLY__
+#include <asm/asm-offsets.h>
+
+#ifdef CONFIG_SHADOW_CALL_STACK
+
+/* Load init_shadow_call_stack to gp. */
+.macro scs_load_init_stack
+ la gp, init_shadow_call_stack
+ XIP_FIXUP_OFFSET gp
+.endm
+
+/* Load task_scs_sp(current) to gp. */
+.macro scs_load_current
+ REG_L gp, TASK_TI_SCS_SP(tp)
+.endm
+
+/* Load task_scs_sp(current) to gp, but only if tp has changed. */
+.macro scs_load_current_if_task_changed prev
+ beq \prev, tp, _skip_scs
+ scs_load_current
+_skip_scs:
+.endm
+
+/* Save gp to task_scs_sp(current). */
+.macro scs_save_current
+ REG_S gp, TASK_TI_SCS_SP(tp)
+.endm
+
+#else /* CONFIG_SHADOW_CALL_STACK */
+
+.macro scs_load_init_stack
+.endm
+.macro scs_load_current
+.endm
+.macro scs_load_current_if_task_changed prev
+.endm
+.macro scs_save_current
+.endm
+
+#endif /* CONFIG_SHADOW_CALL_STACK */
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_SCS_H */
diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
index d18ce0113ca1..574779900bfb 100644
--- a/arch/riscv/include/asm/thread_info.h
+++ b/arch/riscv/include/asm/thread_info.h
@@ -57,8 +57,20 @@ struct thread_info {
long user_sp; /* User stack pointer */
int cpu;
unsigned long syscall_work; /* SYSCALL_WORK_ flags */
+#ifdef CONFIG_SHADOW_CALL_STACK
+ void *scs_base;
+ void *scs_sp;
+#endif
};
+#ifdef CONFIG_SHADOW_CALL_STACK
+#define INIT_SCS \
+ .scs_base = init_shadow_call_stack, \
+ .scs_sp = init_shadow_call_stack,
+#else
+#define INIT_SCS
+#endif
+
/*
* macros/functions for gaining access to the thread information structure
*
@@ -68,6 +80,7 @@ struct thread_info {
{ \
.flags = 0, \
.preempt_count = INIT_PREEMPT_COUNT, \
+ INIT_SCS \
}
void arch_release_task_struct(struct task_struct *tsk);
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index 9f535d5de33f..177cef43a2ee 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -38,6 +38,9 @@ void asm_offsets(void)
OFFSET(TASK_TI_PREEMPT_COUNT, task_struct, thread_info.preempt_count);
OFFSET(TASK_TI_KERNEL_SP, task_struct, thread_info.kernel_sp);
OFFSET(TASK_TI_USER_SP, task_struct, thread_info.user_sp);
+#ifdef CONFIG_SHADOW_CALL_STACK
+ OFFSET(TASK_TI_SCS_SP, task_struct, thread_info.scs_sp);
+#endif
OFFSET(TASK_TI_CPU_NUM, task_struct, thread_info.cpu);
OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 2b4248c6b0a9..ad34507d3c96 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -9,6 +9,7 @@
#include <asm/asm.h>
#include <asm/csr.h>
+#include <asm/scs.h>
#include <asm/unistd.h>
#include <asm/page.h>
#include <asm/thread_info.h>
@@ -77,6 +78,9 @@ _save_context:
/* Load the global pointer */
load_global_pointer
+ /* Load the kernel shadow call stack pointer if coming from userspace */
+ scs_load_current_if_task_changed s5
+
move a0, sp /* pt_regs */
la ra, ret_from_exception
@@ -123,6 +127,9 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
addi s0, sp, PT_SIZE_ON_STACK
REG_S s0, TASK_TI_KERNEL_SP(tp)
+ /* Save the kernel shadow call stack pointer */
+ scs_save_current
+
/*
* Save TP into the scratch register , so we can find the kernel data
* structures again.
@@ -277,6 +284,8 @@ SYM_FUNC_START(__switch_to)
REG_S s9, TASK_THREAD_S9_RA(a3)
REG_S s10, TASK_THREAD_S10_RA(a3)
REG_S s11, TASK_THREAD_S11_RA(a3)
+ /* Save the kernel shadow call stack pointer */
+ scs_save_current
/* Restore context from next->thread */
REG_L ra, TASK_THREAD_RA_RA(a4)
REG_L sp, TASK_THREAD_SP_RA(a4)
@@ -294,6 +303,8 @@ SYM_FUNC_START(__switch_to)
REG_L s11, TASK_THREAD_S11_RA(a4)
/* The offset of thread_info in task_struct is zero. */
move tp, a1
+ /* Switch to the next shadow call stack */
+ scs_load_current
ret
SYM_FUNC_END(__switch_to)
diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 79b5a863c782..c3d0ee77483b 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -14,6 +14,7 @@
#include <asm/cpu_ops_sbi.h>
#include <asm/hwcap.h>
#include <asm/image.h>
+#include <asm/scs.h>
#include <asm/xip_fixup.h>
#include "efi-header.S"
@@ -153,6 +154,7 @@ secondary_start_sbi:
XIP_FIXUP_OFFSET a3
add a3, a3, a1
REG_L sp, (a3)
+ scs_load_current
.Lsecondary_start_common:
@@ -293,6 +295,7 @@ clear_bss_done:
la sp, init_thread_union + THREAD_SIZE
XIP_FIXUP_OFFSET sp
addi sp, sp, -PT_SIZE_ON_STACK
+ scs_load_init_stack
#ifdef CONFIG_BUILTIN_DTB
la a0, __dtb_start
XIP_FIXUP_OFFSET a0
@@ -311,6 +314,7 @@ clear_bss_done:
la tp, init_task
la sp, init_thread_union + THREAD_SIZE
addi sp, sp, -PT_SIZE_ON_STACK
+ scs_load_init_stack
#ifdef CONFIG_KASAN
call kasan_early_init
diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
index 6b1dba11bf6d..48c362c0cb3d 100644
--- a/arch/riscv/kernel/vdso/Makefile
+++ b/arch/riscv/kernel/vdso/Makefile
@@ -36,7 +36,7 @@ CPPFLAGS_vdso.lds += -DHAS_VGETTIMEOFDAY
endif
# Disable -pg to prevent insert call site
-CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS)
# Disable profiling and instrumentation for VDSO code
GCOV_PROFILE := n
diff --git a/arch/riscv/purgatory/Makefile b/arch/riscv/purgatory/Makefile
index dc20e166983e..d5d60c040560 100644
--- a/arch/riscv/purgatory/Makefile
+++ b/arch/riscv/purgatory/Makefile
@@ -77,6 +77,10 @@ ifdef CONFIG_STACKPROTECTOR_STRONG
PURGATORY_CFLAGS_REMOVE += -fstack-protector-strong
endif
+ifdef CONFIG_SHADOW_CALL_STACK
+PURGATORY_CFLAGS_REMOVE += $(CC_FLAGS_SCS)
+endif
+
CFLAGS_REMOVE_purgatory.o += $(PURGATORY_CFLAGS_REMOVE)
CFLAGS_purgatory.o += $(PURGATORY_CFLAGS)
--
2.41.0.640.ga95def55d0-goog
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 4/5] riscv: Implement Shadow Call Stack
@ 2023-08-11 23:36 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:36 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
Implement CONFIG_SHADOW_CALL_STACK for RISC-V. When enabled, the
compiler injects instructions to all non-leaf C functions to
store the return address to the shadow stack and unconditionally
load it again before returning, which makes it harder to corrupt
the return address through a stack overflow, for example.
The active shadow call stack pointer is stored in the gp
register, which makes SCS incompatible with gp relaxation. Use
--no-relax-gp to ensure gp relaxation is disabled and disable
global pointer loading. Add SCS pointers to struct thread_info,
implement SCS initialization, and task switching
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/Kconfig | 6 ++++
arch/riscv/Makefile | 4 +++
arch/riscv/include/asm/asm.h | 6 ++++
arch/riscv/include/asm/scs.h | 47 ++++++++++++++++++++++++++++
arch/riscv/include/asm/thread_info.h | 13 ++++++++
arch/riscv/kernel/asm-offsets.c | 3 ++
arch/riscv/kernel/entry.S | 11 +++++++
arch/riscv/kernel/head.S | 4 +++
arch/riscv/kernel/vdso/Makefile | 2 +-
arch/riscv/purgatory/Makefile | 4 +++
10 files changed, 99 insertions(+), 1 deletion(-)
create mode 100644 arch/riscv/include/asm/scs.h
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 4c07b9189c86..8fe31ec59da4 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -46,6 +46,7 @@ config RISCV
select ARCH_SUPPORTS_HUGETLBFS if MMU
select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
+ select ARCH_SUPPORTS_SHADOW_CALL_STACK if HAVE_SHADOW_CALL_STACK
select ARCH_USE_MEMTEST
select ARCH_USE_QUEUED_RWLOCKS
select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
@@ -169,6 +170,11 @@ config GCC_SUPPORTS_DYNAMIC_FTRACE
def_bool CC_IS_GCC
depends on $(cc-option,-fpatchable-function-entry=8)
+config HAVE_SHADOW_CALL_STACK
+ def_bool $(cc-option,-fsanitize=shadow-call-stack)
+ # https://github.com/riscv-non-isa/riscv-elf-psabi-doc/commit/a484e843e6eeb51f0cb7b8819e50da6d2444d769
+ depends on $(ld-option,--no-relax-gp)
+
config ARCH_MMAP_RND_BITS_MIN
default 18 if 64BIT
default 8
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 6ec6d52a4180..e518a74640fb 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -55,6 +55,10 @@ endif
endif
endif
+ifeq ($(CONFIG_SHADOW_CALL_STACK),y)
+ KBUILD_LDFLAGS += --no-relax-gp
+endif
+
# ISA string setting
riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima
riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index c53388f784a2..ac77738835ae 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -103,6 +103,11 @@
REG_L \dst, 0(\dst)
.endm
+#ifdef CONFIG_SHADOW_CALL_STACK
+/* gp is used as the shadow call stack pointer instead */
+.macro load_global_pointer
+.endm
+#else
/* load __global_pointer to gp */
.macro load_global_pointer
.option push
@@ -110,6 +115,7 @@
la gp, __global_pointer$
.option pop
.endm
+#endif /* CONFIG_SHADOW_CALL_STACK */
/* save all GPs except x1 ~ x5 */
.macro save_from_x6_to_x31
diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
new file mode 100644
index 000000000000..94726ea773e3
--- /dev/null
+++ b/arch/riscv/include/asm/scs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_SCS_H
+#define _ASM_SCS_H
+
+#ifdef __ASSEMBLY__
+#include <asm/asm-offsets.h>
+
+#ifdef CONFIG_SHADOW_CALL_STACK
+
+/* Load init_shadow_call_stack to gp. */
+.macro scs_load_init_stack
+ la gp, init_shadow_call_stack
+ XIP_FIXUP_OFFSET gp
+.endm
+
+/* Load task_scs_sp(current) to gp. */
+.macro scs_load_current
+ REG_L gp, TASK_TI_SCS_SP(tp)
+.endm
+
+/* Load task_scs_sp(current) to gp, but only if tp has changed. */
+.macro scs_load_current_if_task_changed prev
+ beq \prev, tp, _skip_scs
+ scs_load_current
+_skip_scs:
+.endm
+
+/* Save gp to task_scs_sp(current). */
+.macro scs_save_current
+ REG_S gp, TASK_TI_SCS_SP(tp)
+.endm
+
+#else /* CONFIG_SHADOW_CALL_STACK */
+
+.macro scs_load_init_stack
+.endm
+.macro scs_load_current
+.endm
+.macro scs_load_current_if_task_changed prev
+.endm
+.macro scs_save_current
+.endm
+
+#endif /* CONFIG_SHADOW_CALL_STACK */
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_SCS_H */
diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
index d18ce0113ca1..574779900bfb 100644
--- a/arch/riscv/include/asm/thread_info.h
+++ b/arch/riscv/include/asm/thread_info.h
@@ -57,8 +57,20 @@ struct thread_info {
long user_sp; /* User stack pointer */
int cpu;
unsigned long syscall_work; /* SYSCALL_WORK_ flags */
+#ifdef CONFIG_SHADOW_CALL_STACK
+ void *scs_base;
+ void *scs_sp;
+#endif
};
+#ifdef CONFIG_SHADOW_CALL_STACK
+#define INIT_SCS \
+ .scs_base = init_shadow_call_stack, \
+ .scs_sp = init_shadow_call_stack,
+#else
+#define INIT_SCS
+#endif
+
/*
* macros/functions for gaining access to the thread information structure
*
@@ -68,6 +80,7 @@ struct thread_info {
{ \
.flags = 0, \
.preempt_count = INIT_PREEMPT_COUNT, \
+ INIT_SCS \
}
void arch_release_task_struct(struct task_struct *tsk);
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index 9f535d5de33f..177cef43a2ee 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -38,6 +38,9 @@ void asm_offsets(void)
OFFSET(TASK_TI_PREEMPT_COUNT, task_struct, thread_info.preempt_count);
OFFSET(TASK_TI_KERNEL_SP, task_struct, thread_info.kernel_sp);
OFFSET(TASK_TI_USER_SP, task_struct, thread_info.user_sp);
+#ifdef CONFIG_SHADOW_CALL_STACK
+ OFFSET(TASK_TI_SCS_SP, task_struct, thread_info.scs_sp);
+#endif
OFFSET(TASK_TI_CPU_NUM, task_struct, thread_info.cpu);
OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 2b4248c6b0a9..ad34507d3c96 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -9,6 +9,7 @@
#include <asm/asm.h>
#include <asm/csr.h>
+#include <asm/scs.h>
#include <asm/unistd.h>
#include <asm/page.h>
#include <asm/thread_info.h>
@@ -77,6 +78,9 @@ _save_context:
/* Load the global pointer */
load_global_pointer
+ /* Load the kernel shadow call stack pointer if coming from userspace */
+ scs_load_current_if_task_changed s5
+
move a0, sp /* pt_regs */
la ra, ret_from_exception
@@ -123,6 +127,9 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
addi s0, sp, PT_SIZE_ON_STACK
REG_S s0, TASK_TI_KERNEL_SP(tp)
+ /* Save the kernel shadow call stack pointer */
+ scs_save_current
+
/*
* Save TP into the scratch register , so we can find the kernel data
* structures again.
@@ -277,6 +284,8 @@ SYM_FUNC_START(__switch_to)
REG_S s9, TASK_THREAD_S9_RA(a3)
REG_S s10, TASK_THREAD_S10_RA(a3)
REG_S s11, TASK_THREAD_S11_RA(a3)
+ /* Save the kernel shadow call stack pointer */
+ scs_save_current
/* Restore context from next->thread */
REG_L ra, TASK_THREAD_RA_RA(a4)
REG_L sp, TASK_THREAD_SP_RA(a4)
@@ -294,6 +303,8 @@ SYM_FUNC_START(__switch_to)
REG_L s11, TASK_THREAD_S11_RA(a4)
/* The offset of thread_info in task_struct is zero. */
move tp, a1
+ /* Switch to the next shadow call stack */
+ scs_load_current
ret
SYM_FUNC_END(__switch_to)
diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
index 79b5a863c782..c3d0ee77483b 100644
--- a/arch/riscv/kernel/head.S
+++ b/arch/riscv/kernel/head.S
@@ -14,6 +14,7 @@
#include <asm/cpu_ops_sbi.h>
#include <asm/hwcap.h>
#include <asm/image.h>
+#include <asm/scs.h>
#include <asm/xip_fixup.h>
#include "efi-header.S"
@@ -153,6 +154,7 @@ secondary_start_sbi:
XIP_FIXUP_OFFSET a3
add a3, a3, a1
REG_L sp, (a3)
+ scs_load_current
.Lsecondary_start_common:
@@ -293,6 +295,7 @@ clear_bss_done:
la sp, init_thread_union + THREAD_SIZE
XIP_FIXUP_OFFSET sp
addi sp, sp, -PT_SIZE_ON_STACK
+ scs_load_init_stack
#ifdef CONFIG_BUILTIN_DTB
la a0, __dtb_start
XIP_FIXUP_OFFSET a0
@@ -311,6 +314,7 @@ clear_bss_done:
la tp, init_task
la sp, init_thread_union + THREAD_SIZE
addi sp, sp, -PT_SIZE_ON_STACK
+ scs_load_init_stack
#ifdef CONFIG_KASAN
call kasan_early_init
diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
index 6b1dba11bf6d..48c362c0cb3d 100644
--- a/arch/riscv/kernel/vdso/Makefile
+++ b/arch/riscv/kernel/vdso/Makefile
@@ -36,7 +36,7 @@ CPPFLAGS_vdso.lds += -DHAS_VGETTIMEOFDAY
endif
# Disable -pg to prevent insert call site
-CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS)
# Disable profiling and instrumentation for VDSO code
GCOV_PROFILE := n
diff --git a/arch/riscv/purgatory/Makefile b/arch/riscv/purgatory/Makefile
index dc20e166983e..d5d60c040560 100644
--- a/arch/riscv/purgatory/Makefile
+++ b/arch/riscv/purgatory/Makefile
@@ -77,6 +77,10 @@ ifdef CONFIG_STACKPROTECTOR_STRONG
PURGATORY_CFLAGS_REMOVE += -fstack-protector-strong
endif
+ifdef CONFIG_SHADOW_CALL_STACK
+PURGATORY_CFLAGS_REMOVE += $(CC_FLAGS_SCS)
+endif
+
CFLAGS_REMOVE_purgatory.o += $(PURGATORY_CFLAGS_REMOVE)
CFLAGS_purgatory.o += $(PURGATORY_CFLAGS)
--
2.41.0.640.ga95def55d0-goog
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 5/5] riscv: Use separate IRQ shadow call stacks
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-11 23:36 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:36 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
When both CONFIG_IRQ_STACKS and SCS are enabled, also use a separate
per-CPU shadow call stack.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/include/asm/scs.h | 7 +++++++
arch/riscv/kernel/entry.S | 7 +++++++
arch/riscv/kernel/irq.c | 21 +++++++++++++++++++++
3 files changed, 35 insertions(+)
diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
index 94726ea773e3..0e45db78b24b 100644
--- a/arch/riscv/include/asm/scs.h
+++ b/arch/riscv/include/asm/scs.h
@@ -13,6 +13,11 @@
XIP_FIXUP_OFFSET gp
.endm
+/* Load the per-CPU IRQ shadow call stack to gp. */
+.macro scs_load_irq_stack tmp
+ load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
+.endm
+
/* Load task_scs_sp(current) to gp. */
.macro scs_load_current
REG_L gp, TASK_TI_SCS_SP(tp)
@@ -34,6 +39,8 @@
.macro scs_load_init_stack
.endm
+.macro scs_load_irq_stack tmp
+.endm
.macro scs_load_current
.endm
.macro scs_load_current_if_task_changed prev
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index ad34507d3c96..c86b76584d2d 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -238,12 +238,19 @@ SYM_FUNC_START(call_on_irq_stack)
REG_S s0, (sp)
addi s0, sp, 2*RISCV_SZPTR
+ /* Switch to the per-CPU shadow call stack */
+ scs_save_current
+ scs_load_irq_stack t0
+
/* Switch to the per-CPU IRQ stack and call the handler */
load_per_cpu t0, irq_stack_ptr, t1
li t1, IRQ_STACK_SIZE
add sp, t0, t1
jalr a1
+ /* Switch back to the thread shadow call stack */
+ scs_load_current
+
/* Switch back to the thread stack and restore ra and s0 */
addi sp, s0, -2*RISCV_SZPTR
REG_L s0, (sp)
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index 95dafdcbd135..7bfea97ee7e7 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -9,6 +9,7 @@
#include <linux/irqchip.h>
#include <linux/irqdomain.h>
#include <linux/module.h>
+#include <linux/scs.h>
#include <linux/seq_file.h>
#include <asm/sbi.h>
#include <asm/smp.h>
@@ -34,6 +35,24 @@ EXPORT_SYMBOL_GPL(riscv_get_intc_hwnode);
#ifdef CONFIG_IRQ_STACKS
#include <asm/irq_stack.h>
+DECLARE_PER_CPU(ulong *, irq_shadow_call_stack_ptr);
+
+#ifdef CONFIG_SHADOW_CALL_STACK
+DEFINE_PER_CPU(ulong *, irq_shadow_call_stack_ptr);
+#endif
+
+static void init_irq_scs(void)
+{
+ int cpu;
+
+ if (!scs_is_enabled())
+ return;
+
+ for_each_possible_cpu(cpu)
+ per_cpu(irq_shadow_call_stack_ptr, cpu) =
+ scs_alloc(cpu_to_node(cpu));
+}
+
DEFINE_PER_CPU(ulong *, irq_stack_ptr);
#ifdef CONFIG_VMAP_STACK
@@ -76,6 +95,7 @@ void do_softirq_own_stack(void)
#endif /* CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK */
#else
+static void init_irq_scs(void) {}
static void init_irq_stacks(void) {}
#endif /* CONFIG_IRQ_STACKS */
@@ -87,6 +107,7 @@ int arch_show_interrupts(struct seq_file *p, int prec)
void __init init_IRQ(void)
{
+ init_irq_scs();
init_irq_stacks();
irqchip_init();
if (!handle_arch_irq)
--
2.41.0.640.ga95def55d0-goog
^ permalink raw reply related [flat|nested] 30+ messages in thread
* [PATCH 5/5] riscv: Use separate IRQ shadow call stacks
@ 2023-08-11 23:36 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-11 23:36 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: Guo Ren, Deepak Gupta, Nathan Chancellor, Nick Desaulniers,
Fangrui Song, linux-riscv, llvm, linux-kernel, Sami Tolvanen
When both CONFIG_IRQ_STACKS and SCS are enabled, also use a separate
per-CPU shadow call stack.
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
arch/riscv/include/asm/scs.h | 7 +++++++
arch/riscv/kernel/entry.S | 7 +++++++
arch/riscv/kernel/irq.c | 21 +++++++++++++++++++++
3 files changed, 35 insertions(+)
diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
index 94726ea773e3..0e45db78b24b 100644
--- a/arch/riscv/include/asm/scs.h
+++ b/arch/riscv/include/asm/scs.h
@@ -13,6 +13,11 @@
XIP_FIXUP_OFFSET gp
.endm
+/* Load the per-CPU IRQ shadow call stack to gp. */
+.macro scs_load_irq_stack tmp
+ load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
+.endm
+
/* Load task_scs_sp(current) to gp. */
.macro scs_load_current
REG_L gp, TASK_TI_SCS_SP(tp)
@@ -34,6 +39,8 @@
.macro scs_load_init_stack
.endm
+.macro scs_load_irq_stack tmp
+.endm
.macro scs_load_current
.endm
.macro scs_load_current_if_task_changed prev
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index ad34507d3c96..c86b76584d2d 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -238,12 +238,19 @@ SYM_FUNC_START(call_on_irq_stack)
REG_S s0, (sp)
addi s0, sp, 2*RISCV_SZPTR
+ /* Switch to the per-CPU shadow call stack */
+ scs_save_current
+ scs_load_irq_stack t0
+
/* Switch to the per-CPU IRQ stack and call the handler */
load_per_cpu t0, irq_stack_ptr, t1
li t1, IRQ_STACK_SIZE
add sp, t0, t1
jalr a1
+ /* Switch back to the thread shadow call stack */
+ scs_load_current
+
/* Switch back to the thread stack and restore ra and s0 */
addi sp, s0, -2*RISCV_SZPTR
REG_L s0, (sp)
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index 95dafdcbd135..7bfea97ee7e7 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -9,6 +9,7 @@
#include <linux/irqchip.h>
#include <linux/irqdomain.h>
#include <linux/module.h>
+#include <linux/scs.h>
#include <linux/seq_file.h>
#include <asm/sbi.h>
#include <asm/smp.h>
@@ -34,6 +35,24 @@ EXPORT_SYMBOL_GPL(riscv_get_intc_hwnode);
#ifdef CONFIG_IRQ_STACKS
#include <asm/irq_stack.h>
+DECLARE_PER_CPU(ulong *, irq_shadow_call_stack_ptr);
+
+#ifdef CONFIG_SHADOW_CALL_STACK
+DEFINE_PER_CPU(ulong *, irq_shadow_call_stack_ptr);
+#endif
+
+static void init_irq_scs(void)
+{
+ int cpu;
+
+ if (!scs_is_enabled())
+ return;
+
+ for_each_possible_cpu(cpu)
+ per_cpu(irq_shadow_call_stack_ptr, cpu) =
+ scs_alloc(cpu_to_node(cpu));
+}
+
DEFINE_PER_CPU(ulong *, irq_stack_ptr);
#ifdef CONFIG_VMAP_STACK
@@ -76,6 +95,7 @@ void do_softirq_own_stack(void)
#endif /* CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK */
#else
+static void init_irq_scs(void) {}
static void init_irq_stacks(void) {}
#endif /* CONFIG_IRQ_STACKS */
@@ -87,6 +107,7 @@ int arch_show_interrupts(struct seq_file *p, int prec)
void __init init_IRQ(void)
{
+ init_irq_scs();
init_irq_stacks();
irqchip_init();
if (!handle_arch_irq)
--
2.41.0.640.ga95def55d0-goog
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 30+ messages in thread
* Re: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-12 14:35 ` kernel test robot
-1 siblings, 0 replies; 30+ messages in thread
From: kernel test robot @ 2023-08-12 14:35 UTC (permalink / raw)
To: Sami Tolvanen, Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: llvm, oe-kbuild-all, Guo Ren, Deepak Gupta, Nathan Chancellor,
Nick Desaulniers, Fangrui Song, linux-riscv, linux-kernel,
Jisheng Zhang
Hi Sami,
kernel test robot noticed the following build errors:
[auto build test ERROR on 52a93d39b17dc7eb98b6aa3edb93943248e03b2f]
url: https://github.com/intel-lab-lkp/linux/commits/Sami-Tolvanen/riscv-VMAP_STACK-overflow-detection-thread-safe/20230812-073751
base: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
patch link: https://lore.kernel.org/r/20230811233556.97161-8-samitolvanen%40google.com
patch subject: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
config: riscv-randconfig-r042-20230812 (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/config)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
reproduce: (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308122238.XLMMmeL0-lkp@intel.com/
All errors (new ones prefixed by >>):
>> ld.lld: error: undefined symbol: __per_cpu_offset
>>> referenced by arch/riscv/kernel/entry.o:(handle_kernel_stack_overflow) in archive vmlinux.a
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
@ 2023-08-12 14:35 ` kernel test robot
0 siblings, 0 replies; 30+ messages in thread
From: kernel test robot @ 2023-08-12 14:35 UTC (permalink / raw)
To: Sami Tolvanen, Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook
Cc: llvm, oe-kbuild-all, Guo Ren, Deepak Gupta, Nathan Chancellor,
Nick Desaulniers, Fangrui Song, linux-riscv, linux-kernel,
Jisheng Zhang
Hi Sami,
kernel test robot noticed the following build errors:
[auto build test ERROR on 52a93d39b17dc7eb98b6aa3edb93943248e03b2f]
url: https://github.com/intel-lab-lkp/linux/commits/Sami-Tolvanen/riscv-VMAP_STACK-overflow-detection-thread-safe/20230812-073751
base: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
patch link: https://lore.kernel.org/r/20230811233556.97161-8-samitolvanen%40google.com
patch subject: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
config: riscv-randconfig-r042-20230812 (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/config)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
reproduce: (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308122238.XLMMmeL0-lkp@intel.com/
All errors (new ones prefixed by >>):
>> ld.lld: error: undefined symbol: __per_cpu_offset
>>> referenced by arch/riscv/kernel/entry.o:(handle_kernel_stack_overflow) in archive vmlinux.a
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
2023-08-12 14:35 ` kernel test robot
@ 2023-08-13 1:25 ` Guo Ren
-1 siblings, 0 replies; 30+ messages in thread
From: Guo Ren @ 2023-08-13 1:25 UTC (permalink / raw)
To: Sami Tolvanen
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, llvm,
kernel test robot, oe-kbuild-all, Deepak Gupta,
Nathan Chancellor, Nick Desaulniers, Fangrui Song, linux-riscv,
linux-kernel, Jisheng Zhang
On Sat, Aug 12, 2023 at 10:36 PM kernel test robot <lkp@intel.com> wrote:
>
> Hi Sami,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on 52a93d39b17dc7eb98b6aa3edb93943248e03b2f]
>
> url: https://github.com/intel-lab-lkp/linux/commits/Sami-Tolvanen/riscv-VMAP_STACK-overflow-detection-thread-safe/20230812-073751
> base: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
> patch link: https://lore.kernel.org/r/20230811233556.97161-8-samitolvanen%40google.com
> patch subject: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
> config: riscv-randconfig-r042-20230812 (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/config)
> compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
> reproduce: (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/reproduce)
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202308122238.XLMMmeL0-lkp@intel.com/
>
> All errors (new ones prefixed by >>):
>
> >> ld.lld: error: undefined symbol: __per_cpu_offset
> >>> referenced by arch/riscv/kernel/entry.o:(handle_kernel_stack_overflow) in archive vmlinux.a
!CONFIG_SMP missed
>
> --
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
--
Best Regards
Guo Ren
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
@ 2023-08-13 1:25 ` Guo Ren
0 siblings, 0 replies; 30+ messages in thread
From: Guo Ren @ 2023-08-13 1:25 UTC (permalink / raw)
To: Sami Tolvanen
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, llvm,
kernel test robot, oe-kbuild-all, Deepak Gupta,
Nathan Chancellor, Nick Desaulniers, Fangrui Song, linux-riscv,
linux-kernel, Jisheng Zhang
On Sat, Aug 12, 2023 at 10:36 PM kernel test robot <lkp@intel.com> wrote:
>
> Hi Sami,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on 52a93d39b17dc7eb98b6aa3edb93943248e03b2f]
>
> url: https://github.com/intel-lab-lkp/linux/commits/Sami-Tolvanen/riscv-VMAP_STACK-overflow-detection-thread-safe/20230812-073751
> base: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
> patch link: https://lore.kernel.org/r/20230811233556.97161-8-samitolvanen%40google.com
> patch subject: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
> config: riscv-randconfig-r042-20230812 (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/config)
> compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
> reproduce: (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/reproduce)
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202308122238.XLMMmeL0-lkp@intel.com/
>
> All errors (new ones prefixed by >>):
>
> >> ld.lld: error: undefined symbol: __per_cpu_offset
> >>> referenced by arch/riscv/kernel/entry.o:(handle_kernel_stack_overflow) in archive vmlinux.a
!CONFIG_SMP missed
>
> --
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
--
Best Regards
Guo Ren
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
2023-08-13 1:25 ` Guo Ren
@ 2023-08-14 15:34 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 15:34 UTC (permalink / raw)
To: Guo Ren
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, llvm,
kernel test robot, oe-kbuild-all, Deepak Gupta,
Nathan Chancellor, Nick Desaulniers, Fangrui Song, linux-riscv,
linux-kernel, Jisheng Zhang
On Sat, Aug 12, 2023 at 6:25 PM Guo Ren <guoren@kernel.org> wrote:
>
> On Sat, Aug 12, 2023 at 10:36 PM kernel test robot <lkp@intel.com> wrote:
> >
> > Hi Sami,
> >
> > kernel test robot noticed the following build errors:
> >
> > [auto build test ERROR on 52a93d39b17dc7eb98b6aa3edb93943248e03b2f]
> >
> > url: https://github.com/intel-lab-lkp/linux/commits/Sami-Tolvanen/riscv-VMAP_STACK-overflow-detection-thread-safe/20230812-073751
> > base: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
> > patch link: https://lore.kernel.org/r/20230811233556.97161-8-samitolvanen%40google.com
> > patch subject: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
> > config: riscv-randconfig-r042-20230812 (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/config)
> > compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
> > reproduce: (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/reproduce)
> >
> > If you fix the issue in a separate patch/commit (i.e. not just a new version of
> > the same patch/commit), kindly add following tags
> > | Reported-by: kernel test robot <lkp@intel.com>
> > | Closes: https://lore.kernel.org/oe-kbuild-all/202308122238.XLMMmeL0-lkp@intel.com/
> >
> > All errors (new ones prefixed by >>):
> >
> > >> ld.lld: error: undefined symbol: __per_cpu_offset
> > >>> referenced by arch/riscv/kernel/entry.o:(handle_kernel_stack_overflow) in archive vmlinux.a
> !CONFIG_SMP missed
Indeed. I'll fix this in v2.
Sami
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
@ 2023-08-14 15:34 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 15:34 UTC (permalink / raw)
To: Guo Ren
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, llvm,
kernel test robot, oe-kbuild-all, Deepak Gupta,
Nathan Chancellor, Nick Desaulniers, Fangrui Song, linux-riscv,
linux-kernel, Jisheng Zhang
On Sat, Aug 12, 2023 at 6:25 PM Guo Ren <guoren@kernel.org> wrote:
>
> On Sat, Aug 12, 2023 at 10:36 PM kernel test robot <lkp@intel.com> wrote:
> >
> > Hi Sami,
> >
> > kernel test robot noticed the following build errors:
> >
> > [auto build test ERROR on 52a93d39b17dc7eb98b6aa3edb93943248e03b2f]
> >
> > url: https://github.com/intel-lab-lkp/linux/commits/Sami-Tolvanen/riscv-VMAP_STACK-overflow-detection-thread-safe/20230812-073751
> > base: 52a93d39b17dc7eb98b6aa3edb93943248e03b2f
> > patch link: https://lore.kernel.org/r/20230811233556.97161-8-samitolvanen%40google.com
> > patch subject: [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe
> > config: riscv-randconfig-r042-20230812 (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/config)
> > compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a)
> > reproduce: (https://download.01.org/0day-ci/archive/20230812/202308122238.XLMMmeL0-lkp@intel.com/reproduce)
> >
> > If you fix the issue in a separate patch/commit (i.e. not just a new version of
> > the same patch/commit), kindly add following tags
> > | Reported-by: kernel test robot <lkp@intel.com>
> > | Closes: https://lore.kernel.org/oe-kbuild-all/202308122238.XLMMmeL0-lkp@intel.com/
> >
> > All errors (new ones prefixed by >>):
> >
> > >> ld.lld: error: undefined symbol: __per_cpu_offset
> > >>> referenced by arch/riscv/kernel/entry.o:(handle_kernel_stack_overflow) in archive vmlinux.a
> !CONFIG_SMP missed
Indeed. I'll fix this in v2.
Sami
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
2023-08-11 23:35 ` Sami Tolvanen
@ 2023-08-14 17:59 ` Nathan Chancellor
-1 siblings, 0 replies; 30+ messages in thread
From: Nathan Chancellor @ 2023-08-14 17:59 UTC (permalink / raw)
To: Sami Tolvanen
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, Guo Ren,
Deepak Gupta, Nick Desaulniers, Fangrui Song, linux-riscv, llvm,
linux-kernel
Hi Sami,
On Fri, Aug 11, 2023 at 11:35:57PM +0000, Sami Tolvanen wrote:
> Hi folks,
>
> This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
> uses compiler instrumentation to store return addresses in a
> separate shadow stack to protect them against accidental or
> malicious overwrites. More information about SCS can be found
> here:
>
> https://clang.llvm.org/docs/ShadowCallStack.html
>
> Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
> handling by adding support for accessing per-CPU variables
> directly in assembly. The patch is included in this series to
> make IRQ stack switching cleaner with SCS, and I've simply
> rebased it. Patch 2 uses this functionality to clean up the stack
> switching by moving duplicate code into a single function. On
> RISC-V, the compiler uses the gp register for storing the current
> shadow call stack pointer, which is incompatible with global
> pointer relaxation. Patch 3 moves global pointer loading into a
> macro that can be easily disabled with SCS. Patch 4 implements
> SCS register loading and switching, and allows the feature to be
> enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
> when CONFIG_IRQ_STACKS is enabled.
>
> Note that this series requires Clang 17. Earlier Clang versions
> support SCS on RISC-V, but use the x18 register instead of gp,
> which isn't ideal. gcc has SCS support for arm64, but I'm not
> aware of plans to support RISC-V. Once the Zicfiss extension is
> ratified, it's probably preferable to use hardware-backed shadow
> stacks instead of SCS on hardware that supports the extension,
> and we may want to consider implementing CONFIG_DYNAMIC_SCS to
> patch between the implementation at runtime (similarly to the
> arm64 implementation, which switches to SCS when hardware PAC
> support isn't available).
I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
the CFI_BACKWARDS LKDTM test does not pass with
CONFIG_SHADOW_CALL_STACK=y.
[ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
[ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
[ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
[ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
Does the test need to be adjusted or is there some other issue?
Cheers,
Nathan
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
@ 2023-08-14 17:59 ` Nathan Chancellor
0 siblings, 0 replies; 30+ messages in thread
From: Nathan Chancellor @ 2023-08-14 17:59 UTC (permalink / raw)
To: Sami Tolvanen
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, Guo Ren,
Deepak Gupta, Nick Desaulniers, Fangrui Song, linux-riscv, llvm,
linux-kernel
Hi Sami,
On Fri, Aug 11, 2023 at 11:35:57PM +0000, Sami Tolvanen wrote:
> Hi folks,
>
> This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
> uses compiler instrumentation to store return addresses in a
> separate shadow stack to protect them against accidental or
> malicious overwrites. More information about SCS can be found
> here:
>
> https://clang.llvm.org/docs/ShadowCallStack.html
>
> Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
> handling by adding support for accessing per-CPU variables
> directly in assembly. The patch is included in this series to
> make IRQ stack switching cleaner with SCS, and I've simply
> rebased it. Patch 2 uses this functionality to clean up the stack
> switching by moving duplicate code into a single function. On
> RISC-V, the compiler uses the gp register for storing the current
> shadow call stack pointer, which is incompatible with global
> pointer relaxation. Patch 3 moves global pointer loading into a
> macro that can be easily disabled with SCS. Patch 4 implements
> SCS register loading and switching, and allows the feature to be
> enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
> when CONFIG_IRQ_STACKS is enabled.
>
> Note that this series requires Clang 17. Earlier Clang versions
> support SCS on RISC-V, but use the x18 register instead of gp,
> which isn't ideal. gcc has SCS support for arm64, but I'm not
> aware of plans to support RISC-V. Once the Zicfiss extension is
> ratified, it's probably preferable to use hardware-backed shadow
> stacks instead of SCS on hardware that supports the extension,
> and we may want to consider implementing CONFIG_DYNAMIC_SCS to
> patch between the implementation at runtime (similarly to the
> arm64 implementation, which switches to SCS when hardware PAC
> support isn't available).
I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
the CFI_BACKWARDS LKDTM test does not pass with
CONFIG_SHADOW_CALL_STACK=y.
[ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
[ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
[ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
[ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
Does the test need to be adjusted or is there some other issue?
Cheers,
Nathan
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
2023-08-14 17:59 ` Nathan Chancellor
@ 2023-08-14 18:33 ` Kees Cook
-1 siblings, 0 replies; 30+ messages in thread
From: Kees Cook @ 2023-08-14 18:33 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Sami Tolvanen, Paul Walmsley, Palmer Dabbelt, Albert Ou, Guo Ren,
Deepak Gupta, Nick Desaulniers, Fangrui Song, linux-riscv, llvm,
linux-kernel
On Mon, Aug 14, 2023 at 10:59:28AM -0700, Nathan Chancellor wrote:
> Hi Sami,
>
> On Fri, Aug 11, 2023 at 11:35:57PM +0000, Sami Tolvanen wrote:
> > Hi folks,
> >
> > This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
> > uses compiler instrumentation to store return addresses in a
> > separate shadow stack to protect them against accidental or
> > malicious overwrites. More information about SCS can be found
> > here:
> >
> > https://clang.llvm.org/docs/ShadowCallStack.html
> >
> > Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
> > handling by adding support for accessing per-CPU variables
> > directly in assembly. The patch is included in this series to
> > make IRQ stack switching cleaner with SCS, and I've simply
> > rebased it. Patch 2 uses this functionality to clean up the stack
> > switching by moving duplicate code into a single function. On
> > RISC-V, the compiler uses the gp register for storing the current
> > shadow call stack pointer, which is incompatible with global
> > pointer relaxation. Patch 3 moves global pointer loading into a
> > macro that can be easily disabled with SCS. Patch 4 implements
> > SCS register loading and switching, and allows the feature to be
> > enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
> > when CONFIG_IRQ_STACKS is enabled.
> >
> > Note that this series requires Clang 17. Earlier Clang versions
> > support SCS on RISC-V, but use the x18 register instead of gp,
> > which isn't ideal. gcc has SCS support for arm64, but I'm not
> > aware of plans to support RISC-V. Once the Zicfiss extension is
> > ratified, it's probably preferable to use hardware-backed shadow
> > stacks instead of SCS on hardware that supports the extension,
> > and we may want to consider implementing CONFIG_DYNAMIC_SCS to
> > patch between the implementation at runtime (similarly to the
> > arm64 implementation, which switches to SCS when hardware PAC
> > support isn't available).
>
> I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
> within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
> the CFI_BACKWARDS LKDTM test does not pass with
> CONFIG_SHADOW_CALL_STACK=y.
>
> [ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
> [ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
> [ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
> [ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
>
> Does the test need to be adjusted or is there some other issue?
Does it pass without the series? I tried to write it to be
arch-agnostic, but I never tested it on RISC-V. It's very possible that
test needs adjusting for the architecture. Besides the label horrors,
the use of __builtin_frame_address may not work there either...
--
Kees Cook
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
@ 2023-08-14 18:33 ` Kees Cook
0 siblings, 0 replies; 30+ messages in thread
From: Kees Cook @ 2023-08-14 18:33 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Sami Tolvanen, Paul Walmsley, Palmer Dabbelt, Albert Ou, Guo Ren,
Deepak Gupta, Nick Desaulniers, Fangrui Song, linux-riscv, llvm,
linux-kernel
On Mon, Aug 14, 2023 at 10:59:28AM -0700, Nathan Chancellor wrote:
> Hi Sami,
>
> On Fri, Aug 11, 2023 at 11:35:57PM +0000, Sami Tolvanen wrote:
> > Hi folks,
> >
> > This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
> > uses compiler instrumentation to store return addresses in a
> > separate shadow stack to protect them against accidental or
> > malicious overwrites. More information about SCS can be found
> > here:
> >
> > https://clang.llvm.org/docs/ShadowCallStack.html
> >
> > Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
> > handling by adding support for accessing per-CPU variables
> > directly in assembly. The patch is included in this series to
> > make IRQ stack switching cleaner with SCS, and I've simply
> > rebased it. Patch 2 uses this functionality to clean up the stack
> > switching by moving duplicate code into a single function. On
> > RISC-V, the compiler uses the gp register for storing the current
> > shadow call stack pointer, which is incompatible with global
> > pointer relaxation. Patch 3 moves global pointer loading into a
> > macro that can be easily disabled with SCS. Patch 4 implements
> > SCS register loading and switching, and allows the feature to be
> > enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
> > when CONFIG_IRQ_STACKS is enabled.
> >
> > Note that this series requires Clang 17. Earlier Clang versions
> > support SCS on RISC-V, but use the x18 register instead of gp,
> > which isn't ideal. gcc has SCS support for arm64, but I'm not
> > aware of plans to support RISC-V. Once the Zicfiss extension is
> > ratified, it's probably preferable to use hardware-backed shadow
> > stacks instead of SCS on hardware that supports the extension,
> > and we may want to consider implementing CONFIG_DYNAMIC_SCS to
> > patch between the implementation at runtime (similarly to the
> > arm64 implementation, which switches to SCS when hardware PAC
> > support isn't available).
>
> I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
> within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
> the CFI_BACKWARDS LKDTM test does not pass with
> CONFIG_SHADOW_CALL_STACK=y.
>
> [ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
> [ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
> [ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
> [ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
>
> Does the test need to be adjusted or is there some other issue?
Does it pass without the series? I tried to write it to be
arch-agnostic, but I never tested it on RISC-V. It's very possible that
test needs adjusting for the architecture. Besides the label horrors,
the use of __builtin_frame_address may not work there either...
--
Kees Cook
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
2023-08-14 17:59 ` Nathan Chancellor
@ 2023-08-14 18:33 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 18:33 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, Guo Ren,
Deepak Gupta, Nick Desaulniers, Fangrui Song, linux-riscv, llvm,
linux-kernel
On Mon, Aug 14, 2023 at 10:59 AM Nathan Chancellor <nathan@kernel.org> wrote:
> I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
> within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
> the CFI_BACKWARDS LKDTM test does not pass with
> CONFIG_SHADOW_CALL_STACK=y.
>
> [ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
> [ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
> [ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
> [ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
>
> Does the test need to be adjusted or is there some other issue?
The test doesn't work on RISC-V. set_return_addr_unchecked thinks 0x2
is the return address, so I assume the __builtin_frame_address logic
isn't quite right here. Kees, any thoughts?
Sami
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
@ 2023-08-14 18:33 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 18:33 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Kees Cook, Guo Ren,
Deepak Gupta, Nick Desaulniers, Fangrui Song, linux-riscv, llvm,
linux-kernel
On Mon, Aug 14, 2023 at 10:59 AM Nathan Chancellor <nathan@kernel.org> wrote:
> I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
> within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
> the CFI_BACKWARDS LKDTM test does not pass with
> CONFIG_SHADOW_CALL_STACK=y.
>
> [ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
> [ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
> [ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
> [ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
>
> Does the test need to be adjusted or is there some other issue?
The test doesn't work on RISC-V. set_return_addr_unchecked thinks 0x2
is the return address, so I assume the __builtin_frame_address logic
isn't quite right here. Kees, any thoughts?
Sami
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
2023-08-14 18:33 ` Kees Cook
@ 2023-08-14 18:57 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 18:57 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Guo Ren, Deepak Gupta, Nick Desaulniers, Fangrui Song,
linux-riscv, llvm, linux-kernel
On Mon, Aug 14, 2023 at 11:33 AM Kees Cook <keescook@chromium.org> wrote:
>
> On Mon, Aug 14, 2023 at 10:59:28AM -0700, Nathan Chancellor wrote:
> > Hi Sami,
> >
> > On Fri, Aug 11, 2023 at 11:35:57PM +0000, Sami Tolvanen wrote:
> > > Hi folks,
> > >
> > > This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
> > > uses compiler instrumentation to store return addresses in a
> > > separate shadow stack to protect them against accidental or
> > > malicious overwrites. More information about SCS can be found
> > > here:
> > >
> > > https://clang.llvm.org/docs/ShadowCallStack.html
> > >
> > > Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
> > > handling by adding support for accessing per-CPU variables
> > > directly in assembly. The patch is included in this series to
> > > make IRQ stack switching cleaner with SCS, and I've simply
> > > rebased it. Patch 2 uses this functionality to clean up the stack
> > > switching by moving duplicate code into a single function. On
> > > RISC-V, the compiler uses the gp register for storing the current
> > > shadow call stack pointer, which is incompatible with global
> > > pointer relaxation. Patch 3 moves global pointer loading into a
> > > macro that can be easily disabled with SCS. Patch 4 implements
> > > SCS register loading and switching, and allows the feature to be
> > > enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
> > > when CONFIG_IRQ_STACKS is enabled.
> > >
> > > Note that this series requires Clang 17. Earlier Clang versions
> > > support SCS on RISC-V, but use the x18 register instead of gp,
> > > which isn't ideal. gcc has SCS support for arm64, but I'm not
> > > aware of plans to support RISC-V. Once the Zicfiss extension is
> > > ratified, it's probably preferable to use hardware-backed shadow
> > > stacks instead of SCS on hardware that supports the extension,
> > > and we may want to consider implementing CONFIG_DYNAMIC_SCS to
> > > patch between the implementation at runtime (similarly to the
> > > arm64 implementation, which switches to SCS when hardware PAC
> > > support isn't available).
> >
> > I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
> > within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
> > the CFI_BACKWARDS LKDTM test does not pass with
> > CONFIG_SHADOW_CALL_STACK=y.
> >
> > [ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
> > [ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
> > [ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
> > [ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
> >
> > Does the test need to be adjusted or is there some other issue?
>
> Does it pass without the series? I tried to write it to be
> arch-agnostic, but I never tested it on RISC-V. It's very possible that
> test needs adjusting for the architecture. Besides the label horrors,
> the use of __builtin_frame_address may not work there either...
Looks like __builtin_frame_address behaves differently on RISC-V.
After staring at the disassembly a bit, using
__builtin_frame_address(0) - 1 instead of + 1 seems to yield correct
results. WIth that change, here's the test without SCS:
# echo CFI_BACKWARD > /sys/kernel/debug/provoke-crash/DIRECT
[ 16.272028] lkdtm: Performing direct entry CFI_BACKWARD
[ 16.272368] lkdtm: Attempting unchecked stack return address redirection ...
[ 16.272671] lkdtm: ok: redirected stack return address.
[ 16.272885] lkdtm: Attempting checked stack return address redirection ...
[ 16.273384] lkdtm: FAIL: stack return address was redirected!
[ 16.273755] lkdtm: This is probably expected, since this kernel
(6.5.0-rc5-00005-g5a1201f89265-dirty riscv64) was built *without*
CONFIG_ARM64_PTR_AUTH_KERNEL=y nor CONFIG_SHADOW_CALL_STACK=y
And with SCS:
# echo CFI_BACKWARD > /sys/kernel/debug/provoke-crash/DIRECT
[ 17.429551] lkdtm: Performing direct entry CFI_BACKWARD
[ 17.430184] lkdtm: Attempting unchecked stack return address redirection ...
[ 17.431402] lkdtm: ok: redirected stack return address.
[ 17.432031] lkdtm: Attempting checked stack return address redirection ...
[ 17.432861] lkdtm: ok: control flow unchanged.
Sami
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
@ 2023-08-14 18:57 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 18:57 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Guo Ren, Deepak Gupta, Nick Desaulniers, Fangrui Song,
linux-riscv, llvm, linux-kernel
On Mon, Aug 14, 2023 at 11:33 AM Kees Cook <keescook@chromium.org> wrote:
>
> On Mon, Aug 14, 2023 at 10:59:28AM -0700, Nathan Chancellor wrote:
> > Hi Sami,
> >
> > On Fri, Aug 11, 2023 at 11:35:57PM +0000, Sami Tolvanen wrote:
> > > Hi folks,
> > >
> > > This series adds Shadow Call Stack (SCS) support for RISC-V. SCS
> > > uses compiler instrumentation to store return addresses in a
> > > separate shadow stack to protect them against accidental or
> > > malicious overwrites. More information about SCS can be found
> > > here:
> > >
> > > https://clang.llvm.org/docs/ShadowCallStack.html
> > >
> > > Patch 1 is from Deepak, and it simplifies VMAP_STACK overflow
> > > handling by adding support for accessing per-CPU variables
> > > directly in assembly. The patch is included in this series to
> > > make IRQ stack switching cleaner with SCS, and I've simply
> > > rebased it. Patch 2 uses this functionality to clean up the stack
> > > switching by moving duplicate code into a single function. On
> > > RISC-V, the compiler uses the gp register for storing the current
> > > shadow call stack pointer, which is incompatible with global
> > > pointer relaxation. Patch 3 moves global pointer loading into a
> > > macro that can be easily disabled with SCS. Patch 4 implements
> > > SCS register loading and switching, and allows the feature to be
> > > enabled, and patch 5 adds separate per-CPU IRQ shadow call stacks
> > > when CONFIG_IRQ_STACKS is enabled.
> > >
> > > Note that this series requires Clang 17. Earlier Clang versions
> > > support SCS on RISC-V, but use the x18 register instead of gp,
> > > which isn't ideal. gcc has SCS support for arm64, but I'm not
> > > aware of plans to support RISC-V. Once the Zicfiss extension is
> > > ratified, it's probably preferable to use hardware-backed shadow
> > > stacks instead of SCS on hardware that supports the extension,
> > > and we may want to consider implementing CONFIG_DYNAMIC_SCS to
> > > patch between the implementation at runtime (similarly to the
> > > arm64 implementation, which switches to SCS when hardware PAC
> > > support isn't available).
> >
> > I took this series for a spin on top of 6.5-rc6 with both LLVM 18 (built
> > within the past couple of days) and LLVM 17.0.0-rc2 but it seems that
> > the CFI_BACKWARDS LKDTM test does not pass with
> > CONFIG_SHADOW_CALL_STACK=y.
> >
> > [ 73.324652] lkdtm: Performing direct entry CFI_BACKWARD
> > [ 73.324900] lkdtm: Attempting unchecked stack return address redirection ...
> > [ 73.325178] lkdtm: Eek: return address mismatch! 0000000000000002 != ffffffff80614982
> > [ 73.325478] lkdtm: FAIL: stack return address manipulation failed!
> >
> > Does the test need to be adjusted or is there some other issue?
>
> Does it pass without the series? I tried to write it to be
> arch-agnostic, but I never tested it on RISC-V. It's very possible that
> test needs adjusting for the architecture. Besides the label horrors,
> the use of __builtin_frame_address may not work there either...
Looks like __builtin_frame_address behaves differently on RISC-V.
After staring at the disassembly a bit, using
__builtin_frame_address(0) - 1 instead of + 1 seems to yield correct
results. WIth that change, here's the test without SCS:
# echo CFI_BACKWARD > /sys/kernel/debug/provoke-crash/DIRECT
[ 16.272028] lkdtm: Performing direct entry CFI_BACKWARD
[ 16.272368] lkdtm: Attempting unchecked stack return address redirection ...
[ 16.272671] lkdtm: ok: redirected stack return address.
[ 16.272885] lkdtm: Attempting checked stack return address redirection ...
[ 16.273384] lkdtm: FAIL: stack return address was redirected!
[ 16.273755] lkdtm: This is probably expected, since this kernel
(6.5.0-rc5-00005-g5a1201f89265-dirty riscv64) was built *without*
CONFIG_ARM64_PTR_AUTH_KERNEL=y nor CONFIG_SHADOW_CALL_STACK=y
And with SCS:
# echo CFI_BACKWARD > /sys/kernel/debug/provoke-crash/DIRECT
[ 17.429551] lkdtm: Performing direct entry CFI_BACKWARD
[ 17.430184] lkdtm: Attempting unchecked stack return address redirection ...
[ 17.431402] lkdtm: ok: redirected stack return address.
[ 17.432031] lkdtm: Attempting checked stack return address redirection ...
[ 17.432861] lkdtm: ok: control flow unchanged.
Sami
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
2023-08-14 18:57 ` Sami Tolvanen
@ 2023-08-14 20:18 ` Sami Tolvanen
-1 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 20:18 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Guo Ren, Deepak Gupta, Nick Desaulniers, Fangrui Song,
linux-riscv, llvm, linux-kernel
On Mon, Aug 14, 2023 at 11:57 AM Sami Tolvanen <samitolvanen@google.com> wrote:
> Looks like __builtin_frame_address behaves differently on RISC-V.
> After staring at the disassembly a bit, using
> __builtin_frame_address(0) - 1 instead of + 1 seems to yield correct
> results.
Elliott was kind enough to point out to me off-list that this behavior
has been documented here:
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-cc.adoc#frame-pointer-convention
I'll include a patch to fix the test on RISC-V in the next version.
Sami
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
@ 2023-08-14 20:18 ` Sami Tolvanen
0 siblings, 0 replies; 30+ messages in thread
From: Sami Tolvanen @ 2023-08-14 20:18 UTC (permalink / raw)
To: Kees Cook
Cc: Nathan Chancellor, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Guo Ren, Deepak Gupta, Nick Desaulniers, Fangrui Song,
linux-riscv, llvm, linux-kernel
On Mon, Aug 14, 2023 at 11:57 AM Sami Tolvanen <samitolvanen@google.com> wrote:
> Looks like __builtin_frame_address behaves differently on RISC-V.
> After staring at the disassembly a bit, using
> __builtin_frame_address(0) - 1 instead of + 1 seems to yield correct
> results.
Elliott was kind enough to point out to me off-list that this behavior
has been documented here:
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-cc.adoc#frame-pointer-convention
I'll include a patch to fix the test on RISC-V in the next version.
Sami
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
2023-08-14 20:18 ` Sami Tolvanen
@ 2023-08-14 21:09 ` Kees Cook
-1 siblings, 0 replies; 30+ messages in thread
From: Kees Cook @ 2023-08-14 21:09 UTC (permalink / raw)
To: Sami Tolvanen
Cc: Nathan Chancellor, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Guo Ren, Deepak Gupta, Nick Desaulniers, Fangrui Song,
linux-riscv, llvm, linux-kernel
On Mon, Aug 14, 2023 at 01:18:30PM -0700, Sami Tolvanen wrote:
> On Mon, Aug 14, 2023 at 11:57 AM Sami Tolvanen <samitolvanen@google.com> wrote:
> > Looks like __builtin_frame_address behaves differently on RISC-V.
> > After staring at the disassembly a bit, using
> > __builtin_frame_address(0) - 1 instead of + 1 seems to yield correct
> > results.
>
> Elliott was kind enough to point out to me off-list that this behavior
> has been documented here:
>
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-cc.adoc#frame-pointer-convention
>
> I'll include a patch to fix the test on RISC-V in the next version.
Perfect! Thanks for figuring out the offset. :)
--
Kees Cook
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH 0/5] riscv: SCS support
@ 2023-08-14 21:09 ` Kees Cook
0 siblings, 0 replies; 30+ messages in thread
From: Kees Cook @ 2023-08-14 21:09 UTC (permalink / raw)
To: Sami Tolvanen
Cc: Nathan Chancellor, Paul Walmsley, Palmer Dabbelt, Albert Ou,
Guo Ren, Deepak Gupta, Nick Desaulniers, Fangrui Song,
linux-riscv, llvm, linux-kernel
On Mon, Aug 14, 2023 at 01:18:30PM -0700, Sami Tolvanen wrote:
> On Mon, Aug 14, 2023 at 11:57 AM Sami Tolvanen <samitolvanen@google.com> wrote:
> > Looks like __builtin_frame_address behaves differently on RISC-V.
> > After staring at the disassembly a bit, using
> > __builtin_frame_address(0) - 1 instead of + 1 seems to yield correct
> > results.
>
> Elliott was kind enough to point out to me off-list that this behavior
> has been documented here:
>
> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-cc.adoc#frame-pointer-convention
>
> I'll include a patch to fix the test on RISC-V in the next version.
Perfect! Thanks for figuring out the offset. :)
--
Kees Cook
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2023-08-14 21:09 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-11 23:35 [PATCH 0/5] riscv: SCS support Sami Tolvanen
2023-08-11 23:35 ` Sami Tolvanen
2023-08-11 23:35 ` [PATCH 1/5] riscv: VMAP_STACK overflow detection thread-safe Sami Tolvanen
2023-08-11 23:35 ` Sami Tolvanen
2023-08-12 14:35 ` kernel test robot
2023-08-12 14:35 ` kernel test robot
2023-08-13 1:25 ` Guo Ren
2023-08-13 1:25 ` Guo Ren
2023-08-14 15:34 ` Sami Tolvanen
2023-08-14 15:34 ` Sami Tolvanen
2023-08-11 23:35 ` [PATCH 2/5] riscv: Deduplicate IRQ stack switching Sami Tolvanen
2023-08-11 23:35 ` Sami Tolvanen
2023-08-11 23:36 ` [PATCH 3/5] riscv: Move global pointer loading to a macro Sami Tolvanen
2023-08-11 23:36 ` Sami Tolvanen
2023-08-11 23:36 ` [PATCH 4/5] riscv: Implement Shadow Call Stack Sami Tolvanen
2023-08-11 23:36 ` Sami Tolvanen
2023-08-11 23:36 ` [PATCH 5/5] riscv: Use separate IRQ shadow call stacks Sami Tolvanen
2023-08-11 23:36 ` Sami Tolvanen
2023-08-14 17:59 ` [PATCH 0/5] riscv: SCS support Nathan Chancellor
2023-08-14 17:59 ` Nathan Chancellor
2023-08-14 18:33 ` Kees Cook
2023-08-14 18:33 ` Kees Cook
2023-08-14 18:57 ` Sami Tolvanen
2023-08-14 18:57 ` Sami Tolvanen
2023-08-14 20:18 ` Sami Tolvanen
2023-08-14 20:18 ` Sami Tolvanen
2023-08-14 21:09 ` Kees Cook
2023-08-14 21:09 ` Kees Cook
2023-08-14 18:33 ` Sami Tolvanen
2023-08-14 18:33 ` Sami Tolvanen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.