* [RFC PATCH v2 0/3] arm64: dynamic shadow call stack support
@ 2022-05-05 16:10 Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules Ard Biesheuvel
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Ard Biesheuvel @ 2022-05-05 16:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: will, mark.rutland, catalin.marinas, maz, Ard Biesheuvel,
Kees Cook, Sami Tolvanen, Fangrui Song, Nick Desaulniers, Dan Li
Generic kernel images such as Android's GKI typically enable all
security features, which are typically implemented in such a way that
they only take effect if the underlying hardware can support it, but
don't interfere with correct and efficient operation otherwise.
For shadow call stack support, which is always supported by the
hardware, it means it will be enabled even if pointer authentication is
also supported, and enabled for signing return addresses stored on the
stack. The additional security provided by shadow call stack is only
marginal in this case, whereas the performance overhead is not.
Given that return address signing is based on PACIASP/AUTIASP
instructions that implicitly operate on the return address register
(X30) and are not idempotent (i.e., each needs to be emitted exactly
once before the return address is stored on the ordinary stack and after
it has been retrieved from it), we can convert these instruction 1:1
into shadow call stack pushed and pops involving the register X30.
As this is something that can be done at runtime rather than build time,
we can do this conditionally based on whether or not return address
signing is supported on the underlying hardware.
In order to be able to unwind call stacks that involve return address
signing, whether or not the return address is currently signed is
tracked by DWARF CFI directives in the unwinding metadata. This means we
can use this information to locate all PACIASP/AUTIASP instructions in
the binary, instead of having to use brute force and go over all
instructions in the entire program.
This series implements this approach for Clang, which has recently been
fixed to emit all these CFI directives correctly. This series is based
on an older PoC sent out last year [0] that targeted GCC only (due to
this issue). This v2 targets Clang only, as GCC has its own issues with
CFI correctness.
Changes since RFC v1:
- implement boot time check for PAC/BTI support, and only enable dynamic
SCS if neither are supported;
- implement module patching as well;
- switch to Clang, and drop workaround for GCC bug;
[0] https://lore.kernel.org/linux-arm-kernel/20211013152243.2216899-1-ardb@kernel.org/
Cc: Kees Cook <keescook@google.com>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Fangrui Song <maskray@google.com>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Dan Li <ashimida@linux.alibaba.com>
Ard Biesheuvel (3):
arm64: unwind: add asynchronous unwind tables to kernel and modules
scs: add support for dynamic shadow call stacks
arm64: implement dynamic shadow call stack for Clang
Makefile | 2 +
arch/Kconfig | 7 +
arch/arm64/Kconfig | 11 +
arch/arm64/Makefile | 5 +
arch/arm64/include/asm/module.lds.h | 8 +
arch/arm64/include/asm/scs.h | 12 +
arch/arm64/kernel/Makefile | 2 +
arch/arm64/kernel/head.S | 3 +
arch/arm64/kernel/irq.c | 2 +-
arch/arm64/kernel/module.c | 10 +
arch/arm64/kernel/patch-scs.c | 257 ++++++++++++++++++++
arch/arm64/kernel/sdei.c | 2 +-
arch/arm64/kernel/setup.c | 5 +
arch/arm64/kernel/vmlinux.lds.S | 16 ++
drivers/firmware/efi/libstub/Makefile | 1 +
include/linux/scs.h | 10 +
kernel/scs.c | 14 +-
17 files changed, 363 insertions(+), 4 deletions(-)
create mode 100644 arch/arm64/kernel/patch-scs.c
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules
2022-05-05 16:10 [RFC PATCH v2 0/3] arm64: dynamic shadow call stack support Ard Biesheuvel
@ 2022-05-05 16:10 ` Ard Biesheuvel
2022-05-05 20:19 ` Nick Desaulniers
2022-05-05 16:10 ` [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang Ard Biesheuvel
2 siblings, 1 reply; 12+ messages in thread
From: Ard Biesheuvel @ 2022-05-05 16:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: will, mark.rutland, catalin.marinas, maz, Ard Biesheuvel,
Kees Cook, Sami Tolvanen, Fangrui Song, Nick Desaulniers, Dan Li
Enable asynchronous unwind table generation for both the core kernel as
well as modules, and emit the resulting .eh_frame sections as init code
so we can use the unwind directives for code patching at boot or module
load time.
This will be used by dynamic shadow call stack support, which will rely
on code patching rather than compiler codegen to emit the shadow call
stack push and pop instructions.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/arm64/Kconfig | 3 +++
arch/arm64/Makefile | 5 +++++
arch/arm64/include/asm/module.lds.h | 8 ++++++++
arch/arm64/kernel/vmlinux.lds.S | 16 ++++++++++++++++
drivers/firmware/efi/libstub/Makefile | 1 +
5 files changed, 33 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 57c4c995965f..b6302f7cd73f 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -354,6 +354,9 @@ config KASAN_SHADOW_OFFSET
default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS
default 0xffffffffffffffff
+config UNWIND_TABLES
+ bool
+
source "arch/arm64/Kconfig.platforms"
menu "Kernel Features"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 2f1de88651e6..a4c6807ecbaf 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -45,8 +45,13 @@ KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
# Avoid generating .eh_frame* sections.
+ifneq ($(CONFIG_UNWIND_TABLES),y)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
KBUILD_AFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
+else
+KBUILD_CFLAGS += -fasynchronous-unwind-tables
+KBUILD_AFLAGS += -fasynchronous-unwind-tables
+endif
ifeq ($(CONFIG_STACKPROTECTOR_PER_TASK),y)
prepare: stack_protector_prepare
diff --git a/arch/arm64/include/asm/module.lds.h b/arch/arm64/include/asm/module.lds.h
index 094701ec5500..dbba4b7559aa 100644
--- a/arch/arm64/include/asm/module.lds.h
+++ b/arch/arm64/include/asm/module.lds.h
@@ -17,4 +17,12 @@ SECTIONS {
*/
.text.hot : { *(.text.hot) }
#endif
+
+#ifdef CONFIG_UNWIND_TABLES
+ /*
+ * Currently, we only use unwind info at module load time, so we can
+ * put it into the .init allocation.
+ */
+ .init.eh_frame : { *(.eh_frame) }
+#endif
}
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index edaf0faf766f..2f4908c8d152 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -122,6 +122,17 @@ jiffies = jiffies_64;
#define TRAMP_TEXT
#endif
+#ifdef CONFIG_UNWIND_TABLES
+#define UNWIND_DATA_SECTIONS \
+ .eh_frame : { \
+ __eh_frame_start = .; \
+ *(.eh_frame) \
+ __eh_frame_end = .; \
+ }
+#else
+#define UNWIND_DATA_SECTIONS
+#endif
+
/*
* The size of the PE/COFF section that covers the kernel image, which
* runs from _stext to _edata, must be a round multiple of the PE/COFF
@@ -150,6 +161,9 @@ SECTIONS
/DISCARD/ : {
*(.interp .dynamic)
*(.dynsym .dynstr .hash .gnu.hash)
+#ifndef CONFIG_UNWIND_TABLES
+ *(.eh_frame)
+#endif
}
. = KIMAGE_VADDR;
@@ -228,6 +242,8 @@ SECTIONS
__alt_instructions_end = .;
}
+ UNWIND_DATA_SECTIONS
+
. = ALIGN(SEGMENT_ALIGN);
__inittext_end = .;
__initdata_begin = .;
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index d0537573501e..78c46638707a 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -20,6 +20,7 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ \
# disable the stackleak plugin
cflags-$(CONFIG_ARM64) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
-fpie $(DISABLE_STACKLEAK_PLUGIN) \
+ -fno-unwind-tables -fno-asynchronous-unwind-tables \
$(call cc-option,-mbranch-protection=none)
cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
-fno-builtin -fpic \
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks
2022-05-05 16:10 [RFC PATCH v2 0/3] arm64: dynamic shadow call stack support Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules Ard Biesheuvel
@ 2022-05-05 16:10 ` Ard Biesheuvel
2022-05-05 21:14 ` Nick Desaulniers
2022-05-06 0:00 ` Kees Cook
2022-05-05 16:10 ` [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang Ard Biesheuvel
2 siblings, 2 replies; 12+ messages in thread
From: Ard Biesheuvel @ 2022-05-05 16:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: will, mark.rutland, catalin.marinas, maz, Ard Biesheuvel,
Kees Cook, Sami Tolvanen, Fangrui Song, Nick Desaulniers, Dan Li
In order to allow arches to use code patching to conditionally emit the
shadow stack pushes and pops, rather than always taking the performance
hit even on CPUs that implement alternatives such as stack pointer
authentication on arm64, add a Kconfig symbol that can be set by the
arch to omit the SCS codegen itself, without otherwise affecting how
support code for SCS and compiler options (for register reservation, for
instance) are emitted.
Also, add a static key and some plumbing to omit the allocation of
shadow call stack for dynamic SCS configurations if SCS is disabled at
runtime.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
Makefile | 2 ++
arch/Kconfig | 7 +++++++
include/linux/scs.h | 10 ++++++++++
kernel/scs.c | 14 ++++++++++++--
4 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index fa5112a0ec1b..a578fffc0337 100644
--- a/Makefile
+++ b/Makefile
@@ -882,8 +882,10 @@ LDFLAGS_vmlinux += --gc-sections
endif
ifdef CONFIG_SHADOW_CALL_STACK
+ifndef CONFIG_DYNAMIC_SCS
CC_FLAGS_SCS := -fsanitize=shadow-call-stack
KBUILD_CFLAGS += $(CC_FLAGS_SCS)
+endif
export CC_FLAGS_SCS
endif
diff --git a/arch/Kconfig b/arch/Kconfig
index 29b0167c088b..126caa75969a 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -627,6 +627,13 @@ config SHADOW_CALL_STACK
reading and writing arbitrary memory may be able to locate them
and hijack control flow by modifying the stacks.
+config DYNAMIC_SCS
+ bool
+ help
+ Set by the arch code if it relies on code patching to insert the
+ shadow call stack push and pop instructions rather than on the
+ compiler.
+
config LTO
bool
help
diff --git a/include/linux/scs.h b/include/linux/scs.h
index 18122d9e17ff..4cc01f21b17a 100644
--- a/include/linux/scs.h
+++ b/include/linux/scs.h
@@ -53,6 +53,15 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk)
return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC;
}
+DECLARE_STATIC_KEY_TRUE(dynamic_scs_enabled);
+
+static inline bool scs_is_enabled(void)
+{
+ if (!IS_ENABLED(CONFIG_DYNAMIC_SCS))
+ return true;
+ return static_branch_likely(&dynamic_scs_enabled);
+}
+
#else /* CONFIG_SHADOW_CALL_STACK */
static inline void *scs_alloc(int node) { return NULL; }
@@ -62,6 +71,7 @@ static inline void scs_task_reset(struct task_struct *tsk) {}
static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; }
static inline void scs_release(struct task_struct *tsk) {}
static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; }
+static inline bool scs_is_enabled(void) { return false; }
#endif /* CONFIG_SHADOW_CALL_STACK */
diff --git a/kernel/scs.c b/kernel/scs.c
index b7e1b096d906..8826794d2645 100644
--- a/kernel/scs.c
+++ b/kernel/scs.c
@@ -12,6 +12,10 @@
#include <linux/vmalloc.h>
#include <linux/vmstat.h>
+#ifdef CONFIG_DYNAMIC_SCS
+DEFINE_STATIC_KEY_TRUE(dynamic_scs_enabled);
+#endif
+
static void __scs_account(void *s, int account)
{
struct page *scs_page = vmalloc_to_page(s);
@@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu)
void __init scs_init(void)
{
+ if (!scs_is_enabled())
+ return;
cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL,
scs_cleanup);
}
int scs_prepare(struct task_struct *tsk, int node)
{
- void *s = scs_alloc(node);
+ void *s;
+ if (!scs_is_enabled())
+ return 0;
+
+ s = scs_alloc(node);
if (!s)
return -ENOMEM;
@@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk)
{
void *s = task_scs(tsk);
- if (!s)
+ if (!scs_is_enabled() || !s)
return;
WARN(task_scs_end_corrupted(tsk),
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang
2022-05-05 16:10 [RFC PATCH v2 0/3] arm64: dynamic shadow call stack support Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks Ard Biesheuvel
@ 2022-05-05 16:10 ` Ard Biesheuvel
2022-05-05 21:01 ` Nick Desaulniers
2022-05-06 0:00 ` Kees Cook
2 siblings, 2 replies; 12+ messages in thread
From: Ard Biesheuvel @ 2022-05-05 16:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: will, mark.rutland, catalin.marinas, maz, Ard Biesheuvel,
Kees Cook, Sami Tolvanen, Fangrui Song, Nick Desaulniers, Dan Li
Implement dynamic shadow call stack support on Clang, by parsing the
unwind tables at init time to locate all occurrences of PACIASP/AUTIASP
instructions, and replacing them with the shadow call stack push and pop
instructions, respectively.
This is useful because the overhead of the shadow call stack is
difficult to justify on hardware that implements pointer authentication
(PAC), and given that the PAC instructions are executed as NOPs on
hardware that doesn't, we can just replace them without breaking
anything. As PACIASP/AUTIASP are guaranteed to be paired with respect to
manipulations of the return address, replacing them 1:1 with shadow call
stack pushes and pops is guaranteed to result in the expected behavior.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/arm64/Kconfig | 8 +
arch/arm64/include/asm/scs.h | 12 +
arch/arm64/kernel/Makefile | 2 +
arch/arm64/kernel/head.S | 3 +
arch/arm64/kernel/irq.c | 2 +-
arch/arm64/kernel/module.c | 10 +
arch/arm64/kernel/patch-scs.c | 257 ++++++++++++++++++++
arch/arm64/kernel/sdei.c | 2 +-
arch/arm64/kernel/setup.c | 5 +
9 files changed, 299 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b6302f7cd73f..df7a7aff456a 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -357,6 +357,14 @@ config KASAN_SHADOW_OFFSET
config UNWIND_TABLES
bool
+config UNWIND_PATCH_PAC_INTO_SCS
+ def_bool y
+ depends on CC_IS_CLANG && CLANG_VERSION >= 150000
+ depends on SHADOW_CALL_STACK
+ depends on ARM64_PTR_AUTH_KERNEL
+ select UNWIND_TABLES
+ select DYNAMIC_SCS
+
source "arch/arm64/Kconfig.platforms"
menu "Kernel Features"
diff --git a/arch/arm64/include/asm/scs.h b/arch/arm64/include/asm/scs.h
index 8297bccf0784..09aed251e695 100644
--- a/arch/arm64/include/asm/scs.h
+++ b/arch/arm64/include/asm/scs.h
@@ -24,6 +24,18 @@
.endm
#endif /* CONFIG_SHADOW_CALL_STACK */
+
+#else
+
+
+#ifdef CONFIG_UNWIND_PATCH_PAC_TO_SCS
+extern bool should_disable_dynamic_scs;
+#else
+#define should_disable_dynamic_scs (false)
+#endif
+
+int scs_patch(const u8 eh_frame[], int size);
+
#endif /* __ASSEMBLY __ */
#endif /* _ASM_SCS_H */
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 986837d7ec82..d7074470031a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -74,6 +74,8 @@ obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o
obj-$(CONFIG_ARM64_MTE) += mte.o
obj-y += vdso-wrap.o
obj-$(CONFIG_COMPAT_VDSO) += vdso32-wrap.o
+obj-$(CONFIG_UNWIND_PATCH_PAC_INTO_SCS) += patch-scs.o
+CFLAGS_patch-scs.o += -mbranch-protection=none
obj-y += probes/
head-y := head.o
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 6a98f1a38c29..e9601c8a1bcd 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -453,6 +453,9 @@ SYM_FUNC_START_LOCAL(__primary_switched)
mov x0, x21 // pass FDT address in x0
bl early_fdt_map // Try mapping the FDT early
bl init_feature_override // Parse cpu feature overrides
+#ifdef CONFIG_UNWIND_PATCH_PAC_INTO_SCS
+ bl scs_patch_vmlinux
+#endif
#ifdef CONFIG_RANDOMIZE_BASE
tst x23, ~(MIN_KIMG_ALIGN - 1) // already running randomized?
b.ne 0f
diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c
index bda49430c9ea..c284ec35c27c 100644
--- a/arch/arm64/kernel/irq.c
+++ b/arch/arm64/kernel/irq.c
@@ -39,7 +39,7 @@ static void init_irq_scs(void)
{
int cpu;
- if (!IS_ENABLED(CONFIG_SHADOW_CALL_STACK))
+ if (!scs_is_enabled())
return;
for_each_possible_cpu(cpu)
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index f2d4bb14bfab..7e9e63600d28 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -18,6 +18,7 @@
#include <linux/vmalloc.h>
#include <asm/alternative.h>
#include <asm/insn.h>
+#include <asm/scs.h>
#include <asm/sections.h>
void *module_alloc(unsigned long size)
@@ -529,5 +530,14 @@ int module_finalize(const Elf_Ehdr *hdr,
if (s)
apply_alternatives_module((void *)s->sh_addr, s->sh_size);
+ if (IS_ENABLED(CONFIG_UNWIND_PATCH_PAC_INTO_SCS) &&
+ !system_supports_address_auth() &&
+ (!IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) || !system_supports_bti())) {
+
+ s = find_section(hdr, sechdrs, ".init.eh_frame");
+ if (s)
+ scs_patch((void *)s->sh_addr, s->sh_size);
+ }
+
return module_init_ftrace_plt(hdr, sechdrs, me);
}
diff --git a/arch/arm64/kernel/patch-scs.c b/arch/arm64/kernel/patch-scs.c
new file mode 100644
index 000000000000..8c534630c2a1
--- /dev/null
+++ b/arch/arm64/kernel/patch-scs.c
@@ -0,0 +1,257 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2022 - Google LLC
+ * Author: Ard Biesheuvel <ardb@google.com>
+ */
+
+#include <linux/bug.h>
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/printk.h>
+#include <linux/types.h>
+
+#include <asm/cpufeature.h>
+
+#define DW_CFA_nop 0x00
+#define DW_CFA_set_loc 0x01
+#define DW_CFA_advance_loc1 0x02
+#define DW_CFA_advance_loc2 0x03
+#define DW_CFA_advance_loc4 0x04
+#define DW_CFA_offset_extended 0x05
+#define DW_CFA_restore_extended 0x06
+#define DW_CFA_undefined 0x07
+#define DW_CFA_same_value 0x08
+#define DW_CFA_register 0x09
+#define DW_CFA_remember_state 0x0a
+#define DW_CFA_restore_state 0x0b
+#define DW_CFA_def_cfa 0x0c
+#define DW_CFA_def_cfa_register 0x0d
+#define DW_CFA_def_cfa_offset 0x0e
+#define DW_CFA_def_cfa_expression 0x0f
+#define DW_CFA_expression 0x10
+#define DW_CFA_offset_extended_sf 0x11
+#define DW_CFA_def_cfa_sf 0x12
+#define DW_CFA_def_cfa_offset_sf 0x13
+#define DW_CFA_val_offset 0x14
+#define DW_CFA_val_offset_sf 0x15
+#define DW_CFA_val_expression 0x16
+#define DW_CFA_lo_user 0x1c
+#define DW_CFA_negate_ra_state 0x2d
+#define DW_CFA_GNU_args_size 0x2e
+#define DW_CFA_GNU_negative_offset_extended 0x2f
+#define DW_CFA_hi_user 0x3f
+
+extern const u8 __eh_frame_start[], __eh_frame_end[];
+
+struct fde_frame {
+ s32 initial_loc;
+ s32 range;
+};
+
+enum {
+ PACIASP = 0xd503233f,
+ AUTIASP = 0xd50323bf,
+ SCS_PUSH = 0xf800865e,
+ SCS_POP = 0xf85f8e5e,
+};
+
+static void __always_inline scs_patch_loc(u64 loc)
+{
+ u32 insn = le32_to_cpup((void *)loc);
+
+ switch (insn) {
+ case PACIASP:
+ *(u32 *)loc = cpu_to_le32(SCS_PUSH);
+ break;
+ case AUTIASP:
+ *(u32 *)loc = cpu_to_le32(SCS_POP);
+ break;
+ default:
+ /*
+ * While the DW_CFA_negate_ra_state directive is guaranteed to
+ * appear right after a PACIASP/AUTIASP instruction, it may
+ * also appear after a DW_CFA_restore_state directive that
+ * restores a state that is only partially accurate, and is
+ * followed by DW_CFA_negate_ra_state directive to toggle the
+ * PAC bit again. So we permit other instructions here, and ignore
+ * them.
+ */
+ break;
+ }
+}
+
+/*
+ * Skip one uleb128/sleb128 encoded quantity from the opcode stream. All bytes
+ * except the last one have bit #7 set.
+ */
+static int __always_inline skip_xleb128(const u8 **opcode, int size)
+{
+ u8 c;
+
+ do {
+ c = *(*opcode)++;
+ size--;
+ } while (c & 0x80);
+
+ return size;
+}
+
+static int noinstr scs_handle_frame(const u8 eh_frame[], u32 size)
+{
+ const struct fde_frame *fde;
+ const u8 *opcode;
+ u64 loc;
+
+ /*
+ * For patching PAC opcodes, we only care about the FDE records, and
+ * not the CIE, which carries the initial CFA directives but they only
+ * pertain to which register is the stack pointer.
+ * TODO this is not 100% true - we need the augmentation string and the
+ * encoding but they are always the same in practice.
+ */
+ if (*(u32 *)eh_frame == 0)
+ return 0;
+
+ fde = (const struct fde_frame *)(eh_frame + 4);
+ loc = (u64)offset_to_ptr(&fde->initial_loc);
+ opcode = (const u8 *)(fde + 1);
+
+ // TODO check augmentation data
+ WARN_ON(*opcode++);
+ size -= sizeof(u32) + sizeof(*fde) + 1;
+
+ /*
+ * Starting from 'loc', apply the CFA opcodes that advance the location
+ * pointer, and identify the locations of the PAC instructions.
+ */
+ do {
+ switch (*opcode & 0xC0) {
+ case 0:
+ // handle DW_CFA_xxx opcodes
+ switch (*opcode) {
+ case DW_CFA_nop:
+ case DW_CFA_remember_state:
+ case DW_CFA_restore_state:
+ break;
+
+ case DW_CFA_advance_loc1:
+ loc += *++opcode;
+ size--;
+ break;
+
+ case DW_CFA_advance_loc2:
+ loc += *++opcode;
+ loc += *++opcode << 8;
+ size -= 2;
+ break;
+
+ case DW_CFA_def_cfa:
+ opcode++;
+ size = skip_xleb128(&opcode, --size);
+ size = skip_xleb128(&opcode, size);
+ continue;
+ case DW_CFA_def_cfa_offset:
+ case DW_CFA_def_cfa_offset_sf:
+ case DW_CFA_def_cfa_register:
+ case DW_CFA_same_value:
+ opcode++;
+ size = skip_xleb128(&opcode, --size);
+ continue;
+
+ case DW_CFA_negate_ra_state:
+ scs_patch_loc(loc - 4);
+ break;
+
+ default:
+ pr_err("unhandled opcode: %02x in FDE frame %lx\n", *opcode, (uintptr_t)eh_frame);
+ return -ENOEXEC;
+ }
+ opcode++;
+ size--;
+ break;
+
+ case 0x40:
+ // advance loc
+ loc += *opcode++ & 0x3f;
+ size--;
+ break;
+
+ case 0x80:
+ opcode++;
+ size = skip_xleb128(&opcode, --size);
+ continue;
+
+ default:
+ // ignore
+ opcode++;
+ size--;
+ break;
+ }
+ } while (size > 0);
+
+ return 0;
+}
+
+int noinstr scs_patch(const u8 eh_frame[], int size)
+{
+ const u8 *p = eh_frame;
+
+ while (size > 4) {
+ const u32 *frame_size = (const u32 *)p;
+ int ret;
+
+ if (*frame_size != -1 && *frame_size <= size) {
+ ret = scs_handle_frame(p + 4, *frame_size);
+ if (ret)
+ return ret;
+ p += 4 + *frame_size;
+ size -= 4 + *frame_size;
+ }
+ }
+ return 0;
+}
+
+extern struct arm64_ftr_override id_aa64isar1_override;
+extern struct arm64_ftr_override id_aa64isar2_override;
+extern struct arm64_ftr_override id_aa64pfr1_override;
+
+bool __initdata should_disable_dynamic_scs = true;
+
+asmlinkage void __init scs_patch_vmlinux(void)
+{
+ /*
+ * We only enable the shadow call stack dynamically if we are running
+ * on a system that does not implement PAC or BTI. PAC and SCS roughly
+ * provide the same level of protection, and BTI relies on the PACIASP
+ * instructions serving as landing pads, preventing us from patching
+ * those instructions into something else.
+ */
+ u64 reg = read_sysreg_s(SYS_ID_AA64ISAR1_EL1);
+
+ reg &= ~id_aa64isar1_override.mask;
+ reg |= id_aa64isar1_override.val;
+
+ if (reg & ((0xf << ID_AA64ISAR1_APA_SHIFT) |
+ (0xf << ID_AA64ISAR1_API_SHIFT)))
+ return;
+
+ reg = read_sysreg_s(SYS_ID_AA64ISAR2_EL1);
+ reg &= ~id_aa64isar2_override.mask;
+ reg |= id_aa64isar2_override.val;
+
+ if (reg & (0xf << ID_AA64ISAR2_APA3_SHIFT))
+ return;
+
+ if (IS_ENABLED(CONFIG_ARM64_BTI_KERNEL)) {
+ reg = read_sysreg_s(SYS_ID_AA64PFR1_EL1);
+ reg &= ~id_aa64pfr1_override.mask;
+ reg |= id_aa64pfr1_override.val;
+
+ if (reg & (0xf << ID_AA64PFR1_BT_SHIFT))
+ return;
+ }
+
+ WARN_ON(scs_patch(__eh_frame_start, __eh_frame_end - __eh_frame_start));
+ should_disable_dynamic_scs = false;
+}
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index d20620a1c51a..30f3c7563694 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -144,7 +144,7 @@ static int init_sdei_scs(void)
int cpu;
int err = 0;
- if (!IS_ENABLED(CONFIG_SHADOW_CALL_STACK))
+ if (!scs_is_enabled())
return 0;
for_each_possible_cpu(cpu) {
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 3505789cf4bd..17fad5749f4a 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -30,6 +30,7 @@
#include <linux/efi.h>
#include <linux/psci.h>
#include <linux/sched/task.h>
+#include <linux/scs.h>
#include <linux/mm.h>
#include <asm/acpi.h>
@@ -42,6 +43,7 @@
#include <asm/cpu_ops.h>
#include <asm/kasan.h>
#include <asm/numa.h>
+#include <asm/scs.h>
#include <asm/sections.h>
#include <asm/setup.h>
#include <asm/smp_plat.h>
@@ -323,6 +325,9 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
jump_label_init();
parse_early_param();
+ if (should_disable_dynamic_scs)
+ static_branch_disable(&dynamic_scs_enabled);
+
/*
* Unmask asynchronous aborts and fiq after bringing up possible
* earlycon. (Report possible System Errors once we can report this
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules
2022-05-05 16:10 ` [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules Ard Biesheuvel
@ 2022-05-05 20:19 ` Nick Desaulniers
2022-05-06 7:07 ` Ard Biesheuvel
0 siblings, 1 reply; 12+ messages in thread
From: Nick Desaulniers @ 2022-05-05 20:19 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-arm-kernel, will, mark.rutland, catalin.marinas, maz,
Kees Cook, Sami Tolvanen, Fangrui Song, Dan Li
On Thu, May 5, 2022 at 9:10 AM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> Enable asynchronous unwind table generation for both the core kernel as
> well as modules, and emit the resulting .eh_frame sections as init code
> so we can use the unwind directives for code patching at boot or module
> load time.
>
> This will be used by dynamic shadow call stack support, which will rely
> on code patching rather than compiler codegen to emit the shadow call
> stack push and pop instructions.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Two minor nits inline below; I don't feel strongly about them either way.
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> arch/arm64/Kconfig | 3 +++
> arch/arm64/Makefile | 5 +++++
> arch/arm64/include/asm/module.lds.h | 8 ++++++++
> arch/arm64/kernel/vmlinux.lds.S | 16 ++++++++++++++++
> drivers/firmware/efi/libstub/Makefile | 1 +
> 5 files changed, 33 insertions(+)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 57c4c995965f..b6302f7cd73f 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -354,6 +354,9 @@ config KASAN_SHADOW_OFFSET
> default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS
> default 0xffffffffffffffff
>
> +config UNWIND_TABLES
> + bool
> +
> source "arch/arm64/Kconfig.platforms"
>
> menu "Kernel Features"
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 2f1de88651e6..a4c6807ecbaf 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -45,8 +45,13 @@ KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
> KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
>
> # Avoid generating .eh_frame* sections.
> +ifneq ($(CONFIG_UNWIND_TABLES),y)
You know how I feel about negated conditions when there's an else branch...
> KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
> KBUILD_AFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
> +else
> +KBUILD_CFLAGS += -fasynchronous-unwind-tables
> +KBUILD_AFLAGS += -fasynchronous-unwind-tables
> +endif
>
> ifeq ($(CONFIG_STACKPROTECTOR_PER_TASK),y)
> prepare: stack_protector_prepare
> diff --git a/arch/arm64/include/asm/module.lds.h b/arch/arm64/include/asm/module.lds.h
> index 094701ec5500..dbba4b7559aa 100644
> --- a/arch/arm64/include/asm/module.lds.h
> +++ b/arch/arm64/include/asm/module.lds.h
> @@ -17,4 +17,12 @@ SECTIONS {
> */
> .text.hot : { *(.text.hot) }
> #endif
> +
> +#ifdef CONFIG_UNWIND_TABLES
> + /*
> + * Currently, we only use unwind info at module load time, so we can
> + * put it into the .init allocation.
> + */
> + .init.eh_frame : { *(.eh_frame) }
> +#endif
> }
> diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
> index edaf0faf766f..2f4908c8d152 100644
> --- a/arch/arm64/kernel/vmlinux.lds.S
> +++ b/arch/arm64/kernel/vmlinux.lds.S
> @@ -122,6 +122,17 @@ jiffies = jiffies_64;
> #define TRAMP_TEXT
> #endif
>
> +#ifdef CONFIG_UNWIND_TABLES
> +#define UNWIND_DATA_SECTIONS \
> + .eh_frame : { \
> + __eh_frame_start = .; \
> + *(.eh_frame) \
> + __eh_frame_end = .; \
> + }
> +#else
> +#define UNWIND_DATA_SECTIONS
> +#endif
> +
> /*
> * The size of the PE/COFF section that covers the kernel image, which
> * runs from _stext to _edata, must be a round multiple of the PE/COFF
> @@ -150,6 +161,9 @@ SECTIONS
> /DISCARD/ : {
> *(.interp .dynamic)
> *(.dynsym .dynstr .hash .gnu.hash)
> +#ifndef CONFIG_UNWIND_TABLES
> + *(.eh_frame)
> +#endif
If CONFIG_UNWIND_TABLES is unset, then
`-fno-asynchronous-unwind-tables -fno-unwind-tables` should be set; it
would be anomalous otherwise. We could leave this hunk out and we
should get a warning from `--orphan-handling=warn` linker flag which
would warn explicitly about such anomalies, rather than produce but
then silently discard them.
> }
>
> . = KIMAGE_VADDR;
> @@ -228,6 +242,8 @@ SECTIONS
> __alt_instructions_end = .;
> }
>
> + UNWIND_DATA_SECTIONS
> +
> . = ALIGN(SEGMENT_ALIGN);
> __inittext_end = .;
> __initdata_begin = .;
> diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> index d0537573501e..78c46638707a 100644
> --- a/drivers/firmware/efi/libstub/Makefile
> +++ b/drivers/firmware/efi/libstub/Makefile
> @@ -20,6 +20,7 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ \
> # disable the stackleak plugin
> cflags-$(CONFIG_ARM64) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> -fpie $(DISABLE_STACKLEAK_PLUGIN) \
> + -fno-unwind-tables -fno-asynchronous-unwind-tables \
> $(call cc-option,-mbranch-protection=none)
> cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> -fno-builtin -fpic \
> --
> 2.30.2
>
--
Thanks,
~Nick Desaulniers
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang
2022-05-05 16:10 ` [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang Ard Biesheuvel
@ 2022-05-05 21:01 ` Nick Desaulniers
2022-05-18 1:53 ` Sami Tolvanen
2022-05-06 0:00 ` Kees Cook
1 sibling, 1 reply; 12+ messages in thread
From: Nick Desaulniers @ 2022-05-05 21:01 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-arm-kernel, will, mark.rutland, catalin.marinas, maz,
Kees Cook, Sami Tolvanen, Fangrui Song, Dan Li
On Thu, May 5, 2022 at 9:10 AM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index b6302f7cd73f..df7a7aff456a 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -357,6 +357,14 @@ config KASAN_SHADOW_OFFSET
> config UNWIND_TABLES
> bool
>
> +config UNWIND_PATCH_PAC_INTO_SCS
> + def_bool y
> + depends on CC_IS_CLANG && CLANG_VERSION >= 150000
Consider adding a comment that links to the corresponding GCC bug
report, which can be replaced with a version check once fixed.
> + depends on SHADOW_CALL_STACK
> + depends on ARM64_PTR_AUTH_KERNEL
> + select UNWIND_TABLES
> + select DYNAMIC_SCS
> +
> source "arch/arm64/Kconfig.platforms"
>
> menu "Kernel Features"
> diff --git a/arch/arm64/kernel/patch-scs.c b/arch/arm64/kernel/patch-scs.c
> new file mode 100644
> index 000000000000..8c534630c2a1
> --- /dev/null
> +++ b/arch/arm64/kernel/patch-scs.c
> @@ -0,0 +1,257 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2022 - Google LLC
> + * Author: Ard Biesheuvel <ardb@google.com>
> + */
> +
> +#include <linux/bug.h>
> +#include <linux/errno.h>
> +#include <linux/init.h>
> +#include <linux/linkage.h>
> +#include <linux/printk.h>
> +#include <linux/types.h>
> +
> +#include <asm/cpufeature.h>
> +
> +#define DW_CFA_nop 0x00
> +#define DW_CFA_set_loc 0x01
> +#define DW_CFA_advance_loc1 0x02
> +#define DW_CFA_advance_loc2 0x03
> +#define DW_CFA_advance_loc4 0x04
> +#define DW_CFA_offset_extended 0x05
> +#define DW_CFA_restore_extended 0x06
> +#define DW_CFA_undefined 0x07
> +#define DW_CFA_same_value 0x08
> +#define DW_CFA_register 0x09
> +#define DW_CFA_remember_state 0x0a
> +#define DW_CFA_restore_state 0x0b
> +#define DW_CFA_def_cfa 0x0c
> +#define DW_CFA_def_cfa_register 0x0d
> +#define DW_CFA_def_cfa_offset 0x0e
> +#define DW_CFA_def_cfa_expression 0x0f
> +#define DW_CFA_expression 0x10
> +#define DW_CFA_offset_extended_sf 0x11
> +#define DW_CFA_def_cfa_sf 0x12
> +#define DW_CFA_def_cfa_offset_sf 0x13
> +#define DW_CFA_val_offset 0x14
> +#define DW_CFA_val_offset_sf 0x15
> +#define DW_CFA_val_expression 0x16
> +#define DW_CFA_lo_user 0x1c
> +#define DW_CFA_negate_ra_state 0x2d
> +#define DW_CFA_GNU_args_size 0x2e
> +#define DW_CFA_GNU_negative_offset_extended 0x2f
> +#define DW_CFA_hi_user 0x3f
Might be more reusable to put these in their own header. Though, this
is currently the only user, so perhaps "YAGNI."
If there's some documentation to these values, consider adding a
comment with a link.
> +
> +extern const u8 __eh_frame_start[], __eh_frame_end[];
> +
> +struct fde_frame {
> + s32 initial_loc;
> + s32 range;
> +};
> +
> +enum {
> + PACIASP = 0xd503233f,
> + AUTIASP = 0xd50323bf,
> + SCS_PUSH = 0xf800865e,
> + SCS_POP = 0xf85f8e5e,
> +};
Is there anything we can reuse from arch/arm64/include/asm/insn.h
rather than hardcoding these values?
> +
> +static void __always_inline scs_patch_loc(u64 loc)
> +{
> + u32 insn = le32_to_cpup((void *)loc);
> +
> + switch (insn) {
> + case PACIASP:
> + *(u32 *)loc = cpu_to_le32(SCS_PUSH);
> + break;
> + case AUTIASP:
> + *(u32 *)loc = cpu_to_le32(SCS_POP);
> + break;
> + default:
> + /*
> + * While the DW_CFA_negate_ra_state directive is guaranteed to
> + * appear right after a PACIASP/AUTIASP instruction, it may
> + * also appear after a DW_CFA_restore_state directive that
> + * restores a state that is only partially accurate, and is
> + * followed by DW_CFA_negate_ra_state directive to toggle the
> + * PAC bit again. So we permit other instructions here, and ignore
> + * them.
> + */
> + break;
> + }
> +}
> +
> +/*
> + * Skip one uleb128/sleb128 encoded quantity from the opcode stream. All bytes
> + * except the last one have bit #7 set.
Consider using the BIT macro to express that.
> + */
> +static int __always_inline skip_xleb128(const u8 **opcode, int size)
> +{
> + u8 c;
> +
> + do {
> + c = *(*opcode)++;
> + size--;
> + } while (c & 0x80);
> +
> + return size;
> +}
> +
> +static int noinstr scs_handle_frame(const u8 eh_frame[], u32 size)
> +{
> + const struct fde_frame *fde;
> + const u8 *opcode;
> + u64 loc;
> +
> + /*
> + * For patching PAC opcodes, we only care about the FDE records, and
> + * not the CIE, which carries the initial CFA directives but they only
> + * pertain to which register is the stack pointer.
> + * TODO this is not 100% true - we need the augmentation string and the
> + * encoding but they are always the same in practice.
> + */
> + if (*(u32 *)eh_frame == 0)
> + return 0;
> +
> + fde = (const struct fde_frame *)(eh_frame + 4);
> + loc = (u64)offset_to_ptr(&fde->initial_loc);
> + opcode = (const u8 *)(fde + 1);
> +
> + // TODO check augmentation data
> + WARN_ON(*opcode++);
> + size -= sizeof(u32) + sizeof(*fde) + 1;
> +
> + /*
> + * Starting from 'loc', apply the CFA opcodes that advance the location
> + * pointer, and identify the locations of the PAC instructions.
> + */
> + do {
> + switch (*opcode & 0xC0) {
> + case 0:
> + // handle DW_CFA_xxx opcodes
> + switch (*opcode) {
> + case DW_CFA_nop:
> + case DW_CFA_remember_state:
> + case DW_CFA_restore_state:
> + break;
> +
> + case DW_CFA_advance_loc1:
> + loc += *++opcode;
> + size--;
> + break;
> +
> + case DW_CFA_advance_loc2:
> + loc += *++opcode;
> + loc += *++opcode << 8;
> + size -= 2;
> + break;
> +
> + case DW_CFA_def_cfa:
> + opcode++;
> + size = skip_xleb128(&opcode, --size);
> + size = skip_xleb128(&opcode, size);
> + continue;
> + case DW_CFA_def_cfa_offset:
> + case DW_CFA_def_cfa_offset_sf:
> + case DW_CFA_def_cfa_register:
> + case DW_CFA_same_value:
> + opcode++;
> + size = skip_xleb128(&opcode, --size);
> + continue;
> +
> + case DW_CFA_negate_ra_state:
> + scs_patch_loc(loc - 4);
> + break;
> +
> + default:
> + pr_err("unhandled opcode: %02x in FDE frame %lx\n", *opcode, (uintptr_t)eh_frame);
I'm curious, if we made these identifiers enum values, then we could
get coverage from -Wswitch.
Though then there is perhaps a risk that new values do start getting
produced by toolchains, and we'd miss adding the enum values in those
cases.
These nested switch statements make it hard to tell which values are
handled where without doing some arithmetic. Would one level of switch
statements with more cases be more readable? Or is there a goal to
have a smaller initial-level switch table?
> + return -ENOEXEC;
> + }
> + opcode++;
> + size--;
> + break;
> +
> + case 0x40:
> + // advance loc
> + loc += *opcode++ & 0x3f;
> + size--;
> + break;
> +
> + case 0x80:
> + opcode++;
> + size = skip_xleb128(&opcode, --size);
> + continue;
> +
> + default:
> + // ignore
> + opcode++;
> + size--;
> + break;
> + }
> + } while (size > 0);
> +
> + return 0;
> +}
> +
> +int noinstr scs_patch(const u8 eh_frame[], int size)
> +{
> + const u8 *p = eh_frame;
> +
> + while (size > 4) {
> + const u32 *frame_size = (const u32 *)p;
> + int ret;
> +
> + if (*frame_size != -1 && *frame_size <= size) {
> + ret = scs_handle_frame(p + 4, *frame_size);
> + if (ret)
> + return ret;
> + p += 4 + *frame_size;
> + size -= 4 + *frame_size;
> + }
> + }
> + return 0;
> +}
> +
> +extern struct arm64_ftr_override id_aa64isar1_override;
> +extern struct arm64_ftr_override id_aa64isar2_override;
> +extern struct arm64_ftr_override id_aa64pfr1_override;
Are these linker defined symbols, like __eh_frame_start/__eh_frame_end?
--
Thanks,
~Nick Desaulniers
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks
2022-05-05 16:10 ` [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks Ard Biesheuvel
@ 2022-05-05 21:14 ` Nick Desaulniers
2022-05-06 0:00 ` Kees Cook
1 sibling, 0 replies; 12+ messages in thread
From: Nick Desaulniers @ 2022-05-05 21:14 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-arm-kernel, will, mark.rutland, catalin.marinas, maz,
Kees Cook, Sami Tolvanen, Fangrui Song, Dan Li
On Thu, May 5, 2022 at 9:10 AM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> In order to allow arches to use code patching to conditionally emit the
> shadow stack pushes and pops, rather than always taking the performance
> hit even on CPUs that implement alternatives such as stack pointer
> authentication on arm64, add a Kconfig symbol that can be set by the
> arch to omit the SCS codegen itself, without otherwise affecting how
> support code for SCS and compiler options (for register reservation, for
> instance) are emitted.
>
> Also, add a static key and some plumbing to omit the allocation of
> shadow call stack for dynamic SCS configurations if SCS is disabled at
> runtime.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Thanks for the patch!
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Makefile | 2 ++
> arch/Kconfig | 7 +++++++
> include/linux/scs.h | 10 ++++++++++
> kernel/scs.c | 14 ++++++++++++--
> 4 files changed, 31 insertions(+), 2 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index fa5112a0ec1b..a578fffc0337 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -882,8 +882,10 @@ LDFLAGS_vmlinux += --gc-sections
> endif
>
> ifdef CONFIG_SHADOW_CALL_STACK
> +ifndef CONFIG_DYNAMIC_SCS
> CC_FLAGS_SCS := -fsanitize=shadow-call-stack
> KBUILD_CFLAGS += $(CC_FLAGS_SCS)
> +endif
> export CC_FLAGS_SCS
> endif
>
> diff --git a/arch/Kconfig b/arch/Kconfig
> index 29b0167c088b..126caa75969a 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -627,6 +627,13 @@ config SHADOW_CALL_STACK
> reading and writing arbitrary memory may be able to locate them
> and hijack control flow by modifying the stacks.
>
> +config DYNAMIC_SCS
> + bool
> + help
> + Set by the arch code if it relies on code patching to insert the
> + shadow call stack push and pop instructions rather than on the
> + compiler.
> +
> config LTO
> bool
> help
> diff --git a/include/linux/scs.h b/include/linux/scs.h
> index 18122d9e17ff..4cc01f21b17a 100644
> --- a/include/linux/scs.h
> +++ b/include/linux/scs.h
> @@ -53,6 +53,15 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk)
> return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC;
> }
>
> +DECLARE_STATIC_KEY_TRUE(dynamic_scs_enabled);
> +
> +static inline bool scs_is_enabled(void)
> +{
> + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS))
> + return true;
> + return static_branch_likely(&dynamic_scs_enabled);
> +}
> +
> #else /* CONFIG_SHADOW_CALL_STACK */
>
> static inline void *scs_alloc(int node) { return NULL; }
> @@ -62,6 +71,7 @@ static inline void scs_task_reset(struct task_struct *tsk) {}
> static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; }
> static inline void scs_release(struct task_struct *tsk) {}
> static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; }
> +static inline bool scs_is_enabled(void) { return false; }
>
> #endif /* CONFIG_SHADOW_CALL_STACK */
>
> diff --git a/kernel/scs.c b/kernel/scs.c
> index b7e1b096d906..8826794d2645 100644
> --- a/kernel/scs.c
> +++ b/kernel/scs.c
> @@ -12,6 +12,10 @@
> #include <linux/vmalloc.h>
> #include <linux/vmstat.h>
>
> +#ifdef CONFIG_DYNAMIC_SCS
> +DEFINE_STATIC_KEY_TRUE(dynamic_scs_enabled);
> +#endif
> +
> static void __scs_account(void *s, int account)
> {
> struct page *scs_page = vmalloc_to_page(s);
> @@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu)
>
> void __init scs_init(void)
> {
> + if (!scs_is_enabled())
> + return;
> cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL,
> scs_cleanup);
> }
>
> int scs_prepare(struct task_struct *tsk, int node)
> {
> - void *s = scs_alloc(node);
> + void *s;
>
> + if (!scs_is_enabled())
> + return 0;
> +
> + s = scs_alloc(node);
> if (!s)
> return -ENOMEM;
>
> @@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk)
> {
> void *s = task_scs(tsk);
>
> - if (!s)
> + if (!scs_is_enabled() || !s)
> return;
>
> WARN(task_scs_end_corrupted(tsk),
> --
> 2.30.2
>
It is less obvious that the two other functions with extern linkage
defined in this TU don't need scs_is_enabled checks because of guards
in the callers.
--
Thanks,
~Nick Desaulniers
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang
2022-05-05 16:10 ` [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang Ard Biesheuvel
2022-05-05 21:01 ` Nick Desaulniers
@ 2022-05-06 0:00 ` Kees Cook
2022-05-06 6:59 ` Ard Biesheuvel
1 sibling, 1 reply; 12+ messages in thread
From: Kees Cook @ 2022-05-06 0:00 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-arm-kernel, will, mark.rutland, catalin.marinas, maz,
Sami Tolvanen, Fangrui Song, Nick Desaulniers, Dan Li
On Thu, May 05, 2022 at 06:10:11PM +0200, Ard Biesheuvel wrote:
> [...]
> + /*
> + * We only enable the shadow call stack dynamically if we are running
> + * on a system that does not implement PAC or BTI. PAC and SCS roughly
> + * provide the same level of protection, and BTI relies on the PACIASP
> + * instructions serving as landing pads, preventing us from patching
> + * those instructions into something else.
> + */
If BTI relies on PAC, then we only need to check for PAC, yes? I.e.
there isn't going to be a device with BTI but without PAC.
--
Kees Cook
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks
2022-05-05 16:10 ` [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks Ard Biesheuvel
2022-05-05 21:14 ` Nick Desaulniers
@ 2022-05-06 0:00 ` Kees Cook
1 sibling, 0 replies; 12+ messages in thread
From: Kees Cook @ 2022-05-06 0:00 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-arm-kernel, will, mark.rutland, catalin.marinas, maz,
Sami Tolvanen, Fangrui Song, Nick Desaulniers, Dan Li
On Thu, May 05, 2022 at 06:10:10PM +0200, Ard Biesheuvel wrote:
> In order to allow arches to use code patching to conditionally emit the
> shadow stack pushes and pops, rather than always taking the performance
> hit even on CPUs that implement alternatives such as stack pointer
> authentication on arm64, add a Kconfig symbol that can be set by the
> arch to omit the SCS codegen itself, without otherwise affecting how
> support code for SCS and compiler options (for register reservation, for
> instance) are emitted.
>
> Also, add a static key and some plumbing to omit the allocation of
> shadow call stack for dynamic SCS configurations if SCS is disabled at
> runtime.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
--
Kees Cook
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang
2022-05-06 0:00 ` Kees Cook
@ 2022-05-06 6:59 ` Ard Biesheuvel
0 siblings, 0 replies; 12+ messages in thread
From: Ard Biesheuvel @ 2022-05-06 6:59 UTC (permalink / raw)
To: Kees Cook
Cc: Linux ARM, Will Deacon, Mark Rutland, Catalin Marinas,
Marc Zyngier, Sami Tolvanen, Fangrui Song, Nick Desaulniers,
Dan Li
On Fri, 6 May 2022 at 02:00, Kees Cook <keescook@chromium.org> wrote:
>
> On Thu, May 05, 2022 at 06:10:11PM +0200, Ard Biesheuvel wrote:
> > [...]
> > + /*
> > + * We only enable the shadow call stack dynamically if we are running
> > + * on a system that does not implement PAC or BTI. PAC and SCS roughly
> > + * provide the same level of protection, and BTI relies on the PACIASP
> > + * instructions serving as landing pads, preventing us from patching
> > + * those instructions into something else.
> > + */
>
> If BTI relies on PAC, then we only need to check for PAC, yes? I.e.
> there isn't going to be a device with BTI but without PAC.
>
BTI does not rely on PAC, but PACIASP serves as an implicit BTI
instruction as well as a 'sign return address' instruction, given that
it usually appears at the start of a function. So we cannot patch it
away in that case, but I expect this to be a rare case anyway.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules
2022-05-05 20:19 ` Nick Desaulniers
@ 2022-05-06 7:07 ` Ard Biesheuvel
0 siblings, 0 replies; 12+ messages in thread
From: Ard Biesheuvel @ 2022-05-06 7:07 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Linux ARM, Will Deacon, Mark Rutland, Catalin Marinas,
Marc Zyngier, Kees Cook, Sami Tolvanen, Fangrui Song, Dan Li
On Thu, 5 May 2022 at 22:20, Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Thu, May 5, 2022 at 9:10 AM Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > Enable asynchronous unwind table generation for both the core kernel as
> > well as modules, and emit the resulting .eh_frame sections as init code
> > so we can use the unwind directives for code patching at boot or module
> > load time.
> >
> > This will be used by dynamic shadow call stack support, which will rely
> > on code patching rather than compiler codegen to emit the shadow call
> > stack push and pop instructions.
> >
> > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>
> Two minor nits inline below; I don't feel strongly about them either way.
> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
>
> > ---
> > arch/arm64/Kconfig | 3 +++
> > arch/arm64/Makefile | 5 +++++
> > arch/arm64/include/asm/module.lds.h | 8 ++++++++
> > arch/arm64/kernel/vmlinux.lds.S | 16 ++++++++++++++++
> > drivers/firmware/efi/libstub/Makefile | 1 +
> > 5 files changed, 33 insertions(+)
> >
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 57c4c995965f..b6302f7cd73f 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -354,6 +354,9 @@ config KASAN_SHADOW_OFFSET
> > default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS
> > default 0xffffffffffffffff
> >
> > +config UNWIND_TABLES
> > + bool
> > +
> > source "arch/arm64/Kconfig.platforms"
> >
> > menu "Kernel Features"
> > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> > index 2f1de88651e6..a4c6807ecbaf 100644
> > --- a/arch/arm64/Makefile
> > +++ b/arch/arm64/Makefile
> > @@ -45,8 +45,13 @@ KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
> > KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
> >
> > # Avoid generating .eh_frame* sections.
> > +ifneq ($(CONFIG_UNWIND_TABLES),y)
>
> You know how I feel about negated conditions when there's an else branch...
>
> > KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
> > KBUILD_AFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
> > +else
> > +KBUILD_CFLAGS += -fasynchronous-unwind-tables
> > +KBUILD_AFLAGS += -fasynchronous-unwind-tables
> > +endif
> >
> > ifeq ($(CONFIG_STACKPROTECTOR_PER_TASK),y)
> > prepare: stack_protector_prepare
> > diff --git a/arch/arm64/include/asm/module.lds.h b/arch/arm64/include/asm/module.lds.h
> > index 094701ec5500..dbba4b7559aa 100644
> > --- a/arch/arm64/include/asm/module.lds.h
> > +++ b/arch/arm64/include/asm/module.lds.h
> > @@ -17,4 +17,12 @@ SECTIONS {
> > */
> > .text.hot : { *(.text.hot) }
> > #endif
> > +
> > +#ifdef CONFIG_UNWIND_TABLES
> > + /*
> > + * Currently, we only use unwind info at module load time, so we can
> > + * put it into the .init allocation.
> > + */
> > + .init.eh_frame : { *(.eh_frame) }
> > +#endif
> > }
> > diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
> > index edaf0faf766f..2f4908c8d152 100644
> > --- a/arch/arm64/kernel/vmlinux.lds.S
> > +++ b/arch/arm64/kernel/vmlinux.lds.S
> > @@ -122,6 +122,17 @@ jiffies = jiffies_64;
> > #define TRAMP_TEXT
> > #endif
> >
> > +#ifdef CONFIG_UNWIND_TABLES
> > +#define UNWIND_DATA_SECTIONS \
> > + .eh_frame : { \
> > + __eh_frame_start = .; \
> > + *(.eh_frame) \
> > + __eh_frame_end = .; \
> > + }
> > +#else
> > +#define UNWIND_DATA_SECTIONS
> > +#endif
> > +
> > /*
> > * The size of the PE/COFF section that covers the kernel image, which
> > * runs from _stext to _edata, must be a round multiple of the PE/COFF
> > @@ -150,6 +161,9 @@ SECTIONS
> > /DISCARD/ : {
> > *(.interp .dynamic)
> > *(.dynsym .dynstr .hash .gnu.hash)
> > +#ifndef CONFIG_UNWIND_TABLES
> > + *(.eh_frame)
> > +#endif
>
> If CONFIG_UNWIND_TABLES is unset, then
> `-fno-asynchronous-unwind-tables -fno-unwind-tables` should be set; it
> would be anomalous otherwise. We could leave this hunk out and we
> should get a warning from `--orphan-handling=warn` linker flag which
> would warn explicitly about such anomalies, rather than produce but
> then silently discard them.
>
Agreed, I'll drop this bit.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang
2022-05-05 21:01 ` Nick Desaulniers
@ 2022-05-18 1:53 ` Sami Tolvanen
0 siblings, 0 replies; 12+ messages in thread
From: Sami Tolvanen @ 2022-05-18 1:53 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Ard Biesheuvel, linux-arm-kernel, Will Deacon, Mark Rutland,
Catalin Marinas, Marc Zyngier, Kees Cook, Fangrui Song, Dan Li
On Thu, May 5, 2022 at 2:01 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Thu, May 5, 2022 at 9:10 AM Ard Biesheuvel <ardb@kernel.org> wrote:
> > +enum {
> > + PACIASP = 0xd503233f,
> > + AUTIASP = 0xd50323bf,
> > + SCS_PUSH = 0xf800865e,
> > + SCS_POP = 0xf85f8e5e,
> > +};
>
> Is there anything we can reuse from arch/arm64/include/asm/insn.h
> rather than hardcoding these values?
I think hardcoding the instructions should be fine. You could use
aarch64_insn_gen_hint for the first two, but the last time I checked
we didn't have code for generating the SCS push/pop instructions.
Sami
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2022-05-18 1:55 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-05 16:10 [RFC PATCH v2 0/3] arm64: dynamic shadow call stack support Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 1/3] arm64: unwind: add asynchronous unwind tables to kernel and modules Ard Biesheuvel
2022-05-05 20:19 ` Nick Desaulniers
2022-05-06 7:07 ` Ard Biesheuvel
2022-05-05 16:10 ` [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks Ard Biesheuvel
2022-05-05 21:14 ` Nick Desaulniers
2022-05-06 0:00 ` Kees Cook
2022-05-05 16:10 ` [RFC PATCH v2 3/3] arm64: implement dynamic shadow call stack for Clang Ard Biesheuvel
2022-05-05 21:01 ` Nick Desaulniers
2022-05-18 1:53 ` Sami Tolvanen
2022-05-06 0:00 ` Kees Cook
2022-05-06 6:59 ` Ard Biesheuvel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.