From: Will Deacon <will.deacon@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon <will.deacon@arm.com> Subject: [PATCH v2 16/18] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Date: Thu, 30 Nov 2017 16:39:44 +0000 [thread overview] Message-ID: <1512059986-21325-17-git-send-email-will.deacon@arm.com> (raw) In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Allow explicit disabling of the entry trampoline on the kernel command line (kaiser=off) by adding a fake CPU feature (ARM64_UNMAP_KERNEL_AT_EL0) that can be used to toggle the alternative sequences in our entry code and avoid use of the trampoline altogether if desired. This also allows us to make use of a static key in arm64_kernel_unmapped_at_el0(). Signed-off-by: Will Deacon <will.deacon@arm.com> --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu.h | 3 ++- arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/entry.S | 11 +++++++---- 4 files changed, 52 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 2ff7c5e8efab..b4537ffd1018 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -41,7 +41,8 @@ #define ARM64_WORKAROUND_CAVIUM_30115 20 #define ARM64_HAS_DCPOP 21 #define ARM64_SVE 22 +#define ARM64_UNMAP_KERNEL_AT_EL0 23 -#define ARM64_NCAPS 23 +#define ARM64_NCAPS 24 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index c07954638658..da6f12e40714 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -36,7 +36,8 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { - return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && + cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0); } extern void paging_init(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index c5ba0097887f..72fc55d22ddb 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -845,6 +845,40 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus ID_AA64PFR0_FP_SHIFT) < 0; } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __kaiser_forced; /* 0: not forced, >0: forced on, <0: forced off */ + +static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + /* Forced on command line? */ + if (__kaiser_forced) { + pr_info("KAISER forced %s by command line option\n", + __kaiser_forced > 0 ? "ON" : "OFF"); + return __kaiser_forced > 0; + } + + /* Useful for KASLR robustness */ + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + return true; + + return false; +} + +static int __init parse_kaiser(char *str) +{ + bool enabled; + int ret = strtobool(str, &enabled); + + if (ret) + return ret; + + __kaiser_forced = enabled ? 1 : -1; + return 0; +} +__setup("kaiser=", parse_kaiser); +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -931,6 +965,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .def_scope = SCOPE_SYSTEM, .matches = hyp_offset_low, }, +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { + .capability = ARM64_UNMAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = unmap_kernel_at_el0, + }, +#endif { /* FP/SIMD is not implemented */ .capability = ARM64_HAS_NO_FPSIMD, diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a5ec6ab5c711..d8775f55e930 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -74,6 +74,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -82,6 +83,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -323,10 +325,10 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp -#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 - eret -#else .if \el == 0 +alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x30 tramp_alias x30, tramp_exit_native @@ -334,10 +336,11 @@ alternative_else_nop_endif 4: tramp_alias x30, tramp_exit_compat br x30 +alternative_else_nop_endif +#endif .else eret .endif -#endif .endm .macro irq_stack_entry -- 2.1.4
WARNING: multiple messages have this Message-ID (diff)
From: will.deacon@arm.com (Will Deacon) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 16/18] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Date: Thu, 30 Nov 2017 16:39:44 +0000 [thread overview] Message-ID: <1512059986-21325-17-git-send-email-will.deacon@arm.com> (raw) In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Allow explicit disabling of the entry trampoline on the kernel command line (kaiser=off) by adding a fake CPU feature (ARM64_UNMAP_KERNEL_AT_EL0) that can be used to toggle the alternative sequences in our entry code and avoid use of the trampoline altogether if desired. This also allows us to make use of a static key in arm64_kernel_unmapped_at_el0(). Signed-off-by: Will Deacon <will.deacon@arm.com> --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu.h | 3 ++- arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/entry.S | 11 +++++++---- 4 files changed, 52 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 2ff7c5e8efab..b4537ffd1018 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -41,7 +41,8 @@ #define ARM64_WORKAROUND_CAVIUM_30115 20 #define ARM64_HAS_DCPOP 21 #define ARM64_SVE 22 +#define ARM64_UNMAP_KERNEL_AT_EL0 23 -#define ARM64_NCAPS 23 +#define ARM64_NCAPS 24 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index c07954638658..da6f12e40714 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -36,7 +36,8 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { - return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && + cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0); } extern void paging_init(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index c5ba0097887f..72fc55d22ddb 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -845,6 +845,40 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus ID_AA64PFR0_FP_SHIFT) < 0; } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __kaiser_forced; /* 0: not forced, >0: forced on, <0: forced off */ + +static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + /* Forced on command line? */ + if (__kaiser_forced) { + pr_info("KAISER forced %s by command line option\n", + __kaiser_forced > 0 ? "ON" : "OFF"); + return __kaiser_forced > 0; + } + + /* Useful for KASLR robustness */ + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + return true; + + return false; +} + +static int __init parse_kaiser(char *str) +{ + bool enabled; + int ret = strtobool(str, &enabled); + + if (ret) + return ret; + + __kaiser_forced = enabled ? 1 : -1; + return 0; +} +__setup("kaiser=", parse_kaiser); +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -931,6 +965,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .def_scope = SCOPE_SYSTEM, .matches = hyp_offset_low, }, +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { + .capability = ARM64_UNMAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = unmap_kernel_at_el0, + }, +#endif { /* FP/SIMD is not implemented */ .capability = ARM64_HAS_NO_FPSIMD, diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a5ec6ab5c711..d8775f55e930 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -74,6 +74,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -82,6 +83,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -323,10 +325,10 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp -#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 - eret -#else .if \el == 0 +alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x30 tramp_alias x30, tramp_exit_native @@ -334,10 +336,11 @@ alternative_else_nop_endif 4: tramp_alias x30, tramp_exit_compat br x30 +alternative_else_nop_endif +#endif .else eret .endif -#endif .endm .macro irq_stack_entry -- 2.1.4
next prev parent reply other threads:[~2017-11-30 16:40 UTC|newest] Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-11-30 16:39 [PATCH v2 00/18] arm64: Unmap the kernel whilst running in userspace (KAISER) Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 01/18] arm64: mm: Use non-global mappings for kernel space Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 02/18] arm64: mm: Temporarily disable ARM64_SW_TTBR0_PAN Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 03/18] arm64: mm: Move ASID from TTBR0 to TTBR1 Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 17:36 ` Mark Rutland 2017-11-30 17:36 ` Mark Rutland 2017-11-30 16:39 ` [PATCH v2 04/18] arm64: mm: Remove pre_ttbr0_update_workaround for Falkor erratum #E1003 Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 05/18] arm64: mm: Rename post_ttbr0_update_workaround Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 06/18] arm64: mm: Fix and re-enable ARM64_SW_TTBR0_PAN Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-12-01 11:48 ` Mark Rutland 2017-12-01 11:48 ` Mark Rutland 2017-11-30 16:39 ` [PATCH v2 07/18] arm64: mm: Allocate ASIDs in pairs Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 08/18] arm64: mm: Add arm64_kernel_unmapped_at_el0 helper Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 09/18] arm64: mm: Invalidate both kernel and user ASIDs when performing TLBI Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 10/18] arm64: entry: Add exception trampoline page for exceptions from EL0 Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-12-01 13:31 ` Mark Rutland 2017-12-01 13:31 ` Mark Rutland 2017-12-06 10:25 ` Ard Biesheuvel 2017-12-06 10:25 ` Ard Biesheuvel 2017-11-30 16:39 ` [PATCH v2 11/18] arm64: mm: Map entry trampoline into trampoline and kernel page tables Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 18:29 ` Mark Rutland 2017-11-30 18:29 ` Mark Rutland 2017-11-30 16:39 ` [PATCH v2 12/18] arm64: entry: Explicitly pass exception level to kernel_ventry macro Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-12-01 11:58 ` Mark Rutland 2017-12-01 11:58 ` Mark Rutland 2017-12-01 17:51 ` Will Deacon 2017-12-01 17:51 ` Will Deacon 2017-12-01 18:00 ` Mark Rutland 2017-12-01 18:00 ` Mark Rutland 2017-11-30 16:39 ` [PATCH v2 13/18] arm64: entry: Hook up entry trampoline to exception vectors Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-12-01 13:53 ` Mark Rutland 2017-12-01 13:53 ` Mark Rutland 2017-12-01 17:40 ` Will Deacon 2017-12-01 17:40 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 14/18] arm64: erratum: Work around Falkor erratum #E1003 in trampoline code Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 17:06 ` Robin Murphy 2017-11-30 17:06 ` Robin Murphy 2017-11-30 17:19 ` Will Deacon 2017-11-30 17:19 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 15/18] arm64: tls: Avoid unconditional zeroing of tpidrro_el0 for native tasks Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-11-30 16:39 ` Will Deacon [this message] 2017-11-30 16:39 ` [PATCH v2 16/18] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Will Deacon 2017-12-01 13:55 ` Mark Rutland 2017-12-01 13:55 ` Mark Rutland 2017-11-30 16:39 ` [PATCH v2 17/18] arm64: Kconfig: Add CONFIG_UNMAP_KERNEL_AT_EL0 Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-12-12 8:44 ` Geert Uytterhoeven 2017-12-12 8:44 ` Geert Uytterhoeven 2017-12-12 10:28 ` Will Deacon 2017-12-12 10:28 ` Will Deacon 2017-11-30 16:39 ` [PATCH v2 18/18] perf: arm_spe: Disallow userspace profiling when arm_kernel_unmapped_at_el0() Will Deacon 2017-11-30 16:39 ` Will Deacon 2017-12-01 12:15 ` Mark Rutland 2017-12-01 12:15 ` Mark Rutland 2017-12-01 16:49 ` Will Deacon 2017-12-01 16:49 ` Will Deacon 2017-12-01 16:26 ` Stephen Boyd 2017-12-01 16:26 ` Stephen Boyd 2017-12-01 14:04 ` [PATCH v2 00/18] arm64: Unmap the kernel whilst running in userspace (KAISER) Mark Rutland 2017-12-01 14:04 ` Mark Rutland 2017-12-01 17:50 ` Will Deacon 2017-12-01 17:50 ` Will Deacon 2017-12-01 17:58 ` Mark Rutland 2017-12-01 17:58 ` Mark Rutland 2017-12-01 18:02 ` Dave Hansen 2017-12-01 18:02 ` Dave Hansen 2017-12-01 18:14 ` Will Deacon 2017-12-01 18:14 ` Will Deacon 2017-12-11 2:24 ` Shanker Donthineni 2017-12-11 2:24 ` Shanker Donthineni 2017-12-04 23:47 ` Laura Abbott 2017-12-04 23:47 ` Laura Abbott
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1512059986-21325-17-git-send-email-will.deacon@arm.com \ --to=will.deacon@arm.com \ --cc=ard.biesheuvel@linaro.org \ --cc=catalin.marinas@arm.com \ --cc=dave.hansen@linux.intel.com \ --cc=keescook@chromium.org \ --cc=labbott@redhat.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=msalter@redhat.com \ --cc=sboyd@codeaurora.org \ --cc=tglx@linutronix.de \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.