From: marc.zyngier@arm.com (Marc Zyngier)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v5 18/23] arm64: KVM: Add epilogue branching to the vector code
Date: Thu, 1 Mar 2018 15:55:33 +0000 [thread overview]
Message-ID: <20180301155538.26860-19-marc.zyngier@arm.com> (raw)
In-Reply-To: <20180301155538.26860-1-marc.zyngier@arm.com>
We are soon going to have to do some extra work in the BP hardening
vector slots. Instead of doing that work in the vectors themselves
(which would massively reduce the space available to deal with
Spectre v2), let's branch to an epilogue where we can do "stuff".
This has a number of consequences:
- We need some free registers, so we're spilling x0 and x1 on the
stack
- In order to counterbalance this, we branch to the *second* instruction
in the vectors, avoiding the initial store that is already there
(or loading the registers back if we've branched to a panic vector)
This is all controlled by a new capability (ARM64_HARDEN_EL2_VECTORS)
which doesn't get enabled yet.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/include/asm/cpucaps.h | 2 +-
arch/arm64/kernel/bpi.S | 57 +++++++++++++++++++++++++---------------
arch/arm64/kvm/hyp/hyp-entry.S | 2 ++
3 files changed, 39 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 76a43a17449a..d4cc54ed0656 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -32,7 +32,7 @@
#define ARM64_HAS_VIRT_HOST_EXTN 11
#define ARM64_WORKAROUND_CAVIUM_27456 12
#define ARM64_HAS_32BIT_EL0 13
-/* #define ARM64_UNALLOCATED_ENTRY 14 */
+#define ARM64_HARDEN_EL2_VECTORS 14
#define ARM64_MISMATCHED_CACHE_LINE_SIZE 15
#define ARM64_HAS_NO_FPSIMD 16
#define ARM64_WORKAROUND_REPEAT_TLBI 17
diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S
index e5de33513b5d..e000cb390618 100644
--- a/arch/arm64/kernel/bpi.S
+++ b/arch/arm64/kernel/bpi.S
@@ -19,40 +19,55 @@
#include <linux/linkage.h>
#include <linux/arm-smccc.h>
-.macro ventry target
- .rept 31
+.macro hyp_ventry offset
+ .align 7
+ .rept 29
nop
.endr
- b \target
+alternative_if ARM64_HARDEN_EL2_VECTORS
+ stp x0, x1, [sp, #-16]!
+ mov x0, #(\offset + 4)
+ b __kvm_enter_vectors
+alternative_else
+ b __kvm_hyp_vector + \offset
+ nop
+ nop
+alternative_endif
.endm
-.macro vectors target
- ventry \target + 0x000
- ventry \target + 0x080
- ventry \target + 0x100
- ventry \target + 0x180
+.macro generate_vectors
+ hyp_ventry 0x000
+ hyp_ventry 0x080
+ hyp_ventry 0x100
+ hyp_ventry 0x180
- ventry \target + 0x200
- ventry \target + 0x280
- ventry \target + 0x300
- ventry \target + 0x380
+ hyp_ventry 0x200
+ hyp_ventry 0x280
+ hyp_ventry 0x300
+ hyp_ventry 0x380
- ventry \target + 0x400
- ventry \target + 0x480
- ventry \target + 0x500
- ventry \target + 0x580
+ hyp_ventry 0x400
+ hyp_ventry 0x480
+ hyp_ventry 0x500
+ hyp_ventry 0x580
- ventry \target + 0x600
- ventry \target + 0x680
- ventry \target + 0x700
- ventry \target + 0x780
+ hyp_ventry 0x600
+ hyp_ventry 0x680
+ hyp_ventry 0x700
+ hyp_ventry 0x780
.endm
.align 11
ENTRY(__bp_harden_hyp_vecs_start)
.rept 4
- vectors __kvm_hyp_vector
+ generate_vectors
.endr
+
+__kvm_enter_vectors:
+
+ adr_l x1, __kvm_hyp_vector
+ add x0, x1, x0
+ br x0
ENTRY(__bp_harden_hyp_vecs_end)
ENTRY(__qcom_hyp_sanitize_link_stack_start)
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 0f62b5f76aa5..fc6a1006cc08 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -220,6 +220,8 @@ ENDPROC(\label)
.macro invalid_vect target
.align 7
b \target
+ ldp x0, x1, [sp], #16
+ b \target
.endm
ENTRY(__kvm_hyp_vector)
--
2.14.2
next prev parent reply other threads:[~2018-03-01 15:55 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-01 15:55 [PATCH v5 00/23] KVM/arm64: Randomise EL2 mappings (variant 3a mitigation) Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 01/23] arm64: alternatives: Add dynamic patching feature Marc Zyngier
2018-03-07 18:09 ` Catalin Marinas
2018-03-01 15:55 ` [PATCH v5 02/23] arm64: insn: Add N immediate encoding Marc Zyngier
2018-03-07 18:09 ` Catalin Marinas
2018-03-01 15:55 ` [PATCH v5 03/23] arm64: insn: Add encoder for bitwise operations using literals Marc Zyngier
2018-03-07 18:10 ` Catalin Marinas
2018-03-12 14:44 ` Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 04/23] arm64: KVM: Dynamically patch the kernel/hyp VA mask Marc Zyngier
2018-03-07 18:10 ` Catalin Marinas
2018-03-01 15:55 ` [PATCH v5 05/23] arm64: cpufeatures: Drop the ARM64_HYP_OFFSET_LOW feature flag Marc Zyngier
2018-03-07 18:11 ` Catalin Marinas
2018-03-13 8:44 ` Suzuki K Poulose
2018-03-01 15:55 ` [PATCH v5 06/23] KVM: arm/arm64: Do not use kern_hyp_va() with kvm_vgic_global_state Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 07/23] KVM: arm/arm64: Demote HYP VA range display to being a debug feature Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 08/23] KVM: arm/arm64: Move ioremap calls to create_hyp_io_mappings Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 09/23] KVM: arm/arm64: Keep GICv2 HYP VAs in kvm_vgic_global_state Marc Zyngier
2018-03-13 9:35 ` Suzuki K Poulose
2018-03-13 11:40 ` Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 10/23] KVM: arm/arm64: Move HYP IO VAs to the "idmap" range Marc Zyngier
2018-03-09 18:59 ` James Morse
2018-03-12 14:02 ` Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 11/23] arm64; insn: Add encoder for the EXTR instruction Marc Zyngier
2018-03-07 18:12 ` Catalin Marinas
2018-03-01 15:55 ` [PATCH v5 12/23] arm64: insn: Allow ADD/SUB (immediate) with LSL #12 Marc Zyngier
2018-03-07 18:13 ` Catalin Marinas
2018-03-01 15:55 ` [PATCH v5 13/23] arm64: KVM: Dynamically compute the HYP VA mask Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 14/23] arm64: KVM: Introduce EL2 VA randomisation Marc Zyngier
2018-03-13 11:31 ` James Morse
2018-03-13 11:48 ` James Morse
2018-03-01 15:55 ` [PATCH v5 15/23] arm64: Update the KVM memory map documentation Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 16/23] arm64: KVM: Move vector offsetting from hyp-init.S to kvm_get_hyp_vector Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 17/23] arm64: KVM: Move stashing of x0/x1 into the vector code itself Marc Zyngier
2018-03-01 15:55 ` Marc Zyngier [this message]
2018-03-08 13:59 ` [PATCH v5 18/23] arm64: KVM: Add epilogue branching to the vector code Catalin Marinas
2018-03-01 15:55 ` [PATCH v5 19/23] arm64: KVM: Allow far branches from vector slots to the main vectors Marc Zyngier
2018-03-08 13:59 ` Catalin Marinas
2018-03-12 18:27 ` James Morse
2018-03-12 19:43 ` Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 20/23] arm/arm64: KVM: Introduce EL2-specific executable mappings Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 21/23] arm64: Make BP hardening slot counter available Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 22/23] arm64: KVM: Allow mapping of vectors outside of the RAM region Marc Zyngier
2018-03-08 17:54 ` Andrew Jones
2018-03-13 10:30 ` Marc Zyngier
2018-03-13 11:14 ` Andrew Jones
2018-03-09 18:59 ` James Morse
2018-03-12 14:23 ` Marc Zyngier
2018-03-14 11:40 ` James Morse
2018-03-14 12:02 ` Marc Zyngier
2018-03-01 15:55 ` [PATCH v5 23/23] arm64: Enable ARM64_HARDEN_EL2_VECTORS on Cortex-A57 and A72 Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180301155538.26860-19-marc.zyngier@arm.com \
--to=marc.zyngier@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).