* [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537
@ 2019-09-25 11:19 Marc Zyngier
2019-09-25 11:19 ` [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Marc Zyngier
` (4 more replies)
0 siblings, 5 replies; 13+ messages in thread
From: Marc Zyngier @ 2019-09-25 11:19 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Suzuki K Poulose, Catalin Marinas, James Morse,
Will Deacon, Julien Thierry
Similarily to erratum 1165522 that affects Cortex-A76, our good old
friends A57 and A72 respectively suffer from errata 1319367 and
1319537, potentially resulting in TLB corruption if the CPU speculates
an AT instruction while switching guests.
The fix is slightly more involved since we don't have VHE to help us
here, but the idea is the same: When switching a guest in, we must
prevent any speculated AT from being able to parse the page tables
until S2 is up and running. Only at this stage can we allow AT to take
place.
For this, we always restore the guest sysregs first, except for its
SCTLR and TCR registers, which must be set with SCTLR.M=1 and
TCR.EPD{0,1} = {1, 1}, effectively disabling the PTW and TLB
allocation. Once S2 is setup, we restore the guest's SCTLR and
TCR. Similar things must be done on TLB invalidation... Fun.
This has been tested on an AMD Seattle box.
Marc Zyngier (5):
arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions
arm64: KVM: Reorder system register restoration and stage-2 activation
arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs
arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context
arm64: Enable and document ARM errata 1319367 and 1319537
Documentation/arm64/silicon-errata.rst | 4 +++
arch/arm64/Kconfig | 10 ++++++
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/kernel/cpu_errata.c | 13 +++++--
arch/arm64/kvm/hyp/switch.c | 48 ++++++++++++++++++++++----
arch/arm64/kvm/hyp/sysreg-sr.c | 14 ++++++--
arch/arm64/kvm/hyp/tlb.c | 23 ++++++++++++
7 files changed, 103 insertions(+), 12 deletions(-)
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions
2019-09-25 11:19 [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537 Marc Zyngier
@ 2019-09-25 11:19 ` Marc Zyngier
2019-09-27 8:33 ` Suzuki K Poulose
2019-09-27 9:03 ` Catalin Marinas
2019-09-25 11:19 ` [PATCH 2/5] arm64: KVM: Reorder system register restoration and stage-2 activation Marc Zyngier
` (3 subsequent siblings)
4 siblings, 2 replies; 13+ messages in thread
From: Marc Zyngier @ 2019-09-25 11:19 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Suzuki K Poulose, Catalin Marinas, James Morse,
Will Deacon, Julien Thierry
Rework the EL2 vector hardening that is only selected for A57 and A72
so that the table can also be used for ARM64_WORKAROUND_1319367.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/cpucaps.h | 3 ++-
arch/arm64/kernel/cpu_errata.c | 13 ++++++++++---
2 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index f19fe4b9acc4..277e37b2a513 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -52,7 +52,8 @@
#define ARM64_HAS_IRQ_PRIO_MASKING 42
#define ARM64_HAS_DCPODP 43
#define ARM64_WORKAROUND_1463225 44
+#define ARM64_WORKAROUND_1319367 45
-#define ARM64_NCAPS 45
+#define ARM64_NCAPS 46
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 1e43ba5c79b7..1640c988d5a1 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -623,9 +623,9 @@ check_branch_predictor(const struct arm64_cpu_capabilities *entry, int scope)
return (need_wa > 0);
}
-#ifdef CONFIG_HARDEN_EL2_VECTORS
+#if defined(CONFIG_HARDEN_EL2_VECTORS) || defined(CONFIG_ARM64_ERRATUM_1319367)
-static const struct midr_range arm64_harden_el2_vectors[] = {
+static const struct midr_range ca57_a72[] = {
MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
{},
@@ -819,7 +819,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "EL2 vector hardening",
.capability = ARM64_HARDEN_EL2_VECTORS,
- ERRATA_MIDR_RANGE_LIST(arm64_harden_el2_vectors),
+ ERRATA_MIDR_RANGE_LIST(ca57_a72),
},
#endif
{
@@ -851,6 +851,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.matches = has_cortex_a76_erratum_1463225,
},
+#endif
+#ifdef CONFIG_ARM64_ERRATUM_1319367
+ {
+ .desc = "ARM erratum 1319367",
+ .capability = ARM64_WORKAROUND_1319367,
+ ERRATA_MIDR_RANGE_LIST(ca57_a72),
+ },
#endif
{
}
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/5] arm64: KVM: Reorder system register restoration and stage-2 activation
2019-09-25 11:19 [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537 Marc Zyngier
2019-09-25 11:19 ` [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Marc Zyngier
@ 2019-09-25 11:19 ` Marc Zyngier
2019-09-25 11:19 ` [PATCH 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs Marc Zyngier
` (2 subsequent siblings)
4 siblings, 0 replies; 13+ messages in thread
From: Marc Zyngier @ 2019-09-25 11:19 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Suzuki K Poulose, Catalin Marinas, James Morse,
Will Deacon, Julien Thierry
In order to prepare for handling erratum 1319367, we need to make
sure that all system registers (and most importantly the registers
configuring the virtual memory) are set before we enable stage-2
translation.
This results in a minor reorganisation of the load sequence, without
any functional change.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/switch.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index a15baca9aca0..e6adb90c12ae 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -605,18 +605,23 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(host_ctxt);
- __activate_vm(kern_hyp_va(vcpu->kvm));
- __activate_traps(vcpu);
-
- __hyp_vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
-
/*
* We must restore the 32-bit state before the sysregs, thanks
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ *
+ * Also, and in order to be able to deal with erratum #1319537 (A57)
+ * and #1319367 (A72), we must ensure that all VM-related sysreg are
+ * restored before we enable S2 translation.
*/
__sysreg32_restore_state(vcpu);
__sysreg_restore_state_nvhe(guest_ctxt);
+
+ __activate_vm(kern_hyp_va(vcpu->kvm));
+ __activate_traps(vcpu);
+
+ __hyp_vgic_restore_state(vcpu);
+ __timer_enable_traps(vcpu);
+
__debug_switch_to_guest(vcpu);
__set_guest_arch_workaround_state(vcpu);
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs
2019-09-25 11:19 [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537 Marc Zyngier
2019-09-25 11:19 ` [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Marc Zyngier
2019-09-25 11:19 ` [PATCH 2/5] arm64: KVM: Reorder system register restoration and stage-2 activation Marc Zyngier
@ 2019-09-25 11:19 ` Marc Zyngier
2019-10-03 11:10 ` James Morse
2019-09-25 11:19 ` [PATCH 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context Marc Zyngier
2019-09-25 11:19 ` [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537 Marc Zyngier
4 siblings, 1 reply; 13+ messages in thread
From: Marc Zyngier @ 2019-09-25 11:19 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Suzuki K Poulose, Catalin Marinas, James Morse,
Will Deacon, Julien Thierry
When erratum 1319367 is being worked around, special care must
be taken not to allow the page table walker to populate TLBs
while we have the stage-2 translation enabled (which would otherwise
result in a bizare mix of the host S1 and the guest S2).
We enforce this by setting TCR_EL1.EPD{0,1} before restoring the S2
configuration, and clear the same bits after having disabled S2.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/tlb.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
index eb0efc5557f3..4ef0bf0d76a6 100644
--- a/arch/arm64/kvm/hyp/tlb.c
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -63,6 +63,22 @@ static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm,
static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm,
struct tlb_inv_context *cxt)
{
+ if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+ u64 val;
+
+ /*
+ * For CPUs that are affected by ARM 1319367, we need to
+ * avoid a host Stage-1 walk while we have the guest's
+ * Stage-2 set in the VTTBR in order to invalidate TLBs.
+ * We're guaranteed that the S1 MMU is enabled, so we can
+ * simply set the EPD bits to avoid any further TLB fill.
+ */
+ val = cxt->tcr = read_sysreg_el1(SYS_TCR);
+ val |= TCR_EPD1_MASK | TCR_EPD0_MASK;
+ write_sysreg_el1(val, SYS_TCR);
+ isb();
+ }
+
__load_guest_stage2(kvm);
isb();
}
@@ -100,6 +116,13 @@ static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm,
struct tlb_inv_context *cxt)
{
write_sysreg(0, vttbr_el2);
+
+ if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+ /* Ensure stage-2 is actually disabled */
+ isb();
+ /* Restore the host's TCR_EL1 */
+ write_sysreg_el1(cxt->tcr, SYS_TCR);
+ }
}
static void __hyp_text __tlb_switch_to_host(struct kvm *kvm,
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context
2019-09-25 11:19 [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537 Marc Zyngier
` (2 preceding siblings ...)
2019-09-25 11:19 ` [PATCH 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs Marc Zyngier
@ 2019-09-25 11:19 ` Marc Zyngier
2019-10-03 11:09 ` James Morse
2019-09-25 11:19 ` [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537 Marc Zyngier
4 siblings, 1 reply; 13+ messages in thread
From: Marc Zyngier @ 2019-09-25 11:19 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Suzuki K Poulose, Catalin Marinas, James Morse,
Will Deacon, Julien Thierry
When handling erratum 1319367, we must ensure that the page table
walker cannot parse the S1 page tables while the guest is in an
inconsistent state. This is done as follows:
On guest entry:
- TCR_EL1.EPD{0,1} are set, ensuring that no PTW can occur
- all system registers are restored, except for TCR_EL1 and SCTLR_EL1
- stage-2 is restored
- SCTLR_EL1 and TCR_EL1 are restored
On guest exit:
- SCTLR_EL1.M and TCR_EL1.EPD{0,1} are set, ensuring that no PTW can occur
- stage-2 is disabled
- All host system registers are restored
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/hyp/switch.c | 31 +++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/sysreg-sr.c | 14 ++++++++++++--
2 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index e6adb90c12ae..4df47d013bec 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -118,6 +118,20 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
}
write_sysreg(val, cptr_el2);
+
+ if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+ struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
+
+ isb();
+ /*
+ * At this stage, and thanks to the above isb(), S2 is
+ * configured and enabled. We can now restore the guest's S1
+ * configuration: SCTLR, and only then TCR.
+ */
+ write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR);
+ isb();
+ write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR);
+ }
}
static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
@@ -156,6 +170,23 @@ static void __hyp_text __deactivate_traps_nvhe(void)
{
u64 mdcr_el2 = read_sysreg(mdcr_el2);
+ if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+ u64 val;
+
+ /*
+ * Set the TCR and SCTLR registers in the exact opposite
+ * sequence as __activate_traps_nvhe (first prevent walks,
+ * then force the MMU on). A generous sprinkling of isb()
+ * ensure that things happen in this exact order.
+ */
+ val = read_sysreg_el1(SYS_TCR);
+ write_sysreg_el1(val | TCR_EPD1_MASK | TCR_EPD0_MASK, SYS_TCR);
+ isb();
+ val = read_sysreg_el1(SYS_SCTLR);
+ write_sysreg_el1(val | SCTLR_ELx_M, SYS_SCTLR);
+ isb();
+ }
+
__deactivate_traps_common();
mdcr_el2 &= MDCR_EL2_HPMN_MASK;
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 7ddbc849b580..adabdceacc10 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -117,12 +117,22 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
{
write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
- write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR);
+
+ /* Must only be done for guest registers, hence the context test */
+ if (cpus_have_const_cap(ARM64_WORKAROUND_1319367) &&
+ !ctxt->__hyp_running_vcpu) {
+ write_sysreg_el1(ctxt->sys_regs[TCR_EL1] |
+ TCR_EPD1_MASK | TCR_EPD0_MASK, SYS_TCR);
+ isb();
+ } else {
+ write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR);
+ write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR);
+ }
+
write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
write_sysreg_el1(ctxt->sys_regs[CPACR_EL1], SYS_CPACR);
write_sysreg_el1(ctxt->sys_regs[TTBR0_EL1], SYS_TTBR0);
write_sysreg_el1(ctxt->sys_regs[TTBR1_EL1], SYS_TTBR1);
- write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR);
write_sysreg_el1(ctxt->sys_regs[ESR_EL1], SYS_ESR);
write_sysreg_el1(ctxt->sys_regs[AFSR0_EL1], SYS_AFSR0);
write_sysreg_el1(ctxt->sys_regs[AFSR1_EL1], SYS_AFSR1);
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537
2019-09-25 11:19 [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537 Marc Zyngier
` (3 preceding siblings ...)
2019-09-25 11:19 ` [PATCH 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context Marc Zyngier
@ 2019-09-25 11:19 ` Marc Zyngier
2019-09-27 9:03 ` Catalin Marinas
2019-10-03 11:11 ` James Morse
4 siblings, 2 replies; 13+ messages in thread
From: Marc Zyngier @ 2019-09-25 11:19 UTC (permalink / raw)
To: linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Suzuki K Poulose, Catalin Marinas, James Morse,
Will Deacon, Julien Thierry
Now that everything is in place, let's get the ball rolling
by allowing the corresponding config option to be selected.
Also add the required information to silicon_arrata.rst.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
Documentation/arm64/silicon-errata.rst | 4 ++++
arch/arm64/Kconfig | 10 ++++++++++
2 files changed, 14 insertions(+)
diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst
index 3e57d09246e6..b90a977fc748 100644
--- a/Documentation/arm64/silicon-errata.rst
+++ b/Documentation/arm64/silicon-errata.rst
@@ -70,8 +70,12 @@ stable kernels.
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 |
+----------------+-----------------+-----------------+-----------------------------+
+| ARM | Cortex-A57 | #1319537 | ARM64_ERRATUM_1319367 |
++----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A72 | #853709 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
+| ARM | Cortex-A72 | #1319367 | ARM64_ERRATUM_1319367 |
++----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 3adcec05b1f6..c50cd4f83bc4 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -523,6 +523,16 @@ config ARM64_ERRATUM_1286807
invalidated has been observed by other observers. The
workaround repeats the TLBI+DSB operation.
+config ARM64_ERRATUM_1319367
+ bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
+ default y
+ help
+ This option adds work arounds for ARM Cortex-A57 erratum 1319537
+ and A72 erratum 1319367
+
+ Cortex-A57 and A72 cores could end-up with corrupted TLBs by
+ speculating an AT instruction during a guest context switch.
+
If unsure, say Y.
config ARM64_ERRATUM_1463225
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions
2019-09-25 11:19 ` [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Marc Zyngier
@ 2019-09-27 8:33 ` Suzuki K Poulose
2019-09-27 9:03 ` Catalin Marinas
1 sibling, 0 replies; 13+ messages in thread
From: Suzuki K Poulose @ 2019-09-27 8:33 UTC (permalink / raw)
To: Marc Zyngier, linux-arm-kernel, kvmarm, kvm
Cc: Mark Rutland, Catalin Marinas, Will Deacon, James Morse, Julien Thierry
On 25/09/2019 12:19, Marc Zyngier wrote:
> Rework the EL2 vector hardening that is only selected for A57 and A72
> so that the table can also be used for ARM64_WORKAROUND_1319367.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions
2019-09-25 11:19 ` [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Marc Zyngier
2019-09-27 8:33 ` Suzuki K Poulose
@ 2019-09-27 9:03 ` Catalin Marinas
1 sibling, 0 replies; 13+ messages in thread
From: Catalin Marinas @ 2019-09-27 9:03 UTC (permalink / raw)
To: Marc Zyngier
Cc: Mark Rutland, kvm, Suzuki K Poulose, James Morse, Julien Thierry,
Will Deacon, kvmarm, linux-arm-kernel
On Wed, Sep 25, 2019 at 12:19:37PM +0100, Marc Zyngier wrote:
> Rework the EL2 vector hardening that is only selected for A57 and A72
> so that the table can also be used for ARM64_WORKAROUND_1319367.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537
2019-09-25 11:19 ` [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537 Marc Zyngier
@ 2019-09-27 9:03 ` Catalin Marinas
2019-10-03 11:11 ` James Morse
1 sibling, 0 replies; 13+ messages in thread
From: Catalin Marinas @ 2019-09-27 9:03 UTC (permalink / raw)
To: Marc Zyngier
Cc: Mark Rutland, kvm, Suzuki K Poulose, James Morse, Julien Thierry,
Will Deacon, kvmarm, linux-arm-kernel
On Wed, Sep 25, 2019 at 12:19:41PM +0100, Marc Zyngier wrote:
> Now that everything is in place, let's get the ball rolling
> by allowing the corresponding config option to be selected.
> Also add the required information to silicon_arrata.rst.
>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context
2019-09-25 11:19 ` [PATCH 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context Marc Zyngier
@ 2019-10-03 11:09 ` James Morse
0 siblings, 0 replies; 13+ messages in thread
From: James Morse @ 2019-10-03 11:09 UTC (permalink / raw)
To: Marc Zyngier
Cc: Mark Rutland, kvm, Suzuki K Poulose, Catalin Marinas,
Julien Thierry, Will Deacon, kvmarm, linux-arm-kernel
Hi Marc,
On 25/09/2019 12:19, Marc Zyngier wrote:
> When handling erratum 1319367, we must ensure that the page table
> walker cannot parse the S1 page tables while the guest is in an
> inconsistent state. This is done as follows:
>
> On guest entry:
> - TCR_EL1.EPD{0,1} are set, ensuring that no PTW can occur
> - all system registers are restored, except for TCR_EL1 and SCTLR_EL1
> - stage-2 is restored
> - SCTLR_EL1 and TCR_EL1 are restored
>
> On guest exit:
> - SCTLR_EL1.M and TCR_EL1.EPD{0,1} are set, ensuring that no PTW can occur
> - stage-2 is disabled
> - All host system registers are restored
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index e6adb90c12ae..4df47d013bec 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -156,6 +170,23 @@ static void __hyp_text __deactivate_traps_nvhe(void)
> {
> u64 mdcr_el2 = read_sysreg(mdcr_el2);
>
> + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
> + u64 val;
> +
> + /*
> + * Set the TCR and SCTLR registers in the exact opposite
> + * sequence as __activate_traps_nvhe (first prevent walks,
> + * then force the MMU on). A generous sprinkling of isb()
> + * ensure that things happen in this exact order.
> + */
> + val = read_sysreg_el1(SYS_TCR);
> + write_sysreg_el1(val | TCR_EPD1_MASK | TCR_EPD0_MASK, SYS_TCR);
> + isb();
> + val = read_sysreg_el1(SYS_SCTLR);
> + write_sysreg_el1(val | SCTLR_ELx_M, SYS_SCTLR);
> + isb();
> + }
We are exiting the guest, and heading back to the host.
This change forces stage-1 off. Stage-2 is still enabled, but its about to be disabled and
have the host VMID restore in __deactivate_vm(). All good so far.
Then we hit __sysreg_restore_state_nvhe() for the host, which calls
__sysreg_restore_el1_state()...
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index 7ddbc849b580..adabdceacc10 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -117,12 +117,22 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> {
> write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
> write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
> - write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR);
> +
> + /* Must only be done for guest registers, hence the context test */
> + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367) &&
> + !ctxt->__hyp_running_vcpu) {
> + write_sysreg_el1(ctxt->sys_regs[TCR_EL1] |
> + TCR_EPD1_MASK | TCR_EPD0_MASK, SYS_TCR);
> + isb();
> + } else {
... which will come in here.
> + write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR);
> + write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR);
This reverses what we did in __deactivate_traps_nvhe(), but we haven't restored the host
TTBRs yet. I don't think the vttbr_el2 write has been sync'd either.
A speculative AT at this point could see the TCR EPDx bits clear, but the guest's TTBR
values. It may also see the guest-VMID.
I think the change to this function needs splitting up. Restore of guest state needs to be
as you have it here, before the guest TTBRs are written.
Restore of the host state needs to only clear the EPDx bits after the TTBRs are written,
and sync'd.
Assuming I'm making sense ... with that:
Reviewed-by: James Morse <james.morse@arm.com>
for the series.
> + }
> +
> write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> write_sysreg_el1(ctxt->sys_regs[CPACR_EL1], SYS_CPACR);
> write_sysreg_el1(ctxt->sys_regs[TTBR0_EL1], SYS_TTBR0);
> write_sysreg_el1(ctxt->sys_regs[TTBR1_EL1], SYS_TTBR1);
> - write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR);
> write_sysreg_el1(ctxt->sys_regs[ESR_EL1], SYS_ESR);
> write_sysreg_el1(ctxt->sys_regs[AFSR0_EL1], SYS_AFSR0);
> write_sysreg_el1(ctxt->sys_regs[AFSR1_EL1], SYS_AFSR1);
Thanks,
James
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs
2019-09-25 11:19 ` [PATCH 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs Marc Zyngier
@ 2019-10-03 11:10 ` James Morse
0 siblings, 0 replies; 13+ messages in thread
From: James Morse @ 2019-10-03 11:10 UTC (permalink / raw)
To: Marc Zyngier
Cc: Mark Rutland, kvm, Suzuki K Poulose, Catalin Marinas,
Julien Thierry, Will Deacon, kvmarm, linux-arm-kernel
Hi Marc,
On 25/09/2019 12:19, Marc Zyngier wrote:
> When erratum 1319367 is being worked around, special care must
> be taken not to allow the page table walker to populate TLBs
> while we have the stage-2 translation enabled (which would otherwise
> result in a bizare mix of the host S1 and the guest S2).
>
> We enforce this by setting TCR_EL1.EPD{0,1} before restoring the S2
> configuration, and clear the same bits after having disabled S2.
Some comment Nits...
> diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
> index eb0efc5557f3..4ef0bf0d76a6 100644
> --- a/arch/arm64/kvm/hyp/tlb.c
> +++ b/arch/arm64/kvm/hyp/tlb.c
> @@ -63,6 +63,22 @@ static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm,
> static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm,
> struct tlb_inv_context *cxt)
> {
> + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
> + u64 val;
> +
> + /*
> + * For CPUs that are affected by ARM 1319367, we need to
> + * avoid a host Stage-1 walk while we have the guest's
> + * Stage-2 set in the VTTBR in order to invalidate TLBs.
Isn't HCR_EL2.VM==0 for all this? I think its the VMID that matters here:
| ... have the guest's VMID set in VTTBR ...
?
> + * We're guaranteed that the S1 MMU is enabled, so we can
> + * simply set the EPD bits to avoid any further TLB fill.
> + */
> + val = cxt->tcr = read_sysreg_el1(SYS_TCR);
> + val |= TCR_EPD1_MASK | TCR_EPD0_MASK;
> + write_sysreg_el1(val, SYS_TCR);
> + isb();
> + }
> +
> __load_guest_stage2(kvm);
> isb();
> }
> @@ -100,6 +116,13 @@ static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm,
> struct tlb_inv_context *cxt)
> {
> write_sysreg(0, vttbr_el2);
> +
> + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
> + /* Ensure stage-2 is actually disabled */
| Ensure the host's VMID has been written
?
> + isb();
> + /* Restore the host's TCR_EL1 */
> + write_sysreg_el1(cxt->tcr, SYS_TCR);
> + }
> }
Thanks,
James
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537
2019-09-25 11:19 ` [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537 Marc Zyngier
2019-09-27 9:03 ` Catalin Marinas
@ 2019-10-03 11:11 ` James Morse
2019-10-07 9:31 ` Marc Zyngier
1 sibling, 1 reply; 13+ messages in thread
From: James Morse @ 2019-10-03 11:11 UTC (permalink / raw)
To: Marc Zyngier
Cc: Mark Rutland, kvm, Suzuki K Poulose, Catalin Marinas,
Julien Thierry, Will Deacon, kvmarm, linux-arm-kernel
Hi Marc,
On 25/09/2019 12:19, Marc Zyngier wrote:
> Now that everything is in place, let's get the ball rolling
> by allowing the corresponding config option to be selected.
> Also add the required information to silicon_arrata.rst.
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 3adcec05b1f6..c50cd4f83bc4 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -523,6 +523,16 @@ config ARM64_ERRATUM_1286807
> invalidated has been observed by other observers. The
> workaround repeats the TLBI+DSB operation.
>
> +config ARM64_ERRATUM_1319367
> + bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
> + default y
> + help
> + This option adds work arounds for ARM Cortex-A57 erratum 1319537
> + and A72 erratum 1319367
> +
> + Cortex-A57 and A72 cores could end-up with corrupted TLBs by
> + speculating an AT instruction during a guest context switch.
> +
> If unsure, say Y.
>
> config ARM64_ERRATUM_1463225
>
Nit: You pinched someone elses "If unsure, say Y."!
Thanks,
James
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537
2019-10-03 11:11 ` James Morse
@ 2019-10-07 9:31 ` Marc Zyngier
0 siblings, 0 replies; 13+ messages in thread
From: Marc Zyngier @ 2019-10-07 9:31 UTC (permalink / raw)
To: James Morse
Cc: Mark Rutland, kvm, Suzuki K Poulose, Catalin Marinas,
Julien Thierry, Will Deacon, kvmarm, linux-arm-kernel
On Thu, 03 Oct 2019 12:11:00 +0100,
James Morse <james.morse@arm.com> wrote:
>
> Hi Marc,
>
> On 25/09/2019 12:19, Marc Zyngier wrote:
> > Now that everything is in place, let's get the ball rolling
> > by allowing the corresponding config option to be selected.
> > Also add the required information to silicon_arrata.rst.
>
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 3adcec05b1f6..c50cd4f83bc4 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -523,6 +523,16 @@ config ARM64_ERRATUM_1286807
> > invalidated has been observed by other observers. The
> > workaround repeats the TLBI+DSB operation.
> >
> > +config ARM64_ERRATUM_1319367
> > + bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
> > + default y
> > + help
> > + This option adds work arounds for ARM Cortex-A57 erratum 1319537
> > + and A72 erratum 1319367
> > +
> > + Cortex-A57 and A72 cores could end-up with corrupted TLBs by
> > + speculating an AT instruction during a guest context switch.
> > +
> > If unsure, say Y.
> >
> > config ARM64_ERRATUM_1463225
> >
>
> Nit: You pinched someone elses "If unsure, say Y."!
Ah! Well spotted.
Thanks,
M.
--
Jazz is not dead, it just smells funny.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2019-10-07 9:32 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-25 11:19 [PATCH 0/5] arm64: KVM: Add workaround for errata 1319367 and 1319537 Marc Zyngier
2019-09-25 11:19 ` [PATCH 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Marc Zyngier
2019-09-27 8:33 ` Suzuki K Poulose
2019-09-27 9:03 ` Catalin Marinas
2019-09-25 11:19 ` [PATCH 2/5] arm64: KVM: Reorder system register restoration and stage-2 activation Marc Zyngier
2019-09-25 11:19 ` [PATCH 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs Marc Zyngier
2019-10-03 11:10 ` James Morse
2019-09-25 11:19 ` [PATCH 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context Marc Zyngier
2019-10-03 11:09 ` James Morse
2019-09-25 11:19 ` [PATCH 5/5] arm64: Enable and document ARM errata 1319367 and 1319537 Marc Zyngier
2019-09-27 9:03 ` Catalin Marinas
2019-10-03 11:11 ` James Morse
2019-10-07 9:31 ` Marc Zyngier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).