Re: [PATCH 2/2] KVM: arm64: nVHE: Don't consume host SErrors with RAS

[Andrew Scull <ascull@google.com>]

On Wed, Aug 05, 2020 at 03:34:11PM +0100, James Morse wrote:
> Hi Andrew,
> 
> On 30/07/2020 23:31, Andrew Scull wrote:
> > On Thu, Jul 30, 2020 at 04:18:23PM +0100, Andrew Scull wrote:
> >> The ESB at the start of the vectors causes any SErrors to be consumed to
> >> DISR_EL1. If the exception came from the host and the ESB caught an
> >> SError, it would not be noticed until a guest exits and DISR_EL1 is
> >> checked. Further, the SError would be attributed to the guest and not
> >> the host.
> >>
> >> To avoid these problems, use a different exception vector for the host
> >> that does not use an ESB but instead leaves any host SError pending. A
> >> guest will not be entered if an SError is pending so it will always be
> >> the host that will receive and handle it.
> 
> > Thinking further, I'm not sure this actually solves all of the problem.
> > It does prevent hyp from causing a host SError to be consumed but, IIUC,
> > there could be an SError already deferred by the host and logged in
> > DISR_EL1 that hyp would not preserve if a guest is run.
> 
> I think that would be a host bug.
> 
> The ESB-instruction is the only thing that writes to DISR_EL1, and only if PSTATE.A is
> set. The host should:
> * Read DISR_EL1 after running the ESB-instruction,
> * Not call into HYP from SError masked code!

Good to know that this is the intent and not just what appears to happen
today.

> (VHE only does it to match the nVHE behaviour, as KVM isn't prepared to handle these).
> 
> 
> 'ESB-instruction' is pedantry to avoid the risk of it being confused with IESB, which is
> just the barrier bit, not the writes-to-DISR bit.
> 
> 
> > I think the host's DISR_EL1 would need to be saved and restored in the
> > vcpu context switch which, from a cursory read of the ARM, is possible
> > without having to virtualize SErrors for the host.
> 
> ... I thought this was a redirected register. Reads from EL1 when HCR_EL2.AMO is set get
> the value from VDISR_EL2, meaning the guest can't read DISR_EL1 at all.
> (see 'Accessing DISR_EL1' in the register description, "D13.7.1
> DISR_EL1, Deferred Interrupt Status Register" of DDI0487F.a

The host doesn't run with HCR_EL2.AMO set so it uses DISR_EL1 directly,
but hyp also uses DISR_EL1 directly during __guest_exit. That is the
clobbering I was concerned about. It may not be a problem most of the
time given what you said above, but I think something like the diff
below should be enough to be sure it is preserved:

 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
index 28f349288f3a..a34210c1c877 100644
--- a/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
+++ b/arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h
@@ -56,8 +56,12 @@ static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
 	ctxt->regs.pc			= read_sysreg_el2(SYS_ELR);
 	ctxt->regs.pstate		= read_sysreg_el2(SYS_SPSR);
 
-	if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
-		ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
+	if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN)) {
+		if (kvm_is_host_cpu_context(ctxt))
+			ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_DISR_EL1);
+		else
+			ctxt_sys_reg(ctxt, DISR_EL1) = read_sysreg_s(SYS_VDISR_EL2);
+	}
 }
 
 static inline void __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
@@ -152,8 +156,12 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctx
 	write_sysreg_el2(ctxt->regs.pc,			SYS_ELR);
 	write_sysreg_el2(pstate,			SYS_SPSR);
 
-	if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN))
-		write_sysreg_s(ctxt_sys_reg(ctxt, DISR_EL1), SYS_VDISR_EL2);
+	if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN)) {
+		if (kvm_is_host_cpu_context(ctxt))
+			write_sysreg_s(ctxt_sys_reg(ctxt, DISR_EL1), SYS_DISR_EL1);
+		else
+			write_sysreg_s(ctxt_sys_reg(ctxt, DISR_EL1), SYS_VDISR_EL2);
+	}
 }
 
 static inline void __sysreg32_save_state(struct kvm_vcpu *vcpu)


> >> Hyp initialization is now passed the vector that is used for the host
> >> and the vector for guests is stored in a percpu variable as
> >> kvm_get_hyp_vector() is not suitable for calling from nVHE hyp.
> 
> 
> Thanks,
> 
> James
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm