* [PATCH 00/37] Optimize KVM/ARM for VHE systems
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
This series redesigns parts of KVM/ARM to optimize the performance on
VHE systems. The general approach is to try to do as little work as
possible when transitioning betwen the VM and the hypervisor. This has
the benefit of lower latency when waiting for interrupts and delivering
virtual interrupts, and reduces the overhead of emulating behavior and
I/O in the host kernel.
Patches 01 through 04 are not VHE specific, but rework parts of KVM/ARM
that can be generally improved. We then add infrastructure to move more
logic into vcpu_load and vcpu_put, we improve handling of VFP and debug
registers.
We then introduce a new world-switch function for VHE systems, which we
can tweak and optimize for VHE systems. To do that, we rework a lot of
the sytem register save/restore handling and emulation code that may
need access to system registers, so that we can defer as many system
register save/restore operations to vcpu_load and vcpu_put, and move
this logic out of the VHE world switch function.
We then optimize the configuration of traps. On non-VHE systems, both
the host and VM kernels run in EL1, but because the host kernel should
have full access to the underlying hardware, but the VM kernel should
not, we essentially make the host kernel more privieleged than the VM
kernel despite them both running at the same privilege level by enabling
VE traps when entering the VM and disabling those traps when exiting the
VM. On VHE systems, the host kernel runs in EL2 and has full access to
the hardware (as much as allowed by secure side software), and is
unaffected by the trap configuration. That means we can configure the
traps for VMs running in EL1 once, and don't have to switch them on and
off for every entry/exit to/from the VM.
Finally, we improve our VGIC handling by moving all save/restore logic
out of the VHE world-switch, and we make it possible to truly only
evaluate if the AP list is empty and not do *any* VGIC work if that is
the case, and only do the minimal amount of work required in the course
of the VGIC processing when we have virtual interrupts in flight.
The patches are based on v4.14-rc2 plus the timer optimization series
[1] and the patches that use the TPIDR_EL2 for per-cpu offsets in the
host kernel and stops storing the VCPU pointer in TPIDR_EL2 from the
SDEI series [2].
I've given the patches a fair amount of testing on Thunder-X, Mustang,
Seattle, and TC2 (32-bit) for non-VHE testing, and tested VHE
functionality on the Foundation model, running both 64-bit VMs and
32-bit VMs side-by-side and using both GICv3-on-GICv3 and
GICv2-on-GICv3.
The paches are also available in the vhe-optimize branch on my
kernel.org repository [3].
Thanks,
-Christoffer
[1]: https://lists.cs.columbia.edu/pipermail/kvmarm/2017-September/027245.html
[2]: https://lists.cs.columbia.edu/pipermail/kvmarm/2017-September/027230.html
[3]: git://git.kernel.org/pub/scm/linux/kernel/git/cdall/linux.git vhe-optimize
Christoffer Dall (36):
KVM: arm64: Avoid storing the vcpu pointer on the stack
KVM: arm64: Rework hyp_panic for VHE and non-VHE
KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
KVM: Record the executing ioctl number on the vcpu struct
KVM: arm/arm64: Only load/put VCPU state for KVM_RUN
KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
KVM: arm64: Defer restoring host VFP state to vcpu_put
KVM: arm64: Move debug dirty flag calculation out of world switch
KVM: arm64: Slightly improve debug save/restore functions
KVM: arm64: Improve debug register save/restore flow
KVM: arm64: Factor out fault info population and gic workarounds
KVM: arm64: Introduce VHE-specific kvm_vcpu_run
KVM: arm64: Remove kern_hyp_va() use in VHE switch function
KVM: arm64: Don't deactivate VM on VHE systems
KVM: arm64: Remove noop calls to timer save/restore from VHE switch
KVM: arm64: Move userspace system registers into separate function
KVM: arm64: Rewrite sysreg alternatives to static keys
KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore
functions
KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
KVM: arm64: Change 32-bit handling of VM system registers
KVM: arm64: Prepare to handle traps on deferred VM sysregs
KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
KVM: arm64: Defer saving/restoring system registers to vcpu load/put
on VHE
KVM: arm64: Move common VHE/non-VHE trap config in separate functions
KVM: arm64: Configure FPSIMD traps on vcpu load/put for VHE
KVM: arm64: Configure c15, PMU, and debug register traps on cpu
load/put for VHE
KVM: arm64: Separate activate_traps and deactive_traps for VHE and
non-VHE
KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64
KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on
VHE
KVM: arm/arm64: Get rid of vgic_elrsr
KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
Shih-Wei Li (1):
KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag
arch/arm/include/asm/kvm_asm.h | 4 +
arch/arm/include/asm/kvm_emulate.h | 9 +-
arch/arm/include/asm/kvm_host.h | 6 +-
arch/arm/include/asm/kvm_hyp.h | 4 +
arch/arm/kvm/emulate.c | 2 +-
arch/arm/kvm/hyp/Makefile | 1 -
arch/arm/kvm/hyp/switch.c | 14 +-
arch/arm64/include/asm/kvm_arm.h | 4 +-
arch/arm64/include/asm/kvm_asm.h | 22 ++
arch/arm64/include/asm/kvm_emulate.h | 22 +-
arch/arm64/include/asm/kvm_host.h | 21 +-
arch/arm64/include/asm/kvm_hyp.h | 29 +--
arch/arm64/kernel/asm-offsets.c | 2 +
arch/arm64/kvm/debug.c | 9 +
arch/arm64/kvm/hyp/Makefile | 2 +-
arch/arm64/kvm/hyp/debug-sr.c | 84 ++++----
arch/arm64/kvm/hyp/entry.S | 8 +-
arch/arm64/kvm/hyp/hyp-entry.S | 39 ++--
arch/arm64/kvm/hyp/switch.c | 392 ++++++++++++++++++++---------------
arch/arm64/kvm/hyp/sysreg-sr.c | 160 +++++++++++---
arch/arm64/kvm/hyp/vgic-v2-sr.c | 77 +++++++
arch/arm64/kvm/inject_fault.c | 93 ++++++++-
arch/arm64/kvm/sys_regs.c | 76 ++++++-
arch/arm64/kvm/sys_regs_generic_v8.c | 5 +-
arch/x86/kvm/vmx.c | 2 +-
arch/x86/kvm/x86.c | 8 +-
include/kvm/arm_vgic.h | 2 -
include/linux/kvm_host.h | 3 +-
virt/kvm/arm/arm.c | 24 ++-
virt/kvm/arm/hyp/timer-sr.c | 10 +-
virt/kvm/arm/hyp/vgic-v2-sr.c | 162 ---------------
virt/kvm/arm/hyp/vgic-v3-sr.c | 239 ++++++++++++---------
virt/kvm/arm/mmu.c | 6 +-
virt/kvm/arm/vgic/vgic-init.c | 22 +-
virt/kvm/arm/vgic/vgic-v2.c | 72 ++++++-
virt/kvm/arm/vgic/vgic-v3.c | 12 +-
virt/kvm/arm/vgic/vgic.c | 22 ++
virt/kvm/arm/vgic/vgic.h | 5 +
virt/kvm/kvm_main.c | 6 +-
39 files changed, 1046 insertions(+), 634 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c
delete mode 100644 virt/kvm/arm/hyp/vgic-v2-sr.c
--
2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: kvm, Marc Zyngier, Shih-Wei Li, Christoffer Dall
We already have the percpu area for the host cpu state, which points to
the VCPU, so there's no need to store the VCPU pointer on the stack on
every context switch. We can be a little more clever and just use
tpidr_el2 for the percpu offset and load the VCPU pointer from the host
context.
This requires us to have a scratch register though, so we take the
chance to rearrange some of the el1_sync code to only look at the
vttbr_el2 to determine if this is a trap from the guest or an HVC from
the host. We do add an extra check to call the panic code if the kernel
is configured with debugging enabled and we saw a trap from the host
which wasn't an HVC, indicating that we left some EL2 trap configured by
mistake.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kvm/hyp/entry.S | 5 +----
arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
arch/arm64/kvm/hyp/switch.c | 2 +-
5 files changed, 41 insertions(+), 26 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index ab4d0a9..7e48a39 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
#endif
+#ifdef __ASSEMBLY__
+.macro get_host_ctxt reg, tmp
+ /*
+ * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
+ * not be accessible by this address from EL2, hyp_panic() converts
+ * it with kern_hyp_va() before use.
+ */
+ ldr \reg, =kvm_host_cpu_state
+ mrs \tmp, tpidr_el2
+ add \reg, \reg, \tmp
+ kern_hyp_va \reg
+.endm
+
+.macro get_vcpu vcpu, ctxt
+ ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
+ kern_hyp_va \vcpu
+.endm
+
+#endif
+
#endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 71bf088..612021d 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -135,6 +135,7 @@ int main(void)
DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
+ DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
#endif
#ifdef CONFIG_CPU_PM
DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 9a8ab5d..76cd48f 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -62,9 +62,6 @@ ENTRY(__guest_enter)
// Store the host regs
save_callee_saved_regs x1
- // Store host_ctxt and vcpu for use at exit time
- stp x1, x0, [sp, #-16]!
-
add x18, x0, #VCPU_CONTEXT
// Restore guest regs x0-x17
@@ -119,7 +116,7 @@ ENTRY(__guest_exit)
save_callee_saved_regs x1
// Restore the host_ctxt from the stack
- ldr x2, [sp], #16
+ get_host_ctxt x2, x3
// Now restore the host regs
restore_callee_saved_regs x2
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index e4f37b9..2950f26 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
el1_sync: // Guest trapped into EL2
stp x0, x1, [sp, #-16]!
-alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
- mrs x1, esr_el2
-alternative_else
- mrs x1, esr_el1
-alternative_endif
- lsr x0, x1, #ESR_ELx_EC_SHIFT
-
- cmp x0, #ESR_ELx_EC_HVC64
- b.ne el1_trap
-
mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
cbnz x1, el1_trap // called HVC
+#ifdef CONFIG_DEBUG
+ mrs x0, esr_el2
+ lsr x0, x0, #ESR_ELx_EC_SHIFT
+ cmp x0, #ESR_ELx_EC_HVC64
+ b.ne __hyp_panic
+#endif
+
/* Here, we're pretty sure the host called HVC. */
ldp x0, x1, [sp], #16
@@ -101,10 +98,15 @@ alternative_endif
eret
el1_trap:
+ get_host_ctxt x0, x1
+ get_vcpu x1, x0
+
+ mrs x0, esr_el2
+ lsr x0, x0, #ESR_ELx_EC_SHIFT
/*
* x0: ESR_EC
+ * x1: vcpu pointer
*/
- ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
/*
* We trap the first access to the FP/SIMD to save the host context
@@ -122,13 +124,15 @@ alternative_else_nop_endif
el1_irq:
stp x0, x1, [sp, #-16]!
- ldr x1, [sp, #16 + 8]
+ get_host_ctxt x0, x1
+ get_vcpu x1, x0
mov x0, #ARM_EXCEPTION_IRQ
b __guest_exit
el1_error:
stp x0, x1, [sp, #-16]!
- ldr x1, [sp, #16 + 8]
+ get_host_ctxt x0, x1
+ get_vcpu x1, x0
mov x0, #ARM_EXCEPTION_EL1_SERROR
b __guest_exit
@@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
ENDPROC(__hyp_do_panic)
ENTRY(__hyp_panic)
- /*
- * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
- * not be accessible by this address from EL2, hyp_panic() converts
- * it with kern_hyp_va() before use.
- */
- ldr x0, =kvm_host_cpu_state
- mrs x1, tpidr_el2
- add x0, x0, x1
+ get_host_ctxt x0, x1
b hyp_panic
ENDPROC(__hyp_panic)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 69ef24a..a0123ad 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
if (read_sysreg(vttbr_el2)) {
struct kvm_cpu_context *host_ctxt;
- host_ctxt = kern_hyp_va(__host_ctxt);
+ host_ctxt = __host_ctxt;
vcpu = host_ctxt->__hyp_running_vcpu;
__timer_disable_traps(vcpu);
__deactivate_traps(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We already have the percpu area for the host cpu state, which points to
the VCPU, so there's no need to store the VCPU pointer on the stack on
every context switch. We can be a little more clever and just use
tpidr_el2 for the percpu offset and load the VCPU pointer from the host
context.
This requires us to have a scratch register though, so we take the
chance to rearrange some of the el1_sync code to only look at the
vttbr_el2 to determine if this is a trap from the guest or an HVC from
the host. We do add an extra check to call the panic code if the kernel
is configured with debugging enabled and we saw a trap from the host
which wasn't an HVC, indicating that we left some EL2 trap configured by
mistake.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kvm/hyp/entry.S | 5 +----
arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
arch/arm64/kvm/hyp/switch.c | 2 +-
5 files changed, 41 insertions(+), 26 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index ab4d0a9..7e48a39 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
#endif
+#ifdef __ASSEMBLY__
+.macro get_host_ctxt reg, tmp
+ /*
+ * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
+ * not be accessible by this address from EL2, hyp_panic() converts
+ * it with kern_hyp_va() before use.
+ */
+ ldr \reg, =kvm_host_cpu_state
+ mrs \tmp, tpidr_el2
+ add \reg, \reg, \tmp
+ kern_hyp_va \reg
+.endm
+
+.macro get_vcpu vcpu, ctxt
+ ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
+ kern_hyp_va \vcpu
+.endm
+
+#endif
+
#endif /* __ARM_KVM_ASM_H__ */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 71bf088..612021d 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -135,6 +135,7 @@ int main(void)
DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
+ DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
#endif
#ifdef CONFIG_CPU_PM
DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 9a8ab5d..76cd48f 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -62,9 +62,6 @@ ENTRY(__guest_enter)
// Store the host regs
save_callee_saved_regs x1
- // Store host_ctxt and vcpu for use at exit time
- stp x1, x0, [sp, #-16]!
-
add x18, x0, #VCPU_CONTEXT
// Restore guest regs x0-x17
@@ -119,7 +116,7 @@ ENTRY(__guest_exit)
save_callee_saved_regs x1
// Restore the host_ctxt from the stack
- ldr x2, [sp], #16
+ get_host_ctxt x2, x3
// Now restore the host regs
restore_callee_saved_regs x2
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index e4f37b9..2950f26 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
el1_sync: // Guest trapped into EL2
stp x0, x1, [sp, #-16]!
-alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
- mrs x1, esr_el2
-alternative_else
- mrs x1, esr_el1
-alternative_endif
- lsr x0, x1, #ESR_ELx_EC_SHIFT
-
- cmp x0, #ESR_ELx_EC_HVC64
- b.ne el1_trap
-
mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
cbnz x1, el1_trap // called HVC
+#ifdef CONFIG_DEBUG
+ mrs x0, esr_el2
+ lsr x0, x0, #ESR_ELx_EC_SHIFT
+ cmp x0, #ESR_ELx_EC_HVC64
+ b.ne __hyp_panic
+#endif
+
/* Here, we're pretty sure the host called HVC. */
ldp x0, x1, [sp], #16
@@ -101,10 +98,15 @@ alternative_endif
eret
el1_trap:
+ get_host_ctxt x0, x1
+ get_vcpu x1, x0
+
+ mrs x0, esr_el2
+ lsr x0, x0, #ESR_ELx_EC_SHIFT
/*
* x0: ESR_EC
+ * x1: vcpu pointer
*/
- ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
/*
* We trap the first access to the FP/SIMD to save the host context
@@ -122,13 +124,15 @@ alternative_else_nop_endif
el1_irq:
stp x0, x1, [sp, #-16]!
- ldr x1, [sp, #16 + 8]
+ get_host_ctxt x0, x1
+ get_vcpu x1, x0
mov x0, #ARM_EXCEPTION_IRQ
b __guest_exit
el1_error:
stp x0, x1, [sp, #-16]!
- ldr x1, [sp, #16 + 8]
+ get_host_ctxt x0, x1
+ get_vcpu x1, x0
mov x0, #ARM_EXCEPTION_EL1_SERROR
b __guest_exit
@@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
ENDPROC(__hyp_do_panic)
ENTRY(__hyp_panic)
- /*
- * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
- * not be accessible by this address from EL2, hyp_panic() converts
- * it with kern_hyp_va() before use.
- */
- ldr x0, =kvm_host_cpu_state
- mrs x1, tpidr_el2
- add x0, x0, x1
+ get_host_ctxt x0, x1
b hyp_panic
ENDPROC(__hyp_panic)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 69ef24a..a0123ad 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
if (read_sysreg(vttbr_el2)) {
struct kvm_cpu_context *host_ctxt;
- host_ctxt = kern_hyp_va(__host_ctxt);
+ host_ctxt = __host_ctxt;
vcpu = host_ctxt->__hyp_running_vcpu;
__timer_disable_traps(vcpu);
__deactivate_traps(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 15:49 ` Marc Zyngier
-1 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 15:49 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: kvm, Shih-Wei Li
On 12/10/17 11:41, Christoffer Dall wrote:
> We already have the percpu area for the host cpu state, which points to
> the VCPU, so there's no need to store the VCPU pointer on the stack on
> every context switch. We can be a little more clever and just use
> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> context.
>
> This requires us to have a scratch register though, so we take the
> chance to rearrange some of the el1_sync code to only look at the
> vttbr_el2 to determine if this is a trap from the guest or an HVC from
> the host. We do add an extra check to call the panic code if the kernel
> is configured with debugging enabled and we saw a trap from the host
> which wasn't an HVC, indicating that we left some EL2 trap configured by
> mistake.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 5 +----
> arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> arch/arm64/kvm/hyp/switch.c | 2 +-
> 5 files changed, 41 insertions(+), 26 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index ab4d0a9..7e48a39 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>
> #endif
>
> +#ifdef __ASSEMBLY__
> +.macro get_host_ctxt reg, tmp
> + /*
> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> + * not be accessible by this address from EL2, hyp_panic() converts
> + * it with kern_hyp_va() before use.
> + */
This really looks like a stale comment, as there is no hyp_panic
involved here anymore (thankfully!).
> + ldr \reg, =kvm_host_cpu_state
> + mrs \tmp, tpidr_el2
> + add \reg, \reg, \tmp
> + kern_hyp_va \reg
Here, we're trading a load from the stack for a load from the constant
pool. Can't we do something like:
adr_l \reg, kvm_host_cpu_state
msr \tmp, tpidr_el2
add \reg, \reg, \tmp
and that's it? This relies on the property that the kernel/hyp offset is
constant, and that it doesn't matter if we add the offset to a kernel VA
or a HYP VA... Completely untested of course!
> +.endm
> +
> +.macro get_vcpu vcpu, ctxt
> + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> + kern_hyp_va \vcpu
> +.endm
> +
> +#endif
> +
> #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 71bf088..612021d 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -135,6 +135,7 @@ int main(void)
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> #endif
> #ifdef CONFIG_CPU_PM
> DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 9a8ab5d..76cd48f 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> // Store the host regs
> save_callee_saved_regs x1
>
> - // Store host_ctxt and vcpu for use at exit time
> - stp x1, x0, [sp, #-16]!
> -
> add x18, x0, #VCPU_CONTEXT
>
> // Restore guest regs x0-x17
> @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> save_callee_saved_regs x1
>
> // Restore the host_ctxt from the stack
Stale comment again.
> - ldr x2, [sp], #16
> + get_host_ctxt x2, x3
>
> // Now restore the host regs
> restore_callee_saved_regs x2
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index e4f37b9..2950f26 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> el1_sync: // Guest trapped into EL2
> stp x0, x1, [sp, #-16]!
>
> -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> - mrs x1, esr_el2
> -alternative_else
> - mrs x1, esr_el1
> -alternative_endif
> - lsr x0, x1, #ESR_ELx_EC_SHIFT
> -
> - cmp x0, #ESR_ELx_EC_HVC64
> - b.ne el1_trap
> -
> mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> cbnz x1, el1_trap // called HVC
Comment is outdated. Any guest trap will take this path.
>
> +#ifdef CONFIG_DEBUG
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> + cmp x0, #ESR_ELx_EC_HVC64
> + b.ne __hyp_panic
> +#endif
> +
> /* Here, we're pretty sure the host called HVC. */
> ldp x0, x1, [sp], #16
>
> @@ -101,10 +98,15 @@ alternative_endif
> eret
>
> el1_trap:
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> +
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> /*
> * x0: ESR_EC
> + * x1: vcpu pointer
> */
> - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
>
> /*
> * We trap the first access to the FP/SIMD to save the host context
> @@ -122,13 +124,15 @@ alternative_else_nop_endif
>
> el1_irq:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_IRQ
> b __guest_exit
>
> el1_error:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_EL1_SERROR
> b __guest_exit
>
> @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> ENDPROC(__hyp_do_panic)
>
> ENTRY(__hyp_panic)
> - /*
> - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> - * not be accessible by this address from EL2, hyp_panic() converts
> - * it with kern_hyp_va() before use.
> - */
> - ldr x0, =kvm_host_cpu_state
> - mrs x1, tpidr_el2
> - add x0, x0, x1
> + get_host_ctxt x0, x1
> b hyp_panic
> ENDPROC(__hyp_panic)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 69ef24a..a0123ad 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> if (read_sysreg(vttbr_el2)) {
> struct kvm_cpu_context *host_ctxt;
>
> - host_ctxt = kern_hyp_va(__host_ctxt);
> + host_ctxt = __host_ctxt;
Can't we just rename __host_ctxt to host_ctxt and drop the local definition?
> vcpu = host_ctxt->__hyp_running_vcpu;
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
>
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-10-12 15:49 ` Marc Zyngier
0 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 15:49 UTC (permalink / raw)
To: linux-arm-kernel
On 12/10/17 11:41, Christoffer Dall wrote:
> We already have the percpu area for the host cpu state, which points to
> the VCPU, so there's no need to store the VCPU pointer on the stack on
> every context switch. We can be a little more clever and just use
> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> context.
>
> This requires us to have a scratch register though, so we take the
> chance to rearrange some of the el1_sync code to only look at the
> vttbr_el2 to determine if this is a trap from the guest or an HVC from
> the host. We do add an extra check to call the panic code if the kernel
> is configured with debugging enabled and we saw a trap from the host
> which wasn't an HVC, indicating that we left some EL2 trap configured by
> mistake.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 5 +----
> arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> arch/arm64/kvm/hyp/switch.c | 2 +-
> 5 files changed, 41 insertions(+), 26 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index ab4d0a9..7e48a39 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>
> #endif
>
> +#ifdef __ASSEMBLY__
> +.macro get_host_ctxt reg, tmp
> + /*
> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> + * not be accessible by this address from EL2, hyp_panic() converts
> + * it with kern_hyp_va() before use.
> + */
This really looks like a stale comment, as there is no hyp_panic
involved here anymore (thankfully!).
> + ldr \reg, =kvm_host_cpu_state
> + mrs \tmp, tpidr_el2
> + add \reg, \reg, \tmp
> + kern_hyp_va \reg
Here, we're trading a load from the stack for a load from the constant
pool. Can't we do something like:
adr_l \reg, kvm_host_cpu_state
msr \tmp, tpidr_el2
add \reg, \reg, \tmp
and that's it? This relies on the property that the kernel/hyp offset is
constant, and that it doesn't matter if we add the offset to a kernel VA
or a HYP VA... Completely untested of course!
> +.endm
> +
> +.macro get_vcpu vcpu, ctxt
> + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> + kern_hyp_va \vcpu
> +.endm
> +
> +#endif
> +
> #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 71bf088..612021d 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -135,6 +135,7 @@ int main(void)
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> #endif
> #ifdef CONFIG_CPU_PM
> DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 9a8ab5d..76cd48f 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> // Store the host regs
> save_callee_saved_regs x1
>
> - // Store host_ctxt and vcpu for use at exit time
> - stp x1, x0, [sp, #-16]!
> -
> add x18, x0, #VCPU_CONTEXT
>
> // Restore guest regs x0-x17
> @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> save_callee_saved_regs x1
>
> // Restore the host_ctxt from the stack
Stale comment again.
> - ldr x2, [sp], #16
> + get_host_ctxt x2, x3
>
> // Now restore the host regs
> restore_callee_saved_regs x2
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index e4f37b9..2950f26 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> el1_sync: // Guest trapped into EL2
> stp x0, x1, [sp, #-16]!
>
> -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> - mrs x1, esr_el2
> -alternative_else
> - mrs x1, esr_el1
> -alternative_endif
> - lsr x0, x1, #ESR_ELx_EC_SHIFT
> -
> - cmp x0, #ESR_ELx_EC_HVC64
> - b.ne el1_trap
> -
> mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> cbnz x1, el1_trap // called HVC
Comment is outdated. Any guest trap will take this path.
>
> +#ifdef CONFIG_DEBUG
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> + cmp x0, #ESR_ELx_EC_HVC64
> + b.ne __hyp_panic
> +#endif
> +
> /* Here, we're pretty sure the host called HVC. */
> ldp x0, x1, [sp], #16
>
> @@ -101,10 +98,15 @@ alternative_endif
> eret
>
> el1_trap:
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> +
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> /*
> * x0: ESR_EC
> + * x1: vcpu pointer
> */
> - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
>
> /*
> * We trap the first access to the FP/SIMD to save the host context
> @@ -122,13 +124,15 @@ alternative_else_nop_endif
>
> el1_irq:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_IRQ
> b __guest_exit
>
> el1_error:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_EL1_SERROR
> b __guest_exit
>
> @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> ENDPROC(__hyp_do_panic)
>
> ENTRY(__hyp_panic)
> - /*
> - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> - * not be accessible by this address from EL2, hyp_panic() converts
> - * it with kern_hyp_va() before use.
> - */
> - ldr x0, =kvm_host_cpu_state
> - mrs x1, tpidr_el2
> - add x0, x0, x1
> + get_host_ctxt x0, x1
> b hyp_panic
> ENDPROC(__hyp_panic)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 69ef24a..a0123ad 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> if (read_sysreg(vttbr_el2)) {
> struct kvm_cpu_context *host_ctxt;
>
> - host_ctxt = kern_hyp_va(__host_ctxt);
> + host_ctxt = __host_ctxt;
Can't we just rename __host_ctxt to host_ctxt and drop the local definition?
> vcpu = host_ctxt->__hyp_running_vcpu;
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
>
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-10-12 15:49 ` Marc Zyngier
@ 2017-10-12 17:02 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 17:02 UTC (permalink / raw)
To: Marc Zyngier; +Cc: linux-arm-kernel, Shih-Wei Li, kvmarm, kvm
On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> > We already have the percpu area for the host cpu state, which points to
> > the VCPU, so there's no need to store the VCPU pointer on the stack on
> > every context switch. We can be a little more clever and just use
> > tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> > context.
> >
> > This requires us to have a scratch register though, so we take the
> > chance to rearrange some of the el1_sync code to only look at the
> > vttbr_el2 to determine if this is a trap from the guest or an HVC from
> > the host. We do add an extra check to call the panic code if the kernel
> > is configured with debugging enabled and we saw a trap from the host
> > which wasn't an HVC, indicating that we left some EL2 trap configured by
> > mistake.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 5 +----
> > arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> > arch/arm64/kvm/hyp/switch.c | 2 +-
> > 5 files changed, 41 insertions(+), 26 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index ab4d0a9..7e48a39 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >
> > #endif
> >
> > +#ifdef __ASSEMBLY__
> > +.macro get_host_ctxt reg, tmp
> > + /*
> > + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > + * not be accessible by this address from EL2, hyp_panic() converts
> > + * it with kern_hyp_va() before use.
> > + */
>
> This really looks like a stale comment, as there is no hyp_panic
> involved here anymore (thankfully!).
>
yeah, I suppose.
> > + ldr \reg, =kvm_host_cpu_state
> > + mrs \tmp, tpidr_el2
> > + add \reg, \reg, \tmp
> > + kern_hyp_va \reg
>
> Here, we're trading a load from the stack for a load from the constant
> pool. Can't we do something like:
>
> adr_l \reg, kvm_host_cpu_state
> msr \tmp, tpidr_el2
> add \reg, \reg, \tmp
>
> and that's it?
That's definitely what the compiler generates from C code...
> This relies on the property that the kernel/hyp offset is
> constant, and that it doesn't matter if we add the offset to a kernel VA
> or a HYP VA... Completely untested of course!
>
You're the hyp VA expert. Is it valid to rely on that assumption?
> > +.endm
> > +
> > +.macro get_vcpu vcpu, ctxt
> > + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> > + kern_hyp_va \vcpu
> > +.endm
> > +
> > +#endif
> > +
> > #endif /* __ARM_KVM_ASM_H__ */
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 71bf088..612021d 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -135,6 +135,7 @@ int main(void)
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > #endif
> > #ifdef CONFIG_CPU_PM
> > DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index 9a8ab5d..76cd48f 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> > // Store the host regs
> > save_callee_saved_regs x1
> >
> > - // Store host_ctxt and vcpu for use at exit time
> > - stp x1, x0, [sp, #-16]!
> > -
> > add x18, x0, #VCPU_CONTEXT
> >
> > // Restore guest regs x0-x17
> > @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> > save_callee_saved_regs x1
> >
> > // Restore the host_ctxt from the stack
>
> Stale comment again.
>
yeah...
> > - ldr x2, [sp], #16
> > + get_host_ctxt x2, x3
> >
> > // Now restore the host regs
> > restore_callee_saved_regs x2
> > diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> > index e4f37b9..2950f26 100644
> > --- a/arch/arm64/kvm/hyp/hyp-entry.S
> > +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> > @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> > el1_sync: // Guest trapped into EL2
> > stp x0, x1, [sp, #-16]!
> >
> > -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> > - mrs x1, esr_el2
> > -alternative_else
> > - mrs x1, esr_el1
> > -alternative_endif
> > - lsr x0, x1, #ESR_ELx_EC_SHIFT
> > -
> > - cmp x0, #ESR_ELx_EC_HVC64
> > - b.ne el1_trap
> > -
> > mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> > cbnz x1, el1_trap // called HVC
>
> Comment is outdated. Any guest trap will take this path.
>
yeah...
> >
> > +#ifdef CONFIG_DEBUG
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > + cmp x0, #ESR_ELx_EC_HVC64
> > + b.ne __hyp_panic
> > +#endif
> > +
> > /* Here, we're pretty sure the host called HVC. */
> > ldp x0, x1, [sp], #16
> >
> > @@ -101,10 +98,15 @@ alternative_endif
> > eret
> >
> > el1_trap:
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > +
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > /*
> > * x0: ESR_EC
> > + * x1: vcpu pointer
> > */
> > - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
> >
> > /*
> > * We trap the first access to the FP/SIMD to save the host context
> > @@ -122,13 +124,15 @@ alternative_else_nop_endif
> >
> > el1_irq:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_IRQ
> > b __guest_exit
> >
> > el1_error:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_EL1_SERROR
> > b __guest_exit
> >
> > @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> > ENDPROC(__hyp_do_panic)
> >
> > ENTRY(__hyp_panic)
> > - /*
> > - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > - * not be accessible by this address from EL2, hyp_panic() converts
> > - * it with kern_hyp_va() before use.
> > - */
> > - ldr x0, =kvm_host_cpu_state
> > - mrs x1, tpidr_el2
> > - add x0, x0, x1
> > + get_host_ctxt x0, x1
> > b hyp_panic
> > ENDPROC(__hyp_panic)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 69ef24a..a0123ad 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> > if (read_sysreg(vttbr_el2)) {
> > struct kvm_cpu_context *host_ctxt;
> >
> > - host_ctxt = kern_hyp_va(__host_ctxt);
> > + host_ctxt = __host_ctxt;
>
> Can't we just rename __host_ctxt to host_ctxt and drop the local definition?
>
yes, patch splitting snafu. Will fix.
By the way, what I'm going for is anything in the hyp address space has
leading __, and otherwise ot.
> > vcpu = host_ctxt->__hyp_running_vcpu;
> > __timer_disable_traps(vcpu);
> > __deactivate_traps(vcpu);
> >
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-10-12 17:02 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 17:02 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> > We already have the percpu area for the host cpu state, which points to
> > the VCPU, so there's no need to store the VCPU pointer on the stack on
> > every context switch. We can be a little more clever and just use
> > tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> > context.
> >
> > This requires us to have a scratch register though, so we take the
> > chance to rearrange some of the el1_sync code to only look at the
> > vttbr_el2 to determine if this is a trap from the guest or an HVC from
> > the host. We do add an extra check to call the panic code if the kernel
> > is configured with debugging enabled and we saw a trap from the host
> > which wasn't an HVC, indicating that we left some EL2 trap configured by
> > mistake.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 5 +----
> > arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> > arch/arm64/kvm/hyp/switch.c | 2 +-
> > 5 files changed, 41 insertions(+), 26 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index ab4d0a9..7e48a39 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >
> > #endif
> >
> > +#ifdef __ASSEMBLY__
> > +.macro get_host_ctxt reg, tmp
> > + /*
> > + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > + * not be accessible by this address from EL2, hyp_panic() converts
> > + * it with kern_hyp_va() before use.
> > + */
>
> This really looks like a stale comment, as there is no hyp_panic
> involved here anymore (thankfully!).
>
yeah, I suppose.
> > + ldr \reg, =kvm_host_cpu_state
> > + mrs \tmp, tpidr_el2
> > + add \reg, \reg, \tmp
> > + kern_hyp_va \reg
>
> Here, we're trading a load from the stack for a load from the constant
> pool. Can't we do something like:
>
> adr_l \reg, kvm_host_cpu_state
> msr \tmp, tpidr_el2
> add \reg, \reg, \tmp
>
> and that's it?
That's definitely what the compiler generates from C code...
> This relies on the property that the kernel/hyp offset is
> constant, and that it doesn't matter if we add the offset to a kernel VA
> or a HYP VA... Completely untested of course!
>
You're the hyp VA expert. Is it valid to rely on that assumption?
> > +.endm
> > +
> > +.macro get_vcpu vcpu, ctxt
> > + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> > + kern_hyp_va \vcpu
> > +.endm
> > +
> > +#endif
> > +
> > #endif /* __ARM_KVM_ASM_H__ */
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 71bf088..612021d 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -135,6 +135,7 @@ int main(void)
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > #endif
> > #ifdef CONFIG_CPU_PM
> > DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index 9a8ab5d..76cd48f 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> > // Store the host regs
> > save_callee_saved_regs x1
> >
> > - // Store host_ctxt and vcpu for use at exit time
> > - stp x1, x0, [sp, #-16]!
> > -
> > add x18, x0, #VCPU_CONTEXT
> >
> > // Restore guest regs x0-x17
> > @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> > save_callee_saved_regs x1
> >
> > // Restore the host_ctxt from the stack
>
> Stale comment again.
>
yeah...
> > - ldr x2, [sp], #16
> > + get_host_ctxt x2, x3
> >
> > // Now restore the host regs
> > restore_callee_saved_regs x2
> > diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> > index e4f37b9..2950f26 100644
> > --- a/arch/arm64/kvm/hyp/hyp-entry.S
> > +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> > @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> > el1_sync: // Guest trapped into EL2
> > stp x0, x1, [sp, #-16]!
> >
> > -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> > - mrs x1, esr_el2
> > -alternative_else
> > - mrs x1, esr_el1
> > -alternative_endif
> > - lsr x0, x1, #ESR_ELx_EC_SHIFT
> > -
> > - cmp x0, #ESR_ELx_EC_HVC64
> > - b.ne el1_trap
> > -
> > mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> > cbnz x1, el1_trap // called HVC
>
> Comment is outdated. Any guest trap will take this path.
>
yeah...
> >
> > +#ifdef CONFIG_DEBUG
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > + cmp x0, #ESR_ELx_EC_HVC64
> > + b.ne __hyp_panic
> > +#endif
> > +
> > /* Here, we're pretty sure the host called HVC. */
> > ldp x0, x1, [sp], #16
> >
> > @@ -101,10 +98,15 @@ alternative_endif
> > eret
> >
> > el1_trap:
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > +
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > /*
> > * x0: ESR_EC
> > + * x1: vcpu pointer
> > */
> > - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
> >
> > /*
> > * We trap the first access to the FP/SIMD to save the host context
> > @@ -122,13 +124,15 @@ alternative_else_nop_endif
> >
> > el1_irq:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_IRQ
> > b __guest_exit
> >
> > el1_error:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_EL1_SERROR
> > b __guest_exit
> >
> > @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> > ENDPROC(__hyp_do_panic)
> >
> > ENTRY(__hyp_panic)
> > - /*
> > - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > - * not be accessible by this address from EL2, hyp_panic() converts
> > - * it with kern_hyp_va() before use.
> > - */
> > - ldr x0, =kvm_host_cpu_state
> > - mrs x1, tpidr_el2
> > - add x0, x0, x1
> > + get_host_ctxt x0, x1
> > b hyp_panic
> > ENDPROC(__hyp_panic)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 69ef24a..a0123ad 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> > if (read_sysreg(vttbr_el2)) {
> > struct kvm_cpu_context *host_ctxt;
> >
> > - host_ctxt = kern_hyp_va(__host_ctxt);
> > + host_ctxt = __host_ctxt;
>
> Can't we just rename __host_ctxt to host_ctxt and drop the local definition?
>
yes, patch splitting snafu. Will fix.
By the way, what I'm going for is anything in the hyp address space has
leading __, and otherwise ot.
> > vcpu = host_ctxt->__hyp_running_vcpu;
> > __timer_disable_traps(vcpu);
> > __deactivate_traps(vcpu);
> >
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-10-12 17:02 ` Christoffer Dall
@ 2017-10-13 11:31 ` Marc Zyngier
-1 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-13 11:31 UTC (permalink / raw)
To: Christoffer Dall; +Cc: linux-arm-kernel, Shih-Wei Li, kvmarm, kvm
On 12/10/17 18:02, Christoffer Dall wrote:
> On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
>> On 12/10/17 11:41, Christoffer Dall wrote:
>>> We already have the percpu area for the host cpu state, which points to
>>> the VCPU, so there's no need to store the VCPU pointer on the stack on
>>> every context switch. We can be a little more clever and just use
>>> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
>>> context.
>>>
>>> This requires us to have a scratch register though, so we take the
>>> chance to rearrange some of the el1_sync code to only look at the
>>> vttbr_el2 to determine if this is a trap from the guest or an HVC from
>>> the host. We do add an extra check to call the panic code if the kernel
>>> is configured with debugging enabled and we saw a trap from the host
>>> which wasn't an HVC, indicating that we left some EL2 trap configured by
>>> mistake.
>>>
>>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> ---
>>> arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
>>> arch/arm64/kernel/asm-offsets.c | 1 +
>>> arch/arm64/kvm/hyp/entry.S | 5 +----
>>> arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
>>> arch/arm64/kvm/hyp/switch.c | 2 +-
>>> 5 files changed, 41 insertions(+), 26 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
>>> index ab4d0a9..7e48a39 100644
>>> --- a/arch/arm64/include/asm/kvm_asm.h
>>> +++ b/arch/arm64/include/asm/kvm_asm.h
>>> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>>>
>>> #endif
>>>
>>> +#ifdef __ASSEMBLY__
>>> +.macro get_host_ctxt reg, tmp
>>> + /*
>>> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
>>> + * not be accessible by this address from EL2, hyp_panic() converts
>>> + * it with kern_hyp_va() before use.
>>> + */
>>
>> This really looks like a stale comment, as there is no hyp_panic
>> involved here anymore (thankfully!).
>>
>
> yeah, I suppose.
>
>>> + ldr \reg, =kvm_host_cpu_state
>>> + mrs \tmp, tpidr_el2
>>> + add \reg, \reg, \tmp
>>> + kern_hyp_va \reg
>>
>> Here, we're trading a load from the stack for a load from the constant
>> pool. Can't we do something like:
>>
>> adr_l \reg, kvm_host_cpu_state
>> msr \tmp, tpidr_el2
>> add \reg, \reg, \tmp
>>
>> and that's it?
>
> That's definitely what the compiler generates from C code...
>
>> This relies on the property that the kernel/hyp offset is
>> constant, and that it doesn't matter if we add the offset to a kernel VA
>> or a HYP VA... Completely untested of course!
>>
>
> You're the hyp VA expert. Is it valid to rely on that assumption?
Absolutely. Otherwise, we've messed up something really badly.
[...]
>>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>>> index 69ef24a..a0123ad 100644
>>> --- a/arch/arm64/kvm/hyp/switch.c
>>> +++ b/arch/arm64/kvm/hyp/switch.c
>>> @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
>>> if (read_sysreg(vttbr_el2)) {
>>> struct kvm_cpu_context *host_ctxt;
>>>
>>> - host_ctxt = kern_hyp_va(__host_ctxt);
>>> + host_ctxt = __host_ctxt;
>>
>> Can't we just rename __host_ctxt to host_ctxt and drop the local definition?
>>
>
> yes, patch splitting snafu. Will fix.
>
> By the way, what I'm going for is anything in the hyp address space has
> leading __, and otherwise ot.
OK, that's a useful convention, actually.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-10-13 11:31 ` Marc Zyngier
0 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-13 11:31 UTC (permalink / raw)
To: linux-arm-kernel
On 12/10/17 18:02, Christoffer Dall wrote:
> On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
>> On 12/10/17 11:41, Christoffer Dall wrote:
>>> We already have the percpu area for the host cpu state, which points to
>>> the VCPU, so there's no need to store the VCPU pointer on the stack on
>>> every context switch. We can be a little more clever and just use
>>> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
>>> context.
>>>
>>> This requires us to have a scratch register though, so we take the
>>> chance to rearrange some of the el1_sync code to only look at the
>>> vttbr_el2 to determine if this is a trap from the guest or an HVC from
>>> the host. We do add an extra check to call the panic code if the kernel
>>> is configured with debugging enabled and we saw a trap from the host
>>> which wasn't an HVC, indicating that we left some EL2 trap configured by
>>> mistake.
>>>
>>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> ---
>>> arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
>>> arch/arm64/kernel/asm-offsets.c | 1 +
>>> arch/arm64/kvm/hyp/entry.S | 5 +----
>>> arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
>>> arch/arm64/kvm/hyp/switch.c | 2 +-
>>> 5 files changed, 41 insertions(+), 26 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
>>> index ab4d0a9..7e48a39 100644
>>> --- a/arch/arm64/include/asm/kvm_asm.h
>>> +++ b/arch/arm64/include/asm/kvm_asm.h
>>> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>>>
>>> #endif
>>>
>>> +#ifdef __ASSEMBLY__
>>> +.macro get_host_ctxt reg, tmp
>>> + /*
>>> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
>>> + * not be accessible by this address from EL2, hyp_panic() converts
>>> + * it with kern_hyp_va() before use.
>>> + */
>>
>> This really looks like a stale comment, as there is no hyp_panic
>> involved here anymore (thankfully!).
>>
>
> yeah, I suppose.
>
>>> + ldr \reg, =kvm_host_cpu_state
>>> + mrs \tmp, tpidr_el2
>>> + add \reg, \reg, \tmp
>>> + kern_hyp_va \reg
>>
>> Here, we're trading a load from the stack for a load from the constant
>> pool. Can't we do something like:
>>
>> adr_l \reg, kvm_host_cpu_state
>> msr \tmp, tpidr_el2
>> add \reg, \reg, \tmp
>>
>> and that's it?
>
> That's definitely what the compiler generates from C code...
>
>> This relies on the property that the kernel/hyp offset is
>> constant, and that it doesn't matter if we add the offset to a kernel VA
>> or a HYP VA... Completely untested of course!
>>
>
> You're the hyp VA expert. Is it valid to rely on that assumption?
Absolutely. Otherwise, we've messed up something really badly.
[...]
>>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>>> index 69ef24a..a0123ad 100644
>>> --- a/arch/arm64/kvm/hyp/switch.c
>>> +++ b/arch/arm64/kvm/hyp/switch.c
>>> @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
>>> if (read_sysreg(vttbr_el2)) {
>>> struct kvm_cpu_context *host_ctxt;
>>>
>>> - host_ctxt = kern_hyp_va(__host_ctxt);
>>> + host_ctxt = __host_ctxt;
>>
>> Can't we just rename __host_ctxt to host_ctxt and drop the local definition?
>>
>
> yes, patch splitting snafu. Will fix.
>
> By the way, what I'm going for is anything in the hyp address space has
> leading __, and otherwise ot.
OK, that's a useful convention, actually.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-10-12 15:49 ` Marc Zyngier
@ 2017-11-23 20:59 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-23 20:59 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Shih-Wei Li
Hi Marc,
On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> > We already have the percpu area for the host cpu state, which points to
> > the VCPU, so there's no need to store the VCPU pointer on the stack on
> > every context switch. We can be a little more clever and just use
> > tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> > context.
> >
> > This requires us to have a scratch register though, so we take the
> > chance to rearrange some of the el1_sync code to only look at the
> > vttbr_el2 to determine if this is a trap from the guest or an HVC from
> > the host. We do add an extra check to call the panic code if the kernel
> > is configured with debugging enabled and we saw a trap from the host
> > which wasn't an HVC, indicating that we left some EL2 trap configured by
> > mistake.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 5 +----
> > arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> > arch/arm64/kvm/hyp/switch.c | 2 +-
> > 5 files changed, 41 insertions(+), 26 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index ab4d0a9..7e48a39 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >
> > #endif
> >
> > +#ifdef __ASSEMBLY__
> > +.macro get_host_ctxt reg, tmp
> > + /*
> > + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > + * not be accessible by this address from EL2, hyp_panic() converts
> > + * it with kern_hyp_va() before use.
> > + */
>
> This really looks like a stale comment, as there is no hyp_panic
> involved here anymore (thankfully!).
>
> > + ldr \reg, =kvm_host_cpu_state
> > + mrs \tmp, tpidr_el2
> > + add \reg, \reg, \tmp
> > + kern_hyp_va \reg
>
> Here, we're trading a load from the stack for a load from the constant
> pool. Can't we do something like:
>
> adr_l \reg, kvm_host_cpu_state
> msr \tmp, tpidr_el2
> add \reg, \reg, \tmp
>
> and that's it? This relies on the property that the kernel/hyp offset is
> constant, and that it doesn't matter if we add the offset to a kernel VA
> or a HYP VA... Completely untested of course!
>
Coming back to this one, annoyingly, it doesn't seem to work. This is
the code I use for get_host_ctxt:
.macro get_host_ctxt reg, tmp
adr_l \reg, kvm_host_cpu_state
mrs \tmp, tpidr_el2
add \reg, \reg, \tmp
kern_hyp_va \reg
.endm
And this is the disassembly for one of the uses in the hyp code:
adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
add x0, x0, #0x7f0
mrs x1, tpidr_el2
add x0, x0, x1
and x0, x0, #0xffffffffffff
For comparison, the following C-code:
struct kvm_cpu_context *host_ctxt;
host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
host_ctxt = kern_hyp_va(host_ctxt);
Gets compiled into this:
adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
add x0, x0, #0x7d0
mrs x1, tpidr_el1
add x0, x0, #0x20
add x0, x0, x1
and x0, x0, #0xffffffffffff
And, during hyp init we do:
mrs x1, tpidr_el1
msr tpidr_el2, x1
Any ideas what could be going on here?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-11-23 20:59 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-23 20:59 UTC (permalink / raw)
To: linux-arm-kernel
Hi Marc,
On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> > We already have the percpu area for the host cpu state, which points to
> > the VCPU, so there's no need to store the VCPU pointer on the stack on
> > every context switch. We can be a little more clever and just use
> > tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> > context.
> >
> > This requires us to have a scratch register though, so we take the
> > chance to rearrange some of the el1_sync code to only look at the
> > vttbr_el2 to determine if this is a trap from the guest or an HVC from
> > the host. We do add an extra check to call the panic code if the kernel
> > is configured with debugging enabled and we saw a trap from the host
> > which wasn't an HVC, indicating that we left some EL2 trap configured by
> > mistake.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 5 +----
> > arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> > arch/arm64/kvm/hyp/switch.c | 2 +-
> > 5 files changed, 41 insertions(+), 26 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index ab4d0a9..7e48a39 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >
> > #endif
> >
> > +#ifdef __ASSEMBLY__
> > +.macro get_host_ctxt reg, tmp
> > + /*
> > + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > + * not be accessible by this address from EL2, hyp_panic() converts
> > + * it with kern_hyp_va() before use.
> > + */
>
> This really looks like a stale comment, as there is no hyp_panic
> involved here anymore (thankfully!).
>
> > + ldr \reg, =kvm_host_cpu_state
> > + mrs \tmp, tpidr_el2
> > + add \reg, \reg, \tmp
> > + kern_hyp_va \reg
>
> Here, we're trading a load from the stack for a load from the constant
> pool. Can't we do something like:
>
> adr_l \reg, kvm_host_cpu_state
> msr \tmp, tpidr_el2
> add \reg, \reg, \tmp
>
> and that's it? This relies on the property that the kernel/hyp offset is
> constant, and that it doesn't matter if we add the offset to a kernel VA
> or a HYP VA... Completely untested of course!
>
Coming back to this one, annoyingly, it doesn't seem to work. This is
the code I use for get_host_ctxt:
.macro get_host_ctxt reg, tmp
adr_l \reg, kvm_host_cpu_state
mrs \tmp, tpidr_el2
add \reg, \reg, \tmp
kern_hyp_va \reg
.endm
And this is the disassembly for one of the uses in the hyp code:
adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
add x0, x0, #0x7f0
mrs x1, tpidr_el2
add x0, x0, x1
and x0, x0, #0xffffffffffff
For comparison, the following C-code:
struct kvm_cpu_context *host_ctxt;
host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
host_ctxt = kern_hyp_va(host_ctxt);
Gets compiled into this:
adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
add x0, x0, #0x7d0
mrs x1, tpidr_el1
add x0, x0, #0x20
add x0, x0, x1
and x0, x0, #0xffffffffffff
And, during hyp init we do:
mrs x1, tpidr_el1
msr tpidr_el2, x1
Any ideas what could be going on here?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-11-23 20:59 ` Christoffer Dall
@ 2017-11-27 11:11 ` James Morse
-1 siblings, 0 replies; 254+ messages in thread
From: James Morse @ 2017-11-27 11:11 UTC (permalink / raw)
To: Christoffer Dall, Marc Zyngier; +Cc: linux-arm-kernel, Shih-Wei Li, kvmarm, kvm
Hi Christoffer,
On 23/11/17 20:59, Christoffer Dall wrote:
> On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
>> On 12/10/17 11:41, Christoffer Dall wrote:
>>> We already have the percpu area for the host cpu state, which points to
>>> the VCPU, so there's no need to store the VCPU pointer on the stack on
>>> every context switch. We can be a little more clever and just use
>>> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
>>> context.
>>>
>>> This requires us to have a scratch register though, so we take the
>>> chance to rearrange some of the el1_sync code to only look at the
>>> vttbr_el2 to determine if this is a trap from the guest or an HVC from
>>> the host. We do add an extra check to call the panic code if the kernel
>>> is configured with debugging enabled and we saw a trap from the host
>>> which wasn't an HVC, indicating that we left some EL2 trap configured by
>>> mistake.
>>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
>>> index ab4d0a9..7e48a39 100644
>>> --- a/arch/arm64/include/asm/kvm_asm.h
>>> +++ b/arch/arm64/include/asm/kvm_asm.h
>>> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>>>
>>> #endif
>>>
>>> +#ifdef __ASSEMBLY__
>>> +.macro get_host_ctxt reg, tmp
>>> + /*
>>> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
>>> + * not be accessible by this address from EL2, hyp_panic() converts
>>> + * it with kern_hyp_va() before use.
>>> + */
>>
>> This really looks like a stale comment, as there is no hyp_panic
>> involved here anymore (thankfully!).
>>
>>> + ldr \reg, =kvm_host_cpu_state
>>> + mrs \tmp, tpidr_el2
>>> + add \reg, \reg, \tmp
This looks like the arch code's adr_this_cpu.
>>> + kern_hyp_va \reg
>>
>> Here, we're trading a load from the stack for a load from the constant
>> pool. Can't we do something like:
>>
>> adr_l \reg, kvm_host_cpu_state
>> msr \tmp, tpidr_el2
>> add \reg, \reg, \tmp
>>
>> and that's it? This relies on the property that the kernel/hyp offset is
>> constant, and that it doesn't matter if we add the offset to a kernel VA
>> or a HYP VA... Completely untested of course!
>>
>
> Coming back to this one, annoyingly, it doesn't seem to work.
The disassembly looks wrong?, or it generates the wrong address?
> This is the code I use for get_host_ctxt:
>
> .macro get_host_ctxt reg, tmp
> adr_l \reg, kvm_host_cpu_state
> mrs \tmp, tpidr_el2
> add \reg, \reg, \tmp
(adr_this_cpu)
> kern_hyp_va \reg
As we know adr_l used adrp to generate a PC-relative address, when executed at
EL2 it should always generate an EL2 address, so the kern_hyp_va will just mask
out some bits that are already zero.
(this subtly depends on KVM's EL2 code not being a module, and
kvm_host_cpu_state not being percpu_alloc()d)
> .endm
>
> And this is the disassembly for one of the uses in the hyp code:
>
> adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> add x0, x0, #0x7f0
> mrs x1, tpidr_el2
> add x0, x0, x1
> and x0, x0, #0xffffffffffff
(that looks right to me).
> For comparison, the following C-code:
>
> struct kvm_cpu_context *host_ctxt;
> host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
> host_ctxt = kern_hyp_va(host_ctxt);
>
> Gets compiled into this:
>
> adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> add x0, x0, #0x7d0
> mrs x1, tpidr_el1
> add x0, x0, #0x20
> add x0, x0, x1
> and x0, x0, #0xffffffffffff
> Any ideas what could be going on here?
You expected tpidr_el2 in the above disassembly?
The patch 'arm64: alternatives: use tpidr_el2 on VHE hosts'[0] wraps the tpidr
access in adr_this_cpu,ldr_this_cpu and __my_cpu_offset() in
ARM64_HAS_VIRT_HOST_EXTN alternatives.
You should have an altinstr_replacement section that contains the 'mrs x1,
tpidr_el2' for this sequence, which will get patched in by the cpufeature code
when we find VHE.
I'm guessing you want to always use tpidr_el2 as cpu_offset for KVM, even on
v8.0 hardware. To do this you can't use the kernel's 'this_cpu_ptr' as its
defined in percpu-defs.h as:
> SHIFT_PERCPU_PTR(ptr, my_cpu_offset)
... and the arch code provides a static-inline 'my_cpu_offset' that resolves to
the correct tpidr for EL1.
I guess you need an asm-accessor for each per-cpu variable you want to access,
or a kvm_this_per_cpu().
> And, during hyp init we do:
> mrs x1, tpidr_el1
> msr tpidr_el2, x1
In the SDEI series this was so that the asm that used tpidr_el2 directly had the
correct value on non-VHE hardware.
Thanks,
James
[0] https://patchwork.kernel.org/patch/10012641/
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-11-27 11:11 ` James Morse
0 siblings, 0 replies; 254+ messages in thread
From: James Morse @ 2017-11-27 11:11 UTC (permalink / raw)
To: linux-arm-kernel
Hi Christoffer,
On 23/11/17 20:59, Christoffer Dall wrote:
> On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
>> On 12/10/17 11:41, Christoffer Dall wrote:
>>> We already have the percpu area for the host cpu state, which points to
>>> the VCPU, so there's no need to store the VCPU pointer on the stack on
>>> every context switch. We can be a little more clever and just use
>>> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
>>> context.
>>>
>>> This requires us to have a scratch register though, so we take the
>>> chance to rearrange some of the el1_sync code to only look at the
>>> vttbr_el2 to determine if this is a trap from the guest or an HVC from
>>> the host. We do add an extra check to call the panic code if the kernel
>>> is configured with debugging enabled and we saw a trap from the host
>>> which wasn't an HVC, indicating that we left some EL2 trap configured by
>>> mistake.
>>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
>>> index ab4d0a9..7e48a39 100644
>>> --- a/arch/arm64/include/asm/kvm_asm.h
>>> +++ b/arch/arm64/include/asm/kvm_asm.h
>>> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>>>
>>> #endif
>>>
>>> +#ifdef __ASSEMBLY__
>>> +.macro get_host_ctxt reg, tmp
>>> + /*
>>> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
>>> + * not be accessible by this address from EL2, hyp_panic() converts
>>> + * it with kern_hyp_va() before use.
>>> + */
>>
>> This really looks like a stale comment, as there is no hyp_panic
>> involved here anymore (thankfully!).
>>
>>> + ldr \reg, =kvm_host_cpu_state
>>> + mrs \tmp, tpidr_el2
>>> + add \reg, \reg, \tmp
This looks like the arch code's adr_this_cpu.
>>> + kern_hyp_va \reg
>>
>> Here, we're trading a load from the stack for a load from the constant
>> pool. Can't we do something like:
>>
>> adr_l \reg, kvm_host_cpu_state
>> msr \tmp, tpidr_el2
>> add \reg, \reg, \tmp
>>
>> and that's it? This relies on the property that the kernel/hyp offset is
>> constant, and that it doesn't matter if we add the offset to a kernel VA
>> or a HYP VA... Completely untested of course!
>>
>
> Coming back to this one, annoyingly, it doesn't seem to work.
The disassembly looks wrong?, or it generates the wrong address?
> This is the code I use for get_host_ctxt:
>
> .macro get_host_ctxt reg, tmp
> adr_l \reg, kvm_host_cpu_state
> mrs \tmp, tpidr_el2
> add \reg, \reg, \tmp
(adr_this_cpu)
> kern_hyp_va \reg
As we know adr_l used adrp to generate a PC-relative address, when executed at
EL2 it should always generate an EL2 address, so the kern_hyp_va will just mask
out some bits that are already zero.
(this subtly depends on KVM's EL2 code not being a module, and
kvm_host_cpu_state not being percpu_alloc()d)
> .endm
>
> And this is the disassembly for one of the uses in the hyp code:
>
> adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> add x0, x0, #0x7f0
> mrs x1, tpidr_el2
> add x0, x0, x1
> and x0, x0, #0xffffffffffff
(that looks right to me).
> For comparison, the following C-code:
>
> struct kvm_cpu_context *host_ctxt;
> host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
> host_ctxt = kern_hyp_va(host_ctxt);
>
> Gets compiled into this:
>
> adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> add x0, x0, #0x7d0
> mrs x1, tpidr_el1
> add x0, x0, #0x20
> add x0, x0, x1
> and x0, x0, #0xffffffffffff
> Any ideas what could be going on here?
You expected tpidr_el2 in the above disassembly?
The patch 'arm64: alternatives: use tpidr_el2 on VHE hosts'[0] wraps the tpidr
access in adr_this_cpu,ldr_this_cpu and __my_cpu_offset() in
ARM64_HAS_VIRT_HOST_EXTN alternatives.
You should have an altinstr_replacement section that contains the 'mrs x1,
tpidr_el2' for this sequence, which will get patched in by the cpufeature code
when we find VHE.
I'm guessing you want to always use tpidr_el2 as cpu_offset for KVM, even on
v8.0 hardware. To do this you can't use the kernel's 'this_cpu_ptr' as its
defined in percpu-defs.h as:
> SHIFT_PERCPU_PTR(ptr, my_cpu_offset)
... and the arch code provides a static-inline 'my_cpu_offset' that resolves to
the correct tpidr for EL1.
I guess you need an asm-accessor for each per-cpu variable you want to access,
or a kvm_this_per_cpu().
> And, during hyp init we do:
> mrs x1, tpidr_el1
> msr tpidr_el2, x1
In the SDEI series this was so that the asm that used tpidr_el2 directly had the
correct value on non-VHE hardware.
Thanks,
James
[0] https://patchwork.kernel.org/patch/10012641/
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-11-27 11:11 ` James Morse
@ 2017-11-29 18:20 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-29 18:20 UTC (permalink / raw)
To: James Morse; +Cc: linux-arm-kernel, kvm, Marc Zyngier, kvmarm, Shih-Wei Li
Hi James,
On Mon, Nov 27, 2017 at 11:11:20AM +0000, James Morse wrote:
> On 23/11/17 20:59, Christoffer Dall wrote:
> > On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
> >> On 12/10/17 11:41, Christoffer Dall wrote:
> >>> We already have the percpu area for the host cpu state, which points to
> >>> the VCPU, so there's no need to store the VCPU pointer on the stack on
> >>> every context switch. We can be a little more clever and just use
> >>> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> >>> context.
> >>>
> >>> This requires us to have a scratch register though, so we take the
> >>> chance to rearrange some of the el1_sync code to only look at the
> >>> vttbr_el2 to determine if this is a trap from the guest or an HVC from
> >>> the host. We do add an extra check to call the panic code if the kernel
> >>> is configured with debugging enabled and we saw a trap from the host
> >>> which wasn't an HVC, indicating that we left some EL2 trap configured by
> >>> mistake.
>
> >>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> >>> index ab4d0a9..7e48a39 100644
> >>> --- a/arch/arm64/include/asm/kvm_asm.h
> >>> +++ b/arch/arm64/include/asm/kvm_asm.h
> >>> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >>>
> >>> #endif
> >>>
> >>> +#ifdef __ASSEMBLY__
> >>> +.macro get_host_ctxt reg, tmp
> >>> + /*
> >>> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> >>> + * not be accessible by this address from EL2, hyp_panic() converts
> >>> + * it with kern_hyp_va() before use.
> >>> + */
> >>
> >> This really looks like a stale comment, as there is no hyp_panic
> >> involved here anymore (thankfully!).
> >>
> >>> + ldr \reg, =kvm_host_cpu_state
> >>> + mrs \tmp, tpidr_el2
> >>> + add \reg, \reg, \tmp
>
> This looks like the arch code's adr_this_cpu.
>
>
> >>> + kern_hyp_va \reg
> >>
> >> Here, we're trading a load from the stack for a load from the constant
> >> pool. Can't we do something like:
> >>
> >> adr_l \reg, kvm_host_cpu_state
> >> msr \tmp, tpidr_el2
> >> add \reg, \reg, \tmp
> >>
> >> and that's it? This relies on the property that the kernel/hyp offset is
> >> constant, and that it doesn't matter if we add the offset to a kernel VA
> >> or a HYP VA... Completely untested of course!
> >>
> >
> > Coming back to this one, annoyingly, it doesn't seem to work.
>
> The disassembly looks wrong?, or it generates the wrong address?
>
The assembly above was just something Marc suggested. I think it's
wrong (it's should be mrs, not msr in the second line), but I just took
it as inspiration, so that's not part of the problem at hand. Sorry for
the confusion.
>
> > This is the code I use for get_host_ctxt:
> >
> > .macro get_host_ctxt reg, tmp
> > adr_l \reg, kvm_host_cpu_state
> > mrs \tmp, tpidr_el2
> > add \reg, \reg, \tmp
>
> (adr_this_cpu)
>
> > kern_hyp_va \reg
>
> As we know adr_l used adrp to generate a PC-relative address, when executed at
> EL2 it should always generate an EL2 address, so the kern_hyp_va will just mask
> out some bits that are already zero.
yes, that's right
>
> (this subtly depends on KVM's EL2 code not being a module, and
> kvm_host_cpu_state not being percpu_alloc()d)
>
>
yes, and I have your "KVM: arm/arm64: Convert kvm_host_cpu_state to a
static per-cpu allocation" patch.
> > .endm
> >
> > And this is the disassembly for one of the uses in the hyp code:
> >
> > adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> > add x0, x0, #0x7f0
> > mrs x1, tpidr_el2
> > add x0, x0, x1
> > and x0, x0, #0xffffffffffff
>
> (that looks right to me).
>
>
to me too, but it doesn't work :(
> > For comparison, the following C-code:
> >
> > struct kvm_cpu_context *host_ctxt;
> > host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
> > host_ctxt = kern_hyp_va(host_ctxt);
> >
> > Gets compiled into this:
> >
> > adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> > add x0, x0, #0x7d0
> > mrs x1, tpidr_el1
> > add x0, x0, #0x20
> > add x0, x0, x1
> > and x0, x0, #0xffffffffffff
>
> > Any ideas what could be going on here?
>
> You expected tpidr_el2 in the above disassembly?
No, because I'm not on a VHE host, but I expect tpidr_el1 and tpidr_el2
to be the same in the hyp code.
I now realize that I never said that this breaks on a non-VHE host, I
haven't actually tried a VHE host, but it shouldn't matter.
>
> The patch 'arm64: alternatives: use tpidr_el2 on VHE hosts'[0] wraps the tpidr
> access in adr_this_cpu,ldr_this_cpu and __my_cpu_offset() in
> ARM64_HAS_VIRT_HOST_EXTN alternatives.
>
> You should have an altinstr_replacement section that contains the 'mrs x1,
> tpidr_el2' for this sequence, which will get patched in by the cpufeature code
> when we find VHE.
>
Yes, I think all that is fine.
>
> I'm guessing you want to always use tpidr_el2 as cpu_offset for KVM, even on
> v8.0 hardware. To do this you can't use the kernel's 'this_cpu_ptr' as its
> defined in percpu-defs.h as:
> > SHIFT_PERCPU_PTR(ptr, my_cpu_offset)
>
> ... and the arch code provides a static-inline 'my_cpu_offset' that resolves to
> the correct tpidr for EL1.
>
> I guess you need an asm-accessor for each per-cpu variable you want to access,
> or a kvm_this_per_cpu().
>
I was under the impression that we were essentially open-coding this
functionality with the assembly above. What did I miss?
>
> > And, during hyp init we do:
> > mrs x1, tpidr_el1
> > msr tpidr_el2, x1
>
> In the SDEI series this was so that the asm that used tpidr_el2 directly had the
> correct value on non-VHE hardware.
>
>
Yes, and I simply generalized that bit of assembly (the hyp panic logic)
which also needed the vcpu context to all the assembly that needs the
vcpu context.
And it works fine with a load from the constant pool and the mask, but
not with the open-coded this_cpu_ptr() at EL2. On a non-VHE system.
Even though the assembly seems identical, and it should just work (TM).
Thoughts?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-11-29 18:20 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-29 18:20 UTC (permalink / raw)
To: linux-arm-kernel
Hi James,
On Mon, Nov 27, 2017 at 11:11:20AM +0000, James Morse wrote:
> On 23/11/17 20:59, Christoffer Dall wrote:
> > On Thu, Oct 12, 2017 at 04:49:44PM +0100, Marc Zyngier wrote:
> >> On 12/10/17 11:41, Christoffer Dall wrote:
> >>> We already have the percpu area for the host cpu state, which points to
> >>> the VCPU, so there's no need to store the VCPU pointer on the stack on
> >>> every context switch. We can be a little more clever and just use
> >>> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> >>> context.
> >>>
> >>> This requires us to have a scratch register though, so we take the
> >>> chance to rearrange some of the el1_sync code to only look at the
> >>> vttbr_el2 to determine if this is a trap from the guest or an HVC from
> >>> the host. We do add an extra check to call the panic code if the kernel
> >>> is configured with debugging enabled and we saw a trap from the host
> >>> which wasn't an HVC, indicating that we left some EL2 trap configured by
> >>> mistake.
>
> >>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> >>> index ab4d0a9..7e48a39 100644
> >>> --- a/arch/arm64/include/asm/kvm_asm.h
> >>> +++ b/arch/arm64/include/asm/kvm_asm.h
> >>> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >>>
> >>> #endif
> >>>
> >>> +#ifdef __ASSEMBLY__
> >>> +.macro get_host_ctxt reg, tmp
> >>> + /*
> >>> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> >>> + * not be accessible by this address from EL2, hyp_panic() converts
> >>> + * it with kern_hyp_va() before use.
> >>> + */
> >>
> >> This really looks like a stale comment, as there is no hyp_panic
> >> involved here anymore (thankfully!).
> >>
> >>> + ldr \reg, =kvm_host_cpu_state
> >>> + mrs \tmp, tpidr_el2
> >>> + add \reg, \reg, \tmp
>
> This looks like the arch code's adr_this_cpu.
>
>
> >>> + kern_hyp_va \reg
> >>
> >> Here, we're trading a load from the stack for a load from the constant
> >> pool. Can't we do something like:
> >>
> >> adr_l \reg, kvm_host_cpu_state
> >> msr \tmp, tpidr_el2
> >> add \reg, \reg, \tmp
> >>
> >> and that's it? This relies on the property that the kernel/hyp offset is
> >> constant, and that it doesn't matter if we add the offset to a kernel VA
> >> or a HYP VA... Completely untested of course!
> >>
> >
> > Coming back to this one, annoyingly, it doesn't seem to work.
>
> The disassembly looks wrong?, or it generates the wrong address?
>
The assembly above was just something Marc suggested. I think it's
wrong (it's should be mrs, not msr in the second line), but I just took
it as inspiration, so that's not part of the problem at hand. Sorry for
the confusion.
>
> > This is the code I use for get_host_ctxt:
> >
> > .macro get_host_ctxt reg, tmp
> > adr_l \reg, kvm_host_cpu_state
> > mrs \tmp, tpidr_el2
> > add \reg, \reg, \tmp
>
> (adr_this_cpu)
>
> > kern_hyp_va \reg
>
> As we know adr_l used adrp to generate a PC-relative address, when executed at
> EL2 it should always generate an EL2 address, so the kern_hyp_va will just mask
> out some bits that are already zero.
yes, that's right
>
> (this subtly depends on KVM's EL2 code not being a module, and
> kvm_host_cpu_state not being percpu_alloc()d)
>
>
yes, and I have your "KVM: arm/arm64: Convert kvm_host_cpu_state to a
static per-cpu allocation" patch.
> > .endm
> >
> > And this is the disassembly for one of the uses in the hyp code:
> >
> > adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> > add x0, x0, #0x7f0
> > mrs x1, tpidr_el2
> > add x0, x0, x1
> > and x0, x0, #0xffffffffffff
>
> (that looks right to me).
>
>
to me too, but it doesn't work :(
> > For comparison, the following C-code:
> >
> > struct kvm_cpu_context *host_ctxt;
> > host_ctxt = this_cpu_ptr(&kvm_host_cpu_state);
> > host_ctxt = kern_hyp_va(host_ctxt);
> >
> > Gets compiled into this:
> >
> > adrp x0, ffff000008ca9000 <overflow_stack+0xd20>
> > add x0, x0, #0x7d0
> > mrs x1, tpidr_el1
> > add x0, x0, #0x20
> > add x0, x0, x1
> > and x0, x0, #0xffffffffffff
>
> > Any ideas what could be going on here?
>
> You expected tpidr_el2 in the above disassembly?
No, because I'm not on a VHE host, but I expect tpidr_el1 and tpidr_el2
to be the same in the hyp code.
I now realize that I never said that this breaks on a non-VHE host, I
haven't actually tried a VHE host, but it shouldn't matter.
>
> The patch 'arm64: alternatives: use tpidr_el2 on VHE hosts'[0] wraps the tpidr
> access in adr_this_cpu,ldr_this_cpu and __my_cpu_offset() in
> ARM64_HAS_VIRT_HOST_EXTN alternatives.
>
> You should have an altinstr_replacement section that contains the 'mrs x1,
> tpidr_el2' for this sequence, which will get patched in by the cpufeature code
> when we find VHE.
>
Yes, I think all that is fine.
>
> I'm guessing you want to always use tpidr_el2 as cpu_offset for KVM, even on
> v8.0 hardware. To do this you can't use the kernel's 'this_cpu_ptr' as its
> defined in percpu-defs.h as:
> > SHIFT_PERCPU_PTR(ptr, my_cpu_offset)
>
> ... and the arch code provides a static-inline 'my_cpu_offset' that resolves to
> the correct tpidr for EL1.
>
> I guess you need an asm-accessor for each per-cpu variable you want to access,
> or a kvm_this_per_cpu().
>
I was under the impression that we were essentially open-coding this
functionality with the assembly above. What did I miss?
>
> > And, during hyp init we do:
> > mrs x1, tpidr_el1
> > msr tpidr_el2, x1
>
> In the SDEI series this was so that the asm that used tpidr_el2 directly had the
> correct value on non-VHE hardware.
>
>
Yes, and I simply generalized that bit of assembly (the hyp panic logic)
which also needed the vcpu context to all the assembly that needs the
vcpu context.
And it works fine with a load from the constant pool and the mask, but
not with the open-coded this_cpu_ptr() at EL2. On a non-VHE system.
Even though the assembly seems identical, and it should just work (TM).
Thoughts?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-06 17:22 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-06 17:22 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, Marc Zyngier, Shih-Wei Li, kvm
On Thu, Oct 12, 2017 at 12:41:05PM +0200, Christoffer Dall wrote:
> We already have the percpu area for the host cpu state, which points to
> the VCPU, so there's no need to store the VCPU pointer on the stack on
> every context switch. We can be a little more clever and just use
> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> context.
>
> This requires us to have a scratch register though, so we take the
> chance to rearrange some of the el1_sync code to only look at the
> vttbr_el2 to determine if this is a trap from the guest or an HVC from
> the host. We do add an extra check to call the panic code if the kernel
> is configured with debugging enabled and we saw a trap from the host
> which wasn't an HVC, indicating that we left some EL2 trap configured by
> mistake.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 5 +----
> arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> arch/arm64/kvm/hyp/switch.c | 2 +-
> 5 files changed, 41 insertions(+), 26 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index ab4d0a9..7e48a39 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>
> #endif
>
> +#ifdef __ASSEMBLY__
> +.macro get_host_ctxt reg, tmp
> + /*
> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> + * not be accessible by this address from EL2, hyp_panic() converts
> + * it with kern_hyp_va() before use.
> + */
> + ldr \reg, =kvm_host_cpu_state
> + mrs \tmp, tpidr_el2
> + add \reg, \reg, \tmp
> + kern_hyp_va \reg
> +.endm
> +
> +.macro get_vcpu vcpu, ctxt
> + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> + kern_hyp_va \vcpu
> +.endm
To avoid the need for the pattern
get_host_ctxt x0, x1
get_vcpu x1, x0
everywhere this macro is used, how about defining it as
.macro get_vcpu vcpu, tmp
get_host_ctxt \tmp, \vcpu
ldr \vcpu, [\tmp, #HOST_CONTEXT_VCPU]
kern_hyp_va \vcpu
.endm
which also has the side-effect of tmp being ctxt after the call.
> +
> +#endif
> +
> #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 71bf088..612021d 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -135,6 +135,7 @@ int main(void)
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> #endif
> #ifdef CONFIG_CPU_PM
> DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 9a8ab5d..76cd48f 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> // Store the host regs
> save_callee_saved_regs x1
>
> - // Store host_ctxt and vcpu for use at exit time
> - stp x1, x0, [sp, #-16]!
> -
> add x18, x0, #VCPU_CONTEXT
>
> // Restore guest regs x0-x17
> @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> save_callee_saved_regs x1
>
> // Restore the host_ctxt from the stack
> - ldr x2, [sp], #16
> + get_host_ctxt x2, x3
>
> // Now restore the host regs
> restore_callee_saved_regs x2
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index e4f37b9..2950f26 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> el1_sync: // Guest trapped into EL2
> stp x0, x1, [sp, #-16]!
>
> -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> - mrs x1, esr_el2
> -alternative_else
> - mrs x1, esr_el1
> -alternative_endif
> - lsr x0, x1, #ESR_ELx_EC_SHIFT
> -
> - cmp x0, #ESR_ELx_EC_HVC64
> - b.ne el1_trap
> -
> mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> cbnz x1, el1_trap // called HVC
>
> +#ifdef CONFIG_DEBUG
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> + cmp x0, #ESR_ELx_EC_HVC64
> + b.ne __hyp_panic
> +#endif
The dropping of the "alternative_if_not ARM64_HAS_VIRT_HOST_EXTN" stuff
isn't called out in the commit message, but it looks like it was just
a cleanup of code that was never necessary, as esr_el2 aliases esr_el1.
Is that correct?
> +
> /* Here, we're pretty sure the host called HVC. */
> ldp x0, x1, [sp], #16
>
> @@ -101,10 +98,15 @@ alternative_endif
> eret
>
> el1_trap:
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> +
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> /*
> * x0: ESR_EC
> + * x1: vcpu pointer
> */
> - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
>
> /*
> * We trap the first access to the FP/SIMD to save the host context
> @@ -122,13 +124,15 @@ alternative_else_nop_endif
>
> el1_irq:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_IRQ
> b __guest_exit
>
> el1_error:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_EL1_SERROR
> b __guest_exit
>
> @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> ENDPROC(__hyp_do_panic)
>
> ENTRY(__hyp_panic)
> - /*
> - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> - * not be accessible by this address from EL2, hyp_panic() converts
> - * it with kern_hyp_va() before use.
> - */
> - ldr x0, =kvm_host_cpu_state
> - mrs x1, tpidr_el2
> - add x0, x0, x1
> + get_host_ctxt x0, x1
> b hyp_panic
> ENDPROC(__hyp_panic)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 69ef24a..a0123ad 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> if (read_sysreg(vttbr_el2)) {
> struct kvm_cpu_context *host_ctxt;
>
> - host_ctxt = kern_hyp_va(__host_ctxt);
> + host_ctxt = __host_ctxt;
> vcpu = host_ctxt->__hyp_running_vcpu;
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
> --
> 2.9.0
>
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-11-06 17:22 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-06 17:22 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:05PM +0200, Christoffer Dall wrote:
> We already have the percpu area for the host cpu state, which points to
> the VCPU, so there's no need to store the VCPU pointer on the stack on
> every context switch. We can be a little more clever and just use
> tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> context.
>
> This requires us to have a scratch register though, so we take the
> chance to rearrange some of the el1_sync code to only look at the
> vttbr_el2 to determine if this is a trap from the guest or an HVC from
> the host. We do add an extra check to call the panic code if the kernel
> is configured with debugging enabled and we saw a trap from the host
> which wasn't an HVC, indicating that we left some EL2 trap configured by
> mistake.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 5 +----
> arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> arch/arm64/kvm/hyp/switch.c | 2 +-
> 5 files changed, 41 insertions(+), 26 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index ab4d0a9..7e48a39 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
>
> #endif
>
> +#ifdef __ASSEMBLY__
> +.macro get_host_ctxt reg, tmp
> + /*
> + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> + * not be accessible by this address from EL2, hyp_panic() converts
> + * it with kern_hyp_va() before use.
> + */
> + ldr \reg, =kvm_host_cpu_state
> + mrs \tmp, tpidr_el2
> + add \reg, \reg, \tmp
> + kern_hyp_va \reg
> +.endm
> +
> +.macro get_vcpu vcpu, ctxt
> + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> + kern_hyp_va \vcpu
> +.endm
To avoid the need for the pattern
get_host_ctxt x0, x1
get_vcpu x1, x0
everywhere this macro is used, how about defining it as
.macro get_vcpu vcpu, tmp
get_host_ctxt \tmp, \vcpu
ldr \vcpu, [\tmp, #HOST_CONTEXT_VCPU]
kern_hyp_va \vcpu
.endm
which also has the side-effect of tmp being ctxt after the call.
> +
> +#endif
> +
> #endif /* __ARM_KVM_ASM_H__ */
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 71bf088..612021d 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -135,6 +135,7 @@ int main(void)
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> #endif
> #ifdef CONFIG_CPU_PM
> DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 9a8ab5d..76cd48f 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> // Store the host regs
> save_callee_saved_regs x1
>
> - // Store host_ctxt and vcpu for use at exit time
> - stp x1, x0, [sp, #-16]!
> -
> add x18, x0, #VCPU_CONTEXT
>
> // Restore guest regs x0-x17
> @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> save_callee_saved_regs x1
>
> // Restore the host_ctxt from the stack
> - ldr x2, [sp], #16
> + get_host_ctxt x2, x3
>
> // Now restore the host regs
> restore_callee_saved_regs x2
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index e4f37b9..2950f26 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> el1_sync: // Guest trapped into EL2
> stp x0, x1, [sp, #-16]!
>
> -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> - mrs x1, esr_el2
> -alternative_else
> - mrs x1, esr_el1
> -alternative_endif
> - lsr x0, x1, #ESR_ELx_EC_SHIFT
> -
> - cmp x0, #ESR_ELx_EC_HVC64
> - b.ne el1_trap
> -
> mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> cbnz x1, el1_trap // called HVC
>
> +#ifdef CONFIG_DEBUG
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> + cmp x0, #ESR_ELx_EC_HVC64
> + b.ne __hyp_panic
> +#endif
The dropping of the "alternative_if_not ARM64_HAS_VIRT_HOST_EXTN" stuff
isn't called out in the commit message, but it looks like it was just
a cleanup of code that was never necessary, as esr_el2 aliases esr_el1.
Is that correct?
> +
> /* Here, we're pretty sure the host called HVC. */
> ldp x0, x1, [sp], #16
>
> @@ -101,10 +98,15 @@ alternative_endif
> eret
>
> el1_trap:
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> +
> + mrs x0, esr_el2
> + lsr x0, x0, #ESR_ELx_EC_SHIFT
> /*
> * x0: ESR_EC
> + * x1: vcpu pointer
> */
> - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
>
> /*
> * We trap the first access to the FP/SIMD to save the host context
> @@ -122,13 +124,15 @@ alternative_else_nop_endif
>
> el1_irq:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_IRQ
> b __guest_exit
>
> el1_error:
> stp x0, x1, [sp, #-16]!
> - ldr x1, [sp, #16 + 8]
> + get_host_ctxt x0, x1
> + get_vcpu x1, x0
> mov x0, #ARM_EXCEPTION_EL1_SERROR
> b __guest_exit
>
> @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> ENDPROC(__hyp_do_panic)
>
> ENTRY(__hyp_panic)
> - /*
> - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> - * not be accessible by this address from EL2, hyp_panic() converts
> - * it with kern_hyp_va() before use.
> - */
> - ldr x0, =kvm_host_cpu_state
> - mrs x1, tpidr_el2
> - add x0, x0, x1
> + get_host_ctxt x0, x1
> b hyp_panic
> ENDPROC(__hyp_panic)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 69ef24a..a0123ad 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> if (read_sysreg(vttbr_el2)) {
> struct kvm_cpu_context *host_ctxt;
>
> - host_ctxt = kern_hyp_va(__host_ctxt);
> + host_ctxt = __host_ctxt;
> vcpu = host_ctxt->__hyp_running_vcpu;
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
> --
> 2.9.0
>
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
2017-11-06 17:22 ` Andrew Jones
@ 2017-11-07 8:24 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-07 8:24 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Mon, Nov 06, 2017 at 06:22:51PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:05PM +0200, Christoffer Dall wrote:
> > We already have the percpu area for the host cpu state, which points to
> > the VCPU, so there's no need to store the VCPU pointer on the stack on
> > every context switch. We can be a little more clever and just use
> > tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> > context.
> >
> > This requires us to have a scratch register though, so we take the
> > chance to rearrange some of the el1_sync code to only look at the
> > vttbr_el2 to determine if this is a trap from the guest or an HVC from
> > the host. We do add an extra check to call the panic code if the kernel
> > is configured with debugging enabled and we saw a trap from the host
> > which wasn't an HVC, indicating that we left some EL2 trap configured by
> > mistake.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 5 +----
> > arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> > arch/arm64/kvm/hyp/switch.c | 2 +-
> > 5 files changed, 41 insertions(+), 26 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index ab4d0a9..7e48a39 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >
> > #endif
> >
> > +#ifdef __ASSEMBLY__
> > +.macro get_host_ctxt reg, tmp
> > + /*
> > + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > + * not be accessible by this address from EL2, hyp_panic() converts
> > + * it with kern_hyp_va() before use.
> > + */
> > + ldr \reg, =kvm_host_cpu_state
> > + mrs \tmp, tpidr_el2
> > + add \reg, \reg, \tmp
> > + kern_hyp_va \reg
> > +.endm
> > +
> > +.macro get_vcpu vcpu, ctxt
> > + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> > + kern_hyp_va \vcpu
> > +.endm
>
> To avoid the need for the pattern
>
> get_host_ctxt x0, x1
> get_vcpu x1, x0
>
> everywhere this macro is used, how about defining it as
>
> .macro get_vcpu vcpu, tmp
> get_host_ctxt \tmp, \vcpu
> ldr \vcpu, [\tmp, #HOST_CONTEXT_VCPU]
> kern_hyp_va \vcpu
> .endm
>
> which also has the side-effect of tmp being ctxt after the call.
>
I actually prefer it the way it is now, because I think it's clearer
what's going on. In the past we did some information hiding in assembly
macros and that didn't improve maintenance.
If there's an aspect to this I'm missing, I'm of course open to
reconsiderig though.
> > +
> > +#endif
> > +
> > #endif /* __ARM_KVM_ASM_H__ */
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 71bf088..612021d 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -135,6 +135,7 @@ int main(void)
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > #endif
> > #ifdef CONFIG_CPU_PM
> > DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index 9a8ab5d..76cd48f 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> > // Store the host regs
> > save_callee_saved_regs x1
> >
> > - // Store host_ctxt and vcpu for use at exit time
> > - stp x1, x0, [sp, #-16]!
> > -
> > add x18, x0, #VCPU_CONTEXT
> >
> > // Restore guest regs x0-x17
> > @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> > save_callee_saved_regs x1
> >
> > // Restore the host_ctxt from the stack
> > - ldr x2, [sp], #16
> > + get_host_ctxt x2, x3
> >
> > // Now restore the host regs
> > restore_callee_saved_regs x2
> > diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> > index e4f37b9..2950f26 100644
> > --- a/arch/arm64/kvm/hyp/hyp-entry.S
> > +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> > @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> > el1_sync: // Guest trapped into EL2
> > stp x0, x1, [sp, #-16]!
> >
> > -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> > - mrs x1, esr_el2
> > -alternative_else
> > - mrs x1, esr_el1
> > -alternative_endif
> > - lsr x0, x1, #ESR_ELx_EC_SHIFT
> > -
> > - cmp x0, #ESR_ELx_EC_HVC64
> > - b.ne el1_trap
> > -
> > mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> > cbnz x1, el1_trap // called HVC
> >
> > +#ifdef CONFIG_DEBUG
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > + cmp x0, #ESR_ELx_EC_HVC64
> > + b.ne __hyp_panic
> > +#endif
>
> The dropping of the "alternative_if_not ARM64_HAS_VIRT_HOST_EXTN" stuff
> isn't called out in the commit message, but it looks like it was just
> a cleanup of code that was never necessary, as esr_el2 aliases esr_el1.
> Is that correct?
>
I suppose 'rearranging some of the code' is not a very precise
description. With VHE, using the 'mrs x1, esr_el1' instruction, will
actuall read ESR_EL2 into x1, and using the 'mrs x1, esr_el2'
instruction will have the same effect on VHE, so we might as well simply
use that.
I'll adjust the commit message.
Thanks for looking at this!
-Christoffer
> > +
> > /* Here, we're pretty sure the host called HVC. */
> > ldp x0, x1, [sp], #16
> >
> > @@ -101,10 +98,15 @@ alternative_endif
> > eret
> >
> > el1_trap:
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > +
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > /*
> > * x0: ESR_EC
> > + * x1: vcpu pointer
> > */
> > - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
> >
> > /*
> > * We trap the first access to the FP/SIMD to save the host context
> > @@ -122,13 +124,15 @@ alternative_else_nop_endif
> >
> > el1_irq:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_IRQ
> > b __guest_exit
> >
> > el1_error:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_EL1_SERROR
> > b __guest_exit
> >
> > @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> > ENDPROC(__hyp_do_panic)
> >
> > ENTRY(__hyp_panic)
> > - /*
> > - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > - * not be accessible by this address from EL2, hyp_panic() converts
> > - * it with kern_hyp_va() before use.
> > - */
> > - ldr x0, =kvm_host_cpu_state
> > - mrs x1, tpidr_el2
> > - add x0, x0, x1
> > + get_host_ctxt x0, x1
> > b hyp_panic
> > ENDPROC(__hyp_panic)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 69ef24a..a0123ad 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> > if (read_sysreg(vttbr_el2)) {
> > struct kvm_cpu_context *host_ctxt;
> >
> > - host_ctxt = kern_hyp_va(__host_ctxt);
> > + host_ctxt = __host_ctxt;
> > vcpu = host_ctxt->__hyp_running_vcpu;
> > __timer_disable_traps(vcpu);
> > __deactivate_traps(vcpu);
> > --
> > 2.9.0
> >
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 01/37] KVM: arm64: Avoid storing the vcpu pointer on the stack
@ 2017-11-07 8:24 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-07 8:24 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 06, 2017 at 06:22:51PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:05PM +0200, Christoffer Dall wrote:
> > We already have the percpu area for the host cpu state, which points to
> > the VCPU, so there's no need to store the VCPU pointer on the stack on
> > every context switch. We can be a little more clever and just use
> > tpidr_el2 for the percpu offset and load the VCPU pointer from the host
> > context.
> >
> > This requires us to have a scratch register though, so we take the
> > chance to rearrange some of the el1_sync code to only look at the
> > vttbr_el2 to determine if this is a trap from the guest or an HVC from
> > the host. We do add an extra check to call the panic code if the kernel
> > is configured with debugging enabled and we saw a trap from the host
> > which wasn't an HVC, indicating that we left some EL2 trap configured by
> > mistake.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 20 ++++++++++++++++++++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 5 +----
> > arch/arm64/kvm/hyp/hyp-entry.S | 39 ++++++++++++++++++---------------------
> > arch/arm64/kvm/hyp/switch.c | 2 +-
> > 5 files changed, 41 insertions(+), 26 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index ab4d0a9..7e48a39 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -70,4 +70,24 @@ extern u32 __init_stage2_translation(void);
> >
> > #endif
> >
> > +#ifdef __ASSEMBLY__
> > +.macro get_host_ctxt reg, tmp
> > + /*
> > + * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > + * not be accessible by this address from EL2, hyp_panic() converts
> > + * it with kern_hyp_va() before use.
> > + */
> > + ldr \reg, =kvm_host_cpu_state
> > + mrs \tmp, tpidr_el2
> > + add \reg, \reg, \tmp
> > + kern_hyp_va \reg
> > +.endm
> > +
> > +.macro get_vcpu vcpu, ctxt
> > + ldr \vcpu, [\ctxt, #HOST_CONTEXT_VCPU]
> > + kern_hyp_va \vcpu
> > +.endm
>
> To avoid the need for the pattern
>
> get_host_ctxt x0, x1
> get_vcpu x1, x0
>
> everywhere this macro is used, how about defining it as
>
> .macro get_vcpu vcpu, tmp
> get_host_ctxt \tmp, \vcpu
> ldr \vcpu, [\tmp, #HOST_CONTEXT_VCPU]
> kern_hyp_va \vcpu
> .endm
>
> which also has the side-effect of tmp being ctxt after the call.
>
I actually prefer it the way it is now, because I think it's clearer
what's going on. In the past we did some information hiding in assembly
macros and that didn't improve maintenance.
If there's an aspect to this I'm missing, I'm of course open to
reconsiderig though.
> > +
> > +#endif
> > +
> > #endif /* __ARM_KVM_ASM_H__ */
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 71bf088..612021d 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -135,6 +135,7 @@ int main(void)
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > + DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > #endif
> > #ifdef CONFIG_CPU_PM
> > DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index 9a8ab5d..76cd48f 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -62,9 +62,6 @@ ENTRY(__guest_enter)
> > // Store the host regs
> > save_callee_saved_regs x1
> >
> > - // Store host_ctxt and vcpu for use at exit time
> > - stp x1, x0, [sp, #-16]!
> > -
> > add x18, x0, #VCPU_CONTEXT
> >
> > // Restore guest regs x0-x17
> > @@ -119,7 +116,7 @@ ENTRY(__guest_exit)
> > save_callee_saved_regs x1
> >
> > // Restore the host_ctxt from the stack
> > - ldr x2, [sp], #16
> > + get_host_ctxt x2, x3
> >
> > // Now restore the host regs
> > restore_callee_saved_regs x2
> > diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> > index e4f37b9..2950f26 100644
> > --- a/arch/arm64/kvm/hyp/hyp-entry.S
> > +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> > @@ -56,19 +56,16 @@ ENDPROC(__vhe_hyp_call)
> > el1_sync: // Guest trapped into EL2
> > stp x0, x1, [sp, #-16]!
> >
> > -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN
> > - mrs x1, esr_el2
> > -alternative_else
> > - mrs x1, esr_el1
> > -alternative_endif
> > - lsr x0, x1, #ESR_ELx_EC_SHIFT
> > -
> > - cmp x0, #ESR_ELx_EC_HVC64
> > - b.ne el1_trap
> > -
> > mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest
> > cbnz x1, el1_trap // called HVC
> >
> > +#ifdef CONFIG_DEBUG
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > + cmp x0, #ESR_ELx_EC_HVC64
> > + b.ne __hyp_panic
> > +#endif
>
> The dropping of the "alternative_if_not ARM64_HAS_VIRT_HOST_EXTN" stuff
> isn't called out in the commit message, but it looks like it was just
> a cleanup of code that was never necessary, as esr_el2 aliases esr_el1.
> Is that correct?
>
I suppose 'rearranging some of the code' is not a very precise
description. With VHE, using the 'mrs x1, esr_el1' instruction, will
actuall read ESR_EL2 into x1, and using the 'mrs x1, esr_el2'
instruction will have the same effect on VHE, so we might as well simply
use that.
I'll adjust the commit message.
Thanks for looking at this!
-Christoffer
> > +
> > /* Here, we're pretty sure the host called HVC. */
> > ldp x0, x1, [sp], #16
> >
> > @@ -101,10 +98,15 @@ alternative_endif
> > eret
> >
> > el1_trap:
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > +
> > + mrs x0, esr_el2
> > + lsr x0, x0, #ESR_ELx_EC_SHIFT
> > /*
> > * x0: ESR_EC
> > + * x1: vcpu pointer
> > */
> > - ldr x1, [sp, #16 + 8] // vcpu stored by __guest_enter
> >
> > /*
> > * We trap the first access to the FP/SIMD to save the host context
> > @@ -122,13 +124,15 @@ alternative_else_nop_endif
> >
> > el1_irq:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_IRQ
> > b __guest_exit
> >
> > el1_error:
> > stp x0, x1, [sp, #-16]!
> > - ldr x1, [sp, #16 + 8]
> > + get_host_ctxt x0, x1
> > + get_vcpu x1, x0
> > mov x0, #ARM_EXCEPTION_EL1_SERROR
> > b __guest_exit
> >
> > @@ -164,14 +168,7 @@ ENTRY(__hyp_do_panic)
> > ENDPROC(__hyp_do_panic)
> >
> > ENTRY(__hyp_panic)
> > - /*
> > - * '=kvm_host_cpu_state' is a host VA from the constant pool, it may
> > - * not be accessible by this address from EL2, hyp_panic() converts
> > - * it with kern_hyp_va() before use.
> > - */
> > - ldr x0, =kvm_host_cpu_state
> > - mrs x1, tpidr_el2
> > - add x0, x0, x1
> > + get_host_ctxt x0, x1
> > b hyp_panic
> > ENDPROC(__hyp_panic)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 69ef24a..a0123ad 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -435,7 +435,7 @@ void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> > if (read_sysreg(vttbr_el2)) {
> > struct kvm_cpu_context *host_ctxt;
> >
> > - host_ctxt = kern_hyp_va(__host_ctxt);
> > + host_ctxt = __host_ctxt;
> > vcpu = host_ctxt->__hyp_running_vcpu;
> > __timer_disable_traps(vcpu);
> > __deactivate_traps(vcpu);
> > --
> > 2.9.0
> >
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 02/37] KVM: arm64: Rework hyp_panic for VHE and non-VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
VHE actually doesn't rely on clearing the VTTBR when returning to the
hsot kernel, and that is the current key mechanism of hyp_panic to
figure out how to attempt to return to a state good enough to print a
panic statement.
Therefore, we split the hyp_panic function into two functions, a VHE and
a non-VHE, keeping the non-VHE version intact, but changing the VHE
behavior.
The vttbr_el2 check on VHE doesn't really make that much sense, because
the only situation where we can get here on VHE is when the hypervisor
assembly code actually caleld into hyp_panic, which only happens when
VBAR_EL2 has been set to the KVM exception vectors. On VHE, we can
always safely disable the traps and restore the host registers at this
point, so we simply do that unconditionally and call into the panic
function directly.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 45 +++++++++++++++++++++++----------------------
1 file changed, 23 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index a0123ad..a50ddf3 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -394,10 +394,20 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
- struct kvm_vcpu *vcpu)
+ struct kvm_cpu_context *__host_ctxt)
{
+ struct kvm_vcpu *vcpu;
unsigned long str_va;
+ vcpu = __host_ctxt->__hyp_running_vcpu;
+
+ if (read_sysreg(vttbr_el2)) {
+ __timer_disable_traps(vcpu);
+ __deactivate_traps(vcpu);
+ __deactivate_vm(vcpu);
+ __sysreg_restore_host_state(__host_ctxt);
+ }
+
/*
* Force the panic string to be loaded from the literal pool,
* making sure it is a kernel address and not a PC-relative
@@ -411,40 +421,31 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
read_sysreg(hpfar_el2), par, vcpu);
}
-static void __hyp_text __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
- struct kvm_vcpu *vcpu)
+static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
+ struct kvm_cpu_context *host_ctxt)
{
+ struct kvm_vcpu *vcpu;
+ vcpu = host_ctxt->__hyp_running_vcpu;
+
+ __deactivate_traps_vhe();
+ __sysreg_restore_host_state(host_ctxt);
+
panic(__hyp_panic_string,
spsr, elr,
read_sysreg_el2(esr), read_sysreg_el2(far),
read_sysreg(hpfar_el2), par, vcpu);
}
-static hyp_alternate_select(__hyp_call_panic,
- __hyp_call_panic_nvhe, __hyp_call_panic_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
{
- struct kvm_vcpu *vcpu = NULL;
-
u64 spsr = read_sysreg_el2(spsr);
u64 elr = read_sysreg_el2(elr);
u64 par = read_sysreg(par_el1);
- if (read_sysreg(vttbr_el2)) {
- struct kvm_cpu_context *host_ctxt;
-
- host_ctxt = __host_ctxt;
- vcpu = host_ctxt->__hyp_running_vcpu;
- __timer_disable_traps(vcpu);
- __deactivate_traps(vcpu);
- __deactivate_vm(vcpu);
- __sysreg_restore_host_state(host_ctxt);
- }
-
- /* Call panic for real */
- __hyp_call_panic()(spsr, elr, par, vcpu);
+ if (!has_vhe())
+ __hyp_call_panic_nvhe(spsr, elr, par, __host_ctxt);
+ else
+ __hyp_call_panic_vhe(spsr, elr, par, __host_ctxt);
unreachable();
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 02/37] KVM: arm64: Rework hyp_panic for VHE and non-VHE
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
VHE actually doesn't rely on clearing the VTTBR when returning to the
hsot kernel, and that is the current key mechanism of hyp_panic to
figure out how to attempt to return to a state good enough to print a
panic statement.
Therefore, we split the hyp_panic function into two functions, a VHE and
a non-VHE, keeping the non-VHE version intact, but changing the VHE
behavior.
The vttbr_el2 check on VHE doesn't really make that much sense, because
the only situation where we can get here on VHE is when the hypervisor
assembly code actually caleld into hyp_panic, which only happens when
VBAR_EL2 has been set to the KVM exception vectors. On VHE, we can
always safely disable the traps and restore the host registers at this
point, so we simply do that unconditionally and call into the panic
function directly.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 45 +++++++++++++++++++++++----------------------
1 file changed, 23 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index a0123ad..a50ddf3 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -394,10 +394,20 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
- struct kvm_vcpu *vcpu)
+ struct kvm_cpu_context *__host_ctxt)
{
+ struct kvm_vcpu *vcpu;
unsigned long str_va;
+ vcpu = __host_ctxt->__hyp_running_vcpu;
+
+ if (read_sysreg(vttbr_el2)) {
+ __timer_disable_traps(vcpu);
+ __deactivate_traps(vcpu);
+ __deactivate_vm(vcpu);
+ __sysreg_restore_host_state(__host_ctxt);
+ }
+
/*
* Force the panic string to be loaded from the literal pool,
* making sure it is a kernel address and not a PC-relative
@@ -411,40 +421,31 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
read_sysreg(hpfar_el2), par, vcpu);
}
-static void __hyp_text __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
- struct kvm_vcpu *vcpu)
+static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
+ struct kvm_cpu_context *host_ctxt)
{
+ struct kvm_vcpu *vcpu;
+ vcpu = host_ctxt->__hyp_running_vcpu;
+
+ __deactivate_traps_vhe();
+ __sysreg_restore_host_state(host_ctxt);
+
panic(__hyp_panic_string,
spsr, elr,
read_sysreg_el2(esr), read_sysreg_el2(far),
read_sysreg(hpfar_el2), par, vcpu);
}
-static hyp_alternate_select(__hyp_call_panic,
- __hyp_call_panic_nvhe, __hyp_call_panic_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
{
- struct kvm_vcpu *vcpu = NULL;
-
u64 spsr = read_sysreg_el2(spsr);
u64 elr = read_sysreg_el2(elr);
u64 par = read_sysreg(par_el1);
- if (read_sysreg(vttbr_el2)) {
- struct kvm_cpu_context *host_ctxt;
-
- host_ctxt = __host_ctxt;
- vcpu = host_ctxt->__hyp_running_vcpu;
- __timer_disable_traps(vcpu);
- __deactivate_traps(vcpu);
- __deactivate_vm(vcpu);
- __sysreg_restore_host_state(host_ctxt);
- }
-
- /* Call panic for real */
- __hyp_call_panic()(spsr, elr, par, vcpu);
+ if (!has_vhe())
+ __hyp_call_panic_nvhe(spsr, elr, par, __host_ctxt);
+ else
+ __hyp_call_panic_vhe(spsr, elr, par, __host_ctxt);
unreachable();
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 02/37] KVM: arm64: Rework hyp_panic for VHE and non-VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 15:55 ` Marc Zyngier
-1 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 15:55 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: kvm, Shih-Wei Li
On 12/10/17 11:41, Christoffer Dall wrote:
> VHE actually doesn't rely on clearing the VTTBR when returning to the
> hsot kernel, and that is the current key mechanism of hyp_panic to
host
> figure out how to attempt to return to a state good enough to print a
> panic statement.
>
> Therefore, we split the hyp_panic function into two functions, a VHE and
> a non-VHE, keeping the non-VHE version intact, but changing the VHE
> behavior.
>
> The vttbr_el2 check on VHE doesn't really make that much sense, because
> the only situation where we can get here on VHE is when the hypervisor
> assembly code actually caleld into hyp_panic, which only happens when
called
> VBAR_EL2 has been set to the KVM exception vectors. On VHE, we can
> always safely disable the traps and restore the host registers at this
> point, so we simply do that unconditionally and call into the panic
> function directly.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 45 +++++++++++++++++++++++----------------------
> 1 file changed, 23 insertions(+), 22 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index a0123ad..a50ddf3 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -394,10 +394,20 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
>
> static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> - struct kvm_vcpu *vcpu)
> + struct kvm_cpu_context *__host_ctxt)
> {
> + struct kvm_vcpu *vcpu;
> unsigned long str_va;
>
> + vcpu = __host_ctxt->__hyp_running_vcpu;
> +
> + if (read_sysreg(vttbr_el2)) {
> + __timer_disable_traps(vcpu);
> + __deactivate_traps(vcpu);
> + __deactivate_vm(vcpu);
> + __sysreg_restore_host_state(__host_ctxt);
> + }
> +
> /*
> * Force the panic string to be loaded from the literal pool,
> * making sure it is a kernel address and not a PC-relative
> @@ -411,40 +421,31 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> read_sysreg(hpfar_el2), par, vcpu);
> }
>
> -static void __hyp_text __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> - struct kvm_vcpu *vcpu)
> +static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> + struct kvm_cpu_context *host_ctxt)
> {
> + struct kvm_vcpu *vcpu;
> + vcpu = host_ctxt->__hyp_running_vcpu;
> +
> + __deactivate_traps_vhe();
Is there a reason why we can't just call __deactivate_traps(), rather
than the VHE-specific subset? It doesn't really matter, as we're about
to panic, but still...
> + __sysreg_restore_host_state(host_ctxt);
> +
> panic(__hyp_panic_string,
> spsr, elr,
> read_sysreg_el2(esr), read_sysreg_el2(far),
> read_sysreg(hpfar_el2), par, vcpu);
> }
>
> -static hyp_alternate_select(__hyp_call_panic,
> - __hyp_call_panic_nvhe, __hyp_call_panic_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> {
> - struct kvm_vcpu *vcpu = NULL;
> -
> u64 spsr = read_sysreg_el2(spsr);
> u64 elr = read_sysreg_el2(elr);
> u64 par = read_sysreg(par_el1);
>
> - if (read_sysreg(vttbr_el2)) {
> - struct kvm_cpu_context *host_ctxt;
> -
> - host_ctxt = __host_ctxt;
> - vcpu = host_ctxt->__hyp_running_vcpu;
> - __timer_disable_traps(vcpu);
> - __deactivate_traps(vcpu);
> - __deactivate_vm(vcpu);
> - __sysreg_restore_host_state(host_ctxt);
> - }
> -
> - /* Call panic for real */
> - __hyp_call_panic()(spsr, elr, par, vcpu);
> + if (!has_vhe())
> + __hyp_call_panic_nvhe(spsr, elr, par, __host_ctxt);
> + else
> + __hyp_call_panic_vhe(spsr, elr, par, __host_ctxt);
>
> unreachable();
> }
>
Otherwise looks good.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 02/37] KVM: arm64: Rework hyp_panic for VHE and non-VHE
@ 2017-10-12 15:55 ` Marc Zyngier
0 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 15:55 UTC (permalink / raw)
To: linux-arm-kernel
On 12/10/17 11:41, Christoffer Dall wrote:
> VHE actually doesn't rely on clearing the VTTBR when returning to the
> hsot kernel, and that is the current key mechanism of hyp_panic to
host
> figure out how to attempt to return to a state good enough to print a
> panic statement.
>
> Therefore, we split the hyp_panic function into two functions, a VHE and
> a non-VHE, keeping the non-VHE version intact, but changing the VHE
> behavior.
>
> The vttbr_el2 check on VHE doesn't really make that much sense, because
> the only situation where we can get here on VHE is when the hypervisor
> assembly code actually caleld into hyp_panic, which only happens when
called
> VBAR_EL2 has been set to the KVM exception vectors. On VHE, we can
> always safely disable the traps and restore the host registers at this
> point, so we simply do that unconditionally and call into the panic
> function directly.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 45 +++++++++++++++++++++++----------------------
> 1 file changed, 23 insertions(+), 22 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index a0123ad..a50ddf3 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -394,10 +394,20 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
>
> static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> - struct kvm_vcpu *vcpu)
> + struct kvm_cpu_context *__host_ctxt)
> {
> + struct kvm_vcpu *vcpu;
> unsigned long str_va;
>
> + vcpu = __host_ctxt->__hyp_running_vcpu;
> +
> + if (read_sysreg(vttbr_el2)) {
> + __timer_disable_traps(vcpu);
> + __deactivate_traps(vcpu);
> + __deactivate_vm(vcpu);
> + __sysreg_restore_host_state(__host_ctxt);
> + }
> +
> /*
> * Force the panic string to be loaded from the literal pool,
> * making sure it is a kernel address and not a PC-relative
> @@ -411,40 +421,31 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> read_sysreg(hpfar_el2), par, vcpu);
> }
>
> -static void __hyp_text __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> - struct kvm_vcpu *vcpu)
> +static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> + struct kvm_cpu_context *host_ctxt)
> {
> + struct kvm_vcpu *vcpu;
> + vcpu = host_ctxt->__hyp_running_vcpu;
> +
> + __deactivate_traps_vhe();
Is there a reason why we can't just call __deactivate_traps(), rather
than the VHE-specific subset? It doesn't really matter, as we're about
to panic, but still...
> + __sysreg_restore_host_state(host_ctxt);
> +
> panic(__hyp_panic_string,
> spsr, elr,
> read_sysreg_el2(esr), read_sysreg_el2(far),
> read_sysreg(hpfar_el2), par, vcpu);
> }
>
> -static hyp_alternate_select(__hyp_call_panic,
> - __hyp_call_panic_nvhe, __hyp_call_panic_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> {
> - struct kvm_vcpu *vcpu = NULL;
> -
> u64 spsr = read_sysreg_el2(spsr);
> u64 elr = read_sysreg_el2(elr);
> u64 par = read_sysreg(par_el1);
>
> - if (read_sysreg(vttbr_el2)) {
> - struct kvm_cpu_context *host_ctxt;
> -
> - host_ctxt = __host_ctxt;
> - vcpu = host_ctxt->__hyp_running_vcpu;
> - __timer_disable_traps(vcpu);
> - __deactivate_traps(vcpu);
> - __deactivate_vm(vcpu);
> - __sysreg_restore_host_state(host_ctxt);
> - }
> -
> - /* Call panic for real */
> - __hyp_call_panic()(spsr, elr, par, vcpu);
> + if (!has_vhe())
> + __hyp_call_panic_nvhe(spsr, elr, par, __host_ctxt);
> + else
> + __hyp_call_panic_vhe(spsr, elr, par, __host_ctxt);
>
> unreachable();
> }
>
Otherwise looks good.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 02/37] KVM: arm64: Rework hyp_panic for VHE and non-VHE
2017-10-12 15:55 ` Marc Zyngier
@ 2017-10-12 17:06 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 17:06 UTC (permalink / raw)
To: Marc Zyngier; +Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Shih-Wei Li
On Thu, Oct 12, 2017 at 04:55:16PM +0100, Marc Zyngier wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> > VHE actually doesn't rely on clearing the VTTBR when returning to the
> > hsot kernel, and that is the current key mechanism of hyp_panic to
>
> host
>
> > figure out how to attempt to return to a state good enough to print a
> > panic statement.
> >
> > Therefore, we split the hyp_panic function into two functions, a VHE and
> > a non-VHE, keeping the non-VHE version intact, but changing the VHE
> > behavior.
> >
> > The vttbr_el2 check on VHE doesn't really make that much sense, because
> > the only situation where we can get here on VHE is when the hypervisor
> > assembly code actually caleld into hyp_panic, which only happens when
>
> called
>
> > VBAR_EL2 has been set to the KVM exception vectors. On VHE, we can
> > always safely disable the traps and restore the host registers at this
> > point, so we simply do that unconditionally and call into the panic
> > function directly.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 45 +++++++++++++++++++++++----------------------
> > 1 file changed, 23 insertions(+), 22 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index a0123ad..a50ddf3 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -394,10 +394,20 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
> >
> > static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> > - struct kvm_vcpu *vcpu)
> > + struct kvm_cpu_context *__host_ctxt)
> > {
> > + struct kvm_vcpu *vcpu;
> > unsigned long str_va;
> >
> > + vcpu = __host_ctxt->__hyp_running_vcpu;
> > +
> > + if (read_sysreg(vttbr_el2)) {
> > + __timer_disable_traps(vcpu);
> > + __deactivate_traps(vcpu);
> > + __deactivate_vm(vcpu);
> > + __sysreg_restore_host_state(__host_ctxt);
> > + }
> > +
> > /*
> > * Force the panic string to be loaded from the literal pool,
> > * making sure it is a kernel address and not a PC-relative
> > @@ -411,40 +421,31 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> > read_sysreg(hpfar_el2), par, vcpu);
> > }
> >
> > -static void __hyp_text __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> > - struct kvm_vcpu *vcpu)
> > +static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> > + struct kvm_cpu_context *host_ctxt)
> > {
> > + struct kvm_vcpu *vcpu;
> > + vcpu = host_ctxt->__hyp_running_vcpu;
> > +
> > + __deactivate_traps_vhe();
>
> Is there a reason why we can't just call __deactivate_traps(), rather
> than the VHE-specific subset? It doesn't really matter, as we're about
> to panic, but still...
>
It doesn't really matter, especially as later patches will change this,
but this patch would be slightly nicer keeping the __deactivate_traps.
I don't mind changing that around in the next version.
> > + __sysreg_restore_host_state(host_ctxt);
> > +
> > panic(__hyp_panic_string,
> > spsr, elr,
> > read_sysreg_el2(esr), read_sysreg_el2(far),
> > read_sysreg(hpfar_el2), par, vcpu);
> > }
> >
> > -static hyp_alternate_select(__hyp_call_panic,
> > - __hyp_call_panic_nvhe, __hyp_call_panic_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> > {
> > - struct kvm_vcpu *vcpu = NULL;
> > -
> > u64 spsr = read_sysreg_el2(spsr);
> > u64 elr = read_sysreg_el2(elr);
> > u64 par = read_sysreg(par_el1);
> >
> > - if (read_sysreg(vttbr_el2)) {
> > - struct kvm_cpu_context *host_ctxt;
> > -
> > - host_ctxt = __host_ctxt;
> > - vcpu = host_ctxt->__hyp_running_vcpu;
> > - __timer_disable_traps(vcpu);
> > - __deactivate_traps(vcpu);
> > - __deactivate_vm(vcpu);
> > - __sysreg_restore_host_state(host_ctxt);
> > - }
> > -
> > - /* Call panic for real */
> > - __hyp_call_panic()(spsr, elr, par, vcpu);
> > + if (!has_vhe())
> > + __hyp_call_panic_nvhe(spsr, elr, par, __host_ctxt);
> > + else
> > + __hyp_call_panic_vhe(spsr, elr, par, __host_ctxt);
> >
> > unreachable();
> > }
> >
>
> Otherwise looks good.
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 02/37] KVM: arm64: Rework hyp_panic for VHE and non-VHE
@ 2017-10-12 17:06 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 17:06 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 04:55:16PM +0100, Marc Zyngier wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> > VHE actually doesn't rely on clearing the VTTBR when returning to the
> > hsot kernel, and that is the current key mechanism of hyp_panic to
>
> host
>
> > figure out how to attempt to return to a state good enough to print a
> > panic statement.
> >
> > Therefore, we split the hyp_panic function into two functions, a VHE and
> > a non-VHE, keeping the non-VHE version intact, but changing the VHE
> > behavior.
> >
> > The vttbr_el2 check on VHE doesn't really make that much sense, because
> > the only situation where we can get here on VHE is when the hypervisor
> > assembly code actually caleld into hyp_panic, which only happens when
>
> called
>
> > VBAR_EL2 has been set to the KVM exception vectors. On VHE, we can
> > always safely disable the traps and restore the host registers at this
> > point, so we simply do that unconditionally and call into the panic
> > function directly.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 45 +++++++++++++++++++++++----------------------
> > 1 file changed, 23 insertions(+), 22 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index a0123ad..a50ddf3 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -394,10 +394,20 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
> >
> > static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> > - struct kvm_vcpu *vcpu)
> > + struct kvm_cpu_context *__host_ctxt)
> > {
> > + struct kvm_vcpu *vcpu;
> > unsigned long str_va;
> >
> > + vcpu = __host_ctxt->__hyp_running_vcpu;
> > +
> > + if (read_sysreg(vttbr_el2)) {
> > + __timer_disable_traps(vcpu);
> > + __deactivate_traps(vcpu);
> > + __deactivate_vm(vcpu);
> > + __sysreg_restore_host_state(__host_ctxt);
> > + }
> > +
> > /*
> > * Force the panic string to be loaded from the literal pool,
> > * making sure it is a kernel address and not a PC-relative
> > @@ -411,40 +421,31 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> > read_sysreg(hpfar_el2), par, vcpu);
> > }
> >
> > -static void __hyp_text __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> > - struct kvm_vcpu *vcpu)
> > +static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> > + struct kvm_cpu_context *host_ctxt)
> > {
> > + struct kvm_vcpu *vcpu;
> > + vcpu = host_ctxt->__hyp_running_vcpu;
> > +
> > + __deactivate_traps_vhe();
>
> Is there a reason why we can't just call __deactivate_traps(), rather
> than the VHE-specific subset? It doesn't really matter, as we're about
> to panic, but still...
>
It doesn't really matter, especially as later patches will change this,
but this patch would be slightly nicer keeping the __deactivate_traps.
I don't mind changing that around in the next version.
> > + __sysreg_restore_host_state(host_ctxt);
> > +
> > panic(__hyp_panic_string,
> > spsr, elr,
> > read_sysreg_el2(esr), read_sysreg_el2(far),
> > read_sysreg(hpfar_el2), par, vcpu);
> > }
> >
> > -static hyp_alternate_select(__hyp_call_panic,
> > - __hyp_call_panic_nvhe, __hyp_call_panic_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *__host_ctxt)
> > {
> > - struct kvm_vcpu *vcpu = NULL;
> > -
> > u64 spsr = read_sysreg_el2(spsr);
> > u64 elr = read_sysreg_el2(elr);
> > u64 par = read_sysreg(par_el1);
> >
> > - if (read_sysreg(vttbr_el2)) {
> > - struct kvm_cpu_context *host_ctxt;
> > -
> > - host_ctxt = __host_ctxt;
> > - vcpu = host_ctxt->__hyp_running_vcpu;
> > - __timer_disable_traps(vcpu);
> > - __deactivate_traps(vcpu);
> > - __deactivate_vm(vcpu);
> > - __sysreg_restore_host_state(host_ctxt);
> > - }
> > -
> > - /* Call panic for real */
> > - __hyp_call_panic()(spsr, elr, par, vcpu);
> > + if (!has_vhe())
> > + __hyp_call_panic_nvhe(spsr, elr, par, __host_ctxt);
> > + else
> > + __hyp_call_panic_vhe(spsr, elr, par, __host_ctxt);
> >
> > unreachable();
> > }
> >
>
> Otherwise looks good.
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 03/37] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
From: Shih-Wei Li <shihwei@cs.columbia.edu>
We always set the IMO and FMO bits in the HCR_EL2 when running the
guest, regardless if we use the vgic or not. By moving these flags to
HCR_GUEST_FLAGS we can avoid one of the extra save/restore operations of
HCR_EL2 in the world switch code, and we can also soon get rid of the
other one.
Signed-off-by: Shih-Wei Li <shihwei@cs.columbia.edu>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_arm.h | 4 ++--
arch/arm64/kvm/hyp/switch.c | 3 ---
2 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 61d694c..e67e279 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -79,9 +79,9 @@
*/
#define HCR_GUEST_FLAGS (HCR_TSC | HCR_TSW | HCR_TWE | HCR_TWI | HCR_VM | \
HCR_TVM | HCR_BSU_IS | HCR_FB | HCR_TAC | \
- HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW)
+ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | \
+ HCR_FMO | HCR_IMO)
#define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF)
-#define HCR_INT_OVERRIDE (HCR_FMO | HCR_IMO)
#define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
/* TCR_EL2 Registers bits */
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index a50ddf3..bcf1a79 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -164,8 +164,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
__vgic_v3_save_state(vcpu);
else
__vgic_v2_save_state(vcpu);
-
- write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
@@ -173,7 +171,6 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
u64 val;
val = read_sysreg(hcr_el2);
- val |= HCR_INT_OVERRIDE;
val |= vcpu->arch.irq_lines;
write_sysreg(val, hcr_el2);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 03/37] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
From: Shih-Wei Li <shihwei@cs.columbia.edu>
We always set the IMO and FMO bits in the HCR_EL2 when running the
guest, regardless if we use the vgic or not. By moving these flags to
HCR_GUEST_FLAGS we can avoid one of the extra save/restore operations of
HCR_EL2 in the world switch code, and we can also soon get rid of the
other one.
Signed-off-by: Shih-Wei Li <shihwei@cs.columbia.edu>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_arm.h | 4 ++--
arch/arm64/kvm/hyp/switch.c | 3 ---
2 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 61d694c..e67e279 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -79,9 +79,9 @@
*/
#define HCR_GUEST_FLAGS (HCR_TSC | HCR_TSW | HCR_TWE | HCR_TWI | HCR_VM | \
HCR_TVM | HCR_BSU_IS | HCR_FB | HCR_TAC | \
- HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW)
+ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | \
+ HCR_FMO | HCR_IMO)
#define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF)
-#define HCR_INT_OVERRIDE (HCR_FMO | HCR_IMO)
#define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
/* TCR_EL2 Registers bits */
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index a50ddf3..bcf1a79 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -164,8 +164,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
__vgic_v3_save_state(vcpu);
else
__vgic_v2_save_state(vcpu);
-
- write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
@@ -173,7 +171,6 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
u64 val;
val = read_sysreg(hcr_el2);
- val |= HCR_INT_OVERRIDE;
val |= vcpu->arch.irq_lines;
write_sysreg(val, hcr_el2);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* Re: [PATCH 03/37] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 16:20 ` Marc Zyngier
-1 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 16:20 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Shih-Wei Li, kvm
On 12/10/17 11:41, Christoffer Dall wrote:
> From: Shih-Wei Li <shihwei@cs.columbia.edu>
>
> We always set the IMO and FMO bits in the HCR_EL2 when running the
> guest, regardless if we use the vgic or not. By moving these flags to
> HCR_GUEST_FLAGS we can avoid one of the extra save/restore operations of
> HCR_EL2 in the world switch code, and we can also soon get rid of the
> other one.
>
> Signed-off-by: Shih-Wei Li <shihwei@cs.columbia.edu>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_arm.h | 4 ++--
> arch/arm64/kvm/hyp/switch.c | 3 ---
> 2 files changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 61d694c..e67e279 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -79,9 +79,9 @@
> */
> #define HCR_GUEST_FLAGS (HCR_TSC | HCR_TSW | HCR_TWE | HCR_TWI | HCR_VM | \
> HCR_TVM | HCR_BSU_IS | HCR_FB | HCR_TAC | \
> - HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW)
> + HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | \
> + HCR_FMO | HCR_IMO)
> #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF)
> -#define HCR_INT_OVERRIDE (HCR_FMO | HCR_IMO)
> #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
>
> /* TCR_EL2 Registers bits */
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index a50ddf3..bcf1a79 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -164,8 +164,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> __vgic_v3_save_state(vcpu);
> else
> __vgic_v2_save_state(vcpu);
> -
> - write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
> }
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> @@ -173,7 +171,6 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> u64 val;
>
> val = read_sysreg(hcr_el2);
> - val |= HCR_INT_OVERRIDE;
> val |= vcpu->arch.irq_lines;
> write_sysreg(val, hcr_el2);
>
>
To expand on why this is actually safe: IMO/FMO control both taking the
interrupts to EL2 and remapping ICC_*_EL1 to ICV_*_EL1 executed at EL1.
As long as we ensure that these bits are clear when returning to the EL1
host, we're OK.
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 03/37] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag
@ 2017-10-12 16:20 ` Marc Zyngier
0 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 16:20 UTC (permalink / raw)
To: linux-arm-kernel
On 12/10/17 11:41, Christoffer Dall wrote:
> From: Shih-Wei Li <shihwei@cs.columbia.edu>
>
> We always set the IMO and FMO bits in the HCR_EL2 when running the
> guest, regardless if we use the vgic or not. By moving these flags to
> HCR_GUEST_FLAGS we can avoid one of the extra save/restore operations of
> HCR_EL2 in the world switch code, and we can also soon get rid of the
> other one.
>
> Signed-off-by: Shih-Wei Li <shihwei@cs.columbia.edu>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_arm.h | 4 ++--
> arch/arm64/kvm/hyp/switch.c | 3 ---
> 2 files changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 61d694c..e67e279 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -79,9 +79,9 @@
> */
> #define HCR_GUEST_FLAGS (HCR_TSC | HCR_TSW | HCR_TWE | HCR_TWI | HCR_VM | \
> HCR_TVM | HCR_BSU_IS | HCR_FB | HCR_TAC | \
> - HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW)
> + HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | \
> + HCR_FMO | HCR_IMO)
> #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF)
> -#define HCR_INT_OVERRIDE (HCR_FMO | HCR_IMO)
> #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
>
> /* TCR_EL2 Registers bits */
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index a50ddf3..bcf1a79 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -164,8 +164,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> __vgic_v3_save_state(vcpu);
> else
> __vgic_v2_save_state(vcpu);
> -
> - write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
> }
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> @@ -173,7 +171,6 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> u64 val;
>
> val = read_sysreg(hcr_el2);
> - val |= HCR_INT_OVERRIDE;
> val |= vcpu->arch.irq_lines;
> write_sysreg(val, hcr_el2);
>
>
To expand on why this is actually safe: IMO/FMO control both taking the
interrupts to EL2 and remapping ICC_*_EL1 to ICV_*_EL1 executed at EL1.
As long as we ensure that these bits are clear when returning to the EL1
host, we're OK.
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
We currently have a separate read-modify-write of the HCR_EL2 on entry
to the guest for the sole purpose of setting the VF and VI bits, if set.
Since this is most rarely the case (only when using userspace IRQ chip
and interrupts are in flight), let's get rid of this operation and
instead modify the bits in the vcpu->arch.hcr[_el2] directly when
needed.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_emulate.h | 9 ++-------
arch/arm/include/asm/kvm_host.h | 3 ---
arch/arm/kvm/emulate.c | 2 +-
arch/arm/kvm/hyp/switch.c | 2 +-
arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
arch/arm64/include/asm/kvm_host.h | 3 ---
arch/arm64/kvm/hyp/switch.c | 6 ------
arch/arm64/kvm/inject_fault.c | 2 +-
virt/kvm/arm/arm.c | 11 ++++++-----
virt/kvm/arm/mmu.c | 6 +++---
10 files changed, 16 insertions(+), 37 deletions(-)
diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
index 98089ff..34663a8 100644
--- a/arch/arm/include/asm/kvm_emulate.h
+++ b/arch/arm/include/asm/kvm_emulate.h
@@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
vcpu->arch.hcr = HCR_GUEST_MASK;
}
-static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
+static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
{
- return vcpu->arch.hcr;
-}
-
-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
-{
- vcpu->arch.hcr = hcr;
+ return (unsigned long *)&vcpu->arch.hcr;
}
static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 4a879f6..1100170 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
/* HYP trapping configuration */
u32 hcr;
- /* Interrupt related fields */
- u32 irq_lines; /* IRQ and FIQ levels */
-
/* Exception Information */
struct kvm_vcpu_fault_info fault;
diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
index 0064b86..4286a89 100644
--- a/arch/arm/kvm/emulate.c
+++ b/arch/arm/kvm/emulate.c
@@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
*/
void kvm_inject_vabt(struct kvm_vcpu *vcpu)
{
- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
+ *vcpu_hcr(vcpu) |= HCR_VA;
}
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index 330c9ce..c3b9799 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
isb();
}
- write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
+ write_sysreg(vcpu->arch.hcr, HCR);
/* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
write_sysreg(HSTR_T(15), HSTR);
write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index e5df3fc..1fbfe96 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 &= ~HCR_RW;
}
-static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
+static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
{
- return vcpu->arch.hcr_el2;
-}
-
-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
-{
- vcpu->arch.hcr_el2 = hcr;
+ return (unsigned long *)&vcpu->arch.hcr_el2;
}
static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 806ccef..27305e7 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
/* IO related fields */
struct kvm_decode mmio_decode;
- /* Interrupt related fields */
- u64 irq_lines; /* IRQ and FIQ levels */
-
/* Cache some mmu pages needed inside spinlock regions */
struct kvm_mmu_memory_cache mmu_page_cache;
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index bcf1a79..7703d63 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
- u64 val;
-
- val = read_sysreg(hcr_el2);
- val |= vcpu->arch.irq_lines;
- write_sysreg(val, hcr_el2);
-
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_restore_state(vcpu);
else
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index da6a8cf..45c7026 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
*/
void kvm_inject_vabt(struct kvm_vcpu *vcpu)
{
- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
+ *vcpu_hcr(vcpu) |= HCR_VSE;
}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 7f9296a..6e9513e 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
*/
int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
{
- return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
+ bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
+ return ((irq_lines || kvm_vgic_vcpu_pending_irq(v))
&& !v->arch.power_off && !v->arch.pause);
}
@@ -771,18 +772,18 @@ static int vcpu_interrupt_line(struct kvm_vcpu *vcpu, int number, bool level)
{
int bit_index;
bool set;
- unsigned long *ptr;
+ unsigned long *hcr;
if (number == KVM_ARM_IRQ_CPU_IRQ)
bit_index = __ffs(HCR_VI);
else /* KVM_ARM_IRQ_CPU_FIQ */
bit_index = __ffs(HCR_VF);
- ptr = (unsigned long *)&vcpu->arch.irq_lines;
+ hcr = vcpu_hcr(vcpu);
if (level)
- set = test_and_set_bit(bit_index, ptr);
+ set = test_and_set_bit(bit_index, hcr);
else
- set = test_and_clear_bit(bit_index, ptr);
+ set = test_and_clear_bit(bit_index, hcr);
/*
* If we didn't change anything, no need to wake up or kick other CPUs
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index b36945d..d93d56d 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -1987,7 +1987,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
*/
void kvm_set_way_flush(struct kvm_vcpu *vcpu)
{
- unsigned long hcr = vcpu_get_hcr(vcpu);
+ unsigned long hcr = *vcpu_hcr(vcpu);
/*
* If this is the first time we do a S/W operation
@@ -2002,7 +2002,7 @@ void kvm_set_way_flush(struct kvm_vcpu *vcpu)
trace_kvm_set_way_flush(*vcpu_pc(vcpu),
vcpu_has_cache_enabled(vcpu));
stage2_flush_vm(vcpu->kvm);
- vcpu_set_hcr(vcpu, hcr | HCR_TVM);
+ *vcpu_hcr(vcpu) = hcr | HCR_TVM;
}
}
@@ -2020,7 +2020,7 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
/* Caches are now on, stop trapping VM ops (until a S/W op) */
if (now_enabled)
- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) & ~HCR_TVM);
+ *vcpu_hcr(vcpu) &= ~HCR_TVM;
trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We currently have a separate read-modify-write of the HCR_EL2 on entry
to the guest for the sole purpose of setting the VF and VI bits, if set.
Since this is most rarely the case (only when using userspace IRQ chip
and interrupts are in flight), let's get rid of this operation and
instead modify the bits in the vcpu->arch.hcr[_el2] directly when
needed.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_emulate.h | 9 ++-------
arch/arm/include/asm/kvm_host.h | 3 ---
arch/arm/kvm/emulate.c | 2 +-
arch/arm/kvm/hyp/switch.c | 2 +-
arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
arch/arm64/include/asm/kvm_host.h | 3 ---
arch/arm64/kvm/hyp/switch.c | 6 ------
arch/arm64/kvm/inject_fault.c | 2 +-
virt/kvm/arm/arm.c | 11 ++++++-----
virt/kvm/arm/mmu.c | 6 +++---
10 files changed, 16 insertions(+), 37 deletions(-)
diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
index 98089ff..34663a8 100644
--- a/arch/arm/include/asm/kvm_emulate.h
+++ b/arch/arm/include/asm/kvm_emulate.h
@@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
vcpu->arch.hcr = HCR_GUEST_MASK;
}
-static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
+static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
{
- return vcpu->arch.hcr;
-}
-
-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
-{
- vcpu->arch.hcr = hcr;
+ return (unsigned long *)&vcpu->arch.hcr;
}
static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 4a879f6..1100170 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
/* HYP trapping configuration */
u32 hcr;
- /* Interrupt related fields */
- u32 irq_lines; /* IRQ and FIQ levels */
-
/* Exception Information */
struct kvm_vcpu_fault_info fault;
diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
index 0064b86..4286a89 100644
--- a/arch/arm/kvm/emulate.c
+++ b/arch/arm/kvm/emulate.c
@@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
*/
void kvm_inject_vabt(struct kvm_vcpu *vcpu)
{
- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
+ *vcpu_hcr(vcpu) |= HCR_VA;
}
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index 330c9ce..c3b9799 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
isb();
}
- write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
+ write_sysreg(vcpu->arch.hcr, HCR);
/* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
write_sysreg(HSTR_T(15), HSTR);
write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index e5df3fc..1fbfe96 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 &= ~HCR_RW;
}
-static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
+static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
{
- return vcpu->arch.hcr_el2;
-}
-
-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
-{
- vcpu->arch.hcr_el2 = hcr;
+ return (unsigned long *)&vcpu->arch.hcr_el2;
}
static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 806ccef..27305e7 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
/* IO related fields */
struct kvm_decode mmio_decode;
- /* Interrupt related fields */
- u64 irq_lines; /* IRQ and FIQ levels */
-
/* Cache some mmu pages needed inside spinlock regions */
struct kvm_mmu_memory_cache mmu_page_cache;
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index bcf1a79..7703d63 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
- u64 val;
-
- val = read_sysreg(hcr_el2);
- val |= vcpu->arch.irq_lines;
- write_sysreg(val, hcr_el2);
-
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_restore_state(vcpu);
else
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index da6a8cf..45c7026 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
*/
void kvm_inject_vabt(struct kvm_vcpu *vcpu)
{
- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
+ *vcpu_hcr(vcpu) |= HCR_VSE;
}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 7f9296a..6e9513e 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
*/
int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
{
- return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
+ bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
+ return ((irq_lines || kvm_vgic_vcpu_pending_irq(v))
&& !v->arch.power_off && !v->arch.pause);
}
@@ -771,18 +772,18 @@ static int vcpu_interrupt_line(struct kvm_vcpu *vcpu, int number, bool level)
{
int bit_index;
bool set;
- unsigned long *ptr;
+ unsigned long *hcr;
if (number == KVM_ARM_IRQ_CPU_IRQ)
bit_index = __ffs(HCR_VI);
else /* KVM_ARM_IRQ_CPU_FIQ */
bit_index = __ffs(HCR_VF);
- ptr = (unsigned long *)&vcpu->arch.irq_lines;
+ hcr = vcpu_hcr(vcpu);
if (level)
- set = test_and_set_bit(bit_index, ptr);
+ set = test_and_set_bit(bit_index, hcr);
else
- set = test_and_clear_bit(bit_index, ptr);
+ set = test_and_clear_bit(bit_index, hcr);
/*
* If we didn't change anything, no need to wake up or kick other CPUs
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index b36945d..d93d56d 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -1987,7 +1987,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
*/
void kvm_set_way_flush(struct kvm_vcpu *vcpu)
{
- unsigned long hcr = vcpu_get_hcr(vcpu);
+ unsigned long hcr = *vcpu_hcr(vcpu);
/*
* If this is the first time we do a S/W operation
@@ -2002,7 +2002,7 @@ void kvm_set_way_flush(struct kvm_vcpu *vcpu)
trace_kvm_set_way_flush(*vcpu_pc(vcpu),
vcpu_has_cache_enabled(vcpu));
stage2_flush_vm(vcpu->kvm);
- vcpu_set_hcr(vcpu, hcr | HCR_TVM);
+ *vcpu_hcr(vcpu) = hcr | HCR_TVM;
}
}
@@ -2020,7 +2020,7 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
/* Caches are now on, stop trapping VM ops (until a S/W op) */
if (now_enabled)
- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) & ~HCR_TVM);
+ *vcpu_hcr(vcpu) &= ~HCR_TVM;
trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 16:24 ` Marc Zyngier
-1 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 16:24 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: kvm, Shih-Wei Li
On 12/10/17 11:41, Christoffer Dall wrote:
> We currently have a separate read-modify-write of the HCR_EL2 on entry
> to the guest for the sole purpose of setting the VF and VI bits, if set.
> Since this is most rarely the case (only when using userspace IRQ chip
> and interrupts are in flight), let's get rid of this operation and
> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> needed.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm/include/asm/kvm_host.h | 3 ---
> arch/arm/kvm/emulate.c | 2 +-
> arch/arm/kvm/hyp/switch.c | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm64/include/asm/kvm_host.h | 3 ---
> arch/arm64/kvm/hyp/switch.c | 6 ------
> arch/arm64/kvm/inject_fault.c | 2 +-
> virt/kvm/arm/arm.c | 11 ++++++-----
> virt/kvm/arm/mmu.c | 6 +++---
> 10 files changed, 16 insertions(+), 37 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
> index 98089ff..34663a8 100644
> --- a/arch/arm/include/asm/kvm_emulate.h
> +++ b/arch/arm/include/asm/kvm_emulate.h
> @@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr = HCR_GUEST_MASK;
> }
>
> -static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr = hcr;
> + return (unsigned long *)&vcpu->arch.hcr;
> }
>
> static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 4a879f6..1100170 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
> /* HYP trapping configuration */
> u32 hcr;
>
> - /* Interrupt related fields */
> - u32 irq_lines; /* IRQ and FIQ levels */
> -
> /* Exception Information */
> struct kvm_vcpu_fault_info fault;
>
> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> index 0064b86..4286a89 100644
> --- a/arch/arm/kvm/emulate.c
> +++ b/arch/arm/kvm/emulate.c
> @@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
> + *vcpu_hcr(vcpu) |= HCR_VA;
> }
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index 330c9ce..c3b9799 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
> isb();
> }
>
> - write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
> + write_sysreg(vcpu->arch.hcr, HCR);
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(HSTR_T(15), HSTR);
> write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e5df3fc..1fbfe96 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr_el2 &= ~HCR_RW;
> }
>
> -static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr_el2;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr_el2 = hcr;
> + return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 806ccef..27305e7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
> /* IO related fields */
> struct kvm_decode mmio_decode;
>
> - /* Interrupt related fields */
> - u64 irq_lines; /* IRQ and FIQ levels */
> -
> /* Cache some mmu pages needed inside spinlock regions */
> struct kvm_mmu_memory_cache mmu_page_cache;
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index bcf1a79..7703d63 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> - u64 val;
> -
> - val = read_sysreg(hcr_el2);
> - val |= vcpu->arch.irq_lines;
> - write_sysreg(val, hcr_el2);
> -
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> else
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index da6a8cf..45c7026 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
> + *vcpu_hcr(vcpu) |= HCR_VSE;
> }
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 7f9296a..6e9513e 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
> */
> int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
> {
> - return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
> + bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
> + return ((irq_lines || kvm_vgic_vcpu_pending_irq(v))
> && !v->arch.power_off && !v->arch.pause);
> }
>
> @@ -771,18 +772,18 @@ static int vcpu_interrupt_line(struct kvm_vcpu *vcpu, int number, bool level)
> {
> int bit_index;
> bool set;
> - unsigned long *ptr;
> + unsigned long *hcr;
>
> if (number == KVM_ARM_IRQ_CPU_IRQ)
> bit_index = __ffs(HCR_VI);
> else /* KVM_ARM_IRQ_CPU_FIQ */
> bit_index = __ffs(HCR_VF);
>
> - ptr = (unsigned long *)&vcpu->arch.irq_lines;
> + hcr = vcpu_hcr(vcpu);
> if (level)
> - set = test_and_set_bit(bit_index, ptr);
> + set = test_and_set_bit(bit_index, hcr);
> else
> - set = test_and_clear_bit(bit_index, ptr);
> + set = test_and_clear_bit(bit_index, hcr);
>
> /*
> * If we didn't change anything, no need to wake up or kick other CPUs
> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
> index b36945d..d93d56d 100644
> --- a/virt/kvm/arm/mmu.c
> +++ b/virt/kvm/arm/mmu.c
> @@ -1987,7 +1987,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> */
> void kvm_set_way_flush(struct kvm_vcpu *vcpu)
> {
> - unsigned long hcr = vcpu_get_hcr(vcpu);
> + unsigned long hcr = *vcpu_hcr(vcpu);
>
> /*
> * If this is the first time we do a S/W operation
> @@ -2002,7 +2002,7 @@ void kvm_set_way_flush(struct kvm_vcpu *vcpu)
> trace_kvm_set_way_flush(*vcpu_pc(vcpu),
> vcpu_has_cache_enabled(vcpu));
> stage2_flush_vm(vcpu->kvm);
> - vcpu_set_hcr(vcpu, hcr | HCR_TVM);
> + *vcpu_hcr(vcpu) = hcr | HCR_TVM;
> }
> }
>
> @@ -2020,7 +2020,7 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
>
> /* Caches are now on, stop trapping VM ops (until a S/W op) */
> if (now_enabled)
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) & ~HCR_TVM);
> + *vcpu_hcr(vcpu) &= ~HCR_TVM;
>
> trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
> }
>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
@ 2017-10-12 16:24 ` Marc Zyngier
0 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-10-12 16:24 UTC (permalink / raw)
To: linux-arm-kernel
On 12/10/17 11:41, Christoffer Dall wrote:
> We currently have a separate read-modify-write of the HCR_EL2 on entry
> to the guest for the sole purpose of setting the VF and VI bits, if set.
> Since this is most rarely the case (only when using userspace IRQ chip
> and interrupts are in flight), let's get rid of this operation and
> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> needed.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm/include/asm/kvm_host.h | 3 ---
> arch/arm/kvm/emulate.c | 2 +-
> arch/arm/kvm/hyp/switch.c | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm64/include/asm/kvm_host.h | 3 ---
> arch/arm64/kvm/hyp/switch.c | 6 ------
> arch/arm64/kvm/inject_fault.c | 2 +-
> virt/kvm/arm/arm.c | 11 ++++++-----
> virt/kvm/arm/mmu.c | 6 +++---
> 10 files changed, 16 insertions(+), 37 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
> index 98089ff..34663a8 100644
> --- a/arch/arm/include/asm/kvm_emulate.h
> +++ b/arch/arm/include/asm/kvm_emulate.h
> @@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr = HCR_GUEST_MASK;
> }
>
> -static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr = hcr;
> + return (unsigned long *)&vcpu->arch.hcr;
> }
>
> static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 4a879f6..1100170 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
> /* HYP trapping configuration */
> u32 hcr;
>
> - /* Interrupt related fields */
> - u32 irq_lines; /* IRQ and FIQ levels */
> -
> /* Exception Information */
> struct kvm_vcpu_fault_info fault;
>
> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> index 0064b86..4286a89 100644
> --- a/arch/arm/kvm/emulate.c
> +++ b/arch/arm/kvm/emulate.c
> @@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
> + *vcpu_hcr(vcpu) |= HCR_VA;
> }
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index 330c9ce..c3b9799 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
> isb();
> }
>
> - write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
> + write_sysreg(vcpu->arch.hcr, HCR);
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(HSTR_T(15), HSTR);
> write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e5df3fc..1fbfe96 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr_el2 &= ~HCR_RW;
> }
>
> -static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr_el2;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr_el2 = hcr;
> + return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 806ccef..27305e7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
> /* IO related fields */
> struct kvm_decode mmio_decode;
>
> - /* Interrupt related fields */
> - u64 irq_lines; /* IRQ and FIQ levels */
> -
> /* Cache some mmu pages needed inside spinlock regions */
> struct kvm_mmu_memory_cache mmu_page_cache;
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index bcf1a79..7703d63 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> - u64 val;
> -
> - val = read_sysreg(hcr_el2);
> - val |= vcpu->arch.irq_lines;
> - write_sysreg(val, hcr_el2);
> -
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> else
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index da6a8cf..45c7026 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
> + *vcpu_hcr(vcpu) |= HCR_VSE;
> }
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 7f9296a..6e9513e 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
> */
> int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
> {
> - return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
> + bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
> + return ((irq_lines || kvm_vgic_vcpu_pending_irq(v))
> && !v->arch.power_off && !v->arch.pause);
> }
>
> @@ -771,18 +772,18 @@ static int vcpu_interrupt_line(struct kvm_vcpu *vcpu, int number, bool level)
> {
> int bit_index;
> bool set;
> - unsigned long *ptr;
> + unsigned long *hcr;
>
> if (number == KVM_ARM_IRQ_CPU_IRQ)
> bit_index = __ffs(HCR_VI);
> else /* KVM_ARM_IRQ_CPU_FIQ */
> bit_index = __ffs(HCR_VF);
>
> - ptr = (unsigned long *)&vcpu->arch.irq_lines;
> + hcr = vcpu_hcr(vcpu);
> if (level)
> - set = test_and_set_bit(bit_index, ptr);
> + set = test_and_set_bit(bit_index, hcr);
> else
> - set = test_and_clear_bit(bit_index, ptr);
> + set = test_and_clear_bit(bit_index, hcr);
>
> /*
> * If we didn't change anything, no need to wake up or kick other CPUs
> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
> index b36945d..d93d56d 100644
> --- a/virt/kvm/arm/mmu.c
> +++ b/virt/kvm/arm/mmu.c
> @@ -1987,7 +1987,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
> */
> void kvm_set_way_flush(struct kvm_vcpu *vcpu)
> {
> - unsigned long hcr = vcpu_get_hcr(vcpu);
> + unsigned long hcr = *vcpu_hcr(vcpu);
>
> /*
> * If this is the first time we do a S/W operation
> @@ -2002,7 +2002,7 @@ void kvm_set_way_flush(struct kvm_vcpu *vcpu)
> trace_kvm_set_way_flush(*vcpu_pc(vcpu),
> vcpu_has_cache_enabled(vcpu));
> stage2_flush_vm(vcpu->kvm);
> - vcpu_set_hcr(vcpu, hcr | HCR_TVM);
> + *vcpu_hcr(vcpu) = hcr | HCR_TVM;
> }
> }
>
> @@ -2020,7 +2020,7 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled)
>
> /* Caches are now on, stop trapping VM ops (until a S/W op) */
> if (now_enabled)
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) & ~HCR_TVM);
> + *vcpu_hcr(vcpu) &= ~HCR_TVM;
>
> trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled);
> }
>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-06 17:58 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-06 17:58 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, Marc Zyngier, Shih-Wei Li, kvm
On Thu, Oct 12, 2017 at 12:41:08PM +0200, Christoffer Dall wrote:
> We currently have a separate read-modify-write of the HCR_EL2 on entry
> to the guest for the sole purpose of setting the VF and VI bits, if set.
> Since this is most rarely the case (only when using userspace IRQ chip
> and interrupts are in flight), let's get rid of this operation and
> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> needed.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm/include/asm/kvm_host.h | 3 ---
> arch/arm/kvm/emulate.c | 2 +-
> arch/arm/kvm/hyp/switch.c | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm64/include/asm/kvm_host.h | 3 ---
> arch/arm64/kvm/hyp/switch.c | 6 ------
> arch/arm64/kvm/inject_fault.c | 2 +-
> virt/kvm/arm/arm.c | 11 ++++++-----
> virt/kvm/arm/mmu.c | 6 +++---
> 10 files changed, 16 insertions(+), 37 deletions(-)
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
@ 2017-11-06 17:58 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-06 17:58 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:08PM +0200, Christoffer Dall wrote:
> We currently have a separate read-modify-write of the HCR_EL2 on entry
> to the guest for the sole purpose of setting the VF and VI bits, if set.
> Since this is most rarely the case (only when using userspace IRQ chip
> and interrupts are in flight), let's get rid of this operation and
> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> needed.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm/include/asm/kvm_host.h | 3 ---
> arch/arm/kvm/emulate.c | 2 +-
> arch/arm/kvm/hyp/switch.c | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm64/include/asm/kvm_host.h | 3 ---
> arch/arm64/kvm/hyp/switch.c | 6 ------
> arch/arm64/kvm/inject_fault.c | 2 +-
> virt/kvm/arm/arm.c | 11 ++++++-----
> virt/kvm/arm/mmu.c | 6 +++---
> 10 files changed, 16 insertions(+), 37 deletions(-)
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-14 12:17 ` Julien Thierry
-1 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-14 12:17 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Hi Christoffer,
On 12/10/17 11:41, Christoffer Dall wrote:
> We currently have a separate read-modify-write of the HCR_EL2 on entry
> to the guest for the sole purpose of setting the VF and VI bits, if set.
> Since this is most rarely the case (only when using userspace IRQ chip
> and interrupts are in flight), let's get rid of this operation and
> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> needed.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm/include/asm/kvm_host.h | 3 ---
> arch/arm/kvm/emulate.c | 2 +-
> arch/arm/kvm/hyp/switch.c | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm64/include/asm/kvm_host.h | 3 ---
> arch/arm64/kvm/hyp/switch.c | 6 ------
> arch/arm64/kvm/inject_fault.c | 2 +-
> virt/kvm/arm/arm.c | 11 ++++++-----
> virt/kvm/arm/mmu.c | 6 +++---
> 10 files changed, 16 insertions(+), 37 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
> index 98089ff..34663a8 100644
> --- a/arch/arm/include/asm/kvm_emulate.h
> +++ b/arch/arm/include/asm/kvm_emulate.h
> @@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr = HCR_GUEST_MASK;
> }
>
> -static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr = hcr;
> + return (unsigned long *)&vcpu->arch.hcr;
> }
>
> static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 4a879f6..1100170 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
> /* HYP trapping configuration */
> u32 hcr;
>
> - /* Interrupt related fields */
> - u32 irq_lines; /* IRQ and FIQ levels */
> -
> /* Exception Information */
> struct kvm_vcpu_fault_info fault;
>
> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> index 0064b86..4286a89 100644
> --- a/arch/arm/kvm/emulate.c
> +++ b/arch/arm/kvm/emulate.c
> @@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
> + *vcpu_hcr(vcpu) |= HCR_VA;
> }
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index 330c9ce..c3b9799 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
> isb();
> }
>
> - write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
> + write_sysreg(vcpu->arch.hcr, HCR);
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(HSTR_T(15), HSTR);
> write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e5df3fc..1fbfe96 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr_el2 &= ~HCR_RW;
> }
>
> -static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr_el2;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr_el2 = hcr;
> + return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 806ccef..27305e7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
> /* IO related fields */
> struct kvm_decode mmio_decode;
>
> - /* Interrupt related fields */
> - u64 irq_lines; /* IRQ and FIQ levels */
> -
> /* Cache some mmu pages needed inside spinlock regions */
> struct kvm_mmu_memory_cache mmu_page_cache;
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index bcf1a79..7703d63 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> - u64 val;
> -
> - val = read_sysreg(hcr_el2);
> - val |= vcpu->arch.irq_lines;
> - write_sysreg(val, hcr_el2);
> -
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> else
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index da6a8cf..45c7026 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
> + *vcpu_hcr(vcpu) |= HCR_VSE;
> }
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 7f9296a..6e9513e 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
> */
> int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
> {
> - return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
> + bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
Hmmm, I might be nitpicking here, but in my mind bool should be used
only to contain true (1) or false (0) values.
Here the non-false values are never 1.
Not sure if the definition of _Bool guaranties to be able to contain
other values than 1 and 0, although I agree it is unlikely it will be
less than a byte which works in your case.
Other than that:
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Cheers,
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
@ 2017-11-14 12:17 ` Julien Thierry
0 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-14 12:17 UTC (permalink / raw)
To: linux-arm-kernel
Hi Christoffer,
On 12/10/17 11:41, Christoffer Dall wrote:
> We currently have a separate read-modify-write of the HCR_EL2 on entry
> to the guest for the sole purpose of setting the VF and VI bits, if set.
> Since this is most rarely the case (only when using userspace IRQ chip
> and interrupts are in flight), let's get rid of this operation and
> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> needed.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm/include/asm/kvm_host.h | 3 ---
> arch/arm/kvm/emulate.c | 2 +-
> arch/arm/kvm/hyp/switch.c | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> arch/arm64/include/asm/kvm_host.h | 3 ---
> arch/arm64/kvm/hyp/switch.c | 6 ------
> arch/arm64/kvm/inject_fault.c | 2 +-
> virt/kvm/arm/arm.c | 11 ++++++-----
> virt/kvm/arm/mmu.c | 6 +++---
> 10 files changed, 16 insertions(+), 37 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
> index 98089ff..34663a8 100644
> --- a/arch/arm/include/asm/kvm_emulate.h
> +++ b/arch/arm/include/asm/kvm_emulate.h
> @@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr = HCR_GUEST_MASK;
> }
>
> -static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr = hcr;
> + return (unsigned long *)&vcpu->arch.hcr;
> }
>
> static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 4a879f6..1100170 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
> /* HYP trapping configuration */
> u32 hcr;
>
> - /* Interrupt related fields */
> - u32 irq_lines; /* IRQ and FIQ levels */
> -
> /* Exception Information */
> struct kvm_vcpu_fault_info fault;
>
> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> index 0064b86..4286a89 100644
> --- a/arch/arm/kvm/emulate.c
> +++ b/arch/arm/kvm/emulate.c
> @@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
> + *vcpu_hcr(vcpu) |= HCR_VA;
> }
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index 330c9ce..c3b9799 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
> isb();
> }
>
> - write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
> + write_sysreg(vcpu->arch.hcr, HCR);
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(HSTR_T(15), HSTR);
> write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index e5df3fc..1fbfe96 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> vcpu->arch.hcr_el2 &= ~HCR_RW;
> }
>
> -static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
> +static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> {
> - return vcpu->arch.hcr_el2;
> -}
> -
> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> -{
> - vcpu->arch.hcr_el2 = hcr;
> + return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 806ccef..27305e7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
> /* IO related fields */
> struct kvm_decode mmio_decode;
>
> - /* Interrupt related fields */
> - u64 irq_lines; /* IRQ and FIQ levels */
> -
> /* Cache some mmu pages needed inside spinlock regions */
> struct kvm_mmu_memory_cache mmu_page_cache;
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index bcf1a79..7703d63 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> - u64 val;
> -
> - val = read_sysreg(hcr_el2);
> - val |= vcpu->arch.irq_lines;
> - write_sysreg(val, hcr_el2);
> -
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> else
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index da6a8cf..45c7026 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
> */
> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> {
> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
> + *vcpu_hcr(vcpu) |= HCR_VSE;
> }
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index 7f9296a..6e9513e 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
> */
> int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
> {
> - return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
> + bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
Hmmm, I might be nitpicking here, but in my mind bool should be used
only to contain true (1) or false (0) values.
Here the non-false values are never 1.
Not sure if the definition of _Bool guaranties to be able to contain
other values than 1 and 0, although I agree it is unlikely it will be
less than a byte which works in your case.
Other than that:
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Cheers,
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
2017-11-14 12:17 ` Julien Thierry
@ 2017-11-16 16:11 ` Julien Thierry
-1 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-16 16:11 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
On 14/11/17 12:17, Julien Thierry wrote:
> Hi Christoffer,
>
> On 12/10/17 11:41, Christoffer Dall wrote:
>> We currently have a separate read-modify-write of the HCR_EL2 on entry
>> to the guest for the sole purpose of setting the VF and VI bits, if set.
>> Since this is most rarely the case (only when using userspace IRQ chip
>> and interrupts are in flight), let's get rid of this operation and
>> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
>> needed.
>>
>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>> ---
>> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
>> arch/arm/include/asm/kvm_host.h | 3 ---
>> arch/arm/kvm/emulate.c | 2 +-
>> arch/arm/kvm/hyp/switch.c | 2 +-
>> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
>> arch/arm64/include/asm/kvm_host.h | 3 ---
>> arch/arm64/kvm/hyp/switch.c | 6 ------
>> arch/arm64/kvm/inject_fault.c | 2 +-
>> virt/kvm/arm/arm.c | 11 ++++++-----
>> virt/kvm/arm/mmu.c | 6 +++---
>> 10 files changed, 16 insertions(+), 37 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_emulate.h
>> b/arch/arm/include/asm/kvm_emulate.h
>> index 98089ff..34663a8 100644
>> --- a/arch/arm/include/asm/kvm_emulate.h
>> +++ b/arch/arm/include/asm/kvm_emulate.h
>> @@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu
>> *vcpu)
>> vcpu->arch.hcr = HCR_GUEST_MASK;
>> }
>> -static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
>> +static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
>> {
>> - return vcpu->arch.hcr;
>> -}
>> -
>> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long
>> hcr)
>> -{
>> - vcpu->arch.hcr = hcr;
>> + return (unsigned long *)&vcpu->arch.hcr;
>> }
>> static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm/include/asm/kvm_host.h
>> b/arch/arm/include/asm/kvm_host.h
>> index 4a879f6..1100170 100644
>> --- a/arch/arm/include/asm/kvm_host.h
>> +++ b/arch/arm/include/asm/kvm_host.h
>> @@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
>> /* HYP trapping configuration */
>> u32 hcr;
>> - /* Interrupt related fields */
>> - u32 irq_lines; /* IRQ and FIQ levels */
>> -
>> /* Exception Information */
>> struct kvm_vcpu_fault_info fault;
>> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
>> index 0064b86..4286a89 100644
>> --- a/arch/arm/kvm/emulate.c
>> +++ b/arch/arm/kvm/emulate.c
>> @@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu,
>> unsigned long addr)
>> */
>> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
>> {
>> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
>> + *vcpu_hcr(vcpu) |= HCR_VA;
>> }
>> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
>> index 330c9ce..c3b9799 100644
>> --- a/arch/arm/kvm/hyp/switch.c
>> +++ b/arch/arm/kvm/hyp/switch.c
>> @@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct
>> kvm_vcpu *vcpu, u32 *fpexc_host)
>> isb();
>> }
>> - write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
>> + write_sysreg(vcpu->arch.hcr, HCR);
>> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
>> write_sysreg(HSTR_T(15), HSTR);
>> write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h
>> b/arch/arm64/include/asm/kvm_emulate.h
>> index e5df3fc..1fbfe96 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu
>> *vcpu)
>> vcpu->arch.hcr_el2 &= ~HCR_RW;
>> }
>> -static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
>> +static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
>> {
>> - return vcpu->arch.hcr_el2;
>> -}
>> -
>> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long
>> hcr)
>> -{
>> - vcpu->arch.hcr_el2 = hcr;
>> + return (unsigned long *)&vcpu->arch.hcr_el2;
>> }
>> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm64/include/asm/kvm_host.h
>> b/arch/arm64/include/asm/kvm_host.h
>> index 806ccef..27305e7 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
>> /* IO related fields */
>> struct kvm_decode mmio_decode;
>> - /* Interrupt related fields */
>> - u64 irq_lines; /* IRQ and FIQ levels */
>> -
>> /* Cache some mmu pages needed inside spinlock regions */
>> struct kvm_mmu_memory_cache mmu_page_cache;
>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>> index bcf1a79..7703d63 100644
>> --- a/arch/arm64/kvm/hyp/switch.c
>> +++ b/arch/arm64/kvm/hyp/switch.c
>> @@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct
>> kvm_vcpu *vcpu)
>> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>> {
>> - u64 val;
>> -
>> - val = read_sysreg(hcr_el2);
>> - val |= vcpu->arch.irq_lines;
>> - write_sysreg(val, hcr_el2);
>> -
>> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
>> __vgic_v3_restore_state(vcpu);
>> else
>> diff --git a/arch/arm64/kvm/inject_fault.c
>> b/arch/arm64/kvm/inject_fault.c
>> index da6a8cf..45c7026 100644
>> --- a/arch/arm64/kvm/inject_fault.c
>> +++ b/arch/arm64/kvm/inject_fault.c
>> @@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
>> */
>> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
>> {
>> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
>> + *vcpu_hcr(vcpu) |= HCR_VSE;
>> }
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index 7f9296a..6e9513e 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct
>> kvm_vcpu *vcpu,
>> */
>> int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>> {
>> - return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
>> + bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
>
> Hmmm, I might be nitpicking here, but in my mind bool should be used
> only to contain true (1) or false (0) values.
> Here the non-false values are never 1.
>
> Not sure if the definition of _Bool guaranties to be able to contain
> other values than 1 and 0, although I agree it is unlikely it will be
> less than a byte which works in your case.
Forget what I said here, I was not aware that casting to _Bool always
gives true or false.
>
> Other than that:
>
> Reviewed-by: Julien Thierry <julien.thierry@arm.com>
>
> Cheers,
>
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
@ 2017-11-16 16:11 ` Julien Thierry
0 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-16 16:11 UTC (permalink / raw)
To: linux-arm-kernel
On 14/11/17 12:17, Julien Thierry wrote:
> Hi Christoffer,
>
> On 12/10/17 11:41, Christoffer Dall wrote:
>> We currently have a separate read-modify-write of the HCR_EL2 on entry
>> to the guest for the sole purpose of setting the VF and VI bits, if set.
>> Since this is most rarely the case (only when using userspace IRQ chip
>> and interrupts are in flight), let's get rid of this operation and
>> instead modify the bits in the vcpu->arch.hcr[_el2] directly when
>> needed.
>>
>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>> ---
>> arch/arm/include/asm/kvm_emulate.h | 9 ++-------
>> arch/arm/include/asm/kvm_host.h | 3 ---
>> arch/arm/kvm/emulate.c | 2 +-
>> arch/arm/kvm/hyp/switch.c | 2 +-
>> arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
>> arch/arm64/include/asm/kvm_host.h | 3 ---
>> arch/arm64/kvm/hyp/switch.c | 6 ------
>> arch/arm64/kvm/inject_fault.c | 2 +-
>> virt/kvm/arm/arm.c | 11 ++++++-----
>> virt/kvm/arm/mmu.c | 6 +++---
>> 10 files changed, 16 insertions(+), 37 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_emulate.h
>> b/arch/arm/include/asm/kvm_emulate.h
>> index 98089ff..34663a8 100644
>> --- a/arch/arm/include/asm/kvm_emulate.h
>> +++ b/arch/arm/include/asm/kvm_emulate.h
>> @@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu
>> *vcpu)
>> vcpu->arch.hcr = HCR_GUEST_MASK;
>> }
>> -static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
>> +static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
>> {
>> - return vcpu->arch.hcr;
>> -}
>> -
>> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long
>> hcr)
>> -{
>> - vcpu->arch.hcr = hcr;
>> + return (unsigned long *)&vcpu->arch.hcr;
>> }
>> static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm/include/asm/kvm_host.h
>> b/arch/arm/include/asm/kvm_host.h
>> index 4a879f6..1100170 100644
>> --- a/arch/arm/include/asm/kvm_host.h
>> +++ b/arch/arm/include/asm/kvm_host.h
>> @@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
>> /* HYP trapping configuration */
>> u32 hcr;
>> - /* Interrupt related fields */
>> - u32 irq_lines; /* IRQ and FIQ levels */
>> -
>> /* Exception Information */
>> struct kvm_vcpu_fault_info fault;
>> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
>> index 0064b86..4286a89 100644
>> --- a/arch/arm/kvm/emulate.c
>> +++ b/arch/arm/kvm/emulate.c
>> @@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu,
>> unsigned long addr)
>> */
>> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
>> {
>> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
>> + *vcpu_hcr(vcpu) |= HCR_VA;
>> }
>> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
>> index 330c9ce..c3b9799 100644
>> --- a/arch/arm/kvm/hyp/switch.c
>> +++ b/arch/arm/kvm/hyp/switch.c
>> @@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct
>> kvm_vcpu *vcpu, u32 *fpexc_host)
>> isb();
>> }
>> - write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
>> + write_sysreg(vcpu->arch.hcr, HCR);
>> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
>> write_sysreg(HSTR_T(15), HSTR);
>> write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h
>> b/arch/arm64/include/asm/kvm_emulate.h
>> index e5df3fc..1fbfe96 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu
>> *vcpu)
>> vcpu->arch.hcr_el2 &= ~HCR_RW;
>> }
>> -static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
>> +static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
>> {
>> - return vcpu->arch.hcr_el2;
>> -}
>> -
>> -static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long
>> hcr)
>> -{
>> - vcpu->arch.hcr_el2 = hcr;
>> + return (unsigned long *)&vcpu->arch.hcr_el2;
>> }
>> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm64/include/asm/kvm_host.h
>> b/arch/arm64/include/asm/kvm_host.h
>> index 806ccef..27305e7 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
>> /* IO related fields */
>> struct kvm_decode mmio_decode;
>> - /* Interrupt related fields */
>> - u64 irq_lines; /* IRQ and FIQ levels */
>> -
>> /* Cache some mmu pages needed inside spinlock regions */
>> struct kvm_mmu_memory_cache mmu_page_cache;
>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>> index bcf1a79..7703d63 100644
>> --- a/arch/arm64/kvm/hyp/switch.c
>> +++ b/arch/arm64/kvm/hyp/switch.c
>> @@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct
>> kvm_vcpu *vcpu)
>> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
>> {
>> - u64 val;
>> -
>> - val = read_sysreg(hcr_el2);
>> - val |= vcpu->arch.irq_lines;
>> - write_sysreg(val, hcr_el2);
>> -
>> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
>> __vgic_v3_restore_state(vcpu);
>> else
>> diff --git a/arch/arm64/kvm/inject_fault.c
>> b/arch/arm64/kvm/inject_fault.c
>> index da6a8cf..45c7026 100644
>> --- a/arch/arm64/kvm/inject_fault.c
>> +++ b/arch/arm64/kvm/inject_fault.c
>> @@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
>> */
>> void kvm_inject_vabt(struct kvm_vcpu *vcpu)
>> {
>> - vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
>> + *vcpu_hcr(vcpu) |= HCR_VSE;
>> }
>> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
>> index 7f9296a..6e9513e 100644
>> --- a/virt/kvm/arm/arm.c
>> +++ b/virt/kvm/arm/arm.c
>> @@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct
>> kvm_vcpu *vcpu,
>> */
>> int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
>> {
>> - return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
>> + bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
>
> Hmmm, I might be nitpicking here, but in my mind bool should be used
> only to contain true (1) or false (0) values.
> Here the non-false values are never 1.
>
> Not sure if the definition of _Bool guaranties to be able to contain
> other values than 1 and 0, although I agree it is unlikely it will be
> less than a byte which works in your case.
Forget what I said here, I was not aware that casting to _Bool always
gives true or false.
>
> Other than that:
>
> Reviewed-by: Julien Thierry <julien.thierry@arm.com>
>
> Cheers,
>
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
2017-11-14 12:17 ` Julien Thierry
@ 2017-11-26 16:04 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:04 UTC (permalink / raw)
To: Julien Thierry
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Tue, Nov 14, 2017 at 12:17:44PM +0000, Julien Thierry wrote:
> Hi Christoffer,
>
> On 12/10/17 11:41, Christoffer Dall wrote:
> >We currently have a separate read-modify-write of the HCR_EL2 on entry
> >to the guest for the sole purpose of setting the VF and VI bits, if set.
> >Since this is most rarely the case (only when using userspace IRQ chip
> >and interrupts are in flight), let's get rid of this operation and
> >instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> >needed.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >---
> > arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> > arch/arm/include/asm/kvm_host.h | 3 ---
> > arch/arm/kvm/emulate.c | 2 +-
> > arch/arm/kvm/hyp/switch.c | 2 +-
> > arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> > arch/arm64/include/asm/kvm_host.h | 3 ---
> > arch/arm64/kvm/hyp/switch.c | 6 ------
> > arch/arm64/kvm/inject_fault.c | 2 +-
> > virt/kvm/arm/arm.c | 11 ++++++-----
> > virt/kvm/arm/mmu.c | 6 +++---
> > 10 files changed, 16 insertions(+), 37 deletions(-)
> >
> >diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
> >index 98089ff..34663a8 100644
> >--- a/arch/arm/include/asm/kvm_emulate.h
> >+++ b/arch/arm/include/asm/kvm_emulate.h
> >@@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> > vcpu->arch.hcr = HCR_GUEST_MASK;
> > }
> >-static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
> >+static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
> > {
> >- return vcpu->arch.hcr;
> >-}
> >-
> >-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> >-{
> >- vcpu->arch.hcr = hcr;
> >+ return (unsigned long *)&vcpu->arch.hcr;
> > }
> > static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
> >diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> >index 4a879f6..1100170 100644
> >--- a/arch/arm/include/asm/kvm_host.h
> >+++ b/arch/arm/include/asm/kvm_host.h
> >@@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
> > /* HYP trapping configuration */
> > u32 hcr;
> >- /* Interrupt related fields */
> >- u32 irq_lines; /* IRQ and FIQ levels */
> >-
> > /* Exception Information */
> > struct kvm_vcpu_fault_info fault;
> >diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> >index 0064b86..4286a89 100644
> >--- a/arch/arm/kvm/emulate.c
> >+++ b/arch/arm/kvm/emulate.c
> >@@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
> > */
> > void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> > {
> >- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
> >+ *vcpu_hcr(vcpu) |= HCR_VA;
> > }
> >diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> >index 330c9ce..c3b9799 100644
> >--- a/arch/arm/kvm/hyp/switch.c
> >+++ b/arch/arm/kvm/hyp/switch.c
> >@@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
> > isb();
> > }
> >- write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
> >+ write_sysreg(vcpu->arch.hcr, HCR);
> > /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> > write_sysreg(HSTR_T(15), HSTR);
> > write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
> >diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> >index e5df3fc..1fbfe96 100644
> >--- a/arch/arm64/include/asm/kvm_emulate.h
> >+++ b/arch/arm64/include/asm/kvm_emulate.h
> >@@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> > vcpu->arch.hcr_el2 &= ~HCR_RW;
> > }
> >-static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
> >+static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > {
> >- return vcpu->arch.hcr_el2;
> >-}
> >-
> >-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> >-{
> >- vcpu->arch.hcr_el2 = hcr;
> >+ return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> >diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> >index 806ccef..27305e7 100644
> >--- a/arch/arm64/include/asm/kvm_host.h
> >+++ b/arch/arm64/include/asm/kvm_host.h
> >@@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
> > /* IO related fields */
> > struct kvm_decode mmio_decode;
> >- /* Interrupt related fields */
> >- u64 irq_lines; /* IRQ and FIQ levels */
> >-
> > /* Cache some mmu pages needed inside spinlock regions */
> > struct kvm_mmu_memory_cache mmu_page_cache;
> >diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> >index bcf1a79..7703d63 100644
> >--- a/arch/arm64/kvm/hyp/switch.c
> >+++ b/arch/arm64/kvm/hyp/switch.c
> >@@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> > static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> > {
> >- u64 val;
> >-
> >- val = read_sysreg(hcr_el2);
> >- val |= vcpu->arch.irq_lines;
> >- write_sysreg(val, hcr_el2);
> >-
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_restore_state(vcpu);
> > else
> >diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> >index da6a8cf..45c7026 100644
> >--- a/arch/arm64/kvm/inject_fault.c
> >+++ b/arch/arm64/kvm/inject_fault.c
> >@@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
> > */
> > void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> > {
> >- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
> >+ *vcpu_hcr(vcpu) |= HCR_VSE;
> > }
> >diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> >index 7f9296a..6e9513e 100644
> >--- a/virt/kvm/arm/arm.c
> >+++ b/virt/kvm/arm/arm.c
> >@@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
> > */
> > int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
> > {
> >- return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
> >+ bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
>
> Hmmm, I might be nitpicking here, but in my mind bool should be used
> only to contain true (1) or false (0) values.
> Here the non-false values are never 1.
>
> Not sure if the definition of _Bool guaranties to be able to contain
> other values than 1 and 0, although I agree it is unlikely it will
> be less than a byte which works in your case.
As you found out, this is fine as long as we use _Bool. If we had used
an integer type, it would be bad practice indeed.
>
> Other than that:
>
> Reviewed-by: Julien Thierry <julien.thierry@arm.com>
>
Thanks for the review!
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 04/37] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines
@ 2017-11-26 16:04 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:04 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 14, 2017 at 12:17:44PM +0000, Julien Thierry wrote:
> Hi Christoffer,
>
> On 12/10/17 11:41, Christoffer Dall wrote:
> >We currently have a separate read-modify-write of the HCR_EL2 on entry
> >to the guest for the sole purpose of setting the VF and VI bits, if set.
> >Since this is most rarely the case (only when using userspace IRQ chip
> >and interrupts are in flight), let's get rid of this operation and
> >instead modify the bits in the vcpu->arch.hcr[_el2] directly when
> >needed.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >---
> > arch/arm/include/asm/kvm_emulate.h | 9 ++-------
> > arch/arm/include/asm/kvm_host.h | 3 ---
> > arch/arm/kvm/emulate.c | 2 +-
> > arch/arm/kvm/hyp/switch.c | 2 +-
> > arch/arm64/include/asm/kvm_emulate.h | 9 ++-------
> > arch/arm64/include/asm/kvm_host.h | 3 ---
> > arch/arm64/kvm/hyp/switch.c | 6 ------
> > arch/arm64/kvm/inject_fault.c | 2 +-
> > virt/kvm/arm/arm.c | 11 ++++++-----
> > virt/kvm/arm/mmu.c | 6 +++---
> > 10 files changed, 16 insertions(+), 37 deletions(-)
> >
> >diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
> >index 98089ff..34663a8 100644
> >--- a/arch/arm/include/asm/kvm_emulate.h
> >+++ b/arch/arm/include/asm/kvm_emulate.h
> >@@ -62,14 +62,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> > vcpu->arch.hcr = HCR_GUEST_MASK;
> > }
> >-static inline unsigned long vcpu_get_hcr(const struct kvm_vcpu *vcpu)
> >+static inline unsigned long *vcpu_hcr(const struct kvm_vcpu *vcpu)
> > {
> >- return vcpu->arch.hcr;
> >-}
> >-
> >-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> >-{
> >- vcpu->arch.hcr = hcr;
> >+ return (unsigned long *)&vcpu->arch.hcr;
> > }
> > static inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
> >diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> >index 4a879f6..1100170 100644
> >--- a/arch/arm/include/asm/kvm_host.h
> >+++ b/arch/arm/include/asm/kvm_host.h
> >@@ -153,9 +153,6 @@ struct kvm_vcpu_arch {
> > /* HYP trapping configuration */
> > u32 hcr;
> >- /* Interrupt related fields */
> >- u32 irq_lines; /* IRQ and FIQ levels */
> >-
> > /* Exception Information */
> > struct kvm_vcpu_fault_info fault;
> >diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> >index 0064b86..4286a89 100644
> >--- a/arch/arm/kvm/emulate.c
> >+++ b/arch/arm/kvm/emulate.c
> >@@ -313,5 +313,5 @@ void kvm_inject_pabt(struct kvm_vcpu *vcpu, unsigned long addr)
> > */
> > void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> > {
> >- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VA);
> >+ *vcpu_hcr(vcpu) |= HCR_VA;
> > }
> >diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> >index 330c9ce..c3b9799 100644
> >--- a/arch/arm/kvm/hyp/switch.c
> >+++ b/arch/arm/kvm/hyp/switch.c
> >@@ -43,7 +43,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu, u32 *fpexc_host)
> > isb();
> > }
> >- write_sysreg(vcpu->arch.hcr | vcpu->arch.irq_lines, HCR);
> >+ write_sysreg(vcpu->arch.hcr, HCR);
> > /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> > write_sysreg(HSTR_T(15), HSTR);
> > write_sysreg(HCPTR_TTA | HCPTR_TCP(10) | HCPTR_TCP(11), HCPTR);
> >diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> >index e5df3fc..1fbfe96 100644
> >--- a/arch/arm64/include/asm/kvm_emulate.h
> >+++ b/arch/arm64/include/asm/kvm_emulate.h
> >@@ -51,14 +51,9 @@ static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu)
> > vcpu->arch.hcr_el2 &= ~HCR_RW;
> > }
> >-static inline unsigned long vcpu_get_hcr(struct kvm_vcpu *vcpu)
> >+static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > {
> >- return vcpu->arch.hcr_el2;
> >-}
> >-
> >-static inline void vcpu_set_hcr(struct kvm_vcpu *vcpu, unsigned long hcr)
> >-{
> >- vcpu->arch.hcr_el2 = hcr;
> >+ return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> >diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> >index 806ccef..27305e7 100644
> >--- a/arch/arm64/include/asm/kvm_host.h
> >+++ b/arch/arm64/include/asm/kvm_host.h
> >@@ -266,9 +266,6 @@ struct kvm_vcpu_arch {
> > /* IO related fields */
> > struct kvm_decode mmio_decode;
> >- /* Interrupt related fields */
> >- u64 irq_lines; /* IRQ and FIQ levels */
> >-
> > /* Cache some mmu pages needed inside spinlock regions */
> > struct kvm_mmu_memory_cache mmu_page_cache;
> >diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> >index bcf1a79..7703d63 100644
> >--- a/arch/arm64/kvm/hyp/switch.c
> >+++ b/arch/arm64/kvm/hyp/switch.c
> >@@ -168,12 +168,6 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> > static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> > {
> >- u64 val;
> >-
> >- val = read_sysreg(hcr_el2);
> >- val |= vcpu->arch.irq_lines;
> >- write_sysreg(val, hcr_el2);
> >-
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_restore_state(vcpu);
> > else
> >diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> >index da6a8cf..45c7026 100644
> >--- a/arch/arm64/kvm/inject_fault.c
> >+++ b/arch/arm64/kvm/inject_fault.c
> >@@ -241,5 +241,5 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)
> > */
> > void kvm_inject_vabt(struct kvm_vcpu *vcpu)
> > {
> >- vcpu_set_hcr(vcpu, vcpu_get_hcr(vcpu) | HCR_VSE);
> >+ *vcpu_hcr(vcpu) |= HCR_VSE;
> > }
> >diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> >index 7f9296a..6e9513e 100644
> >--- a/virt/kvm/arm/arm.c
> >+++ b/virt/kvm/arm/arm.c
> >@@ -411,7 +411,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
> > */
> > int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
> > {
> >- return ((!!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v))
> >+ bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF);
>
> Hmmm, I might be nitpicking here, but in my mind bool should be used
> only to contain true (1) or false (0) values.
> Here the non-false values are never 1.
>
> Not sure if the definition of _Bool guaranties to be able to contain
> other values than 1 and 0, although I agree it is unlikely it will
> be less than a byte which works in your case.
As you found out, this is fine as long as we use _Bool. If we had used
an integer type, it would be bad practice indeed.
>
> Other than that:
>
> Reviewed-by: Julien Thierry <julien.thierry@arm.com>
>
Thanks for the review!
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: kvm, Marc Zyngier, Shih-Wei Li, Paolo Bonzini
Some architectures may decide to do different things during
kvm_arch_vcpu_load depending on the ioctl being executed. For example,
arm64 is about to do significant work in vcpu load/put when running a
vcpu, but not when doing things like KVM_SET_ONE_REG or
KVM_SET_MP_STATE.
Therefore, store the ioctl number that we are executing on the VCPU
during the first vcpu_load() which succeeds in getting the vcpu->mutex
and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
successfully loading the vcpu.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/x86/kvm/vmx.c | 2 +-
arch/x86/kvm/x86.c | 8 ++++----
include/linux/kvm_host.h | 3 ++-
virt/kvm/kvm_main.c | 6 ++++--
4 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 6970249..d729889 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -9493,7 +9493,7 @@ static void vmx_free_vcpu_nested(struct kvm_vcpu *vcpu)
struct vcpu_vmx *vmx = to_vmx(vcpu);
int r;
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
BUG_ON(r);
vmx_switch_vmcs(vcpu, &vmx->vmcs01);
free_nested(vmx);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index cd17b7d..68d9c95 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7723,7 +7723,7 @@ int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu)
int r;
kvm_vcpu_mtrr_init(vcpu);
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
if (r)
return r;
kvm_vcpu_reset(vcpu, false);
@@ -7739,7 +7739,7 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
kvm_hv_vcpu_postcreate(vcpu);
- if (vcpu_load(vcpu))
+ if (vcpu_load(vcpu, vcpu->ioctl))
return;
msr.data = 0x0;
msr.index = MSR_IA32_TSC;
@@ -7759,7 +7759,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
int r;
vcpu->arch.apf.msr_val = 0;
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
BUG_ON(r);
kvm_mmu_unload(vcpu);
vcpu_put(vcpu);
@@ -8116,7 +8116,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
static void kvm_unload_vcpu_mmu(struct kvm_vcpu *vcpu)
{
int r;
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
BUG_ON(r);
kvm_mmu_unload(vcpu);
vcpu_put(vcpu);
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6882538..da0acc0 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -274,6 +274,7 @@ struct kvm_vcpu {
bool preempted;
struct kvm_vcpu_arch arch;
struct dentry *debugfs_dentry;
+ unsigned int ioctl; /* ioctl currently executing or 0 */
};
static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu)
@@ -533,7 +534,7 @@ static inline int kvm_vcpu_get_idx(struct kvm_vcpu *vcpu)
int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id);
void kvm_vcpu_uninit(struct kvm_vcpu *vcpu);
-int __must_check vcpu_load(struct kvm_vcpu *vcpu);
+int __must_check vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl);
void vcpu_put(struct kvm_vcpu *vcpu);
#ifdef __KVM_HAVE_IOAPIC
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 9deb5a2..1911ef0 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -147,12 +147,13 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
/*
* Switches to specified vcpu, until a matching vcpu_put()
*/
-int vcpu_load(struct kvm_vcpu *vcpu)
+int vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl)
{
int cpu;
if (mutex_lock_killable(&vcpu->mutex))
return -EINTR;
+ vcpu->ioctl = ioctl;
cpu = get_cpu();
preempt_notifier_register(&vcpu->preempt_notifier);
kvm_arch_vcpu_load(vcpu, cpu);
@@ -2529,7 +2530,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
#endif
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, ioctl);
if (r)
return r;
switch (ioctl) {
@@ -2704,6 +2705,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
}
out:
vcpu_put(vcpu);
+ vcpu->ioctl = 0;
kfree(fpu);
kfree(kvm_sregs);
return r;
--
2.9.0
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Some architectures may decide to do different things during
kvm_arch_vcpu_load depending on the ioctl being executed. For example,
arm64 is about to do significant work in vcpu load/put when running a
vcpu, but not when doing things like KVM_SET_ONE_REG or
KVM_SET_MP_STATE.
Therefore, store the ioctl number that we are executing on the VCPU
during the first vcpu_load() which succeeds in getting the vcpu->mutex
and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
successfully loading the vcpu.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Kr?m?? <rkrcmar@redhat.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/x86/kvm/vmx.c | 2 +-
arch/x86/kvm/x86.c | 8 ++++----
include/linux/kvm_host.h | 3 ++-
virt/kvm/kvm_main.c | 6 ++++--
4 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 6970249..d729889 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -9493,7 +9493,7 @@ static void vmx_free_vcpu_nested(struct kvm_vcpu *vcpu)
struct vcpu_vmx *vmx = to_vmx(vcpu);
int r;
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
BUG_ON(r);
vmx_switch_vmcs(vcpu, &vmx->vmcs01);
free_nested(vmx);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index cd17b7d..68d9c95 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7723,7 +7723,7 @@ int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu)
int r;
kvm_vcpu_mtrr_init(vcpu);
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
if (r)
return r;
kvm_vcpu_reset(vcpu, false);
@@ -7739,7 +7739,7 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
kvm_hv_vcpu_postcreate(vcpu);
- if (vcpu_load(vcpu))
+ if (vcpu_load(vcpu, vcpu->ioctl))
return;
msr.data = 0x0;
msr.index = MSR_IA32_TSC;
@@ -7759,7 +7759,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
int r;
vcpu->arch.apf.msr_val = 0;
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
BUG_ON(r);
kvm_mmu_unload(vcpu);
vcpu_put(vcpu);
@@ -8116,7 +8116,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
static void kvm_unload_vcpu_mmu(struct kvm_vcpu *vcpu)
{
int r;
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, vcpu->ioctl);
BUG_ON(r);
kvm_mmu_unload(vcpu);
vcpu_put(vcpu);
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6882538..da0acc0 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -274,6 +274,7 @@ struct kvm_vcpu {
bool preempted;
struct kvm_vcpu_arch arch;
struct dentry *debugfs_dentry;
+ unsigned int ioctl; /* ioctl currently executing or 0 */
};
static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu)
@@ -533,7 +534,7 @@ static inline int kvm_vcpu_get_idx(struct kvm_vcpu *vcpu)
int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id);
void kvm_vcpu_uninit(struct kvm_vcpu *vcpu);
-int __must_check vcpu_load(struct kvm_vcpu *vcpu);
+int __must_check vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl);
void vcpu_put(struct kvm_vcpu *vcpu);
#ifdef __KVM_HAVE_IOAPIC
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 9deb5a2..1911ef0 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -147,12 +147,13 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
/*
* Switches to specified vcpu, until a matching vcpu_put()
*/
-int vcpu_load(struct kvm_vcpu *vcpu)
+int vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl)
{
int cpu;
if (mutex_lock_killable(&vcpu->mutex))
return -EINTR;
+ vcpu->ioctl = ioctl;
cpu = get_cpu();
preempt_notifier_register(&vcpu->preempt_notifier);
kvm_arch_vcpu_load(vcpu, cpu);
@@ -2529,7 +2530,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
#endif
- r = vcpu_load(vcpu);
+ r = vcpu_load(vcpu, ioctl);
if (r)
return r;
switch (ioctl) {
@@ -2704,6 +2705,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
}
out:
vcpu_put(vcpu);
+ vcpu->ioctl = 0;
kfree(fpu);
kfree(kvm_sregs);
return r;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-13 17:13 ` Radim Krčmář
-1 siblings, 0 replies; 254+ messages in thread
From: Radim Krčmář @ 2017-10-13 17:13 UTC (permalink / raw)
To: Christoffer Dall
Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li, Paolo Bonzini
2017-10-12 12:41+0200, Christoffer Dall:
> Some architectures may decide to do different things during
> kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> arm64 is about to do significant work in vcpu load/put when running a
> vcpu, but not when doing things like KVM_SET_ONE_REG or
> KVM_SET_MP_STATE.
>
> Therefore, store the ioctl number that we are executing on the VCPU
> during the first vcpu_load() which succeeds in getting the vcpu->mutex
> and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> successfully loading the vcpu.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> @@ -147,12 +147,13 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
> /*
> * Switches to specified vcpu, until a matching vcpu_put()
> */
> -int vcpu_load(struct kvm_vcpu *vcpu)
> +int vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl)
> {
> int cpu;
>
> if (mutex_lock_killable(&vcpu->mutex))
> return -EINTR;
> + vcpu->ioctl = ioctl;
This seems to prevent races by protecting the store by a mutex, but
> cpu = get_cpu();
> preempt_notifier_register(&vcpu->preempt_notifier);
> kvm_arch_vcpu_load(vcpu, cpu);
> @@ -2529,7 +2530,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> #endif
>
>
> - r = vcpu_load(vcpu);
> + r = vcpu_load(vcpu, ioctl);
> if (r)
> return r;
> switch (ioctl) {
> @@ -2704,6 +2705,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> }
> out:
> vcpu_put(vcpu);
> + vcpu->ioctl = 0;
we should still have a race as we clear ioctl only after releasing the
lock. For example malicious userspace could break KVM terms of use and
issue !KVM_RUN followed by KVM_RUN, so we would have these races:
!KVM_RUN | KVM_RUN
mutex_lock_killable(&vcpu->mutex); |
vcpu->ioctl = !KVM_RUN; |
... | mutex_lock_killable(&vcpu->mutex);
mutex_unlock(&vcpu->mutex); |
| vcpu->ioctl = KVM_RUN;
| kvm_arch_vcpu_load() // variant 1
vcpu->ioctl = 0; | ...
| kvm_arch_vcpu_load() // variant 2
| vcpu_put()
where the observed value of vcpu->ioctl in vcpu_put() would not
correctly pair with vcpu_load() or worse, kvm_arch_arch_load() in
KVM_RUN would execute with vcpu->ioctl = 0.
I think that other (special) callsites of vcpu_load()/vcpu_put() have a
well defined IOCTL that can be used instead of vcpu->ioctl, so we could
just pass the ioctl value all the way to arch code and never save it
anywhere,
thanks.
> kfree(fpu);
> kfree(kvm_sregs);
> return r;
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-10-13 17:13 ` Radim Krčmář
0 siblings, 0 replies; 254+ messages in thread
From: Radim Krčmář @ 2017-10-13 17:13 UTC (permalink / raw)
To: linux-arm-kernel
2017-10-12 12:41+0200, Christoffer Dall:
> Some architectures may decide to do different things during
> kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> arm64 is about to do significant work in vcpu load/put when running a
> vcpu, but not when doing things like KVM_SET_ONE_REG or
> KVM_SET_MP_STATE.
>
> Therefore, store the ioctl number that we are executing on the VCPU
> during the first vcpu_load() which succeeds in getting the vcpu->mutex
> and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> successfully loading the vcpu.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Kr?m?? <rkrcmar@redhat.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> @@ -147,12 +147,13 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
> /*
> * Switches to specified vcpu, until a matching vcpu_put()
> */
> -int vcpu_load(struct kvm_vcpu *vcpu)
> +int vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl)
> {
> int cpu;
>
> if (mutex_lock_killable(&vcpu->mutex))
> return -EINTR;
> + vcpu->ioctl = ioctl;
This seems to prevent races by protecting the store by a mutex, but
> cpu = get_cpu();
> preempt_notifier_register(&vcpu->preempt_notifier);
> kvm_arch_vcpu_load(vcpu, cpu);
> @@ -2529,7 +2530,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> #endif
>
>
> - r = vcpu_load(vcpu);
> + r = vcpu_load(vcpu, ioctl);
> if (r)
> return r;
> switch (ioctl) {
> @@ -2704,6 +2705,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> }
> out:
> vcpu_put(vcpu);
> + vcpu->ioctl = 0;
we should still have a race as we clear ioctl only after releasing the
lock. For example malicious userspace could break KVM terms of use and
issue !KVM_RUN followed by KVM_RUN, so we would have these races:
!KVM_RUN | KVM_RUN
mutex_lock_killable(&vcpu->mutex); |
vcpu->ioctl = !KVM_RUN; |
... | mutex_lock_killable(&vcpu->mutex);
mutex_unlock(&vcpu->mutex); |
| vcpu->ioctl = KVM_RUN;
| kvm_arch_vcpu_load() // variant 1
vcpu->ioctl = 0; | ...
| kvm_arch_vcpu_load() // variant 2
| vcpu_put()
where the observed value of vcpu->ioctl in vcpu_put() would not
correctly pair with vcpu_load() or worse, kvm_arch_arch_load() in
KVM_RUN would execute with vcpu->ioctl = 0.
I think that other (special) callsites of vcpu_load()/vcpu_put() have a
well defined IOCTL that can be used instead of vcpu->ioctl, so we could
just pass the ioctl value all the way to arch code and never save it
anywhere,
thanks.
> kfree(fpu);
> kfree(kvm_sregs);
> return r;
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-10-13 17:13 ` Radim Krčmář
@ 2017-10-13 17:31 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-13 17:31 UTC (permalink / raw)
To: Radim Krčmář
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li, Paolo Bonzini
On Fri, Oct 13, 2017 at 07:13:07PM +0200, Radim Krčmář wrote:
> 2017-10-12 12:41+0200, Christoffer Dall:
> > Some architectures may decide to do different things during
> > kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> > arm64 is about to do significant work in vcpu load/put when running a
> > vcpu, but not when doing things like KVM_SET_ONE_REG or
> > KVM_SET_MP_STATE.
> >
> > Therefore, store the ioctl number that we are executing on the VCPU
> > during the first vcpu_load() which succeeds in getting the vcpu->mutex
> > and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> > successfully loading the vcpu.
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Krčmář <rkrcmar@redhat.com>
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> > @@ -147,12 +147,13 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
> > /*
> > * Switches to specified vcpu, until a matching vcpu_put()
> > */
> > -int vcpu_load(struct kvm_vcpu *vcpu)
> > +int vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl)
> > {
> > int cpu;
> >
> > if (mutex_lock_killable(&vcpu->mutex))
> > return -EINTR;
> > + vcpu->ioctl = ioctl;
>
> This seems to prevent races by protecting the store by a mutex, but
>
> > cpu = get_cpu();
> > preempt_notifier_register(&vcpu->preempt_notifier);
> > kvm_arch_vcpu_load(vcpu, cpu);
> > @@ -2529,7 +2530,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> > #endif
> >
> >
> > - r = vcpu_load(vcpu);
> > + r = vcpu_load(vcpu, ioctl);
> > if (r)
> > return r;
> > switch (ioctl) {
> > @@ -2704,6 +2705,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> > }
> > out:
> > vcpu_put(vcpu);
> > + vcpu->ioctl = 0;
>
> we should still have a race as we clear ioctl only after releasing the
> lock. For example malicious userspace could break KVM terms of use and
> issue !KVM_RUN followed by KVM_RUN, so we would have these races:
>
> !KVM_RUN | KVM_RUN
>
> mutex_lock_killable(&vcpu->mutex); |
> vcpu->ioctl = !KVM_RUN; |
> ... | mutex_lock_killable(&vcpu->mutex);
> mutex_unlock(&vcpu->mutex); |
> | vcpu->ioctl = KVM_RUN;
> | kvm_arch_vcpu_load() // variant 1
> vcpu->ioctl = 0; | ...
> | kvm_arch_vcpu_load() // variant 2
> | vcpu_put()
>
> where the observed value of vcpu->ioctl in vcpu_put() would not
> correctly pair with vcpu_load() or worse, kvm_arch_arch_load() in
> KVM_RUN would execute with vcpu->ioctl = 0.
Yeah, this is super racy, thanks for pointing that out.
>
> I think that other (special) callsites of vcpu_load()/vcpu_put() have a
> well defined IOCTL that can be used instead of vcpu->ioctl, so we could
> just pass the ioctl value all the way to arch code and never save it
> anywhere,
I don't think that works; what would you do with preempt notifier calls?
One solution is to add a parameter to vcpu_put, lie for vcpu_load, which
also sets the ioctl, and other callers than the final vcpu_put in
kvm_vcpu_ioctl() just pass the existing value, where the kvm_vcpu_ioctl
call can pass 0 which gets set before releasing the mutex.
Can you think of a more elegant solution?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-10-13 17:31 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-13 17:31 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 13, 2017 at 07:13:07PM +0200, Radim Kr?m?? wrote:
> 2017-10-12 12:41+0200, Christoffer Dall:
> > Some architectures may decide to do different things during
> > kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> > arm64 is about to do significant work in vcpu load/put when running a
> > vcpu, but not when doing things like KVM_SET_ONE_REG or
> > KVM_SET_MP_STATE.
> >
> > Therefore, store the ioctl number that we are executing on the VCPU
> > during the first vcpu_load() which succeeds in getting the vcpu->mutex
> > and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> > successfully loading the vcpu.
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Kr?m?? <rkrcmar@redhat.com>
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> > @@ -147,12 +147,13 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
> > /*
> > * Switches to specified vcpu, until a matching vcpu_put()
> > */
> > -int vcpu_load(struct kvm_vcpu *vcpu)
> > +int vcpu_load(struct kvm_vcpu *vcpu, unsigned int ioctl)
> > {
> > int cpu;
> >
> > if (mutex_lock_killable(&vcpu->mutex))
> > return -EINTR;
> > + vcpu->ioctl = ioctl;
>
> This seems to prevent races by protecting the store by a mutex, but
>
> > cpu = get_cpu();
> > preempt_notifier_register(&vcpu->preempt_notifier);
> > kvm_arch_vcpu_load(vcpu, cpu);
> > @@ -2529,7 +2530,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> > #endif
> >
> >
> > - r = vcpu_load(vcpu);
> > + r = vcpu_load(vcpu, ioctl);
> > if (r)
> > return r;
> > switch (ioctl) {
> > @@ -2704,6 +2705,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> > }
> > out:
> > vcpu_put(vcpu);
> > + vcpu->ioctl = 0;
>
> we should still have a race as we clear ioctl only after releasing the
> lock. For example malicious userspace could break KVM terms of use and
> issue !KVM_RUN followed by KVM_RUN, so we would have these races:
>
> !KVM_RUN | KVM_RUN
>
> mutex_lock_killable(&vcpu->mutex); |
> vcpu->ioctl = !KVM_RUN; |
> ... | mutex_lock_killable(&vcpu->mutex);
> mutex_unlock(&vcpu->mutex); |
> | vcpu->ioctl = KVM_RUN;
> | kvm_arch_vcpu_load() // variant 1
> vcpu->ioctl = 0; | ...
> | kvm_arch_vcpu_load() // variant 2
> | vcpu_put()
>
> where the observed value of vcpu->ioctl in vcpu_put() would not
> correctly pair with vcpu_load() or worse, kvm_arch_arch_load() in
> KVM_RUN would execute with vcpu->ioctl = 0.
Yeah, this is super racy, thanks for pointing that out.
>
> I think that other (special) callsites of vcpu_load()/vcpu_put() have a
> well defined IOCTL that can be used instead of vcpu->ioctl, so we could
> just pass the ioctl value all the way to arch code and never save it
> anywhere,
I don't think that works; what would you do with preempt notifier calls?
One solution is to add a parameter to vcpu_put, lie for vcpu_load, which
also sets the ioctl, and other callers than the final vcpu_put in
kvm_vcpu_ioctl() just pass the existing value, where the kvm_vcpu_ioctl
call can pass 0 which gets set before releasing the mutex.
Can you think of a more elegant solution?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-10-13 17:31 ` Christoffer Dall
@ 2017-10-13 18:38 ` Radim Krčmář
-1 siblings, 0 replies; 254+ messages in thread
From: Radim Krčmář @ 2017-10-13 18:38 UTC (permalink / raw)
To: Christoffer Dall
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li, Paolo Bonzini
2017-10-13 19:31+0200, Christoffer Dall:
> On Fri, Oct 13, 2017 at 07:13:07PM +0200, Radim Krčmář wrote:
> > I think that other (special) callsites of vcpu_load()/vcpu_put() have a
> > well defined IOCTL that can be used instead of vcpu->ioctl, so we could
> > just pass the ioctl value all the way to arch code and never save it
> > anywhere,
>
> I don't think that works; what would you do with preempt notifier calls?
Right, BUG :), I didn't consider them before and they need to know.
> One solution is to add a parameter to vcpu_put, lie for vcpu_load, which
> also sets the ioctl, and other callers than the final vcpu_put in
> kvm_vcpu_ioctl() just pass the existing value, where the kvm_vcpu_ioctl
> call can pass 0 which gets set before releasing the mutex.
>
> Can you think of a more elegant solution?
Not really, only thought of touching preempt notifiers and it seems to
be more complicated.
I think we shouldn't restore ioctl on vcpu_put() at all -- the value
isn't well defined outside of the mutex, so there is no point in looking
and we can just zero the ioctl.
Actually, I wouldn't rely on the existing value at all because that.
The need for load/put depends on the current code path, not on the one
we race with.
x86 seems to be the only user of vcpu_load() outside of kvm_vcpu_ioctl()
and the callers are either under a VM ioctl or under a VM destruction
paths (invalid IOCTL) and we can just hardcode that.
Passing 0 to all other vcpu_load()s and unconditionally zeroing ioctl
before mutex_unlock() should work.
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-10-13 18:38 ` Radim Krčmář
0 siblings, 0 replies; 254+ messages in thread
From: Radim Krčmář @ 2017-10-13 18:38 UTC (permalink / raw)
To: linux-arm-kernel
2017-10-13 19:31+0200, Christoffer Dall:
> On Fri, Oct 13, 2017 at 07:13:07PM +0200, Radim Kr?m?? wrote:
> > I think that other (special) callsites of vcpu_load()/vcpu_put() have a
> > well defined IOCTL that can be used instead of vcpu->ioctl, so we could
> > just pass the ioctl value all the way to arch code and never save it
> > anywhere,
>
> I don't think that works; what would you do with preempt notifier calls?
Right, BUG :), I didn't consider them before and they need to know.
> One solution is to add a parameter to vcpu_put, lie for vcpu_load, which
> also sets the ioctl, and other callers than the final vcpu_put in
> kvm_vcpu_ioctl() just pass the existing value, where the kvm_vcpu_ioctl
> call can pass 0 which gets set before releasing the mutex.
>
> Can you think of a more elegant solution?
Not really, only thought of touching preempt notifiers and it seems to
be more complicated.
I think we shouldn't restore ioctl on vcpu_put() at all -- the value
isn't well defined outside of the mutex, so there is no point in looking
and we can just zero the ioctl.
Actually, I wouldn't rely on the existing value at all because that.
The need for load/put depends on the current code path, not on the one
we race with.
x86 seems to be the only user of vcpu_load() outside of kvm_vcpu_ioctl()
and the callers are either under a VM ioctl or under a VM destruction
paths (invalid IOCTL) and we can just hardcode that.
Passing 0 to all other vcpu_load()s and unconditionally zeroing ioctl
before mutex_unlock() should work.
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-10-13 18:38 ` Radim Krčmář
@ 2017-10-13 18:51 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-13 18:51 UTC (permalink / raw)
To: Radim Krčmář
Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li, Paolo Bonzini
On Fri, Oct 13, 2017 at 8:38 PM, Radim Krčmář <rkrcmar@redhat.com> wrote:
> 2017-10-13 19:31+0200, Christoffer Dall:
>> On Fri, Oct 13, 2017 at 07:13:07PM +0200, Radim Krčmář wrote:
>> > I think that other (special) callsites of vcpu_load()/vcpu_put() have a
>> > well defined IOCTL that can be used instead of vcpu->ioctl, so we could
>> > just pass the ioctl value all the way to arch code and never save it
>> > anywhere,
>>
>> I don't think that works; what would you do with preempt notifier calls?
>
> Right, BUG :), I didn't consider them before and they need to know.
>
>> One solution is to add a parameter to vcpu_put, lie for vcpu_load, which
>> also sets the ioctl, and other callers than the final vcpu_put in
>> kvm_vcpu_ioctl() just pass the existing value, where the kvm_vcpu_ioctl
>> call can pass 0 which gets set before releasing the mutex.
>>
>> Can you think of a more elegant solution?
>
> Not really, only thought of touching preempt notifiers and it seems to
> be more complicated.
>
> I think we shouldn't restore ioctl on vcpu_put() at all -- the value
> isn't well defined outside of the mutex, so there is no point in looking
> and we can just zero the ioctl.
I agree, that fits with the semantics of vcpu_put() quit nicely actually.
>
> Actually, I wouldn't rely on the existing value at all because that.
> The need for load/put depends on the current code path, not on the one
> we race with.
>
> x86 seems to be the only user of vcpu_load() outside of kvm_vcpu_ioctl()
> and the callers are either under a VM ioctl or under a VM destruction
> paths (invalid IOCTL) and we can just hardcode that.
Yes, we should. I honestly just wanted feedback on this before
tracing through all call paths to the vcpu_load() calls on the x86
side.
>
> Passing 0 to all other vcpu_load()s and unconditionally zeroing ioctl
> before mutex_unlock() should work.
Agreed. I'll do that for the next version.
Thanks for having a look!
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-10-13 18:51 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-13 18:51 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 13, 2017 at 8:38 PM, Radim Kr?m?? <rkrcmar@redhat.com> wrote:
> 2017-10-13 19:31+0200, Christoffer Dall:
>> On Fri, Oct 13, 2017 at 07:13:07PM +0200, Radim Kr?m?? wrote:
>> > I think that other (special) callsites of vcpu_load()/vcpu_put() have a
>> > well defined IOCTL that can be used instead of vcpu->ioctl, so we could
>> > just pass the ioctl value all the way to arch code and never save it
>> > anywhere,
>>
>> I don't think that works; what would you do with preempt notifier calls?
>
> Right, BUG :), I didn't consider them before and they need to know.
>
>> One solution is to add a parameter to vcpu_put, lie for vcpu_load, which
>> also sets the ioctl, and other callers than the final vcpu_put in
>> kvm_vcpu_ioctl() just pass the existing value, where the kvm_vcpu_ioctl
>> call can pass 0 which gets set before releasing the mutex.
>>
>> Can you think of a more elegant solution?
>
> Not really, only thought of touching preempt notifiers and it seems to
> be more complicated.
>
> I think we shouldn't restore ioctl on vcpu_put() at all -- the value
> isn't well defined outside of the mutex, so there is no point in looking
> and we can just zero the ioctl.
I agree, that fits with the semantics of vcpu_put() quit nicely actually.
>
> Actually, I wouldn't rely on the existing value at all because that.
> The need for load/put depends on the current code path, not on the one
> we race with.
>
> x86 seems to be the only user of vcpu_load() outside of kvm_vcpu_ioctl()
> and the callers are either under a VM ioctl or under a VM destruction
> paths (invalid IOCTL) and we can just hardcode that.
Yes, we should. I honestly just wanted feedback on this before
tracing through all call paths to the vcpu_load() calls on the x86
side.
>
> Passing 0 to all other vcpu_load()s and unconditionally zeroing ioctl
> before mutex_unlock() should work.
Agreed. I'll do that for the next version.
Thanks for having a look!
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 10:45 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 10:45 UTC (permalink / raw)
To: Christoffer Dall
Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li,
Paolo Bonzini, Radim Krčmář
On Thu, Oct 12, 2017 at 12:41:09PM +0200, Christoffer Dall wrote:
> Some architectures may decide to do different things during
> kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> arm64 is about to do significant work in vcpu load/put when running a
> vcpu, but not when doing things like KVM_SET_ONE_REG or
> KVM_SET_MP_STATE.
>
> Therefore, store the ioctl number that we are executing on the VCPU
> during the first vcpu_load() which succeeds in getting the vcpu->mutex
> and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> successfully loading the vcpu.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/x86/kvm/vmx.c | 2 +-
> arch/x86/kvm/x86.c | 8 ++++----
> include/linux/kvm_host.h | 3 ++-
> virt/kvm/kvm_main.c | 6 ++++--
> 4 files changed, 11 insertions(+), 8 deletions(-)
>
I wonder if enough other architectures would be able to benefit from this
for most/all of their non-RUN VCPU ioctls. If so, then maybe we should
consider doing something like
int __vcpu_load(struct kvm_vcpu *vcpu)
{
int cpu;
cpu = get_cpu();
preempt_notifier_register(&vcpu->preempt_notifier);
kvm_arch_vcpu_load(vcpu, cpu);
put_cpu();
return 0;
}
int vcpu_load(struct kvm_vcpu *vcpu)
{
if (mutex_lock_killable(&vcpu->mutex))
return -EINTR;
return __vcpu_load(vcpu);
}
and the equivalent for vcpu_put.
Then just take the lock in kvm_vcpu_ioctl and leave it to the
kvm_arch_vcpu_ioctl_* functions to call __vcpu_load/__vcpu_put
if necessary.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-11-07 10:45 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 10:45 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:09PM +0200, Christoffer Dall wrote:
> Some architectures may decide to do different things during
> kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> arm64 is about to do significant work in vcpu load/put when running a
> vcpu, but not when doing things like KVM_SET_ONE_REG or
> KVM_SET_MP_STATE.
>
> Therefore, store the ioctl number that we are executing on the VCPU
> during the first vcpu_load() which succeeds in getting the vcpu->mutex
> and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> successfully loading the vcpu.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Kr?m?? <rkrcmar@redhat.com>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/x86/kvm/vmx.c | 2 +-
> arch/x86/kvm/x86.c | 8 ++++----
> include/linux/kvm_host.h | 3 ++-
> virt/kvm/kvm_main.c | 6 ++++--
> 4 files changed, 11 insertions(+), 8 deletions(-)
>
I wonder if enough other architectures would be able to benefit from this
for most/all of their non-RUN VCPU ioctls. If so, then maybe we should
consider doing something like
int __vcpu_load(struct kvm_vcpu *vcpu)
{
int cpu;
cpu = get_cpu();
preempt_notifier_register(&vcpu->preempt_notifier);
kvm_arch_vcpu_load(vcpu, cpu);
put_cpu();
return 0;
}
int vcpu_load(struct kvm_vcpu *vcpu)
{
if (mutex_lock_killable(&vcpu->mutex))
return -EINTR;
return __vcpu_load(vcpu);
}
and the equivalent for vcpu_put.
Then just take the lock in kvm_vcpu_ioctl and leave it to the
kvm_arch_vcpu_ioctl_* functions to call __vcpu_load/__vcpu_put
if necessary.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
2017-11-07 10:45 ` Andrew Jones
@ 2017-11-22 20:28 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-22 20:28 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li, Paolo Bonzini, Radim Krčmář
On Tue, Nov 07, 2017 at 11:45:01AM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:09PM +0200, Christoffer Dall wrote:
> > Some architectures may decide to do different things during
> > kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> > arm64 is about to do significant work in vcpu load/put when running a
> > vcpu, but not when doing things like KVM_SET_ONE_REG or
> > KVM_SET_MP_STATE.
> >
> > Therefore, store the ioctl number that we are executing on the VCPU
> > during the first vcpu_load() which succeeds in getting the vcpu->mutex
> > and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> > successfully loading the vcpu.
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Krčmář <rkrcmar@redhat.com>
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/x86/kvm/vmx.c | 2 +-
> > arch/x86/kvm/x86.c | 8 ++++----
> > include/linux/kvm_host.h | 3 ++-
> > virt/kvm/kvm_main.c | 6 ++++--
> > 4 files changed, 11 insertions(+), 8 deletions(-)
> >
>
> I wonder if enough other architectures would be able to benefit from this
> for most/all of their non-RUN VCPU ioctls. If so, then maybe we should
> consider doing something like
>
> int __vcpu_load(struct kvm_vcpu *vcpu)
> {
> int cpu;
>
> cpu = get_cpu();
> preempt_notifier_register(&vcpu->preempt_notifier);
> kvm_arch_vcpu_load(vcpu, cpu);
> put_cpu();
> return 0;
> }
> int vcpu_load(struct kvm_vcpu *vcpu)
> {
> if (mutex_lock_killable(&vcpu->mutex))
> return -EINTR;
> return __vcpu_load(vcpu);
> }
>
> and the equivalent for vcpu_put.
>
> Then just take the lock in kvm_vcpu_ioctl and leave it to the
> kvm_arch_vcpu_ioctl_* functions to call __vcpu_load/__vcpu_put
> if necessary.
>
That seems like it would be a pretty invasive change, because we change
the semantics of when kvm_arch_vcpu_load() is called, and I'm not really
sure when it then does/doesn't make sense to call __vcpu_load/__vcpu_put
for each architecture.
But on ARM we only need preempt_notifiers for KVM_RUN, so if it's
similar on other architectures, it might not be that bad.
Was your intention about vcpu_load() then that x86 could remain
unmodified?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 05/37] KVM: Record the executing ioctl number on the vcpu struct
@ 2017-11-22 20:28 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-22 20:28 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 11:45:01AM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:09PM +0200, Christoffer Dall wrote:
> > Some architectures may decide to do different things during
> > kvm_arch_vcpu_load depending on the ioctl being executed. For example,
> > arm64 is about to do significant work in vcpu load/put when running a
> > vcpu, but not when doing things like KVM_SET_ONE_REG or
> > KVM_SET_MP_STATE.
> >
> > Therefore, store the ioctl number that we are executing on the VCPU
> > during the first vcpu_load() which succeeds in getting the vcpu->mutex
> > and set the ioctl number to 0 when exiting kvm_vcpu_ioctl() after
> > successfully loading the vcpu.
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Kr?m?? <rkrcmar@redhat.com>
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/x86/kvm/vmx.c | 2 +-
> > arch/x86/kvm/x86.c | 8 ++++----
> > include/linux/kvm_host.h | 3 ++-
> > virt/kvm/kvm_main.c | 6 ++++--
> > 4 files changed, 11 insertions(+), 8 deletions(-)
> >
>
> I wonder if enough other architectures would be able to benefit from this
> for most/all of their non-RUN VCPU ioctls. If so, then maybe we should
> consider doing something like
>
> int __vcpu_load(struct kvm_vcpu *vcpu)
> {
> int cpu;
>
> cpu = get_cpu();
> preempt_notifier_register(&vcpu->preempt_notifier);
> kvm_arch_vcpu_load(vcpu, cpu);
> put_cpu();
> return 0;
> }
> int vcpu_load(struct kvm_vcpu *vcpu)
> {
> if (mutex_lock_killable(&vcpu->mutex))
> return -EINTR;
> return __vcpu_load(vcpu);
> }
>
> and the equivalent for vcpu_put.
>
> Then just take the lock in kvm_vcpu_ioctl and leave it to the
> kvm_arch_vcpu_ioctl_* functions to call __vcpu_load/__vcpu_put
> if necessary.
>
That seems like it would be a pretty invasive change, because we change
the semantics of when kvm_arch_vcpu_load() is called, and I'm not really
sure when it then does/doesn't make sense to call __vcpu_load/__vcpu_put
for each architecture.
But on ARM we only need preempt_notifiers for KVM_RUN, so if it's
similar on other architectures, it might not be that bad.
Was your intention about vcpu_load() then that x86 could remain
unmodified?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 06/37] KVM: arm/arm64: Only load/put VCPU state for KVM_RUN
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: kvm, Marc Zyngier, Shih-Wei Li, Christoffer Dall
We only want to do things like invalidating TLBs or load timer state
onto the physical CPU if we really intend to run the VCPU, not if we are
simply retrieving some in-kernel value from userspace, for example.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
virt/kvm/arm/arm.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 6e9513e..d495453 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -338,6 +338,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
int *last_ran;
+ if (vcpu->ioctl != KVM_RUN)
+ return;
+
last_ran = this_cpu_ptr(vcpu->kvm->arch.last_vcpu_ran);
/*
@@ -359,6 +362,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
+ if (vcpu->ioctl != KVM_RUN)
+ return;
+
kvm_timer_vcpu_put(vcpu);
kvm_vgic_put(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 06/37] KVM: arm/arm64: Only load/put VCPU state for KVM_RUN
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We only want to do things like invalidating TLBs or load timer state
onto the physical CPU if we really intend to run the VCPU, not if we are
simply retrieving some in-kernel value from userspace, for example.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
virt/kvm/arm/arm.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 6e9513e..d495453 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -338,6 +338,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
int *last_ran;
+ if (vcpu->ioctl != KVM_RUN)
+ return;
+
last_ran = this_cpu_ptr(vcpu->kvm->arch.last_vcpu_ran);
/*
@@ -359,6 +362,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
+ if (vcpu->ioctl != KVM_RUN)
+ return;
+
kvm_timer_vcpu_put(vcpu);
kvm_vgic_put(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
As we are about to move a buch of save/restore logic for VHE kernels to
the load and put functions, we need some infrastructure to do this.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_host.h | 3 +++
arch/arm64/include/asm/kvm_host.h | 3 +++
arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
virt/kvm/arm/arm.c | 2 ++
4 files changed, 35 insertions(+)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 1100170..13f8165 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
struct kvm_device_attr *attr);
+static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
+static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
+
#endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 27305e7..7d3bfa7 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
"PARange is %d bits, unsupported configuration!", parange);
}
+void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
+void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
+
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index c54cc2a..b7438c8 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
}
+
+/**
+ * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * If the kernel runs in EL2 then load the system register state for the VCPU
+ * for EL1 onto the physical CPU so that we can go back and foward between the
+ * VM and the hypervisor without switching all this state around.
+ */
+void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
+{
+}
+
+/**
+ * kvm_vcpu_put_sysregs - Restore host system register state to physical CPU
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * If the kernel runs in EL2 and the physical register state belongs to the
+ * VCPU, then restore the system register state for the host for EL1 onto the
+ * physical CPU so that we can run userspace and other threads on this
+ * physical CPU.
+ */
+void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
+{
+}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index d495453..cf121b2 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -358,6 +358,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
kvm_arm_set_running_vcpu(vcpu);
kvm_vgic_load(vcpu);
kvm_timer_vcpu_load(vcpu);
+ kvm_vcpu_load_sysregs(vcpu);
}
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
@@ -365,6 +366,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
if (vcpu->ioctl != KVM_RUN)
return;
+ kvm_vcpu_put_sysregs(vcpu);
kvm_timer_vcpu_put(vcpu);
kvm_vgic_put(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
As we are about to move a buch of save/restore logic for VHE kernels to
the load and put functions, we need some infrastructure to do this.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_host.h | 3 +++
arch/arm64/include/asm/kvm_host.h | 3 +++
arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
virt/kvm/arm/arm.c | 2 ++
4 files changed, 35 insertions(+)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 1100170..13f8165 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
struct kvm_device_attr *attr);
+static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
+static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
+
#endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 27305e7..7d3bfa7 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
"PARange is %d bits, unsupported configuration!", parange);
}
+void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
+void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
+
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index c54cc2a..b7438c8 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
}
+
+/**
+ * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * If the kernel runs in EL2 then load the system register state for the VCPU
+ * for EL1 onto the physical CPU so that we can go back and foward between the
+ * VM and the hypervisor without switching all this state around.
+ */
+void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
+{
+}
+
+/**
+ * kvm_vcpu_put_sysregs - Restore host system register state to physical CPU
+ *
+ * @vcpu: The VCPU pointer
+ *
+ * If the kernel runs in EL2 and the physical register state belongs to the
+ * VCPU, then restore the system register state for the host for EL1 onto the
+ * physical CPU so that we can run userspace and other threads on this
+ * physical CPU.
+ */
+void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
+{
+}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index d495453..cf121b2 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -358,6 +358,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
kvm_arm_set_running_vcpu(vcpu);
kvm_vgic_load(vcpu);
kvm_timer_vcpu_load(vcpu);
+ kvm_vcpu_load_sysregs(vcpu);
}
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
@@ -365,6 +366,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
if (vcpu->ioctl != KVM_RUN)
return;
+ kvm_vcpu_put_sysregs(vcpu);
kvm_timer_vcpu_put(vcpu);
kvm_vgic_put(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 10:56 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 10:56 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel, kvm
On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> As we are about to move a buch of save/restore logic for VHE kernels to
> the load and put functions, we need some infrastructure to do this.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> virt/kvm/arm/arm.c | 2 ++
> 4 files changed, 35 insertions(+)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 1100170..13f8165 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 27305e7..7d3bfa7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index c54cc2a..b7438c8 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> }
> +
> +/**
> + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
system registers to the physical CPU
> + *
> + * @vcpu: The VCPU pointer
> + *
> + * If the kernel runs in EL2 then load the system register state for the VCPU
> + * for EL1 onto the physical CPU so that we can go back and foward between the
/foward/forth/
> + * VM and the hypervisor without switching all this state around.
> + */
> +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> +{
> +}
> +
> +/**
> + * kvm_vcpu_put_sysregs - Restore host system register state to physical CPU
system registers to the physical CPU
> + *
> + * @vcpu: The VCPU pointer
> + *
> + * If the kernel runs in EL2 and the physical register state belongs to the
> + * VCPU, then restore the system register state for the host for EL1 onto the
> + * physical CPU so that we can run userspace and other threads on this
> + * physical CPU.
> + */
> +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> +{
> +}
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index d495453..cf121b2 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -358,6 +358,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> kvm_arm_set_running_vcpu(vcpu);
> kvm_vgic_load(vcpu);
> kvm_timer_vcpu_load(vcpu);
> + kvm_vcpu_load_sysregs(vcpu);
> }
>
> void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> @@ -365,6 +366,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> if (vcpu->ioctl != KVM_RUN)
> return;
>
> + kvm_vcpu_put_sysregs(vcpu);
> kvm_timer_vcpu_put(vcpu);
> kvm_vgic_put(vcpu);
>
> --
> 2.9.0
>
I'm not sure this needs to be a separate patch, but it was a nice relaxing
review :-)
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
@ 2017-11-07 10:56 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 10:56 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> As we are about to move a buch of save/restore logic for VHE kernels to
> the load and put functions, we need some infrastructure to do this.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> virt/kvm/arm/arm.c | 2 ++
> 4 files changed, 35 insertions(+)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 1100170..13f8165 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 27305e7..7d3bfa7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index c54cc2a..b7438c8 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> }
> +
> +/**
> + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
system registers to the physical CPU
> + *
> + * @vcpu: The VCPU pointer
> + *
> + * If the kernel runs in EL2 then load the system register state for the VCPU
> + * for EL1 onto the physical CPU so that we can go back and foward between the
/foward/forth/
> + * VM and the hypervisor without switching all this state around.
> + */
> +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> +{
> +}
> +
> +/**
> + * kvm_vcpu_put_sysregs - Restore host system register state to physical CPU
system registers to the physical CPU
> + *
> + * @vcpu: The VCPU pointer
> + *
> + * If the kernel runs in EL2 and the physical register state belongs to the
> + * VCPU, then restore the system register state for the host for EL1 onto the
> + * physical CPU so that we can run userspace and other threads on this
> + * physical CPU.
> + */
> +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> +{
> +}
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index d495453..cf121b2 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -358,6 +358,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> kvm_arm_set_running_vcpu(vcpu);
> kvm_vgic_load(vcpu);
> kvm_timer_vcpu_load(vcpu);
> + kvm_vcpu_load_sysregs(vcpu);
> }
>
> void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> @@ -365,6 +366,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> if (vcpu->ioctl != KVM_RUN)
> return;
>
> + kvm_vcpu_put_sysregs(vcpu);
> kvm_timer_vcpu_put(vcpu);
> kvm_vgic_put(vcpu);
>
> --
> 2.9.0
>
I'm not sure this needs to be a separate patch, but it was a nice relaxing
review :-)
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 11:10 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 11:10 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, Marc Zyngier, Shih-Wei Li, kvm
On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> As we are about to move a buch of save/restore logic for VHE kernels to
> the load and put functions, we need some infrastructure to do this.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> virt/kvm/arm/arm.c | 2 ++
> 4 files changed, 35 insertions(+)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 1100170..13f8165 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 27305e7..7d3bfa7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index c54cc2a..b7438c8 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> }
> +
> +/**
> + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
> + *
> + * @vcpu: The VCPU pointer
> + *
> + * If the kernel runs in EL2 then load the system register state for the VCPU
> + * for EL1 onto the physical CPU so that we can go back and foward between the
> + * VM and the hypervisor without switching all this state around.
Actually, on second thought, I'm a bit confused by this comment. Maybe
it'll be clearer after the code that goes here will be added, but, ATM,
I'm not sure why we want to specifically load EL1 VCPU state here. Will
that always be the case, even with nested? Also, I'm not sure what
alternative loading scheme we're avoiding in order to ensure the state
isn't "switched around", which I don't really understand either.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
@ 2017-11-07 11:10 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 11:10 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> As we are about to move a buch of save/restore logic for VHE kernels to
> the load and put functions, we need some infrastructure to do this.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> virt/kvm/arm/arm.c | 2 ++
> 4 files changed, 35 insertions(+)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 1100170..13f8165 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 27305e7..7d3bfa7 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index c54cc2a..b7438c8 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> }
> +
> +/**
> + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
> + *
> + * @vcpu: The VCPU pointer
> + *
> + * If the kernel runs in EL2 then load the system register state for the VCPU
> + * for EL1 onto the physical CPU so that we can go back and foward between the
> + * VM and the hypervisor without switching all this state around.
Actually, on second thought, I'm a bit confused by this comment. Maybe
it'll be clearer after the code that goes here will be added, but, ATM,
I'm not sure why we want to specifically load EL1 VCPU state here. Will
that always be the case, even with nested? Also, I'm not sure what
alternative loading scheme we're avoiding in order to ensure the state
isn't "switched around", which I don't really understand either.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
2017-11-07 11:10 ` Andrew Jones
@ 2017-11-22 20:34 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-22 20:34 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Tue, Nov 07, 2017 at 12:10:00PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> > As we are about to move a buch of save/restore logic for VHE kernels to
> > the load and put functions, we need some infrastructure to do this.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_host.h | 3 +++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> > virt/kvm/arm/arm.c | 2 ++
> > 4 files changed, 35 insertions(+)
> >
> > diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> > index 1100170..13f8165 100644
> > --- a/arch/arm/include/asm/kvm_host.h
> > +++ b/arch/arm/include/asm/kvm_host.h
> > @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> > int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> > struct kvm_device_attr *attr);
> >
> > +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> > +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> > +
> > #endif /* __ARM_KVM_HOST_H__ */
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 27305e7..7d3bfa7 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> > "PARange is %d bits, unsupported configuration!", parange);
> > }
> >
> > +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> > +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> > +
> > #endif /* __ARM64_KVM_HOST_H__ */
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index c54cc2a..b7438c8 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> > }
> > +
> > +/**
> > + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
> > + *
> > + * @vcpu: The VCPU pointer
> > + *
> > + * If the kernel runs in EL2 then load the system register state for the VCPU
> > + * for EL1 onto the physical CPU so that we can go back and foward between the
> > + * VM and the hypervisor without switching all this state around.
>
> Actually, on second thought, I'm a bit confused by this comment. Maybe
> it'll be clearer after the code that goes here will be added, but, ATM,
> I'm not sure why we want to specifically load EL1 VCPU state here. Will
> that always be the case, even with nested? Also, I'm not sure what
> alternative loading scheme we're avoiding in order to ensure the state
> isn't "switched around", which I don't really understand either.
>
I was referring to the EL1 hardware state. Even for nesting, that's the
best we can do, because we can't load EL2 state if we're running in EL2,
or EL1 state if we're running in EL1, because we'd be shooting ourselves
in the foot.
The alternative loading scheme is what we have today, where we save and
restore everything every time we exit/enter the VM, even though we don't
have to.
How about:
/*
* Load system registers that do not affect the host's execution, for
* example EL1 system registers on a VHE system where the host kernel
* runs at EL2. This function is called from KVM's vcpu_load() function
* and loading system register state early avoids having to load them on
* every entry to the VM.
*/
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
@ 2017-11-22 20:34 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-22 20:34 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 12:10:00PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> > As we are about to move a buch of save/restore logic for VHE kernels to
> > the load and put functions, we need some infrastructure to do this.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_host.h | 3 +++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> > virt/kvm/arm/arm.c | 2 ++
> > 4 files changed, 35 insertions(+)
> >
> > diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> > index 1100170..13f8165 100644
> > --- a/arch/arm/include/asm/kvm_host.h
> > +++ b/arch/arm/include/asm/kvm_host.h
> > @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> > int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> > struct kvm_device_attr *attr);
> >
> > +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> > +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> > +
> > #endif /* __ARM_KVM_HOST_H__ */
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 27305e7..7d3bfa7 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> > "PARange is %d bits, unsupported configuration!", parange);
> > }
> >
> > +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> > +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> > +
> > #endif /* __ARM64_KVM_HOST_H__ */
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index c54cc2a..b7438c8 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> > }
> > +
> > +/**
> > + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
> > + *
> > + * @vcpu: The VCPU pointer
> > + *
> > + * If the kernel runs in EL2 then load the system register state for the VCPU
> > + * for EL1 onto the physical CPU so that we can go back and foward between the
> > + * VM and the hypervisor without switching all this state around.
>
> Actually, on second thought, I'm a bit confused by this comment. Maybe
> it'll be clearer after the code that goes here will be added, but, ATM,
> I'm not sure why we want to specifically load EL1 VCPU state here. Will
> that always be the case, even with nested? Also, I'm not sure what
> alternative loading scheme we're avoiding in order to ensure the state
> isn't "switched around", which I don't really understand either.
>
I was referring to the EL1 hardware state. Even for nesting, that's the
best we can do, because we can't load EL2 state if we're running in EL2,
or EL1 state if we're running in EL1, because we'd be shooting ourselves
in the foot.
The alternative loading scheme is what we have today, where we save and
restore everything every time we exit/enter the VM, even though we don't
have to.
How about:
/*
* Load system registers that do not affect the host's execution, for
* example EL1 system registers on a VHE system where the host kernel
* runs at EL2. This function is called from KVM's vcpu_load() function
* and loading system register state early avoids having to load them on
* every entry to the VM.
*/
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
2017-11-22 20:34 ` Christoffer Dall
@ 2017-11-23 11:08 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-23 11:08 UTC (permalink / raw)
To: Christoffer Dall
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Wed, Nov 22, 2017 at 09:34:17PM +0100, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 12:10:00PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> > > As we are about to move a buch of save/restore logic for VHE kernels to
> > > the load and put functions, we need some infrastructure to do this.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm/include/asm/kvm_host.h | 3 +++
> > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> > > virt/kvm/arm/arm.c | 2 ++
> > > 4 files changed, 35 insertions(+)
> > >
> > > diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> > > index 1100170..13f8165 100644
> > > --- a/arch/arm/include/asm/kvm_host.h
> > > +++ b/arch/arm/include/asm/kvm_host.h
> > > @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> > > int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> > > struct kvm_device_attr *attr);
> > >
> > > +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> > > +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> > > +
> > > #endif /* __ARM_KVM_HOST_H__ */
> > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > index 27305e7..7d3bfa7 100644
> > > --- a/arch/arm64/include/asm/kvm_host.h
> > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> > > "PARange is %d bits, unsupported configuration!", parange);
> > > }
> > >
> > > +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> > > +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> > > +
> > > #endif /* __ARM64_KVM_HOST_H__ */
> > > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > > index c54cc2a..b7438c8 100644
> > > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > > @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> > > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > > write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> > > }
> > > +
> > > +/**
> > > + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
> > > + *
> > > + * @vcpu: The VCPU pointer
> > > + *
> > > + * If the kernel runs in EL2 then load the system register state for the VCPU
> > > + * for EL1 onto the physical CPU so that we can go back and foward between the
> > > + * VM and the hypervisor without switching all this state around.
> >
> > Actually, on second thought, I'm a bit confused by this comment. Maybe
> > it'll be clearer after the code that goes here will be added, but, ATM,
> > I'm not sure why we want to specifically load EL1 VCPU state here. Will
> > that always be the case, even with nested? Also, I'm not sure what
> > alternative loading scheme we're avoiding in order to ensure the state
> > isn't "switched around", which I don't really understand either.
> >
>
> I was referring to the EL1 hardware state. Even for nesting, that's the
> best we can do, because we can't load EL2 state if we're running in EL2,
> or EL1 state if we're running in EL1, because we'd be shooting ourselves
> in the foot.
>
> The alternative loading scheme is what we have today, where we save and
> restore everything every time we exit/enter the VM, even though we don't
> have to.
>
> How about:
>
> /*
> * Load system registers that do not affect the host's execution, for
> * example EL1 system registers on a VHE system where the host kernel
> * runs at EL2. This function is called from KVM's vcpu_load() function
> * and loading system register state early avoids having to load them on
> * every entry to the VM.
> */
Yeah, I like this. Thanks for the clarification.
(I guess it's time for me to return to reviewing this series. Sorry for
stalling out on it.)
drew
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 07/37] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs
@ 2017-11-23 11:08 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-23 11:08 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 22, 2017 at 09:34:17PM +0100, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 12:10:00PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:11PM +0200, Christoffer Dall wrote:
> > > As we are about to move a buch of save/restore logic for VHE kernels to
> > > the load and put functions, we need some infrastructure to do this.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm/include/asm/kvm_host.h | 3 +++
> > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > arch/arm64/kvm/hyp/sysreg-sr.c | 27 +++++++++++++++++++++++++++
> > > virt/kvm/arm/arm.c | 2 ++
> > > 4 files changed, 35 insertions(+)
> > >
> > > diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> > > index 1100170..13f8165 100644
> > > --- a/arch/arm/include/asm/kvm_host.h
> > > +++ b/arch/arm/include/asm/kvm_host.h
> > > @@ -290,4 +290,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> > > int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> > > struct kvm_device_attr *attr);
> > >
> > > +static inline void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu) {}
> > > +static inline void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu) {}
> > > +
> > > #endif /* __ARM_KVM_HOST_H__ */
> > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > index 27305e7..7d3bfa7 100644
> > > --- a/arch/arm64/include/asm/kvm_host.h
> > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > @@ -383,4 +383,7 @@ static inline void __cpu_init_stage2(void)
> > > "PARange is %d bits, unsupported configuration!", parange);
> > > }
> > >
> > > +void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu);
> > > +void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu);
> > > +
> > > #endif /* __ARM64_KVM_HOST_H__ */
> > > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > > index c54cc2a..b7438c8 100644
> > > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > > @@ -183,3 +183,30 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> > > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > > write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> > > }
> > > +
> > > +/**
> > > + * kvm_vcpu_load_sysregs - Load guest system register to physical CPU
> > > + *
> > > + * @vcpu: The VCPU pointer
> > > + *
> > > + * If the kernel runs in EL2 then load the system register state for the VCPU
> > > + * for EL1 onto the physical CPU so that we can go back and foward between the
> > > + * VM and the hypervisor without switching all this state around.
> >
> > Actually, on second thought, I'm a bit confused by this comment. Maybe
> > it'll be clearer after the code that goes here will be added, but, ATM,
> > I'm not sure why we want to specifically load EL1 VCPU state here. Will
> > that always be the case, even with nested? Also, I'm not sure what
> > alternative loading scheme we're avoiding in order to ensure the state
> > isn't "switched around", which I don't really understand either.
> >
>
> I was referring to the EL1 hardware state. Even for nesting, that's the
> best we can do, because we can't load EL2 state if we're running in EL2,
> or EL1 state if we're running in EL1, because we'd be shooting ourselves
> in the foot.
>
> The alternative loading scheme is what we have today, where we save and
> restore everything every time we exit/enter the VM, even though we don't
> have to.
>
> How about:
>
> /*
> * Load system registers that do not affect the host's execution, for
> * example EL1 system registers on a VHE system where the host kernel
> * runs at EL2. This function is called from KVM's vcpu_load() function
> * and loading system register state early avoids having to load them on
> * every entry to the VM.
> */
Yeah, I like this. Thanks for the clarification.
(I guess it's time for me to return to reviewing this series. Sorry for
stalling out on it.)
drew
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Avoid saving the guest VFP registers and restoring the host VFP
registers on every exit from the VM. Only when we're about to run
userspace or other threads in the kernel do we really have to switch the
state back to the host state.
We still initially configure the VFP registers to trap when entering the
VM, but the difference is that we now leave the guest state in the
hardware registers while running the VM.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_emulate.h | 5 ++++
arch/arm64/include/asm/kvm_host.h | 3 +++
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kvm/hyp/entry.S | 3 +++
arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
6 files changed, 44 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 1fbfe96..630dd60 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
return (unsigned long *)&vcpu->arch.hcr_el2;
}
+static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
+{
+ return (!(vcpu->arch.hcr_el2 & HCR_RW));
+}
+
static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
{
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 7d3bfa7..5e09eb9 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
/* Guest debug state */
u64 debug_flags;
+ /* 1 if the guest VFP state is loaded into the hardware */
+ u64 guest_vfp_loaded;
+
/*
* We maintain more than a single set of debug registers to support
* debugging the guest from the host and to maintain separate host and
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 612021d..9946732 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -133,6 +133,7 @@ int main(void)
DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
+ DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 76cd48f..b3e7191 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -185,6 +185,9 @@ alternative_endif
add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
bl __fpsimd_restore_state
+ mov x0, #1
+ str x0, [x3, #VCPU_GUEST_VFP_LOADED]
+
// Skip restoring fpexc32 for AArch64 guests
mrs x1, hcr_el2
tbnz x1, #HCR_RW_SHIFT, 1f
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 7703d63..ef05c59 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -23,43 +23,31 @@
#include <asm/kvm_hyp.h>
#include <asm/fpsimd.h>
-static bool __hyp_text __fpsimd_enabled_nvhe(void)
-{
- return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
-}
-
-static bool __hyp_text __fpsimd_enabled_vhe(void)
-{
- return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
-}
-
-static hyp_alternate_select(__fpsimd_is_enabled,
- __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
-bool __hyp_text __fpsimd_enabled(void)
-{
- return __fpsimd_is_enabled()();
-}
-
-static void __hyp_text __activate_traps_vhe(void)
+static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
{
u64 val;
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
- val &= ~CPACR_EL1_FPEN;
+ if (vcpu->arch.guest_vfp_loaded)
+ val |= CPACR_EL1_FPEN;
+ else
+ val &= ~CPACR_EL1_FPEN;
write_sysreg(val, cpacr_el1);
write_sysreg(__kvm_hyp_vector, vbar_el1);
}
-static void __hyp_text __activate_traps_nvhe(void)
+static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
{
u64 val;
val = CPTR_EL2_DEFAULT;
- val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
+ val |= CPTR_EL2_TTA;
+ if (vcpu->arch.guest_vfp_loaded)
+ val &= ~CPTR_EL2_TFP;
+ else
+ val |= CPTR_EL2_TFP;
write_sysreg(val, cptr_el2);
}
@@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
* it will cause an exception.
*/
val = vcpu->arch.hcr_el2;
- if (!(val & HCR_RW) && system_supports_fpsimd()) {
+ if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
+ !vcpu->arch.guest_vfp_loaded) {
write_sysreg(1 << 30, fpexc32_el2);
isb();
}
@@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
write_sysreg(0, pmselr_el0);
write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
- __activate_traps_arch()();
+ __activate_traps_arch()(vcpu);
}
static void __hyp_text __deactivate_traps_vhe(void)
@@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
- bool fp_enabled;
u64 exit_code;
vcpu = kern_hyp_va(vcpu);
@@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
/* 0 falls through to be handled out of EL2 */
}
- fp_enabled = __fpsimd_enabled();
-
__sysreg_save_guest_state(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
@@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_restore_host_state(host_ctxt);
- if (fp_enabled) {
- __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
- __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
- }
-
__debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index b7438c8..c4a3714 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -19,6 +19,7 @@
#include <linux/kvm_host.h>
#include <asm/kvm_asm.h>
+#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
/* Yes, this does nothing, on purpose */
@@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
__sysreg_restore_common_state(ctxt);
}
+static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
+{
+ ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
+}
+
void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
{
u64 *spsr, *sysreg;
@@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
- if (__fpsimd_enabled())
- sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
-
if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
}
@@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
*/
void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
{
+ struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
+ struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
+
+ /* Restore host FP/SIMD state */
+ if (vcpu->arch.guest_vfp_loaded) {
+ if (vcpu_el1_is_32bit(vcpu))
+ kvm_call_hyp(__fpsimd32_save_state,
+ kern_hyp_va(guest_ctxt));
+ __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
+ __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
+ vcpu->arch.guest_vfp_loaded = 0;
+ }
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Avoid saving the guest VFP registers and restoring the host VFP
registers on every exit from the VM. Only when we're about to run
userspace or other threads in the kernel do we really have to switch the
state back to the host state.
We still initially configure the VFP registers to trap when entering the
VM, but the difference is that we now leave the guest state in the
hardware registers while running the VM.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_emulate.h | 5 ++++
arch/arm64/include/asm/kvm_host.h | 3 +++
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kvm/hyp/entry.S | 3 +++
arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
6 files changed, 44 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 1fbfe96..630dd60 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
return (unsigned long *)&vcpu->arch.hcr_el2;
}
+static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
+{
+ return (!(vcpu->arch.hcr_el2 & HCR_RW));
+}
+
static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
{
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 7d3bfa7..5e09eb9 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
/* Guest debug state */
u64 debug_flags;
+ /* 1 if the guest VFP state is loaded into the hardware */
+ u64 guest_vfp_loaded;
+
/*
* We maintain more than a single set of debug registers to support
* debugging the guest from the host and to maintain separate host and
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 612021d..9946732 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -133,6 +133,7 @@ int main(void)
DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
+ DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 76cd48f..b3e7191 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -185,6 +185,9 @@ alternative_endif
add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
bl __fpsimd_restore_state
+ mov x0, #1
+ str x0, [x3, #VCPU_GUEST_VFP_LOADED]
+
// Skip restoring fpexc32 for AArch64 guests
mrs x1, hcr_el2
tbnz x1, #HCR_RW_SHIFT, 1f
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 7703d63..ef05c59 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -23,43 +23,31 @@
#include <asm/kvm_hyp.h>
#include <asm/fpsimd.h>
-static bool __hyp_text __fpsimd_enabled_nvhe(void)
-{
- return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
-}
-
-static bool __hyp_text __fpsimd_enabled_vhe(void)
-{
- return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
-}
-
-static hyp_alternate_select(__fpsimd_is_enabled,
- __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
-bool __hyp_text __fpsimd_enabled(void)
-{
- return __fpsimd_is_enabled()();
-}
-
-static void __hyp_text __activate_traps_vhe(void)
+static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
{
u64 val;
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
- val &= ~CPACR_EL1_FPEN;
+ if (vcpu->arch.guest_vfp_loaded)
+ val |= CPACR_EL1_FPEN;
+ else
+ val &= ~CPACR_EL1_FPEN;
write_sysreg(val, cpacr_el1);
write_sysreg(__kvm_hyp_vector, vbar_el1);
}
-static void __hyp_text __activate_traps_nvhe(void)
+static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
{
u64 val;
val = CPTR_EL2_DEFAULT;
- val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
+ val |= CPTR_EL2_TTA;
+ if (vcpu->arch.guest_vfp_loaded)
+ val &= ~CPTR_EL2_TFP;
+ else
+ val |= CPTR_EL2_TFP;
write_sysreg(val, cptr_el2);
}
@@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
* it will cause an exception.
*/
val = vcpu->arch.hcr_el2;
- if (!(val & HCR_RW) && system_supports_fpsimd()) {
+ if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
+ !vcpu->arch.guest_vfp_loaded) {
write_sysreg(1 << 30, fpexc32_el2);
isb();
}
@@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
write_sysreg(0, pmselr_el0);
write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
- __activate_traps_arch()();
+ __activate_traps_arch()(vcpu);
}
static void __hyp_text __deactivate_traps_vhe(void)
@@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
- bool fp_enabled;
u64 exit_code;
vcpu = kern_hyp_va(vcpu);
@@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
/* 0 falls through to be handled out of EL2 */
}
- fp_enabled = __fpsimd_enabled();
-
__sysreg_save_guest_state(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
@@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_restore_host_state(host_ctxt);
- if (fp_enabled) {
- __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
- __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
- }
-
__debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index b7438c8..c4a3714 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -19,6 +19,7 @@
#include <linux/kvm_host.h>
#include <asm/kvm_asm.h>
+#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
/* Yes, this does nothing, on purpose */
@@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
__sysreg_restore_common_state(ctxt);
}
+static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
+{
+ ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
+}
+
void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
{
u64 *spsr, *sysreg;
@@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
- if (__fpsimd_enabled())
- sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
-
if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
}
@@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
*/
void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
{
+ struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
+ struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
+
+ /* Restore host FP/SIMD state */
+ if (vcpu->arch.guest_vfp_loaded) {
+ if (vcpu_el1_is_32bit(vcpu))
+ kvm_call_hyp(__fpsimd32_save_state,
+ kern_hyp_va(guest_ctxt));
+ __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
+ __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
+ vcpu->arch.guest_vfp_loaded = 0;
+ }
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 13:15 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 13:15 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, Marc Zyngier, Shih-Wei Li, kvm
On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> Avoid saving the guest VFP registers and restoring the host VFP
> registers on every exit from the VM. Only when we're about to run
> userspace or other threads in the kernel do we really have to switch the
> state back to the host state.
>
> We still initially configure the VFP registers to trap when entering the
> VM, but the difference is that we now leave the guest state in the
> hardware registers while running the VM.
running the host.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 3 +++
> arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> 6 files changed, 44 insertions(+), 36 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 1fbfe96..630dd60 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> +{
> + return (!(vcpu->arch.hcr_el2 & HCR_RW));
nit: no need for the outer ().
> +}
> +
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> {
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 7d3bfa7..5e09eb9 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> /* Guest debug state */
> u64 debug_flags;
>
> + /* 1 if the guest VFP state is loaded into the hardware */
> + u64 guest_vfp_loaded;
> +
Is there a chance we'll want other flags like this? Should we just make
this a lazy state flags field with the (currently only) flag VFP? If not,
then a bool would be nicer, although I see below the u64 was chosen in
order for the 'str' to be used.
> /*
> * We maintain more than a single set of debug registers to support
> * debugging the guest from the host and to maintain separate host and
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 612021d..9946732 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -133,6 +133,7 @@ int main(void)
> DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 76cd48f..b3e7191 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -185,6 +185,9 @@ alternative_endif
> add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> bl __fpsimd_restore_state
>
> + mov x0, #1
> + str x0, [x3, #VCPU_GUEST_VFP_LOADED]
> +
> // Skip restoring fpexc32 for AArch64 guests
> mrs x1, hcr_el2
> tbnz x1, #HCR_RW_SHIFT, 1f
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7703d63..ef05c59 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -23,43 +23,31 @@
> #include <asm/kvm_hyp.h>
> #include <asm/fpsimd.h>
>
> -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> -{
> - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> -}
> -
> -static bool __hyp_text __fpsimd_enabled_vhe(void)
> -{
> - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> -}
> -
> -static hyp_alternate_select(__fpsimd_is_enabled,
> - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> -bool __hyp_text __fpsimd_enabled(void)
> -{
> - return __fpsimd_is_enabled()();
> -}
> -
> -static void __hyp_text __activate_traps_vhe(void)
> +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + if (vcpu->arch.guest_vfp_loaded)
> + val |= CPACR_EL1_FPEN;
> + else
> + val &= ~CPACR_EL1_FPEN;
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> }
>
> -static void __hyp_text __activate_traps_nvhe(void)
> +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA;
> + if (vcpu->arch.guest_vfp_loaded)
> + val &= ~CPTR_EL2_TFP;
> + else
> + val |= CPTR_EL2_TFP;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> + !vcpu->arch.guest_vfp_loaded) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> @@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> write_sysreg(0, pmselr_el0);
> write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> - __activate_traps_arch()();
> + __activate_traps_arch()(vcpu);
> }
>
> static void __hyp_text __deactivate_traps_vhe(void)
> @@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> struct kvm_cpu_context *guest_ctxt;
> - bool fp_enabled;
> u64 exit_code;
>
> vcpu = kern_hyp_va(vcpu);
> @@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> /* 0 falls through to be handled out of EL2 */
> }
>
> - fp_enabled = __fpsimd_enabled();
> -
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> @@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_host_state(host_ctxt);
>
> - if (fp_enabled) {
> - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> - }
> -
> __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index b7438c8..c4a3714 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -19,6 +19,7 @@
> #include <linux/kvm_host.h>
>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> /* Yes, this does nothing, on purpose */
> @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> __sysreg_restore_common_state(ctxt);
> }
>
> +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> +{
> + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> +}
> +
> void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> {
> u64 *spsr, *sysreg;
> @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
>
> - if (__fpsimd_enabled())
> - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> -
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> }
> @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> */
> void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> +
> + /* Restore host FP/SIMD state */
> + if (vcpu->arch.guest_vfp_loaded) {
> + if (vcpu_el1_is_32bit(vcpu))
> + kvm_call_hyp(__fpsimd32_save_state,
> + kern_hyp_va(guest_ctxt));
nit: might be nice to use {} since we need two lines.
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + vcpu->arch.guest_vfp_loaded = 0;
> + }
> }
> --
> 2.9.0
>
Otherwise,
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-07 13:15 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 13:15 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> Avoid saving the guest VFP registers and restoring the host VFP
> registers on every exit from the VM. Only when we're about to run
> userspace or other threads in the kernel do we really have to switch the
> state back to the host state.
>
> We still initially configure the VFP registers to trap when entering the
> VM, but the difference is that we now leave the guest state in the
> hardware registers while running the VM.
running the host.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 3 +++
> arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> 6 files changed, 44 insertions(+), 36 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 1fbfe96..630dd60 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> +{
> + return (!(vcpu->arch.hcr_el2 & HCR_RW));
nit: no need for the outer ().
> +}
> +
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> {
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 7d3bfa7..5e09eb9 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> /* Guest debug state */
> u64 debug_flags;
>
> + /* 1 if the guest VFP state is loaded into the hardware */
> + u64 guest_vfp_loaded;
> +
Is there a chance we'll want other flags like this? Should we just make
this a lazy state flags field with the (currently only) flag VFP? If not,
then a bool would be nicer, although I see below the u64 was chosen in
order for the 'str' to be used.
> /*
> * We maintain more than a single set of debug registers to support
> * debugging the guest from the host and to maintain separate host and
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 612021d..9946732 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -133,6 +133,7 @@ int main(void)
> DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 76cd48f..b3e7191 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -185,6 +185,9 @@ alternative_endif
> add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> bl __fpsimd_restore_state
>
> + mov x0, #1
> + str x0, [x3, #VCPU_GUEST_VFP_LOADED]
> +
> // Skip restoring fpexc32 for AArch64 guests
> mrs x1, hcr_el2
> tbnz x1, #HCR_RW_SHIFT, 1f
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7703d63..ef05c59 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -23,43 +23,31 @@
> #include <asm/kvm_hyp.h>
> #include <asm/fpsimd.h>
>
> -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> -{
> - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> -}
> -
> -static bool __hyp_text __fpsimd_enabled_vhe(void)
> -{
> - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> -}
> -
> -static hyp_alternate_select(__fpsimd_is_enabled,
> - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> -bool __hyp_text __fpsimd_enabled(void)
> -{
> - return __fpsimd_is_enabled()();
> -}
> -
> -static void __hyp_text __activate_traps_vhe(void)
> +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + if (vcpu->arch.guest_vfp_loaded)
> + val |= CPACR_EL1_FPEN;
> + else
> + val &= ~CPACR_EL1_FPEN;
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> }
>
> -static void __hyp_text __activate_traps_nvhe(void)
> +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA;
> + if (vcpu->arch.guest_vfp_loaded)
> + val &= ~CPTR_EL2_TFP;
> + else
> + val |= CPTR_EL2_TFP;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> + !vcpu->arch.guest_vfp_loaded) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> @@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> write_sysreg(0, pmselr_el0);
> write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> - __activate_traps_arch()();
> + __activate_traps_arch()(vcpu);
> }
>
> static void __hyp_text __deactivate_traps_vhe(void)
> @@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> struct kvm_cpu_context *guest_ctxt;
> - bool fp_enabled;
> u64 exit_code;
>
> vcpu = kern_hyp_va(vcpu);
> @@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> /* 0 falls through to be handled out of EL2 */
> }
>
> - fp_enabled = __fpsimd_enabled();
> -
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> @@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_host_state(host_ctxt);
>
> - if (fp_enabled) {
> - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> - }
> -
> __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index b7438c8..c4a3714 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -19,6 +19,7 @@
> #include <linux/kvm_host.h>
>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> /* Yes, this does nothing, on purpose */
> @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> __sysreg_restore_common_state(ctxt);
> }
>
> +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> +{
> + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> +}
> +
> void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> {
> u64 *spsr, *sysreg;
> @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
>
> - if (__fpsimd_enabled())
> - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> -
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> }
> @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> */
> void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> +
> + /* Restore host FP/SIMD state */
> + if (vcpu->arch.guest_vfp_loaded) {
> + if (vcpu_el1_is_32bit(vcpu))
> + kvm_call_hyp(__fpsimd32_save_state,
> + kern_hyp_va(guest_ctxt));
nit: might be nice to use {} since we need two lines.
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + vcpu->arch.guest_vfp_loaded = 0;
> + }
> }
> --
> 2.9.0
>
Otherwise,
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-07 13:15 ` Andrew Jones
@ 2017-11-26 16:24 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:24 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Tue, Nov 07, 2017 at 02:15:50PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
> >
> > We still initially configure the VFP registers to trap when entering the
> > VM, but the difference is that we now leave the guest state in the
> > hardware registers while running the VM.
>
> running the host.
>
I actually did mean the VM, but I should clarify to mean as long as
we're running the VCPU on this physical CPU, even if we trap to the
host.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 3 +++
> > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > 6 files changed, 44 insertions(+), 36 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 1fbfe96..630dd60 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> >
> > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > +{
> > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
>
> nit: no need for the outer ().
>
> > +}
> > +
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 7d3bfa7..5e09eb9 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > /* Guest debug state */
> > u64 debug_flags;
> >
> > + /* 1 if the guest VFP state is loaded into the hardware */
> > + u64 guest_vfp_loaded;
> > +
>
> Is there a chance we'll want other flags like this? Should we just make
> this a lazy state flags field with the (currently only) flag VFP? If not,
> then a bool would be nicer, although I see below the u64 was chosen in
> order for the 'str' to be used.
>
See my reply to Yury. In terms of merging flags I thought about merging
it with the debug flags, but I didn't think it would look very nice, and
I couldn't come up with a name for the variable that would describe the
logic.
Honestly, I didn't care about the few extra bytes per CPU, and much prefer
clarity, but it may make sense to combine this with for example the
sysreg and timer state later, I'll have a look.
> > /*
> > * We maintain more than a single set of debug registers to support
> > * debugging the guest from the host and to maintain separate host and
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 612021d..9946732 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -133,6 +133,7 @@ int main(void)
> > DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> > DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index 76cd48f..b3e7191 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -185,6 +185,9 @@ alternative_endif
> > add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> > bl __fpsimd_restore_state
> >
> > + mov x0, #1
> > + str x0, [x3, #VCPU_GUEST_VFP_LOADED]
> > +
> > // Skip restoring fpexc32 for AArch64 guests
> > mrs x1, hcr_el2
> > tbnz x1, #HCR_RW_SHIFT, 1f
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 7703d63..ef05c59 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -23,43 +23,31 @@
> > #include <asm/kvm_hyp.h>
> > #include <asm/fpsimd.h>
> >
> > -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> > -{
> > - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> > -}
> > -
> > -static bool __hyp_text __fpsimd_enabled_vhe(void)
> > -{
> > - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> > -}
> > -
> > -static hyp_alternate_select(__fpsimd_is_enabled,
> > - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > -bool __hyp_text __fpsimd_enabled(void)
> > -{
> > - return __fpsimd_is_enabled()();
> > -}
> > -
> > -static void __hyp_text __activate_traps_vhe(void)
> > +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> > {
> > u64 val;
> >
> > val = read_sysreg(cpacr_el1);
> > val |= CPACR_EL1_TTA;
> > - val &= ~CPACR_EL1_FPEN;
> > + if (vcpu->arch.guest_vfp_loaded)
> > + val |= CPACR_EL1_FPEN;
> > + else
> > + val &= ~CPACR_EL1_FPEN;
> > write_sysreg(val, cpacr_el1);
> >
> > write_sysreg(__kvm_hyp_vector, vbar_el1);
> > }
> >
> > -static void __hyp_text __activate_traps_nvhe(void)
> > +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> > {
> > u64 val;
> >
> > val = CPTR_EL2_DEFAULT;
> > - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> > + val |= CPTR_EL2_TTA;
> > + if (vcpu->arch.guest_vfp_loaded)
> > + val &= ~CPTR_EL2_TFP;
> > + else
> > + val |= CPTR_EL2_TFP;
> > write_sysreg(val, cptr_el2);
> > }
> >
> > @@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > * it will cause an exception.
> > */
> > val = vcpu->arch.hcr_el2;
> > - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> > + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> > + !vcpu->arch.guest_vfp_loaded) {
> > write_sysreg(1 << 30, fpexc32_el2);
> > isb();
> > }
> > @@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > write_sysreg(0, pmselr_el0);
> > write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> > write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> > - __activate_traps_arch()();
> > + __activate_traps_arch()(vcpu);
> > }
> >
> > static void __hyp_text __deactivate_traps_vhe(void)
> > @@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > {
> > struct kvm_cpu_context *host_ctxt;
> > struct kvm_cpu_context *guest_ctxt;
> > - bool fp_enabled;
> > u64 exit_code;
> >
> > vcpu = kern_hyp_va(vcpu);
> > @@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > /* 0 falls through to be handled out of EL2 */
> > }
> >
> > - fp_enabled = __fpsimd_enabled();
> > -
> > __sysreg_save_guest_state(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > __timer_disable_traps(vcpu);
> > @@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > - if (fp_enabled) {
> > - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > - }
> > -
> > __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index b7438c8..c4a3714 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -19,6 +19,7 @@
> > #include <linux/kvm_host.h>
> >
> > #include <asm/kvm_asm.h>
> > +#include <asm/kvm_emulate.h>
> > #include <asm/kvm_hyp.h>
> >
> > /* Yes, this does nothing, on purpose */
> > @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_common_state(ctxt);
> > }
> >
> > +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > +{
> > + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> > +}
> > +
> > void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> > {
> > u64 *spsr, *sysreg;
> > @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> > sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> > sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> >
> > - if (__fpsimd_enabled())
> > - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> > -
> > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> > }
> > @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > */
> > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + /* Restore host FP/SIMD state */
> > + if (vcpu->arch.guest_vfp_loaded) {
> > + if (vcpu_el1_is_32bit(vcpu))
> > + kvm_call_hyp(__fpsimd32_save_state,
> > + kern_hyp_va(guest_ctxt));
>
> nit: might be nice to use {} since we need two lines.
>
sure.
> > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > + vcpu->arch.guest_vfp_loaded = 0;
> > + }
> > }
> > --
> > 2.9.0
> >
>
> Otherwise,
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-26 16:24 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:24 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 02:15:50PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
> >
> > We still initially configure the VFP registers to trap when entering the
> > VM, but the difference is that we now leave the guest state in the
> > hardware registers while running the VM.
>
> running the host.
>
I actually did mean the VM, but I should clarify to mean as long as
we're running the VCPU on this physical CPU, even if we trap to the
host.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 3 +++
> > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > 6 files changed, 44 insertions(+), 36 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 1fbfe96..630dd60 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> >
> > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > +{
> > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
>
> nit: no need for the outer ().
>
> > +}
> > +
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 7d3bfa7..5e09eb9 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > /* Guest debug state */
> > u64 debug_flags;
> >
> > + /* 1 if the guest VFP state is loaded into the hardware */
> > + u64 guest_vfp_loaded;
> > +
>
> Is there a chance we'll want other flags like this? Should we just make
> this a lazy state flags field with the (currently only) flag VFP? If not,
> then a bool would be nicer, although I see below the u64 was chosen in
> order for the 'str' to be used.
>
See my reply to Yury. In terms of merging flags I thought about merging
it with the debug flags, but I didn't think it would look very nice, and
I couldn't come up with a name for the variable that would describe the
logic.
Honestly, I didn't care about the few extra bytes per CPU, and much prefer
clarity, but it may make sense to combine this with for example the
sysreg and timer state later, I'll have a look.
> > /*
> > * We maintain more than a single set of debug registers to support
> > * debugging the guest from the host and to maintain separate host and
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 612021d..9946732 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -133,6 +133,7 @@ int main(void)
> > DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> > DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index 76cd48f..b3e7191 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -185,6 +185,9 @@ alternative_endif
> > add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> > bl __fpsimd_restore_state
> >
> > + mov x0, #1
> > + str x0, [x3, #VCPU_GUEST_VFP_LOADED]
> > +
> > // Skip restoring fpexc32 for AArch64 guests
> > mrs x1, hcr_el2
> > tbnz x1, #HCR_RW_SHIFT, 1f
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 7703d63..ef05c59 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -23,43 +23,31 @@
> > #include <asm/kvm_hyp.h>
> > #include <asm/fpsimd.h>
> >
> > -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> > -{
> > - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> > -}
> > -
> > -static bool __hyp_text __fpsimd_enabled_vhe(void)
> > -{
> > - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> > -}
> > -
> > -static hyp_alternate_select(__fpsimd_is_enabled,
> > - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > -bool __hyp_text __fpsimd_enabled(void)
> > -{
> > - return __fpsimd_is_enabled()();
> > -}
> > -
> > -static void __hyp_text __activate_traps_vhe(void)
> > +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> > {
> > u64 val;
> >
> > val = read_sysreg(cpacr_el1);
> > val |= CPACR_EL1_TTA;
> > - val &= ~CPACR_EL1_FPEN;
> > + if (vcpu->arch.guest_vfp_loaded)
> > + val |= CPACR_EL1_FPEN;
> > + else
> > + val &= ~CPACR_EL1_FPEN;
> > write_sysreg(val, cpacr_el1);
> >
> > write_sysreg(__kvm_hyp_vector, vbar_el1);
> > }
> >
> > -static void __hyp_text __activate_traps_nvhe(void)
> > +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> > {
> > u64 val;
> >
> > val = CPTR_EL2_DEFAULT;
> > - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> > + val |= CPTR_EL2_TTA;
> > + if (vcpu->arch.guest_vfp_loaded)
> > + val &= ~CPTR_EL2_TFP;
> > + else
> > + val |= CPTR_EL2_TFP;
> > write_sysreg(val, cptr_el2);
> > }
> >
> > @@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > * it will cause an exception.
> > */
> > val = vcpu->arch.hcr_el2;
> > - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> > + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> > + !vcpu->arch.guest_vfp_loaded) {
> > write_sysreg(1 << 30, fpexc32_el2);
> > isb();
> > }
> > @@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > write_sysreg(0, pmselr_el0);
> > write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> > write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> > - __activate_traps_arch()();
> > + __activate_traps_arch()(vcpu);
> > }
> >
> > static void __hyp_text __deactivate_traps_vhe(void)
> > @@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > {
> > struct kvm_cpu_context *host_ctxt;
> > struct kvm_cpu_context *guest_ctxt;
> > - bool fp_enabled;
> > u64 exit_code;
> >
> > vcpu = kern_hyp_va(vcpu);
> > @@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > /* 0 falls through to be handled out of EL2 */
> > }
> >
> > - fp_enabled = __fpsimd_enabled();
> > -
> > __sysreg_save_guest_state(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > __timer_disable_traps(vcpu);
> > @@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > - if (fp_enabled) {
> > - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > - }
> > -
> > __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index b7438c8..c4a3714 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -19,6 +19,7 @@
> > #include <linux/kvm_host.h>
> >
> > #include <asm/kvm_asm.h>
> > +#include <asm/kvm_emulate.h>
> > #include <asm/kvm_hyp.h>
> >
> > /* Yes, this does nothing, on purpose */
> > @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_common_state(ctxt);
> > }
> >
> > +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > +{
> > + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> > +}
> > +
> > void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> > {
> > u64 *spsr, *sysreg;
> > @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> > sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> > sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> >
> > - if (__fpsimd_enabled())
> > - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> > -
> > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> > }
> > @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > */
> > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + /* Restore host FP/SIMD state */
> > + if (vcpu->arch.guest_vfp_loaded) {
> > + if (vcpu_el1_is_32bit(vcpu))
> > + kvm_call_hyp(__fpsimd32_save_state,
> > + kern_hyp_va(guest_ctxt));
>
> nit: might be nice to use {} since we need two lines.
>
sure.
> > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > + vcpu->arch.guest_vfp_loaded = 0;
> > + }
> > }
> > --
> > 2.9.0
> >
>
> Otherwise,
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-15 16:04 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-15 16:04 UTC (permalink / raw)
To: Christoffer Dall
Cc: kvmarm, linux-arm-kernel, Marc Zyngier, Shih-Wei Li, kvm, riel
On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> Avoid saving the guest VFP registers and restoring the host VFP
> registers on every exit from the VM. Only when we're about to run
> userspace or other threads in the kernel do we really have to switch the
> state back to the host state.
Rik van Riel's recently post patch "[PATCH v2 0/2] x86,kvm: move qemu/guest
FPU switching out to kvm_arch_vcpu_ioctl_run" indicates that for x86 they
only need to swap guest and userspace VFP registers before exiting VCPU_RUN
to userspace, not for running other threads. I imagine that's the same for
ARM as well.
If so, then I think this hunk
> @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> */
> void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> +
> + /* Restore host FP/SIMD state */
> + if (vcpu->arch.guest_vfp_loaded) {
> + if (vcpu_el1_is_32bit(vcpu))
> + kvm_call_hyp(__fpsimd32_save_state,
> + kern_hyp_va(guest_ctxt));
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + vcpu->arch.guest_vfp_loaded = 0;
> + }
> }
could be moved to the return of kvm_arch_vcpu_ioctl_run().
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-15 16:04 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-15 16:04 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> Avoid saving the guest VFP registers and restoring the host VFP
> registers on every exit from the VM. Only when we're about to run
> userspace or other threads in the kernel do we really have to switch the
> state back to the host state.
Rik van Riel's recently post patch "[PATCH v2 0/2] x86,kvm: move qemu/guest
FPU switching out to kvm_arch_vcpu_ioctl_run" indicates that for x86 they
only need to swap guest and userspace VFP registers before exiting VCPU_RUN
to userspace, not for running other threads. I imagine that's the same for
ARM as well.
If so, then I think this hunk
> @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> */
> void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> +
> + /* Restore host FP/SIMD state */
> + if (vcpu->arch.guest_vfp_loaded) {
> + if (vcpu_el1_is_32bit(vcpu))
> + kvm_call_hyp(__fpsimd32_save_state,
> + kern_hyp_va(guest_ctxt));
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + vcpu->arch.guest_vfp_loaded = 0;
> + }
> }
could be moved to the return of kvm_arch_vcpu_ioctl_run().
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-15 16:04 ` Andrew Jones
@ 2017-11-26 16:17 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:17 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm, riel
Hi Drew,
On Wed, Nov 15, 2017 at 05:04:40PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
>
> Rik van Riel's recently post patch "[PATCH v2 0/2] x86,kvm: move qemu/guest
> FPU switching out to kvm_arch_vcpu_ioctl_run" indicates that for x86 they
> only need to swap guest and userspace VFP registers before exiting VCPU_RUN
> to userspace, not for running other threads. I imagine that's the same for
> ARM as well.
>
> If so, then I think this hunk
>
> > @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > */
> > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + /* Restore host FP/SIMD state */
> > + if (vcpu->arch.guest_vfp_loaded) {
> > + if (vcpu_el1_is_32bit(vcpu))
> > + kvm_call_hyp(__fpsimd32_save_state,
> > + kern_hyp_va(guest_ctxt));
> > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > + vcpu->arch.guest_vfp_loaded = 0;
> > + }
> > }
>
> could be moved to the return of kvm_arch_vcpu_ioctl_run().
>
That sounds cool.
I'll keep this patch as it is now, and look at Rik's patches and post a
follow up later, does that sound ok?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-26 16:17 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:17 UTC (permalink / raw)
To: linux-arm-kernel
Hi Drew,
On Wed, Nov 15, 2017 at 05:04:40PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
>
> Rik van Riel's recently post patch "[PATCH v2 0/2] x86,kvm: move qemu/guest
> FPU switching out to kvm_arch_vcpu_ioctl_run" indicates that for x86 they
> only need to swap guest and userspace VFP registers before exiting VCPU_RUN
> to userspace, not for running other threads. I imagine that's the same for
> ARM as well.
>
> If so, then I think this hunk
>
> > @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > */
> > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + /* Restore host FP/SIMD state */
> > + if (vcpu->arch.guest_vfp_loaded) {
> > + if (vcpu_el1_is_32bit(vcpu))
> > + kvm_call_hyp(__fpsimd32_save_state,
> > + kern_hyp_va(guest_ctxt));
> > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > + vcpu->arch.guest_vfp_loaded = 0;
> > + }
> > }
>
> could be moved to the return of kvm_arch_vcpu_ioctl_run().
>
That sounds cool.
I'll keep this patch as it is now, and look at Rik's patches and post a
follow up later, does that sound ok?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-26 16:17 ` Christoffer Dall
@ 2017-11-27 8:32 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-27 8:32 UTC (permalink / raw)
To: Christoffer Dall
Cc: riel, kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Sun, Nov 26, 2017 at 05:17:56PM +0100, Christoffer Dall wrote:
> Hi Drew,
>
> On Wed, Nov 15, 2017 at 05:04:40PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > Avoid saving the guest VFP registers and restoring the host VFP
> > > registers on every exit from the VM. Only when we're about to run
> > > userspace or other threads in the kernel do we really have to switch the
> > > state back to the host state.
> >
> > Rik van Riel's recently post patch "[PATCH v2 0/2] x86,kvm: move qemu/guest
> > FPU switching out to kvm_arch_vcpu_ioctl_run" indicates that for x86 they
> > only need to swap guest and userspace VFP registers before exiting VCPU_RUN
> > to userspace, not for running other threads. I imagine that's the same for
> > ARM as well.
> >
> > If so, then I think this hunk
> >
> > > @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > > */
> > > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > > {
> > > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > > +
> > > + /* Restore host FP/SIMD state */
> > > + if (vcpu->arch.guest_vfp_loaded) {
> > > + if (vcpu_el1_is_32bit(vcpu))
> > > + kvm_call_hyp(__fpsimd32_save_state,
> > > + kern_hyp_va(guest_ctxt));
> > > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > > + vcpu->arch.guest_vfp_loaded = 0;
> > > + }
> > > }
> >
> > could be moved to the return of kvm_arch_vcpu_ioctl_run().
> >
> That sounds cool.
>
> I'll keep this patch as it is now, and look at Rik's patches and post a
> follow up later, does that sound ok?
Works for me. I'm guessing there's also going to be some overlap with
Dave Martin's SVE work. So whichever series goes second can probably
be the one to put the most consideration into it.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-27 8:32 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-27 8:32 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 05:17:56PM +0100, Christoffer Dall wrote:
> Hi Drew,
>
> On Wed, Nov 15, 2017 at 05:04:40PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > Avoid saving the guest VFP registers and restoring the host VFP
> > > registers on every exit from the VM. Only when we're about to run
> > > userspace or other threads in the kernel do we really have to switch the
> > > state back to the host state.
> >
> > Rik van Riel's recently post patch "[PATCH v2 0/2] x86,kvm: move qemu/guest
> > FPU switching out to kvm_arch_vcpu_ioctl_run" indicates that for x86 they
> > only need to swap guest and userspace VFP registers before exiting VCPU_RUN
> > to userspace, not for running other threads. I imagine that's the same for
> > ARM as well.
> >
> > If so, then I think this hunk
> >
> > > @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > > */
> > > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > > {
> > > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > > +
> > > + /* Restore host FP/SIMD state */
> > > + if (vcpu->arch.guest_vfp_loaded) {
> > > + if (vcpu_el1_is_32bit(vcpu))
> > > + kvm_call_hyp(__fpsimd32_save_state,
> > > + kern_hyp_va(guest_ctxt));
> > > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > > + vcpu->arch.guest_vfp_loaded = 0;
> > > + }
> > > }
> >
> > could be moved to the return of kvm_arch_vcpu_ioctl_run().
> >
> That sounds cool.
>
> I'll keep this patch as it is now, and look at Rik's patches and post a
> follow up later, does that sound ok?
Works for me. I'm guessing there's also going to be some overlap with
Dave Martin's SVE work. So whichever series goes second can probably
be the one to put the most consideration into it.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-25 7:52 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-25 7:52 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
Hi Christoffer,
On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> Avoid saving the guest VFP registers and restoring the host VFP
> registers on every exit from the VM. Only when we're about to run
> userspace or other threads in the kernel do we really have to switch the
> state back to the host state.
>
> We still initially configure the VFP registers to trap when entering the
> VM, but the difference is that we now leave the guest state in the
> hardware registers while running the VM.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 3 +++
> arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> 6 files changed, 44 insertions(+), 36 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 1fbfe96..630dd60 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> +{
> + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> +}
> +
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> {
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 7d3bfa7..5e09eb9 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> /* Guest debug state */
> u64 debug_flags;
>
> + /* 1 if the guest VFP state is loaded into the hardware */
> + u64 guest_vfp_loaded;
May it be just u8/bool?
Yury
> +
> /*
> * We maintain more than a single set of debug registers to support
> * debugging the guest from the host and to maintain separate host and
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 612021d..9946732 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -133,6 +133,7 @@ int main(void)
> DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 76cd48f..b3e7191 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -185,6 +185,9 @@ alternative_endif
> add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> bl __fpsimd_restore_state
>
> + mov x0, #1
> + str x0, [x3, #VCPU_GUEST_VFP_LOADED]
> +
> // Skip restoring fpexc32 for AArch64 guests
> mrs x1, hcr_el2
> tbnz x1, #HCR_RW_SHIFT, 1f
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7703d63..ef05c59 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -23,43 +23,31 @@
> #include <asm/kvm_hyp.h>
> #include <asm/fpsimd.h>
>
> -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> -{
> - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> -}
> -
> -static bool __hyp_text __fpsimd_enabled_vhe(void)
> -{
> - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> -}
> -
> -static hyp_alternate_select(__fpsimd_is_enabled,
> - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> -bool __hyp_text __fpsimd_enabled(void)
> -{
> - return __fpsimd_is_enabled()();
> -}
> -
> -static void __hyp_text __activate_traps_vhe(void)
> +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + if (vcpu->arch.guest_vfp_loaded)
> + val |= CPACR_EL1_FPEN;
> + else
> + val &= ~CPACR_EL1_FPEN;
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> }
>
> -static void __hyp_text __activate_traps_nvhe(void)
> +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA;
> + if (vcpu->arch.guest_vfp_loaded)
> + val &= ~CPTR_EL2_TFP;
> + else
> + val |= CPTR_EL2_TFP;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> + !vcpu->arch.guest_vfp_loaded) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> @@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> write_sysreg(0, pmselr_el0);
> write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> - __activate_traps_arch()();
> + __activate_traps_arch()(vcpu);
> }
>
> static void __hyp_text __deactivate_traps_vhe(void)
> @@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> struct kvm_cpu_context *guest_ctxt;
> - bool fp_enabled;
> u64 exit_code;
>
> vcpu = kern_hyp_va(vcpu);
> @@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> /* 0 falls through to be handled out of EL2 */
> }
>
> - fp_enabled = __fpsimd_enabled();
> -
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> @@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_host_state(host_ctxt);
>
> - if (fp_enabled) {
> - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> - }
> -
> __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index b7438c8..c4a3714 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -19,6 +19,7 @@
> #include <linux/kvm_host.h>
>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> /* Yes, this does nothing, on purpose */
> @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> __sysreg_restore_common_state(ctxt);
> }
>
> +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> +{
> + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> +}
> +
> void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> {
> u64 *spsr, *sysreg;
> @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
>
> - if (__fpsimd_enabled())
> - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> -
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> }
> @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> */
> void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> +
> + /* Restore host FP/SIMD state */
> + if (vcpu->arch.guest_vfp_loaded) {
> + if (vcpu_el1_is_32bit(vcpu))
> + kvm_call_hyp(__fpsimd32_save_state,
> + kern_hyp_va(guest_ctxt));
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + vcpu->arch.guest_vfp_loaded = 0;
> + }
> }
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-25 7:52 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-25 7:52 UTC (permalink / raw)
To: linux-arm-kernel
Hi Christoffer,
On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> Avoid saving the guest VFP registers and restoring the host VFP
> registers on every exit from the VM. Only when we're about to run
> userspace or other threads in the kernel do we really have to switch the
> state back to the host state.
>
> We still initially configure the VFP registers to trap when entering the
> VM, but the difference is that we now leave the guest state in the
> hardware registers while running the VM.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> arch/arm64/include/asm/kvm_host.h | 3 +++
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 3 +++
> arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> 6 files changed, 44 insertions(+), 36 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 1fbfe96..630dd60 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> return (unsigned long *)&vcpu->arch.hcr_el2;
> }
>
> +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> +{
> + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> +}
> +
> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> {
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 7d3bfa7..5e09eb9 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> /* Guest debug state */
> u64 debug_flags;
>
> + /* 1 if the guest VFP state is loaded into the hardware */
> + u64 guest_vfp_loaded;
May it be just u8/bool?
Yury
> +
> /*
> * We maintain more than a single set of debug registers to support
> * debugging the guest from the host and to maintain separate host and
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 612021d..9946732 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -133,6 +133,7 @@ int main(void)
> DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 76cd48f..b3e7191 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -185,6 +185,9 @@ alternative_endif
> add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> bl __fpsimd_restore_state
>
> + mov x0, #1
> + str x0, [x3, #VCPU_GUEST_VFP_LOADED]
> +
> // Skip restoring fpexc32 for AArch64 guests
> mrs x1, hcr_el2
> tbnz x1, #HCR_RW_SHIFT, 1f
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7703d63..ef05c59 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -23,43 +23,31 @@
> #include <asm/kvm_hyp.h>
> #include <asm/fpsimd.h>
>
> -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> -{
> - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> -}
> -
> -static bool __hyp_text __fpsimd_enabled_vhe(void)
> -{
> - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> -}
> -
> -static hyp_alternate_select(__fpsimd_is_enabled,
> - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> -bool __hyp_text __fpsimd_enabled(void)
> -{
> - return __fpsimd_is_enabled()();
> -}
> -
> -static void __hyp_text __activate_traps_vhe(void)
> +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + if (vcpu->arch.guest_vfp_loaded)
> + val |= CPACR_EL1_FPEN;
> + else
> + val &= ~CPACR_EL1_FPEN;
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> }
>
> -static void __hyp_text __activate_traps_nvhe(void)
> +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA;
> + if (vcpu->arch.guest_vfp_loaded)
> + val &= ~CPTR_EL2_TFP;
> + else
> + val |= CPTR_EL2_TFP;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -81,7 +69,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> + !vcpu->arch.guest_vfp_loaded) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> @@ -97,7 +86,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> write_sysreg(0, pmselr_el0);
> write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> - __activate_traps_arch()();
> + __activate_traps_arch()(vcpu);
> }
>
> static void __hyp_text __deactivate_traps_vhe(void)
> @@ -273,7 +262,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> struct kvm_cpu_context *guest_ctxt;
> - bool fp_enabled;
> u64 exit_code;
>
> vcpu = kern_hyp_va(vcpu);
> @@ -355,8 +343,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> /* 0 falls through to be handled out of EL2 */
> }
>
> - fp_enabled = __fpsimd_enabled();
> -
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> @@ -367,11 +353,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_host_state(host_ctxt);
>
> - if (fp_enabled) {
> - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> - }
> -
> __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index b7438c8..c4a3714 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -19,6 +19,7 @@
> #include <linux/kvm_host.h>
>
> #include <asm/kvm_asm.h>
> +#include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> /* Yes, this does nothing, on purpose */
> @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> __sysreg_restore_common_state(ctxt);
> }
>
> +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> +{
> + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> +}
> +
> void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> {
> u64 *spsr, *sysreg;
> @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
>
> - if (__fpsimd_enabled())
> - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> -
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> }
> @@ -209,4 +212,16 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> */
> void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> +
> + /* Restore host FP/SIMD state */
> + if (vcpu->arch.guest_vfp_loaded) {
> + if (vcpu_el1_is_32bit(vcpu))
> + kvm_call_hyp(__fpsimd32_save_state,
> + kern_hyp_va(guest_ctxt));
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + vcpu->arch.guest_vfp_loaded = 0;
> + }
> }
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-25 7:52 ` Yury Norov
@ 2017-11-26 16:17 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:17 UTC (permalink / raw)
To: Yury Norov
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
Hi Yury,
On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
>
> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
> >
> > We still initially configure the VFP registers to trap when entering the
> > VM, but the difference is that we now leave the guest state in the
> > hardware registers while running the VM.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 3 +++
> > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > 6 files changed, 44 insertions(+), 36 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 1fbfe96..630dd60 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> >
> > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > +{
> > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > +}
> > +
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 7d3bfa7..5e09eb9 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > /* Guest debug state */
> > u64 debug_flags;
> >
> > + /* 1 if the guest VFP state is loaded into the hardware */
> > + u64 guest_vfp_loaded;
>
> May it be just u8/bool?
>
This particular field is accessed from assembly code, and I'm not sure
what guarantees the compiler makes in terms of how a u8/bool is
allocated with respect to padding and alignment, and I think that's why
we've been using u64 fields in the past.
I don't actually remember the details, but I'd rather err on the side of
caution than trying to save a few bytes. However, if someone can
convince me there's a completely safe way to do this, then I'm happy to
change it.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-26 16:17 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 16:17 UTC (permalink / raw)
To: linux-arm-kernel
Hi Yury,
On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
>
> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
> >
> > We still initially configure the VFP registers to trap when entering the
> > VM, but the difference is that we now leave the guest state in the
> > hardware registers while running the VM.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 3 +++
> > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > 6 files changed, 44 insertions(+), 36 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 1fbfe96..630dd60 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> >
> > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > +{
> > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > +}
> > +
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 7d3bfa7..5e09eb9 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > /* Guest debug state */
> > u64 debug_flags;
> >
> > + /* 1 if the guest VFP state is loaded into the hardware */
> > + u64 guest_vfp_loaded;
>
> May it be just u8/bool?
>
This particular field is accessed from assembly code, and I'm not sure
what guarantees the compiler makes in terms of how a u8/bool is
allocated with respect to padding and alignment, and I think that's why
we've been using u64 fields in the past.
I don't actually remember the details, but I'd rather err on the side of
caution than trying to save a few bytes. However, if someone can
convince me there's a completely safe way to do this, then I'm happy to
change it.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-26 16:17 ` Christoffer Dall
@ 2017-11-26 18:58 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 18:58 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
> Hi Yury,
>
> On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
> >
> > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > Avoid saving the guest VFP registers and restoring the host VFP
> > > registers on every exit from the VM. Only when we're about to run
> > > userspace or other threads in the kernel do we really have to switch the
> > > state back to the host state.
> > >
> > > We still initially configure the VFP registers to trap when entering the
> > > VM, but the difference is that we now leave the guest state in the
> > > hardware registers while running the VM.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > arch/arm64/kernel/asm-offsets.c | 1 +
> > > arch/arm64/kvm/hyp/entry.S | 3 +++
> > > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > > 6 files changed, 44 insertions(+), 36 deletions(-)
> > >
> > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > > index 1fbfe96..630dd60 100644
> > > --- a/arch/arm64/include/asm/kvm_emulate.h
> > > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > > return (unsigned long *)&vcpu->arch.hcr_el2;
> > > }
> > >
> > > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > > +{
> > > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > > +}
> > > +
> > > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > > {
> > > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > index 7d3bfa7..5e09eb9 100644
> > > --- a/arch/arm64/include/asm/kvm_host.h
> > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > > /* Guest debug state */
> > > u64 debug_flags;
> > >
> > > + /* 1 if the guest VFP state is loaded into the hardware */
> > > + u64 guest_vfp_loaded;
> >
> > May it be just u8/bool?
> >
> This particular field is accessed from assembly code, and I'm not sure
> what guarantees the compiler makes in terms of how a u8/bool is
> allocated with respect to padding and alignment, and I think that's why
> we've been using u64 fields in the past.
>
> I don't actually remember the details, but I'd rather err on the side of
> caution than trying to save a few bytes. However, if someone can
> convince me there's a completely safe way to do this, then I'm happy to
> change it.
'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
C6.6.181 STRB (register) in ARM64 ARM.
The only thing I would recommend is to reorder fields in kvm_vcpu_arch
to avoid unneeded holes in the structure. It already spend 10 bytes for
nothing in 3 holes.
Yury
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-26 18:58 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 18:58 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
> Hi Yury,
>
> On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
> >
> > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > Avoid saving the guest VFP registers and restoring the host VFP
> > > registers on every exit from the VM. Only when we're about to run
> > > userspace or other threads in the kernel do we really have to switch the
> > > state back to the host state.
> > >
> > > We still initially configure the VFP registers to trap when entering the
> > > VM, but the difference is that we now leave the guest state in the
> > > hardware registers while running the VM.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > arch/arm64/kernel/asm-offsets.c | 1 +
> > > arch/arm64/kvm/hyp/entry.S | 3 +++
> > > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > > 6 files changed, 44 insertions(+), 36 deletions(-)
> > >
> > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > > index 1fbfe96..630dd60 100644
> > > --- a/arch/arm64/include/asm/kvm_emulate.h
> > > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > > return (unsigned long *)&vcpu->arch.hcr_el2;
> > > }
> > >
> > > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > > +{
> > > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > > +}
> > > +
> > > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > > {
> > > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > index 7d3bfa7..5e09eb9 100644
> > > --- a/arch/arm64/include/asm/kvm_host.h
> > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > > /* Guest debug state */
> > > u64 debug_flags;
> > >
> > > + /* 1 if the guest VFP state is loaded into the hardware */
> > > + u64 guest_vfp_loaded;
> >
> > May it be just u8/bool?
> >
> This particular field is accessed from assembly code, and I'm not sure
> what guarantees the compiler makes in terms of how a u8/bool is
> allocated with respect to padding and alignment, and I think that's why
> we've been using u64 fields in the past.
>
> I don't actually remember the details, but I'd rather err on the side of
> caution than trying to save a few bytes. However, if someone can
> convince me there's a completely safe way to do this, then I'm happy to
> change it.
'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
C6.6.181 STRB (register) in ARM64 ARM.
The only thing I would recommend is to reorder fields in kvm_vcpu_arch
to avoid unneeded holes in the structure. It already spend 10 bytes for
nothing in 3 holes.
Yury
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-26 18:58 ` Yury Norov
@ 2017-11-26 19:18 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:18 UTC (permalink / raw)
To: Yury Norov; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Sun, Nov 26, 2017 at 09:58:52PM +0300, Yury Norov wrote:
> On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
> > Hi Yury,
> >
> > On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
> > >
> > > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > > Avoid saving the guest VFP registers and restoring the host VFP
> > > > registers on every exit from the VM. Only when we're about to run
> > > > userspace or other threads in the kernel do we really have to switch the
> > > > state back to the host state.
> > > >
> > > > We still initially configure the VFP registers to trap when entering the
> > > > VM, but the difference is that we now leave the guest state in the
> > > > hardware registers while running the VM.
> > > >
> > > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > > ---
> > > > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > > arch/arm64/kernel/asm-offsets.c | 1 +
> > > > arch/arm64/kvm/hyp/entry.S | 3 +++
> > > > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > > > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > > > 6 files changed, 44 insertions(+), 36 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > > > index 1fbfe96..630dd60 100644
> > > > --- a/arch/arm64/include/asm/kvm_emulate.h
> > > > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > > > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > > > return (unsigned long *)&vcpu->arch.hcr_el2;
> > > > }
> > > >
> > > > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > > > +{
> > > > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > > > +}
> > > > +
> > > > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > > > {
> > > > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > > index 7d3bfa7..5e09eb9 100644
> > > > --- a/arch/arm64/include/asm/kvm_host.h
> > > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > > > /* Guest debug state */
> > > > u64 debug_flags;
> > > >
> > > > + /* 1 if the guest VFP state is loaded into the hardware */
> > > > + u64 guest_vfp_loaded;
> > >
> > > May it be just u8/bool?
> > >
> > This particular field is accessed from assembly code, and I'm not sure
> > what guarantees the compiler makes in terms of how a u8/bool is
> > allocated with respect to padding and alignment, and I think that's why
> > we've been using u64 fields in the past.
> >
> > I don't actually remember the details, but I'd rather err on the side of
> > caution than trying to save a few bytes. However, if someone can
> > convince me there's a completely safe way to do this, then I'm happy to
> > change it.
>
> 'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
> C6.6.181 STRB (register) in ARM64 ARM.
I'm well aware of this instruction. Thank you though.
The concern was that we haven't done this in the past. I think that was
because the size of a _Bool is not well-defined and we really didn't
care about a couple of handful of bytes when talking about vcpu
structures. Really.
A u8 should work though, but probably this will all be moot if I combine
the flags into a single field.
>
> The only thing I would recommend is to reorder fields in kvm_vcpu_arch
> to avoid unneeded holes in the structure. It already spend 10 bytes for
> nothing in 3 holes.
>
Patches are welcome.
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-26 19:18 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:18 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 09:58:52PM +0300, Yury Norov wrote:
> On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
> > Hi Yury,
> >
> > On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
> > >
> > > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > > Avoid saving the guest VFP registers and restoring the host VFP
> > > > registers on every exit from the VM. Only when we're about to run
> > > > userspace or other threads in the kernel do we really have to switch the
> > > > state back to the host state.
> > > >
> > > > We still initially configure the VFP registers to trap when entering the
> > > > VM, but the difference is that we now leave the guest state in the
> > > > hardware registers while running the VM.
> > > >
> > > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > > ---
> > > > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > > arch/arm64/kernel/asm-offsets.c | 1 +
> > > > arch/arm64/kvm/hyp/entry.S | 3 +++
> > > > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > > > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > > > 6 files changed, 44 insertions(+), 36 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > > > index 1fbfe96..630dd60 100644
> > > > --- a/arch/arm64/include/asm/kvm_emulate.h
> > > > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > > > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > > > return (unsigned long *)&vcpu->arch.hcr_el2;
> > > > }
> > > >
> > > > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > > > +{
> > > > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > > > +}
> > > > +
> > > > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > > > {
> > > > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > > index 7d3bfa7..5e09eb9 100644
> > > > --- a/arch/arm64/include/asm/kvm_host.h
> > > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > > > /* Guest debug state */
> > > > u64 debug_flags;
> > > >
> > > > + /* 1 if the guest VFP state is loaded into the hardware */
> > > > + u64 guest_vfp_loaded;
> > >
> > > May it be just u8/bool?
> > >
> > This particular field is accessed from assembly code, and I'm not sure
> > what guarantees the compiler makes in terms of how a u8/bool is
> > allocated with respect to padding and alignment, and I think that's why
> > we've been using u64 fields in the past.
> >
> > I don't actually remember the details, but I'd rather err on the side of
> > caution than trying to save a few bytes. However, if someone can
> > convince me there's a completely safe way to do this, then I'm happy to
> > change it.
>
> 'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
> C6.6.181 STRB (register) in ARM64 ARM.
I'm well aware of this instruction. Thank you though.
The concern was that we haven't done this in the past. I think that was
because the size of a _Bool is not well-defined and we really didn't
care about a couple of handful of bytes when talking about vcpu
structures. Really.
A u8 should work though, but probably this will all be moot if I combine
the flags into a single field.
>
> The only thing I would recommend is to reorder fields in kvm_vcpu_arch
> to avoid unneeded holes in the structure. It already spend 10 bytes for
> nothing in 3 holes.
>
Patches are welcome.
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-26 19:18 ` Christoffer Dall
@ 2017-11-27 6:25 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-27 6:25 UTC (permalink / raw)
To: Christoffer Dall
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Sun, Nov 26, 2017 at 08:18:34PM +0100, Christoffer Dall wrote:
> On Sun, Nov 26, 2017 at 09:58:52PM +0300, Yury Norov wrote:
> > On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
> > > Hi Yury,
> > >
> > > On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
> > > >
> > > > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > > > Avoid saving the guest VFP registers and restoring the host VFP
> > > > > registers on every exit from the VM. Only when we're about to run
> > > > > userspace or other threads in the kernel do we really have to switch the
> > > > > state back to the host state.
> > > > >
> > > > > We still initially configure the VFP registers to trap when entering the
> > > > > VM, but the difference is that we now leave the guest state in the
> > > > > hardware registers while running the VM.
> > > > >
> > > > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > > > ---
> > > > > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > > > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > > > arch/arm64/kernel/asm-offsets.c | 1 +
> > > > > arch/arm64/kvm/hyp/entry.S | 3 +++
> > > > > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > > > > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > > > > 6 files changed, 44 insertions(+), 36 deletions(-)
> > > > >
> > > > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > > > > index 1fbfe96..630dd60 100644
> > > > > --- a/arch/arm64/include/asm/kvm_emulate.h
> > > > > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > > > > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > > > > return (unsigned long *)&vcpu->arch.hcr_el2;
> > > > > }
> > > > >
> > > > > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > > > > +{
> > > > > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > > > > +}
> > > > > +
> > > > > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > > > > {
> > > > > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > > > index 7d3bfa7..5e09eb9 100644
> > > > > --- a/arch/arm64/include/asm/kvm_host.h
> > > > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > > > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > > > > /* Guest debug state */
> > > > > u64 debug_flags;
> > > > >
> > > > > + /* 1 if the guest VFP state is loaded into the hardware */
> > > > > + u64 guest_vfp_loaded;
> > > >
> > > > May it be just u8/bool?
> > > >
> > > This particular field is accessed from assembly code, and I'm not sure
> > > what guarantees the compiler makes in terms of how a u8/bool is
> > > allocated with respect to padding and alignment, and I think that's why
> > > we've been using u64 fields in the past.
> > >
> > > I don't actually remember the details, but I'd rather err on the side of
> > > caution than trying to save a few bytes. However, if someone can
> > > convince me there's a completely safe way to do this, then I'm happy to
> > > change it.
> >
> > 'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
> > C6.6.181 STRB (register) in ARM64 ARM.
>
> I'm well aware of this instruction. Thank you though.
>
> The concern was that we haven't done this in the past. I think that was
> because the size of a _Bool is not well-defined and we really didn't
> care about a couple of handful of bytes when talking about vcpu
> structures. Really.
>
> A u8 should work though, but probably this will all be moot if I combine
> the flags into a single field.
>
> >
> > The only thing I would recommend is to reorder fields in kvm_vcpu_arch
> > to avoid unneeded holes in the structure. It already spend 10 bytes for
> > nothing in 3 holes.
> >
> Patches are welcome.
Heh :) I meant reordering only this field if it is changed.
If you want me to reorder the whole structure and remove all holes...
Patches of that sort (I mean moving fields here and there just to save
couple of bytes) are looking weird. At most because there is general
assumption that the hole exists because author prefers to have clean
logic in field order even with the cost of few holes. But if you give
me indulgence...
Nevertheless, for this specific structure:
Before:
/* size: 8176, cachelines: 128, members: 23 */
/* sum members: 8152, holes: 3, sum holes: 10 */
/* padding: 14 */
/* last cacheline: 48 bytes */
After:
/* size: 8160, cachelines: 128, members: 23 */
/* padding: 8 */
/* last cacheline: 32 bytes */
The patch is below.
Yury
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index dcded44b4180..3739471c39ac 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -200,10 +200,6 @@ typedef struct kvm_cpu_context kvm_cpu_context_t;
struct kvm_vcpu_arch {
struct kvm_cpu_context ctxt;
- /* HYP configuration */
- u64 hcr_el2;
- u32 mdcr_el2;
-
/* Exception Information */
struct kvm_vcpu_fault_info fault;
@@ -249,6 +245,20 @@ struct kvm_vcpu_arch {
* here.
*/
+ /* IO related fields */
+ struct kvm_decode mmio_decode;
+
+ /* Cache some mmu pages needed inside spinlock regions */
+ struct kvm_mmu_memory_cache mmu_page_cache;
+
+ /* HYP configuration */
+ u64 hcr_el2;
+ u32 mdcr_el2;
+
+ /* Target CPU and feature flags */
+ int target;
+ DECLARE_BITMAP(features, KVM_VCPU_MAX_FEATURES);
+
/*
* Guest registers we preserve during guest debugging.
*
@@ -266,16 +276,6 @@ struct kvm_vcpu_arch {
/* Don't run the guest (internal implementation need) */
bool pause;
- /* IO related fields */
- struct kvm_decode mmio_decode;
-
- /* Cache some mmu pages needed inside spinlock regions */
- struct kvm_mmu_memory_cache mmu_page_cache;
-
- /* Target CPU and feature flags */
- int target;
- DECLARE_BITMAP(features, KVM_VCPU_MAX_FEATURES);
-
/* Detect first run of a vcpu */
bool has_run_once;
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-27 6:25 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-27 6:25 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 08:18:34PM +0100, Christoffer Dall wrote:
> On Sun, Nov 26, 2017 at 09:58:52PM +0300, Yury Norov wrote:
> > On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
> > > Hi Yury,
> > >
> > > On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
> > > >
> > > > On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
> > > > > Avoid saving the guest VFP registers and restoring the host VFP
> > > > > registers on every exit from the VM. Only when we're about to run
> > > > > userspace or other threads in the kernel do we really have to switch the
> > > > > state back to the host state.
> > > > >
> > > > > We still initially configure the VFP registers to trap when entering the
> > > > > VM, but the difference is that we now leave the guest state in the
> > > > > hardware registers while running the VM.
> > > > >
> > > > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > > > ---
> > > > > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > > > > arch/arm64/include/asm/kvm_host.h | 3 +++
> > > > > arch/arm64/kernel/asm-offsets.c | 1 +
> > > > > arch/arm64/kvm/hyp/entry.S | 3 +++
> > > > > arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
> > > > > arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
> > > > > 6 files changed, 44 insertions(+), 36 deletions(-)
> > > > >
> > > > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > > > > index 1fbfe96..630dd60 100644
> > > > > --- a/arch/arm64/include/asm/kvm_emulate.h
> > > > > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > > > > @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > > > > return (unsigned long *)&vcpu->arch.hcr_el2;
> > > > > }
> > > > >
> > > > > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > > > > +{
> > > > > + return (!(vcpu->arch.hcr_el2 & HCR_RW));
> > > > > +}
> > > > > +
> > > > > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > > > > {
> > > > > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > > > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > > > > index 7d3bfa7..5e09eb9 100644
> > > > > --- a/arch/arm64/include/asm/kvm_host.h
> > > > > +++ b/arch/arm64/include/asm/kvm_host.h
> > > > > @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
> > > > > /* Guest debug state */
> > > > > u64 debug_flags;
> > > > >
> > > > > + /* 1 if the guest VFP state is loaded into the hardware */
> > > > > + u64 guest_vfp_loaded;
> > > >
> > > > May it be just u8/bool?
> > > >
> > > This particular field is accessed from assembly code, and I'm not sure
> > > what guarantees the compiler makes in terms of how a u8/bool is
> > > allocated with respect to padding and alignment, and I think that's why
> > > we've been using u64 fields in the past.
> > >
> > > I don't actually remember the details, but I'd rather err on the side of
> > > caution than trying to save a few bytes. However, if someone can
> > > convince me there's a completely safe way to do this, then I'm happy to
> > > change it.
> >
> > 'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
> > C6.6.181 STRB (register) in ARM64 ARM.
>
> I'm well aware of this instruction. Thank you though.
>
> The concern was that we haven't done this in the past. I think that was
> because the size of a _Bool is not well-defined and we really didn't
> care about a couple of handful of bytes when talking about vcpu
> structures. Really.
>
> A u8 should work though, but probably this will all be moot if I combine
> the flags into a single field.
>
> >
> > The only thing I would recommend is to reorder fields in kvm_vcpu_arch
> > to avoid unneeded holes in the structure. It already spend 10 bytes for
> > nothing in 3 holes.
> >
> Patches are welcome.
Heh :) I meant reordering only this field if it is changed.
If you want me to reorder the whole structure and remove all holes...
Patches of that sort (I mean moving fields here and there just to save
couple of bytes) are looking weird. At most because there is general
assumption that the hole exists because author prefers to have clean
logic in field order even with the cost of few holes. But if you give
me indulgence...
Nevertheless, for this specific structure:
Before:
/* size: 8176, cachelines: 128, members: 23 */
/* sum members: 8152, holes: 3, sum holes: 10 */
/* padding: 14 */
/* last cacheline: 48 bytes */
After:
/* size: 8160, cachelines: 128, members: 23 */
/* padding: 8 */
/* last cacheline: 32 bytes */
The patch is below.
Yury
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index dcded44b4180..3739471c39ac 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -200,10 +200,6 @@ typedef struct kvm_cpu_context kvm_cpu_context_t;
struct kvm_vcpu_arch {
struct kvm_cpu_context ctxt;
- /* HYP configuration */
- u64 hcr_el2;
- u32 mdcr_el2;
-
/* Exception Information */
struct kvm_vcpu_fault_info fault;
@@ -249,6 +245,20 @@ struct kvm_vcpu_arch {
* here.
*/
+ /* IO related fields */
+ struct kvm_decode mmio_decode;
+
+ /* Cache some mmu pages needed inside spinlock regions */
+ struct kvm_mmu_memory_cache mmu_page_cache;
+
+ /* HYP configuration */
+ u64 hcr_el2;
+ u32 mdcr_el2;
+
+ /* Target CPU and feature flags */
+ int target;
+ DECLARE_BITMAP(features, KVM_VCPU_MAX_FEATURES);
+
/*
* Guest registers we preserve during guest debugging.
*
@@ -266,16 +276,6 @@ struct kvm_vcpu_arch {
/* Don't run the guest (internal implementation need) */
bool pause;
- /* IO related fields */
- struct kvm_decode mmio_decode;
-
- /* Cache some mmu pages needed inside spinlock regions */
- struct kvm_mmu_memory_cache mmu_page_cache;
-
- /* Target CPU and feature flags */
- int target;
- DECLARE_BITMAP(features, KVM_VCPU_MAX_FEATURES);
-
/* Detect first run of a vcpu */
bool has_run_once;
^ permalink raw reply related [flat|nested] 254+ messages in thread
* Re: [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
2017-11-26 18:58 ` Yury Norov
@ 2017-11-30 19:07 ` Marc Zyngier
-1 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-11-30 19:07 UTC (permalink / raw)
To: Yury Norov, Christoffer Dall
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Shih-Wei Li
On 26/11/17 18:58, Yury Norov wrote:
> On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
>> Hi Yury,
>>
>> On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
>>>
>>> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
>>>> Avoid saving the guest VFP registers and restoring the host VFP
>>>> registers on every exit from the VM. Only when we're about to run
>>>> userspace or other threads in the kernel do we really have to switch the
>>>> state back to the host state.
>>>>
>>>> We still initially configure the VFP registers to trap when entering the
>>>> VM, but the difference is that we now leave the guest state in the
>>>> hardware registers while running the VM.
>>>>
>>>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> ---
>>>> arch/arm64/include/asm/kvm_emulate.h | 5 ++++
>>>> arch/arm64/include/asm/kvm_host.h | 3 +++
>>>> arch/arm64/kernel/asm-offsets.c | 1 +
>>>> arch/arm64/kvm/hyp/entry.S | 3 +++
>>>> arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
>>>> arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
>>>> 6 files changed, 44 insertions(+), 36 deletions(-)
>>>>
>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>>>> index 1fbfe96..630dd60 100644
>>>> --- a/arch/arm64/include/asm/kvm_emulate.h
>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>>>> @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
>>>> return (unsigned long *)&vcpu->arch.hcr_el2;
>>>> }
>>>>
>>>> +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
>>>> +{
>>>> + return (!(vcpu->arch.hcr_el2 & HCR_RW));
>>>> +}
>>>> +
>>>> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
>>>> {
>>>> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
>>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>>>> index 7d3bfa7..5e09eb9 100644
>>>> --- a/arch/arm64/include/asm/kvm_host.h
>>>> +++ b/arch/arm64/include/asm/kvm_host.h
>>>> @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
>>>> /* Guest debug state */
>>>> u64 debug_flags;
>>>>
>>>> + /* 1 if the guest VFP state is loaded into the hardware */
>>>> + u64 guest_vfp_loaded;
>>>
>>> May it be just u8/bool?
>>>
>> This particular field is accessed from assembly code, and I'm not sure
>> what guarantees the compiler makes in terms of how a u8/bool is
>> allocated with respect to padding and alignment, and I think that's why
>> we've been using u64 fields in the past.
>>
>> I don't actually remember the details, but I'd rather err on the side of
>> caution than trying to save a few bytes. However, if someone can
>> convince me there's a completely safe way to do this, then I'm happy to
>> change it.
>
> 'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
> C6.6.181 STRB (register) in ARM64 ARM.
>
> The only thing I would recommend is to reorder fields in kvm_vcpu_arch
> to avoid unneeded holes in the structure. It already spend 10 bytes for
> nothing in 3 holes.
Terrifying. How many vcpu are you going to run before this becomes a
real bottleneck? KVM on a 6502? ;-)
Now, when it comes to reordering fields, please keep in mind that the
order of the fields in the structure does matter. We want the hottest
fields grouped together so that they are fetched in the same cache line.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 08/37] KVM: arm64: Defer restoring host VFP state to vcpu_put
@ 2017-11-30 19:07 ` Marc Zyngier
0 siblings, 0 replies; 254+ messages in thread
From: Marc Zyngier @ 2017-11-30 19:07 UTC (permalink / raw)
To: linux-arm-kernel
On 26/11/17 18:58, Yury Norov wrote:
> On Sun, Nov 26, 2017 at 05:17:16PM +0100, Christoffer Dall wrote:
>> Hi Yury,
>>
>> On Sat, Nov 25, 2017 at 10:52:21AM +0300, Yury Norov wrote:
>>>
>>> On Thu, Oct 12, 2017 at 12:41:12PM +0200, Christoffer Dall wrote:
>>>> Avoid saving the guest VFP registers and restoring the host VFP
>>>> registers on every exit from the VM. Only when we're about to run
>>>> userspace or other threads in the kernel do we really have to switch the
>>>> state back to the host state.
>>>>
>>>> We still initially configure the VFP registers to trap when entering the
>>>> VM, but the difference is that we now leave the guest state in the
>>>> hardware registers while running the VM.
>>>>
>>>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>>>> ---
>>>> arch/arm64/include/asm/kvm_emulate.h | 5 ++++
>>>> arch/arm64/include/asm/kvm_host.h | 3 +++
>>>> arch/arm64/kernel/asm-offsets.c | 1 +
>>>> arch/arm64/kvm/hyp/entry.S | 3 +++
>>>> arch/arm64/kvm/hyp/switch.c | 47 +++++++++++-------------------------
>>>> arch/arm64/kvm/hyp/sysreg-sr.c | 21 +++++++++++++---
>>>> 6 files changed, 44 insertions(+), 36 deletions(-)
>>>>
>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>>>> index 1fbfe96..630dd60 100644
>>>> --- a/arch/arm64/include/asm/kvm_emulate.h
>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>>>> @@ -56,6 +56,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
>>>> return (unsigned long *)&vcpu->arch.hcr_el2;
>>>> }
>>>>
>>>> +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
>>>> +{
>>>> + return (!(vcpu->arch.hcr_el2 & HCR_RW));
>>>> +}
>>>> +
>>>> static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
>>>> {
>>>> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
>>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>>>> index 7d3bfa7..5e09eb9 100644
>>>> --- a/arch/arm64/include/asm/kvm_host.h
>>>> +++ b/arch/arm64/include/asm/kvm_host.h
>>>> @@ -210,6 +210,9 @@ struct kvm_vcpu_arch {
>>>> /* Guest debug state */
>>>> u64 debug_flags;
>>>>
>>>> + /* 1 if the guest VFP state is loaded into the hardware */
>>>> + u64 guest_vfp_loaded;
>>>
>>> May it be just u8/bool?
>>>
>> This particular field is accessed from assembly code, and I'm not sure
>> what guarantees the compiler makes in terms of how a u8/bool is
>> allocated with respect to padding and alignment, and I think that's why
>> we've been using u64 fields in the past.
>>
>> I don't actually remember the details, but I'd rather err on the side of
>> caution than trying to save a few bytes. However, if someone can
>> convince me there's a completely safe way to do this, then I'm happy to
>> change it.
>
> 'strb w0, [x3, #VCPU_GUEST_VFP_LOADED]' would work. See
> C6.6.181 STRB (register) in ARM64 ARM.
>
> The only thing I would recommend is to reorder fields in kvm_vcpu_arch
> to avoid unneeded holes in the structure. It already spend 10 bytes for
> nothing in 3 holes.
Terrifying. How many vcpu are you going to run before this becomes a
real bottleneck? KVM on a 6502? ;-)
Now, when it comes to reordering fields, please keep in mind that the
order of the fields in the structure does matter. We want the hottest
fields grouped together so that they are fetched in the same cache line.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
There is no need to figure out inside the world-switch if we should
save/restore the debug registers or not, we can might as well do that in
the higher level debug setup code, making it easier to optimize down the
line.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/debug.c | 9 +++++++++
arch/arm64/kvm/hyp/debug-sr.c | 6 ------
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index dbadfaf..62550de19 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
if (trap_debug)
vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
+ /*
+ * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
+ * a full save/restore cycle.
+ */
+ if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
+ (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
+ vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
+
+
trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_sys_reg(vcpu, MDSCR_EL1));
}
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index f5154ed..0fc0758 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -172,12 +172,6 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
{
- /* If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
- * a full save/restore cycle. */
- if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
- (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_MDE))
- vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
-
__debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
kern_hyp_va(vcpu->arch.host_cpu_context));
__debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
There is no need to figure out inside the world-switch if we should
save/restore the debug registers or not, we can might as well do that in
the higher level debug setup code, making it easier to optimize down the
line.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/debug.c | 9 +++++++++
arch/arm64/kvm/hyp/debug-sr.c | 6 ------
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
index dbadfaf..62550de19 100644
--- a/arch/arm64/kvm/debug.c
+++ b/arch/arm64/kvm/debug.c
@@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
if (trap_debug)
vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
+ /*
+ * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
+ * a full save/restore cycle.
+ */
+ if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
+ (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
+ vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
+
+
trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_sys_reg(vcpu, MDSCR_EL1));
}
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index f5154ed..0fc0758 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -172,12 +172,6 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
{
- /* If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
- * a full save/restore cycle. */
- if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
- (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_MDE))
- vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
-
__debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
kern_hyp_va(vcpu->arch.host_cpu_context));
__debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 14:09 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 14:09 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> There is no need to figure out inside the world-switch if we should
> save/restore the debug registers or not, we can might as well do that in
> the higher level debug setup code, making it easier to optimize down the
> line.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/debug.c | 9 +++++++++
> arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> 2 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> index dbadfaf..62550de19 100644
> --- a/arch/arm64/kvm/debug.c
> +++ b/arch/arm64/kvm/debug.c
> @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> if (trap_debug)
> vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
>
> + /*
> + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> + * a full save/restore cycle.
The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
tests KDE and MDE.
> + */
> + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
nit: could also write as
if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> +
It looks like there's only one flag for debug_flags - this dirty flag,
which I guess is also used to trigger trapping. So maybe this could be a
second flag of a "lazy state" field, as I suggested earlier?
> +
> trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
> trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_sys_reg(vcpu, MDSCR_EL1));
> }
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index f5154ed..0fc0758 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -172,12 +172,6 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
>
> void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> {
> - /* If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> - * a full save/restore cycle. */
> - if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> - (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_MDE))
> - vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> -
> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
> __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> --
> 2.9.0
>
Also, while grepping, I noticed __debug_cond_restore_host_state() unsets
KVM_ARM64_DEBUG_DIRTY on that condition that it's set. A micro
optimization could be to do it unconditionally, removing the branch.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
@ 2017-11-07 14:09 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 14:09 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> There is no need to figure out inside the world-switch if we should
> save/restore the debug registers or not, we can might as well do that in
> the higher level debug setup code, making it easier to optimize down the
> line.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/debug.c | 9 +++++++++
> arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> 2 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> index dbadfaf..62550de19 100644
> --- a/arch/arm64/kvm/debug.c
> +++ b/arch/arm64/kvm/debug.c
> @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> if (trap_debug)
> vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
>
> + /*
> + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> + * a full save/restore cycle.
The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
tests KDE and MDE.
> + */
> + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
nit: could also write as
if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> +
It looks like there's only one flag for debug_flags - this dirty flag,
which I guess is also used to trigger trapping. So maybe this could be a
second flag of a "lazy state" field, as I suggested earlier?
> +
> trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
> trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_sys_reg(vcpu, MDSCR_EL1));
> }
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index f5154ed..0fc0758 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -172,12 +172,6 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
>
> void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> {
> - /* If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> - * a full save/restore cycle. */
> - if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> - (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_MDE))
> - vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> -
> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
> __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> --
> 2.9.0
>
Also, while grepping, I noticed __debug_cond_restore_host_state() unsets
KVM_ARM64_DEBUG_DIRTY on that condition that it's set. A micro
optimization could be to do it unconditionally, removing the branch.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
2017-11-07 14:09 ` Andrew Jones
@ 2017-11-25 8:09 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-25 8:09 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 03:09:19PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> > There is no need to figure out inside the world-switch if we should
> > save/restore the debug registers or not, we can might as well do that in
> > the higher level debug setup code, making it easier to optimize down the
> > line.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/debug.c | 9 +++++++++
> > arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> > 2 files changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> > index dbadfaf..62550de19 100644
> > --- a/arch/arm64/kvm/debug.c
> > +++ b/arch/arm64/kvm/debug.c
> > @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> > if (trap_debug)
> > vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
> >
> > + /*
> > + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > + * a full save/restore cycle.
>
> The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
> tests KDE and MDE.
>
> > + */
> > + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> > + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
>
> nit: could also write as
>
> if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
Another nit: two empty lines at the end of this chunk.
> > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > +
>
> It looks like there's only one flag for debug_flags - this dirty flag,
> which I guess is also used to trigger trapping. So maybe this could be a
> second flag of a "lazy state" field, as I suggested earlier?
>
> > +
> > trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
> > trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_sys_reg(vcpu, MDSCR_EL1));
> > }
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > index f5154ed..0fc0758 100644
> > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -172,12 +172,6 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> >
> > void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > {
> > - /* If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > - * a full save/restore cycle. */
> > - if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> > - (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_MDE))
> > - vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > -
> > __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> > __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> > --
> > 2.9.0
> >
>
> Also, while grepping, I noticed __debug_cond_restore_host_state() unsets
> KVM_ARM64_DEBUG_DIRTY on that condition that it's set. A micro
> optimization could be to do it unconditionally, removing the branch.
>
> Thanks,
> drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
@ 2017-11-25 8:09 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-25 8:09 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 03:09:19PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> > There is no need to figure out inside the world-switch if we should
> > save/restore the debug registers or not, we can might as well do that in
> > the higher level debug setup code, making it easier to optimize down the
> > line.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/debug.c | 9 +++++++++
> > arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> > 2 files changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> > index dbadfaf..62550de19 100644
> > --- a/arch/arm64/kvm/debug.c
> > +++ b/arch/arm64/kvm/debug.c
> > @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> > if (trap_debug)
> > vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
> >
> > + /*
> > + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > + * a full save/restore cycle.
>
> The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
> tests KDE and MDE.
>
> > + */
> > + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> > + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
>
> nit: could also write as
>
> if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
Another nit: two empty lines at the end of this chunk.
> > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > +
>
> It looks like there's only one flag for debug_flags - this dirty flag,
> which I guess is also used to trigger trapping. So maybe this could be a
> second flag of a "lazy state" field, as I suggested earlier?
>
> > +
> > trace_kvm_arm_set_dreg32("MDCR_EL2", vcpu->arch.mdcr_el2);
> > trace_kvm_arm_set_dreg32("MDSCR_EL1", vcpu_sys_reg(vcpu, MDSCR_EL1));
> > }
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > index f5154ed..0fc0758 100644
> > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -172,12 +172,6 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> >
> > void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > {
> > - /* If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > - * a full save/restore cycle. */
> > - if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> > - (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_MDE))
> > - vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > -
> > __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> > __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> > --
> > 2.9.0
> >
>
> Also, while grepping, I noticed __debug_cond_restore_host_state() unsets
> KVM_ARM64_DEBUG_DIRTY on that condition that it's set. A micro
> optimization could be to do it unconditionally, removing the branch.
>
> Thanks,
> drew
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
2017-11-07 14:09 ` Andrew Jones
@ 2017-12-01 17:25 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 17:25 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 03:09:19PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> > There is no need to figure out inside the world-switch if we should
> > save/restore the debug registers or not, we can might as well do that in
> > the higher level debug setup code, making it easier to optimize down the
> > line.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/debug.c | 9 +++++++++
> > arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> > 2 files changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> > index dbadfaf..62550de19 100644
> > --- a/arch/arm64/kvm/debug.c
> > +++ b/arch/arm64/kvm/debug.c
> > @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> > if (trap_debug)
> > vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
> >
> > + /*
> > + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > + * a full save/restore cycle.
>
> The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
> tests KDE and MDE.
>
You meant the comment, right?
> > + */
> > + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> > + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
>
> nit: could also write as
>
> if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
>
I actually prefer it the other way, and I think the compiler will figure
out what to do in terms of efficiency.
> > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > +
>
> It looks like there's only one flag for debug_flags - this dirty flag,
> which I guess is also used to trigger trapping. So maybe this could be a
> second flag of a "lazy state" field, as I suggested earlier?
>
I'm going to leave this one for now, but we can improve that later on if
we want to save a little space in the vcpu struct, or rather if we want
to rearrange things to make frequently accessed fields fit in the same
cache line.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
@ 2017-12-01 17:25 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 17:25 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 03:09:19PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> > There is no need to figure out inside the world-switch if we should
> > save/restore the debug registers or not, we can might as well do that in
> > the higher level debug setup code, making it easier to optimize down the
> > line.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/debug.c | 9 +++++++++
> > arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> > 2 files changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> > index dbadfaf..62550de19 100644
> > --- a/arch/arm64/kvm/debug.c
> > +++ b/arch/arm64/kvm/debug.c
> > @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> > if (trap_debug)
> > vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
> >
> > + /*
> > + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > + * a full save/restore cycle.
>
> The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
> tests KDE and MDE.
>
You meant the comment, right?
> > + */
> > + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> > + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
>
> nit: could also write as
>
> if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
>
I actually prefer it the other way, and I think the compiler will figure
out what to do in terms of efficiency.
> > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > +
>
> It looks like there's only one flag for debug_flags - this dirty flag,
> which I guess is also used to trigger trapping. So maybe this could be a
> second flag of a "lazy state" field, as I suggested earlier?
>
I'm going to leave this one for now, but we can improve that later on if
we want to save a little space in the vcpu struct, or rather if we want
to rearrange things to make frequently accessed fields fit in the same
cache line.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
2017-12-01 17:25 ` Christoffer Dall
@ 2017-12-03 13:17 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-12-03 13:17 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Fri, Dec 01, 2017 at 06:25:46PM +0100, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 03:09:19PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> > > There is no need to figure out inside the world-switch if we should
> > > save/restore the debug registers or not, we can might as well do that in
> > > the higher level debug setup code, making it easier to optimize down the
> > > line.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm64/kvm/debug.c | 9 +++++++++
> > > arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> > > 2 files changed, 9 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> > > index dbadfaf..62550de19 100644
> > > --- a/arch/arm64/kvm/debug.c
> > > +++ b/arch/arm64/kvm/debug.c
> > > @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> > > if (trap_debug)
> > > vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
> > >
> > > + /*
> > > + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > > + * a full save/restore cycle.
> >
> > The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
> > tests KDE and MDE.
> >
>
> You meant the comment, right?
yup
>
> > > + */
> > > + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> > > + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
> >
> > nit: could also write as
> >
> > if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
> >
>
> I actually prefer it the other way, and I think the compiler will figure
> out what to do in terms of efficiency.
I won't argue about code aesthetics (although I disagree here :-). I will
point out that in this case the compiler will no doubt do the right thing,
as vcpu_sys_reg() is just a macro, but for a pointer taking function in
general there would be potential to get both calls emitted, as the
compiler wouldn't be sure there would be no side effect.
>
> > > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > > +
> >
> > It looks like there's only one flag for debug_flags - this dirty flag,
> > which I guess is also used to trigger trapping. So maybe this could be a
> > second flag of a "lazy state" field, as I suggested earlier?
> >
>
> I'm going to leave this one for now, but we can improve that later on if
> we want to save a little space in the vcpu struct, or rather if we want
> to rearrange things to make frequently accessed fields fit in the same
> cache line.
>
OK
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 09/37] KVM: arm64: Move debug dirty flag calculation out of world switch
@ 2017-12-03 13:17 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-12-03 13:17 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Dec 01, 2017 at 06:25:46PM +0100, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 03:09:19PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:13PM +0200, Christoffer Dall wrote:
> > > There is no need to figure out inside the world-switch if we should
> > > save/restore the debug registers or not, we can might as well do that in
> > > the higher level debug setup code, making it easier to optimize down the
> > > line.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm64/kvm/debug.c | 9 +++++++++
> > > arch/arm64/kvm/hyp/debug-sr.c | 6 ------
> > > 2 files changed, 9 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c
> > > index dbadfaf..62550de19 100644
> > > --- a/arch/arm64/kvm/debug.c
> > > +++ b/arch/arm64/kvm/debug.c
> > > @@ -193,6 +193,15 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)
> > > if (trap_debug)
> > > vcpu->arch.mdcr_el2 |= MDCR_EL2_TDA;
> > >
> > > + /*
> > > + * If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY is set, perform
> > > + * a full save/restore cycle.
> >
> > The commit message implies testing KVM_ARM64_DEBUG_DIRTY, but it only
> > tests KDE and MDE.
> >
>
> You meant the comment, right?
yup
>
> > > + */
> > > + if ((vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_KDE) ||
> > > + (vcpu_sys_reg(vcpu, MDSCR_EL1) & DBG_MDSCR_MDE))
> >
> > nit: could also write as
> >
> > if (vcpu_sys_reg(vcpu, MDSCR_EL1) & (DBG_MDSCR_KDE | DBG_MDSCR_MDE))
> >
>
> I actually prefer it the other way, and I think the compiler will figure
> out what to do in terms of efficiency.
I won't argue about code aesthetics (although I disagree here :-). I will
point out that in this case the compiler will no doubt do the right thing,
as vcpu_sys_reg() is just a macro, but for a pointer taking function in
general there would be potential to get both calls emitted, as the
compiler wouldn't be sure there would be no side effect.
>
> > > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > > +
> >
> > It looks like there's only one flag for debug_flags - this dirty flag,
> > which I guess is also used to trigger trapping. So maybe this could be a
> > second flag of a "lazy state" field, as I suggested earlier?
> >
>
> I'm going to leave this one for now, but we can improve that later on if
> we want to save a little space in the vcpu struct, or rather if we want
> to rearrange things to make frequently accessed fields fit in the same
> cache line.
>
OK
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
The debug save/restore functions can be improved by using the has_vhe()
static key instead of the instruction alternative. Using the static key
uses the same paradigm as we're going to use elsewhere, it makes the
code more readable, and it generates slightly better code (no
stack setups and function calls unless necessary).
We also use a static key on the restore path, because it will be
marginally faster than loading a value from memory.
Finally, we don't have to conditionally clear the debug dirty flag if
it's set, we can just clear it.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
1 file changed, 9 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index 0fc0758..a2291b6 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -75,11 +75,6 @@
#define psb_csync() asm volatile("hint #17")
-static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
-{
- /* The vcpu can run. but it can't hide. */
-}
-
static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
{
u64 reg;
@@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
dsb(nsh);
}
-static hyp_alternate_select(__debug_save_spe,
- __debug_save_spe_nvhe, __debug_save_spe_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
{
if (!pmscr_el1)
@@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
{
__debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
kern_hyp_va(vcpu->arch.host_cpu_context));
- __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
+
+ /* Non-VHE: Disable and flush SPE data generation
+ * VHE: The vcpu can run. but it can't hide. */
+ if (!has_vhe())
+ __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
}
void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
{
- __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
+ if (!has_vhe())
+ __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
+
__debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
kern_hyp_va(vcpu->arch.host_cpu_context));
- if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
- vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
+ vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
}
u32 __hyp_text __kvm_get_mdcr_el2(void)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
The debug save/restore functions can be improved by using the has_vhe()
static key instead of the instruction alternative. Using the static key
uses the same paradigm as we're going to use elsewhere, it makes the
code more readable, and it generates slightly better code (no
stack setups and function calls unless necessary).
We also use a static key on the restore path, because it will be
marginally faster than loading a value from memory.
Finally, we don't have to conditionally clear the debug dirty flag if
it's set, we can just clear it.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
1 file changed, 9 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index 0fc0758..a2291b6 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -75,11 +75,6 @@
#define psb_csync() asm volatile("hint #17")
-static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
-{
- /* The vcpu can run. but it can't hide. */
-}
-
static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
{
u64 reg;
@@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
dsb(nsh);
}
-static hyp_alternate_select(__debug_save_spe,
- __debug_save_spe_nvhe, __debug_save_spe_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
{
if (!pmscr_el1)
@@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
{
__debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
kern_hyp_va(vcpu->arch.host_cpu_context));
- __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
+
+ /* Non-VHE: Disable and flush SPE data generation
+ * VHE: The vcpu can run. but it can't hide. */
+ if (!has_vhe())
+ __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
}
void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
{
- __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
+ if (!has_vhe())
+ __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
+
__debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
kern_hyp_va(vcpu->arch.host_cpu_context));
- if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
- vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
+ vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
}
u32 __hyp_text __kvm_get_mdcr_el2(void)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 14:22 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 14:22 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:14PM +0200, Christoffer Dall wrote:
> The debug save/restore functions can be improved by using the has_vhe()
> static key instead of the instruction alternative. Using the static key
> uses the same paradigm as we're going to use elsewhere, it makes the
> code more readable, and it generates slightly better code (no
> stack setups and function calls unless necessary).
>
> We also use a static key on the restore path, because it will be
> marginally faster than loading a value from memory.
>
> Finally, we don't have to conditionally clear the debug dirty flag if
> it's set, we can just clear it.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> 1 file changed, 9 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 0fc0758..a2291b6 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -75,11 +75,6 @@
>
> #define psb_csync() asm volatile("hint #17")
>
> -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> -{
> - /* The vcpu can run. but it can't hide. */
> -}
> -
> static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> {
> u64 reg;
> @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> dsb(nsh);
> }
>
> -static hyp_alternate_select(__debug_save_spe,
> - __debug_save_spe_nvhe, __debug_save_spe_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> {
> if (!pmscr_el1)
> @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> {
> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
> - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> +
> + /* Non-VHE: Disable and flush SPE data generation
> + * VHE: The vcpu can run. but it can't hide. */
Not the standard comment format.
s/./,/
I'm glad you kept the funny comment :-)
> + if (!has_vhe())
> + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> }
>
> void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> {
> - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> + if (!has_vhe())
> + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> +
> __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
>
> - if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> - vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
Guess I should have read ahead before commenting on this in the last
patch :-)
> }
>
> u32 __hyp_text __kvm_get_mdcr_el2(void)
> --
> 2.9.0
>
Do we still need to pass vcpu->arch.host_debug_state.pmscr_el1 as a
parameter to __debug_save_spe_nvhe and __debug_restore_spe? Or can
we just pass the vcpu and remove the "if (!pmscr_el1) return" in
__debug_restore_spe? Should __debug_restore_spe be renamed to have
a _nvhe for consistency?
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
@ 2017-11-07 14:22 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 14:22 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:14PM +0200, Christoffer Dall wrote:
> The debug save/restore functions can be improved by using the has_vhe()
> static key instead of the instruction alternative. Using the static key
> uses the same paradigm as we're going to use elsewhere, it makes the
> code more readable, and it generates slightly better code (no
> stack setups and function calls unless necessary).
>
> We also use a static key on the restore path, because it will be
> marginally faster than loading a value from memory.
>
> Finally, we don't have to conditionally clear the debug dirty flag if
> it's set, we can just clear it.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> 1 file changed, 9 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 0fc0758..a2291b6 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -75,11 +75,6 @@
>
> #define psb_csync() asm volatile("hint #17")
>
> -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> -{
> - /* The vcpu can run. but it can't hide. */
> -}
> -
> static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> {
> u64 reg;
> @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> dsb(nsh);
> }
>
> -static hyp_alternate_select(__debug_save_spe,
> - __debug_save_spe_nvhe, __debug_save_spe_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> {
> if (!pmscr_el1)
> @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> {
> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
> - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> +
> + /* Non-VHE: Disable and flush SPE data generation
> + * VHE: The vcpu can run. but it can't hide. */
Not the standard comment format.
s/./,/
I'm glad you kept the funny comment :-)
> + if (!has_vhe())
> + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> }
>
> void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> {
> - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> + if (!has_vhe())
> + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> +
> __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
>
> - if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> - vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
Guess I should have read ahead before commenting on this in the last
patch :-)
> }
>
> u32 __hyp_text __kvm_get_mdcr_el2(void)
> --
> 2.9.0
>
Do we still need to pass vcpu->arch.host_debug_state.pmscr_el1 as a
parameter to __debug_save_spe_nvhe and __debug_restore_spe? Or can
we just pass the vcpu and remove the "if (!pmscr_el1) return" in
__debug_restore_spe? Should __debug_restore_spe be renamed to have
a _nvhe for consistency?
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
2017-11-07 14:22 ` Andrew Jones
@ 2017-12-01 17:51 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 17:51 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 03:22:29PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:14PM +0200, Christoffer Dall wrote:
> > The debug save/restore functions can be improved by using the has_vhe()
> > static key instead of the instruction alternative. Using the static key
> > uses the same paradigm as we're going to use elsewhere, it makes the
> > code more readable, and it generates slightly better code (no
> > stack setups and function calls unless necessary).
> >
> > We also use a static key on the restore path, because it will be
> > marginally faster than loading a value from memory.
> >
> > Finally, we don't have to conditionally clear the debug dirty flag if
> > it's set, we can just clear it.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> > 1 file changed, 9 insertions(+), 13 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > index 0fc0758..a2291b6 100644
> > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -75,11 +75,6 @@
> >
> > #define psb_csync() asm volatile("hint #17")
> >
> > -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> > -{
> > - /* The vcpu can run. but it can't hide. */
> > -}
> > -
> > static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > {
> > u64 reg;
> > @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > dsb(nsh);
> > }
> >
> > -static hyp_alternate_select(__debug_save_spe,
> > - __debug_save_spe_nvhe, __debug_save_spe_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > {
> > if (!pmscr_el1)
> > @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > {
> > __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> > - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> > +
> > + /* Non-VHE: Disable and flush SPE data generation
> > + * VHE: The vcpu can run. but it can't hide. */
>
> Not the standard comment format.
> s/./,/
>
> I'm glad you kept the funny comment :-)
>
>
> > + if (!has_vhe())
> > + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > }
> >
> > void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > {
> > - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > + if (!has_vhe())
> > + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > +
> > __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> >
> > - if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > - vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
>
> Guess I should have read ahead before commenting on this in the last
> patch :-)
>
> > }
> >
> > u32 __hyp_text __kvm_get_mdcr_el2(void)
> > --
> > 2.9.0
> >
>
> Do we still need to pass vcpu->arch.host_debug_state.pmscr_el1 as a
> parameter to __debug_save_spe_nvhe and __debug_restore_spe? Or can
> we just pass the vcpu
We could change that, but that's unrelated to any optimization and I
believe the idea behind doing the code this way is that it can be reused
for another context later on if it should become relevant.
> and remove the "if (!pmscr_el1) return" in
> __debug_restore_spe?
That would change the logic (hint: the debug function takes a value, not
a pointer), so I don't think so.
> Should __debug_restore_spe be renamed to have
> a _nvhe for consistency?
>
Yes.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
@ 2017-12-01 17:51 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 17:51 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 03:22:29PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:14PM +0200, Christoffer Dall wrote:
> > The debug save/restore functions can be improved by using the has_vhe()
> > static key instead of the instruction alternative. Using the static key
> > uses the same paradigm as we're going to use elsewhere, it makes the
> > code more readable, and it generates slightly better code (no
> > stack setups and function calls unless necessary).
> >
> > We also use a static key on the restore path, because it will be
> > marginally faster than loading a value from memory.
> >
> > Finally, we don't have to conditionally clear the debug dirty flag if
> > it's set, we can just clear it.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> > 1 file changed, 9 insertions(+), 13 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > index 0fc0758..a2291b6 100644
> > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -75,11 +75,6 @@
> >
> > #define psb_csync() asm volatile("hint #17")
> >
> > -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> > -{
> > - /* The vcpu can run. but it can't hide. */
> > -}
> > -
> > static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > {
> > u64 reg;
> > @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > dsb(nsh);
> > }
> >
> > -static hyp_alternate_select(__debug_save_spe,
> > - __debug_save_spe_nvhe, __debug_save_spe_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > {
> > if (!pmscr_el1)
> > @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > {
> > __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> > - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> > +
> > + /* Non-VHE: Disable and flush SPE data generation
> > + * VHE: The vcpu can run. but it can't hide. */
>
> Not the standard comment format.
> s/./,/
>
> I'm glad you kept the funny comment :-)
>
>
> > + if (!has_vhe())
> > + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > }
> >
> > void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > {
> > - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > + if (!has_vhe())
> > + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > +
> > __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> >
> > - if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > - vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
>
> Guess I should have read ahead before commenting on this in the last
> patch :-)
>
> > }
> >
> > u32 __hyp_text __kvm_get_mdcr_el2(void)
> > --
> > 2.9.0
> >
>
> Do we still need to pass vcpu->arch.host_debug_state.pmscr_el1 as a
> parameter to __debug_save_spe_nvhe and __debug_restore_spe? Or can
> we just pass the vcpu
We could change that, but that's unrelated to any optimization and I
believe the idea behind doing the code this way is that it can be reused
for another context later on if it should become relevant.
> and remove the "if (!pmscr_el1) return" in
> __debug_restore_spe?
That would change the logic (hint: the debug function takes a value, not
a pointer), so I don't think so.
> Should __debug_restore_spe be renamed to have
> a _nvhe for consistency?
>
Yes.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-14 16:42 ` Julien Thierry
-1 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-14 16:42 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Hi Christopher,
On 12/10/17 11:41, Christoffer Dall wrote:
> The debug save/restore functions can be improved by using the has_vhe()
> static key instead of the instruction alternative. Using the static key
> uses the same paradigm as we're going to use elsewhere, it makes the
> code more readable, and it generates slightly better code (no
> stack setups and function calls unless necessary).
>
> We also use a static key on the restore path, because it will be
> marginally faster than loading a value from memory.
>
> Finally, we don't have to conditionally clear the debug dirty flag if
> it's set, we can just clear it.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> 1 file changed, 9 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 0fc0758..a2291b6 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -75,11 +75,6 @@
>
> #define psb_csync() asm volatile("hint #17")
>
> -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> -{
> - /* The vcpu can run. but it can't hide. */
> -}
> -
> static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> {
> u64 reg;
> @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> dsb(nsh);
> }
>
> -static hyp_alternate_select(__debug_save_spe,
> - __debug_save_spe_nvhe, __debug_save_spe_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> {
> if (!pmscr_el1)
> @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> {
> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
> - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> +
> + /* Non-VHE: Disable and flush SPE data generation
> + * VHE: The vcpu can run. but it can't hide. */
> + if (!has_vhe())
> + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> }
>
> void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> {
> - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> + if (!has_vhe())
> + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
For consistency, would it be worth naming that function
'__debug_restore_spe_nvhe' ?
Also, looking at __debug_save_spe_nvhe, I'm not sure how we guarantee
that we might not end up using stale data during the restore_spe
(though, if this is an issue, it existed before this change).
The save function might exit without setting a value to saved pmscr_el1.
Basically I'm wondering if the following scenario (in non VHE) is
possible and/or whether it is problematic:
- save spe
- restore spe
- host starts using spi -> !(PMBLIMITR_EL1 & PMBLIMITR_EL1_E)
- save spe -> returns early without setting pmscr_el1
- restore spe with old save instead of doing nothing
Cheers,
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
@ 2017-11-14 16:42 ` Julien Thierry
0 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-14 16:42 UTC (permalink / raw)
To: linux-arm-kernel
Hi Christopher,
On 12/10/17 11:41, Christoffer Dall wrote:
> The debug save/restore functions can be improved by using the has_vhe()
> static key instead of the instruction alternative. Using the static key
> uses the same paradigm as we're going to use elsewhere, it makes the
> code more readable, and it generates slightly better code (no
> stack setups and function calls unless necessary).
>
> We also use a static key on the restore path, because it will be
> marginally faster than loading a value from memory.
>
> Finally, we don't have to conditionally clear the debug dirty flag if
> it's set, we can just clear it.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> 1 file changed, 9 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 0fc0758..a2291b6 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -75,11 +75,6 @@
>
> #define psb_csync() asm volatile("hint #17")
>
> -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> -{
> - /* The vcpu can run. but it can't hide. */
> -}
> -
> static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> {
> u64 reg;
> @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> dsb(nsh);
> }
>
> -static hyp_alternate_select(__debug_save_spe,
> - __debug_save_spe_nvhe, __debug_save_spe_vhe,
> - ARM64_HAS_VIRT_HOST_EXTN);
> -
> static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> {
> if (!pmscr_el1)
> @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> {
> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> kern_hyp_va(vcpu->arch.host_cpu_context));
> - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> +
> + /* Non-VHE: Disable and flush SPE data generation
> + * VHE: The vcpu can run. but it can't hide. */
> + if (!has_vhe())
> + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> }
>
> void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> {
> - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> + if (!has_vhe())
> + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
For consistency, would it be worth naming that function
'__debug_restore_spe_nvhe' ?
Also, looking at __debug_save_spe_nvhe, I'm not sure how we guarantee
that we might not end up using stale data during the restore_spe
(though, if this is an issue, it existed before this change).
The save function might exit without setting a value to saved pmscr_el1.
Basically I'm wondering if the following scenario (in non VHE) is
possible and/or whether it is problematic:
- save spe
- restore spe
- host starts using spi -> !(PMBLIMITR_EL1 & PMBLIMITR_EL1_E)
- save spe -> returns early without setting pmscr_el1
- restore spe with old save instead of doing nothing
Cheers,
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
2017-11-14 16:42 ` Julien Thierry
@ 2017-12-01 15:19 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 15:19 UTC (permalink / raw)
To: Julien Thierry; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
Hi Julien,
On Tue, Nov 14, 2017 at 04:42:13PM +0000, Julien Thierry wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> >The debug save/restore functions can be improved by using the has_vhe()
> >static key instead of the instruction alternative. Using the static key
> >uses the same paradigm as we're going to use elsewhere, it makes the
> >code more readable, and it generates slightly better code (no
> >stack setups and function calls unless necessary).
> >
> >We also use a static key on the restore path, because it will be
> >marginally faster than loading a value from memory.
> >
> >Finally, we don't have to conditionally clear the debug dirty flag if
> >it's set, we can just clear it.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >---
> > arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> > 1 file changed, 9 insertions(+), 13 deletions(-)
> >
> >diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> >index 0fc0758..a2291b6 100644
> >--- a/arch/arm64/kvm/hyp/debug-sr.c
> >+++ b/arch/arm64/kvm/hyp/debug-sr.c
> >@@ -75,11 +75,6 @@
> > #define psb_csync() asm volatile("hint #17")
> >-static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> >-{
> >- /* The vcpu can run. but it can't hide. */
> >-}
> >-
> > static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > {
> > u64 reg;
> >@@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > dsb(nsh);
> > }
> >-static hyp_alternate_select(__debug_save_spe,
> >- __debug_save_spe_nvhe, __debug_save_spe_vhe,
> >- ARM64_HAS_VIRT_HOST_EXTN);
> >-
> > static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > {
> > if (!pmscr_el1)
> >@@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > {
> > __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> >- __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> >+
> >+ /* Non-VHE: Disable and flush SPE data generation
> >+ * VHE: The vcpu can run. but it can't hide. */
> >+ if (!has_vhe())
> >+ __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > }
> > void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > {
> >- __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> >+ if (!has_vhe())
> >+ __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>
> For consistency, would it be worth naming that function
> '__debug_restore_spe_nvhe' ?
Yes.
>
> Also, looking at __debug_save_spe_nvhe, I'm not sure how we guarantee that
> we might not end up using stale data during the restore_spe (though, if this
> is an issue, it existed before this change).
> The save function might exit without setting a value to saved pmscr_el1.
>
> Basically I'm wondering if the following scenario (in non VHE) is possible
> and/or whether it is problematic:
>
> - save spe
> - restore spe
> - host starts using spi -> !(PMBLIMITR_EL1 & PMBLIMITR_EL1_E)
spi ?
> - save spe -> returns early without setting pmscr_el1
> - restore spe with old save instead of doing nothing
>
I think I see what you mean. Basically you're asking if we need this:
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index 4112160..8ab3510 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -106,7 +106,7 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
{
- if (!pmscr_el1)
+ if (*pmscr_el1 != 0)
return;
/* The host page table is installed, but not yet synchronised */
@@ -114,6 +114,7 @@ static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
/* Re-enable data generation */
write_sysreg_s(pmscr_el1, PMSCR_EL1);
+ *pmscr_el1 = 0;
}
void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
I think we do, and I think this is a separate fix. Would you like to
write a patch and cc Will and Marc (original author and committer) to
fix this? Probably worth a cc stable as well.
Thanks,
-Christoffer
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
@ 2017-12-01 15:19 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 15:19 UTC (permalink / raw)
To: linux-arm-kernel
Hi Julien,
On Tue, Nov 14, 2017 at 04:42:13PM +0000, Julien Thierry wrote:
> On 12/10/17 11:41, Christoffer Dall wrote:
> >The debug save/restore functions can be improved by using the has_vhe()
> >static key instead of the instruction alternative. Using the static key
> >uses the same paradigm as we're going to use elsewhere, it makes the
> >code more readable, and it generates slightly better code (no
> >stack setups and function calls unless necessary).
> >
> >We also use a static key on the restore path, because it will be
> >marginally faster than loading a value from memory.
> >
> >Finally, we don't have to conditionally clear the debug dirty flag if
> >it's set, we can just clear it.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >---
> > arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
> > 1 file changed, 9 insertions(+), 13 deletions(-)
> >
> >diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> >index 0fc0758..a2291b6 100644
> >--- a/arch/arm64/kvm/hyp/debug-sr.c
> >+++ b/arch/arm64/kvm/hyp/debug-sr.c
> >@@ -75,11 +75,6 @@
> > #define psb_csync() asm volatile("hint #17")
> >-static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
> >-{
> >- /* The vcpu can run. but it can't hide. */
> >-}
> >-
> > static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > {
> > u64 reg;
> >@@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
> > dsb(nsh);
> > }
> >-static hyp_alternate_select(__debug_save_spe,
> >- __debug_save_spe_nvhe, __debug_save_spe_vhe,
> >- ARM64_HAS_VIRT_HOST_EXTN);
> >-
> > static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > {
> > if (!pmscr_el1)
> >@@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > {
> > __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > kern_hyp_va(vcpu->arch.host_cpu_context));
> >- __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
> >+
> >+ /* Non-VHE: Disable and flush SPE data generation
> >+ * VHE: The vcpu can run. but it can't hide. */
> >+ if (!has_vhe())
> >+ __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > }
> > void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > {
> >- __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> >+ if (!has_vhe())
> >+ __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>
> For consistency, would it be worth naming that function
> '__debug_restore_spe_nvhe' ?
Yes.
>
> Also, looking at __debug_save_spe_nvhe, I'm not sure how we guarantee that
> we might not end up using stale data during the restore_spe (though, if this
> is an issue, it existed before this change).
> The save function might exit without setting a value to saved pmscr_el1.
>
> Basically I'm wondering if the following scenario (in non VHE) is possible
> and/or whether it is problematic:
>
> - save spe
> - restore spe
> - host starts using spi -> !(PMBLIMITR_EL1 & PMBLIMITR_EL1_E)
spi ?
> - save spe -> returns early without setting pmscr_el1
> - restore spe with old save instead of doing nothing
>
I think I see what you mean. Basically you're asking if we need this:
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index 4112160..8ab3510 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -106,7 +106,7 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
{
- if (!pmscr_el1)
+ if (*pmscr_el1 != 0)
return;
/* The host page table is installed, but not yet synchronised */
@@ -114,6 +114,7 @@ static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
/* Re-enable data generation */
write_sysreg_s(pmscr_el1, PMSCR_EL1);
+ *pmscr_el1 = 0;
}
void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
I think we do, and I think this is a separate fix. Would you like to
write a patch and cc Will and Marc (original author and committer) to
fix this? Probably worth a cc stable as well.
Thanks,
-Christoffer
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
2017-12-01 15:19 ` Christoffer Dall
@ 2017-12-06 15:38 ` Julien Thierry
-1 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-12-06 15:38 UTC (permalink / raw)
To: Christoffer Dall
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On 01/12/17 15:19, Christoffer Dall wrote:
> Hi Julien,
>
> On Tue, Nov 14, 2017 at 04:42:13PM +0000, Julien Thierry wrote:
>> On 12/10/17 11:41, Christoffer Dall wrote:
>>> The debug save/restore functions can be improved by using the has_vhe()
>>> static key instead of the instruction alternative. Using the static key
>>> uses the same paradigm as we're going to use elsewhere, it makes the
>>> code more readable, and it generates slightly better code (no
>>> stack setups and function calls unless necessary).
>>>
>>> We also use a static key on the restore path, because it will be
>>> marginally faster than loading a value from memory.
>>>
>>> Finally, we don't have to conditionally clear the debug dirty flag if
>>> it's set, we can just clear it.
>>>
>>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> ---
>>> arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
>>> 1 file changed, 9 insertions(+), 13 deletions(-)
>>>
>>> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
>>> index 0fc0758..a2291b6 100644
>>> --- a/arch/arm64/kvm/hyp/debug-sr.c
>>> +++ b/arch/arm64/kvm/hyp/debug-sr.c
>>> @@ -75,11 +75,6 @@
>>> #define psb_csync() asm volatile("hint #17")
>>> -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
>>> -{
>>> - /* The vcpu can run. but it can't hide. */
>>> -}
>>> -
>>> static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
>>> {
>>> u64 reg;
>>> @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
>>> dsb(nsh);
>>> }
>>> -static hyp_alternate_select(__debug_save_spe,
>>> - __debug_save_spe_nvhe, __debug_save_spe_vhe,
>>> - ARM64_HAS_VIRT_HOST_EXTN);
>>> -
>>> static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
>>> {
>>> if (!pmscr_el1)
>>> @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>>> {
>>> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
>>> kern_hyp_va(vcpu->arch.host_cpu_context));
>>> - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
>>> +
>>> + /* Non-VHE: Disable and flush SPE data generation
>>> + * VHE: The vcpu can run. but it can't hide. */
>>> + if (!has_vhe())
>>> + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
>>> }
>>> void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
>>> {
>>> - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>>> + if (!has_vhe())
>>> + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>>
>> For consistency, would it be worth naming that function
>> '__debug_restore_spe_nvhe' ?
>
> Yes.
>
>>
>> Also, looking at __debug_save_spe_nvhe, I'm not sure how we guarantee that
>> we might not end up using stale data during the restore_spe (though, if this
>> is an issue, it existed before this change).
>> The save function might exit without setting a value to saved pmscr_el1.
>>
>> Basically I'm wondering if the following scenario (in non VHE) is possible
>> and/or whether it is problematic:
>>
>> - save spe
>> - restore spe
>> - host starts using spi -> !(PMBLIMITR_EL1 & PMBLIMITR_EL1_E)
>
> spi ?
spe*
>
>> - save spe -> returns early without setting pmscr_el1
>> - restore spe with old save instead of doing nothing
>>
>
> I think I see what you mean. Basically you're asking if we need this:
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 4112160..8ab3510 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -106,7 +106,7 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
>
> static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
> {
> - if (!pmscr_el1)
> + if (*pmscr_el1 != 0)
> return;
>
> /* The host page table is installed, but not yet synchronised */
> @@ -114,6 +114,7 @@ static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
>
> /* Re-enable data generation */
> write_sysreg_s(pmscr_el1, PMSCR_EL1);
> + *pmscr_el1 = 0;
> }
>
> void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
>
> I think we do, and I think this is a separate fix. Would you like to
> write a patch and cc Will and Marc (original author and committer) to
> fix this? Probably worth a cc stable as well.
>
Yes, this is what I was referring to. I agree it is a separate fix. I'll
make a patch for this.
Thanks,
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 10/37] KVM: arm64: Slightly improve debug save/restore functions
@ 2017-12-06 15:38 ` Julien Thierry
0 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-12-06 15:38 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/17 15:19, Christoffer Dall wrote:
> Hi Julien,
>
> On Tue, Nov 14, 2017 at 04:42:13PM +0000, Julien Thierry wrote:
>> On 12/10/17 11:41, Christoffer Dall wrote:
>>> The debug save/restore functions can be improved by using the has_vhe()
>>> static key instead of the instruction alternative. Using the static key
>>> uses the same paradigm as we're going to use elsewhere, it makes the
>>> code more readable, and it generates slightly better code (no
>>> stack setups and function calls unless necessary).
>>>
>>> We also use a static key on the restore path, because it will be
>>> marginally faster than loading a value from memory.
>>>
>>> Finally, we don't have to conditionally clear the debug dirty flag if
>>> it's set, we can just clear it.
>>>
>>> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>>> ---
>>> arch/arm64/kvm/hyp/debug-sr.c | 22 +++++++++-------------
>>> 1 file changed, 9 insertions(+), 13 deletions(-)
>>>
>>> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
>>> index 0fc0758..a2291b6 100644
>>> --- a/arch/arm64/kvm/hyp/debug-sr.c
>>> +++ b/arch/arm64/kvm/hyp/debug-sr.c
>>> @@ -75,11 +75,6 @@
>>> #define psb_csync() asm volatile("hint #17")
>>> -static void __hyp_text __debug_save_spe_vhe(u64 *pmscr_el1)
>>> -{
>>> - /* The vcpu can run. but it can't hide. */
>>> -}
>>> -
>>> static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
>>> {
>>> u64 reg;
>>> @@ -109,10 +104,6 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
>>> dsb(nsh);
>>> }
>>> -static hyp_alternate_select(__debug_save_spe,
>>> - __debug_save_spe_nvhe, __debug_save_spe_vhe,
>>> - ARM64_HAS_VIRT_HOST_EXTN);
>>> -
>>> static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
>>> {
>>> if (!pmscr_el1)
>>> @@ -174,17 +165,22 @@ void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>>> {
>>> __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
>>> kern_hyp_va(vcpu->arch.host_cpu_context));
>>> - __debug_save_spe()(&vcpu->arch.host_debug_state.pmscr_el1);
>>> +
>>> + /* Non-VHE: Disable and flush SPE data generation
>>> + * VHE: The vcpu can run. but it can't hide. */
>>> + if (!has_vhe())
>>> + __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
>>> }
>>> void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
>>> {
>>> - __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>>> + if (!has_vhe())
>>> + __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>>
>> For consistency, would it be worth naming that function
>> '__debug_restore_spe_nvhe' ?
>
> Yes.
>
>>
>> Also, looking at __debug_save_spe_nvhe, I'm not sure how we guarantee that
>> we might not end up using stale data during the restore_spe (though, if this
>> is an issue, it existed before this change).
>> The save function might exit without setting a value to saved pmscr_el1.
>>
>> Basically I'm wondering if the following scenario (in non VHE) is possible
>> and/or whether it is problematic:
>>
>> - save spe
>> - restore spe
>> - host starts using spi -> !(PMBLIMITR_EL1 & PMBLIMITR_EL1_E)
>
> spi ?
spe*
>
>> - save spe -> returns early without setting pmscr_el1
>> - restore spe with old save instead of doing nothing
>>
>
> I think I see what you mean. Basically you're asking if we need this:
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 4112160..8ab3510 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -106,7 +106,7 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
>
> static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
> {
> - if (!pmscr_el1)
> + if (*pmscr_el1 != 0)
> return;
>
> /* The host page table is installed, but not yet synchronised */
> @@ -114,6 +114,7 @@ static void __hyp_text __debug_restore_spe_nvhe(u64 &pmscr_el1)
>
> /* Re-enable data generation */
> write_sysreg_s(pmscr_el1, PMSCR_EL1);
> + *pmscr_el1 = 0;
> }
>
> void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
>
> I think we do, and I think this is a separate fix. Would you like to
> write a patch and cc Will and Marc (original author and committer) to
> fix this? Probably worth a cc stable as well.
>
Yes, this is what I was referring to. I agree it is a separate fix. I'll
make a patch for this.
Thanks,
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Instead of having multiple calls from the world switch path to the debug
logic, each figuring out if the dirty bit is set and if we should
save/restore the debug registes, let's just provide two hooks to the
debug save/restore functionality, one for switching to the guest
context, and one for switching to the host context, and we get the
benefit of only having to evaluate the dirty flag once on each path,
plus we give the compiler some more room to inline some of this
functionality.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
arch/arm64/kvm/hyp/switch.c | 6 ++---
3 files changed, 42 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 08d3bb6..a0e5a70 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
void __sysreg32_save_state(struct kvm_vcpu *vcpu);
void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
-void __debug_save_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt);
-void __debug_restore_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt);
-void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
-void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
+void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
+void __debug_switch_to_host(struct kvm_vcpu *vcpu);
void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index a2291b6..b4cd8e0 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
write_sysreg_s(pmscr_el1, PMSCR_EL1);
}
-void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt)
+static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt)
{
u64 aa64dfr0;
int brps, wrps;
- if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
- return;
-
aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
brps = (aa64dfr0 >> 12) & 0xf;
wrps = (aa64dfr0 >> 20) & 0xf;
@@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
}
-void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt)
+static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt)
{
u64 aa64dfr0;
int brps, wrps;
- if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
- return;
-
aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
brps = (aa64dfr0 >> 12) & 0xf;
@@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
}
-void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
+void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
{
- __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
- kern_hyp_va(vcpu->arch.host_cpu_context));
+ struct kvm_cpu_context *__host_ctxt;
+ struct kvm_cpu_context *__guest_ctxt;
+ struct kvm_guest_debug_arch *__host_dbg;
+ struct kvm_guest_debug_arch *__guest_dbg;
/* Non-VHE: Disable and flush SPE data generation
* VHE: The vcpu can run. but it can't hide. */
if (!has_vhe())
__debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
+
+ if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
+ return;
+
+ __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ __guest_ctxt = &vcpu->arch.ctxt;
+ __host_dbg = &vcpu->arch.host_debug_state.regs;
+ __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
+
+ __debug_save_state(vcpu, __host_dbg, __host_ctxt);
+ __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
}
-void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
+void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
{
+ struct kvm_cpu_context *__host_ctxt;
+ struct kvm_cpu_context *__guest_ctxt;
+ struct kvm_guest_debug_arch *__host_dbg;
+ struct kvm_guest_debug_arch *__guest_dbg;
+
if (!has_vhe())
__debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
- __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
- kern_hyp_va(vcpu->arch.host_cpu_context));
+ if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
+ return;
+
+ __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ __guest_ctxt = &vcpu->arch.ctxt;
+ __host_dbg = &vcpu->arch.host_debug_state.regs;
+ __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
+
+ __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
+ __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
}
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index ef05c59..e270cba 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
guest_ctxt = &vcpu->arch.ctxt;
__sysreg_save_host_state(host_ctxt);
- __debug_cond_save_host_state(vcpu);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
*/
__sysreg32_restore_state(vcpu);
__sysreg_restore_guest_state(guest_ctxt);
- __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
+ __debug_switch_to_guest(vcpu);
/* Jump in the fire! */
again:
@@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_restore_host_state(host_ctxt);
- __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
* system may enable SPE here and make use of the TTBRs.
*/
- __debug_cond_restore_host_state(vcpu);
+ __debug_switch_to_host(vcpu);
return exit_code;
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Instead of having multiple calls from the world switch path to the debug
logic, each figuring out if the dirty bit is set and if we should
save/restore the debug registes, let's just provide two hooks to the
debug save/restore functionality, one for switching to the guest
context, and one for switching to the host context, and we get the
benefit of only having to evaluate the dirty flag once on each path,
plus we give the compiler some more room to inline some of this
functionality.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
arch/arm64/kvm/hyp/switch.c | 6 ++---
3 files changed, 42 insertions(+), 30 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 08d3bb6..a0e5a70 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
void __sysreg32_save_state(struct kvm_vcpu *vcpu);
void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
-void __debug_save_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt);
-void __debug_restore_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt);
-void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
-void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
+void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
+void __debug_switch_to_host(struct kvm_vcpu *vcpu);
void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index a2291b6..b4cd8e0 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
write_sysreg_s(pmscr_el1, PMSCR_EL1);
}
-void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt)
+static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt)
{
u64 aa64dfr0;
int brps, wrps;
- if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
- return;
-
aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
brps = (aa64dfr0 >> 12) & 0xf;
wrps = (aa64dfr0 >> 20) & 0xf;
@@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
}
-void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
- struct kvm_guest_debug_arch *dbg,
- struct kvm_cpu_context *ctxt)
+static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt)
{
u64 aa64dfr0;
int brps, wrps;
- if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
- return;
-
aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
brps = (aa64dfr0 >> 12) & 0xf;
@@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
}
-void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
+void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
{
- __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
- kern_hyp_va(vcpu->arch.host_cpu_context));
+ struct kvm_cpu_context *__host_ctxt;
+ struct kvm_cpu_context *__guest_ctxt;
+ struct kvm_guest_debug_arch *__host_dbg;
+ struct kvm_guest_debug_arch *__guest_dbg;
/* Non-VHE: Disable and flush SPE data generation
* VHE: The vcpu can run. but it can't hide. */
if (!has_vhe())
__debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
+
+ if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
+ return;
+
+ __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ __guest_ctxt = &vcpu->arch.ctxt;
+ __host_dbg = &vcpu->arch.host_debug_state.regs;
+ __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
+
+ __debug_save_state(vcpu, __host_dbg, __host_ctxt);
+ __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
}
-void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
+void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
{
+ struct kvm_cpu_context *__host_ctxt;
+ struct kvm_cpu_context *__guest_ctxt;
+ struct kvm_guest_debug_arch *__host_dbg;
+ struct kvm_guest_debug_arch *__guest_dbg;
+
if (!has_vhe())
__debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
- __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
- kern_hyp_va(vcpu->arch.host_cpu_context));
+ if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
+ return;
+
+ __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ __guest_ctxt = &vcpu->arch.ctxt;
+ __host_dbg = &vcpu->arch.host_debug_state.regs;
+ __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
+
+ __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
+ __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
}
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index ef05c59..e270cba 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
guest_ctxt = &vcpu->arch.ctxt;
__sysreg_save_host_state(host_ctxt);
- __debug_cond_save_host_state(vcpu);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
*/
__sysreg32_restore_state(vcpu);
__sysreg_restore_guest_state(guest_ctxt);
- __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
+ __debug_switch_to_guest(vcpu);
/* Jump in the fire! */
again:
@@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_restore_host_state(host_ctxt);
- __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
* system may enable SPE here and make use of the TTBRs.
*/
- __debug_cond_restore_host_state(vcpu);
+ __debug_switch_to_host(vcpu);
return exit_code;
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 14:48 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 14:48 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> Instead of having multiple calls from the world switch path to the debug
> logic, each figuring out if the dirty bit is set and if we should
> save/restore the debug registes, let's just provide two hooks to the
> debug save/restore functionality, one for switching to the guest
> context, and one for switching to the host context, and we get the
> benefit of only having to evaluate the dirty flag once on each path,
> plus we give the compiler some more room to inline some of this
> functionality.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> arch/arm64/kvm/hyp/switch.c | 6 ++---
> 3 files changed, 42 insertions(+), 30 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index 08d3bb6..a0e5a70 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> -void __debug_save_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt);
> -void __debug_restore_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt);
> -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
>
> void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index a2291b6..b4cd8e0 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> write_sysreg_s(pmscr_el1, PMSCR_EL1);
> }
>
> -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt)
> +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> {
> u64 aa64dfr0;
> int brps, wrps;
>
> - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> - return;
> -
> aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> brps = (aa64dfr0 >> 12) & 0xf;
> wrps = (aa64dfr0 >> 20) & 0xf;
> @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> }
>
> -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt)
> +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> {
> u64 aa64dfr0;
> int brps, wrps;
>
> - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> - return;
> -
> aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>
> brps = (aa64dfr0 >> 12) & 0xf;
> @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> }
>
> -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> {
> - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> - kern_hyp_va(vcpu->arch.host_cpu_context));
> + struct kvm_cpu_context *__host_ctxt;
> + struct kvm_cpu_context *__guest_ctxt;
> + struct kvm_guest_debug_arch *__host_dbg;
> + struct kvm_guest_debug_arch *__guest_dbg;
I caught in your reply to Marc that the __ prefix here is for hyp mode
accessible code and data, but do we also need to use it for stack data?
No big deal, but it's not very pretty.
>
> /* Non-VHE: Disable and flush SPE data generation
> * VHE: The vcpu can run. but it can't hide. */
> if (!has_vhe())
> __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> +
> + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> + return;
> +
> + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + __guest_ctxt = &vcpu->arch.ctxt;
> + __host_dbg = &vcpu->arch.host_debug_state.regs;
> + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> +
> + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> }
>
> -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *__host_ctxt;
> + struct kvm_cpu_context *__guest_ctxt;
> + struct kvm_guest_debug_arch *__host_dbg;
> + struct kvm_guest_debug_arch *__guest_dbg;
> +
> if (!has_vhe())
> __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>
> - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> - kern_hyp_va(vcpu->arch.host_cpu_context));
> + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> + return;
> +
> + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + __guest_ctxt = &vcpu->arch.ctxt;
> + __host_dbg = &vcpu->arch.host_debug_state.regs;
> + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> +
> + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
>
> vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> }
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index ef05c59..e270cba 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> guest_ctxt = &vcpu->arch.ctxt;
>
> __sysreg_save_host_state(host_ctxt);
> - __debug_cond_save_host_state(vcpu);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> */
> __sysreg32_restore_state(vcpu);
> __sysreg_restore_guest_state(guest_ctxt);
> - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> + __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> again:
> @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_host_state(host_ctxt);
>
> - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> * system may enable SPE here and make use of the TTBRs.
> */
> - __debug_cond_restore_host_state(vcpu);
> + __debug_switch_to_host(vcpu);
>
> return exit_code;
> }
> --
> 2.9.0
>
This looks like a nice cleanup, but can you please add a note to the
commit message about why we don't need to use the
save-host-state
activate-traps-and-vm
restore-guest-state
and the reverse, patterns for the debug registers? As of this patch, that
pattern still exists for the sysregs.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
@ 2017-11-07 14:48 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 14:48 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> Instead of having multiple calls from the world switch path to the debug
> logic, each figuring out if the dirty bit is set and if we should
> save/restore the debug registes, let's just provide two hooks to the
> debug save/restore functionality, one for switching to the guest
> context, and one for switching to the host context, and we get the
> benefit of only having to evaluate the dirty flag once on each path,
> plus we give the compiler some more room to inline some of this
> functionality.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> arch/arm64/kvm/hyp/switch.c | 6 ++---
> 3 files changed, 42 insertions(+), 30 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index 08d3bb6..a0e5a70 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> -void __debug_save_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt);
> -void __debug_restore_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt);
> -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
>
> void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index a2291b6..b4cd8e0 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> write_sysreg_s(pmscr_el1, PMSCR_EL1);
> }
>
> -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt)
> +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> {
> u64 aa64dfr0;
> int brps, wrps;
>
> - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> - return;
> -
> aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> brps = (aa64dfr0 >> 12) & 0xf;
> wrps = (aa64dfr0 >> 20) & 0xf;
> @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> }
>
> -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> - struct kvm_guest_debug_arch *dbg,
> - struct kvm_cpu_context *ctxt)
> +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> {
> u64 aa64dfr0;
> int brps, wrps;
>
> - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> - return;
> -
> aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>
> brps = (aa64dfr0 >> 12) & 0xf;
> @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> }
>
> -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> {
> - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> - kern_hyp_va(vcpu->arch.host_cpu_context));
> + struct kvm_cpu_context *__host_ctxt;
> + struct kvm_cpu_context *__guest_ctxt;
> + struct kvm_guest_debug_arch *__host_dbg;
> + struct kvm_guest_debug_arch *__guest_dbg;
I caught in your reply to Marc that the __ prefix here is for hyp mode
accessible code and data, but do we also need to use it for stack data?
No big deal, but it's not very pretty.
>
> /* Non-VHE: Disable and flush SPE data generation
> * VHE: The vcpu can run. but it can't hide. */
> if (!has_vhe())
> __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> +
> + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> + return;
> +
> + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + __guest_ctxt = &vcpu->arch.ctxt;
> + __host_dbg = &vcpu->arch.host_debug_state.regs;
> + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> +
> + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> }
>
> -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> {
> + struct kvm_cpu_context *__host_ctxt;
> + struct kvm_cpu_context *__guest_ctxt;
> + struct kvm_guest_debug_arch *__host_dbg;
> + struct kvm_guest_debug_arch *__guest_dbg;
> +
> if (!has_vhe())
> __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
>
> - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> - kern_hyp_va(vcpu->arch.host_cpu_context));
> + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> + return;
> +
> + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + __guest_ctxt = &vcpu->arch.ctxt;
> + __host_dbg = &vcpu->arch.host_debug_state.regs;
> + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> +
> + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
>
> vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> }
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index ef05c59..e270cba 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> guest_ctxt = &vcpu->arch.ctxt;
>
> __sysreg_save_host_state(host_ctxt);
> - __debug_cond_save_host_state(vcpu);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> */
> __sysreg32_restore_state(vcpu);
> __sysreg_restore_guest_state(guest_ctxt);
> - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> + __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> again:
> @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_host_state(host_ctxt);
>
> - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> * system may enable SPE here and make use of the TTBRs.
> */
> - __debug_cond_restore_host_state(vcpu);
> + __debug_switch_to_host(vcpu);
>
> return exit_code;
> }
> --
> 2.9.0
>
This looks like a nice cleanup, but can you please add a note to the
commit message about why we don't need to use the
save-host-state
activate-traps-and-vm
restore-guest-state
and the reverse, patterns for the debug registers? As of this patch, that
pattern still exists for the sysregs.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
2017-11-07 14:48 ` Andrew Jones
@ 2017-12-01 17:52 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 17:52 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 03:48:57PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> > Instead of having multiple calls from the world switch path to the debug
> > logic, each figuring out if the dirty bit is set and if we should
> > save/restore the debug registes, let's just provide two hooks to the
> > debug save/restore functionality, one for switching to the guest
> > context, and one for switching to the host context, and we get the
> > benefit of only having to evaluate the dirty flag once on each path,
> > plus we give the compiler some more room to inline some of this
> > functionality.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> > arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> > arch/arm64/kvm/hyp/switch.c | 6 ++---
> > 3 files changed, 42 insertions(+), 30 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 08d3bb6..a0e5a70 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> >
> > -void __debug_save_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt);
> > -void __debug_restore_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt);
> > -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> > +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
> >
> > void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> > void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > index a2291b6..b4cd8e0 100644
> > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > write_sysreg_s(pmscr_el1, PMSCR_EL1);
> > }
> >
> > -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt)
> > {
> > u64 aa64dfr0;
> > int brps, wrps;
> >
> > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > - return;
> > -
> > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > brps = (aa64dfr0 >> 12) & 0xf;
> > wrps = (aa64dfr0 >> 20) & 0xf;
> > @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > }
> >
> > -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt)
> > {
> > u64 aa64dfr0;
> > int brps, wrps;
> >
> > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > - return;
> > -
> > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> >
> > brps = (aa64dfr0 >> 12) & 0xf;
> > @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > }
> >
> > -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> > {
> > - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > + struct kvm_cpu_context *__host_ctxt;
> > + struct kvm_cpu_context *__guest_ctxt;
> > + struct kvm_guest_debug_arch *__host_dbg;
> > + struct kvm_guest_debug_arch *__guest_dbg;
>
> I caught in your reply to Marc that the __ prefix here is for hyp mode
> accessible code and data, but do we also need to use it for stack data?
> No big deal, but it's not very pretty.
>
sure.
> >
> > /* Non-VHE: Disable and flush SPE data generation
> > * VHE: The vcpu can run. but it can't hide. */
> > if (!has_vhe())
> > __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > +
> > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > + return;
> > +
> > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > + __guest_ctxt = &vcpu->arch.ctxt;
> > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > +
> > + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> > + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> > }
> >
> > -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *__host_ctxt;
> > + struct kvm_cpu_context *__guest_ctxt;
> > + struct kvm_guest_debug_arch *__host_dbg;
> > + struct kvm_guest_debug_arch *__guest_dbg;
> > +
> > if (!has_vhe())
> > __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> >
> > - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > + return;
> > +
> > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > + __guest_ctxt = &vcpu->arch.ctxt;
> > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > +
> > + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> > + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
> >
> > vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > }
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index ef05c59..e270cba 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > guest_ctxt = &vcpu->arch.ctxt;
> >
> > __sysreg_save_host_state(host_ctxt);
> > - __debug_cond_save_host_state(vcpu);
> >
> > __activate_traps(vcpu);
> > __activate_vm(vcpu);
> > @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > */
> > __sysreg32_restore_state(vcpu);
> > __sysreg_restore_guest_state(guest_ctxt);
> > - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > + __debug_switch_to_guest(vcpu);
> >
> > /* Jump in the fire! */
> > again:
> > @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > * system may enable SPE here and make use of the TTBRs.
> > */
> > - __debug_cond_restore_host_state(vcpu);
> > + __debug_switch_to_host(vcpu);
> >
> > return exit_code;
> > }
> > --
> > 2.9.0
> >
>
> This looks like a nice cleanup, but can you please add a note to the
> commit message about why we don't need to use the
>
> save-host-state
> activate-traps-and-vm
> restore-guest-state
>
> and the reverse, patterns for the debug registers?
I think the current commit message motivates the change fine.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
@ 2017-12-01 17:52 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 17:52 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 03:48:57PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> > Instead of having multiple calls from the world switch path to the debug
> > logic, each figuring out if the dirty bit is set and if we should
> > save/restore the debug registes, let's just provide two hooks to the
> > debug save/restore functionality, one for switching to the guest
> > context, and one for switching to the host context, and we get the
> > benefit of only having to evaluate the dirty flag once on each path,
> > plus we give the compiler some more room to inline some of this
> > functionality.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> > arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> > arch/arm64/kvm/hyp/switch.c | 6 ++---
> > 3 files changed, 42 insertions(+), 30 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 08d3bb6..a0e5a70 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> >
> > -void __debug_save_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt);
> > -void __debug_restore_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt);
> > -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> > +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
> >
> > void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> > void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > index a2291b6..b4cd8e0 100644
> > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > write_sysreg_s(pmscr_el1, PMSCR_EL1);
> > }
> >
> > -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt)
> > {
> > u64 aa64dfr0;
> > int brps, wrps;
> >
> > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > - return;
> > -
> > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > brps = (aa64dfr0 >> 12) & 0xf;
> > wrps = (aa64dfr0 >> 20) & 0xf;
> > @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > }
> >
> > -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > - struct kvm_guest_debug_arch *dbg,
> > - struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt)
> > {
> > u64 aa64dfr0;
> > int brps, wrps;
> >
> > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > - return;
> > -
> > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> >
> > brps = (aa64dfr0 >> 12) & 0xf;
> > @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > }
> >
> > -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> > {
> > - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > + struct kvm_cpu_context *__host_ctxt;
> > + struct kvm_cpu_context *__guest_ctxt;
> > + struct kvm_guest_debug_arch *__host_dbg;
> > + struct kvm_guest_debug_arch *__guest_dbg;
>
> I caught in your reply to Marc that the __ prefix here is for hyp mode
> accessible code and data, but do we also need to use it for stack data?
> No big deal, but it's not very pretty.
>
sure.
> >
> > /* Non-VHE: Disable and flush SPE data generation
> > * VHE: The vcpu can run. but it can't hide. */
> > if (!has_vhe())
> > __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > +
> > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > + return;
> > +
> > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > + __guest_ctxt = &vcpu->arch.ctxt;
> > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > +
> > + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> > + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> > }
> >
> > -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *__host_ctxt;
> > + struct kvm_cpu_context *__guest_ctxt;
> > + struct kvm_guest_debug_arch *__host_dbg;
> > + struct kvm_guest_debug_arch *__guest_dbg;
> > +
> > if (!has_vhe())
> > __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> >
> > - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > + return;
> > +
> > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > + __guest_ctxt = &vcpu->arch.ctxt;
> > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > +
> > + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> > + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
> >
> > vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > }
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index ef05c59..e270cba 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > guest_ctxt = &vcpu->arch.ctxt;
> >
> > __sysreg_save_host_state(host_ctxt);
> > - __debug_cond_save_host_state(vcpu);
> >
> > __activate_traps(vcpu);
> > __activate_vm(vcpu);
> > @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > */
> > __sysreg32_restore_state(vcpu);
> > __sysreg_restore_guest_state(guest_ctxt);
> > - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > + __debug_switch_to_guest(vcpu);
> >
> > /* Jump in the fire! */
> > again:
> > @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > * system may enable SPE here and make use of the TTBRs.
> > */
> > - __debug_cond_restore_host_state(vcpu);
> > + __debug_switch_to_host(vcpu);
> >
> > return exit_code;
> > }
> > --
> > 2.9.0
> >
>
> This looks like a nice cleanup, but can you please add a note to the
> commit message about why we don't need to use the
>
> save-host-state
> activate-traps-and-vm
> restore-guest-state
>
> and the reverse, patterns for the debug registers?
I think the current commit message motivates the change fine.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
2017-12-01 17:52 ` Christoffer Dall
@ 2017-12-03 13:49 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-12-03 13:49 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Fri, Dec 01, 2017 at 06:52:03PM +0100, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 03:48:57PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> > > Instead of having multiple calls from the world switch path to the debug
> > > logic, each figuring out if the dirty bit is set and if we should
> > > save/restore the debug registes, let's just provide two hooks to the
> > > debug save/restore functionality, one for switching to the guest
> > > context, and one for switching to the host context, and we get the
> > > benefit of only having to evaluate the dirty flag once on each path,
> > > plus we give the compiler some more room to inline some of this
> > > functionality.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> > > arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> > > arch/arm64/kvm/hyp/switch.c | 6 ++---
> > > 3 files changed, 42 insertions(+), 30 deletions(-)
> > >
> > > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > > index 08d3bb6..a0e5a70 100644
> > > --- a/arch/arm64/include/asm/kvm_hyp.h
> > > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > > @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> > > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> > >
> > > -void __debug_save_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt);
> > > -void __debug_restore_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt);
> > > -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > > -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > > +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> > > +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
> > >
> > > void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> > > void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> > > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > > index a2291b6..b4cd8e0 100644
> > > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > > @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > > write_sysreg_s(pmscr_el1, PMSCR_EL1);
> > > }
> > >
> > > -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt)
> > > +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > + struct kvm_guest_debug_arch *dbg,
> > > + struct kvm_cpu_context *ctxt)
> > > {
> > > u64 aa64dfr0;
> > > int brps, wrps;
> > >
> > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > - return;
> > > -
> > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > > brps = (aa64dfr0 >> 12) & 0xf;
> > > wrps = (aa64dfr0 >> 20) & 0xf;
> > > @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > > }
> > >
> > > -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt)
> > > +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > + struct kvm_guest_debug_arch *dbg,
> > > + struct kvm_cpu_context *ctxt)
> > > {
> > > u64 aa64dfr0;
> > > int brps, wrps;
> > >
> > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > - return;
> > > -
> > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > >
> > > brps = (aa64dfr0 >> 12) & 0xf;
> > > @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > > }
> > >
> > > -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > > +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> > > {
> > > - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > + struct kvm_cpu_context *__host_ctxt;
> > > + struct kvm_cpu_context *__guest_ctxt;
> > > + struct kvm_guest_debug_arch *__host_dbg;
> > > + struct kvm_guest_debug_arch *__guest_dbg;
> >
> > I caught in your reply to Marc that the __ prefix here is for hyp mode
> > accessible code and data, but do we also need to use it for stack data?
> > No big deal, but it's not very pretty.
> >
>
> sure.
>
> > >
> > > /* Non-VHE: Disable and flush SPE data generation
> > > * VHE: The vcpu can run. but it can't hide. */
> > > if (!has_vhe())
> > > __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > > +
> > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > + return;
> > > +
> > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > +
> > > + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> > > + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> > > }
> > >
> > > -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > > +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> > > {
> > > + struct kvm_cpu_context *__host_ctxt;
> > > + struct kvm_cpu_context *__guest_ctxt;
> > > + struct kvm_guest_debug_arch *__host_dbg;
> > > + struct kvm_guest_debug_arch *__guest_dbg;
> > > +
> > > if (!has_vhe())
> > > __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > >
> > > - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > + return;
> > > +
> > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > +
> > > + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> > > + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
> > >
> > > vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > > }
> > > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > > index ef05c59..e270cba 100644
> > > --- a/arch/arm64/kvm/hyp/switch.c
> > > +++ b/arch/arm64/kvm/hyp/switch.c
> > > @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > guest_ctxt = &vcpu->arch.ctxt;
> > >
> > > __sysreg_save_host_state(host_ctxt);
> > > - __debug_cond_save_host_state(vcpu);
> > >
> > > __activate_traps(vcpu);
> > > __activate_vm(vcpu);
> > > @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > */
> > > __sysreg32_restore_state(vcpu);
> > > __sysreg_restore_guest_state(guest_ctxt);
> > > - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > + __debug_switch_to_guest(vcpu);
> > >
> > > /* Jump in the fire! */
> > > again:
> > > @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > >
> > > __sysreg_restore_host_state(host_ctxt);
> > >
> > > - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > /*
> > > * This must come after restoring the host sysregs, since a non-VHE
> > > * system may enable SPE here and make use of the TTBRs.
> > > */
> > > - __debug_cond_restore_host_state(vcpu);
> > > + __debug_switch_to_host(vcpu);
> > >
> > > return exit_code;
> > > }
> > > --
> > > 2.9.0
> > >
> >
> > This looks like a nice cleanup, but can you please add a note to the
> > commit message about why we don't need to use the
> >
> > save-host-state
> > activate-traps-and-vm
> > restore-guest-state
> >
> > and the reverse, patterns for the debug registers?
>
> I think the current commit message motivates the change fine.
>
The motivation is obvious, the justification for an additional change
is not. How does it justify changing the sequence
save-debug-host-state
activate-debug-traps /* and other stuff in between */
restore-debug-guest-state
to
activate-debug-traps /* and other stuff in between */
save-debug-host-state
restore-debug-guest-state
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
@ 2017-12-03 13:49 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-12-03 13:49 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Dec 01, 2017 at 06:52:03PM +0100, Christoffer Dall wrote:
> On Tue, Nov 07, 2017 at 03:48:57PM +0100, Andrew Jones wrote:
> > On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> > > Instead of having multiple calls from the world switch path to the debug
> > > logic, each figuring out if the dirty bit is set and if we should
> > > save/restore the debug registes, let's just provide two hooks to the
> > > debug save/restore functionality, one for switching to the guest
> > > context, and one for switching to the host context, and we get the
> > > benefit of only having to evaluate the dirty flag once on each path,
> > > plus we give the compiler some more room to inline some of this
> > > functionality.
> > >
> > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > ---
> > > arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> > > arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> > > arch/arm64/kvm/hyp/switch.c | 6 ++---
> > > 3 files changed, 42 insertions(+), 30 deletions(-)
> > >
> > > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > > index 08d3bb6..a0e5a70 100644
> > > --- a/arch/arm64/include/asm/kvm_hyp.h
> > > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > > @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> > > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> > >
> > > -void __debug_save_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt);
> > > -void __debug_restore_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt);
> > > -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > > -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > > +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> > > +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
> > >
> > > void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> > > void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> > > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > > index a2291b6..b4cd8e0 100644
> > > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > > @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > > write_sysreg_s(pmscr_el1, PMSCR_EL1);
> > > }
> > >
> > > -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt)
> > > +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > + struct kvm_guest_debug_arch *dbg,
> > > + struct kvm_cpu_context *ctxt)
> > > {
> > > u64 aa64dfr0;
> > > int brps, wrps;
> > >
> > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > - return;
> > > -
> > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > > brps = (aa64dfr0 >> 12) & 0xf;
> > > wrps = (aa64dfr0 >> 20) & 0xf;
> > > @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > > }
> > >
> > > -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > - struct kvm_guest_debug_arch *dbg,
> > > - struct kvm_cpu_context *ctxt)
> > > +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > + struct kvm_guest_debug_arch *dbg,
> > > + struct kvm_cpu_context *ctxt)
> > > {
> > > u64 aa64dfr0;
> > > int brps, wrps;
> > >
> > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > - return;
> > > -
> > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > >
> > > brps = (aa64dfr0 >> 12) & 0xf;
> > > @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > > }
> > >
> > > -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > > +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> > > {
> > > - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > + struct kvm_cpu_context *__host_ctxt;
> > > + struct kvm_cpu_context *__guest_ctxt;
> > > + struct kvm_guest_debug_arch *__host_dbg;
> > > + struct kvm_guest_debug_arch *__guest_dbg;
> >
> > I caught in your reply to Marc that the __ prefix here is for hyp mode
> > accessible code and data, but do we also need to use it for stack data?
> > No big deal, but it's not very pretty.
> >
>
> sure.
>
> > >
> > > /* Non-VHE: Disable and flush SPE data generation
> > > * VHE: The vcpu can run. but it can't hide. */
> > > if (!has_vhe())
> > > __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > > +
> > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > + return;
> > > +
> > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > +
> > > + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> > > + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> > > }
> > >
> > > -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > > +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> > > {
> > > + struct kvm_cpu_context *__host_ctxt;
> > > + struct kvm_cpu_context *__guest_ctxt;
> > > + struct kvm_guest_debug_arch *__host_dbg;
> > > + struct kvm_guest_debug_arch *__guest_dbg;
> > > +
> > > if (!has_vhe())
> > > __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > >
> > > - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > + return;
> > > +
> > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > +
> > > + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> > > + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
> > >
> > > vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > > }
> > > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > > index ef05c59..e270cba 100644
> > > --- a/arch/arm64/kvm/hyp/switch.c
> > > +++ b/arch/arm64/kvm/hyp/switch.c
> > > @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > guest_ctxt = &vcpu->arch.ctxt;
> > >
> > > __sysreg_save_host_state(host_ctxt);
> > > - __debug_cond_save_host_state(vcpu);
> > >
> > > __activate_traps(vcpu);
> > > __activate_vm(vcpu);
> > > @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > */
> > > __sysreg32_restore_state(vcpu);
> > > __sysreg_restore_guest_state(guest_ctxt);
> > > - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > + __debug_switch_to_guest(vcpu);
> > >
> > > /* Jump in the fire! */
> > > again:
> > > @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > >
> > > __sysreg_restore_host_state(host_ctxt);
> > >
> > > - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > /*
> > > * This must come after restoring the host sysregs, since a non-VHE
> > > * system may enable SPE here and make use of the TTBRs.
> > > */
> > > - __debug_cond_restore_host_state(vcpu);
> > > + __debug_switch_to_host(vcpu);
> > >
> > > return exit_code;
> > > }
> > > --
> > > 2.9.0
> > >
> >
> > This looks like a nice cleanup, but can you please add a note to the
> > commit message about why we don't need to use the
> >
> > save-host-state
> > activate-traps-and-vm
> > restore-guest-state
> >
> > and the reverse, patterns for the debug registers?
>
> I think the current commit message motivates the change fine.
>
The motivation is obvious, the justification for an additional change
is not. How does it justify changing the sequence
save-debug-host-state
activate-debug-traps /* and other stuff in between */
restore-debug-guest-state
to
activate-debug-traps /* and other stuff in between */
save-debug-host-state
restore-debug-guest-state
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
2017-12-03 13:49 ` Andrew Jones
@ 2017-12-03 20:47 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:47 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, Christoffer Dall, kvmarm, linux-arm-kernel,
kvm, Marc Zyngier, Shih-Wei Li
On Sun, Dec 03, 2017 at 02:49:58PM +0100, Andrew Jones wrote:
> On Fri, Dec 01, 2017 at 06:52:03PM +0100, Christoffer Dall wrote:
> > On Tue, Nov 07, 2017 at 03:48:57PM +0100, Andrew Jones wrote:
> > > On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> > > > Instead of having multiple calls from the world switch path to the debug
> > > > logic, each figuring out if the dirty bit is set and if we should
> > > > save/restore the debug registes, let's just provide two hooks to the
> > > > debug save/restore functionality, one for switching to the guest
> > > > context, and one for switching to the host context, and we get the
> > > > benefit of only having to evaluate the dirty flag once on each path,
> > > > plus we give the compiler some more room to inline some of this
> > > > functionality.
> > > >
> > > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > > ---
> > > > arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> > > > arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> > > > arch/arm64/kvm/hyp/switch.c | 6 ++---
> > > > 3 files changed, 42 insertions(+), 30 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > > > index 08d3bb6..a0e5a70 100644
> > > > --- a/arch/arm64/include/asm/kvm_hyp.h
> > > > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > > > @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> > > > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > > > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> > > >
> > > > -void __debug_save_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt);
> > > > -void __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt);
> > > > -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > > > -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > > > +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> > > > +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
> > > >
> > > > void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> > > > void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> > > > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > > > index a2291b6..b4cd8e0 100644
> > > > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > > > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > > > @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > > > write_sysreg_s(pmscr_el1, PMSCR_EL1);
> > > > }
> > > >
> > > > -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt)
> > > > +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > > + struct kvm_guest_debug_arch *dbg,
> > > > + struct kvm_cpu_context *ctxt)
> > > > {
> > > > u64 aa64dfr0;
> > > > int brps, wrps;
> > > >
> > > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > - return;
> > > > -
> > > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > > > brps = (aa64dfr0 >> 12) & 0xf;
> > > > wrps = (aa64dfr0 >> 20) & 0xf;
> > > > @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > > ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > > > }
> > > >
> > > > -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt)
> > > > +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > + struct kvm_guest_debug_arch *dbg,
> > > > + struct kvm_cpu_context *ctxt)
> > > > {
> > > > u64 aa64dfr0;
> > > > int brps, wrps;
> > > >
> > > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > - return;
> > > > -
> > > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > > >
> > > > brps = (aa64dfr0 >> 12) & 0xf;
> > > > @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > > > }
> > > >
> > > > -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > > > +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> > > > {
> > > > - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > > + struct kvm_cpu_context *__host_ctxt;
> > > > + struct kvm_cpu_context *__guest_ctxt;
> > > > + struct kvm_guest_debug_arch *__host_dbg;
> > > > + struct kvm_guest_debug_arch *__guest_dbg;
> > >
> > > I caught in your reply to Marc that the __ prefix here is for hyp mode
> > > accessible code and data, but do we also need to use it for stack data?
> > > No big deal, but it's not very pretty.
> > >
> >
> > sure.
> >
> > > >
> > > > /* Non-VHE: Disable and flush SPE data generation
> > > > * VHE: The vcpu can run. but it can't hide. */
> > > > if (!has_vhe())
> > > > __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > > > +
> > > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > + return;
> > > > +
> > > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > > +
> > > > + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> > > > + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> > > > }
> > > >
> > > > -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > > > +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> > > > {
> > > > + struct kvm_cpu_context *__host_ctxt;
> > > > + struct kvm_cpu_context *__guest_ctxt;
> > > > + struct kvm_guest_debug_arch *__host_dbg;
> > > > + struct kvm_guest_debug_arch *__guest_dbg;
> > > > +
> > > > if (!has_vhe())
> > > > __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > > >
> > > > - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > + return;
> > > > +
> > > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > > +
> > > > + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> > > > + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
> > > >
> > > > vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > > > }
> > > > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > > > index ef05c59..e270cba 100644
> > > > --- a/arch/arm64/kvm/hyp/switch.c
> > > > +++ b/arch/arm64/kvm/hyp/switch.c
> > > > @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > > guest_ctxt = &vcpu->arch.ctxt;
> > > >
> > > > __sysreg_save_host_state(host_ctxt);
> > > > - __debug_cond_save_host_state(vcpu);
> > > >
> > > > __activate_traps(vcpu);
> > > > __activate_vm(vcpu);
> > > > @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > > */
> > > > __sysreg32_restore_state(vcpu);
> > > > __sysreg_restore_guest_state(guest_ctxt);
> > > > - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > > + __debug_switch_to_guest(vcpu);
> > > >
> > > > /* Jump in the fire! */
> > > > again:
> > > > @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > >
> > > > __sysreg_restore_host_state(host_ctxt);
> > > >
> > > > - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > > /*
> > > > * This must come after restoring the host sysregs, since a non-VHE
> > > > * system may enable SPE here and make use of the TTBRs.
> > > > */
> > > > - __debug_cond_restore_host_state(vcpu);
> > > > + __debug_switch_to_host(vcpu);
> > > >
> > > > return exit_code;
> > > > }
> > > > --
> > > > 2.9.0
> > > >
> > >
> > > This looks like a nice cleanup, but can you please add a note to the
> > > commit message about why we don't need to use the
> > >
> > > save-host-state
> > > activate-traps-and-vm
> > > restore-guest-state
> > >
> > > and the reverse, patterns for the debug registers?
> >
> > I think the current commit message motivates the change fine.
> >
>
> The motivation is obvious, the justification for an additional change
> is not. How does it justify changing the sequence
>
> save-debug-host-state
> activate-debug-traps /* and other stuff in between */
> restore-debug-guest-state
>
> to
>
> activate-debug-traps /* and other stuff in between */
> save-debug-host-state
> restore-debug-guest-state
>
It doesn't, and I don't think it has to. The code is there, and I can't
argue all possible interdependencies between all code lines in English
and explain why it's safe, that requires looking at the code. The
commit message should motivate the change, not explain code which
requires understanding the software and the architecture.
If there was a clear rule that we do 'save, activate, restore' for
everything, it would make sense to explain why we can break that, but
that is not how I see the implementation. For example, there is no
particular reason why we restore the vgic before the timer, or the vgic
after we activate the stage 2 translation. The key here is
understanding the flexibility the architecture gives you.
You can then argue that since this bothers you, I should just add
something in the commit message, but I think that can be just as
misleading, because it would imply a total order, which doesn't exist.
Also, since you clearly understood the flow and what is going on, we're
probably fine as it is.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 11/37] KVM: arm64: Improve debug register save/restore flow
@ 2017-12-03 20:47 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:47 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Dec 03, 2017 at 02:49:58PM +0100, Andrew Jones wrote:
> On Fri, Dec 01, 2017 at 06:52:03PM +0100, Christoffer Dall wrote:
> > On Tue, Nov 07, 2017 at 03:48:57PM +0100, Andrew Jones wrote:
> > > On Thu, Oct 12, 2017 at 12:41:15PM +0200, Christoffer Dall wrote:
> > > > Instead of having multiple calls from the world switch path to the debug
> > > > logic, each figuring out if the dirty bit is set and if we should
> > > > save/restore the debug registes, let's just provide two hooks to the
> > > > debug save/restore functionality, one for switching to the guest
> > > > context, and one for switching to the host context, and we get the
> > > > benefit of only having to evaluate the dirty flag once on each path,
> > > > plus we give the compiler some more room to inline some of this
> > > > functionality.
> > > >
> > > > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > > > ---
> > > > arch/arm64/include/asm/kvm_hyp.h | 10 ++-----
> > > > arch/arm64/kvm/hyp/debug-sr.c | 56 +++++++++++++++++++++++++++-------------
> > > > arch/arm64/kvm/hyp/switch.c | 6 ++---
> > > > 3 files changed, 42 insertions(+), 30 deletions(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > > > index 08d3bb6..a0e5a70 100644
> > > > --- a/arch/arm64/include/asm/kvm_hyp.h
> > > > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > > > @@ -139,14 +139,8 @@ void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> > > > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > > > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> > > >
> > > > -void __debug_save_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt);
> > > > -void __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt);
> > > > -void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > > > -void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > > > +void __debug_switch_to_guest(struct kvm_vcpu *vcpu);
> > > > +void __debug_switch_to_host(struct kvm_vcpu *vcpu);
> > > >
> > > > void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> > > > void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> > > > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > > > index a2291b6..b4cd8e0 100644
> > > > --- a/arch/arm64/kvm/hyp/debug-sr.c
> > > > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > > > @@ -116,16 +116,13 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)
> > > > write_sysreg_s(pmscr_el1, PMSCR_EL1);
> > > > }
> > > >
> > > > -void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt)
> > > > +static void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > > + struct kvm_guest_debug_arch *dbg,
> > > > + struct kvm_cpu_context *ctxt)
> > > > {
> > > > u64 aa64dfr0;
> > > > int brps, wrps;
> > > >
> > > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > - return;
> > > > -
> > > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > > > brps = (aa64dfr0 >> 12) & 0xf;
> > > > wrps = (aa64dfr0 >> 20) & 0xf;
> > > > @@ -138,16 +135,13 @@ void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > > > ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > > > }
> > > >
> > > > -void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > - struct kvm_guest_debug_arch *dbg,
> > > > - struct kvm_cpu_context *ctxt)
> > > > +static void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > + struct kvm_guest_debug_arch *dbg,
> > > > + struct kvm_cpu_context *ctxt)
> > > > {
> > > > u64 aa64dfr0;
> > > > int brps, wrps;
> > > >
> > > > - if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > - return;
> > > > -
> > > > aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > > >
> > > > brps = (aa64dfr0 >> 12) & 0xf;
> > > > @@ -161,24 +155,50 @@ void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > > > write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > > > }
> > > >
> > > > -void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > > > +void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu)
> > > > {
> > > > - __debug_save_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > > + struct kvm_cpu_context *__host_ctxt;
> > > > + struct kvm_cpu_context *__guest_ctxt;
> > > > + struct kvm_guest_debug_arch *__host_dbg;
> > > > + struct kvm_guest_debug_arch *__guest_dbg;
> > >
> > > I caught in your reply to Marc that the __ prefix here is for hyp mode
> > > accessible code and data, but do we also need to use it for stack data?
> > > No big deal, but it's not very pretty.
> > >
> >
> > sure.
> >
> > > >
> > > > /* Non-VHE: Disable and flush SPE data generation
> > > > * VHE: The vcpu can run. but it can't hide. */
> > > > if (!has_vhe())
> > > > __debug_save_spe_nvhe(&vcpu->arch.host_debug_state.pmscr_el1);
> > > > +
> > > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > + return;
> > > > +
> > > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > > +
> > > > + __debug_save_state(vcpu, __host_dbg, __host_ctxt);
> > > > + __debug_restore_state(vcpu, __guest_dbg, __guest_ctxt);
> > > > }
> > > >
> > > > -void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > > > +void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu)
> > > > {
> > > > + struct kvm_cpu_context *__host_ctxt;
> > > > + struct kvm_cpu_context *__guest_ctxt;
> > > > + struct kvm_guest_debug_arch *__host_dbg;
> > > > + struct kvm_guest_debug_arch *__guest_dbg;
> > > > +
> > > > if (!has_vhe())
> > > > __debug_restore_spe(vcpu->arch.host_debug_state.pmscr_el1);
> > > >
> > > > - __debug_restore_state(vcpu, &vcpu->arch.host_debug_state.regs,
> > > > - kern_hyp_va(vcpu->arch.host_cpu_context));
> > > > + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> > > > + return;
> > > > +
> > > > + __host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > > > + __guest_ctxt = &vcpu->arch.ctxt;
> > > > + __host_dbg = &vcpu->arch.host_debug_state.regs;
> > > > + __guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr);
> > > > +
> > > > + __debug_save_state(vcpu, __guest_dbg, __guest_ctxt);
> > > > + __debug_restore_state(vcpu, __host_dbg, __host_ctxt);
> > > >
> > > > vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > > > }
> > > > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > > > index ef05c59..e270cba 100644
> > > > --- a/arch/arm64/kvm/hyp/switch.c
> > > > +++ b/arch/arm64/kvm/hyp/switch.c
> > > > @@ -271,7 +271,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > > guest_ctxt = &vcpu->arch.ctxt;
> > > >
> > > > __sysreg_save_host_state(host_ctxt);
> > > > - __debug_cond_save_host_state(vcpu);
> > > >
> > > > __activate_traps(vcpu);
> > > > __activate_vm(vcpu);
> > > > @@ -285,7 +284,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > > */
> > > > __sysreg32_restore_state(vcpu);
> > > > __sysreg_restore_guest_state(guest_ctxt);
> > > > - __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > > + __debug_switch_to_guest(vcpu);
> > > >
> > > > /* Jump in the fire! */
> > > > again:
> > > > @@ -353,12 +352,11 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > > >
> > > > __sysreg_restore_host_state(host_ctxt);
> > > >
> > > > - __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > > > /*
> > > > * This must come after restoring the host sysregs, since a non-VHE
> > > > * system may enable SPE here and make use of the TTBRs.
> > > > */
> > > > - __debug_cond_restore_host_state(vcpu);
> > > > + __debug_switch_to_host(vcpu);
> > > >
> > > > return exit_code;
> > > > }
> > > > --
> > > > 2.9.0
> > > >
> > >
> > > This looks like a nice cleanup, but can you please add a note to the
> > > commit message about why we don't need to use the
> > >
> > > save-host-state
> > > activate-traps-and-vm
> > > restore-guest-state
> > >
> > > and the reverse, patterns for the debug registers?
> >
> > I think the current commit message motivates the change fine.
> >
>
> The motivation is obvious, the justification for an additional change
> is not. How does it justify changing the sequence
>
> save-debug-host-state
> activate-debug-traps /* and other stuff in between */
> restore-debug-guest-state
>
> to
>
> activate-debug-traps /* and other stuff in between */
> save-debug-host-state
> restore-debug-guest-state
>
It doesn't, and I don't think it has to. The code is there, and I can't
argue all possible interdependencies between all code lines in English
and explain why it's safe, that requires looking at the code. The
commit message should motivate the change, not explain code which
requires understanding the software and the architecture.
If there was a clear rule that we do 'save, activate, restore' for
everything, it would make sense to explain why we can break that, but
that is not how I see the implementation. For example, there is no
particular reason why we restore the vgic before the timer, or the vgic
after we activate the stage 2 translation. The key here is
understanding the flexibility the architecture gives you.
You can then argue that since this bothers you, I should just add
something in the commit message, but I think that can be just as
misleading, because it would imply a total order, which doesn't exist.
Also, since you clearly understood the flow and what is going on, we're
probably fine as it is.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 12/37] KVM: arm64: Factor out fault info population and gic workarounds
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
The current world-switch function has functionality to detect a number
of cases where we need to fixup some part of the exit condition and
possibly run the guest again, before having restored the host state.
This includes populating missing fault info, emulating GICv2 CPU
interface accesses when mapped at unaligned addresses, and emulating
the GICv3 CPU interfaceon systems that need that.
As we are about to have an alternative switch function for VHE systems,
but VHE systems still need the same early fixup logic, factor out this
logic into a separate function that can be shared by both switch
functions.
No functional change.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 91 ++++++++++++++++++++++++++-------------------
1 file changed, 52 insertions(+), 39 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index e270cba..ed30af5 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -258,50 +258,24 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)
write_sysreg_el2(*vcpu_pc(vcpu), elr);
}
-int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
+/*
+ * Return true when we were able to fixup the guest exit and should return to
+ * the guest, false when we should restore the host state and return to the
+ * main run loop.
+ */
+static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
{
- struct kvm_cpu_context *host_ctxt;
- struct kvm_cpu_context *guest_ctxt;
- u64 exit_code;
-
- vcpu = kern_hyp_va(vcpu);
-
- host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
- host_ctxt->__hyp_running_vcpu = vcpu;
- guest_ctxt = &vcpu->arch.ctxt;
-
- __sysreg_save_host_state(host_ctxt);
-
- __activate_traps(vcpu);
- __activate_vm(vcpu);
-
- __vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
-
- /*
- * We must restore the 32-bit state before the sysregs, thanks
- * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
- */
- __sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state(guest_ctxt);
- __debug_switch_to_guest(vcpu);
-
- /* Jump in the fire! */
-again:
- exit_code = __guest_enter(vcpu, host_ctxt);
- /* And we're baaack! */
-
/*
* We're using the raw exception code in order to only process
* the trap if no SError is pending. We will come back to the
* same PC once the SError has been injected, and replay the
* trapping instruction.
*/
- if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
- goto again;
+ if (*exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
+ return true;
if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
- exit_code == ARM_EXCEPTION_TRAP) {
+ *exit_code == ARM_EXCEPTION_TRAP) {
bool valid;
valid = kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_DABT_LOW &&
@@ -315,13 +289,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (ret == 1) {
__skip_instr(vcpu);
- goto again;
+ return true;
}
if (ret == -1) {
/* Promote an illegal access to an SError */
__skip_instr(vcpu);
- exit_code = ARM_EXCEPTION_EL1_SERROR;
+ *exit_code = ARM_EXCEPTION_EL1_SERROR;
}
/* 0 falls through to be handler out of EL2 */
@@ -329,19 +303,58 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
}
if (static_branch_unlikely(&vgic_v3_cpuif_trap) &&
- exit_code == ARM_EXCEPTION_TRAP &&
+ *exit_code == ARM_EXCEPTION_TRAP &&
(kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 ||
kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_CP15_32)) {
int ret = __vgic_v3_perform_cpuif_access(vcpu);
if (ret == 1) {
__skip_instr(vcpu);
- goto again;
+ return true;
}
/* 0 falls through to be handled out of EL2 */
}
+ return false;
+}
+
+int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *host_ctxt;
+ struct kvm_cpu_context *guest_ctxt;
+ u64 exit_code;
+
+ vcpu = kern_hyp_va(vcpu);
+
+ host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ host_ctxt->__hyp_running_vcpu = vcpu;
+ guest_ctxt = &vcpu->arch.ctxt;
+
+ __sysreg_save_host_state(host_ctxt);
+
+ __activate_traps(vcpu);
+ __activate_vm(vcpu);
+
+ __vgic_restore_state(vcpu);
+ __timer_enable_traps(vcpu);
+
+ /*
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_guest_state(guest_ctxt);
+ __debug_switch_to_guest(vcpu);
+
+ /* Jump in the fire! */
+again:
+ exit_code = __guest_enter(vcpu, host_ctxt);
+ /* And we're baaack! */
+
+ if (fixup_guest_exit(vcpu, &exit_code))
+ goto again;
+
__sysreg_save_guest_state(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 12/37] KVM: arm64: Factor out fault info population and gic workarounds
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
The current world-switch function has functionality to detect a number
of cases where we need to fixup some part of the exit condition and
possibly run the guest again, before having restored the host state.
This includes populating missing fault info, emulating GICv2 CPU
interface accesses when mapped at unaligned addresses, and emulating
the GICv3 CPU interfaceon systems that need that.
As we are about to have an alternative switch function for VHE systems,
but VHE systems still need the same early fixup logic, factor out this
logic into a separate function that can be shared by both switch
functions.
No functional change.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 91 ++++++++++++++++++++++++++-------------------
1 file changed, 52 insertions(+), 39 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index e270cba..ed30af5 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -258,50 +258,24 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)
write_sysreg_el2(*vcpu_pc(vcpu), elr);
}
-int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
+/*
+ * Return true when we were able to fixup the guest exit and should return to
+ * the guest, false when we should restore the host state and return to the
+ * main run loop.
+ */
+static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
{
- struct kvm_cpu_context *host_ctxt;
- struct kvm_cpu_context *guest_ctxt;
- u64 exit_code;
-
- vcpu = kern_hyp_va(vcpu);
-
- host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
- host_ctxt->__hyp_running_vcpu = vcpu;
- guest_ctxt = &vcpu->arch.ctxt;
-
- __sysreg_save_host_state(host_ctxt);
-
- __activate_traps(vcpu);
- __activate_vm(vcpu);
-
- __vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
-
- /*
- * We must restore the 32-bit state before the sysregs, thanks
- * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
- */
- __sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state(guest_ctxt);
- __debug_switch_to_guest(vcpu);
-
- /* Jump in the fire! */
-again:
- exit_code = __guest_enter(vcpu, host_ctxt);
- /* And we're baaack! */
-
/*
* We're using the raw exception code in order to only process
* the trap if no SError is pending. We will come back to the
* same PC once the SError has been injected, and replay the
* trapping instruction.
*/
- if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
- goto again;
+ if (*exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
+ return true;
if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
- exit_code == ARM_EXCEPTION_TRAP) {
+ *exit_code == ARM_EXCEPTION_TRAP) {
bool valid;
valid = kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_DABT_LOW &&
@@ -315,13 +289,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (ret == 1) {
__skip_instr(vcpu);
- goto again;
+ return true;
}
if (ret == -1) {
/* Promote an illegal access to an SError */
__skip_instr(vcpu);
- exit_code = ARM_EXCEPTION_EL1_SERROR;
+ *exit_code = ARM_EXCEPTION_EL1_SERROR;
}
/* 0 falls through to be handler out of EL2 */
@@ -329,19 +303,58 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
}
if (static_branch_unlikely(&vgic_v3_cpuif_trap) &&
- exit_code == ARM_EXCEPTION_TRAP &&
+ *exit_code == ARM_EXCEPTION_TRAP &&
(kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 ||
kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_CP15_32)) {
int ret = __vgic_v3_perform_cpuif_access(vcpu);
if (ret == 1) {
__skip_instr(vcpu);
- goto again;
+ return true;
}
/* 0 falls through to be handled out of EL2 */
}
+ return false;
+}
+
+int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *host_ctxt;
+ struct kvm_cpu_context *guest_ctxt;
+ u64 exit_code;
+
+ vcpu = kern_hyp_va(vcpu);
+
+ host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ host_ctxt->__hyp_running_vcpu = vcpu;
+ guest_ctxt = &vcpu->arch.ctxt;
+
+ __sysreg_save_host_state(host_ctxt);
+
+ __activate_traps(vcpu);
+ __activate_vm(vcpu);
+
+ __vgic_restore_state(vcpu);
+ __timer_enable_traps(vcpu);
+
+ /*
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_guest_state(guest_ctxt);
+ __debug_switch_to_guest(vcpu);
+
+ /* Jump in the fire! */
+again:
+ exit_code = __guest_enter(vcpu, host_ctxt);
+ /* And we're baaack! */
+
+ if (fixup_guest_exit(vcpu, &exit_code))
+ goto again;
+
__sysreg_save_guest_state(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 12/37] KVM: arm64: Factor out fault info population and gic workarounds
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 15:12 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 15:12 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:16PM +0200, Christoffer Dall wrote:
> The current world-switch function has functionality to detect a number
> of cases where we need to fixup some part of the exit condition and
> possibly run the guest again, before having restored the host state.
>
> This includes populating missing fault info, emulating GICv2 CPU
> interface accesses when mapped at unaligned addresses, and emulating
> the GICv3 CPU interfaceon systems that need that.
interface on
need it.
>
> As we are about to have an alternative switch function for VHE systems,
> but VHE systems still need the same early fixup logic, factor out this
> logic into a separate function that can be shared by both switch
> functions.
>
> No functional change.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 91 ++++++++++++++++++++++++++-------------------
> 1 file changed, 52 insertions(+), 39 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index e270cba..ed30af5 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -258,50 +258,24 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)
> write_sysreg_el2(*vcpu_pc(vcpu), elr);
> }
>
> -int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> +/*
> + * Return true when we were able to fixup the guest exit and should return to
> + * the guest, false when we should restore the host state and return to the
> + * main run loop.
> + */
> +static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> {
> - struct kvm_cpu_context *host_ctxt;
> - struct kvm_cpu_context *guest_ctxt;
> - u64 exit_code;
> -
> - vcpu = kern_hyp_va(vcpu);
> -
> - host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> - host_ctxt->__hyp_running_vcpu = vcpu;
> - guest_ctxt = &vcpu->arch.ctxt;
> -
> - __sysreg_save_host_state(host_ctxt);
> -
> - __activate_traps(vcpu);
> - __activate_vm(vcpu);
> -
> - __vgic_restore_state(vcpu);
> - __timer_enable_traps(vcpu);
> -
> - /*
> - * We must restore the 32-bit state before the sysregs, thanks
> - * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> - */
> - __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state(guest_ctxt);
> - __debug_switch_to_guest(vcpu);
> -
> - /* Jump in the fire! */
> -again:
> - exit_code = __guest_enter(vcpu, host_ctxt);
> - /* And we're baaack! */
> -
> /*
> * We're using the raw exception code in order to only process
> * the trap if no SError is pending. We will come back to the
> * same PC once the SError has been injected, and replay the
> * trapping instruction.
> */
> - if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
> - goto again;
> + if (*exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
> + return true;
>
> if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
> - exit_code == ARM_EXCEPTION_TRAP) {
> + *exit_code == ARM_EXCEPTION_TRAP) {
> bool valid;
>
> valid = kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_DABT_LOW &&
> @@ -315,13 +289,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> if (ret == 1) {
> __skip_instr(vcpu);
> - goto again;
> + return true;
> }
>
> if (ret == -1) {
> /* Promote an illegal access to an SError */
> __skip_instr(vcpu);
> - exit_code = ARM_EXCEPTION_EL1_SERROR;
> + *exit_code = ARM_EXCEPTION_EL1_SERROR;
> }
>
> /* 0 falls through to be handler out of EL2 */
> @@ -329,19 +303,58 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> }
>
> if (static_branch_unlikely(&vgic_v3_cpuif_trap) &&
> - exit_code == ARM_EXCEPTION_TRAP &&
> + *exit_code == ARM_EXCEPTION_TRAP &&
> (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 ||
> kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_CP15_32)) {
> int ret = __vgic_v3_perform_cpuif_access(vcpu);
>
> if (ret == 1) {
> __skip_instr(vcpu);
> - goto again;
> + return true;
> }
>
> /* 0 falls through to be handled out of EL2 */
I'm not sure the "0 falls through..." comments are as easy to understand
now that we only fall through to a 'return false'. Maybe they should be
modified?
> }
>
> + return false;
> +}
> +
> +int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *host_ctxt;
> + struct kvm_cpu_context *guest_ctxt;
> + u64 exit_code;
> +
> + vcpu = kern_hyp_va(vcpu);
> +
> + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + host_ctxt->__hyp_running_vcpu = vcpu;
> + guest_ctxt = &vcpu->arch.ctxt;
> +
> + __sysreg_save_host_state(host_ctxt);
> +
> + __activate_traps(vcpu);
> + __activate_vm(vcpu);
> +
> + __vgic_restore_state(vcpu);
> + __timer_enable_traps(vcpu);
> +
> + /*
> + * We must restore the 32-bit state before the sysregs, thanks
> + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> + */
> + __sysreg32_restore_state(vcpu);
> + __sysreg_restore_guest_state(guest_ctxt);
> + __debug_switch_to_guest(vcpu);
> +
> + /* Jump in the fire! */
> +again:
> + exit_code = __guest_enter(vcpu, host_ctxt);
> + /* And we're baaack! */
> +
> + if (fixup_guest_exit(vcpu, &exit_code))
> + goto again;
We can change this goto to a do-while now.
> +
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> --
> 2.9.0
>
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 12/37] KVM: arm64: Factor out fault info population and gic workarounds
@ 2017-11-07 15:12 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 15:12 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:16PM +0200, Christoffer Dall wrote:
> The current world-switch function has functionality to detect a number
> of cases where we need to fixup some part of the exit condition and
> possibly run the guest again, before having restored the host state.
>
> This includes populating missing fault info, emulating GICv2 CPU
> interface accesses when mapped at unaligned addresses, and emulating
> the GICv3 CPU interfaceon systems that need that.
interface on
need it.
>
> As we are about to have an alternative switch function for VHE systems,
> but VHE systems still need the same early fixup logic, factor out this
> logic into a separate function that can be shared by both switch
> functions.
>
> No functional change.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 91 ++++++++++++++++++++++++++-------------------
> 1 file changed, 52 insertions(+), 39 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index e270cba..ed30af5 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -258,50 +258,24 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)
> write_sysreg_el2(*vcpu_pc(vcpu), elr);
> }
>
> -int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> +/*
> + * Return true when we were able to fixup the guest exit and should return to
> + * the guest, false when we should restore the host state and return to the
> + * main run loop.
> + */
> +static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> {
> - struct kvm_cpu_context *host_ctxt;
> - struct kvm_cpu_context *guest_ctxt;
> - u64 exit_code;
> -
> - vcpu = kern_hyp_va(vcpu);
> -
> - host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> - host_ctxt->__hyp_running_vcpu = vcpu;
> - guest_ctxt = &vcpu->arch.ctxt;
> -
> - __sysreg_save_host_state(host_ctxt);
> -
> - __activate_traps(vcpu);
> - __activate_vm(vcpu);
> -
> - __vgic_restore_state(vcpu);
> - __timer_enable_traps(vcpu);
> -
> - /*
> - * We must restore the 32-bit state before the sysregs, thanks
> - * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> - */
> - __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state(guest_ctxt);
> - __debug_switch_to_guest(vcpu);
> -
> - /* Jump in the fire! */
> -again:
> - exit_code = __guest_enter(vcpu, host_ctxt);
> - /* And we're baaack! */
> -
> /*
> * We're using the raw exception code in order to only process
> * the trap if no SError is pending. We will come back to the
> * same PC once the SError has been injected, and replay the
> * trapping instruction.
> */
> - if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
> - goto again;
> + if (*exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu))
> + return true;
>
> if (static_branch_unlikely(&vgic_v2_cpuif_trap) &&
> - exit_code == ARM_EXCEPTION_TRAP) {
> + *exit_code == ARM_EXCEPTION_TRAP) {
> bool valid;
>
> valid = kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_DABT_LOW &&
> @@ -315,13 +289,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> if (ret == 1) {
> __skip_instr(vcpu);
> - goto again;
> + return true;
> }
>
> if (ret == -1) {
> /* Promote an illegal access to an SError */
> __skip_instr(vcpu);
> - exit_code = ARM_EXCEPTION_EL1_SERROR;
> + *exit_code = ARM_EXCEPTION_EL1_SERROR;
> }
>
> /* 0 falls through to be handler out of EL2 */
> @@ -329,19 +303,58 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> }
>
> if (static_branch_unlikely(&vgic_v3_cpuif_trap) &&
> - exit_code == ARM_EXCEPTION_TRAP &&
> + *exit_code == ARM_EXCEPTION_TRAP &&
> (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 ||
> kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_CP15_32)) {
> int ret = __vgic_v3_perform_cpuif_access(vcpu);
>
> if (ret == 1) {
> __skip_instr(vcpu);
> - goto again;
> + return true;
> }
>
> /* 0 falls through to be handled out of EL2 */
I'm not sure the "0 falls through..." comments are as easy to understand
now that we only fall through to a 'return false'. Maybe they should be
modified?
> }
>
> + return false;
> +}
> +
> +int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *host_ctxt;
> + struct kvm_cpu_context *guest_ctxt;
> + u64 exit_code;
> +
> + vcpu = kern_hyp_va(vcpu);
> +
> + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + host_ctxt->__hyp_running_vcpu = vcpu;
> + guest_ctxt = &vcpu->arch.ctxt;
> +
> + __sysreg_save_host_state(host_ctxt);
> +
> + __activate_traps(vcpu);
> + __activate_vm(vcpu);
> +
> + __vgic_restore_state(vcpu);
> + __timer_enable_traps(vcpu);
> +
> + /*
> + * We must restore the 32-bit state before the sysregs, thanks
> + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> + */
> + __sysreg32_restore_state(vcpu);
> + __sysreg_restore_guest_state(guest_ctxt);
> + __debug_switch_to_guest(vcpu);
> +
> + /* Jump in the fire! */
> +again:
> + exit_code = __guest_enter(vcpu, host_ctxt);
> + /* And we're baaack! */
> +
> + if (fixup_guest_exit(vcpu, &exit_code))
> + goto again;
We can change this goto to a do-while now.
> +
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> --
> 2.9.0
>
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 13/37] KVM: arm64: Introduce VHE-specific kvm_vcpu_run
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
So far this is just a copy of the legacy non-VHE switch function, where
we only change the existing calls to has_vhe() in both the original and
new functions.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_asm.h | 4 +++
arch/arm64/include/asm/kvm_asm.h | 2 ++
arch/arm64/kvm/hyp/switch.c | 57 ++++++++++++++++++++++++++++++++++++++++
virt/kvm/arm/arm.c | 5 +++-
4 files changed, 67 insertions(+), 1 deletion(-)
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 36dd296..1a7bc5f 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -70,8 +70,12 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
+/* no VHE on 32-bit :( */
+static inline int kvm_vcpu_run(struct kvm_vcpu *vcpu) { return 0; }
+
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
extern void __init_stage2_translation(void);
extern u64 __vgic_v3_get_ich_vtr_el2(void);
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 7e48a39..2eb5b23 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -57,6 +57,8 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
+extern int kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
extern u64 __vgic_v3_get_ich_vtr_el2(void);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index ed30af5..8a0f38f 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -319,6 +319,63 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
return false;
}
+/* Switch to the guest for VHE systems running in EL2 */
+int kvm_vcpu_run(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *host_ctxt;
+ struct kvm_cpu_context *guest_ctxt;
+ u64 exit_code;
+
+ vcpu = kern_hyp_va(vcpu);
+
+ host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ host_ctxt->__hyp_running_vcpu = vcpu;
+ guest_ctxt = &vcpu->arch.ctxt;
+
+ __sysreg_save_host_state(host_ctxt);
+
+ __activate_traps(vcpu);
+ __activate_vm(vcpu);
+
+ __vgic_restore_state(vcpu);
+ __timer_enable_traps(vcpu);
+
+ /*
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_guest_state(guest_ctxt);
+ __debug_switch_to_guest(vcpu);
+
+ /* Jump in the fire! */
+again:
+ exit_code = __guest_enter(vcpu, host_ctxt);
+ /* And we're baaack! */
+
+ if (fixup_guest_exit(vcpu, &exit_code))
+ goto again;
+
+ __sysreg_save_guest_state(guest_ctxt);
+ __sysreg32_save_state(vcpu);
+ __timer_disable_traps(vcpu);
+ __vgic_save_state(vcpu);
+
+ __deactivate_traps(vcpu);
+ __deactivate_vm(vcpu);
+
+ __sysreg_restore_host_state(host_ctxt);
+
+ /*
+ * This must come after restoring the host sysregs, since a non-VHE
+ * system may enable SPE here and make use of the TTBRs.
+ */
+ __debug_switch_to_host(vcpu);
+
+ return exit_code;
+}
+
+/* Switch to the guest for legacy non-VHE systems */
int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index cf121b2..b11647a 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -706,7 +706,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
trace_kvm_entry(*vcpu_pc(vcpu));
guest_enter_irqoff();
- ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
+ if (has_vhe())
+ ret = kvm_vcpu_run(vcpu);
+ else
+ ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 13/37] KVM: arm64: Introduce VHE-specific kvm_vcpu_run
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
So far this is just a copy of the legacy non-VHE switch function, where
we only change the existing calls to has_vhe() in both the original and
new functions.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_asm.h | 4 +++
arch/arm64/include/asm/kvm_asm.h | 2 ++
arch/arm64/kvm/hyp/switch.c | 57 ++++++++++++++++++++++++++++++++++++++++
virt/kvm/arm/arm.c | 5 +++-
4 files changed, 67 insertions(+), 1 deletion(-)
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 36dd296..1a7bc5f 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -70,8 +70,12 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
+/* no VHE on 32-bit :( */
+static inline int kvm_vcpu_run(struct kvm_vcpu *vcpu) { return 0; }
+
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
extern void __init_stage2_translation(void);
extern u64 __vgic_v3_get_ich_vtr_el2(void);
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 7e48a39..2eb5b23 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -57,6 +57,8 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
+extern int kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
extern u64 __vgic_v3_get_ich_vtr_el2(void);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index ed30af5..8a0f38f 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -319,6 +319,63 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
return false;
}
+/* Switch to the guest for VHE systems running in EL2 */
+int kvm_vcpu_run(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *host_ctxt;
+ struct kvm_cpu_context *guest_ctxt;
+ u64 exit_code;
+
+ vcpu = kern_hyp_va(vcpu);
+
+ host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ host_ctxt->__hyp_running_vcpu = vcpu;
+ guest_ctxt = &vcpu->arch.ctxt;
+
+ __sysreg_save_host_state(host_ctxt);
+
+ __activate_traps(vcpu);
+ __activate_vm(vcpu);
+
+ __vgic_restore_state(vcpu);
+ __timer_enable_traps(vcpu);
+
+ /*
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_guest_state(guest_ctxt);
+ __debug_switch_to_guest(vcpu);
+
+ /* Jump in the fire! */
+again:
+ exit_code = __guest_enter(vcpu, host_ctxt);
+ /* And we're baaack! */
+
+ if (fixup_guest_exit(vcpu, &exit_code))
+ goto again;
+
+ __sysreg_save_guest_state(guest_ctxt);
+ __sysreg32_save_state(vcpu);
+ __timer_disable_traps(vcpu);
+ __vgic_save_state(vcpu);
+
+ __deactivate_traps(vcpu);
+ __deactivate_vm(vcpu);
+
+ __sysreg_restore_host_state(host_ctxt);
+
+ /*
+ * This must come after restoring the host sysregs, since a non-VHE
+ * system may enable SPE here and make use of the TTBRs.
+ */
+ __debug_switch_to_host(vcpu);
+
+ return exit_code;
+}
+
+/* Switch to the guest for legacy non-VHE systems */
int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index cf121b2..b11647a 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -706,7 +706,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
trace_kvm_entry(*vcpu_pc(vcpu));
guest_enter_irqoff();
- ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
+ if (has_vhe())
+ ret = kvm_vcpu_run(vcpu);
+ else
+ ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 13/37] KVM: arm64: Introduce VHE-specific kvm_vcpu_run
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 15:25 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 15:25 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:17PM +0200, Christoffer Dall wrote:
> So far this is just a copy of the legacy non-VHE switch function, where
> we only change the existing calls to has_vhe() in both the original and
> new functions.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_asm.h | 4 +++
> arch/arm64/include/asm/kvm_asm.h | 2 ++
> arch/arm64/kvm/hyp/switch.c | 57 ++++++++++++++++++++++++++++++++++++++++
> virt/kvm/arm/arm.c | 5 +++-
> 4 files changed, 67 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> index 36dd296..1a7bc5f 100644
> --- a/arch/arm/include/asm/kvm_asm.h
> +++ b/arch/arm/include/asm/kvm_asm.h
> @@ -70,8 +70,12 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
>
> extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
>
> +/* no VHE on 32-bit :( */
> +static inline int kvm_vcpu_run(struct kvm_vcpu *vcpu) { return 0; }
> +
> extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>
> +
stray new blank line
> extern void __init_stage2_translation(void);
>
> extern u64 __vgic_v3_get_ich_vtr_el2(void);
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index 7e48a39..2eb5b23 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -57,6 +57,8 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
>
> extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
>
> +extern int kvm_vcpu_run(struct kvm_vcpu *vcpu);
> +
> extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>
> extern u64 __vgic_v3_get_ich_vtr_el2(void);
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index ed30af5..8a0f38f 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -319,6 +319,63 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> return false;
> }
>
> +/* Switch to the guest for VHE systems running in EL2 */
> +int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *host_ctxt;
> + struct kvm_cpu_context *guest_ctxt;
> + u64 exit_code;
> +
> + vcpu = kern_hyp_va(vcpu);
> +
> + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + host_ctxt->__hyp_running_vcpu = vcpu;
> + guest_ctxt = &vcpu->arch.ctxt;
> +
> + __sysreg_save_host_state(host_ctxt);
> +
> + __activate_traps(vcpu);
> + __activate_vm(vcpu);
> +
> + __vgic_restore_state(vcpu);
> + __timer_enable_traps(vcpu);
> +
> + /*
> + * We must restore the 32-bit state before the sysregs, thanks
> + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> + */
> + __sysreg32_restore_state(vcpu);
> + __sysreg_restore_guest_state(guest_ctxt);
> + __debug_switch_to_guest(vcpu);
> +
> + /* Jump in the fire! */
> +again:
> + exit_code = __guest_enter(vcpu, host_ctxt);
> + /* And we're baaack! */
> +
> + if (fixup_guest_exit(vcpu, &exit_code))
> + goto again;
> +
> + __sysreg_save_guest_state(guest_ctxt);
> + __sysreg32_save_state(vcpu);
> + __timer_disable_traps(vcpu);
> + __vgic_save_state(vcpu);
> +
> + __deactivate_traps(vcpu);
> + __deactivate_vm(vcpu);
> +
> + __sysreg_restore_host_state(host_ctxt);
> +
> + /*
> + * This must come after restoring the host sysregs, since a non-VHE
> + * system may enable SPE here and make use of the TTBRs.
> + */
> + __debug_switch_to_host(vcpu);
> +
> + return exit_code;
> +}
> +
> +/* Switch to the guest for legacy non-VHE systems */
> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index cf121b2..b11647a 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -706,7 +706,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> trace_kvm_entry(*vcpu_pc(vcpu));
> guest_enter_irqoff();
>
> - ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
> + if (has_vhe())
> + ret = kvm_vcpu_run(vcpu);
> + else
> + ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
So kvm_vcpu_run() is going to be VHE only, but the only way for people
stepping through the code to know that is by seeing this callsite or by
reading the comment above it. Everywhere else we make it much more clear
with the _vhe and _nvhe suffixes. I wonder if we should keep that pattern
by adding
static int kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
if (has_vhe())
ret = kvm_vcpu_run_vhe(vcpu);
else
ret = kvm_call_hyp(__kvm_vcpu_run_nvhe, vcpu);
}
to virt/kvm/arm/arm.c and then calling that from here.
I'm afraid the __ prefix is too ambiguous in the kernel to interpret
__foo() as the hyp mode non-VHE version of foo(), the hyp mode VHE
version.
Thanks,
drew
>
> vcpu->mode = OUTSIDE_GUEST_MODE;
> vcpu->stat.exits++;
> --
> 2.9.0
>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 13/37] KVM: arm64: Introduce VHE-specific kvm_vcpu_run
@ 2017-11-07 15:25 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 15:25 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:17PM +0200, Christoffer Dall wrote:
> So far this is just a copy of the legacy non-VHE switch function, where
> we only change the existing calls to has_vhe() in both the original and
> new functions.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_asm.h | 4 +++
> arch/arm64/include/asm/kvm_asm.h | 2 ++
> arch/arm64/kvm/hyp/switch.c | 57 ++++++++++++++++++++++++++++++++++++++++
> virt/kvm/arm/arm.c | 5 +++-
> 4 files changed, 67 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> index 36dd296..1a7bc5f 100644
> --- a/arch/arm/include/asm/kvm_asm.h
> +++ b/arch/arm/include/asm/kvm_asm.h
> @@ -70,8 +70,12 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
>
> extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
>
> +/* no VHE on 32-bit :( */
> +static inline int kvm_vcpu_run(struct kvm_vcpu *vcpu) { return 0; }
> +
> extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>
> +
stray new blank line
> extern void __init_stage2_translation(void);
>
> extern u64 __vgic_v3_get_ich_vtr_el2(void);
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index 7e48a39..2eb5b23 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -57,6 +57,8 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
>
> extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
>
> +extern int kvm_vcpu_run(struct kvm_vcpu *vcpu);
> +
> extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>
> extern u64 __vgic_v3_get_ich_vtr_el2(void);
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index ed30af5..8a0f38f 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -319,6 +319,63 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> return false;
> }
>
> +/* Switch to the guest for VHE systems running in EL2 */
> +int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *host_ctxt;
> + struct kvm_cpu_context *guest_ctxt;
> + u64 exit_code;
> +
> + vcpu = kern_hyp_va(vcpu);
> +
> + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + host_ctxt->__hyp_running_vcpu = vcpu;
> + guest_ctxt = &vcpu->arch.ctxt;
> +
> + __sysreg_save_host_state(host_ctxt);
> +
> + __activate_traps(vcpu);
> + __activate_vm(vcpu);
> +
> + __vgic_restore_state(vcpu);
> + __timer_enable_traps(vcpu);
> +
> + /*
> + * We must restore the 32-bit state before the sysregs, thanks
> + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> + */
> + __sysreg32_restore_state(vcpu);
> + __sysreg_restore_guest_state(guest_ctxt);
> + __debug_switch_to_guest(vcpu);
> +
> + /* Jump in the fire! */
> +again:
> + exit_code = __guest_enter(vcpu, host_ctxt);
> + /* And we're baaack! */
> +
> + if (fixup_guest_exit(vcpu, &exit_code))
> + goto again;
> +
> + __sysreg_save_guest_state(guest_ctxt);
> + __sysreg32_save_state(vcpu);
> + __timer_disable_traps(vcpu);
> + __vgic_save_state(vcpu);
> +
> + __deactivate_traps(vcpu);
> + __deactivate_vm(vcpu);
> +
> + __sysreg_restore_host_state(host_ctxt);
> +
> + /*
> + * This must come after restoring the host sysregs, since a non-VHE
> + * system may enable SPE here and make use of the TTBRs.
> + */
> + __debug_switch_to_host(vcpu);
> +
> + return exit_code;
> +}
> +
> +/* Switch to the guest for legacy non-VHE systems */
> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index cf121b2..b11647a 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -706,7 +706,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> trace_kvm_entry(*vcpu_pc(vcpu));
> guest_enter_irqoff();
>
> - ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
> + if (has_vhe())
> + ret = kvm_vcpu_run(vcpu);
> + else
> + ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
So kvm_vcpu_run() is going to be VHE only, but the only way for people
stepping through the code to know that is by seeing this callsite or by
reading the comment above it. Everywhere else we make it much more clear
with the _vhe and _nvhe suffixes. I wonder if we should keep that pattern
by adding
static int kvm_vcpu_run(struct kvm_vcpu *vcpu)
{
if (has_vhe())
ret = kvm_vcpu_run_vhe(vcpu);
else
ret = kvm_call_hyp(__kvm_vcpu_run_nvhe, vcpu);
}
to virt/kvm/arm/arm.c and then calling that from here.
I'm afraid the __ prefix is too ambiguous in the kernel to interpret
__foo() as the hyp mode non-VHE version of foo(), the hyp mode VHE
version.
Thanks,
drew
>
> vcpu->mode = OUTSIDE_GUEST_MODE;
> vcpu->stat.exits++;
> --
> 2.9.0
>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 13/37] KVM: arm64: Introduce VHE-specific kvm_vcpu_run
2017-11-07 15:25 ` Andrew Jones
@ 2017-12-01 18:10 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 18:10 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 04:25:29PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:17PM +0200, Christoffer Dall wrote:
> > So far this is just a copy of the legacy non-VHE switch function, where
> > we only change the existing calls to has_vhe() in both the original and
> > new functions.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_asm.h | 4 +++
> > arch/arm64/include/asm/kvm_asm.h | 2 ++
> > arch/arm64/kvm/hyp/switch.c | 57 ++++++++++++++++++++++++++++++++++++++++
> > virt/kvm/arm/arm.c | 5 +++-
> > 4 files changed, 67 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> > index 36dd296..1a7bc5f 100644
> > --- a/arch/arm/include/asm/kvm_asm.h
> > +++ b/arch/arm/include/asm/kvm_asm.h
> > @@ -70,8 +70,12 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
> >
> > extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
> >
> > +/* no VHE on 32-bit :( */
> > +static inline int kvm_vcpu_run(struct kvm_vcpu *vcpu) { return 0; }
> > +
> > extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> >
> > +
>
> stray new blank line
>
> > extern void __init_stage2_translation(void);
> >
> > extern u64 __vgic_v3_get_ich_vtr_el2(void);
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index 7e48a39..2eb5b23 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -57,6 +57,8 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
> >
> > extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
> >
> > +extern int kvm_vcpu_run(struct kvm_vcpu *vcpu);
> > +
> > extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> >
> > extern u64 __vgic_v3_get_ich_vtr_el2(void);
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index ed30af5..8a0f38f 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -319,6 +319,63 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> > return false;
> > }
> >
> > +/* Switch to the guest for VHE systems running in EL2 */
> > +int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm_cpu_context *host_ctxt;
> > + struct kvm_cpu_context *guest_ctxt;
> > + u64 exit_code;
> > +
> > + vcpu = kern_hyp_va(vcpu);
> > +
> > + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > + host_ctxt->__hyp_running_vcpu = vcpu;
> > + guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + __sysreg_save_host_state(host_ctxt);
> > +
> > + __activate_traps(vcpu);
> > + __activate_vm(vcpu);
> > +
> > + __vgic_restore_state(vcpu);
> > + __timer_enable_traps(vcpu);
> > +
> > + /*
> > + * We must restore the 32-bit state before the sysregs, thanks
> > + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> > + */
> > + __sysreg32_restore_state(vcpu);
> > + __sysreg_restore_guest_state(guest_ctxt);
> > + __debug_switch_to_guest(vcpu);
> > +
> > + /* Jump in the fire! */
> > +again:
> > + exit_code = __guest_enter(vcpu, host_ctxt);
> > + /* And we're baaack! */
> > +
> > + if (fixup_guest_exit(vcpu, &exit_code))
> > + goto again;
> > +
> > + __sysreg_save_guest_state(guest_ctxt);
> > + __sysreg32_save_state(vcpu);
> > + __timer_disable_traps(vcpu);
> > + __vgic_save_state(vcpu);
> > +
> > + __deactivate_traps(vcpu);
> > + __deactivate_vm(vcpu);
> > +
> > + __sysreg_restore_host_state(host_ctxt);
> > +
> > + /*
> > + * This must come after restoring the host sysregs, since a non-VHE
> > + * system may enable SPE here and make use of the TTBRs.
> > + */
> > + __debug_switch_to_host(vcpu);
> > +
> > + return exit_code;
> > +}
> > +
> > +/* Switch to the guest for legacy non-VHE systems */
> > int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > {
> > struct kvm_cpu_context *host_ctxt;
> > diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> > index cf121b2..b11647a 100644
> > --- a/virt/kvm/arm/arm.c
> > +++ b/virt/kvm/arm/arm.c
> > @@ -706,7 +706,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > trace_kvm_entry(*vcpu_pc(vcpu));
> > guest_enter_irqoff();
> >
> > - ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
> > + if (has_vhe())
> > + ret = kvm_vcpu_run(vcpu);
> > + else
> > + ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
>
> So kvm_vcpu_run() is going to be VHE only, but the only way for people
> stepping through the code to know that is by seeing this callsite or by
> reading the comment above it. Everywhere else we make it much more clear
> with the _vhe and _nvhe suffixes. I wonder if we should keep that pattern
> by adding
>
> static int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> if (has_vhe())
> ret = kvm_vcpu_run_vhe(vcpu);
> else
> ret = kvm_call_hyp(__kvm_vcpu_run_nvhe, vcpu);
> }
>
> to virt/kvm/arm/arm.c and then calling that from here.
>
> I'm afraid the __ prefix is too ambiguous in the kernel to interpret
> __foo() as the hyp mode non-VHE version of foo(), the hyp mode VHE
> version.
>
Sure.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 13/37] KVM: arm64: Introduce VHE-specific kvm_vcpu_run
@ 2017-12-01 18:10 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-01 18:10 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 04:25:29PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:17PM +0200, Christoffer Dall wrote:
> > So far this is just a copy of the legacy non-VHE switch function, where
> > we only change the existing calls to has_vhe() in both the original and
> > new functions.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_asm.h | 4 +++
> > arch/arm64/include/asm/kvm_asm.h | 2 ++
> > arch/arm64/kvm/hyp/switch.c | 57 ++++++++++++++++++++++++++++++++++++++++
> > virt/kvm/arm/arm.c | 5 +++-
> > 4 files changed, 67 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
> > index 36dd296..1a7bc5f 100644
> > --- a/arch/arm/include/asm/kvm_asm.h
> > +++ b/arch/arm/include/asm/kvm_asm.h
> > @@ -70,8 +70,12 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
> >
> > extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
> >
> > +/* no VHE on 32-bit :( */
> > +static inline int kvm_vcpu_run(struct kvm_vcpu *vcpu) { return 0; }
> > +
> > extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> >
> > +
>
> stray new blank line
>
> > extern void __init_stage2_translation(void);
> >
> > extern u64 __vgic_v3_get_ich_vtr_el2(void);
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index 7e48a39..2eb5b23 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -57,6 +57,8 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);
> >
> > extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);
> >
> > +extern int kvm_vcpu_run(struct kvm_vcpu *vcpu);
> > +
> > extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> >
> > extern u64 __vgic_v3_get_ich_vtr_el2(void);
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index ed30af5..8a0f38f 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -319,6 +319,63 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> > return false;
> > }
> >
> > +/* Switch to the guest for VHE systems running in EL2 */
> > +int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm_cpu_context *host_ctxt;
> > + struct kvm_cpu_context *guest_ctxt;
> > + u64 exit_code;
> > +
> > + vcpu = kern_hyp_va(vcpu);
> > +
> > + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> > + host_ctxt->__hyp_running_vcpu = vcpu;
> > + guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + __sysreg_save_host_state(host_ctxt);
> > +
> > + __activate_traps(vcpu);
> > + __activate_vm(vcpu);
> > +
> > + __vgic_restore_state(vcpu);
> > + __timer_enable_traps(vcpu);
> > +
> > + /*
> > + * We must restore the 32-bit state before the sysregs, thanks
> > + * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> > + */
> > + __sysreg32_restore_state(vcpu);
> > + __sysreg_restore_guest_state(guest_ctxt);
> > + __debug_switch_to_guest(vcpu);
> > +
> > + /* Jump in the fire! */
> > +again:
> > + exit_code = __guest_enter(vcpu, host_ctxt);
> > + /* And we're baaack! */
> > +
> > + if (fixup_guest_exit(vcpu, &exit_code))
> > + goto again;
> > +
> > + __sysreg_save_guest_state(guest_ctxt);
> > + __sysreg32_save_state(vcpu);
> > + __timer_disable_traps(vcpu);
> > + __vgic_save_state(vcpu);
> > +
> > + __deactivate_traps(vcpu);
> > + __deactivate_vm(vcpu);
> > +
> > + __sysreg_restore_host_state(host_ctxt);
> > +
> > + /*
> > + * This must come after restoring the host sysregs, since a non-VHE
> > + * system may enable SPE here and make use of the TTBRs.
> > + */
> > + __debug_switch_to_host(vcpu);
> > +
> > + return exit_code;
> > +}
> > +
> > +/* Switch to the guest for legacy non-VHE systems */
> > int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > {
> > struct kvm_cpu_context *host_ctxt;
> > diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> > index cf121b2..b11647a 100644
> > --- a/virt/kvm/arm/arm.c
> > +++ b/virt/kvm/arm/arm.c
> > @@ -706,7 +706,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > trace_kvm_entry(*vcpu_pc(vcpu));
> > guest_enter_irqoff();
> >
> > - ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
> > + if (has_vhe())
> > + ret = kvm_vcpu_run(vcpu);
> > + else
> > + ret = kvm_call_hyp(__kvm_vcpu_run, vcpu);
>
> So kvm_vcpu_run() is going to be VHE only, but the only way for people
> stepping through the code to know that is by seeing this callsite or by
> reading the comment above it. Everywhere else we make it much more clear
> with the _vhe and _nvhe suffixes. I wonder if we should keep that pattern
> by adding
>
> static int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> {
> if (has_vhe())
> ret = kvm_vcpu_run_vhe(vcpu);
> else
> ret = kvm_call_hyp(__kvm_vcpu_run_nvhe, vcpu);
> }
>
> to virt/kvm/arm/arm.c and then calling that from here.
>
> I'm afraid the __ prefix is too ambiguous in the kernel to interpret
> __foo() as the hyp mode non-VHE version of foo(), the hyp mode VHE
> version.
>
Sure.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 14/37] KVM: arm64: Remove kern_hyp_va() use in VHE switch function
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
VHE kernels run completely in EL2 and therefore don't have a notion of
kernel and hyp addresses, they are all just kernel addresses. Therefore
don't call kern_hyp_va() in the VHE switch function.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 8a0f38f..b72dc66 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -326,9 +326,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
struct kvm_cpu_context *guest_ctxt;
u64 exit_code;
- vcpu = kern_hyp_va(vcpu);
-
- host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ host_ctxt = vcpu->arch.host_cpu_context;
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 14/37] KVM: arm64: Remove kern_hyp_va() use in VHE switch function
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
VHE kernels run completely in EL2 and therefore don't have a notion of
kernel and hyp addresses, they are all just kernel addresses. Therefore
don't call kern_hyp_va() in the VHE switch function.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 8a0f38f..b72dc66 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -326,9 +326,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
struct kvm_cpu_context *guest_ctxt;
u64 exit_code;
- vcpu = kern_hyp_va(vcpu);
-
- host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ host_ctxt = vcpu->arch.host_cpu_context;
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* Re: [PATCH 14/37] KVM: arm64: Remove kern_hyp_va() use in VHE switch function
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 16:07 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 16:07 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:18PM +0200, Christoffer Dall wrote:
> VHE kernels run completely in EL2 and therefore don't have a notion of
> kernel and hyp addresses, they are all just kernel addresses. Therefore
> don't call kern_hyp_va() in the VHE switch function.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 8a0f38f..b72dc66 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -326,9 +326,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> struct kvm_cpu_context *guest_ctxt;
> u64 exit_code;
>
> - vcpu = kern_hyp_va(vcpu);
> -
> - host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + host_ctxt = vcpu->arch.host_cpu_context;
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 14/37] KVM: arm64: Remove kern_hyp_va() use in VHE switch function
@ 2017-11-07 16:07 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 16:07 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:18PM +0200, Christoffer Dall wrote:
> VHE kernels run completely in EL2 and therefore don't have a notion of
> kernel and hyp addresses, they are all just kernel addresses. Therefore
> don't call kern_hyp_va() in the VHE switch function.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 8a0f38f..b72dc66 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -326,9 +326,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> struct kvm_cpu_context *guest_ctxt;
> u64 exit_code;
>
> - vcpu = kern_hyp_va(vcpu);
> -
> - host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + host_ctxt = vcpu->arch.host_cpu_context;
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 15/37] KVM: arm64: Don't deactivate VM on VHE systems
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
There is no need to reset the VTTBR to zero when exiting the guest on
VHE systems. VHE systems don't use stage 2 translations for the EL2&0
translation regime used by the host.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b72dc66..2cedf12 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -136,13 +136,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
write_sysreg(0, pmuserenr_el0);
}
-static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
+static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = kern_hyp_va(vcpu->kvm);
write_sysreg(kvm->arch.vttbr, vttbr_el2);
}
-static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
+static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
{
write_sysreg(0, vttbr_el2);
}
@@ -360,7 +360,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
- __deactivate_vm(vcpu);
__sysreg_restore_host_state(host_ctxt);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 15/37] KVM: arm64: Don't deactivate VM on VHE systems
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
There is no need to reset the VTTBR to zero when exiting the guest on
VHE systems. VHE systems don't use stage 2 translations for the EL2&0
translation regime used by the host.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b72dc66..2cedf12 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -136,13 +136,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
write_sysreg(0, pmuserenr_el0);
}
-static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
+static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = kern_hyp_va(vcpu->kvm);
write_sysreg(kvm->arch.vttbr, vttbr_el2);
}
-static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
+static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
{
write_sysreg(0, vttbr_el2);
}
@@ -360,7 +360,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
- __deactivate_vm(vcpu);
__sysreg_restore_host_state(host_ctxt);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 15/37] KVM: arm64: Don't deactivate VM on VHE systems
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 16:14 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 16:14 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:19PM +0200, Christoffer Dall wrote:
> There is no need to reset the VTTBR to zero when exiting the guest on
> VHE systems. VHE systems don't use stage 2 translations for the EL2&0
> translation regime used by the host.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index b72dc66..2cedf12 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -136,13 +136,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> write_sysreg(0, pmuserenr_el0);
> }
>
> -static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> +static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> {
> struct kvm *kvm = kern_hyp_va(vcpu->kvm);
Hmm, should we change __activate_vm to take a kvm pointer instead of a
vcpu, and then call the function from the VHE kvm_vcpu_run normally, but
from the non-VHE kvm_vcpu_run with kern_hyp_va(vcpu->kvm)? I only ask,
because it appears to be the last kern_hyp_va() that VHE code would still
invoke, at least considering the code in arch/arm64/kvm/hyp/switch.c
> write_sysreg(kvm->arch.vttbr, vttbr_el2);
> }
>
> -static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> +static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
Adding these 'inline' attributes is unrelated to this patch, and probably
unnecessary, as the compiler probably knew to do so anyway, but whatever.
> {
> write_sysreg(0, vttbr_el2);
> }
> @@ -360,7 +360,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __vgic_save_state(vcpu);
>
> __deactivate_traps(vcpu);
> - __deactivate_vm(vcpu);
>
> __sysreg_restore_host_state(host_ctxt);
>
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 15/37] KVM: arm64: Don't deactivate VM on VHE systems
@ 2017-11-07 16:14 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 16:14 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:19PM +0200, Christoffer Dall wrote:
> There is no need to reset the VTTBR to zero when exiting the guest on
> VHE systems. VHE systems don't use stage 2 translations for the EL2&0
> translation regime used by the host.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index b72dc66..2cedf12 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -136,13 +136,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> write_sysreg(0, pmuserenr_el0);
> }
>
> -static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> +static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> {
> struct kvm *kvm = kern_hyp_va(vcpu->kvm);
Hmm, should we change __activate_vm to take a kvm pointer instead of a
vcpu, and then call the function from the VHE kvm_vcpu_run normally, but
from the non-VHE kvm_vcpu_run with kern_hyp_va(vcpu->kvm)? I only ask,
because it appears to be the last kern_hyp_va() that VHE code would still
invoke, at least considering the code in arch/arm64/kvm/hyp/switch.c
> write_sysreg(kvm->arch.vttbr, vttbr_el2);
> }
>
> -static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> +static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
Adding these 'inline' attributes is unrelated to this patch, and probably
unnecessary, as the compiler probably knew to do so anyway, but whatever.
> {
> write_sysreg(0, vttbr_el2);
> }
> @@ -360,7 +360,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __vgic_save_state(vcpu);
>
> __deactivate_traps(vcpu);
> - __deactivate_vm(vcpu);
>
> __sysreg_restore_host_state(host_ctxt);
>
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 15/37] KVM: arm64: Don't deactivate VM on VHE systems
2017-11-07 16:14 ` Andrew Jones
@ 2017-12-03 19:27 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:27 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 05:14:50PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:19PM +0200, Christoffer Dall wrote:
> > There is no need to reset the VTTBR to zero when exiting the guest on
> > VHE systems. VHE systems don't use stage 2 translations for the EL2&0
> > translation regime used by the host.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index b72dc66..2cedf12 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -136,13 +136,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> > write_sysreg(0, pmuserenr_el0);
> > }
> >
> > -static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> > +static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> > {
> > struct kvm *kvm = kern_hyp_va(vcpu->kvm);
>
> Hmm, should we change __activate_vm to take a kvm pointer instead of a
> vcpu, and then call the function from the VHE kvm_vcpu_run normally, but
> from the non-VHE kvm_vcpu_run with kern_hyp_va(vcpu->kvm)? I only ask,
> because it appears to be the last kern_hyp_va() that VHE code would still
> invoke, at least considering the code in arch/arm64/kvm/hyp/switch.c
>
Yes, that's a reasonable point.
> > write_sysreg(kvm->arch.vttbr, vttbr_el2);
> > }
> >
> > -static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> > +static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>
> Adding these 'inline' attributes is unrelated to this patch, and probably
> unnecessary, as the compiler probably knew to do so anyway, but whatever.
>
I actually saw an additional function being inlined on the compiler I
used with the inline attribute, but not without, but I cannot reproduce
currently, so I'll drop these attributes.
> > {
> > write_sysreg(0, vttbr_el2);
> > }
> > @@ -360,7 +360,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > __vgic_save_state(vcpu);
> >
> > __deactivate_traps(vcpu);
> > - __deactivate_vm(vcpu);
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > --
> > 2.9.0
> >
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 15/37] KVM: arm64: Don't deactivate VM on VHE systems
@ 2017-12-03 19:27 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:27 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 05:14:50PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:19PM +0200, Christoffer Dall wrote:
> > There is no need to reset the VTTBR to zero when exiting the guest on
> > VHE systems. VHE systems don't use stage 2 translations for the EL2&0
> > translation regime used by the host.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index b72dc66..2cedf12 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -136,13 +136,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> > write_sysreg(0, pmuserenr_el0);
> > }
> >
> > -static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> > +static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> > {
> > struct kvm *kvm = kern_hyp_va(vcpu->kvm);
>
> Hmm, should we change __activate_vm to take a kvm pointer instead of a
> vcpu, and then call the function from the VHE kvm_vcpu_run normally, but
> from the non-VHE kvm_vcpu_run with kern_hyp_va(vcpu->kvm)? I only ask,
> because it appears to be the last kern_hyp_va() that VHE code would still
> invoke, at least considering the code in arch/arm64/kvm/hyp/switch.c
>
Yes, that's a reasonable point.
> > write_sysreg(kvm->arch.vttbr, vttbr_el2);
> > }
> >
> > -static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> > +static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
>
> Adding these 'inline' attributes is unrelated to this patch, and probably
> unnecessary, as the compiler probably knew to do so anyway, but whatever.
>
I actually saw an additional function being inlined on the compiler I
used with the inline attribute, but not without, but I cannot reproduce
currently, so I'll drop these attributes.
> > {
> > write_sysreg(0, vttbr_el2);
> > }
> > @@ -360,7 +360,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > __vgic_save_state(vcpu);
> >
> > __deactivate_traps(vcpu);
> > - __deactivate_vm(vcpu);
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > --
> > 2.9.0
> >
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 16/37] KVM: arm64: Remove noop calls to timer save/restore from VHE switch
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
The VHE switch function calls __timer_enable_traps and
__timer_disable_traps which don't do anything on VHE systems.
Therefore, simply remove these calls from the VHE switch function and
make the functions non-conditional as they are now only called from the
non-VHE switch path.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 2 --
virt/kvm/arm/hyp/timer-sr.c | 10 ++--------
2 files changed, 2 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 2cedf12..b98b73b 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -336,7 +336,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__activate_vm(vcpu);
__vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
/*
* We must restore the 32-bit state before the sysregs, thanks
@@ -356,7 +355,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_guest_state(guest_ctxt);
__sysreg32_save_state(vcpu);
- __timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index f398616..82c217e 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -53,16 +53,10 @@ void __hyp_text disable_el1_phys_timer_access(void)
void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)
{
- /*
- * We don't need to do this for VHE since the host kernel runs in EL2
- * with HCR_EL2.TGE ==1, which makes those bits have no impact.
- */
- if (!has_vhe())
- enable_el1_phys_timer_access();
+ enable_el1_phys_timer_access();
}
void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)
{
- if (!has_vhe())
- disable_el1_phys_timer_access();
+ disable_el1_phys_timer_access();
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 16/37] KVM: arm64: Remove noop calls to timer save/restore from VHE switch
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
The VHE switch function calls __timer_enable_traps and
__timer_disable_traps which don't do anything on VHE systems.
Therefore, simply remove these calls from the VHE switch function and
make the functions non-conditional as they are now only called from the
non-VHE switch path.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 2 --
virt/kvm/arm/hyp/timer-sr.c | 10 ++--------
2 files changed, 2 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 2cedf12..b98b73b 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -336,7 +336,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__activate_vm(vcpu);
__vgic_restore_state(vcpu);
- __timer_enable_traps(vcpu);
/*
* We must restore the 32-bit state before the sysregs, thanks
@@ -356,7 +355,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_guest_state(guest_ctxt);
__sysreg32_save_state(vcpu);
- __timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index f398616..82c217e 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -53,16 +53,10 @@ void __hyp_text disable_el1_phys_timer_access(void)
void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)
{
- /*
- * We don't need to do this for VHE since the host kernel runs in EL2
- * with HCR_EL2.TGE ==1, which makes those bits have no impact.
- */
- if (!has_vhe())
- enable_el1_phys_timer_access();
+ enable_el1_phys_timer_access();
}
void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)
{
- if (!has_vhe())
- disable_el1_phys_timer_access();
+ disable_el1_phys_timer_access();
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 16/37] KVM: arm64: Remove noop calls to timer save/restore from VHE switch
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-07 16:25 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 16:25 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:20PM +0200, Christoffer Dall wrote:
> The VHE switch function calls __timer_enable_traps and
> __timer_disable_traps which don't do anything on VHE systems.
> Therefore, simply remove these calls from the VHE switch function and
> make the functions non-conditional as they are now only called from the
> non-VHE switch path.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 2 --
> virt/kvm/arm/hyp/timer-sr.c | 10 ++--------
> 2 files changed, 2 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 2cedf12..b98b73b 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -336,7 +336,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __activate_vm(vcpu);
>
> __vgic_restore_state(vcpu);
> - __timer_enable_traps(vcpu);
>
> /*
> * We must restore the 32-bit state before the sysregs, thanks
> @@ -356,7 +355,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> - __timer_disable_traps(vcpu);
> __vgic_save_state(vcpu);
>
> __deactivate_traps(vcpu);
> diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
> index f398616..82c217e 100644
> --- a/virt/kvm/arm/hyp/timer-sr.c
> +++ b/virt/kvm/arm/hyp/timer-sr.c
> @@ -53,16 +53,10 @@ void __hyp_text disable_el1_phys_timer_access(void)
>
> void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)
> {
> - /*
> - * We don't need to do this for VHE since the host kernel runs in EL2
> - * with HCR_EL2.TGE ==1, which makes those bits have no impact.
> - */
I was about to suggest that we should move this comment, instead of remove
it, but it seems misleading anyway. We do call
enable/disable_el1_phys_timer_access on VHE, but at VCPU load/put time
instead of VM enter/exit time. So I guess removing it is best.
> - if (!has_vhe())
> - enable_el1_phys_timer_access();
> + enable_el1_phys_timer_access();
> }
>
> void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)
> {
> - if (!has_vhe())
> - disable_el1_phys_timer_access();
> + disable_el1_phys_timer_access();
> }
> --
> 2.9.0
>
Should we just call enable/disable_el1_phys_timer_access directly from
__kvm_vcpu_run for non-VHE and drop this extra function level?
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 16/37] KVM: arm64: Remove noop calls to timer save/restore from VHE switch
@ 2017-11-07 16:25 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-07 16:25 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:20PM +0200, Christoffer Dall wrote:
> The VHE switch function calls __timer_enable_traps and
> __timer_disable_traps which don't do anything on VHE systems.
> Therefore, simply remove these calls from the VHE switch function and
> make the functions non-conditional as they are now only called from the
> non-VHE switch path.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 2 --
> virt/kvm/arm/hyp/timer-sr.c | 10 ++--------
> 2 files changed, 2 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 2cedf12..b98b73b 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -336,7 +336,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __activate_vm(vcpu);
>
> __vgic_restore_state(vcpu);
> - __timer_enable_traps(vcpu);
>
> /*
> * We must restore the 32-bit state before the sysregs, thanks
> @@ -356,7 +355,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
>
> __sysreg_save_guest_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> - __timer_disable_traps(vcpu);
> __vgic_save_state(vcpu);
>
> __deactivate_traps(vcpu);
> diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
> index f398616..82c217e 100644
> --- a/virt/kvm/arm/hyp/timer-sr.c
> +++ b/virt/kvm/arm/hyp/timer-sr.c
> @@ -53,16 +53,10 @@ void __hyp_text disable_el1_phys_timer_access(void)
>
> void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)
> {
> - /*
> - * We don't need to do this for VHE since the host kernel runs in EL2
> - * with HCR_EL2.TGE ==1, which makes those bits have no impact.
> - */
I was about to suggest that we should move this comment, instead of remove
it, but it seems misleading anyway. We do call
enable/disable_el1_phys_timer_access on VHE, but at VCPU load/put time
instead of VM enter/exit time. So I guess removing it is best.
> - if (!has_vhe())
> - enable_el1_phys_timer_access();
> + enable_el1_phys_timer_access();
> }
>
> void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)
> {
> - if (!has_vhe())
> - disable_el1_phys_timer_access();
> + disable_el1_phys_timer_access();
> }
> --
> 2.9.0
>
Should we just call enable/disable_el1_phys_timer_access directly from
__kvm_vcpu_run for non-VHE and drop this extra function level?
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 16/37] KVM: arm64: Remove noop calls to timer save/restore from VHE switch
2017-11-07 16:25 ` Andrew Jones
@ 2017-12-03 19:27 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:27 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Tue, Nov 07, 2017 at 05:25:28PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:20PM +0200, Christoffer Dall wrote:
> > The VHE switch function calls __timer_enable_traps and
> > __timer_disable_traps which don't do anything on VHE systems.
> > Therefore, simply remove these calls from the VHE switch function and
> > make the functions non-conditional as they are now only called from the
> > non-VHE switch path.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 2 --
> > virt/kvm/arm/hyp/timer-sr.c | 10 ++--------
> > 2 files changed, 2 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 2cedf12..b98b73b 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -336,7 +336,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > __activate_vm(vcpu);
> >
> > __vgic_restore_state(vcpu);
> > - __timer_enable_traps(vcpu);
> >
> > /*
> > * We must restore the 32-bit state before the sysregs, thanks
> > @@ -356,7 +355,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_save_guest_state(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > - __timer_disable_traps(vcpu);
> > __vgic_save_state(vcpu);
> >
> > __deactivate_traps(vcpu);
> > diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
> > index f398616..82c217e 100644
> > --- a/virt/kvm/arm/hyp/timer-sr.c
> > +++ b/virt/kvm/arm/hyp/timer-sr.c
> > @@ -53,16 +53,10 @@ void __hyp_text disable_el1_phys_timer_access(void)
> >
> > void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)
> > {
> > - /*
> > - * We don't need to do this for VHE since the host kernel runs in EL2
> > - * with HCR_EL2.TGE ==1, which makes those bits have no impact.
> > - */
>
> I was about to suggest that we should move this comment, instead of remove
> it, but it seems misleading anyway. We do call
> enable/disable_el1_phys_timer_access on VHE, but at VCPU load/put time
> instead of VM enter/exit time. So I guess removing it is best.
>
That was actually a bug in the timer series, which has now been fixed.
In any case, I don't think the comment is necessary as it's already
explained in kvm_timer_init_vhe() in arch_timer.c.
> > - if (!has_vhe())
> > - enable_el1_phys_timer_access();
> > + enable_el1_phys_timer_access();
> > }
> >
> > void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)
> > {
> > - if (!has_vhe())
> > - disable_el1_phys_timer_access();
> > + disable_el1_phys_timer_access();
> > }
> > --
> > 2.9.0
> >
>
> Should we just call enable/disable_el1_phys_timer_access directly from
> __kvm_vcpu_run for non-VHE and drop this extra function level?
>
I don't think there's a problem with this indirection and we could
imagine having more traps in the future.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 16/37] KVM: arm64: Remove noop calls to timer save/restore from VHE switch
@ 2017-12-03 19:27 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:27 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 07, 2017 at 05:25:28PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:20PM +0200, Christoffer Dall wrote:
> > The VHE switch function calls __timer_enable_traps and
> > __timer_disable_traps which don't do anything on VHE systems.
> > Therefore, simply remove these calls from the VHE switch function and
> > make the functions non-conditional as they are now only called from the
> > non-VHE switch path.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 2 --
> > virt/kvm/arm/hyp/timer-sr.c | 10 ++--------
> > 2 files changed, 2 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 2cedf12..b98b73b 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -336,7 +336,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > __activate_vm(vcpu);
> >
> > __vgic_restore_state(vcpu);
> > - __timer_enable_traps(vcpu);
> >
> > /*
> > * We must restore the 32-bit state before the sysregs, thanks
> > @@ -356,7 +355,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_save_guest_state(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > - __timer_disable_traps(vcpu);
> > __vgic_save_state(vcpu);
> >
> > __deactivate_traps(vcpu);
> > diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
> > index f398616..82c217e 100644
> > --- a/virt/kvm/arm/hyp/timer-sr.c
> > +++ b/virt/kvm/arm/hyp/timer-sr.c
> > @@ -53,16 +53,10 @@ void __hyp_text disable_el1_phys_timer_access(void)
> >
> > void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)
> > {
> > - /*
> > - * We don't need to do this for VHE since the host kernel runs in EL2
> > - * with HCR_EL2.TGE ==1, which makes those bits have no impact.
> > - */
>
> I was about to suggest that we should move this comment, instead of remove
> it, but it seems misleading anyway. We do call
> enable/disable_el1_phys_timer_access on VHE, but at VCPU load/put time
> instead of VM enter/exit time. So I guess removing it is best.
>
That was actually a bug in the timer series, which has now been fixed.
In any case, I don't think the comment is necessary as it's already
explained in kvm_timer_init_vhe() in arch_timer.c.
> > - if (!has_vhe())
> > - enable_el1_phys_timer_access();
> > + enable_el1_phys_timer_access();
> > }
> >
> > void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)
> > {
> > - if (!has_vhe())
> > - disable_el1_phys_timer_access();
> > + disable_el1_phys_timer_access();
> > }
> > --
> > 2.9.0
> >
>
> Should we just call enable/disable_el1_phys_timer_access directly from
> __kvm_vcpu_run for non-VHE and drop this extra function level?
>
I don't think there's a problem with this indirection and we could
imagine having more traps in the future.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 17/37] KVM: arm64: Move userspace system registers into separate function
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
There's a semantic difference between the EL1 registers that control
operation of a kernel running in EL1 and EL1 registers that only control
userspace execution in EL0. Since we can defer saving/restoring the
latter, move them into their own function.
We also take this chance to rename the function saving/restoring the
remaining system register to make it clear this function deals with
the EL1 system registers.
No functional change.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/sysreg-sr.c | 34 +++++++++++++++++++++++-----------
1 file changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index c4a3714..193c2b0 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -34,14 +34,18 @@ static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
{
+ ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
+ ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
+}
+
+static void __hyp_text __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
+{
ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
- ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
- ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
}
-static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
+static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
{
ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
@@ -70,31 +74,37 @@ static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
}
static hyp_alternate_select(__sysreg_call_save_host_state,
- __sysreg_save_state, __sysreg_do_nothing,
+ __sysreg_save_el1_state, __sysreg_do_nothing,
ARM64_HAS_VIRT_HOST_EXTN);
void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
{
__sysreg_call_save_host_state()(ctxt);
__sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
}
void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_save_state(ctxt);
+ __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
}
static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
{
- write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
- write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
- write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
}
-static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
+static void __hyp_text __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
+{
+ write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
+ write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
+ write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
+}
+
+static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
{
write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
@@ -123,19 +133,21 @@ static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
}
static hyp_alternate_select(__sysreg_call_restore_host_state,
- __sysreg_restore_state, __sysreg_do_nothing,
+ __sysreg_restore_el1_state, __sysreg_do_nothing,
ARM64_HAS_VIRT_HOST_EXTN);
void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
{
__sysreg_call_restore_host_state()(ctxt);
__sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
}
void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_state(ctxt);
+ __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
}
static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 17/37] KVM: arm64: Move userspace system registers into separate function
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
There's a semantic difference between the EL1 registers that control
operation of a kernel running in EL1 and EL1 registers that only control
userspace execution in EL0. Since we can defer saving/restoring the
latter, move them into their own function.
We also take this chance to rename the function saving/restoring the
remaining system register to make it clear this function deals with
the EL1 system registers.
No functional change.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/sysreg-sr.c | 34 +++++++++++++++++++++++-----------
1 file changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index c4a3714..193c2b0 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -34,14 +34,18 @@ static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
{
+ ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
+ ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
+}
+
+static void __hyp_text __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
+{
ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
- ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
- ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
}
-static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
+static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
{
ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
@@ -70,31 +74,37 @@ static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
}
static hyp_alternate_select(__sysreg_call_save_host_state,
- __sysreg_save_state, __sysreg_do_nothing,
+ __sysreg_save_el1_state, __sysreg_do_nothing,
ARM64_HAS_VIRT_HOST_EXTN);
void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
{
__sysreg_call_save_host_state()(ctxt);
__sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
}
void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_save_state(ctxt);
+ __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
}
static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
{
- write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
- write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
- write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
}
-static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
+static void __hyp_text __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
+{
+ write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
+ write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
+ write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
+}
+
+static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
{
write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
@@ -123,19 +133,21 @@ static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
}
static hyp_alternate_select(__sysreg_call_restore_host_state,
- __sysreg_restore_state, __sysreg_do_nothing,
+ __sysreg_restore_el1_state, __sysreg_do_nothing,
ARM64_HAS_VIRT_HOST_EXTN);
void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
{
__sysreg_call_restore_host_state()(ctxt);
__sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
}
void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_state(ctxt);
+ __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
}
static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 17/37] KVM: arm64: Move userspace system registers into separate function
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-08 9:32 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 9:32 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:21PM +0200, Christoffer Dall wrote:
> There's a semantic difference between the EL1 registers that control
> operation of a kernel running in EL1 and EL1 registers that only control
> userspace execution in EL0. Since we can defer saving/restoring the
> latter, move them into their own function.
>
> We also take this chance to rename the function saving/restoring the
> remaining system register to make it clear this function deals with
> the EL1 system registers.
>
> No functional change.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/sysreg-sr.c | 34 +++++++++++++++++++++++-----------
> 1 file changed, 23 insertions(+), 11 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index c4a3714..193c2b0 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -34,14 +34,18 @@ static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
>
> static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
> {
> + ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> + ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
Maybe a comment stating that sp_el0 is here, instead of down in user
state, because arm64 Linux chooses to use it for the 'current' pointer.
At least that's why I think it's been promoted to common state, but maybe
there was a different reason I missed.
> +}
> +
> +static void __hyp_text __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
> +{
> ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
> ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
> ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
> - ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> - ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
> }
>
> -static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> +static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> {
> ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
> ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
> @@ -70,31 +74,37 @@ static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> }
>
> static hyp_alternate_select(__sysreg_call_save_host_state,
> - __sysreg_save_state, __sysreg_do_nothing,
> + __sysreg_save_el1_state, __sysreg_do_nothing,
> ARM64_HAS_VIRT_HOST_EXTN);
>
> void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
> {
> __sysreg_call_save_host_state()(ctxt);
> __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> }
>
> void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
> {
> - __sysreg_save_state(ctxt);
> + __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> }
>
> static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> {
> - write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> - write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> - write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
> write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
> }
>
> -static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> +static void __hyp_text __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
> +{
> + write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> + write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> + write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> +}
> +
> +static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> {
> write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
> write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
> @@ -123,19 +133,21 @@ static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> }
>
> static hyp_alternate_select(__sysreg_call_restore_host_state,
> - __sysreg_restore_state, __sysreg_do_nothing,
> + __sysreg_restore_el1_state, __sysreg_do_nothing,
> ARM64_HAS_VIRT_HOST_EXTN);
>
> void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
> {
> __sysreg_call_restore_host_state()(ctxt);
> __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> }
>
> void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> {
> - __sysreg_restore_state(ctxt);
> + __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> }
>
> static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> --
> 2.9.0
>
Otherwise
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 17/37] KVM: arm64: Move userspace system registers into separate function
@ 2017-11-08 9:32 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 9:32 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:21PM +0200, Christoffer Dall wrote:
> There's a semantic difference between the EL1 registers that control
> operation of a kernel running in EL1 and EL1 registers that only control
> userspace execution in EL0. Since we can defer saving/restoring the
> latter, move them into their own function.
>
> We also take this chance to rename the function saving/restoring the
> remaining system register to make it clear this function deals with
> the EL1 system registers.
>
> No functional change.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/sysreg-sr.c | 34 +++++++++++++++++++++++-----------
> 1 file changed, 23 insertions(+), 11 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index c4a3714..193c2b0 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -34,14 +34,18 @@ static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
>
> static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
> {
> + ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> + ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
Maybe a comment stating that sp_el0 is here, instead of down in user
state, because arm64 Linux chooses to use it for the 'current' pointer.
At least that's why I think it's been promoted to common state, but maybe
there was a different reason I missed.
> +}
> +
> +static void __hyp_text __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
> +{
> ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
> ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
> ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
> - ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> - ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
> }
>
> -static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> +static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> {
> ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
> ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
> @@ -70,31 +74,37 @@ static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> }
>
> static hyp_alternate_select(__sysreg_call_save_host_state,
> - __sysreg_save_state, __sysreg_do_nothing,
> + __sysreg_save_el1_state, __sysreg_do_nothing,
> ARM64_HAS_VIRT_HOST_EXTN);
>
> void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
> {
> __sysreg_call_save_host_state()(ctxt);
> __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> }
>
> void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
> {
> - __sysreg_save_state(ctxt);
> + __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> }
>
> static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> {
> - write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> - write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> - write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
> write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
> }
>
> -static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> +static void __hyp_text __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
> +{
> + write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> + write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> + write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> +}
> +
> +static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> {
> write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
> write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
> @@ -123,19 +133,21 @@ static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> }
>
> static hyp_alternate_select(__sysreg_call_restore_host_state,
> - __sysreg_restore_state, __sysreg_do_nothing,
> + __sysreg_restore_el1_state, __sysreg_do_nothing,
> ARM64_HAS_VIRT_HOST_EXTN);
>
> void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
> {
> __sysreg_call_restore_host_state()(ctxt);
> __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> }
>
> void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> {
> - __sysreg_restore_state(ctxt);
> + __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> }
>
> static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> --
> 2.9.0
>
Otherwise
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 17/37] KVM: arm64: Move userspace system registers into separate function
2017-11-08 9:32 ` Andrew Jones
@ 2017-12-03 19:36 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:36 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Wed, Nov 08, 2017 at 10:32:30AM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:21PM +0200, Christoffer Dall wrote:
> > There's a semantic difference between the EL1 registers that control
> > operation of a kernel running in EL1 and EL1 registers that only control
> > userspace execution in EL0. Since we can defer saving/restoring the
> > latter, move them into their own function.
> >
> > We also take this chance to rename the function saving/restoring the
> > remaining system register to make it clear this function deals with
> > the EL1 system registers.
> >
> > No functional change.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/sysreg-sr.c | 34 +++++++++++++++++++++++-----------
> > 1 file changed, 23 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index c4a3714..193c2b0 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -34,14 +34,18 @@ static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
> >
> > static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
> > {
> > + ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> > + ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
>
> Maybe a comment stating that sp_el0 is here, instead of down in user
> state, because arm64 Linux chooses to use it for the 'current' pointer.
> At least that's why I think it's been promoted to common state, but maybe
> there was a different reason I missed.
>
That is the reason. I can add a comment.
> > +}
> > +
> > +static void __hyp_text __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
> > +{
> > ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
> > ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
> > ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
> > - ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> > - ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
> > }
> >
> > -static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> > {
> > ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
> > ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
> > @@ -70,31 +74,37 @@ static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> > }
> >
> > static hyp_alternate_select(__sysreg_call_save_host_state,
> > - __sysreg_save_state, __sysreg_do_nothing,
> > + __sysreg_save_el1_state, __sysreg_do_nothing,
> > ARM64_HAS_VIRT_HOST_EXTN);
> >
> > void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_call_save_host_state()(ctxt);
> > __sysreg_save_common_state(ctxt);
> > + __sysreg_save_user_state(ctxt);
> > }
> >
> > void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
> > {
> > - __sysreg_save_state(ctxt);
> > + __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > + __sysreg_save_user_state(ctxt);
> > }
> >
> > static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> > {
> > - write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> > - write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> > - write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> > write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
> > write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
> > }
> >
> > -static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
> > +{
> > + write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> > + write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> > + write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> > +}
> > +
> > +static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> > {
> > write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
> > write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
> > @@ -123,19 +133,21 @@ static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> > }
> >
> > static hyp_alternate_select(__sysreg_call_restore_host_state,
> > - __sysreg_restore_state, __sysreg_do_nothing,
> > + __sysreg_restore_el1_state, __sysreg_do_nothing,
> > ARM64_HAS_VIRT_HOST_EXTN);
> >
> > void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_call_restore_host_state()(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > + __sysreg_restore_user_state(ctxt);
> > }
> >
> > void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> > {
> > - __sysreg_restore_state(ctxt);
> > + __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > + __sysreg_restore_user_state(ctxt);
> > }
> >
> > static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > --
> > 2.9.0
> >
>
> Otherwise
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 17/37] KVM: arm64: Move userspace system registers into separate function
@ 2017-12-03 19:36 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:36 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 08, 2017 at 10:32:30AM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:21PM +0200, Christoffer Dall wrote:
> > There's a semantic difference between the EL1 registers that control
> > operation of a kernel running in EL1 and EL1 registers that only control
> > userspace execution in EL0. Since we can defer saving/restoring the
> > latter, move them into their own function.
> >
> > We also take this chance to rename the function saving/restoring the
> > remaining system register to make it clear this function deals with
> > the EL1 system registers.
> >
> > No functional change.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/sysreg-sr.c | 34 +++++++++++++++++++++++-----------
> > 1 file changed, 23 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index c4a3714..193c2b0 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -34,14 +34,18 @@ static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
> >
> > static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
> > {
> > + ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> > + ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
>
> Maybe a comment stating that sp_el0 is here, instead of down in user
> state, because arm64 Linux chooses to use it for the 'current' pointer.
> At least that's why I think it's been promoted to common state, but maybe
> there was a different reason I missed.
>
That is the reason. I can add a comment.
> > +}
> > +
> > +static void __hyp_text __sysreg_save_user_state(struct kvm_cpu_context *ctxt)
> > +{
> > ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
> > ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
> > ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
> > - ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> > - ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
> > }
> >
> > -static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> > {
> > ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
> > ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
> > @@ -70,31 +74,37 @@ static void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> > }
> >
> > static hyp_alternate_select(__sysreg_call_save_host_state,
> > - __sysreg_save_state, __sysreg_do_nothing,
> > + __sysreg_save_el1_state, __sysreg_do_nothing,
> > ARM64_HAS_VIRT_HOST_EXTN);
> >
> > void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_call_save_host_state()(ctxt);
> > __sysreg_save_common_state(ctxt);
> > + __sysreg_save_user_state(ctxt);
> > }
> >
> > void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
> > {
> > - __sysreg_save_state(ctxt);
> > + __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > + __sysreg_save_user_state(ctxt);
> > }
> >
> > static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> > {
> > - write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> > - write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> > - write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> > write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
> > write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
> > }
> >
> > -static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> > +static void __hyp_text __sysreg_restore_user_state(struct kvm_cpu_context *ctxt)
> > +{
> > + write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> > + write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> > + write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> > +}
> > +
> > +static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> > {
> > write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
> > write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
> > @@ -123,19 +133,21 @@ static void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> > }
> >
> > static hyp_alternate_select(__sysreg_call_restore_host_state,
> > - __sysreg_restore_state, __sysreg_do_nothing,
> > + __sysreg_restore_el1_state, __sysreg_do_nothing,
> > ARM64_HAS_VIRT_HOST_EXTN);
> >
> > void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_call_restore_host_state()(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > + __sysreg_restore_user_state(ctxt);
> > }
> >
> > void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> > {
> > - __sysreg_restore_state(ctxt);
> > + __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > + __sysreg_restore_user_state(ctxt);
> > }
> >
> > static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > --
> > 2.9.0
> >
>
> Otherwise
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 18/37] KVM: arm64: Rewrite sysreg alternatives to static keys
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
As we are about to move calls around in the sysreg save/restore logic,
let's first rewrite the alternative function callers, because it is
going to make the next patches much easier to read.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/sysreg-sr.c | 17 ++++-------------
1 file changed, 4 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 193c2b0..b55c4ad 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -22,9 +22,6 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
-/* Yes, this does nothing, on purpose */
-static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
-
/*
* Non-VHE: Both host and guest must save everything.
*
@@ -73,13 +70,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
-static hyp_alternate_select(__sysreg_call_save_host_state,
- __sysreg_save_el1_state, __sysreg_do_nothing,
- ARM64_HAS_VIRT_HOST_EXTN);
-
void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_call_save_host_state()(ctxt);
+ if (!has_vhe())
+ __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
}
@@ -132,13 +126,10 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
-static hyp_alternate_select(__sysreg_call_restore_host_state,
- __sysreg_restore_el1_state, __sysreg_do_nothing,
- ARM64_HAS_VIRT_HOST_EXTN);
-
void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_call_restore_host_state()(ctxt);
+ if (!has_vhe())
+ __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 18/37] KVM: arm64: Rewrite sysreg alternatives to static keys
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
As we are about to move calls around in the sysreg save/restore logic,
let's first rewrite the alternative function callers, because it is
going to make the next patches much easier to read.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/sysreg-sr.c | 17 ++++-------------
1 file changed, 4 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 193c2b0..b55c4ad 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -22,9 +22,6 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
-/* Yes, this does nothing, on purpose */
-static void __hyp_text __sysreg_do_nothing(struct kvm_cpu_context *ctxt) { }
-
/*
* Non-VHE: Both host and guest must save everything.
*
@@ -73,13 +70,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
-static hyp_alternate_select(__sysreg_call_save_host_state,
- __sysreg_save_el1_state, __sysreg_do_nothing,
- ARM64_HAS_VIRT_HOST_EXTN);
-
void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_call_save_host_state()(ctxt);
+ if (!has_vhe())
+ __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
}
@@ -132,13 +126,10 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
-static hyp_alternate_select(__sysreg_call_restore_host_state,
- __sysreg_restore_el1_state, __sysreg_do_nothing,
- ARM64_HAS_VIRT_HOST_EXTN);
-
void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
{
- __sysreg_call_restore_host_state()(ctxt);
+ if (!has_vhe())
+ __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 19/37] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
As we are about to handle system registers quite differently between VHE
and non-VHE systems. In preparation for that, we need to split some of
the handling functions between VHE and non-VHE functionality.
For now, we simply copy the non-VHE functions, but we do change the use
of static keys for VHE and non-VHE functionality now that we have
separate functions.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 12 ++++++++----
arch/arm64/kvm/hyp/switch.c | 20 ++++++++++----------
arch/arm64/kvm/hyp/sysreg-sr.c | 40 ++++++++++++++++++++++++++++++++--------
3 files changed, 50 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index a0e5a70..998152d 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -132,10 +132,14 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-void __sysreg_save_host_state(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_host_state(struct kvm_cpu_context *ctxt);
-void __sysreg_save_guest_state(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
+void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
+void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
+void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
+void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
+void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt);
void __sysreg32_save_state(struct kvm_vcpu *vcpu);
void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b98b73b..7c4d430 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -330,7 +330,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
- __sysreg_save_host_state(host_ctxt);
+ sysreg_save_host_state_vhe(host_ctxt);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -342,7 +342,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
*/
__sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state(guest_ctxt);
+ sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
/* Jump in the fire! */
@@ -353,13 +353,13 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (fixup_guest_exit(vcpu, &exit_code))
goto again;
- __sysreg_save_guest_state(guest_ctxt);
+ sysreg_save_guest_state_vhe(guest_ctxt);
__sysreg32_save_state(vcpu);
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
- __sysreg_restore_host_state(host_ctxt);
+ sysreg_restore_host_state_vhe(host_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
@@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
- __sysreg_save_host_state(host_ctxt);
+ __sysreg_save_host_state_nvhe(host_ctxt);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
*/
__sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state(guest_ctxt);
+ __sysreg_restore_guest_state_nvhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
/* Jump in the fire! */
@@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (fixup_guest_exit(vcpu, &exit_code))
goto again;
- __sysreg_save_guest_state(guest_ctxt);
+ __sysreg_save_guest_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
@@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state(host_ctxt);
+ __sysreg_restore_host_state_nvhe(host_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
@@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
__timer_disable_traps(vcpu);
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state(__host_ctxt);
+ __sysreg_restore_host_state_nvhe(__host_ctxt);
}
/*
@@ -463,7 +463,7 @@ static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
vcpu = host_ctxt->__hyp_running_vcpu;
__deactivate_traps_vhe();
- __sysreg_restore_host_state(host_ctxt);
+ sysreg_restore_host_state_vhe(host_ctxt);
panic(__hyp_panic_string,
spsr, elr,
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index b55c4ad..d8c42de 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -70,15 +70,27 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
-void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_save_el1_state(ctxt);
+ __sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
+}
+
+void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_save_el1_state(ctxt);
+ __sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
+}
+
+void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
{
- if (!has_vhe())
- __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
}
-void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
+void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
@@ -126,15 +138,27 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
-void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
+}
+
+void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
+}
+
+void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
{
- if (!has_vhe())
- __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
}
-void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
+void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 19/37] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
As we are about to handle system registers quite differently between VHE
and non-VHE systems. In preparation for that, we need to split some of
the handling functions between VHE and non-VHE functionality.
For now, we simply copy the non-VHE functions, but we do change the use
of static keys for VHE and non-VHE functionality now that we have
separate functions.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 12 ++++++++----
arch/arm64/kvm/hyp/switch.c | 20 ++++++++++----------
arch/arm64/kvm/hyp/sysreg-sr.c | 40 ++++++++++++++++++++++++++++++++--------
3 files changed, 50 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index a0e5a70..998152d 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -132,10 +132,14 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-void __sysreg_save_host_state(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_host_state(struct kvm_cpu_context *ctxt);
-void __sysreg_save_guest_state(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
+void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
+void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
+void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
+void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
+void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt);
void __sysreg32_save_state(struct kvm_vcpu *vcpu);
void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b98b73b..7c4d430 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -330,7 +330,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
- __sysreg_save_host_state(host_ctxt);
+ sysreg_save_host_state_vhe(host_ctxt);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -342,7 +342,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
*/
__sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state(guest_ctxt);
+ sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
/* Jump in the fire! */
@@ -353,13 +353,13 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (fixup_guest_exit(vcpu, &exit_code))
goto again;
- __sysreg_save_guest_state(guest_ctxt);
+ sysreg_save_guest_state_vhe(guest_ctxt);
__sysreg32_save_state(vcpu);
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
- __sysreg_restore_host_state(host_ctxt);
+ sysreg_restore_host_state_vhe(host_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
@@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
- __sysreg_save_host_state(host_ctxt);
+ __sysreg_save_host_state_nvhe(host_ctxt);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
*/
__sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state(guest_ctxt);
+ __sysreg_restore_guest_state_nvhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
/* Jump in the fire! */
@@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (fixup_guest_exit(vcpu, &exit_code))
goto again;
- __sysreg_save_guest_state(guest_ctxt);
+ __sysreg_save_guest_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
@@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state(host_ctxt);
+ __sysreg_restore_host_state_nvhe(host_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
@@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
__timer_disable_traps(vcpu);
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state(__host_ctxt);
+ __sysreg_restore_host_state_nvhe(__host_ctxt);
}
/*
@@ -463,7 +463,7 @@ static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
vcpu = host_ctxt->__hyp_running_vcpu;
__deactivate_traps_vhe();
- __sysreg_restore_host_state(host_ctxt);
+ sysreg_restore_host_state_vhe(host_ctxt);
panic(__hyp_panic_string,
spsr, elr,
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index b55c4ad..d8c42de 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -70,15 +70,27 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
-void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_save_el1_state(ctxt);
+ __sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
+}
+
+void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_save_el1_state(ctxt);
+ __sysreg_save_common_state(ctxt);
+ __sysreg_save_user_state(ctxt);
+}
+
+void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
{
- if (!has_vhe())
- __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
}
-void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
+void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
@@ -126,15 +138,27 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
-void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
+}
+
+void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+{
+ __sysreg_restore_el1_state(ctxt);
+ __sysreg_restore_common_state(ctxt);
+ __sysreg_restore_user_state(ctxt);
+}
+
+void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
{
- if (!has_vhe())
- __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
}
-void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
+void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 19/37] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-08 10:31 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 10:31 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:23PM +0200, Christoffer Dall wrote:
> As we are about to handle system registers quite differently between VHE
> and non-VHE systems. In preparation for that, we need to split some of
> the handling functions between VHE and non-VHE functionality.
>
> For now, we simply copy the non-VHE functions, but we do change the use
> of static keys for VHE and non-VHE functionality now that we have
> separate functions.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_hyp.h | 12 ++++++++----
> arch/arm64/kvm/hyp/switch.c | 20 ++++++++++----------
> arch/arm64/kvm/hyp/sysreg-sr.c | 40 ++++++++++++++++++++++++++++++++--------
> 3 files changed, 50 insertions(+), 22 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index a0e5a70..998152d 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -132,10 +132,14 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> void __timer_enable_traps(struct kvm_vcpu *vcpu);
> void __timer_disable_traps(struct kvm_vcpu *vcpu);
>
> -void __sysreg_save_host_state(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_host_state(struct kvm_cpu_context *ctxt);
> -void __sysreg_save_guest_state(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> +void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> +void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
> +void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
> +void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
> +void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt);
> void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index b98b73b..7c4d430 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -330,7 +330,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> - __sysreg_save_host_state(host_ctxt);
> + sysreg_save_host_state_vhe(host_ctxt);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -342,7 +342,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> */
> __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state(guest_ctxt);
> + sysreg_restore_guest_state_vhe(guest_ctxt);
> __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> @@ -353,13 +353,13 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> if (fixup_guest_exit(vcpu, &exit_code))
> goto again;
>
> - __sysreg_save_guest_state(guest_ctxt);
> + sysreg_save_guest_state_vhe(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __vgic_save_state(vcpu);
>
> __deactivate_traps(vcpu);
>
> - __sysreg_restore_host_state(host_ctxt);
> + sysreg_restore_host_state_vhe(host_ctxt);
>
> /*
> * This must come after restoring the host sysregs, since a non-VHE
This comment is a carry over from the non-VHE function. It's a bit
confusing in a VHE-only function, so it should probably be removed,
perhaps with its own patch though.
> @@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> - __sysreg_save_host_state(host_ctxt);
> + __sysreg_save_host_state_nvhe(host_ctxt);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> */
> __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state(guest_ctxt);
> + __sysreg_restore_guest_state_nvhe(guest_ctxt);
> __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> @@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> if (fixup_guest_exit(vcpu, &exit_code))
> goto again;
>
> - __sysreg_save_guest_state(guest_ctxt);
> + __sysreg_save_guest_state_nvhe(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> __vgic_save_state(vcpu);
> @@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
>
> - __sysreg_restore_host_state(host_ctxt);
> + __sysreg_restore_host_state_nvhe(host_ctxt);
>
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> @@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
> - __sysreg_restore_host_state(__host_ctxt);
> + __sysreg_restore_host_state_nvhe(__host_ctxt);
> }
>
> /*
> @@ -463,7 +463,7 @@ static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> vcpu = host_ctxt->__hyp_running_vcpu;
>
> __deactivate_traps_vhe();
> - __sysreg_restore_host_state(host_ctxt);
> + sysreg_restore_host_state_vhe(host_ctxt);
>
> panic(__hyp_panic_string,
> spsr, elr,
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index b55c4ad..d8c42de 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -70,15 +70,27 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> }
>
> -void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_save_el1_state(ctxt);
> + __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> +}
> +
> +void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_save_el1_state(ctxt);
> + __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> +}
> +
> +void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
> {
> - if (!has_vhe())
> - __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> __sysreg_save_user_state(ctxt);
> }
>
> -void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
> +void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> @@ -126,15 +138,27 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> }
>
> -void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_restore_el1_state(ctxt);
> + __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> +}
> +
> +void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_restore_el1_state(ctxt);
> + __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> +}
> +
> +void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
> {
> - if (!has_vhe())
> - __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> __sysreg_restore_user_state(ctxt);
> }
>
> -void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> +void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 19/37] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions
@ 2017-11-08 10:31 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 10:31 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:23PM +0200, Christoffer Dall wrote:
> As we are about to handle system registers quite differently between VHE
> and non-VHE systems. In preparation for that, we need to split some of
> the handling functions between VHE and non-VHE functionality.
>
> For now, we simply copy the non-VHE functions, but we do change the use
> of static keys for VHE and non-VHE functionality now that we have
> separate functions.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_hyp.h | 12 ++++++++----
> arch/arm64/kvm/hyp/switch.c | 20 ++++++++++----------
> arch/arm64/kvm/hyp/sysreg-sr.c | 40 ++++++++++++++++++++++++++++++++--------
> 3 files changed, 50 insertions(+), 22 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index a0e5a70..998152d 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -132,10 +132,14 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> void __timer_enable_traps(struct kvm_vcpu *vcpu);
> void __timer_disable_traps(struct kvm_vcpu *vcpu);
>
> -void __sysreg_save_host_state(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_host_state(struct kvm_cpu_context *ctxt);
> -void __sysreg_save_guest_state(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt);
> +void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> +void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
> +void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
> +void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
> +void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt);
> void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index b98b73b..7c4d430 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -330,7 +330,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> - __sysreg_save_host_state(host_ctxt);
> + sysreg_save_host_state_vhe(host_ctxt);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -342,7 +342,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> */
> __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state(guest_ctxt);
> + sysreg_restore_guest_state_vhe(guest_ctxt);
> __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> @@ -353,13 +353,13 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
> if (fixup_guest_exit(vcpu, &exit_code))
> goto again;
>
> - __sysreg_save_guest_state(guest_ctxt);
> + sysreg_save_guest_state_vhe(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __vgic_save_state(vcpu);
>
> __deactivate_traps(vcpu);
>
> - __sysreg_restore_host_state(host_ctxt);
> + sysreg_restore_host_state_vhe(host_ctxt);
>
> /*
> * This must come after restoring the host sysregs, since a non-VHE
This comment is a carry over from the non-VHE function. It's a bit
confusing in a VHE-only function, so it should probably be removed,
perhaps with its own patch though.
> @@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> - __sysreg_save_host_state(host_ctxt);
> + __sysreg_save_host_state_nvhe(host_ctxt);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> */
> __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state(guest_ctxt);
> + __sysreg_restore_guest_state_nvhe(guest_ctxt);
> __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> @@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> if (fixup_guest_exit(vcpu, &exit_code))
> goto again;
>
> - __sysreg_save_guest_state(guest_ctxt);
> + __sysreg_save_guest_state_nvhe(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> __vgic_save_state(vcpu);
> @@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
>
> - __sysreg_restore_host_state(host_ctxt);
> + __sysreg_restore_host_state_nvhe(host_ctxt);
>
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> @@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
> - __sysreg_restore_host_state(__host_ctxt);
> + __sysreg_restore_host_state_nvhe(__host_ctxt);
> }
>
> /*
> @@ -463,7 +463,7 @@ static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
> vcpu = host_ctxt->__hyp_running_vcpu;
>
> __deactivate_traps_vhe();
> - __sysreg_restore_host_state(host_ctxt);
> + sysreg_restore_host_state_vhe(host_ctxt);
>
> panic(__hyp_panic_string,
> spsr, elr,
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index b55c4ad..d8c42de 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -70,15 +70,27 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> }
>
> -void __hyp_text __sysreg_save_host_state(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_save_el1_state(ctxt);
> + __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> +}
> +
> +void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_save_el1_state(ctxt);
> + __sysreg_save_common_state(ctxt);
> + __sysreg_save_user_state(ctxt);
> +}
> +
> +void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
> {
> - if (!has_vhe())
> - __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> __sysreg_save_user_state(ctxt);
> }
>
> -void __hyp_text __sysreg_save_guest_state(struct kvm_cpu_context *ctxt)
> +void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> @@ -126,15 +138,27 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> }
>
> -void __hyp_text __sysreg_restore_host_state(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_restore_el1_state(ctxt);
> + __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> +}
> +
> +void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +{
> + __sysreg_restore_el1_state(ctxt);
> + __sysreg_restore_common_state(ctxt);
> + __sysreg_restore_user_state(ctxt);
> +}
> +
> +void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
> {
> - if (!has_vhe())
> - __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> __sysreg_restore_user_state(ctxt);
> }
>
> -void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> +void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 20/37] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: kvm, Marc Zyngier, Shih-Wei Li, Christoffer Dall
There is no need to have multiple identical functions with different
names for saving host and guest state. When saving and restoring state
for the host and guest, the state is the same for both contexts, and
that's why we have the kvm_cpu_context structure. Delete one
version and rename the other into simply save/restore.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 6 ++----
arch/arm64/kvm/hyp/switch.c | 10 +++++-----
arch/arm64/kvm/hyp/sysreg-sr.c | 18 ++----------------
3 files changed, 9 insertions(+), 25 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 998152d..3f54c55 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -132,10 +132,8 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
-void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt);
void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 7c4d430..6356bec 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
- __sysreg_save_host_state_nvhe(host_ctxt);
+ __sysreg_save_state_nvhe(host_ctxt);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
*/
__sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state_nvhe(guest_ctxt);
+ __sysreg_restore_state_nvhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
/* Jump in the fire! */
@@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (fixup_guest_exit(vcpu, &exit_code))
goto again;
- __sysreg_save_guest_state_nvhe(guest_ctxt);
+ __sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
@@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state_nvhe(host_ctxt);
+ __sysreg_restore_state_nvhe(host_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
@@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
__timer_disable_traps(vcpu);
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state_nvhe(__host_ctxt);
+ __sysreg_restore_state_nvhe(__host_ctxt);
}
/*
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index d8c42de..f5c1b44 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -70,14 +70,7 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
-void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
-{
- __sysreg_save_el1_state(ctxt);
- __sysreg_save_common_state(ctxt);
- __sysreg_save_user_state(ctxt);
-}
-
-void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
{
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
@@ -138,14 +131,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
-void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
-{
- __sysreg_restore_el1_state(ctxt);
- __sysreg_restore_common_state(ctxt);
- __sysreg_restore_user_state(ctxt);
-}
-
-void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
{
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 20/37] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
There is no need to have multiple identical functions with different
names for saving host and guest state. When saving and restoring state
for the host and guest, the state is the same for both contexts, and
that's why we have the kvm_cpu_context structure. Delete one
version and rename the other into simply save/restore.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 6 ++----
arch/arm64/kvm/hyp/switch.c | 10 +++++-----
arch/arm64/kvm/hyp/sysreg-sr.c | 18 ++----------------
3 files changed, 9 insertions(+), 25 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 998152d..3f54c55 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -132,10 +132,8 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __timer_enable_traps(struct kvm_vcpu *vcpu);
void __timer_disable_traps(struct kvm_vcpu *vcpu);
-void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
-void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
-void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt);
void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 7c4d430..6356bec 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
host_ctxt->__hyp_running_vcpu = vcpu;
guest_ctxt = &vcpu->arch.ctxt;
- __sysreg_save_host_state_nvhe(host_ctxt);
+ __sysreg_save_state_nvhe(host_ctxt);
__activate_traps(vcpu);
__activate_vm(vcpu);
@@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
* to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
*/
__sysreg32_restore_state(vcpu);
- __sysreg_restore_guest_state_nvhe(guest_ctxt);
+ __sysreg_restore_state_nvhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
/* Jump in the fire! */
@@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
if (fixup_guest_exit(vcpu, &exit_code))
goto again;
- __sysreg_save_guest_state_nvhe(guest_ctxt);
+ __sysreg_save_state_nvhe(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
@@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state_nvhe(host_ctxt);
+ __sysreg_restore_state_nvhe(host_ctxt);
/*
* This must come after restoring the host sysregs, since a non-VHE
@@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
__timer_disable_traps(vcpu);
__deactivate_traps(vcpu);
__deactivate_vm(vcpu);
- __sysreg_restore_host_state_nvhe(__host_ctxt);
+ __sysreg_restore_state_nvhe(__host_ctxt);
}
/*
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index d8c42de..f5c1b44 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -70,14 +70,7 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
-void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
-{
- __sysreg_save_el1_state(ctxt);
- __sysreg_save_common_state(ctxt);
- __sysreg_save_user_state(ctxt);
-}
-
-void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
{
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
@@ -138,14 +131,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
-void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
-{
- __sysreg_restore_el1_state(ctxt);
- __sysreg_restore_common_state(ctxt);
- __sysreg_restore_user_state(ctxt);
-}
-
-void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
+void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
{
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 20/37] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-08 10:39 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 10:39 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel, kvm
On Thu, Oct 12, 2017 at 12:41:24PM +0200, Christoffer Dall wrote:
> There is no need to have multiple identical functions with different
> names for saving host and guest state. When saving and restoring state
> for the host and guest, the state is the same for both contexts, and
> that's why we have the kvm_cpu_context structure. Delete one
> version and rename the other into simply save/restore.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_hyp.h | 6 ++----
> arch/arm64/kvm/hyp/switch.c | 10 +++++-----
> arch/arm64/kvm/hyp/sysreg-sr.c | 18 ++----------------
> 3 files changed, 9 insertions(+), 25 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index 998152d..3f54c55 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -132,10 +132,8 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> void __timer_enable_traps(struct kvm_vcpu *vcpu);
> void __timer_disable_traps(struct kvm_vcpu *vcpu);
>
> -void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
> -void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt);
> void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
> void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
> void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7c4d430..6356bec 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> - __sysreg_save_host_state_nvhe(host_ctxt);
> + __sysreg_save_state_nvhe(host_ctxt);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> */
> __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state_nvhe(guest_ctxt);
> + __sysreg_restore_state_nvhe(guest_ctxt);
> __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> @@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> if (fixup_guest_exit(vcpu, &exit_code))
> goto again;
>
> - __sysreg_save_guest_state_nvhe(guest_ctxt);
> + __sysreg_save_state_nvhe(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> __vgic_save_state(vcpu);
> @@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
>
> - __sysreg_restore_host_state_nvhe(host_ctxt);
> + __sysreg_restore_state_nvhe(host_ctxt);
>
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> @@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
> - __sysreg_restore_host_state_nvhe(__host_ctxt);
> + __sysreg_restore_state_nvhe(__host_ctxt);
> }
>
> /*
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index d8c42de..f5c1b44 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -70,14 +70,7 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> }
>
> -void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
> -{
> - __sysreg_save_el1_state(ctxt);
> - __sysreg_save_common_state(ctxt);
> - __sysreg_save_user_state(ctxt);
> -}
> -
> -void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> @@ -138,14 +131,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> }
>
> -void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
> -{
> - __sysreg_restore_el1_state(ctxt);
> - __sysreg_restore_common_state(ctxt);
> - __sysreg_restore_user_state(ctxt);
> -}
> -
> -void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> --
> 2.9.0
>
I think this patch could be squashed into the last one without
complicating the review. Anyway
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 20/37] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
@ 2017-11-08 10:39 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 10:39 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:24PM +0200, Christoffer Dall wrote:
> There is no need to have multiple identical functions with different
> names for saving host and guest state. When saving and restoring state
> for the host and guest, the state is the same for both contexts, and
> that's why we have the kvm_cpu_context structure. Delete one
> version and rename the other into simply save/restore.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_hyp.h | 6 ++----
> arch/arm64/kvm/hyp/switch.c | 10 +++++-----
> arch/arm64/kvm/hyp/sysreg-sr.c | 18 ++----------------
> 3 files changed, 9 insertions(+), 25 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index 998152d..3f54c55 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -132,10 +132,8 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> void __timer_enable_traps(struct kvm_vcpu *vcpu);
> void __timer_disable_traps(struct kvm_vcpu *vcpu);
>
> -void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
> -void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> -void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt);
> void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
> void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
> void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7c4d430..6356bec 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> host_ctxt->__hyp_running_vcpu = vcpu;
> guest_ctxt = &vcpu->arch.ctxt;
>
> - __sysreg_save_host_state_nvhe(host_ctxt);
> + __sysreg_save_state_nvhe(host_ctxt);
>
> __activate_traps(vcpu);
> __activate_vm(vcpu);
> @@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> */
> __sysreg32_restore_state(vcpu);
> - __sysreg_restore_guest_state_nvhe(guest_ctxt);
> + __sysreg_restore_state_nvhe(guest_ctxt);
> __debug_switch_to_guest(vcpu);
>
> /* Jump in the fire! */
> @@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> if (fixup_guest_exit(vcpu, &exit_code))
> goto again;
>
> - __sysreg_save_guest_state_nvhe(guest_ctxt);
> + __sysreg_save_state_nvhe(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_disable_traps(vcpu);
> __vgic_save_state(vcpu);
> @@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
>
> - __sysreg_restore_host_state_nvhe(host_ctxt);
> + __sysreg_restore_state_nvhe(host_ctxt);
>
> /*
> * This must come after restoring the host sysregs, since a non-VHE
> @@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> __timer_disable_traps(vcpu);
> __deactivate_traps(vcpu);
> __deactivate_vm(vcpu);
> - __sysreg_restore_host_state_nvhe(__host_ctxt);
> + __sysreg_restore_state_nvhe(__host_ctxt);
> }
>
> /*
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index d8c42de..f5c1b44 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -70,14 +70,7 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> }
>
> -void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
> -{
> - __sysreg_save_el1_state(ctxt);
> - __sysreg_save_common_state(ctxt);
> - __sysreg_save_user_state(ctxt);
> -}
> -
> -void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> @@ -138,14 +131,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> }
>
> -void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
> -{
> - __sysreg_restore_el1_state(ctxt);
> - __sysreg_restore_common_state(ctxt);
> - __sysreg_restore_user_state(ctxt);
> -}
> -
> -void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> +void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> {
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> --
> 2.9.0
>
I think this patch could be squashed into the last one without
complicating the review. Anyway
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 20/37] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
2017-11-08 10:39 ` Andrew Jones
@ 2017-12-03 19:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:41 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Wed, Nov 08, 2017 at 11:39:06AM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:24PM +0200, Christoffer Dall wrote:
> > There is no need to have multiple identical functions with different
> > names for saving host and guest state. When saving and restoring state
> > for the host and guest, the state is the same for both contexts, and
> > that's why we have the kvm_cpu_context structure. Delete one
> > version and rename the other into simply save/restore.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_hyp.h | 6 ++----
> > arch/arm64/kvm/hyp/switch.c | 10 +++++-----
> > arch/arm64/kvm/hyp/sysreg-sr.c | 18 ++----------------
> > 3 files changed, 9 insertions(+), 25 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 998152d..3f54c55 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -132,10 +132,8 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> > void __timer_enable_traps(struct kvm_vcpu *vcpu);
> > void __timer_disable_traps(struct kvm_vcpu *vcpu);
> >
> > -void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
> > -void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
> > -void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> > -void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> > +void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
> > +void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt);
> > void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
> > void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
> > void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 7c4d430..6356bec 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > host_ctxt->__hyp_running_vcpu = vcpu;
> > guest_ctxt = &vcpu->arch.ctxt;
> >
> > - __sysreg_save_host_state_nvhe(host_ctxt);
> > + __sysreg_save_state_nvhe(host_ctxt);
> >
> > __activate_traps(vcpu);
> > __activate_vm(vcpu);
> > @@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> > */
> > __sysreg32_restore_state(vcpu);
> > - __sysreg_restore_guest_state_nvhe(guest_ctxt);
> > + __sysreg_restore_state_nvhe(guest_ctxt);
> > __debug_switch_to_guest(vcpu);
> >
> > /* Jump in the fire! */
> > @@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > if (fixup_guest_exit(vcpu, &exit_code))
> > goto again;
> >
> > - __sysreg_save_guest_state_nvhe(guest_ctxt);
> > + __sysreg_save_state_nvhe(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > __timer_disable_traps(vcpu);
> > __vgic_save_state(vcpu);
> > @@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > __deactivate_traps(vcpu);
> > __deactivate_vm(vcpu);
> >
> > - __sysreg_restore_host_state_nvhe(host_ctxt);
> > + __sysreg_restore_state_nvhe(host_ctxt);
> >
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > @@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> > __timer_disable_traps(vcpu);
> > __deactivate_traps(vcpu);
> > __deactivate_vm(vcpu);
> > - __sysreg_restore_host_state_nvhe(__host_ctxt);
> > + __sysreg_restore_state_nvhe(__host_ctxt);
> > }
> >
> > /*
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index d8c42de..f5c1b44 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -70,14 +70,7 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> > ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> > }
> >
> > -void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
> > -{
> > - __sysreg_save_el1_state(ctxt);
> > - __sysreg_save_common_state(ctxt);
> > - __sysreg_save_user_state(ctxt);
> > -}
> > -
> > -void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> > +void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > @@ -138,14 +131,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> > write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> > }
> >
> > -void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
> > -{
> > - __sysreg_restore_el1_state(ctxt);
> > - __sysreg_restore_common_state(ctxt);
> > - __sysreg_restore_user_state(ctxt);
> > -}
> > -
> > -void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> > +void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > --
> > 2.9.0
> >
>
> I think this patch could be squashed into the last one without
> complicating the review. Anyway
This series was hard to debug, so I made changes as small as possible to
maintain my sanity when bisecting.
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 20/37] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions
@ 2017-12-03 19:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:41 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 08, 2017 at 11:39:06AM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:24PM +0200, Christoffer Dall wrote:
> > There is no need to have multiple identical functions with different
> > names for saving host and guest state. When saving and restoring state
> > for the host and guest, the state is the same for both contexts, and
> > that's why we have the kvm_cpu_context structure. Delete one
> > version and rename the other into simply save/restore.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_hyp.h | 6 ++----
> > arch/arm64/kvm/hyp/switch.c | 10 +++++-----
> > arch/arm64/kvm/hyp/sysreg-sr.c | 18 ++----------------
> > 3 files changed, 9 insertions(+), 25 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 998152d..3f54c55 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -132,10 +132,8 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
> > void __timer_enable_traps(struct kvm_vcpu *vcpu);
> > void __timer_disable_traps(struct kvm_vcpu *vcpu);
> >
> > -void __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt);
> > -void __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt);
> > -void __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> > -void __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt);
> > +void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt);
> > +void __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt);
> > void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt);
> > void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt);
> > void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt);
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 7c4d430..6356bec 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -383,7 +383,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > host_ctxt->__hyp_running_vcpu = vcpu;
> > guest_ctxt = &vcpu->arch.ctxt;
> >
> > - __sysreg_save_host_state_nvhe(host_ctxt);
> > + __sysreg_save_state_nvhe(host_ctxt);
> >
> > __activate_traps(vcpu);
> > __activate_vm(vcpu);
> > @@ -396,7 +396,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
> > */
> > __sysreg32_restore_state(vcpu);
> > - __sysreg_restore_guest_state_nvhe(guest_ctxt);
> > + __sysreg_restore_state_nvhe(guest_ctxt);
> > __debug_switch_to_guest(vcpu);
> >
> > /* Jump in the fire! */
> > @@ -407,7 +407,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > if (fixup_guest_exit(vcpu, &exit_code))
> > goto again;
> >
> > - __sysreg_save_guest_state_nvhe(guest_ctxt);
> > + __sysreg_save_state_nvhe(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > __timer_disable_traps(vcpu);
> > __vgic_save_state(vcpu);
> > @@ -415,7 +415,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > __deactivate_traps(vcpu);
> > __deactivate_vm(vcpu);
> >
> > - __sysreg_restore_host_state_nvhe(host_ctxt);
> > + __sysreg_restore_state_nvhe(host_ctxt);
> >
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > @@ -440,7 +440,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
> > __timer_disable_traps(vcpu);
> > __deactivate_traps(vcpu);
> > __deactivate_vm(vcpu);
> > - __sysreg_restore_host_state_nvhe(__host_ctxt);
> > + __sysreg_restore_state_nvhe(__host_ctxt);
> > }
> >
> > /*
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index d8c42de..f5c1b44 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -70,14 +70,7 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> > ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> > }
> >
> > -void __hyp_text __sysreg_save_host_state_nvhe(struct kvm_cpu_context *ctxt)
> > -{
> > - __sysreg_save_el1_state(ctxt);
> > - __sysreg_save_common_state(ctxt);
> > - __sysreg_save_user_state(ctxt);
> > -}
> > -
> > -void __hyp_text __sysreg_save_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> > +void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > @@ -138,14 +131,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> > write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> > }
> >
> > -void __hyp_text __sysreg_restore_host_state_nvhe(struct kvm_cpu_context *ctxt)
> > -{
> > - __sysreg_restore_el1_state(ctxt);
> > - __sysreg_restore_common_state(ctxt);
> > - __sysreg_restore_user_state(ctxt);
> > -}
> > -
> > -void __hyp_text __sysreg_restore_guest_state_nvhe(struct kvm_cpu_context *ctxt)
> > +void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> > {
> > __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > --
> > 2.9.0
> >
>
> I think this patch could be squashed into the last one without
> complicating the review. Anyway
This series was hard to debug, so I made changes as small as possible to
maintain my sanity when bisecting.
>
> Reviewed-by: Andrew Jones <drjones@redhat.com>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 21/37] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
can return to the host in EL1 in the same state and location where we
issued a hypercall to EL2, but these registers don't contain anything
important on VHE, because all of the host runs in EL2. Therefore,
exclude them when saving/restoring the host state.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/sysreg-sr.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index f5c1b44..354ca02 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -66,6 +66,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
ctxt->gp_regs.elr_el1 = read_sysreg_el1(elr);
ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg_el1(spsr);
+}
+
+static void __hyp_text __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
+{
ctxt->gp_regs.regs.pc = read_sysreg_el2(elr);
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
@@ -75,6 +79,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
+ __sysreg_save_el2_return_state(ctxt);
}
void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
@@ -88,6 +93,7 @@ void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
+ __sysreg_save_el2_return_state(ctxt);
}
static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
@@ -127,6 +133,11 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
write_sysreg_el1(ctxt->gp_regs.elr_el1, elr);
write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],spsr);
+}
+
+static void __hyp_text
+__sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
+{
write_sysreg_el2(ctxt->gp_regs.regs.pc, elr);
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
@@ -136,6 +147,7 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
+ __sysreg_restore_el2_return_state(ctxt);
}
void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
@@ -149,6 +161,7 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
+ __sysreg_restore_el2_return_state(ctxt);
}
static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 21/37] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
can return to the host in EL1 in the same state and location where we
issued a hypercall to EL2, but these registers don't contain anything
important on VHE, because all of the host runs in EL2. Therefore,
exclude them when saving/restoring the host state.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/sysreg-sr.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index f5c1b44..354ca02 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -66,6 +66,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
ctxt->gp_regs.elr_el1 = read_sysreg_el1(elr);
ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg_el1(spsr);
+}
+
+static void __hyp_text __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
+{
ctxt->gp_regs.regs.pc = read_sysreg_el2(elr);
ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
}
@@ -75,6 +79,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
+ __sysreg_save_el2_return_state(ctxt);
}
void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
@@ -88,6 +93,7 @@ void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
__sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
__sysreg_save_user_state(ctxt);
+ __sysreg_save_el2_return_state(ctxt);
}
static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
@@ -127,6 +133,11 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
write_sysreg_el1(ctxt->gp_regs.elr_el1, elr);
write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],spsr);
+}
+
+static void __hyp_text
+__sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
+{
write_sysreg_el2(ctxt->gp_regs.regs.pc, elr);
write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
}
@@ -136,6 +147,7 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
+ __sysreg_restore_el2_return_state(ctxt);
}
void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
@@ -149,6 +161,7 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
__sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
__sysreg_restore_user_state(ctxt);
+ __sysreg_restore_el2_return_state(ctxt);
}
static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 21/37] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-08 17:03 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 17:03 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:25PM +0200, Christoffer Dall wrote:
> On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
> can return to the host in EL1 in the same state and location where we
> issued a hypercall to EL2, but these registers don't contain anything
> important on VHE, because all of the host runs in EL2. Therefore,
> exclude them when saving/restoring the host state.
This commit message implies we start excluding now, but this patch
actually has no functional change. It just prepares things for
patch 27/37 "KVM: arm64: Defer saving/restoring system registers to
vcpu load/put on VHE". Can you please clarify it?
Thanks,
drew
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/sysreg-sr.c | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index f5c1b44..354ca02 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -66,6 +66,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
> ctxt->gp_regs.elr_el1 = read_sysreg_el1(elr);
> ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg_el1(spsr);
> +}
> +
> +static void __hyp_text __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
> +{
> ctxt->gp_regs.regs.pc = read_sysreg_el2(elr);
> ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> }
> @@ -75,6 +79,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> __sysreg_save_user_state(ctxt);
> + __sysreg_save_el2_return_state(ctxt);
> }
>
> void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
> @@ -88,6 +93,7 @@ void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> __sysreg_save_user_state(ctxt);
> + __sysreg_save_el2_return_state(ctxt);
> }
>
> static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> @@ -127,6 +133,11 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
> write_sysreg_el1(ctxt->gp_regs.elr_el1, elr);
> write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],spsr);
> +}
> +
> +static void __hyp_text
> +__sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
> +{
> write_sysreg_el2(ctxt->gp_regs.regs.pc, elr);
> write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> }
> @@ -136,6 +147,7 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> __sysreg_restore_user_state(ctxt);
> + __sysreg_restore_el2_return_state(ctxt);
> }
>
> void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
> @@ -149,6 +161,7 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> __sysreg_restore_user_state(ctxt);
> + __sysreg_restore_el2_return_state(ctxt);
> }
>
> static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> --
> 2.9.0
>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 21/37] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
@ 2017-11-08 17:03 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-08 17:03 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:25PM +0200, Christoffer Dall wrote:
> On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
> can return to the host in EL1 in the same state and location where we
> issued a hypercall to EL2, but these registers don't contain anything
> important on VHE, because all of the host runs in EL2. Therefore,
> exclude them when saving/restoring the host state.
This commit message implies we start excluding now, but this patch
actually has no functional change. It just prepares things for
patch 27/37 "KVM: arm64: Defer saving/restoring system registers to
vcpu load/put on VHE". Can you please clarify it?
Thanks,
drew
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/sysreg-sr.c | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index f5c1b44..354ca02 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -66,6 +66,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
> ctxt->gp_regs.elr_el1 = read_sysreg_el1(elr);
> ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg_el1(spsr);
> +}
> +
> +static void __hyp_text __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
> +{
> ctxt->gp_regs.regs.pc = read_sysreg_el2(elr);
> ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> }
> @@ -75,6 +79,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> __sysreg_save_user_state(ctxt);
> + __sysreg_save_el2_return_state(ctxt);
> }
>
> void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
> @@ -88,6 +93,7 @@ void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
> __sysreg_save_el1_state(ctxt);
> __sysreg_save_common_state(ctxt);
> __sysreg_save_user_state(ctxt);
> + __sysreg_save_el2_return_state(ctxt);
> }
>
> static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> @@ -127,6 +133,11 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
> write_sysreg_el1(ctxt->gp_regs.elr_el1, elr);
> write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],spsr);
> +}
> +
> +static void __hyp_text
> +__sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
> +{
> write_sysreg_el2(ctxt->gp_regs.regs.pc, elr);
> write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> }
> @@ -136,6 +147,7 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> __sysreg_restore_user_state(ctxt);
> + __sysreg_restore_el2_return_state(ctxt);
> }
>
> void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
> @@ -149,6 +161,7 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
> __sysreg_restore_el1_state(ctxt);
> __sysreg_restore_common_state(ctxt);
> __sysreg_restore_user_state(ctxt);
> + __sysreg_restore_el2_return_state(ctxt);
> }
>
> static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> --
> 2.9.0
>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 21/37] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
2017-11-08 17:03 ` Andrew Jones
@ 2017-12-03 19:45 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:45 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Wed, Nov 08, 2017 at 06:03:23PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:25PM +0200, Christoffer Dall wrote:
> > On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
> > can return to the host in EL1 in the same state and location where we
> > issued a hypercall to EL2, but these registers don't contain anything
> > important on VHE, because all of the host runs in EL2. Therefore,
> > exclude them when saving/restoring the host state.
>
> This commit message implies we start excluding now, but this patch
> actually has no functional change. It just prepares things for
> patch 27/37 "KVM: arm64: Defer saving/restoring system registers to
> vcpu load/put on VHE". Can you please clarify it?
>
Sure:
On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
can return to the host in EL1 in the same state and location where we
issued a hypercall to EL2, but these registers don't contain anything
important on VHE, because all of the host runs in EL2. Therefore,
factor out these registers into separate save/restore functions, making
it easy to exclude them from the VHE world-switch path later on.
Thanks,
-Christoffer
>
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/sysreg-sr.c | 13 +++++++++++++
> > 1 file changed, 13 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index f5c1b44..354ca02 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -66,6 +66,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> > ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
> > ctxt->gp_regs.elr_el1 = read_sysreg_el1(elr);
> > ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg_el1(spsr);
> > +}
> > +
> > +static void __hyp_text __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
> > +{
> > ctxt->gp_regs.regs.pc = read_sysreg_el2(elr);
> > ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> > }
> > @@ -75,6 +79,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> > __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > __sysreg_save_user_state(ctxt);
> > + __sysreg_save_el2_return_state(ctxt);
> > }
> >
> > void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
> > @@ -88,6 +93,7 @@ void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
> > __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > __sysreg_save_user_state(ctxt);
> > + __sysreg_save_el2_return_state(ctxt);
> > }
> >
> > static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> > @@ -127,6 +133,11 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> > write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
> > write_sysreg_el1(ctxt->gp_regs.elr_el1, elr);
> > write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],spsr);
> > +}
> > +
> > +static void __hyp_text
> > +__sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
> > +{
> > write_sysreg_el2(ctxt->gp_regs.regs.pc, elr);
> > write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> > }
> > @@ -136,6 +147,7 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > __sysreg_restore_user_state(ctxt);
> > + __sysreg_restore_el2_return_state(ctxt);
> > }
> >
> > void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
> > @@ -149,6 +161,7 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > __sysreg_restore_user_state(ctxt);
> > + __sysreg_restore_el2_return_state(ctxt);
> > }
> >
> > static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > --
> > 2.9.0
> >
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 21/37] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems
@ 2017-12-03 19:45 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:45 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 08, 2017 at 06:03:23PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:25PM +0200, Christoffer Dall wrote:
> > On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
> > can return to the host in EL1 in the same state and location where we
> > issued a hypercall to EL2, but these registers don't contain anything
> > important on VHE, because all of the host runs in EL2. Therefore,
> > exclude them when saving/restoring the host state.
>
> This commit message implies we start excluding now, but this patch
> actually has no functional change. It just prepares things for
> patch 27/37 "KVM: arm64: Defer saving/restoring system registers to
> vcpu load/put on VHE". Can you please clarify it?
>
Sure:
On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we
can return to the host in EL1 in the same state and location where we
issued a hypercall to EL2, but these registers don't contain anything
important on VHE, because all of the host runs in EL2. Therefore,
factor out these registers into separate save/restore functions, making
it easy to exclude them from the VHE world-switch path later on.
Thanks,
-Christoffer
>
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/sysreg-sr.c | 13 +++++++++++++
> > 1 file changed, 13 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index f5c1b44..354ca02 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -66,6 +66,10 @@ static void __hyp_text __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
> > ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
> > ctxt->gp_regs.elr_el1 = read_sysreg_el1(elr);
> > ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg_el1(spsr);
> > +}
> > +
> > +static void __hyp_text __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
> > +{
> > ctxt->gp_regs.regs.pc = read_sysreg_el2(elr);
> > ctxt->gp_regs.regs.pstate = read_sysreg_el2(spsr);
> > }
> > @@ -75,6 +79,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
> > __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > __sysreg_save_user_state(ctxt);
> > + __sysreg_save_el2_return_state(ctxt);
> > }
> >
> > void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
> > @@ -88,6 +93,7 @@ void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
> > __sysreg_save_el1_state(ctxt);
> > __sysreg_save_common_state(ctxt);
> > __sysreg_save_user_state(ctxt);
> > + __sysreg_save_el2_return_state(ctxt);
> > }
> >
> > static void __hyp_text __sysreg_restore_common_state(struct kvm_cpu_context *ctxt)
> > @@ -127,6 +133,11 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
> > write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
> > write_sysreg_el1(ctxt->gp_regs.elr_el1, elr);
> > write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],spsr);
> > +}
> > +
> > +static void __hyp_text
> > +__sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)
> > +{
> > write_sysreg_el2(ctxt->gp_regs.regs.pc, elr);
> > write_sysreg_el2(ctxt->gp_regs.regs.pstate, spsr);
> > }
> > @@ -136,6 +147,7 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > __sysreg_restore_user_state(ctxt);
> > + __sysreg_restore_el2_return_state(ctxt);
> > }
> >
> > void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
> > @@ -149,6 +161,7 @@ void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_el1_state(ctxt);
> > __sysreg_restore_common_state(ctxt);
> > __sysreg_restore_user_state(ctxt);
> > + __sysreg_restore_el2_return_state(ctxt);
> > }
> >
> > static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > --
> > 2.9.0
> >
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 22/37] KVM: arm64: Change 32-bit handling of VM system registers
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
We currently handle 32-bit accesses to trapped VM system registers using
the 32-bit index into the coproc array on the vcpu structure, which is a
union of the coprog array and the sysreg array.
Since all the 32-bit coproc indicies are created to correspond to the
architectural mapping between 64-bit system registers and 32-bit
coprocessor registers, and because the AArch64 system registers are the
double in size of the AArch32 coprocessor registers, we can always find
the system register entry that we must update by dividing the 32-bit
coproc index by 2.
This is going to make our lives much easier when we have to start
accessing system registers that uses deferred save/restore and might
have to be read directly from the physical CPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_host.h | 8 --------
arch/arm64/kvm/sys_regs.c | 20 +++++++++++++++-----
2 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 5e09eb9..9f5761f 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -289,14 +289,6 @@ struct kvm_vcpu_arch {
#define vcpu_cp14(v,r) ((v)->arch.ctxt.copro[(r)])
#define vcpu_cp15(v,r) ((v)->arch.ctxt.copro[(r)])
-#ifdef CONFIG_CPU_BIG_ENDIAN
-#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r))
-#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r) + 1)
-#else
-#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r) + 1)
-#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r))
-#endif
-
struct kvm_vm_stat {
ulong remote_tlb_flush;
};
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index bb0e41b..dbe35fd 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -120,16 +120,26 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
const struct sys_reg_desc *r)
{
bool was_enabled = vcpu_has_cache_enabled(vcpu);
+ u64 val;
+ int reg = r->reg;
BUG_ON(!p->is_write);
- if (!p->is_aarch32) {
- vcpu_sys_reg(vcpu, r->reg) = p->regval;
+ /* See the 32bit mapping in kvm_host.h */
+ if (p->is_aarch32)
+ reg = r->reg / 2;
+
+ if (!p->is_aarch32 || !p->is_32bit) {
+ val = p->regval;
} else {
- if (!p->is_32bit)
- vcpu_cp15_64_high(vcpu, r->reg) = upper_32_bits(p->regval);
- vcpu_cp15_64_low(vcpu, r->reg) = lower_32_bits(p->regval);
+ val = vcpu_sys_reg(vcpu, reg);
+ if (r->reg % 2)
+ val = (p->regval << 32) | (u64)lower_32_bits(val);
+ else
+ val = ((u64)upper_32_bits(val) << 32) |
+ (u64)lower_32_bits(p->regval);
}
+ vcpu_sys_reg(vcpu, reg) = val;
kvm_toggle_cache(vcpu, was_enabled);
return true;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 22/37] KVM: arm64: Change 32-bit handling of VM system registers
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We currently handle 32-bit accesses to trapped VM system registers using
the 32-bit index into the coproc array on the vcpu structure, which is a
union of the coprog array and the sysreg array.
Since all the 32-bit coproc indicies are created to correspond to the
architectural mapping between 64-bit system registers and 32-bit
coprocessor registers, and because the AArch64 system registers are the
double in size of the AArch32 coprocessor registers, we can always find
the system register entry that we must update by dividing the 32-bit
coproc index by 2.
This is going to make our lives much easier when we have to start
accessing system registers that uses deferred save/restore and might
have to be read directly from the physical CPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_host.h | 8 --------
arch/arm64/kvm/sys_regs.c | 20 +++++++++++++++-----
2 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 5e09eb9..9f5761f 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -289,14 +289,6 @@ struct kvm_vcpu_arch {
#define vcpu_cp14(v,r) ((v)->arch.ctxt.copro[(r)])
#define vcpu_cp15(v,r) ((v)->arch.ctxt.copro[(r)])
-#ifdef CONFIG_CPU_BIG_ENDIAN
-#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r))
-#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r) + 1)
-#else
-#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r) + 1)
-#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r))
-#endif
-
struct kvm_vm_stat {
ulong remote_tlb_flush;
};
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index bb0e41b..dbe35fd 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -120,16 +120,26 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
const struct sys_reg_desc *r)
{
bool was_enabled = vcpu_has_cache_enabled(vcpu);
+ u64 val;
+ int reg = r->reg;
BUG_ON(!p->is_write);
- if (!p->is_aarch32) {
- vcpu_sys_reg(vcpu, r->reg) = p->regval;
+ /* See the 32bit mapping in kvm_host.h */
+ if (p->is_aarch32)
+ reg = r->reg / 2;
+
+ if (!p->is_aarch32 || !p->is_32bit) {
+ val = p->regval;
} else {
- if (!p->is_32bit)
- vcpu_cp15_64_high(vcpu, r->reg) = upper_32_bits(p->regval);
- vcpu_cp15_64_low(vcpu, r->reg) = lower_32_bits(p->regval);
+ val = vcpu_sys_reg(vcpu, reg);
+ if (r->reg % 2)
+ val = (p->regval << 32) | (u64)lower_32_bits(val);
+ else
+ val = ((u64)upper_32_bits(val) << 32) |
+ (u64)lower_32_bits(p->regval);
}
+ vcpu_sys_reg(vcpu, reg) = val;
kvm_toggle_cache(vcpu, was_enabled);
return true;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 22/37] KVM: arm64: Change 32-bit handling of VM system registers
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-13 16:25 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 16:25 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:26PM +0200, Christoffer Dall wrote:
> We currently handle 32-bit accesses to trapped VM system registers using
> the 32-bit index into the coproc array on the vcpu structure, which is a
> union of the coprog array and the sysreg array.
coproc
>
> Since all the 32-bit coproc indicies are created to correspond to the
> architectural mapping between 64-bit system registers and 32-bit
> coprocessor registers, and because the AArch64 system registers are the
> double in size of the AArch32 coprocessor registers, we can always find
> the system register entry that we must update by dividing the 32-bit
> coproc index by 2.
>
> This is going to make our lives much easier when we have to start
> accessing system registers that uses deferred save/restore and might
use
> have to be read directly from the physical CPU.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 8 --------
> arch/arm64/kvm/sys_regs.c | 20 +++++++++++++++-----
> 2 files changed, 15 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 5e09eb9..9f5761f 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -289,14 +289,6 @@ struct kvm_vcpu_arch {
> #define vcpu_cp14(v,r) ((v)->arch.ctxt.copro[(r)])
> #define vcpu_cp15(v,r) ((v)->arch.ctxt.copro[(r)])
>
> -#ifdef CONFIG_CPU_BIG_ENDIAN
> -#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r))
> -#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r) + 1)
> -#else
> -#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r) + 1)
> -#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r))
> -#endif
> -
> struct kvm_vm_stat {
> ulong remote_tlb_flush;
> };
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index bb0e41b..dbe35fd 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -120,16 +120,26 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> const struct sys_reg_desc *r)
> {
> bool was_enabled = vcpu_has_cache_enabled(vcpu);
> + u64 val;
> + int reg = r->reg;
>
> BUG_ON(!p->is_write);
>
> - if (!p->is_aarch32) {
> - vcpu_sys_reg(vcpu, r->reg) = p->regval;
> + /* See the 32bit mapping in kvm_host.h */
> + if (p->is_aarch32)
> + reg = r->reg / 2;
> +
> + if (!p->is_aarch32 || !p->is_32bit) {
> + val = p->regval;
> } else {
> - if (!p->is_32bit)
> - vcpu_cp15_64_high(vcpu, r->reg) = upper_32_bits(p->regval);
> - vcpu_cp15_64_low(vcpu, r->reg) = lower_32_bits(p->regval);
> + val = vcpu_sys_reg(vcpu, reg);
> + if (r->reg % 2)
> + val = (p->regval << 32) | (u64)lower_32_bits(val);
> + else
> + val = ((u64)upper_32_bits(val) << 32) |
> + (u64)lower_32_bits(p->regval);
> }
> + vcpu_sys_reg(vcpu, reg) = val;
>
> kvm_toggle_cache(vcpu, was_enabled);
> return true;
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 22/37] KVM: arm64: Change 32-bit handling of VM system registers
@ 2017-11-13 16:25 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 16:25 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:26PM +0200, Christoffer Dall wrote:
> We currently handle 32-bit accesses to trapped VM system registers using
> the 32-bit index into the coproc array on the vcpu structure, which is a
> union of the coprog array and the sysreg array.
coproc
>
> Since all the 32-bit coproc indicies are created to correspond to the
> architectural mapping between 64-bit system registers and 32-bit
> coprocessor registers, and because the AArch64 system registers are the
> double in size of the AArch32 coprocessor registers, we can always find
> the system register entry that we must update by dividing the 32-bit
> coproc index by 2.
>
> This is going to make our lives much easier when we have to start
> accessing system registers that uses deferred save/restore and might
use
> have to be read directly from the physical CPU.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 8 --------
> arch/arm64/kvm/sys_regs.c | 20 +++++++++++++++-----
> 2 files changed, 15 insertions(+), 13 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 5e09eb9..9f5761f 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -289,14 +289,6 @@ struct kvm_vcpu_arch {
> #define vcpu_cp14(v,r) ((v)->arch.ctxt.copro[(r)])
> #define vcpu_cp15(v,r) ((v)->arch.ctxt.copro[(r)])
>
> -#ifdef CONFIG_CPU_BIG_ENDIAN
> -#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r))
> -#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r) + 1)
> -#else
> -#define vcpu_cp15_64_high(v,r) vcpu_cp15((v),(r) + 1)
> -#define vcpu_cp15_64_low(v,r) vcpu_cp15((v),(r))
> -#endif
> -
> struct kvm_vm_stat {
> ulong remote_tlb_flush;
> };
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index bb0e41b..dbe35fd 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -120,16 +120,26 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> const struct sys_reg_desc *r)
> {
> bool was_enabled = vcpu_has_cache_enabled(vcpu);
> + u64 val;
> + int reg = r->reg;
>
> BUG_ON(!p->is_write);
>
> - if (!p->is_aarch32) {
> - vcpu_sys_reg(vcpu, r->reg) = p->regval;
> + /* See the 32bit mapping in kvm_host.h */
> + if (p->is_aarch32)
> + reg = r->reg / 2;
> +
> + if (!p->is_aarch32 || !p->is_32bit) {
> + val = p->regval;
> } else {
> - if (!p->is_32bit)
> - vcpu_cp15_64_high(vcpu, r->reg) = upper_32_bits(p->regval);
> - vcpu_cp15_64_low(vcpu, r->reg) = lower_32_bits(p->regval);
> + val = vcpu_sys_reg(vcpu, reg);
> + if (r->reg % 2)
> + val = (p->regval << 32) | (u64)lower_32_bits(val);
> + else
> + val = ((u64)upper_32_bits(val) << 32) |
> + (u64)lower_32_bits(p->regval);
> }
> + vcpu_sys_reg(vcpu, reg) = val;
>
> kvm_toggle_cache(vcpu, was_enabled);
> return true;
> --
> 2.9.0
>
Reviewed-by: Andrew Jones <drjones@redhat.com>
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
When we defer the save/restore of system registers to vcpu_load and
vcpu_put, we need to take care of the emulation code that handles traps
to these registers, since simply reading the memory array will return
stale data.
Therefore, introduce two functions to directly read/write the registers
from the physical CPU when we're on a VHE system that has loaded the
system registers onto the physical CPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +++
arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++++++--
2 files changed, 56 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9f5761f..dcded44 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -278,6 +278,10 @@ struct kvm_vcpu_arch {
/* Detect first run of a vcpu */
bool has_run_once;
+
+ /* True when deferrable sysregs are loaded on the physical CPU,
+ * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */
+ bool sysregs_loaded_on_cpu;
};
#define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index dbe35fd..f7887dd 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -34,6 +34,7 @@
#include <asm/kvm_coproc.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_host.h>
+#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
#include <asm/perf_event.h>
#include <asm/sysreg.h>
@@ -110,8 +111,57 @@ static bool access_dcsw(struct kvm_vcpu *vcpu,
return true;
}
+static u64 read_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ switch (reg) {
+ case SCTLR_EL1: return read_sysreg_el1(sctlr);
+ case TTBR0_EL1: return read_sysreg_el1(ttbr0);
+ case TTBR1_EL1: return read_sysreg_el1(ttbr1);
+ case TCR_EL1: return read_sysreg_el1(tcr);
+ case ESR_EL1: return read_sysreg_el1(esr);
+ case FAR_EL1: return read_sysreg_el1(far);
+ case AFSR0_EL1: return read_sysreg_el1(afsr0);
+ case AFSR1_EL1: return read_sysreg_el1(afsr1);
+ case MAIR_EL1: return read_sysreg_el1(mair);
+ case AMAIR_EL1: return read_sysreg_el1(amair);
+ case CONTEXTIDR_EL1: return read_sysreg_el1(contextidr);
+ case DACR32_EL2: return read_sysreg(dacr32_el2);
+ case IFSR32_EL2: return read_sysreg(ifsr32_el2);
+ default: BUG();
+ }
+ }
+
+ return vcpu_sys_reg(vcpu, reg);
+}
+
+static void write_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg, u64 val)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ switch (reg) {
+ case SCTLR_EL1: write_sysreg_el1(val, sctlr); return;
+ case TTBR0_EL1: write_sysreg_el1(val, ttbr0); return;
+ case TTBR1_EL1: write_sysreg_el1(val, ttbr1); return;
+ case TCR_EL1: write_sysreg_el1(val, tcr); return;
+ case ESR_EL1: write_sysreg_el1(val, esr); return;
+ case FAR_EL1: write_sysreg_el1(val, far); return;
+ case AFSR0_EL1: write_sysreg_el1(val, afsr0); return;
+ case AFSR1_EL1: write_sysreg_el1(val, afsr1); return;
+ case MAIR_EL1: write_sysreg_el1(val, mair); return;
+ case AMAIR_EL1: write_sysreg_el1(val, amair); return;
+ case CONTEXTIDR_EL1: write_sysreg_el1(val, contextidr); return;
+ case DACR32_EL2: write_sysreg(val, dacr32_el2); return;
+ case IFSR32_EL2: write_sysreg(val, ifsr32_el2); return;
+ default: BUG();
+ }
+ }
+
+ vcpu_sys_reg(vcpu, reg) = val;
+}
+
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
+ *
* is set. If the guest enables the MMU, we stop trapping the VM
* sys_regs and leave it in complete control of the caches.
*/
@@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
if (!p->is_aarch32 || !p->is_32bit) {
val = p->regval;
} else {
- val = vcpu_sys_reg(vcpu, reg);
+ val = read_deferrable_vm_reg(vcpu, reg);
if (r->reg % 2)
val = (p->regval << 32) | (u64)lower_32_bits(val);
else
val = ((u64)upper_32_bits(val) << 32) |
(u64)lower_32_bits(p->regval);
}
- vcpu_sys_reg(vcpu, reg) = val;
+ write_deferrable_vm_reg(vcpu, reg, val);
kvm_toggle_cache(vcpu, was_enabled);
return true;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
When we defer the save/restore of system registers to vcpu_load and
vcpu_put, we need to take care of the emulation code that handles traps
to these registers, since simply reading the memory array will return
stale data.
Therefore, introduce two functions to directly read/write the registers
from the physical CPU when we're on a VHE system that has loaded the
system registers onto the physical CPU.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_host.h | 4 +++
arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++++++--
2 files changed, 56 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9f5761f..dcded44 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -278,6 +278,10 @@ struct kvm_vcpu_arch {
/* Detect first run of a vcpu */
bool has_run_once;
+
+ /* True when deferrable sysregs are loaded on the physical CPU,
+ * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */
+ bool sysregs_loaded_on_cpu;
};
#define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index dbe35fd..f7887dd 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -34,6 +34,7 @@
#include <asm/kvm_coproc.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_host.h>
+#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
#include <asm/perf_event.h>
#include <asm/sysreg.h>
@@ -110,8 +111,57 @@ static bool access_dcsw(struct kvm_vcpu *vcpu,
return true;
}
+static u64 read_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ switch (reg) {
+ case SCTLR_EL1: return read_sysreg_el1(sctlr);
+ case TTBR0_EL1: return read_sysreg_el1(ttbr0);
+ case TTBR1_EL1: return read_sysreg_el1(ttbr1);
+ case TCR_EL1: return read_sysreg_el1(tcr);
+ case ESR_EL1: return read_sysreg_el1(esr);
+ case FAR_EL1: return read_sysreg_el1(far);
+ case AFSR0_EL1: return read_sysreg_el1(afsr0);
+ case AFSR1_EL1: return read_sysreg_el1(afsr1);
+ case MAIR_EL1: return read_sysreg_el1(mair);
+ case AMAIR_EL1: return read_sysreg_el1(amair);
+ case CONTEXTIDR_EL1: return read_sysreg_el1(contextidr);
+ case DACR32_EL2: return read_sysreg(dacr32_el2);
+ case IFSR32_EL2: return read_sysreg(ifsr32_el2);
+ default: BUG();
+ }
+ }
+
+ return vcpu_sys_reg(vcpu, reg);
+}
+
+static void write_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg, u64 val)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ switch (reg) {
+ case SCTLR_EL1: write_sysreg_el1(val, sctlr); return;
+ case TTBR0_EL1: write_sysreg_el1(val, ttbr0); return;
+ case TTBR1_EL1: write_sysreg_el1(val, ttbr1); return;
+ case TCR_EL1: write_sysreg_el1(val, tcr); return;
+ case ESR_EL1: write_sysreg_el1(val, esr); return;
+ case FAR_EL1: write_sysreg_el1(val, far); return;
+ case AFSR0_EL1: write_sysreg_el1(val, afsr0); return;
+ case AFSR1_EL1: write_sysreg_el1(val, afsr1); return;
+ case MAIR_EL1: write_sysreg_el1(val, mair); return;
+ case AMAIR_EL1: write_sysreg_el1(val, amair); return;
+ case CONTEXTIDR_EL1: write_sysreg_el1(val, contextidr); return;
+ case DACR32_EL2: write_sysreg(val, dacr32_el2); return;
+ case IFSR32_EL2: write_sysreg(val, ifsr32_el2); return;
+ default: BUG();
+ }
+ }
+
+ vcpu_sys_reg(vcpu, reg) = val;
+}
+
/*
* Generic accessor for VM registers. Only called as long as HCR_TVM
+ *
* is set. If the guest enables the MMU, we stop trapping the VM
* sys_regs and leave it in complete control of the caches.
*/
@@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
if (!p->is_aarch32 || !p->is_32bit) {
val = p->regval;
} else {
- val = vcpu_sys_reg(vcpu, reg);
+ val = read_deferrable_vm_reg(vcpu, reg);
if (r->reg % 2)
val = (p->regval << 32) | (u64)lower_32_bits(val);
else
val = ((u64)upper_32_bits(val) << 32) |
(u64)lower_32_bits(p->regval);
}
- vcpu_sys_reg(vcpu, reg) = val;
+ write_deferrable_vm_reg(vcpu, reg, val);
kvm_toggle_cache(vcpu, was_enabled);
return true;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-13 17:54 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 17:54 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:27PM +0200, Christoffer Dall wrote:
> When we defer the save/restore of system registers to vcpu_load and
> vcpu_put, we need to take care of the emulation code that handles traps
> to these registers, since simply reading the memory array will return
> stale data.
>
> Therefore, introduce two functions to directly read/write the registers
> from the physical CPU when we're on a VHE system that has loaded the
> system registers onto the physical CPU.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 4 +++
> arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++++++--
> 2 files changed, 56 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 9f5761f..dcded44 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -278,6 +278,10 @@ struct kvm_vcpu_arch {
>
> /* Detect first run of a vcpu */
> bool has_run_once;
> +
> + /* True when deferrable sysregs are loaded on the physical CPU,
> + * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */
> + bool sysregs_loaded_on_cpu;
To be sure I understand correctly, this should only be false when we're
servicing ioctls that don't do a full vcpu_load first (non-KVM_RUN) or
when we're not running on a CPU with VHE. If we didn't need to worry
about non-KVM_RUN ioctls, then we could use the static key has_vhe(),
right?
> };
>
> #define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index dbe35fd..f7887dd 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -34,6 +34,7 @@
> #include <asm/kvm_coproc.h>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_host.h>
> +#include <asm/kvm_hyp.h>
> #include <asm/kvm_mmu.h>
> #include <asm/perf_event.h>
> #include <asm/sysreg.h>
> @@ -110,8 +111,57 @@ static bool access_dcsw(struct kvm_vcpu *vcpu,
> return true;
> }
>
> +static u64 read_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + switch (reg) {
> + case SCTLR_EL1: return read_sysreg_el1(sctlr);
> + case TTBR0_EL1: return read_sysreg_el1(ttbr0);
> + case TTBR1_EL1: return read_sysreg_el1(ttbr1);
> + case TCR_EL1: return read_sysreg_el1(tcr);
> + case ESR_EL1: return read_sysreg_el1(esr);
> + case FAR_EL1: return read_sysreg_el1(far);
> + case AFSR0_EL1: return read_sysreg_el1(afsr0);
> + case AFSR1_EL1: return read_sysreg_el1(afsr1);
> + case MAIR_EL1: return read_sysreg_el1(mair);
> + case AMAIR_EL1: return read_sysreg_el1(amair);
> + case CONTEXTIDR_EL1: return read_sysreg_el1(contextidr);
> + case DACR32_EL2: return read_sysreg(dacr32_el2);
> + case IFSR32_EL2: return read_sysreg(ifsr32_el2);
> + default: BUG();
> + }
> + }
> +
> + return vcpu_sys_reg(vcpu, reg);
> +}
> +
> +static void write_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg, u64 val)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + switch (reg) {
> + case SCTLR_EL1: write_sysreg_el1(val, sctlr); return;
> + case TTBR0_EL1: write_sysreg_el1(val, ttbr0); return;
> + case TTBR1_EL1: write_sysreg_el1(val, ttbr1); return;
> + case TCR_EL1: write_sysreg_el1(val, tcr); return;
> + case ESR_EL1: write_sysreg_el1(val, esr); return;
> + case FAR_EL1: write_sysreg_el1(val, far); return;
> + case AFSR0_EL1: write_sysreg_el1(val, afsr0); return;
> + case AFSR1_EL1: write_sysreg_el1(val, afsr1); return;
> + case MAIR_EL1: write_sysreg_el1(val, mair); return;
> + case AMAIR_EL1: write_sysreg_el1(val, amair); return;
> + case CONTEXTIDR_EL1: write_sysreg_el1(val, contextidr); return;
> + case DACR32_EL2: write_sysreg(val, dacr32_el2); return;
> + case IFSR32_EL2: write_sysreg(val, ifsr32_el2); return;
> + default: BUG();
> + }
> + }
> +
> + vcpu_sys_reg(vcpu, reg) = val;
> +}
> +
> /*
> * Generic accessor for VM registers. Only called as long as HCR_TVM
> + *
> * is set. If the guest enables the MMU, we stop trapping the VM
> * sys_regs and leave it in complete control of the caches.
> */
> @@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> if (!p->is_aarch32 || !p->is_32bit) {
> val = p->regval;
> } else {
> - val = vcpu_sys_reg(vcpu, reg);
> + val = read_deferrable_vm_reg(vcpu, reg);
> if (r->reg % 2)
> val = (p->regval << 32) | (u64)lower_32_bits(val);
> else
> val = ((u64)upper_32_bits(val) << 32) |
> (u64)lower_32_bits(p->regval);
> }
> - vcpu_sys_reg(vcpu, reg) = val;
> + write_deferrable_vm_reg(vcpu, reg, val);
>
> kvm_toggle_cache(vcpu, was_enabled);
> return true;
> --
> 2.9.0
>
I read ahead and see other wrappers that check sysregs_loaded_on_cpu are
added, but only write_deferrable_vm_reg() has 'deferrable' in its name.
Should it just be named read/write_vm_reg?
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
@ 2017-11-13 17:54 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 17:54 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:27PM +0200, Christoffer Dall wrote:
> When we defer the save/restore of system registers to vcpu_load and
> vcpu_put, we need to take care of the emulation code that handles traps
> to these registers, since simply reading the memory array will return
> stale data.
>
> Therefore, introduce two functions to directly read/write the registers
> from the physical CPU when we're on a VHE system that has loaded the
> system registers onto the physical CPU.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_host.h | 4 +++
> arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++++++--
> 2 files changed, 56 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 9f5761f..dcded44 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -278,6 +278,10 @@ struct kvm_vcpu_arch {
>
> /* Detect first run of a vcpu */
> bool has_run_once;
> +
> + /* True when deferrable sysregs are loaded on the physical CPU,
> + * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */
> + bool sysregs_loaded_on_cpu;
To be sure I understand correctly, this should only be false when we're
servicing ioctls that don't do a full vcpu_load first (non-KVM_RUN) or
when we're not running on a CPU with VHE. If we didn't need to worry
about non-KVM_RUN ioctls, then we could use the static key has_vhe(),
right?
> };
>
> #define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index dbe35fd..f7887dd 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -34,6 +34,7 @@
> #include <asm/kvm_coproc.h>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_host.h>
> +#include <asm/kvm_hyp.h>
> #include <asm/kvm_mmu.h>
> #include <asm/perf_event.h>
> #include <asm/sysreg.h>
> @@ -110,8 +111,57 @@ static bool access_dcsw(struct kvm_vcpu *vcpu,
> return true;
> }
>
> +static u64 read_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + switch (reg) {
> + case SCTLR_EL1: return read_sysreg_el1(sctlr);
> + case TTBR0_EL1: return read_sysreg_el1(ttbr0);
> + case TTBR1_EL1: return read_sysreg_el1(ttbr1);
> + case TCR_EL1: return read_sysreg_el1(tcr);
> + case ESR_EL1: return read_sysreg_el1(esr);
> + case FAR_EL1: return read_sysreg_el1(far);
> + case AFSR0_EL1: return read_sysreg_el1(afsr0);
> + case AFSR1_EL1: return read_sysreg_el1(afsr1);
> + case MAIR_EL1: return read_sysreg_el1(mair);
> + case AMAIR_EL1: return read_sysreg_el1(amair);
> + case CONTEXTIDR_EL1: return read_sysreg_el1(contextidr);
> + case DACR32_EL2: return read_sysreg(dacr32_el2);
> + case IFSR32_EL2: return read_sysreg(ifsr32_el2);
> + default: BUG();
> + }
> + }
> +
> + return vcpu_sys_reg(vcpu, reg);
> +}
> +
> +static void write_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg, u64 val)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + switch (reg) {
> + case SCTLR_EL1: write_sysreg_el1(val, sctlr); return;
> + case TTBR0_EL1: write_sysreg_el1(val, ttbr0); return;
> + case TTBR1_EL1: write_sysreg_el1(val, ttbr1); return;
> + case TCR_EL1: write_sysreg_el1(val, tcr); return;
> + case ESR_EL1: write_sysreg_el1(val, esr); return;
> + case FAR_EL1: write_sysreg_el1(val, far); return;
> + case AFSR0_EL1: write_sysreg_el1(val, afsr0); return;
> + case AFSR1_EL1: write_sysreg_el1(val, afsr1); return;
> + case MAIR_EL1: write_sysreg_el1(val, mair); return;
> + case AMAIR_EL1: write_sysreg_el1(val, amair); return;
> + case CONTEXTIDR_EL1: write_sysreg_el1(val, contextidr); return;
> + case DACR32_EL2: write_sysreg(val, dacr32_el2); return;
> + case IFSR32_EL2: write_sysreg(val, ifsr32_el2); return;
> + default: BUG();
> + }
> + }
> +
> + vcpu_sys_reg(vcpu, reg) = val;
> +}
> +
> /*
> * Generic accessor for VM registers. Only called as long as HCR_TVM
> + *
> * is set. If the guest enables the MMU, we stop trapping the VM
> * sys_regs and leave it in complete control of the caches.
> */
> @@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> if (!p->is_aarch32 || !p->is_32bit) {
> val = p->regval;
> } else {
> - val = vcpu_sys_reg(vcpu, reg);
> + val = read_deferrable_vm_reg(vcpu, reg);
> if (r->reg % 2)
> val = (p->regval << 32) | (u64)lower_32_bits(val);
> else
> val = ((u64)upper_32_bits(val) << 32) |
> (u64)lower_32_bits(p->regval);
> }
> - vcpu_sys_reg(vcpu, reg) = val;
> + write_deferrable_vm_reg(vcpu, reg, val);
>
> kvm_toggle_cache(vcpu, was_enabled);
> return true;
> --
> 2.9.0
>
I read ahead and see other wrappers that check sysregs_loaded_on_cpu are
added, but only write_deferrable_vm_reg() has 'deferrable' in its name.
Should it just be named read/write_vm_reg?
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
2017-11-13 17:54 ` Andrew Jones
@ 2017-12-03 19:50 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:50 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Mon, Nov 13, 2017 at 06:54:02PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:27PM +0200, Christoffer Dall wrote:
> > When we defer the save/restore of system registers to vcpu_load and
> > vcpu_put, we need to take care of the emulation code that handles traps
> > to these registers, since simply reading the memory array will return
> > stale data.
> >
> > Therefore, introduce two functions to directly read/write the registers
> > from the physical CPU when we're on a VHE system that has loaded the
> > system registers onto the physical CPU.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_host.h | 4 +++
> > arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++++++--
> > 2 files changed, 56 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 9f5761f..dcded44 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -278,6 +278,10 @@ struct kvm_vcpu_arch {
> >
> > /* Detect first run of a vcpu */
> > bool has_run_once;
> > +
> > + /* True when deferrable sysregs are loaded on the physical CPU,
> > + * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */
> > + bool sysregs_loaded_on_cpu;
>
> To be sure I understand correctly, this should only be false when we're
> servicing ioctls that don't do a full vcpu_load first (non-KVM_RUN) or
> when we're not running on a CPU with VHE. If we didn't need to worry
> about non-KVM_RUN ioctls, then we could use the static key has_vhe(),
> right?
>
That is correct, if we also liked the idea of replacing the entire CPU
state every time we read/write a single register from user space.
However, as this isn't performance critical, I'd much rather have a flag
that tells me where to go look for the latest value of a system
register.
> > };
> >
> > #define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index dbe35fd..f7887dd 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -34,6 +34,7 @@
> > #include <asm/kvm_coproc.h>
> > #include <asm/kvm_emulate.h>
> > #include <asm/kvm_host.h>
> > +#include <asm/kvm_hyp.h>
> > #include <asm/kvm_mmu.h>
> > #include <asm/perf_event.h>
> > #include <asm/sysreg.h>
> > @@ -110,8 +111,57 @@ static bool access_dcsw(struct kvm_vcpu *vcpu,
> > return true;
> > }
> >
> > +static u64 read_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + switch (reg) {
> > + case SCTLR_EL1: return read_sysreg_el1(sctlr);
> > + case TTBR0_EL1: return read_sysreg_el1(ttbr0);
> > + case TTBR1_EL1: return read_sysreg_el1(ttbr1);
> > + case TCR_EL1: return read_sysreg_el1(tcr);
> > + case ESR_EL1: return read_sysreg_el1(esr);
> > + case FAR_EL1: return read_sysreg_el1(far);
> > + case AFSR0_EL1: return read_sysreg_el1(afsr0);
> > + case AFSR1_EL1: return read_sysreg_el1(afsr1);
> > + case MAIR_EL1: return read_sysreg_el1(mair);
> > + case AMAIR_EL1: return read_sysreg_el1(amair);
> > + case CONTEXTIDR_EL1: return read_sysreg_el1(contextidr);
> > + case DACR32_EL2: return read_sysreg(dacr32_el2);
> > + case IFSR32_EL2: return read_sysreg(ifsr32_el2);
> > + default: BUG();
> > + }
> > + }
> > +
> > + return vcpu_sys_reg(vcpu, reg);
> > +}
> > +
> > +static void write_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg, u64 val)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + switch (reg) {
> > + case SCTLR_EL1: write_sysreg_el1(val, sctlr); return;
> > + case TTBR0_EL1: write_sysreg_el1(val, ttbr0); return;
> > + case TTBR1_EL1: write_sysreg_el1(val, ttbr1); return;
> > + case TCR_EL1: write_sysreg_el1(val, tcr); return;
> > + case ESR_EL1: write_sysreg_el1(val, esr); return;
> > + case FAR_EL1: write_sysreg_el1(val, far); return;
> > + case AFSR0_EL1: write_sysreg_el1(val, afsr0); return;
> > + case AFSR1_EL1: write_sysreg_el1(val, afsr1); return;
> > + case MAIR_EL1: write_sysreg_el1(val, mair); return;
> > + case AMAIR_EL1: write_sysreg_el1(val, amair); return;
> > + case CONTEXTIDR_EL1: write_sysreg_el1(val, contextidr); return;
> > + case DACR32_EL2: write_sysreg(val, dacr32_el2); return;
> > + case IFSR32_EL2: write_sysreg(val, ifsr32_el2); return;
> > + default: BUG();
> > + }
> > + }
> > +
> > + vcpu_sys_reg(vcpu, reg) = val;
> > +}
> > +
> > /*
> > * Generic accessor for VM registers. Only called as long as HCR_TVM
> > + *
> > * is set. If the guest enables the MMU, we stop trapping the VM
> > * sys_regs and leave it in complete control of the caches.
> > */
> > @@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> > if (!p->is_aarch32 || !p->is_32bit) {
> > val = p->regval;
> > } else {
> > - val = vcpu_sys_reg(vcpu, reg);
> > + val = read_deferrable_vm_reg(vcpu, reg);
> > if (r->reg % 2)
> > val = (p->regval << 32) | (u64)lower_32_bits(val);
> > else
> > val = ((u64)upper_32_bits(val) << 32) |
> > (u64)lower_32_bits(p->regval);
> > }
> > - vcpu_sys_reg(vcpu, reg) = val;
> > + write_deferrable_vm_reg(vcpu, reg, val);
> >
> > kvm_toggle_cache(vcpu, was_enabled);
> > return true;
> > --
> > 2.9.0
> >
>
> I read ahead and see other wrappers that check sysregs_loaded_on_cpu are
> added, but only write_deferrable_vm_reg() has 'deferrable' in its name.
> Should it just be named read/write_vm_reg?
>
I chose this name to avoid implying that this function was universally
supported for all (current and future) VM registers.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
@ 2017-12-03 19:50 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:50 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 13, 2017 at 06:54:02PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:27PM +0200, Christoffer Dall wrote:
> > When we defer the save/restore of system registers to vcpu_load and
> > vcpu_put, we need to take care of the emulation code that handles traps
> > to these registers, since simply reading the memory array will return
> > stale data.
> >
> > Therefore, introduce two functions to directly read/write the registers
> > from the physical CPU when we're on a VHE system that has loaded the
> > system registers onto the physical CPU.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_host.h | 4 +++
> > arch/arm64/kvm/sys_regs.c | 54 +++++++++++++++++++++++++++++++++++++--
> > 2 files changed, 56 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 9f5761f..dcded44 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -278,6 +278,10 @@ struct kvm_vcpu_arch {
> >
> > /* Detect first run of a vcpu */
> > bool has_run_once;
> > +
> > + /* True when deferrable sysregs are loaded on the physical CPU,
> > + * see kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs. */
> > + bool sysregs_loaded_on_cpu;
>
> To be sure I understand correctly, this should only be false when we're
> servicing ioctls that don't do a full vcpu_load first (non-KVM_RUN) or
> when we're not running on a CPU with VHE. If we didn't need to worry
> about non-KVM_RUN ioctls, then we could use the static key has_vhe(),
> right?
>
That is correct, if we also liked the idea of replacing the entire CPU
state every time we read/write a single register from user space.
However, as this isn't performance critical, I'd much rather have a flag
that tells me where to go look for the latest value of a system
register.
> > };
> >
> > #define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index dbe35fd..f7887dd 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -34,6 +34,7 @@
> > #include <asm/kvm_coproc.h>
> > #include <asm/kvm_emulate.h>
> > #include <asm/kvm_host.h>
> > +#include <asm/kvm_hyp.h>
> > #include <asm/kvm_mmu.h>
> > #include <asm/perf_event.h>
> > #include <asm/sysreg.h>
> > @@ -110,8 +111,57 @@ static bool access_dcsw(struct kvm_vcpu *vcpu,
> > return true;
> > }
> >
> > +static u64 read_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + switch (reg) {
> > + case SCTLR_EL1: return read_sysreg_el1(sctlr);
> > + case TTBR0_EL1: return read_sysreg_el1(ttbr0);
> > + case TTBR1_EL1: return read_sysreg_el1(ttbr1);
> > + case TCR_EL1: return read_sysreg_el1(tcr);
> > + case ESR_EL1: return read_sysreg_el1(esr);
> > + case FAR_EL1: return read_sysreg_el1(far);
> > + case AFSR0_EL1: return read_sysreg_el1(afsr0);
> > + case AFSR1_EL1: return read_sysreg_el1(afsr1);
> > + case MAIR_EL1: return read_sysreg_el1(mair);
> > + case AMAIR_EL1: return read_sysreg_el1(amair);
> > + case CONTEXTIDR_EL1: return read_sysreg_el1(contextidr);
> > + case DACR32_EL2: return read_sysreg(dacr32_el2);
> > + case IFSR32_EL2: return read_sysreg(ifsr32_el2);
> > + default: BUG();
> > + }
> > + }
> > +
> > + return vcpu_sys_reg(vcpu, reg);
> > +}
> > +
> > +static void write_deferrable_vm_reg(struct kvm_vcpu *vcpu, int reg, u64 val)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + switch (reg) {
> > + case SCTLR_EL1: write_sysreg_el1(val, sctlr); return;
> > + case TTBR0_EL1: write_sysreg_el1(val, ttbr0); return;
> > + case TTBR1_EL1: write_sysreg_el1(val, ttbr1); return;
> > + case TCR_EL1: write_sysreg_el1(val, tcr); return;
> > + case ESR_EL1: write_sysreg_el1(val, esr); return;
> > + case FAR_EL1: write_sysreg_el1(val, far); return;
> > + case AFSR0_EL1: write_sysreg_el1(val, afsr0); return;
> > + case AFSR1_EL1: write_sysreg_el1(val, afsr1); return;
> > + case MAIR_EL1: write_sysreg_el1(val, mair); return;
> > + case AMAIR_EL1: write_sysreg_el1(val, amair); return;
> > + case CONTEXTIDR_EL1: write_sysreg_el1(val, contextidr); return;
> > + case DACR32_EL2: write_sysreg(val, dacr32_el2); return;
> > + case IFSR32_EL2: write_sysreg(val, ifsr32_el2); return;
> > + default: BUG();
> > + }
> > + }
> > +
> > + vcpu_sys_reg(vcpu, reg) = val;
> > +}
> > +
> > /*
> > * Generic accessor for VM registers. Only called as long as HCR_TVM
> > + *
> > * is set. If the guest enables the MMU, we stop trapping the VM
> > * sys_regs and leave it in complete control of the caches.
> > */
> > @@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> > if (!p->is_aarch32 || !p->is_32bit) {
> > val = p->regval;
> > } else {
> > - val = vcpu_sys_reg(vcpu, reg);
> > + val = read_deferrable_vm_reg(vcpu, reg);
> > if (r->reg % 2)
> > val = (p->regval << 32) | (u64)lower_32_bits(val);
> > else
> > val = ((u64)upper_32_bits(val) << 32) |
> > (u64)lower_32_bits(p->regval);
> > }
> > - vcpu_sys_reg(vcpu, reg) = val;
> > + write_deferrable_vm_reg(vcpu, reg, val);
> >
> > kvm_toggle_cache(vcpu, was_enabled);
> > return true;
> > --
> > 2.9.0
> >
>
> I read ahead and see other wrappers that check sysregs_loaded_on_cpu are
> added, but only write_deferrable_vm_reg() has 'deferrable' in its name.
> Should it just be named read/write_vm_reg?
>
I chose this name to avoid implying that this function was universally
supported for all (current and future) VM registers.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
2017-12-03 19:50 ` Christoffer Dall
@ 2017-12-04 10:05 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-12-04 10:05 UTC (permalink / raw)
To: Christoffer Dall
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Sun, Dec 03, 2017 at 08:50:26PM +0100, Christoffer Dall wrote:
> On Mon, Nov 13, 2017 at 06:54:02PM +0100, Andrew Jones wrote:
...
> > > + }
> > > +
> > > + vcpu_sys_reg(vcpu, reg) = val;
> > > +}
> > > +
> > > /*
> > > * Generic accessor for VM registers. Only called as long as HCR_TVM
> > > + *
just noticed this stray blank line added here
> > > * is set. If the guest enables the MMU, we stop trapping the VM
> > > * sys_regs and leave it in complete control of the caches.
> > > */
> > > @@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> > > if (!p->is_aarch32 || !p->is_32bit) {
> > > val = p->regval;
> > > } else {
> > > - val = vcpu_sys_reg(vcpu, reg);
> > > + val = read_deferrable_vm_reg(vcpu, reg);
> > > if (r->reg % 2)
> > > val = (p->regval << 32) | (u64)lower_32_bits(val);
> > > else
> > > val = ((u64)upper_32_bits(val) << 32) |
> > > (u64)lower_32_bits(p->regval);
> > > }
> > > - vcpu_sys_reg(vcpu, reg) = val;
> > > + write_deferrable_vm_reg(vcpu, reg, val);
> > >
> > > kvm_toggle_cache(vcpu, was_enabled);
> > > return true;
> > > --
> > > 2.9.0
> > >
> >
> > I read ahead and see other wrappers that check sysregs_loaded_on_cpu are
> > added, but only write_deferrable_vm_reg() has 'deferrable' in its name.
> > Should it just be named read/write_vm_reg?
> >
>
> I chose this name to avoid implying that this function was universally
> supported for all (current and future) VM registers.
OK
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 23/37] KVM: arm64: Prepare to handle traps on deferred VM sysregs
@ 2017-12-04 10:05 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-12-04 10:05 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Dec 03, 2017 at 08:50:26PM +0100, Christoffer Dall wrote:
> On Mon, Nov 13, 2017 at 06:54:02PM +0100, Andrew Jones wrote:
...
> > > + }
> > > +
> > > + vcpu_sys_reg(vcpu, reg) = val;
> > > +}
> > > +
> > > /*
> > > * Generic accessor for VM registers. Only called as long as HCR_TVM
> > > + *
just noticed this stray blank line added here
> > > * is set. If the guest enables the MMU, we stop trapping the VM
> > > * sys_regs and leave it in complete control of the caches.
> > > */
> > > @@ -132,14 +182,14 @@ static bool access_vm_reg(struct kvm_vcpu *vcpu,
> > > if (!p->is_aarch32 || !p->is_32bit) {
> > > val = p->regval;
> > > } else {
> > > - val = vcpu_sys_reg(vcpu, reg);
> > > + val = read_deferrable_vm_reg(vcpu, reg);
> > > if (r->reg % 2)
> > > val = (p->regval << 32) | (u64)lower_32_bits(val);
> > > else
> > > val = ((u64)upper_32_bits(val) << 32) |
> > > (u64)lower_32_bits(p->regval);
> > > }
> > > - vcpu_sys_reg(vcpu, reg) = val;
> > > + write_deferrable_vm_reg(vcpu, reg, val);
> > >
> > > kvm_toggle_cache(vcpu, was_enabled);
> > > return true;
> > > --
> > > 2.9.0
> > >
> >
> > I read ahead and see other wrappers that check sysregs_loaded_on_cpu are
> > added, but only write_deferrable_vm_reg() has 'deferrable' in its name.
> > Should it just be named read/write_vm_reg?
> >
>
> I chose this name to avoid implying that this function was universally
> supported for all (current and future) VM registers.
OK
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 24/37] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
We can trap access to ACTLR_EL1 which we can later defer to only
save/restore during vcpu_load and vcpu_put, so let's read the value
directly from the CPU when necessary.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/sys_regs_generic_v8.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs_generic_v8.c b/arch/arm64/kvm/sys_regs_generic_v8.c
index 969ade1..348f227 100644
--- a/arch/arm64/kvm/sys_regs_generic_v8.c
+++ b/arch/arm64/kvm/sys_regs_generic_v8.c
@@ -38,7 +38,10 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
if (p->is_write)
return ignore_write(vcpu, p);
- p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ read_sysreg(actlr_el1);
+ else
+ p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
return true;
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 24/37] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We can trap access to ACTLR_EL1 which we can later defer to only
save/restore during vcpu_load and vcpu_put, so let's read the value
directly from the CPU when necessary.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/sys_regs_generic_v8.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs_generic_v8.c b/arch/arm64/kvm/sys_regs_generic_v8.c
index 969ade1..348f227 100644
--- a/arch/arm64/kvm/sys_regs_generic_v8.c
+++ b/arch/arm64/kvm/sys_regs_generic_v8.c
@@ -38,7 +38,10 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
if (p->is_write)
return ignore_write(vcpu, p);
- p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ read_sysreg(actlr_el1);
+ else
+ p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
return true;
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* Re: [PATCH 24/37] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-15 9:25 ` Julien Thierry
-1 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-15 9:25 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
On 12/10/17 11:41, Christoffer Dall wrote:
> We can trap access to ACTLR_EL1 which we can later defer to only
> save/restore during vcpu_load and vcpu_put, so let's read the value
> directly from the CPU when necessary.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/sys_regs_generic_v8.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/sys_regs_generic_v8.c b/arch/arm64/kvm/sys_regs_generic_v8.c
> index 969ade1..348f227 100644
> --- a/arch/arm64/kvm/sys_regs_generic_v8.c
> +++ b/arch/arm64/kvm/sys_regs_generic_v8.c
> @@ -38,7 +38,10 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
> if (p->is_write)
> return ignore_write(vcpu, p);
>
> - p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + read_sysreg(actlr_el1);
Did you mean "p->regval = read_sysreg(actlr_el1);" ?
> + else
> + p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> return true;
> }
>
>
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 24/37] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
@ 2017-11-15 9:25 ` Julien Thierry
0 siblings, 0 replies; 254+ messages in thread
From: Julien Thierry @ 2017-11-15 9:25 UTC (permalink / raw)
To: linux-arm-kernel
On 12/10/17 11:41, Christoffer Dall wrote:
> We can trap access to ACTLR_EL1 which we can later defer to only
> save/restore during vcpu_load and vcpu_put, so let's read the value
> directly from the CPU when necessary.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/sys_regs_generic_v8.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/sys_regs_generic_v8.c b/arch/arm64/kvm/sys_regs_generic_v8.c
> index 969ade1..348f227 100644
> --- a/arch/arm64/kvm/sys_regs_generic_v8.c
> +++ b/arch/arm64/kvm/sys_regs_generic_v8.c
> @@ -38,7 +38,10 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
> if (p->is_write)
> return ignore_write(vcpu, p);
>
> - p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + read_sysreg(actlr_el1);
Did you mean "p->regval = read_sysreg(actlr_el1);" ?
> + else
> + p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> return true;
> }
>
>
--
Julien Thierry
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 24/37] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
2017-11-15 9:25 ` Julien Thierry
@ 2017-12-03 19:51 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:51 UTC (permalink / raw)
To: Julien Thierry
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Wed, Nov 15, 2017 at 09:25:08AM +0000, Julien Thierry wrote:
>
>
> On 12/10/17 11:41, Christoffer Dall wrote:
> >We can trap access to ACTLR_EL1 which we can later defer to only
> >save/restore during vcpu_load and vcpu_put, so let's read the value
> >directly from the CPU when necessary.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >---
> > arch/arm64/kvm/sys_regs_generic_v8.c | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> >diff --git a/arch/arm64/kvm/sys_regs_generic_v8.c b/arch/arm64/kvm/sys_regs_generic_v8.c
> >index 969ade1..348f227 100644
> >--- a/arch/arm64/kvm/sys_regs_generic_v8.c
> >+++ b/arch/arm64/kvm/sys_regs_generic_v8.c
> >@@ -38,7 +38,10 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
> > if (p->is_write)
> > return ignore_write(vcpu, p);
> >- p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> >+ if (vcpu->arch.sysregs_loaded_on_cpu)
> >+ read_sysreg(actlr_el1);
>
> Did you mean "p->regval = read_sysreg(actlr_el1);" ?
>
> >+ else
> >+ p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> > return true;
> > }
> >
>
Yes, most certainly. Nice catch.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 24/37] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs
@ 2017-12-03 19:51 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 19:51 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 15, 2017 at 09:25:08AM +0000, Julien Thierry wrote:
>
>
> On 12/10/17 11:41, Christoffer Dall wrote:
> >We can trap access to ACTLR_EL1 which we can later defer to only
> >save/restore during vcpu_load and vcpu_put, so let's read the value
> >directly from the CPU when necessary.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >---
> > arch/arm64/kvm/sys_regs_generic_v8.c | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> >diff --git a/arch/arm64/kvm/sys_regs_generic_v8.c b/arch/arm64/kvm/sys_regs_generic_v8.c
> >index 969ade1..348f227 100644
> >--- a/arch/arm64/kvm/sys_regs_generic_v8.c
> >+++ b/arch/arm64/kvm/sys_regs_generic_v8.c
> >@@ -38,7 +38,10 @@ static bool access_actlr(struct kvm_vcpu *vcpu,
> > if (p->is_write)
> > return ignore_write(vcpu, p);
> >- p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> >+ if (vcpu->arch.sysregs_loaded_on_cpu)
> >+ read_sysreg(actlr_el1);
>
> Did you mean "p->regval = read_sysreg(actlr_el1);" ?
>
> >+ else
> >+ p->regval = vcpu_sys_reg(vcpu, ACTLR_EL1);
> > return true;
> > }
> >
>
Yes, most certainly. Nice catch.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 25/37] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Handle accesses during traps to any remaining EL1 registers which can be
deferred to vcpu_load and vcpu_put, by either accessing them directly on
the physical CPU when the latest version is stored there, or by
synchronizing the memory representation with the CPU state.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_emulate.h | 14 -------
arch/arm64/kvm/inject_fault.c | 79 ++++++++++++++++++++++++++++++++----
arch/arm64/kvm/sys_regs.c | 6 ++-
3 files changed, 76 insertions(+), 23 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 630dd60..69bb40d 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -66,11 +66,6 @@ static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
}
-static inline unsigned long *vcpu_elr_el1(const struct kvm_vcpu *vcpu)
-{
- return (unsigned long *)&vcpu_gp_regs(vcpu)->elr_el1;
-}
-
static inline unsigned long *vcpu_cpsr(const struct kvm_vcpu *vcpu)
{
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pstate;
@@ -120,15 +115,6 @@ static inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
vcpu_gp_regs(vcpu)->regs.regs[reg_num] = val;
}
-/* Get vcpu SPSR for current mode */
-static inline unsigned long *vcpu_spsr(const struct kvm_vcpu *vcpu)
-{
- if (vcpu_mode_is_32bit(vcpu))
- return vcpu_spsr32(vcpu);
-
- return (unsigned long *)&vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1];
-}
-
static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
{
u32 mode;
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index 45c7026..f4513fc 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -23,6 +23,7 @@
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_hyp.h>
#include <asm/esr.h>
#define PSTATE_FAULT_BITS_64 (PSR_MODE_EL1h | PSR_A_BIT | PSR_F_BIT | \
@@ -33,13 +34,55 @@
#define LOWER_EL_AArch64_VECTOR 0x400
#define LOWER_EL_AArch32_VECTOR 0x600
+static u64 vcpu_get_vbar_el1(struct kvm_vcpu *vcpu)
+{
+ unsigned long vbar;
+
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ vbar = read_sysreg_el1(vbar);
+ else
+ vbar = vcpu_sys_reg(vcpu, VBAR_EL1);
+
+ if (vcpu_el1_is_32bit(vcpu))
+ return lower_32_bits(vbar);
+ return vbar;
+}
+
+static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ write_sysreg_el1(val, elr);
+ else
+ vcpu_gp_regs(vcpu)->elr_el1 = val;
+}
+
+/* Set the SPSR for the current mode */
+static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
+{
+ if (vcpu_mode_is_32bit(vcpu))
+ *vcpu_spsr32(vcpu) = val;
+
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ write_sysreg_el1(val, spsr);
+ else
+ vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1] = val;
+}
+
+static u32 vcpu_get_c1_sctlr(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ return lower_32_bits(read_sysreg_el1(sctlr));
+ else
+ return vcpu_cp15(vcpu, c1_SCTLR);
+}
+
static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
{
unsigned long cpsr;
unsigned long new_spsr_value = *vcpu_cpsr(vcpu);
bool is_thumb = (new_spsr_value & COMPAT_PSR_T_BIT);
u32 return_offset = (is_thumb) ? 4 : 0;
- u32 sctlr = vcpu_cp15(vcpu, c1_SCTLR);
+ u32 sctlr = vcpu_get_c1_sctlr(vcpu);
cpsr = mode | COMPAT_PSR_I_BIT;
@@ -51,14 +94,14 @@ static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
*vcpu_cpsr(vcpu) = cpsr;
/* Note: These now point to the banked copies */
- *vcpu_spsr(vcpu) = new_spsr_value;
+ vcpu_set_spsr(vcpu, new_spsr_value);
*vcpu_reg32(vcpu, 14) = *vcpu_pc(vcpu) + return_offset;
/* Branch to exception vector */
if (sctlr & (1 << 13))
vect_offset += 0xffff0000;
else /* always have security exceptions */
- vect_offset += vcpu_cp15(vcpu, c12_VBAR);
+ vect_offset += vcpu_get_vbar_el1(vcpu);
*vcpu_pc(vcpu) = vect_offset;
}
@@ -79,6 +122,20 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
u32 *far, *fsr;
bool is_lpae;
+ /*
+ * We are going to need the latest values of the following system
+ * regiters:
+ * DFAR: mapped to FAR_EL1
+ * IFAR: mapped to FAR_EL1
+ * DFSR: mapped to ESR_EL1
+ * TTBCR: mapped to TCR_EL1
+ */
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
+ vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
+ vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
+ }
+
if (is_pabt) {
vect_offset = 12;
far = &vcpu_cp15(vcpu, c6_IFAR);
@@ -99,6 +156,12 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
*fsr = 1 << 9 | 0x34;
else
*fsr = 0x14;
+
+ /* Sync back any registers we may have changed */
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
+ write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
+ }
}
enum exception_type {
@@ -126,7 +189,7 @@ static u64 get_except_vector(struct kvm_vcpu *vcpu, enum exception_type type)
exc_offset = LOWER_EL_AArch32_VECTOR;
}
- return vcpu_sys_reg(vcpu, VBAR_EL1) + exc_offset + type;
+ return vcpu_get_vbar_el1(vcpu) + exc_offset + type;
}
static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
@@ -135,11 +198,11 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
u32 esr = 0;
- *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
+ vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
*vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
*vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
- *vcpu_spsr(vcpu) = cpsr;
+ vcpu_set_spsr(vcpu, cpsr);
vcpu_sys_reg(vcpu, FAR_EL1) = addr;
@@ -170,11 +233,11 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
unsigned long cpsr = *vcpu_cpsr(vcpu);
u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
- *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
+ vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
*vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
*vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
- *vcpu_spsr(vcpu) = cpsr;
+ vcpu_set_spsr(vcpu, cpsr);
/*
* Build an unknown exception, depending on the instruction
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index f7887dd..60d1660 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -86,12 +86,16 @@ static u32 cache_levels;
static u32 get_ccsidr(u32 csselr)
{
u32 ccsidr;
+ u32 csselr_preserve;
- /* Make sure noone else changes CSSELR during this! */
+ /* Make sure noone else changes CSSELR during this and preserve any
+ * existing value in the CSSELR! */
local_irq_disable();
+ csselr_preserve = read_sysreg(csselr_el1);
write_sysreg(csselr, csselr_el1);
isb();
ccsidr = read_sysreg(ccsidr_el1);
+ write_sysreg(csselr_preserve, csselr_el1);
local_irq_enable();
return ccsidr;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 25/37] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Handle accesses during traps to any remaining EL1 registers which can be
deferred to vcpu_load and vcpu_put, by either accessing them directly on
the physical CPU when the latest version is stored there, or by
synchronizing the memory representation with the CPU state.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_emulate.h | 14 -------
arch/arm64/kvm/inject_fault.c | 79 ++++++++++++++++++++++++++++++++----
arch/arm64/kvm/sys_regs.c | 6 ++-
3 files changed, 76 insertions(+), 23 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 630dd60..69bb40d 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -66,11 +66,6 @@ static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
}
-static inline unsigned long *vcpu_elr_el1(const struct kvm_vcpu *vcpu)
-{
- return (unsigned long *)&vcpu_gp_regs(vcpu)->elr_el1;
-}
-
static inline unsigned long *vcpu_cpsr(const struct kvm_vcpu *vcpu)
{
return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pstate;
@@ -120,15 +115,6 @@ static inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
vcpu_gp_regs(vcpu)->regs.regs[reg_num] = val;
}
-/* Get vcpu SPSR for current mode */
-static inline unsigned long *vcpu_spsr(const struct kvm_vcpu *vcpu)
-{
- if (vcpu_mode_is_32bit(vcpu))
- return vcpu_spsr32(vcpu);
-
- return (unsigned long *)&vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1];
-}
-
static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
{
u32 mode;
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index 45c7026..f4513fc 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -23,6 +23,7 @@
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_hyp.h>
#include <asm/esr.h>
#define PSTATE_FAULT_BITS_64 (PSR_MODE_EL1h | PSR_A_BIT | PSR_F_BIT | \
@@ -33,13 +34,55 @@
#define LOWER_EL_AArch64_VECTOR 0x400
#define LOWER_EL_AArch32_VECTOR 0x600
+static u64 vcpu_get_vbar_el1(struct kvm_vcpu *vcpu)
+{
+ unsigned long vbar;
+
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ vbar = read_sysreg_el1(vbar);
+ else
+ vbar = vcpu_sys_reg(vcpu, VBAR_EL1);
+
+ if (vcpu_el1_is_32bit(vcpu))
+ return lower_32_bits(vbar);
+ return vbar;
+}
+
+static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ write_sysreg_el1(val, elr);
+ else
+ vcpu_gp_regs(vcpu)->elr_el1 = val;
+}
+
+/* Set the SPSR for the current mode */
+static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
+{
+ if (vcpu_mode_is_32bit(vcpu))
+ *vcpu_spsr32(vcpu) = val;
+
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ write_sysreg_el1(val, spsr);
+ else
+ vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1] = val;
+}
+
+static u32 vcpu_get_c1_sctlr(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ return lower_32_bits(read_sysreg_el1(sctlr));
+ else
+ return vcpu_cp15(vcpu, c1_SCTLR);
+}
+
static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
{
unsigned long cpsr;
unsigned long new_spsr_value = *vcpu_cpsr(vcpu);
bool is_thumb = (new_spsr_value & COMPAT_PSR_T_BIT);
u32 return_offset = (is_thumb) ? 4 : 0;
- u32 sctlr = vcpu_cp15(vcpu, c1_SCTLR);
+ u32 sctlr = vcpu_get_c1_sctlr(vcpu);
cpsr = mode | COMPAT_PSR_I_BIT;
@@ -51,14 +94,14 @@ static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
*vcpu_cpsr(vcpu) = cpsr;
/* Note: These now point to the banked copies */
- *vcpu_spsr(vcpu) = new_spsr_value;
+ vcpu_set_spsr(vcpu, new_spsr_value);
*vcpu_reg32(vcpu, 14) = *vcpu_pc(vcpu) + return_offset;
/* Branch to exception vector */
if (sctlr & (1 << 13))
vect_offset += 0xffff0000;
else /* always have security exceptions */
- vect_offset += vcpu_cp15(vcpu, c12_VBAR);
+ vect_offset += vcpu_get_vbar_el1(vcpu);
*vcpu_pc(vcpu) = vect_offset;
}
@@ -79,6 +122,20 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
u32 *far, *fsr;
bool is_lpae;
+ /*
+ * We are going to need the latest values of the following system
+ * regiters:
+ * DFAR: mapped to FAR_EL1
+ * IFAR: mapped to FAR_EL1
+ * DFSR: mapped to ESR_EL1
+ * TTBCR: mapped to TCR_EL1
+ */
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
+ vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
+ vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
+ }
+
if (is_pabt) {
vect_offset = 12;
far = &vcpu_cp15(vcpu, c6_IFAR);
@@ -99,6 +156,12 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
*fsr = 1 << 9 | 0x34;
else
*fsr = 0x14;
+
+ /* Sync back any registers we may have changed */
+ if (vcpu->arch.sysregs_loaded_on_cpu) {
+ write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
+ write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
+ }
}
enum exception_type {
@@ -126,7 +189,7 @@ static u64 get_except_vector(struct kvm_vcpu *vcpu, enum exception_type type)
exc_offset = LOWER_EL_AArch32_VECTOR;
}
- return vcpu_sys_reg(vcpu, VBAR_EL1) + exc_offset + type;
+ return vcpu_get_vbar_el1(vcpu) + exc_offset + type;
}
static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
@@ -135,11 +198,11 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
u32 esr = 0;
- *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
+ vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
*vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
*vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
- *vcpu_spsr(vcpu) = cpsr;
+ vcpu_set_spsr(vcpu, cpsr);
vcpu_sys_reg(vcpu, FAR_EL1) = addr;
@@ -170,11 +233,11 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
unsigned long cpsr = *vcpu_cpsr(vcpu);
u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
- *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
+ vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
*vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
*vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
- *vcpu_spsr(vcpu) = cpsr;
+ vcpu_set_spsr(vcpu, cpsr);
/*
* Build an unknown exception, depending on the instruction
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index f7887dd..60d1660 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -86,12 +86,16 @@ static u32 cache_levels;
static u32 get_ccsidr(u32 csselr)
{
u32 ccsidr;
+ u32 csselr_preserve;
- /* Make sure noone else changes CSSELR during this! */
+ /* Make sure noone else changes CSSELR during this and preserve any
+ * existing value in the CSSELR! */
local_irq_disable();
+ csselr_preserve = read_sysreg(csselr_el1);
write_sysreg(csselr, csselr_el1);
isb();
ccsidr = read_sysreg(ccsidr_el1);
+ write_sysreg(csselr_preserve, csselr_el1);
local_irq_enable();
return ccsidr;
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 25/37] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-13 18:56 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 18:56 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:29PM +0200, Christoffer Dall wrote:
> Handle accesses during traps to any remaining EL1 registers which can be
> deferred to vcpu_load and vcpu_put, by either accessing them directly on
> the physical CPU when the latest version is stored there, or by
> synchronizing the memory representation with the CPU state.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_emulate.h | 14 -------
> arch/arm64/kvm/inject_fault.c | 79 ++++++++++++++++++++++++++++++++----
> arch/arm64/kvm/sys_regs.c | 6 ++-
> 3 files changed, 76 insertions(+), 23 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 630dd60..69bb40d 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -66,11 +66,6 @@ static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> }
>
> -static inline unsigned long *vcpu_elr_el1(const struct kvm_vcpu *vcpu)
> -{
> - return (unsigned long *)&vcpu_gp_regs(vcpu)->elr_el1;
> -}
> -
> static inline unsigned long *vcpu_cpsr(const struct kvm_vcpu *vcpu)
> {
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pstate;
> @@ -120,15 +115,6 @@ static inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
> vcpu_gp_regs(vcpu)->regs.regs[reg_num] = val;
> }
>
> -/* Get vcpu SPSR for current mode */
> -static inline unsigned long *vcpu_spsr(const struct kvm_vcpu *vcpu)
> -{
> - if (vcpu_mode_is_32bit(vcpu))
> - return vcpu_spsr32(vcpu);
> -
> - return (unsigned long *)&vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1];
> -}
> -
> static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
> {
> u32 mode;
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index 45c7026..f4513fc 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -23,6 +23,7 @@
>
> #include <linux/kvm_host.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_hyp.h>
> #include <asm/esr.h>
>
> #define PSTATE_FAULT_BITS_64 (PSR_MODE_EL1h | PSR_A_BIT | PSR_F_BIT | \
> @@ -33,13 +34,55 @@
> #define LOWER_EL_AArch64_VECTOR 0x400
> #define LOWER_EL_AArch32_VECTOR 0x600
>
> +static u64 vcpu_get_vbar_el1(struct kvm_vcpu *vcpu)
> +{
> + unsigned long vbar;
> +
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + vbar = read_sysreg_el1(vbar);
> + else
> + vbar = vcpu_sys_reg(vcpu, VBAR_EL1);
> +
> + if (vcpu_el1_is_32bit(vcpu))
> + return lower_32_bits(vbar);
> + return vbar;
> +}
> +
> +static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + write_sysreg_el1(val, elr);
> + else
> + vcpu_gp_regs(vcpu)->elr_el1 = val;
> +}
> +
> +/* Set the SPSR for the current mode */
> +static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> +{
> + if (vcpu_mode_is_32bit(vcpu))
> + *vcpu_spsr32(vcpu) = val;
> +
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + write_sysreg_el1(val, spsr);
> + else
> + vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1] = val;
> +}
> +
> +static u32 vcpu_get_c1_sctlr(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + return lower_32_bits(read_sysreg_el1(sctlr));
> + else
> + return vcpu_cp15(vcpu, c1_SCTLR);
> +}
> +
> static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> {
> unsigned long cpsr;
> unsigned long new_spsr_value = *vcpu_cpsr(vcpu);
> bool is_thumb = (new_spsr_value & COMPAT_PSR_T_BIT);
> u32 return_offset = (is_thumb) ? 4 : 0;
> - u32 sctlr = vcpu_cp15(vcpu, c1_SCTLR);
> + u32 sctlr = vcpu_get_c1_sctlr(vcpu);
>
> cpsr = mode | COMPAT_PSR_I_BIT;
>
> @@ -51,14 +94,14 @@ static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> *vcpu_cpsr(vcpu) = cpsr;
>
> /* Note: These now point to the banked copies */
> - *vcpu_spsr(vcpu) = new_spsr_value;
> + vcpu_set_spsr(vcpu, new_spsr_value);
> *vcpu_reg32(vcpu, 14) = *vcpu_pc(vcpu) + return_offset;
>
> /* Branch to exception vector */
> if (sctlr & (1 << 13))
> vect_offset += 0xffff0000;
> else /* always have security exceptions */
> - vect_offset += vcpu_cp15(vcpu, c12_VBAR);
> + vect_offset += vcpu_get_vbar_el1(vcpu);
>
> *vcpu_pc(vcpu) = vect_offset;
> }
> @@ -79,6 +122,20 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> u32 *far, *fsr;
> bool is_lpae;
>
> + /*
> + * We are going to need the latest values of the following system
> + * regiters:
registers
> + * DFAR: mapped to FAR_EL1
FAR_EL1[31:0]
> + * IFAR: mapped to FAR_EL1
FAR_EL1[63:32]
> + * DFSR: mapped to ESR_EL1
> + * TTBCR: mapped to TCR_EL1
> + */
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> + vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> + vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> + }
> +
> if (is_pabt) {
> vect_offset = 12;
> far = &vcpu_cp15(vcpu, c6_IFAR);
> @@ -99,6 +156,12 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> *fsr = 1 << 9 | 0x34;
> else
> *fsr = 0x14;
> +
> + /* Sync back any registers we may have changed */
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> + }
> }
>
> enum exception_type {
> @@ -126,7 +189,7 @@ static u64 get_except_vector(struct kvm_vcpu *vcpu, enum exception_type type)
> exc_offset = LOWER_EL_AArch32_VECTOR;
> }
>
> - return vcpu_sys_reg(vcpu, VBAR_EL1) + exc_offset + type;
> + return vcpu_get_vbar_el1(vcpu) + exc_offset + type;
> }
>
> static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> @@ -135,11 +198,11 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> u32 esr = 0;
>
> - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
>
> *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> - *vcpu_spsr(vcpu) = cpsr;
> + vcpu_set_spsr(vcpu, cpsr);
>
> vcpu_sys_reg(vcpu, FAR_EL1) = addr;
>
> @@ -170,11 +233,11 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> unsigned long cpsr = *vcpu_cpsr(vcpu);
> u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
>
> - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
>
> *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> - *vcpu_spsr(vcpu) = cpsr;
> + vcpu_set_spsr(vcpu, cpsr);
>
I'm concerned the maintenance of emulation code will become more
difficult now that some registers have special accessors, while
others don't, and some functions have save/restore lists, that
will need to stay maintained with the emulation. The only alternative
that pops to mind, though, is adding get/set members to the register
descriptors and encapsulating the decision in them, but that might be
overkill.
> /*
> * Build an unknown exception, depending on the instruction
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index f7887dd..60d1660 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -86,12 +86,16 @@ static u32 cache_levels;
> static u32 get_ccsidr(u32 csselr)
> {
> u32 ccsidr;
> + u32 csselr_preserve;
>
> - /* Make sure noone else changes CSSELR during this! */
> + /* Make sure noone else changes CSSELR during this and preserve any
> + * existing value in the CSSELR! */
> local_irq_disable();
> + csselr_preserve = read_sysreg(csselr_el1);
> write_sysreg(csselr, csselr_el1);
> isb();
> ccsidr = read_sysreg(ccsidr_el1);
> + write_sysreg(csselr_preserve, csselr_el1);
> local_irq_enable();
>
> return ccsidr;
This looks like an unrelated fix.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 25/37] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
@ 2017-11-13 18:56 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 18:56 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:29PM +0200, Christoffer Dall wrote:
> Handle accesses during traps to any remaining EL1 registers which can be
> deferred to vcpu_load and vcpu_put, by either accessing them directly on
> the physical CPU when the latest version is stored there, or by
> synchronizing the memory representation with the CPU state.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/include/asm/kvm_emulate.h | 14 -------
> arch/arm64/kvm/inject_fault.c | 79 ++++++++++++++++++++++++++++++++----
> arch/arm64/kvm/sys_regs.c | 6 ++-
> 3 files changed, 76 insertions(+), 23 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 630dd60..69bb40d 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -66,11 +66,6 @@ static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> }
>
> -static inline unsigned long *vcpu_elr_el1(const struct kvm_vcpu *vcpu)
> -{
> - return (unsigned long *)&vcpu_gp_regs(vcpu)->elr_el1;
> -}
> -
> static inline unsigned long *vcpu_cpsr(const struct kvm_vcpu *vcpu)
> {
> return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pstate;
> @@ -120,15 +115,6 @@ static inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
> vcpu_gp_regs(vcpu)->regs.regs[reg_num] = val;
> }
>
> -/* Get vcpu SPSR for current mode */
> -static inline unsigned long *vcpu_spsr(const struct kvm_vcpu *vcpu)
> -{
> - if (vcpu_mode_is_32bit(vcpu))
> - return vcpu_spsr32(vcpu);
> -
> - return (unsigned long *)&vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1];
> -}
> -
> static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
> {
> u32 mode;
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index 45c7026..f4513fc 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -23,6 +23,7 @@
>
> #include <linux/kvm_host.h>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_hyp.h>
> #include <asm/esr.h>
>
> #define PSTATE_FAULT_BITS_64 (PSR_MODE_EL1h | PSR_A_BIT | PSR_F_BIT | \
> @@ -33,13 +34,55 @@
> #define LOWER_EL_AArch64_VECTOR 0x400
> #define LOWER_EL_AArch32_VECTOR 0x600
>
> +static u64 vcpu_get_vbar_el1(struct kvm_vcpu *vcpu)
> +{
> + unsigned long vbar;
> +
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + vbar = read_sysreg_el1(vbar);
> + else
> + vbar = vcpu_sys_reg(vcpu, VBAR_EL1);
> +
> + if (vcpu_el1_is_32bit(vcpu))
> + return lower_32_bits(vbar);
> + return vbar;
> +}
> +
> +static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + write_sysreg_el1(val, elr);
> + else
> + vcpu_gp_regs(vcpu)->elr_el1 = val;
> +}
> +
> +/* Set the SPSR for the current mode */
> +static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> +{
> + if (vcpu_mode_is_32bit(vcpu))
> + *vcpu_spsr32(vcpu) = val;
> +
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + write_sysreg_el1(val, spsr);
> + else
> + vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1] = val;
> +}
> +
> +static u32 vcpu_get_c1_sctlr(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + return lower_32_bits(read_sysreg_el1(sctlr));
> + else
> + return vcpu_cp15(vcpu, c1_SCTLR);
> +}
> +
> static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> {
> unsigned long cpsr;
> unsigned long new_spsr_value = *vcpu_cpsr(vcpu);
> bool is_thumb = (new_spsr_value & COMPAT_PSR_T_BIT);
> u32 return_offset = (is_thumb) ? 4 : 0;
> - u32 sctlr = vcpu_cp15(vcpu, c1_SCTLR);
> + u32 sctlr = vcpu_get_c1_sctlr(vcpu);
>
> cpsr = mode | COMPAT_PSR_I_BIT;
>
> @@ -51,14 +94,14 @@ static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> *vcpu_cpsr(vcpu) = cpsr;
>
> /* Note: These now point to the banked copies */
> - *vcpu_spsr(vcpu) = new_spsr_value;
> + vcpu_set_spsr(vcpu, new_spsr_value);
> *vcpu_reg32(vcpu, 14) = *vcpu_pc(vcpu) + return_offset;
>
> /* Branch to exception vector */
> if (sctlr & (1 << 13))
> vect_offset += 0xffff0000;
> else /* always have security exceptions */
> - vect_offset += vcpu_cp15(vcpu, c12_VBAR);
> + vect_offset += vcpu_get_vbar_el1(vcpu);
>
> *vcpu_pc(vcpu) = vect_offset;
> }
> @@ -79,6 +122,20 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> u32 *far, *fsr;
> bool is_lpae;
>
> + /*
> + * We are going to need the latest values of the following system
> + * regiters:
registers
> + * DFAR: mapped to FAR_EL1
FAR_EL1[31:0]
> + * IFAR: mapped to FAR_EL1
FAR_EL1[63:32]
> + * DFSR: mapped to ESR_EL1
> + * TTBCR: mapped to TCR_EL1
> + */
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> + vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> + vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> + }
> +
> if (is_pabt) {
> vect_offset = 12;
> far = &vcpu_cp15(vcpu, c6_IFAR);
> @@ -99,6 +156,12 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> *fsr = 1 << 9 | 0x34;
> else
> *fsr = 0x14;
> +
> + /* Sync back any registers we may have changed */
> + if (vcpu->arch.sysregs_loaded_on_cpu) {
> + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> + }
> }
>
> enum exception_type {
> @@ -126,7 +189,7 @@ static u64 get_except_vector(struct kvm_vcpu *vcpu, enum exception_type type)
> exc_offset = LOWER_EL_AArch32_VECTOR;
> }
>
> - return vcpu_sys_reg(vcpu, VBAR_EL1) + exc_offset + type;
> + return vcpu_get_vbar_el1(vcpu) + exc_offset + type;
> }
>
> static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> @@ -135,11 +198,11 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> u32 esr = 0;
>
> - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
>
> *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> - *vcpu_spsr(vcpu) = cpsr;
> + vcpu_set_spsr(vcpu, cpsr);
>
> vcpu_sys_reg(vcpu, FAR_EL1) = addr;
>
> @@ -170,11 +233,11 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> unsigned long cpsr = *vcpu_cpsr(vcpu);
> u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
>
> - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
>
> *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> - *vcpu_spsr(vcpu) = cpsr;
> + vcpu_set_spsr(vcpu, cpsr);
>
I'm concerned the maintenance of emulation code will become more
difficult now that some registers have special accessors, while
others don't, and some functions have save/restore lists, that
will need to stay maintained with the emulation. The only alternative
that pops to mind, though, is adding get/set members to the register
descriptors and encapsulating the decision in them, but that might be
overkill.
> /*
> * Build an unknown exception, depending on the instruction
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index f7887dd..60d1660 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -86,12 +86,16 @@ static u32 cache_levels;
> static u32 get_ccsidr(u32 csselr)
> {
> u32 ccsidr;
> + u32 csselr_preserve;
>
> - /* Make sure noone else changes CSSELR during this! */
> + /* Make sure noone else changes CSSELR during this and preserve any
> + * existing value in the CSSELR! */
> local_irq_disable();
> + csselr_preserve = read_sysreg(csselr_el1);
> write_sysreg(csselr, csselr_el1);
> isb();
> ccsidr = read_sysreg(ccsidr_el1);
> + write_sysreg(csselr_preserve, csselr_el1);
> local_irq_enable();
>
> return ccsidr;
This looks like an unrelated fix.
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 25/37] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
2017-11-13 18:56 ` Andrew Jones
@ 2017-12-03 20:29 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:29 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Mon, Nov 13, 2017 at 07:56:14PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:29PM +0200, Christoffer Dall wrote:
> > Handle accesses during traps to any remaining EL1 registers which can be
> > deferred to vcpu_load and vcpu_put, by either accessing them directly on
> > the physical CPU when the latest version is stored there, or by
> > synchronizing the memory representation with the CPU state.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_emulate.h | 14 -------
> > arch/arm64/kvm/inject_fault.c | 79 ++++++++++++++++++++++++++++++++----
> > arch/arm64/kvm/sys_regs.c | 6 ++-
> > 3 files changed, 76 insertions(+), 23 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 630dd60..69bb40d 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -66,11 +66,6 @@ static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > }
> >
> > -static inline unsigned long *vcpu_elr_el1(const struct kvm_vcpu *vcpu)
> > -{
> > - return (unsigned long *)&vcpu_gp_regs(vcpu)->elr_el1;
> > -}
> > -
> > static inline unsigned long *vcpu_cpsr(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pstate;
> > @@ -120,15 +115,6 @@ static inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
> > vcpu_gp_regs(vcpu)->regs.regs[reg_num] = val;
> > }
> >
> > -/* Get vcpu SPSR for current mode */
> > -static inline unsigned long *vcpu_spsr(const struct kvm_vcpu *vcpu)
> > -{
> > - if (vcpu_mode_is_32bit(vcpu))
> > - return vcpu_spsr32(vcpu);
> > -
> > - return (unsigned long *)&vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1];
> > -}
> > -
> > static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
> > {
> > u32 mode;
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index 45c7026..f4513fc 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -23,6 +23,7 @@
> >
> > #include <linux/kvm_host.h>
> > #include <asm/kvm_emulate.h>
> > +#include <asm/kvm_hyp.h>
> > #include <asm/esr.h>
> >
> > #define PSTATE_FAULT_BITS_64 (PSR_MODE_EL1h | PSR_A_BIT | PSR_F_BIT | \
> > @@ -33,13 +34,55 @@
> > #define LOWER_EL_AArch64_VECTOR 0x400
> > #define LOWER_EL_AArch32_VECTOR 0x600
> >
> > +static u64 vcpu_get_vbar_el1(struct kvm_vcpu *vcpu)
> > +{
> > + unsigned long vbar;
> > +
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + vbar = read_sysreg_el1(vbar);
> > + else
> > + vbar = vcpu_sys_reg(vcpu, VBAR_EL1);
> > +
> > + if (vcpu_el1_is_32bit(vcpu))
> > + return lower_32_bits(vbar);
> > + return vbar;
> > +}
> > +
> > +static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + write_sysreg_el1(val, elr);
> > + else
> > + vcpu_gp_regs(vcpu)->elr_el1 = val;
> > +}
> > +
> > +/* Set the SPSR for the current mode */
> > +static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> > +{
> > + if (vcpu_mode_is_32bit(vcpu))
> > + *vcpu_spsr32(vcpu) = val;
> > +
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + write_sysreg_el1(val, spsr);
> > + else
> > + vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1] = val;
> > +}
> > +
> > +static u32 vcpu_get_c1_sctlr(struct kvm_vcpu *vcpu)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + return lower_32_bits(read_sysreg_el1(sctlr));
> > + else
> > + return vcpu_cp15(vcpu, c1_SCTLR);
> > +}
> > +
> > static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> > {
> > unsigned long cpsr;
> > unsigned long new_spsr_value = *vcpu_cpsr(vcpu);
> > bool is_thumb = (new_spsr_value & COMPAT_PSR_T_BIT);
> > u32 return_offset = (is_thumb) ? 4 : 0;
> > - u32 sctlr = vcpu_cp15(vcpu, c1_SCTLR);
> > + u32 sctlr = vcpu_get_c1_sctlr(vcpu);
> >
> > cpsr = mode | COMPAT_PSR_I_BIT;
> >
> > @@ -51,14 +94,14 @@ static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> > *vcpu_cpsr(vcpu) = cpsr;
> >
> > /* Note: These now point to the banked copies */
> > - *vcpu_spsr(vcpu) = new_spsr_value;
> > + vcpu_set_spsr(vcpu, new_spsr_value);
> > *vcpu_reg32(vcpu, 14) = *vcpu_pc(vcpu) + return_offset;
> >
> > /* Branch to exception vector */
> > if (sctlr & (1 << 13))
> > vect_offset += 0xffff0000;
> > else /* always have security exceptions */
> > - vect_offset += vcpu_cp15(vcpu, c12_VBAR);
> > + vect_offset += vcpu_get_vbar_el1(vcpu);
> >
> > *vcpu_pc(vcpu) = vect_offset;
> > }
> > @@ -79,6 +122,20 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > u32 *far, *fsr;
> > bool is_lpae;
> >
> > + /*
> > + * We are going to need the latest values of the following system
> > + * regiters:
>
> registers
>
> > + * DFAR: mapped to FAR_EL1
>
> FAR_EL1[31:0]
>
> > + * IFAR: mapped to FAR_EL1
>
> FAR_EL1[63:32]
>
I had to rework this after the 32-bit fault injection code has been
reworked, so I got rid of this comment (as it's now being shared with
the 32-bit side, and I use a bigger hammer to solve the problem).
> > + * DFSR: mapped to ESR_EL1
> > + * TTBCR: mapped to TCR_EL1
> > + */
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> > + vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> > + vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> > + }
> > +
> > if (is_pabt) {
> > vect_offset = 12;
> > far = &vcpu_cp15(vcpu, c6_IFAR);
> > @@ -99,6 +156,12 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > *fsr = 1 << 9 | 0x34;
> > else
> > *fsr = 0x14;
> > +
> > + /* Sync back any registers we may have changed */
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> > + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> > + }
> > }
> >
> > enum exception_type {
> > @@ -126,7 +189,7 @@ static u64 get_except_vector(struct kvm_vcpu *vcpu, enum exception_type type)
> > exc_offset = LOWER_EL_AArch32_VECTOR;
> > }
> >
> > - return vcpu_sys_reg(vcpu, VBAR_EL1) + exc_offset + type;
> > + return vcpu_get_vbar_el1(vcpu) + exc_offset + type;
> > }
> >
> > static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> > @@ -135,11 +198,11 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> > bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> > u32 esr = 0;
> >
> > - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> > + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> > *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
> >
> > *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> > - *vcpu_spsr(vcpu) = cpsr;
> > + vcpu_set_spsr(vcpu, cpsr);
> >
> > vcpu_sys_reg(vcpu, FAR_EL1) = addr;
> >
> > @@ -170,11 +233,11 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> > unsigned long cpsr = *vcpu_cpsr(vcpu);
> > u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
> >
> > - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> > + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> > *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
> >
> > *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> > - *vcpu_spsr(vcpu) = cpsr;
> > + vcpu_set_spsr(vcpu, cpsr);
> >
>
> I'm concerned the maintenance of emulation code will become more
> difficult now that some registers have special accessors, while
> others don't, and some functions have save/restore lists, that
> will need to stay maintained with the emulation. The only alternative
> that pops to mind, though, is adding get/set members to the register
> descriptors and encapsulating the decision in them, but that might be
> overkill.
>
I was concerned initially as well, and had rewritten the entire code to
use get/set accessors everywhere, and that was not a pretty set of
patches to review, and in fact seemed less intuitive and easy to
maintain.
This is an interesting discussion; my original standpoint was very
similar to yours and I had similar concernes, but this rework has left
me feeling that maintaining this code base requires an intimate
understanding of the flow and state the CPU/VM can be in at any one
point, and it's therefore actually easier to maintain something that
does the required things in the right places, as opposed to creating
larger abstractions, at least to some degree.
I'm curious what Marc thinks here, and I'm willing to produce an
alternative patch with everything abstracted as get/set functions, with
giant switch statements to access hardware registers (which will only
actually hit in a few of the cases), but I think I know what the final
answer will be already.
> > /*
> > * Build an unknown exception, depending on the instruction
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index f7887dd..60d1660 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -86,12 +86,16 @@ static u32 cache_levels;
> > static u32 get_ccsidr(u32 csselr)
> > {
> > u32 ccsidr;
> > + u32 csselr_preserve;
> >
> > - /* Make sure noone else changes CSSELR during this! */
> > + /* Make sure noone else changes CSSELR during this and preserve any
> > + * existing value in the CSSELR! */
> > local_irq_disable();
> > + csselr_preserve = read_sysreg(csselr_el1);
> > write_sysreg(csselr, csselr_el1);
> > isb();
> > ccsidr = read_sysreg(ccsidr_el1);
> > + write_sysreg(csselr_preserve, csselr_el1);
> > local_irq_enable();
> >
> > return ccsidr;
>
> This looks like an unrelated fix.
>
Not really. The point is that the host kernel doesn't make any
assumptions about the csselr and will set it to select the value it
wants to read when using this feature.
Until we start deferring el1 registers for the guest, we can therefore
do the same, as the guest will always get its values restore.
However, when we introduce lazy save/restore of guest state, we cannot
just step on the guest state, but we have to preserve the guest state.
We could make it conditional on the sysregs loaded flag, but that
doesn't get us much, and we might as well do this.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 25/37] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs
@ 2017-12-03 20:29 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:29 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 13, 2017 at 07:56:14PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:29PM +0200, Christoffer Dall wrote:
> > Handle accesses during traps to any remaining EL1 registers which can be
> > deferred to vcpu_load and vcpu_put, by either accessing them directly on
> > the physical CPU when the latest version is stored there, or by
> > synchronizing the memory representation with the CPU state.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/include/asm/kvm_emulate.h | 14 -------
> > arch/arm64/kvm/inject_fault.c | 79 ++++++++++++++++++++++++++++++++----
> > arch/arm64/kvm/sys_regs.c | 6 ++-
> > 3 files changed, 76 insertions(+), 23 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index 630dd60..69bb40d 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -66,11 +66,6 @@ static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > }
> >
> > -static inline unsigned long *vcpu_elr_el1(const struct kvm_vcpu *vcpu)
> > -{
> > - return (unsigned long *)&vcpu_gp_regs(vcpu)->elr_el1;
> > -}
> > -
> > static inline unsigned long *vcpu_cpsr(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pstate;
> > @@ -120,15 +115,6 @@ static inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,
> > vcpu_gp_regs(vcpu)->regs.regs[reg_num] = val;
> > }
> >
> > -/* Get vcpu SPSR for current mode */
> > -static inline unsigned long *vcpu_spsr(const struct kvm_vcpu *vcpu)
> > -{
> > - if (vcpu_mode_is_32bit(vcpu))
> > - return vcpu_spsr32(vcpu);
> > -
> > - return (unsigned long *)&vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1];
> > -}
> > -
> > static inline bool vcpu_mode_priv(const struct kvm_vcpu *vcpu)
> > {
> > u32 mode;
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index 45c7026..f4513fc 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -23,6 +23,7 @@
> >
> > #include <linux/kvm_host.h>
> > #include <asm/kvm_emulate.h>
> > +#include <asm/kvm_hyp.h>
> > #include <asm/esr.h>
> >
> > #define PSTATE_FAULT_BITS_64 (PSR_MODE_EL1h | PSR_A_BIT | PSR_F_BIT | \
> > @@ -33,13 +34,55 @@
> > #define LOWER_EL_AArch64_VECTOR 0x400
> > #define LOWER_EL_AArch32_VECTOR 0x600
> >
> > +static u64 vcpu_get_vbar_el1(struct kvm_vcpu *vcpu)
> > +{
> > + unsigned long vbar;
> > +
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + vbar = read_sysreg_el1(vbar);
> > + else
> > + vbar = vcpu_sys_reg(vcpu, VBAR_EL1);
> > +
> > + if (vcpu_el1_is_32bit(vcpu))
> > + return lower_32_bits(vbar);
> > + return vbar;
> > +}
> > +
> > +static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + write_sysreg_el1(val, elr);
> > + else
> > + vcpu_gp_regs(vcpu)->elr_el1 = val;
> > +}
> > +
> > +/* Set the SPSR for the current mode */
> > +static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> > +{
> > + if (vcpu_mode_is_32bit(vcpu))
> > + *vcpu_spsr32(vcpu) = val;
> > +
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + write_sysreg_el1(val, spsr);
> > + else
> > + vcpu_gp_regs(vcpu)->spsr[KVM_SPSR_EL1] = val;
> > +}
> > +
> > +static u32 vcpu_get_c1_sctlr(struct kvm_vcpu *vcpu)
> > +{
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + return lower_32_bits(read_sysreg_el1(sctlr));
> > + else
> > + return vcpu_cp15(vcpu, c1_SCTLR);
> > +}
> > +
> > static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> > {
> > unsigned long cpsr;
> > unsigned long new_spsr_value = *vcpu_cpsr(vcpu);
> > bool is_thumb = (new_spsr_value & COMPAT_PSR_T_BIT);
> > u32 return_offset = (is_thumb) ? 4 : 0;
> > - u32 sctlr = vcpu_cp15(vcpu, c1_SCTLR);
> > + u32 sctlr = vcpu_get_c1_sctlr(vcpu);
> >
> > cpsr = mode | COMPAT_PSR_I_BIT;
> >
> > @@ -51,14 +94,14 @@ static void prepare_fault32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)
> > *vcpu_cpsr(vcpu) = cpsr;
> >
> > /* Note: These now point to the banked copies */
> > - *vcpu_spsr(vcpu) = new_spsr_value;
> > + vcpu_set_spsr(vcpu, new_spsr_value);
> > *vcpu_reg32(vcpu, 14) = *vcpu_pc(vcpu) + return_offset;
> >
> > /* Branch to exception vector */
> > if (sctlr & (1 << 13))
> > vect_offset += 0xffff0000;
> > else /* always have security exceptions */
> > - vect_offset += vcpu_cp15(vcpu, c12_VBAR);
> > + vect_offset += vcpu_get_vbar_el1(vcpu);
> >
> > *vcpu_pc(vcpu) = vect_offset;
> > }
> > @@ -79,6 +122,20 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > u32 *far, *fsr;
> > bool is_lpae;
> >
> > + /*
> > + * We are going to need the latest values of the following system
> > + * regiters:
>
> registers
>
> > + * DFAR: mapped to FAR_EL1
>
> FAR_EL1[31:0]
>
> > + * IFAR: mapped to FAR_EL1
>
> FAR_EL1[63:32]
>
I had to rework this after the 32-bit fault injection code has been
reworked, so I got rid of this comment (as it's now being shared with
the 32-bit side, and I use a bigger hammer to solve the problem).
> > + * DFSR: mapped to ESR_EL1
> > + * TTBCR: mapped to TCR_EL1
> > + */
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> > + vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> > + vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> > + }
> > +
> > if (is_pabt) {
> > vect_offset = 12;
> > far = &vcpu_cp15(vcpu, c6_IFAR);
> > @@ -99,6 +156,12 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > *fsr = 1 << 9 | 0x34;
> > else
> > *fsr = 0x14;
> > +
> > + /* Sync back any registers we may have changed */
> > + if (vcpu->arch.sysregs_loaded_on_cpu) {
> > + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> > + write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> > + }
> > }
> >
> > enum exception_type {
> > @@ -126,7 +189,7 @@ static u64 get_except_vector(struct kvm_vcpu *vcpu, enum exception_type type)
> > exc_offset = LOWER_EL_AArch32_VECTOR;
> > }
> >
> > - return vcpu_sys_reg(vcpu, VBAR_EL1) + exc_offset + type;
> > + return vcpu_get_vbar_el1(vcpu) + exc_offset + type;
> > }
> >
> > static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr)
> > @@ -135,11 +198,11 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr
> > bool is_aarch32 = vcpu_mode_is_32bit(vcpu);
> > u32 esr = 0;
> >
> > - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> > + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> > *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
> >
> > *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> > - *vcpu_spsr(vcpu) = cpsr;
> > + vcpu_set_spsr(vcpu, cpsr);
> >
> > vcpu_sys_reg(vcpu, FAR_EL1) = addr;
> >
> > @@ -170,11 +233,11 @@ static void inject_undef64(struct kvm_vcpu *vcpu)
> > unsigned long cpsr = *vcpu_cpsr(vcpu);
> > u32 esr = (ESR_ELx_EC_UNKNOWN << ESR_ELx_EC_SHIFT);
> >
> > - *vcpu_elr_el1(vcpu) = *vcpu_pc(vcpu);
> > + vcpu_set_elr_el1(vcpu, *vcpu_pc(vcpu));
> > *vcpu_pc(vcpu) = get_except_vector(vcpu, except_type_sync);
> >
> > *vcpu_cpsr(vcpu) = PSTATE_FAULT_BITS_64;
> > - *vcpu_spsr(vcpu) = cpsr;
> > + vcpu_set_spsr(vcpu, cpsr);
> >
>
> I'm concerned the maintenance of emulation code will become more
> difficult now that some registers have special accessors, while
> others don't, and some functions have save/restore lists, that
> will need to stay maintained with the emulation. The only alternative
> that pops to mind, though, is adding get/set members to the register
> descriptors and encapsulating the decision in them, but that might be
> overkill.
>
I was concerned initially as well, and had rewritten the entire code to
use get/set accessors everywhere, and that was not a pretty set of
patches to review, and in fact seemed less intuitive and easy to
maintain.
This is an interesting discussion; my original standpoint was very
similar to yours and I had similar concernes, but this rework has left
me feeling that maintaining this code base requires an intimate
understanding of the flow and state the CPU/VM can be in at any one
point, and it's therefore actually easier to maintain something that
does the required things in the right places, as opposed to creating
larger abstractions, at least to some degree.
I'm curious what Marc thinks here, and I'm willing to produce an
alternative patch with everything abstracted as get/set functions, with
giant switch statements to access hardware registers (which will only
actually hit in a few of the cases), but I think I know what the final
answer will be already.
> > /*
> > * Build an unknown exception, depending on the instruction
> > diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> > index f7887dd..60d1660 100644
> > --- a/arch/arm64/kvm/sys_regs.c
> > +++ b/arch/arm64/kvm/sys_regs.c
> > @@ -86,12 +86,16 @@ static u32 cache_levels;
> > static u32 get_ccsidr(u32 csselr)
> > {
> > u32 ccsidr;
> > + u32 csselr_preserve;
> >
> > - /* Make sure noone else changes CSSELR during this! */
> > + /* Make sure noone else changes CSSELR during this and preserve any
> > + * existing value in the CSSELR! */
> > local_irq_disable();
> > + csselr_preserve = read_sysreg(csselr_el1);
> > write_sysreg(csselr, csselr_el1);
> > isb();
> > ccsidr = read_sysreg(ccsidr_el1);
> > + write_sysreg(csselr_preserve, csselr_el1);
> > local_irq_enable();
> >
> > return ccsidr;
>
> This looks like an unrelated fix.
>
Not really. The point is that the host kernel doesn't make any
assumptions about the csselr and will set it to select the value it
wants to read when using this feature.
Until we start deferring el1 registers for the guest, we can therefore
do the same, as the guest will always get its values restore.
However, when we introduce lazy save/restore of guest state, we cannot
just step on the guest state, but we have to preserve the guest state.
We could make it conditional on the sysregs loaded flag, but that
doesn't get us much, and we might as well do this.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 26/37] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Handle accesses to any AArch32 EL1 system registers where we can defer
saving and restoring them to vcpu_load and vcpu_put, and which are
stored in special EL2 registers only used support 32-bit guests.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/inject_fault.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index f4513fc..02990f5 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -59,9 +59,18 @@ static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
/* Set the SPSR for the current mode */
static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
{
- if (vcpu_mode_is_32bit(vcpu))
+ if (vcpu_mode_is_32bit(vcpu)) {
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ __sysreg32_save_state(vcpu);
+
*vcpu_spsr32(vcpu) = val;
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ __sysreg32_restore_state(vcpu);
+
+ return;
+ }
+
if (vcpu->arch.sysregs_loaded_on_cpu)
write_sysreg_el1(val, spsr);
else
@@ -129,11 +138,13 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
* IFAR: mapped to FAR_EL1
* DFSR: mapped to ESR_EL1
* TTBCR: mapped to TCR_EL1
+ * IFSR: stored in IFSR32_EL2
*/
if (vcpu->arch.sysregs_loaded_on_cpu) {
vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
+ vcpu->arch.ctxt.sys_regs[IFSR32_EL2] = read_sysreg(ifsr32_el2);
}
if (is_pabt) {
@@ -161,6 +172,7 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
if (vcpu->arch.sysregs_loaded_on_cpu) {
write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
+ write_sysreg(vcpu->arch.ctxt.sys_regs[IFSR32_EL2], ifsr32_el2);
}
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 26/37] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Handle accesses to any AArch32 EL1 system registers where we can defer
saving and restoring them to vcpu_load and vcpu_put, and which are
stored in special EL2 registers only used support 32-bit guests.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/inject_fault.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
index f4513fc..02990f5 100644
--- a/arch/arm64/kvm/inject_fault.c
+++ b/arch/arm64/kvm/inject_fault.c
@@ -59,9 +59,18 @@ static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
/* Set the SPSR for the current mode */
static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
{
- if (vcpu_mode_is_32bit(vcpu))
+ if (vcpu_mode_is_32bit(vcpu)) {
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ __sysreg32_save_state(vcpu);
+
*vcpu_spsr32(vcpu) = val;
+ if (vcpu->arch.sysregs_loaded_on_cpu)
+ __sysreg32_restore_state(vcpu);
+
+ return;
+ }
+
if (vcpu->arch.sysregs_loaded_on_cpu)
write_sysreg_el1(val, spsr);
else
@@ -129,11 +138,13 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
* IFAR: mapped to FAR_EL1
* DFSR: mapped to ESR_EL1
* TTBCR: mapped to TCR_EL1
+ * IFSR: stored in IFSR32_EL2
*/
if (vcpu->arch.sysregs_loaded_on_cpu) {
vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
+ vcpu->arch.ctxt.sys_regs[IFSR32_EL2] = read_sysreg(ifsr32_el2);
}
if (is_pabt) {
@@ -161,6 +172,7 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
if (vcpu->arch.sysregs_loaded_on_cpu) {
write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
+ write_sysreg(vcpu->arch.ctxt.sys_regs[IFSR32_EL2], ifsr32_el2);
}
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 26/37] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-13 19:07 ` Andrew Jones
-1 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 19:07 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel, kvm
On Thu, Oct 12, 2017 at 12:41:30PM +0200, Christoffer Dall wrote:
> Handle accesses to any AArch32 EL1 system registers where we can defer
> saving and restoring them to vcpu_load and vcpu_put, and which are
> stored in special EL2 registers only used support 32-bit guests.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/inject_fault.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index f4513fc..02990f5 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -59,9 +59,18 @@ static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> /* Set the SPSR for the current mode */
> static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> {
> - if (vcpu_mode_is_32bit(vcpu))
> + if (vcpu_mode_is_32bit(vcpu)) {
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + __sysreg32_save_state(vcpu);
> +
> *vcpu_spsr32(vcpu) = val;
>
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + __sysreg32_restore_state(vcpu);
> +
> + return;
Is this a fix? I don't understand why it's necessary now, but it wasn't
before.
> + }
> +
> if (vcpu->arch.sysregs_loaded_on_cpu)
> write_sysreg_el1(val, spsr);
> else
> @@ -129,11 +138,13 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> * IFAR: mapped to FAR_EL1
> * DFSR: mapped to ESR_EL1
> * TTBCR: mapped to TCR_EL1
> + * IFSR: stored in IFSR32_EL2
> */
> if (vcpu->arch.sysregs_loaded_on_cpu) {
> vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> + vcpu->arch.ctxt.sys_regs[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> }
>
> if (is_pabt) {
> @@ -161,6 +172,7 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> if (vcpu->arch.sysregs_loaded_on_cpu) {
> write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> + write_sysreg(vcpu->arch.ctxt.sys_regs[IFSR32_EL2], ifsr32_el2);
This appears to be a fix. Why not squash it into the patch that
save/restore the other registers?
> }
> }
>
> --
> 2.9.0
>
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 26/37] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
@ 2017-11-13 19:07 ` Andrew Jones
0 siblings, 0 replies; 254+ messages in thread
From: Andrew Jones @ 2017-11-13 19:07 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:30PM +0200, Christoffer Dall wrote:
> Handle accesses to any AArch32 EL1 system registers where we can defer
> saving and restoring them to vcpu_load and vcpu_put, and which are
> stored in special EL2 registers only used support 32-bit guests.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/inject_fault.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> index f4513fc..02990f5 100644
> --- a/arch/arm64/kvm/inject_fault.c
> +++ b/arch/arm64/kvm/inject_fault.c
> @@ -59,9 +59,18 @@ static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> /* Set the SPSR for the current mode */
> static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> {
> - if (vcpu_mode_is_32bit(vcpu))
> + if (vcpu_mode_is_32bit(vcpu)) {
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + __sysreg32_save_state(vcpu);
> +
> *vcpu_spsr32(vcpu) = val;
>
> + if (vcpu->arch.sysregs_loaded_on_cpu)
> + __sysreg32_restore_state(vcpu);
> +
> + return;
Is this a fix? I don't understand why it's necessary now, but it wasn't
before.
> + }
> +
> if (vcpu->arch.sysregs_loaded_on_cpu)
> write_sysreg_el1(val, spsr);
> else
> @@ -129,11 +138,13 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> * IFAR: mapped to FAR_EL1
> * DFSR: mapped to ESR_EL1
> * TTBCR: mapped to TCR_EL1
> + * IFSR: stored in IFSR32_EL2
> */
> if (vcpu->arch.sysregs_loaded_on_cpu) {
> vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> + vcpu->arch.ctxt.sys_regs[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> }
>
> if (is_pabt) {
> @@ -161,6 +172,7 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> if (vcpu->arch.sysregs_loaded_on_cpu) {
> write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> + write_sysreg(vcpu->arch.ctxt.sys_regs[IFSR32_EL2], ifsr32_el2);
This appears to be a fix. Why not squash it into the patch that
save/restore the other registers?
> }
> }
>
> --
> 2.9.0
>
Thanks,
drew
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 26/37] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
2017-11-13 19:07 ` Andrew Jones
@ 2017-12-03 20:35 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:35 UTC (permalink / raw)
To: Andrew Jones
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Mon, Nov 13, 2017 at 08:07:01PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:30PM +0200, Christoffer Dall wrote:
> > Handle accesses to any AArch32 EL1 system registers where we can defer
> > saving and restoring them to vcpu_load and vcpu_put, and which are
> > stored in special EL2 registers only used support 32-bit guests.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/inject_fault.c | 14 +++++++++++++-
> > 1 file changed, 13 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index f4513fc..02990f5 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -59,9 +59,18 @@ static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> > /* Set the SPSR for the current mode */
> > static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> > {
> > - if (vcpu_mode_is_32bit(vcpu))
> > + if (vcpu_mode_is_32bit(vcpu)) {
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + __sysreg32_save_state(vcpu);
> > +
> > *vcpu_spsr32(vcpu) = val;
> >
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + __sysreg32_restore_state(vcpu);
> > +
> > + return;
>
> Is this a fix? I don't understand why it's necessary now, but it wasn't
> before.
>
All of these patches are not necessary before "KVM: arm64: Defer
saving/restoring system registers to vcpu load/put on VHE", but I needed
to find a way to split the patches into some smaller pieces, both for
debugging/bisecting and for easing the review.
> > + }
> > +
> > if (vcpu->arch.sysregs_loaded_on_cpu)
> > write_sysreg_el1(val, spsr);
> > else
> > @@ -129,11 +138,13 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > * IFAR: mapped to FAR_EL1
> > * DFSR: mapped to ESR_EL1
> > * TTBCR: mapped to TCR_EL1
> > + * IFSR: stored in IFSR32_EL2
> > */
> > if (vcpu->arch.sysregs_loaded_on_cpu) {
> > vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> > vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> > vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> > + vcpu->arch.ctxt.sys_regs[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> > }
> >
> > if (is_pabt) {
> > @@ -161,6 +172,7 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > if (vcpu->arch.sysregs_loaded_on_cpu) {
> > write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> > write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> > + write_sysreg(vcpu->arch.ctxt.sys_regs[IFSR32_EL2], ifsr32_el2);
>
> This appears to be a fix. Why not squash it into the patch that
> save/restore the other registers?
>
It's just more work we can do before we start being lazy.
I thought it would be easier to review in pieces, and I thought the
sequence of patches begining with "Prepare..." was a commonly used
theme, bt apparently not.
In any cae, this hunk has now gone away due to the shared logic of
32-bit fault injection, so it should be more obvious next time around.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 26/37] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs
@ 2017-12-03 20:35 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:35 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 13, 2017 at 08:07:01PM +0100, Andrew Jones wrote:
> On Thu, Oct 12, 2017 at 12:41:30PM +0200, Christoffer Dall wrote:
> > Handle accesses to any AArch32 EL1 system registers where we can defer
> > saving and restoring them to vcpu_load and vcpu_put, and which are
> > stored in special EL2 registers only used support 32-bit guests.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/inject_fault.c | 14 +++++++++++++-
> > 1 file changed, 13 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/kvm/inject_fault.c b/arch/arm64/kvm/inject_fault.c
> > index f4513fc..02990f5 100644
> > --- a/arch/arm64/kvm/inject_fault.c
> > +++ b/arch/arm64/kvm/inject_fault.c
> > @@ -59,9 +59,18 @@ static void vcpu_set_elr_el1(struct kvm_vcpu *vcpu, u64 val)
> > /* Set the SPSR for the current mode */
> > static void vcpu_set_spsr(struct kvm_vcpu *vcpu, u64 val)
> > {
> > - if (vcpu_mode_is_32bit(vcpu))
> > + if (vcpu_mode_is_32bit(vcpu)) {
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + __sysreg32_save_state(vcpu);
> > +
> > *vcpu_spsr32(vcpu) = val;
> >
> > + if (vcpu->arch.sysregs_loaded_on_cpu)
> > + __sysreg32_restore_state(vcpu);
> > +
> > + return;
>
> Is this a fix? I don't understand why it's necessary now, but it wasn't
> before.
>
All of these patches are not necessary before "KVM: arm64: Defer
saving/restoring system registers to vcpu load/put on VHE", but I needed
to find a way to split the patches into some smaller pieces, both for
debugging/bisecting and for easing the review.
> > + }
> > +
> > if (vcpu->arch.sysregs_loaded_on_cpu)
> > write_sysreg_el1(val, spsr);
> > else
> > @@ -129,11 +138,13 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > * IFAR: mapped to FAR_EL1
> > * DFSR: mapped to ESR_EL1
> > * TTBCR: mapped to TCR_EL1
> > + * IFSR: stored in IFSR32_EL2
> > */
> > if (vcpu->arch.sysregs_loaded_on_cpu) {
> > vcpu->arch.ctxt.sys_regs[FAR_EL1] = read_sysreg_el1(far);
> > vcpu->arch.ctxt.sys_regs[ESR_EL1] = read_sysreg_el1(esr);
> > vcpu->arch.ctxt.sys_regs[TCR_EL1] = read_sysreg_el1(tcr);
> > + vcpu->arch.ctxt.sys_regs[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> > }
> >
> > if (is_pabt) {
> > @@ -161,6 +172,7 @@ static void inject_abt32(struct kvm_vcpu *vcpu, bool is_pabt,
> > if (vcpu->arch.sysregs_loaded_on_cpu) {
> > write_sysreg_el1(vcpu->arch.ctxt.sys_regs[FAR_EL1], far);
> > write_sysreg_el1(vcpu->arch.ctxt.sys_regs[ESR_EL1], esr);
> > + write_sysreg(vcpu->arch.ctxt.sys_regs[IFSR32_EL2], ifsr32_el2);
>
> This appears to be a fix. Why not squash it into the patch that
> save/restore the other registers?
>
It's just more work we can do before we start being lazy.
I thought it would be easier to review in pieces, and I thought the
sequence of patches begining with "Prepare..." was a commonly used
theme, bt apparently not.
In any cae, this hunk has now gone away due to the shared logic of
32-bit fault injection, so it should be more obvious next time around.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 27/37] KVM: arm64: Defer saving/restoring system registers to vcpu load/put on VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Some system registers do not affect the host kernel's execution and can
therefore be loaded when we are about to run a VCPU and we don't have to
restore the host state to the hardware before the time when we are
actually about to return to userspace or schedule out the VCPU thread.
The EL1 system registers and the userspace state registers, which only
affect EL0 execution, do not affect the host kernel's execution.
The 32-bit system registers are not used by a VHE host kernel and
therefore don't need to be saved/restored on every entry/exit to/from
the guest, but can be deferred to vcpu_load and vcpu_put, respectively.
We have already prepared the trap handling code which accesses any of
these registers to directly access the registers on the physical CPU or
to sync the registers when needed.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 6 ------
arch/arm64/kvm/hyp/sysreg-sr.c | 46 ++++++++++++++++++++++++++++++++++--------
2 files changed, 38 insertions(+), 14 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 6356bec..6a12504 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -337,11 +337,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__vgic_restore_state(vcpu);
- /*
- * We must restore the 32-bit state before the sysregs, thanks
- * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
- */
- __sysreg32_restore_state(vcpu);
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
@@ -354,7 +349,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
goto again;
sysreg_save_guest_state_vhe(guest_ctxt);
- __sysreg32_save_state(vcpu);
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 354ca02..6ff1ce5 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -25,8 +25,12 @@
/*
* Non-VHE: Both host and guest must save everything.
*
- * VHE: Host must save tpidr*_el0, actlr_el1, mdscr_el1, sp_el0,
- * and guest must save everything.
+ * VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and pstate,
+ * which are handled as part of the el2 return state) on every switch.
+ * tpidr_el0, tpidrro_el0, and actlr_el1 only need to be switched when going
+ * to host userspace or a different VCPU. EL1 registers only need to be
+ * switched when potentially going to run a different VCPU. The latter two
+ * classes are handled as part of kvm_arch_vcpu_load and kvm_arch_vcpu_put.
*/
static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
@@ -85,14 +89,11 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_save_common_state(ctxt);
- __sysreg_save_user_state(ctxt);
}
void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
- __sysreg_save_user_state(ctxt);
__sysreg_save_el2_return_state(ctxt);
}
@@ -153,14 +154,11 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_restore_common_state(ctxt);
- __sysreg_restore_user_state(ctxt);
}
void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
- __sysreg_restore_user_state(ctxt);
__sysreg_restore_el2_return_state(ctxt);
}
@@ -224,6 +222,26 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
*/
void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
{
+ struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
+ struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
+
+ if (!has_vhe())
+ return;
+
+ __sysreg_save_user_state(host_ctxt);
+
+
+ /*
+ * Load guest EL1 and user state
+ *
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_user_state(guest_ctxt);
+ __sysreg_restore_el1_state(guest_ctxt);
+
+ vcpu->arch.sysregs_loaded_on_cpu = true;
}
/**
@@ -250,4 +268,16 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
__fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
vcpu->arch.guest_vfp_loaded = 0;
}
+
+ if (!has_vhe())
+ return;
+
+ __sysreg_save_el1_state(guest_ctxt);
+ __sysreg_save_user_state(guest_ctxt);
+ __sysreg32_save_state(vcpu);
+
+ /* Restore host user state */
+ __sysreg_restore_user_state(host_ctxt);
+
+ vcpu->arch.sysregs_loaded_on_cpu = false;
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 27/37] KVM: arm64: Defer saving/restoring system registers to vcpu load/put on VHE
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Some system registers do not affect the host kernel's execution and can
therefore be loaded when we are about to run a VCPU and we don't have to
restore the host state to the hardware before the time when we are
actually about to return to userspace or schedule out the VCPU thread.
The EL1 system registers and the userspace state registers, which only
affect EL0 execution, do not affect the host kernel's execution.
The 32-bit system registers are not used by a VHE host kernel and
therefore don't need to be saved/restored on every entry/exit to/from
the guest, but can be deferred to vcpu_load and vcpu_put, respectively.
We have already prepared the trap handling code which accesses any of
these registers to directly access the registers on the physical CPU or
to sync the registers when needed.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 6 ------
arch/arm64/kvm/hyp/sysreg-sr.c | 46 ++++++++++++++++++++++++++++++++++--------
2 files changed, 38 insertions(+), 14 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 6356bec..6a12504 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -337,11 +337,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
__vgic_restore_state(vcpu);
- /*
- * We must restore the 32-bit state before the sysregs, thanks
- * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
- */
- __sysreg32_restore_state(vcpu);
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
@@ -354,7 +349,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
goto again;
sysreg_save_guest_state_vhe(guest_ctxt);
- __sysreg32_save_state(vcpu);
__vgic_save_state(vcpu);
__deactivate_traps(vcpu);
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 354ca02..6ff1ce5 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -25,8 +25,12 @@
/*
* Non-VHE: Both host and guest must save everything.
*
- * VHE: Host must save tpidr*_el0, actlr_el1, mdscr_el1, sp_el0,
- * and guest must save everything.
+ * VHE: Host and guest must save mdscr_el1 and sp_el0 (and the PC and pstate,
+ * which are handled as part of the el2 return state) on every switch.
+ * tpidr_el0, tpidrro_el0, and actlr_el1 only need to be switched when going
+ * to host userspace or a different VCPU. EL1 registers only need to be
+ * switched when potentially going to run a different VCPU. The latter two
+ * classes are handled as part of kvm_arch_vcpu_load and kvm_arch_vcpu_put.
*/
static void __hyp_text __sysreg_save_common_state(struct kvm_cpu_context *ctxt)
@@ -85,14 +89,11 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)
void sysreg_save_host_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_save_common_state(ctxt);
- __sysreg_save_user_state(ctxt);
}
void sysreg_save_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_save_el1_state(ctxt);
__sysreg_save_common_state(ctxt);
- __sysreg_save_user_state(ctxt);
__sysreg_save_el2_return_state(ctxt);
}
@@ -153,14 +154,11 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)
void sysreg_restore_host_state_vhe(struct kvm_cpu_context *ctxt)
{
__sysreg_restore_common_state(ctxt);
- __sysreg_restore_user_state(ctxt);
}
void sysreg_restore_guest_state_vhe(struct kvm_cpu_context *ctxt)
{
- __sysreg_restore_el1_state(ctxt);
__sysreg_restore_common_state(ctxt);
- __sysreg_restore_user_state(ctxt);
__sysreg_restore_el2_return_state(ctxt);
}
@@ -224,6 +222,26 @@ void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
*/
void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
{
+ struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
+ struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
+
+ if (!has_vhe())
+ return;
+
+ __sysreg_save_user_state(host_ctxt);
+
+
+ /*
+ * Load guest EL1 and user state
+ *
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72).
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_user_state(guest_ctxt);
+ __sysreg_restore_el1_state(guest_ctxt);
+
+ vcpu->arch.sysregs_loaded_on_cpu = true;
}
/**
@@ -250,4 +268,16 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
__fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
vcpu->arch.guest_vfp_loaded = 0;
}
+
+ if (!has_vhe())
+ return;
+
+ __sysreg_save_el1_state(guest_ctxt);
+ __sysreg_save_user_state(guest_ctxt);
+ __sysreg32_save_state(vcpu);
+
+ /* Restore host user state */
+ __sysreg_restore_user_state(host_ctxt);
+
+ vcpu->arch.sysregs_loaded_on_cpu = false;
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 28/37] KVM: arm64: Move common VHE/non-VHE trap config in separate functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
As we are about to be more lazy with some of the trap configuration
register read/writes for VHE systems, move the logic that is currently
shared between VHE and non-VHE into a separate function which can be
called from either the world-switch path or from vcpu_load/vcpu_put.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 70 +++++++++++++++++++++++++--------------------
1 file changed, 39 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 6a12504..c587416 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -23,6 +23,43 @@
#include <asm/kvm_hyp.h>
#include <asm/fpsimd.h>
+static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
+{
+ /*
+ * We are about to set CPTR_EL2.TFP to trap all floating point
+ * register accesses to EL2, however, the ARM ARM clearly states that
+ * traps are only taken to EL2 if the operation would not otherwise
+ * trap to EL1. Therefore, always make sure that for 32-bit guests,
+ * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
+ * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
+ * it will cause an exception.
+ */
+ if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
+ !vcpu->arch.guest_vfp_loaded) {
+ write_sysreg(1 << 30, fpexc32_el2);
+ isb();
+ }
+ write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
+
+ /* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
+ write_sysreg(1 << 15, hstr_el2);
+ /*
+ * Make sure we trap PMU access from EL0 to EL2. Also sanitize
+ * PMSELR_EL0 to make sure it never contains the cycle
+ * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
+ * EL1 instead of being trapped to EL2.
+ */
+ write_sysreg(0, pmselr_el0);
+ write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
+ write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+}
+
+static void __hyp_text __deactivate_traps_common(void)
+{
+ write_sysreg(0, hstr_el2);
+ write_sysreg(0, pmuserenr_el0);
+}
+
static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
{
u64 val;
@@ -57,35 +94,7 @@ static hyp_alternate_select(__activate_traps_arch,
static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
{
- u64 val;
-
- /*
- * We are about to set CPTR_EL2.TFP to trap all floating point
- * register accesses to EL2, however, the ARM ARM clearly states that
- * traps are only taken to EL2 if the operation would not otherwise
- * trap to EL1. Therefore, always make sure that for 32-bit guests,
- * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
- * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
- * it will cause an exception.
- */
- val = vcpu->arch.hcr_el2;
- if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
- !vcpu->arch.guest_vfp_loaded) {
- write_sysreg(1 << 30, fpexc32_el2);
- isb();
- }
- write_sysreg(val, hcr_el2);
- /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
- write_sysreg(1 << 15, hstr_el2);
- /*
- * Make sure we trap PMU access from EL0 to EL2. Also sanitize
- * PMSELR_EL0 to make sure it never contains the cycle
- * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
- * EL1 instead of being trapped to EL2.
- */
- write_sysreg(0, pmselr_el0);
- write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
- write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+ __activate_traps_common(vcpu);
__activate_traps_arch()(vcpu);
}
@@ -131,9 +140,8 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
if (vcpu->arch.hcr_el2 & HCR_VSE)
vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
+ __deactivate_traps_common();
__deactivate_traps_arch()();
- write_sysreg(0, hstr_el2);
- write_sysreg(0, pmuserenr_el0);
}
static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 28/37] KVM: arm64: Move common VHE/non-VHE trap config in separate functions
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
As we are about to be more lazy with some of the trap configuration
register read/writes for VHE systems, move the logic that is currently
shared between VHE and non-VHE into a separate function which can be
called from either the world-switch path or from vcpu_load/vcpu_put.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 70 +++++++++++++++++++++++++--------------------
1 file changed, 39 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 6a12504..c587416 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -23,6 +23,43 @@
#include <asm/kvm_hyp.h>
#include <asm/fpsimd.h>
+static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
+{
+ /*
+ * We are about to set CPTR_EL2.TFP to trap all floating point
+ * register accesses to EL2, however, the ARM ARM clearly states that
+ * traps are only taken to EL2 if the operation would not otherwise
+ * trap to EL1. Therefore, always make sure that for 32-bit guests,
+ * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
+ * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
+ * it will cause an exception.
+ */
+ if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
+ !vcpu->arch.guest_vfp_loaded) {
+ write_sysreg(1 << 30, fpexc32_el2);
+ isb();
+ }
+ write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
+
+ /* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
+ write_sysreg(1 << 15, hstr_el2);
+ /*
+ * Make sure we trap PMU access from EL0 to EL2. Also sanitize
+ * PMSELR_EL0 to make sure it never contains the cycle
+ * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
+ * EL1 instead of being trapped to EL2.
+ */
+ write_sysreg(0, pmselr_el0);
+ write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
+ write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+}
+
+static void __hyp_text __deactivate_traps_common(void)
+{
+ write_sysreg(0, hstr_el2);
+ write_sysreg(0, pmuserenr_el0);
+}
+
static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
{
u64 val;
@@ -57,35 +94,7 @@ static hyp_alternate_select(__activate_traps_arch,
static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
{
- u64 val;
-
- /*
- * We are about to set CPTR_EL2.TFP to trap all floating point
- * register accesses to EL2, however, the ARM ARM clearly states that
- * traps are only taken to EL2 if the operation would not otherwise
- * trap to EL1. Therefore, always make sure that for 32-bit guests,
- * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
- * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
- * it will cause an exception.
- */
- val = vcpu->arch.hcr_el2;
- if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
- !vcpu->arch.guest_vfp_loaded) {
- write_sysreg(1 << 30, fpexc32_el2);
- isb();
- }
- write_sysreg(val, hcr_el2);
- /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
- write_sysreg(1 << 15, hstr_el2);
- /*
- * Make sure we trap PMU access from EL0 to EL2. Also sanitize
- * PMSELR_EL0 to make sure it never contains the cycle
- * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
- * EL1 instead of being trapped to EL2.
- */
- write_sysreg(0, pmselr_el0);
- write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
- write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+ __activate_traps_common(vcpu);
__activate_traps_arch()(vcpu);
}
@@ -131,9 +140,8 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
if (vcpu->arch.hcr_el2 & HCR_VSE)
vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
+ __deactivate_traps_common();
__deactivate_traps_arch()();
- write_sysreg(0, hstr_el2);
- write_sysreg(0, pmuserenr_el0);
}
static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 28/37] KVM: arm64: Move common VHE/non-VHE trap config in separate functions
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-25 10:43 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-25 10:43 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel, kvm
On Thu, Oct 12, 2017 at 12:41:32PM +0200, Christoffer Dall wrote:
> As we are about to be more lazy with some of the trap configuration
> register read/writes for VHE systems, move the logic that is currently
> shared between VHE and non-VHE into a separate function which can be
> called from either the world-switch path or from vcpu_load/vcpu_put.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 70 +++++++++++++++++++++++++--------------------
> 1 file changed, 39 insertions(+), 31 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 6a12504..c587416 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -23,6 +23,43 @@
> #include <asm/kvm_hyp.h>
> #include <asm/fpsimd.h>
>
> +static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
> +{
> + /*
> + * We are about to set CPTR_EL2.TFP to trap all floating point
> + * register accesses to EL2, however, the ARM ARM clearly states that
ARM ARM: typo?
> + * traps are only taken to EL2 if the operation would not otherwise
> + * trap to EL1. Therefore, always make sure that for 32-bit guests,
> + * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
> + * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
> + * it will cause an exception.
> + */
> + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> + !vcpu->arch.guest_vfp_loaded) {
> + write_sysreg(1 << 30, fpexc32_el2);
> + isb();
> + }
> + write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
> +
> + /* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
> + write_sysreg(1 << 15, hstr_el2);
> + /*
> + * Make sure we trap PMU access from EL0 to EL2. Also sanitize
> + * PMSELR_EL0 to make sure it never contains the cycle
> + * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
> + * EL1 instead of being trapped to EL2.
> + */
> + write_sysreg(0, pmselr_el0);
> + write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> + write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> +}
> +
> +static void __hyp_text __deactivate_traps_common(void)
> +{
> + write_sysreg(0, hstr_el2);
> + write_sysreg(0, pmuserenr_el0);
> +}
> +
> static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
> @@ -57,35 +94,7 @@ static hyp_alternate_select(__activate_traps_arch,
>
> static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> {
> - u64 val;
> -
> - /*
> - * We are about to set CPTR_EL2.TFP to trap all floating point
> - * register accesses to EL2, however, the ARM ARM clearly states that
> - * traps are only taken to EL2 if the operation would not otherwise
> - * trap to EL1. Therefore, always make sure that for 32-bit guests,
> - * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
> - * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
> - * it will cause an exception.
> - */
> - val = vcpu->arch.hcr_el2;
> - if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> - !vcpu->arch.guest_vfp_loaded) {
> - write_sysreg(1 << 30, fpexc32_el2);
> - isb();
> - }
> - write_sysreg(val, hcr_el2);
> - /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> - write_sysreg(1 << 15, hstr_el2);
> - /*
> - * Make sure we trap PMU access from EL0 to EL2. Also sanitize
> - * PMSELR_EL0 to make sure it never contains the cycle
> - * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
> - * EL1 instead of being trapped to EL2.
> - */
> - write_sysreg(0, pmselr_el0);
> - write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> - write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> + __activate_traps_common(vcpu);
> __activate_traps_arch()(vcpu);
> }
>
> @@ -131,9 +140,8 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> if (vcpu->arch.hcr_el2 & HCR_VSE)
> vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
>
> + __deactivate_traps_common();
> __deactivate_traps_arch()();
> - write_sysreg(0, hstr_el2);
> - write_sysreg(0, pmuserenr_el0);
> }
>
> static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 28/37] KVM: arm64: Move common VHE/non-VHE trap config in separate functions
@ 2017-11-25 10:43 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-25 10:43 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:32PM +0200, Christoffer Dall wrote:
> As we are about to be more lazy with some of the trap configuration
> register read/writes for VHE systems, move the logic that is currently
> shared between VHE and non-VHE into a separate function which can be
> called from either the world-switch path or from vcpu_load/vcpu_put.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm64/kvm/hyp/switch.c | 70 +++++++++++++++++++++++++--------------------
> 1 file changed, 39 insertions(+), 31 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 6a12504..c587416 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -23,6 +23,43 @@
> #include <asm/kvm_hyp.h>
> #include <asm/fpsimd.h>
>
> +static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
> +{
> + /*
> + * We are about to set CPTR_EL2.TFP to trap all floating point
> + * register accesses to EL2, however, the ARM ARM clearly states that
ARM ARM: typo?
> + * traps are only taken to EL2 if the operation would not otherwise
> + * trap to EL1. Therefore, always make sure that for 32-bit guests,
> + * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
> + * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
> + * it will cause an exception.
> + */
> + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> + !vcpu->arch.guest_vfp_loaded) {
> + write_sysreg(1 << 30, fpexc32_el2);
> + isb();
> + }
> + write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
> +
> + /* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
> + write_sysreg(1 << 15, hstr_el2);
> + /*
> + * Make sure we trap PMU access from EL0 to EL2. Also sanitize
> + * PMSELR_EL0 to make sure it never contains the cycle
> + * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
> + * EL1 instead of being trapped to EL2.
> + */
> + write_sysreg(0, pmselr_el0);
> + write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> + write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> +}
> +
> +static void __hyp_text __deactivate_traps_common(void)
> +{
> + write_sysreg(0, hstr_el2);
> + write_sysreg(0, pmuserenr_el0);
> +}
> +
> static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> {
> u64 val;
> @@ -57,35 +94,7 @@ static hyp_alternate_select(__activate_traps_arch,
>
> static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> {
> - u64 val;
> -
> - /*
> - * We are about to set CPTR_EL2.TFP to trap all floating point
> - * register accesses to EL2, however, the ARM ARM clearly states that
> - * traps are only taken to EL2 if the operation would not otherwise
> - * trap to EL1. Therefore, always make sure that for 32-bit guests,
> - * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
> - * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
> - * it will cause an exception.
> - */
> - val = vcpu->arch.hcr_el2;
> - if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> - !vcpu->arch.guest_vfp_loaded) {
> - write_sysreg(1 << 30, fpexc32_el2);
> - isb();
> - }
> - write_sysreg(val, hcr_el2);
> - /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> - write_sysreg(1 << 15, hstr_el2);
> - /*
> - * Make sure we trap PMU access from EL0 to EL2. Also sanitize
> - * PMSELR_EL0 to make sure it never contains the cycle
> - * counter, which could make a PMXEVCNTR_EL0 access UNDEF at
> - * EL1 instead of being trapped to EL2.
> - */
> - write_sysreg(0, pmselr_el0);
> - write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> - write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> + __activate_traps_common(vcpu);
> __activate_traps_arch()(vcpu);
> }
>
> @@ -131,9 +140,8 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> if (vcpu->arch.hcr_el2 & HCR_VSE)
> vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
>
> + __deactivate_traps_common();
> __deactivate_traps_arch()();
> - write_sysreg(0, hstr_el2);
> - write_sysreg(0, pmuserenr_el0);
> }
>
> static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 28/37] KVM: arm64: Move common VHE/non-VHE trap config in separate functions
2017-11-25 10:43 ` Yury Norov
@ 2017-11-25 10:49 ` Russell King - ARM Linux
-1 siblings, 0 replies; 254+ messages in thread
From: Russell King - ARM Linux @ 2017-11-25 10:49 UTC (permalink / raw)
To: Yury Norov; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Sat, Nov 25, 2017 at 01:43:47PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:32PM +0200, Christoffer Dall wrote:
> > As we are about to be more lazy with some of the trap configuration
> > register read/writes for VHE systems, move the logic that is currently
> > shared between VHE and non-VHE into a separate function which can be
> > called from either the world-switch path or from vcpu_load/vcpu_put.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 70 +++++++++++++++++++++++++--------------------
> > 1 file changed, 39 insertions(+), 31 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 6a12504..c587416 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -23,6 +23,43 @@
> > #include <asm/kvm_hyp.h>
> > #include <asm/fpsimd.h>
> >
> > +static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
> > +{
> > + /*
> > + * We are about to set CPTR_EL2.TFP to trap all floating point
> > + * register accesses to EL2, however, the ARM ARM clearly states that
>
> ARM ARM: typo?
No. ARM _A_rchitecture _R_eference _M_anual.
--
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 28/37] KVM: arm64: Move common VHE/non-VHE trap config in separate functions
@ 2017-11-25 10:49 ` Russell King - ARM Linux
0 siblings, 0 replies; 254+ messages in thread
From: Russell King - ARM Linux @ 2017-11-25 10:49 UTC (permalink / raw)
To: linux-arm-kernel
On Sat, Nov 25, 2017 at 01:43:47PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:32PM +0200, Christoffer Dall wrote:
> > As we are about to be more lazy with some of the trap configuration
> > register read/writes for VHE systems, move the logic that is currently
> > shared between VHE and non-VHE into a separate function which can be
> > called from either the world-switch path or from vcpu_load/vcpu_put.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm64/kvm/hyp/switch.c | 70 +++++++++++++++++++++++++--------------------
> > 1 file changed, 39 insertions(+), 31 deletions(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 6a12504..c587416 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -23,6 +23,43 @@
> > #include <asm/kvm_hyp.h>
> > #include <asm/fpsimd.h>
> >
> > +static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
> > +{
> > + /*
> > + * We are about to set CPTR_EL2.TFP to trap all floating point
> > + * register accesses to EL2, however, the ARM ARM clearly states that
>
> ARM ARM: typo?
No. ARM _A_rchitecture _R_eference _M_anual.
--
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 29/37] KVM: arm64: Configure FPSIMD traps on vcpu load/put for VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
There is no need to enable/disable traps to FP registers on every switch
to/from the VM, because the host kernel does not use this resource
without calling vcpu_put. We can therefore move things around enough
that we still always write FPEXC32_EL2 before programming CPTR_EL2 but
only program these during vcpu load/put.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 3 +++
arch/arm64/kvm/hyp/switch.c | 34 ++++++++++++++++++++++++----------
arch/arm64/kvm/hyp/sysreg-sr.c | 4 ++++
3 files changed, 31 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 3f54c55..28d5f3c 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -148,6 +148,9 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
bool __fpsimd_enabled(void);
+void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
+void deactivate_traps_vhe_put(void);
+
u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
void __noreturn __hyp_do_panic(unsigned long, ...);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index c587416..09be10f 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -23,22 +23,25 @@
#include <asm/kvm_hyp.h>
#include <asm/fpsimd.h>
-static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
+static void __hyp_text __activate_traps_fpsimd32(struct kvm_vcpu *vcpu)
{
/*
- * We are about to set CPTR_EL2.TFP to trap all floating point
- * register accesses to EL2, however, the ARM ARM clearly states that
- * traps are only taken to EL2 if the operation would not otherwise
- * trap to EL1. Therefore, always make sure that for 32-bit guests,
- * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
- * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
- * it will cause an exception.
+ * We are about to trap all floating point register accesses to EL2,
+ * however, traps are only taken to EL2 if the operation would not
+ * otherwise trap to EL1. Therefore, always make sure that for 32-bit
+ * guests, we set FPEXC.EN to prevent traps to EL1, when setting the
+ * TFP bit. If FP/ASIMD is not implemented, FPEXC is UNDEFINED and
+ * any access to it will cause an exception.
*/
if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
!vcpu->arch.guest_vfp_loaded) {
write_sysreg(1 << 30, fpexc32_el2);
isb();
}
+}
+
+static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
+{
write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
/* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
@@ -60,10 +63,12 @@ static void __hyp_text __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
-static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
+void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
{
u64 val;
+ __activate_traps_fpsimd32(vcpu);
+
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -71,7 +76,15 @@ static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
else
val &= ~CPACR_EL1_FPEN;
write_sysreg(val, cpacr_el1);
+}
+void deactivate_traps_vhe_put(void)
+{
+ write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
+}
+
+static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
+{
write_sysreg(__kvm_hyp_vector, vbar_el1);
}
@@ -79,6 +92,8 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
{
u64 val;
+ __activate_traps_fpsimd32(vcpu);
+
val = CPTR_EL2_DEFAULT;
val |= CPTR_EL2_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -109,7 +124,6 @@ static void __hyp_text __deactivate_traps_vhe(void)
write_sysreg(mdcr_el2, mdcr_el2);
write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
- write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
write_sysreg(vectors, vbar_el1);
}
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 6ff1ce5..54a4f55 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -242,6 +242,8 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
__sysreg_restore_el1_state(guest_ctxt);
vcpu->arch.sysregs_loaded_on_cpu = true;
+
+ activate_traps_vhe_load(vcpu);
}
/**
@@ -272,6 +274,8 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
if (!has_vhe())
return;
+ deactivate_traps_vhe_put();
+
__sysreg_save_el1_state(guest_ctxt);
__sysreg_save_user_state(guest_ctxt);
__sysreg32_save_state(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 29/37] KVM: arm64: Configure FPSIMD traps on vcpu load/put for VHE
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
There is no need to enable/disable traps to FP registers on every switch
to/from the VM, because the host kernel does not use this resource
without calling vcpu_put. We can therefore move things around enough
that we still always write FPEXC32_EL2 before programming CPTR_EL2 but
only program these during vcpu load/put.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 3 +++
arch/arm64/kvm/hyp/switch.c | 34 ++++++++++++++++++++++++----------
arch/arm64/kvm/hyp/sysreg-sr.c | 4 ++++
3 files changed, 31 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 3f54c55..28d5f3c 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -148,6 +148,9 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
bool __fpsimd_enabled(void);
+void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
+void deactivate_traps_vhe_put(void);
+
u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
void __noreturn __hyp_do_panic(unsigned long, ...);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index c587416..09be10f 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -23,22 +23,25 @@
#include <asm/kvm_hyp.h>
#include <asm/fpsimd.h>
-static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
+static void __hyp_text __activate_traps_fpsimd32(struct kvm_vcpu *vcpu)
{
/*
- * We are about to set CPTR_EL2.TFP to trap all floating point
- * register accesses to EL2, however, the ARM ARM clearly states that
- * traps are only taken to EL2 if the operation would not otherwise
- * trap to EL1. Therefore, always make sure that for 32-bit guests,
- * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
- * If FP/ASIMD is not implemented, FPEXC is UNDEFINED and any access to
- * it will cause an exception.
+ * We are about to trap all floating point register accesses to EL2,
+ * however, traps are only taken to EL2 if the operation would not
+ * otherwise trap to EL1. Therefore, always make sure that for 32-bit
+ * guests, we set FPEXC.EN to prevent traps to EL1, when setting the
+ * TFP bit. If FP/ASIMD is not implemented, FPEXC is UNDEFINED and
+ * any access to it will cause an exception.
*/
if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
!vcpu->arch.guest_vfp_loaded) {
write_sysreg(1 << 30, fpexc32_el2);
isb();
}
+}
+
+static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
+{
write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
/* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
@@ -60,10 +63,12 @@ static void __hyp_text __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
-static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
+void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
{
u64 val;
+ __activate_traps_fpsimd32(vcpu);
+
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -71,7 +76,15 @@ static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
else
val &= ~CPACR_EL1_FPEN;
write_sysreg(val, cpacr_el1);
+}
+void deactivate_traps_vhe_put(void)
+{
+ write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
+}
+
+static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
+{
write_sysreg(__kvm_hyp_vector, vbar_el1);
}
@@ -79,6 +92,8 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
{
u64 val;
+ __activate_traps_fpsimd32(vcpu);
+
val = CPTR_EL2_DEFAULT;
val |= CPTR_EL2_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -109,7 +124,6 @@ static void __hyp_text __deactivate_traps_vhe(void)
write_sysreg(mdcr_el2, mdcr_el2);
write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
- write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
write_sysreg(vectors, vbar_el1);
}
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 6ff1ce5..54a4f55 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -242,6 +242,8 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
__sysreg_restore_el1_state(guest_ctxt);
vcpu->arch.sysregs_loaded_on_cpu = true;
+
+ activate_traps_vhe_load(vcpu);
}
/**
@@ -272,6 +274,8 @@ void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
if (!has_vhe())
return;
+ deactivate_traps_vhe_put();
+
__sysreg_save_el1_state(guest_ctxt);
__sysreg_save_user_state(guest_ctxt);
__sysreg32_save_state(vcpu);
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 30/37] KVM: arm64: Configure c15, PMU, and debug register traps on cpu load/put for VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
We do not have to change the c15 trap setting on each switch to/from the
guest on VHE systems, because this setting only affects EL0.
The PMU and debug trap configuration can also be done on vcpu load/put
instead, because they don't affect how the host kernel can access the
debug registers while executing KVM kernel code and KVM doesn't use
floating point itself.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 35 ++++++++++++++++++++++++++---------
1 file changed, 26 insertions(+), 9 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 09be10f..13e137e 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -42,8 +42,6 @@ static void __hyp_text __activate_traps_fpsimd32(struct kvm_vcpu *vcpu)
static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
{
- write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
-
/* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
write_sysreg(1 << 15, hstr_el2);
/*
@@ -63,12 +61,15 @@ static void __hyp_text __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
+/* Activate the traps we can during vcpu_load with VHE */
void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
{
u64 val;
+ /* Make sure 32-bit guests trap VFP */
__activate_traps_fpsimd32(vcpu);
+ /* Trap VFP accesses on a VHE system */
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -76,11 +77,28 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
else
val &= ~CPACR_EL1_FPEN;
write_sysreg(val, cpacr_el1);
+
+ /* Activate traps on impdef sysregs, PMU, and debug */
+ __activate_traps_common(vcpu);
}
+/* Deactivate the traps we can during vcpu_put with VHE */
void deactivate_traps_vhe_put(void)
{
+ u64 mdcr_el2;
+
+ /* Re-enable host VFP access */
write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
+
+ /* Re-enable host access to impdef sysregs and the PMU */
+ __deactivate_traps_common();
+
+ /* Re-enable host access to the debug regs */
+ mdcr_el2 = read_sysreg(mdcr_el2);
+ mdcr_el2 &= MDCR_EL2_HPMN_MASK |
+ MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT |
+ MDCR_EL2_TPMS;
+ write_sysreg(mdcr_el2, mdcr_el2);
}
static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
@@ -92,8 +110,13 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
{
u64 val;
+ /* Activate traps on impdef sysregs, PMU, and debug */
+ __activate_traps_common(vcpu);
+
+ /* Make sure 32-bit guests trap VFP */
__activate_traps_fpsimd32(vcpu);
+ /* Trap VFP accesses on a non-VHE system */
val = CPTR_EL2_DEFAULT;
val |= CPTR_EL2_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -109,20 +132,14 @@ static hyp_alternate_select(__activate_traps_arch,
static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
{
- __activate_traps_common(vcpu);
__activate_traps_arch()(vcpu);
+ write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
}
static void __hyp_text __deactivate_traps_vhe(void)
{
extern char vectors[]; /* kernel exception vectors */
- u64 mdcr_el2 = read_sysreg(mdcr_el2);
- mdcr_el2 &= MDCR_EL2_HPMN_MASK |
- MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT |
- MDCR_EL2_TPMS;
-
- write_sysreg(mdcr_el2, mdcr_el2);
write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
write_sysreg(vectors, vbar_el1);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 30/37] KVM: arm64: Configure c15, PMU, and debug register traps on cpu load/put for VHE
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We do not have to change the c15 trap setting on each switch to/from the
guest on VHE systems, because this setting only affects EL0.
The PMU and debug trap configuration can also be done on vcpu load/put
instead, because they don't affect how the host kernel can access the
debug registers while executing KVM kernel code and KVM doesn't use
floating point itself.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 35 ++++++++++++++++++++++++++---------
1 file changed, 26 insertions(+), 9 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 09be10f..13e137e 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -42,8 +42,6 @@ static void __hyp_text __activate_traps_fpsimd32(struct kvm_vcpu *vcpu)
static void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)
{
- write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
-
/* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */
write_sysreg(1 << 15, hstr_el2);
/*
@@ -63,12 +61,15 @@ static void __hyp_text __deactivate_traps_common(void)
write_sysreg(0, pmuserenr_el0);
}
+/* Activate the traps we can during vcpu_load with VHE */
void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
{
u64 val;
+ /* Make sure 32-bit guests trap VFP */
__activate_traps_fpsimd32(vcpu);
+ /* Trap VFP accesses on a VHE system */
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -76,11 +77,28 @@ void activate_traps_vhe_load(struct kvm_vcpu *vcpu)
else
val &= ~CPACR_EL1_FPEN;
write_sysreg(val, cpacr_el1);
+
+ /* Activate traps on impdef sysregs, PMU, and debug */
+ __activate_traps_common(vcpu);
}
+/* Deactivate the traps we can during vcpu_put with VHE */
void deactivate_traps_vhe_put(void)
{
+ u64 mdcr_el2;
+
+ /* Re-enable host VFP access */
write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
+
+ /* Re-enable host access to impdef sysregs and the PMU */
+ __deactivate_traps_common();
+
+ /* Re-enable host access to the debug regs */
+ mdcr_el2 = read_sysreg(mdcr_el2);
+ mdcr_el2 &= MDCR_EL2_HPMN_MASK |
+ MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT |
+ MDCR_EL2_TPMS;
+ write_sysreg(mdcr_el2, mdcr_el2);
}
static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
@@ -92,8 +110,13 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
{
u64 val;
+ /* Activate traps on impdef sysregs, PMU, and debug */
+ __activate_traps_common(vcpu);
+
+ /* Make sure 32-bit guests trap VFP */
__activate_traps_fpsimd32(vcpu);
+ /* Trap VFP accesses on a non-VHE system */
val = CPTR_EL2_DEFAULT;
val |= CPTR_EL2_TTA;
if (vcpu->arch.guest_vfp_loaded)
@@ -109,20 +132,14 @@ static hyp_alternate_select(__activate_traps_arch,
static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
{
- __activate_traps_common(vcpu);
__activate_traps_arch()(vcpu);
+ write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
}
static void __hyp_text __deactivate_traps_vhe(void)
{
extern char vectors[]; /* kernel exception vectors */
- u64 mdcr_el2 = read_sysreg(mdcr_el2);
- mdcr_el2 &= MDCR_EL2_HPMN_MASK |
- MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT |
- MDCR_EL2_TPMS;
-
- write_sysreg(mdcr_el2, mdcr_el2);
write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
write_sysreg(vectors, vbar_el1);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 31/37] KVM: arm64: Separate activate_traps and deactive_traps for VHE and non-VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
To make the code more readable and to avoid the overhead of a function
call, let's get rid of a pair of the alternative function selectors and
explicitly call the VHE and non-VHE functions instead, telling the
compiler to try to inline the static function if it can.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 78 +++++++++++++++++++++------------------------
1 file changed, 37 insertions(+), 41 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 13e137e..5692aa0 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -101,9 +101,27 @@ void deactivate_traps_vhe_put(void)
write_sysreg(mdcr_el2, mdcr_el2);
}
-static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
+static inline void activate_traps_vhe(struct kvm_vcpu *vcpu)
{
- write_sysreg(__kvm_hyp_vector, vbar_el1);
+ write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
+ write_sysreg_el2(__kvm_hyp_vector, vbar);
+}
+
+static inline void deactivate_traps_vhe(struct kvm_vcpu *vcpu)
+{
+ extern char vectors[]; /* kernel exception vectors */
+
+ /*
+ * If we pended a virtual abort, preserve it until it gets
+ * cleared. See D1.14.3 (Virtual Interrupts) for details, but
+ * the crucial bit is "On taking a vSError interrupt,
+ * HCR_EL2.VSE is cleared to 0."
+ */
+ if (vcpu->arch.hcr_el2 & HCR_VSE)
+ vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
+
+ write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
+ write_sysreg(vectors, vbar_el1);
}
static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
@@ -124,44 +142,15 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
else
val |= CPTR_EL2_TFP;
write_sysreg(val, cptr_el2);
-}
-static hyp_alternate_select(__activate_traps_arch,
- __activate_traps_nvhe, __activate_traps_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
-static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
-{
- __activate_traps_arch()(vcpu);
+ /* Configure all other hypervisor traps and features */
write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
}
-static void __hyp_text __deactivate_traps_vhe(void)
-{
- extern char vectors[]; /* kernel exception vectors */
-
- write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
- write_sysreg(vectors, vbar_el1);
-}
-
-static void __hyp_text __deactivate_traps_nvhe(void)
+static void __hyp_text __deactivate_traps_nvhe(struct kvm_vcpu *vcpu)
{
- u64 mdcr_el2 = read_sysreg(mdcr_el2);
-
- mdcr_el2 &= MDCR_EL2_HPMN_MASK;
- mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
-
- write_sysreg(mdcr_el2, mdcr_el2);
- write_sysreg(HCR_RW, hcr_el2);
- write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
-}
-
-static hyp_alternate_select(__deactivate_traps_arch,
- __deactivate_traps_nvhe, __deactivate_traps_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
+ u64 mdcr_el2;
-static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
-{
/*
* If we pended a virtual abort, preserve it until it gets
* cleared. See D1.14.3 (Virtual Interrupts) for details, but
@@ -172,7 +161,14 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
__deactivate_traps_common();
- __deactivate_traps_arch()();
+
+ mdcr_el2 = read_sysreg(mdcr_el2);
+ mdcr_el2 &= MDCR_EL2_HPMN_MASK;
+ mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
+
+ write_sysreg(mdcr_el2, mdcr_el2);
+ write_sysreg(HCR_RW, hcr_el2);
+ write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
}
static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
@@ -371,7 +367,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
sysreg_save_host_state_vhe(host_ctxt);
- __activate_traps(vcpu);
+ activate_traps_vhe(vcpu);
__activate_vm(vcpu);
__vgic_restore_state(vcpu);
@@ -390,7 +386,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
sysreg_save_guest_state_vhe(guest_ctxt);
__vgic_save_state(vcpu);
- __deactivate_traps(vcpu);
+ deactivate_traps_vhe(vcpu);
sysreg_restore_host_state_vhe(host_ctxt);
@@ -418,7 +414,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(host_ctxt);
- __activate_traps(vcpu);
+ __activate_traps_nvhe(vcpu);
__activate_vm(vcpu);
__vgic_restore_state(vcpu);
@@ -445,7 +441,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
- __deactivate_traps(vcpu);
+ __deactivate_traps_nvhe(vcpu);
__deactivate_vm(vcpu);
__sysreg_restore_state_nvhe(host_ctxt);
@@ -471,7 +467,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
if (read_sysreg(vttbr_el2)) {
__timer_disable_traps(vcpu);
- __deactivate_traps(vcpu);
+ __deactivate_traps_nvhe(vcpu);
__deactivate_vm(vcpu);
__sysreg_restore_state_nvhe(__host_ctxt);
}
@@ -495,7 +491,7 @@ static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
struct kvm_vcpu *vcpu;
vcpu = host_ctxt->__hyp_running_vcpu;
- __deactivate_traps_vhe();
+ deactivate_traps_vhe(vcpu);
sysreg_restore_host_state_vhe(host_ctxt);
panic(__hyp_panic_string,
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 31/37] KVM: arm64: Separate activate_traps and deactive_traps for VHE and non-VHE
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
To make the code more readable and to avoid the overhead of a function
call, let's get rid of a pair of the alternative function selectors and
explicitly call the VHE and non-VHE functions instead, telling the
compiler to try to inline the static function if it can.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 78 +++++++++++++++++++++------------------------
1 file changed, 37 insertions(+), 41 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 13e137e..5692aa0 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -101,9 +101,27 @@ void deactivate_traps_vhe_put(void)
write_sysreg(mdcr_el2, mdcr_el2);
}
-static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
+static inline void activate_traps_vhe(struct kvm_vcpu *vcpu)
{
- write_sysreg(__kvm_hyp_vector, vbar_el1);
+ write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
+ write_sysreg_el2(__kvm_hyp_vector, vbar);
+}
+
+static inline void deactivate_traps_vhe(struct kvm_vcpu *vcpu)
+{
+ extern char vectors[]; /* kernel exception vectors */
+
+ /*
+ * If we pended a virtual abort, preserve it until it gets
+ * cleared. See D1.14.3 (Virtual Interrupts) for details, but
+ * the crucial bit is "On taking a vSError interrupt,
+ * HCR_EL2.VSE is cleared to 0."
+ */
+ if (vcpu->arch.hcr_el2 & HCR_VSE)
+ vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
+
+ write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
+ write_sysreg(vectors, vbar_el1);
}
static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
@@ -124,44 +142,15 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
else
val |= CPTR_EL2_TFP;
write_sysreg(val, cptr_el2);
-}
-static hyp_alternate_select(__activate_traps_arch,
- __activate_traps_nvhe, __activate_traps_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
-
-static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
-{
- __activate_traps_arch()(vcpu);
+ /* Configure all other hypervisor traps and features */
write_sysreg(vcpu->arch.hcr_el2, hcr_el2);
}
-static void __hyp_text __deactivate_traps_vhe(void)
-{
- extern char vectors[]; /* kernel exception vectors */
-
- write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
- write_sysreg(vectors, vbar_el1);
-}
-
-static void __hyp_text __deactivate_traps_nvhe(void)
+static void __hyp_text __deactivate_traps_nvhe(struct kvm_vcpu *vcpu)
{
- u64 mdcr_el2 = read_sysreg(mdcr_el2);
-
- mdcr_el2 &= MDCR_EL2_HPMN_MASK;
- mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
-
- write_sysreg(mdcr_el2, mdcr_el2);
- write_sysreg(HCR_RW, hcr_el2);
- write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
-}
-
-static hyp_alternate_select(__deactivate_traps_arch,
- __deactivate_traps_nvhe, __deactivate_traps_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
+ u64 mdcr_el2;
-static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
-{
/*
* If we pended a virtual abort, preserve it until it gets
* cleared. See D1.14.3 (Virtual Interrupts) for details, but
@@ -172,7 +161,14 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
vcpu->arch.hcr_el2 = read_sysreg(hcr_el2);
__deactivate_traps_common();
- __deactivate_traps_arch()();
+
+ mdcr_el2 = read_sysreg(mdcr_el2);
+ mdcr_el2 &= MDCR_EL2_HPMN_MASK;
+ mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT;
+
+ write_sysreg(mdcr_el2, mdcr_el2);
+ write_sysreg(HCR_RW, hcr_el2);
+ write_sysreg(CPTR_EL2_DEFAULT, cptr_el2);
}
static inline void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
@@ -371,7 +367,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
sysreg_save_host_state_vhe(host_ctxt);
- __activate_traps(vcpu);
+ activate_traps_vhe(vcpu);
__activate_vm(vcpu);
__vgic_restore_state(vcpu);
@@ -390,7 +386,7 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
sysreg_save_guest_state_vhe(guest_ctxt);
__vgic_save_state(vcpu);
- __deactivate_traps(vcpu);
+ deactivate_traps_vhe(vcpu);
sysreg_restore_host_state_vhe(host_ctxt);
@@ -418,7 +414,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__sysreg_save_state_nvhe(host_ctxt);
- __activate_traps(vcpu);
+ __activate_traps_nvhe(vcpu);
__activate_vm(vcpu);
__vgic_restore_state(vcpu);
@@ -445,7 +441,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__timer_disable_traps(vcpu);
__vgic_save_state(vcpu);
- __deactivate_traps(vcpu);
+ __deactivate_traps_nvhe(vcpu);
__deactivate_vm(vcpu);
__sysreg_restore_state_nvhe(host_ctxt);
@@ -471,7 +467,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,
if (read_sysreg(vttbr_el2)) {
__timer_disable_traps(vcpu);
- __deactivate_traps(vcpu);
+ __deactivate_traps_nvhe(vcpu);
__deactivate_vm(vcpu);
__sysreg_restore_state_nvhe(__host_ctxt);
}
@@ -495,7 +491,7 @@ static void __hyp_call_panic_vhe(u64 spsr, u64 elr, u64 par,
struct kvm_vcpu *vcpu;
vcpu = host_ctxt->__hyp_running_vcpu;
- __deactivate_traps_vhe();
+ deactivate_traps_vhe(vcpu);
sysreg_restore_host_state_vhe(host_ctxt);
panic(__hyp_panic_string,
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
We can program the GICv2 hypervisor control interface logic directly
from the core vgic code and can instead do the save/restore directly
from the flush/sync functions, which can lead to a number of future
optimizations.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/kvm/hyp/switch.c | 4 --
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/kvm/hyp/switch.c | 4 --
virt/kvm/arm/hyp/vgic-v2-sr.c | 83 ------------------------------------
virt/kvm/arm/vgic/vgic-init.c | 22 ++++++----
virt/kvm/arm/vgic/vgic-v2.c | 92 ++++++++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic.c | 21 ++++++++-
virt/kvm/arm/vgic/vgic.h | 5 +++
8 files changed, 130 insertions(+), 103 deletions(-)
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index c3b9799..0d834f8 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -91,16 +91,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_save_state(vcpu);
- else
- __vgic_v2_save_state(vcpu);
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_restore_state(vcpu);
- else
- __vgic_v2_restore_state(vcpu);
}
static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 28d5f3c..bd3fe64 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -121,8 +121,6 @@ typeof(orig) * __hyp_text fname(void) \
return val; \
}
-void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
-void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 5692aa0..90da506 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -186,16 +186,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_save_state(vcpu);
- else
- __vgic_v2_save_state(vcpu);
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_restore_state(vcpu);
- else
- __vgic_v2_restore_state(vcpu);
}
static bool __hyp_text __true_value(void)
diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/virt/kvm/arm/hyp/vgic-v2-sr.c
index a3f18d3..b433257 100644
--- a/virt/kvm/arm/hyp/vgic-v2-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v2-sr.c
@@ -22,89 +22,6 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
-static void __hyp_text save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
-{
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int nr_lr = (kern_hyp_va(&kvm_vgic_global_state))->nr_lr;
- u32 elrsr0, elrsr1;
-
- elrsr0 = readl_relaxed(base + GICH_ELRSR0);
- if (unlikely(nr_lr > 32))
- elrsr1 = readl_relaxed(base + GICH_ELRSR1);
- else
- elrsr1 = 0;
-
-#ifdef CONFIG_CPU_BIG_ENDIAN
- cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
-#else
- cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
-#endif
-}
-
-static void __hyp_text save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
-{
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int i;
- u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
-
- for (i = 0; i < used_lrs; i++) {
- if (cpu_if->vgic_elrsr & (1UL << i))
- cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
- else
- cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
-
- writel_relaxed(0, base + GICH_LR0 + (i * 4));
- }
-}
-
-/* vcpu is already in the HYP VA space */
-void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
-{
- struct kvm *kvm = kern_hyp_va(vcpu->kvm);
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- struct vgic_dist *vgic = &kvm->arch.vgic;
- void __iomem *base = kern_hyp_va(vgic->vctrl_base);
- u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
-
- if (!base)
- return;
-
- if (used_lrs) {
- cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
-
- save_elrsr(vcpu, base);
- save_lrs(vcpu, base);
-
- writel_relaxed(0, base + GICH_HCR);
- } else {
- cpu_if->vgic_elrsr = ~0UL;
- cpu_if->vgic_apr = 0;
- }
-}
-
-/* vcpu is already in the HYP VA space */
-void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
-{
- struct kvm *kvm = kern_hyp_va(vcpu->kvm);
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- struct vgic_dist *vgic = &kvm->arch.vgic;
- void __iomem *base = kern_hyp_va(vgic->vctrl_base);
- int i;
- u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
-
- if (!base)
- return;
-
- if (used_lrs) {
- writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
- writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
- for (i = 0; i < used_lrs; i++) {
- writel_relaxed(cpu_if->vgic_lr[i],
- base + GICH_LR0 + (i * 4));
- }
- }
-}
-
#ifdef CONFIG_ARM64
/*
* __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 5801261f..fa2b565 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -425,14 +425,16 @@ static int vgic_init_cpu_dying(unsigned int cpu)
return 0;
}
-static irqreturn_t vgic_maintenance_handler(int irq, void *data)
+static irqreturn_t vgic_v3_maintenance_handler(int irq, void *data)
{
- /*
- * We cannot rely on the vgic maintenance interrupt to be
- * delivered synchronously. This means we can only use it to
- * exit the VM, and we perform the handling of EOIed
- * interrupts on the exit path (see vgic_process_maintenance).
- */
+ BUG(); /* Not implemented lazy save/restore on GICv3 */
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t vgic_v2_maintenance_handler(int irq, void *data)
+{
+ struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data;
+ vgic_v2_handle_maintenance(vcpu);
return IRQ_HANDLED;
}
@@ -464,6 +466,7 @@ void kvm_vgic_init_cpu_hardware(void)
int kvm_vgic_hyp_init(void)
{
const struct gic_kvm_info *gic_kvm_info;
+ irqreturn_t (*handler)(int irq, void *data);
int ret;
gic_kvm_info = gic_get_kvm_info();
@@ -478,6 +481,7 @@ int kvm_vgic_hyp_init(void)
switch (gic_kvm_info->type) {
case GIC_V2:
ret = vgic_v2_probe(gic_kvm_info);
+ handler = vgic_v2_maintenance_handler;
break;
case GIC_V3:
ret = vgic_v3_probe(gic_kvm_info);
@@ -485,6 +489,7 @@ int kvm_vgic_hyp_init(void)
static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
kvm_info("GIC system register CPU interface enabled\n");
}
+ handler = vgic_v3_maintenance_handler;
break;
default:
ret = -ENODEV;
@@ -494,8 +499,7 @@ int kvm_vgic_hyp_init(void)
return ret;
kvm_vgic_global_state.maint_irq = gic_kvm_info->maint_irq;
- ret = request_percpu_irq(kvm_vgic_global_state.maint_irq,
- vgic_maintenance_handler,
+ ret = request_percpu_irq(kvm_vgic_global_state.maint_irq, handler,
"vgic", kvm_get_running_vcpus());
if (ret) {
kvm_err("Cannot register interrupt %d\n",
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 8089710..259079b 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -37,6 +37,17 @@ void vgic_v2_init_lrs(void)
vgic_v2_write_lr(i, 0);
}
+void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu)
+{
+ void __iomem *base = kvm_vgic_global_state.vctrl_base;
+
+ /*
+ * Disable maintenance interrupt as we only use it to generate an exit
+ * from the VM.
+ */
+ writel_relaxed(0, base + GICH_HCR);
+}
+
void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
{
struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
@@ -393,6 +404,87 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
return ret;
}
+static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
+{
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ int nr_lr = kvm_vgic_global_state.nr_lr;
+ u32 elrsr0, elrsr1;
+
+ elrsr0 = readl_relaxed(base + GICH_ELRSR0);
+ if (unlikely(nr_lr > 32))
+ elrsr1 = readl_relaxed(base + GICH_ELRSR1);
+ else
+ elrsr1 = 0;
+
+#ifdef CONFIG_CPU_BIG_ENDIAN
+ cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
+#else
+ cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
+#endif
+}
+
+static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
+{
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ int i;
+ u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+
+ for (i = 0; i < used_lrs; i++) {
+ if (cpu_if->vgic_elrsr & (1UL << i))
+ cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
+ else
+ cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
+
+ writel_relaxed(0, base + GICH_LR0 + (i * 4));
+ }
+}
+
+void vgic_v2_save_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ void __iomem *base = vgic->vctrl_base;
+ u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+
+ if (!base)
+ return;
+
+ if (used_lrs) {
+ cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
+
+ save_elrsr(vcpu, base);
+ save_lrs(vcpu, base);
+
+ writel_relaxed(0, base + GICH_HCR);
+ } else {
+ cpu_if->vgic_elrsr = ~0UL;
+ cpu_if->vgic_apr = 0;
+ }
+}
+
+void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ void __iomem *base = vgic->vctrl_base;
+ u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+ int i;
+
+ if (!base)
+ return;
+
+ if (used_lrs) {
+ writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
+ writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
+ for (i = 0; i < used_lrs; i++) {
+ writel_relaxed(cpu_if->vgic_lr[i],
+ base + GICH_LR0 + (i * 4));
+ }
+ }
+}
+
void vgic_v2_load(struct kvm_vcpu *vcpu)
{
struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index b1bd238..790fd66 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -18,6 +18,8 @@
#include <linux/kvm_host.h>
#include <linux/list_sort.h>
+#include <asm/kvm_hyp.h>
+
#include "vgic.h"
#define CREATE_TRACE_POINTS
@@ -683,11 +685,19 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
vgic_clear_lr(vcpu, count);
}
+static inline void vgic_save_state(struct kvm_vcpu *vcpu)
+{
+ if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ vgic_v2_save_state(vcpu);
+}
+
/* Sync back the hardware VGIC state into our emulation after a guest's run. */
void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ vgic_save_state(vcpu);
+
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
@@ -697,6 +707,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
vgic_prune_ap_list(vcpu);
}
+static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
+{
+ if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ vgic_v2_restore_state(vcpu);
+}
+
/* Flush our emulation state into the GIC hardware before entering the guest. */
void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
{
@@ -710,13 +726,16 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
* this.
*/
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
- return;
+ goto out;
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
spin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);
vgic_flush_lr_state(vcpu);
spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);
+
+out:
+ vgic_restore_state(vcpu);
}
void kvm_vgic_load(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 4f8aecb..9895396 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -167,6 +167,11 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
void vgic_v2_init_lrs(void);
void vgic_v2_load(struct kvm_vcpu *vcpu);
void vgic_v2_put(struct kvm_vcpu *vcpu);
+void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu);
+
+void vgic_v2_save_state(struct kvm_vcpu *vcpu);
+void vgic_v2_restore_state(struct kvm_vcpu *vcpu);
+
static inline void vgic_get_irq_kref(struct vgic_irq *irq)
{
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We can program the GICv2 hypervisor control interface logic directly
from the core vgic code and can instead do the save/restore directly
from the flush/sync functions, which can lead to a number of future
optimizations.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/kvm/hyp/switch.c | 4 --
arch/arm64/include/asm/kvm_hyp.h | 2 -
arch/arm64/kvm/hyp/switch.c | 4 --
virt/kvm/arm/hyp/vgic-v2-sr.c | 83 ------------------------------------
virt/kvm/arm/vgic/vgic-init.c | 22 ++++++----
virt/kvm/arm/vgic/vgic-v2.c | 92 ++++++++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic.c | 21 ++++++++-
virt/kvm/arm/vgic/vgic.h | 5 +++
8 files changed, 130 insertions(+), 103 deletions(-)
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index c3b9799..0d834f8 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -91,16 +91,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_save_state(vcpu);
- else
- __vgic_v2_save_state(vcpu);
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_restore_state(vcpu);
- else
- __vgic_v2_restore_state(vcpu);
}
static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 28d5f3c..bd3fe64 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -121,8 +121,6 @@ typeof(orig) * __hyp_text fname(void) \
return val; \
}
-void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
-void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 5692aa0..90da506 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -186,16 +186,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_save_state(vcpu);
- else
- __vgic_v2_save_state(vcpu);
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
__vgic_v3_restore_state(vcpu);
- else
- __vgic_v2_restore_state(vcpu);
}
static bool __hyp_text __true_value(void)
diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/virt/kvm/arm/hyp/vgic-v2-sr.c
index a3f18d3..b433257 100644
--- a/virt/kvm/arm/hyp/vgic-v2-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v2-sr.c
@@ -22,89 +22,6 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
-static void __hyp_text save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
-{
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int nr_lr = (kern_hyp_va(&kvm_vgic_global_state))->nr_lr;
- u32 elrsr0, elrsr1;
-
- elrsr0 = readl_relaxed(base + GICH_ELRSR0);
- if (unlikely(nr_lr > 32))
- elrsr1 = readl_relaxed(base + GICH_ELRSR1);
- else
- elrsr1 = 0;
-
-#ifdef CONFIG_CPU_BIG_ENDIAN
- cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
-#else
- cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
-#endif
-}
-
-static void __hyp_text save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
-{
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int i;
- u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
-
- for (i = 0; i < used_lrs; i++) {
- if (cpu_if->vgic_elrsr & (1UL << i))
- cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
- else
- cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
-
- writel_relaxed(0, base + GICH_LR0 + (i * 4));
- }
-}
-
-/* vcpu is already in the HYP VA space */
-void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
-{
- struct kvm *kvm = kern_hyp_va(vcpu->kvm);
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- struct vgic_dist *vgic = &kvm->arch.vgic;
- void __iomem *base = kern_hyp_va(vgic->vctrl_base);
- u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
-
- if (!base)
- return;
-
- if (used_lrs) {
- cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
-
- save_elrsr(vcpu, base);
- save_lrs(vcpu, base);
-
- writel_relaxed(0, base + GICH_HCR);
- } else {
- cpu_if->vgic_elrsr = ~0UL;
- cpu_if->vgic_apr = 0;
- }
-}
-
-/* vcpu is already in the HYP VA space */
-void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
-{
- struct kvm *kvm = kern_hyp_va(vcpu->kvm);
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- struct vgic_dist *vgic = &kvm->arch.vgic;
- void __iomem *base = kern_hyp_va(vgic->vctrl_base);
- int i;
- u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
-
- if (!base)
- return;
-
- if (used_lrs) {
- writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
- writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
- for (i = 0; i < used_lrs; i++) {
- writel_relaxed(cpu_if->vgic_lr[i],
- base + GICH_LR0 + (i * 4));
- }
- }
-}
-
#ifdef CONFIG_ARM64
/*
* __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index 5801261f..fa2b565 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -425,14 +425,16 @@ static int vgic_init_cpu_dying(unsigned int cpu)
return 0;
}
-static irqreturn_t vgic_maintenance_handler(int irq, void *data)
+static irqreturn_t vgic_v3_maintenance_handler(int irq, void *data)
{
- /*
- * We cannot rely on the vgic maintenance interrupt to be
- * delivered synchronously. This means we can only use it to
- * exit the VM, and we perform the handling of EOIed
- * interrupts on the exit path (see vgic_process_maintenance).
- */
+ BUG(); /* Not implemented lazy save/restore on GICv3 */
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t vgic_v2_maintenance_handler(int irq, void *data)
+{
+ struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data;
+ vgic_v2_handle_maintenance(vcpu);
return IRQ_HANDLED;
}
@@ -464,6 +466,7 @@ void kvm_vgic_init_cpu_hardware(void)
int kvm_vgic_hyp_init(void)
{
const struct gic_kvm_info *gic_kvm_info;
+ irqreturn_t (*handler)(int irq, void *data);
int ret;
gic_kvm_info = gic_get_kvm_info();
@@ -478,6 +481,7 @@ int kvm_vgic_hyp_init(void)
switch (gic_kvm_info->type) {
case GIC_V2:
ret = vgic_v2_probe(gic_kvm_info);
+ handler = vgic_v2_maintenance_handler;
break;
case GIC_V3:
ret = vgic_v3_probe(gic_kvm_info);
@@ -485,6 +489,7 @@ int kvm_vgic_hyp_init(void)
static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
kvm_info("GIC system register CPU interface enabled\n");
}
+ handler = vgic_v3_maintenance_handler;
break;
default:
ret = -ENODEV;
@@ -494,8 +499,7 @@ int kvm_vgic_hyp_init(void)
return ret;
kvm_vgic_global_state.maint_irq = gic_kvm_info->maint_irq;
- ret = request_percpu_irq(kvm_vgic_global_state.maint_irq,
- vgic_maintenance_handler,
+ ret = request_percpu_irq(kvm_vgic_global_state.maint_irq, handler,
"vgic", kvm_get_running_vcpus());
if (ret) {
kvm_err("Cannot register interrupt %d\n",
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 8089710..259079b 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -37,6 +37,17 @@ void vgic_v2_init_lrs(void)
vgic_v2_write_lr(i, 0);
}
+void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu)
+{
+ void __iomem *base = kvm_vgic_global_state.vctrl_base;
+
+ /*
+ * Disable maintenance interrupt as we only use it to generate an exit
+ * from the VM.
+ */
+ writel_relaxed(0, base + GICH_HCR);
+}
+
void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
{
struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
@@ -393,6 +404,87 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
return ret;
}
+static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
+{
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ int nr_lr = kvm_vgic_global_state.nr_lr;
+ u32 elrsr0, elrsr1;
+
+ elrsr0 = readl_relaxed(base + GICH_ELRSR0);
+ if (unlikely(nr_lr > 32))
+ elrsr1 = readl_relaxed(base + GICH_ELRSR1);
+ else
+ elrsr1 = 0;
+
+#ifdef CONFIG_CPU_BIG_ENDIAN
+ cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
+#else
+ cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
+#endif
+}
+
+static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
+{
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ int i;
+ u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+
+ for (i = 0; i < used_lrs; i++) {
+ if (cpu_if->vgic_elrsr & (1UL << i))
+ cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
+ else
+ cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
+
+ writel_relaxed(0, base + GICH_LR0 + (i * 4));
+ }
+}
+
+void vgic_v2_save_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ void __iomem *base = vgic->vctrl_base;
+ u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+
+ if (!base)
+ return;
+
+ if (used_lrs) {
+ cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
+
+ save_elrsr(vcpu, base);
+ save_lrs(vcpu, base);
+
+ writel_relaxed(0, base + GICH_HCR);
+ } else {
+ cpu_if->vgic_elrsr = ~0UL;
+ cpu_if->vgic_apr = 0;
+ }
+}
+
+void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ void __iomem *base = vgic->vctrl_base;
+ u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+ int i;
+
+ if (!base)
+ return;
+
+ if (used_lrs) {
+ writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
+ writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
+ for (i = 0; i < used_lrs; i++) {
+ writel_relaxed(cpu_if->vgic_lr[i],
+ base + GICH_LR0 + (i * 4));
+ }
+ }
+}
+
void vgic_v2_load(struct kvm_vcpu *vcpu)
{
struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index b1bd238..790fd66 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -18,6 +18,8 @@
#include <linux/kvm_host.h>
#include <linux/list_sort.h>
+#include <asm/kvm_hyp.h>
+
#include "vgic.h"
#define CREATE_TRACE_POINTS
@@ -683,11 +685,19 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
vgic_clear_lr(vcpu, count);
}
+static inline void vgic_save_state(struct kvm_vcpu *vcpu)
+{
+ if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ vgic_v2_save_state(vcpu);
+}
+
/* Sync back the hardware VGIC state into our emulation after a guest's run. */
void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ vgic_save_state(vcpu);
+
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
@@ -697,6 +707,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
vgic_prune_ap_list(vcpu);
}
+static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
+{
+ if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ vgic_v2_restore_state(vcpu);
+}
+
/* Flush our emulation state into the GIC hardware before entering the guest. */
void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
{
@@ -710,13 +726,16 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
* this.
*/
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
- return;
+ goto out;
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
spin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);
vgic_flush_lr_state(vcpu);
spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);
+
+out:
+ vgic_restore_state(vcpu);
}
void kvm_vgic_load(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 4f8aecb..9895396 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -167,6 +167,11 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
void vgic_v2_init_lrs(void);
void vgic_v2_load(struct kvm_vcpu *vcpu);
void vgic_v2_put(struct kvm_vcpu *vcpu);
+void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu);
+
+void vgic_v2_save_state(struct kvm_vcpu *vcpu);
+void vgic_v2_restore_state(struct kvm_vcpu *vcpu);
+
static inline void vgic_get_irq_kref(struct vgic_irq *irq)
{
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-15 17:50 ` Andre Przywara
-1 siblings, 0 replies; 254+ messages in thread
From: Andre Przywara @ 2017-11-15 17:50 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Hi,
those last few patches are actually helpful for the Xen port ...
On 12/10/17 11:41, Christoffer Dall wrote:
> We can program the GICv2 hypervisor control interface logic directly
> from the core vgic code and can instead do the save/restore directly
> from the flush/sync functions, which can lead to a number of future
> optimizations.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/kvm/hyp/switch.c | 4 --
> arch/arm64/include/asm/kvm_hyp.h | 2 -
> arch/arm64/kvm/hyp/switch.c | 4 --
> virt/kvm/arm/hyp/vgic-v2-sr.c | 83 ------------------------------------
> virt/kvm/arm/vgic/vgic-init.c | 22 ++++++----
> virt/kvm/arm/vgic/vgic-v2.c | 92 ++++++++++++++++++++++++++++++++++++++++
> virt/kvm/arm/vgic/vgic.c | 21 ++++++++-
> virt/kvm/arm/vgic/vgic.h | 5 +++
> 8 files changed, 130 insertions(+), 103 deletions(-)
>
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index c3b9799..0d834f8 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -91,16 +91,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_save_state(vcpu);
> - else
> - __vgic_v2_save_state(vcpu);
> }
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> - else
> - __vgic_v2_restore_state(vcpu);
> }
>
> static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index 28d5f3c..bd3fe64 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -121,8 +121,6 @@ typeof(orig) * __hyp_text fname(void) \
> return val; \
> }
>
> -void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> -void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
>
> void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 5692aa0..90da506 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -186,16 +186,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_save_state(vcpu);
> - else
> - __vgic_v2_save_state(vcpu);
> }
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> - else
> - __vgic_v2_restore_state(vcpu);
> }
>
> static bool __hyp_text __true_value(void)
> diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/virt/kvm/arm/hyp/vgic-v2-sr.c
> index a3f18d3..b433257 100644
> --- a/virt/kvm/arm/hyp/vgic-v2-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v2-sr.c
> @@ -22,89 +22,6 @@
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> -static void __hyp_text save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> -{
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int nr_lr = (kern_hyp_va(&kvm_vgic_global_state))->nr_lr;
> - u32 elrsr0, elrsr1;
> -
> - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> - if (unlikely(nr_lr > 32))
> - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> - else
> - elrsr1 = 0;
> -
> -#ifdef CONFIG_CPU_BIG_ENDIAN
> - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> -#else
> - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> -#endif
> -}
> -
> -static void __hyp_text save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> -{
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int i;
> - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> -
> - for (i = 0; i < used_lrs; i++) {
> - if (cpu_if->vgic_elrsr & (1UL << i))
> - cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> - else
> - cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> -
> - writel_relaxed(0, base + GICH_LR0 + (i * 4));
> - }
> -}
> -
> -/* vcpu is already in the HYP VA space */
> -void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
> -{
> - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - struct vgic_dist *vgic = &kvm->arch.vgic;
> - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> -
> - if (!base)
> - return;
> -
> - if (used_lrs) {
> - cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> -
> - save_elrsr(vcpu, base);
> - save_lrs(vcpu, base);
> -
> - writel_relaxed(0, base + GICH_HCR);
> - } else {
> - cpu_if->vgic_elrsr = ~0UL;
> - cpu_if->vgic_apr = 0;
> - }
> -}
> -
> -/* vcpu is already in the HYP VA space */
> -void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> -{
> - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - struct vgic_dist *vgic = &kvm->arch.vgic;
> - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> - int i;
> - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> -
> - if (!base)
> - return;
> -
> - if (used_lrs) {
> - writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> - writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> - for (i = 0; i < used_lrs; i++) {
> - writel_relaxed(cpu_if->vgic_lr[i],
> - base + GICH_LR0 + (i * 4));
> - }
> - }
> -}
> -
> #ifdef CONFIG_ARM64
> /*
> * __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f..fa2b565 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -425,14 +425,16 @@ static int vgic_init_cpu_dying(unsigned int cpu)
> return 0;
> }
>
> -static irqreturn_t vgic_maintenance_handler(int irq, void *data)
> +static irqreturn_t vgic_v3_maintenance_handler(int irq, void *data)
> {
> - /*
> - * We cannot rely on the vgic maintenance interrupt to be
> - * delivered synchronously. This means we can only use it to
> - * exit the VM, and we perform the handling of EOIed
> - * interrupts on the exit path (see vgic_process_maintenance).
> - */
I always found this comment quite enlightening, especially as it points
out that we need to deviate somewhat from the architectural idea here.
I see that you have shortened it below. Is it no longer true? Can we
keep the more elaborate version? If not here, then below?
> + BUG(); /* Not implemented lazy save/restore on GICv3 */
> + return IRQ_HANDLED;
> +}
> +
> +static irqreturn_t vgic_v2_maintenance_handler(int irq, void *data)
> +{
> + struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data;
I think you need an empty line here, to separate variable declaration
from code.
> + vgic_v2_handle_maintenance(vcpu);
> return IRQ_HANDLED;
> }
>
> @@ -464,6 +466,7 @@ void kvm_vgic_init_cpu_hardware(void)
> int kvm_vgic_hyp_init(void)
> {
> const struct gic_kvm_info *gic_kvm_info;
> + irqreturn_t (*handler)(int irq, void *data);
> int ret;
>
> gic_kvm_info = gic_get_kvm_info();
> @@ -478,6 +481,7 @@ int kvm_vgic_hyp_init(void)
> switch (gic_kvm_info->type) {
> case GIC_V2:
> ret = vgic_v2_probe(gic_kvm_info);
> + handler = vgic_v2_maintenance_handler;
> break;
1> case GIC_V3:
> ret = vgic_v3_probe(gic_kvm_info);
> @@ -485,6 +489,7 @@ int kvm_vgic_hyp_init(void)
> static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
> kvm_info("GIC system register CPU interface enabled\n");
> }
> + handler = vgic_v3_maintenance_handler;
> break;
> default:
> ret = -ENODEV;
> @@ -494,8 +499,7 @@ int kvm_vgic_hyp_init(void)
> return ret;
>
> kvm_vgic_global_state.maint_irq = gic_kvm_info->maint_irq;
> - ret = request_percpu_irq(kvm_vgic_global_state.maint_irq,
> - vgic_maintenance_handler,
> + ret = request_percpu_irq(kvm_vgic_global_state.maint_irq, handler,
> "vgic", kvm_get_running_vcpus());
> if (ret) {
> kvm_err("Cannot register interrupt %d\n",
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index 8089710..259079b 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -37,6 +37,17 @@ void vgic_v2_init_lrs(void)
> vgic_v2_write_lr(i, 0);
> }
>
> +void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu)
> +{
> + void __iomem *base = kvm_vgic_global_state.vctrl_base;
> +
> + /*
> + * Disable maintenance interrupt as we only use it to generate an exit
> + * from the VM.
> + */
Isn't that comment a bit misleading, as writing 0 to HCR not only
disables all interrupt sources, but also the whole GICV interface
altogether (bit 0: EN)?
I see that it gets enabled later on when writing ->vgic_hcr into the
register, but this function here looks a bit surprising to me.
In general these changes to the interrupt handling leave me a bit
puzzled. Should this be a separate patch? Or explained in the commit
message?
> + writel_relaxed(0, base + GICH_HCR);
> +}
> +
> void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
> {
> struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
> @@ -393,6 +404,87 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> return ret;
> }
>
> +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> +{
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + int nr_lr = kvm_vgic_global_state.nr_lr;
> + u32 elrsr0, elrsr1;
> +
> + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> + if (unlikely(nr_lr > 32))
> + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> + else
> + elrsr1 = 0;
> +
> +#ifdef CONFIG_CPU_BIG_ENDIAN
> + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> +#else
> + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> +#endif
I have some gut feeling that this is really broken, since we mix up
endian *byte* ordering with *bit* ordering here, don't we?
I understand it's just copied and gets removed later on, so I was
wondering if you could actually move patch 35/37 ("Get rid of
vgic_elrsr") before this patch here, to avoid copying bogus code around?
Or does 35/37 depend on 34/37 to be correct?
> +}
> +
> +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> +{
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + int i;
> + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> +
> + for (i = 0; i < used_lrs; i++) {
> + if (cpu_if->vgic_elrsr & (1UL << i))
> + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> + else
> + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> +
> + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> + }
> +}
> +
> +void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct vgic_dist *vgic = &kvm->arch.vgic;
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + void __iomem *base = vgic->vctrl_base;
> + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> +
> + if (!base)
> + return;
> +
> + if (used_lrs) {
> + cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> +
> + save_elrsr(vcpu, base);
> + save_lrs(vcpu, base);
> +
> + writel_relaxed(0, base + GICH_HCR);
> + } else {
> + cpu_if->vgic_elrsr = ~0UL;
> + cpu_if->vgic_apr = 0;
> + }
> +}
> +
> +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct vgic_dist *vgic = &kvm->arch.vgic;
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + void __iomem *base = vgic->vctrl_base;
> + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> + int i;
> +
> + if (!base)
> + return;
> +
> + if (used_lrs) {
> + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> + for (i = 0; i < used_lrs; i++) {
> + writel_relaxed(cpu_if->vgic_lr[i],
> + base + GICH_LR0 + (i * 4));
> + }
> + }
> +}
> +
> void vgic_v2_load(struct kvm_vcpu *vcpu)
> {
> struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index b1bd238..790fd66 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -18,6 +18,8 @@
> #include <linux/kvm_host.h>
> #include <linux/list_sort.h>
>
> +#include <asm/kvm_hyp.h>
Why do you need that? Commenting this out seems to compile anyway for me.
> +
> #include "vgic.h"
>
> #define CREATE_TRACE_POINTS
> @@ -683,11 +685,19 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
> vgic_clear_lr(vcpu, count);
> }
>
> +static inline void vgic_save_state(struct kvm_vcpu *vcpu)
Isn't "inline" frowned upon in .c files?
> +{
> + if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> + vgic_v2_save_state(vcpu);
> +}
> +
> /* Sync back the hardware VGIC state into our emulation after a guest's run. */
> void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> {
> struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>
> + vgic_save_state(vcpu);
> +
> /* An empty ap_list_head implies used_lrs == 0 */
> if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
> return;
> @@ -697,6 +707,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> vgic_prune_ap_list(vcpu);
> }
>
> +static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
Same here.
Apart from those rather cosmetic issues I have at least verified that
the code is actually moved from vgic-v2-sr.c to vgic-v2.c, plus/minus
the required changes when moving this from HYP to vGIC/EL1.
Cheers,
Andre.
> +{
> + if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> + vgic_v2_restore_state(vcpu);
> +}
> +
> /* Flush our emulation state into the GIC hardware before entering the guest. */
> void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
> {
> @@ -710,13 +726,16 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
> * this.
> */
> if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
> - return;
> + goto out;
>
> DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
>
> spin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);
> vgic_flush_lr_state(vcpu);
> spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);
> +
> +out:
> + vgic_restore_state(vcpu);
> }
>
> void kvm_vgic_load(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 4f8aecb..9895396 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -167,6 +167,11 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
> void vgic_v2_init_lrs(void);
> void vgic_v2_load(struct kvm_vcpu *vcpu);
> void vgic_v2_put(struct kvm_vcpu *vcpu);
> +void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu);
> +
> +void vgic_v2_save_state(struct kvm_vcpu *vcpu);
> +void vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> +
>
> static inline void vgic_get_irq_kref(struct vgic_irq *irq)
> {
>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
@ 2017-11-15 17:50 ` Andre Przywara
0 siblings, 0 replies; 254+ messages in thread
From: Andre Przywara @ 2017-11-15 17:50 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
those last few patches are actually helpful for the Xen port ...
On 12/10/17 11:41, Christoffer Dall wrote:
> We can program the GICv2 hypervisor control interface logic directly
> from the core vgic code and can instead do the save/restore directly
> from the flush/sync functions, which can lead to a number of future
> optimizations.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/kvm/hyp/switch.c | 4 --
> arch/arm64/include/asm/kvm_hyp.h | 2 -
> arch/arm64/kvm/hyp/switch.c | 4 --
> virt/kvm/arm/hyp/vgic-v2-sr.c | 83 ------------------------------------
> virt/kvm/arm/vgic/vgic-init.c | 22 ++++++----
> virt/kvm/arm/vgic/vgic-v2.c | 92 ++++++++++++++++++++++++++++++++++++++++
> virt/kvm/arm/vgic/vgic.c | 21 ++++++++-
> virt/kvm/arm/vgic/vgic.h | 5 +++
> 8 files changed, 130 insertions(+), 103 deletions(-)
>
> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> index c3b9799..0d834f8 100644
> --- a/arch/arm/kvm/hyp/switch.c
> +++ b/arch/arm/kvm/hyp/switch.c
> @@ -91,16 +91,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_save_state(vcpu);
> - else
> - __vgic_v2_save_state(vcpu);
> }
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> - else
> - __vgic_v2_restore_state(vcpu);
> }
>
> static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> index 28d5f3c..bd3fe64 100644
> --- a/arch/arm64/include/asm/kvm_hyp.h
> +++ b/arch/arm64/include/asm/kvm_hyp.h
> @@ -121,8 +121,6 @@ typeof(orig) * __hyp_text fname(void) \
> return val; \
> }
>
> -void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> -void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
>
> void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 5692aa0..90da506 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -186,16 +186,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_save_state(vcpu);
> - else
> - __vgic_v2_save_state(vcpu);
> }
>
> static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> {
> if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> __vgic_v3_restore_state(vcpu);
> - else
> - __vgic_v2_restore_state(vcpu);
> }
>
> static bool __hyp_text __true_value(void)
> diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/virt/kvm/arm/hyp/vgic-v2-sr.c
> index a3f18d3..b433257 100644
> --- a/virt/kvm/arm/hyp/vgic-v2-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v2-sr.c
> @@ -22,89 +22,6 @@
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> -static void __hyp_text save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> -{
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int nr_lr = (kern_hyp_va(&kvm_vgic_global_state))->nr_lr;
> - u32 elrsr0, elrsr1;
> -
> - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> - if (unlikely(nr_lr > 32))
> - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> - else
> - elrsr1 = 0;
> -
> -#ifdef CONFIG_CPU_BIG_ENDIAN
> - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> -#else
> - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> -#endif
> -}
> -
> -static void __hyp_text save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> -{
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int i;
> - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> -
> - for (i = 0; i < used_lrs; i++) {
> - if (cpu_if->vgic_elrsr & (1UL << i))
> - cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> - else
> - cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> -
> - writel_relaxed(0, base + GICH_LR0 + (i * 4));
> - }
> -}
> -
> -/* vcpu is already in the HYP VA space */
> -void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
> -{
> - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - struct vgic_dist *vgic = &kvm->arch.vgic;
> - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> -
> - if (!base)
> - return;
> -
> - if (used_lrs) {
> - cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> -
> - save_elrsr(vcpu, base);
> - save_lrs(vcpu, base);
> -
> - writel_relaxed(0, base + GICH_HCR);
> - } else {
> - cpu_if->vgic_elrsr = ~0UL;
> - cpu_if->vgic_apr = 0;
> - }
> -}
> -
> -/* vcpu is already in the HYP VA space */
> -void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> -{
> - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - struct vgic_dist *vgic = &kvm->arch.vgic;
> - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> - int i;
> - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> -
> - if (!base)
> - return;
> -
> - if (used_lrs) {
> - writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> - writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> - for (i = 0; i < used_lrs; i++) {
> - writel_relaxed(cpu_if->vgic_lr[i],
> - base + GICH_LR0 + (i * 4));
> - }
> - }
> -}
> -
> #ifdef CONFIG_ARM64
> /*
> * __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
> diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> index 5801261f..fa2b565 100644
> --- a/virt/kvm/arm/vgic/vgic-init.c
> +++ b/virt/kvm/arm/vgic/vgic-init.c
> @@ -425,14 +425,16 @@ static int vgic_init_cpu_dying(unsigned int cpu)
> return 0;
> }
>
> -static irqreturn_t vgic_maintenance_handler(int irq, void *data)
> +static irqreturn_t vgic_v3_maintenance_handler(int irq, void *data)
> {
> - /*
> - * We cannot rely on the vgic maintenance interrupt to be
> - * delivered synchronously. This means we can only use it to
> - * exit the VM, and we perform the handling of EOIed
> - * interrupts on the exit path (see vgic_process_maintenance).
> - */
I always found this comment quite enlightening, especially as it points
out that we need to deviate somewhat from the architectural idea here.
I see that you have shortened it below. Is it no longer true? Can we
keep the more elaborate version? If not here, then below?
> + BUG(); /* Not implemented lazy save/restore on GICv3 */
> + return IRQ_HANDLED;
> +}
> +
> +static irqreturn_t vgic_v2_maintenance_handler(int irq, void *data)
> +{
> + struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data;
I think you need an empty line here, to separate variable declaration
from code.
> + vgic_v2_handle_maintenance(vcpu);
> return IRQ_HANDLED;
> }
>
> @@ -464,6 +466,7 @@ void kvm_vgic_init_cpu_hardware(void)
> int kvm_vgic_hyp_init(void)
> {
> const struct gic_kvm_info *gic_kvm_info;
> + irqreturn_t (*handler)(int irq, void *data);
> int ret;
>
> gic_kvm_info = gic_get_kvm_info();
> @@ -478,6 +481,7 @@ int kvm_vgic_hyp_init(void)
> switch (gic_kvm_info->type) {
> case GIC_V2:
> ret = vgic_v2_probe(gic_kvm_info);
> + handler = vgic_v2_maintenance_handler;
> break;
1> case GIC_V3:
> ret = vgic_v3_probe(gic_kvm_info);
> @@ -485,6 +489,7 @@ int kvm_vgic_hyp_init(void)
> static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
> kvm_info("GIC system register CPU interface enabled\n");
> }
> + handler = vgic_v3_maintenance_handler;
> break;
> default:
> ret = -ENODEV;
> @@ -494,8 +499,7 @@ int kvm_vgic_hyp_init(void)
> return ret;
>
> kvm_vgic_global_state.maint_irq = gic_kvm_info->maint_irq;
> - ret = request_percpu_irq(kvm_vgic_global_state.maint_irq,
> - vgic_maintenance_handler,
> + ret = request_percpu_irq(kvm_vgic_global_state.maint_irq, handler,
> "vgic", kvm_get_running_vcpus());
> if (ret) {
> kvm_err("Cannot register interrupt %d\n",
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index 8089710..259079b 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -37,6 +37,17 @@ void vgic_v2_init_lrs(void)
> vgic_v2_write_lr(i, 0);
> }
>
> +void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu)
> +{
> + void __iomem *base = kvm_vgic_global_state.vctrl_base;
> +
> + /*
> + * Disable maintenance interrupt as we only use it to generate an exit
> + * from the VM.
> + */
Isn't that comment a bit misleading, as writing 0 to HCR not only
disables all interrupt sources, but also the whole GICV interface
altogether (bit 0: EN)?
I see that it gets enabled later on when writing ->vgic_hcr into the
register, but this function here looks a bit surprising to me.
In general these changes to the interrupt handling leave me a bit
puzzled. Should this be a separate patch? Or explained in the commit
message?
> + writel_relaxed(0, base + GICH_HCR);
> +}
> +
> void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
> {
> struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
> @@ -393,6 +404,87 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> return ret;
> }
>
> +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> +{
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + int nr_lr = kvm_vgic_global_state.nr_lr;
> + u32 elrsr0, elrsr1;
> +
> + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> + if (unlikely(nr_lr > 32))
> + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> + else
> + elrsr1 = 0;
> +
> +#ifdef CONFIG_CPU_BIG_ENDIAN
> + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> +#else
> + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> +#endif
I have some gut feeling that this is really broken, since we mix up
endian *byte* ordering with *bit* ordering here, don't we?
I understand it's just copied and gets removed later on, so I was
wondering if you could actually move patch 35/37 ("Get rid of
vgic_elrsr") before this patch here, to avoid copying bogus code around?
Or does 35/37 depend on 34/37 to be correct?
> +}
> +
> +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> +{
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + int i;
> + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> +
> + for (i = 0; i < used_lrs; i++) {
> + if (cpu_if->vgic_elrsr & (1UL << i))
> + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> + else
> + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> +
> + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> + }
> +}
> +
> +void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct vgic_dist *vgic = &kvm->arch.vgic;
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + void __iomem *base = vgic->vctrl_base;
> + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> +
> + if (!base)
> + return;
> +
> + if (used_lrs) {
> + cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> +
> + save_elrsr(vcpu, base);
> + save_lrs(vcpu, base);
> +
> + writel_relaxed(0, base + GICH_HCR);
> + } else {
> + cpu_if->vgic_elrsr = ~0UL;
> + cpu_if->vgic_apr = 0;
> + }
> +}
> +
> +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = vcpu->kvm;
> + struct vgic_dist *vgic = &kvm->arch.vgic;
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + void __iomem *base = vgic->vctrl_base;
> + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> + int i;
> +
> + if (!base)
> + return;
> +
> + if (used_lrs) {
> + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> + for (i = 0; i < used_lrs; i++) {
> + writel_relaxed(cpu_if->vgic_lr[i],
> + base + GICH_LR0 + (i * 4));
> + }
> + }
> +}
> +
> void vgic_v2_load(struct kvm_vcpu *vcpu)
> {
> struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> index b1bd238..790fd66 100644
> --- a/virt/kvm/arm/vgic/vgic.c
> +++ b/virt/kvm/arm/vgic/vgic.c
> @@ -18,6 +18,8 @@
> #include <linux/kvm_host.h>
> #include <linux/list_sort.h>
>
> +#include <asm/kvm_hyp.h>
Why do you need that? Commenting this out seems to compile anyway for me.
> +
> #include "vgic.h"
>
> #define CREATE_TRACE_POINTS
> @@ -683,11 +685,19 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
> vgic_clear_lr(vcpu, count);
> }
>
> +static inline void vgic_save_state(struct kvm_vcpu *vcpu)
Isn't "inline" frowned upon in .c files?
> +{
> + if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> + vgic_v2_save_state(vcpu);
> +}
> +
> /* Sync back the hardware VGIC state into our emulation after a guest's run. */
> void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> {
> struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>
> + vgic_save_state(vcpu);
> +
> /* An empty ap_list_head implies used_lrs == 0 */
> if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
> return;
> @@ -697,6 +707,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> vgic_prune_ap_list(vcpu);
> }
>
> +static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
Same here.
Apart from those rather cosmetic issues I have at least verified that
the code is actually moved from vgic-v2-sr.c to vgic-v2.c, plus/minus
the required changes when moving this from HYP to vGIC/EL1.
Cheers,
Andre.
> +{
> + if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> + vgic_v2_restore_state(vcpu);
> +}
> +
> /* Flush our emulation state into the GIC hardware before entering the guest. */
> void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
> {
> @@ -710,13 +726,16 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
> * this.
> */
> if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
> - return;
> + goto out;
>
> DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
>
> spin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);
> vgic_flush_lr_state(vcpu);
> spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);
> +
> +out:
> + vgic_restore_state(vcpu);
> }
>
> void kvm_vgic_load(struct kvm_vcpu *vcpu)
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 4f8aecb..9895396 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -167,6 +167,11 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,
> void vgic_v2_init_lrs(void);
> void vgic_v2_load(struct kvm_vcpu *vcpu);
> void vgic_v2_put(struct kvm_vcpu *vcpu);
> +void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu);
> +
> +void vgic_v2_save_state(struct kvm_vcpu *vcpu);
> +void vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> +
>
> static inline void vgic_get_irq_kref(struct vgic_irq *irq)
> {
>
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
2017-11-15 17:50 ` Andre Przywara
@ 2017-11-26 10:29 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 10:29 UTC (permalink / raw)
To: Andre Przywara; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> Hi,
>
> those last few patches are actually helpful for the Xen port ...
[...]
> > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > + u32 elrsr0, elrsr1;
> > +
> > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > + if (unlikely(nr_lr > 32))
> > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > + else
> > + elrsr1 = 0;
> > +
> > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > +#else
> > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > +#endif
>
> I have some gut feeling that this is really broken, since we mix up
> endian *byte* ordering with *bit* ordering here, don't we?
Good feeling indeed. :)
We have bitmap_{from,to)_u32array for things like this. But it was
considered bad-designed, and I proposed new bitmap_{from,to)_arr32().
https://lkml.org/lkml/2017/11/15/592
What else I have in mind, to introduce something like bitmap_{from,to}_pair_32()
as most of current users of bitmap_{from,to)_u32array(), (and those who should
use it but don't, like this one) have only 2 32-bit halfwords to be copied
from/to bitmap.
Also, it will be complementary to bitmap_from_u64().
More reading about bitmap/array conversion is in comment to BITMAP_FROM_U64
macro.
> I understand it's just copied and gets removed later on, so I was
> wondering if you could actually move patch 35/37 ("Get rid of
> vgic_elrsr") before this patch here, to avoid copying bogus code around?
> Or does 35/37 depend on 34/37 to be correct?
>
> > +}
> > +
> > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int i;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > +
> > + for (i = 0; i < used_lrs; i++) {
> > + if (cpu_if->vgic_elrsr & (1UL << i))
So, the vgic_elrsr is naturally bitmap, and bitmap API is preferred if no
other considerations:
if (test_bit(i, cpu_if->vgic_elrsr))
> > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > + else
> > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > +
> > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > + }
> > +}
I'd also headscratch about using for_each_clear_bit() here:
/*
* Setup default vgic_lr values somewhere earlier.
* Not needed at all if you take my suggestion for
* vgic_v2_restore_state() below
*/
for (i = 0; i < used_lrs; i++)
cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
{
[...]
for_each_clear_bit (i, cpu_if->vgic_elrsr, used_lrs)
cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
for (i = 0; i < used_lrs; i++)
writel_relaxed(0, base + GICH_LR0 + (i * 4));
}
Not sure how performance-critical this path is, but sometimes things
get really faster with bitmaps.
[...]
> > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + void __iomem *base = vgic->vctrl_base;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > + int i;
> > +
> > + if (!base)
> > + return;
> > +
> > + if (used_lrs) {
> > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > + for (i = 0; i < used_lrs; i++) {
> > + writel_relaxed(cpu_if->vgic_lr[i],
> > + base + GICH_LR0 + (i * 4));
> > + }
> > + }
> > +}
The alternative approach would be:
for (i = 0; i < used_lrs; i++) {
if (test_bit(i, cpu_if->vgic_elrsr))
writel_relaxed(~GICH_LR_STATE, base + GICH_LR0 + (i * 4));
else
writel_relaxed(cpu_if->vgic_lr[i], base + GICH_LR0 + (i * 4));
}
If cpu_if->vgic_elrsr is untouched in-between of course. It will make
save_lrs() simpler and this function more verbose.
Thanks,
Yury
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
@ 2017-11-26 10:29 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 10:29 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> Hi,
>
> those last few patches are actually helpful for the Xen port ...
[...]
> > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > + u32 elrsr0, elrsr1;
> > +
> > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > + if (unlikely(nr_lr > 32))
> > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > + else
> > + elrsr1 = 0;
> > +
> > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > +#else
> > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > +#endif
>
> I have some gut feeling that this is really broken, since we mix up
> endian *byte* ordering with *bit* ordering here, don't we?
Good feeling indeed. :)
We have bitmap_{from,to)_u32array for things like this. But it was
considered bad-designed, and I proposed new bitmap_{from,to)_arr32().
https://lkml.org/lkml/2017/11/15/592
What else I have in mind, to introduce something like bitmap_{from,to}_pair_32()
as most of current users of bitmap_{from,to)_u32array(), (and those who should
use it but don't, like this one) have only 2 32-bit halfwords to be copied
from/to bitmap.
Also, it will be complementary to bitmap_from_u64().
More reading about bitmap/array conversion is in comment to BITMAP_FROM_U64
macro.
> I understand it's just copied and gets removed later on, so I was
> wondering if you could actually move patch 35/37 ("Get rid of
> vgic_elrsr") before this patch here, to avoid copying bogus code around?
> Or does 35/37 depend on 34/37 to be correct?
>
> > +}
> > +
> > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int i;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > +
> > + for (i = 0; i < used_lrs; i++) {
> > + if (cpu_if->vgic_elrsr & (1UL << i))
So, the vgic_elrsr is naturally bitmap, and bitmap API is preferred if no
other considerations:
if (test_bit(i, cpu_if->vgic_elrsr))
> > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > + else
> > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > +
> > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > + }
> > +}
I'd also headscratch about using for_each_clear_bit() here:
/*
* Setup default vgic_lr values somewhere earlier.
* Not needed at all if you take my suggestion for
* vgic_v2_restore_state() below
*/
for (i = 0; i < used_lrs; i++)
cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
{
[...]
for_each_clear_bit (i, cpu_if->vgic_elrsr, used_lrs)
cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
for (i = 0; i < used_lrs; i++)
writel_relaxed(0, base + GICH_LR0 + (i * 4));
}
Not sure how performance-critical this path is, but sometimes things
get really faster with bitmaps.
[...]
> > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + void __iomem *base = vgic->vctrl_base;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > + int i;
> > +
> > + if (!base)
> > + return;
> > +
> > + if (used_lrs) {
> > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > + for (i = 0; i < used_lrs; i++) {
> > + writel_relaxed(cpu_if->vgic_lr[i],
> > + base + GICH_LR0 + (i * 4));
> > + }
> > + }
> > +}
The alternative approach would be:
for (i = 0; i < used_lrs; i++) {
if (test_bit(i, cpu_if->vgic_elrsr))
writel_relaxed(~GICH_LR_STATE, base + GICH_LR0 + (i * 4));
else
writel_relaxed(cpu_if->vgic_lr[i], base + GICH_LR0 + (i * 4));
}
If cpu_if->vgic_elrsr is untouched in-between of course. It will make
save_lrs() simpler and this function more verbose.
Thanks,
Yury
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
2017-11-26 10:29 ` Yury Norov
@ 2017-11-26 19:46 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:46 UTC (permalink / raw)
To: Yury Norov
Cc: Andre Przywara, Christoffer Dall, kvmarm, linux-arm-kernel,
Marc Zyngier, kvm, Shih-Wei Li
On Sun, Nov 26, 2017 at 01:29:30PM +0300, Yury Norov wrote:
> On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> > Hi,
> >
> > those last few patches are actually helpful for the Xen port ...
>
> [...]
>
> > > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > > +{
> > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > > + u32 elrsr0, elrsr1;
> > > +
> > > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > > + if (unlikely(nr_lr > 32))
> > > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > > + else
> > > + elrsr1 = 0;
> > > +
> > > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > > +#else
> > > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > > +#endif
> >
> > I have some gut feeling that this is really broken, since we mix up
> > endian *byte* ordering with *bit* ordering here, don't we?
>
> Good feeling indeed. :)
>
> We have bitmap_{from,to)_u32array for things like this. But it was
> considered bad-designed, and I proposed new bitmap_{from,to)_arr32().
>
> https://lkml.org/lkml/2017/11/15/592
>
> What else I have in mind, to introduce something like bitmap_{from,to}_pair_32()
> as most of current users of bitmap_{from,to)_u32array(), (and those who should
> use it but don't, like this one) have only 2 32-bit halfwords to be copied
> from/to bitmap.
>
> Also, it will be complementary to bitmap_from_u64().
>
> More reading about bitmap/array conversion is in comment to BITMAP_FROM_U64
> macro.
>
I have no idea what you want to introduce here. If you have an idea on
how to improve the code, patches are welcome.
Please keep in mind, that the purpose of this patch is to move code
around to improve the GIC handling performance, not changing the
lower-level details of the code.
> > I understand it's just copied and gets removed later on, so I was
> > wondering if you could actually move patch 35/37 ("Get rid of
> > vgic_elrsr") before this patch here, to avoid copying bogus code around?
> > Or does 35/37 depend on 34/37 to be correct?
> >
> > > +}
> > > +
> > > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > > +{
> > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > + int i;
> > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > +
> > > + for (i = 0; i < used_lrs; i++) {
> > > + if (cpu_if->vgic_elrsr & (1UL << i))
>
> So, the vgic_elrsr is naturally bitmap, and bitmap API is preferred if no
> other considerations:
> if (test_bit(i, cpu_if->vgic_elrsr))
>
> > > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > > + else
> > > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > > +
> > > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > > + }
> > > +}
>
> I'd also headscratch about using for_each_clear_bit() here:
>
> /*
> * Setup default vgic_lr values somewhere earlier.
Not sure what the 'default' values are.
> * Not needed at all if you take my suggestion for
> * vgic_v2_restore_state() below
> */
> for (i = 0; i < used_lrs; i++)
> cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
>
> static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> {
> [...]
>
> for_each_clear_bit (i, cpu_if->vgic_elrsr, used_lrs)
> cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
>
> for (i = 0; i < used_lrs; i++)
> writel_relaxed(0, base + GICH_LR0 + (i * 4));
> }
>
> Not sure how performance-critical this path is, but sometimes things
> get really faster with bitmaps.
>
Your suggestion below would require us to maintain elrsr when we setup
list registers, and I don't really see the benefit.
> [...]
>
> > > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > > +{
> > > + struct kvm *kvm = vcpu->kvm;
> > > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > + void __iomem *base = vgic->vctrl_base;
> > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > + int i;
> > > +
> > > + if (!base)
> > > + return;
> > > +
> > > + if (used_lrs) {
> > > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > > + for (i = 0; i < used_lrs; i++) {
> > > + writel_relaxed(cpu_if->vgic_lr[i],
> > > + base + GICH_LR0 + (i * 4));
> > > + }
> > > + }
> > > +}
>
> The alternative approach would be:
> for (i = 0; i < used_lrs; i++) {
> if (test_bit(i, cpu_if->vgic_elrsr))
> writel_relaxed(~GICH_LR_STATE, base + GICH_LR0 + (i * 4));
> else
> writel_relaxed(cpu_if->vgic_lr[i], base + GICH_LR0 + (i * 4));
> }
>
> If cpu_if->vgic_elrsr is untouched in-between of course. It will make
> save_lrs() simpler and this function more verbose.
>
I don't understand your suggestion. As you will see later, we will get
rid of storing the elrsr completely with a measureable performance
improvement.
If you think you can improve the code beyond that, a follow-up patch
would be most welcome.
Note that on all the implementations I'm familiar with, the maximum
number of LRs is four, so we're not wading through massive bitmaps in
practice here.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
@ 2017-11-26 19:46 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:46 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 01:29:30PM +0300, Yury Norov wrote:
> On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> > Hi,
> >
> > those last few patches are actually helpful for the Xen port ...
>
> [...]
>
> > > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > > +{
> > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > > + u32 elrsr0, elrsr1;
> > > +
> > > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > > + if (unlikely(nr_lr > 32))
> > > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > > + else
> > > + elrsr1 = 0;
> > > +
> > > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > > +#else
> > > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > > +#endif
> >
> > I have some gut feeling that this is really broken, since we mix up
> > endian *byte* ordering with *bit* ordering here, don't we?
>
> Good feeling indeed. :)
>
> We have bitmap_{from,to)_u32array for things like this. But it was
> considered bad-designed, and I proposed new bitmap_{from,to)_arr32().
>
> https://lkml.org/lkml/2017/11/15/592
>
> What else I have in mind, to introduce something like bitmap_{from,to}_pair_32()
> as most of current users of bitmap_{from,to)_u32array(), (and those who should
> use it but don't, like this one) have only 2 32-bit halfwords to be copied
> from/to bitmap.
>
> Also, it will be complementary to bitmap_from_u64().
>
> More reading about bitmap/array conversion is in comment to BITMAP_FROM_U64
> macro.
>
I have no idea what you want to introduce here. If you have an idea on
how to improve the code, patches are welcome.
Please keep in mind, that the purpose of this patch is to move code
around to improve the GIC handling performance, not changing the
lower-level details of the code.
> > I understand it's just copied and gets removed later on, so I was
> > wondering if you could actually move patch 35/37 ("Get rid of
> > vgic_elrsr") before this patch here, to avoid copying bogus code around?
> > Or does 35/37 depend on 34/37 to be correct?
> >
> > > +}
> > > +
> > > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > > +{
> > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > + int i;
> > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > +
> > > + for (i = 0; i < used_lrs; i++) {
> > > + if (cpu_if->vgic_elrsr & (1UL << i))
>
> So, the vgic_elrsr is naturally bitmap, and bitmap API is preferred if no
> other considerations:
> if (test_bit(i, cpu_if->vgic_elrsr))
>
> > > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > > + else
> > > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > > +
> > > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > > + }
> > > +}
>
> I'd also headscratch about using for_each_clear_bit() here:
>
> /*
> * Setup default vgic_lr values somewhere earlier.
Not sure what the 'default' values are.
> * Not needed at all if you take my suggestion for
> * vgic_v2_restore_state() below
> */
> for (i = 0; i < used_lrs; i++)
> cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
>
> static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> {
> [...]
>
> for_each_clear_bit (i, cpu_if->vgic_elrsr, used_lrs)
> cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
>
> for (i = 0; i < used_lrs; i++)
> writel_relaxed(0, base + GICH_LR0 + (i * 4));
> }
>
> Not sure how performance-critical this path is, but sometimes things
> get really faster with bitmaps.
>
Your suggestion below would require us to maintain elrsr when we setup
list registers, and I don't really see the benefit.
> [...]
>
> > > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > > +{
> > > + struct kvm *kvm = vcpu->kvm;
> > > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > + void __iomem *base = vgic->vctrl_base;
> > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > + int i;
> > > +
> > > + if (!base)
> > > + return;
> > > +
> > > + if (used_lrs) {
> > > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > > + for (i = 0; i < used_lrs; i++) {
> > > + writel_relaxed(cpu_if->vgic_lr[i],
> > > + base + GICH_LR0 + (i * 4));
> > > + }
> > > + }
> > > +}
>
> The alternative approach would be:
> for (i = 0; i < used_lrs; i++) {
> if (test_bit(i, cpu_if->vgic_elrsr))
> writel_relaxed(~GICH_LR_STATE, base + GICH_LR0 + (i * 4));
> else
> writel_relaxed(cpu_if->vgic_lr[i], base + GICH_LR0 + (i * 4));
> }
>
> If cpu_if->vgic_elrsr is untouched in-between of course. It will make
> save_lrs() simpler and this function more verbose.
>
I don't understand your suggestion. As you will see later, we will get
rid of storing the elrsr completely with a measureable performance
improvement.
If you think you can improve the code beyond that, a follow-up patch
would be most welcome.
Note that on all the implementations I'm familiar with, the maximum
number of LRs is four, so we're not wading through massive bitmaps in
practice here.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
2017-11-26 19:46 ` Christoffer Dall
@ 2017-11-30 12:09 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-30 12:09 UTC (permalink / raw)
To: Christoffer Dall
Cc: Andre Przywara, Christoffer Dall, kvmarm, linux-arm-kernel,
Marc Zyngier, kvm, Shih-Wei Li
On Sun, Nov 26, 2017 at 08:46:41PM +0100, Christoffer Dall wrote:
> On Sun, Nov 26, 2017 at 01:29:30PM +0300, Yury Norov wrote:
> > On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> > > Hi,
> > >
> > > those last few patches are actually helpful for the Xen port ...
> >
> > [...]
> >
> > > > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > > > +{
> > > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > > > + u32 elrsr0, elrsr1;
> > > > +
> > > > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > > > + if (unlikely(nr_lr > 32))
> > > > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > > > + else
> > > > + elrsr1 = 0;
> > > > +
> > > > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > > > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > > > +#else
> > > > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > > > +#endif
> > >
> > > I have some gut feeling that this is really broken, since we mix up
> > > endian *byte* ordering with *bit* ordering here, don't we?
> >
> > Good feeling indeed. :)
> >
> > We have bitmap_{from,to)_u32array for things like this. But it was
> > considered bad-designed, and I proposed new bitmap_{from,to)_arr32().
> >
> > https://lkml.org/lkml/2017/11/15/592
> >
> > What else I have in mind, to introduce something like bitmap_{from,to}_pair_32()
> > as most of current users of bitmap_{from,to)_u32array(), (and those who should
> > use it but don't, like this one) have only 2 32-bit halfwords to be copied
> > from/to bitmap.
> >
> > Also, it will be complementary to bitmap_from_u64().
> >
> > More reading about bitmap/array conversion is in comment to BITMAP_FROM_U64
> > macro.
> >
>
> I have no idea what you want to introduce here. If you have an idea on
> how to improve the code, patches are welcome.
That's about Andre's gut feeling, not about your patch. I have some
ideas related to it, and just want to share it to him - that's all.
> Please keep in mind, that the purpose of this patch is to move code
> around to improve the GIC handling performance, not changing the
> lower-level details of the code.
>
> > > I understand it's just copied and gets removed later on, so I was
> > > wondering if you could actually move patch 35/37 ("Get rid of
> > > vgic_elrsr") before this patch here, to avoid copying bogus code around?
> > > Or does 35/37 depend on 34/37 to be correct?
> > >
> > > > +}
> > > > +
> > > > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > > > +{
> > > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > > + int i;
> > > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > > +
> > > > + for (i = 0; i < used_lrs; i++) {
> > > > + if (cpu_if->vgic_elrsr & (1UL << i))
> >
> > So, the vgic_elrsr is naturally bitmap, and bitmap API is preferred if no
> > other considerations:
> > if (test_bit(i, cpu_if->vgic_elrsr))
> >
> > > > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > > > + else
> > > > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > > > +
> > > > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > > > + }
> > > > +}
> >
> > I'd also headscratch about using for_each_clear_bit() here:
> >
> > /*
> > * Setup default vgic_lr values somewhere earlier.
>
> Not sure what the 'default' values are.
>
> > * Not needed at all if you take my suggestion for
> > * vgic_v2_restore_state() below
> > */
> > for (i = 0; i < used_lrs; i++)
> > cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> >
> > static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > {
> > [...]
> >
> > for_each_clear_bit (i, cpu_if->vgic_elrsr, used_lrs)
> > cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> >
> > for (i = 0; i < used_lrs; i++)
> > writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > }
> >
> > Not sure how performance-critical this path is, but sometimes things
> > get really faster with bitmaps.
> >
>
> Your suggestion below would require us to maintain elrsr when we setup
> list registers, and I don't really see the benefit.
That's what I asked - is it maintained or not. If not then it will not
work.
> > [...]
> >
> > > > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > > > +{
> > > > + struct kvm *kvm = vcpu->kvm;
> > > > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > > + void __iomem *base = vgic->vctrl_base;
> > > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > > + int i;
> > > > +
> > > > + if (!base)
> > > > + return;
> > > > +
> > > > + if (used_lrs) {
> > > > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > > > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > > > + for (i = 0; i < used_lrs; i++) {
> > > > + writel_relaxed(cpu_if->vgic_lr[i],
> > > > + base + GICH_LR0 + (i * 4));
> > > > + }
> > > > + }
> > > > +}
> >
> > The alternative approach would be:
> > for (i = 0; i < used_lrs; i++) {
> > if (test_bit(i, cpu_if->vgic_elrsr))
> > writel_relaxed(~GICH_LR_STATE, base + GICH_LR0 + (i * 4));
> > else
> > writel_relaxed(cpu_if->vgic_lr[i], base + GICH_LR0 + (i * 4));
> > }
> >
> > If cpu_if->vgic_elrsr is untouched in-between of course. It will make
> > save_lrs() simpler and this function more verbose.
> >
> I don't understand your suggestion. As you will see later, we will get
> rid of storing the elrsr completely with a measureable performance
> improvement.
OK, now I see. Sorry for stupid questions - I just start learning
codebase. By the way, can you share the technique that you use to
measure performance? It would be great if I can reproduce your
results.
> If you think you can improve the code beyond that, a follow-up patch
> would be most welcome.
>
> Note that on all the implementations I'm familiar with, the maximum
> number of LRs is four, so we're not wading through massive bitmaps in
> practice here.
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
@ 2017-11-30 12:09 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-30 12:09 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 08:46:41PM +0100, Christoffer Dall wrote:
> On Sun, Nov 26, 2017 at 01:29:30PM +0300, Yury Norov wrote:
> > On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> > > Hi,
> > >
> > > those last few patches are actually helpful for the Xen port ...
> >
> > [...]
> >
> > > > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > > > +{
> > > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > > > + u32 elrsr0, elrsr1;
> > > > +
> > > > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > > > + if (unlikely(nr_lr > 32))
> > > > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > > > + else
> > > > + elrsr1 = 0;
> > > > +
> > > > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > > > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > > > +#else
> > > > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > > > +#endif
> > >
> > > I have some gut feeling that this is really broken, since we mix up
> > > endian *byte* ordering with *bit* ordering here, don't we?
> >
> > Good feeling indeed. :)
> >
> > We have bitmap_{from,to)_u32array for things like this. But it was
> > considered bad-designed, and I proposed new bitmap_{from,to)_arr32().
> >
> > https://lkml.org/lkml/2017/11/15/592
> >
> > What else I have in mind, to introduce something like bitmap_{from,to}_pair_32()
> > as most of current users of bitmap_{from,to)_u32array(), (and those who should
> > use it but don't, like this one) have only 2 32-bit halfwords to be copied
> > from/to bitmap.
> >
> > Also, it will be complementary to bitmap_from_u64().
> >
> > More reading about bitmap/array conversion is in comment to BITMAP_FROM_U64
> > macro.
> >
>
> I have no idea what you want to introduce here. If you have an idea on
> how to improve the code, patches are welcome.
That's about Andre's gut feeling, not about your patch. I have some
ideas related to it, and just want to share it to him - that's all.
> Please keep in mind, that the purpose of this patch is to move code
> around to improve the GIC handling performance, not changing the
> lower-level details of the code.
>
> > > I understand it's just copied and gets removed later on, so I was
> > > wondering if you could actually move patch 35/37 ("Get rid of
> > > vgic_elrsr") before this patch here, to avoid copying bogus code around?
> > > Or does 35/37 depend on 34/37 to be correct?
> > >
> > > > +}
> > > > +
> > > > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > > > +{
> > > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > > + int i;
> > > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > > +
> > > > + for (i = 0; i < used_lrs; i++) {
> > > > + if (cpu_if->vgic_elrsr & (1UL << i))
> >
> > So, the vgic_elrsr is naturally bitmap, and bitmap API is preferred if no
> > other considerations:
> > if (test_bit(i, cpu_if->vgic_elrsr))
> >
> > > > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > > > + else
> > > > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > > > +
> > > > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > > > + }
> > > > +}
> >
> > I'd also headscratch about using for_each_clear_bit() here:
> >
> > /*
> > * Setup default vgic_lr values somewhere earlier.
>
> Not sure what the 'default' values are.
>
> > * Not needed at all if you take my suggestion for
> > * vgic_v2_restore_state() below
> > */
> > for (i = 0; i < used_lrs; i++)
> > cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> >
> > static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > {
> > [...]
> >
> > for_each_clear_bit (i, cpu_if->vgic_elrsr, used_lrs)
> > cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> >
> > for (i = 0; i < used_lrs; i++)
> > writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > }
> >
> > Not sure how performance-critical this path is, but sometimes things
> > get really faster with bitmaps.
> >
>
> Your suggestion below would require us to maintain elrsr when we setup
> list registers, and I don't really see the benefit.
That's what I asked - is it maintained or not. If not then it will not
work.
> > [...]
> >
> > > > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > > > +{
> > > > + struct kvm *kvm = vcpu->kvm;
> > > > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > > > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > > > + void __iomem *base = vgic->vctrl_base;
> > > > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > > > + int i;
> > > > +
> > > > + if (!base)
> > > > + return;
> > > > +
> > > > + if (used_lrs) {
> > > > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > > > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > > > + for (i = 0; i < used_lrs; i++) {
> > > > + writel_relaxed(cpu_if->vgic_lr[i],
> > > > + base + GICH_LR0 + (i * 4));
> > > > + }
> > > > + }
> > > > +}
> >
> > The alternative approach would be:
> > for (i = 0; i < used_lrs; i++) {
> > if (test_bit(i, cpu_if->vgic_elrsr))
> > writel_relaxed(~GICH_LR_STATE, base + GICH_LR0 + (i * 4));
> > else
> > writel_relaxed(cpu_if->vgic_lr[i], base + GICH_LR0 + (i * 4));
> > }
> >
> > If cpu_if->vgic_elrsr is untouched in-between of course. It will make
> > save_lrs() simpler and this function more verbose.
> >
> I don't understand your suggestion. As you will see later, we will get
> rid of storing the elrsr completely with a measureable performance
> improvement.
OK, now I see. Sorry for stupid questions - I just start learning
codebase. By the way, can you share the technique that you use to
measure performance? It would be great if I can reproduce your
results.
> If you think you can improve the code beyond that, a follow-up patch
> would be most welcome.
>
> Note that on all the implementations I'm familiar with, the maximum
> number of LRs is four, so we're not wading through massive bitmaps in
> practice here.
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* Re: [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
2017-11-15 17:50 ` Andre Przywara
@ 2017-11-26 19:37 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:37 UTC (permalink / raw)
To: Andre Przywara
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, Marc Zyngier,
Shih-Wei Li, kvm
On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> Hi,
>
> those last few patches are actually helpful for the Xen port ...
cool!
>
> On 12/10/17 11:41, Christoffer Dall wrote:
> > We can program the GICv2 hypervisor control interface logic directly
> > from the core vgic code and can instead do the save/restore directly
> > from the flush/sync functions, which can lead to a number of future
> > optimizations.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/kvm/hyp/switch.c | 4 --
> > arch/arm64/include/asm/kvm_hyp.h | 2 -
> > arch/arm64/kvm/hyp/switch.c | 4 --
> > virt/kvm/arm/hyp/vgic-v2-sr.c | 83 ------------------------------------
> > virt/kvm/arm/vgic/vgic-init.c | 22 ++++++----
> > virt/kvm/arm/vgic/vgic-v2.c | 92 ++++++++++++++++++++++++++++++++++++++++
> > virt/kvm/arm/vgic/vgic.c | 21 ++++++++-
> > virt/kvm/arm/vgic/vgic.h | 5 +++
> > 8 files changed, 130 insertions(+), 103 deletions(-)
> >
> > diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> > index c3b9799..0d834f8 100644
> > --- a/arch/arm/kvm/hyp/switch.c
> > +++ b/arch/arm/kvm/hyp/switch.c
> > @@ -91,16 +91,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_save_state(vcpu);
> > - else
> > - __vgic_v2_save_state(vcpu);
> > }
> >
> > static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_restore_state(vcpu);
> > - else
> > - __vgic_v2_restore_state(vcpu);
> > }
> >
> > static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 28d5f3c..bd3fe64 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -121,8 +121,6 @@ typeof(orig) * __hyp_text fname(void) \
> > return val; \
> > }
> >
> > -void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> > -void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> > int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
> >
> > void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 5692aa0..90da506 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -186,16 +186,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_save_state(vcpu);
> > - else
> > - __vgic_v2_save_state(vcpu);
> > }
> >
> > static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_restore_state(vcpu);
> > - else
> > - __vgic_v2_restore_state(vcpu);
> > }
> >
> > static bool __hyp_text __true_value(void)
> > diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/virt/kvm/arm/hyp/vgic-v2-sr.c
> > index a3f18d3..b433257 100644
> > --- a/virt/kvm/arm/hyp/vgic-v2-sr.c
> > +++ b/virt/kvm/arm/hyp/vgic-v2-sr.c
> > @@ -22,89 +22,6 @@
> > #include <asm/kvm_emulate.h>
> > #include <asm/kvm_hyp.h>
> >
> > -static void __hyp_text save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > -{
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int nr_lr = (kern_hyp_va(&kvm_vgic_global_state))->nr_lr;
> > - u32 elrsr0, elrsr1;
> > -
> > - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > - if (unlikely(nr_lr > 32))
> > - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > - else
> > - elrsr1 = 0;
> > -
> > -#ifdef CONFIG_CPU_BIG_ENDIAN
> > - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > -#else
> > - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > -#endif
> > -}
> > -
> > -static void __hyp_text save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > -{
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int i;
> > - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > -
> > - for (i = 0; i < used_lrs; i++) {
> > - if (cpu_if->vgic_elrsr & (1UL << i))
> > - cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > - else
> > - cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > -
> > - writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > - }
> > -}
> > -
> > -/* vcpu is already in the HYP VA space */
> > -void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
> > -{
> > - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - struct vgic_dist *vgic = &kvm->arch.vgic;
> > - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> > - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > -
> > - if (!base)
> > - return;
> > -
> > - if (used_lrs) {
> > - cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> > -
> > - save_elrsr(vcpu, base);
> > - save_lrs(vcpu, base);
> > -
> > - writel_relaxed(0, base + GICH_HCR);
> > - } else {
> > - cpu_if->vgic_elrsr = ~0UL;
> > - cpu_if->vgic_apr = 0;
> > - }
> > -}
> > -
> > -/* vcpu is already in the HYP VA space */
> > -void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > -{
> > - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - struct vgic_dist *vgic = &kvm->arch.vgic;
> > - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> > - int i;
> > - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > -
> > - if (!base)
> > - return;
> > -
> > - if (used_lrs) {
> > - writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > - writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > - for (i = 0; i < used_lrs; i++) {
> > - writel_relaxed(cpu_if->vgic_lr[i],
> > - base + GICH_LR0 + (i * 4));
> > - }
> > - }
> > -}
> > -
> > #ifdef CONFIG_ARM64
> > /*
> > * __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
> > diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> > index 5801261f..fa2b565 100644
> > --- a/virt/kvm/arm/vgic/vgic-init.c
> > +++ b/virt/kvm/arm/vgic/vgic-init.c
> > @@ -425,14 +425,16 @@ static int vgic_init_cpu_dying(unsigned int cpu)
> > return 0;
> > }
> >
> > -static irqreturn_t vgic_maintenance_handler(int irq, void *data)
> > +static irqreturn_t vgic_v3_maintenance_handler(int irq, void *data)
> > {
> > - /*
> > - * We cannot rely on the vgic maintenance interrupt to be
> > - * delivered synchronously. This means we can only use it to
> > - * exit the VM, and we perform the handling of EOIed
> > - * interrupts on the exit path (see vgic_process_maintenance).
> > - */
>
> I always found this comment quite enlightening, especially as it points
> out that we need to deviate somewhat from the architectural idea here.
> I see that you have shortened it below. Is it no longer true? Can we
> keep the more elaborate version? If not here, then below?
>
I think I found it a bit misleading or harder to understand now that we
may actually do some work in the maintenance handler (disable the vgic).
But if you think it's better to keep it, I'm happy to make that change
for v2.
> > + BUG(); /* Not implemented lazy save/restore on GICv3 */
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static irqreturn_t vgic_v2_maintenance_handler(int irq, void *data)
> > +{
> > + struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data;
>
> I think you need an empty line here, to separate variable declaration
> from code.
>
sure
> > + vgic_v2_handle_maintenance(vcpu);
>
> > return IRQ_HANDLED;
> > }
> >
> > @@ -464,6 +466,7 @@ void kvm_vgic_init_cpu_hardware(void)
> > int kvm_vgic_hyp_init(void)
> > {
> > const struct gic_kvm_info *gic_kvm_info;
> > + irqreturn_t (*handler)(int irq, void *data);
> > int ret;
> >
> > gic_kvm_info = gic_get_kvm_info();
> > @@ -478,6 +481,7 @@ int kvm_vgic_hyp_init(void)
> > switch (gic_kvm_info->type) {
> > case GIC_V2:
> > ret = vgic_v2_probe(gic_kvm_info);
> > + handler = vgic_v2_maintenance_handler;
> > break;
> 1> case GIC_V3:
> > ret = vgic_v3_probe(gic_kvm_info);
> > @@ -485,6 +489,7 @@ int kvm_vgic_hyp_init(void)
> > static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
> > kvm_info("GIC system register CPU interface enabled\n");
> > }
> > + handler = vgic_v3_maintenance_handler;
> > break;
> > default:
> > ret = -ENODEV;
> > @@ -494,8 +499,7 @@ int kvm_vgic_hyp_init(void)
> > return ret;
> >
> > kvm_vgic_global_state.maint_irq = gic_kvm_info->maint_irq;
> > - ret = request_percpu_irq(kvm_vgic_global_state.maint_irq,
> > - vgic_maintenance_handler,
> > + ret = request_percpu_irq(kvm_vgic_global_state.maint_irq, handler,
> > "vgic", kvm_get_running_vcpus());
> > if (ret) {
> > kvm_err("Cannot register interrupt %d\n",
> > diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> > index 8089710..259079b 100644
> > --- a/virt/kvm/arm/vgic/vgic-v2.c
> > +++ b/virt/kvm/arm/vgic/vgic-v2.c
> > @@ -37,6 +37,17 @@ void vgic_v2_init_lrs(void)
> > vgic_v2_write_lr(i, 0);
> > }
> >
> > +void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu)
> > +{
> > + void __iomem *base = kvm_vgic_global_state.vctrl_base;
> > +
> > + /*
> > + * Disable maintenance interrupt as we only use it to generate an exit
> > + * from the VM.
> > + */
>
> Isn't that comment a bit misleading, as writing 0 to HCR not only
> disables all interrupt sources, but also the whole GICV interface
> altogether (bit 0: EN)?
> I see that it gets enabled later on when writing ->vgic_hcr into the
> register, but this function here looks a bit surprising to me.
I don't think the comment is misleading and I'm not trying to explain
how the whole hardware works, but the intention behind this code.
I'll try to expand this comment and the commit message.
The point is that we want to get back out of the hyp code (or just back
to the vgic sync function on VHE) so that we in the case of no
interrupts in flight, only check a single variable, and then in the case
of having interrupts in flight, manage the hardware then. That means
that we leave the VGIC on, which in turn means that we may now actually
see the maintenance interrupt firing, and in that case, we want to
disable the VGIC to prevent continuous maintenance interrupts from
firing, but we still only handle the actual maintenance (resample the
line) in the sync function.
I'll try to find some nice way of explaining this.
>
> In general these changes to the interrupt handling leave me a bit
> puzzled. Should this be a separate patch? Or explained in the commit
> message?
>
Should definitely be explained. I tried splitting the logic, but
couldn't find a nice working way to do that. I'll have another look.
> > + writel_relaxed(0, base + GICH_HCR);
> > +}
> > +
> > void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
> > @@ -393,6 +404,87 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> > return ret;
> > }
> >
> > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > + u32 elrsr0, elrsr1;
> > +
> > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > + if (unlikely(nr_lr > 32))
> > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > + else
> > + elrsr1 = 0;
> > +
> > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > +#else
> > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > +#endif
>
> I have some gut feeling that this is really broken, since we mix up
> endian *byte* ordering with *bit* ordering here, don't we?
Yes, it's broken, as I also noted in the commit message later.
>
> I understand it's just copied and gets removed later on, so I was
> wondering if you could actually move patch 35/37 ("Get rid of
> vgic_elrsr") before this patch here, to avoid copying bogus code around?
> Or does 35/37 depend on 34/37 to be correct?
>
I don't remember. I'll give it a try.
> > +}
> > +
> > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int i;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > +
> > + for (i = 0; i < used_lrs; i++) {
> > + if (cpu_if->vgic_elrsr & (1UL << i))
> > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > + else
> > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > +
> > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > + }
> > +}
> > +
> > +void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + void __iomem *base = vgic->vctrl_base;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > +
> > + if (!base)
> > + return;
> > +
> > + if (used_lrs) {
> > + cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> > +
> > + save_elrsr(vcpu, base);
> > + save_lrs(vcpu, base);
> > +
> > + writel_relaxed(0, base + GICH_HCR);
> > + } else {
> > + cpu_if->vgic_elrsr = ~0UL;
> > + cpu_if->vgic_apr = 0;
> > + }
> > +}
> > +
> > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + void __iomem *base = vgic->vctrl_base;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > + int i;
> > +
> > + if (!base)
> > + return;
> > +
> > + if (used_lrs) {
> > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > + for (i = 0; i < used_lrs; i++) {
> > + writel_relaxed(cpu_if->vgic_lr[i],
> > + base + GICH_LR0 + (i * 4));
> > + }
> > + }
> > +}
> > +
> > void vgic_v2_load(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> > index b1bd238..790fd66 100644
> > --- a/virt/kvm/arm/vgic/vgic.c
> > +++ b/virt/kvm/arm/vgic/vgic.c
> > @@ -18,6 +18,8 @@
> > #include <linux/kvm_host.h>
> > #include <linux/list_sort.h>
> >
> > +#include <asm/kvm_hyp.h>
>
> Why do you need that? Commenting this out seems to compile anyway for me.
>
Probably while I was rearranging patches this got in there somehow.
> > +
> > #include "vgic.h"
> >
> > #define CREATE_TRACE_POINTS
> > @@ -683,11 +685,19 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
> > vgic_clear_lr(vcpu, count);
> > }
> >
> > +static inline void vgic_save_state(struct kvm_vcpu *vcpu)
>
> Isn't "inline" frowned upon in .c files?
>
Opinions seem to vary on this one. I actually observed this making a
difference in the compilation for some of these smaller functions with
my compiler so I included it, and there are certainly examples of this
in the kernel.
If there's a policy I missed or someone has a strong argument that I
have to remove the inline word here, then let me know.
> > +{
> > + if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > + vgic_v2_save_state(vcpu);
> > +}
> > +
> > /* Sync back the hardware VGIC state into our emulation after a guest's run. */
> > void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> >
> > + vgic_save_state(vcpu);
> > +
> > /* An empty ap_list_head implies used_lrs == 0 */
> > if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
> > return;
> > @@ -697,6 +707,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> > vgic_prune_ap_list(vcpu);
> > }
> >
> > +static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
>
> Same here.
>
> Apart from those rather cosmetic issues I have at least verified that
> the code is actually moved from vgic-v2-sr.c to vgic-v2.c, plus/minus
> the required changes when moving this from HYP to vGIC/EL1.
>
Thanks for having a look!
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 32/37] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code
@ 2017-11-26 19:37 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:37 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 15, 2017 at 05:50:07PM +0000, Andre Przywara wrote:
> Hi,
>
> those last few patches are actually helpful for the Xen port ...
cool!
>
> On 12/10/17 11:41, Christoffer Dall wrote:
> > We can program the GICv2 hypervisor control interface logic directly
> > from the core vgic code and can instead do the save/restore directly
> > from the flush/sync functions, which can lead to a number of future
> > optimizations.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/kvm/hyp/switch.c | 4 --
> > arch/arm64/include/asm/kvm_hyp.h | 2 -
> > arch/arm64/kvm/hyp/switch.c | 4 --
> > virt/kvm/arm/hyp/vgic-v2-sr.c | 83 ------------------------------------
> > virt/kvm/arm/vgic/vgic-init.c | 22 ++++++----
> > virt/kvm/arm/vgic/vgic-v2.c | 92 ++++++++++++++++++++++++++++++++++++++++
> > virt/kvm/arm/vgic/vgic.c | 21 ++++++++-
> > virt/kvm/arm/vgic/vgic.h | 5 +++
> > 8 files changed, 130 insertions(+), 103 deletions(-)
> >
> > diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
> > index c3b9799..0d834f8 100644
> > --- a/arch/arm/kvm/hyp/switch.c
> > +++ b/arch/arm/kvm/hyp/switch.c
> > @@ -91,16 +91,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_save_state(vcpu);
> > - else
> > - __vgic_v2_save_state(vcpu);
> > }
> >
> > static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_restore_state(vcpu);
> > - else
> > - __vgic_v2_restore_state(vcpu);
> > }
> >
> > static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 28d5f3c..bd3fe64 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -121,8 +121,6 @@ typeof(orig) * __hyp_text fname(void) \
> > return val; \
> > }
> >
> > -void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> > -void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> > int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
> >
> > void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 5692aa0..90da506 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -186,16 +186,12 @@ static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_save_state(vcpu);
> > - else
> > - __vgic_v2_save_state(vcpu);
> > }
> >
> > static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> > {
> > if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > __vgic_v3_restore_state(vcpu);
> > - else
> > - __vgic_v2_restore_state(vcpu);
> > }
> >
> > static bool __hyp_text __true_value(void)
> > diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/virt/kvm/arm/hyp/vgic-v2-sr.c
> > index a3f18d3..b433257 100644
> > --- a/virt/kvm/arm/hyp/vgic-v2-sr.c
> > +++ b/virt/kvm/arm/hyp/vgic-v2-sr.c
> > @@ -22,89 +22,6 @@
> > #include <asm/kvm_emulate.h>
> > #include <asm/kvm_hyp.h>
> >
> > -static void __hyp_text save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > -{
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int nr_lr = (kern_hyp_va(&kvm_vgic_global_state))->nr_lr;
> > - u32 elrsr0, elrsr1;
> > -
> > - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > - if (unlikely(nr_lr > 32))
> > - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > - else
> > - elrsr1 = 0;
> > -
> > -#ifdef CONFIG_CPU_BIG_ENDIAN
> > - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > -#else
> > - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > -#endif
> > -}
> > -
> > -static void __hyp_text save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > -{
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int i;
> > - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > -
> > - for (i = 0; i < used_lrs; i++) {
> > - if (cpu_if->vgic_elrsr & (1UL << i))
> > - cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > - else
> > - cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > -
> > - writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > - }
> > -}
> > -
> > -/* vcpu is already in the HYP VA space */
> > -void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
> > -{
> > - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - struct vgic_dist *vgic = &kvm->arch.vgic;
> > - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> > - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > -
> > - if (!base)
> > - return;
> > -
> > - if (used_lrs) {
> > - cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> > -
> > - save_elrsr(vcpu, base);
> > - save_lrs(vcpu, base);
> > -
> > - writel_relaxed(0, base + GICH_HCR);
> > - } else {
> > - cpu_if->vgic_elrsr = ~0UL;
> > - cpu_if->vgic_apr = 0;
> > - }
> > -}
> > -
> > -/* vcpu is already in the HYP VA space */
> > -void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > -{
> > - struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - struct vgic_dist *vgic = &kvm->arch.vgic;
> > - void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> > - int i;
> > - u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > -
> > - if (!base)
> > - return;
> > -
> > - if (used_lrs) {
> > - writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > - writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > - for (i = 0; i < used_lrs; i++) {
> > - writel_relaxed(cpu_if->vgic_lr[i],
> > - base + GICH_LR0 + (i * 4));
> > - }
> > - }
> > -}
> > -
> > #ifdef CONFIG_ARM64
> > /*
> > * __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
> > diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
> > index 5801261f..fa2b565 100644
> > --- a/virt/kvm/arm/vgic/vgic-init.c
> > +++ b/virt/kvm/arm/vgic/vgic-init.c
> > @@ -425,14 +425,16 @@ static int vgic_init_cpu_dying(unsigned int cpu)
> > return 0;
> > }
> >
> > -static irqreturn_t vgic_maintenance_handler(int irq, void *data)
> > +static irqreturn_t vgic_v3_maintenance_handler(int irq, void *data)
> > {
> > - /*
> > - * We cannot rely on the vgic maintenance interrupt to be
> > - * delivered synchronously. This means we can only use it to
> > - * exit the VM, and we perform the handling of EOIed
> > - * interrupts on the exit path (see vgic_process_maintenance).
> > - */
>
> I always found this comment quite enlightening, especially as it points
> out that we need to deviate somewhat from the architectural idea here.
> I see that you have shortened it below. Is it no longer true? Can we
> keep the more elaborate version? If not here, then below?
>
I think I found it a bit misleading or harder to understand now that we
may actually do some work in the maintenance handler (disable the vgic).
But if you think it's better to keep it, I'm happy to make that change
for v2.
> > + BUG(); /* Not implemented lazy save/restore on GICv3 */
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static irqreturn_t vgic_v2_maintenance_handler(int irq, void *data)
> > +{
> > + struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data;
>
> I think you need an empty line here, to separate variable declaration
> from code.
>
sure
> > + vgic_v2_handle_maintenance(vcpu);
>
> > return IRQ_HANDLED;
> > }
> >
> > @@ -464,6 +466,7 @@ void kvm_vgic_init_cpu_hardware(void)
> > int kvm_vgic_hyp_init(void)
> > {
> > const struct gic_kvm_info *gic_kvm_info;
> > + irqreturn_t (*handler)(int irq, void *data);
> > int ret;
> >
> > gic_kvm_info = gic_get_kvm_info();
> > @@ -478,6 +481,7 @@ int kvm_vgic_hyp_init(void)
> > switch (gic_kvm_info->type) {
> > case GIC_V2:
> > ret = vgic_v2_probe(gic_kvm_info);
> > + handler = vgic_v2_maintenance_handler;
> > break;
> 1> case GIC_V3:
> > ret = vgic_v3_probe(gic_kvm_info);
> > @@ -485,6 +489,7 @@ int kvm_vgic_hyp_init(void)
> > static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
> > kvm_info("GIC system register CPU interface enabled\n");
> > }
> > + handler = vgic_v3_maintenance_handler;
> > break;
> > default:
> > ret = -ENODEV;
> > @@ -494,8 +499,7 @@ int kvm_vgic_hyp_init(void)
> > return ret;
> >
> > kvm_vgic_global_state.maint_irq = gic_kvm_info->maint_irq;
> > - ret = request_percpu_irq(kvm_vgic_global_state.maint_irq,
> > - vgic_maintenance_handler,
> > + ret = request_percpu_irq(kvm_vgic_global_state.maint_irq, handler,
> > "vgic", kvm_get_running_vcpus());
> > if (ret) {
> > kvm_err("Cannot register interrupt %d\n",
> > diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> > index 8089710..259079b 100644
> > --- a/virt/kvm/arm/vgic/vgic-v2.c
> > +++ b/virt/kvm/arm/vgic/vgic-v2.c
> > @@ -37,6 +37,17 @@ void vgic_v2_init_lrs(void)
> > vgic_v2_write_lr(i, 0);
> > }
> >
> > +void vgic_v2_handle_maintenance(struct kvm_vcpu *vcpu)
> > +{
> > + void __iomem *base = kvm_vgic_global_state.vctrl_base;
> > +
> > + /*
> > + * Disable maintenance interrupt as we only use it to generate an exit
> > + * from the VM.
> > + */
>
> Isn't that comment a bit misleading, as writing 0 to HCR not only
> disables all interrupt sources, but also the whole GICV interface
> altogether (bit 0: EN)?
> I see that it gets enabled later on when writing ->vgic_hcr into the
> register, but this function here looks a bit surprising to me.
I don't think the comment is misleading and I'm not trying to explain
how the whole hardware works, but the intention behind this code.
I'll try to expand this comment and the commit message.
The point is that we want to get back out of the hyp code (or just back
to the vgic sync function on VHE) so that we in the case of no
interrupts in flight, only check a single variable, and then in the case
of having interrupts in flight, manage the hardware then. That means
that we leave the VGIC on, which in turn means that we may now actually
see the maintenance interrupt firing, and in that case, we want to
disable the VGIC to prevent continuous maintenance interrupts from
firing, but we still only handle the actual maintenance (resample the
line) in the sync function.
I'll try to find some nice way of explaining this.
>
> In general these changes to the interrupt handling leave me a bit
> puzzled. Should this be a separate patch? Or explained in the commit
> message?
>
Should definitely be explained. I tried splitting the logic, but
couldn't find a nice working way to do that. I'll have another look.
> > + writel_relaxed(0, base + GICH_HCR);
> > +}
> > +
> > void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
> > @@ -393,6 +404,87 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> > return ret;
> > }
> >
> > +static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int nr_lr = kvm_vgic_global_state.nr_lr;
> > + u32 elrsr0, elrsr1;
> > +
> > + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > + if (unlikely(nr_lr > 32))
> > + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > + else
> > + elrsr1 = 0;
> > +
> > +#ifdef CONFIG_CPU_BIG_ENDIAN
> > + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > +#else
> > + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > +#endif
>
> I have some gut feeling that this is really broken, since we mix up
> endian *byte* ordering with *bit* ordering here, don't we?
Yes, it's broken, as I also noted in the commit message later.
>
> I understand it's just copied and gets removed later on, so I was
> wondering if you could actually move patch 35/37 ("Get rid of
> vgic_elrsr") before this patch here, to avoid copying bogus code around?
> Or does 35/37 depend on 34/37 to be correct?
>
I don't remember. I'll give it a try.
> > +}
> > +
> > +static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > +{
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + int i;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > +
> > + for (i = 0; i < used_lrs; i++) {
> > + if (cpu_if->vgic_elrsr & (1UL << i))
> > + cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > + else
> > + cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > +
> > + writel_relaxed(0, base + GICH_LR0 + (i * 4));
> > + }
> > +}
> > +
> > +void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + void __iomem *base = vgic->vctrl_base;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > +
> > + if (!base)
> > + return;
> > +
> > + if (used_lrs) {
> > + cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> > +
> > + save_elrsr(vcpu, base);
> > + save_lrs(vcpu, base);
> > +
> > + writel_relaxed(0, base + GICH_HCR);
> > + } else {
> > + cpu_if->vgic_elrsr = ~0UL;
> > + cpu_if->vgic_apr = 0;
> > + }
> > +}
> > +
> > +void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> > +{
> > + struct kvm *kvm = vcpu->kvm;
> > + struct vgic_dist *vgic = &kvm->arch.vgic;
> > + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > + void __iomem *base = vgic->vctrl_base;
> > + u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > + int i;
> > +
> > + if (!base)
> > + return;
> > +
> > + if (used_lrs) {
> > + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> > + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> > + for (i = 0; i < used_lrs; i++) {
> > + writel_relaxed(cpu_if->vgic_lr[i],
> > + base + GICH_LR0 + (i * 4));
> > + }
> > + }
> > +}
> > +
> > void vgic_v2_load(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
> > index b1bd238..790fd66 100644
> > --- a/virt/kvm/arm/vgic/vgic.c
> > +++ b/virt/kvm/arm/vgic/vgic.c
> > @@ -18,6 +18,8 @@
> > #include <linux/kvm_host.h>
> > #include <linux/list_sort.h>
> >
> > +#include <asm/kvm_hyp.h>
>
> Why do you need that? Commenting this out seems to compile anyway for me.
>
Probably while I was rearranging patches this got in there somehow.
> > +
> > #include "vgic.h"
> >
> > #define CREATE_TRACE_POINTS
> > @@ -683,11 +685,19 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
> > vgic_clear_lr(vcpu, count);
> > }
> >
> > +static inline void vgic_save_state(struct kvm_vcpu *vcpu)
>
> Isn't "inline" frowned upon in .c files?
>
Opinions seem to vary on this one. I actually observed this making a
difference in the compilation for some of these smaller functions with
my compiler so I included it, and there are certainly examples of this
in the kernel.
If there's a policy I missed or someone has a strong argument that I
have to remove the inline word here, then let me know.
> > +{
> > + if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
> > + vgic_v2_save_state(vcpu);
> > +}
> > +
> > /* Sync back the hardware VGIC state into our emulation after a guest's run. */
> > void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> >
> > + vgic_save_state(vcpu);
> > +
> > /* An empty ap_list_head implies used_lrs == 0 */
> > if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
> > return;
> > @@ -697,6 +707,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
> > vgic_prune_ap_list(vcpu);
> > }
> >
> > +static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
>
> Same here.
>
> Apart from those rather cosmetic issues I have at least verified that
> the code is actually moved from vgic-v2-sr.c to vgic-v2.c, plus/minus
> the required changes when moving this from HYP to vGIC/EL1.
>
Thanks for having a look!
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 33/37] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
The vgic-v2-sr.c file now only contains the logic to replay unaligned
accesses to the virtual CPU interface on 16K and 64K page systems, which
is only relevant on 64-bit platforms. Therefore move this file to the
arm64 KVM tree, remove the compile directive from the 32-bit side
makefile, and remove the ifdef in the C file.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/kvm/hyp/Makefile | 1 -
arch/arm64/kvm/hyp/Makefile | 2 +-
{virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c | 2 --
3 files changed, 1 insertion(+), 4 deletions(-)
rename {virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c (98%)
diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
index 8679405..4be5061 100644
--- a/arch/arm/kvm/hyp/Makefile
+++ b/arch/arm/kvm/hyp/Makefile
@@ -6,7 +6,6 @@ ccflags-y += -fno-stack-protector
KVM=../../../../virt/kvm
-obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 14c4e3b..b746fa5 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -6,10 +6,10 @@ ccflags-y += -fno-stack-protector
KVM=../../../../virt/kvm
-obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
similarity index 98%
rename from virt/kvm/arm/hyp/vgic-v2-sr.c
rename to arch/arm64/kvm/hyp/vgic-v2-sr.c
index b433257..fcd7b4e 100644
--- a/virt/kvm/arm/hyp/vgic-v2-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
@@ -22,7 +22,6 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
-#ifdef CONFIG_ARM64
/*
* __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
* guest.
@@ -76,4 +75,3 @@ int __hyp_text __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu)
return 1;
}
-#endif
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 33/37] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
The vgic-v2-sr.c file now only contains the logic to replay unaligned
accesses to the virtual CPU interface on 16K and 64K page systems, which
is only relevant on 64-bit platforms. Therefore move this file to the
arm64 KVM tree, remove the compile directive from the 32-bit side
makefile, and remove the ifdef in the C file.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/kvm/hyp/Makefile | 1 -
arch/arm64/kvm/hyp/Makefile | 2 +-
{virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c | 2 --
3 files changed, 1 insertion(+), 4 deletions(-)
rename {virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c (98%)
diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
index 8679405..4be5061 100644
--- a/arch/arm/kvm/hyp/Makefile
+++ b/arch/arm/kvm/hyp/Makefile
@@ -6,7 +6,6 @@ ccflags-y += -fno-stack-protector
KVM=../../../../virt/kvm
-obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 14c4e3b..b746fa5 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -6,10 +6,10 @@ ccflags-y += -fno-stack-protector
KVM=../../../../virt/kvm
-obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
similarity index 98%
rename from virt/kvm/arm/hyp/vgic-v2-sr.c
rename to arch/arm64/kvm/hyp/vgic-v2-sr.c
index b433257..fcd7b4e 100644
--- a/virt/kvm/arm/hyp/vgic-v2-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
@@ -22,7 +22,6 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
-#ifdef CONFIG_ARM64
/*
* __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
* guest.
@@ -76,4 +75,3 @@ int __hyp_text __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu)
return 1;
}
-#endif
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 33/37] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-15 17:52 ` Andre Przywara
-1 siblings, 0 replies; 254+ messages in thread
From: Andre Przywara @ 2017-11-15 17:52 UTC (permalink / raw)
To: Christoffer Dall, kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, kvm, Shih-Wei Li
Hi,
On 12/10/17 11:41, Christoffer Dall wrote:
> The vgic-v2-sr.c file now only contains the logic to replay unaligned
> accesses to the virtual CPU interface on 16K and 64K page systems, which
> is only relevant on 64-bit platforms. Therefore move this file to the
> arm64 KVM tree, remove the compile directive from the 32-bit side
> makefile, and remove the ifdef in the C file.
I like that one!
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Cheers,
Andre.
> ---
> arch/arm/kvm/hyp/Makefile | 1 -
> arch/arm64/kvm/hyp/Makefile | 2 +-
> {virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c | 2 --
> 3 files changed, 1 insertion(+), 4 deletions(-)
> rename {virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c (98%)
>
> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> index 8679405..4be5061 100644
> --- a/arch/arm/kvm/hyp/Makefile
> +++ b/arch/arm/kvm/hyp/Makefile
> @@ -6,7 +6,6 @@ ccflags-y += -fno-stack-protector
>
> KVM=../../../../virt/kvm
>
> -obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 14c4e3b..b746fa5 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -6,10 +6,10 @@ ccflags-y += -fno-stack-protector
>
> KVM=../../../../virt/kvm
>
> -obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>
> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
> similarity index 98%
> rename from virt/kvm/arm/hyp/vgic-v2-sr.c
> rename to arch/arm64/kvm/hyp/vgic-v2-sr.c
> index b433257..fcd7b4e 100644
> --- a/virt/kvm/arm/hyp/vgic-v2-sr.c
> +++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
> @@ -22,7 +22,6 @@
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> -#ifdef CONFIG_ARM64
> /*
> * __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
> * guest.
> @@ -76,4 +75,3 @@ int __hyp_text __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu)
>
> return 1;
> }
> -#endif
>
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 33/37] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64
@ 2017-11-15 17:52 ` Andre Przywara
0 siblings, 0 replies; 254+ messages in thread
From: Andre Przywara @ 2017-11-15 17:52 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On 12/10/17 11:41, Christoffer Dall wrote:
> The vgic-v2-sr.c file now only contains the logic to replay unaligned
> accesses to the virtual CPU interface on 16K and 64K page systems, which
> is only relevant on 64-bit platforms. Therefore move this file to the
> arm64 KVM tree, remove the compile directive from the 32-bit side
> makefile, and remove the ifdef in the C file.
I like that one!
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Cheers,
Andre.
> ---
> arch/arm/kvm/hyp/Makefile | 1 -
> arch/arm64/kvm/hyp/Makefile | 2 +-
> {virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c | 2 --
> 3 files changed, 1 insertion(+), 4 deletions(-)
> rename {virt/kvm/arm => arch/arm64/kvm}/hyp/vgic-v2-sr.c (98%)
>
> diff --git a/arch/arm/kvm/hyp/Makefile b/arch/arm/kvm/hyp/Makefile
> index 8679405..4be5061 100644
> --- a/arch/arm/kvm/hyp/Makefile
> +++ b/arch/arm/kvm/hyp/Makefile
> @@ -6,7 +6,6 @@ ccflags-y += -fno-stack-protector
>
> KVM=../../../../virt/kvm
>
> -obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 14c4e3b..b746fa5 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -6,10 +6,10 @@ ccflags-y += -fno-stack-protector
>
> KVM=../../../../virt/kvm
>
> -obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/hyp/timer-sr.o
>
> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> diff --git a/virt/kvm/arm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
> similarity index 98%
> rename from virt/kvm/arm/hyp/vgic-v2-sr.c
> rename to arch/arm64/kvm/hyp/vgic-v2-sr.c
> index b433257..fcd7b4e 100644
> --- a/virt/kvm/arm/hyp/vgic-v2-sr.c
> +++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
> @@ -22,7 +22,6 @@
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_hyp.h>
>
> -#ifdef CONFIG_ARM64
> /*
> * __vgic_v2_perform_cpuif_access -- perform a GICV access on behalf of the
> * guest.
> @@ -76,4 +75,3 @@ int __hyp_text __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu)
>
> return 1;
> }
> -#endif
>
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 34/37] KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on VHE
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
Just like we can program the GICv2 hypervisor control interface directly
from the core vgic code, we can do the same for the GICv3 hypervisor
control interface on VHE systems.
We do this by simply calling the save/restore functions when we have VHE
and we can then get rid of the save/restore function calls from the VHE
world switch function.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 3 ---
virt/kvm/arm/vgic/vgic.c | 4 ++++
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 90da506..0cdf6ae 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -366,8 +366,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
activate_traps_vhe(vcpu);
__activate_vm(vcpu);
- __vgic_restore_state(vcpu);
-
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
@@ -380,7 +378,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
goto again;
sysreg_save_guest_state_vhe(guest_ctxt);
- __vgic_save_state(vcpu);
deactivate_traps_vhe(vcpu);
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 790fd66..f0072f7 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -689,6 +689,8 @@ static inline void vgic_save_state(struct kvm_vcpu *vcpu)
{
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
vgic_v2_save_state(vcpu);
+ else if (has_vhe())
+ __vgic_v3_save_state(vcpu);
}
/* Sync back the hardware VGIC state into our emulation after a guest's run. */
@@ -711,6 +713,8 @@ static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
{
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
vgic_v2_restore_state(vcpu);
+ else if (has_vhe())
+ __vgic_v3_restore_state(vcpu);
}
/* Flush our emulation state into the GIC hardware before entering the guest. */
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 34/37] KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on VHE
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
Just like we can program the GICv2 hypervisor control interface directly
from the core vgic code, we can do the same for the GICv3 hypervisor
control interface on VHE systems.
We do this by simply calling the save/restore functions when we have VHE
and we can then get rid of the save/restore function calls from the VHE
world switch function.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/hyp/switch.c | 3 ---
virt/kvm/arm/vgic/vgic.c | 4 ++++
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 90da506..0cdf6ae 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -366,8 +366,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
activate_traps_vhe(vcpu);
__activate_vm(vcpu);
- __vgic_restore_state(vcpu);
-
sysreg_restore_guest_state_vhe(guest_ctxt);
__debug_switch_to_guest(vcpu);
@@ -380,7 +378,6 @@ int kvm_vcpu_run(struct kvm_vcpu *vcpu)
goto again;
sysreg_save_guest_state_vhe(guest_ctxt);
- __vgic_save_state(vcpu);
deactivate_traps_vhe(vcpu);
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 790fd66..f0072f7 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -689,6 +689,8 @@ static inline void vgic_save_state(struct kvm_vcpu *vcpu)
{
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
vgic_v2_save_state(vcpu);
+ else if (has_vhe())
+ __vgic_v3_save_state(vcpu);
}
/* Sync back the hardware VGIC state into our emulation after a guest's run. */
@@ -711,6 +713,8 @@ static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
{
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
vgic_v2_restore_state(vcpu);
+ else if (has_vhe())
+ __vgic_v3_restore_state(vcpu);
}
/* Flush our emulation state into the GIC hardware before entering the guest. */
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread
* [PATCH 35/37] KVM: arm/arm64: Get rid of vgic_elrsr
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
There is really no need to store the vgic_elrsr on the VGIC data
structures as the only need we have for the elrsr is to figure out if an
LR is inavtive when we save the VGIC state upon returning from the
guest. We can might as well store this in a temporary local variable.
This also gets rid of the endianness conversion in the VGIC save
function, which is completely unnecessary and would actually result in
incorrect functionality on big-endian systems, because we are only using
typed values here and not converting pointers and reading different
types here.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
include/kvm/arm_vgic.h | 2 --
virt/kvm/arm/hyp/vgic-v3-sr.c | 6 +++---
virt/kvm/arm/vgic/vgic-v2.c | 33 +++++++--------------------------
virt/kvm/arm/vgic/vgic-v3.c | 1 -
4 files changed, 10 insertions(+), 32 deletions(-)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 34dba51..79c9e67 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -237,7 +237,6 @@ struct vgic_dist {
struct vgic_v2_cpu_if {
u32 vgic_hcr;
u32 vgic_vmcr;
- u64 vgic_elrsr; /* Saved only */
u32 vgic_apr;
u32 vgic_lr[VGIC_V2_MAX_LRS];
};
@@ -246,7 +245,6 @@ struct vgic_v3_cpu_if {
u32 vgic_hcr;
u32 vgic_vmcr;
u32 vgic_sre; /* Restored only, change ignored */
- u32 vgic_elrsr; /* Saved only */
u32 vgic_ap0r[4];
u32 vgic_ap1r[4];
u64 vgic_lr[VGIC_V3_MAX_LRS];
diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 91728fa..05548b2 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -222,15 +222,16 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
int i;
u32 nr_pre_bits;
+ u32 elrsr;
- cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
+ elrsr = read_gicreg(ICH_ELSR_EL2);
write_gicreg(0, ICH_HCR_EL2);
val = read_gicreg(ICH_VTR_EL2);
nr_pre_bits = vtr_to_nr_pre_bits(val);
for (i = 0; i < used_lrs; i++) {
- if (cpu_if->vgic_elrsr & (1 << i))
+ if (elrsr & (1 << i))
cpu_if->vgic_lr[i] &= ~ICH_LR_STATE;
else
cpu_if->vgic_lr[i] = __gic_v3_get_lr(i);
@@ -261,7 +262,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
if (static_branch_unlikely(&vgic_v3_cpuif_trap))
write_gicreg(0, ICH_HCR_EL2);
- cpu_if->vgic_elrsr = 0xffff;
cpu_if->vgic_ap0r[0] = 0;
cpu_if->vgic_ap0r[1] = 0;
cpu_if->vgic_ap0r[2] = 0;
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 259079b..df7e03b 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -247,7 +247,6 @@ void vgic_v2_enable(struct kvm_vcpu *vcpu)
* anyway.
*/
vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
- vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr = ~0;
/* Get the show on the road... */
vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
@@ -404,33 +403,19 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
return ret;
}
-static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
-{
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int nr_lr = kvm_vgic_global_state.nr_lr;
- u32 elrsr0, elrsr1;
-
- elrsr0 = readl_relaxed(base + GICH_ELRSR0);
- if (unlikely(nr_lr > 32))
- elrsr1 = readl_relaxed(base + GICH_ELRSR1);
- else
- elrsr1 = 0;
-
-#ifdef CONFIG_CPU_BIG_ENDIAN
- cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
-#else
- cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
-#endif
-}
-
static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
{
struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int i;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+ u64 elrsr;
+ int i;
+
+ elrsr = readl_relaxed(base + GICH_ELRSR0);
+ if (unlikely(used_lrs > 32))
+ elrsr |= ((u64)readl_relaxed(base + GICH_ELRSR1)) << 32;
for (i = 0; i < used_lrs; i++) {
- if (cpu_if->vgic_elrsr & (1UL << i))
+ if (elrsr & (1UL << i))
cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
else
cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
@@ -452,13 +437,9 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
-
- save_elrsr(vcpu, base);
save_lrs(vcpu, base);
-
writel_relaxed(0, base + GICH_HCR);
} else {
- cpu_if->vgic_elrsr = ~0UL;
cpu_if->vgic_apr = 0;
}
}
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 863351c..0900649 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -237,7 +237,6 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
* anyway.
*/
vgic_v3->vgic_vmcr = 0;
- vgic_v3->vgic_elrsr = ~0;
/*
* If we are emulating a GICv3, we do it in an non-GICv2-compatible
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 35/37] KVM: arm/arm64: Get rid of vgic_elrsr
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
There is really no need to store the vgic_elrsr on the VGIC data
structures as the only need we have for the elrsr is to figure out if an
LR is inavtive when we save the VGIC state upon returning from the
guest. We can might as well store this in a temporary local variable.
This also gets rid of the endianness conversion in the VGIC save
function, which is completely unnecessary and would actually result in
incorrect functionality on big-endian systems, because we are only using
typed values here and not converting pointers and reading different
types here.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
include/kvm/arm_vgic.h | 2 --
virt/kvm/arm/hyp/vgic-v3-sr.c | 6 +++---
virt/kvm/arm/vgic/vgic-v2.c | 33 +++++++--------------------------
virt/kvm/arm/vgic/vgic-v3.c | 1 -
4 files changed, 10 insertions(+), 32 deletions(-)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 34dba51..79c9e67 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -237,7 +237,6 @@ struct vgic_dist {
struct vgic_v2_cpu_if {
u32 vgic_hcr;
u32 vgic_vmcr;
- u64 vgic_elrsr; /* Saved only */
u32 vgic_apr;
u32 vgic_lr[VGIC_V2_MAX_LRS];
};
@@ -246,7 +245,6 @@ struct vgic_v3_cpu_if {
u32 vgic_hcr;
u32 vgic_vmcr;
u32 vgic_sre; /* Restored only, change ignored */
- u32 vgic_elrsr; /* Saved only */
u32 vgic_ap0r[4];
u32 vgic_ap1r[4];
u64 vgic_lr[VGIC_V3_MAX_LRS];
diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 91728fa..05548b2 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -222,15 +222,16 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
int i;
u32 nr_pre_bits;
+ u32 elrsr;
- cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
+ elrsr = read_gicreg(ICH_ELSR_EL2);
write_gicreg(0, ICH_HCR_EL2);
val = read_gicreg(ICH_VTR_EL2);
nr_pre_bits = vtr_to_nr_pre_bits(val);
for (i = 0; i < used_lrs; i++) {
- if (cpu_if->vgic_elrsr & (1 << i))
+ if (elrsr & (1 << i))
cpu_if->vgic_lr[i] &= ~ICH_LR_STATE;
else
cpu_if->vgic_lr[i] = __gic_v3_get_lr(i);
@@ -261,7 +262,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
if (static_branch_unlikely(&vgic_v3_cpuif_trap))
write_gicreg(0, ICH_HCR_EL2);
- cpu_if->vgic_elrsr = 0xffff;
cpu_if->vgic_ap0r[0] = 0;
cpu_if->vgic_ap0r[1] = 0;
cpu_if->vgic_ap0r[2] = 0;
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 259079b..df7e03b 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -247,7 +247,6 @@ void vgic_v2_enable(struct kvm_vcpu *vcpu)
* anyway.
*/
vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
- vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr = ~0;
/* Get the show on the road... */
vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
@@ -404,33 +403,19 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
return ret;
}
-static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
-{
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int nr_lr = kvm_vgic_global_state.nr_lr;
- u32 elrsr0, elrsr1;
-
- elrsr0 = readl_relaxed(base + GICH_ELRSR0);
- if (unlikely(nr_lr > 32))
- elrsr1 = readl_relaxed(base + GICH_ELRSR1);
- else
- elrsr1 = 0;
-
-#ifdef CONFIG_CPU_BIG_ENDIAN
- cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
-#else
- cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
-#endif
-}
-
static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
{
struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
- int i;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
+ u64 elrsr;
+ int i;
+
+ elrsr = readl_relaxed(base + GICH_ELRSR0);
+ if (unlikely(used_lrs > 32))
+ elrsr |= ((u64)readl_relaxed(base + GICH_ELRSR1)) << 32;
for (i = 0; i < used_lrs; i++) {
- if (cpu_if->vgic_elrsr & (1UL << i))
+ if (elrsr & (1UL << i))
cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
else
cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
@@ -452,13 +437,9 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
-
- save_elrsr(vcpu, base);
save_lrs(vcpu, base);
-
writel_relaxed(0, base + GICH_HCR);
} else {
- cpu_if->vgic_elrsr = ~0UL;
cpu_if->vgic_apr = 0;
}
}
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 863351c..0900649 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -237,7 +237,6 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
* anyway.
*/
vgic_v3->vgic_vmcr = 0;
- vgic_v3->vgic_elrsr = ~0;
/*
* If we are emulating a GICv3, we do it in an non-GICv2-compatible
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 35/37] KVM: arm/arm64: Get rid of vgic_elrsr
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-26 14:39 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 14:39 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel, kvm
On Thu, Oct 12, 2017 at 12:41:39PM +0200, Christoffer Dall wrote:
> There is really no need to store the vgic_elrsr on the VGIC data
> structures as the only need we have for the elrsr is to figure out if an
> LR is inavtive when we save the VGIC state upon returning from the
> guest. We can might as well store this in a temporary local variable.
>
> This also gets rid of the endianness conversion in the VGIC save
> function, which is completely unnecessary and would actually result in
> incorrect functionality on big-endian systems,
Does it mean that existing code in mainline is broken for BE systems?
If so, I think that it should be fixed in separated patch...
> because we are only using
> typed values here and not converting pointers and reading different
> types here.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> include/kvm/arm_vgic.h | 2 --
> virt/kvm/arm/hyp/vgic-v3-sr.c | 6 +++---
> virt/kvm/arm/vgic/vgic-v2.c | 33 +++++++--------------------------
> virt/kvm/arm/vgic/vgic-v3.c | 1 -
> 4 files changed, 10 insertions(+), 32 deletions(-)
>
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 34dba51..79c9e67 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -237,7 +237,6 @@ struct vgic_dist {
> struct vgic_v2_cpu_if {
> u32 vgic_hcr;
> u32 vgic_vmcr;
> - u64 vgic_elrsr; /* Saved only */
> u32 vgic_apr;
> u32 vgic_lr[VGIC_V2_MAX_LRS];
> };
> @@ -246,7 +245,6 @@ struct vgic_v3_cpu_if {
> u32 vgic_hcr;
> u32 vgic_vmcr;
> u32 vgic_sre; /* Restored only, change ignored */
> - u32 vgic_elrsr; /* Saved only */
> u32 vgic_ap0r[4];
> u32 vgic_ap1r[4];
> u64 vgic_lr[VGIC_V3_MAX_LRS];
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index 91728fa..05548b2 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -222,15 +222,16 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> if (used_lrs) {
> int i;
> u32 nr_pre_bits;
> + u32 elrsr;
>
> - cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> + elrsr = read_gicreg(ICH_ELSR_EL2);
>
> write_gicreg(0, ICH_HCR_EL2);
> val = read_gicreg(ICH_VTR_EL2);
> nr_pre_bits = vtr_to_nr_pre_bits(val);
>
> for (i = 0; i < used_lrs; i++) {
> - if (cpu_if->vgic_elrsr & (1 << i))
> + if (elrsr & (1 << i))
Same comments as to patch 28. Here should be test_bit(), I think.
And if it's possible, for set bits in elrsr it's simpler not to write
~ICH_LR_STATE to cpu_if->vgic_lr, but directly to IO memory in
__vgic_v3_restore_state().
> cpu_if->vgic_lr[i] &= ~ICH_LR_STATE;
> else
> cpu_if->vgic_lr[i] = __gic_v3_get_lr(i);
> @@ -261,7 +262,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> write_gicreg(0, ICH_HCR_EL2);
>
> - cpu_if->vgic_elrsr = 0xffff;
> cpu_if->vgic_ap0r[0] = 0;
> cpu_if->vgic_ap0r[1] = 0;
> cpu_if->vgic_ap0r[2] = 0;
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index 259079b..df7e03b 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -247,7 +247,6 @@ void vgic_v2_enable(struct kvm_vcpu *vcpu)
> * anyway.
> */
> vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
> - vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr = ~0;
>
> /* Get the show on the road... */
> vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
> @@ -404,33 +403,19 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> return ret;
> }
>
> -static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> -{
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int nr_lr = kvm_vgic_global_state.nr_lr;
> - u32 elrsr0, elrsr1;
> -
> - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> - if (unlikely(nr_lr > 32))
> - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> - else
> - elrsr1 = 0;
> -
> -#ifdef CONFIG_CPU_BIG_ENDIAN
> - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> -#else
> - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> -#endif
> -}
> -
> static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> {
> struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int i;
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
Not to this patch, but anyway, what for to reserve u64 for used_lrs
here and in struct vgic_cpu, when maximum value for it is 64?
> + u64 elrsr;
> + int i;
Nit. What for you move 'int i' declaration?
> +
> + elrsr = readl_relaxed(base + GICH_ELRSR0);
> + if (unlikely(used_lrs > 32))
> + elrsr |= ((u64)readl_relaxed(base + GICH_ELRSR1)) << 32;
>
> for (i = 0; i < used_lrs; i++) {
> - if (cpu_if->vgic_elrsr & (1UL << i))
> + if (elrsr & (1UL << i))
test_bit()
> cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> else
> cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> @@ -452,13 +437,9 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
>
> if (used_lrs) {
> cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> -
> - save_elrsr(vcpu, base);
> save_lrs(vcpu, base);
> -
> writel_relaxed(0, base + GICH_HCR);
> } else {
> - cpu_if->vgic_elrsr = ~0UL;
> cpu_if->vgic_apr = 0;
> }
> }
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 863351c..0900649 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -237,7 +237,6 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> * anyway.
> */
> vgic_v3->vgic_vmcr = 0;
> - vgic_v3->vgic_elrsr = ~0;
>
> /*
> * If we are emulating a GICv3, we do it in an non-GICv2-compatible
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 35/37] KVM: arm/arm64: Get rid of vgic_elrsr
@ 2017-11-26 14:39 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 14:39 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:39PM +0200, Christoffer Dall wrote:
> There is really no need to store the vgic_elrsr on the VGIC data
> structures as the only need we have for the elrsr is to figure out if an
> LR is inavtive when we save the VGIC state upon returning from the
> guest. We can might as well store this in a temporary local variable.
>
> This also gets rid of the endianness conversion in the VGIC save
> function, which is completely unnecessary and would actually result in
> incorrect functionality on big-endian systems,
Does it mean that existing code in mainline is broken for BE systems?
If so, I think that it should be fixed in separated patch...
> because we are only using
> typed values here and not converting pointers and reading different
> types here.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> include/kvm/arm_vgic.h | 2 --
> virt/kvm/arm/hyp/vgic-v3-sr.c | 6 +++---
> virt/kvm/arm/vgic/vgic-v2.c | 33 +++++++--------------------------
> virt/kvm/arm/vgic/vgic-v3.c | 1 -
> 4 files changed, 10 insertions(+), 32 deletions(-)
>
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 34dba51..79c9e67 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -237,7 +237,6 @@ struct vgic_dist {
> struct vgic_v2_cpu_if {
> u32 vgic_hcr;
> u32 vgic_vmcr;
> - u64 vgic_elrsr; /* Saved only */
> u32 vgic_apr;
> u32 vgic_lr[VGIC_V2_MAX_LRS];
> };
> @@ -246,7 +245,6 @@ struct vgic_v3_cpu_if {
> u32 vgic_hcr;
> u32 vgic_vmcr;
> u32 vgic_sre; /* Restored only, change ignored */
> - u32 vgic_elrsr; /* Saved only */
> u32 vgic_ap0r[4];
> u32 vgic_ap1r[4];
> u64 vgic_lr[VGIC_V3_MAX_LRS];
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index 91728fa..05548b2 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -222,15 +222,16 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> if (used_lrs) {
> int i;
> u32 nr_pre_bits;
> + u32 elrsr;
>
> - cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> + elrsr = read_gicreg(ICH_ELSR_EL2);
>
> write_gicreg(0, ICH_HCR_EL2);
> val = read_gicreg(ICH_VTR_EL2);
> nr_pre_bits = vtr_to_nr_pre_bits(val);
>
> for (i = 0; i < used_lrs; i++) {
> - if (cpu_if->vgic_elrsr & (1 << i))
> + if (elrsr & (1 << i))
Same comments as to patch 28. Here should be test_bit(), I think.
And if it's possible, for set bits in elrsr it's simpler not to write
~ICH_LR_STATE to cpu_if->vgic_lr, but directly to IO memory in
__vgic_v3_restore_state().
> cpu_if->vgic_lr[i] &= ~ICH_LR_STATE;
> else
> cpu_if->vgic_lr[i] = __gic_v3_get_lr(i);
> @@ -261,7 +262,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> write_gicreg(0, ICH_HCR_EL2);
>
> - cpu_if->vgic_elrsr = 0xffff;
> cpu_if->vgic_ap0r[0] = 0;
> cpu_if->vgic_ap0r[1] = 0;
> cpu_if->vgic_ap0r[2] = 0;
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index 259079b..df7e03b 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -247,7 +247,6 @@ void vgic_v2_enable(struct kvm_vcpu *vcpu)
> * anyway.
> */
> vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
> - vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr = ~0;
>
> /* Get the show on the road... */
> vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
> @@ -404,33 +403,19 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> return ret;
> }
>
> -static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> -{
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int nr_lr = kvm_vgic_global_state.nr_lr;
> - u32 elrsr0, elrsr1;
> -
> - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> - if (unlikely(nr_lr > 32))
> - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> - else
> - elrsr1 = 0;
> -
> -#ifdef CONFIG_CPU_BIG_ENDIAN
> - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> -#else
> - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> -#endif
> -}
> -
> static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> {
> struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> - int i;
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
Not to this patch, but anyway, what for to reserve u64 for used_lrs
here and in struct vgic_cpu, when maximum value for it is 64?
> + u64 elrsr;
> + int i;
Nit. What for you move 'int i' declaration?
> +
> + elrsr = readl_relaxed(base + GICH_ELRSR0);
> + if (unlikely(used_lrs > 32))
> + elrsr |= ((u64)readl_relaxed(base + GICH_ELRSR1)) << 32;
>
> for (i = 0; i < used_lrs; i++) {
> - if (cpu_if->vgic_elrsr & (1UL << i))
> + if (elrsr & (1UL << i))
test_bit()
> cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> else
> cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> @@ -452,13 +437,9 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
>
> if (used_lrs) {
> cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> -
> - save_elrsr(vcpu, base);
> save_lrs(vcpu, base);
> -
> writel_relaxed(0, base + GICH_HCR);
> } else {
> - cpu_if->vgic_elrsr = ~0UL;
> cpu_if->vgic_apr = 0;
> }
> }
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 863351c..0900649 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -237,7 +237,6 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> * anyway.
> */
> vgic_v3->vgic_vmcr = 0;
> - vgic_v3->vgic_elrsr = ~0;
>
> /*
> * If we are emulating a GICv3, we do it in an non-GICv2-compatible
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 35/37] KVM: arm/arm64: Get rid of vgic_elrsr
2017-11-26 14:39 ` Yury Norov
@ 2017-11-26 19:53 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:53 UTC (permalink / raw)
To: Yury Norov; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
On Sun, Nov 26, 2017 at 05:39:02PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:39PM +0200, Christoffer Dall wrote:
> > There is really no need to store the vgic_elrsr on the VGIC data
> > structures as the only need we have for the elrsr is to figure out if an
> > LR is inavtive when we save the VGIC state upon returning from the
> > guest. We can might as well store this in a temporary local variable.
> >
> > This also gets rid of the endianness conversion in the VGIC save
> > function, which is completely unnecessary and would actually result in
> > incorrect functionality on big-endian systems,
>
> Does it mean that existing code in mainline is broken for BE systems?
> If so, I think that it should be fixed in separated patch...
>
I think it's broken yes. But I was looking for someone to actually tell
me wether they agreed on this or not.
Despite the rare use of big endian hosts, it may be ware to submit this
is a separate fix indeed.
> > because we are only using
> > typed values here and not converting pointers and reading different
> > types here.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > include/kvm/arm_vgic.h | 2 --
> > virt/kvm/arm/hyp/vgic-v3-sr.c | 6 +++---
> > virt/kvm/arm/vgic/vgic-v2.c | 33 +++++++--------------------------
> > virt/kvm/arm/vgic/vgic-v3.c | 1 -
> > 4 files changed, 10 insertions(+), 32 deletions(-)
> >
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index 34dba51..79c9e67 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -237,7 +237,6 @@ struct vgic_dist {
> > struct vgic_v2_cpu_if {
> > u32 vgic_hcr;
> > u32 vgic_vmcr;
> > - u64 vgic_elrsr; /* Saved only */
> > u32 vgic_apr;
> > u32 vgic_lr[VGIC_V2_MAX_LRS];
> > };
> > @@ -246,7 +245,6 @@ struct vgic_v3_cpu_if {
> > u32 vgic_hcr;
> > u32 vgic_vmcr;
> > u32 vgic_sre; /* Restored only, change ignored */
> > - u32 vgic_elrsr; /* Saved only */
> > u32 vgic_ap0r[4];
> > u32 vgic_ap1r[4];
> > u64 vgic_lr[VGIC_V3_MAX_LRS];
> > diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > index 91728fa..05548b2 100644
> > --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> > +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > @@ -222,15 +222,16 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > if (used_lrs) {
> > int i;
> > u32 nr_pre_bits;
> > + u32 elrsr;
> >
> > - cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> > + elrsr = read_gicreg(ICH_ELSR_EL2);
> >
> > write_gicreg(0, ICH_HCR_EL2);
> > val = read_gicreg(ICH_VTR_EL2);
> > nr_pre_bits = vtr_to_nr_pre_bits(val);
> >
> > for (i = 0; i < used_lrs; i++) {
> > - if (cpu_if->vgic_elrsr & (1 << i))
> > + if (elrsr & (1 << i))
>
> Same comments as to patch 28. Here should be test_bit(), I think.
> And if it's possible, for set bits in elrsr it's simpler not to write
> ~ICH_LR_STATE to cpu_if->vgic_lr, but directly to IO memory in
> __vgic_v3_restore_state().
>
I think you're missing how the VGIC works. We need to clear the
LR_STATE on the memory representation of the LRs, because that's how we
know the guest has processed an interrupt and is ready to take a new
one.
Also, there's no IO memory for the GICv3 LRs, they're system registers.
> > cpu_if->vgic_lr[i] &= ~ICH_LR_STATE;
> > else
> > cpu_if->vgic_lr[i] = __gic_v3_get_lr(i);
> > @@ -261,7 +262,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> > write_gicreg(0, ICH_HCR_EL2);
> >
> > - cpu_if->vgic_elrsr = 0xffff;
> > cpu_if->vgic_ap0r[0] = 0;
> > cpu_if->vgic_ap0r[1] = 0;
> > cpu_if->vgic_ap0r[2] = 0;
> > diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> > index 259079b..df7e03b 100644
> > --- a/virt/kvm/arm/vgic/vgic-v2.c
> > +++ b/virt/kvm/arm/vgic/vgic-v2.c
> > @@ -247,7 +247,6 @@ void vgic_v2_enable(struct kvm_vcpu *vcpu)
> > * anyway.
> > */
> > vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
> > - vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr = ~0;
> >
> > /* Get the show on the road... */
> > vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
> > @@ -404,33 +403,19 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> > return ret;
> > }
> >
> > -static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > -{
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int nr_lr = kvm_vgic_global_state.nr_lr;
> > - u32 elrsr0, elrsr1;
> > -
> > - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > - if (unlikely(nr_lr > 32))
> > - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > - else
> > - elrsr1 = 0;
> > -
> > -#ifdef CONFIG_CPU_BIG_ENDIAN
> > - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > -#else
> > - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > -#endif
> > -}
> > -
> > static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > {
> > struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int i;
> > u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
>
> Not to this patch, but anyway, what for to reserve u64 for used_lrs
> here and in struct vgic_cpu, when maximum value for it is 64?
>
What's your concern? A couple of extra bytes in the vcpu structure? I
wouldn't be opposed to changing the data type and moving things around
if someone wants to micro-optimize the KVM memory footprint. Patches
are welxome.
> > + u64 elrsr;
> > + int i;
>
> Nit. What for you move 'int i' declaration?
>
It looks nicer that with the long two lines.
> > +
> > + elrsr = readl_relaxed(base + GICH_ELRSR0);
> > + if (unlikely(used_lrs > 32))
> > + elrsr |= ((u64)readl_relaxed(base + GICH_ELRSR1)) << 32;
> >
> > for (i = 0; i < used_lrs; i++) {
> > - if (cpu_if->vgic_elrsr & (1UL << i))
> > + if (elrsr & (1UL << i))
>
> test_bit()
>
Given that test_bit is more complicated because it can work on arbitrary
length bitmaps and we're operating on a single word here, what is the
benefit? Is it just that you prefer reading test_bit for readability or
do you have a claim of performance benefits?
-Christoffer
> > cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > else
> > cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > @@ -452,13 +437,9 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> >
> > if (used_lrs) {
> > cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> > -
> > - save_elrsr(vcpu, base);
> > save_lrs(vcpu, base);
> > -
> > writel_relaxed(0, base + GICH_HCR);
> > } else {
> > - cpu_if->vgic_elrsr = ~0UL;
> > cpu_if->vgic_apr = 0;
> > }
> > }
> > diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> > index 863351c..0900649 100644
> > --- a/virt/kvm/arm/vgic/vgic-v3.c
> > +++ b/virt/kvm/arm/vgic/vgic-v3.c
> > @@ -237,7 +237,6 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> > * anyway.
> > */
> > vgic_v3->vgic_vmcr = 0;
> > - vgic_v3->vgic_elrsr = ~0;
> >
> > /*
> > * If we are emulating a GICv3, we do it in an non-GICv2-compatible
> > --
> > 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 35/37] KVM: arm/arm64: Get rid of vgic_elrsr
@ 2017-11-26 19:53 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:53 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 05:39:02PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:39PM +0200, Christoffer Dall wrote:
> > There is really no need to store the vgic_elrsr on the VGIC data
> > structures as the only need we have for the elrsr is to figure out if an
> > LR is inavtive when we save the VGIC state upon returning from the
> > guest. We can might as well store this in a temporary local variable.
> >
> > This also gets rid of the endianness conversion in the VGIC save
> > function, which is completely unnecessary and would actually result in
> > incorrect functionality on big-endian systems,
>
> Does it mean that existing code in mainline is broken for BE systems?
> If so, I think that it should be fixed in separated patch...
>
I think it's broken yes. But I was looking for someone to actually tell
me wether they agreed on this or not.
Despite the rare use of big endian hosts, it may be ware to submit this
is a separate fix indeed.
> > because we are only using
> > typed values here and not converting pointers and reading different
> > types here.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > include/kvm/arm_vgic.h | 2 --
> > virt/kvm/arm/hyp/vgic-v3-sr.c | 6 +++---
> > virt/kvm/arm/vgic/vgic-v2.c | 33 +++++++--------------------------
> > virt/kvm/arm/vgic/vgic-v3.c | 1 -
> > 4 files changed, 10 insertions(+), 32 deletions(-)
> >
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index 34dba51..79c9e67 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -237,7 +237,6 @@ struct vgic_dist {
> > struct vgic_v2_cpu_if {
> > u32 vgic_hcr;
> > u32 vgic_vmcr;
> > - u64 vgic_elrsr; /* Saved only */
> > u32 vgic_apr;
> > u32 vgic_lr[VGIC_V2_MAX_LRS];
> > };
> > @@ -246,7 +245,6 @@ struct vgic_v3_cpu_if {
> > u32 vgic_hcr;
> > u32 vgic_vmcr;
> > u32 vgic_sre; /* Restored only, change ignored */
> > - u32 vgic_elrsr; /* Saved only */
> > u32 vgic_ap0r[4];
> > u32 vgic_ap1r[4];
> > u64 vgic_lr[VGIC_V3_MAX_LRS];
> > diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > index 91728fa..05548b2 100644
> > --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> > +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > @@ -222,15 +222,16 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > if (used_lrs) {
> > int i;
> > u32 nr_pre_bits;
> > + u32 elrsr;
> >
> > - cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> > + elrsr = read_gicreg(ICH_ELSR_EL2);
> >
> > write_gicreg(0, ICH_HCR_EL2);
> > val = read_gicreg(ICH_VTR_EL2);
> > nr_pre_bits = vtr_to_nr_pre_bits(val);
> >
> > for (i = 0; i < used_lrs; i++) {
> > - if (cpu_if->vgic_elrsr & (1 << i))
> > + if (elrsr & (1 << i))
>
> Same comments as to patch 28. Here should be test_bit(), I think.
> And if it's possible, for set bits in elrsr it's simpler not to write
> ~ICH_LR_STATE to cpu_if->vgic_lr, but directly to IO memory in
> __vgic_v3_restore_state().
>
I think you're missing how the VGIC works. We need to clear the
LR_STATE on the memory representation of the LRs, because that's how we
know the guest has processed an interrupt and is ready to take a new
one.
Also, there's no IO memory for the GICv3 LRs, they're system registers.
> > cpu_if->vgic_lr[i] &= ~ICH_LR_STATE;
> > else
> > cpu_if->vgic_lr[i] = __gic_v3_get_lr(i);
> > @@ -261,7 +262,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> > write_gicreg(0, ICH_HCR_EL2);
> >
> > - cpu_if->vgic_elrsr = 0xffff;
> > cpu_if->vgic_ap0r[0] = 0;
> > cpu_if->vgic_ap0r[1] = 0;
> > cpu_if->vgic_ap0r[2] = 0;
> > diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> > index 259079b..df7e03b 100644
> > --- a/virt/kvm/arm/vgic/vgic-v2.c
> > +++ b/virt/kvm/arm/vgic/vgic-v2.c
> > @@ -247,7 +247,6 @@ void vgic_v2_enable(struct kvm_vcpu *vcpu)
> > * anyway.
> > */
> > vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
> > - vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr = ~0;
> >
> > /* Get the show on the road... */
> > vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
> > @@ -404,33 +403,19 @@ int vgic_v2_probe(const struct gic_kvm_info *info)
> > return ret;
> > }
> >
> > -static void save_elrsr(struct kvm_vcpu *vcpu, void __iomem *base)
> > -{
> > - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int nr_lr = kvm_vgic_global_state.nr_lr;
> > - u32 elrsr0, elrsr1;
> > -
> > - elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> > - if (unlikely(nr_lr > 32))
> > - elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> > - else
> > - elrsr1 = 0;
> > -
> > -#ifdef CONFIG_CPU_BIG_ENDIAN
> > - cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> > -#else
> > - cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> > -#endif
> > -}
> > -
> > static void save_lrs(struct kvm_vcpu *vcpu, void __iomem *base)
> > {
> > struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> > - int i;
> > u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
>
> Not to this patch, but anyway, what for to reserve u64 for used_lrs
> here and in struct vgic_cpu, when maximum value for it is 64?
>
What's your concern? A couple of extra bytes in the vcpu structure? I
wouldn't be opposed to changing the data type and moving things around
if someone wants to micro-optimize the KVM memory footprint. Patches
are welxome.
> > + u64 elrsr;
> > + int i;
>
> Nit. What for you move 'int i' declaration?
>
It looks nicer that with the long two lines.
> > +
> > + elrsr = readl_relaxed(base + GICH_ELRSR0);
> > + if (unlikely(used_lrs > 32))
> > + elrsr |= ((u64)readl_relaxed(base + GICH_ELRSR1)) << 32;
> >
> > for (i = 0; i < used_lrs; i++) {
> > - if (cpu_if->vgic_elrsr & (1UL << i))
> > + if (elrsr & (1UL << i))
>
> test_bit()
>
Given that test_bit is more complicated because it can work on arbitrary
length bitmaps and we're operating on a single word here, what is the
benefit? Is it just that you prefer reading test_bit for readability or
do you have a claim of performance benefits?
-Christoffer
> > cpu_if->vgic_lr[i] &= ~GICH_LR_STATE;
> > else
> > cpu_if->vgic_lr[i] = readl_relaxed(base + GICH_LR0 + (i * 4));
> > @@ -452,13 +437,9 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> >
> > if (used_lrs) {
> > cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> > -
> > - save_elrsr(vcpu, base);
> > save_lrs(vcpu, base);
> > -
> > writel_relaxed(0, base + GICH_HCR);
> > } else {
> > - cpu_if->vgic_elrsr = ~0UL;
> > cpu_if->vgic_apr = 0;
> > }
> > }
> > diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> > index 863351c..0900649 100644
> > --- a/virt/kvm/arm/vgic/vgic-v3.c
> > +++ b/virt/kvm/arm/vgic/vgic-v3.c
> > @@ -237,7 +237,6 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> > * anyway.
> > */
> > vgic_v3->vgic_vmcr = 0;
> > - vgic_v3->vgic_elrsr = ~0;
> >
> > /*
> > * If we are emulating a GICv3, we do it in an non-GICv2-compatible
> > --
> > 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 36/37] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
The APRs can only have bits set when the guest acknowledges an interrupt
in the LR and can only have a bit cleared when the guest EOIs an
interrupt in the LR. Therefore, if we have no LRs with any
pending/active interrupts, the APR cannot change value and there is no
need to clear it on every exit from the VM (hint: it will have already
been cleared when we exited the guest the last time with the LRs all
EOIed).
The only case we need to take care of is when we migrate the VCPU away
from a CPU or migrate a new VCPU onto a CPU, or when we return to
userspace to capture the state of the VCPU for migration. To make sure
this works, factor out the APR save/restore functionality into separate
functions called from the VCPU (and by extension VGIC) put/load hooks.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_hyp.h | 2 +
arch/arm64/include/asm/kvm_hyp.h | 2 +
virt/kvm/arm/hyp/vgic-v3-sr.c | 123 +++++++++++++++++++++------------------
virt/kvm/arm/vgic/vgic-v2.c | 7 +--
virt/kvm/arm/vgic/vgic-v3.c | 5 ++
5 files changed, 77 insertions(+), 62 deletions(-)
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index ab20ffa..b3dd4f4 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -109,6 +109,8 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
asmlinkage void __vfp_save_state(struct vfp_hard_struct *vfp);
asmlinkage void __vfp_restore_state(struct vfp_hard_struct *vfp);
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index bd3fe64..62872ce 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -125,6 +125,8 @@ int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __timer_enable_traps(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 05548b2..ed5da75 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -221,14 +221,11 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
int i;
- u32 nr_pre_bits;
u32 elrsr;
elrsr = read_gicreg(ICH_ELSR_EL2);
write_gicreg(0, ICH_HCR_EL2);
- val = read_gicreg(ICH_VTR_EL2);
- nr_pre_bits = vtr_to_nr_pre_bits(val);
for (i = 0; i < used_lrs; i++) {
if (elrsr & (1 << i))
@@ -238,38 +235,9 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
__gic_v3_set_lr(0, i);
}
-
- switch (nr_pre_bits) {
- case 7:
- cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
- cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
- case 6:
- cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
- default:
- cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
- }
-
- switch (nr_pre_bits) {
- case 7:
- cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
- cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
- case 6:
- cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
- default:
- cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
- }
} else {
if (static_branch_unlikely(&vgic_v3_cpuif_trap))
write_gicreg(0, ICH_HCR_EL2);
-
- cpu_if->vgic_ap0r[0] = 0;
- cpu_if->vgic_ap0r[1] = 0;
- cpu_if->vgic_ap0r[2] = 0;
- cpu_if->vgic_ap0r[3] = 0;
- cpu_if->vgic_ap1r[0] = 0;
- cpu_if->vgic_ap1r[1] = 0;
- cpu_if->vgic_ap1r[2] = 0;
- cpu_if->vgic_ap1r[3] = 0;
}
val = read_gicreg(ICC_SRE_EL2);
@@ -286,8 +254,6 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
- u64 val;
- u32 nr_pre_bits;
int i;
/*
@@ -305,32 +271,9 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
}
- val = read_gicreg(ICH_VTR_EL2);
- nr_pre_bits = vtr_to_nr_pre_bits(val);
-
if (used_lrs) {
write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
- switch (nr_pre_bits) {
- case 7:
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
- case 6:
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
- default:
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
- }
-
- switch (nr_pre_bits) {
- case 7:
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
- case 6:
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
- default:
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
- }
-
for (i = 0; i < used_lrs; i++)
__gic_v3_set_lr(cpu_if->vgic_lr[i], i);
} else {
@@ -361,6 +304,72 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
ICC_SRE_EL2);
}
+void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if;
+ u64 val;
+ u32 nr_pre_bits;
+
+ vcpu = kern_hyp_va(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ val = read_gicreg(ICH_VTR_EL2);
+ nr_pre_bits = vtr_to_nr_pre_bits(val);
+
+ switch (nr_pre_bits) {
+ case 7:
+ cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
+ cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
+ case 6:
+ cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
+ default:
+ cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
+ }
+
+ switch (nr_pre_bits) {
+ case 7:
+ cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
+ cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
+ case 6:
+ cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
+ default:
+ cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
+ }
+}
+
+void __hyp_text __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if;
+ u64 val;
+ u32 nr_pre_bits;
+
+ vcpu = kern_hyp_va(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ val = read_gicreg(ICH_VTR_EL2);
+ nr_pre_bits = vtr_to_nr_pre_bits(val);
+
+ switch (nr_pre_bits) {
+ case 7:
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
+ case 6:
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
+ default:
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
+ }
+
+ switch (nr_pre_bits) {
+ case 7:
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
+ case 6:
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
+ default:
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
+ }
+}
+
void __hyp_text __vgic_v3_init_lrs(void)
{
int max_lr_idx = vtr_to_max_lr_idx(read_gicreg(ICH_VTR_EL2));
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index df7e03b..c04c3475 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -428,7 +428,6 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = vcpu->kvm;
struct vgic_dist *vgic = &kvm->arch.vgic;
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
void __iomem *base = vgic->vctrl_base;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
@@ -436,11 +435,8 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
return;
if (used_lrs) {
- cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
save_lrs(vcpu, base);
writel_relaxed(0, base + GICH_HCR);
- } else {
- cpu_if->vgic_apr = 0;
}
}
@@ -458,7 +454,6 @@ void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
- writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
for (i = 0; i < used_lrs; i++) {
writel_relaxed(cpu_if->vgic_lr[i],
base + GICH_LR0 + (i * 4));
@@ -472,6 +467,7 @@ void vgic_v2_load(struct kvm_vcpu *vcpu)
struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
writel_relaxed(cpu_if->vgic_vmcr, vgic->vctrl_base + GICH_VMCR);
+ writel_relaxed(cpu_if->vgic_apr, vgic->vctrl_base + GICH_APR);
}
void vgic_v2_put(struct kvm_vcpu *vcpu)
@@ -480,4 +476,5 @@ void vgic_v2_put(struct kvm_vcpu *vcpu)
struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
cpu_if->vgic_vmcr = readl_relaxed(vgic->vctrl_base + GICH_VMCR);
+ cpu_if->vgic_apr = readl_relaxed(vgic->vctrl_base + GICH_APR);
}
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 0900649..df3a222 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -16,6 +16,7 @@
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <kvm/arm_vgic.h>
+#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
#include <asm/kvm_asm.h>
@@ -544,6 +545,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
*/
if (likely(cpu_if->vgic_sre))
kvm_call_hyp(__vgic_v3_write_vmcr, cpu_if->vgic_vmcr);
+
+ kvm_call_hyp(__vgic_v3_restore_aprs, vcpu);
}
void vgic_v3_put(struct kvm_vcpu *vcpu)
@@ -552,4 +555,6 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
if (likely(cpu_if->vgic_sre))
cpu_if->vgic_vmcr = kvm_call_hyp(__vgic_v3_read_vmcr);
+
+ kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 36/37] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
The APRs can only have bits set when the guest acknowledges an interrupt
in the LR and can only have a bit cleared when the guest EOIs an
interrupt in the LR. Therefore, if we have no LRs with any
pending/active interrupts, the APR cannot change value and there is no
need to clear it on every exit from the VM (hint: it will have already
been cleared when we exited the guest the last time with the LRs all
EOIed).
The only case we need to take care of is when we migrate the VCPU away
from a CPU or migrate a new VCPU onto a CPU, or when we return to
userspace to capture the state of the VCPU for migration. To make sure
this works, factor out the APR save/restore functionality into separate
functions called from the VCPU (and by extension VGIC) put/load hooks.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_hyp.h | 2 +
arch/arm64/include/asm/kvm_hyp.h | 2 +
virt/kvm/arm/hyp/vgic-v3-sr.c | 123 +++++++++++++++++++++------------------
virt/kvm/arm/vgic/vgic-v2.c | 7 +--
virt/kvm/arm/vgic/vgic-v3.c | 5 ++
5 files changed, 77 insertions(+), 62 deletions(-)
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index ab20ffa..b3dd4f4 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -109,6 +109,8 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
asmlinkage void __vfp_save_state(struct vfp_hard_struct *vfp);
asmlinkage void __vfp_restore_state(struct vfp_hard_struct *vfp);
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index bd3fe64..62872ce 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -125,6 +125,8 @@ int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __timer_enable_traps(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index 05548b2..ed5da75 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -221,14 +221,11 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
int i;
- u32 nr_pre_bits;
u32 elrsr;
elrsr = read_gicreg(ICH_ELSR_EL2);
write_gicreg(0, ICH_HCR_EL2);
- val = read_gicreg(ICH_VTR_EL2);
- nr_pre_bits = vtr_to_nr_pre_bits(val);
for (i = 0; i < used_lrs; i++) {
if (elrsr & (1 << i))
@@ -238,38 +235,9 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
__gic_v3_set_lr(0, i);
}
-
- switch (nr_pre_bits) {
- case 7:
- cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
- cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
- case 6:
- cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
- default:
- cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
- }
-
- switch (nr_pre_bits) {
- case 7:
- cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
- cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
- case 6:
- cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
- default:
- cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
- }
} else {
if (static_branch_unlikely(&vgic_v3_cpuif_trap))
write_gicreg(0, ICH_HCR_EL2);
-
- cpu_if->vgic_ap0r[0] = 0;
- cpu_if->vgic_ap0r[1] = 0;
- cpu_if->vgic_ap0r[2] = 0;
- cpu_if->vgic_ap0r[3] = 0;
- cpu_if->vgic_ap1r[0] = 0;
- cpu_if->vgic_ap1r[1] = 0;
- cpu_if->vgic_ap1r[2] = 0;
- cpu_if->vgic_ap1r[3] = 0;
}
val = read_gicreg(ICC_SRE_EL2);
@@ -286,8 +254,6 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
- u64 val;
- u32 nr_pre_bits;
int i;
/*
@@ -305,32 +271,9 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
}
- val = read_gicreg(ICH_VTR_EL2);
- nr_pre_bits = vtr_to_nr_pre_bits(val);
-
if (used_lrs) {
write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
- switch (nr_pre_bits) {
- case 7:
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
- case 6:
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
- default:
- __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
- }
-
- switch (nr_pre_bits) {
- case 7:
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
- case 6:
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
- default:
- __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
- }
-
for (i = 0; i < used_lrs; i++)
__gic_v3_set_lr(cpu_if->vgic_lr[i], i);
} else {
@@ -361,6 +304,72 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
ICC_SRE_EL2);
}
+void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if;
+ u64 val;
+ u32 nr_pre_bits;
+
+ vcpu = kern_hyp_va(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ val = read_gicreg(ICH_VTR_EL2);
+ nr_pre_bits = vtr_to_nr_pre_bits(val);
+
+ switch (nr_pre_bits) {
+ case 7:
+ cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
+ cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
+ case 6:
+ cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
+ default:
+ cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
+ }
+
+ switch (nr_pre_bits) {
+ case 7:
+ cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
+ cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
+ case 6:
+ cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
+ default:
+ cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
+ }
+}
+
+void __hyp_text __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if;
+ u64 val;
+ u32 nr_pre_bits;
+
+ vcpu = kern_hyp_va(vcpu);
+ cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ val = read_gicreg(ICH_VTR_EL2);
+ nr_pre_bits = vtr_to_nr_pre_bits(val);
+
+ switch (nr_pre_bits) {
+ case 7:
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
+ case 6:
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
+ default:
+ __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
+ }
+
+ switch (nr_pre_bits) {
+ case 7:
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
+ case 6:
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
+ default:
+ __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
+ }
+}
+
void __hyp_text __vgic_v3_init_lrs(void)
{
int max_lr_idx = vtr_to_max_lr_idx(read_gicreg(ICH_VTR_EL2));
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index df7e03b..c04c3475 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -428,7 +428,6 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = vcpu->kvm;
struct vgic_dist *vgic = &kvm->arch.vgic;
- struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
void __iomem *base = vgic->vctrl_base;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
@@ -436,11 +435,8 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
return;
if (used_lrs) {
- cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
save_lrs(vcpu, base);
writel_relaxed(0, base + GICH_HCR);
- } else {
- cpu_if->vgic_apr = 0;
}
}
@@ -458,7 +454,6 @@ void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
if (used_lrs) {
writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
- writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
for (i = 0; i < used_lrs; i++) {
writel_relaxed(cpu_if->vgic_lr[i],
base + GICH_LR0 + (i * 4));
@@ -472,6 +467,7 @@ void vgic_v2_load(struct kvm_vcpu *vcpu)
struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
writel_relaxed(cpu_if->vgic_vmcr, vgic->vctrl_base + GICH_VMCR);
+ writel_relaxed(cpu_if->vgic_apr, vgic->vctrl_base + GICH_APR);
}
void vgic_v2_put(struct kvm_vcpu *vcpu)
@@ -480,4 +476,5 @@ void vgic_v2_put(struct kvm_vcpu *vcpu)
struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
cpu_if->vgic_vmcr = readl_relaxed(vgic->vctrl_base + GICH_VMCR);
+ cpu_if->vgic_apr = readl_relaxed(vgic->vctrl_base + GICH_APR);
}
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 0900649..df3a222 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -16,6 +16,7 @@
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <kvm/arm_vgic.h>
+#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
#include <asm/kvm_asm.h>
@@ -544,6 +545,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
*/
if (likely(cpu_if->vgic_sre))
kvm_call_hyp(__vgic_v3_write_vmcr, cpu_if->vgic_vmcr);
+
+ kvm_call_hyp(__vgic_v3_restore_aprs, vcpu);
}
void vgic_v3_put(struct kvm_vcpu *vcpu)
@@ -552,4 +555,6 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
if (likely(cpu_if->vgic_sre))
cpu_if->vgic_vmcr = kvm_call_hyp(__vgic_v3_read_vmcr);
+
+ kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 36/37] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-26 15:09 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 15:09 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel, kvm
On Thu, Oct 12, 2017 at 12:41:40PM +0200, Christoffer Dall wrote:
> The APRs can only have bits set when the guest acknowledges an interrupt
> in the LR and can only have a bit cleared when the guest EOIs an
> interrupt in the LR. Therefore, if we have no LRs with any
> pending/active interrupts, the APR cannot change value and there is no
> need to clear it on every exit from the VM (hint: it will have already
> been cleared when we exited the guest the last time with the LRs all
> EOIed).
>
> The only case we need to take care of is when we migrate the VCPU away
> from a CPU or migrate a new VCPU onto a CPU, or when we return to
> userspace to capture the state of the VCPU for migration. To make sure
> this works, factor out the APR save/restore functionality into separate
> functions called from the VCPU (and by extension VGIC) put/load hooks.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> virt/kvm/arm/hyp/vgic-v3-sr.c | 123 +++++++++++++++++++++------------------
> virt/kvm/arm/vgic/vgic-v2.c | 7 +--
> virt/kvm/arm/vgic/vgic-v3.c | 5 ++
> 5 files changed, 77 insertions(+), 62 deletions(-)
[...]
> @@ -361,6 +304,72 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> ICC_SRE_EL2);
> }
>
> +void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if;
> + u64 val;
> + u32 nr_pre_bits;
> +
> + vcpu = kern_hyp_va(vcpu);
> + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_pre_bits = vtr_to_nr_pre_bits(val);
> +
> + switch (nr_pre_bits) {
> + case 7:
> + cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
> + cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
> + case 6:
> + cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
> + default:
> + cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
> + }
> +
> + switch (nr_pre_bits) {
> + case 7:
> + cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
> + cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
> + case 6:
> + cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
> + default:
> + cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
> + }
> +}
What for 2 switch-cases here? At first glance, you can join them:
switch (nr_pre_bits) {
case 7:
cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
case 6:
cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
default:
cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
}
> +
> +void __hyp_text __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if;
> + u64 val;
> + u32 nr_pre_bits;
> +
> + vcpu = kern_hyp_va(vcpu);
> + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_pre_bits = vtr_to_nr_pre_bits(val);
> +
> + switch (nr_pre_bits) {
> + case 7:
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
> + case 6:
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
> + default:
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
> + }
> +
> + switch (nr_pre_bits) {
> + case 7:
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
> + case 6:
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
> + default:
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
> + }
And here
Thanks,
Yury
> +}
> +
> void __hyp_text __vgic_v3_init_lrs(void)
> {
> int max_lr_idx = vtr_to_max_lr_idx(read_gicreg(ICH_VTR_EL2));
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index df7e03b..c04c3475 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -428,7 +428,6 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> {
> struct kvm *kvm = vcpu->kvm;
> struct vgic_dist *vgic = &kvm->arch.vgic;
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> void __iomem *base = vgic->vctrl_base;
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
>
> @@ -436,11 +435,8 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> return;
>
> if (used_lrs) {
> - cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> save_lrs(vcpu, base);
> writel_relaxed(0, base + GICH_HCR);
> - } else {
> - cpu_if->vgic_apr = 0;
> }
> }
>
> @@ -458,7 +454,6 @@ void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
>
> if (used_lrs) {
> writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> - writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> for (i = 0; i < used_lrs; i++) {
> writel_relaxed(cpu_if->vgic_lr[i],
> base + GICH_LR0 + (i * 4));
> @@ -472,6 +467,7 @@ void vgic_v2_load(struct kvm_vcpu *vcpu)
> struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
>
> writel_relaxed(cpu_if->vgic_vmcr, vgic->vctrl_base + GICH_VMCR);
> + writel_relaxed(cpu_if->vgic_apr, vgic->vctrl_base + GICH_APR);
> }
>
> void vgic_v2_put(struct kvm_vcpu *vcpu)
> @@ -480,4 +476,5 @@ void vgic_v2_put(struct kvm_vcpu *vcpu)
> struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
>
> cpu_if->vgic_vmcr = readl_relaxed(vgic->vctrl_base + GICH_VMCR);
> + cpu_if->vgic_apr = readl_relaxed(vgic->vctrl_base + GICH_APR);
> }
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 0900649..df3a222 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -16,6 +16,7 @@
> #include <linux/kvm.h>
> #include <linux/kvm_host.h>
> #include <kvm/arm_vgic.h>
> +#include <asm/kvm_hyp.h>
> #include <asm/kvm_mmu.h>
> #include <asm/kvm_asm.h>
>
> @@ -544,6 +545,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
> */
> if (likely(cpu_if->vgic_sre))
> kvm_call_hyp(__vgic_v3_write_vmcr, cpu_if->vgic_vmcr);
> +
> + kvm_call_hyp(__vgic_v3_restore_aprs, vcpu);
> }
>
> void vgic_v3_put(struct kvm_vcpu *vcpu)
> @@ -552,4 +555,6 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
>
> if (likely(cpu_if->vgic_sre))
> cpu_if->vgic_vmcr = kvm_call_hyp(__vgic_v3_read_vmcr);
> +
> + kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
> }
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 36/37] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
@ 2017-11-26 15:09 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-26 15:09 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:40PM +0200, Christoffer Dall wrote:
> The APRs can only have bits set when the guest acknowledges an interrupt
> in the LR and can only have a bit cleared when the guest EOIs an
> interrupt in the LR. Therefore, if we have no LRs with any
> pending/active interrupts, the APR cannot change value and there is no
> need to clear it on every exit from the VM (hint: it will have already
> been cleared when we exited the guest the last time with the LRs all
> EOIed).
>
> The only case we need to take care of is when we migrate the VCPU away
> from a CPU or migrate a new VCPU onto a CPU, or when we return to
> userspace to capture the state of the VCPU for migration. To make sure
> this works, factor out the APR save/restore functionality into separate
> functions called from the VCPU (and by extension VGIC) put/load hooks.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> virt/kvm/arm/hyp/vgic-v3-sr.c | 123 +++++++++++++++++++++------------------
> virt/kvm/arm/vgic/vgic-v2.c | 7 +--
> virt/kvm/arm/vgic/vgic-v3.c | 5 ++
> 5 files changed, 77 insertions(+), 62 deletions(-)
[...]
> @@ -361,6 +304,72 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> ICC_SRE_EL2);
> }
>
> +void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if;
> + u64 val;
> + u32 nr_pre_bits;
> +
> + vcpu = kern_hyp_va(vcpu);
> + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_pre_bits = vtr_to_nr_pre_bits(val);
> +
> + switch (nr_pre_bits) {
> + case 7:
> + cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
> + cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
> + case 6:
> + cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
> + default:
> + cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
> + }
> +
> + switch (nr_pre_bits) {
> + case 7:
> + cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
> + cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
> + case 6:
> + cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
> + default:
> + cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
> + }
> +}
What for 2 switch-cases here? At first glance, you can join them:
switch (nr_pre_bits) {
case 7:
cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
case 6:
cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
default:
cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
}
> +
> +void __hyp_text __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if;
> + u64 val;
> + u32 nr_pre_bits;
> +
> + vcpu = kern_hyp_va(vcpu);
> + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_pre_bits = vtr_to_nr_pre_bits(val);
> +
> + switch (nr_pre_bits) {
> + case 7:
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
> + case 6:
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
> + default:
> + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
> + }
> +
> + switch (nr_pre_bits) {
> + case 7:
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
> + case 6:
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
> + default:
> + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
> + }
And here
Thanks,
Yury
> +}
> +
> void __hyp_text __vgic_v3_init_lrs(void)
> {
> int max_lr_idx = vtr_to_max_lr_idx(read_gicreg(ICH_VTR_EL2));
> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
> index df7e03b..c04c3475 100644
> --- a/virt/kvm/arm/vgic/vgic-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-v2.c
> @@ -428,7 +428,6 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> {
> struct kvm *kvm = vcpu->kvm;
> struct vgic_dist *vgic = &kvm->arch.vgic;
> - struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> void __iomem *base = vgic->vctrl_base;
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
>
> @@ -436,11 +435,8 @@ void vgic_v2_save_state(struct kvm_vcpu *vcpu)
> return;
>
> if (used_lrs) {
> - cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> save_lrs(vcpu, base);
> writel_relaxed(0, base + GICH_HCR);
> - } else {
> - cpu_if->vgic_apr = 0;
> }
> }
>
> @@ -458,7 +454,6 @@ void vgic_v2_restore_state(struct kvm_vcpu *vcpu)
>
> if (used_lrs) {
> writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> - writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> for (i = 0; i < used_lrs; i++) {
> writel_relaxed(cpu_if->vgic_lr[i],
> base + GICH_LR0 + (i * 4));
> @@ -472,6 +467,7 @@ void vgic_v2_load(struct kvm_vcpu *vcpu)
> struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
>
> writel_relaxed(cpu_if->vgic_vmcr, vgic->vctrl_base + GICH_VMCR);
> + writel_relaxed(cpu_if->vgic_apr, vgic->vctrl_base + GICH_APR);
> }
>
> void vgic_v2_put(struct kvm_vcpu *vcpu)
> @@ -480,4 +476,5 @@ void vgic_v2_put(struct kvm_vcpu *vcpu)
> struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
>
> cpu_if->vgic_vmcr = readl_relaxed(vgic->vctrl_base + GICH_VMCR);
> + cpu_if->vgic_apr = readl_relaxed(vgic->vctrl_base + GICH_APR);
> }
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 0900649..df3a222 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -16,6 +16,7 @@
> #include <linux/kvm.h>
> #include <linux/kvm_host.h>
> #include <kvm/arm_vgic.h>
> +#include <asm/kvm_hyp.h>
> #include <asm/kvm_mmu.h>
> #include <asm/kvm_asm.h>
>
> @@ -544,6 +545,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
> */
> if (likely(cpu_if->vgic_sre))
> kvm_call_hyp(__vgic_v3_write_vmcr, cpu_if->vgic_vmcr);
> +
> + kvm_call_hyp(__vgic_v3_restore_aprs, vcpu);
> }
>
> void vgic_v3_put(struct kvm_vcpu *vcpu)
> @@ -552,4 +555,6 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
>
> if (likely(cpu_if->vgic_sre))
> cpu_if->vgic_vmcr = kvm_call_hyp(__vgic_v3_read_vmcr);
> +
> + kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
> }
> --
> 2.9.0
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 36/37] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
2017-11-26 15:09 ` Yury Norov
@ 2017-11-26 19:55 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:55 UTC (permalink / raw)
To: Yury Norov
Cc: Christoffer Dall, kvmarm, linux-arm-kernel, kvm, Marc Zyngier,
Shih-Wei Li
On Sun, Nov 26, 2017 at 06:09:25PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:40PM +0200, Christoffer Dall wrote:
> > The APRs can only have bits set when the guest acknowledges an interrupt
> > in the LR and can only have a bit cleared when the guest EOIs an
> > interrupt in the LR. Therefore, if we have no LRs with any
> > pending/active interrupts, the APR cannot change value and there is no
> > need to clear it on every exit from the VM (hint: it will have already
> > been cleared when we exited the guest the last time with the LRs all
> > EOIed).
> >
> > The only case we need to take care of is when we migrate the VCPU away
> > from a CPU or migrate a new VCPU onto a CPU, or when we return to
> > userspace to capture the state of the VCPU for migration. To make sure
> > this works, factor out the APR save/restore functionality into separate
> > functions called from the VCPU (and by extension VGIC) put/load hooks.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > virt/kvm/arm/hyp/vgic-v3-sr.c | 123 +++++++++++++++++++++------------------
> > virt/kvm/arm/vgic/vgic-v2.c | 7 +--
> > virt/kvm/arm/vgic/vgic-v3.c | 5 ++
> > 5 files changed, 77 insertions(+), 62 deletions(-)
>
> [...]
>
> > @@ -361,6 +304,72 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> > ICC_SRE_EL2);
> > }
> >
> > +void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
> > +{
> > + struct vgic_v3_cpu_if *cpu_if;
> > + u64 val;
> > + u32 nr_pre_bits;
> > +
> > + vcpu = kern_hyp_va(vcpu);
> > + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > +
> > + val = read_gicreg(ICH_VTR_EL2);
> > + nr_pre_bits = vtr_to_nr_pre_bits(val);
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
> > + cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
> > + case 6:
> > + cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
> > + default:
> > + cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
> > + }
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
> > + cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
> > + case 6:
> > + cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
> > + default:
> > + cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
> > + }
> > +}
>
> What for 2 switch-cases here? At first glance, you can join them:
> switch (nr_pre_bits) {
> case 7:
> cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
> cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
> cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
> cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
> case 6:
> cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
> cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
> default:
> cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
> cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
> }
>
> > +
> > +void __hyp_text __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu)
> > +{
> > + struct vgic_v3_cpu_if *cpu_if;
> > + u64 val;
> > + u32 nr_pre_bits;
> > +
> > + vcpu = kern_hyp_va(vcpu);
> > + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > +
> > + val = read_gicreg(ICH_VTR_EL2);
> > + nr_pre_bits = vtr_to_nr_pre_bits(val);
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
> > + case 6:
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
> > + default:
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
> > + }
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
> > + case 6:
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
> > + default:
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
> > + }
>
> And here
>
Sure.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 36/37] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load
@ 2017-11-26 19:55 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-11-26 19:55 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 26, 2017 at 06:09:25PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:40PM +0200, Christoffer Dall wrote:
> > The APRs can only have bits set when the guest acknowledges an interrupt
> > in the LR and can only have a bit cleared when the guest EOIs an
> > interrupt in the LR. Therefore, if we have no LRs with any
> > pending/active interrupts, the APR cannot change value and there is no
> > need to clear it on every exit from the VM (hint: it will have already
> > been cleared when we exited the guest the last time with the LRs all
> > EOIed).
> >
> > The only case we need to take care of is when we migrate the VCPU away
> > from a CPU or migrate a new VCPU onto a CPU, or when we return to
> > userspace to capture the state of the VCPU for migration. To make sure
> > this works, factor out the APR save/restore functionality into separate
> > functions called from the VCPU (and by extension VGIC) put/load hooks.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > virt/kvm/arm/hyp/vgic-v3-sr.c | 123 +++++++++++++++++++++------------------
> > virt/kvm/arm/vgic/vgic-v2.c | 7 +--
> > virt/kvm/arm/vgic/vgic-v3.c | 5 ++
> > 5 files changed, 77 insertions(+), 62 deletions(-)
>
> [...]
>
> > @@ -361,6 +304,72 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> > ICC_SRE_EL2);
> > }
> >
> > +void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
> > +{
> > + struct vgic_v3_cpu_if *cpu_if;
> > + u64 val;
> > + u32 nr_pre_bits;
> > +
> > + vcpu = kern_hyp_va(vcpu);
> > + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > +
> > + val = read_gicreg(ICH_VTR_EL2);
> > + nr_pre_bits = vtr_to_nr_pre_bits(val);
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
> > + cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
> > + case 6:
> > + cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
> > + default:
> > + cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
> > + }
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
> > + cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
> > + case 6:
> > + cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
> > + default:
> > + cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
> > + }
> > +}
>
> What for 2 switch-cases here? At first glance, you can join them:
> switch (nr_pre_bits) {
> case 7:
> cpu_if->vgic_ap0r[3] = __vgic_v3_read_ap0rn(3);
> cpu_if->vgic_ap0r[2] = __vgic_v3_read_ap0rn(2);
> cpu_if->vgic_ap1r[3] = __vgic_v3_read_ap1rn(3);
> cpu_if->vgic_ap1r[2] = __vgic_v3_read_ap1rn(2);
> case 6:
> cpu_if->vgic_ap0r[1] = __vgic_v3_read_ap0rn(1);
> cpu_if->vgic_ap1r[1] = __vgic_v3_read_ap1rn(1);
> default:
> cpu_if->vgic_ap1r[0] = __vgic_v3_read_ap1rn(0);
> cpu_if->vgic_ap0r[0] = __vgic_v3_read_ap0rn(0);
> }
>
> > +
> > +void __hyp_text __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu)
> > +{
> > + struct vgic_v3_cpu_if *cpu_if;
> > + u64 val;
> > + u32 nr_pre_bits;
> > +
> > + vcpu = kern_hyp_va(vcpu);
> > + cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > +
> > + val = read_gicreg(ICH_VTR_EL2);
> > + nr_pre_bits = vtr_to_nr_pre_bits(val);
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[3], 3);
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[2], 2);
> > + case 6:
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[1], 1);
> > + default:
> > + __vgic_v3_write_ap0rn(cpu_if->vgic_ap0r[0], 0);
> > + }
> > +
> > + switch (nr_pre_bits) {
> > + case 7:
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[3], 3);
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[2], 2);
> > + case 6:
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[1], 1);
> > + default:
> > + __vgic_v3_write_ap1rn(cpu_if->vgic_ap1r[0], 0);
> > + }
>
> And here
>
Sure.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread
* [PATCH 37/37] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-10-12 10:41 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: kvmarm, linux-arm-kernel; +Cc: Marc Zyngier, Shih-Wei Li, kvm
We can finally get completely rid of any calls to the VGICv3
save/restore functions when the AP lists are empty on VHE systems. This
requires carefully factoring out trap configuration from saving and
restoring state, and carefully choosing what to do on the VHE and
non-VHE path.
One of the challenges is that we cannot save/restore the VMCR lazily
because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
up being delivered as FIQ.
To solve this problem, and still provide fast performance in the fast
path of exiting a VM when no interrupts are pending (which also
optimized the latency for actually delivering virtual interrupts coming
from physical interrupts), we orchestrate a dance of only doing the
activate/deactivate traps in vgic load/put for VHE systems (which can
have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
configuration on every round-trip on non-VHE systems.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_hyp.h | 2 +
arch/arm/kvm/hyp/switch.c | 8 ++-
arch/arm64/include/asm/kvm_hyp.h | 2 +
arch/arm64/kvm/hyp/switch.c | 8 ++-
virt/kvm/arm/hyp/vgic-v3-sr.c | 116 +++++++++++++++++++++++++--------------
virt/kvm/arm/vgic/vgic-v3.c | 6 ++
virt/kvm/arm/vgic/vgic.c | 7 +--
7 files changed, 101 insertions(+), 48 deletions(-)
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index b3dd4f4..d01676e 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -109,6 +109,8 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_activate_traps(struct kvm_vcpu *vcpu);
+void __vgic_v3_deactivate_traps(struct kvm_vcpu *vcpu);
void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index 0d834f8..6b487c2 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -89,14 +89,18 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
__vgic_v3_save_state(vcpu);
+ __vgic_v3_deactivate_traps(vcpu);
+ }
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
+ __vgic_v3_activate_traps(vcpu);
__vgic_v3_restore_state(vcpu);
+ }
}
static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 62872ce..1c21a70 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -125,6 +125,8 @@ int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_activate_traps(struct kvm_vcpu *vcpu);
+void __vgic_v3_deactivate_traps(struct kvm_vcpu *vcpu);
void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 0cdf6ae..42e6d56 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -184,14 +184,18 @@ static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
__vgic_v3_save_state(vcpu);
+ __vgic_v3_deactivate_traps(vcpu);
+ }
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
+ __vgic_v3_activate_traps(vcpu);
__vgic_v3_restore_state(vcpu);
+ }
}
static bool __hyp_text __true_value(void)
diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index ed5da75..34d71d2 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -208,15 +208,15 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
- u64 val;
/*
* Make sure stores to the GIC via the memory mapped interface
- * are now visible to the system register interface.
+ * are now visible to the system register interface when reading the
+ * LRs, and when reading back the VMCR on non-VHE systems.
*/
- if (!cpu_if->vgic_sre) {
- dsb(st);
- cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
+ if (used_lrs || !has_vhe()) {
+ if (!cpu_if->vgic_sre)
+ dsb(st);
}
if (used_lrs) {
@@ -225,7 +225,7 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
elrsr = read_gicreg(ICH_ELSR_EL2);
- write_gicreg(0, ICH_HCR_EL2);
+ write_gicreg(cpu_if->vgic_hcr & ~ICH_HCR_EN, ICH_HCR_EL2);
for (i = 0; i < used_lrs; i++) {
if (elrsr & (1 << i))
@@ -235,18 +235,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
__gic_v3_set_lr(0, i);
}
- } else {
- if (static_branch_unlikely(&vgic_v3_cpuif_trap))
- write_gicreg(0, ICH_HCR_EL2);
- }
-
- val = read_gicreg(ICC_SRE_EL2);
- write_gicreg(val | ICC_SRE_EL2_ENABLE, ICC_SRE_EL2);
-
- if (!cpu_if->vgic_sre) {
- /* Make sure ENABLE is set at EL2 before setting SRE at EL1 */
- isb();
- write_gicreg(1, ICC_SRE_EL1);
}
}
@@ -256,6 +244,31 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
int i;
+ if (used_lrs) {
+ write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+
+ for (i = 0; i < used_lrs; i++)
+ __gic_v3_set_lr(cpu_if->vgic_lr[i], i);
+ }
+
+ /*
+ * Ensure that writes to the LRs, and on non-VHE systems ensure that
+ * the write to the VMCR in __vgic_v3_activate_traps(), will have
+ * reached the (re)distributors. This ensure the guest will read the
+ * correct values from the memory-mapped interface.
+ */
+ if (used_lrs || !has_vhe()) {
+ if (!cpu_if->vgic_sre) {
+ isb();
+ dsb(sy);
+ }
+ }
+}
+
+void __hyp_text __vgic_v3_activate_traps(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+
/*
* VFIQEn is RES1 if ICC_SRE_EL1.SRE is 1. This causes a
* Group0 interrupt (as generated in GICv2 mode) to be
@@ -263,45 +276,68 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
* consequences. So we must make sure that ICC_SRE_EL1 has
* been actually programmed with the value we want before
* starting to mess with the rest of the GIC, and VMCR_EL2 in
- * particular.
+ * particular. This logic must be called before
+ * __vgic_v3_restore_state().
*/
if (!cpu_if->vgic_sre) {
write_gicreg(0, ICC_SRE_EL1);
isb();
write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
- }
- if (used_lrs) {
- write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
- for (i = 0; i < used_lrs; i++)
- __gic_v3_set_lr(cpu_if->vgic_lr[i], i);
- } else {
- /*
- * If we need to trap system registers, we must write
- * ICH_HCR_EL2 anyway, even if no interrupts are being
- * injected,
- */
- if (static_branch_unlikely(&vgic_v3_cpuif_trap))
- write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+ if (has_vhe()) {
+ /*
+ * Ensure that the write to the VMCR will have reached
+ * the (re)distributors. This ensure the guest will
+ * read the correct values from the memory-mapped
+ * interface.
+ */
+ isb();
+ dsb(sy);
+ }
}
/*
- * Ensures that the above will have reached the
- * (re)distributors. This ensure the guest will read the
- * correct values from the memory-mapped interface.
+ * Prevent the guest from touching the GIC system registers if
+ * SRE isn't enabled for GICv3 emulation.
+ */
+ write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
+ ICC_SRE_EL2);
+
+ /*
+ * If we need to trap system registers, we must write
+ * ICH_HCR_EL2 anyway, even if no interrupts are being
+ * injected,
*/
+ if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+ write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+}
+
+void __hyp_text __vgic_v3_deactivate_traps(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ u64 val;
+
if (!cpu_if->vgic_sre) {
+ cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
+ }
+
+
+ val = read_gicreg(ICC_SRE_EL2);
+ write_gicreg(val | ICC_SRE_EL2_ENABLE, ICC_SRE_EL2);
+
+ if (!cpu_if->vgic_sre) {
+ /* Make sure ENABLE is set at EL2 before setting SRE at EL1 */
isb();
- dsb(sy);
+ write_gicreg(1, ICC_SRE_EL1);
}
/*
- * Prevent the guest from touching the GIC system registers if
- * SRE isn't enabled for GICv3 emulation.
+ * If we were trapping system registers, we enabled the VGIC even if
+ * no interrupts were being injected, and we disable it again here.
*/
- write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
- ICC_SRE_EL2);
+ if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+ write_gicreg(0, ICH_HCR_EL2);
}
void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index df3a222..964c647 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -547,6 +547,9 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
kvm_call_hyp(__vgic_v3_write_vmcr, cpu_if->vgic_vmcr);
kvm_call_hyp(__vgic_v3_restore_aprs, vcpu);
+
+ if (has_vhe())
+ __vgic_v3_activate_traps(vcpu);
}
void vgic_v3_put(struct kvm_vcpu *vcpu)
@@ -557,4 +560,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
cpu_if->vgic_vmcr = kvm_call_hyp(__vgic_v3_read_vmcr);
kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
+
+ if (has_vhe())
+ __vgic_v3_deactivate_traps(vcpu);
}
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index f0072f7..3b8d092 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -698,12 +698,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
- vgic_save_state(vcpu);
-
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
+ vgic_save_state(vcpu);
+
if (vgic_cpu->used_lrs)
vgic_fold_lr_state(vcpu);
vgic_prune_ap_list(vcpu);
@@ -730,7 +730,7 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
* this.
*/
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
- goto out;
+ return;
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
@@ -738,7 +738,6 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
vgic_flush_lr_state(vcpu);
spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);
-out:
vgic_restore_state(vcpu);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* [PATCH 37/37] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
@ 2017-10-12 10:41 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-10-12 10:41 UTC (permalink / raw)
To: linux-arm-kernel
We can finally get completely rid of any calls to the VGICv3
save/restore functions when the AP lists are empty on VHE systems. This
requires carefully factoring out trap configuration from saving and
restoring state, and carefully choosing what to do on the VHE and
non-VHE path.
One of the challenges is that we cannot save/restore the VMCR lazily
because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
up being delivered as FIQ.
To solve this problem, and still provide fast performance in the fast
path of exiting a VM when no interrupts are pending (which also
optimized the latency for actually delivering virtual interrupts coming
from physical interrupts), we orchestrate a dance of only doing the
activate/deactivate traps in vgic load/put for VHE systems (which can
have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
configuration on every round-trip on non-VHE systems.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/include/asm/kvm_hyp.h | 2 +
arch/arm/kvm/hyp/switch.c | 8 ++-
arch/arm64/include/asm/kvm_hyp.h | 2 +
arch/arm64/kvm/hyp/switch.c | 8 ++-
virt/kvm/arm/hyp/vgic-v3-sr.c | 116 +++++++++++++++++++++++++--------------
virt/kvm/arm/vgic/vgic-v3.c | 6 ++
virt/kvm/arm/vgic/vgic.c | 7 +--
7 files changed, 101 insertions(+), 48 deletions(-)
diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h
index b3dd4f4..d01676e 100644
--- a/arch/arm/include/asm/kvm_hyp.h
+++ b/arch/arm/include/asm/kvm_hyp.h
@@ -109,6 +109,8 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_activate_traps(struct kvm_vcpu *vcpu);
+void __vgic_v3_deactivate_traps(struct kvm_vcpu *vcpu);
void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c
index 0d834f8..6b487c2 100644
--- a/arch/arm/kvm/hyp/switch.c
+++ b/arch/arm/kvm/hyp/switch.c
@@ -89,14 +89,18 @@ static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
__vgic_v3_save_state(vcpu);
+ __vgic_v3_deactivate_traps(vcpu);
+ }
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
+ __vgic_v3_activate_traps(vcpu);
__vgic_v3_restore_state(vcpu);
+ }
}
static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 62872ce..1c21a70 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -125,6 +125,8 @@ int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_activate_traps(struct kvm_vcpu *vcpu);
+void __vgic_v3_deactivate_traps(struct kvm_vcpu *vcpu);
void __vgic_v3_save_aprs(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_aprs(struct kvm_vcpu *vcpu);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 0cdf6ae..42e6d56 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -184,14 +184,18 @@ static inline void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
__vgic_v3_save_state(vcpu);
+ __vgic_v3_deactivate_traps(vcpu);
+ }
}
static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
{
- if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
+ __vgic_v3_activate_traps(vcpu);
__vgic_v3_restore_state(vcpu);
+ }
}
static bool __hyp_text __true_value(void)
diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
index ed5da75..34d71d2 100644
--- a/virt/kvm/arm/hyp/vgic-v3-sr.c
+++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
@@ -208,15 +208,15 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
- u64 val;
/*
* Make sure stores to the GIC via the memory mapped interface
- * are now visible to the system register interface.
+ * are now visible to the system register interface when reading the
+ * LRs, and when reading back the VMCR on non-VHE systems.
*/
- if (!cpu_if->vgic_sre) {
- dsb(st);
- cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
+ if (used_lrs || !has_vhe()) {
+ if (!cpu_if->vgic_sre)
+ dsb(st);
}
if (used_lrs) {
@@ -225,7 +225,7 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
elrsr = read_gicreg(ICH_ELSR_EL2);
- write_gicreg(0, ICH_HCR_EL2);
+ write_gicreg(cpu_if->vgic_hcr & ~ICH_HCR_EN, ICH_HCR_EL2);
for (i = 0; i < used_lrs; i++) {
if (elrsr & (1 << i))
@@ -235,18 +235,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
__gic_v3_set_lr(0, i);
}
- } else {
- if (static_branch_unlikely(&vgic_v3_cpuif_trap))
- write_gicreg(0, ICH_HCR_EL2);
- }
-
- val = read_gicreg(ICC_SRE_EL2);
- write_gicreg(val | ICC_SRE_EL2_ENABLE, ICC_SRE_EL2);
-
- if (!cpu_if->vgic_sre) {
- /* Make sure ENABLE is set at EL2 before setting SRE at EL1 */
- isb();
- write_gicreg(1, ICC_SRE_EL1);
}
}
@@ -256,6 +244,31 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
int i;
+ if (used_lrs) {
+ write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+
+ for (i = 0; i < used_lrs; i++)
+ __gic_v3_set_lr(cpu_if->vgic_lr[i], i);
+ }
+
+ /*
+ * Ensure that writes to the LRs, and on non-VHE systems ensure that
+ * the write to the VMCR in __vgic_v3_activate_traps(), will have
+ * reached the (re)distributors. This ensure the guest will read the
+ * correct values from the memory-mapped interface.
+ */
+ if (used_lrs || !has_vhe()) {
+ if (!cpu_if->vgic_sre) {
+ isb();
+ dsb(sy);
+ }
+ }
+}
+
+void __hyp_text __vgic_v3_activate_traps(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+
/*
* VFIQEn is RES1 if ICC_SRE_EL1.SRE is 1. This causes a
* Group0 interrupt (as generated in GICv2 mode) to be
@@ -263,45 +276,68 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
* consequences. So we must make sure that ICC_SRE_EL1 has
* been actually programmed with the value we want before
* starting to mess with the rest of the GIC, and VMCR_EL2 in
- * particular.
+ * particular. This logic must be called before
+ * __vgic_v3_restore_state().
*/
if (!cpu_if->vgic_sre) {
write_gicreg(0, ICC_SRE_EL1);
isb();
write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
- }
- if (used_lrs) {
- write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
- for (i = 0; i < used_lrs; i++)
- __gic_v3_set_lr(cpu_if->vgic_lr[i], i);
- } else {
- /*
- * If we need to trap system registers, we must write
- * ICH_HCR_EL2 anyway, even if no interrupts are being
- * injected,
- */
- if (static_branch_unlikely(&vgic_v3_cpuif_trap))
- write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+ if (has_vhe()) {
+ /*
+ * Ensure that the write to the VMCR will have reached
+ * the (re)distributors. This ensure the guest will
+ * read the correct values from the memory-mapped
+ * interface.
+ */
+ isb();
+ dsb(sy);
+ }
}
/*
- * Ensures that the above will have reached the
- * (re)distributors. This ensure the guest will read the
- * correct values from the memory-mapped interface.
+ * Prevent the guest from touching the GIC system registers if
+ * SRE isn't enabled for GICv3 emulation.
+ */
+ write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
+ ICC_SRE_EL2);
+
+ /*
+ * If we need to trap system registers, we must write
+ * ICH_HCR_EL2 anyway, even if no interrupts are being
+ * injected,
*/
+ if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+ write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+}
+
+void __hyp_text __vgic_v3_deactivate_traps(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ u64 val;
+
if (!cpu_if->vgic_sre) {
+ cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
+ }
+
+
+ val = read_gicreg(ICC_SRE_EL2);
+ write_gicreg(val | ICC_SRE_EL2_ENABLE, ICC_SRE_EL2);
+
+ if (!cpu_if->vgic_sre) {
+ /* Make sure ENABLE is set at EL2 before setting SRE at EL1 */
isb();
- dsb(sy);
+ write_gicreg(1, ICC_SRE_EL1);
}
/*
- * Prevent the guest from touching the GIC system registers if
- * SRE isn't enabled for GICv3 emulation.
+ * If we were trapping system registers, we enabled the VGIC even if
+ * no interrupts were being injected, and we disable it again here.
*/
- write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
- ICC_SRE_EL2);
+ if (static_branch_unlikely(&vgic_v3_cpuif_trap))
+ write_gicreg(0, ICH_HCR_EL2);
}
void __hyp_text __vgic_v3_save_aprs(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index df3a222..964c647 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -547,6 +547,9 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
kvm_call_hyp(__vgic_v3_write_vmcr, cpu_if->vgic_vmcr);
kvm_call_hyp(__vgic_v3_restore_aprs, vcpu);
+
+ if (has_vhe())
+ __vgic_v3_activate_traps(vcpu);
}
void vgic_v3_put(struct kvm_vcpu *vcpu)
@@ -557,4 +560,7 @@ void vgic_v3_put(struct kvm_vcpu *vcpu)
cpu_if->vgic_vmcr = kvm_call_hyp(__vgic_v3_read_vmcr);
kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
+
+ if (has_vhe())
+ __vgic_v3_deactivate_traps(vcpu);
}
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index f0072f7..3b8d092 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -698,12 +698,12 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
- vgic_save_state(vcpu);
-
/* An empty ap_list_head implies used_lrs == 0 */
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
return;
+ vgic_save_state(vcpu);
+
if (vgic_cpu->used_lrs)
vgic_fold_lr_state(vcpu);
vgic_prune_ap_list(vcpu);
@@ -730,7 +730,7 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
* this.
*/
if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
- goto out;
+ return;
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
@@ -738,7 +738,6 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
vgic_flush_lr_state(vcpu);
spin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);
-out:
vgic_restore_state(vcpu);
}
--
2.9.0
^ permalink raw reply related [flat|nested] 254+ messages in thread* Re: [PATCH 37/37] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
2017-10-12 10:41 ` Christoffer Dall
@ 2017-11-30 18:33 ` Yury Norov
-1 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-30 18:33 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, linux-arm-kernel, kvm, Marc Zyngier, Shih-Wei Li
On Thu, Oct 12, 2017 at 12:41:41PM +0200, Christoffer Dall wrote:
> We can finally get completely rid of any calls to the VGICv3
> save/restore functions when the AP lists are empty on VHE systems. This
> requires carefully factoring out trap configuration from saving and
> restoring state, and carefully choosing what to do on the VHE and
> non-VHE path.
>
> One of the challenges is that we cannot save/restore the VMCR lazily
> because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
> emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
> up being delivered as FIQ.
>
> To solve this problem, and still provide fast performance in the fast
> path of exiting a VM when no interrupts are pending (which also
> optimized the latency for actually delivering virtual interrupts coming
> from physical interrupts), we orchestrate a dance of only doing the
> activate/deactivate traps in vgic load/put for VHE systems (which can
> have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
> configuration on every round-trip on non-VHE systems.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_hyp.h | 2 +
> arch/arm/kvm/hyp/switch.c | 8 ++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/kvm/hyp/switch.c | 8 ++-
> virt/kvm/arm/hyp/vgic-v3-sr.c | 116 +++++++++++++++++++++++++--------------
> virt/kvm/arm/vgic/vgic-v3.c | 6 ++
> virt/kvm/arm/vgic/vgic.c | 7 +--
> 7 files changed, 101 insertions(+), 48 deletions(-)
[...]
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index ed5da75..34d71d2 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -208,15 +208,15 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> {
> struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> - u64 val;
>
> /*
> * Make sure stores to the GIC via the memory mapped interface
> - * are now visible to the system register interface.
> + * are now visible to the system register interface when reading the
> + * LRs, and when reading back the VMCR on non-VHE systems.
> */
> - if (!cpu_if->vgic_sre) {
> - dsb(st);
> - cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> + if (used_lrs || !has_vhe()) {
> + if (!cpu_if->vgic_sre)
> + dsb(st);
> }
Nit:
if ((used_lrs || !has_vhe()) && !cpu_if->vgic_sre)
dsb(st);
>
> if (used_lrs) {
> @@ -225,7 +225,7 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>
> elrsr = read_gicreg(ICH_ELSR_EL2);
>
> - write_gicreg(0, ICH_HCR_EL2);
> + write_gicreg(cpu_if->vgic_hcr & ~ICH_HCR_EN, ICH_HCR_EL2);
>
> for (i = 0; i < used_lrs; i++) {
> if (elrsr & (1 << i))
> @@ -235,18 +235,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>
> __gic_v3_set_lr(0, i);
> }
> - } else {
> - if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> - write_gicreg(0, ICH_HCR_EL2);
> - }
> -
> - val = read_gicreg(ICC_SRE_EL2);
> - write_gicreg(val | ICC_SRE_EL2_ENABLE, ICC_SRE_EL2);
> -
> - if (!cpu_if->vgic_sre) {
> - /* Make sure ENABLE is set at EL2 before setting SRE at EL1 */
> - isb();
> - write_gicreg(1, ICC_SRE_EL1);
> }
> }
>
> @@ -256,6 +244,31 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> int i;
>
> + if (used_lrs) {
> + write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
> +
> + for (i = 0; i < used_lrs; i++)
> + __gic_v3_set_lr(cpu_if->vgic_lr[i], i);
> + }
> +
> + /*
> + * Ensure that writes to the LRs, and on non-VHE systems ensure that
> + * the write to the VMCR in __vgic_v3_activate_traps(), will have
> + * reached the (re)distributors. This ensure the guest will read the
> + * correct values from the memory-mapped interface.
> + */
> + if (used_lrs || !has_vhe()) {
> + if (!cpu_if->vgic_sre) {
> + isb();
> + dsb(sy);
> + }
> + }
And here
> +}
> +
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 37/37] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
@ 2017-11-30 18:33 ` Yury Norov
0 siblings, 0 replies; 254+ messages in thread
From: Yury Norov @ 2017-11-30 18:33 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 12, 2017 at 12:41:41PM +0200, Christoffer Dall wrote:
> We can finally get completely rid of any calls to the VGICv3
> save/restore functions when the AP lists are empty on VHE systems. This
> requires carefully factoring out trap configuration from saving and
> restoring state, and carefully choosing what to do on the VHE and
> non-VHE path.
>
> One of the challenges is that we cannot save/restore the VMCR lazily
> because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
> emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
> up being delivered as FIQ.
>
> To solve this problem, and still provide fast performance in the fast
> path of exiting a VM when no interrupts are pending (which also
> optimized the latency for actually delivering virtual interrupts coming
> from physical interrupts), we orchestrate a dance of only doing the
> activate/deactivate traps in vgic load/put for VHE systems (which can
> have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
> configuration on every round-trip on non-VHE systems.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> ---
> arch/arm/include/asm/kvm_hyp.h | 2 +
> arch/arm/kvm/hyp/switch.c | 8 ++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/kvm/hyp/switch.c | 8 ++-
> virt/kvm/arm/hyp/vgic-v3-sr.c | 116 +++++++++++++++++++++++++--------------
> virt/kvm/arm/vgic/vgic-v3.c | 6 ++
> virt/kvm/arm/vgic/vgic.c | 7 +--
> 7 files changed, 101 insertions(+), 48 deletions(-)
[...]
> diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> index ed5da75..34d71d2 100644
> --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> @@ -208,15 +208,15 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> {
> struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> - u64 val;
>
> /*
> * Make sure stores to the GIC via the memory mapped interface
> - * are now visible to the system register interface.
> + * are now visible to the system register interface when reading the
> + * LRs, and when reading back the VMCR on non-VHE systems.
> */
> - if (!cpu_if->vgic_sre) {
> - dsb(st);
> - cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> + if (used_lrs || !has_vhe()) {
> + if (!cpu_if->vgic_sre)
> + dsb(st);
> }
Nit:
if ((used_lrs || !has_vhe()) && !cpu_if->vgic_sre)
dsb(st);
>
> if (used_lrs) {
> @@ -225,7 +225,7 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>
> elrsr = read_gicreg(ICH_ELSR_EL2);
>
> - write_gicreg(0, ICH_HCR_EL2);
> + write_gicreg(cpu_if->vgic_hcr & ~ICH_HCR_EN, ICH_HCR_EL2);
>
> for (i = 0; i < used_lrs; i++) {
> if (elrsr & (1 << i))
> @@ -235,18 +235,6 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>
> __gic_v3_set_lr(0, i);
> }
> - } else {
> - if (static_branch_unlikely(&vgic_v3_cpuif_trap))
> - write_gicreg(0, ICH_HCR_EL2);
> - }
> -
> - val = read_gicreg(ICC_SRE_EL2);
> - write_gicreg(val | ICC_SRE_EL2_ENABLE, ICC_SRE_EL2);
> -
> - if (!cpu_if->vgic_sre) {
> - /* Make sure ENABLE is set at EL2 before setting SRE at EL1 */
> - isb();
> - write_gicreg(1, ICC_SRE_EL1);
> }
> }
>
> @@ -256,6 +244,31 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> int i;
>
> + if (used_lrs) {
> + write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
> +
> + for (i = 0; i < used_lrs; i++)
> + __gic_v3_set_lr(cpu_if->vgic_lr[i], i);
> + }
> +
> + /*
> + * Ensure that writes to the LRs, and on non-VHE systems ensure that
> + * the write to the VMCR in __vgic_v3_activate_traps(), will have
> + * reached the (re)distributors. This ensure the guest will read the
> + * correct values from the memory-mapped interface.
> + */
> + if (used_lrs || !has_vhe()) {
> + if (!cpu_if->vgic_sre) {
> + isb();
> + dsb(sy);
> + }
> + }
And here
> +}
> +
^ permalink raw reply [flat|nested] 254+ messages in thread* Re: [PATCH 37/37] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
2017-11-30 18:33 ` Yury Norov
@ 2017-12-03 20:38 ` Christoffer Dall
-1 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:38 UTC (permalink / raw)
To: Yury Norov; +Cc: kvm, Marc Zyngier, Shih-Wei Li, kvmarm, linux-arm-kernel
Hi Yury,
On Thu, Nov 30, 2017 at 09:33:30PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:41PM +0200, Christoffer Dall wrote:
> > We can finally get completely rid of any calls to the VGICv3
> > save/restore functions when the AP lists are empty on VHE systems. This
> > requires carefully factoring out trap configuration from saving and
> > restoring state, and carefully choosing what to do on the VHE and
> > non-VHE path.
> >
> > One of the challenges is that we cannot save/restore the VMCR lazily
> > because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
> > emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
> > up being delivered as FIQ.
> >
> > To solve this problem, and still provide fast performance in the fast
> > path of exiting a VM when no interrupts are pending (which also
> > optimized the latency for actually delivering virtual interrupts coming
> > from physical interrupts), we orchestrate a dance of only doing the
> > activate/deactivate traps in vgic load/put for VHE systems (which can
> > have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
> > configuration on every round-trip on non-VHE systems.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_hyp.h | 2 +
> > arch/arm/kvm/hyp/switch.c | 8 ++-
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/kvm/hyp/switch.c | 8 ++-
> > virt/kvm/arm/hyp/vgic-v3-sr.c | 116 +++++++++++++++++++++++++--------------
> > virt/kvm/arm/vgic/vgic-v3.c | 6 ++
> > virt/kvm/arm/vgic/vgic.c | 7 +--
> > 7 files changed, 101 insertions(+), 48 deletions(-)
>
> [...]
>
> > diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > index ed5da75..34d71d2 100644
> > --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> > +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > @@ -208,15 +208,15 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > - u64 val;
> >
> > /*
> > * Make sure stores to the GIC via the memory mapped interface
> > - * are now visible to the system register interface.
> > + * are now visible to the system register interface when reading the
> > + * LRs, and when reading back the VMCR on non-VHE systems.
> > */
> > - if (!cpu_if->vgic_sre) {
> > - dsb(st);
> > - cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> > + if (used_lrs || !has_vhe()) {
> > + if (!cpu_if->vgic_sre)
> > + dsb(st);
> > }
>
> Nit:
> if ((used_lrs || !has_vhe()) && !cpu_if->vgic_sre)
> dsb(st);
>
I don't think that's easier to read and I'm not paying for vertical
space, so I'm going to keep it as is.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread* [PATCH 37/37] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs
@ 2017-12-03 20:38 ` Christoffer Dall
0 siblings, 0 replies; 254+ messages in thread
From: Christoffer Dall @ 2017-12-03 20:38 UTC (permalink / raw)
To: linux-arm-kernel
Hi Yury,
On Thu, Nov 30, 2017 at 09:33:30PM +0300, Yury Norov wrote:
> On Thu, Oct 12, 2017 at 12:41:41PM +0200, Christoffer Dall wrote:
> > We can finally get completely rid of any calls to the VGICv3
> > save/restore functions when the AP lists are empty on VHE systems. This
> > requires carefully factoring out trap configuration from saving and
> > restoring state, and carefully choosing what to do on the VHE and
> > non-VHE path.
> >
> > One of the challenges is that we cannot save/restore the VMCR lazily
> > because we can only write the VMCR when ICC_SRE_EL1.SRE is cleared when
> > emulating a GICv2-on-GICv3, since otherwise all Group-0 interrupts end
> > up being delivered as FIQ.
> >
> > To solve this problem, and still provide fast performance in the fast
> > path of exiting a VM when no interrupts are pending (which also
> > optimized the latency for actually delivering virtual interrupts coming
> > from physical interrupts), we orchestrate a dance of only doing the
> > activate/deactivate traps in vgic load/put for VHE systems (which can
> > have ICC_SRE_EL1.SRE cleared when running in the host), and doing the
> > configuration on every round-trip on non-VHE systems.
> >
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> > arch/arm/include/asm/kvm_hyp.h | 2 +
> > arch/arm/kvm/hyp/switch.c | 8 ++-
> > arch/arm64/include/asm/kvm_hyp.h | 2 +
> > arch/arm64/kvm/hyp/switch.c | 8 ++-
> > virt/kvm/arm/hyp/vgic-v3-sr.c | 116 +++++++++++++++++++++++++--------------
> > virt/kvm/arm/vgic/vgic-v3.c | 6 ++
> > virt/kvm/arm/vgic/vgic.c | 7 +--
> > 7 files changed, 101 insertions(+), 48 deletions(-)
>
> [...]
>
> > diff --git a/virt/kvm/arm/hyp/vgic-v3-sr.c b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > index ed5da75..34d71d2 100644
> > --- a/virt/kvm/arm/hyp/vgic-v3-sr.c
> > +++ b/virt/kvm/arm/hyp/vgic-v3-sr.c
> > @@ -208,15 +208,15 @@ void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > {
> > struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > u64 used_lrs = vcpu->arch.vgic_cpu.used_lrs;
> > - u64 val;
> >
> > /*
> > * Make sure stores to the GIC via the memory mapped interface
> > - * are now visible to the system register interface.
> > + * are now visible to the system register interface when reading the
> > + * LRs, and when reading back the VMCR on non-VHE systems.
> > */
> > - if (!cpu_if->vgic_sre) {
> > - dsb(st);
> > - cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> > + if (used_lrs || !has_vhe()) {
> > + if (!cpu_if->vgic_sre)
> > + dsb(st);
> > }
>
> Nit:
> if ((used_lrs || !has_vhe()) && !cpu_if->vgic_sre)
> dsb(st);
>
I don't think that's easier to read and I'm not paying for vertical
space, so I'm going to keep it as is.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 254+ messages in thread