From: Marc Zyngier <maz@kernel.org>
To: kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org
Cc: dave.martin@arm.com, daniel.kiss@arm.com,
Will Deacon <will@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
broonie@kernel.org, kernel-team@android.com
Subject: [PATCH 09/10] KVM: arm64: Save/restore SVE state for nVHE
Date: Tue, 16 Mar 2021 10:13:11 +0000 [thread overview]
Message-ID: <20210316101312.102925-10-maz@kernel.org> (raw)
In-Reply-To: <20210316101312.102925-1-maz@kernel.org>
Implement the SVE save/restore for nVHE, following a similar
logic to that of the VHE implementation:
- the SVE state is switched on trap from EL1 to EL2
- no further changes to ZCR_EL2 occur as long as the guest isn't
preempted or exit to userspace
- on vcpu_put(), ZCR_EL2 is reset to its default value, and ZCR_EL1
restored to the default guest value
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kvm/fpsimd.c | 15 ++++++++--
arch/arm64/kvm/hyp/include/hyp/switch.h | 37 +++++++++----------------
arch/arm64/kvm/hyp/nvhe/switch.c | 4 +--
3 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index b5f95abd23f5..cc6cdea69596 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -121,11 +121,22 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
local_irq_save(flags);
if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) {
- if (guest_has_sve)
+ if (guest_has_sve) {
__vcpu_sys_reg(vcpu, ZCR_EL1) = read_sysreg_el1(SYS_ZCR);
+ /* Restore the VL that was saved when bound to the CPU */
+ if (!has_vhe()) {
+ u64 zcr;
+
+ kvm_call_hyp(__kvm_reset_sve_vq);
+ zcr = vcpu_sve_vq(vcpu) - 1;
+ if (read_sysreg_el1(SYS_ZCR) != zcr)
+ write_sysreg_el1(zcr, SYS_ZCR);
+ }
+ }
+
fpsimd_save_and_flush_cpu_state();
- } else if (host_has_sve) {
+ } else if (has_vhe() && host_has_sve) {
/*
* The FPSIMD/SVE state in the CPU has not been touched, and we
* have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index d34dc220a1ce..777b9bfa4478 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -218,25 +218,19 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
/* Check for an FPSIMD/SVE trap and handle as appropriate */
static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
{
- bool vhe, sve_guest, sve_host;
+ bool sve_guest, sve_host;
u8 esr_ec;
+ u64 reg;
if (!system_supports_fpsimd())
return false;
- /*
- * Currently system_supports_sve() currently implies has_vhe(),
- * so the check is redundant. However, has_vhe() can be determined
- * statically and helps the compiler remove dead code.
- */
- if (has_vhe() && system_supports_sve()) {
+ if (system_supports_sve()) {
sve_guest = vcpu_has_sve(vcpu);
sve_host = vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE;
- vhe = true;
} else {
sve_guest = false;
sve_host = false;
- vhe = has_vhe();
}
esr_ec = kvm_vcpu_trap_get_class(vcpu);
@@ -245,31 +239,26 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
return false;
/* Don't handle SVE traps for non-SVE vcpus here: */
- if (!sve_guest)
- if (esr_ec != ESR_ELx_EC_FP_ASIMD)
- return false;
+ if (!sve_guest && esr_ec != ESR_ELx_EC_FP_ASIMD)
+ return false;
/* Valid trap. Switch the context: */
-
- if (vhe) {
- u64 reg = read_sysreg(cpacr_el1) | CPACR_EL1_FPEN;
-
+ if (has_vhe()) {
+ reg = CPACR_EL1_FPEN;
if (sve_guest)
reg |= CPACR_EL1_ZEN;
- write_sysreg(reg, cpacr_el1);
+ sysreg_clear_set(cpacr_el1, 0, reg);
} else {
- write_sysreg(read_sysreg(cptr_el2) & ~(u64)CPTR_EL2_TFP,
- cptr_el2);
- }
+ reg = CPTR_EL2_TFP;
+ if (sve_guest)
+ reg |= CPTR_EL2_TZ;
+ sysreg_clear_set(cptr_el2, reg, 0);
+ }
isb();
if (vcpu->arch.flags & KVM_ARM64_FP_HOST) {
- /*
- * In the SVE case, VHE is assumed: it is enforced by
- * Kconfig and kvm_arch_init().
- */
if (sve_host)
__hyp_sve_save_host(vcpu);
else
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index f3d0e9eca56c..ef47abcef63f 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -41,9 +41,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
__activate_traps_common(vcpu);
val = CPTR_EL2_DEFAULT;
- val |= CPTR_EL2_TTA | CPTR_EL2_TZ | CPTR_EL2_TAM;
+ val |= CPTR_EL2_TTA | CPTR_EL2_TAM;
if (!update_fp_enabled(vcpu)) {
- val |= CPTR_EL2_TFP;
+ val |= CPTR_EL2_TFP | CPTR_EL2_TZ;
__activate_traps_fpsimd32(vcpu);
}
--
2.29.2
next prev parent reply other threads:[~2021-03-16 10:14 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-16 10:13 [PATCH 00/10] KVM: arm64: Enable SVE support on nVHE systems Marc Zyngier
2021-03-16 10:13 ` [PATCH 01/10] KVM: arm64: Provide KVM's own save/restore SVE primitives Marc Zyngier
2021-03-16 10:31 ` Quentin Perret
2021-03-16 12:17 ` Marc Zyngier
2021-03-17 14:30 ` Will Deacon
2021-03-16 10:13 ` [PATCH 02/10] KVM: arm64: Use {read,write}_sysreg_el1 to access ZCR_EL1 Marc Zyngier
2021-03-17 14:31 ` Will Deacon
2021-03-16 10:13 ` [PATCH 03/10] KVM: arm64: Let vcpu_sve_pffr() handle HYP VAs Marc Zyngier
2021-03-17 14:31 ` Will Deacon
2021-03-16 10:13 ` [PATCH 04/10] KVM: arm64: Introduce vcpu_sve_vq() helper Marc Zyngier
2021-03-17 14:01 ` Will Deacon
2021-03-16 10:13 ` [PATCH 05/10] KVM: arm64: Rework SVE host-save/guest-restore Marc Zyngier
2021-03-17 14:29 ` Will Deacon
2021-03-17 14:54 ` Marc Zyngier
2021-03-16 10:13 ` [PATCH 06/10] KVM: arm64: Map SVE context at EL2 when available Marc Zyngier
2021-03-17 16:01 ` Will Deacon
2021-03-18 8:56 ` Marc Zyngier
2021-03-16 10:13 ` [PATCH 07/10] KVM: arm64: Save guest's ZCR_EL1 before saving the FPSIMD state Marc Zyngier
2021-03-17 17:17 ` Will Deacon
2021-03-17 17:20 ` Will Deacon
2021-03-16 10:13 ` [PATCH 08/10] KVM: arm64: Add a nVHE-specific SVE VQ reset hypercall Marc Zyngier
2021-03-16 10:45 ` Quentin Perret
2021-03-16 12:18 ` Marc Zyngier
2021-03-16 14:24 ` Andrew Scull
2021-03-16 15:00 ` Marc Zyngier
2021-03-16 10:13 ` Marc Zyngier [this message]
2021-03-17 17:57 ` [PATCH 09/10] KVM: arm64: Save/restore SVE state for nVHE Will Deacon
2021-03-18 9:12 ` Marc Zyngier
2021-03-16 10:13 ` [PATCH 10/10] KVM: arm64: Enable SVE support " Marc Zyngier
2021-03-17 18:00 ` Will Deacon
2021-03-18 9:14 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210316101312.102925-10-maz@kernel.org \
--to=maz@kernel.org \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=daniel.kiss@arm.com \
--cc=dave.martin@arm.com \
--cc=james.morse@arm.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kernel-team@android.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).