[PATCH v2 0/5] KVM: arm64: Rework FPSIMD/SVE tracking

[PATCH v2 0/5] KVM: arm64: Rework FPSIMD/SVE tracking

[Marc Zyngier]

This is v2 of this series aiming at simplifying the FP handling.

It recently became apparent that we are mapping each vcpu thread's
thread_info structure at EL2 for the sole purpose of checking on the
TIF_FOREIGN_FPSTATE flag.

Given that this looks like a slightly over-engineered way of sharing a
single bit of information, let's move to a slightly more obvious
implementation by maintaining a vcpu-private shadow flag that
represents the same state.

In the same vein, it appears that the code that deals with saving the
host SVE state when used by the guest can never run, and that's by
construction. This is actually a good thing, because it be guaranteed
to explode on nVHE. Let's get rid of it.

I also take this opportunity to add what looks like a missing, and
nonetheless crucial piece of information to the FPSIMD code regarding
the way KVM (ab)uses the TIF_FOREIGN_FPSTATE.

Lightly tested on an A53 box with a bunch of paranoia instances
running in both host and guests, and more heavily on a FVP to check
the SVE behaviour (using the sve-test selftest running in both host
and guest at the same time).

* From v1 [1]:
  - New patch getting rid of the host SVE save code
  - Reworded the documentation update patch

[1] https://lore.kernel.org/r/20211021151124.3098113-1-maz@kernel.org

Marc Zyngier (5):
  KVM: arm64: Reorder vcpu flag definitions
  KVM: arm64: Get rid of host SVE tracking/saving
  KVM: arm64: Introduce flag shadowing TIF_FOREIGN_FPSTATE
  KVM: arm64: Stop mapping current thread_info at EL2
  arm64/fpsimd: Document the use of TIF_FOREIGN_FPSTATE by KVM

 arch/arm64/include/asm/kvm_host.h       | 29 ++++++++++---------
 arch/arm64/kernel/fpsimd.c              |  6 +++-
 arch/arm64/kvm/arm.c                    |  1 +
 arch/arm64/kvm/fpsimd.c                 | 37 ++++++++++---------------
 arch/arm64/kvm/hyp/include/hyp/switch.h | 30 +++-----------------
 arch/arm64/kvm/hyp/nvhe/switch.c        |  1 -
 arch/arm64/kvm/hyp/vhe/switch.c         |  1 -
 7 files changed, 38 insertions(+), 67 deletions(-)

-- 
2.30.2

[PATCH v2 1/5] KVM: arm64: Reorder vcpu flag definitions

[Marc Zyngier]

The vcpu arch flags are in an interesting, semi random order.
As I have made the mistake of reusing a flag once, let's rework
this in an order that I find a bit less confusing.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index f8be56d5342b..1363c1ff66fb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -416,14 +416,12 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_VCPU_SVE_FINALIZED	(1 << 6) /* SVE config completed */
 #define KVM_ARM64_GUEST_HAS_PTRAUTH	(1 << 7) /* PTRAUTH exposed to guest */
 #define KVM_ARM64_PENDING_EXCEPTION	(1 << 8) /* Exception pending */
+/*
+ * Overlaps with KVM_ARM64_EXCEPT_MASK on purpose so that it can't be
+ * set together with an exception...
+ */
+#define KVM_ARM64_INCREMENT_PC		(1 << 9) /* Increment PC */
 #define KVM_ARM64_EXCEPT_MASK		(7 << 9) /* Target EL/MODE */
-#define KVM_ARM64_DEBUG_STATE_SAVE_SPE	(1 << 12) /* Save SPE context if active  */
-#define KVM_ARM64_DEBUG_STATE_SAVE_TRBE	(1 << 13) /* Save TRBE context if active  */
-
-#define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
-				 KVM_GUESTDBG_USE_SW_BP | \
-				 KVM_GUESTDBG_USE_HW | \
-				 KVM_GUESTDBG_SINGLESTEP)
 /*
  * When KVM_ARM64_PENDING_EXCEPTION is set, KVM_ARM64_EXCEPT_MASK can
  * take the following values:
@@ -441,11 +439,13 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_EXCEPT_AA64_EL1	(0 << 11)
 #define KVM_ARM64_EXCEPT_AA64_EL2	(1 << 11)
 
-/*
- * Overlaps with KVM_ARM64_EXCEPT_MASK on purpose so that it can't be
- * set together with an exception...
- */
-#define KVM_ARM64_INCREMENT_PC		(1 << 9) /* Increment PC */
+#define KVM_ARM64_DEBUG_STATE_SAVE_SPE	(1 << 12) /* Save SPE context if active  */
+#define KVM_ARM64_DEBUG_STATE_SAVE_TRBE	(1 << 13) /* Save TRBE context if active  */
+
+#define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
+				 KVM_GUESTDBG_USE_SW_BP | \
+				 KVM_GUESTDBG_USE_HW | \
+				 KVM_GUESTDBG_SINGLESTEP)
 
 #define vcpu_has_sve(vcpu) (system_supports_sve() &&			\
 			    ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))
-- 
2.30.2

[PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Marc Zyngier]

The SVE host tracking in KVM is pretty involved. It relies on a
set of flags tracking the ownership of the SVE register, as well
as that of the EL0 access.

It is also pretty scary: __hyp_sve_save_host() computes
a thread_struct pointer and obtains a sve_state which gets directly
accessed without further ado, even on nVHE. How can this even work?

The answer to that is that it doesn't, and that this is mostly dead
code. Closer examination shows that on executing a syscall, userspace
loses its SVE state entirely. This is part of the ABI. Another
thing to notice is that although the kernel provides helpers such as
kernel_neon_begin()/end(), they only deal with the FP/NEON state,
and not SVE.

Given that you can only execute a guest as the result of a syscall,
and that the kernel cannot use SVE by itself, it becomes pretty
obvious that there is never any host SVE state to save, and that
this code is only there to increase confusion.

Get rid of the TIF_SVE tracking and host save infrastructure altogether.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_host.h       |  1 -
 arch/arm64/kvm/fpsimd.c                 | 17 +++++-----------
 arch/arm64/kvm/hyp/include/hyp/switch.h | 27 +++----------------------
 3 files changed, 8 insertions(+), 37 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 1363c1ff66fb..e24d960244d9 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -410,7 +410,6 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_DEBUG_DIRTY		(1 << 0)
 #define KVM_ARM64_FP_ENABLED		(1 << 1) /* guest FP regs loaded */
 #define KVM_ARM64_FP_HOST		(1 << 2) /* host FP regs loaded */
-#define KVM_ARM64_HOST_SVE_IN_USE	(1 << 3) /* backup for host TIF_SVE */
 #define KVM_ARM64_HOST_SVE_ENABLED	(1 << 4) /* SVE enabled for EL0 */
 #define KVM_ARM64_GUEST_HAS_SVE		(1 << 5) /* SVE exposed to guest */
 #define KVM_ARM64_VCPU_SVE_FINALIZED	(1 << 6) /* SVE config completed */
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index 5621020b28de..38ca332c10fe 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -73,15 +73,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 {
 	BUG_ON(!current->mm);
+	BUG_ON(test_thread_flag(TIF_SVE));
 
-	vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
-			      KVM_ARM64_HOST_SVE_IN_USE |
-			      KVM_ARM64_HOST_SVE_ENABLED);
+	vcpu->arch.flags &= ~KVM_ARM64_FP_ENABLED;
 	vcpu->arch.flags |= KVM_ARM64_FP_HOST;
 
-	if (test_thread_flag(TIF_SVE))
-		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_IN_USE;
-
 	if (read_sysreg(cpacr_el1) & CPACR_EL1_ZEN_EL0EN)
 		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_ENABLED;
 }
@@ -115,13 +111,11 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 {
 	unsigned long flags;
-	bool host_has_sve = system_supports_sve();
-	bool guest_has_sve = vcpu_has_sve(vcpu);
 
 	local_irq_save(flags);
 
 	if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) {
-		if (guest_has_sve) {
+		if (vcpu_has_sve(vcpu)) {
 			__vcpu_sys_reg(vcpu, ZCR_EL1) = read_sysreg_el1(SYS_ZCR);
 
 			/* Restore the VL that was saved when bound to the CPU */
@@ -131,7 +125,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 		}
 
 		fpsimd_save_and_flush_cpu_state();
-	} else if (has_vhe() && host_has_sve) {
+	} else if (has_vhe() && system_supports_sve()) {
 		/*
 		 * The FPSIMD/SVE state in the CPU has not been touched, and we
 		 * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been
@@ -145,8 +139,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 			sysreg_clear_set(CPACR_EL1, CPACR_EL1_ZEN_EL0EN, 0);
 	}
 
-	update_thread_flag(TIF_SVE,
-			   vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE);
+	update_thread_flag(TIF_SVE, 0);
 
 	local_irq_restore(flags);
 }
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index a0e78a6027be..722dfde7f1aa 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -207,16 +207,6 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
 	return __get_fault_info(esr, &vcpu->arch.fault);
 }
 
-static inline void __hyp_sve_save_host(struct kvm_vcpu *vcpu)
-{
-	struct thread_struct *thread;
-
-	thread = container_of(vcpu->arch.host_fpsimd_state, struct thread_struct,
-			      uw.fpsimd_state);
-
-	__sve_save_state(sve_pffr(thread), &vcpu->arch.host_fpsimd_state->fpsr);
-}
-
 static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 {
 	sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2);
@@ -228,21 +218,14 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 /* Check for an FPSIMD/SVE trap and handle as appropriate */
 static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
 {
-	bool sve_guest, sve_host;
+	bool sve_guest;
 	u8 esr_ec;
 	u64 reg;
 
 	if (!system_supports_fpsimd())
 		return false;
 
-	if (system_supports_sve()) {
-		sve_guest = vcpu_has_sve(vcpu);
-		sve_host = vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE;
-	} else {
-		sve_guest = false;
-		sve_host = false;
-	}
-
+	sve_guest = vcpu_has_sve(vcpu);
 	esr_ec = kvm_vcpu_trap_get_class(vcpu);
 	if (esr_ec != ESR_ELx_EC_FP_ASIMD &&
 	    esr_ec != ESR_ELx_EC_SVE)
@@ -269,11 +252,7 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
 	isb();
 
 	if (vcpu->arch.flags & KVM_ARM64_FP_HOST) {
-		if (sve_host)
-			__hyp_sve_save_host(vcpu);
-		else
-			__fpsimd_save_state(vcpu->arch.host_fpsimd_state);
-
+		__fpsimd_save_state(vcpu->arch.host_fpsimd_state);
 		vcpu->arch.flags &= ~KVM_ARM64_FP_HOST;
 	}
 
-- 
2.30.2

[PATCH v2 3/5] KVM: arm64: Introduce flag shadowing TIF_FOREIGN_FPSTATE

[Marc Zyngier]

We currently have to maintain a mapping the thread_info structure
at EL2 in order to be able to check the TIF_FOREIGN_FPSTATE flag.

In order to eventually get rid of this, start with a vcpu flag that
shadows the thread flag on each entry into the hypervisor.

Reviewed-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_host.h       | 2 ++
 arch/arm64/kvm/arm.c                    | 1 +
 arch/arm64/kvm/fpsimd.c                 | 8 ++++++++
 arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +-
 4 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e24d960244d9..0db523e447a5 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -440,6 +440,7 @@ struct kvm_vcpu_arch {
 
 #define KVM_ARM64_DEBUG_STATE_SAVE_SPE	(1 << 12) /* Save SPE context if active  */
 #define KVM_ARM64_DEBUG_STATE_SAVE_TRBE	(1 << 13) /* Save TRBE context if active  */
+#define KVM_ARM64_FP_FOREIGN_FPSTATE	(1 << 14)
 
 #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
 				 KVM_GUESTDBG_USE_SW_BP | \
@@ -735,6 +736,7 @@ long kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm,
 /* Guest/host FPSIMD coordination helpers */
 int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
+void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu);
 
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index fe102cd2e518..4afb84e994e2 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -840,6 +840,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 		}
 
 		kvm_arm_setup_debug(vcpu);
+		kvm_arch_vcpu_ctxflush_fp(vcpu);
 
 		/**************************************************************
 		 * Enter the guest
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index 38ca332c10fe..f464db2e9e47 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -82,6 +82,14 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_ENABLED;
 }
 
+void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu)
+{
+	if (test_thread_flag(TIF_FOREIGN_FPSTATE))
+		vcpu->arch.flags |= KVM_ARM64_FP_FOREIGN_FPSTATE;
+	else
+		vcpu->arch.flags &= ~KVM_ARM64_FP_FOREIGN_FPSTATE;
+}
+
 /*
  * If the guest FPSIMD state was loaded, update the host's context
  * tracking data mark the CPU FPSIMD regs as dirty and belonging to vcpu
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index 722dfde7f1aa..d49b1b3725f5 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -44,7 +44,7 @@ static inline bool update_fp_enabled(struct kvm_vcpu *vcpu)
 	 * trap the accesses.
 	 */
 	if (!system_supports_fpsimd() ||
-	    vcpu->arch.host_thread_info->flags & _TIF_FOREIGN_FPSTATE)
+	    vcpu->arch.flags & KVM_ARM64_FP_FOREIGN_FPSTATE)
 		vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
 				      KVM_ARM64_FP_HOST);
 
-- 
2.30.2

[PATCH v2 4/5] KVM: arm64: Stop mapping current thread_info at EL2

[Marc Zyngier]

Now that we can track an equivalent of TIF_FOREIGN_FPSTATE, drop
the mapping of current's thread_info at EL2.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/include/asm/kvm_host.h       |  2 --
 arch/arm64/kvm/fpsimd.c                 | 12 +-----------
 arch/arm64/kvm/hyp/include/hyp/switch.h |  1 -
 arch/arm64/kvm/hyp/nvhe/switch.c        |  1 -
 arch/arm64/kvm/hyp/vhe/switch.c         |  1 -
 5 files changed, 1 insertion(+), 16 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 0db523e447a5..da4a82d8ce5d 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -26,7 +26,6 @@
 #include <asm/fpsimd.h>
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
-#include <asm/thread_info.h>
 
 #define __KVM_HAVE_ARCH_INTC_INITIALIZED
 
@@ -320,7 +319,6 @@ struct kvm_vcpu_arch {
 	struct kvm_guest_debug_arch vcpu_debug_state;
 	struct kvm_guest_debug_arch external_debug_state;
 
-	struct thread_info *host_thread_info;	/* hyp VA */
 	struct user_fpsimd_state *host_fpsimd_state;	/* hyp VA */
 
 	struct {
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index f464db2e9e47..f54d6a1eaff6 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -7,7 +7,6 @@
  */
 #include <linux/irqflags.h>
 #include <linux/sched.h>
-#include <linux/thread_info.h>
 #include <linux/kvm_host.h>
 #include <asm/fpsimd.h>
 #include <asm/kvm_asm.h>
@@ -28,17 +27,9 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 {
 	int ret;
 
-	struct thread_info *ti = &current->thread_info;
 	struct user_fpsimd_state *fpsimd = &current->thread.uw.fpsimd_state;
 
-	/*
-	 * Make sure the host task thread flags and fpsimd state are
-	 * visible to hyp:
-	 */
-	ret = create_hyp_mappings(ti, ti + 1, PAGE_HYP);
-	if (ret)
-		goto error;
-
+	/* Make sure the host task fpsimd state is visible to hyp: */
 	ret = create_hyp_mappings(fpsimd, fpsimd + 1, PAGE_HYP);
 	if (ret)
 		goto error;
@@ -54,7 +45,6 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 			goto error;
 	}
 
-	vcpu->arch.host_thread_info = kern_hyp_va(ti);
 	vcpu->arch.host_fpsimd_state = kern_hyp_va(fpsimd);
 error:
 	return ret;
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index d49b1b3725f5..d2d7b72de84a 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -28,7 +28,6 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
-#include <asm/thread_info.h>
 
 extern struct exception_table_entry __start___kvm_ex_table;
 extern struct exception_table_entry __stop___kvm_ex_table;
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
index a34b01cc8ab9..012890c2af1b 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -25,7 +25,6 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
-#include <asm/thread_info.h>
 
 #include <nvhe/mem_protect.h>
 
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index ded2c66675f0..c7fc04b11673 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -24,7 +24,6 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
-#include <asm/thread_info.h>
 
 /* VHE specific context */
 DEFINE_PER_CPU(struct kvm_host_data, kvm_host_data);
-- 
2.30.2

[PATCH v2 5/5] arm64/fpsimd: Document the use of TIF_FOREIGN_FPSTATE by KVM

[Marc Zyngier]

The bit of documentation that talks about TIF_FOREIGN_FPSTATE
does not mention the ungodly tricks that KVM plays with this flag.

Try and document this for the posterity.

Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kernel/fpsimd.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index ff4962750b3d..1fbd6ba7dbac 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -78,7 +78,11 @@
  * indicate whether or not the userland FPSIMD state of the current task is
  * present in the registers. The flag is set unless the FPSIMD registers of this
  * CPU currently contain the most recent userland FPSIMD state of the current
- * task.
+ * task. If the task is behaving as a VMM, then this is will be managed by
+ * KVM which will clear it to indicate that the vcpu FPSIMD state is currently
+ * loaded on the CPU, allowing the state to be saved if a FPSIMD-aware
+ * softirq kicks in. Upon vcpu_put(), KVM will save the vcpu FP state and
+ * flag the register state as invalid.
  *
  * In order to allow softirq handlers to use FPSIMD, kernel_neon_begin() may
  * save the task's FPSIMD context back to task_struct from softirq context.
-- 
2.30.2

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

On Thu, Oct 28, 2021 at 12:16:37PM +0100, Marc Zyngier wrote:
> The SVE host tracking in KVM is pretty involved. It relies on a
> set of flags tracking the ownership of the SVE register, as well
> as that of the EL0 access.

> It is also pretty scary: __hyp_sve_save_host() computes
> a thread_struct pointer and obtains a sve_state which gets directly
> accessed without further ado, even on nVHE. How can this even work?

> The answer to that is that it doesn't, and that this is mostly dead
> code. Closer examination shows that on executing a syscall, userspace
> loses its SVE state entirely. This is part of the ABI. Another
> thing to notice is that although the kernel provides helpers such as
> kernel_neon_begin()/end(), they only deal with the FP/NEON state,
> and not SVE.

> Given that you can only execute a guest as the result of a syscall,
> and that the kernel cannot use SVE by itself, it becomes pretty
> obvious that there is never any host SVE state to save, and that
> this code is only there to increase confusion.

Ah, this explains a lot and does in fact make life a lot easier, though
we're going to get some of the fun back for SME since the ABI does not
invalidate ZA on syscall.  That said there we have a register we can
check to see if the state is live rather than having to track what's
going on with TIF.  I've also currently got changes in the SME patch set
which do mean that we won't clear TIF_SVE on syscall entry while SME is
active, however I can rework that to fit in with this change easily
enough which given the simplifications introduced seems like it is
clearly the right thing to do so:

Reviewed-by: Mark Brown <broonie@kernel.org>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 5/5] arm64/fpsimd: Document the use of TIF_FOREIGN_FPSTATE by KVM

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

On Thu, Oct 28, 2021 at 12:16:40PM +0100, Marc Zyngier wrote:
> The bit of documentation that talks about TIF_FOREIGN_FPSTATE
> does not mention the ungodly tricks that KVM plays with this flag.
> 
> Try and document this for the posterity.

Reviwed-by: Mark Brown <broonie@kernel.org>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Zenghui Yu]

Hi Marc,

On 2021/10/28 19:16, Marc Zyngier wrote:
> The SVE host tracking in KVM is pretty involved. It relies on a
> set of flags tracking the ownership of the SVE register, as well
> as that of the EL0 access.
> 
> It is also pretty scary: __hyp_sve_save_host() computes
> a thread_struct pointer and obtains a sve_state which gets directly
> accessed without further ado, even on nVHE. How can this even work?
> 
> The answer to that is that it doesn't, and that this is mostly dead
> code. Closer examination shows that on executing a syscall, userspace
> loses its SVE state entirely. This is part of the ABI. Another
> thing to notice is that although the kernel provides helpers such as
> kernel_neon_begin()/end(), they only deal with the FP/NEON state,
> and not SVE.
> 
> Given that you can only execute a guest as the result of a syscall,
> and that the kernel cannot use SVE by itself, it becomes pretty
> obvious that there is never any host SVE state to save, and that
> this code is only there to increase confusion.
> 
> Get rid of the TIF_SVE tracking and host save infrastructure altogether.
> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
> index 5621020b28de..38ca332c10fe 100644
> --- a/arch/arm64/kvm/fpsimd.c
> +++ b/arch/arm64/kvm/fpsimd.c
> @@ -73,15 +73,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
>  void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
>  {
>  	BUG_ON(!current->mm);
> +	BUG_ON(test_thread_flag(TIF_SVE));
>  
> -	vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
> -			      KVM_ARM64_HOST_SVE_IN_USE |
> -			      KVM_ARM64_HOST_SVE_ENABLED);
> +	vcpu->arch.flags &= ~KVM_ARM64_FP_ENABLED;
>  	vcpu->arch.flags |= KVM_ARM64_FP_HOST;
>  
> -	if (test_thread_flag(TIF_SVE))
> -		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_IN_USE;

The comment about TIF_SVE on top of kvm_arch_vcpu_load_fp() becomes
obsolete now. Maybe worth removing it?

| *
| * TIF_SVE is backed up here, since it may get clobbered with guest state.
| * This flag is restored by kvm_arch_vcpu_put_fp(vcpu).

> diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
> index a0e78a6027be..722dfde7f1aa 100644
> --- a/arch/arm64/kvm/hyp/include/hyp/switch.h
> +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
> @@ -207,16 +207,6 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
>  	return __get_fault_info(esr, &vcpu->arch.fault);
>  }
>  
> -static inline void __hyp_sve_save_host(struct kvm_vcpu *vcpu)
> -{
> -	struct thread_struct *thread;
> -
> -	thread = container_of(vcpu->arch.host_fpsimd_state, struct thread_struct,
> -			      uw.fpsimd_state);
> -
> -	__sve_save_state(sve_pffr(thread), &vcpu->arch.host_fpsimd_state->fpsr);
> -}

Nit: This removes the only user of __sve_save_state() helper. Should we
still keep it in fpsimd.S?

Thanks,
Zenghui

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Marc Zyngier]

Hi Zenghui,

On Wed, 10 Nov 2021 13:19:23 +0000,
Zenghui Yu <yuzenghui@huawei.com> wrote:
> 
> Hi Marc,
> 
> On 2021/10/28 19:16, Marc Zyngier wrote:
> > The SVE host tracking in KVM is pretty involved. It relies on a
> > set of flags tracking the ownership of the SVE register, as well
> > as that of the EL0 access.
> > 
> > It is also pretty scary: __hyp_sve_save_host() computes
> > a thread_struct pointer and obtains a sve_state which gets directly
> > accessed without further ado, even on nVHE. How can this even work?
> > 
> > The answer to that is that it doesn't, and that this is mostly dead
> > code. Closer examination shows that on executing a syscall, userspace
> > loses its SVE state entirely. This is part of the ABI. Another
> > thing to notice is that although the kernel provides helpers such as
> > kernel_neon_begin()/end(), they only deal with the FP/NEON state,
> > and not SVE.
> > 
> > Given that you can only execute a guest as the result of a syscall,
> > and that the kernel cannot use SVE by itself, it becomes pretty
> > obvious that there is never any host SVE state to save, and that
> > this code is only there to increase confusion.
> > 
> > Get rid of the TIF_SVE tracking and host save infrastructure altogether.
> > 
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
> > index 5621020b28de..38ca332c10fe 100644
> > --- a/arch/arm64/kvm/fpsimd.c
> > +++ b/arch/arm64/kvm/fpsimd.c
> > @@ -73,15 +73,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
> >  void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
> >  {
> >  	BUG_ON(!current->mm);
> > +	BUG_ON(test_thread_flag(TIF_SVE));
> >  -	vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
> > -			      KVM_ARM64_HOST_SVE_IN_USE |
> > -			      KVM_ARM64_HOST_SVE_ENABLED);
> > +	vcpu->arch.flags &= ~KVM_ARM64_FP_ENABLED;
> >  	vcpu->arch.flags |= KVM_ARM64_FP_HOST;
> >  -	if (test_thread_flag(TIF_SVE))
> > -		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_IN_USE;
> 
> The comment about TIF_SVE on top of kvm_arch_vcpu_load_fp() becomes
> obsolete now. Maybe worth removing it?
> 
> | *
> | * TIF_SVE is backed up here, since it may get clobbered with guest state.
> | * This flag is restored by kvm_arch_vcpu_put_fp(vcpu).

Indeed. Now gone.

> 
> > diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
> > index a0e78a6027be..722dfde7f1aa 100644
> > --- a/arch/arm64/kvm/hyp/include/hyp/switch.h
> > +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
> > @@ -207,16 +207,6 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
> >  	return __get_fault_info(esr, &vcpu->arch.fault);
> >  }
> >  -static inline void __hyp_sve_save_host(struct kvm_vcpu *vcpu)
> > -{
> > -	struct thread_struct *thread;
> > -
> > -	thread = container_of(vcpu->arch.host_fpsimd_state, struct thread_struct,
> > -			      uw.fpsimd_state);
> > -
> > -	__sve_save_state(sve_pffr(thread), &vcpu->arch.host_fpsimd_state->fpsr);
> > -}
> 
> Nit: This removes the only user of __sve_save_state() helper. Should we
> still keep it in fpsimd.S?

I was in two minds about that, as I'd like to eventually be able to
use SVE for protected guests, where the hypervisor itself has to be in
charge of the FP/SVE save-restore.

But that's probably several months away, and I can always revert a
deletion patch if I need to, so let's get rid of it now.

Thanks for the suggestions.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]

On Mon, Nov 22, 2021 at 03:57:32PM +0000, Marc Zyngier wrote:
> Zenghui Yu <yuzenghui@huawei.com> wrote:

> > Nit: This removes the only user of __sve_save_state() helper. Should we
> > still keep it in fpsimd.S?

> I was in two minds about that, as I'd like to eventually be able to
> use SVE for protected guests, where the hypervisor itself has to be in
> charge of the FP/SVE save-restore.

> But that's probably several months away, and I can always revert a
> deletion patch if I need to, so let's get rid of it now.

While we're on the subject of potential future work we might in future
want to not disable SVE on every syscall if (as seems likely) it turns
out that that's more performant for small vector lengths which would
mean some minor reshuffling here to do something like convert the saved
state to FPSIMD and drop TIF_SVE in _vcpu_load_fp().  As with using SVE
in protected guests that can just be done when needed though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Marc Zyngier]

On Mon, 22 Nov 2021 17:58:00 +0000,
Mark Brown <broonie@kernel.org> wrote:
> 
> On Mon, Nov 22, 2021 at 03:57:32PM +0000, Marc Zyngier wrote:
> > Zenghui Yu <yuzenghui@huawei.com> wrote:
> 
> > > Nit: This removes the only user of __sve_save_state() helper. Should we
> > > still keep it in fpsimd.S?
> 
> > I was in two minds about that, as I'd like to eventually be able to
> > use SVE for protected guests, where the hypervisor itself has to be in
> > charge of the FP/SVE save-restore.
> 
> > But that's probably several months away, and I can always revert a
> > deletion patch if I need to, so let's get rid of it now.
> 
> While we're on the subject of potential future work we might in future
> want to not disable SVE on every syscall if (as seems likely) it turns
> out that that's more performant for small vector lengths

[...]

How are you going to retrofit that into userspace? This would be an
ABI change, and I'm not sure how you'd want to deal with that
transition...

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

On Mon, Nov 22, 2021 at 06:10:25PM +0000, Marc Zyngier wrote:
> Mark Brown <broonie@kernel.org> wrote:

> > While we're on the subject of potential future work we might in future
> > want to not disable SVE on every syscall if (as seems likely) it turns
> > out that that's more performant for small vector lengths

> How are you going to retrofit that into userspace? This would be an
> ABI change, and I'm not sure how you'd want to deal with that
> transition...

We don't need to change the ABI, the ABI just says we zero the registers
that aren't shared with FPSIMD.  Instead of doing that on taking a SVE
access trap to reenable SVE after having disabled TIF_SVE we could do
that during the syscall, userspace can't tell the difference other than
via the different formats we use to report the SVE register set via
ptrace if it single steps over a syscall.  Even then I'm struggling to
think of a scenario where userspace would be relying on that.

You could also implement a similar optimisation by forcing on TIF_SVE
whenever we return to userspace but that would create a cost for
userspace tasks that don't use SVE on SVE capable hardware so doesn't
seem as good.  In any case it's not an issue for now since anything here
will need benchmarking on a reasonable range of hardware.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Marc Zyngier]

On Mon, 22 Nov 2021 18:30:16 +0000,
Mark Brown <broonie@kernel.org> wrote:
> 
> On Mon, Nov 22, 2021 at 06:10:25PM +0000, Marc Zyngier wrote:
> > Mark Brown <broonie@kernel.org> wrote:
> 
> > > While we're on the subject of potential future work we might in future
> > > want to not disable SVE on every syscall if (as seems likely) it turns
> > > out that that's more performant for small vector lengths
> 
> > How are you going to retrofit that into userspace? This would be an
> > ABI change, and I'm not sure how you'd want to deal with that
> > transition...
> 
> We don't need to change the ABI, the ABI just says we zero the registers
> that aren't shared with FPSIMD.  Instead of doing that on taking a SVE
> access trap to reenable SVE after having disabled TIF_SVE we could do
> that during the syscall, userspace can't tell the difference other than
> via the different formats we use to report the SVE register set via
> ptrace if it single steps over a syscall.  Even then I'm struggling to
> think of a scenario where userspace would be relying on that.

That's not the point I'm trying to make.

Userspace expects to have lost SVE information over a syscall (even if
the VL is 128, it expects to have lost P0..P15 and FFR). How do you
plan to tell userspace that this behaviour has changed?

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

On Tue, Nov 23, 2021 at 10:11:33AM +0000, Marc Zyngier wrote:
> Mark Brown <broonie@kernel.org> wrote:

> > We don't need to change the ABI, the ABI just says we zero the registers
> > that aren't shared with FPSIMD.  Instead of doing that on taking a SVE
> > access trap to reenable SVE after having disabled TIF_SVE we could do

> That's not the point I'm trying to make.

> Userspace expects to have lost SVE information over a syscall (even if
> the VL is 128, it expects to have lost P0..P15 and FFR). How do you
> plan to tell userspace that this behaviour has changed?

My point is that this doesn't need to change.  Userspace can't tell if
we zeroed the non-shared state on syscall or on some later access trap.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]