* [PATCH v4 01/25] KVM: Documentation: Document arm64 core registers in detail
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 02/25] arm64: fpsimd: Always set TIF_FOREIGN_FPSTATE on task state flush Dave Martin
` (23 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
Since the the sizes of individual members of the core arm64
registers vary, the list of register encodings that make sense is
not a simple linear sequence.
To clarify which encodings to use, this patch adds a brief list
to the documentation.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Documentation/virtual/kvm/api.txt | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 356156f..097b8ba 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2090,6 +2090,30 @@ contains elements ranging from 32 to 128 bits. The index is a 32bit
value in the kvm_regs structure seen as a 32bit array.
0x60x0 0000 0010 <index into the kvm_regs struct:16>
+Specifically:
+ Encoding Register Bits kvm_regs member
+----------------------------------------------------------------
+ 0x6030 0000 0010 0000 X0 64 regs.regs[0]
+ 0x6030 0000 0010 0002 X1 64 regs.regs[1]
+ ...
+ 0x6030 0000 0010 003c X30 64 regs.regs[30]
+ 0x6030 0000 0010 003e SP 64 regs.sp
+ 0x6030 0000 0010 0040 PC 64 regs.pc
+ 0x6030 0000 0010 0042 PSTATE 64 regs.pstate
+ 0x6030 0000 0010 0044 SP_EL1 64 sp_el1
+ 0x6030 0000 0010 0046 ELR_EL1 64 elr_el1
+ 0x6030 0000 0010 0048 SPSR_EL1 64 spsr[KVM_SPSR_EL1] (alias SPSR_SVC)
+ 0x6030 0000 0010 004a SPSR_ABT 64 spsr[KVM_SPSR_ABT]
+ 0x6030 0000 0010 004c SPSR_UND 64 spsr[KVM_SPSR_UND]
+ 0x6030 0000 0010 004e SPSR_IRQ 64 spsr[KVM_SPSR_IRQ]
+ 0x6060 0000 0010 0050 SPSR_FIQ 64 spsr[KVM_SPSR_FIQ]
+ 0x6040 0000 0010 0054 V0 128 fp_regs.vregs[0]
+ 0x6040 0000 0010 0058 V1 128 fp_regs.vregs[1]
+ ...
+ 0x6040 0000 0010 00d0 V31 128 fp_regs.vregs[31]
+ 0x6020 0000 0010 00d4 FPSR 32 fp_regs.fpsr
+ 0x6020 0000 0010 00d5 FPCR 32 fp_regs.fpcr
+
arm64 CCSIDR registers are demultiplexed by CSSELR value:
0x6020 0000 0011 00 <csselr:8>
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 02/25] arm64: fpsimd: Always set TIF_FOREIGN_FPSTATE on task state flush
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
2019-01-17 20:33 ` [PATCH v4 01/25] KVM: Documentation: Document arm64 core registers in detail Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 03/25] KVM: arm64: Delete orphaned declaration for __fpsimd_enabled() Dave Martin
` (22 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
This patch updates fpsimd_flush_task_state() to mirror the new
semantics of fpsimd_flush_cpu_state() introduced by commit
d8ad71fa38a9 ("arm64: fpsimd: Fix TIF_FOREIGN_FPSTATE after
invalidating cpu regs"). Both functions now implicitly set
TIF_FOREIGN_FPSTATE to indicate that the task's FPSIMD state is not
loaded into the cpu.
As a side-effect, fpsimd_flush_task_state() now sets
TIF_FOREIGN_FPSTATE even for non-running tasks. In the case of
non-running tasks this is not useful but also harmless, because the
flag is live only while the corresponding task is running. This
function is not called from fast paths, so special-casing this for
the task == current case is not really worth it.
Compiler barriers previously present in restore_sve_fpsimd_context()
are pulled into fpsimd_flush_task_state() so that it can be safely
called with preemption enabled if necessary.
Explicit calls to set TIF_FOREIGN_FPSTATE that accompany
fpsimd_flush_task_state() calls and are now redundant are removed
as appropriate.
fpsimd_flush_task_state() is used to get exclusive access to the
representation of the task's state via task_struct, for the purpose
of replacing the state. Thus, the call to this function should
happen before manipulating fpsimd_state or sve_state etc. in
task_struct. Anomalous cases are reordered appropriately in order
to make the code more consistent, although there should be no
functional difference since these cases are protected by
local_bh_disable() anyway.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/kernel/fpsimd.c | 25 +++++++++++++++++++------
arch/arm64/kernel/signal.c | 5 -----
2 files changed, 19 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 5ebe73b..62c37f0 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -550,7 +550,6 @@ int sve_set_vector_length(struct task_struct *task,
local_bh_disable();
fpsimd_save();
- set_thread_flag(TIF_FOREIGN_FPSTATE);
}
fpsimd_flush_task_state(task);
@@ -816,12 +815,11 @@ asmlinkage void do_sve_acc(unsigned int esr, struct pt_regs *regs)
local_bh_disable();
fpsimd_save();
- fpsimd_to_sve(current);
/* Force ret_to_user to reload the registers: */
fpsimd_flush_task_state(current);
- set_thread_flag(TIF_FOREIGN_FPSTATE);
+ fpsimd_to_sve(current);
if (test_and_set_thread_flag(TIF_SVE))
WARN_ON(1); /* SVE access shouldn't have trapped */
@@ -894,9 +892,9 @@ void fpsimd_flush_thread(void)
local_bh_disable();
+ fpsimd_flush_task_state(current);
memset(¤t->thread.uw.fpsimd_state, 0,
sizeof(current->thread.uw.fpsimd_state));
- fpsimd_flush_task_state(current);
if (system_supports_sve()) {
clear_thread_flag(TIF_SVE);
@@ -933,8 +931,6 @@ void fpsimd_flush_thread(void)
current->thread.sve_vl_onexec = 0;
}
- set_thread_flag(TIF_FOREIGN_FPSTATE);
-
local_bh_enable();
}
@@ -1043,12 +1039,29 @@ void fpsimd_update_current_state(struct user_fpsimd_state const *state)
/*
* Invalidate live CPU copies of task t's FPSIMD state
+ *
+ * This function may be called with preemption enabled. The barrier()
+ * ensures that the assignment to fpsimd_cpu is visible to any
+ * preemption/softirq that could race with set_tsk_thread_flag(), so
+ * that TIF_FOREIGN_FPSTATE cannot be spuriously re-cleared.
+ *
+ * The final barrier ensures that TIF_FOREIGN_FPSTATE is seen set by any
+ * subsequent code.
*/
void fpsimd_flush_task_state(struct task_struct *t)
{
t->thread.fpsimd_cpu = NR_CPUS;
+
+ barrier();
+ set_tsk_thread_flag(t, TIF_FOREIGN_FPSTATE);
+
+ barrier();
}
+/*
+ * Invalidate any task's FPSIMD state that is present on this cpu.
+ * This function must be called with softirqs disabled.
+ */
void fpsimd_flush_cpu_state(void)
{
__this_cpu_write(fpsimd_last_state.st, NULL);
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 867a7ce..a9b0485 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -296,11 +296,6 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user)
*/
fpsimd_flush_task_state(current);
- barrier();
- /* From now, fpsimd_thread_switch() won't clear TIF_FOREIGN_FPSTATE */
-
- set_thread_flag(TIF_FOREIGN_FPSTATE);
- barrier();
/* From now, fpsimd_thread_switch() won't touch thread.sve_state */
sve_alloc(current);
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 03/25] KVM: arm64: Delete orphaned declaration for __fpsimd_enabled()
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
2019-01-17 20:33 ` [PATCH v4 01/25] KVM: Documentation: Document arm64 core registers in detail Dave Martin
2019-01-17 20:33 ` [PATCH v4 02/25] arm64: fpsimd: Always set TIF_FOREIGN_FPSTATE on task state flush Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 04/25] KVM: arm64: Refactor kvm_arm_num_regs() for easier maintenance Dave Martin
` (21 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
__fpsimd_enabled() no longer exists, but a dangling declaration has
survived in kvm_hyp.h.
This patch gets rid of it.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/include/asm/kvm_hyp.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index a80a7ef..bd7a055 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -148,7 +148,6 @@ void __debug_switch_to_host(struct kvm_vcpu *vcpu);
void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
-bool __fpsimd_enabled(void);
void activate_traps_vhe_load(struct kvm_vcpu *vcpu);
void deactivate_traps_vhe_put(void);
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 04/25] KVM: arm64: Refactor kvm_arm_num_regs() for easier maintenance
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (2 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 03/25] KVM: arm64: Delete orphaned declaration for __fpsimd_enabled() Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 05/25] KVM: arm64: Add missing #include of <linux/bitmap.h> to kvm_host.h Dave Martin
` (20 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
kvm_arm_num_regs() adds together various partial register counts in
a freeform sum expression, which makes it harder than necessary to
read diffs that add, modify or remove a single term in the sum
(which is expected to the common case under maintenance).
This patch refactors the code to add the term one per line, for
maximum readability.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/kvm/guest.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 46eb867..65f4338 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -312,8 +312,14 @@ static int get_timer_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
*/
unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu)
{
- return num_core_regs() + kvm_arm_num_sys_reg_descs(vcpu)
- + kvm_arm_get_fw_num_regs(vcpu) + NUM_TIMER_REGS;
+ unsigned long res = 0;
+
+ res += num_core_regs();
+ res += kvm_arm_num_sys_reg_descs(vcpu);
+ res += kvm_arm_get_fw_num_regs(vcpu);
+ res += NUM_TIMER_REGS;
+
+ return res;
}
/**
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 05/25] KVM: arm64: Add missing #include of <linux/bitmap.h> to kvm_host.h
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (3 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 04/25] KVM: arm64: Refactor kvm_arm_num_regs() for easier maintenance Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 06/25] arm64/sve: Check SVE virtualisability Dave Martin
` (19 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
kvm_host.h uses DECLARE_BITMAP() to declare the features member of
struct vcpu_arch, but the corresponding #include for this is
missing.
This patch adds a suitable #include for <linux/bitmap.h>. Although
the header builds without it today, this should help to avoid
future surprises.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/include/asm/kvm_host.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 7732d0b..84056a4 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -22,6 +22,7 @@
#ifndef __ARM64_KVM_HOST_H__
#define __ARM64_KVM_HOST_H__
+#include <linux/bitmap.h>
#include <linux/types.h>
#include <linux/kvm_types.h>
#include <asm/cpufeature.h>
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 06/25] arm64/sve: Check SVE virtualisability
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (4 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 05/25] KVM: arm64: Add missing #include of <linux/bitmap.h> to kvm_host.h Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 07/25] arm64/sve: Clarify role of the VQ map maintenance functions Dave Martin
` (18 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
Due to the way the effective SVE vector length is controlled and
trapped at different exception levels, certain mismatches in the
sets of vector lengths supported by different physical CPUs in the
system may prevent straightforward virtualisation of SVE at parity
with the host.
This patch analyses the extent to which SVE can be virtualised
safely without interfering with migration of vcpus between physical
CPUs, and rejects late secondary CPUs that would erode the
situation further.
It is left up to KVM to decide what to do with this information.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/fpsimd.h | 1 +
arch/arm64/kernel/cpufeature.c | 2 +-
arch/arm64/kernel/fpsimd.c | 86 ++++++++++++++++++++++++++++++++++-------
3 files changed, 73 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index dd1ad39..964adc9 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -87,6 +87,7 @@ extern void sve_kernel_enable(const struct arm64_cpu_capabilities *__unused);
extern u64 read_zcr_features(void);
extern int __ro_after_init sve_max_vl;
+extern int __ro_after_init sve_max_virtualisable_vl;
#ifdef CONFIG_ARM64_SVE
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index f6d84e2..5eaacb4 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1825,7 +1825,7 @@ static void verify_sve_features(void)
unsigned int len = zcr & ZCR_ELx_LEN_MASK;
if (len < safe_len || sve_verify_vq_map()) {
- pr_crit("CPU%d: SVE: required vector length(s) missing\n",
+ pr_crit("CPU%d: SVE: vector length support mismatch\n",
smp_processor_id());
cpu_die_early();
}
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 62c37f0..64729e2 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -18,6 +18,7 @@
*/
#include <linux/bitmap.h>
+#include <linux/bitops.h>
#include <linux/bottom_half.h>
#include <linux/bug.h>
#include <linux/cache.h>
@@ -48,6 +49,7 @@
#include <asm/sigcontext.h>
#include <asm/sysreg.h>
#include <asm/traps.h>
+#include <asm/virt.h>
#define FPEXC_IOF (1 << 0)
#define FPEXC_DZF (1 << 1)
@@ -130,14 +132,18 @@ static int sve_default_vl = -1;
/* Maximum supported vector length across all CPUs (initially poisoned) */
int __ro_after_init sve_max_vl = SVE_VL_MIN;
+int __ro_after_init sve_max_virtualisable_vl = SVE_VL_MIN;
/* Set of available vector lengths, as vq_to_bit(vq): */
static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+/* Set of vector lengths present on at least one cpu: */
+static __ro_after_init DECLARE_BITMAP(sve_vq_partial_map, SVE_VQ_MAX);
static void __percpu *efi_sve_state;
#else /* ! CONFIG_ARM64_SVE */
/* Dummy declaration for code that will be optimised out: */
extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+extern __ro_after_init DECLARE_BITMAP(sve_vq_partial_map, SVE_VQ_MAX);
extern void __percpu *efi_sve_state;
#endif /* ! CONFIG_ARM64_SVE */
@@ -623,12 +629,6 @@ int sve_get_current_vl(void)
return sve_prctl_status(0);
}
-/*
- * Bitmap for temporary storage of the per-CPU set of supported vector lengths
- * during secondary boot.
- */
-static DECLARE_BITMAP(sve_secondary_vq_map, SVE_VQ_MAX);
-
static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
{
unsigned int vq, vl;
@@ -650,6 +650,7 @@ static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
void __init sve_init_vq_map(void)
{
sve_probe_vqs(sve_vq_map);
+ bitmap_copy(sve_vq_partial_map, sve_vq_map, SVE_VQ_MAX);
}
/*
@@ -658,25 +659,58 @@ void __init sve_init_vq_map(void)
*/
void sve_update_vq_map(void)
{
- sve_probe_vqs(sve_secondary_vq_map);
- bitmap_and(sve_vq_map, sve_vq_map, sve_secondary_vq_map, SVE_VQ_MAX);
+ DECLARE_BITMAP(tmp_map, SVE_VQ_MAX);
+
+ sve_probe_vqs(tmp_map);
+ bitmap_and(sve_vq_map, sve_vq_map, tmp_map, SVE_VQ_MAX);
+ bitmap_or(sve_vq_partial_map, sve_vq_partial_map, tmp_map, SVE_VQ_MAX);
}
/* Check whether the current CPU supports all VQs in the committed set */
int sve_verify_vq_map(void)
{
- int ret = 0;
+ DECLARE_BITMAP(tmp_map, SVE_VQ_MAX);
+ unsigned long b;
- sve_probe_vqs(sve_secondary_vq_map);
- bitmap_andnot(sve_secondary_vq_map, sve_vq_map, sve_secondary_vq_map,
- SVE_VQ_MAX);
- if (!bitmap_empty(sve_secondary_vq_map, SVE_VQ_MAX)) {
+ sve_probe_vqs(tmp_map);
+
+ bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX);
+ if (bitmap_intersects(tmp_map, sve_vq_map, SVE_VQ_MAX)) {
pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
smp_processor_id());
- ret = -EINVAL;
+ return -EINVAL;
}
- return ret;
+ if (!IS_ENABLED(CONFIG_KVM) || !is_hyp_mode_available())
+ return 0;
+
+ /*
+ * For KVM, it is necessary to ensure that this CPU doesn't
+ * support any vector length that guests may have probed as
+ * unsupported.
+ */
+
+ /* Recover the set of supported VQs: */
+ bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX);
+ /* Find VQs supported that are not globally supported: */
+ bitmap_andnot(tmp_map, tmp_map, sve_vq_map, SVE_VQ_MAX);
+
+ /* Find the lowest such VQ, if any: */
+ b = find_last_bit(tmp_map, SVE_VQ_MAX);
+ if (b >= SVE_VQ_MAX)
+ return 0; /* no mismatches */
+
+ /*
+ * Mismatches above sve_max_virtualisable_vl are fine, since
+ * no guest is allowed to configure ZCR_EL2.LEN to exceed this:
+ */
+ if (sve_vl_from_vq(bit_to_vq(b)) <= sve_max_virtualisable_vl) {
+ pr_warn("SVE: cpu%d: Unsupported vector length(s) present\n",
+ smp_processor_id());
+ return -EINVAL;
+ }
+
+ return 0;
}
static void __init sve_efi_setup(void)
@@ -743,6 +777,8 @@ u64 read_zcr_features(void)
void __init sve_setup(void)
{
u64 zcr;
+ DECLARE_BITMAP(tmp_map, SVE_VQ_MAX);
+ unsigned long b;
if (!system_supports_sve())
return;
@@ -771,11 +807,31 @@ void __init sve_setup(void)
*/
sve_default_vl = find_supported_vector_length(64);
+ bitmap_andnot(tmp_map, sve_vq_partial_map, sve_vq_map,
+ SVE_VQ_MAX);
+
+ b = find_last_bit(tmp_map, SVE_VQ_MAX);
+ if (b >= SVE_VQ_MAX)
+ /* No non-virtualisable VLs found */
+ sve_max_virtualisable_vl = SVE_VQ_MAX;
+ else if (WARN_ON(b == SVE_VQ_MAX - 1))
+ /* No virtualisable VLs? This is architecturally forbidden. */
+ sve_max_virtualisable_vl = SVE_VQ_MIN;
+ else /* b + 1 < SVE_VQ_MAX */
+ sve_max_virtualisable_vl = sve_vl_from_vq(bit_to_vq(b + 1));
+
+ if (sve_max_virtualisable_vl > sve_max_vl)
+ sve_max_virtualisable_vl = sve_max_vl;
+
pr_info("SVE: maximum available vector length %u bytes per vector\n",
sve_max_vl);
pr_info("SVE: default vector length %u bytes per vector\n",
sve_default_vl);
+ /* KVM decides whether to support mismatched systems. Just warn here: */
+ if (sve_max_virtualisable_vl < sve_max_vl)
+ pr_info("SVE: unvirtualisable vector lengths present\n");
+
sve_efi_setup();
}
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 07/25] arm64/sve: Clarify role of the VQ map maintenance functions
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (5 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 06/25] arm64/sve: Check SVE virtualisability Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 08/25] arm64/sve: Enable SVE state tracking for non-task contexts Dave Martin
` (17 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
The roles of sve_init_vq_map(), sve_update_vq_map() and
sve_verify_vq_map() are highly non-obvious to anyone who has not dug
through cpufeatures.c in detail.
Since the way these functions interact with each other is more
important here than a full understanding of the cpufeatures code, this
patch adds comments to make the functions' roles clearer.
No functional change.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kernel/fpsimd.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 64729e2..92c2331 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -647,6 +647,10 @@ static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
}
}
+/*
+ * Initialise the set of known supported VQs for the boot CPU.
+ * This is called during kernel boot, before secondary CPUs are brought up.
+ */
void __init sve_init_vq_map(void)
{
sve_probe_vqs(sve_vq_map);
@@ -656,6 +660,7 @@ void __init sve_init_vq_map(void)
/*
* If we haven't committed to the set of supported VQs yet, filter out
* those not supported by the current CPU.
+ * This function is called during the bring-up of early secondary CPUs only.
*/
void sve_update_vq_map(void)
{
@@ -666,7 +671,10 @@ void sve_update_vq_map(void)
bitmap_or(sve_vq_partial_map, sve_vq_partial_map, tmp_map, SVE_VQ_MAX);
}
-/* Check whether the current CPU supports all VQs in the committed set */
+/*
+ * Check whether the current CPU supports all VQs in the committed set.
+ * This function is called during the bring-up of late secondary CPUs only.
+ */
int sve_verify_vq_map(void)
{
DECLARE_BITMAP(tmp_map, SVE_VQ_MAX);
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 08/25] arm64/sve: Enable SVE state tracking for non-task contexts
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (6 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 07/25] arm64/sve: Clarify role of the VQ map maintenance functions Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 09/25] KVM: arm64: Add a vcpu flag to control SVE visibility for the guest Dave Martin
` (16 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
The current FPSIMD/SVE context handling support for non-task (i.e.,
KVM vcpu) contexts does not take SVE into account. This means that
only task contexts can safely use SVE at present.
In preparation for enabling KVM guests to use SVE, it is necessary
to keep track of SVE state for non-task contexts too.
This patch adds the necessary support, removing assumptions from
the context switch code about the location of the SVE context
storage.
When binding a vcpu context, its vector length is arbitrarily
specified as SVE_VL_MIN for now. In any case, because TIF_SVE is
presently cleared at vcpu context bind time, the specified vector
length will not be used for anything yet. In later patches TIF_SVE
will be set here as appropriate, and the appropriate maximum vector
length for the vcpu will be passed when binding.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
Changes since RFC v3:
* Add missing include of <asm/fpsimd.h> in arch/arm64/kvm/fpsimd.c.
* Pass SVE_VL_MIN instead of sve_max_vl to fpsimd_bind_state_to_cpu(),
to avoid a link error in the CONFNIG_ARM64_SVE=n case. The value is
in any case not used for anything in practice, and the code is
replaced with the real vector length in a subsequent patch,
resolving the link error.
---
arch/arm64/include/asm/fpsimd.h | 3 ++-
arch/arm64/kernel/fpsimd.c | 20 +++++++++++++++-----
arch/arm64/kvm/fpsimd.c | 5 ++++-
3 files changed, 21 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 964adc9..df7a143 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -56,7 +56,8 @@ extern void fpsimd_restore_current_state(void);
extern void fpsimd_update_current_state(struct user_fpsimd_state const *state);
extern void fpsimd_bind_task_to_cpu(void);
-extern void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *state);
+extern void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *state,
+ void *sve_state, unsigned int sve_vl);
extern void fpsimd_flush_task_state(struct task_struct *target);
extern void fpsimd_flush_cpu_state(void);
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 92c2331..09ee264 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -121,6 +121,8 @@
*/
struct fpsimd_last_state_struct {
struct user_fpsimd_state *st;
+ void *sve_state;
+ unsigned int sve_vl;
};
static DEFINE_PER_CPU(struct fpsimd_last_state_struct, fpsimd_last_state);
@@ -241,14 +243,15 @@ static void task_fpsimd_load(void)
*/
void fpsimd_save(void)
{
- struct user_fpsimd_state *st = __this_cpu_read(fpsimd_last_state.st);
+ struct fpsimd_last_state_struct const *last =
+ this_cpu_ptr(&fpsimd_last_state);
/* set by fpsimd_bind_task_to_cpu() or fpsimd_bind_state_to_cpu() */
WARN_ON(!in_softirq() && !irqs_disabled());
if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) {
if (system_supports_sve() && test_thread_flag(TIF_SVE)) {
- if (WARN_ON(sve_get_vl() != current->thread.sve_vl)) {
+ if (WARN_ON(sve_get_vl() != last->sve_vl)) {
/*
* Can't save the user regs, so current would
* re-enter user with corrupt state.
@@ -258,9 +261,11 @@ void fpsimd_save(void)
return;
}
- sve_save_state(sve_pffr(¤t->thread), &st->fpsr);
+ sve_save_state((char *)last->sve_state +
+ sve_ffr_offset(last->sve_vl),
+ &last->st->fpsr);
} else
- fpsimd_save_state(st);
+ fpsimd_save_state(last->st);
}
}
@@ -1034,6 +1039,8 @@ void fpsimd_bind_task_to_cpu(void)
this_cpu_ptr(&fpsimd_last_state);
last->st = ¤t->thread.uw.fpsimd_state;
+ last->sve_state = current->thread.sve_state;
+ last->sve_vl = current->thread.sve_vl;
current->thread.fpsimd_cpu = smp_processor_id();
if (system_supports_sve()) {
@@ -1047,7 +1054,8 @@ void fpsimd_bind_task_to_cpu(void)
}
}
-void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st)
+void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st, void *sve_state,
+ unsigned int sve_vl)
{
struct fpsimd_last_state_struct *last =
this_cpu_ptr(&fpsimd_last_state);
@@ -1055,6 +1063,8 @@ void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st)
WARN_ON(!in_softirq() && !irqs_disabled());
last->st = st;
+ last->sve_state = sve_state;
+ last->sve_vl = sve_vl;
}
/*
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index aac7808..1cf4f02 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -9,6 +9,7 @@
#include <linux/sched.h>
#include <linux/thread_info.h>
#include <linux/kvm_host.h>
+#include <asm/fpsimd.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_host.h>
#include <asm/kvm_mmu.h>
@@ -85,7 +86,9 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
WARN_ON_ONCE(!irqs_disabled());
if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) {
- fpsimd_bind_state_to_cpu(&vcpu->arch.ctxt.gp_regs.fp_regs);
+ fpsimd_bind_state_to_cpu(&vcpu->arch.ctxt.gp_regs.fp_regs,
+ NULL, SVE_VL_MIN);
+
clear_thread_flag(TIF_FOREIGN_FPSTATE);
clear_thread_flag(TIF_SVE);
}
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 09/25] KVM: arm64: Add a vcpu flag to control SVE visibility for the guest
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (7 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 08/25] arm64/sve: Enable SVE state tracking for non-task contexts Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 10/25] KVM: arm64: Propagate vcpu into read_id_reg() Dave Martin
` (15 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
Since SVE will be enabled or disabled on a per-vcpu basis, a flag
is needed in order to track which vcpus have it enabled.
This patch adds a suitable flag and a helper for checking it.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/include/asm/kvm_host.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 84056a4..af625a8 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -309,6 +309,10 @@ struct kvm_vcpu_arch {
#define KVM_ARM64_FP_HOST (1 << 2) /* host FP regs loaded */
#define KVM_ARM64_HOST_SVE_IN_USE (1 << 3) /* backup for host TIF_SVE */
#define KVM_ARM64_HOST_SVE_ENABLED (1 << 4) /* SVE enabled for EL0 */
+#define KVM_ARM64_GUEST_HAS_SVE (1 << 5) /* SVE exposed to guest */
+
+#define vcpu_has_sve(vcpu) (system_supports_sve() && \
+ ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))
#define vcpu_gp_regs(v) (&(v)->arch.ctxt.gp_regs)
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 10/25] KVM: arm64: Propagate vcpu into read_id_reg()
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (8 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 09/25] KVM: arm64: Add a vcpu flag to control SVE visibility for the guest Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 11/25] KVM: arm64: Extend reset_unknown() to handle mixed RES0/UNKNOWN registers Dave Martin
` (14 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
Architecture features that are conditionally visible to the guest
will require run-time checks in the ID register accessor functions.
In particular, read_id_reg() will need to perform checks in order
to generate the correct emulated value for certain ID register
fields such as ID_AA64PFR0_EL1.SVE for example.
This patch propagates vcpu into read_id_reg() so that future
patches can add run-time checks on the guest configuration here.
For now, there is no functional change.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/kvm/sys_regs.c | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e3e3722..71c5825 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1029,7 +1029,8 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
}
/* Read a sanitised cpufeature ID register by sys_reg_desc */
-static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+static u64 read_id_reg(const struct kvm_vcpu *vcpu,
+ struct sys_reg_desc const *r, bool raz)
{
u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
(u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
@@ -1068,7 +1069,7 @@ static bool __access_id_reg(struct kvm_vcpu *vcpu,
if (p->is_write)
return write_to_read_only(vcpu, p, r);
- p->regval = read_id_reg(r, raz);
+ p->regval = read_id_reg(vcpu, r, raz);
return true;
}
@@ -1097,16 +1098,18 @@ static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
* are stored, and for set_id_reg() we don't allow the effective value
* to be changed.
*/
-static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
+static int __get_id_reg(const struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd, void __user *uaddr,
bool raz)
{
const u64 id = sys_reg_to_index(rd);
- const u64 val = read_id_reg(rd, raz);
+ const u64 val = read_id_reg(vcpu, rd, raz);
return reg_to_user(uaddr, &val, id);
}
-static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
+static int __set_id_reg(const struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd, void __user *uaddr,
bool raz)
{
const u64 id = sys_reg_to_index(rd);
@@ -1118,7 +1121,7 @@ static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
return err;
/* This is what we mean by invariant: you can't change it. */
- if (val != read_id_reg(rd, raz))
+ if (val != read_id_reg(vcpu, rd, raz))
return -EINVAL;
return 0;
@@ -1127,25 +1130,25 @@ static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr)
{
- return __get_id_reg(rd, uaddr, false);
+ return __get_id_reg(vcpu, rd, uaddr, false);
}
static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr)
{
- return __set_id_reg(rd, uaddr, false);
+ return __set_id_reg(vcpu, rd, uaddr, false);
}
static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr)
{
- return __get_id_reg(rd, uaddr, true);
+ return __get_id_reg(vcpu, rd, uaddr, true);
}
static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr)
{
- return __set_id_reg(rd, uaddr, true);
+ return __set_id_reg(vcpu, rd, uaddr, true);
}
/* sys_reg_desc initialiser for known cpufeature ID registers */
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 11/25] KVM: arm64: Extend reset_unknown() to handle mixed RES0/UNKNOWN registers
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (9 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 10/25] KVM: arm64: Propagate vcpu into read_id_reg() Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 12/25] KVM: arm64: Support runtime sysreg filtering for KVM_GET_REG_LIST Dave Martin
` (13 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
The reset_unknown() system register helper initialises a guest
register to a distinctive junk value on vcpu reset, to help expose
and debug deficient register initialisation within the guest.
Some registers such as the SVE control register ZCR_EL1 contain a
mixture of UNKNOWN fields and RES0 bits. For these,
reset_unknown() does not work at present, since it sets all bits to
junk values instead of just the wanted bits.
There is no need to craft another special helper just for that,
since reset_unknown() almost does the appropriate thing anyway.
This patch takes advantage of the unused val field in struct
sys_reg_desc to specify a mask of bits that should be initialised
to zero instead of junk.
All existing users of reset_unknown() do not (and should not)
define a value for val, so they will implicitly set it to zero,
resulting in all bits being made UNKNOWN by this function: thus,
this patch makes no functional change for currently defined
registers.
Future patches will make use of non-zero val.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kvm/sys_regs.h | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index 3b1bc7f..174ffc0 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -56,7 +56,12 @@ struct sys_reg_desc {
/* Index into sys_reg[], or 0 if we don't need to save it. */
int reg;
- /* Value (usually reset value) */
+ /*
+ * Value (usually reset value)
+ * For reset_unknown, each bit set to 1 in val is treated as
+ * RES0 in the register: the corresponding register bit is
+ * reset to 0 instead of "unknown".
+ */
u64 val;
/* Custom get/set_user functions, fallback to generic if NULL */
@@ -92,7 +97,9 @@ static inline void reset_unknown(struct kvm_vcpu *vcpu,
{
BUG_ON(!r->reg);
BUG_ON(r->reg >= NR_SYS_REGS);
- __vcpu_sys_reg(vcpu, r->reg) = 0x1de7ec7edbadc0deULL;
+
+ /* If non-zero, r->val specifies which register bits are RES0: */
+ __vcpu_sys_reg(vcpu, r->reg) = 0x1de7ec7edbadc0deULL & ~r->val;
}
static inline void reset_val(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 12/25] KVM: arm64: Support runtime sysreg filtering for KVM_GET_REG_LIST
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (10 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 11/25] KVM: arm64: Extend reset_unknown() to handle mixed RES0/UNKNOWN registers Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 13/25] KVM: arm64/sve: System register context switch and access support Dave Martin
` (12 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
KVM_GET_REG_LIST should only enumerate registers that are actually
accessible, so it is necessary to filter out any register that is
not exposed to the guest. For features that are configured at
runtime, this will require a dynamic check.
For example, ZCR_EL1 and ID_AA64ZFR0_EL1 would need to be hidden
if SVE is not enabled for the guest.
Special-casing walk_one_sys_reg() for specific registers will make
the code unnecessarily messy, so this patch adds a new sysreg
method check_present() that, if defined, indicates whether the
sysreg should be enumerated. If the guest runtime configuration
may require a particular system register to be hidden,
check_present should point to a function that returns true or false
to enable or disable enumeration of that register respectively.
Currently check_present() is not used for any other purpose, but it
may be a useful foundation for abstracting other parts of the code
to handle conditionally-present sysregs, if required.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kvm/sys_regs.c | 10 +++++++---
arch/arm64/kvm/sys_regs.h | 4 ++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 71c5825..bca32d5 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -2447,7 +2447,8 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
return true;
}
-static int walk_one_sys_reg(const struct sys_reg_desc *rd,
+static int walk_one_sys_reg(const struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd,
u64 __user **uind,
unsigned int *total)
{
@@ -2458,6 +2459,9 @@ static int walk_one_sys_reg(const struct sys_reg_desc *rd,
if (!(rd->reg || rd->get_user))
return 0;
+ if (rd->check_present && !rd->check_present(vcpu, rd))
+ return 0;
+
if (!copy_reg_to_user(rd, uind))
return -EFAULT;
@@ -2486,9 +2490,9 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
int cmp = cmp_sys_reg(i1, i2);
/* target-specific overrides generic entry. */
if (cmp <= 0)
- err = walk_one_sys_reg(i1, &uind, &total);
+ err = walk_one_sys_reg(vcpu, i1, &uind, &total);
else
- err = walk_one_sys_reg(i2, &uind, &total);
+ err = walk_one_sys_reg(vcpu, i2, &uind, &total);
if (err)
return err;
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index 174ffc0..472e8f1 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -69,6 +69,10 @@ struct sys_reg_desc {
const struct kvm_one_reg *reg, void __user *uaddr);
int (*set_user)(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
const struct kvm_one_reg *reg, void __user *uaddr);
+
+ /* Return true iff the register exists; assume present if NULL */
+ bool (*check_present)(const struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd);
};
static inline void print_sys_reg_instr(const struct sys_reg_params *p)
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 13/25] KVM: arm64/sve: System register context switch and access support
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (11 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 12/25] KVM: arm64: Support runtime sysreg filtering for KVM_GET_REG_LIST Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-18 16:42 ` Marc Zyngier
2019-01-17 20:33 ` [PATCH v4 14/25] KVM: arm64/sve: Context switch the SVE registers Dave Martin
` (11 subsequent siblings)
24 siblings, 1 reply; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
This patch adds the necessary support for context switching ZCR_EL1
for each vcpu.
ZCR_EL1 is trapped alongside the FPSIMD/SVE registers, so it makes
sense for it to be handled as part of the guest FPSIMD/SVE context
for context switch purposes instead of handling it as a general
system register. This means that it can be switched in lazily at
the appropriate time. No effort is made to track host context for
this register, since SVE requires VHE: thus the hosts's value for
this register lives permanently in ZCR_EL2 and does not alias the
guest's value at any time.
The Hyp switch and fpsimd context handling code is extended
appropriately.
Accessors are added in sys_regs.c to expose the SVE system
registers and ID register fields. Because these need to be
conditionally visible based on the guest configuration, they are
implemented separately for now rather than by use of the generic
system register helpers. This may be abstracted better later on
when/if there are more features requiring this model.
ID_AA64ZFR0_EL1 is RO-RAZ for MRS/MSR when SVE is disabled for the
guest, but for compatibility with non-SVE aware KVM implementations
the register should not be enumerated at all for KVM_GET_REG_LIST
in this case. For consistency we also reject ioctl access to the
register. This ensures that a non-SVE-enabled guest looks the same
to userspace, irrespective of whether the kernel KVM implementation
supports SVE.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/sysreg.h | 3 ++
arch/arm64/kvm/fpsimd.c | 8 ++-
arch/arm64/kvm/hyp/switch.c | 3 ++
arch/arm64/kvm/sys_regs.c | 111 ++++++++++++++++++++++++++++++++++++--
5 files changed, 121 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index af625a8..c32f195 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -112,6 +112,7 @@ enum vcpu_sysreg {
SCTLR_EL1, /* System Control Register */
ACTLR_EL1, /* Auxiliary Control Register */
CPACR_EL1, /* Coprocessor Access Control */
+ ZCR_EL1, /* SVE Control */
TTBR0_EL1, /* Translation Table Base Register 0 */
TTBR1_EL1, /* Translation Table Base Register 1 */
TCR_EL1, /* Translation Control Register */
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 72dc4c0..da38491 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -449,6 +449,9 @@
#define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
#define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
+/* VHE encodings for architectural EL0/1 system registers */
+#define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
+
/* Common SCTLR_ELx flags. */
#define SCTLR_ELx_DSSBS (_BITUL(44))
#define SCTLR_ELx_ENIA (_BITUL(31))
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index 1cf4f02..5ff2d90 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -103,6 +103,8 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
{
unsigned long flags;
+ bool host_has_sve = system_supports_sve();
+ bool guest_has_sve = vcpu_has_sve(vcpu);
local_irq_save(flags);
@@ -110,7 +112,11 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
/* Clean guest FP state to memory and invalidate cpu view */
fpsimd_save();
fpsimd_flush_cpu_state();
- } else if (system_supports_sve()) {
+
+ if (guest_has_sve)
+ vcpu->arch.ctxt.sys_regs[ZCR_EL1] =
+ read_sysreg_s(SYS_ZCR_EL12);
+ } else if (host_has_sve) {
/*
* The FPSIMD/SVE state in the CPU has not been touched, and we
* have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index b0b1478..9f07403 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -347,6 +347,9 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
__fpsimd_restore_state(&vcpu->arch.ctxt.gp_regs.fp_regs);
+ if (vcpu_has_sve(vcpu))
+ write_sysreg_s(vcpu->arch.ctxt.sys_regs[ZCR_EL1], SYS_ZCR_EL12);
+
/* Skip restoring fpexc32 for AArch64 guests */
if (!(read_sysreg(hcr_el2) & HCR_RW))
write_sysreg(vcpu->arch.ctxt.sys_regs[FPEXC32_EL2],
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index bca32d5..c8df730 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1036,10 +1036,7 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
(u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
- if (id == SYS_ID_AA64PFR0_EL1) {
- if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
- kvm_debug("SVE unsupported for guests, suppressing\n");
-
+ if (id == SYS_ID_AA64PFR0_EL1 && !vcpu_has_sve(vcpu)) {
val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
} else if (id == SYS_ID_AA64ISAR1_EL1) {
const u64 ptrauth_mask = (0xfUL << ID_AA64ISAR1_APA_SHIFT) |
@@ -1091,6 +1088,105 @@ static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
+#ifdef CONFIG_ARM64_SVE
+static bool sve_check_present(const struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd)
+{
+ return vcpu_has_sve(vcpu);
+}
+
+static bool access_zcr_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *rd)
+{
+ /*
+ * ZCR_EL1 access is handled directly in Hyp as part of the FPSIMD/SVE
+ * context, so we should only arrive here for non-SVE guests:
+ */
+ WARN_ON(vcpu_has_sve(vcpu));
+
+ kvm_inject_undefined(vcpu);
+ return false;
+}
+
+static int get_zcr_el1(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ if (!vcpu_has_sve(vcpu))
+ return -ENOENT;
+
+ return reg_to_user(uaddr, &vcpu->arch.ctxt.sys_regs[ZCR_EL1],
+ reg->id);
+}
+
+static int set_zcr_el1(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ if (!vcpu_has_sve(vcpu))
+ return -ENOENT;
+
+ return reg_from_user(&vcpu->arch.ctxt.sys_regs[ZCR_EL1], uaddr,
+ reg->id);
+}
+
+/* Generate the emulated ID_AA64ZFR0_EL1 value exposed to the guest */
+static u64 guest_id_aa64zfr0_el1(const struct kvm_vcpu *vcpu)
+{
+ if (!vcpu_has_sve(vcpu))
+ return 0;
+
+ return read_sanitised_ftr_reg(SYS_ID_AA64ZFR0_EL1);
+}
+
+static bool access_id_aa64zfr0_el1(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *rd)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, rd);
+
+ p->regval = guest_id_aa64zfr0_el1(vcpu);
+ return true;
+}
+
+static int get_id_aa64zfr0_el1(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ u64 val;
+
+ if (!vcpu_has_sve(vcpu))
+ return -ENOENT;
+
+ val = guest_id_aa64zfr0_el1(vcpu);
+ return reg_to_user(uaddr, &val, reg->id);
+}
+
+static int set_id_aa64zfr0_el1(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ const u64 id = sys_reg_to_index(rd);
+ int err;
+ u64 val;
+
+ if (!vcpu_has_sve(vcpu))
+ return -ENOENT;
+
+ err = reg_from_user(&val, uaddr, id);
+ if (err)
+ return err;
+
+ /* This is what we mean by invariant: you can't change it. */
+ if (val != guest_id_aa64zfr0_el1(vcpu))
+ return -EINVAL;
+
+ return 0;
+}
+#endif /* CONFIG_ARM64_SVE */
+
/*
* cpufeature ID register user accessors
*
@@ -1278,7 +1374,11 @@ static const struct sys_reg_desc sys_reg_descs[] = {
ID_SANITISED(ID_AA64PFR1_EL1),
ID_UNALLOCATED(4,2),
ID_UNALLOCATED(4,3),
+#ifdef CONFIG_ARM64_SVE
+ { SYS_DESC(SYS_ID_AA64ZFR0_EL1), access_id_aa64zfr0_el1, .get_user = get_id_aa64zfr0_el1, .set_user = set_id_aa64zfr0_el1, .check_present = sve_check_present },
+#else
ID_UNALLOCATED(4,4),
+#endif
ID_UNALLOCATED(4,5),
ID_UNALLOCATED(4,6),
ID_UNALLOCATED(4,7),
@@ -1315,6 +1415,9 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
+#ifdef CONFIG_ARM64_SVE
+ { SYS_DESC(SYS_ZCR_EL1), access_zcr_el1, reset_unknown, ZCR_EL1, ~0xfUL, .get_user = get_zcr_el1, .set_user = set_zcr_el1, .check_present = sve_check_present },
+#endif
{ SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
{ SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 },
{ SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 },
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v4 13/25] KVM: arm64/sve: System register context switch and access support
2019-01-17 20:33 ` [PATCH v4 13/25] KVM: arm64/sve: System register context switch and access support Dave Martin
@ 2019-01-18 16:42 ` Marc Zyngier
2019-01-22 16:27 ` Dave Martin
0 siblings, 1 reply; 32+ messages in thread
From: Marc Zyngier @ 2019-01-18 16:42 UTC (permalink / raw)
To: Dave Martin, kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Catalin Marinas, Will Deacon, Julien Grall,
Alex Bennée, linux-arm-kernel
On 17/01/2019 20:33, Dave Martin wrote:
> This patch adds the necessary support for context switching ZCR_EL1
> for each vcpu.
>
> ZCR_EL1 is trapped alongside the FPSIMD/SVE registers, so it makes
> sense for it to be handled as part of the guest FPSIMD/SVE context
> for context switch purposes instead of handling it as a general
> system register. This means that it can be switched in lazily at
> the appropriate time. No effort is made to track host context for
> this register, since SVE requires VHE: thus the hosts's value for
> this register lives permanently in ZCR_EL2 and does not alias the
> guest's value at any time.
>
> The Hyp switch and fpsimd context handling code is extended
> appropriately.
>
> Accessors are added in sys_regs.c to expose the SVE system
> registers and ID register fields. Because these need to be
> conditionally visible based on the guest configuration, they are
> implemented separately for now rather than by use of the generic
> system register helpers. This may be abstracted better later on
> when/if there are more features requiring this model.
>
> ID_AA64ZFR0_EL1 is RO-RAZ for MRS/MSR when SVE is disabled for the
> guest, but for compatibility with non-SVE aware KVM implementations
> the register should not be enumerated at all for KVM_GET_REG_LIST
> in this case. For consistency we also reject ioctl access to the
> register. This ensures that a non-SVE-enabled guest looks the same
> to userspace, irrespective of whether the kernel KVM implementation
> supports SVE.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/include/asm/kvm_host.h | 1 +
> arch/arm64/include/asm/sysreg.h | 3 ++
> arch/arm64/kvm/fpsimd.c | 8 ++-
> arch/arm64/kvm/hyp/switch.c | 3 ++
> arch/arm64/kvm/sys_regs.c | 111 ++++++++++++++++++++++++++++++++++++--
> 5 files changed, 121 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index af625a8..c32f195 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -112,6 +112,7 @@ enum vcpu_sysreg {
> SCTLR_EL1, /* System Control Register */
> ACTLR_EL1, /* Auxiliary Control Register */
> CPACR_EL1, /* Coprocessor Access Control */
> + ZCR_EL1, /* SVE Control */
> TTBR0_EL1, /* Translation Table Base Register 0 */
> TTBR1_EL1, /* Translation Table Base Register 1 */
> TCR_EL1, /* Translation Control Register */
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 72dc4c0..da38491 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -449,6 +449,9 @@
> #define SYS_ICH_LR14_EL2 __SYS__LR8_EL2(6)
> #define SYS_ICH_LR15_EL2 __SYS__LR8_EL2(7)
>
> +/* VHE encodings for architectural EL0/1 system registers */
> +#define SYS_ZCR_EL12 sys_reg(3, 5, 1, 2, 0)
> +
> /* Common SCTLR_ELx flags. */
> #define SCTLR_ELx_DSSBS (_BITUL(44))
> #define SCTLR_ELx_ENIA (_BITUL(31))
> diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
> index 1cf4f02..5ff2d90 100644
> --- a/arch/arm64/kvm/fpsimd.c
> +++ b/arch/arm64/kvm/fpsimd.c
> @@ -103,6 +103,8 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
> void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
> {
> unsigned long flags;
> + bool host_has_sve = system_supports_sve();
> + bool guest_has_sve = vcpu_has_sve(vcpu);
>
> local_irq_save(flags);
>
> @@ -110,7 +112,11 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
> /* Clean guest FP state to memory and invalidate cpu view */
> fpsimd_save();
> fpsimd_flush_cpu_state();
> - } else if (system_supports_sve()) {
> +
> + if (guest_has_sve)
> + vcpu->arch.ctxt.sys_regs[ZCR_EL1] =
> + read_sysreg_s(SYS_ZCR_EL12);
nit: Please keep assignments on a single line.
> + } else if (host_has_sve) {
> /*
> * The FPSIMD/SVE state in the CPU has not been touched, and we
> * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index b0b1478..9f07403 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -347,6 +347,9 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
>
> __fpsimd_restore_state(&vcpu->arch.ctxt.gp_regs.fp_regs);
>
> + if (vcpu_has_sve(vcpu))
> + write_sysreg_s(vcpu->arch.ctxt.sys_regs[ZCR_EL1], SYS_ZCR_EL12);
> +
> /* Skip restoring fpexc32 for AArch64 guests */
> if (!(read_sysreg(hcr_el2) & HCR_RW))
> write_sysreg(vcpu->arch.ctxt.sys_regs[FPEXC32_EL2],
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index bca32d5..c8df730 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -1036,10 +1036,7 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
> (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
>
> - if (id == SYS_ID_AA64PFR0_EL1) {
> - if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
> - kvm_debug("SVE unsupported for guests, suppressing\n");
> -
> + if (id == SYS_ID_AA64PFR0_EL1 && !vcpu_has_sve(vcpu)) {
> val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> } else if (id == SYS_ID_AA64ISAR1_EL1) {
> const u64 ptrauth_mask = (0xfUL << ID_AA64ISAR1_APA_SHIFT) |
> @@ -1091,6 +1088,105 @@ static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
> static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
> static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
>
> +#ifdef CONFIG_ARM64_SVE
> +static bool sve_check_present(const struct kvm_vcpu *vcpu,
> + const struct sys_reg_desc *rd)
> +{
> + return vcpu_has_sve(vcpu);
> +}
> +
> +static bool access_zcr_el1(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *rd)
> +{
> + /*
> + * ZCR_EL1 access is handled directly in Hyp as part of the FPSIMD/SVE
> + * context, so we should only arrive here for non-SVE guests:
> + */
> + WARN_ON(vcpu_has_sve(vcpu));
> +
> + kvm_inject_undefined(vcpu);
> + return false;
> +}
> +
> +static int get_zcr_el1(struct kvm_vcpu *vcpu,
> + const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + if (!vcpu_has_sve(vcpu))
> + return -ENOENT;
> +
> + return reg_to_user(uaddr, &vcpu->arch.ctxt.sys_regs[ZCR_EL1],
> + reg->id);
> +}
> +
> +static int set_zcr_el1(struct kvm_vcpu *vcpu,
> + const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + if (!vcpu_has_sve(vcpu))
> + return -ENOENT;
> +
> + return reg_from_user(&vcpu->arch.ctxt.sys_regs[ZCR_EL1], uaddr,
> + reg->id);
> +}
> +
> +/* Generate the emulated ID_AA64ZFR0_EL1 value exposed to the guest */
> +static u64 guest_id_aa64zfr0_el1(const struct kvm_vcpu *vcpu)
> +{
> + if (!vcpu_has_sve(vcpu))
> + return 0;
> +
> + return read_sanitised_ftr_reg(SYS_ID_AA64ZFR0_EL1);
> +}
> +
> +static bool access_id_aa64zfr0_el1(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *rd)
> +{
> + if (p->is_write)
> + return write_to_read_only(vcpu, p, rd);
> +
> + p->regval = guest_id_aa64zfr0_el1(vcpu);
> + return true;
> +}
> +
> +static int get_id_aa64zfr0_el1(struct kvm_vcpu *vcpu,
> + const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + u64 val;
> +
> + if (!vcpu_has_sve(vcpu))
> + return -ENOENT;
> +
> + val = guest_id_aa64zfr0_el1(vcpu);
> + return reg_to_user(uaddr, &val, reg->id);
> +}
> +
> +static int set_id_aa64zfr0_el1(struct kvm_vcpu *vcpu,
> + const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + int err;
> + u64 val;
> +
> + if (!vcpu_has_sve(vcpu))
> + return -ENOENT;
> +
> + err = reg_from_user(&val, uaddr, id);
> + if (err)
> + return err;
> +
> + /* This is what we mean by invariant: you can't change it. */
> + if (val != guest_id_aa64zfr0_el1(vcpu))
> + return -EINVAL;
> +
> + return 0;
> +}
> +#endif /* CONFIG_ARM64_SVE */
> +
> /*
> * cpufeature ID register user accessors
> *
> @@ -1278,7 +1374,11 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> ID_SANITISED(ID_AA64PFR1_EL1),
> ID_UNALLOCATED(4,2),
> ID_UNALLOCATED(4,3),
> +#ifdef CONFIG_ARM64_SVE
> + { SYS_DESC(SYS_ID_AA64ZFR0_EL1), access_id_aa64zfr0_el1, .get_user = get_id_aa64zfr0_el1, .set_user = set_id_aa64zfr0_el1, .check_present = sve_check_present },
> +#else
> ID_UNALLOCATED(4,4),
> +#endif
Can't we always have this code present and fallback to the "unallocated"
behaviour at runtime?
> ID_UNALLOCATED(4,5),
> ID_UNALLOCATED(4,6),
> ID_UNALLOCATED(4,7),
> @@ -1315,6 +1415,9 @@ static const struct sys_reg_desc sys_reg_descs[] = {
>
> { SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
> { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
> +#ifdef CONFIG_ARM64_SVE
> + { SYS_DESC(SYS_ZCR_EL1), access_zcr_el1, reset_unknown, ZCR_EL1, ~0xfUL, .get_user = get_zcr_el1, .set_user = set_zcr_el1, .check_present = sve_check_present },
> +#endif
> { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
> { SYS_DESC(SYS_TTBR1_EL1), access_vm_reg, reset_unknown, TTBR1_EL1 },
> { SYS_DESC(SYS_TCR_EL1), access_vm_reg, reset_val, TCR_EL1, 0 },
>
Thanks,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v4 13/25] KVM: arm64/sve: System register context switch and access support
2019-01-18 16:42 ` Marc Zyngier
@ 2019-01-22 16:27 ` Dave Martin
0 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-22 16:27 UTC (permalink / raw)
To: Marc Zyngier
Cc: Okamoto Takayuki, Christoffer Dall, Ard Biesheuvel,
Catalin Marinas, Will Deacon, kvmarm, linux-arm-kernel
On Fri, Jan 18, 2019 at 04:42:07PM +0000, Marc Zyngier wrote:
> On 17/01/2019 20:33, Dave Martin wrote:
> > This patch adds the necessary support for context switching ZCR_EL1
> > for each vcpu.
> >
> > ZCR_EL1 is trapped alongside the FPSIMD/SVE registers, so it makes
> > sense for it to be handled as part of the guest FPSIMD/SVE context
> > for context switch purposes instead of handling it as a general
> > system register. This means that it can be switched in lazily at
> > the appropriate time. No effort is made to track host context for
> > this register, since SVE requires VHE: thus the hosts's value for
> > this register lives permanently in ZCR_EL2 and does not alias the
> > guest's value at any time.
> >
> > The Hyp switch and fpsimd context handling code is extended
> > appropriately.
> >
> > Accessors are added in sys_regs.c to expose the SVE system
> > registers and ID register fields. Because these need to be
> > conditionally visible based on the guest configuration, they are
> > implemented separately for now rather than by use of the generic
> > system register helpers. This may be abstracted better later on
> > when/if there are more features requiring this model.
> >
> > ID_AA64ZFR0_EL1 is RO-RAZ for MRS/MSR when SVE is disabled for the
> > guest, but for compatibility with non-SVE aware KVM implementations
> > the register should not be enumerated at all for KVM_GET_REG_LIST
> > in this case. For consistency we also reject ioctl access to the
> > register. This ensures that a non-SVE-enabled guest looks the same
> > to userspace, irrespective of whether the kernel KVM implementation
> > supports SVE.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
[...]
> > diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
> > index 1cf4f02..5ff2d90 100644
> > --- a/arch/arm64/kvm/fpsimd.c
> > +++ b/arch/arm64/kvm/fpsimd.c
> > @@ -103,6 +103,8 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
> > void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
> > {
> > unsigned long flags;
> > + bool host_has_sve = system_supports_sve();
> > + bool guest_has_sve = vcpu_has_sve(vcpu);
> >
> > local_irq_save(flags);
> >
> > @@ -110,7 +112,11 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
> > /* Clean guest FP state to memory and invalidate cpu view */
> > fpsimd_save();
> > fpsimd_flush_cpu_state();
> > - } else if (system_supports_sve()) {
> > +
> > + if (guest_has_sve)
> > + vcpu->arch.ctxt.sys_regs[ZCR_EL1] =
> > + read_sysreg_s(SYS_ZCR_EL12);
>
> nit: Please keep assignments on a single line.
My antipathy for long lines got the better of me here.
If I can come up with a non-ridiculous way of shortening this line I'll
do that, otherwise I'm can join these lines and live with it.
>
> > + } else if (host_has_sve) {
> > /*
> > * The FPSIMD/SVE state in the CPU has not been touched, and we
> > * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been
[...]
> > /*
> > * cpufeature ID register user accessors
> > *
> > @@ -1278,7 +1374,11 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> > ID_SANITISED(ID_AA64PFR1_EL1),
> > ID_UNALLOCATED(4,2),
> > ID_UNALLOCATED(4,3),
> > +#ifdef CONFIG_ARM64_SVE
> > + { SYS_DESC(SYS_ID_AA64ZFR0_EL1), access_id_aa64zfr0_el1, .get_user = get_id_aa64zfr0_el1, .set_user = set_id_aa64zfr0_el1, .check_present = sve_check_present },
> > +#else
> > ID_UNALLOCATED(4,4),
> > +#endif
>
> Can't we always have this code present and fallback to the "unallocated"
> behaviour at runtime?
This may actually be preferable, since "check_present()" sounds like it
ought to be used to check whether the register is present for all
purposes, whereas right now it's only called to check whether to
enumerate the register in KVM_GET_REG_LIST. This behaviour may confuse
us later on...
There may have been some difficulty with this, but if so I can't
remember what. I'll take a look.
[...]
Cheers
---Dave
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v4 14/25] KVM: arm64/sve: Context switch the SVE registers
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (12 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 13/25] KVM: arm64/sve: System register context switch and access support Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-18 17:15 ` Marc Zyngier
2019-01-17 20:33 ` [PATCH v4 15/25] KVM: Allow 2048-bit register access via ioctl interface Dave Martin
` (10 subsequent siblings)
24 siblings, 1 reply; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
In order to give each vcpu its own view of the SVE registers, this
patch adds context storage via a new sve_state pointer in struct
vcpu_arch. An additional member sve_max_vl is also added for each
vcpu, to determine the maximum vector length visible to the guest
and thus the value to be configured in ZCR_EL2.LEN while the is
active. This also determines the layout and size of the storage in
sve_state, which is read and written by the same backend functions
that are used for context-switching the SVE state for host tasks.
On SVE-enabled vcpus, SVE access traps are now handled by switching
in the vcpu's SVE context and disabling the trap before returning
to the guest. On other vcpus, the trap is not handled and an exit
back to the host occurs, where the handle_sve() fallback path
reflects an undefined instruction exception back to the guest,
consistently with the behaviour of non-SVE-capable hardware (as was
done unconditionally prior to this patch).
No SVE handling is added on non-VHE-only paths, since VHE is an
architectural and Kconfig prerequisite of SVE.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since RFC v3:
* Ensure guest_has_sve is initialised before use. Previously the
order of the code allowed this to be used before initilaisation
on some paths. Mysteriously the compiler failed to warn.
---
arch/arm64/include/asm/kvm_host.h | 6 ++++
arch/arm64/kvm/fpsimd.c | 5 +--
arch/arm64/kvm/hyp/switch.c | 70 ++++++++++++++++++++++++++++++---------
3 files changed, 64 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index c32f195..77b6f3e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -212,6 +212,8 @@ typedef struct kvm_cpu_context kvm_cpu_context_t;
struct kvm_vcpu_arch {
struct kvm_cpu_context ctxt;
+ void *sve_state;
+ unsigned int sve_max_vl;
/* HYP configuration */
u64 hcr_el2;
@@ -304,6 +306,10 @@ struct kvm_vcpu_arch {
bool sysregs_loaded_on_cpu;
};
+/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */
+#define vcpu_sve_pffr(vcpu) ((void *)((char *)((vcpu)->arch.sve_state) + \
+ sve_ffr_offset((vcpu)->arch.sve_max_vl)))
+
/* vcpu_arch flags field values: */
#define KVM_ARM64_DEBUG_DIRTY (1 << 0)
#define KVM_ARM64_FP_ENABLED (1 << 1) /* guest FP regs loaded */
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index 5ff2d90..9cc57d4 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -87,10 +87,11 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) {
fpsimd_bind_state_to_cpu(&vcpu->arch.ctxt.gp_regs.fp_regs,
- NULL, SVE_VL_MIN);
+ vcpu->arch.sve_state,
+ vcpu->arch.sve_max_vl);
clear_thread_flag(TIF_FOREIGN_FPSTATE);
- clear_thread_flag(TIF_SVE);
+ update_thread_flag(TIF_SVE, vcpu_has_sve(vcpu));
}
}
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 9f07403..f7a8e3b 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -98,7 +98,10 @@ static void activate_traps_vhe(struct kvm_vcpu *vcpu)
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
val &= ~CPACR_EL1_ZEN;
- if (!update_fp_enabled(vcpu)) {
+ if (update_fp_enabled(vcpu)) {
+ if (vcpu_has_sve(vcpu))
+ val |= CPACR_EL1_ZEN;
+ } else {
val &= ~CPACR_EL1_FPEN;
__activate_traps_fpsimd32(vcpu);
}
@@ -313,26 +316,59 @@ static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
return true;
}
-static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
+/*
+ * if () with a gating check for SVE support to minimise branch
+ * mispredictions in non-SVE systems.
+ * (system_supports_sve() is resolved at build time or via a static key.)
+ */
+#define if_sve(cond) if (system_supports_sve() && (cond))
+
+/* Check for an FPSIMD/SVE trap and handle as appropriate */
+static bool __hyp_text __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
{
- struct user_fpsimd_state *host_fpsimd = vcpu->arch.host_fpsimd_state;
+ u8 trap_class;
+ bool guest_has_sve;
- if (has_vhe())
- write_sysreg(read_sysreg(cpacr_el1) | CPACR_EL1_FPEN,
- cpacr_el1);
- else
+ if (!system_supports_fpsimd())
+ return false;
+
+ guest_has_sve = vcpu_has_sve(vcpu);
+ trap_class = kvm_vcpu_trap_get_class(vcpu);
+
+ if (trap_class == ESR_ELx_EC_FP_ASIMD)
+ goto handle;
+
+ if_sve (guest_has_sve && trap_class == ESR_ELx_EC_SVE)
+ goto handle;
+
+ return false;
+
+handle:
+ /* The trap is an FPSIMD/SVE trap: switch the context */
+
+ if (has_vhe()) {
+ u64 reg = read_sysreg(cpacr_el1) | CPACR_EL1_FPEN;
+
+ if_sve (guest_has_sve)
+ reg |= CPACR_EL1_ZEN;
+
+ write_sysreg(reg, cpacr_el1);
+ } else {
write_sysreg(read_sysreg(cptr_el2) & ~(u64)CPTR_EL2_TFP,
cptr_el2);
+ }
isb();
if (vcpu->arch.flags & KVM_ARM64_FP_HOST) {
+ struct user_fpsimd_state *host_fpsimd =
+ vcpu->arch.host_fpsimd_state;
+
/*
* In the SVE case, VHE is assumed: it is enforced by
* Kconfig and kvm_arch_init().
*/
- if (system_supports_sve() &&
- (vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE)) {
+ if_sve (vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE) {
struct thread_struct *thread = container_of(
host_fpsimd,
struct thread_struct, uw.fpsimd_state);
@@ -345,10 +381,14 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
vcpu->arch.flags &= ~KVM_ARM64_FP_HOST;
}
- __fpsimd_restore_state(&vcpu->arch.ctxt.gp_regs.fp_regs);
-
- if (vcpu_has_sve(vcpu))
+ if_sve (guest_has_sve) {
+ sve_load_state(vcpu_sve_pffr(vcpu),
+ &vcpu->arch.ctxt.gp_regs.fp_regs.fpsr,
+ sve_vq_from_vl(vcpu->arch.sve_max_vl) - 1);
write_sysreg_s(vcpu->arch.ctxt.sys_regs[ZCR_EL1], SYS_ZCR_EL12);
+ } else {
+ __fpsimd_restore_state(&vcpu->arch.ctxt.gp_regs.fp_regs);
+ }
/* Skip restoring fpexc32 for AArch64 guests */
if (!(read_sysreg(hcr_el2) & HCR_RW))
@@ -384,10 +424,10 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
* and restore the guest context lazily.
* If FP/SIMD is not implemented, handle the trap and inject an
* undefined instruction exception to the guest.
+ * Similarly for trapped SVE accesses.
*/
- if (system_supports_fpsimd() &&
- kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_FP_ASIMD)
- return __hyp_switch_fpsimd(vcpu);
+ if (__hyp_handle_fpsimd(vcpu))
+ return true;
if (!__populate_fault_info(vcpu))
return true;
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v4 14/25] KVM: arm64/sve: Context switch the SVE registers
2019-01-17 20:33 ` [PATCH v4 14/25] KVM: arm64/sve: Context switch the SVE registers Dave Martin
@ 2019-01-18 17:15 ` Marc Zyngier
2019-01-22 17:12 ` Dave Martin
0 siblings, 1 reply; 32+ messages in thread
From: Marc Zyngier @ 2019-01-18 17:15 UTC (permalink / raw)
To: Dave Martin, kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Catalin Marinas, Will Deacon, Julien Grall,
Alex Bennée, linux-arm-kernel
On 17/01/2019 20:33, Dave Martin wrote:
> In order to give each vcpu its own view of the SVE registers, this
> patch adds context storage via a new sve_state pointer in struct
> vcpu_arch. An additional member sve_max_vl is also added for each
> vcpu, to determine the maximum vector length visible to the guest
> and thus the value to be configured in ZCR_EL2.LEN while the is
> active. This also determines the layout and size of the storage in
> sve_state, which is read and written by the same backend functions
> that are used for context-switching the SVE state for host tasks.
>
> On SVE-enabled vcpus, SVE access traps are now handled by switching
> in the vcpu's SVE context and disabling the trap before returning
> to the guest. On other vcpus, the trap is not handled and an exit
> back to the host occurs, where the handle_sve() fallback path
> reflects an undefined instruction exception back to the guest,
> consistently with the behaviour of non-SVE-capable hardware (as was
> done unconditionally prior to this patch).
>
> No SVE handling is added on non-VHE-only paths, since VHE is an
> architectural and Kconfig prerequisite of SVE.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>
> ---
>
> Changes since RFC v3:
>
> * Ensure guest_has_sve is initialised before use. Previously the
> order of the code allowed this to be used before initilaisation
> on some paths. Mysteriously the compiler failed to warn.
> ---
> arch/arm64/include/asm/kvm_host.h | 6 ++++
> arch/arm64/kvm/fpsimd.c | 5 +--
> arch/arm64/kvm/hyp/switch.c | 70 ++++++++++++++++++++++++++++++---------
> 3 files changed, 64 insertions(+), 17 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index c32f195..77b6f3e 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -212,6 +212,8 @@ typedef struct kvm_cpu_context kvm_cpu_context_t;
>
> struct kvm_vcpu_arch {
> struct kvm_cpu_context ctxt;
> + void *sve_state;
> + unsigned int sve_max_vl;
>
> /* HYP configuration */
> u64 hcr_el2;
> @@ -304,6 +306,10 @@ struct kvm_vcpu_arch {
> bool sysregs_loaded_on_cpu;
> };
>
> +/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */
> +#define vcpu_sve_pffr(vcpu) ((void *)((char *)((vcpu)->arch.sve_state) + \
> + sve_ffr_offset((vcpu)->arch.sve_max_vl)))
> +
> /* vcpu_arch flags field values: */
> #define KVM_ARM64_DEBUG_DIRTY (1 << 0)
> #define KVM_ARM64_FP_ENABLED (1 << 1) /* guest FP regs loaded */
> diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
> index 5ff2d90..9cc57d4 100644
> --- a/arch/arm64/kvm/fpsimd.c
> +++ b/arch/arm64/kvm/fpsimd.c
> @@ -87,10 +87,11 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
>
> if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) {
> fpsimd_bind_state_to_cpu(&vcpu->arch.ctxt.gp_regs.fp_regs,
> - NULL, SVE_VL_MIN);
> + vcpu->arch.sve_state,
> + vcpu->arch.sve_max_vl);
>
> clear_thread_flag(TIF_FOREIGN_FPSTATE);
> - clear_thread_flag(TIF_SVE);
> + update_thread_flag(TIF_SVE, vcpu_has_sve(vcpu));
> }
> }
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 9f07403..f7a8e3b 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -98,7 +98,10 @@ static void activate_traps_vhe(struct kvm_vcpu *vcpu)
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> val &= ~CPACR_EL1_ZEN;
> - if (!update_fp_enabled(vcpu)) {
> + if (update_fp_enabled(vcpu)) {
> + if (vcpu_has_sve(vcpu))
> + val |= CPACR_EL1_ZEN;
> + } else {
> val &= ~CPACR_EL1_FPEN;
> __activate_traps_fpsimd32(vcpu);
> }
> @@ -313,26 +316,59 @@ static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> return true;
> }
>
> -static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
> +/*
> + * if () with a gating check for SVE support to minimise branch
> + * mispredictions in non-SVE systems.
> + * (system_supports_sve() is resolved at build time or via a static key.)
> + */
> +#define if_sve(cond) if (system_supports_sve() && (cond))
I really hate this kind of construct, as while it provides an apparent
benefit, it actually leads to code that is harder to read. Do we have
any data showing that this adversely impacts any known workload on
non-SVE systems, given how rarely this exception is triggered?
At any rate, I'd rather see system_supports_sve() as part of the condition.
> +
> +/* Check for an FPSIMD/SVE trap and handle as appropriate */
> +static bool __hyp_text __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
> {
> - struct user_fpsimd_state *host_fpsimd = vcpu->arch.host_fpsimd_state;
> + u8 trap_class;
> + bool guest_has_sve;
>
> - if (has_vhe())
> - write_sysreg(read_sysreg(cpacr_el1) | CPACR_EL1_FPEN,
> - cpacr_el1);
> - else
> + if (!system_supports_fpsimd())
> + return false;
> +
> + guest_has_sve = vcpu_has_sve(vcpu);
> + trap_class = kvm_vcpu_trap_get_class(vcpu);
> +
> + if (trap_class == ESR_ELx_EC_FP_ASIMD)
> + goto handle;
> +
> + if_sve (guest_has_sve && trap_class == ESR_ELx_EC_SVE)
> + goto handle;
> +
> + return false;
In the end, this is the stuff we want to handle quickly: nothing to do
at all. Given that vcpu_has_sve() is already gated by
system_supports_sve(), I'm pretty sure that there is no difference in
code generation between your if_sve() construct and a simple if().
> +
> +handle:
> + /* The trap is an FPSIMD/SVE trap: switch the context */
> +
> + if (has_vhe()) {
> + u64 reg = read_sysreg(cpacr_el1) | CPACR_EL1_FPEN;
> +
> + if_sve (guest_has_sve)
> + reg |= CPACR_EL1_ZEN;
> +
> + write_sysreg(reg, cpacr_el1);
> + } else {
> write_sysreg(read_sysreg(cptr_el2) & ~(u64)CPTR_EL2_TFP,
> cptr_el2);
> + }
>
> isb();
>
> if (vcpu->arch.flags & KVM_ARM64_FP_HOST) {
> + struct user_fpsimd_state *host_fpsimd =
> + vcpu->arch.host_fpsimd_state;
> +
Single line, please. Or split declaration and assignment.
> /*
> * In the SVE case, VHE is assumed: it is enforced by
> * Kconfig and kvm_arch_init().
> */
> - if (system_supports_sve() &&
> - (vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE)) {
> + if_sve (vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE) {
> struct thread_struct *thread = container_of(
> host_fpsimd,
> struct thread_struct, uw.fpsimd_state);
> @@ -345,10 +381,14 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
> vcpu->arch.flags &= ~KVM_ARM64_FP_HOST;
> }
>
> - __fpsimd_restore_state(&vcpu->arch.ctxt.gp_regs.fp_regs);
> -
> - if (vcpu_has_sve(vcpu))
> + if_sve (guest_has_sve) {
> + sve_load_state(vcpu_sve_pffr(vcpu),
> + &vcpu->arch.ctxt.gp_regs.fp_regs.fpsr,
> + sve_vq_from_vl(vcpu->arch.sve_max_vl) - 1);
> write_sysreg_s(vcpu->arch.ctxt.sys_regs[ZCR_EL1], SYS_ZCR_EL12);
> + } else {
> + __fpsimd_restore_state(&vcpu->arch.ctxt.gp_regs.fp_regs);
> + }
>
> /* Skip restoring fpexc32 for AArch64 guests */
> if (!(read_sysreg(hcr_el2) & HCR_RW))
> @@ -384,10 +424,10 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
> * and restore the guest context lazily.
> * If FP/SIMD is not implemented, handle the trap and inject an
> * undefined instruction exception to the guest.
> + * Similarly for trapped SVE accesses.
> */
> - if (system_supports_fpsimd() &&
> - kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_FP_ASIMD)
> - return __hyp_switch_fpsimd(vcpu);
> + if (__hyp_handle_fpsimd(vcpu))
> + return true;
>
> if (!__populate_fault_info(vcpu))
> return true;
>
Thanks,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v4 14/25] KVM: arm64/sve: Context switch the SVE registers
2019-01-18 17:15 ` Marc Zyngier
@ 2019-01-22 17:12 ` Dave Martin
0 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-22 17:12 UTC (permalink / raw)
To: Marc Zyngier
Cc: Okamoto Takayuki, Christoffer Dall, Ard Biesheuvel,
Catalin Marinas, Will Deacon, kvmarm, linux-arm-kernel
On Fri, Jan 18, 2019 at 05:15:07PM +0000, Marc Zyngier wrote:
> On 17/01/2019 20:33, Dave Martin wrote:
> > In order to give each vcpu its own view of the SVE registers, this
> > patch adds context storage via a new sve_state pointer in struct
> > vcpu_arch. An additional member sve_max_vl is also added for each
> > vcpu, to determine the maximum vector length visible to the guest
> > and thus the value to be configured in ZCR_EL2.LEN while the is
> > active. This also determines the layout and size of the storage in
> > sve_state, which is read and written by the same backend functions
> > that are used for context-switching the SVE state for host tasks.
> >
> > On SVE-enabled vcpus, SVE access traps are now handled by switching
> > in the vcpu's SVE context and disabling the trap before returning
> > to the guest. On other vcpus, the trap is not handled and an exit
> > back to the host occurs, where the handle_sve() fallback path
> > reflects an undefined instruction exception back to the guest,
> > consistently with the behaviour of non-SVE-capable hardware (as was
> > done unconditionally prior to this patch).
> >
> > No SVE handling is added on non-VHE-only paths, since VHE is an
> > architectural and Kconfig prerequisite of SVE.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> >
> > ---
> >
> > Changes since RFC v3:
> >
> > * Ensure guest_has_sve is initialised before use. Previously the
> > order of the code allowed this to be used before initilaisation
> > on some paths. Mysteriously the compiler failed to warn.
> > ---
> > arch/arm64/include/asm/kvm_host.h | 6 ++++
> > arch/arm64/kvm/fpsimd.c | 5 +--
> > arch/arm64/kvm/hyp/switch.c | 70 ++++++++++++++++++++++++++++++---------
> > 3 files changed, 64 insertions(+), 17 deletions(-)
[...]
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 9f07403..f7a8e3b 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -98,7 +98,10 @@ static void activate_traps_vhe(struct kvm_vcpu *vcpu)
> > val = read_sysreg(cpacr_el1);
> > val |= CPACR_EL1_TTA;
> > val &= ~CPACR_EL1_ZEN;
> > - if (!update_fp_enabled(vcpu)) {
> > + if (update_fp_enabled(vcpu)) {
> > + if (vcpu_has_sve(vcpu))
> > + val |= CPACR_EL1_ZEN;
> > + } else {
> > val &= ~CPACR_EL1_FPEN;
> > __activate_traps_fpsimd32(vcpu);
> > }
> > @@ -313,26 +316,59 @@ static bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu)
> > return true;
> > }
> >
> > -static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
> > +/*
> > + * if () with a gating check for SVE support to minimise branch
> > + * mispredictions in non-SVE systems.
> > + * (system_supports_sve() is resolved at build time or via a static key.)
> > + */
> > +#define if_sve(cond) if (system_supports_sve() && (cond))
>
> I really hate this kind of construct, as while it provides an apparent
> benefit, it actually leads to code that is harder to read. Do we have
> any data showing that this adversely impacts any known workload on
> non-SVE systems, given how rarely this exception is triggered?
>
> At any rate, I'd rather see system_supports_sve() as part of the condition.
The alternative was to have a lot of if (system_supports_sve() && foo),
which makes for more branchy code.
But I'll agree that we don't have good evidence to justify this
if_sve () thing either.
I'll see if I can come up with something saner.
Is it worth at least annotating paths with likely/unlikely, or is that
again something we shouldn't do without some benchmarking that backs
it up?
> > +
> > +/* Check for an FPSIMD/SVE trap and handle as appropriate */
> > +static bool __hyp_text __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
> > {
> > - struct user_fpsimd_state *host_fpsimd = vcpu->arch.host_fpsimd_state;
> > + u8 trap_class;
> > + bool guest_has_sve;
> >
> > - if (has_vhe())
> > - write_sysreg(read_sysreg(cpacr_el1) | CPACR_EL1_FPEN,
> > - cpacr_el1);
> > - else
> > + if (!system_supports_fpsimd())
> > + return false;
> > +
> > + guest_has_sve = vcpu_has_sve(vcpu);
> > + trap_class = kvm_vcpu_trap_get_class(vcpu);
> > +
> > + if (trap_class == ESR_ELx_EC_FP_ASIMD)
> > + goto handle;
> > +
> > + if_sve (guest_has_sve && trap_class == ESR_ELx_EC_SVE)
> > + goto handle;
> > +
> > + return false;
>
> In the end, this is the stuff we want to handle quickly: nothing to do
> at all. Given that vcpu_has_sve() is already gated by
True -- would it be worth renaming the function __try_handle_fpsimd() or
similar, to make it clearer that the function may actually not handle
FPSIMD (or do anything material) in the common case?
I may have led myself astray with the current naming.
> system_supports_sve(), I'm pretty sure that there is no difference in
> code generation between your if_sve() construct and a simple if().
Maybe so. At least, if_sve () may make the code more obscure in
practice for no good reason.
> > +
> > +handle:
> > + /* The trap is an FPSIMD/SVE trap: switch the context */
> > +
> > + if (has_vhe()) {
> > + u64 reg = read_sysreg(cpacr_el1) | CPACR_EL1_FPEN;
> > +
> > + if_sve (guest_has_sve)
> > + reg |= CPACR_EL1_ZEN;
> > +
> > + write_sysreg(reg, cpacr_el1);
> > + } else {
> > write_sysreg(read_sysreg(cptr_el2) & ~(u64)CPTR_EL2_TFP,
> > cptr_el2);
> > + }
> >
> > isb();
> >
> > if (vcpu->arch.flags & KVM_ARM64_FP_HOST) {
> > + struct user_fpsimd_state *host_fpsimd =
> > + vcpu->arch.host_fpsimd_state;
> > +
>
> Single line, please. Or split declaration and assignment.
Can do.
> > /*
> > * In the SVE case, VHE is assumed: it is enforced by
> > * Kconfig and kvm_arch_init().
> > */
> > - if (system_supports_sve() &&
> > - (vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE)) {
> > + if_sve (vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE) {
> > struct thread_struct *thread = container_of(
> > host_fpsimd,
> > struct thread_struct, uw.fpsimd_state);
> > @@ -345,10 +381,14 @@ static bool __hyp_text __hyp_switch_fpsimd(struct kvm_vcpu *vcpu)
> > vcpu->arch.flags &= ~KVM_ARM64_FP_HOST;
> > }
[...]
Cheers
---Dave
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v4 15/25] KVM: Allow 2048-bit register access via ioctl interface
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (13 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 14/25] KVM: arm64/sve: Context switch the SVE registers Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 16/25] KVM: arm64: Reject ioctl access to FPSIMD V-regs on SVE vcpus Dave Martin
` (9 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
The Arm SVE architecture defines registers that are up to 2048 bits
in size (with some possibility of further future expansion).
In order to avoid the need for an excessively large number of
ioctls when saving and restoring a vcpu's registers, this patch
adds a #define to make support for individual 2048-bit registers
through the KVM_{GET,SET}_ONE_REG ioctl interface official. This
will allow each SVE register to be accessed in a single call.
There are sufficient spare bits in the register id size field for
this change, so there is no ABI impact providing that
KVM_GET_REG_LIST does not enumerate any 2048-bit register unless
userspace explicitly opts in to the relevant architecture-specific
features.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
include/uapi/linux/kvm.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 6d4ea4b..dc77a5a 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1145,6 +1145,7 @@ struct kvm_dirty_tlb {
#define KVM_REG_SIZE_U256 0x0050000000000000ULL
#define KVM_REG_SIZE_U512 0x0060000000000000ULL
#define KVM_REG_SIZE_U1024 0x0070000000000000ULL
+#define KVM_REG_SIZE_U2048 0x0080000000000000ULL
struct kvm_reg_list {
__u64 n; /* number of regs */
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 16/25] KVM: arm64: Reject ioctl access to FPSIMD V-regs on SVE vcpus
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (14 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 15/25] KVM: Allow 2048-bit register access via ioctl interface Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 17/25] KVM: arm64/sve: Add SVE support to register access ioctl interface Dave Martin
` (8 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
In order to avoid the pointless complexity of maintaining two ioctl
register access views of the same data, this patch blocks ioctl
access to the FPSIMD V-registers on vcpus that support SVE.
This will make it more straightforward to add SVE register access
support.
Since SVE is an opt-in feature for userspace, this will not affect
existing users.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kvm/guest.c | 38 +++++++++++++++++++++++++++-----------
1 file changed, 27 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 65f4338..ffa38d4 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -94,7 +94,14 @@ static int core_reg_size_from_offset(u64 off)
return -EINVAL;
}
-static int validate_core_offset(const struct kvm_one_reg *reg)
+static bool core_reg_offset_is_vreg(u64 off)
+{
+ return off >= KVM_REG_ARM_CORE_REG(fp_regs.vregs) &&
+ off < KVM_REG_ARM_CORE_REG(fp_regs.fpsr);
+}
+
+static int validate_core_offset(const struct kvm_vcpu *vcpu,
+ const struct kvm_one_reg *reg)
{
u64 off = core_reg_offset_from_id(reg->id);
int size = core_reg_size_from_offset(off);
@@ -102,10 +109,18 @@ static int validate_core_offset(const struct kvm_one_reg *reg)
if (size < 0)
return -EINVAL;
- if (KVM_REG_SIZE(reg->id) == size)
- return 0;
+ if (KVM_REG_SIZE(reg->id) != size)
+ return -EINVAL;
- return -EINVAL;
+ /*
+ * The KVM_REG_ARM64_SVE regs must be used instead of
+ * KVM_REG_ARM_CORE for accessing the FPSIMD V-registers on
+ * SVE-enabled vcpus:
+ */
+ if (vcpu_has_sve(vcpu) && core_reg_offset_is_vreg(off))
+ return -EINVAL;
+
+ return 0;
}
static int get_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
@@ -127,7 +142,7 @@ static int get_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
(off + (KVM_REG_SIZE(reg->id) / sizeof(__u32))) >= nr_regs)
return -ENOENT;
- if (validate_core_offset(reg))
+ if (validate_core_offset(vcpu, reg))
return -EINVAL;
if (copy_to_user(uaddr, ((u32 *)regs) + off, KVM_REG_SIZE(reg->id)))
@@ -152,7 +167,7 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
(off + (KVM_REG_SIZE(reg->id) / sizeof(__u32))) >= nr_regs)
return -ENOENT;
- if (validate_core_offset(reg))
+ if (validate_core_offset(vcpu, reg))
return -EINVAL;
if (KVM_REG_SIZE(reg->id) > sizeof(tmp))
@@ -205,7 +220,8 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
return -EINVAL;
}
-static int copy_core_reg_indices(u64 __user **uind)
+static int copy_core_reg_indices(const struct kvm_vcpu *vcpu,
+ u64 __user **uind)
{
unsigned int i;
int n = 0;
@@ -247,9 +263,9 @@ static int copy_core_reg_indices(u64 __user **uind)
return n;
}
-static unsigned long num_core_regs(void)
+static unsigned long num_core_regs(const struct kvm_vcpu *vcpu)
{
- return copy_core_reg_indices(NULL);
+ return copy_core_reg_indices(vcpu, NULL);
}
/**
@@ -314,7 +330,7 @@ unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu)
{
unsigned long res = 0;
- res += num_core_regs();
+ res += num_core_regs(vcpu);
res += kvm_arm_num_sys_reg_descs(vcpu);
res += kvm_arm_get_fw_num_regs(vcpu);
res += NUM_TIMER_REGS;
@@ -331,7 +347,7 @@ int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
{
int ret;
- ret = copy_core_reg_indices(&uindices);
+ ret = copy_core_reg_indices(vcpu, &uindices);
if (ret < 0)
return ret;
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 17/25] KVM: arm64/sve: Add SVE support to register access ioctl interface
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (15 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 16/25] KVM: arm64: Reject ioctl access to FPSIMD V-regs on SVE vcpus Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-18 17:58 ` Marc Zyngier
2019-01-17 20:33 ` [PATCH v4 18/25] KVM: arm64: Enumerate SVE register indices for KVM_GET_REG_LIST Dave Martin
` (7 subsequent siblings)
24 siblings, 1 reply; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
This patch adds the following registers for access via the
KVM_{GET,SET}_ONE_REG interface:
* KVM_REG_ARM64_SVE_ZREG(n, i) (n = 0..31) (in 2048-bit slices)
* KVM_REG_ARM64_SVE_PREG(n, i) (n = 0..15) (in 256-bit slices)
* KVM_REG_ARM64_SVE_FFR(i) (in 256-bit slices)
In order to adapt gracefully to future architectural extensions,
the registers are divided up into slices as noted above: the i
parameter denotes the slice index.
For simplicity, bits or slices that exceed the maximum vector
length supported for the vcpu are ignored for KVM_SET_ONE_REG, and
read as zero for KVM_GET_ONE_REG.
For the current architecture, only slice i = 0 is significant. The
interface design allows i to increase to up to 31 in the future if
required by future architectural amendments.
The registers are only visible for vcpus that have SVE enabled.
They are not enumerated by KVM_GET_REG_LIST on vcpus that do not
have SVE. In all cases, surplus slices are not enumerated by
KVM_GET_REG_LIST.
Accesses to the FPSIMD registers via KVM_REG_ARM_CORE is not
allowed for SVE-enabled vcpus: SVE-aware userspace can use the
KVM_REG_ARM64_SVE_ZREG() interface instead to access the same
register state. This avoids some complex and pointless emluation
in the kernel.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/uapi/asm/kvm.h | 10 +++
arch/arm64/kvm/guest.c | 131 ++++++++++++++++++++++++++++++++++----
2 files changed, 129 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 97c3478..1ff68fa 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -226,6 +226,16 @@ struct kvm_vcpu_events {
KVM_REG_ARM_FW | ((r) & 0xffff))
#define KVM_REG_ARM_PSCI_VERSION KVM_REG_ARM_FW_REG(0)
+/* SVE registers */
+#define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT)
+#define KVM_REG_ARM64_SVE_ZREG(n, i) (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
+ KVM_REG_SIZE_U2048 | \
+ ((n) << 5) | (i))
+#define KVM_REG_ARM64_SVE_PREG(n, i) (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
+ KVM_REG_SIZE_U256 | \
+ ((n) << 5) | (i) | 0x400)
+#define KVM_REG_ARM64_SVE_FFR(i) KVM_REG_ARM64_SVE_PREG(16, i)
+
/* Device Control API: ARM VGIC */
#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index ffa38d4..b8f9c1e 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -19,8 +19,10 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/bits.h>
#include <linux/errno.h>
#include <linux/err.h>
+#include <linux/kernel.h>
#include <linux/kvm_host.h>
#include <linux/module.h>
#include <linux/vmalloc.h>
@@ -28,9 +30,12 @@
#include <kvm/arm_psci.h>
#include <asm/cputype.h>
#include <linux/uaccess.h>
+#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_coproc.h>
+#include <asm/kvm_host.h>
+#include <asm/sigcontext.h>
#include "trace.h"
@@ -210,6 +215,108 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
return err;
}
+struct kreg_region {
+ char *kptr;
+ size_t size;
+ size_t zeropad;
+};
+
+#define SVE_REG_SLICE_SHIFT 0
+#define SVE_REG_SLICE_BITS 5
+#define SVE_REG_ID_SHIFT (SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS)
+#define SVE_REG_ID_BITS 5
+
+#define SVE_REG_SLICE_MASK \
+ GENMASK(SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS - 1, \
+ SVE_REG_SLICE_SHIFT)
+#define SVE_REG_ID_MASK \
+ GENMASK(SVE_REG_ID_SHIFT + SVE_REG_ID_BITS - 1, SVE_REG_ID_SHIFT)
+
+#define SVE_NUM_SLICES (1 << SVE_REG_SLICE_BITS)
+
+static int sve_reg_region(struct kreg_region *b,
+ const struct kvm_vcpu *vcpu,
+ const struct kvm_one_reg *reg)
+{
+ const unsigned int vl = vcpu->arch.sve_max_vl;
+ const unsigned int vq = sve_vq_from_vl(vl);
+
+ const unsigned int reg_num =
+ (reg->id & SVE_REG_ID_MASK) >> SVE_REG_ID_SHIFT;
+ const unsigned int slice_num =
+ (reg->id & SVE_REG_SLICE_MASK) >> SVE_REG_SLICE_SHIFT;
+
+ unsigned int slice_size, offset, limit;
+
+ if (reg->id >= KVM_REG_ARM64_SVE_ZREG(0, 0) &&
+ reg->id <= KVM_REG_ARM64_SVE_ZREG(SVE_NUM_ZREGS - 1,
+ SVE_NUM_SLICES - 1)) {
+ slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_ZREG(0, 0));
+
+ /* Compute start and end of the register: */
+ offset = SVE_SIG_ZREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
+ limit = offset + SVE_SIG_ZREG_SIZE(vq);
+
+ offset += slice_size * slice_num; /* start of requested slice */
+
+ } else if (reg->id >= KVM_REG_ARM64_SVE_PREG(0, 0) &&
+ reg->id <= KVM_REG_ARM64_SVE_FFR(SVE_NUM_SLICES - 1)) {
+ /* (FFR is P16 for our purposes) */
+
+ slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_PREG(0, 0));
+
+ /* Compute start and end of the register: */
+ offset = SVE_SIG_PREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
+ limit = offset + SVE_SIG_PREG_SIZE(vq);
+
+ offset += slice_size * slice_num; /* start of requested slice */
+
+ } else {
+ return -ENOENT;
+ }
+
+ b->kptr = (char *)vcpu->arch.sve_state + offset;
+
+ /*
+ * If the slice starts after the end of the reg, just pad.
+ * Otherwise, copy as much as possible up to slice_size and pad
+ * the remainder:
+ */
+ b->size = offset >= limit ? 0 : min(limit - offset, slice_size);
+ b->zeropad = slice_size - b->size;
+
+ return 0;
+}
+
+static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
+{
+ struct kreg_region kreg;
+ char __user *uptr = (char __user *)reg->addr;
+
+ if (!vcpu_has_sve(vcpu) || sve_reg_region(&kreg, vcpu, reg))
+ return -ENOENT;
+
+ if (copy_to_user(uptr, kreg.kptr, kreg.size) ||
+ clear_user(uptr + kreg.size, kreg.zeropad))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
+{
+ struct kreg_region kreg;
+ char __user *uptr = (char __user *)reg->addr;
+
+ if (!vcpu_has_sve(vcpu) || sve_reg_region(&kreg, vcpu, reg))
+ return -ENOENT;
+
+ if (copy_from_user(kreg.kptr, uptr, kreg.size))
+ return -EFAULT;
+
+ return 0;
+}
+
int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
{
return -EINVAL;
@@ -370,12 +477,12 @@ int kvm_arm_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
if ((reg->id & ~KVM_REG_SIZE_MASK) >> 32 != KVM_REG_ARM64 >> 32)
return -EINVAL;
- /* Register group 16 means we want a core register. */
- if ((reg->id & KVM_REG_ARM_COPROC_MASK) == KVM_REG_ARM_CORE)
- return get_core_reg(vcpu, reg);
-
- if ((reg->id & KVM_REG_ARM_COPROC_MASK) == KVM_REG_ARM_FW)
- return kvm_arm_get_fw_reg(vcpu, reg);
+ switch (reg->id & KVM_REG_ARM_COPROC_MASK) {
+ case KVM_REG_ARM_CORE: return get_core_reg(vcpu, reg);
+ case KVM_REG_ARM_FW: return kvm_arm_get_fw_reg(vcpu, reg);
+ case KVM_REG_ARM64_SVE: return get_sve_reg(vcpu, reg);
+ default: break; /* fall through */
+ }
if (is_timer_reg(reg->id))
return get_timer_reg(vcpu, reg);
@@ -389,12 +496,12 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
if ((reg->id & ~KVM_REG_SIZE_MASK) >> 32 != KVM_REG_ARM64 >> 32)
return -EINVAL;
- /* Register group 16 means we set a core register. */
- if ((reg->id & KVM_REG_ARM_COPROC_MASK) == KVM_REG_ARM_CORE)
- return set_core_reg(vcpu, reg);
-
- if ((reg->id & KVM_REG_ARM_COPROC_MASK) == KVM_REG_ARM_FW)
- return kvm_arm_set_fw_reg(vcpu, reg);
+ switch (reg->id & KVM_REG_ARM_COPROC_MASK) {
+ case KVM_REG_ARM_CORE: return set_core_reg(vcpu, reg);
+ case KVM_REG_ARM_FW: return kvm_arm_set_fw_reg(vcpu, reg);
+ case KVM_REG_ARM64_SVE: return set_sve_reg(vcpu, reg);
+ default: break; /* fall through */
+ }
if (is_timer_reg(reg->id))
return set_timer_reg(vcpu, reg);
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v4 17/25] KVM: arm64/sve: Add SVE support to register access ioctl interface
2019-01-17 20:33 ` [PATCH v4 17/25] KVM: arm64/sve: Add SVE support to register access ioctl interface Dave Martin
@ 2019-01-18 17:58 ` Marc Zyngier
2019-01-22 17:24 ` Dave Martin
0 siblings, 1 reply; 32+ messages in thread
From: Marc Zyngier @ 2019-01-18 17:58 UTC (permalink / raw)
To: Dave Martin, kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Catalin Marinas, Will Deacon, Julien Grall,
Alex Bennée, linux-arm-kernel
On 17/01/2019 20:33, Dave Martin wrote:
> This patch adds the following registers for access via the
> KVM_{GET,SET}_ONE_REG interface:
>
> * KVM_REG_ARM64_SVE_ZREG(n, i) (n = 0..31) (in 2048-bit slices)
> * KVM_REG_ARM64_SVE_PREG(n, i) (n = 0..15) (in 256-bit slices)
> * KVM_REG_ARM64_SVE_FFR(i) (in 256-bit slices)
>
> In order to adapt gracefully to future architectural extensions,
> the registers are divided up into slices as noted above: the i
> parameter denotes the slice index.
>
> For simplicity, bits or slices that exceed the maximum vector
> length supported for the vcpu are ignored for KVM_SET_ONE_REG, and
> read as zero for KVM_GET_ONE_REG.
>
> For the current architecture, only slice i = 0 is significant. The
> interface design allows i to increase to up to 31 in the future if
> required by future architectural amendments.
>
> The registers are only visible for vcpus that have SVE enabled.
> They are not enumerated by KVM_GET_REG_LIST on vcpus that do not
> have SVE. In all cases, surplus slices are not enumerated by
> KVM_GET_REG_LIST.
>
> Accesses to the FPSIMD registers via KVM_REG_ARM_CORE is not
> allowed for SVE-enabled vcpus: SVE-aware userspace can use the
> KVM_REG_ARM64_SVE_ZREG() interface instead to access the same
> register state. This avoids some complex and pointless emluation
nit: emulation.
> in the kernel.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 10 +++
> arch/arm64/kvm/guest.c | 131 ++++++++++++++++++++++++++++++++++----
> 2 files changed, 129 insertions(+), 12 deletions(-)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 97c3478..1ff68fa 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -226,6 +226,16 @@ struct kvm_vcpu_events {
> KVM_REG_ARM_FW | ((r) & 0xffff))
> #define KVM_REG_ARM_PSCI_VERSION KVM_REG_ARM_FW_REG(0)
>
> +/* SVE registers */
> +#define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT)
> +#define KVM_REG_ARM64_SVE_ZREG(n, i) (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
> + KVM_REG_SIZE_U2048 | \
> + ((n) << 5) | (i))
> +#define KVM_REG_ARM64_SVE_PREG(n, i) (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
> + KVM_REG_SIZE_U256 | \
> + ((n) << 5) | (i) | 0x400)
Can we please have a name for this 0x400 bit?
> +#define KVM_REG_ARM64_SVE_FFR(i) KVM_REG_ARM64_SVE_PREG(16, i)
> +
> /* Device Control API: ARM VGIC */
> #define KVM_DEV_ARM_VGIC_GRP_ADDR 0
> #define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
> diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
> index ffa38d4..b8f9c1e 100644
> --- a/arch/arm64/kvm/guest.c
> +++ b/arch/arm64/kvm/guest.c
> @@ -19,8 +19,10 @@
> * along with this program. If not, see <http://www.gnu.org/licenses/>.
> */
>
> +#include <linux/bits.h>
> #include <linux/errno.h>
> #include <linux/err.h>
> +#include <linux/kernel.h>
> #include <linux/kvm_host.h>
> #include <linux/module.h>
> #include <linux/vmalloc.h>
> @@ -28,9 +30,12 @@
> #include <kvm/arm_psci.h>
> #include <asm/cputype.h>
> #include <linux/uaccess.h>
> +#include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_coproc.h>
> +#include <asm/kvm_host.h>
> +#include <asm/sigcontext.h>
>
> #include "trace.h"
>
> @@ -210,6 +215,108 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
> return err;
> }
>
> +struct kreg_region {
> + char *kptr;
Should this rather be void * instead?
> + size_t size;
> + size_t zeropad;
> +};
> +
> +#define SVE_REG_SLICE_SHIFT 0
> +#define SVE_REG_SLICE_BITS 5
> +#define SVE_REG_ID_SHIFT (SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS)
> +#define SVE_REG_ID_BITS 5
> +
> +#define SVE_REG_SLICE_MASK \
> + GENMASK(SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS - 1, \
> + SVE_REG_SLICE_SHIFT)
> +#define SVE_REG_ID_MASK \
> + GENMASK(SVE_REG_ID_SHIFT + SVE_REG_ID_BITS - 1, SVE_REG_ID_SHIFT)
> +
> +#define SVE_NUM_SLICES (1 << SVE_REG_SLICE_BITS)
> +
> +static int sve_reg_region(struct kreg_region *b,
> + const struct kvm_vcpu *vcpu,
> + const struct kvm_one_reg *reg)
> +{
> + const unsigned int vl = vcpu->arch.sve_max_vl;
> + const unsigned int vq = sve_vq_from_vl(vl);
> +
> + const unsigned int reg_num =
> + (reg->id & SVE_REG_ID_MASK) >> SVE_REG_ID_SHIFT;
> + const unsigned int slice_num =
> + (reg->id & SVE_REG_SLICE_MASK) >> SVE_REG_SLICE_SHIFT;
> +
> + unsigned int slice_size, offset, limit;
> +
> + if (reg->id >= KVM_REG_ARM64_SVE_ZREG(0, 0) &&
> + reg->id <= KVM_REG_ARM64_SVE_ZREG(SVE_NUM_ZREGS - 1,
> + SVE_NUM_SLICES - 1)) {
> + slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_ZREG(0, 0));
> +
> + /* Compute start and end of the register: */
> + offset = SVE_SIG_ZREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
> + limit = offset + SVE_SIG_ZREG_SIZE(vq);
> +
> + offset += slice_size * slice_num; /* start of requested slice */
> +
> + } else if (reg->id >= KVM_REG_ARM64_SVE_PREG(0, 0) &&
> + reg->id <= KVM_REG_ARM64_SVE_FFR(SVE_NUM_SLICES - 1)) {
> + /* (FFR is P16 for our purposes) */
> +
> + slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_PREG(0, 0));
> +
> + /* Compute start and end of the register: */
> + offset = SVE_SIG_PREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
> + limit = offset + SVE_SIG_PREG_SIZE(vq);
> +
> + offset += slice_size * slice_num; /* start of requested slice */
> +
> + } else {
> + return -ENOENT;
> + }
> +
> + b->kptr = (char *)vcpu->arch.sve_state + offset;
I'm very uneasy with this pointer that, at this stage, may point to an
arbitrary location in memory, given that the slice number is controlled
by userspace. It feels like a disaster waiting to happen as a potential
spectre-v1 gadget.
And actually, we don't have any requirement to handle non-zero slices,
do we? The important thing is that you've nicely reserved extra space in
the encoding, and that we can expand the API in a very straightforward way.
So at this stage, I'd drop all support for non-zero slices and return an
error instead.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v4 17/25] KVM: arm64/sve: Add SVE support to register access ioctl interface
2019-01-18 17:58 ` Marc Zyngier
@ 2019-01-22 17:24 ` Dave Martin
0 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-22 17:24 UTC (permalink / raw)
To: Marc Zyngier
Cc: Okamoto Takayuki, Christoffer Dall, Ard Biesheuvel,
Catalin Marinas, Will Deacon, kvmarm, linux-arm-kernel
On Fri, Jan 18, 2019 at 05:58:33PM +0000, Marc Zyngier wrote:
> On 17/01/2019 20:33, Dave Martin wrote:
> > This patch adds the following registers for access via the
> > KVM_{GET,SET}_ONE_REG interface:
> >
> > * KVM_REG_ARM64_SVE_ZREG(n, i) (n = 0..31) (in 2048-bit slices)
> > * KVM_REG_ARM64_SVE_PREG(n, i) (n = 0..15) (in 256-bit slices)
> > * KVM_REG_ARM64_SVE_FFR(i) (in 256-bit slices)
> >
> > In order to adapt gracefully to future architectural extensions,
> > the registers are divided up into slices as noted above: the i
> > parameter denotes the slice index.
> >
> > For simplicity, bits or slices that exceed the maximum vector
> > length supported for the vcpu are ignored for KVM_SET_ONE_REG, and
> > read as zero for KVM_GET_ONE_REG.
> >
> > For the current architecture, only slice i = 0 is significant. The
> > interface design allows i to increase to up to 31 in the future if
> > required by future architectural amendments.
> >
> > The registers are only visible for vcpus that have SVE enabled.
> > They are not enumerated by KVM_GET_REG_LIST on vcpus that do not
> > have SVE. In all cases, surplus slices are not enumerated by
> > KVM_GET_REG_LIST.
> >
> > Accesses to the FPSIMD registers via KVM_REG_ARM_CORE is not
> > allowed for SVE-enabled vcpus: SVE-aware userspace can use the
> > KVM_REG_ARM64_SVE_ZREG() interface instead to access the same
> > register state. This avoids some complex and pointless emluation
>
> nit: emulation.
>
> > in the kernel.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > ---
> > arch/arm64/include/uapi/asm/kvm.h | 10 +++
> > arch/arm64/kvm/guest.c | 131 ++++++++++++++++++++++++++++++++++----
> > 2 files changed, 129 insertions(+), 12 deletions(-)
> >
> > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> > index 97c3478..1ff68fa 100644
> > --- a/arch/arm64/include/uapi/asm/kvm.h
> > +++ b/arch/arm64/include/uapi/asm/kvm.h
> > @@ -226,6 +226,16 @@ struct kvm_vcpu_events {
> > KVM_REG_ARM_FW | ((r) & 0xffff))
> > #define KVM_REG_ARM_PSCI_VERSION KVM_REG_ARM_FW_REG(0)
> >
> > +/* SVE registers */
> > +#define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT)
> > +#define KVM_REG_ARM64_SVE_ZREG(n, i) (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
> > + KVM_REG_SIZE_U2048 | \
> > + ((n) << 5) | (i))
> > +#define KVM_REG_ARM64_SVE_PREG(n, i) (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
> > + KVM_REG_SIZE_U256 | \
> > + ((n) << 5) | (i) | 0x400)
>
> Can we please have a name for this 0x400 bit?
>
>
> > +#define KVM_REG_ARM64_SVE_FFR(i) KVM_REG_ARM64_SVE_PREG(16, i)
> > +
> > /* Device Control API: ARM VGIC */
> > #define KVM_DEV_ARM_VGIC_GRP_ADDR 0
> > #define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
> > diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
> > index ffa38d4..b8f9c1e 100644
> > --- a/arch/arm64/kvm/guest.c
> > +++ b/arch/arm64/kvm/guest.c
> > @@ -19,8 +19,10 @@
> > * along with this program. If not, see <http://www.gnu.org/licenses/>.
> > */
> >
> > +#include <linux/bits.h>
> > #include <linux/errno.h>
> > #include <linux/err.h>
> > +#include <linux/kernel.h>
> > #include <linux/kvm_host.h>
> > #include <linux/module.h>
> > #include <linux/vmalloc.h>
> > @@ -28,9 +30,12 @@
> > #include <kvm/arm_psci.h>
> > #include <asm/cputype.h>
> > #include <linux/uaccess.h>
> > +#include <asm/fpsimd.h>
> > #include <asm/kvm.h>
> > #include <asm/kvm_emulate.h>
> > #include <asm/kvm_coproc.h>
> > +#include <asm/kvm_host.h>
> > +#include <asm/sigcontext.h>
> >
> > #include "trace.h"
> >
> > @@ -210,6 +215,108 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
> > return err;
> > }
> >
> > +struct kreg_region {
> > + char *kptr;
>
> Should this rather be void * instead?
This was to support address arithmetic on kptr without explicit casting.
Maybe this is ill-advised. GCC supports pointer arithmetic on void *
with equivalent behaviour as if the pointer were to a character type,
without any cast.
If we rely elsewhere on this working, then we can do it here.
However... [see below]
> > + size_t size;
> > + size_t zeropad;
> > +};
> > +
> > +#define SVE_REG_SLICE_SHIFT 0
> > +#define SVE_REG_SLICE_BITS 5
> > +#define SVE_REG_ID_SHIFT (SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS)
> > +#define SVE_REG_ID_BITS 5
> > +
> > +#define SVE_REG_SLICE_MASK \
> > + GENMASK(SVE_REG_SLICE_SHIFT + SVE_REG_SLICE_BITS - 1, \
> > + SVE_REG_SLICE_SHIFT)
> > +#define SVE_REG_ID_MASK \
> > + GENMASK(SVE_REG_ID_SHIFT + SVE_REG_ID_BITS - 1, SVE_REG_ID_SHIFT)
> > +
> > +#define SVE_NUM_SLICES (1 << SVE_REG_SLICE_BITS)
> > +
> > +static int sve_reg_region(struct kreg_region *b,
> > + const struct kvm_vcpu *vcpu,
> > + const struct kvm_one_reg *reg)
> > +{
> > + const unsigned int vl = vcpu->arch.sve_max_vl;
> > + const unsigned int vq = sve_vq_from_vl(vl);
> > +
> > + const unsigned int reg_num =
> > + (reg->id & SVE_REG_ID_MASK) >> SVE_REG_ID_SHIFT;
> > + const unsigned int slice_num =
> > + (reg->id & SVE_REG_SLICE_MASK) >> SVE_REG_SLICE_SHIFT;
> > +
> > + unsigned int slice_size, offset, limit;
> > +
> > + if (reg->id >= KVM_REG_ARM64_SVE_ZREG(0, 0) &&
> > + reg->id <= KVM_REG_ARM64_SVE_ZREG(SVE_NUM_ZREGS - 1,
> > + SVE_NUM_SLICES - 1)) {
> > + slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_ZREG(0, 0));
> > +
> > + /* Compute start and end of the register: */
> > + offset = SVE_SIG_ZREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
> > + limit = offset + SVE_SIG_ZREG_SIZE(vq);
> > +
> > + offset += slice_size * slice_num; /* start of requested slice */
> > +
> > + } else if (reg->id >= KVM_REG_ARM64_SVE_PREG(0, 0) &&
> > + reg->id <= KVM_REG_ARM64_SVE_FFR(SVE_NUM_SLICES - 1)) {
> > + /* (FFR is P16 for our purposes) */
> > +
> > + slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_PREG(0, 0));
> > +
> > + /* Compute start and end of the register: */
> > + offset = SVE_SIG_PREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
> > + limit = offset + SVE_SIG_PREG_SIZE(vq);
> > +
> > + offset += slice_size * slice_num; /* start of requested slice */
> > +
> > + } else {
> > + return -ENOENT;
> > + }
> > +
> > + b->kptr = (char *)vcpu->arch.sve_state + offset;
>
> I'm very uneasy with this pointer that, at this stage, may point to an
> arbitrary location in memory, given that the slice number is controlled
> by userspace. It feels like a disaster waiting to happen as a potential
> spectre-v1 gadget.
Hmmm, yes, that does seem a concern. We only guard against accessing
that address with an if(), so while the compiler cannot emit an explicit
access, the CPU can emit a speculative read.
So, a speculation-safe bounds check would be appropriate here, at
minimum.
> And actually, we don't have any requirement to handle non-zero slices,
> do we? The important thing is that you've nicely reserved extra space in
> the encoding, and that we can expand the API in a very straightforward way.
>
> So at this stage, I'd drop all support for non-zero slices and return an
> error instead.
Since the extra-slices thing is for the far future (or never), we can
probably drop this code for now.
This means that we can drop the enumeration of extra slices too.
If the vector length is > 256 bytes, I'll do pr_warn() and clamp the
vector lengths seen by the guest to <= 256 so that no extra slices are
needed.
In any case, we can keep the space reserved in the register ID encoding
to allow larger vectors to be supported later if need be.
I'll write up the rule for determining the number of slices somewhere,
but in userspace it will never be effectively tested. Too bad, I
guess: we might add an explicit capability for larger vectors if we
eventually risk hitting incompatibilities here.
Cheers
---Dave
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v4 18/25] KVM: arm64: Enumerate SVE register indices for KVM_GET_REG_LIST
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (16 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 17/25] KVM: arm64/sve: Add SVE support to register access ioctl interface Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 19/25] arm64/sve: In-kernel vector length availability query interface Dave Martin
` (6 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
This patch includes the SVE register IDs in the list returned by
KVM_GET_REG_LIST, as appropriate.
On a non-SVE-enabled vcpu, no new IDs are added.
On an SVE-enabled vcpu, IDs for the FPSIMD V-registers are removed
from the list, since userspace is required to access the Z-
registers instead to access their context. For the variably-
sized SVE registers, the appropriate set of slice IDs are
enumerated, depending on the maximum vector length for the vcpu.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since RFC v3:
* Don't enumerate FPSIMD V-regs for SVE vcpus.
---
arch/arm64/kvm/guest.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 58 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index b8f9c1e..2d248e7 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -234,6 +234,11 @@ struct kreg_region {
#define SVE_NUM_SLICES (1 << SVE_REG_SLICE_BITS)
+#define KVM_SVE_ZREG_SIZE KVM_REG_SIZE(KVM_REG_ARM64_SVE_ZREG(0, 0))
+#define KVM_SVE_PREG_SIZE KVM_REG_SIZE(KVM_REG_ARM64_SVE_PREG(0, 0))
+#define KVM_SVE_SLICES(vcpu) \
+ DIV_ROUND_UP((vcpu)->arch.sve_max_vl, KVM_SVE_ZREG_SIZE)
+
static int sve_reg_region(struct kreg_region *b,
const struct kvm_vcpu *vcpu,
const struct kvm_one_reg *reg)
@@ -251,7 +256,7 @@ static int sve_reg_region(struct kreg_region *b,
if (reg->id >= KVM_REG_ARM64_SVE_ZREG(0, 0) &&
reg->id <= KVM_REG_ARM64_SVE_ZREG(SVE_NUM_ZREGS - 1,
SVE_NUM_SLICES - 1)) {
- slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_ZREG(0, 0));
+ slice_size = KVM_SVE_ZREG_SIZE;
/* Compute start and end of the register: */
offset = SVE_SIG_ZREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
@@ -263,7 +268,7 @@ static int sve_reg_region(struct kreg_region *b,
reg->id <= KVM_REG_ARM64_SVE_FFR(SVE_NUM_SLICES - 1)) {
/* (FFR is P16 for our purposes) */
- slice_size = KVM_REG_SIZE(KVM_REG_ARM64_SVE_PREG(0, 0));
+ slice_size = KVM_SVE_PREG_SIZE;
/* Compute start and end of the register: */
offset = SVE_SIG_PREG_OFFSET(vq, reg_num) - SVE_SIG_REGS_OFFSET;
@@ -358,6 +363,14 @@ static int copy_core_reg_indices(const struct kvm_vcpu *vcpu,
continue;
}
+ /*
+ * The KVM_REG_ARM64_SVE regs must be used instead of
+ * KVM_REG_ARM_CORE for accessing the FPSIMD V-registers on
+ * SVE-enabled vcpus:
+ */
+ if (vcpu_has_sve(vcpu) && core_reg_offset_is_vreg(i))
+ continue;
+
if (uind) {
if (put_user(reg, *uind))
return -EFAULT;
@@ -428,6 +441,44 @@ static int get_timer_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
return copy_to_user(uaddr, &val, KVM_REG_SIZE(reg->id)) ? -EFAULT : 0;
}
+static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu)
+{
+ unsigned int slices;
+
+ if (!vcpu_has_sve(vcpu))
+ return 0;
+
+ slices = KVM_SVE_SLICES(vcpu);
+ return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */);
+}
+
+static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu, u64 __user **uind)
+{
+ unsigned int slices, i, n;
+
+ if (!vcpu_has_sve(vcpu))
+ return 0;
+
+ slices = KVM_SVE_SLICES(vcpu);
+
+ for (i = 0; i < slices; i++) {
+ for (n = 0; n < SVE_NUM_ZREGS; n++) {
+ if (put_user(KVM_REG_ARM64_SVE_ZREG(n, i), (*uind)++))
+ return -EFAULT;
+ }
+
+ for (n = 0; n < SVE_NUM_PREGS; n++) {
+ if (put_user(KVM_REG_ARM64_SVE_PREG(n, i), (*uind)++))
+ return -EFAULT;
+ }
+
+ if (put_user(KVM_REG_ARM64_SVE_FFR(i), (*uind)++))
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
/**
* kvm_arm_num_regs - how many registers do we present via KVM_GET_ONE_REG
*
@@ -438,6 +489,7 @@ unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu)
unsigned long res = 0;
res += num_core_regs(vcpu);
+ res += num_sve_regs(vcpu);
res += kvm_arm_num_sys_reg_descs(vcpu);
res += kvm_arm_get_fw_num_regs(vcpu);
res += NUM_TIMER_REGS;
@@ -458,6 +510,10 @@ int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
if (ret < 0)
return ret;
+ ret = copy_sve_reg_indices(vcpu, &uindices);
+ if (ret)
+ return ret;
+
ret = kvm_arm_copy_fw_reg_indices(vcpu, uindices);
if (ret)
return ret;
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 19/25] arm64/sve: In-kernel vector length availability query interface
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (17 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 18/25] KVM: arm64: Enumerate SVE register indices for KVM_GET_REG_LIST Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 20/25] KVM: arm/arm64: Add hook to finalize the vcpu configuration Dave Martin
` (5 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
KVM will need to interrogate the set of SVE vector lengths
available on the system.
This patch exposes the relevant bits to the kernel, along with a
sve_vq_available() helper to check whether a particular vector
length is supported.
vq_to_bit() and bit_to_vq() are not intended for use outside these
functions.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/include/asm/fpsimd.h | 29 +++++++++++++++++++++++++++++
arch/arm64/kernel/fpsimd.c | 35 ++++++++---------------------------
2 files changed, 37 insertions(+), 27 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index df7a143..ad6d2e4 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -24,10 +24,13 @@
#ifndef __ASSEMBLY__
+#include <linux/bitmap.h>
#include <linux/build_bug.h>
+#include <linux/bug.h>
#include <linux/cache.h>
#include <linux/init.h>
#include <linux/stddef.h>
+#include <linux/types.h>
#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
/* Masks for extracting the FPSR and FPCR from the FPSCR */
@@ -89,6 +92,32 @@ extern u64 read_zcr_features(void);
extern int __ro_after_init sve_max_vl;
extern int __ro_after_init sve_max_virtualisable_vl;
+/* Set of available vector lengths, as vq_to_bit(vq): */
+extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+
+/*
+ * Helpers to translate bit indices in sve_vq_map to VQ values (and
+ * vice versa). This allows find_next_bit() to be used to find the
+ * _maximum_ VQ not exceeding a certain value.
+ */
+static inline unsigned int __vq_to_bit(unsigned int vq)
+{
+ return SVE_VQ_MAX - vq;
+}
+
+static inline unsigned int __bit_to_vq(unsigned int bit)
+{
+ if (WARN_ON(bit >= SVE_VQ_MAX))
+ bit = SVE_VQ_MAX - 1;
+
+ return SVE_VQ_MAX - bit;
+}
+
+/* Ensure vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX before calling this function */
+static inline bool sve_vq_available(unsigned int vq)
+{
+ return test_bit(__vq_to_bit(vq), sve_vq_map);
+}
#ifdef CONFIG_ARM64_SVE
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 09ee264..ac003cb 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -136,7 +136,7 @@ static int sve_default_vl = -1;
int __ro_after_init sve_max_vl = SVE_VL_MIN;
int __ro_after_init sve_max_virtualisable_vl = SVE_VL_MIN;
/* Set of available vector lengths, as vq_to_bit(vq): */
-static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+__ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
/* Set of vector lengths present on at least one cpu: */
static __ro_after_init DECLARE_BITMAP(sve_vq_partial_map, SVE_VQ_MAX);
static void __percpu *efi_sve_state;
@@ -270,25 +270,6 @@ void fpsimd_save(void)
}
/*
- * Helpers to translate bit indices in sve_vq_map to VQ values (and
- * vice versa). This allows find_next_bit() to be used to find the
- * _maximum_ VQ not exceeding a certain value.
- */
-
-static unsigned int vq_to_bit(unsigned int vq)
-{
- return SVE_VQ_MAX - vq;
-}
-
-static unsigned int bit_to_vq(unsigned int bit)
-{
- if (WARN_ON(bit >= SVE_VQ_MAX))
- bit = SVE_VQ_MAX - 1;
-
- return SVE_VQ_MAX - bit;
-}
-
-/*
* All vector length selection from userspace comes through here.
* We're on a slow path, so some sanity-checks are included.
* If things go wrong there's a bug somewhere, but try to fall back to a
@@ -309,8 +290,8 @@ static unsigned int find_supported_vector_length(unsigned int vl)
vl = max_vl;
bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
- vq_to_bit(sve_vq_from_vl(vl)));
- return sve_vl_from_vq(bit_to_vq(bit));
+ __vq_to_bit(sve_vq_from_vl(vl)));
+ return sve_vl_from_vq(__bit_to_vq(bit));
}
#ifdef CONFIG_SYSCTL
@@ -648,7 +629,7 @@ static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
write_sysreg_s(zcr | (vq - 1), SYS_ZCR_EL1); /* self-syncing */
vl = sve_get_vl();
vq = sve_vq_from_vl(vl); /* skip intervening lengths */
- set_bit(vq_to_bit(vq), map);
+ set_bit(__vq_to_bit(vq), map);
}
}
@@ -717,7 +698,7 @@ int sve_verify_vq_map(void)
* Mismatches above sve_max_virtualisable_vl are fine, since
* no guest is allowed to configure ZCR_EL2.LEN to exceed this:
*/
- if (sve_vl_from_vq(bit_to_vq(b)) <= sve_max_virtualisable_vl) {
+ if (sve_vl_from_vq(__bit_to_vq(b)) <= sve_max_virtualisable_vl) {
pr_warn("SVE: cpu%d: Unsupported vector length(s) present\n",
smp_processor_id());
return -EINVAL;
@@ -801,8 +782,8 @@ void __init sve_setup(void)
* so sve_vq_map must have at least SVE_VQ_MIN set.
* If something went wrong, at least try to patch it up:
*/
- if (WARN_ON(!test_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
- set_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map);
+ if (WARN_ON(!test_bit(__vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
+ set_bit(__vq_to_bit(SVE_VQ_MIN), sve_vq_map);
zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
sve_max_vl = sve_vl_from_vq((zcr & ZCR_ELx_LEN_MASK) + 1);
@@ -831,7 +812,7 @@ void __init sve_setup(void)
/* No virtualisable VLs? This is architecturally forbidden. */
sve_max_virtualisable_vl = SVE_VQ_MIN;
else /* b + 1 < SVE_VQ_MAX */
- sve_max_virtualisable_vl = sve_vl_from_vq(bit_to_vq(b + 1));
+ sve_max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1));
if (sve_max_virtualisable_vl > sve_max_vl)
sve_max_virtualisable_vl = sve_max_vl;
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 20/25] KVM: arm/arm64: Add hook to finalize the vcpu configuration
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (18 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 19/25] arm64/sve: In-kernel vector length availability query interface Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 21/25] KVM: arm64/sve: Add pseudo-register for the guest's vector lengths Dave Martin
` (4 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
Some aspects of vcpu configuration can't be completed inside
KVM_VCPU_INIT, but still change API behaviour visible to userspace.
Where such configuration choices affect the register list visible
to userspace, we will need to track whether we have made a
commitment to userspace regarding the list of vcpu registers.
This patch adds a new hook kvm_arm_vcpu_finalize() to capture this
commitment, along with a corresponding check
kvm_arm_vcpu_finalized(). We commit to the register list when
userspace reads it via KVM_GET_REG_LIST, or when the vcpu is fired
up via KVM_RUN.
kvm_arm_vcpu_finalize() is currently a no-op, but future patches
will amend this to handle SVE on arm64.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm/include/asm/kvm_host.h | 4 ++++
arch/arm64/include/asm/kvm_host.h | 4 ++++
virt/kvm/arm/arm.c | 8 ++++++++
3 files changed, 16 insertions(+)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index ca56537..ab7c76b 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -360,4 +360,8 @@ static inline int kvm_arm_setup_stage2(struct kvm *kvm, unsigned long type)
return 0;
}
+/* Commit to the set of vcpu registers currently configured: */
+static inline int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu) { return 0; }
+#define kvm_arm_vcpu_finalized(vcpu) true
+
#endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 77b6f3e..55bf9d0 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -539,4 +539,8 @@ void kvm_arch_free_vm(struct kvm *kvm);
int kvm_arm_setup_stage2(struct kvm *kvm, unsigned long type);
+/* Commit to the set of vcpu registers currently configured: */
+static inline int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu) { return 0; }
+#define kvm_arm_vcpu_finalized(vcpu) true
+
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 9e350fd3..12f9dc2 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -560,6 +560,10 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
if (likely(vcpu->arch.has_run_once))
return 0;
+ ret = kvm_arm_vcpu_finalize(vcpu);
+ if (ret)
+ return ret;
+
vcpu->arch.has_run_once = true;
if (likely(irqchip_in_kernel(kvm))) {
@@ -1121,6 +1125,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
if (unlikely(!kvm_vcpu_initialized(vcpu)))
break;
+ r = kvm_arm_vcpu_finalize(vcpu);
+ if (r)
+ break;
+
r = -EFAULT;
if (copy_from_user(®_list, user_list, sizeof(reg_list)))
break;
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 21/25] KVM: arm64/sve: Add pseudo-register for the guest's vector lengths
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (19 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 20/25] KVM: arm/arm64: Add hook to finalize the vcpu configuration Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 22/25] KVM: arm64/sve: Allow userspace to enable SVE for vcpus Dave Martin
` (3 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
This patch adds a new pseudo-register KVM_REG_ARM64_SVE_VLS to
allow userspace to set and query the set of vector lengths visible
to the guest, along with corresponding storage in struct
kvm_vcpu_arch.
Once the number of SVE register slices visible through the ioctl
interface has been determined, we cannot allow the vector length
set to be changed any more. For this reason, this patch adds
support to track vcpu finalization explicitly.
The new pseudo-register is not exposed yet. Subsequent patches
will allow SVE to be turned on for guest vcpus, making it visible.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/kvm_host.h | 8 ++-
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kvm/guest.c | 108 +++++++++++++++++++++++++++++++++++---
arch/arm64/kvm/reset.c | 9 ++++
4 files changed, 119 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 55bf9d0..82a99f6 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -24,6 +24,7 @@
#include <linux/bitmap.h>
#include <linux/types.h>
+#include <linux/kernel.h>
#include <linux/kvm_types.h>
#include <asm/cpufeature.h>
#include <asm/daifflags.h>
@@ -214,6 +215,7 @@ struct kvm_vcpu_arch {
struct kvm_cpu_context ctxt;
void *sve_state;
unsigned int sve_max_vl;
+ u64 sve_vqs[DIV_ROUND_UP(SVE_VQ_MAX - SVE_VQ_MIN + 1, 64)];
/* HYP configuration */
u64 hcr_el2;
@@ -317,6 +319,7 @@ struct kvm_vcpu_arch {
#define KVM_ARM64_HOST_SVE_IN_USE (1 << 3) /* backup for host TIF_SVE */
#define KVM_ARM64_HOST_SVE_ENABLED (1 << 4) /* SVE enabled for EL0 */
#define KVM_ARM64_GUEST_HAS_SVE (1 << 5) /* SVE exposed to guest */
+#define KVM_ARM64_VCPU_FINALIZED (1 << 6) /* vcpu config completed */
#define vcpu_has_sve(vcpu) (system_supports_sve() && \
((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))
@@ -540,7 +543,8 @@ void kvm_arch_free_vm(struct kvm *kvm);
int kvm_arm_setup_stage2(struct kvm *kvm, unsigned long type);
/* Commit to the set of vcpu registers currently configured: */
-static inline int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu) { return 0; }
-#define kvm_arm_vcpu_finalized(vcpu) true
+int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu);
+#define kvm_arm_vcpu_finalized(vcpu) \
+ ((vcpu)->arch.flags & KVM_ARM64_VCPU_FINALIZED)
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 1ff68fa..6dfbfa3 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -235,6 +235,8 @@ struct kvm_vcpu_events {
KVM_REG_SIZE_U256 | \
((n) << 5) | (i) | 0x400)
#define KVM_REG_ARM64_SVE_FFR(i) KVM_REG_ARM64_SVE_PREG(16, i)
+#define KVM_REG_ARM64_SVE_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \
+ KVM_REG_SIZE_U512 | 0xffff)
/* Device Control API: ARM VGIC */
#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 2d248e7..a636330 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -215,6 +215,65 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
return err;
}
+static bool vq_present(
+ const u64 (*vqs)[DIV_ROUND_UP(SVE_VQ_MAX - SVE_VQ_MIN + 1, 64)],
+ unsigned int vq)
+{
+ unsigned int i = vq - SVE_VQ_MIN;
+
+ return (*vqs)[i / 64] & ((u64)1 << (i % 64));
+}
+
+static int get_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
+{
+ if (WARN_ON(sizeof(vcpu->arch.sve_vqs) != KVM_REG_SIZE(reg->id) ||
+ !sve_vl_valid(vcpu->arch.sve_max_vl)))
+ return -EINVAL;
+
+ if (copy_to_user((void __user *)reg->addr, vcpu->arch.sve_vqs,
+ sizeof(vcpu->arch.sve_vqs)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int set_sve_vls(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
+{
+ unsigned int vq, max_vq;
+
+ u64 vqs[DIV_ROUND_UP(SVE_VQ_MAX - SVE_VQ_MIN + 1, 64)];
+
+ if (kvm_arm_vcpu_finalized(vcpu))
+ return -EPERM; /* too late! */
+
+ if (WARN_ON(sizeof(vcpu->arch.sve_vqs) != KVM_REG_SIZE(reg->id) ||
+ sizeof(vcpu->arch.sve_vqs) != sizeof(vqs) ||
+ !sve_vl_valid(vcpu->arch.sve_max_vl) ||
+ vcpu->arch.sve_state))
+ return -EINVAL;
+
+ if (copy_from_user(vqs, (const void __user *)reg->addr, sizeof(vqs)))
+ return -EFAULT;
+
+ max_vq = 0;
+ for (vq = SVE_VQ_MIN; vq <= SVE_VQ_MAX; ++vq)
+ if (vq_present(&vqs, vq))
+ max_vq = vq;
+
+ for (vq = SVE_VQ_MIN; vq <= max_vq; ++vq)
+ if (vq_present(&vqs, vq) != sve_vq_available(vq))
+ return -EINVAL;
+
+ /* Can't run with no vector lengths at all: */
+ if (max_vq < SVE_VQ_MIN)
+ return -EINVAL;
+
+ vcpu->arch.sve_max_vl = sve_vl_from_vq(max_vq);
+ memcpy(vcpu->arch.sve_vqs, vqs, sizeof(vcpu->arch.sve_vqs));
+
+ return 0;
+}
+
struct kreg_region {
char *kptr;
size_t size;
@@ -296,9 +355,21 @@ static int sve_reg_region(struct kreg_region *b,
static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
{
struct kreg_region kreg;
+ int ret;
char __user *uptr = (char __user *)reg->addr;
- if (!vcpu_has_sve(vcpu) || sve_reg_region(&kreg, vcpu, reg))
+ if (!vcpu_has_sve(vcpu))
+ return -ENOENT;
+
+ if (reg->id == KVM_REG_ARM64_SVE_VLS)
+ return get_sve_vls(vcpu, reg);
+
+ /* Finalize the number of slices per SVE register: */
+ ret = kvm_arm_vcpu_finalize(vcpu);
+ if (ret)
+ return ret;
+
+ if (sve_reg_region(&kreg, vcpu, reg))
return -ENOENT;
if (copy_to_user(uptr, kreg.kptr, kreg.size) ||
@@ -311,9 +382,21 @@ static int get_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
static int set_sve_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
{
struct kreg_region kreg;
+ int ret;
char __user *uptr = (char __user *)reg->addr;
- if (!vcpu_has_sve(vcpu) || sve_reg_region(&kreg, vcpu, reg))
+ if (!vcpu_has_sve(vcpu))
+ return -ENOENT;
+
+ if (reg->id == KVM_REG_ARM64_SVE_VLS)
+ return set_sve_vls(vcpu, reg);
+
+ /* Finalize the number of slices per SVE register: */
+ ret = kvm_arm_vcpu_finalize(vcpu);
+ if (ret)
+ return ret;
+
+ if (sve_reg_region(&kreg, vcpu, reg))
return -ENOENT;
if (copy_from_user(kreg.kptr, uptr, kreg.size))
@@ -449,30 +532,43 @@ static unsigned long num_sve_regs(const struct kvm_vcpu *vcpu)
return 0;
slices = KVM_SVE_SLICES(vcpu);
- return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */);
+ return slices * (SVE_NUM_PREGS + SVE_NUM_ZREGS + 1 /* FFR */)
+ + 1; /* KVM_REG_ARM64_SVE_VLS */
}
static int copy_sve_reg_indices(const struct kvm_vcpu *vcpu, u64 __user **uind)
{
+ u64 reg;
unsigned int slices, i, n;
if (!vcpu_has_sve(vcpu))
return 0;
+ /*
+ * Enumerate this first, so that userspace can save/restore in
+ * the order reported by KVM_GET_REG_LIST:
+ */
+ reg = KVM_REG_ARM64_SVE_VLS;
+ if (put_user(reg, (*uind)++))
+ return -EFAULT;
+
slices = KVM_SVE_SLICES(vcpu);
for (i = 0; i < slices; i++) {
for (n = 0; n < SVE_NUM_ZREGS; n++) {
- if (put_user(KVM_REG_ARM64_SVE_ZREG(n, i), (*uind)++))
+ reg = KVM_REG_ARM64_SVE_ZREG(n, i);
+ if (put_user(reg, (*uind)++))
return -EFAULT;
}
for (n = 0; n < SVE_NUM_PREGS; n++) {
- if (put_user(KVM_REG_ARM64_SVE_PREG(n, i), (*uind)++))
+ reg = KVM_REG_ARM64_SVE_PREG(n, i);
+ if (put_user(reg, (*uind)++))
return -EFAULT;
}
- if (put_user(KVM_REG_ARM64_SVE_FFR(i), (*uind)++))
+ reg = KVM_REG_ARM64_SVE_FFR(i);
+ if (put_user(reg, (*uind)++))
return -EFAULT;
}
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index b72a3dd..1379fb2 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -98,6 +98,15 @@ int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
return r;
}
+int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu)
+{
+ if (likely(kvm_arm_vcpu_finalized(vcpu)))
+ return 0;
+
+ vcpu->arch.flags |= KVM_ARM64_VCPU_FINALIZED;
+ return 0;
+}
+
/**
* kvm_reset_vcpu - sets core registers and sys_regs to reset value
* @vcpu: The VCPU pointer
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 22/25] KVM: arm64/sve: Allow userspace to enable SVE for vcpus
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (20 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 21/25] KVM: arm64/sve: Add pseudo-register for the guest's vector lengths Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 23/25] KVM: arm64: Add a capabillity to advertise SVE support Dave Martin
` (2 subsequent siblings)
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
Now that all the pieces are in place, this patch offers a new flag
KVM_ARM_VCPU_SVE that userspace can pass to KVM_ARM_VCPU_INIT to
turn on SVE for the guest, on a per-vcpu basis.
As part of this, support for initialisation and reset of the SVE
vector length set and registers is added in the appropriate places.
Allocation SVE registers is deferred until kvm_arm_vcpu_finalize(),
by which time the size of the registers is known.
Setting the vector lengths supported by the vcpu is considered
configuration of the emulated hardware rather than runtime
configuration, so no support is offered for changing the vector
lengths of an existing vcpu across reset.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/kvm_host.h | 2 +-
arch/arm64/include/uapi/asm/kvm.h | 1 +
arch/arm64/kvm/reset.c | 78 ++++++++++++++++++++++++++++++++++++++-
3 files changed, 79 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 82a99f6..f77b780 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -45,7 +45,7 @@
#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS
-#define KVM_VCPU_MAX_FEATURES 4
+#define KVM_VCPU_MAX_FEATURES 5
#define KVM_REQ_SLEEP \
KVM_ARCH_REQ_FLAGS(0, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 6dfbfa3..fc613af 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -102,6 +102,7 @@ struct kvm_regs {
#define KVM_ARM_VCPU_EL1_32BIT 1 /* CPU running a 32bit VM */
#define KVM_ARM_VCPU_PSCI_0_2 2 /* CPU uses PSCI v0.2 */
#define KVM_ARM_VCPU_PMU_V3 3 /* Support guest PMUv3 */
+#define KVM_ARM_VCPU_SVE 4 /* enable SVE for this CPU */
struct kvm_vcpu_init {
__u32 target;
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 1379fb2..5ff2360 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -23,11 +23,13 @@
#include <linux/kvm_host.h>
#include <linux/kvm.h>
#include <linux/hw_breakpoint.h>
+#include <linux/slab.h>
#include <kvm/arm_arch_timer.h>
#include <asm/cpufeature.h>
#include <asm/cputype.h>
+#include <asm/fpsimd.h>
#include <asm/ptrace.h>
#include <asm/kvm_arm.h>
#include <asm/kvm_asm.h>
@@ -98,11 +100,77 @@ int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
return r;
}
+static size_t vcpu_sve_state_size(struct kvm_vcpu *vcpu)
+{
+ if (WARN_ON(!sve_vl_valid(vcpu->arch.sve_max_vl)))
+ return 0;
+
+ return SVE_SIG_REGS_SIZE(sve_vq_from_vl(vcpu->arch.sve_max_vl));
+}
+
+static int kvm_reset_sve(struct kvm_vcpu *vcpu)
+{
+ unsigned int vq;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* If resetting an already-configured vcpu, just zero the SVE regs: */
+ if (vcpu->arch.sve_state) {
+ size_t size = vcpu_sve_state_size(vcpu);
+
+ if (!size)
+ return -EINVAL;
+
+ if (WARN_ON(!vcpu_has_sve(vcpu)))
+ return -EINVAL;
+
+ memset(vcpu->arch.sve_state, 0, size);
+ return 0;
+ }
+
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ return -EINVAL;
+
+ /* If the full set of host vector lengths cannot be used, give up: */
+ if (sve_max_virtualisable_vl < sve_max_vl)
+ return -EINVAL;
+
+ /* Default to the set of vector lengths supported by the host */
+ vcpu->arch.sve_max_vl = sve_max_vl;
+ for (vq = SVE_VQ_MIN; vq <= sve_vq_from_vl(sve_max_vl); ++vq) {
+ unsigned int i = vq - SVE_VQ_MIN;
+
+ if (sve_vq_available(vq))
+ vcpu->arch.sve_vqs[i / 64] |= (u64)1 << (i % 64);
+ }
+
+ /*
+ * Userspace can still customize the vector lengths by writing
+ * KVM_REG_ARM64_SVE_VLS. Allocation is deferred until
+ * kvm_arm_vcpu_finalize(), which freezes the configuration.
+ */
+ vcpu->arch.flags |= KVM_ARM64_GUEST_HAS_SVE;
+
+ return 0;
+}
+
int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu)
{
if (likely(kvm_arm_vcpu_finalized(vcpu)))
return 0;
+ if (vcpu_has_sve(vcpu)) {
+ size_t size = vcpu_sve_state_size(vcpu);
+
+ if (!size)
+ return -EINVAL;
+
+ vcpu->arch.sve_state = kzalloc(size, GFP_KERNEL);
+ if (!vcpu->arch.sve_state)
+ return -ENOMEM;
+ }
+
vcpu->arch.flags |= KVM_ARM64_VCPU_FINALIZED;
return 0;
}
@@ -113,12 +181,20 @@ int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu)
*
* This function finds the right table above and sets the registers on
* the virtual CPU struct to their architecturally defined reset
- * values.
+ * values, except for registers whose reset is deferred until
+ * kvm_arm_vcpu_finalize().
*/
int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
{
+ int ret;
const struct kvm_regs *cpu_reset;
+ if (test_bit(KVM_ARM_VCPU_SVE, vcpu->arch.features)) {
+ ret = kvm_reset_sve(vcpu);
+ if (ret)
+ return ret;
+ }
+
switch (vcpu->arch.target) {
default:
if (test_bit(KVM_ARM_VCPU_EL1_32BIT, vcpu->arch.features)) {
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 23/25] KVM: arm64: Add a capabillity to advertise SVE support
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (21 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 22/25] KVM: arm64/sve: Allow userspace to enable SVE for vcpus Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 24/25] KVM: Document errors for KVM_GET_ONE_REG and KVM_SET_ONE_REG Dave Martin
2019-01-17 20:33 ` [PATCH v4 25/25] KVM: arm64/sve: Document KVM API extensions for SVE Dave Martin
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
To provide a uniform way to check for KVM SVE support amongst other
features, this patch adds a suitable capability KVM_CAP_ARM_SVE,
and reports it as present when SVE is available.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kvm/reset.c | 8 ++++++++
include/uapi/linux/kvm.h | 1 +
2 files changed, 9 insertions(+)
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 5ff2360..6e3482e 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -35,6 +35,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_coproc.h>
#include <asm/kvm_mmu.h>
+#include <asm/virt.h>
/* Maximum phys_shift supported for any VM on this host */
static u32 kvm_ipa_limit;
@@ -93,6 +94,9 @@ int kvm_arch_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_ARM_VM_IPA_SIZE:
r = kvm_ipa_limit;
break;
+ case KVM_CAP_ARM_SVE:
+ r = system_supports_sve();
+ break;
default:
r = 0;
}
@@ -115,6 +119,10 @@ static int kvm_reset_sve(struct kvm_vcpu *vcpu)
if (!system_supports_sve())
return -EINVAL;
+ /* Verify that KVM startup enforced this when SVE was detected: */
+ if (WARN_ON(!has_vhe()))
+ return -EINVAL;
+
/* If resetting an already-configured vcpu, just zero the SVE regs: */
if (vcpu->arch.sve_state) {
size_t size = vcpu_sve_state_size(vcpu);
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index dc77a5a..4ea0d92 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -988,6 +988,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ARM_VM_IPA_SIZE 165
#define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166
#define KVM_CAP_HYPERV_CPUID 167
+#define KVM_CAP_ARM_SVE 168
#ifdef KVM_CAP_IRQ_ROUTING
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 24/25] KVM: Document errors for KVM_GET_ONE_REG and KVM_SET_ONE_REG
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (22 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 23/25] KVM: arm64: Add a capabillity to advertise SVE support Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
2019-01-17 20:33 ` [PATCH v4 25/25] KVM: arm64/sve: Document KVM API extensions for SVE Dave Martin
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
KVM_GET_ONE_REG and KVM_SET_ONE_REG return some error codes that
are not documented (but hopefully not surprising either). To give
an indication of what these may mean, this patch adds brief
documentation.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Documentation/virtual/kvm/api.txt | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 097b8ba..cb15f2a 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1854,6 +1854,9 @@ Architectures: all
Type: vcpu ioctl
Parameters: struct kvm_one_reg (in)
Returns: 0 on success, negative value on failure
+Errors:
+ ENOENT: no such register
+ EINVAL: other errors, such as bad size encoding for a known register
struct kvm_one_reg {
__u64 id;
@@ -2175,6 +2178,9 @@ Architectures: all
Type: vcpu ioctl
Parameters: struct kvm_one_reg (in and out)
Returns: 0 on success, negative value on failure
+Errors:
+ ENOENT: no such register
+ EINVAL: other errors, such as bad size encoding for a known register
This ioctl allows to receive the value of a single register implemented
in a vcpu. The register to read is indicated by the "id" field of the
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4 25/25] KVM: arm64/sve: Document KVM API extensions for SVE
2019-01-17 20:33 [PATCH v4 00/25] KVM: arm64: SVE guest support Dave Martin
` (23 preceding siblings ...)
2019-01-17 20:33 ` [PATCH v4 24/25] KVM: Document errors for KVM_GET_ONE_REG and KVM_SET_ONE_REG Dave Martin
@ 2019-01-17 20:33 ` Dave Martin
24 siblings, 0 replies; 32+ messages in thread
From: Dave Martin @ 2019-01-17 20:33 UTC (permalink / raw)
To: kvmarm
Cc: Peter Maydell, Okamoto Takayuki, Christoffer Dall,
Ard Biesheuvel, Marc Zyngier, Catalin Marinas, Will Deacon,
Julien Grall, Alex Bennée, linux-arm-kernel
This patch adds sections to the KVM API documentation describing
the extensions for supporting the Scalable Vector Extension (SVE)
in guests.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since RFC v3:
* Rewrote the bulk of the documentation to document the new vector
length set control interface via KVM_REG_ARM64_SVE_VLS, and describe
the new vcpu init / register access semantics.
* Fix documentation regarding which SVE Zn register bits must be
accessed in order to get at Vn on an SVE-enabled vcpu.
---
Documentation/virtual/kvm/api.txt | 61 +++++++++++++++++++++++++++++++++++++--
1 file changed, 58 insertions(+), 3 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index cb15f2a..d45e93c 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1856,6 +1856,7 @@ Parameters: struct kvm_one_reg (in)
Returns: 0 on success, negative value on failure
Errors:
ENOENT: no such register
+ EPERM: register access forbidden for architecture-dependent reasons
EINVAL: other errors, such as bad size encoding for a known register
struct kvm_one_reg {
@@ -2110,13 +2111,20 @@ Specifically:
0x6030 0000 0010 004c SPSR_UND 64 spsr[KVM_SPSR_UND]
0x6030 0000 0010 004e SPSR_IRQ 64 spsr[KVM_SPSR_IRQ]
0x6060 0000 0010 0050 SPSR_FIQ 64 spsr[KVM_SPSR_FIQ]
- 0x6040 0000 0010 0054 V0 128 fp_regs.vregs[0]
- 0x6040 0000 0010 0058 V1 128 fp_regs.vregs[1]
+ 0x6040 0000 0010 0054 V0 128 fp_regs.vregs[0] (*)
+ 0x6040 0000 0010 0058 V1 128 fp_regs.vregs[1] (*)
...
- 0x6040 0000 0010 00d0 V31 128 fp_regs.vregs[31]
+ 0x6040 0000 0010 00d0 V31 128 fp_regs.vregs[31] (*)
0x6020 0000 0010 00d4 FPSR 32 fp_regs.fpsr
0x6020 0000 0010 00d5 FPCR 32 fp_regs.fpcr
+(*) These encodings are not accepted for SVE-enabled vcpus. See
+ KVM_ARM_VCPU_INIT.
+
+ The equivalent register content can be accessed via bits [127:0] of
+ the corresponding SVE Zn registers instead for vcpus that have SVE
+ enabled (see below).
+
arm64 CCSIDR registers are demultiplexed by CSSELR value:
0x6020 0000 0011 00 <csselr:8>
@@ -2126,6 +2134,50 @@ arm64 system registers have the following id bit patterns:
arm64 firmware pseudo-registers have the following bit pattern:
0x6030 0000 0014 <regno:16>
+arm64 SVE registers have the following bit patterns:
+ 0x6080 0000 0015 00 <n:5> <slice:5> Zn bits[2048*slice + 2047 : 2048*slice]
+ 0x6050 0000 0015 04 <n:4> <slice:5> Pn bits[256*slice + 255 : 256*slice]
+ 0x6050 0000 0015 060 <slice:5> FFR bits[256*slice + 255 : 256*slice]
+ 0x6060 0000 0015 ffff KVM_REG_ARM64_SVE_VLS pseudo-register
+
+These registers are only accessible on vcpus for which SVE is enabled.
+See KVM_ARM_VCPU_INIT for details.
+
+KVM_REG_ARM64_SVE_VLS is a pseudo-register that allows the set of vector
+lengths supported by the vcpu to be discovered and configured by
+userspace. When transferred to or from user memory via KVM_GET_ONE_REG
+or KVM_SET_ONE_REG, the value of this register is of type __u64[8], and
+encodes the set of vector lengths as follows:
+
+__u64 vector_lengths[8];
+
+if (vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX &&
+ ((vector_lengths[(vq - 1) / 64] >> ((vq - 1) % 64)) & 1))
+ /* Vector length vq * 16 bytes supported */
+else
+ /* Vector length vq * 16 bytes not supported */
+
+(See Documentation/arm64/sve.txt for an explanation of the "vq"
+nomenclature.)
+
+KVM_REG_ARM64_SVE_VLS is only accessible after KVM_ARM_VCPU_INIT.
+KVM_ARM_VCPU_INIT initialises it to the best set of vector lengths that
+the host supports.
+
+Userspace may subsequently modify it if desired until the vcpu
+configuration is finalized by accessing the guest's SVE registers, or
+enumerating them via KVM_GET_REG_LIST, or setting the cpu running via
+KVM_RUN.
+
+Apart from simply removing all vector lengths from the host set that
+exceed some value, support for arbitrarily chosen sets of vector lengths
+is hardware-dependent and may not be available. Attempting to configure
+an invalid set of vector lengths via KVM_SET_ONE_REG will fail with
+EINVAL.
+
+After the vcpu's configuration is finalized, further attempts to write
+this register will fail with EPERM.
+
MIPS registers are mapped using the lower 32 bits. The upper 16 of that is
the register group type:
@@ -2180,6 +2232,7 @@ Parameters: struct kvm_one_reg (in and out)
Returns: 0 on success, negative value on failure
Errors:
ENOENT: no such register
+ EPERM: register access forbidden for architecture-dependent reasons
EINVAL: other errors, such as bad size encoding for a known register
This ioctl allows to receive the value of a single register implemented
@@ -2672,6 +2725,8 @@ Possible features:
Depends on KVM_CAP_ARM_PSCI_0_2.
- KVM_ARM_VCPU_PMU_V3: Emulate PMUv3 for the CPU.
Depends on KVM_CAP_ARM_PMU_V3.
+ - KVM_ARM_VCPU_SVE: Enables SVE for the CPU (arm64 only).
+ Depends on KVM_CAP_ARM_SVE.
4.83 KVM_ARM_PREFERRED_TARGET
--
2.1.4
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread