From: Julien Thierry <julien.thierry@arm.com>
To: Dave Martin <Dave.Martin@arm.com>, kvmarm@lists.cs.columbia.edu
Cc: Okamoto Takayuki <tokamoto@jp.fujitsu.com>,
Christoffer Dall <cdall@kernel.org>,
Ard Biesheuvel <ard.biesheuvel@linaro.org>,
Marc Zyngier <marc.zyngier@arm.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will.deacon@arm.com>,
Zhang Lei <zhang.lei@jp.fujitsu.com>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v5 06/26] arm64/sve: Check SVE virtualisability
Date: Wed, 20 Feb 2019 11:12:49 +0000 [thread overview]
Message-ID: <64764e35-c243-e642-a868-5b9e221813c7@arm.com> (raw)
In-Reply-To: <1550519559-15915-7-git-send-email-Dave.Martin@arm.com>
Hi Dave,
On 18/02/2019 19:52, Dave Martin wrote:
> Due to the way the effective SVE vector length is controlled and
> trapped at different exception levels, certain mismatches in the
> sets of vector lengths supported by different physical CPUs in the
> system may prevent straightforward virtualisation of SVE at parity
> with the host.
>
> This patch analyses the extent to which SVE can be virtualised
> safely without interfering with migration of vcpus between physical
> CPUs, and rejects late secondary CPUs that would erode the
> situation further.
>
> It is left up to KVM to decide what to do with this information.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/include/asm/fpsimd.h | 1 +
> arch/arm64/kernel/cpufeature.c | 2 +-
> arch/arm64/kernel/fpsimd.c | 86 ++++++++++++++++++++++++++++++++++-------
> 3 files changed, 73 insertions(+), 16 deletions(-)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index dd1ad39..964adc9 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -87,6 +87,7 @@ extern void sve_kernel_enable(const struct arm64_cpu_capabilities *__unused);
> extern u64 read_zcr_features(void);
>
> extern int __ro_after_init sve_max_vl;
> +extern int __ro_after_init sve_max_virtualisable_vl;
>
> #ifdef CONFIG_ARM64_SVE
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index f6d84e2..5eaacb4 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -1825,7 +1825,7 @@ static void verify_sve_features(void)
> unsigned int len = zcr & ZCR_ELx_LEN_MASK;
>
> if (len < safe_len || sve_verify_vq_map()) {
> - pr_crit("CPU%d: SVE: required vector length(s) missing\n",
> + pr_crit("CPU%d: SVE: vector length support mismatch\n",
> smp_processor_id());
> cpu_die_early();
> }
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 62c37f0..64729e2 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -18,6 +18,7 @@
> */
>
> #include <linux/bitmap.h>
> +#include <linux/bitops.h>
> #include <linux/bottom_half.h>
> #include <linux/bug.h>
> #include <linux/cache.h>
> @@ -48,6 +49,7 @@
> #include <asm/sigcontext.h>
> #include <asm/sysreg.h>
> #include <asm/traps.h>
> +#include <asm/virt.h>
>
> #define FPEXC_IOF (1 << 0)
> #define FPEXC_DZF (1 << 1)
> @@ -130,14 +132,18 @@ static int sve_default_vl = -1;
>
> /* Maximum supported vector length across all CPUs (initially poisoned) */
> int __ro_after_init sve_max_vl = SVE_VL_MIN;
> +int __ro_after_init sve_max_virtualisable_vl = SVE_VL_MIN;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +/* Set of vector lengths present on at least one cpu: */
> +static __ro_after_init DECLARE_BITMAP(sve_vq_partial_map, SVE_VQ_MAX);
> static void __percpu *efi_sve_state;
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern __ro_after_init DECLARE_BITMAP(sve_vq_partial_map, SVE_VQ_MAX);
> extern void __percpu *efi_sve_state;
>
> #endif /* ! CONFIG_ARM64_SVE */
> @@ -623,12 +629,6 @@ int sve_get_current_vl(void)
> return sve_prctl_status(0);
> }
>
> -/*
> - * Bitmap for temporary storage of the per-CPU set of supported vector lengths
> - * during secondary boot.
> - */
> -static DECLARE_BITMAP(sve_secondary_vq_map, SVE_VQ_MAX);
> -
> static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
> {
> unsigned int vq, vl;
> @@ -650,6 +650,7 @@ static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
> void __init sve_init_vq_map(void)
> {
> sve_probe_vqs(sve_vq_map);
> + bitmap_copy(sve_vq_partial_map, sve_vq_map, SVE_VQ_MAX);
> }
>
> /*
> @@ -658,25 +659,58 @@ void __init sve_init_vq_map(void)
> */
> void sve_update_vq_map(void)
> {
> - sve_probe_vqs(sve_secondary_vq_map);
> - bitmap_and(sve_vq_map, sve_vq_map, sve_secondary_vq_map, SVE_VQ_MAX);
> + DECLARE_BITMAP(tmp_map, SVE_VQ_MAX);
> +
> + sve_probe_vqs(tmp_map);
> + bitmap_and(sve_vq_map, sve_vq_map, tmp_map, SVE_VQ_MAX);
> + bitmap_or(sve_vq_partial_map, sve_vq_partial_map, tmp_map, SVE_VQ_MAX);
> }
>
> /* Check whether the current CPU supports all VQs in the committed set */
> int sve_verify_vq_map(void)
> {
> - int ret = 0;
> + DECLARE_BITMAP(tmp_map, SVE_VQ_MAX);
> + unsigned long b;
>
> - sve_probe_vqs(sve_secondary_vq_map);
> - bitmap_andnot(sve_secondary_vq_map, sve_vq_map, sve_secondary_vq_map,
> - SVE_VQ_MAX);
> - if (!bitmap_empty(sve_secondary_vq_map, SVE_VQ_MAX)) {
> + sve_probe_vqs(tmp_map);
> +
> + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX);
> + if (bitmap_intersects(tmp_map, sve_vq_map, SVE_VQ_MAX)) {
> pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
> smp_processor_id());
> - ret = -EINVAL;
> + return -EINVAL;
> }
>
> - return ret;
> + if (!IS_ENABLED(CONFIG_KVM) || !is_hyp_mode_available())
> + return 0;
> +
> + /*
> + * For KVM, it is necessary to ensure that this CPU doesn't
> + * support any vector length that guests may have probed as
> + * unsupported.
> + */
> +
> + /* Recover the set of supported VQs: */
> + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX);
> + /* Find VQs supported that are not globally supported: */
> + bitmap_andnot(tmp_map, tmp_map, sve_vq_map, SVE_VQ_MAX);
> +
> + /* Find the lowest such VQ, if any: */
> + b = find_last_bit(tmp_map, SVE_VQ_MAX);
> + if (b >= SVE_VQ_MAX)
> + return 0; /* no mismatches */
> +
> + /*
> + * Mismatches above sve_max_virtualisable_vl are fine, since
> + * no guest is allowed to configure ZCR_EL2.LEN to exceed this:
> + */
> + if (sve_vl_from_vq(bit_to_vq(b)) <= sve_max_virtualisable_vl) {
> + pr_warn("SVE: cpu%d: Unsupported vector length(s) present\n",
Nit: might be good to specify that the vector length is unsupported for
virtualisation.
Also, since KVM is the one deciding what to do with the information,
should we have a warning here? But I can understand that knowing which
CPUs are introducing unsupported vector length, maybe using pr_devel()
instead of pr_warn()
In any case, the logic looks good to me:
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Cheers,
--
Julien Thierry
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-02-20 11:13 UTC|newest]
Thread overview: 96+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-18 19:52 [PATCH v5 00/26] KVM: arm64: SVE guest support Dave Martin
2019-02-18 19:52 ` [PATCH v5 01/26] KVM: Documentation: Document arm64 core registers in detail Dave Martin
2019-02-21 11:48 ` Julien Grall
2019-02-26 12:05 ` Dave Martin
2019-02-21 11:57 ` Peter Maydell
2019-02-18 19:52 ` [PATCH v5 02/26] arm64: fpsimd: Always set TIF_FOREIGN_FPSTATE on task state flush Dave Martin
2019-02-21 12:39 ` Julien Grall
2019-02-26 12:06 ` Dave Martin
2019-02-26 12:35 ` Julien Grall
2019-02-18 19:52 ` [PATCH v5 03/26] KVM: arm64: Delete orphaned declaration for __fpsimd_enabled() Dave Martin
2019-02-18 19:52 ` [PATCH v5 04/26] KVM: arm64: Refactor kvm_arm_num_regs() for easier maintenance Dave Martin
2019-02-18 19:52 ` [PATCH v5 05/26] KVM: arm64: Add missing #include of <linux/bitmap.h> to kvm_host.h Dave Martin
2019-02-20 15:23 ` Mark Rutland
2019-02-26 12:06 ` Dave Martin
2019-02-26 12:31 ` Mark Rutland
2019-02-26 12:33 ` Dave Martin
2019-02-26 12:40 ` Mark Rutland
2019-02-18 19:52 ` [PATCH v5 06/26] arm64/sve: Check SVE virtualisability Dave Martin
2019-02-20 11:12 ` Julien Thierry [this message]
2019-02-26 12:06 ` Dave Martin
2019-03-01 12:39 ` Julien Thierry
2019-03-01 14:44 ` Dave Martin
2019-02-21 13:36 ` Julien Grall
2019-02-26 12:06 ` Dave Martin
2019-02-26 15:43 ` Julien Grall
2019-02-18 19:52 ` [PATCH v5 07/26] arm64/sve: Clarify role of the VQ map maintenance functions Dave Martin
2019-02-20 11:43 ` Julien Thierry
2019-02-26 12:06 ` Dave Martin
2019-02-21 13:46 ` Julien Grall
2019-02-26 12:07 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 08/26] arm64/sve: Enable SVE state tracking for non-task contexts Dave Martin
2019-02-22 15:26 ` Julien Grall
2019-02-26 12:07 ` Dave Martin
2019-02-26 15:49 ` Julien Grall
2019-02-26 15:58 ` Dave Martin
2019-02-26 15:59 ` Julien Grall
2019-02-26 16:03 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 09/26] KVM: arm64: Add a vcpu flag to control SVE visibility for the guest Dave Martin
2019-02-18 19:52 ` [PATCH v5 10/26] KVM: arm64: Propagate vcpu into read_id_reg() Dave Martin
2019-02-18 19:52 ` [PATCH v5 11/26] KVM: arm64: Extend reset_unknown() to handle mixed RES0/UNKNOWN registers Dave Martin
2019-02-20 13:33 ` Julien Thierry
2019-02-26 12:07 ` Dave Martin
2019-02-22 16:04 ` Julien Grall
2019-02-18 19:52 ` [PATCH v5 12/26] KVM: arm64: Support runtime sysreg visibility filtering Dave Martin
2019-02-20 14:33 ` Julien Thierry
2019-02-26 12:07 ` Dave Martin
2019-02-20 15:37 ` Mark Rutland
2019-02-26 12:12 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 13/26] KVM: arm64/sve: System register context switch and access support Dave Martin
2019-02-20 16:48 ` Julien Thierry
2019-02-26 16:32 ` Julien Grall
2019-02-26 17:01 ` Dave Martin
2019-02-27 12:02 ` Julien Grall
2019-02-27 13:50 ` Dave Martin
2019-02-27 14:17 ` Julien Grall
2019-02-27 14:38 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 14/26] KVM: arm64/sve: Context switch the SVE registers Dave Martin
2019-02-20 16:19 ` Mark Rutland
2019-02-26 12:13 ` Dave Martin
2019-02-20 16:46 ` Julien Thierry
2019-02-26 12:13 ` Dave Martin
2019-02-26 16:56 ` Julien Grall
2019-02-27 13:37 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 15/26] KVM: Allow 2048-bit register access via ioctl interface Dave Martin
2019-02-18 19:52 ` [PATCH v5 16/26] KVM: arm64: Add missing #include of <linux/string.h> in guest.c Dave Martin
2019-02-18 19:52 ` [PATCH v5 17/26] KVM: arm64: Reject ioctl access to FPSIMD V-regs on SVE vcpus Dave Martin
2019-02-21 12:06 ` Julien Thierry
2019-02-26 12:13 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 18/26] KVM: arm64/sve: Add SVE support to register access ioctl interface Dave Martin
2019-02-21 15:23 ` Julien Thierry
2019-02-26 12:13 ` Dave Martin
2019-03-01 13:03 ` Julien Thierry
2019-03-01 14:45 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 19/26] KVM: arm64: Enumerate SVE register indices for KVM_GET_REG_LIST Dave Martin
2019-02-21 16:28 ` Julien Thierry
2019-02-18 19:52 ` [PATCH v5 20/26] arm64/sve: In-kernel vector length availability query interface Dave Martin
2019-02-18 19:52 ` [PATCH v5 21/26] KVM: arm/arm64: Add hook to finalize the vcpu configuration Dave Martin
2019-02-18 19:52 ` [PATCH v5 22/26] KVM: arm64/sve: Add pseudo-register for the guest's vector lengths Dave Martin
2019-02-21 17:48 ` Julien Thierry
2019-02-26 12:13 ` Dave Martin
2019-03-01 13:28 ` Julien Thierry
2019-03-01 14:55 ` Dave Martin
2019-03-07 13:47 ` Marc Zyngier
2019-03-07 15:30 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 23/26] KVM: arm64/sve: Allow userspace to enable SVE for vcpus Dave Martin
2019-02-22 9:05 ` Julien Thierry
2019-02-26 12:13 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 24/26] KVM: arm64: Add a capabillity to advertise SVE support Dave Martin
2019-02-22 9:10 ` Julien Thierry
2019-02-26 12:14 ` Dave Martin
2019-02-18 19:52 ` [PATCH v5 25/26] KVM: Document errors for KVM_GET_ONE_REG and KVM_SET_ONE_REG Dave Martin
2019-02-18 19:52 ` [PATCH v5 26/26] KVM: arm64/sve: Document KVM API extensions for SVE Dave Martin
2019-02-20 15:47 ` [PATCH v5 00/26] KVM: arm64: SVE guest support Dave Martin
2019-03-03 2:40 ` Zhang, Lei
2019-03-05 9:47 ` Dave Martin
2019-03-08 7:06 ` Zhang, Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=64764e35-c243-e642-a868-5b9e221813c7@arm.com \
--to=julien.thierry@arm.com \
--cc=Dave.Martin@arm.com \
--cc=ard.biesheuvel@linaro.org \
--cc=catalin.marinas@arm.com \
--cc=cdall@kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=marc.zyngier@arm.com \
--cc=tokamoto@jp.fujitsu.com \
--cc=will.deacon@arm.com \
--cc=zhang.lei@jp.fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).