From: Mark Brown <broonie@kernel.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Shuah Khan <skhan@linuxfoundation.org>, Shuah Khan <shuah@kernel.org> Cc: Alan Hayward <alan.hayward@arm.com>, Luis Machado <luis.machado@arm.com>, Salil Akerkar <Salil.Akerkar@arm.com>, Basant Kumar Dwivedi <Basant.KumarDwivedi@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown <broonie@kernel.org> Subject: [PATCH v2 33/42] arm64/sme: Save and restore streaming mode over EFI runtime calls Date: Mon, 18 Oct 2021 20:08:49 +0100 [thread overview] Message-ID: <20211018190858.2119209-34-broonie@kernel.org> (raw) In-Reply-To: <20211018190858.2119209-1-broonie@kernel.org> When saving and restoring the floating point state over an EFI runtime call ensure that we handle streaming mode, only handling FFR if we are not in streaming mode and ensuring that we are in normal mode over the call into runtime services. We currently assume that ZA will not be modified by runtime services, the specification is not yet finalised so this may need updating if that changes. Signed-off-by: Mark Brown <broonie@kernel.org> --- arch/arm64/kernel/fpsimd.c | 47 +++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index d3665d7f6e78..bd9a0eddf533 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1049,21 +1049,25 @@ int vec_verify_vq_map(enum vec_type type) static void __init sve_efi_setup(void) { - struct vl_info *info = &vl_info[ARM64_VEC_SVE]; + int max_vl = 0; + int i; if (!IS_ENABLED(CONFIG_EFI)) return; + for (i = 0; i < ARRAY_SIZE(vl_info); i++) + max_vl = max(vl_info[i].max_vl, max_vl); + /* * alloc_percpu() warns and prints a backtrace if this goes wrong. * This is evidence of a crippled system and we are returning void, * so no attempt is made to handle this situation here. */ - if (!sve_vl_valid(info->max_vl)) + if (!sve_vl_valid(max_vl)) goto fail; efi_sve_state = __alloc_percpu( - SVE_SIG_REGS_SIZE(sve_vq_from_vl(info->max_vl)), SVE_VQ_BYTES); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(max_vl)), SVE_VQ_BYTES); if (!efi_sve_state) goto fail; @@ -1830,6 +1834,7 @@ EXPORT_SYMBOL(kernel_neon_end); static DEFINE_PER_CPU(struct user_fpsimd_state, efi_fpsimd_state); static DEFINE_PER_CPU(bool, efi_fpsimd_state_used); static DEFINE_PER_CPU(bool, efi_sve_state_used); +static DEFINE_PER_CPU(bool, efi_sm_state); /* * EFI runtime services support functions @@ -1864,12 +1869,28 @@ void __efi_fpsimd_begin(void) */ if (system_supports_sve() && likely(efi_sve_state)) { char *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + u64 svcr; __this_cpu_write(efi_sve_state_used, true); + /* If we are in streaming mode don't touch FFR */ + if (system_supports_sme()) { + svcr = read_sysreg_s(SYS_SVCR_EL0); + + ffr = svcr & SYS_SVCR_EL0_SM_MASK; + + __this_cpu_write(efi_sm_state, ffr); + } + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); + + if (system_supports_sme()) + sysreg_clear_set_s(SYS_SVCR_EL0, + SYS_SVCR_EL0_SM_MASK, 0); + } else { fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state)); } @@ -1892,11 +1913,25 @@ void __efi_fpsimd_end(void) if (system_supports_sve() && likely(__this_cpu_read(efi_sve_state_used))) { char const *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + + /* + * Restore streaming mode; EFI calls are + * normal function calls so should not return in + * streaming mode. + */ + if (system_supports_sme()) { + if (__this_cpu_read(efi_sm_state)) { + sysreg_clear_set_s(SYS_SVCR_EL0, + 0, + SYS_SVCR_EL0_SM_MASK); + ffr = false; + } + } - sve_set_vq(sve_vq_from_vl(sve_get_vl()) - 1); sve_load_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); __this_cpu_write(efi_sve_state_used, false); } else { -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Mark Brown <broonie@kernel.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Shuah Khan <skhan@linuxfoundation.org>, Shuah Khan <shuah@kernel.org> Cc: Alan Hayward <alan.hayward@arm.com>, Luis Machado <luis.machado@arm.com>, Salil Akerkar <Salil.Akerkar@arm.com>, Basant Kumar Dwivedi <Basant.KumarDwivedi@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown <broonie@kernel.org> Subject: [PATCH v2 33/42] arm64/sme: Save and restore streaming mode over EFI runtime calls Date: Mon, 18 Oct 2021 20:08:49 +0100 [thread overview] Message-ID: <20211018190858.2119209-34-broonie@kernel.org> (raw) In-Reply-To: <20211018190858.2119209-1-broonie@kernel.org> When saving and restoring the floating point state over an EFI runtime call ensure that we handle streaming mode, only handling FFR if we are not in streaming mode and ensuring that we are in normal mode over the call into runtime services. We currently assume that ZA will not be modified by runtime services, the specification is not yet finalised so this may need updating if that changes. Signed-off-by: Mark Brown <broonie@kernel.org> --- arch/arm64/kernel/fpsimd.c | 47 +++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index d3665d7f6e78..bd9a0eddf533 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1049,21 +1049,25 @@ int vec_verify_vq_map(enum vec_type type) static void __init sve_efi_setup(void) { - struct vl_info *info = &vl_info[ARM64_VEC_SVE]; + int max_vl = 0; + int i; if (!IS_ENABLED(CONFIG_EFI)) return; + for (i = 0; i < ARRAY_SIZE(vl_info); i++) + max_vl = max(vl_info[i].max_vl, max_vl); + /* * alloc_percpu() warns and prints a backtrace if this goes wrong. * This is evidence of a crippled system and we are returning void, * so no attempt is made to handle this situation here. */ - if (!sve_vl_valid(info->max_vl)) + if (!sve_vl_valid(max_vl)) goto fail; efi_sve_state = __alloc_percpu( - SVE_SIG_REGS_SIZE(sve_vq_from_vl(info->max_vl)), SVE_VQ_BYTES); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(max_vl)), SVE_VQ_BYTES); if (!efi_sve_state) goto fail; @@ -1830,6 +1834,7 @@ EXPORT_SYMBOL(kernel_neon_end); static DEFINE_PER_CPU(struct user_fpsimd_state, efi_fpsimd_state); static DEFINE_PER_CPU(bool, efi_fpsimd_state_used); static DEFINE_PER_CPU(bool, efi_sve_state_used); +static DEFINE_PER_CPU(bool, efi_sm_state); /* * EFI runtime services support functions @@ -1864,12 +1869,28 @@ void __efi_fpsimd_begin(void) */ if (system_supports_sve() && likely(efi_sve_state)) { char *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + u64 svcr; __this_cpu_write(efi_sve_state_used, true); + /* If we are in streaming mode don't touch FFR */ + if (system_supports_sme()) { + svcr = read_sysreg_s(SYS_SVCR_EL0); + + ffr = svcr & SYS_SVCR_EL0_SM_MASK; + + __this_cpu_write(efi_sm_state, ffr); + } + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); + + if (system_supports_sme()) + sysreg_clear_set_s(SYS_SVCR_EL0, + SYS_SVCR_EL0_SM_MASK, 0); + } else { fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state)); } @@ -1892,11 +1913,25 @@ void __efi_fpsimd_end(void) if (system_supports_sve() && likely(__this_cpu_read(efi_sve_state_used))) { char const *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + + /* + * Restore streaming mode; EFI calls are + * normal function calls so should not return in + * streaming mode. + */ + if (system_supports_sme()) { + if (__this_cpu_read(efi_sm_state)) { + sysreg_clear_set_s(SYS_SVCR_EL0, + 0, + SYS_SVCR_EL0_SM_MASK); + ffr = false; + } + } - sve_set_vq(sve_vq_from_vl(sve_get_vl()) - 1); sve_load_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); __this_cpu_write(efi_sve_state_used, false); } else { -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-10-18 19:11 UTC|newest] Thread overview: 96+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-10-18 19:08 [PATCH v2 00/42] arm64/sme: Initial support for the Scalable Matrix Extension Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 01/42] arm64/fp: Reindent fpsimd_save() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 02/42] arm64/sve: Remove sve_load_from_fpsimd_state() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 03/42] arm64/sve: Make sve_state_size() static Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 04/42] arm64/sve: Make access to FFR optional Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-19 10:14 ` Will Deacon 2021-10-19 10:14 ` Will Deacon 2021-10-19 13:17 ` Mark Brown 2021-10-19 13:17 ` Mark Brown 2021-10-19 14:39 ` Mark Rutland 2021-10-19 14:39 ` Mark Rutland 2021-10-19 15:31 ` Mark Brown 2021-10-19 15:31 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 05/42] arm64/sve: Rename find_supported_vector_length() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 06/42] arm64/sve: Use accessor functions for vector lengths in thread_struct Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 07/42] arm64/sve: Put system wide vector length information into structs Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 08/42] arm64/sve: Explicitly load vector length when restoring SVE state Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 09/42] arm64/sve: Track vector lengths for tasks in an array Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 10/42] arm64/sve: Make sysctl interface for SVE reusable by SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 11/42] arm64/sve: Generalise vector length configuration prctl() for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 12/42] kselftest/arm64: Parameterise ptrace vector length information Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 13/42] kselftest/arm64: Allow signal tests to trigger from a function Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 14/42] tools/nolibc: Implement gettid() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-19 5:06 ` Willy Tarreau 2021-10-19 5:06 ` Willy Tarreau 2021-10-18 19:08 ` [PATCH v2 15/42] arm64/sme: Provide ABI documentation for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 16/42] arm64/sme: System register and exception syndrome definitions Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 17/42] arm64/sme: Define macros for manually encoding SME instructions Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 18/42] arm64/sme: Early CPU setup for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 19/42] arm64/sme: Basic enumeration support Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 20/42] arm64/sme: Identify supported SME vector lengths at boot Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 21/42] arm64/sme: Implement sysctl to set the default vector length Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 22/42] arm64/sme: Implement vector length configuration prctl()s Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 23/42] arm64/sme: Implement support for TPIDR2 Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 24/42] arm64/sme: Implement SVCR context switching Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 25/42] arm64/sme: Implement streaming SVE " Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 26/42] arm64/sme: Implement ZA " Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 27/42] arm64/sme: Implement traps and syscall handling for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 28/42] arm64/sme: Implement streaming SVE signal handling Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 29/42] arm64/sme: Implement ZA " Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 30/42] arm64/sme: Implement ptrace support for streaming mode SVE registers Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 31/42] arm64/sme: Add ptrace support for ZA Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 32/42] arm64/sme: Disable streaming mode and ZA when flushing CPU state Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` Mark Brown [this message] 2021-10-18 19:08 ` [PATCH v2 33/42] arm64/sme: Save and restore streaming mode over EFI runtime calls Mark Brown 2021-10-18 19:08 ` [PATCH v2 34/42] arm64/sme: Provide Kconfig for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 35/42] kselftest/arm64: sme: Add streaming SME support to vlset Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 36/42] kselftest/arm64: Add tests for TPIDR2 Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 37/42] kselftest/arm64: Extend vector configuration API tests to cover SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 38/42] kselftest/arm64: sme: Provide streaming mode SVE stress test Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 39/42] kselftest/arm64: Add stress test for SME ZA context switching Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 40/42] kselftest/arm64: signal: Add SME signal handling tests Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 41/42] kselftest/arm64: Add streaming SVE to SVE ptrace tests Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 42/42] kselftest/arm64: Add coverage for the ZA ptrace interface Mark Brown 2021-10-18 19:08 ` Mark Brown
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211018190858.2119209-34-broonie@kernel.org \ --to=broonie@kernel.org \ --cc=Basant.KumarDwivedi@arm.com \ --cc=Salil.Akerkar@arm.com \ --cc=alan.hayward@arm.com \ --cc=catalin.marinas@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=luis.machado@arm.com \ --cc=shuah@kernel.org \ --cc=skhan@linuxfoundation.org \ --cc=szabolcs.nagy@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.