From: Mark Brown <broonie@kernel.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Shuah Khan <skhan@linuxfoundation.org>, Shuah Khan <shuah@kernel.org> Cc: Alan Hayward <alan.hayward@arm.com>, Luis Machado <luis.machado@arm.com>, Salil Akerkar <Salil.Akerkar@arm.com>, Basant Kumar Dwivedi <Basant.KumarDwivedi@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown <broonie@kernel.org> Subject: [PATCH v2 08/42] arm64/sve: Explicitly load vector length when restoring SVE state Date: Mon, 18 Oct 2021 20:08:24 +0100 [thread overview] Message-ID: <20211018190858.2119209-9-broonie@kernel.org> (raw) In-Reply-To: <20211018190858.2119209-1-broonie@kernel.org> Currently when restoring the SVE state we supply the SVE vector length as an argument to sve_load_state() and the underlying macros. This becomes inconvenient with the addition of SME since we may need to restore any combination of SVE and SME vector lengths, and we already separately restore the vector length in the KVM code. We don't need to know the vector length during the actual register load since the SME load instructions can index into the data array for us. Refactor the interface so we explicitly set the vector length separately to restoring the SVE registers in preparation for adding SME support, no functional change should be involved. Signed-off-by: Mark Brown <broonie@kernel.org> --- arch/arm64/include/asm/fpsimd.h | 2 +- arch/arm64/include/asm/fpsimdmacros.h | 7 +------ arch/arm64/kernel/entry-fpsimd.S | 3 +-- arch/arm64/kernel/fpsimd.c | 13 +++++++------ arch/arm64/kvm/hyp/fpsimd.S | 2 +- 5 files changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 5a1f79a4a500..1d0b5fa253a0 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -67,7 +67,7 @@ static inline void *sve_pffr(struct thread_struct *thread) extern void sve_save_state(void *state, u32 *pfpsr, int save_ffr); extern void sve_load_state(void const *state, u32 const *pfpsr, - int restore_ffr, unsigned long vq_minus_1); + int restore_ffr); extern void sve_flush_live(bool flush_ffr, unsigned long vq_minus_1); extern unsigned int sve_get_vl(void); extern void sve_set_vq(unsigned long vq_minus_1); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 84d8cb7b07fa..b22538a6137e 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -241,7 +241,7 @@ str w\nxtmp, [\xpfpsr, #4] .endm -.macro __sve_load nxbase, xpfpsr, restore_ffr, nxtmp +.macro sve_load nxbase, xpfpsr, restore_ffr, nxtmp _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34 cbz \restore_ffr, 921f _sve_ldr_p 0, \nxbase @@ -254,8 +254,3 @@ ldr w\nxtmp, [\xpfpsr, #4] msr fpcr, x\nxtmp .endm - -.macro sve_load nxbase, xpfpsr, restore_ffr, xvqminus1, nxtmp, xtmp2 - sve_load_vq \xvqminus1, x\nxtmp, \xtmp2 - __sve_load \nxbase, \xpfpsr, \restore_ffr, \nxtmp -.endm diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S index 13c27465bfa8..2339d370bfe1 100644 --- a/arch/arm64/kernel/entry-fpsimd.S +++ b/arch/arm64/kernel/entry-fpsimd.S @@ -51,10 +51,9 @@ SYM_FUNC_END(sve_save_state) * x0 - pointer to buffer for state * x1 - pointer to storage for FPSR * x2 - Restore FFR if non-zero - * x3 - VQ-1 */ SYM_FUNC_START(sve_load_state) - sve_load 0, x1, x2, x3, 4, x5 + sve_load 0, x1, x2, 4 ret SYM_FUNC_END(sve_load_state) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 9248c9efe5fd..e75dd20a40cf 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -318,12 +318,13 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); - if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { + sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); sve_load_state(sve_pffr(¤t->thread), - ¤t->thread.uw.fpsimd_state.fpsr, true, - sve_vq_from_vl(task_get_sve_vl(current)) - 1); - else + ¤t->thread.uw.fpsimd_state.fpsr, true); + } else { fpsimd_load_state(¤t->thread.uw.fpsimd_state); + } } /* @@ -1423,10 +1424,10 @@ void __efi_fpsimd_end(void) likely(__this_cpu_read(efi_sve_state_used))) { char const *sve_state = this_cpu_ptr(efi_sve_state); + sve_set_vq(sve_vq_from_vl(sve_get_vl()) - 1); sve_load_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true, - sve_vq_from_vl(sve_get_vl()) - 1); + true); __this_cpu_write(efi_sve_state_used, false); } else { diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index 1bb3b04b84e6..e950875e31ce 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -22,7 +22,7 @@ SYM_FUNC_END(__fpsimd_restore_state) SYM_FUNC_START(__sve_restore_state) mov x2, #1 - __sve_load 0, x1, x2, 3 + sve_load 0, x1, x2, 3 ret SYM_FUNC_END(__sve_restore_state) -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Mark Brown <broonie@kernel.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Shuah Khan <skhan@linuxfoundation.org>, Shuah Khan <shuah@kernel.org> Cc: Alan Hayward <alan.hayward@arm.com>, Luis Machado <luis.machado@arm.com>, Salil Akerkar <Salil.Akerkar@arm.com>, Basant Kumar Dwivedi <Basant.KumarDwivedi@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown <broonie@kernel.org> Subject: [PATCH v2 08/42] arm64/sve: Explicitly load vector length when restoring SVE state Date: Mon, 18 Oct 2021 20:08:24 +0100 [thread overview] Message-ID: <20211018190858.2119209-9-broonie@kernel.org> (raw) In-Reply-To: <20211018190858.2119209-1-broonie@kernel.org> Currently when restoring the SVE state we supply the SVE vector length as an argument to sve_load_state() and the underlying macros. This becomes inconvenient with the addition of SME since we may need to restore any combination of SVE and SME vector lengths, and we already separately restore the vector length in the KVM code. We don't need to know the vector length during the actual register load since the SME load instructions can index into the data array for us. Refactor the interface so we explicitly set the vector length separately to restoring the SVE registers in preparation for adding SME support, no functional change should be involved. Signed-off-by: Mark Brown <broonie@kernel.org> --- arch/arm64/include/asm/fpsimd.h | 2 +- arch/arm64/include/asm/fpsimdmacros.h | 7 +------ arch/arm64/kernel/entry-fpsimd.S | 3 +-- arch/arm64/kernel/fpsimd.c | 13 +++++++------ arch/arm64/kvm/hyp/fpsimd.S | 2 +- 5 files changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 5a1f79a4a500..1d0b5fa253a0 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -67,7 +67,7 @@ static inline void *sve_pffr(struct thread_struct *thread) extern void sve_save_state(void *state, u32 *pfpsr, int save_ffr); extern void sve_load_state(void const *state, u32 const *pfpsr, - int restore_ffr, unsigned long vq_minus_1); + int restore_ffr); extern void sve_flush_live(bool flush_ffr, unsigned long vq_minus_1); extern unsigned int sve_get_vl(void); extern void sve_set_vq(unsigned long vq_minus_1); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 84d8cb7b07fa..b22538a6137e 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -241,7 +241,7 @@ str w\nxtmp, [\xpfpsr, #4] .endm -.macro __sve_load nxbase, xpfpsr, restore_ffr, nxtmp +.macro sve_load nxbase, xpfpsr, restore_ffr, nxtmp _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34 cbz \restore_ffr, 921f _sve_ldr_p 0, \nxbase @@ -254,8 +254,3 @@ ldr w\nxtmp, [\xpfpsr, #4] msr fpcr, x\nxtmp .endm - -.macro sve_load nxbase, xpfpsr, restore_ffr, xvqminus1, nxtmp, xtmp2 - sve_load_vq \xvqminus1, x\nxtmp, \xtmp2 - __sve_load \nxbase, \xpfpsr, \restore_ffr, \nxtmp -.endm diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S index 13c27465bfa8..2339d370bfe1 100644 --- a/arch/arm64/kernel/entry-fpsimd.S +++ b/arch/arm64/kernel/entry-fpsimd.S @@ -51,10 +51,9 @@ SYM_FUNC_END(sve_save_state) * x0 - pointer to buffer for state * x1 - pointer to storage for FPSR * x2 - Restore FFR if non-zero - * x3 - VQ-1 */ SYM_FUNC_START(sve_load_state) - sve_load 0, x1, x2, x3, 4, x5 + sve_load 0, x1, x2, 4 ret SYM_FUNC_END(sve_load_state) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 9248c9efe5fd..e75dd20a40cf 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -318,12 +318,13 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); - if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { + sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); sve_load_state(sve_pffr(¤t->thread), - ¤t->thread.uw.fpsimd_state.fpsr, true, - sve_vq_from_vl(task_get_sve_vl(current)) - 1); - else + ¤t->thread.uw.fpsimd_state.fpsr, true); + } else { fpsimd_load_state(¤t->thread.uw.fpsimd_state); + } } /* @@ -1423,10 +1424,10 @@ void __efi_fpsimd_end(void) likely(__this_cpu_read(efi_sve_state_used))) { char const *sve_state = this_cpu_ptr(efi_sve_state); + sve_set_vq(sve_vq_from_vl(sve_get_vl()) - 1); sve_load_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true, - sve_vq_from_vl(sve_get_vl()) - 1); + true); __this_cpu_write(efi_sve_state_used, false); } else { diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index 1bb3b04b84e6..e950875e31ce 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -22,7 +22,7 @@ SYM_FUNC_END(__fpsimd_restore_state) SYM_FUNC_START(__sve_restore_state) mov x2, #1 - __sve_load 0, x1, x2, 3 + sve_load 0, x1, x2, 3 ret SYM_FUNC_END(__sve_restore_state) -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-10-18 19:10 UTC|newest] Thread overview: 96+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-10-18 19:08 [PATCH v2 00/42] arm64/sme: Initial support for the Scalable Matrix Extension Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 01/42] arm64/fp: Reindent fpsimd_save() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 02/42] arm64/sve: Remove sve_load_from_fpsimd_state() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 03/42] arm64/sve: Make sve_state_size() static Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 04/42] arm64/sve: Make access to FFR optional Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-19 10:14 ` Will Deacon 2021-10-19 10:14 ` Will Deacon 2021-10-19 13:17 ` Mark Brown 2021-10-19 13:17 ` Mark Brown 2021-10-19 14:39 ` Mark Rutland 2021-10-19 14:39 ` Mark Rutland 2021-10-19 15:31 ` Mark Brown 2021-10-19 15:31 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 05/42] arm64/sve: Rename find_supported_vector_length() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 06/42] arm64/sve: Use accessor functions for vector lengths in thread_struct Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 07/42] arm64/sve: Put system wide vector length information into structs Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` Mark Brown [this message] 2021-10-18 19:08 ` [PATCH v2 08/42] arm64/sve: Explicitly load vector length when restoring SVE state Mark Brown 2021-10-18 19:08 ` [PATCH v2 09/42] arm64/sve: Track vector lengths for tasks in an array Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 10/42] arm64/sve: Make sysctl interface for SVE reusable by SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 11/42] arm64/sve: Generalise vector length configuration prctl() for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 12/42] kselftest/arm64: Parameterise ptrace vector length information Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 13/42] kselftest/arm64: Allow signal tests to trigger from a function Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 14/42] tools/nolibc: Implement gettid() Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-19 5:06 ` Willy Tarreau 2021-10-19 5:06 ` Willy Tarreau 2021-10-18 19:08 ` [PATCH v2 15/42] arm64/sme: Provide ABI documentation for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 16/42] arm64/sme: System register and exception syndrome definitions Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 17/42] arm64/sme: Define macros for manually encoding SME instructions Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 18/42] arm64/sme: Early CPU setup for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 19/42] arm64/sme: Basic enumeration support Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 20/42] arm64/sme: Identify supported SME vector lengths at boot Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 21/42] arm64/sme: Implement sysctl to set the default vector length Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 22/42] arm64/sme: Implement vector length configuration prctl()s Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 23/42] arm64/sme: Implement support for TPIDR2 Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 24/42] arm64/sme: Implement SVCR context switching Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 25/42] arm64/sme: Implement streaming SVE " Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 26/42] arm64/sme: Implement ZA " Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 27/42] arm64/sme: Implement traps and syscall handling for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 28/42] arm64/sme: Implement streaming SVE signal handling Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 29/42] arm64/sme: Implement ZA " Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 30/42] arm64/sme: Implement ptrace support for streaming mode SVE registers Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 31/42] arm64/sme: Add ptrace support for ZA Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 32/42] arm64/sme: Disable streaming mode and ZA when flushing CPU state Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 33/42] arm64/sme: Save and restore streaming mode over EFI runtime calls Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 34/42] arm64/sme: Provide Kconfig for SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 35/42] kselftest/arm64: sme: Add streaming SME support to vlset Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 36/42] kselftest/arm64: Add tests for TPIDR2 Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 37/42] kselftest/arm64: Extend vector configuration API tests to cover SME Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 38/42] kselftest/arm64: sme: Provide streaming mode SVE stress test Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 39/42] kselftest/arm64: Add stress test for SME ZA context switching Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 40/42] kselftest/arm64: signal: Add SME signal handling tests Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 41/42] kselftest/arm64: Add streaming SVE to SVE ptrace tests Mark Brown 2021-10-18 19:08 ` Mark Brown 2021-10-18 19:08 ` [PATCH v2 42/42] kselftest/arm64: Add coverage for the ZA ptrace interface Mark Brown 2021-10-18 19:08 ` Mark Brown
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211018190858.2119209-9-broonie@kernel.org \ --to=broonie@kernel.org \ --cc=Basant.KumarDwivedi@arm.com \ --cc=Salil.Akerkar@arm.com \ --cc=alan.hayward@arm.com \ --cc=catalin.marinas@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=luis.machado@arm.com \ --cc=shuah@kernel.org \ --cc=skhan@linuxfoundation.org \ --cc=szabolcs.nagy@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.