From: Mark Brown <broonie@kernel.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Shuah Khan <skhan@linuxfoundation.org>, Shuah Khan <shuah@kernel.org> Cc: Alan Hayward <alan.hayward@arm.com>, Luis Machado <luis.machado@arm.com>, Salil Akerkar <Salil.Akerkar@arm.com>, Basant Kumar Dwivedi <Basant.KumarDwivedi@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown <broonie@kernel.org> Subject: [PATCH v7 18/37] arm64/sme: Implement SVCR context switching Date: Fri, 10 Dec 2021 18:41:14 +0000 [thread overview] Message-ID: <20211210184133.320748-19-broonie@kernel.org> (raw) In-Reply-To: <20211210184133.320748-1-broonie@kernel.org> In SME the use of both streaming SVE mode and ZA are tracked through PSTATE.SM and PSTATE.ZA, visible through the system register SVCR. In order to context switch the floating point state for SME we need to context switch the contents of this register as part of context switching the floating point state. Since changing the vector length exits streaming SVE mode and disables ZA we also make sure we update SVCR appropriately when setting vector length, and similarly ensure that new threads have streaming SVE mode and ZA disabled. Signed-off-by: Mark Brown <broonie@kernel.org> --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/thread_info.h | 1 + arch/arm64/kernel/fpsimd.c | 11 +++++++++++ arch/arm64/kernel/process.c | 2 ++ 4 files changed, 15 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 008a1767ebff..7e08a4d48c24 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -168,6 +168,7 @@ struct thread_struct { u64 mte_ctrl; #endif u64 sctlr_user; + u64 svcr; u64 tpidr2_el0; }; diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 4e6b58dcd6f9..848739c15de8 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -82,6 +82,7 @@ int arch_dup_task_struct(struct task_struct *dst, #define TIF_SVE_VL_INHERIT 24 /* Inherit SVE vl_onexec across exec */ #define TIF_SSBD 25 /* Wants SSB mitigation */ #define TIF_TAGGED_ADDR 26 /* Allow tagged user addresses */ +#define TIF_SME 27 /* SME in use */ #define TIF_SME_VL_INHERIT 28 /* Inherit SME vl_onexec across exec */ #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 0af82c518979..d6e6bec6b490 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -355,6 +355,9 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); + if (IS_ENABLED(CONFIG_ARM64_SME) && test_thread_flag(TIF_SME)) + write_sysreg_s(current->thread.svcr, SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); sve_load_state(sve_pffr(¤t->thread), @@ -380,6 +383,10 @@ static void fpsimd_save(void) if (test_thread_flag(TIF_FOREIGN_FPSTATE)) return; + if (IS_ENABLED(CONFIG_ARM64_SME) && + test_thread_flag(TIF_SME)) + current->thread.svcr = read_sysreg_s(SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { if (WARN_ON(sve_get_vl() != last->sve_vl)) { @@ -731,6 +738,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, if (test_and_clear_tsk_thread_flag(task, TIF_SVE)) sve_to_fpsimd(task); + if (system_supports_sme() && type == ARM64_VEC_SME) + task->thread.svcr &= ~(SYS_SVCR_EL0_SM_MASK | + SYS_SVCR_EL0_ZA_MASK); + if (task == current) put_cpu_fpsimd_context(); diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 3adca0123943..eff50e02b4e2 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -309,6 +309,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) dst->thread.sve_state = NULL; clear_tsk_thread_flag(dst, TIF_SVE); + dst->thread.svcr = 0; + /* clear any pending asynchronous tag fault raised by the parent */ clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); -- 2.30.2
WARNING: multiple messages have this Message-ID (diff)
From: Mark Brown <broonie@kernel.org> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Shuah Khan <skhan@linuxfoundation.org>, Shuah Khan <shuah@kernel.org> Cc: Alan Hayward <alan.hayward@arm.com>, Luis Machado <luis.machado@arm.com>, Salil Akerkar <Salil.Akerkar@arm.com>, Basant Kumar Dwivedi <Basant.KumarDwivedi@arm.com>, Szabolcs Nagy <szabolcs.nagy@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown <broonie@kernel.org> Subject: [PATCH v7 18/37] arm64/sme: Implement SVCR context switching Date: Fri, 10 Dec 2021 18:41:14 +0000 [thread overview] Message-ID: <20211210184133.320748-19-broonie@kernel.org> (raw) In-Reply-To: <20211210184133.320748-1-broonie@kernel.org> In SME the use of both streaming SVE mode and ZA are tracked through PSTATE.SM and PSTATE.ZA, visible through the system register SVCR. In order to context switch the floating point state for SME we need to context switch the contents of this register as part of context switching the floating point state. Since changing the vector length exits streaming SVE mode and disables ZA we also make sure we update SVCR appropriately when setting vector length, and similarly ensure that new threads have streaming SVE mode and ZA disabled. Signed-off-by: Mark Brown <broonie@kernel.org> --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/thread_info.h | 1 + arch/arm64/kernel/fpsimd.c | 11 +++++++++++ arch/arm64/kernel/process.c | 2 ++ 4 files changed, 15 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 008a1767ebff..7e08a4d48c24 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -168,6 +168,7 @@ struct thread_struct { u64 mte_ctrl; #endif u64 sctlr_user; + u64 svcr; u64 tpidr2_el0; }; diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 4e6b58dcd6f9..848739c15de8 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -82,6 +82,7 @@ int arch_dup_task_struct(struct task_struct *dst, #define TIF_SVE_VL_INHERIT 24 /* Inherit SVE vl_onexec across exec */ #define TIF_SSBD 25 /* Wants SSB mitigation */ #define TIF_TAGGED_ADDR 26 /* Allow tagged user addresses */ +#define TIF_SME 27 /* SME in use */ #define TIF_SME_VL_INHERIT 28 /* Inherit SME vl_onexec across exec */ #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 0af82c518979..d6e6bec6b490 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -355,6 +355,9 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); + if (IS_ENABLED(CONFIG_ARM64_SME) && test_thread_flag(TIF_SME)) + write_sysreg_s(current->thread.svcr, SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); sve_load_state(sve_pffr(¤t->thread), @@ -380,6 +383,10 @@ static void fpsimd_save(void) if (test_thread_flag(TIF_FOREIGN_FPSTATE)) return; + if (IS_ENABLED(CONFIG_ARM64_SME) && + test_thread_flag(TIF_SME)) + current->thread.svcr = read_sysreg_s(SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { if (WARN_ON(sve_get_vl() != last->sve_vl)) { @@ -731,6 +738,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, if (test_and_clear_tsk_thread_flag(task, TIF_SVE)) sve_to_fpsimd(task); + if (system_supports_sme() && type == ARM64_VEC_SME) + task->thread.svcr &= ~(SYS_SVCR_EL0_SM_MASK | + SYS_SVCR_EL0_ZA_MASK); + if (task == current) put_cpu_fpsimd_context(); diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 3adca0123943..eff50e02b4e2 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -309,6 +309,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) dst->thread.sve_state = NULL; clear_tsk_thread_flag(dst, TIF_SVE); + dst->thread.svcr = 0; + /* clear any pending asynchronous tag fault raised by the parent */ clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-12-10 18:44 UTC|newest] Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-12-10 18:40 [PATCH v7 00/37] arm64/sme: Initial support for the Scalable Matrix Extension Mark Brown 2021-12-10 18:40 ` Mark Brown 2021-12-10 18:40 ` [PATCH v7 01/37] arm64/sve: Make sysctl interface for SVE reusable by SME Mark Brown 2021-12-10 18:40 ` Mark Brown 2021-12-10 18:40 ` [PATCH v7 02/37] arm64/sve: Generalise vector length configuration prctl() for SME Mark Brown 2021-12-10 18:40 ` Mark Brown 2021-12-10 18:40 ` [PATCH v7 03/37] arm64/sve: Minor clarification of ABI documentation Mark Brown 2021-12-10 18:40 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 04/37] kselftest/arm64: Parameterise ptrace vector length information Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 05/37] kselftest/arm64: Allow signal tests to trigger from a function Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 06/37] kselftest/arm64: Add a test program to exercise the syscall ABI Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 07/37] arm64: cpufeature: Always specify and use a field width for capabilities Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 08/37] tools/nolibc: Implement gettid() Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 09/37] arm64/sme: Provide ABI documentation for SME Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 10/37] arm64/sme: System register and exception syndrome definitions Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 11/37] arm64/sme: Define macros for manually encoding SME instructions Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 12/37] arm64/sme: Early CPU setup for SME Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 13/37] arm64/sme: Basic enumeration support Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 14/37] arm64/sme: Identify supported SME vector lengths at boot Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 15/37] arm64/sme: Implement sysctl to set the default vector length Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 16/37] arm64/sme: Implement vector length configuration prctl()s Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 17/37] arm64/sme: Implement support for TPIDR2 Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` Mark Brown [this message] 2021-12-10 18:41 ` [PATCH v7 18/37] arm64/sme: Implement SVCR context switching Mark Brown 2021-12-10 18:41 ` [PATCH v7 19/37] arm64/sme: Implement streaming SVE " Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 20/37] arm64/sme: Implement ZA " Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 21/37] arm64/sme: Implement traps and syscall handling for SME Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 22/37] arm64/sme: Implement streaming SVE signal handling Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 23/37] arm64/sme: Implement ZA " Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 24/37] arm64/sme: Implement ptrace support for streaming mode SVE registers Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 25/37] arm64/sme: Add ptrace support for ZA Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 26/37] arm64/sme: Disable streaming mode and ZA when flushing CPU state Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 27/37] arm64/sme: Save and restore streaming mode over EFI runtime calls Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 28/37] arm64/sme: Provide Kconfig for SME Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 29/37] kselftest/arm64: sme: Add streaming SME support to vlset Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 30/37] kselftest/arm64: Add tests for TPIDR2 Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 31/37] kselftest/arm64: Extend vector configuration API tests to cover SME Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 32/37] kselftest/arm64: sme: Provide streaming mode SVE stress test Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 33/37] kselftest/arm64: Add stress test for SME ZA context switching Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 34/37] kselftest/arm64: signal: Add SME signal handling tests Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 35/37] kselftest/arm64: Add streaming SVE to SVE ptrace tests Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 36/37] kselftest/arm64: Add coverage for the ZA ptrace interface Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-10 18:41 ` [PATCH v7 37/37] kselftest/arm64: Add SME support to syscall ABI test Mark Brown 2021-12-10 18:41 ` Mark Brown 2021-12-14 19:17 ` (subset) [PATCH v7 00/37] arm64/sme: Initial support for the Scalable Matrix Extension Catalin Marinas 2021-12-14 19:17 ` Catalin Marinas 2021-12-14 19:18 ` Catalin Marinas 2021-12-14 19:18 ` Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211210184133.320748-19-broonie@kernel.org \ --to=broonie@kernel.org \ --cc=Basant.KumarDwivedi@arm.com \ --cc=Salil.Akerkar@arm.com \ --cc=alan.hayward@arm.com \ --cc=catalin.marinas@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=luis.machado@arm.com \ --cc=shuah@kernel.org \ --cc=skhan@linuxfoundation.org \ --cc=szabolcs.nagy@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.