From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBF52C433F5 for ; Fri, 10 Dec 2021 18:44:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238751AbhLJSsI (ORCPT ); Fri, 10 Dec 2021 13:48:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245518AbhLJSsI (ORCPT ); Fri, 10 Dec 2021 13:48:08 -0500 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5486C061746 for ; Fri, 10 Dec 2021 10:44:32 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id F0ADECE2C78 for ; Fri, 10 Dec 2021 18:44:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA864C341CC; Fri, 10 Dec 2021 18:44:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1639161869; bh=Iz1TB8HTnr7e9Ekco1erA6sCTxhOFwnd1nCY4980Kgc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MyOW7jZwroKWJSGV7mRuDauFftIuZ78vXqA8qw/soEwa4neaAZZeu3D5JRW3o7qke DBkCKlj386B4wKuqBCYzbwZyYFz886nFld/mPCm/07f/dZEveCrO/yl/MjAUUOh3ht jsqnRSs0a61QwDn5ApiqG/wGmKqdLsNkCy+lQZMJIZkR6bnQXKaQW0mE/HMNQ/hpLZ MghhlPeXn9NnZ+GTU/YQhvp8xRPNJOE5j94UVPZ1sjV+K8dcWPqkWkmXR9hvPFIx4A D4z627lZMDdjw+e+AR3qcTRIy2kn0MNxm45cKTBaBOB5RI1WcC+JAdhL8lXqFRnSg2 nV0FENwpn1R3Q== From: Mark Brown To: Catalin Marinas , Will Deacon , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown Subject: [PATCH v7 18/37] arm64/sme: Implement SVCR context switching Date: Fri, 10 Dec 2021 18:41:14 +0000 Message-Id: <20211210184133.320748-19-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210184133.320748-1-broonie@kernel.org> References: <20211210184133.320748-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3708; h=from:subject; bh=Iz1TB8HTnr7e9Ekco1erA6sCTxhOFwnd1nCY4980Kgc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBhs59Mlraut7rom3l+pQGzMjle5o9o8GsYbmgcTDV7 lQfMBxWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYbOfTAAKCRAk1otyXVSH0EQfCA CC4BHjXz/Yot4Xa16aYgKcefvW2qBxJ72+tcooFvjhIU/B0QtvT0kyLLvG0GyZ1zL4uxl6O5b/BPpq Kvb4rs4YkXjXz5pbpXq3a0vwDHykOO/lOT2F5f8FPNDjMJgl+0X10t/mfiwINrhx6oFC8C4jBs7h6g +4DsHBTPqQEhM3QuS96hNZ6b/+TKWF0KVvcvls6O7EkQ2MFyZPTtkmBiCCdmDH5Atsbuto34eFybDo 3wTEVwgsJeqh68PWCtr/tfbK4Jgu/6w61WnFA/llqgN3Nu7s/BbEHACSnA9x01drQD9R+mlYGkBX8q GKv7cNjV9J6Y8a2M0ZNVLaNLvTeiIj X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org In SME the use of both streaming SVE mode and ZA are tracked through PSTATE.SM and PSTATE.ZA, visible through the system register SVCR. In order to context switch the floating point state for SME we need to context switch the contents of this register as part of context switching the floating point state. Since changing the vector length exits streaming SVE mode and disables ZA we also make sure we update SVCR appropriately when setting vector length, and similarly ensure that new threads have streaming SVE mode and ZA disabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/thread_info.h | 1 + arch/arm64/kernel/fpsimd.c | 11 +++++++++++ arch/arm64/kernel/process.c | 2 ++ 4 files changed, 15 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 008a1767ebff..7e08a4d48c24 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -168,6 +168,7 @@ struct thread_struct { u64 mte_ctrl; #endif u64 sctlr_user; + u64 svcr; u64 tpidr2_el0; }; diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 4e6b58dcd6f9..848739c15de8 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -82,6 +82,7 @@ int arch_dup_task_struct(struct task_struct *dst, #define TIF_SVE_VL_INHERIT 24 /* Inherit SVE vl_onexec across exec */ #define TIF_SSBD 25 /* Wants SSB mitigation */ #define TIF_TAGGED_ADDR 26 /* Allow tagged user addresses */ +#define TIF_SME 27 /* SME in use */ #define TIF_SME_VL_INHERIT 28 /* Inherit SME vl_onexec across exec */ #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 0af82c518979..d6e6bec6b490 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -355,6 +355,9 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); + if (IS_ENABLED(CONFIG_ARM64_SME) && test_thread_flag(TIF_SME)) + write_sysreg_s(current->thread.svcr, SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); sve_load_state(sve_pffr(¤t->thread), @@ -380,6 +383,10 @@ static void fpsimd_save(void) if (test_thread_flag(TIF_FOREIGN_FPSTATE)) return; + if (IS_ENABLED(CONFIG_ARM64_SME) && + test_thread_flag(TIF_SME)) + current->thread.svcr = read_sysreg_s(SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { if (WARN_ON(sve_get_vl() != last->sve_vl)) { @@ -731,6 +738,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, if (test_and_clear_tsk_thread_flag(task, TIF_SVE)) sve_to_fpsimd(task); + if (system_supports_sme() && type == ARM64_VEC_SME) + task->thread.svcr &= ~(SYS_SVCR_EL0_SM_MASK | + SYS_SVCR_EL0_ZA_MASK); + if (task == current) put_cpu_fpsimd_context(); diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 3adca0123943..eff50e02b4e2 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -309,6 +309,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) dst->thread.sve_state = NULL; clear_tsk_thread_flag(dst, TIF_SVE); + dst->thread.svcr = 0; + /* clear any pending asynchronous tag fault raised by the parent */ clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); -- 2.30.2 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1507C433EF for ; Fri, 10 Dec 2021 19:00:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hhYYKQRSbPE++LBe3XVYeS0bLBwyJhJseZ4qKP8gFdo=; b=nkrEV0uGZ3DNzg zvkR+JaX+WvegthsAJ0Yki1igVjgPxpZaxNQlfA552sttGsae/YwroafUAF0fhmE77eO/yzV6C+c4 tOuyRNBZSrmpBSeDAMdK5dwvLW3Cp+ZRkOQWy1tqyHk0he96QHZXwHzlYyHl2bmsxa4oHXyvtYGER lIfNtlc7kZw/VEDKWYcxdBLIRmCzsQ6T+ll+Zn23pLKl62/LfxXgmpGNM6zUfNvapyFAzfy7AA3f7 B1o6g9SUwdYZYhwuopKvEggyAvZinZfxeaePJxyWzqoq1Q25ey/pYkkXuUoJNHw8kevcWpZPb6d2w hT8EOhi0R687amDbDDsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mvl5u-003L8m-32; Fri, 10 Dec 2021 18:58:11 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mvksi-003FY7-KF for linux-arm-kernel@lists.infradead.org; Fri, 10 Dec 2021 18:44:34 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id F224CCE2C90; Fri, 10 Dec 2021 18:44:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA864C341CC; Fri, 10 Dec 2021 18:44:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1639161869; bh=Iz1TB8HTnr7e9Ekco1erA6sCTxhOFwnd1nCY4980Kgc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MyOW7jZwroKWJSGV7mRuDauFftIuZ78vXqA8qw/soEwa4neaAZZeu3D5JRW3o7qke DBkCKlj386B4wKuqBCYzbwZyYFz886nFld/mPCm/07f/dZEveCrO/yl/MjAUUOh3ht jsqnRSs0a61QwDn5ApiqG/wGmKqdLsNkCy+lQZMJIZkR6bnQXKaQW0mE/HMNQ/hpLZ MghhlPeXn9NnZ+GTU/YQhvp8xRPNJOE5j94UVPZ1sjV+K8dcWPqkWkmXR9hvPFIx4A D4z627lZMDdjw+e+AR3qcTRIy2kn0MNxm45cKTBaBOB5RI1WcC+JAdhL8lXqFRnSg2 nV0FENwpn1R3Q== From: Mark Brown To: Catalin Marinas , Will Deacon , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown Subject: [PATCH v7 18/37] arm64/sme: Implement SVCR context switching Date: Fri, 10 Dec 2021 18:41:14 +0000 Message-Id: <20211210184133.320748-19-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211210184133.320748-1-broonie@kernel.org> References: <20211210184133.320748-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3708; h=from:subject; bh=Iz1TB8HTnr7e9Ekco1erA6sCTxhOFwnd1nCY4980Kgc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBhs59Mlraut7rom3l+pQGzMjle5o9o8GsYbmgcTDV7 lQfMBxWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYbOfTAAKCRAk1otyXVSH0EQfCA CC4BHjXz/Yot4Xa16aYgKcefvW2qBxJ72+tcooFvjhIU/B0QtvT0kyLLvG0GyZ1zL4uxl6O5b/BPpq Kvb4rs4YkXjXz5pbpXq3a0vwDHykOO/lOT2F5f8FPNDjMJgl+0X10t/mfiwINrhx6oFC8C4jBs7h6g +4DsHBTPqQEhM3QuS96hNZ6b/+TKWF0KVvcvls6O7EkQ2MFyZPTtkmBiCCdmDH5Atsbuto34eFybDo 3wTEVwgsJeqh68PWCtr/tfbK4Jgu/6w61WnFA/llqgN3Nu7s/BbEHACSnA9x01drQD9R+mlYGkBX8q GKv7cNjV9J6Y8a2M0ZNVLaNLvTeiIj X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211210_104433_091400_A99BD31E X-CRM114-Status: GOOD ( 13.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In SME the use of both streaming SVE mode and ZA are tracked through PSTATE.SM and PSTATE.ZA, visible through the system register SVCR. In order to context switch the floating point state for SME we need to context switch the contents of this register as part of context switching the floating point state. Since changing the vector length exits streaming SVE mode and disables ZA we also make sure we update SVCR appropriately when setting vector length, and similarly ensure that new threads have streaming SVE mode and ZA disabled. Signed-off-by: Mark Brown --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/thread_info.h | 1 + arch/arm64/kernel/fpsimd.c | 11 +++++++++++ arch/arm64/kernel/process.c | 2 ++ 4 files changed, 15 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 008a1767ebff..7e08a4d48c24 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -168,6 +168,7 @@ struct thread_struct { u64 mte_ctrl; #endif u64 sctlr_user; + u64 svcr; u64 tpidr2_el0; }; diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 4e6b58dcd6f9..848739c15de8 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -82,6 +82,7 @@ int arch_dup_task_struct(struct task_struct *dst, #define TIF_SVE_VL_INHERIT 24 /* Inherit SVE vl_onexec across exec */ #define TIF_SSBD 25 /* Wants SSB mitigation */ #define TIF_TAGGED_ADDR 26 /* Allow tagged user addresses */ +#define TIF_SME 27 /* SME in use */ #define TIF_SME_VL_INHERIT 28 /* Inherit SME vl_onexec across exec */ #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 0af82c518979..d6e6bec6b490 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -355,6 +355,9 @@ static void task_fpsimd_load(void) WARN_ON(!system_supports_fpsimd()); WARN_ON(!have_cpu_fpsimd_context()); + if (IS_ENABLED(CONFIG_ARM64_SME) && test_thread_flag(TIF_SME)) + write_sysreg_s(current->thread.svcr, SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); sve_load_state(sve_pffr(¤t->thread), @@ -380,6 +383,10 @@ static void fpsimd_save(void) if (test_thread_flag(TIF_FOREIGN_FPSTATE)) return; + if (IS_ENABLED(CONFIG_ARM64_SME) && + test_thread_flag(TIF_SME)) + current->thread.svcr = read_sysreg_s(SYS_SVCR_EL0); + if (IS_ENABLED(CONFIG_ARM64_SVE) && test_thread_flag(TIF_SVE)) { if (WARN_ON(sve_get_vl() != last->sve_vl)) { @@ -731,6 +738,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, if (test_and_clear_tsk_thread_flag(task, TIF_SVE)) sve_to_fpsimd(task); + if (system_supports_sme() && type == ARM64_VEC_SME) + task->thread.svcr &= ~(SYS_SVCR_EL0_SM_MASK | + SYS_SVCR_EL0_ZA_MASK); + if (task == current) put_cpu_fpsimd_context(); diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 3adca0123943..eff50e02b4e2 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -309,6 +309,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) dst->thread.sve_state = NULL; clear_tsk_thread_flag(dst, TIF_SVE); + dst->thread.svcr = 0; + /* clear any pending asynchronous tag fault raised by the parent */ clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel