From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE644C2BA4C for ; Wed, 26 Jan 2022 15:30:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242827AbiAZPax (ORCPT ); Wed, 26 Jan 2022 10:30:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242823AbiAZPax (ORCPT ); Wed, 26 Jan 2022 10:30:53 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A802AC06161C for ; Wed, 26 Jan 2022 07:30:52 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6F3D1B81EA8 for ; Wed, 26 Jan 2022 15:30:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9FC5C340EF; Wed, 26 Jan 2022 15:30:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1643211050; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pZ0sR/3Zt9K27rLzPc61vQxEDfg2RrlFY31bRvO6N6OFIPp4g4zRB+zFGkwKJnnly Po0hzpvXu8fonKYIjLTMxcyj3DE4dD+QhyPzoeQTyegSgUo+DHvfiymjC4kFQgrCji 8QD2UZ/8oeqoUT9R5avbvFTLR9/c6HrhZ/TU1PreurJTBITxHFEEa5q58pDv2dGXqC trOiX47wtIn3DicD96e/nlzZWs3rN3Nh8tMRaojlUe1oRf36MtAcs13ntdhqkw+B9G GrCgK4FSlbF6VqtJvgZxMQato9ihlaTc3F0sQzkrXJ5yeJsrq9R0vmXIJEurOPOqKo 2pY7+oXd9cz6A== From: Mark Brown To: Catalin Marinas , Will Deacon , Marc Zyngier , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, kvmarm@lists.cs.columbia.edu, Mark Brown Subject: [PATCH v10 07/39] arm64/sme: Manually encode SME instructions Date: Wed, 26 Jan 2022 15:27:17 +0000 Message-Id: <20220126152749.233712-8-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126152749.233712-1-broonie@kernel.org> References: <20220126152749.233712-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3671; h=from:subject; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBh8WhXU4t0P2TtzTYSd2s2Hen3CRw27ngQIyNfDOJC 4/nuJxWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYfFoVwAKCRAk1otyXVSH0BgkB/ 0d0z9/jukPse1lSLUm4iTeNDZVoeOnKgOotMSZdM9ZTZGvWTOIS+FmjBTXJkNwjV3iv5OrYDzC3t6D ZsXgVEKZ3bkJy4uS/LGDPwDqYcoGxHJUaeIm6wWqbdpoLEfrney6A7vv139KvIpCyzACxVR+V23mi2 8K8A2GSBhg4jjFnV3N8uijBrSQObswB0HWPNCDVs6tMIj4FQczO2IxpKMhvWtQnlCU2sTrzPjNZ3PX P0DOhUTeFz/Y0yzB4KipXkHcqWMIGI3ebHhRVt/oqvWODbFbFAHp+cpQe7WucTm4cZpTMba+PVrBAC UXOqz1I13k9SijtuBy+BrAP3A8bjLG X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org As with SVE rather than impose ambitious toolchain requirements for SME we manually encode the few instructions which we require in order to perform the work the kernel needs to do. The instructions used to save and restore context are provided as assembler macros while those for entering and leaving streaming mode are done in asm volatile blocks since they are expected to be used from C. We could do the SMSTART and SMSTOP operations with read/modify/write cycles on SVCR but using the aliases provided for individual field accesses should be slightly faster. These instructions are aliases for MSR but since our minimum toolchain requirements are old enough to mean that we can't use the sX_X_cX_cX_X form and they always use xzr rather than taking a value like write_sysreg_s() wants we just use .inst. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 25 +++++++++++++ arch/arm64/include/asm/fpsimdmacros.h | 53 +++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index cb24385e3632..c90f7f99a768 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -249,6 +249,31 @@ static inline void sve_setup(void) { } #endif /* ! CONFIG_ARM64_SVE */ +#ifdef CONFIG_ARM64_SME + +static inline void sme_smstart_sm(void) +{ + asm volatile(".inst 0xd503437f"); +} + +static inline void sme_smstop_sm(void) +{ + asm volatile(".inst 0xd503427f"); +} + +static inline void sme_smstop(void) +{ + asm volatile(".inst 0xd503467f"); +} + +#else + +static inline void sme_smstart_sm(void) { } +static inline void sme_smstop_sm(void) { } +static inline void sme_smstop(void) { } + +#endif /* ! CONFIG_ARM64_SME */ + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 2509d7dde55a..11c426ddd62c 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -93,6 +93,12 @@ .endif .endm +.macro _sme_check_wv v + .if (\v) < 12 || (\v) > 15 + .error "Bad vector select register \v." + .endif +.endm + /* SVE instruction encodings for non-SVE-capable assemblers */ /* (pre binutils 2.28, all kernel capable clang versions support SVE) */ @@ -174,6 +180,53 @@ | (\np) .endm +/* SME instruction encodings for non-SME-capable assemblers */ + +/* RDSVL X\nx, #\imm */ +.macro _sme_rdsvl nx, imm + _check_general_reg \nx + _check_num (\imm), -0x20, 0x1f + .inst 0x04bf5800 \ + | (\nx) \ + | (((\imm) & 0x3f) << 5) +.endm + +/* + * STR (vector from ZA array): + * STR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_str_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1200000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * LDR (vector to ZA array): + * LDR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_ldr_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1000000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * Zero the entire ZA array + * ZERO ZA + */ +.macro zero_za + .inst 0xc00800ff +.endm + .macro __for from:req, to:req .if (\from) == (\to) _for__body %\from -- 2.30.2 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id D664AC63686 for ; Wed, 26 Jan 2022 15:30:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 8CBC949E33; Wed, 26 Jan 2022 10:30:55 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bhDWuD2QWwlq; Wed, 26 Jan 2022 10:30:54 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 416AB49EDF; Wed, 26 Jan 2022 10:30:54 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 79A1449EEB for ; Wed, 26 Jan 2022 10:30:52 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hpf0kPnlplwG for ; Wed, 26 Jan 2022 10:30:51 -0500 (EST) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 4831149EDF for ; Wed, 26 Jan 2022 10:30:51 -0500 (EST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D23D66171D; Wed, 26 Jan 2022 15:30:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9FC5C340EF; Wed, 26 Jan 2022 15:30:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1643211050; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pZ0sR/3Zt9K27rLzPc61vQxEDfg2RrlFY31bRvO6N6OFIPp4g4zRB+zFGkwKJnnly Po0hzpvXu8fonKYIjLTMxcyj3DE4dD+QhyPzoeQTyegSgUo+DHvfiymjC4kFQgrCji 8QD2UZ/8oeqoUT9R5avbvFTLR9/c6HrhZ/TU1PreurJTBITxHFEEa5q58pDv2dGXqC trOiX47wtIn3DicD96e/nlzZWs3rN3Nh8tMRaojlUe1oRf36MtAcs13ntdhqkw+B9G GrCgK4FSlbF6VqtJvgZxMQato9ihlaTc3F0sQzkrXJ5yeJsrq9R0vmXIJEurOPOqKo 2pY7+oXd9cz6A== From: Mark Brown To: Catalin Marinas , Will Deacon , Marc Zyngier , Shuah Khan , Shuah Khan Subject: [PATCH v10 07/39] arm64/sme: Manually encode SME instructions Date: Wed, 26 Jan 2022 15:27:17 +0000 Message-Id: <20220126152749.233712-8-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126152749.233712-1-broonie@kernel.org> References: <20220126152749.233712-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3671; h=from:subject; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBh8WhXU4t0P2TtzTYSd2s2Hen3CRw27ngQIyNfDOJC 4/nuJxWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYfFoVwAKCRAk1otyXVSH0BgkB/ 0d0z9/jukPse1lSLUm4iTeNDZVoeOnKgOotMSZdM9ZTZGvWTOIS+FmjBTXJkNwjV3iv5OrYDzC3t6D ZsXgVEKZ3bkJy4uS/LGDPwDqYcoGxHJUaeIm6wWqbdpoLEfrney6A7vv139KvIpCyzACxVR+V23mi2 8K8A2GSBhg4jjFnV3N8uijBrSQObswB0HWPNCDVs6tMIj4FQczO2IxpKMhvWtQnlCU2sTrzPjNZ3PX P0DOhUTeFz/Y0yzB4KipXkHcqWMIGI3ebHhRVt/oqvWODbFbFAHp+cpQe7WucTm4cZpTMba+PVrBAC UXOqz1I13k9SijtuBy+BrAP3A8bjLG X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Cc: Basant Kumar Dwivedi , Luis Machado , Szabolcs Nagy , Mark Brown , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Alan Hayward , kvmarm@lists.cs.columbia.edu, Salil Akerkar X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu As with SVE rather than impose ambitious toolchain requirements for SME we manually encode the few instructions which we require in order to perform the work the kernel needs to do. The instructions used to save and restore context are provided as assembler macros while those for entering and leaving streaming mode are done in asm volatile blocks since they are expected to be used from C. We could do the SMSTART and SMSTOP operations with read/modify/write cycles on SVCR but using the aliases provided for individual field accesses should be slightly faster. These instructions are aliases for MSR but since our minimum toolchain requirements are old enough to mean that we can't use the sX_X_cX_cX_X form and they always use xzr rather than taking a value like write_sysreg_s() wants we just use .inst. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 25 +++++++++++++ arch/arm64/include/asm/fpsimdmacros.h | 53 +++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index cb24385e3632..c90f7f99a768 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -249,6 +249,31 @@ static inline void sve_setup(void) { } #endif /* ! CONFIG_ARM64_SVE */ +#ifdef CONFIG_ARM64_SME + +static inline void sme_smstart_sm(void) +{ + asm volatile(".inst 0xd503437f"); +} + +static inline void sme_smstop_sm(void) +{ + asm volatile(".inst 0xd503427f"); +} + +static inline void sme_smstop(void) +{ + asm volatile(".inst 0xd503467f"); +} + +#else + +static inline void sme_smstart_sm(void) { } +static inline void sme_smstop_sm(void) { } +static inline void sme_smstop(void) { } + +#endif /* ! CONFIG_ARM64_SME */ + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 2509d7dde55a..11c426ddd62c 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -93,6 +93,12 @@ .endif .endm +.macro _sme_check_wv v + .if (\v) < 12 || (\v) > 15 + .error "Bad vector select register \v." + .endif +.endm + /* SVE instruction encodings for non-SVE-capable assemblers */ /* (pre binutils 2.28, all kernel capable clang versions support SVE) */ @@ -174,6 +180,53 @@ | (\np) .endm +/* SME instruction encodings for non-SME-capable assemblers */ + +/* RDSVL X\nx, #\imm */ +.macro _sme_rdsvl nx, imm + _check_general_reg \nx + _check_num (\imm), -0x20, 0x1f + .inst 0x04bf5800 \ + | (\nx) \ + | (((\imm) & 0x3f) << 5) +.endm + +/* + * STR (vector from ZA array): + * STR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_str_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1200000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * LDR (vector to ZA array): + * LDR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_ldr_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1000000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * Zero the entire ZA array + * ZERO ZA + */ +.macro zero_za + .inst 0xc00800ff +.endm + .macro __for from:req, to:req .if (\from) == (\to) _for__body %\from -- 2.30.2 _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 23F20C2BA4C for ; Wed, 26 Jan 2022 16:22:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6ds+H5FoGKdNJAeQQL+aeA0PS1zUNHSknGjbbrV5Dpo=; b=GzV6ts4S2Jw9No PEkxblXkehTUPajikG5c768qRqust/9/RmKtxjhZ1aVV8xp1ZWQkiOlOvgLtXi3ONUubNAr4Ow4Ta e63pD/vj3nzlTf4w8mcc4gOL8vn3UtJa4Sikz1f/6DemcJsZQZEKRtyVxeNsVyszF97DGtfkUn7zk PzOpXiJjqBCfoeQmaS2wlyZqcqVWxEHAwVlubnSFhRdWn/7nYbKpMKVpYM/LvIkeiP/UK0bCLXxmg OjJqfQWbd6V7oc/StZOzcP4mT0MwB9KmchSLSWN4KSNucD0UorgF61q7vC8r4LbT9O5dU2wcni1tf ZpLqMNThwuxVREfL7hYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nCl2t-00CbHS-MD; Wed, 26 Jan 2022 16:21:20 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nCkG3-00CJ9G-9f for linux-arm-kernel@lists.infradead.org; Wed, 26 Jan 2022 15:30:52 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D23D66171D; Wed, 26 Jan 2022 15:30:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9FC5C340EF; Wed, 26 Jan 2022 15:30:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1643211050; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pZ0sR/3Zt9K27rLzPc61vQxEDfg2RrlFY31bRvO6N6OFIPp4g4zRB+zFGkwKJnnly Po0hzpvXu8fonKYIjLTMxcyj3DE4dD+QhyPzoeQTyegSgUo+DHvfiymjC4kFQgrCji 8QD2UZ/8oeqoUT9R5avbvFTLR9/c6HrhZ/TU1PreurJTBITxHFEEa5q58pDv2dGXqC trOiX47wtIn3DicD96e/nlzZWs3rN3Nh8tMRaojlUe1oRf36MtAcs13ntdhqkw+B9G GrCgK4FSlbF6VqtJvgZxMQato9ihlaTc3F0sQzkrXJ5yeJsrq9R0vmXIJEurOPOqKo 2pY7+oXd9cz6A== From: Mark Brown To: Catalin Marinas , Will Deacon , Marc Zyngier , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, kvmarm@lists.cs.columbia.edu, Mark Brown Subject: [PATCH v10 07/39] arm64/sme: Manually encode SME instructions Date: Wed, 26 Jan 2022 15:27:17 +0000 Message-Id: <20220126152749.233712-8-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220126152749.233712-1-broonie@kernel.org> References: <20220126152749.233712-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3671; h=from:subject; bh=HMQHyHN36InZca7SOj753D+ZgRaa/ByOBuanPgEEWxc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBh8WhXU4t0P2TtzTYSd2s2Hen3CRw27ngQIyNfDOJC 4/nuJxWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYfFoVwAKCRAk1otyXVSH0BgkB/ 0d0z9/jukPse1lSLUm4iTeNDZVoeOnKgOotMSZdM9ZTZGvWTOIS+FmjBTXJkNwjV3iv5OrYDzC3t6D ZsXgVEKZ3bkJy4uS/LGDPwDqYcoGxHJUaeIm6wWqbdpoLEfrney6A7vv139KvIpCyzACxVR+V23mi2 8K8A2GSBhg4jjFnV3N8uijBrSQObswB0HWPNCDVs6tMIj4FQczO2IxpKMhvWtQnlCU2sTrzPjNZ3PX P0DOhUTeFz/Y0yzB4KipXkHcqWMIGI3ebHhRVt/oqvWODbFbFAHp+cpQe7WucTm4cZpTMba+PVrBAC UXOqz1I13k9SijtuBy+BrAP3A8bjLG X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220126_073051_437299_23822260 X-CRM114-Status: GOOD ( 14.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As with SVE rather than impose ambitious toolchain requirements for SME we manually encode the few instructions which we require in order to perform the work the kernel needs to do. The instructions used to save and restore context are provided as assembler macros while those for entering and leaving streaming mode are done in asm volatile blocks since they are expected to be used from C. We could do the SMSTART and SMSTOP operations with read/modify/write cycles on SVCR but using the aliases provided for individual field accesses should be slightly faster. These instructions are aliases for MSR but since our minimum toolchain requirements are old enough to mean that we can't use the sX_X_cX_cX_X form and they always use xzr rather than taking a value like write_sysreg_s() wants we just use .inst. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 25 +++++++++++++ arch/arm64/include/asm/fpsimdmacros.h | 53 +++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index cb24385e3632..c90f7f99a768 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -249,6 +249,31 @@ static inline void sve_setup(void) { } #endif /* ! CONFIG_ARM64_SVE */ +#ifdef CONFIG_ARM64_SME + +static inline void sme_smstart_sm(void) +{ + asm volatile(".inst 0xd503437f"); +} + +static inline void sme_smstop_sm(void) +{ + asm volatile(".inst 0xd503427f"); +} + +static inline void sme_smstop(void) +{ + asm volatile(".inst 0xd503467f"); +} + +#else + +static inline void sme_smstart_sm(void) { } +static inline void sme_smstop_sm(void) { } +static inline void sme_smstop(void) { } + +#endif /* ! CONFIG_ARM64_SME */ + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 2509d7dde55a..11c426ddd62c 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -93,6 +93,12 @@ .endif .endm +.macro _sme_check_wv v + .if (\v) < 12 || (\v) > 15 + .error "Bad vector select register \v." + .endif +.endm + /* SVE instruction encodings for non-SVE-capable assemblers */ /* (pre binutils 2.28, all kernel capable clang versions support SVE) */ @@ -174,6 +180,53 @@ | (\np) .endm +/* SME instruction encodings for non-SME-capable assemblers */ + +/* RDSVL X\nx, #\imm */ +.macro _sme_rdsvl nx, imm + _check_general_reg \nx + _check_num (\imm), -0x20, 0x1f + .inst 0x04bf5800 \ + | (\nx) \ + | (((\imm) & 0x3f) << 5) +.endm + +/* + * STR (vector from ZA array): + * STR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_str_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1200000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * LDR (vector to ZA array): + * LDR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_ldr_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1000000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * Zero the entire ZA array + * ZERO ZA + */ +.macro zero_za + .inst 0xc00800ff +.endm + .macro __for from:req, to:req .if (\from) == (\to) _for__body %\from -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel