From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48097C433F5 for ; Fri, 25 Feb 2022 17:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fi7eS7iAl57NVoFdIsPpU4PU58eErfV/t1OXtIW4mmE=; b=jV3t2ULFG3Cc09 jq9KF7tB3DTWGgExvD/ltMk7R3aoVqfBRkfciTZL8Xe1XbjyDtFaGJ6EdZ2XnoJ0DpQJO/rDoSq6p sLuvKcK6EJU9eoUp/n90mh4e0H7kQzyZzJ++FiISoUT35uvfQEecBMQmQBzMvsAu8QsY8JutT+PyU k6RBKZqmEspDY76oMbZRL+UmjOC1PmkxtLVyc7iXGSIsiJ6DMpxyM6KlSwe9I2xsrNSNtN5Pb2ko+ sgh8ZGcc7FzOa1nfpudGYc7LrBjH+wXUSSlGdlLOJQYAGxR5RqaQUMoNGFnK5YjAhwfqcx9EhDfI8 tAn2G4esGfT5/AA5mcSg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nNebN-006J7K-OI; Fri, 25 Feb 2022 17:41:58 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nNeUZ-006GWa-Ky for linux-arm-kernel@lists.infradead.org; Fri, 25 Feb 2022 17:34:57 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1F482B832CB; Fri, 25 Feb 2022 17:34:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62496C340F4; Fri, 25 Feb 2022 17:34:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1645810493; bh=eCPdmodXk+aP9UavloJwBpsd9SlzvLKpb59yNybNpEM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IgzdLcyiYPLPBqGG0/+ISOTFfOqKaDHxYyryMcjm/oX1w+AJp8Gkqiq5Y6i6a/vy5 svE25R5LGP/h/2shpw8KuaTxcozvtCpLL0x5LYR35YFeG99mvtWYW6V2cmzPnxVlzQ SkYxImxhjm5Y0voN9j1uYaP+GBRz7/TczBxx6upPIUdGcGCGsNHCInRXuXSAqJOhC9 nImSEMNSmw7KwbyMWLfDdLWNomhm4C+WvS/t6Fo4+ZJ3ieEnMq5VkYUODQeomYL7Ej 2QFEhrgxAzflntpLAeXTcgS2nkbSnrSQdFO6cmdn7j/4Lzv6y+S4Qv1NeV/pyHhljT uLqFx8qIwipfA== From: Mark Brown To: Catalin Marinas , Will Deacon , Marc Zyngier , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, kvmarm@lists.cs.columbia.edu, Mark Brown Subject: [PATCH v12 08/40] arm64/sme: Manually encode SME instructions Date: Fri, 25 Feb 2022 16:58:51 +0000 Message-Id: <20220225165923.1474372-9-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220225165923.1474372-1-broonie@kernel.org> References: <20220225165923.1474372-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3770; h=from:subject; bh=eCPdmodXk+aP9UavloJwBpsd9SlzvLKpb59yNybNpEM=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBiGQrQRUGTP7JuPbFmFOZFlojowhZE0kuj7Ac8c0yG /18YqKyJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYhkK0AAKCRAk1otyXVSH0PxdB/ 4iOtNgrVzBDIhGYCnla6mQTY7DE0BQHucpbWTNuWMdbqpoAOELXrteJs2oiVLdy68xqJL91jdVy6vu VwTgIyaSbw9xCQ4HSBZWiiLf44XCUubldbCZgzEQvYXYboA63OnFyEJVXnOx8PB1ljtECQn2gsZQa3 Zlzly0uuoesFD7R+Z7f4WGno2pyn4Tn7jot+PC5cTc31ch7dwp0eXfD7riQ80bib7Ret/KTm+WqRCe riEdUkoGRgy2Vm4ol+iuyBTnM+NwlI0lrzhp+N/UbvDmwRzuiVHLFntkGJ2FjIF0IGL3s5WcmpSsAQ DjFZcBDDvp35TgpMVWZob1wJJPTavP X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220225_093456_040447_CF77236D X-CRM114-Status: GOOD ( 14.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As with SVE rather than impose ambitious toolchain requirements for SME we manually encode the few instructions which we require in order to perform the work the kernel needs to do. The instructions used to save and restore context are provided as assembler macros while those for entering and leaving streaming mode are done in asm volatile blocks since they are expected to be used from C. We could do the SMSTART and SMSTOP operations with read/modify/write cycles on SVCR but using the aliases provided for individual field accesses should be slightly faster. These instructions are aliases for MSR but since our minimum toolchain requirements are old enough to mean that we can't use the sX_X_cX_cX_X form and they always use xzr rather than taking a value like write_sysreg_s() wants we just use .inst. Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 25 +++++++++++++ arch/arm64/include/asm/fpsimdmacros.h | 54 +++++++++++++++++++++++++++ 2 files changed, 79 insertions(+) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index cb24385e3632..6e2dc9dcbf49 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -249,6 +249,31 @@ static inline void sve_setup(void) { } #endif /* ! CONFIG_ARM64_SVE */ +#ifdef CONFIG_ARM64_SME + +static inline void sme_smstart_sm(void) +{ + asm volatile(__msr_s(SYS_SVCR_SMSTART_SM_EL0, "xzr")); +} + +static inline void sme_smstop_sm(void) +{ + asm volatile(__msr_s(SYS_SVCR_SMSTOP_SM_EL0, "xzr")); +} + +static inline void sme_smstop(void) +{ + asm volatile(__msr_s(SYS_SVCR_SMSTOP_SMZA_EL0, "xzr")); +} + +#else + +static inline void sme_smstart_sm(void) { } +static inline void sme_smstop_sm(void) { } +static inline void sme_smstop(void) { } + +#endif /* ! CONFIG_ARM64_SME */ + /* For use by EFI runtime services calls only */ extern void __efi_fpsimd_begin(void); extern void __efi_fpsimd_end(void); diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h index 2509d7dde55a..2e9a33155081 100644 --- a/arch/arm64/include/asm/fpsimdmacros.h +++ b/arch/arm64/include/asm/fpsimdmacros.h @@ -93,6 +93,12 @@ .endif .endm +.macro _sme_check_wv v + .if (\v) < 12 || (\v) > 15 + .error "Bad vector select register \v." + .endif +.endm + /* SVE instruction encodings for non-SVE-capable assemblers */ /* (pre binutils 2.28, all kernel capable clang versions support SVE) */ @@ -174,6 +180,54 @@ | (\np) .endm +/* SME instruction encodings for non-SME-capable assemblers */ +/* (pre binutils 2.38/LLVM 13) */ + +/* RDSVL X\nx, #\imm */ +.macro _sme_rdsvl nx, imm + _check_general_reg \nx + _check_num (\imm), -0x20, 0x1f + .inst 0x04bf5800 \ + | (\nx) \ + | (((\imm) & 0x3f) << 5) +.endm + +/* + * STR (vector from ZA array): + * STR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_str_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1200000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * LDR (vector to ZA array): + * LDR ZA[\nw, #\offset], [X\nxbase, #\offset, MUL VL] + */ +.macro _sme_ldr_zav nw, nxbase, offset=0 + _sme_check_wv \nw + _check_general_reg \nxbase + _check_num (\offset), -0x100, 0xff + .inst 0xe1000000 \ + | (((\nw) & 3) << 13) \ + | ((\nxbase) << 5) \ + | ((\offset) & 7) +.endm + +/* + * Zero the entire ZA array + * ZERO ZA + */ +.macro zero_za + .inst 0xc00800ff +.endm + .macro __for from:req, to:req .if (\from) == (\to) _for__body %\from -- 2.30.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel