From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B2B3C47082 for ; Thu, 3 Jun 2021 18:43:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 157C4613EE for ; Thu, 3 Jun 2021 18:43:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 157C4613EE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=tbSbbnt2gtCfWLrJRdRjgH2BEP3/wMOUrsDreBTBo1c=; b=ur8WmV4qX0VP4U m3KR9CPglCw7V07FGVdmZ9eKPi0w2U1STYLKwP8o9YmM2gtv5EPbhXi+zJH8mAHRllV4vlHtSIVPO KRyCN6eH2p/65L9Dos9WfV4FnNZf/UF1ltcNsIdpjjlk4tD5pSNPwhHF0tmweqA/Yko2ezxvT+u6T PsXF46xhlL39YY0BctA75LnQbp4bKdzG83REClB3XUWUmTF14SGRAY41PVJW4malKKlverXsxHUKR RbrXkGsCjTlEjRx7rarzbJk9sjgRYYKt3TSSmiuXS/qsyHNUAHs/3dsPfy190M/zTpbZhUAiot3cZ 7Ai23XMdM9LH9lmdVVuA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1losHh-00AECn-Ur; Thu, 03 Jun 2021 18:41:38 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1losHe-00AECK-5G for linux-arm-kernel@lists.infradead.org; Thu, 03 Jun 2021 18:41:35 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id C8C30613E3; Thu, 3 Jun 2021 18:41:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1622745693; bh=JxehT3P5awbYBJ05zV8hUKtwpwTXBYjKl4Y8f+6O5jc=; h=From:To:Cc:Subject:Date:From; b=CA71RcrQEkgIy4HPMgT0QDo3GnwOt9tSkZj7ipy/i5IPO21XF8d1ZZxcf47JwSkCq 5GU7kdUYNYiSCNhjAxFv/1Ugjy1BBeVV/h+EhO6HcKkFlck01xp4jmX408TbTOdsgW GfieiKEhji/EpNqH8LdqfoYwD3+ZvFpWvAIj1rZcNn7hUWqIH3z62TA6TU/xX9NoAg qxcnVuWXyRVrToIHG6seI5KBUvuWGYcdGxVOgs1uDTsHARZoI90b33Mh4ar/9o2v2L aL+MbpuRbrMkSxx0Sp41HmbpB1J1QCA2oUnBQTCcd1i6NH7tOknp/mVSRkXE4XjRLX ZAA+qtcBVmmEg== From: Mark Brown To: Catalin Marinas , Will Deacon Cc: Lorenzo Pieralisi , Sudeep Holla , linux-arm-kernel@lists.infradead.org, Mark Brown , Ard Biesheuvel , Marc Zyngier Subject: [PATCH v6] arm64: smccc: Support SMCCC v1.3 SVE register saving hint Date: Thu, 3 Jun 2021 19:41:18 +0100 Message-Id: <20210603184118.15090-1-broonie@kernel.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6418; h=from:subject; bh=JxehT3P5awbYBJ05zV8hUKtwpwTXBYjKl4Y8f+6O5jc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBguSHHbrAnSVRmnEh/Nw99z6d0rAmgZo3bxF4RXiJt T612oUGJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYLkhxwAKCRAk1otyXVSH0Ii6B/ 9DSCscaHyjxVo/3wBZQjskc/6+wTZ/yiMZHhtYptpEeEkdiEW7O2L1Tos2jSYUl4gzesGM3CE6USBd e9E0O68HFvUwVQ+kbeGWRG/jwqWqsajaTKLktR/ld0KX/8mTqXHSpP3PJiEpai6XKbRQ3Qd4GfwfpM FKZ8EJojo49NvwdPwy/t3U2njwajEgbwbSScpBLOy/sGErv/NpWdaQ8zfKIr1EK6H6QZusIcI4UTfz o83/0C9k90JApeF9njus36RJu0OWf8KVOmzfR+OLzXiQvfOgIJkhTAKTi8/1Ol4m9obao0ibAzxLAJ zbUWRtGJP3FB0OsSiI/XLlKuoBU9en X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210603_114134_261527_8549B99C X-CRM114-Status: GOOD ( 22.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SMCCC v1.2 requires that all SVE state be preserved over SMC calls which introduces substantial overhead in the common case where there is no SVE state in the registers. To avoid this SMCCC v1.3 introduces a flag which allows the caller to say that there is no state that needs to be preserved in the registers. Make use of this flag, setting it if the SMCCC version indicates support for it and the TIF_ flags indicate that there is no live SVE state in the registers, this avoids placing any constraints on when SMCCC calls can be done or triggering extra saving and reloading of SVE register state in the kernel. This would be straightforward enough except for the rather entertaining inline assembly we use to do SMCCC v1.1 calls to allow us to take advantage of the limited number of registers it clobbers. Deal with this by having a function which we call immediately before issuing the SMCCC call to make our checks and set the flag. Using alternatives the overhead if SVE is supported but not detected at runtime can be reduced to a single NOP. Signed-off-by: Mark Brown Reviewed-by: Ard Biesheuvel Reviewed-by: Marc Zyngier --- v6: - Prototype __arm_smccc_sve_check add add the _arm to the name to fix issues with symbol versioning. v5: - Use x30 rather than lr in the clobber for wider toolchain compatibility. v4: - Also clobber cc as per suggestion from Ard. - Rebase onto v5.13-rc3. arch/arm64/kernel/smccc-call.S | 26 ++++++++++++++++++++++++++ drivers/firmware/smccc/smccc.c | 4 ++++ include/linux/arm-smccc.h | 33 +++++++++++++++++++++++++++++++-- 3 files changed, 61 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/smccc-call.S b/arch/arm64/kernel/smccc-call.S index d62447964ed9..0d0a03dd0488 100644 --- a/arch/arm64/kernel/smccc-call.S +++ b/arch/arm64/kernel/smccc-call.S @@ -7,8 +7,34 @@ #include #include +#include + +/* + * If we have SMCCC v1.3 and (as is likely) no SVE state in + * the registers then set the SMCCC hint bit to say there's no + * need to preserve it. Do this by directly adjusting the SMCCC + * function value which is already stored in x0 ready to be called. + */ +SYM_FUNC_START(__arm_smccc_sve_check) + + ldr_l x16, smccc_has_sve_hint + cbz x16, 2f + + get_current_task x16 + ldr x16, [x16, #TSK_TI_FLAGS] + tbnz x16, #TIF_FOREIGN_FPSTATE, 1f // Any live FP state? + tbnz x16, #TIF_SVE, 2f // Does that state include SVE? + +1: orr x0, x0, ARM_SMCCC_1_3_SVE_HINT + +2: ret +SYM_FUNC_END(__arm_smccc_sve_check) +EXPORT_SYMBOL(__arm_smccc_sve_check) .macro SMCCC instr +alternative_if ARM64_SVE + bl __arm_smccc_sve_check +alternative_else_nop_endif \instr #0 ldr x4, [sp] stp x0, x1, [x4, #ARM_SMCCC_RES_X0_OFFS] diff --git a/drivers/firmware/smccc/smccc.c b/drivers/firmware/smccc/smccc.c index 028f81d702cc..9f937b125ab0 100644 --- a/drivers/firmware/smccc/smccc.c +++ b/drivers/firmware/smccc/smccc.c @@ -15,6 +15,7 @@ static u32 smccc_version = ARM_SMCCC_VERSION_1_0; static enum arm_smccc_conduit smccc_conduit = SMCCC_CONDUIT_NONE; bool __ro_after_init smccc_trng_available = false; +u64 __ro_after_init smccc_has_sve_hint = false; void __init arm_smccc_version_init(u32 version, enum arm_smccc_conduit conduit) { @@ -22,6 +23,9 @@ void __init arm_smccc_version_init(u32 version, enum arm_smccc_conduit conduit) smccc_conduit = conduit; smccc_trng_available = smccc_probe_trng(); + if (IS_ENABLED(CONFIG_ARM64_SVE) && + smccc_version >= ARM_SMCCC_VERSION_1_3) + smccc_has_sve_hint = true; } enum arm_smccc_conduit arm_smccc_1_1_get_conduit(void) diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 6861489a1890..1dec3e3ac81c 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -63,6 +63,9 @@ #define ARM_SMCCC_VERSION_1_0 0x10000 #define ARM_SMCCC_VERSION_1_1 0x10001 #define ARM_SMCCC_VERSION_1_2 0x10002 +#define ARM_SMCCC_VERSION_1_3 0x10003 + +#define ARM_SMCCC_1_3_SVE_HINT 0x10000 #define ARM_SMCCC_VERSION_FUNC_ID \ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ @@ -216,6 +219,8 @@ u32 arm_smccc_get_version(void); void __init arm_smccc_version_init(u32 version, enum arm_smccc_conduit conduit); +extern u64 smccc_has_sve_hint; + /** * struct arm_smccc_res - Result from SMC/HVC call * @a0-a3 result values from registers 0 to 3 @@ -240,6 +245,15 @@ struct arm_smccc_quirk { } state; }; +/** + * __arm_smccc_sve_check() - Set the SVE hint bit when doing SMC calls + * + * Sets the SMCCC hint bit to indicate if there is live state in the SVE + * registers, this modifies x0 in place and should never be called from C + * code. + */ +asmlinkage unsigned long __arm_smccc_sve_check(unsigned long x0); + /** * __arm_smccc_smc() - make SMC calls * @a0-a7: arguments passed in registers 0 to 7 @@ -297,6 +311,20 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, #endif +/* nVHE hypervisor doesn't have a current thread so needs separate checks */ +#if defined(CONFIG_ARM64_SVE) && !defined(__KVM_NVHE_HYPERVISOR__) + +#define SMCCC_SVE_CHECK ALTERNATIVE("nop \n", "bl __arm_smccc_sve_check \n", \ + ARM64_SVE) +#define smccc_sve_clobbers "x16", "x30", "cc", + +#else + +#define SMCCC_SVE_CHECK +#define smccc_sve_clobbers + +#endif + #define ___count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x #define __count_args(...) \ @@ -364,7 +392,7 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, #define ___constraints(count) \ : __constraint_read_ ## count \ - : "memory" + : smccc_sve_clobbers "memory" #define __constraints(count) ___constraints(count) /* @@ -379,7 +407,8 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, register unsigned long r2 asm("r2"); \ register unsigned long r3 asm("r3"); \ __declare_args(__count_args(__VA_ARGS__), __VA_ARGS__); \ - asm volatile(inst "\n" : \ + asm volatile(SMCCC_SVE_CHECK \ + inst "\n" : \ "=r" (r0), "=r" (r1), "=r" (r2), "=r" (r3) \ __constraints(__count_args(__VA_ARGS__))); \ if (___res) \ -- 2.20.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel