From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A46EC56201 for ; Fri, 6 Nov 2020 19:39:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1479C21D81 for ; Fri, 6 Nov 2020 19:39:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="HFyIveJD"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wX4D0b+o" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1479C21D81 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SnMWf67bpJAhZzJuLj3/gjnOl0b5Tu1jTRLevsUMOnc=; b=HFyIveJDX9hwit/gKe7EEj1Ru cK9T/bXsgmzu2CyM+ZG3awZ7ip1ifvu5d50qkQm+JgdETA0iZ/hxSOI2+vLYbUHBx6fuch7SD2L0h TikjchpT3aY9NVcikgkZJ74+2Tx5rbz4NE96lZ/r4Kd6AiAN6yfR4+4MAped3oyqOfkZ6pyHV/Pkr PmE6xBqF4rMTw7zyHgiCkLg0uo8+Hz49Ebpf+s1aQVBI1/gzoWM5fCSyABu3P8oBaOhFGXf8gV5e2 LdMG+ALBFAI0qRVWyPgSLpxPp3fwFQD8/ccfo/c1G0RD6zMoNie4NvCmjngwe3UqJgjNuOwT40AIf hJE7tHVGA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kb7Yn-0001e6-Fj; Fri, 06 Nov 2020 19:38:09 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kb7Yi-0001cv-3u for linux-arm-kernel@lists.infradead.org; Fri, 06 Nov 2020 19:38:05 +0000 Received: from localhost (fw-tnat.cambridge.arm.com [217.140.96.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CE36B21D46; Fri, 6 Nov 2020 19:38:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604691483; bh=V0qT3KeyEMwMhWYgEu63jQVIT4d9Bnd4iqvIuR8h+pM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wX4D0b+oGKMNS+UWjTyyFRXf0n6+93TVtzMjlnoU4m24mIQcy0k0qoBGIRWxI/n9R 2tQMsQVdLRoK+ZnGTj01LYNNCDHHGApGq+0qOhniGtYJYWP5CPHT2Ku5Y56/9E3RXK a0umInEhDe/ettAvFYLrQ0aLL1DqShP2zInuONMQ= From: Mark Brown To: Catalin Marinas , Will Deacon Subject: [PATCH v5 2/2] arm64/sve: Rework SVE trap access to use TIF_SVE_NEEDS_FLUSH Date: Fri, 6 Nov 2020 19:35:53 +0000 Message-Id: <20201106193553.22946-3-broonie@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201106193553.22946-1-broonie@kernel.org> References: <20201106193553.22946-1-broonie@kernel.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=0b72bb7c9192ed6aded2b5095c751f1f86f40b41; i=epBjqFTkXjOKKDZ2e6QR+vI4aZkqr5uepApQAeRfX8M=; m=X9KjM75MiD+i3dld7Gp65+fmPLO7Pt3CqUzlQ3ms9hE=; p=QUlhUcXkBogOvfyfK40pJG783tY9G0NsCQ2XWU2avzU= X-Patch-Sig: m=pgp; i=julien.grall@arm.com; s=0xC3F436CA30F5D8EB; b=iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAl+lpZgACgkQJNaLcl1Uh9BaSgf+Kvd ClIuh148OC0p46KJvH9X2GEDw37axtNYS2PfX5temIH3Ug+on3+Q/fuuXHbDgoWhEKjkVx+KQIfko /it4hNc5KC6yyMVewi4LCD2hZA6Zgy1Yt9X9gaZ144AgDYryp6seCwE9MveIesgkV7FYvADA4aCBa ChmjGnRI/Hmu/TX8d9QgOljWmTx0WFmMGqdXUPXbh0M2gaS1VOmgNUvOVP1j5OJdclpmHOHSZX0Al 1TvRUQ55isttTlVTe0OSo76gtU128cM4et/3qECoGYCUveB5YuUxguFvPO7CL+S8bzdb5b/bC1bnA qBiFXMJ2At0GsWLNol2e0olADR6CYJg== X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201106_143804_403845_679CAD8A X-CRM114-Status: GOOD ( 22.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Grall , Zhang Lei , Julien Grall , Mark Brown , Dave Martin , linux-arm-kernel@lists.infradead.org, Daniel Kiss Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Julien Grall SVE state will be flushed on the first SVE access trap. At the moment, the SVE state will be generated from the FPSIMD state in software and then loaded in memory. It is possible to use the newly introduce flag TIF_SVE_NEEDS_FLUSH to avoid a lot of memory access. If the FPSIMD state is in memory, the SVE state will be loaded on return to userspace from the FPSIMD state. If the FPSIMD state is loaded, then we need to set the vector-length before relying on return to userspace to flush the SVE registers. This is because the vector length is only set when loading from memory. We also need to rebind the task to the CPU so the newly allocated SVE state is used when the task is saved. Signed-off-by: Julien Grall Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 2 ++ arch/arm64/kernel/entry-fpsimd.S | 5 +++++ arch/arm64/kernel/fpsimd.c | 32 +++++++++++++++++++++----------- 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index bec5f14b622a..e60aa4ebb351 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -74,6 +74,8 @@ extern void sve_load_from_fpsimd_state(struct user_fpsimd_state const *state, unsigned long vq_minus_1); extern unsigned int sve_get_vl(void); +extern void sve_set_vq(unsigned long vq_minus_1); + struct arm64_cpu_capabilities; extern void sve_kernel_enable(const struct arm64_cpu_capabilities *__unused); diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S index 2ca395c25448..3ecec60d3295 100644 --- a/arch/arm64/kernel/entry-fpsimd.S +++ b/arch/arm64/kernel/entry-fpsimd.S @@ -48,6 +48,11 @@ SYM_FUNC_START(sve_get_vl) ret SYM_FUNC_END(sve_get_vl) +SYM_FUNC_START(sve_set_vq) + sve_load_vq x0, x1, x2 + ret +SYM_FUNC_END(sve_set_vq) + /* * Load SVE state from FPSIMD state. * diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index e2dbc23ecf19..b320371e1404 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -974,10 +974,10 @@ void fpsimd_release_task(struct task_struct *dead_task) /* * Trapped SVE access * - * Storage is allocated for the full SVE state, the current FPSIMD - * register contents are migrated across, and TIF_SVE is set so that - * the SVE access trap will be disabled the next time this task - * reaches ret_to_user. + * Storage is allocated for the full SVE state so that the code + * running subsequently has somewhere to save the SVE registers to. We + * then rely on ret_to_user to actually convert the FPSIMD registers + * to SVE state by flushing as required. * * TIF_SVE should be clear on entry: otherwise, fpsimd_restore_current_state() * would have disabled the SVE access trap for userspace during @@ -995,14 +995,24 @@ void do_sve_acc(unsigned int esr, struct pt_regs *regs) get_cpu_fpsimd_context(); - fpsimd_save(); - - /* Force ret_to_user to reload the registers: */ - fpsimd_flush_task_state(current); + set_thread_flag(TIF_SVE_NEEDS_FLUSH); + /* + * We should not be here with SVE enabled. TIF_SVE will be set + * before returning to userspace by fpsimd_restore_current_state(). + */ + WARN_ON(test_thread_flag(TIF_SVE)); - fpsimd_to_sve(current); - if (test_and_set_thread_flag(TIF_SVE)) - WARN_ON(1); /* SVE access shouldn't have trapped */ + /* + * When the FPSIMD state is loaded: + * - The return path (see fpsimd_restore_current_state) requires + * the vector length to be loaded beforehand. + * - We need to rebind the task to the CPU so the newly allocated + * SVE state is used when the task is saved. + */ + if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) { + sve_set_vq(sve_vq_from_vl(current->thread.sve_vl) - 1); + fpsimd_bind_task_to_cpu(); + } put_cpu_fpsimd_context(); } -- 2.20.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel