From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D121C43334 for ; Mon, 11 Jul 2022 09:41:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 067164B857; Mon, 11 Jul 2022 05:41:00 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rsgGv4U8B2lT; Mon, 11 Jul 2022 05:40:58 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 3ABA74B8E4; Mon, 11 Jul 2022 05:40:58 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 5046F4B857 for ; Mon, 11 Jul 2022 05:40:56 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TNDlP-sCB+DW for ; Mon, 11 Jul 2022 05:40:54 -0400 (EDT) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 8D6874B856 for ; Mon, 11 Jul 2022 05:40:54 -0400 (EDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 88A8D60F13; Mon, 11 Jul 2022 09:40:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39A8BC34115; Mon, 11 Jul 2022 09:40:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657532453; bh=JiArFBRu09eCdxnEj/hpCwPAGswEzEbHzz0CHgP2+qQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=EsqVYHaB4xUDWYyCp3OhJ7yRsU2g76+P0Kd40DY5m3CUWYksDVJSqOZWFIFrawHCs ltKE0ndMyblETSXxRj+dlkx7E4NsruBjg87LKn3wyFF0stbUNjj2wqRHhU4hpeS/fT ZEZGLA+Ku1e4sqqYavy3PT7ZIRsuJrlrrmtLamOTw/Z4VW/Ha/KOtTHHlnnIiYQDQq q6u635/HCnS5KO3GwDnlLlTBUKcCSDuDMkjGG0yVPrL9yHfT47Lqakae1IWBNhf3wl XVhb8+fLRT6n45rGY4YYbmT6kR89wjPp6gjtXX1Pxa/RDN60J1+enNBIlHpUCQ8jmp 2ANsmtDCgx3dQ== Received: from [213.208.244.172] (helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oApuM-006cap-R7; Mon, 11 Jul 2022 10:40:51 +0100 Date: Mon, 11 Jul 2022 10:40:50 +0100 Message-ID: <87ilo4kmvx.wl-maz@kernel.org> From: Marc Zyngier To: Mark Brown Subject: Re: [PATCH v2 2/7] arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE In-Reply-To: <20220620124158.482039-3-broonie@kernel.org> References: <20220620124158.482039-1-broonie@kernel.org> <20220620124158.482039-3-broonie@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 213.208.244.172 X-SA-Exim-Rcpt-To: broonie@kernel.org, catalin.marinas@arm.com, will@kernel.org, zhang.lei@jp.fujitsu.com, james.morse@arm.com, alexandru.elisei@arm.com, andre.przywara@arm.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: Catalin Marinas , Zhang Lei , Andre Przywara , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Mon, 20 Jun 2022 13:41:53 +0100, Mark Brown wrote: > > When we save the state for the floating point registers this can be done > in the form visible through either the FPSIMD V registers or the SVE Z and > P registers. At present we track which format is currently used based on > TIF_SVE and the SME streaming mode state but particularly in the SVE case > this limits our options for optimising things, especially around syscalls. > Introduce a new enum in thread_struct which explicitly states which format > is active and keep it up to date when we change it. > > At present we do not use this state except to verify that it has the > expected value when loading the state, future patches will introduce > functional changes. > > Signed-off-by: Mark Brown > --- > arch/arm64/include/asm/fpsimd.h | 2 +- > arch/arm64/include/asm/kvm_host.h | 1 + > arch/arm64/include/asm/processor.h | 6 ++++ > arch/arm64/kernel/fpsimd.c | 57 ++++++++++++++++++++++-------- > arch/arm64/kernel/process.c | 2 ++ > arch/arm64/kernel/ptrace.c | 6 ++-- > arch/arm64/kernel/signal.c | 3 ++ > arch/arm64/kvm/fpsimd.c | 3 +- > 8 files changed, 61 insertions(+), 19 deletions(-) > > diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h > index 847fd119cdb8..5762419fdcc0 100644 > --- a/arch/arm64/include/asm/fpsimd.h > +++ b/arch/arm64/include/asm/fpsimd.h > @@ -61,7 +61,7 @@ extern void fpsimd_kvm_prepare(void); > extern void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *state, > void *sve_state, unsigned int sve_vl, > void *za_state, unsigned int sme_vl, > - u64 *svcr); > + u64 *svcr, enum fp_state *type); > > extern void fpsimd_flush_task_state(struct task_struct *target); > extern void fpsimd_save_and_flush_cpu_state(void); > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index de32152cea04..250e23f221c4 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -310,6 +310,7 @@ struct kvm_vcpu_arch { > void *sve_state; > unsigned int sve_max_vl; > u64 svcr; > + enum fp_state fp_type; > > /* Stage 2 paging state used by the hardware on next switch */ > struct kvm_s2_mmu *hw_mmu; > diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h > index 9e58749db21d..192986509a8e 100644 > --- a/arch/arm64/include/asm/processor.h > +++ b/arch/arm64/include/asm/processor.h > @@ -122,6 +122,11 @@ enum vec_type { > ARM64_VEC_MAX, > }; > > +enum fp_state { > + FP_STATE_FPSIMD, > + FP_STATE_SVE, > +}; > + > struct cpu_context { > unsigned long x19; > unsigned long x20; > @@ -152,6 +157,7 @@ struct thread_struct { > struct user_fpsimd_state fpsimd_state; > } uw; > > + enum fp_state fp_type; /* registers FPSIMD or SVE? */ > unsigned int fpsimd_cpu; > void *sve_state; /* SVE registers, if any */ > void *za_state; /* ZA register, if any */ > diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c > index d67e658f1e24..fdb2925becdf 100644 > --- a/arch/arm64/kernel/fpsimd.c > +++ b/arch/arm64/kernel/fpsimd.c > @@ -125,6 +125,7 @@ struct fpsimd_last_state_struct { > u64 *svcr; > unsigned int sve_vl; > unsigned int sme_vl; > + enum fp_state *type; For consistency: s/type/fp_type/ ? > }; > > static DEFINE_PER_CPU(struct fpsimd_last_state_struct, fpsimd_last_state); > @@ -330,15 +331,6 @@ void task_set_vl_onexec(struct task_struct *task, enum vec_type type, > * The task can execute SVE instructions while in userspace without > * trapping to the kernel. > * > - * When stored, Z0-Z31 (incorporating Vn in bits[127:0] or the > - * corresponding Zn), P0-P15 and FFR are encoded in > - * task->thread.sve_state, formatted appropriately for vector > - * length task->thread.sve_vl or, if SVCR.SM is set, > - * task->thread.sme_vl. > - * > - * task->thread.sve_state must point to a valid buffer at least > - * sve_state_size(task) bytes in size. > - * > * During any syscall, the kernel may optionally clear TIF_SVE and > * discard the vector state except for the FPSIMD subset. > * > @@ -348,7 +340,15 @@ void task_set_vl_onexec(struct task_struct *task, enum vec_type type, > * do_sve_acc() to be called, which does some preparation and then > * sets TIF_SVE. > * > - * When stored, FPSIMD registers V0-V31 are encoded in > + * During any syscall, the kernel may optionally clear TIF_SVE and > + * discard the vector state except for the FPSIMD subset. > + * > + * The data will be stored in one of two formats: > + * > + * * FPSIMD only - FP_STATE_FPSIMD: > + * > + * When the FPSIMD only state stored task->thread.fp_type is set to > + * FP_STATE_FPSIMD, the FPSIMD registers V0-V31 are encoded in > * task->thread.uw.fpsimd_state; bits [max : 128] for each of Z0-Z31 are > * logically zero but not stored anywhere; P0-P15 and FFR are not > * stored and have unspecified values from userspace's point of > @@ -358,6 +358,18 @@ void task_set_vl_onexec(struct task_struct *task, enum vec_type type, > * task->thread.sve_state does not need to be non-NULL, valid or any > * particular size: it must not be dereferenced. > * > + * * SVE state - FP_STATE_SVE: > + * > + * When the full SVE state is stored task->thread.fp_type is set to > + * FP_STATE_SVE and Z0-Z31 (incorporating Vn in bits[127:0] or the > + * corresponding Zn), P0-P15 and FFR are encoded in in > + * task->thread.sve_state, formatted appropriately for vector > + * length task->thread.sve_vl or, if SVCR.SM is set, > + * task->thread.sme_vl. Can you add a commend about the fate of the Vn data in fpsimd_state? My understanding is that they are stale data as soon as FP_STATE_SVE is set, but I'd really like this being clarified. > + * > + * task->thread.sve_state must point to a valid buffer at least > + * sve_state_size(task) bytes in size. > + * > * * FPSR and FPCR are always stored in task->thread.uw.fpsimd_state > * irrespective of whether TIF_SVE is clear or set, since these are > * not vector length dependent. > @@ -404,12 +416,15 @@ static void task_fpsimd_load(void) > } > } > > - if (restore_sve_regs) > + if (restore_sve_regs) { > + WARN_ON_ONCE(current->thread.fp_type != FP_STATE_SVE); > sve_load_state(sve_pffr(¤t->thread), > ¤t->thread.uw.fpsimd_state.fpsr, > restore_ffr); > - else > + } else { > + WARN_ON_ONCE(current->thread.fp_type != FP_STATE_FPSIMD); > fpsimd_load_state(¤t->thread.uw.fpsimd_state); > + } > } > > /* > @@ -475,8 +490,10 @@ static void fpsimd_save(void) > sve_save_state((char *)last->sve_state + > sve_ffr_offset(vl), > &last->st->fpsr, save_ffr); > + *last->type = FP_STATE_SVE; > } else { > fpsimd_save_state(last->st); > + *last->type = FP_STATE_FPSIMD; > } > } > > @@ -847,8 +864,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, > > fpsimd_flush_task_state(task); > if (test_and_clear_tsk_thread_flag(task, TIF_SVE) || > - thread_sm_enabled(&task->thread)) > + thread_sm_enabled(&task->thread)) { > sve_to_fpsimd(task); > + task->thread.fp_type = FP_STATE_FPSIMD; Can you move this assignment into the sve_to_fpsimd() helper? > + } > > if (system_supports_sme() && type == ARM64_VEC_SME) { > task->thread.svcr &= ~(SVCR_SM_MASK | > @@ -1367,6 +1386,7 @@ static void sve_init_regs(void) > fpsimd_bind_task_to_cpu(); > } else { > fpsimd_to_sve(current); > + current->thread.fp_type = FP_STATE_SVE; Same thing here. > } > } > > @@ -1606,6 +1626,8 @@ void fpsimd_flush_thread(void) > current->thread.svcr = 0; > } > > + current->thread.fp_type = FP_STATE_FPSIMD; > + > put_cpu_fpsimd_context(); > kfree(sve_state); > kfree(za_state); > @@ -1654,8 +1676,10 @@ void fpsimd_kvm_prepare(void) > */ > get_cpu_fpsimd_context(); > > - if (test_and_clear_thread_flag(TIF_SVE)) > + if (test_and_clear_thread_flag(TIF_SVE)) { > sve_to_fpsimd(current); > + current->thread.fp_type = FP_STATE_FPSIMD; > + } > > put_cpu_fpsimd_context(); > } > @@ -1677,6 +1701,7 @@ static void fpsimd_bind_task_to_cpu(void) > last->sve_vl = task_get_sve_vl(current); > last->sme_vl = task_get_sme_vl(current); > last->svcr = ¤t->thread.svcr; > + last->type = ¤t->thread.fp_type; > current->thread.fpsimd_cpu = smp_processor_id(); > > /* > @@ -1700,7 +1725,8 @@ static void fpsimd_bind_task_to_cpu(void) > > void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st, void *sve_state, > unsigned int sve_vl, void *za_state, > - unsigned int sme_vl, u64 *svcr) > + unsigned int sme_vl, u64 *svcr, > + enum fp_state *type) > { > struct fpsimd_last_state_struct *last = > this_cpu_ptr(&fpsimd_last_state); > @@ -1714,6 +1740,7 @@ void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st, void *sve_state, > last->za_state = za_state; > last->sve_vl = sve_vl; > last->sme_vl = sme_vl; > + last->type = type; > } > > /* > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > index 92bcc1768f0b..944d782d581b 100644 > --- a/arch/arm64/kernel/process.c > +++ b/arch/arm64/kernel/process.c > @@ -335,6 +335,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) > clear_tsk_thread_flag(dst, TIF_SME); > } > > + dst->thread.fp_type = FP_STATE_FPSIMD; > + > /* clear any pending asynchronous tag fault raised by the parent */ > clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); > > diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c > index 21da83187a60..e0233f322af3 100644 > --- a/arch/arm64/kernel/ptrace.c > +++ b/arch/arm64/kernel/ptrace.c > @@ -892,8 +892,7 @@ static int sve_set_common(struct task_struct *target, > ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, > SVE_PT_FPSIMD_OFFSET); > clear_tsk_thread_flag(target, TIF_SVE); > - if (type == ARM64_VEC_SME) > - fpsimd_force_sync_to_sve(target); I don't get this particular change. Can you please clarify? > + target->thread.fp_type = FP_STATE_FPSIMD; > goto out; > } > > @@ -916,6 +915,7 @@ static int sve_set_common(struct task_struct *target, > if (!target->thread.sve_state) { > ret = -ENOMEM; > clear_tsk_thread_flag(target, TIF_SVE); > + target->thread.fp_type = FP_STATE_FPSIMD; > goto out; > } > > @@ -954,6 +954,8 @@ static int sve_set_common(struct task_struct *target, > &target->thread.uw.fpsimd_state.fpsr, > start, end); > > + target->thread.fp_type = FP_STATE_SVE; > + > out: > fpsimd_flush_task_state(target); > return ret; > diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c > index b0980fbb6bc7..23dd4c3bcfed 100644 > --- a/arch/arm64/kernel/signal.c > +++ b/arch/arm64/kernel/signal.c > @@ -207,6 +207,7 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx) > __get_user_error(fpsimd.fpcr, &ctx->fpcr, err); > > clear_thread_flag(TIF_SVE); > + current->thread.fp_type = FP_STATE_FPSIMD; > > /* load the hardware registers from the fpsimd_state structure */ > if (!err) > @@ -289,6 +290,7 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) > if (sve.head.size <= sizeof(*user->sve)) { > clear_thread_flag(TIF_SVE); > current->thread.svcr &= ~SVCR_SM_MASK; > + current->thread.fp_type = FP_STATE_FPSIMD; > goto fpsimd_only; > } > > @@ -324,6 +326,7 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) > current->thread.svcr |= SVCR_SM_MASK; > else > set_thread_flag(TIF_SVE); > + current->thread.fp_type = FP_STATE_SVE; > > fpsimd_only: > /* copy the FP and status/control registers */ > diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c > index a433ee8da232..be3ddb214ab1 100644 > --- a/arch/arm64/kvm/fpsimd.c > +++ b/arch/arm64/kvm/fpsimd.c > @@ -139,7 +139,8 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) > fpsimd_bind_state_to_cpu(&vcpu->arch.ctxt.fp_regs, > vcpu->arch.sve_state, > vcpu->arch.sve_max_vl, > - NULL, 0, &vcpu->arch.svcr); > + NULL, 0, &vcpu->arch.svcr, > + &vcpu->arch.fp_type); > > clear_thread_flag(TIF_FOREIGN_FPSTATE); > update_thread_flag(TIF_SVE, vcpu_has_sve(vcpu)); Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB105C43334 for ; Mon, 11 Jul 2022 09:42:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=23YOAJ1+YZ5XvbAQw6el39dCmUXD6jkW0r8ezxoQIKU=; b=b4X0bw8G1Y9m26 Hx0601/BoYTKVnFIe0FzYpNbPAxC4TyNKmJbT31irWJhuXhpky5Jnaxxiw2qCza4eUv+e6x97yfXr f8kWUmzyBkv06z5rlHfITdVsgy20nAA/WlyRWTWJHIfcbb5qRMcn1R0z6cG+cOdwSb3F5gO3eDCUQ GzuCtRMXGnPEWv2QYyIxVkkpclQwFBJO6hd7aiPasfEClMptMCpNbVjSWtWZ5KyhvpWd94/L4oKaF DNi8Om6Rn1hzdAPAEaG9MHnM79o4M78kPkCTW4K1XwmqUcy5xLSNmKwOyS9OYfsYvXGzh6xsK00Rm mSU4F9Jh4ulRje8L4Ibg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oApuT-000DiP-Kz; Mon, 11 Jul 2022 09:40:57 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oApuQ-000DgG-7n for linux-arm-kernel@lists.infradead.org; Mon, 11 Jul 2022 09:40:56 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 88A8D60F13; Mon, 11 Jul 2022 09:40:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39A8BC34115; Mon, 11 Jul 2022 09:40:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657532453; bh=JiArFBRu09eCdxnEj/hpCwPAGswEzEbHzz0CHgP2+qQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=EsqVYHaB4xUDWYyCp3OhJ7yRsU2g76+P0Kd40DY5m3CUWYksDVJSqOZWFIFrawHCs ltKE0ndMyblETSXxRj+dlkx7E4NsruBjg87LKn3wyFF0stbUNjj2wqRHhU4hpeS/fT ZEZGLA+Ku1e4sqqYavy3PT7ZIRsuJrlrrmtLamOTw/Z4VW/Ha/KOtTHHlnnIiYQDQq q6u635/HCnS5KO3GwDnlLlTBUKcCSDuDMkjGG0yVPrL9yHfT47Lqakae1IWBNhf3wl XVhb8+fLRT6n45rGY4YYbmT6kR89wjPp6gjtXX1Pxa/RDN60J1+enNBIlHpUCQ8jmp 2ANsmtDCgx3dQ== Received: from [213.208.244.172] (helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oApuM-006cap-R7; Mon, 11 Jul 2022 10:40:51 +0100 Date: Mon, 11 Jul 2022 10:40:50 +0100 Message-ID: <87ilo4kmvx.wl-maz@kernel.org> From: Marc Zyngier To: Mark Brown Cc: Catalin Marinas , Will Deacon , Zhang Lei , James Morse , Alexandru Elisei , Andre Przywara , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v2 2/7] arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE In-Reply-To: <20220620124158.482039-3-broonie@kernel.org> References: <20220620124158.482039-1-broonie@kernel.org> <20220620124158.482039-3-broonie@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 213.208.244.172 X-SA-Exim-Rcpt-To: broonie@kernel.org, catalin.marinas@arm.com, will@kernel.org, zhang.lei@jp.fujitsu.com, james.morse@arm.com, alexandru.elisei@arm.com, andre.przywara@arm.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220711_024054_386687_9F09C28F X-CRM114-Status: GOOD ( 47.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 20 Jun 2022 13:41:53 +0100, Mark Brown wrote: > > When we save the state for the floating point registers this can be done > in the form visible through either the FPSIMD V registers or the SVE Z and > P registers. At present we track which format is currently used based on > TIF_SVE and the SME streaming mode state but particularly in the SVE case > this limits our options for optimising things, especially around syscalls. > Introduce a new enum in thread_struct which explicitly states which format > is active and keep it up to date when we change it. > > At present we do not use this state except to verify that it has the > expected value when loading the state, future patches will introduce > functional changes. > > Signed-off-by: Mark Brown > --- > arch/arm64/include/asm/fpsimd.h | 2 +- > arch/arm64/include/asm/kvm_host.h | 1 + > arch/arm64/include/asm/processor.h | 6 ++++ > arch/arm64/kernel/fpsimd.c | 57 ++++++++++++++++++++++-------- > arch/arm64/kernel/process.c | 2 ++ > arch/arm64/kernel/ptrace.c | 6 ++-- > arch/arm64/kernel/signal.c | 3 ++ > arch/arm64/kvm/fpsimd.c | 3 +- > 8 files changed, 61 insertions(+), 19 deletions(-) > > diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h > index 847fd119cdb8..5762419fdcc0 100644 > --- a/arch/arm64/include/asm/fpsimd.h > +++ b/arch/arm64/include/asm/fpsimd.h > @@ -61,7 +61,7 @@ extern void fpsimd_kvm_prepare(void); > extern void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *state, > void *sve_state, unsigned int sve_vl, > void *za_state, unsigned int sme_vl, > - u64 *svcr); > + u64 *svcr, enum fp_state *type); > > extern void fpsimd_flush_task_state(struct task_struct *target); > extern void fpsimd_save_and_flush_cpu_state(void); > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index de32152cea04..250e23f221c4 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -310,6 +310,7 @@ struct kvm_vcpu_arch { > void *sve_state; > unsigned int sve_max_vl; > u64 svcr; > + enum fp_state fp_type; > > /* Stage 2 paging state used by the hardware on next switch */ > struct kvm_s2_mmu *hw_mmu; > diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h > index 9e58749db21d..192986509a8e 100644 > --- a/arch/arm64/include/asm/processor.h > +++ b/arch/arm64/include/asm/processor.h > @@ -122,6 +122,11 @@ enum vec_type { > ARM64_VEC_MAX, > }; > > +enum fp_state { > + FP_STATE_FPSIMD, > + FP_STATE_SVE, > +}; > + > struct cpu_context { > unsigned long x19; > unsigned long x20; > @@ -152,6 +157,7 @@ struct thread_struct { > struct user_fpsimd_state fpsimd_state; > } uw; > > + enum fp_state fp_type; /* registers FPSIMD or SVE? */ > unsigned int fpsimd_cpu; > void *sve_state; /* SVE registers, if any */ > void *za_state; /* ZA register, if any */ > diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c > index d67e658f1e24..fdb2925becdf 100644 > --- a/arch/arm64/kernel/fpsimd.c > +++ b/arch/arm64/kernel/fpsimd.c > @@ -125,6 +125,7 @@ struct fpsimd_last_state_struct { > u64 *svcr; > unsigned int sve_vl; > unsigned int sme_vl; > + enum fp_state *type; For consistency: s/type/fp_type/ ? > }; > > static DEFINE_PER_CPU(struct fpsimd_last_state_struct, fpsimd_last_state); > @@ -330,15 +331,6 @@ void task_set_vl_onexec(struct task_struct *task, enum vec_type type, > * The task can execute SVE instructions while in userspace without > * trapping to the kernel. > * > - * When stored, Z0-Z31 (incorporating Vn in bits[127:0] or the > - * corresponding Zn), P0-P15 and FFR are encoded in > - * task->thread.sve_state, formatted appropriately for vector > - * length task->thread.sve_vl or, if SVCR.SM is set, > - * task->thread.sme_vl. > - * > - * task->thread.sve_state must point to a valid buffer at least > - * sve_state_size(task) bytes in size. > - * > * During any syscall, the kernel may optionally clear TIF_SVE and > * discard the vector state except for the FPSIMD subset. > * > @@ -348,7 +340,15 @@ void task_set_vl_onexec(struct task_struct *task, enum vec_type type, > * do_sve_acc() to be called, which does some preparation and then > * sets TIF_SVE. > * > - * When stored, FPSIMD registers V0-V31 are encoded in > + * During any syscall, the kernel may optionally clear TIF_SVE and > + * discard the vector state except for the FPSIMD subset. > + * > + * The data will be stored in one of two formats: > + * > + * * FPSIMD only - FP_STATE_FPSIMD: > + * > + * When the FPSIMD only state stored task->thread.fp_type is set to > + * FP_STATE_FPSIMD, the FPSIMD registers V0-V31 are encoded in > * task->thread.uw.fpsimd_state; bits [max : 128] for each of Z0-Z31 are > * logically zero but not stored anywhere; P0-P15 and FFR are not > * stored and have unspecified values from userspace's point of > @@ -358,6 +358,18 @@ void task_set_vl_onexec(struct task_struct *task, enum vec_type type, > * task->thread.sve_state does not need to be non-NULL, valid or any > * particular size: it must not be dereferenced. > * > + * * SVE state - FP_STATE_SVE: > + * > + * When the full SVE state is stored task->thread.fp_type is set to > + * FP_STATE_SVE and Z0-Z31 (incorporating Vn in bits[127:0] or the > + * corresponding Zn), P0-P15 and FFR are encoded in in > + * task->thread.sve_state, formatted appropriately for vector > + * length task->thread.sve_vl or, if SVCR.SM is set, > + * task->thread.sme_vl. Can you add a commend about the fate of the Vn data in fpsimd_state? My understanding is that they are stale data as soon as FP_STATE_SVE is set, but I'd really like this being clarified. > + * > + * task->thread.sve_state must point to a valid buffer at least > + * sve_state_size(task) bytes in size. > + * > * * FPSR and FPCR are always stored in task->thread.uw.fpsimd_state > * irrespective of whether TIF_SVE is clear or set, since these are > * not vector length dependent. > @@ -404,12 +416,15 @@ static void task_fpsimd_load(void) > } > } > > - if (restore_sve_regs) > + if (restore_sve_regs) { > + WARN_ON_ONCE(current->thread.fp_type != FP_STATE_SVE); > sve_load_state(sve_pffr(¤t->thread), > ¤t->thread.uw.fpsimd_state.fpsr, > restore_ffr); > - else > + } else { > + WARN_ON_ONCE(current->thread.fp_type != FP_STATE_FPSIMD); > fpsimd_load_state(¤t->thread.uw.fpsimd_state); > + } > } > > /* > @@ -475,8 +490,10 @@ static void fpsimd_save(void) > sve_save_state((char *)last->sve_state + > sve_ffr_offset(vl), > &last->st->fpsr, save_ffr); > + *last->type = FP_STATE_SVE; > } else { > fpsimd_save_state(last->st); > + *last->type = FP_STATE_FPSIMD; > } > } > > @@ -847,8 +864,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, > > fpsimd_flush_task_state(task); > if (test_and_clear_tsk_thread_flag(task, TIF_SVE) || > - thread_sm_enabled(&task->thread)) > + thread_sm_enabled(&task->thread)) { > sve_to_fpsimd(task); > + task->thread.fp_type = FP_STATE_FPSIMD; Can you move this assignment into the sve_to_fpsimd() helper? > + } > > if (system_supports_sme() && type == ARM64_VEC_SME) { > task->thread.svcr &= ~(SVCR_SM_MASK | > @@ -1367,6 +1386,7 @@ static void sve_init_regs(void) > fpsimd_bind_task_to_cpu(); > } else { > fpsimd_to_sve(current); > + current->thread.fp_type = FP_STATE_SVE; Same thing here. > } > } > > @@ -1606,6 +1626,8 @@ void fpsimd_flush_thread(void) > current->thread.svcr = 0; > } > > + current->thread.fp_type = FP_STATE_FPSIMD; > + > put_cpu_fpsimd_context(); > kfree(sve_state); > kfree(za_state); > @@ -1654,8 +1676,10 @@ void fpsimd_kvm_prepare(void) > */ > get_cpu_fpsimd_context(); > > - if (test_and_clear_thread_flag(TIF_SVE)) > + if (test_and_clear_thread_flag(TIF_SVE)) { > sve_to_fpsimd(current); > + current->thread.fp_type = FP_STATE_FPSIMD; > + } > > put_cpu_fpsimd_context(); > } > @@ -1677,6 +1701,7 @@ static void fpsimd_bind_task_to_cpu(void) > last->sve_vl = task_get_sve_vl(current); > last->sme_vl = task_get_sme_vl(current); > last->svcr = ¤t->thread.svcr; > + last->type = ¤t->thread.fp_type; > current->thread.fpsimd_cpu = smp_processor_id(); > > /* > @@ -1700,7 +1725,8 @@ static void fpsimd_bind_task_to_cpu(void) > > void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st, void *sve_state, > unsigned int sve_vl, void *za_state, > - unsigned int sme_vl, u64 *svcr) > + unsigned int sme_vl, u64 *svcr, > + enum fp_state *type) > { > struct fpsimd_last_state_struct *last = > this_cpu_ptr(&fpsimd_last_state); > @@ -1714,6 +1740,7 @@ void fpsimd_bind_state_to_cpu(struct user_fpsimd_state *st, void *sve_state, > last->za_state = za_state; > last->sve_vl = sve_vl; > last->sme_vl = sme_vl; > + last->type = type; > } > > /* > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > index 92bcc1768f0b..944d782d581b 100644 > --- a/arch/arm64/kernel/process.c > +++ b/arch/arm64/kernel/process.c > @@ -335,6 +335,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) > clear_tsk_thread_flag(dst, TIF_SME); > } > > + dst->thread.fp_type = FP_STATE_FPSIMD; > + > /* clear any pending asynchronous tag fault raised by the parent */ > clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); > > diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c > index 21da83187a60..e0233f322af3 100644 > --- a/arch/arm64/kernel/ptrace.c > +++ b/arch/arm64/kernel/ptrace.c > @@ -892,8 +892,7 @@ static int sve_set_common(struct task_struct *target, > ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, > SVE_PT_FPSIMD_OFFSET); > clear_tsk_thread_flag(target, TIF_SVE); > - if (type == ARM64_VEC_SME) > - fpsimd_force_sync_to_sve(target); I don't get this particular change. Can you please clarify? > + target->thread.fp_type = FP_STATE_FPSIMD; > goto out; > } > > @@ -916,6 +915,7 @@ static int sve_set_common(struct task_struct *target, > if (!target->thread.sve_state) { > ret = -ENOMEM; > clear_tsk_thread_flag(target, TIF_SVE); > + target->thread.fp_type = FP_STATE_FPSIMD; > goto out; > } > > @@ -954,6 +954,8 @@ static int sve_set_common(struct task_struct *target, > &target->thread.uw.fpsimd_state.fpsr, > start, end); > > + target->thread.fp_type = FP_STATE_SVE; > + > out: > fpsimd_flush_task_state(target); > return ret; > diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c > index b0980fbb6bc7..23dd4c3bcfed 100644 > --- a/arch/arm64/kernel/signal.c > +++ b/arch/arm64/kernel/signal.c > @@ -207,6 +207,7 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx) > __get_user_error(fpsimd.fpcr, &ctx->fpcr, err); > > clear_thread_flag(TIF_SVE); > + current->thread.fp_type = FP_STATE_FPSIMD; > > /* load the hardware registers from the fpsimd_state structure */ > if (!err) > @@ -289,6 +290,7 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) > if (sve.head.size <= sizeof(*user->sve)) { > clear_thread_flag(TIF_SVE); > current->thread.svcr &= ~SVCR_SM_MASK; > + current->thread.fp_type = FP_STATE_FPSIMD; > goto fpsimd_only; > } > > @@ -324,6 +326,7 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) > current->thread.svcr |= SVCR_SM_MASK; > else > set_thread_flag(TIF_SVE); > + current->thread.fp_type = FP_STATE_SVE; > > fpsimd_only: > /* copy the FP and status/control registers */ > diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c > index a433ee8da232..be3ddb214ab1 100644 > --- a/arch/arm64/kvm/fpsimd.c > +++ b/arch/arm64/kvm/fpsimd.c > @@ -139,7 +139,8 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu) > fpsimd_bind_state_to_cpu(&vcpu->arch.ctxt.fp_regs, > vcpu->arch.sve_state, > vcpu->arch.sve_max_vl, > - NULL, 0, &vcpu->arch.svcr); > + NULL, 0, &vcpu->arch.svcr, > + &vcpu->arch.fp_type); > > clear_thread_flag(TIF_FOREIGN_FPSTATE); > update_thread_flag(TIF_SVE, vcpu_has_sve(vcpu)); Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel