From: Andy Chiu <andy.chiu@sifive.com> To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Vincent Chen <vincent.chen@sifive.com>, Andy Chiu <andy.chiu@sifive.com>, Paul Walmsley <paul.walmsley@sifive.com>, Albert Ou <aou@eecs.berkeley.edu>, Guo Ren <guoren@kernel.org>, Richard Henderson <richard.henderson@linaro.org> Subject: [PATCH -next v13 08/19] riscv: Introduce struct/helpers to save/restore per-task Vector state Date: Wed, 25 Jan 2023 14:20:45 +0000 [thread overview] Message-ID: <20230125142056.18356-9-andy.chiu@sifive.com> (raw) In-Reply-To: <20230125142056.18356-1-andy.chiu@sifive.com> From: Greentime Hu <greentime.hu@sifive.com> Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved separately. This is not yet wired into final thread_struct - will be done when __switch_to actually starts doing this in later patches. Given the variable (and potentially large) size of regfile, they are saved in dynamically allocated memory, pointed to by datap pointer in __riscv_v_state. Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> [vineetg: merged bits from 2 different patches] Signed-off-by: Andy Chiu <andy.chiu@sifive.com> [andy.chiu: use inline asm to save/restore context, remove asm vaiant] --- arch/riscv/include/asm/vector.h | 84 ++++++++++++++++++++++++++++ arch/riscv/include/uapi/asm/ptrace.h | 17 ++++++ 2 files changed, 101 insertions(+) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 16cb4a1c1230..842a859609b5 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -12,6 +12,9 @@ #include <asm/hwcap.h> #include <asm/csr.h> +#include <asm/asm.h> + +#define CSR_STR(x) __ASM_STR(x) extern unsigned long riscv_vsize; @@ -20,6 +23,26 @@ static __always_inline bool has_vector(void) return static_branch_likely(&riscv_isa_ext_keys[RISCV_ISA_EXT_KEY_VECTOR]); } +static inline void __vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_CLEAN; +} + +static inline void vstate_off(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void vstate_on(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_INITIAL; +} + +static inline bool vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + static __always_inline void rvv_enable(void) { csr_set(CSR_SSTATUS, SR_VS); @@ -30,10 +53,71 @@ static __always_inline void rvv_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline void __vstate_csr_save(struct __riscv_v_state *dest) +{ + asm volatile ( + "csrr %0, " CSR_STR(CSR_VSTART) "\n\t" + "csrr %1, " CSR_STR(CSR_VTYPE) "\n\t" + "csrr %2, " CSR_STR(CSR_VL) "\n\t" + "csrr %3, " CSR_STR(CSR_VCSR) "\n\t" + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), + "=r" (dest->vcsr) : :); +} + +static __always_inline void __vstate_csr_restore(struct __riscv_v_state *src) +{ + asm volatile ( + "vsetvl x0, %2, %1\n\t" + "csrw " CSR_STR(CSR_VSTART) ", %0\n\t" + "csrw " CSR_STR(CSR_VCSR) ", %3\n\t" + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), + "r" (src->vcsr) :); +} + +static inline void __vstate_save(struct __riscv_v_state *save_to, void *datap) +{ + rvv_enable(); + __vstate_csr_save(save_to); + asm volatile ( + "vsetvli t4, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v8, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v16, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v24, (%0)\n\t" + : : "r" (datap) : "t4", "memory"); + rvv_disable(); +} + +static inline void __vstate_restore(struct __riscv_v_state *restore_from, + void *datap) +{ + rvv_enable(); + asm volatile ( + "vsetvli t4, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v8, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v16, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v24, (%0)\n\t" + : : "r" (datap) : "t4"); + __vstate_csr_restore(restore_from); + rvv_disable(); +} + #else /* ! CONFIG_RISCV_ISA_V */ +struct pt_regs; + static __always_inline bool has_vector(void) { return false; } +static inline bool vstate_query(struct pt_regs *regs) { return false; } #define riscv_vsize (0) +#define vstate_off(regs) do {} while (0) +#define vstate_on(regs) do {} while (0) #endif /* CONFIG_RISCV_ISA_V */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..6ee1ca2edfa7 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */ -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Andy Chiu <andy.chiu@sifive.com> To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Vincent Chen <vincent.chen@sifive.com>, Andy Chiu <andy.chiu@sifive.com>, Paul Walmsley <paul.walmsley@sifive.com>, Albert Ou <aou@eecs.berkeley.edu>, Guo Ren <guoren@kernel.org>, Richard Henderson <richard.henderson@linaro.org> Subject: [PATCH -next v13 08/19] riscv: Introduce struct/helpers to save/restore per-task Vector state Date: Wed, 25 Jan 2023 14:20:45 +0000 [thread overview] Message-ID: <20230125142056.18356-9-andy.chiu@sifive.com> (raw) In-Reply-To: <20230125142056.18356-1-andy.chiu@sifive.com> From: Greentime Hu <greentime.hu@sifive.com> Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved separately. This is not yet wired into final thread_struct - will be done when __switch_to actually starts doing this in later patches. Given the variable (and potentially large) size of regfile, they are saved in dynamically allocated memory, pointed to by datap pointer in __riscv_v_state. Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> [vineetg: merged bits from 2 different patches] Signed-off-by: Andy Chiu <andy.chiu@sifive.com> [andy.chiu: use inline asm to save/restore context, remove asm vaiant] --- arch/riscv/include/asm/vector.h | 84 ++++++++++++++++++++++++++++ arch/riscv/include/uapi/asm/ptrace.h | 17 ++++++ 2 files changed, 101 insertions(+) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 16cb4a1c1230..842a859609b5 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -12,6 +12,9 @@ #include <asm/hwcap.h> #include <asm/csr.h> +#include <asm/asm.h> + +#define CSR_STR(x) __ASM_STR(x) extern unsigned long riscv_vsize; @@ -20,6 +23,26 @@ static __always_inline bool has_vector(void) return static_branch_likely(&riscv_isa_ext_keys[RISCV_ISA_EXT_KEY_VECTOR]); } +static inline void __vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_CLEAN; +} + +static inline void vstate_off(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void vstate_on(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_INITIAL; +} + +static inline bool vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + static __always_inline void rvv_enable(void) { csr_set(CSR_SSTATUS, SR_VS); @@ -30,10 +53,71 @@ static __always_inline void rvv_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline void __vstate_csr_save(struct __riscv_v_state *dest) +{ + asm volatile ( + "csrr %0, " CSR_STR(CSR_VSTART) "\n\t" + "csrr %1, " CSR_STR(CSR_VTYPE) "\n\t" + "csrr %2, " CSR_STR(CSR_VL) "\n\t" + "csrr %3, " CSR_STR(CSR_VCSR) "\n\t" + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), + "=r" (dest->vcsr) : :); +} + +static __always_inline void __vstate_csr_restore(struct __riscv_v_state *src) +{ + asm volatile ( + "vsetvl x0, %2, %1\n\t" + "csrw " CSR_STR(CSR_VSTART) ", %0\n\t" + "csrw " CSR_STR(CSR_VCSR) ", %3\n\t" + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), + "r" (src->vcsr) :); +} + +static inline void __vstate_save(struct __riscv_v_state *save_to, void *datap) +{ + rvv_enable(); + __vstate_csr_save(save_to); + asm volatile ( + "vsetvli t4, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v8, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v16, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v24, (%0)\n\t" + : : "r" (datap) : "t4", "memory"); + rvv_disable(); +} + +static inline void __vstate_restore(struct __riscv_v_state *restore_from, + void *datap) +{ + rvv_enable(); + asm volatile ( + "vsetvli t4, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v8, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v16, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v24, (%0)\n\t" + : : "r" (datap) : "t4"); + __vstate_csr_restore(restore_from); + rvv_disable(); +} + #else /* ! CONFIG_RISCV_ISA_V */ +struct pt_regs; + static __always_inline bool has_vector(void) { return false; } +static inline bool vstate_query(struct pt_regs *regs) { return false; } #define riscv_vsize (0) +#define vstate_off(regs) do {} while (0) +#define vstate_on(regs) do {} while (0) #endif /* CONFIG_RISCV_ISA_V */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..6ee1ca2edfa7 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */ -- 2.17.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2023-01-25 14:21 UTC|newest] Thread overview: 128+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-01-25 14:20 [PATCH -next v13 00/19] riscv: Add vector ISA support Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 01/19] riscv: Rename __switch_to_aux -> fpu Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 21:15 ` Conor Dooley 2023-01-25 21:15 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 02/19] riscv: Extending cpufeature.c to detect V-extension Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 21:33 ` Conor Dooley 2023-01-25 21:33 ` Conor Dooley 2023-01-28 7:09 ` Guo Ren 2023-01-28 7:09 ` Guo Ren 2023-01-28 10:28 ` Conor Dooley 2023-01-28 10:28 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 03/19] riscv: Add new csr defines related to vector extension Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 22:16 ` Conor Dooley 2023-01-25 22:16 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 04/19] riscv: Clear vector regfile on bootup Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 21:54 ` Conor Dooley 2023-01-25 21:54 ` Conor Dooley 2023-01-25 21:57 ` Vineet Gupta 2023-01-25 21:57 ` Vineet Gupta 2023-01-25 22:18 ` Conor Dooley 2023-01-25 22:18 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 05/19] riscv: Disable Vector Instructions for kernel itself Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 21:51 ` Conor Dooley 2023-01-25 21:51 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 06/19] riscv: Introduce Vector enable/disable helpers Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-26 21:06 ` Conor Dooley 2023-01-26 21:06 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 07/19] riscv: Introduce riscv_vsize to record size of Vector context Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-26 21:24 ` Conor Dooley 2023-01-26 21:24 ` Conor Dooley 2023-01-25 14:20 ` Andy Chiu [this message] 2023-01-25 14:20 ` [PATCH -next v13 08/19] riscv: Introduce struct/helpers to save/restore per-task Vector state Andy Chiu 2023-01-26 21:32 ` Conor Dooley 2023-01-26 21:32 ` Conor Dooley 2023-01-25 14:20 ` [PATCH -next v13 09/19] riscv: Add task switch support for vector Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-26 21:44 ` Conor Dooley 2023-01-26 21:44 ` Conor Dooley 2023-01-31 2:55 ` Vineet Gupta 2023-01-31 2:55 ` Vineet Gupta 2023-01-25 14:20 ` [PATCH -next v13 10/19] riscv: Allocate user's vector context in the first-use trap Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-26 23:11 ` Conor Dooley 2023-01-26 23:11 ` Conor Dooley 2023-02-06 12:00 ` Andy Chiu 2023-02-06 12:00 ` Andy Chiu 2023-02-06 13:40 ` Conor Dooley 2023-02-06 13:40 ` Conor Dooley 2023-02-10 12:00 ` Andy Chiu 2023-02-10 12:00 ` Andy Chiu 2023-02-07 14:36 ` Björn Töpel 2023-02-07 14:36 ` Björn Töpel 2023-02-13 22:54 ` Vineet Gupta 2023-02-13 22:54 ` Vineet Gupta 2023-02-14 6:43 ` Björn Töpel 2023-02-14 6:43 ` Björn Töpel 2023-02-14 15:36 ` Andy Chiu 2023-02-14 15:36 ` Andy Chiu 2023-02-14 16:50 ` Björn Töpel 2023-02-14 16:50 ` Björn Töpel 2023-02-14 17:24 ` Vineet Gupta 2023-02-14 17:24 ` Vineet Gupta 2023-02-15 7:14 ` Björn Töpel 2023-02-15 7:14 ` Björn Töpel 2023-02-15 14:39 ` Andy Chiu 2023-02-15 14:39 ` Andy Chiu 2023-02-07 21:18 ` Vineet Gupta 2023-02-07 21:18 ` Vineet Gupta 2023-02-08 9:20 ` Björn Töpel 2023-02-08 9:20 ` Björn Töpel 2023-01-25 14:20 ` [PATCH -next v13 11/19] riscv: Add ptrace vector support Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 12/19] riscv: signal: check fp-reserved words unconditionally Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 13/19] riscv: signal: Add sigcontext save/restore for vector Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 14/19] riscv: signal: Report signal frame size to userspace via auxv Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-26 23:19 ` Conor Dooley 2023-01-26 23:19 ` Conor Dooley 2023-01-31 12:34 ` Andy Chiu 2023-01-31 12:34 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 15/19] riscv: Fix a kernel panic issue if $s2 is set to a specific value before entering Linux Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-27 20:31 ` Conor Dooley 2023-01-27 20:31 ` Conor Dooley 2023-01-31 12:34 ` Andy Chiu 2023-01-31 12:34 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 16/19] riscv: Add V extension to KVM ISA Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-27 20:43 ` Conor Dooley 2023-01-27 20:43 ` Conor Dooley 2023-01-30 9:58 ` Andy Chiu 2023-01-30 9:58 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 17/19] riscv: KVM: Add vector lazy save/restore support Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 18/19] riscv: kvm: redirect illegal instruction traps to guests Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-27 11:28 ` Anup Patel 2023-01-27 11:28 ` Anup Patel 2023-01-30 8:18 ` Andy Chiu 2023-01-30 8:18 ` Andy Chiu 2023-01-25 14:20 ` [PATCH -next v13 19/19] riscv: Enable Vector code to be built Andy Chiu 2023-01-25 14:20 ` Andy Chiu 2023-01-25 21:04 ` Conor Dooley 2023-01-25 21:04 ` Conor Dooley 2023-01-25 21:38 ` Jessica Clarke 2023-01-25 21:38 ` Jessica Clarke 2023-01-25 22:24 ` Conor Dooley 2023-01-25 22:24 ` Conor Dooley 2023-01-30 6:38 ` Andy Chiu 2023-01-30 6:38 ` Andy Chiu 2023-01-30 18:38 ` Vineet Gupta 2023-01-30 18:38 ` Vineet Gupta 2023-01-30 7:46 ` Andy Chiu 2023-01-30 7:46 ` Andy Chiu 2023-01-30 8:13 ` Conor Dooley 2023-01-30 8:13 ` Conor Dooley 2023-02-08 18:19 ` Conor Dooley 2023-02-08 18:19 ` Conor Dooley
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230125142056.18356-9-andy.chiu@sifive.com \ --to=andy.chiu@sifive.com \ --cc=anup@brainfault.org \ --cc=aou@eecs.berkeley.edu \ --cc=atishp@atishpatra.org \ --cc=greentime.hu@sifive.com \ --cc=guoren@kernel.org \ --cc=guoren@linux.alibaba.com \ --cc=kvm-riscv@lists.infradead.org \ --cc=kvm@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@dabbelt.com \ --cc=paul.walmsley@sifive.com \ --cc=richard.henderson@linaro.org \ --cc=vincent.chen@sifive.com \ --cc=vineetg@rivosinc.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.