From: Samuel Holland <samuel.holland@sifive.com> To: Palmer Dabbelt <palmer@dabbelt.com>, linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti <alexghiti@rivosinc.com>, Jisheng Zhang <jszhang@kernel.org>, Yunhui Cui <cuiyunhui@bytedance.com>, Samuel Holland <samuel.holland@sifive.com> Subject: [PATCH v6 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 26 Mar 2024 21:49:48 -0700 [thread overview] Message-ID: <20240327045035.368512-8-samuel.holland@sifive.com> (raw) In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland <samuel.holland@sifive.com> --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 1f2dbfb8a8bf..35ce26899960 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0901aa47b58f..ad7bdcfcc219 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include <asm/sbi.h> #include <asm/mmu_context.h> -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. -- 2.43.1
WARNING: multiple messages have this Message-ID (diff)
From: Samuel Holland <samuel.holland@sifive.com> To: Palmer Dabbelt <palmer@dabbelt.com>, linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti <alexghiti@rivosinc.com>, Jisheng Zhang <jszhang@kernel.org>, Yunhui Cui <cuiyunhui@bytedance.com>, Samuel Holland <samuel.holland@sifive.com> Subject: [PATCH v6 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 26 Mar 2024 21:49:48 -0700 [thread overview] Message-ID: <20240327045035.368512-8-samuel.holland@sifive.com> (raw) In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland <samuel.holland@sifive.com> --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 1f2dbfb8a8bf..35ce26899960 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0901aa47b58f..ad7bdcfcc219 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include <asm/sbi.h> #include <asm/mmu_context.h> -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. -- 2.43.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2024-03-27 4:50 UTC|newest] Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-03-27 4:49 [PATCH v6 00/13] riscv: ASID-related and UP-related TLB flush enhancements Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 01/13] riscv: Flush the instruction cache during SMP bringup Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-04-24 20:50 ` Alexandre Ghiti 2024-04-24 20:50 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 02/13] riscv: Factor out page table TLB synchronization Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-04-04 7:48 ` Alexandre Ghiti 2024-04-04 7:48 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-04-04 7:56 ` Alexandre Ghiti 2024-04-04 7:56 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 05/13] riscv: Only send remote fences when some other CPU is online Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 6:16 ` [External] " yunhui cui 2024-03-27 6:16 ` yunhui cui 2024-03-27 20:14 ` Samuel Holland 2024-03-27 20:14 ` Samuel Holland 2024-03-28 2:21 ` yunhui cui 2024-03-28 2:21 ` yunhui cui 2024-04-04 8:04 ` Alexandre Ghiti 2024-04-04 8:04 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 06/13] riscv: mm: Combine the SMP and UP TLB flush code Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 6:23 ` [External] " yunhui cui 2024-03-27 6:23 ` yunhui cui 2024-03-27 4:49 ` Samuel Holland [this message] 2024-03-27 4:49 ` [PATCH v6 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Samuel Holland 2024-03-27 4:49 ` [PATCH v6 08/13] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 6:27 ` [External] " yunhui cui 2024-03-27 6:27 ` yunhui cui 2024-03-27 4:49 ` [PATCH v6 09/13] riscv: mm: Introduce cntx2asid/cntx2version helper macros Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 10/13] riscv: mm: Use a fixed layout for the MM context ID Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 11/13] riscv: mm: Make asid_bits a local variable Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 12/13] riscv: mm: Preserve global TLB entries when switching contexts Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 13/13] riscv: mm: Always use an ASID to flush mm contexts Samuel Holland 2024-03-27 4:49 ` Samuel Holland
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20240327045035.368512-8-samuel.holland@sifive.com \ --to=samuel.holland@sifive.com \ --cc=alexghiti@rivosinc.com \ --cc=cuiyunhui@bytedance.com \ --cc=jszhang@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@dabbelt.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.