From: Samuel Holland <samuel.holland@sifive.com> To: Palmer Dabbelt <palmer@dabbelt.com>, linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti <alexghiti@rivosinc.com>, Jisheng Zhang <jszhang@kernel.org>, Yunhui Cui <cuiyunhui@bytedance.com>, Samuel Holland <samuel.holland@sifive.com> Subject: [PATCH v6 06/13] riscv: mm: Combine the SMP and UP TLB flush code Date: Tue, 26 Mar 2024 21:49:47 -0700 [thread overview] Message-ID: <20240327045035.368512-7-samuel.holland@sifive.com> (raw) In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Samuel Holland <samuel.holland@sifive.com> --- (no changes since v4) Changes in v4: - Merge the two copies of __flush_tlb_range() and rely on the compiler to optimize out the broadcast path (both clang and gcc do this) - Merge the two copies of flush_tlb_all() and rely on constant folding Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/Kconfig | 2 +- arch/riscv/include/asm/tlbflush.h | 31 +++---------------------------- arch/riscv/mm/Makefile | 5 +---- 3 files changed, 5 insertions(+), 33 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56..442532819a44 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -60,7 +60,7 @@ config RISCV select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_USES_CFI_TRAPS if CFI_CLANG - select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if SMP && MMU + select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if MMU select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4112cc8d1d69..4f86424b1ba5 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,12 +27,7 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, @@ -54,28 +49,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); - -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#define local_flush_tlb_kernel_range(start, end) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 2c869f8026a8..cbe4d775ef56 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,14 +13,11 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o -- 2.43.1
WARNING: multiple messages have this Message-ID (diff)
From: Samuel Holland <samuel.holland@sifive.com> To: Palmer Dabbelt <palmer@dabbelt.com>, linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti <alexghiti@rivosinc.com>, Jisheng Zhang <jszhang@kernel.org>, Yunhui Cui <cuiyunhui@bytedance.com>, Samuel Holland <samuel.holland@sifive.com> Subject: [PATCH v6 06/13] riscv: mm: Combine the SMP and UP TLB flush code Date: Tue, 26 Mar 2024 21:49:47 -0700 [thread overview] Message-ID: <20240327045035.368512-7-samuel.holland@sifive.com> (raw) In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Samuel Holland <samuel.holland@sifive.com> --- (no changes since v4) Changes in v4: - Merge the two copies of __flush_tlb_range() and rely on the compiler to optimize out the broadcast path (both clang and gcc do this) - Merge the two copies of flush_tlb_all() and rely on constant folding Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/Kconfig | 2 +- arch/riscv/include/asm/tlbflush.h | 31 +++---------------------------- arch/riscv/mm/Makefile | 5 +---- 3 files changed, 5 insertions(+), 33 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56..442532819a44 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -60,7 +60,7 @@ config RISCV select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_USES_CFI_TRAPS if CFI_CLANG - select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if SMP && MMU + select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if MMU select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4112cc8d1d69..4f86424b1ba5 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,12 +27,7 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, @@ -54,28 +49,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); - -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#define local_flush_tlb_kernel_range(start, end) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 2c869f8026a8..cbe4d775ef56 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,14 +13,11 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o -- 2.43.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2024-03-27 4:50 UTC|newest] Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-03-27 4:49 [PATCH v6 00/13] riscv: ASID-related and UP-related TLB flush enhancements Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 01/13] riscv: Flush the instruction cache during SMP bringup Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-04-24 20:50 ` Alexandre Ghiti 2024-04-24 20:50 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 02/13] riscv: Factor out page table TLB synchronization Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-04-04 7:48 ` Alexandre Ghiti 2024-04-04 7:48 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-04-04 7:56 ` Alexandre Ghiti 2024-04-04 7:56 ` Alexandre Ghiti 2024-03-27 4:49 ` [PATCH v6 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 05/13] riscv: Only send remote fences when some other CPU is online Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 6:16 ` [External] " yunhui cui 2024-03-27 6:16 ` yunhui cui 2024-03-27 20:14 ` Samuel Holland 2024-03-27 20:14 ` Samuel Holland 2024-03-28 2:21 ` yunhui cui 2024-03-28 2:21 ` yunhui cui 2024-04-04 8:04 ` Alexandre Ghiti 2024-04-04 8:04 ` Alexandre Ghiti 2024-03-27 4:49 ` Samuel Holland [this message] 2024-03-27 4:49 ` [PATCH v6 06/13] riscv: mm: Combine the SMP and UP TLB flush code Samuel Holland 2024-03-27 6:23 ` [External] " yunhui cui 2024-03-27 6:23 ` yunhui cui 2024-03-27 4:49 ` [PATCH v6 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 08/13] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 6:27 ` [External] " yunhui cui 2024-03-27 6:27 ` yunhui cui 2024-03-27 4:49 ` [PATCH v6 09/13] riscv: mm: Introduce cntx2asid/cntx2version helper macros Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 10/13] riscv: mm: Use a fixed layout for the MM context ID Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 11/13] riscv: mm: Make asid_bits a local variable Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 12/13] riscv: mm: Preserve global TLB entries when switching contexts Samuel Holland 2024-03-27 4:49 ` Samuel Holland 2024-03-27 4:49 ` [PATCH v6 13/13] riscv: mm: Always use an ASID to flush mm contexts Samuel Holland 2024-03-27 4:49 ` Samuel Holland
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20240327045035.368512-7-samuel.holland@sifive.com \ --to=samuel.holland@sifive.com \ --cc=alexghiti@rivosinc.com \ --cc=cuiyunhui@bytedance.com \ --cc=jszhang@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@dabbelt.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.