From: Samuel Holland <samuel.holland@sifive.com>
To: Palmer Dabbelt <palmer@dabbelt.com>, linux-riscv@lists.infradead.org
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Alexandre Ghiti <alexghiti@rivosinc.com>,
Jisheng Zhang <jszhang@kernel.org>,
Yunhui Cui <cuiyunhui@bytedance.com>,
Samuel Holland <samuel.holland@sifive.com>
Subject: [PATCH v5 00/13] riscv: ASID-related and UP-related TLB flush enhancements
Date: Thu, 29 Feb 2024 15:21:41 -0800 [thread overview]
Message-ID: <20240229232211.161961-1-samuel.holland@sifive.com> (raw)
While reviewing Alexandre Ghiti's "riscv: tlb flush improvements"
series[1], I noticed that most TLB flush functions end up as a call to
local_flush_tlb_all() when SMP is disabled. This series resolves that,
and also optimizes the scenario where SMP is enabled but only one CPU is
present or online. Along the way, I realized that we should be using
single-ASID flushes wherever possible, so I implemented that as well.
Here are some numbers from D1 (with SMP disabled) which show the
performance impact:
v6.8-rc6 + riscv/fixes + riscv/for-next:
System Benchmarks Partial Index BASELINE RESULT INDEX
Execl Throughput 43.0 223.5 52.0
File Copy 1024 bufsize 2000 maxblocks 3960.0 62563.4 158.0
File Copy 256 bufsize 500 maxblocks 1655.0 17869.2 108.0
File Copy 4096 bufsize 8000 maxblocks 5800.0 164915.1 284.3
Pipe Throughput 12440.0 161368.5 129.7
Pipe-based Context Switching 4000.0 22247.1 55.6
Process Creation 126.0 546.9 43.4
Shell Scripts (1 concurrent) 42.4 599.6 141.4
Shell Scripts (16 concurrent) --- 39.3 ---
Shell Scripts (8 concurrent) 6.0 79.1 131.9
System Call Overhead 15000.0 246019.0 164.0
========
System Benchmarks Index Score (Partial Only) 109.2
v6.8-rc6 + riscv/fixes + riscv/for-next + this patch series:
System Benchmarks Partial Index BASELINE RESULT INDEX
Execl Throughput 43.0 223.1 51.9
File Copy 1024 bufsize 2000 maxblocks 3960.0 71982.9 181.8
File Copy 256 bufsize 500 maxblocks 1655.0 18436.9 111.4
File Copy 4096 bufsize 8000 maxblocks 5800.0 184955.2 318.9
Pipe Throughput 12440.0 162622.9 130.7
Pipe-based Context Switching 4000.0 22082.5 55.2
Process Creation 126.0 546.4 43.4
Shell Scripts (1 concurrent) 42.4 598.1 141.1
Shell Scripts (16 concurrent) --- 38.8 ---
Shell Scripts (8 concurrent) 6.0 78.6 131.0
System Call Overhead 15000.0 258529.3 172.4
========
System Benchmarks Index Score (Partial Only) 112.8
[1]: https://lore.kernel.org/linux-riscv/20231030133027.19542-1-alexghiti@rivosinc.com/
Changes in v5:
- Rebase on v6.8-rc1 + riscv/for-next (for the fast GUP implementation)
- Add patch for minor refactoring in asm/pgalloc.h
- Also switch to riscv_use_sbi_for_rfence() in asm/pgalloc.h
- Leave use_asid_allocator declared in asm/mmu_context.h
Changes in v4:
- Fix a possible race between flush_icache_*() and SMP bringup
- Refactor riscv_use_ipi_for_rfence() to make later changes cleaner
- Optimize kernel TLB flushes with only one CPU online
- Optimize global cache/TLB flushes with only one CPU online
- Merge the two copies of __flush_tlb_range() and rely on the compiler
to optimize out the broadcast path (both clang and gcc do this)
- Merge the two copies of flush_tlb_all() and rely on constant folding
- Only set tlb_flush_all_threshold when CONFIG_MMU=y.
Changes in v3:
- Fixed a performance regression caused by executing sfence.vma in a
loop on implementations affected by SiFive CIP-1200
- Rebased on v6.7-rc1
Changes in v2:
- Move the SMP/UP merge earlier in the series to avoid build issues
- Make a copy of __flush_tlb_range() instead of adding ifdefs inside
- local_flush_tlb_all() is the only function used on !MMU (smpboot.c)
Samuel Holland (13):
riscv: Flush the instruction cache during SMP bringup
riscv: Factor out page table TLB synchronization
riscv: Use IPIs for remote cache/TLB flushes by default
riscv: mm: Broadcast kernel TLB flushes only when needed
riscv: Only send remote fences when some other CPU is online
riscv: mm: Combine the SMP and UP TLB flush code
riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma
riscv: Avoid TLB flush loops when affected by SiFive CIP-1200
riscv: mm: Introduce cntx2asid/cntx2version helper macros
riscv: mm: Use a fixed layout for the MM context ID
riscv: mm: Make asid_bits a local variable
riscv: mm: Preserve global TLB entries when switching contexts
riscv: mm: Always use an ASID to flush mm contexts
arch/riscv/Kconfig | 2 +-
arch/riscv/errata/sifive/errata.c | 5 ++
arch/riscv/include/asm/errata_list.h | 12 ++++-
arch/riscv/include/asm/mmu.h | 3 ++
arch/riscv/include/asm/pgalloc.h | 32 ++++++------
arch/riscv/include/asm/sbi.h | 4 ++
arch/riscv/include/asm/smp.h | 15 +-----
arch/riscv/include/asm/tlbflush.h | 51 ++++++++-----------
arch/riscv/kernel/sbi-ipi.c | 11 +++-
arch/riscv/kernel/smp.c | 11 +---
arch/riscv/kernel/smpboot.c | 7 +--
arch/riscv/mm/Makefile | 5 +-
arch/riscv/mm/cacheflush.c | 7 +--
arch/riscv/mm/context.c | 23 ++++-----
arch/riscv/mm/tlbflush.c | 75 ++++++++--------------------
drivers/clocksource/timer-clint.c | 2 +-
16 files changed, 114 insertions(+), 151 deletions(-)
--
2.43.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
next reply other threads:[~2024-02-29 23:22 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-29 23:21 Samuel Holland [this message]
2024-02-29 23:21 ` [PATCH v5 01/13] riscv: Flush the instruction cache during SMP bringup Samuel Holland
2024-02-29 23:21 ` [PATCH v5 02/13] riscv: Factor out page table TLB synchronization Samuel Holland
2024-02-29 23:21 ` [PATCH v5 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Samuel Holland
2024-03-11 3:06 ` Stefan O'Rear
2024-03-11 4:04 ` Anup Patel
2024-03-11 4:12 ` Anup Patel
2024-03-11 4:42 ` Samuel Holland
2024-02-29 23:21 ` [PATCH v5 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Samuel Holland
2024-02-29 23:21 ` [PATCH v5 05/13] riscv: Only send remote fences when some other CPU is online Samuel Holland
2024-02-29 23:21 ` [PATCH v5 06/13] riscv: mm: Combine the SMP and UP TLB flush code Samuel Holland
2024-03-01 2:12 ` [External] " yunhui cui
2024-03-01 2:34 ` Samuel Holland
2024-02-29 23:21 ` [PATCH v5 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Samuel Holland
2024-02-29 23:21 ` [PATCH v5 08/13] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Samuel Holland
2024-03-01 2:48 ` [External] " yunhui cui
2024-03-12 0:35 ` Samuel Holland
2024-03-12 1:51 ` yunhui cui
2024-02-29 23:21 ` [PATCH v5 09/13] riscv: mm: Introduce cntx2asid/cntx2version helper macros Samuel Holland
2024-02-29 23:21 ` [PATCH v5 10/13] riscv: mm: Use a fixed layout for the MM context ID Samuel Holland
2024-02-29 23:21 ` [PATCH v5 11/13] riscv: mm: Make asid_bits a local variable Samuel Holland
2024-02-29 23:21 ` [PATCH v5 12/13] riscv: mm: Preserve global TLB entries when switching contexts Samuel Holland
2024-02-29 23:21 ` [PATCH v5 13/13] riscv: mm: Always use an ASID to flush mm contexts Samuel Holland
2024-03-01 9:31 ` [PATCH v5 00/13] riscv: ASID-related and UP-related TLB flush enhancements Conor Dooley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240229232211.161961-1-samuel.holland@sifive.com \
--to=samuel.holland@sifive.com \
--cc=alexghiti@rivosinc.com \
--cc=cuiyunhui@bytedance.com \
--cc=jszhang@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=palmer@dabbelt.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).