From: Atish Patra <atish.patra@wdc.com> To: linux-kernel@vger.kernel.org Cc: Atish Patra <atish.patra@wdc.com>, Albert Ou <aou@eecs.berkeley.edu>, linux-riscv@lists.infradead.org, Palmer Dabbelt <palmer@sifive.com>, Paul Walmsley <paul.walmsley@sifive.com>, Christoph Hellwig <hch@lst.de>, Anup Patel <Anup.Patel@wdc.com> Subject: [PATCH v4 2/3] RISC-V: Issue a local tlbflush if possible. Date: Thu, 22 Aug 2019 00:51:50 -0700 [thread overview] Message-ID: <20190822075151.24838-3-atish.patra@wdc.com> (raw) In-Reply-To: <20190822075151.24838-1-atish.patra@wdc.com> In RISC-V, tlb flush happens via SBI which is expensive. If the local cpu is the only cpu in cpumask, there is no need to invoke a SBI call. Just do a local flush and return. Signed-off-by: Atish Patra <atish.patra@wdc.com> --- arch/riscv/mm/tlbflush.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 1293b8017ee0..8172fbf46123 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -2,6 +2,7 @@ #include <linux/mm.h> #include <linux/smp.h> +#include <linux/sched.h> #include <asm/sbi.h> void flush_tlb_all(void) @@ -9,16 +10,30 @@ void flush_tlb_all(void) sbi_remote_sfence_vma(NULL, 0, -1); } +/* + * This function must not be called with cmask being null. + * Kernel may panic if cmask is NULL. + */ static void __sbi_tlb_flush_range(struct cpumask *cmask, unsigned long start, unsigned long size) { struct cpumask hmask; + unsigned int cpuid; if (cpumask_empty(cmask)) return; - riscv_cpuid_to_hartid_mask(cmask, &hmask); - sbi_remote_sfence_vma(hmask.bits, start, size); + cpuid = get_cpu(); + + if (cpumask_any_but(cmask, cpuid) >= nr_cpu_ids) { + /* local cpu is the only cpu present in cpumask */ + local_flush_tlb_all(); + } else { + riscv_cpuid_to_hartid_mask(cmask, &hmask); + sbi_remote_sfence_vma(cpumask_bits(&hmask), start, size); + } + + put_cpu(); } void flush_tlb_mm(struct mm_struct *mm) -- 2.21.0
WARNING: multiple messages have this Message-ID (diff)
From: Atish Patra <atish.patra@wdc.com> To: linux-kernel@vger.kernel.org Cc: Albert Ou <aou@eecs.berkeley.edu>, Palmer Dabbelt <palmer@sifive.com>, Atish Patra <atish.patra@wdc.com>, Anup Patel <Anup.Patel@wdc.com>, Paul Walmsley <paul.walmsley@sifive.com>, linux-riscv@lists.infradead.org, Christoph Hellwig <hch@lst.de> Subject: [PATCH v4 2/3] RISC-V: Issue a local tlbflush if possible. Date: Thu, 22 Aug 2019 00:51:50 -0700 [thread overview] Message-ID: <20190822075151.24838-3-atish.patra@wdc.com> (raw) In-Reply-To: <20190822075151.24838-1-atish.patra@wdc.com> In RISC-V, tlb flush happens via SBI which is expensive. If the local cpu is the only cpu in cpumask, there is no need to invoke a SBI call. Just do a local flush and return. Signed-off-by: Atish Patra <atish.patra@wdc.com> --- arch/riscv/mm/tlbflush.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 1293b8017ee0..8172fbf46123 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -2,6 +2,7 @@ #include <linux/mm.h> #include <linux/smp.h> +#include <linux/sched.h> #include <asm/sbi.h> void flush_tlb_all(void) @@ -9,16 +10,30 @@ void flush_tlb_all(void) sbi_remote_sfence_vma(NULL, 0, -1); } +/* + * This function must not be called with cmask being null. + * Kernel may panic if cmask is NULL. + */ static void __sbi_tlb_flush_range(struct cpumask *cmask, unsigned long start, unsigned long size) { struct cpumask hmask; + unsigned int cpuid; if (cpumask_empty(cmask)) return; - riscv_cpuid_to_hartid_mask(cmask, &hmask); - sbi_remote_sfence_vma(hmask.bits, start, size); + cpuid = get_cpu(); + + if (cpumask_any_but(cmask, cpuid) >= nr_cpu_ids) { + /* local cpu is the only cpu present in cpumask */ + local_flush_tlb_all(); + } else { + riscv_cpuid_to_hartid_mask(cmask, &hmask); + sbi_remote_sfence_vma(cpumask_bits(&hmask), start, size); + } + + put_cpu(); } void flush_tlb_mm(struct mm_struct *mm) -- 2.21.0 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2019-08-22 7:59 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-22 7:51 [PATCH v4 0/3] Optimize tlbflush path Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 7:51 ` [PATCH v4 1/3] RISC-V: Do not invoke SBI call if cpumask is empty Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 7:51 ` Atish Patra [this message] 2019-08-22 7:51 ` [PATCH v4 2/3] RISC-V: Issue a local tlbflush if possible Atish Patra 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 8:10 ` Christoph Hellwig 2019-08-22 7:51 ` [PATCH v4 3/3] RISC-V: Issue a tlb page flush " Atish Patra 2019-08-22 7:51 ` Atish Patra 2019-08-22 8:11 ` Christoph Hellwig 2019-08-22 8:11 ` Christoph Hellwig 2019-08-22 18:31 ` Atish Patra 2019-08-22 18:31 ` Atish Patra 2019-08-31 2:50 ` [PATCH v4 0/3] Optimize tlbflush path Paul Walmsley 2019-08-31 2:50 ` Paul Walmsley 2019-09-28 4:23 ` Atish Patra 2019-09-28 4:23 ` Atish Patra 2019-10-25 3:09 ` Paul Walmsley 2019-10-25 3:09 ` Paul Walmsley 2019-10-31 15:13 ` Atish Patra 2019-10-31 15:13 ` Atish Patra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190822075151.24838-3-atish.patra@wdc.com \ --to=atish.patra@wdc.com \ --cc=Anup.Patel@wdc.com \ --cc=aou@eecs.berkeley.edu \ --cc=hch@lst.de \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@sifive.com \ --cc=paul.walmsley@sifive.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.