From: Jisheng Zhang <jszhang@kernel.org> To: Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, Will Deacon <will@kernel.org>, "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, Nick Piggin <npiggin@gmail.com>, Peter Zijlstra <peterz@infradead.org> Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 0/4] riscv: support fast gup Date: Wed, 20 Dec 2023 01:50:42 +0800 [thread overview] Message-ID: <20231219175046.2496-1-jszhang@kernel.org> (raw) This series adds fast gup support to riscv. The First patch fixes a bug in __p*d_free_tlb(). Per the riscv privileged spec, if non-leaf PTEs I.E pmd, pud or p4d is modified, a sfence.vma is a must. The 2nd patch is a preparation patch. The last two patches do the real work: In order to implement fast gup we need to ensure that the page table walker is protected from page table pages being freed from under it. riscv situation is more complicated than other architectures: some riscv platforms may use IPI to perform TLB shootdown, for example, those platforms which support AIA, usually the riscv_ipi_for_rfence is true on these platforms; some riscv platforms may rely on the SBI to perform TLB shootdown, usually the riscv_ipi_for_rfence is false on these platforms. To keep software pagetable walkers safe in this case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h for more details. This patch enables MMU_GATHER_RCU_TABLE_FREE, then use *tlb_remove_page_ptdesc() for those platforms which use IPI to perform TLB shootdown; *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB shootdown; Both case mean that disabling interrupts will block the free and protect the fast gup page walker. So after the 3rd patch, everything is well prepared, let's select HAVE_FAST_GUP if MMU. Jisheng Zhang (4): riscv: tlb: fix __p*d_free_tlb() riscv: tlb: convert __p*d_free_tlb() to inline functions riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU riscv: enable HAVE_FAST_GUP if MMU arch/riscv/Kconfig | 2 ++ arch/riscv/include/asm/pgalloc.h | 53 +++++++++++++++++++++++++++----- arch/riscv/include/asm/pgtable.h | 6 ++++ arch/riscv/include/asm/tlb.h | 18 +++++++++++ 4 files changed, 71 insertions(+), 8 deletions(-) -- 2.40.0
WARNING: multiple messages have this Message-ID (diff)
From: Jisheng Zhang <jszhang@kernel.org> To: Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, Will Deacon <will@kernel.org>, "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, Nick Piggin <npiggin@gmail.com>, Peter Zijlstra <peterz@infradead.org> Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 0/4] riscv: support fast gup Date: Wed, 20 Dec 2023 01:50:42 +0800 [thread overview] Message-ID: <20231219175046.2496-1-jszhang@kernel.org> (raw) This series adds fast gup support to riscv. The First patch fixes a bug in __p*d_free_tlb(). Per the riscv privileged spec, if non-leaf PTEs I.E pmd, pud or p4d is modified, a sfence.vma is a must. The 2nd patch is a preparation patch. The last two patches do the real work: In order to implement fast gup we need to ensure that the page table walker is protected from page table pages being freed from under it. riscv situation is more complicated than other architectures: some riscv platforms may use IPI to perform TLB shootdown, for example, those platforms which support AIA, usually the riscv_ipi_for_rfence is true on these platforms; some riscv platforms may rely on the SBI to perform TLB shootdown, usually the riscv_ipi_for_rfence is false on these platforms. To keep software pagetable walkers safe in this case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h for more details. This patch enables MMU_GATHER_RCU_TABLE_FREE, then use *tlb_remove_page_ptdesc() for those platforms which use IPI to perform TLB shootdown; *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB shootdown; Both case mean that disabling interrupts will block the free and protect the fast gup page walker. So after the 3rd patch, everything is well prepared, let's select HAVE_FAST_GUP if MMU. Jisheng Zhang (4): riscv: tlb: fix __p*d_free_tlb() riscv: tlb: convert __p*d_free_tlb() to inline functions riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU riscv: enable HAVE_FAST_GUP if MMU arch/riscv/Kconfig | 2 ++ arch/riscv/include/asm/pgalloc.h | 53 +++++++++++++++++++++++++++----- arch/riscv/include/asm/pgtable.h | 6 ++++ arch/riscv/include/asm/tlb.h | 18 +++++++++++ 4 files changed, 71 insertions(+), 8 deletions(-) -- 2.40.0 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next reply other threads:[~2023-12-19 18:03 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-12-19 17:50 Jisheng Zhang [this message] 2023-12-19 17:50 ` [PATCH 0/4] riscv: support fast gup Jisheng Zhang 2023-12-19 17:50 ` [PATCH 1/4] riscv: tlb: fix __p*d_free_tlb() Jisheng Zhang 2023-12-19 17:50 ` Jisheng Zhang 2023-12-31 6:21 ` Alexandre Ghiti 2023-12-31 6:21 ` Alexandre Ghiti 2024-02-07 7:28 ` Alexandre Ghiti 2024-02-07 7:28 ` Alexandre Ghiti 2024-01-04 10:55 ` Alexandre Ghiti 2024-01-04 10:55 ` Alexandre Ghiti 2024-01-10 14:52 ` Palmer Dabbelt 2024-01-10 14:52 ` Palmer Dabbelt 2023-12-19 17:50 ` [PATCH 2/4] riscv: tlb: convert __p*d_free_tlb() to inline functions Jisheng Zhang 2023-12-19 17:50 ` Jisheng Zhang 2023-12-20 2:59 ` [External] " yunhui cui 2023-12-20 2:59 ` yunhui cui 2023-12-20 12:57 ` Jisheng Zhang 2023-12-20 12:57 ` Jisheng Zhang 2023-12-31 6:24 ` Alexandre Ghiti 2023-12-31 6:24 ` Alexandre Ghiti 2023-12-19 17:50 ` [PATCH 3/4] riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU Jisheng Zhang 2023-12-19 17:50 ` Jisheng Zhang 2023-12-31 6:32 ` Alexandre Ghiti 2023-12-31 6:32 ` Alexandre Ghiti 2024-01-02 3:23 ` Jisheng Zhang 2024-01-02 3:23 ` Jisheng Zhang 2024-01-04 10:45 ` Alexandre Ghiti 2024-01-04 10:45 ` Alexandre Ghiti 2023-12-19 17:50 ` [PATCH 4/4] riscv: enable HAVE_FAST_GUP if MMU Jisheng Zhang 2023-12-19 17:50 ` Jisheng Zhang 2023-12-31 6:37 ` Alexandre Ghiti 2023-12-31 6:37 ` Alexandre Ghiti 2024-01-02 3:25 ` Jisheng Zhang 2024-01-02 3:25 ` Jisheng Zhang 2024-01-04 10:46 ` Alexandre Ghiti 2024-01-04 10:46 ` Alexandre Ghiti 2024-01-25 21:30 ` [PATCH 0/4] riscv: support fast gup patchwork-bot+linux-riscv 2024-01-25 21:30 ` patchwork-bot+linux-riscv
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20231219175046.2496-1-jszhang@kernel.org \ --to=jszhang@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=aneesh.kumar@linux.ibm.com \ --cc=aou@eecs.berkeley.edu \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-riscv@lists.infradead.org \ --cc=npiggin@gmail.com \ --cc=palmer@dabbelt.com \ --cc=paul.walmsley@sifive.com \ --cc=peterz@infradead.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.