From: Nicholas Piggin <npiggin@gmail.com> To: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org, Nadav Amit <nadav.amit@gmail.com>, Mel Gorman <mgorman@techsingularity.net>, Linus Torvalds <torvalds@linux-foundation.org>, Nicholas Piggin <npiggin@gmail.com>, Minchan Kim <minchan@kernel.org>, "Aneesh Kumar K . V" <aneesh.kumar@linux.vnet.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 3/3] powerpc/64s/radix: optimise TLB flush with precise TLB ranges in mmu_gather Date: Tue, 12 Jun 2018 17:16:21 +1000 [thread overview] Message-ID: <20180612071621.26775-4-npiggin@gmail.com> (raw) In-Reply-To: <20180612071621.26775-1-npiggin@gmail.com> Use the page_start and page_end fields of mmu_gather to implement more precise TLB flushing. (start, end) covers the entire TLB and page table range that has been invalidated, for architectures that do not have explicit page walk cache management. page_start and page_end are just for ranges that may have TLB entries. A tlb_flush may have no pages in this range, but still requires PWC to be flushed. That is handled properly. This brings the number of tlbiel instructions required by a kernel compile from 33M to 25M, most avoided from exec->shift_arg_pages. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arch/powerpc/mm/tlb-radix.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c index 67a6e86d3e7e..06452ad701cf 100644 --- a/arch/powerpc/mm/tlb-radix.c +++ b/arch/powerpc/mm/tlb-radix.c @@ -853,8 +853,11 @@ void radix__tlb_flush(struct mmu_gather *tlb) else radix__flush_all_mm(mm); } else { - unsigned long start = tlb->start; - unsigned long end = tlb->end; + unsigned long start = tlb->page_start; + unsigned long end = tlb->page_end; + + if (end < start) + end = start; if (!tlb->need_flush_all) radix__flush_tlb_range_psize(mm, start, end, psize); -- 2.17.0
WARNING: multiple messages have this Message-ID (diff)
From: Nicholas Piggin <npiggin@gmail.com> To: linux-mm@kvack.org Cc: Nicholas Piggin <npiggin@gmail.com>, linuxppc-dev@lists.ozlabs.org, linux-arch@vger.kernel.org, "Aneesh Kumar K . V" <aneesh.kumar@linux.vnet.ibm.com>, Minchan Kim <minchan@kernel.org>, Mel Gorman <mgorman@techsingularity.net>, Nadav Amit <nadav.amit@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, Linus Torvalds <torvalds@linux-foundation.org> Subject: [RFC PATCH 3/3] powerpc/64s/radix: optimise TLB flush with precise TLB ranges in mmu_gather Date: Tue, 12 Jun 2018 17:16:21 +1000 [thread overview] Message-ID: <20180612071621.26775-4-npiggin@gmail.com> (raw) Message-ID: <20180612071621.bjaHBi2ajsPQ_6ggyeV17XbjnPnac8Z2HQd0O-TBE08@z> (raw) In-Reply-To: <20180612071621.26775-1-npiggin@gmail.com> Use the page_start and page_end fields of mmu_gather to implement more precise TLB flushing. (start, end) covers the entire TLB and page table range that has been invalidated, for architectures that do not have explicit page walk cache management. page_start and page_end are just for ranges that may have TLB entries. A tlb_flush may have no pages in this range, but still requires PWC to be flushed. That is handled properly. This brings the number of tlbiel instructions required by a kernel compile from 33M to 25M, most avoided from exec->shift_arg_pages. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arch/powerpc/mm/tlb-radix.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c index 67a6e86d3e7e..06452ad701cf 100644 --- a/arch/powerpc/mm/tlb-radix.c +++ b/arch/powerpc/mm/tlb-radix.c @@ -853,8 +853,11 @@ void radix__tlb_flush(struct mmu_gather *tlb) else radix__flush_all_mm(mm); } else { - unsigned long start = tlb->start; - unsigned long end = tlb->end; + unsigned long start = tlb->page_start; + unsigned long end = tlb->page_end; + + if (end < start) + end = start; if (!tlb->need_flush_all) radix__flush_tlb_range_psize(mm, start, end, psize); -- 2.17.0
next prev parent reply other threads:[~2018-06-12 7:16 UTC|newest] Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-06-12 7:16 [RFC PATCH 0/3] couple of TLB flush optimisations Nicholas Piggin 2018-06-12 7:16 ` Nicholas Piggin 2018-06-12 7:16 ` [RFC PATCH 1/3] Revert "mm: always flush VMA ranges affected by zap_page_range" Nicholas Piggin 2018-06-12 7:16 ` Nicholas Piggin 2018-06-12 13:53 ` Aneesh Kumar K.V 2018-06-12 13:53 ` Aneesh Kumar K.V 2018-06-12 18:52 ` Nadav Amit 2018-06-12 18:52 ` Nadav Amit 2018-06-12 18:52 ` Nadav Amit 2018-06-12 7:16 ` [RFC PATCH 2/3] mm: mmu_gather track of invalidated TLB ranges explicitly for more precise flushing Nicholas Piggin 2018-06-12 7:16 ` Nicholas Piggin 2018-06-12 18:14 ` Linus Torvalds 2018-06-12 18:14 ` Linus Torvalds 2018-06-12 7:16 ` Nicholas Piggin [this message] 2018-06-12 7:16 ` [RFC PATCH 3/3] powerpc/64s/radix: optimise TLB flush with precise TLB ranges in mmu_gather Nicholas Piggin 2018-06-12 18:18 ` Linus Torvalds 2018-06-12 18:18 ` Linus Torvalds 2018-06-12 22:31 ` Nicholas Piggin 2018-06-12 22:31 ` Nicholas Piggin 2018-06-12 22:31 ` Nicholas Piggin 2018-06-12 22:42 ` Linus Torvalds 2018-06-12 22:42 ` Linus Torvalds 2018-06-12 22:42 ` Linus Torvalds 2018-06-12 23:09 ` Nicholas Piggin 2018-06-12 23:09 ` Nicholas Piggin 2018-06-12 23:09 ` Nicholas Piggin 2018-06-12 23:26 ` Linus Torvalds 2018-06-12 23:26 ` Linus Torvalds 2018-06-12 23:39 ` Linus Torvalds 2018-06-12 23:39 ` Linus Torvalds 2018-06-13 0:12 ` Nicholas Piggin 2018-06-13 0:12 ` Nicholas Piggin 2018-06-13 1:10 ` Linus Torvalds 2018-06-13 1:10 ` Linus Torvalds 2018-06-14 2:49 ` Nicholas Piggin 2018-06-14 2:49 ` Nicholas Piggin 2018-06-14 6:15 ` Linus Torvalds 2018-06-14 6:15 ` Linus Torvalds 2018-06-14 6:51 ` Nicholas Piggin 2018-06-14 6:51 ` Nicholas Piggin 2018-06-12 23:53 ` Nicholas Piggin 2018-06-12 23:53 ` Nicholas Piggin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180612071621.26775-4-npiggin@gmail.com \ --to=npiggin@gmail.com \ --cc=akpm@linux-foundation.org \ --cc=aneesh.kumar@linux.vnet.ibm.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mgorman@techsingularity.net \ --cc=minchan@kernel.org \ --cc=nadav.amit@gmail.com \ --cc=torvalds@linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.