From: Mel Gorman <mgorman@suse.de> To: Linux-MM <linux-mm@kvack.org> Cc: Rik van Riel <riel@redhat.com>, Johannes Weiner <hannes@cmpxchg.org>, Dave Hansen <dave.hansen@intel.com>, Andi Kleen <andi@firstfloor.org>, LKML <linux-kernel@vger.kernel.org>, Mel Gorman <mgorman@suse.de> Subject: [RFC PATCH 0/4] TLB flush multiple pages with a single IPI Date: Wed, 15 Apr 2015 11:42:52 +0100 [thread overview] Message-ID: <1429094576-5877-1-git-send-email-mgorman@suse.de> (raw) When unmapping pages it is necessary to flush the TLB. If that page was accessed by another CPU then an IPI is used to flush the remote CPU. That is a lot of IPIs if kswapd is scanning and unmapping >100K pages per second. There already is a window between when a page is unmapped and when it is TLB flushed. This series simply increases the window so multiple pages can be flushed using a single IPI. Patch 1 simply made the rest of the series easier to write as ftrace could identify all the senders of TLB flush IPIS. Patch 2 collects a list of PFNs and sends one IPI to flush them all Patch 3 uses more memory so further defer when the IPI gets sent Patch 4 uses the same infrastructure as patch 2 to batch IPIs sent during page migration. The performance impact is documented in the changelogs but in the optimistic case on a 4-socket machine the full series reduces interrupts from 900K interrupts/second to 60K interrupts/second. Last minute note: It occured to me just before sending that a TLB flush cannot be batched if the PTE was dirty at unmap time as the page lock is released before the TLB flush occurs. That allows IO to be started in parallel while writes can still take place through a cached entry. I decided not to delay the series as it's RFC and I want to see if there is interest in this. Note however that there is a difficult-to-hit potential corruption race here. arch/x86/Kconfig | 1 + arch/x86/include/asm/tlbflush.h | 2 + arch/x86/mm/tlb.c | 1 + include/linux/init_task.h | 8 ++++ include/linux/mm_types.h | 1 + include/linux/rmap.h | 3 ++ include/linux/sched.h | 20 ++++++++++ include/trace/events/tlb.h | 3 +- init/Kconfig | 5 +++ kernel/fork.c | 5 +++ kernel/sched/core.c | 3 ++ mm/internal.h | 16 ++++++++ mm/migrate.c | 8 +++- mm/rmap.c | 85 ++++++++++++++++++++++++++++++++++++++++- mm/vmscan.c | 29 +++++++++++++- 15 files changed, 186 insertions(+), 4 deletions(-) -- 2.1.2
WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de> To: Linux-MM <linux-mm@kvack.org> Cc: Rik van Riel <riel@redhat.com>, Johannes Weiner <hannes@cmpxchg.org>, Dave Hansen <dave.hansen@intel.com>, Andi Kleen <andi@firstfloor.org>, LKML <linux-kernel@vger.kernel.org>, Mel Gorman <mgorman@suse.de> Subject: [RFC PATCH 0/4] TLB flush multiple pages with a single IPI Date: Wed, 15 Apr 2015 11:42:52 +0100 [thread overview] Message-ID: <1429094576-5877-1-git-send-email-mgorman@suse.de> (raw) When unmapping pages it is necessary to flush the TLB. If that page was accessed by another CPU then an IPI is used to flush the remote CPU. That is a lot of IPIs if kswapd is scanning and unmapping >100K pages per second. There already is a window between when a page is unmapped and when it is TLB flushed. This series simply increases the window so multiple pages can be flushed using a single IPI. Patch 1 simply made the rest of the series easier to write as ftrace could identify all the senders of TLB flush IPIS. Patch 2 collects a list of PFNs and sends one IPI to flush them all Patch 3 uses more memory so further defer when the IPI gets sent Patch 4 uses the same infrastructure as patch 2 to batch IPIs sent during page migration. The performance impact is documented in the changelogs but in the optimistic case on a 4-socket machine the full series reduces interrupts from 900K interrupts/second to 60K interrupts/second. Last minute note: It occured to me just before sending that a TLB flush cannot be batched if the PTE was dirty at unmap time as the page lock is released before the TLB flush occurs. That allows IO to be started in parallel while writes can still take place through a cached entry. I decided not to delay the series as it's RFC and I want to see if there is interest in this. Note however that there is a difficult-to-hit potential corruption race here. arch/x86/Kconfig | 1 + arch/x86/include/asm/tlbflush.h | 2 + arch/x86/mm/tlb.c | 1 + include/linux/init_task.h | 8 ++++ include/linux/mm_types.h | 1 + include/linux/rmap.h | 3 ++ include/linux/sched.h | 20 ++++++++++ include/trace/events/tlb.h | 3 +- init/Kconfig | 5 +++ kernel/fork.c | 5 +++ kernel/sched/core.c | 3 ++ mm/internal.h | 16 ++++++++ mm/migrate.c | 8 +++- mm/rmap.c | 85 ++++++++++++++++++++++++++++++++++++++++- mm/vmscan.c | 29 +++++++++++++- 15 files changed, 186 insertions(+), 4 deletions(-) -- 2.1.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next reply other threads:[~2015-04-15 10:43 UTC|newest] Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-04-15 10:42 Mel Gorman [this message] 2015-04-15 10:42 ` [RFC PATCH 0/4] TLB flush multiple pages with a single IPI Mel Gorman 2015-04-15 10:42 ` [PATCH 1/4] x86, mm: Trace when an IPI is about to be sent Mel Gorman 2015-04-15 10:42 ` Mel Gorman 2015-04-15 10:42 ` [PATCH 2/4] mm: Send a single IPI to TLB flush multiple pages when unmapping Mel Gorman 2015-04-15 10:42 ` Mel Gorman 2015-04-15 21:03 ` Rik van Riel 2015-04-15 21:03 ` Rik van Riel 2015-04-15 21:16 ` Hugh Dickins 2015-04-15 21:16 ` Hugh Dickins 2015-04-15 21:28 ` Mel Gorman 2015-04-15 21:28 ` Mel Gorman 2015-04-15 21:32 ` Dave Hansen 2015-04-15 21:32 ` Dave Hansen 2015-04-16 6:38 ` Minchan Kim 2015-04-16 6:38 ` Minchan Kim 2015-04-16 8:07 ` Mel Gorman 2015-04-16 8:07 ` Mel Gorman 2015-04-16 8:29 ` Minchan Kim 2015-04-16 8:29 ` Minchan Kim 2015-04-16 9:19 ` Mel Gorman 2015-04-16 9:19 ` Mel Gorman 2015-04-16 23:30 ` Minchan Kim 2015-04-16 23:30 ` Minchan Kim 2015-04-15 22:20 ` Andi Kleen 2015-04-15 22:20 ` Andi Kleen 2015-04-15 22:53 ` Mel Gorman 2015-04-15 22:53 ` Mel Gorman 2015-04-15 10:42 ` [PATCH 3/4] mm: Gather more PFNs before sending a TLB to flush unmapped pages Mel Gorman 2015-04-15 10:42 ` Mel Gorman 2015-04-15 11:42 ` Peter Zijlstra 2015-04-15 11:42 ` Peter Zijlstra 2015-04-15 12:15 ` Mel Gorman 2015-04-15 12:15 ` Mel Gorman 2015-04-15 12:24 ` Peter Zijlstra 2015-04-15 12:24 ` Peter Zijlstra 2015-04-15 12:56 ` Mel Gorman 2015-04-15 12:56 ` Mel Gorman 2015-04-15 10:42 ` [PATCH 4/4] mm: migrate: Batch TLB flushing when unmapping pages for migration Mel Gorman 2015-04-15 10:42 ` Mel Gorman 2015-04-15 21:06 ` Hugh Dickins 2015-04-15 21:06 ` Hugh Dickins 2015-04-15 21:44 ` Mel Gorman 2015-04-15 21:44 ` Mel Gorman 2015-04-15 23:50 ` Hugh Dickins 2015-04-15 23:50 ` Hugh Dickins
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1429094576-5877-1-git-send-email-mgorman@suse.de \ --to=mgorman@suse.de \ --cc=andi@firstfloor.org \ --cc=dave.hansen@intel.com \ --cc=hannes@cmpxchg.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=riel@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.