From: Aaron Lu <aaron.lu@intel.com> To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Dave Hansen <dave.hansen@intel.com>, Tim Chen <tim.c.chen@intel.com>, Andrew Morton <akpm@linux-foundation.org>, Ying Huang <ying.huang@intel.com>, Aaron Lu <aaron.lu@intel.com> Subject: [PATCH v2 1/5] mm: add tlb_flush_mmu_free_batches Date: Wed, 15 Mar 2017 17:00:00 +0800 [thread overview] Message-ID: <1489568404-7817-2-git-send-email-aaron.lu@intel.com> (raw) In-Reply-To: <1489568404-7817-1-git-send-email-aaron.lu@intel.com> There are two places doing page free related to struct mmu_gather_batch: 1 in tlb_flush_mmu_free, where pages gathered in mmu_gather_batch list are freed; 2 in tlb_flush_mmu_finish, where pages for the mmu_gather_batch structure(let's call it the batch page) are freed. There will be yet another place in the parallel free worker thread introduced in the following patch to free both the pages pointed to by the mmu_gather_batch list and the batch pages themselves. To avoid code duplication, add a new function named tlb_flush_mmu_free_batches for this purpose. Another reason to add this function is that after the following patch, cond_resched will need to be added at places where more than 10K pages can be freed, i.e. in tlb_flush_mmu_free and the worker function. Instead of adding cond_resched at multiple places, using a single function to reduce code duplication. There should be no functionality change. Signed-off-by: Aaron Lu <aaron.lu@intel.com> --- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 14fc0b40f0bb..cdb2a53f251f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -250,14 +250,25 @@ static void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) __tlb_reset_range(tlb); } -static void tlb_flush_mmu_free(struct mmu_gather *tlb) +static void tlb_flush_mmu_free_batches(struct mmu_gather_batch *batch_start, + bool free_batch_page) { - struct mmu_gather_batch *batch; + struct mmu_gather_batch *batch, *next; - for (batch = &tlb->local; batch && batch->nr; batch = batch->next) { - free_pages_and_swap_cache(batch->pages, batch->nr); - batch->nr = 0; + for (batch = batch_start; batch; batch = next) { + next = batch->next; + if (batch->nr) { + free_pages_and_swap_cache(batch->pages, batch->nr); + batch->nr = 0; + } + if (free_batch_page) + free_pages((unsigned long)batch, 0); } +} + +static void tlb_flush_mmu_free(struct mmu_gather *tlb) +{ + tlb_flush_mmu_free_batches(&tlb->local, false); tlb->active = &tlb->local; } @@ -273,17 +284,12 @@ void tlb_flush_mmu(struct mmu_gather *tlb) */ void tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end) { - struct mmu_gather_batch *batch, *next; - tlb_flush_mmu(tlb); /* keep the page table cache within bounds */ check_pgt_cache(); - for (batch = tlb->local.next; batch; batch = next) { - next = batch->next; - free_pages((unsigned long)batch, 0); - } + tlb_flush_mmu_free_batches(tlb->local.next, true); tlb->local.next = NULL; } -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: Aaron Lu <aaron.lu@intel.com> To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Dave Hansen <dave.hansen@intel.com>, Tim Chen <tim.c.chen@intel.com>, Andrew Morton <akpm@linux-foundation.org>, Ying Huang <ying.huang@intel.com>, Aaron Lu <aaron.lu@intel.com> Subject: [PATCH v2 1/5] mm: add tlb_flush_mmu_free_batches Date: Wed, 15 Mar 2017 17:00:00 +0800 [thread overview] Message-ID: <1489568404-7817-2-git-send-email-aaron.lu@intel.com> (raw) In-Reply-To: <1489568404-7817-1-git-send-email-aaron.lu@intel.com> There are two places doing page free related to struct mmu_gather_batch: 1 in tlb_flush_mmu_free, where pages gathered in mmu_gather_batch list are freed; 2 in tlb_flush_mmu_finish, where pages for the mmu_gather_batch structure(let's call it the batch page) are freed. There will be yet another place in the parallel free worker thread introduced in the following patch to free both the pages pointed to by the mmu_gather_batch list and the batch pages themselves. To avoid code duplication, add a new function named tlb_flush_mmu_free_batches for this purpose. Another reason to add this function is that after the following patch, cond_resched will need to be added at places where more than 10K pages can be freed, i.e. in tlb_flush_mmu_free and the worker function. Instead of adding cond_resched at multiple places, using a single function to reduce code duplication. There should be no functionality change. Signed-off-by: Aaron Lu <aaron.lu@intel.com> --- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 14fc0b40f0bb..cdb2a53f251f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -250,14 +250,25 @@ static void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) __tlb_reset_range(tlb); } -static void tlb_flush_mmu_free(struct mmu_gather *tlb) +static void tlb_flush_mmu_free_batches(struct mmu_gather_batch *batch_start, + bool free_batch_page) { - struct mmu_gather_batch *batch; + struct mmu_gather_batch *batch, *next; - for (batch = &tlb->local; batch && batch->nr; batch = batch->next) { - free_pages_and_swap_cache(batch->pages, batch->nr); - batch->nr = 0; + for (batch = batch_start; batch; batch = next) { + next = batch->next; + if (batch->nr) { + free_pages_and_swap_cache(batch->pages, batch->nr); + batch->nr = 0; + } + if (free_batch_page) + free_pages((unsigned long)batch, 0); } +} + +static void tlb_flush_mmu_free(struct mmu_gather *tlb) +{ + tlb_flush_mmu_free_batches(&tlb->local, false); tlb->active = &tlb->local; } @@ -273,17 +284,12 @@ void tlb_flush_mmu(struct mmu_gather *tlb) */ void tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end) { - struct mmu_gather_batch *batch, *next; - tlb_flush_mmu(tlb); /* keep the page table cache within bounds */ check_pgt_cache(); - for (batch = tlb->local.next; batch; batch = next) { - next = batch->next; - free_pages((unsigned long)batch, 0); - } + tlb_flush_mmu_free_batches(tlb->local.next, true); tlb->local.next = NULL; } -- 2.7.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-03-15 9:00 UTC|newest] Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-03-15 8:59 [PATCH v2 0/5] mm: support parallel free of memory Aaron Lu 2017-03-15 8:59 ` Aaron Lu 2017-03-15 9:00 ` Aaron Lu [this message] 2017-03-15 9:00 ` [PATCH v2 1/5] mm: add tlb_flush_mmu_free_batches Aaron Lu 2017-03-15 9:00 ` [PATCH v2 2/5] mm: parallel free pages Aaron Lu 2017-03-15 9:00 ` Aaron Lu 2017-03-15 9:42 ` Hillf Danton 2017-03-15 9:42 ` Hillf Danton 2017-03-15 11:54 ` Aaron Lu 2017-03-15 11:54 ` Aaron Lu 2017-03-15 9:00 ` [PATCH v2 3/5] mm: use a dedicated workqueue for the free workers Aaron Lu 2017-03-15 9:00 ` Aaron Lu 2017-03-22 6:33 ` Minchan Kim 2017-03-22 6:33 ` Minchan Kim 2017-03-22 8:41 ` Aaron Lu 2017-03-22 8:41 ` Aaron Lu 2017-03-22 8:55 ` Minchan Kim 2017-03-22 8:55 ` Minchan Kim 2017-03-22 13:43 ` Aaron Lu 2017-03-22 13:43 ` Aaron Lu 2017-03-23 5:53 ` Minchan Kim 2017-03-23 5:53 ` Minchan Kim 2017-03-23 15:38 ` Dave Hansen 2017-03-23 15:38 ` Dave Hansen 2017-03-24 12:37 ` Aaron Lu 2017-03-24 12:37 ` Aaron Lu 2017-03-15 9:00 ` [PATCH v2 4/5] mm: add force_free_pages in zap_pte_range Aaron Lu 2017-03-15 9:00 ` Aaron Lu 2017-03-15 9:00 ` [PATCH v2 5/5] mm: add debugfs interface for parallel free tuning Aaron Lu 2017-03-15 9:00 ` Aaron Lu 2017-03-15 14:18 ` [PATCH v2 0/5] mm: support parallel free of memory Michal Hocko 2017-03-15 14:18 ` Michal Hocko 2017-03-15 15:44 ` Aaron Lu 2017-03-15 15:44 ` Aaron Lu 2017-03-15 16:28 ` Michal Hocko 2017-03-15 16:28 ` Michal Hocko 2017-03-15 21:38 ` Tim Chen 2017-03-15 21:38 ` Tim Chen 2017-03-16 9:07 ` Michal Hocko 2017-03-16 9:07 ` Michal Hocko 2017-03-16 18:36 ` Tim Chen 2017-03-16 18:36 ` Tim Chen 2017-03-17 7:47 ` Michal Hocko 2017-03-17 7:47 ` Michal Hocko 2017-03-17 8:07 ` Minchan Kim 2017-03-17 8:07 ` Minchan Kim 2017-03-17 12:33 ` Aaron Lu 2017-03-17 12:33 ` Aaron Lu 2017-03-17 12:59 ` Michal Hocko 2017-03-17 12:59 ` Michal Hocko 2017-03-17 13:16 ` Peter Zijlstra 2017-03-17 13:16 ` Peter Zijlstra 2017-03-17 12:53 ` Peter Zijlstra 2017-03-17 12:53 ` Peter Zijlstra 2017-03-17 13:05 ` Michal Hocko 2017-03-17 13:05 ` Michal Hocko 2017-03-21 14:54 ` Dave Hansen 2017-03-21 14:54 ` Dave Hansen 2017-03-22 8:02 ` Aaron Lu 2017-03-22 8:02 ` Aaron Lu 2017-03-24 7:04 ` Aaron Lu 2017-03-24 7:04 ` Aaron Lu 2017-03-21 15:18 ` Tim Chen 2017-03-21 15:18 ` Tim Chen 2017-03-16 6:54 ` Aaron Lu 2017-03-16 6:54 ` Aaron Lu 2017-03-16 7:34 ` Aaron Lu 2017-03-16 7:34 ` Aaron Lu 2017-03-16 13:51 ` Aaron Lu 2017-03-16 13:51 ` Aaron Lu 2017-03-16 14:14 ` Aaron Lu 2017-03-16 14:14 ` Aaron Lu 2017-03-15 14:56 ` Vlastimil Babka 2017-03-15 14:56 ` Vlastimil Babka 2017-03-15 15:50 ` Aaron Lu 2017-03-15 15:50 ` Aaron Lu 2017-03-17 3:10 ` Aaron Lu 2017-03-17 3:10 ` Aaron Lu 2017-03-16 19:38 ` Alex Thorlton 2017-03-16 19:38 ` Alex Thorlton 2017-03-17 2:21 ` Aaron Lu 2017-03-17 2:21 ` Aaron Lu 2017-03-20 19:15 ` Alex Thorlton 2017-03-20 19:15 ` Alex Thorlton
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1489568404-7817-2-git-send-email-aaron.lu@intel.com \ --to=aaron.lu@intel.com \ --cc=akpm@linux-foundation.org \ --cc=dave.hansen@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=tim.c.chen@intel.com \ --cc=ying.huang@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.