From: Vlastimil Babka <vbabka@suse.cz> To: linux-mm@kvack.org, Johannes Weiner <hannes@cmpxchg.org> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>, David Rientjes <rientjes@google.com>, Mel Gorman <mgorman@techsingularity.net>, linux-kernel@vger.kernel.org, kernel-team@fb.com, Vlastimil Babka <vbabka@suse.cz> Subject: [PATCH v2 07/10] mm, compaction: restrict async compaction to pageblocks of same migratetype Date: Fri, 10 Feb 2017 18:23:40 +0100 [thread overview] Message-ID: <20170210172343.30283-8-vbabka@suse.cz> (raw) In-Reply-To: <20170210172343.30283-1-vbabka@suse.cz> The migrate scanner in async compaction is currently limited to MIGRATE_MOVABLE pageblocks. This is a heuristic intended to reduce latency, based on the assumption that non-MOVABLE pageblocks are unlikely to contain movable pages. However, with the exception of THP's, most high-order allocations are not movable. Should the async compaction succeed, this increases the chance that the non-MOVABLE allocations will fallback to a MOVABLE pageblock, making the long-term fragmentation worse. This patch attempts to help the situation by changing async direct compaction so that the migrate scanner only scans the pageblocks of the requested migratetype. If it's a non-MOVABLE type and there are such pageblocks that do contain movable pages, chances are that the allocation can succeed within one of such pageblocks, removing the need for a fallback. If that fails, the subsequent sync attempt will ignore this restriction. Signed-off-by: Vlastimil Babka <vbabka@suse.cz> --- mm/compaction.c | 11 +++++++++-- mm/page_alloc.c | 20 +++++++++++++------- 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index b7094700712b..84ef44c3b1c9 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -994,10 +994,17 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn, static bool suitable_migration_source(struct compact_control *cc, struct page *page) { - if (cc->mode != MIGRATE_ASYNC) + int block_mt; + + if ((cc->mode != MIGRATE_ASYNC) || !cc->direct_compaction) return true; - return is_migrate_movable(get_pageblock_migratetype(page)); + block_mt = get_pageblock_migratetype(page); + + if (cc->migratetype == MIGRATE_MOVABLE) + return is_migrate_movable(block_mt); + else + return block_mt == cc->migratetype; } /* Returns true if the page is within a block suitable for migration to */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a7d33818610f..6d9ba640a12d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3523,6 +3523,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, struct alloc_context *ac) { bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; + const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER; struct page *page = NULL; unsigned int alloc_flags; unsigned long did_some_progress; @@ -3572,12 +3573,17 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, /* * For costly allocations, try direct compaction first, as it's likely - * that we have enough base pages and don't need to reclaim. Don't try - * that for allocations that are allowed to ignore watermarks, as the - * ALLOC_NO_WATERMARKS attempt didn't yet happen. + * that we have enough base pages and don't need to reclaim. For non- + * movable high-order allocations, do that as well, as compaction will + * try prevent permanent fragmentation by migrating from blocks of the + * same migratetype. + * Don't try this for allocations that are allowed to ignore + * watermarks, as the ALLOC_NO_WATERMARKS attempt didn't yet happen. */ - if (can_direct_reclaim && order > PAGE_ALLOC_COSTLY_ORDER && - !gfp_pfmemalloc_allowed(gfp_mask)) { + if (can_direct_reclaim && + (costly_order || + (order > 0 && ac->migratetype != MIGRATE_MOVABLE)) + && !gfp_pfmemalloc_allowed(gfp_mask)) { page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags, ac, INIT_COMPACT_PRIORITY, @@ -3589,7 +3595,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, * Checks for costly allocations with __GFP_NORETRY, which * includes THP page fault allocations */ - if (gfp_mask & __GFP_NORETRY) { + if (costly_order && (gfp_mask & __GFP_NORETRY)) { /* * If compaction is deferred for high-order allocations, * it is because sync compaction recently failed. If @@ -3684,7 +3690,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, * Do not retry costly high order allocations unless they are * __GFP_REPEAT */ - if (order > PAGE_ALLOC_COSTLY_ORDER && !(gfp_mask & __GFP_REPEAT)) + if (costly_order && !(gfp_mask & __GFP_REPEAT)) goto nopage; /* Make sure we know about allocations which stall for too long */ -- 2.11.0
WARNING: multiple messages have this Message-ID (diff)
From: Vlastimil Babka <vbabka@suse.cz> To: linux-mm@kvack.org, Johannes Weiner <hannes@cmpxchg.org> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>, David Rientjes <rientjes@google.com>, Mel Gorman <mgorman@techsingularity.net>, linux-kernel@vger.kernel.org, kernel-team@fb.com, Vlastimil Babka <vbabka@suse.cz> Subject: [PATCH v2 07/10] mm, compaction: restrict async compaction to pageblocks of same migratetype Date: Fri, 10 Feb 2017 18:23:40 +0100 [thread overview] Message-ID: <20170210172343.30283-8-vbabka@suse.cz> (raw) In-Reply-To: <20170210172343.30283-1-vbabka@suse.cz> The migrate scanner in async compaction is currently limited to MIGRATE_MOVABLE pageblocks. This is a heuristic intended to reduce latency, based on the assumption that non-MOVABLE pageblocks are unlikely to contain movable pages. However, with the exception of THP's, most high-order allocations are not movable. Should the async compaction succeed, this increases the chance that the non-MOVABLE allocations will fallback to a MOVABLE pageblock, making the long-term fragmentation worse. This patch attempts to help the situation by changing async direct compaction so that the migrate scanner only scans the pageblocks of the requested migratetype. If it's a non-MOVABLE type and there are such pageblocks that do contain movable pages, chances are that the allocation can succeed within one of such pageblocks, removing the need for a fallback. If that fails, the subsequent sync attempt will ignore this restriction. Signed-off-by: Vlastimil Babka <vbabka@suse.cz> --- mm/compaction.c | 11 +++++++++-- mm/page_alloc.c | 20 +++++++++++++------- 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index b7094700712b..84ef44c3b1c9 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -994,10 +994,17 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn, static bool suitable_migration_source(struct compact_control *cc, struct page *page) { - if (cc->mode != MIGRATE_ASYNC) + int block_mt; + + if ((cc->mode != MIGRATE_ASYNC) || !cc->direct_compaction) return true; - return is_migrate_movable(get_pageblock_migratetype(page)); + block_mt = get_pageblock_migratetype(page); + + if (cc->migratetype == MIGRATE_MOVABLE) + return is_migrate_movable(block_mt); + else + return block_mt == cc->migratetype; } /* Returns true if the page is within a block suitable for migration to */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a7d33818610f..6d9ba640a12d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3523,6 +3523,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, struct alloc_context *ac) { bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; + const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER; struct page *page = NULL; unsigned int alloc_flags; unsigned long did_some_progress; @@ -3572,12 +3573,17 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, /* * For costly allocations, try direct compaction first, as it's likely - * that we have enough base pages and don't need to reclaim. Don't try - * that for allocations that are allowed to ignore watermarks, as the - * ALLOC_NO_WATERMARKS attempt didn't yet happen. + * that we have enough base pages and don't need to reclaim. For non- + * movable high-order allocations, do that as well, as compaction will + * try prevent permanent fragmentation by migrating from blocks of the + * same migratetype. + * Don't try this for allocations that are allowed to ignore + * watermarks, as the ALLOC_NO_WATERMARKS attempt didn't yet happen. */ - if (can_direct_reclaim && order > PAGE_ALLOC_COSTLY_ORDER && - !gfp_pfmemalloc_allowed(gfp_mask)) { + if (can_direct_reclaim && + (costly_order || + (order > 0 && ac->migratetype != MIGRATE_MOVABLE)) + && !gfp_pfmemalloc_allowed(gfp_mask)) { page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags, ac, INIT_COMPACT_PRIORITY, @@ -3589,7 +3595,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, * Checks for costly allocations with __GFP_NORETRY, which * includes THP page fault allocations */ - if (gfp_mask & __GFP_NORETRY) { + if (costly_order && (gfp_mask & __GFP_NORETRY)) { /* * If compaction is deferred for high-order allocations, * it is because sync compaction recently failed. If @@ -3684,7 +3690,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, * Do not retry costly high order allocations unless they are * __GFP_REPEAT */ - if (order > PAGE_ALLOC_COSTLY_ORDER && !(gfp_mask & __GFP_REPEAT)) + if (costly_order && !(gfp_mask & __GFP_REPEAT)) goto nopage; /* Make sure we know about allocations which stall for too long */ -- 2.11.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-02-10 18:16 UTC|newest] Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-02-10 17:23 [PATCH v2 00/10] try to reduce fragmenting fallbacks Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-10 17:23 ` [PATCH v2 01/10] mm, compaction: reorder fields in struct compact_control Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:49 ` Mel Gorman 2017-02-13 10:49 ` Mel Gorman 2017-02-14 16:33 ` Johannes Weiner 2017-02-14 16:33 ` Johannes Weiner 2017-02-10 17:23 ` [PATCH v2 02/10] mm, compaction: remove redundant watermark check in compact_finished() Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:49 ` Mel Gorman 2017-02-13 10:49 ` Mel Gorman 2017-02-14 16:34 ` Johannes Weiner 2017-02-14 16:34 ` Johannes Weiner 2017-02-10 17:23 ` [PATCH v2 03/10] mm, page_alloc: split smallest stolen page in fallback Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:51 ` Mel Gorman 2017-02-13 10:51 ` Mel Gorman 2017-02-13 10:54 ` Vlastimil Babka 2017-02-13 10:54 ` Vlastimil Babka 2017-02-14 16:59 ` Johannes Weiner 2017-02-14 16:59 ` Johannes Weiner 2017-02-10 17:23 ` [PATCH v2 04/10] mm, page_alloc: count movable pages when stealing from pageblock Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:53 ` Mel Gorman 2017-02-13 10:53 ` Mel Gorman 2017-02-14 10:07 ` Xishi Qiu 2017-02-14 10:07 ` Xishi Qiu 2017-02-15 10:47 ` Vlastimil Babka 2017-02-15 10:47 ` Vlastimil Babka 2017-02-15 11:56 ` Xishi Qiu 2017-02-15 11:56 ` Xishi Qiu 2017-02-17 16:21 ` Vlastimil Babka 2017-02-17 16:21 ` Vlastimil Babka 2017-02-14 18:10 ` Johannes Weiner 2017-02-14 18:10 ` Johannes Weiner 2017-02-17 16:09 ` Vlastimil Babka 2017-02-17 16:09 ` Vlastimil Babka 2017-02-10 17:23 ` [PATCH v2 05/10] mm, compaction: change migrate_async_suitable() to suitable_migration_source() Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:53 ` Mel Gorman 2017-02-13 10:53 ` Mel Gorman 2017-02-14 18:12 ` Johannes Weiner 2017-02-14 18:12 ` Johannes Weiner 2017-02-10 17:23 ` [PATCH v2 06/10] mm, compaction: add migratetype to compact_control Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:53 ` Mel Gorman 2017-02-13 10:53 ` Mel Gorman 2017-02-14 18:15 ` Johannes Weiner 2017-02-14 18:15 ` Johannes Weiner 2017-02-10 17:23 ` Vlastimil Babka [this message] 2017-02-10 17:23 ` [PATCH v2 07/10] mm, compaction: restrict async compaction to pageblocks of same migratetype Vlastimil Babka 2017-02-13 10:56 ` Mel Gorman 2017-02-13 10:56 ` Mel Gorman 2017-02-14 20:10 ` Johannes Weiner 2017-02-14 20:10 ` Johannes Weiner 2017-02-17 16:32 ` Vlastimil Babka 2017-02-17 16:32 ` Vlastimil Babka 2017-02-17 17:39 ` Johannes Weiner 2017-02-17 17:39 ` Johannes Weiner 2017-02-10 17:23 ` [PATCH v2 08/10] mm, compaction: finish whole pageblock to reduce fragmentation Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-13 10:57 ` Mel Gorman 2017-02-13 10:57 ` Mel Gorman 2017-02-16 11:44 ` Johannes Weiner 2017-02-16 11:44 ` Johannes Weiner 2017-02-10 17:23 ` [RFC v2 09/10] mm, page_alloc: disallow migratetype fallback in fastpath Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-02-10 17:23 ` [RFC v2 10/10] mm, page_alloc: introduce MIGRATE_MIXED migratetype Vlastimil Babka 2017-02-10 17:23 ` Vlastimil Babka 2017-03-08 2:16 ` Yisheng Xie 2017-03-08 2:16 ` Yisheng Xie 2017-03-08 7:07 ` Vlastimil Babka 2017-03-08 7:07 ` Vlastimil Babka 2017-03-13 2:16 ` Yisheng Xie 2017-03-13 2:16 ` Yisheng Xie 2017-02-13 11:07 ` [PATCH v2 00/10] try to reduce fragmenting fallbacks Mel Gorman 2017-02-13 11:07 ` Mel Gorman 2017-02-15 14:29 ` Vlastimil Babka 2017-02-15 14:29 ` Vlastimil Babka 2017-02-15 16:11 ` Vlastimil Babka 2017-02-15 16:11 ` Vlastimil Babka 2017-02-15 20:11 ` Vlastimil Babka 2017-02-15 20:11 ` Vlastimil Babka 2017-02-16 15:12 ` Vlastimil Babka 2017-02-16 15:12 ` Vlastimil Babka 2017-02-17 15:24 ` Vlastimil Babka 2017-02-17 15:24 ` Vlastimil Babka 2017-02-20 12:30 ` Vlastimil Babka 2017-02-20 12:30 ` Vlastimil Babka 2017-02-23 16:01 ` Mel Gorman 2017-02-23 16:01 ` Mel Gorman
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170210172343.30283-8-vbabka@suse.cz \ --to=vbabka@suse.cz \ --cc=hannes@cmpxchg.org \ --cc=iamjoonsoo.kim@lge.com \ --cc=kernel-team@fb.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@techsingularity.net \ --cc=rientjes@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.