From: Vlastimil Babka <vbabka@suse.cz> To: Michal Hocko <mhocko@kernel.org>, Andrew Morton <akpm@linux-foundation.org> Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Mel Gorman <mgorman@techsingularity.net>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, David Rientjes <rientjes@google.com>, Rik van Riel <riel@redhat.com>, Vlastimil Babka <vbabka@suse.cz> Subject: [PATCH v2 03/18] mm, page_alloc: don't retry initial attempt in slowpath Date: Tue, 31 May 2016 15:08:03 +0200 [thread overview] Message-ID: <20160531130818.28724-4-vbabka@suse.cz> (raw) In-Reply-To: <20160531130818.28724-1-vbabka@suse.cz> After __alloc_pages_slowpath() sets up new alloc_flags and wakes up kswapd, it first tries get_page_from_freelist() with the new alloc_flags, as it may succeed e.g. due to using min watermark instead of low watermark. This attempt does not have to be retried on each loop, since direct reclaim, direct compaction and oom call get_page_from_freelist() themselves. There is a corner case where direct reclaim does not attempt allocation when there is no reclaim progress, but that is trivial to adjust. This patch therefore moves the initial attempt above the retry label. The ALLOC_NO_WATERMARKS attempt is kept under retry label as it's special and should be retried after each loop. Kswapd wakeups are also done on each retry to be safe from potential races resulting in kswapd going to sleep while a process (that may not be able to reclaim by itself) is still looping. Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Acked-by: Michal Hocko <mhocko@suse.com> --- mm/page_alloc.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index da3a62a94b4a..9f83259a18a8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3367,10 +3367,9 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, bool drained = false; *did_some_progress = __perform_reclaim(gfp_mask, order, ac); - if (unlikely(!(*did_some_progress))) - return NULL; retry: + /* We attempt even when no progress, as kswapd might have done some */ page = get_page_from_freelist(gfp_mask, order, alloc_flags, ac); /* @@ -3378,7 +3377,7 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, * pages are pinned on the per-cpu lists or in high alloc reserves. * Shrink them them and try again */ - if (!page && !drained) { + if (!page && *did_some_progress && !drained) { unreserve_highatomic_pageblock(ac); drain_all_pages(NULL); drained = true; @@ -3592,15 +3591,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, */ alloc_flags = gfp_to_alloc_flags(gfp_mask); -retry: if (gfp_mask & __GFP_KSWAPD_RECLAIM) wake_all_kswapds(order, ac); - /* This is the last chance, in general, before the goto nopage. */ + /* + * The adjusted alloc_flags might result in immediate success, so try + * that first + */ page = get_page_from_freelist(gfp_mask, order, alloc_flags, ac); if (page) goto got_pg; +retry: + /* Ensure kswapd doesn't accidentally go to sleep as long as we loop */ + if (gfp_mask & __GFP_KSWAPD_RECLAIM) + wake_all_kswapds(order, ac); + /* Allocate without watermarks if the context allows */ if (gfp_pfmemalloc_allowed(gfp_mask)) { /* -- 2.8.3
WARNING: multiple messages have this Message-ID (diff)
From: Vlastimil Babka <vbabka@suse.cz> To: Michal Hocko <mhocko@kernel.org>, Andrew Morton <akpm@linux-foundation.org> Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Mel Gorman <mgorman@techsingularity.net>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, David Rientjes <rientjes@google.com>, Rik van Riel <riel@redhat.com>, Vlastimil Babka <vbabka@suse.cz> Subject: [PATCH v2 03/18] mm, page_alloc: don't retry initial attempt in slowpath Date: Tue, 31 May 2016 15:08:03 +0200 [thread overview] Message-ID: <20160531130818.28724-4-vbabka@suse.cz> (raw) In-Reply-To: <20160531130818.28724-1-vbabka@suse.cz> After __alloc_pages_slowpath() sets up new alloc_flags and wakes up kswapd, it first tries get_page_from_freelist() with the new alloc_flags, as it may succeed e.g. due to using min watermark instead of low watermark. This attempt does not have to be retried on each loop, since direct reclaim, direct compaction and oom call get_page_from_freelist() themselves. There is a corner case where direct reclaim does not attempt allocation when there is no reclaim progress, but that is trivial to adjust. This patch therefore moves the initial attempt above the retry label. The ALLOC_NO_WATERMARKS attempt is kept under retry label as it's special and should be retried after each loop. Kswapd wakeups are also done on each retry to be safe from potential races resulting in kswapd going to sleep while a process (that may not be able to reclaim by itself) is still looping. Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Acked-by: Michal Hocko <mhocko@suse.com> --- mm/page_alloc.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index da3a62a94b4a..9f83259a18a8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3367,10 +3367,9 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, bool drained = false; *did_some_progress = __perform_reclaim(gfp_mask, order, ac); - if (unlikely(!(*did_some_progress))) - return NULL; retry: + /* We attempt even when no progress, as kswapd might have done some */ page = get_page_from_freelist(gfp_mask, order, alloc_flags, ac); /* @@ -3378,7 +3377,7 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, * pages are pinned on the per-cpu lists or in high alloc reserves. * Shrink them them and try again */ - if (!page && !drained) { + if (!page && *did_some_progress && !drained) { unreserve_highatomic_pageblock(ac); drain_all_pages(NULL); drained = true; @@ -3592,15 +3591,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, */ alloc_flags = gfp_to_alloc_flags(gfp_mask); -retry: if (gfp_mask & __GFP_KSWAPD_RECLAIM) wake_all_kswapds(order, ac); - /* This is the last chance, in general, before the goto nopage. */ + /* + * The adjusted alloc_flags might result in immediate success, so try + * that first + */ page = get_page_from_freelist(gfp_mask, order, alloc_flags, ac); if (page) goto got_pg; +retry: + /* Ensure kswapd doesn't accidentally go to sleep as long as we loop */ + if (gfp_mask & __GFP_KSWAPD_RECLAIM) + wake_all_kswapds(order, ac); + /* Allocate without watermarks if the context allows */ if (gfp_pfmemalloc_allowed(gfp_mask)) { /* -- 2.8.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-05-31 13:08 UTC|newest] Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-05-31 13:08 [PATCH v2 00/18] make direct compaction more deterministic Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 01/18] mm, compaction: don't isolate PageWriteback pages in MIGRATE_SYNC_LIGHT mode Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 02/18] mm, page_alloc: set alloc_flags only once in slowpath Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka [this message] 2016-05-31 13:08 ` [PATCH v2 03/18] mm, page_alloc: don't retry initial attempt " Vlastimil Babka 2016-06-01 13:26 ` Michal Hocko 2016-06-01 13:26 ` Michal Hocko 2016-06-01 14:58 ` Vlastimil Babka 2016-06-01 14:58 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 04/18] mm, page_alloc: restructure direct compaction handling " Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 05/18] mm, page_alloc: make THP-specific decisions more generic Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 06/18] mm, thp: remove __GFP_NORETRY from khugepaged and madvised allocations Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 13:33 ` Michal Hocko 2016-06-01 13:33 ` Michal Hocko 2016-05-31 13:08 ` [PATCH v2 07/18] mm, compaction: introduce direct compaction priority Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 08/18] mm, compaction: simplify contended compaction handling Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 09/18] mm, compaction: make whole_zone flag ignore cached scanner positions Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 10/18] mm, compaction: cleanup unused functions Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 13:45 ` Michal Hocko 2016-06-01 13:45 ` Michal Hocko 2016-05-31 13:08 ` [PATCH v2 11/18] mm, compaction: add the ultimate direct compaction priority Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 12/18] mm, compaction: more reliably increase " Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 13:51 ` Michal Hocko 2016-06-01 13:51 ` Michal Hocko 2016-06-23 14:41 ` Vlastimil Babka 2016-06-23 14:41 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 13/18] mm, compaction: use correct watermark when checking allocation success Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 13:59 ` Michal Hocko 2016-06-01 13:59 ` Michal Hocko 2016-05-31 13:08 ` [PATCH v2 14/18] mm, compaction: create compact_gap wrapper Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 14:02 ` Michal Hocko 2016-06-01 14:02 ` Michal Hocko 2016-05-31 13:08 ` [PATCH v2 15/18] mm, compaction: use proper alloc_flags in __compaction_suitable() Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-05-31 13:08 ` [PATCH v2 16/18] mm, compaction: require only min watermarks for non-costly orders Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 14:08 ` Michal Hocko 2016-06-01 14:08 ` Michal Hocko 2016-05-31 13:08 ` [PATCH v2 17/18] mm, vmscan: make compaction_ready() more accurate and readable Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 14:14 ` Michal Hocko 2016-06-01 14:14 ` Michal Hocko 2016-05-31 13:08 ` [PATCH v2 18/18] mm, vmscan: use proper classzone_idx in should_continue_reclaim() Vlastimil Babka 2016-05-31 13:08 ` Vlastimil Babka 2016-06-01 14:21 ` Michal Hocko 2016-06-01 14:21 ` Michal Hocko 2016-06-01 15:19 ` Vlastimil Babka 2016-06-01 15:19 ` Vlastimil Babka 2016-06-01 15:45 ` Michal Hocko 2016-06-01 15:45 ` Michal Hocko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20160531130818.28724-4-vbabka@suse.cz \ --to=vbabka@suse.cz \ --cc=akpm@linux-foundation.org \ --cc=iamjoonsoo.kim@lge.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@techsingularity.net \ --cc=mhocko@kernel.org \ --cc=riel@redhat.com \ --cc=rientjes@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.