From: Andrew Morton <akpm@linux-foundation.org>
To: rientjes@google.com, mhocko@suse.com, mgorman@suse.de,
kirill@shutemov.name, aarcange@redhat.com, vbabka@suse.cz,
akpm@linux-foundation.org, linux-mm@kvack.org,
mm-commits@vger.kernel.org, torvalds@linux-foundation.org
Subject: [patch 01/11] mm, thp: tweak reclaim/compaction effort of local-only and all-node allocations
Date: Mon, 13 Jan 2020 16:29:04 -0800 [thread overview]
Message-ID: <20200114002904.h9bSo%akpm__21904.1851743654$1578961831$gmane$org@linux-foundation.org> (raw)
In-Reply-To: <20200113162831.f7d69e11e9e673c40005c9b0@linux-foundation.org>
From: Vlastimil Babka <vbabka@suse.cz>
Subject: mm, thp: tweak reclaim/compaction effort of local-only and all-node allocations
THP page faults now attempt a __GFP_THISNODE allocation first, which
should only compact existing free memory, followed by another attempt that
can allocate from any node using reclaim/compaction effort specified by
global defrag setting and madvise.
This patch makes the following changes to the scheme:
- before the patch, the first allocation relies on a check for pageblock
order and __GFP_IO to prevent excessive reclaim. This however affects
also the second attempt, which is not limited to single node. Instead
of that, reuse the existing check for costly order __GFP_NORETRY
allocations, and make sure the first THP attempt uses __GFP_NORETRY. As
a side-effect, all costly order __GFP_NORETRY allocations will bail out
if compaction needs reclaim, while previously they only bailed out when
compaction was deferred due to previous failures. This should be still
acceptable within the __GFP_NORETRY semantics.
- before the patch, the second allocation attempt (on all nodes) was
passing __GFP_NORETRY. This is redundant as the check for pageblock
order (discussed above) was stronger. It's also contrary to
madvise(MADV_HUGEPAGE) which means some effort to allocate THP is
requested. After this patch, the second attempt doesn't pass
__GFP_THISNODE nor __GFP_NORETRY.
To sum up, THP page faults now try the following attempts:
1. local node only THP allocation with no reclaim, just compaction.
2. for madvised VMA's or when synchronous compaction is enabled always - THP
allocation from any node with effort determined by global defrag setting
and VMA madvise
3. fallback to base pages on any node
Link: http://lkml.kernel.org/r/08a3f4dd-c3ce-0009-86c5-9ee51aba8557@suse.cz
Fixes: b39d0ee2632d ("mm, page_alloc: avoid expensive reclaim when compaction may not succeed")
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Kirill A. Shutemov" <kirill@shutemov.name>
Cc: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
mm/mempolicy.c | 10 +++++++---
mm/page_alloc.c | 24 +++++-------------------
2 files changed, 12 insertions(+), 22 deletions(-)
--- a/mm/mempolicy.c~mm-thp-tweak-reclaim-compaction-effort-of-local-only-and-all-node-allocations
+++ a/mm/mempolicy.c
@@ -2148,18 +2148,22 @@ alloc_pages_vma(gfp_t gfp, int order, st
nmask = policy_nodemask(gfp, pol);
if (!nmask || node_isset(hpage_node, *nmask)) {
mpol_cond_put(pol);
+ /*
+ * First, try to allocate THP only on local node, but
+ * don't reclaim unnecessarily, just compact.
+ */
page = __alloc_pages_node(hpage_node,
- gfp | __GFP_THISNODE, order);
+ gfp | __GFP_THISNODE | __GFP_NORETRY, order);
/*
* If hugepage allocations are configured to always
* synchronous compact or the vma has been madvised
* to prefer hugepage backing, retry allowing remote
- * memory as well.
+ * memory with both reclaim and compact as well.
*/
if (!page && (gfp & __GFP_DIRECT_RECLAIM))
page = __alloc_pages_node(hpage_node,
- gfp | __GFP_NORETRY, order);
+ gfp, order);
goto out;
}
--- a/mm/page_alloc.c~mm-thp-tweak-reclaim-compaction-effort-of-local-only-and-all-node-allocations
+++ a/mm/page_alloc.c
@@ -4476,8 +4476,11 @@ retry_cpuset:
if (page)
goto got_pg;
- if (order >= pageblock_order && (gfp_mask & __GFP_IO) &&
- !(gfp_mask & __GFP_RETRY_MAYFAIL)) {
+ /*
+ * Checks for costly allocations with __GFP_NORETRY, which
+ * includes some THP page fault allocations
+ */
+ if (costly_order && (gfp_mask & __GFP_NORETRY)) {
/*
* If allocating entire pageblock(s) and compaction
* failed because all zones are below low watermarks
@@ -4498,23 +4501,6 @@ retry_cpuset:
if (compact_result == COMPACT_SKIPPED ||
compact_result == COMPACT_DEFERRED)
goto nopage;
- }
next prev parent reply other threads:[~2020-01-14 0:29 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-14 0:28 incoming Andrew Morton
2020-01-14 0:29 ` Andrew Morton [this message]
2020-01-14 0:29 ` [patch 02/11] mm/memory_hotplug: don't free usage map when removing a re-added early section Andrew Morton
2020-01-14 0:29 ` [patch 03/11] mm/huge_memory.c: thp: fix conflict of above-47bit hint address and PMD alignment Andrew Morton
2020-01-14 0:29 ` [patch 04/11] mm/shmem.c: thp, shmem: " Andrew Morton
2020-01-14 0:29 ` [patch 05/11] mm: memcg/slab: fix percpu slab vmstats flushing Andrew Morton
2020-01-14 0:29 ` [patch 06/11] mm, debug_pagealloc: don't rely on static keys too early Andrew Morton
2020-01-14 0:29 ` [patch 07/11] mm/page-writeback.c: avoid potential division by zero in wb_min_max_ratio() Andrew Morton
2020-01-14 0:29 ` [patch 08/11] mm/page-writeback.c: use div64_ul() for u64-by-unsigned-long divide Andrew Morton
2020-01-14 0:29 ` [patch 09/11] mm/page-writeback.c: improve arithmetic divisions Andrew Morton
2020-01-14 0:29 ` [patch 10/11] mm: memcg/slab: call flush_memcg_workqueue() only if memcg workqueue is valid Andrew Morton
2020-01-14 0:29 ` [patch 11/11] mm: khugepaged: add trace status description for SCAN_PAGE_HAS_PRIVATE Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='20200114002904.h9bSo%akpm__21904.1851743654$1578961831$gmane$org@linux-foundation.org' \
--to=akpm@linux-foundation.org \
--cc=aarcange@redhat.com \
--cc=kirill@shutemov.name \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.com \
--cc=mm-commits@vger.kernel.org \
--cc=rientjes@google.com \
--cc=torvalds@linux-foundation.org \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).