mm-commits Archive on lore.kernel.org
 help / color / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, linux-mm@kvack.org,
	mgorman@techsingularity.net, mhocko@kernel.org,
	mm-commits@vger.kernel.org, riel@surriel.com,
	shy828301@gmail.com, stable@vger.kernel.org,
	torvalds@linux-foundation.org, vbabka@suse.cz, ziy@nvidia.com
Subject: [patch 01/14] mm/compaction: count pages and stop correctly during page isolation
Date: Fri, 13 Nov 2020 22:51:40 -0800
Message-ID: <20201114065140.81J4LaSck%akpm@linux-foundation.org> (raw)
In-Reply-To: <20201113225115.b24faebc85f710d5aff55aa7@linux-foundation.org>

From: Zi Yan <ziy@nvidia.com>
Subject: mm/compaction: count pages and stop correctly during page isolation

In isolate_migratepages_block, when cc->alloc_contig is true, we are able
to isolate compound pages, nr_migratepages and nr_isolated did not count
compound pages correctly, causing us to isolate more pages than we
thought.  Count compound pages as the number of base pages they contain. 
Otherwise, we might be trapped in too_many_isolated while loop, since the
actual isolated pages can go up to COMPACT_CLUSTER_MAX*512=16384, where
COMPACT_CLUSTER_MAX is 32, since we stop isolation after
cc->nr_migratepages reaches to COMPACT_CLUSTER_MAX.

In addition, after we fix the issue above, cc->nr_migratepages could never
be equal to COMPACT_CLUSTER_MAX if compound pages are isolated, thus page
isolation could not stop as we intended.  Change the isolation stop
condition to >=.

The issue can be triggered as follows: In a system with 16GB memory and an
8GB CMA region reserved by hugetlb_cma, if we first allocate 10GB THPs and
mlock them (so some THPs are allocated in the CMA region and mlocked),
reserving 6 1GB hugetlb pages via
/sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages will get stuck
(looping in too_many_isolated function) until we kill either task.  With
the patch applied, oom will kill the application with 10GB THPs and let
hugetlb page reservation finish.

[ziy@nvidia.com: v3]
  Link: https://lkml.kernel.org/r/20201030183809.3616803-1-zi.yan@sent.com
Link: https://lkml.kernel.org/r/20201029200435.3386066-1-zi.yan@sent.com
Fixes: 1da2f328fa64 ("cmm,thp,compaction,cma: allow THP migration for CMA allocations")
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Yang Shi <shy828301@gmail.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Rik van Riel <riel@surriel.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/compaction.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/mm/compaction.c~mm-compaction-count-pages-and-stop-correctly-during-page-isolation
+++ a/mm/compaction.c
@@ -1012,8 +1012,8 @@ isolate_migratepages_block(struct compac
 
 isolate_success:
 		list_add(&page->lru, &cc->migratepages);
-		cc->nr_migratepages++;
-		nr_isolated++;
+		cc->nr_migratepages += compound_nr(page);
+		nr_isolated += compound_nr(page);
 
 		/*
 		 * Avoid isolating too much unless this block is being
@@ -1021,7 +1021,7 @@ isolate_success:
 		 * or a lock is contended. For contention, isolate quickly to
 		 * potentially remove one source of contention.
 		 */
-		if (cc->nr_migratepages == COMPACT_CLUSTER_MAX &&
+		if (cc->nr_migratepages >= COMPACT_CLUSTER_MAX &&
 		    !cc->rescan && !cc->contended) {
 			++low_pfn;
 			break;
@@ -1132,7 +1132,7 @@ isolate_migratepages_range(struct compac
 		if (!pfn)
 			break;
 
-		if (cc->nr_migratepages == COMPACT_CLUSTER_MAX)
+		if (cc->nr_migratepages >= COMPACT_CLUSTER_MAX)
 			break;
 	}
 
_

  reply index

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-14  6:51 incoming Andrew Morton
2020-11-14  6:51 ` Andrew Morton [this message]
2020-11-14  6:51 ` [patch 02/14] mm/compaction: stop isolation if too many pages are isolated and we have pages to migrate Andrew Morton
2020-11-14  6:51 ` [patch 03/14] mm/vmscan: fix NR_ISOLATED_FILE corruption on 64-bit Andrew Morton
2020-11-14 21:39   ` Linus Torvalds
2020-11-14 22:14     ` Matthew Wilcox
2020-11-14  6:51 ` [patch 04/14] mailmap: fix entry for Dmitry Baryshkov/Eremin-Solenikov Andrew Morton
2020-11-14  6:51 ` [patch 05/14] mm/slub: fix panic in slab_alloc_node() Andrew Morton
2020-11-14  6:51 ` [patch 06/14] mm/gup: use unpin_user_pages() in __gup_longterm_locked() Andrew Morton
2020-11-14  6:51 ` [patch 07/14] compiler.h: fix barrier_data() on clang Andrew Morton
2020-11-14  6:52 ` [patch 08/14] Revert "kernel/reboot.c: convert simple_strtoul to kstrtoint" Andrew Morton
2020-11-14  6:52 ` [patch 09/14] reboot: fix overflow parsing reboot cpu number Andrew Morton
2020-11-14  6:52 ` [patch 10/14] kernel/watchdog: fix watchdog_allowed_mask not used warning Andrew Morton
2020-11-14  6:52 ` [patch 11/14] mm: memcontrol: fix missing wakeup polling thread Andrew Morton
2020-11-14  6:52 ` [patch 12/14] hugetlbfs: fix anon huge page migration race Andrew Morton
2020-11-14  6:52 ` [patch 13/14] panic: don't dump stack twice on warn Andrew Morton
2020-11-14  6:52 ` [patch 14/14] ocfs2: initialize ip_next_orphan Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201114065140.81J4LaSck%akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@kernel.org \
    --cc=mm-commits@vger.kernel.org \
    --cc=riel@surriel.com \
    --cc=shy828301@gmail.com \
    --cc=stable@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=vbabka@suse.cz \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

mm-commits Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/mm-commits/0 mm-commits/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 mm-commits mm-commits/ https://lore.kernel.org/mm-commits \
		mm-commits@vger.kernel.org
	public-inbox-index mm-commits

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.mm-commits


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git