All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mel Gorman <mgorman@techsingularity.net>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: David Rientjes <rientjes@google.com>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Vlastimil Babka <vbabka@suse.cz>,
	Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
	Linux-MM <linux-mm@kvack.org>,
	Mel Gorman <mgorman@techsingularity.net>
Subject: [PATCH 12/22] mm, compaction: Avoid rescanning the same pageblock multiple times
Date: Fri, 18 Jan 2019 17:51:26 +0000	[thread overview]
Message-ID: <20190118175136.31341-13-mgorman@techsingularity.net> (raw)
In-Reply-To: <20190118175136.31341-1-mgorman@techsingularity.net>

Pageblocks are marked for skip when no pages are isolated after a scan.
However, it's possible to hit corner cases where the migration scanner
gets stuck near the boundary between the source and target scanner. Due
to pages being migrated in blocks of COMPACT_CLUSTER_MAX, pages that
are migrated can be reallocated before the pageblock is complete. The
pageblock is not necessarily skipped so it can be rescanned multiple
times. Similarly, a pageblock with some dirty/writeback pages may fail
to migrate and be rescanned until writeback completes which is wasteful.

This patch tracks if a pageblock is being rescanned. If so, then the entire
pageblock will be migrated as one operation. This narrows the race window
during which pages can be reallocated during migration. Secondly, if there
are pages that cannot be isolated then the pageblock will still be fully
scanned and marked for skipping. On the second rescan, the pageblock skip
is set and the migration scanner makes progress.

                                     5.0.0-rc1              5.0.0-rc1
                                findfree-v3r16         norescan-v3r16
Amean     fault-both-1         0.00 (   0.00%)        0.00 *   0.00%*
Amean     fault-both-3      3200.68 (   0.00%)     3002.07 (   6.21%)
Amean     fault-both-5      4847.75 (   0.00%)     4684.47 (   3.37%)
Amean     fault-both-7      6658.92 (   0.00%)     6815.54 (  -2.35%)
Amean     fault-both-12    11077.62 (   0.00%)    10864.02 (   1.93%)
Amean     fault-both-18    12403.97 (   0.00%)    12247.52 (   1.26%)
Amean     fault-both-24    15607.10 (   0.00%)    15683.99 (  -0.49%)
Amean     fault-both-30    18752.27 (   0.00%)    18620.02 (   0.71%)
Amean     fault-both-32    21207.54 (   0.00%)    19250.28 *   9.23%*

                                5.0.0-rc1              5.0.0-rc1
                           findfree-v3r16         norescan-v3r16
Percentage huge-3        96.86 (   0.00%)       95.00 (  -1.91%)
Percentage huge-5        93.72 (   0.00%)       94.22 (   0.53%)
Percentage huge-7        94.31 (   0.00%)       92.35 (  -2.08%)
Percentage huge-12       92.66 (   0.00%)       91.90 (  -0.82%)
Percentage huge-18       91.51 (   0.00%)       89.58 (  -2.11%)
Percentage huge-24       90.50 (   0.00%)       90.03 (  -0.52%)
Percentage huge-30       91.57 (   0.00%)       89.14 (  -2.65%)
Percentage huge-32       91.00 (   0.00%)       90.58 (  -0.46%)

Negligible difference but this was likely a case when the specific corner
case was not hit. A previous run of the same patch based on an earlier
iteration of the series showed large differences where migration rates
could be halved when the corner case was hit.

The specific corner case where migration scan rates go through the roof
was due to a dirty/writeback pageblock located at the boundary of the
migration/free scanner did not happen in this case. When it does happen,
the scan rates multipled by massive margins.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
---
 mm/compaction.c | 32 ++++++++++++++++++++++++++------
 mm/internal.h   |  1 +
 2 files changed, 27 insertions(+), 6 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index 19fea4a7b3f4..a31fea7b3f96 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -949,8 +949,11 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		cc->nr_migratepages++;
 		nr_isolated++;
 
-		/* Avoid isolating too much */
-		if (cc->nr_migratepages == COMPACT_CLUSTER_MAX) {
+		/*
+		 * Avoid isolating too much unless this block is being
+		 * rescanned (e.g. dirty/writeback pages, parallel allocation).
+		 */
+		if (cc->nr_migratepages == COMPACT_CLUSTER_MAX && !cc->rescan) {
 			++low_pfn;
 			break;
 		}
@@ -997,11 +1000,14 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		spin_unlock_irqrestore(zone_lru_lock(zone), flags);
 
 	/*
-	 * Updated the cached scanner pfn if the pageblock was scanned
-	 * without isolating a page. The pageblock may not be marked
-	 * skipped already if there were no LRU pages in the block.
+	 * Updated the cached scanner pfn once the pageblock has been scanned
+	 * Pages will either be migrated in which case there is no point
+	 * scanning in the near future or migration failed in which case the
+	 * failure reason may persist. The block is marked for skipping if
+	 * there were no pages isolated in the block or if the block is
+	 * rescanned twice in a row.
 	 */
-	if (low_pfn == end_pfn && !nr_isolated) {
+	if (low_pfn == end_pfn && (!nr_isolated || cc->rescan)) {
 		if (valid_page && !skip_updated)
 			set_pageblock_skip(valid_page);
 		update_cached_migrate(cc, low_pfn);
@@ -2033,6 +2039,20 @@ static enum compact_result compact_zone(struct compact_control *cc)
 		int err;
 		unsigned long start_pfn = cc->migrate_pfn;
 
+		/*
+		 * Avoid multiple rescans which can happen if a page cannot be
+		 * isolated (dirty/writeback in async mode) or if the migrated
+		 * pages are being allocated before the pageblock is cleared.
+		 * The first rescan will capture the entire pageblock for
+		 * migration. If it fails, it'll be marked skip and scanning
+		 * will proceed as normal.
+		 */
+		cc->rescan = false;
+		if (pageblock_start_pfn(last_migrated_pfn) ==
+		    pageblock_start_pfn(start_pfn)) {
+			cc->rescan = true;
+		}
+
 		switch (isolate_migratepages(cc->zone, cc)) {
 		case ISOLATE_ABORT:
 			ret = COMPACT_CONTENDED;
diff --git a/mm/internal.h b/mm/internal.h
index 983cb975545f..d5b999e5eb5f 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -205,6 +205,7 @@ struct compact_control {
 	bool direct_compaction;		/* False from kcompactd or /proc/... */
 	bool whole_zone;		/* Whole zone should/has been scanned */
 	bool contended;			/* Signal lock or sched contention */
+	bool rescan;			/* Rescanning the same pageblock */
 };
 
 unsigned long
-- 
2.16.4


  parent reply	other threads:[~2019-01-18 17:53 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-18 17:51 [PATCH 00/22] Increase success rates and reduce latency of compaction v3 Mel Gorman
2019-01-18 17:51 ` [PATCH 01/22] mm, compaction: Shrink compact_control Mel Gorman
2019-01-18 17:51 ` [PATCH 02/22] mm, compaction: Rearrange compact_control Mel Gorman
2019-01-18 17:51 ` [PATCH 03/22] mm, compaction: Remove last_migrated_pfn from compact_control Mel Gorman
2019-01-18 17:51 ` [PATCH 04/22] mm, compaction: Remove unnecessary zone parameter in some instances Mel Gorman
2019-01-18 17:51 ` [PATCH 05/22] mm, compaction: Rename map_pages to split_map_pages Mel Gorman
2019-01-18 17:51 ` [PATCH 06/22] mm, migrate: Immediately fail migration of a page with no migration handler Mel Gorman
2019-01-18 17:51 ` [PATCH 07/22] mm, compaction: Always finish scanning of a full pageblock Mel Gorman
2019-01-18 17:51 ` [PATCH 08/22] mm, compaction: Ignore the fragmentation avoidance boost for isolation and compaction Mel Gorman
2019-01-18 17:51 ` [PATCH 09/22] mm, compaction: Use free lists to quickly locate a migration source Mel Gorman
2019-01-31 13:55   ` Vlastimil Babka
2019-01-31 14:12     ` Vlastimil Babka
2019-02-01 15:06     ` Mel Gorman
2019-02-04  8:55       ` [PATCH] mm, compaction: Use free lists to quickly locate a migration source -fix Mel Gorman
2019-02-08 17:10   ` [PATCH 09/22] mm, compaction: Use free lists to quickly locate a migration source Vlastimil Babka
2019-01-18 17:51 ` [PATCH 10/22] mm, compaction: Keep migration source private to a single compaction instance Mel Gorman
2019-01-18 17:51 ` [PATCH 11/22] mm, compaction: Use free lists to quickly locate a migration target Mel Gorman
2019-01-31 14:52   ` Vlastimil Babka
2019-02-01 14:51     ` Mel Gorman
2019-02-01 14:58       ` Vlastimil Babka
2019-02-04 12:01         ` [PATCH] mm, compaction: Use free lists to quickly locate a migration source -fix Mel Gorman
2021-01-12  5:19     ` [PATCH] mm, compaction: make sure we isolate a valid freepage when high_pfn is used Rokudo Yan
2021-01-12  9:10       ` Mel Gorman
2021-01-12  9:47         ` [PATCH] mm, compaction: move high_pfn to the for loop scope Rokudo Yan
2021-01-12 10:45           ` Mel Gorman
2021-01-12 10:48           ` Vlastimil Babka
2021-01-12 22:27           ` Andrew Morton
2021-01-18  7:41             ` Rokudo Yan
2021-01-18  9:42               ` Mel Gorman
2019-01-18 17:51 ` Mel Gorman [this message]
2019-01-18 17:51 ` [PATCH 13/22] mm, compaction: Finish pageblock scanning on contention Mel Gorman
2019-01-18 17:51 ` [PATCH 14/22] mm, compaction: Check early for huge pages encountered by the migration scanner Mel Gorman
2019-01-18 17:51 ` [PATCH 15/22] mm, compaction: Keep cached migration PFNs synced for unusable pageblocks Mel Gorman
2019-01-18 17:51 ` [PATCH 16/22] mm, compaction: Rework compact_should_abort as compact_check_resched Mel Gorman
2019-01-18 17:51 ` [PATCH 17/22] mm, compaction: Do not consider a need to reschedule as contention Mel Gorman
2019-01-18 17:51 ` [PATCH 18/22] mm, compaction: Reduce premature advancement of the migration target scanner Mel Gorman
2019-01-18 17:51 ` [PATCH 19/22] mm, compaction: Round-robin the order while searching the free lists for a target Mel Gorman
2019-01-18 17:51 ` [PATCH 20/22] mm, compaction: Sample pageblocks for free pages Mel Gorman
2019-01-31 15:39   ` Vlastimil Babka
2019-01-18 17:51 ` [PATCH 21/22] mm, compaction: Be selective about what pageblocks to clear skip hints Mel Gorman
2019-01-31 15:45   ` Vlastimil Babka
2019-01-18 17:51 ` [PATCH 22/22] mm, compaction: Capture a page under direct compaction Mel Gorman
2019-01-31 16:11   ` Vlastimil Babka
2019-02-01 14:38     ` [PATCH] mm, compaction: Capture a page under direct compaction -fix Mel Gorman
2019-01-24  8:53 ` [PATCH 00/22] Increase success rates and reduce latency of compaction v3 Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190118175136.31341-13-mgorman@techsingularity.net \
    --to=mgorman@techsingularity.net \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=rientjes@google.com \
    --cc=vbabka@suse.cz \
    --subject='Re: [PATCH 12/22] mm, compaction: Avoid rescanning the same pageblock multiple times' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.