All of lore.kernel.org
 help / color / mirror / Atom feed
* [GIT PULL] mm/compaction functional fixes for v5.1-rc4
@ 2019-04-05 13:51 Mel Gorman
  2019-04-05 13:51 ` [PATCH 1/2] mm/compaction.c: correct zone boundary handling when resetting pageblock skip hints Mel Gorman
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Mel Gorman @ 2019-04-05 13:51 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Andrew Morton, Linux-MM, LKML, Mel Gorman

The following changes since commit 79a3aaa7b82e3106be97842dedfd8429248896e6:

  Linux 5.1-rc3 (2019-03-31 14:39:29 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mel/linux.git tags/mm-compaction-5.1-rc4

for you to fetch changes up to 5b56d996dd50a9d2ca87c25ebb50c07b255b7e04:

  mm/compaction.c: abort search if isolation fails (2019-04-04 11:56:15 +0100)

----------------

Hi Linus,

The merge window for 5.1 introduced a number of compaction-related patches
authored by myself.  There are intermittent reports of corruption and
functional issues based on them due to sloppy checking of zone boundaries
and a corner case where the free lists are overrun.

Reports are not common but at least two users and 0-day have tripped over them.
There is a chance that one of the syzbot reports are related but it has not
been confirmed properly.

The normal submission path is through Andrew but it's now late on a Friday
and I do not know if a round of updates are coming your way or not and
these patches have been floating around for a while. Given the nature
of the fixes, I really would prefer to avoid another RC with corruption
issues creating duplicate reports.

All of these have been successfully tested on older RC windows. This will
make this branch look like a rebase but they've simply been cherry-picked
from Andrew's tree and placed on a fresh branch. I've no reason to
believe that this has invalidated the testing given the lack of change
in compaction and the nature of the fixes.

Note that you may still receive these from Andrew and there are other
compaction-related fixes in his tree that are less critical. I do not
expect them to conflict but there is a non-zero risk of confusion. If
you get a bunch of patches from Andrew then please ignore this entirely
so the normal submission path is preserved. Otherwise, please either git
pull this or pick up the patches directly at your discretion.

Mel Gorman (1):
  mm/compaction.c: correct zone boundary handling when resetting
    pageblock skip hints

Qian Cai (1):
  mm/compaction.c: abort search if isolation fails

 mm/compaction.c | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

-- 
2.16.4


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/2] mm/compaction.c: correct zone boundary handling when resetting pageblock skip hints
  2019-04-05 13:51 [GIT PULL] mm/compaction functional fixes for v5.1-rc4 Mel Gorman
@ 2019-04-05 13:51 ` Mel Gorman
  2019-04-05 13:51 ` [PATCH 2/2] mm/compaction.c: abort search if isolation fails Mel Gorman
  2019-04-05 16:15 ` [GIT PULL] mm/compaction functional fixes for v5.1-rc4 pr-tracker-bot
  2 siblings, 0 replies; 4+ messages in thread
From: Mel Gorman @ 2019-04-05 13:51 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Andrew Morton, Linux-MM, LKML, Mel Gorman

Mikhail Gavrilo reported the following bug being triggered in a Fedora
kernel based on 5.1-rc1 but it is relevant to a vanilla kernel.

 kernel: page dumped because: VM_BUG_ON_PAGE(PagePoisoned(p))
 kernel: ------------[ cut here ]------------
 kernel: kernel BUG at include/linux/mm.h:1021!
 kernel: invalid opcode: 0000 [#1] SMP NOPTI
 kernel: CPU: 6 PID: 116 Comm: kswapd0 Tainted: G         C        5.1.0-0.rc1.git1.3.fc31.x86_64 #1
 kernel: Hardware name: System manufacturer System Product Name/ROG STRIX X470-I GAMING, BIOS 1201 12/07/2018
 kernel: RIP: 0010:__reset_isolation_pfn+0x244/0x2b0
 kernel: Code: fe 06 e8 0f 8e fc ff 44 0f b6 4c 24 04 48 85 c0 0f 85 dc fe ff ff e9 68 fe ff ff 48 c7 c6 58 b7 2e 8c 4c 89 ff e8 0c 75 00 00 <0f> 0b 48 c7 c6 58 b7 2e 8c e8 fe 74 00 00 0f 0b 48 89 fa 41 b8 01
 kernel: RSP: 0018:ffff9e2d03f0fde8 EFLAGS: 00010246
 kernel: RAX: 0000000000000034 RBX: 000000000081f380 RCX: ffff8cffbddd6c20
 kernel: RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffff8cffbddd6c20
 kernel: RBP: 0000000000000001 R08: 0000009898b94613 R09: 0000000000000000
 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000100000
 kernel: R13: 0000000000100000 R14: 0000000000000001 R15: ffffca7de07ce000
 kernel: FS:  0000000000000000(0000) GS:ffff8cffbdc00000(0000) knlGS:0000000000000000
 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 kernel: CR2: 00007fc1670e9000 CR3: 00000007f5276000 CR4: 00000000003406e0
 kernel: Call Trace:
 kernel:  __reset_isolation_suitable+0x62/0x120
 kernel:  reset_isolation_suitable+0x3b/0x40
 kernel:  kswapd+0x147/0x540
 kernel:  ? finish_wait+0x90/0x90
 kernel:  kthread+0x108/0x140
 kernel:  ? balance_pgdat+0x560/0x560
 kernel:  ? kthread_park+0x90/0x90
 kernel:  ret_from_fork+0x27/0x50

He bisected it down to e332f741a8dd ("mm, compaction: be selective about
what pageblocks to clear skip hints").  The problem is that the patch in
question was sloppy with respect to the handling of zone boundaries.  In
some instances, it was possible for PFNs outside of a zone to be examined
and if those were not properly initialised or poisoned then it would
trigger the VM_BUG_ON.  This patch corrects the zone boundary issues when
resetting pageblock skip hints and Mikhail reported that the bug did not
trigger after 30 hours of testing.

Link: http://lkml.kernel.org/r/20190327085424.GL3189@techsingularity.net
Fixes: e332f741a8dd ("mm, compaction: be selective about what pageblocks to clear skip hints")
Reported-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Qian Cai <cai@lca.pw>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
 mm/compaction.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index f171a83707ce..b4930bf93c8a 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -242,6 +242,7 @@ __reset_isolation_pfn(struct zone *zone, unsigned long pfn, bool check_source,
 							bool check_target)
 {
 	struct page *page = pfn_to_online_page(pfn);
+	struct page *block_page;
 	struct page *end_page;
 	unsigned long block_pfn;
 
@@ -267,20 +268,26 @@ __reset_isolation_pfn(struct zone *zone, unsigned long pfn, bool check_source,
 	    get_pageblock_migratetype(page) != MIGRATE_MOVABLE)
 		return false;
 
+	/* Ensure the start of the pageblock or zone is online and valid */
+	block_pfn = pageblock_start_pfn(pfn);
+	block_page = pfn_to_online_page(max(block_pfn, zone->zone_start_pfn));
+	if (block_page) {
+		page = block_page;
+		pfn = block_pfn;
+	}
+
+	/* Ensure the end of the pageblock or zone is online and valid */
+	block_pfn += pageblock_nr_pages;
+	block_pfn = min(block_pfn, zone_end_pfn(zone) - 1);
+	end_page = pfn_to_online_page(block_pfn);
+	if (!end_page)
+		return false;
+
 	/*
 	 * Only clear the hint if a sample indicates there is either a
 	 * free page or an LRU page in the block. One or other condition
 	 * is necessary for the block to be a migration source/target.
 	 */
-	block_pfn = pageblock_start_pfn(pfn);
-	pfn = max(block_pfn, zone->zone_start_pfn);
-	page = pfn_to_page(pfn);
-	if (zone != page_zone(page))
-		return false;
-	pfn = block_pfn + pageblock_nr_pages;
-	pfn = min(pfn, zone_end_pfn(zone));
-	end_page = pfn_to_page(pfn);
-
 	do {
 		if (pfn_valid_within(pfn)) {
 			if (check_source && PageLRU(page)) {
@@ -309,7 +316,7 @@ __reset_isolation_pfn(struct zone *zone, unsigned long pfn, bool check_source,
 static void __reset_isolation_suitable(struct zone *zone)
 {
 	unsigned long migrate_pfn = zone->zone_start_pfn;
-	unsigned long free_pfn = zone_end_pfn(zone);
+	unsigned long free_pfn = zone_end_pfn(zone) - 1;
 	unsigned long reset_migrate = free_pfn;
 	unsigned long reset_free = migrate_pfn;
 	bool source_set = false;
-- 
2.16.4


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 2/2] mm/compaction.c: abort search if isolation fails
  2019-04-05 13:51 [GIT PULL] mm/compaction functional fixes for v5.1-rc4 Mel Gorman
  2019-04-05 13:51 ` [PATCH 1/2] mm/compaction.c: correct zone boundary handling when resetting pageblock skip hints Mel Gorman
@ 2019-04-05 13:51 ` Mel Gorman
  2019-04-05 16:15 ` [GIT PULL] mm/compaction functional fixes for v5.1-rc4 pr-tracker-bot
  2 siblings, 0 replies; 4+ messages in thread
From: Mel Gorman @ 2019-04-05 13:51 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Andrew Morton, Linux-MM, LKML, Mel Gorman

From: Qian Cai <cai@lca.pw>

Running LTP oom01 in a tight loop or memory stress testing put the system
in a low-memory situation could triggers random memory corruption like
page flag corruption below due to in fast_isolate_freepages(), if
isolation fails, next_search_order() does not abort the search immediately
could lead to improper accesses.

UBSAN: Undefined behaviour in ./include/linux/mm.h:1195:50
index 7 is out of range for type 'zone [5]'
Call Trace:
 dump_stack+0x62/0x9a
 ubsan_epilogue+0xd/0x7f
 __ubsan_handle_out_of_bounds+0x14d/0x192
 __isolate_free_page+0x52c/0x600
 compaction_alloc+0x886/0x25f0
 unmap_and_move+0x37/0x1e70
 migrate_pages+0x2ca/0xb20
 compact_zone+0x19cb/0x3620
 kcompactd_do_work+0x2df/0x680
 kcompactd+0x1d8/0x6c0
 kthread+0x32c/0x3f0
 ret_from_fork+0x35/0x40
------------[ cut here ]------------
kernel BUG at mm/page_alloc.c:3124!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC KASAN PTI
RIP: 0010:__isolate_free_page+0x464/0x600
RSP: 0000:ffff888b9e1af848 EFLAGS: 00010007
RAX: 0000000030000000 RBX: ffff888c39fcf0f8 RCX: 0000000000000000
RDX: 1ffff111873f9e25 RSI: 0000000000000004 RDI: ffffed1173c35ef6
RBP: ffff888b9e1af898 R08: fffffbfff4fc2461 R09: fffffbfff4fc2460
R10: fffffbfff4fc2460 R11: ffffffffa7e12303 R12: 0000000000000008
R13: dffffc0000000000 R14: 0000000000000000 R15: 0000000000000007
FS:  0000000000000000(0000) GS:ffff888ba8e80000(0000)
knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc7abc00000 CR3: 0000000752416004 CR4: 00000000001606a0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 compaction_alloc+0x886/0x25f0
 unmap_and_move+0x37/0x1e70
 migrate_pages+0x2ca/0xb20
 compact_zone+0x19cb/0x3620
 kcompactd_do_work+0x2df/0x680
 kcompactd+0x1d8/0x6c0
 kthread+0x32c/0x3f0
 ret_from_fork+0x35/0x40

Link: http://lkml.kernel.org/r/20190320192648.52499-1-cai@lca.pw
Fixes: dbe2d4e4f12e ("mm, compaction: round-robin the order while searching the free lists for a target")
Signed-off-by: Qian Cai <cai@lca.pw>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
 mm/compaction.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index b4930bf93c8a..3319e0872d01 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1370,7 +1370,7 @@ fast_isolate_freepages(struct compact_control *cc)
 				count_compact_events(COMPACTISOLATED, nr_isolated);
 			} else {
 				/* If isolation fails, abort the search */
-				order = -1;
+				order = cc->search_order + 1;
 				page = NULL;
 			}
 		}
-- 
2.16.4


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [GIT PULL] mm/compaction functional fixes for v5.1-rc4
  2019-04-05 13:51 [GIT PULL] mm/compaction functional fixes for v5.1-rc4 Mel Gorman
  2019-04-05 13:51 ` [PATCH 1/2] mm/compaction.c: correct zone boundary handling when resetting pageblock skip hints Mel Gorman
  2019-04-05 13:51 ` [PATCH 2/2] mm/compaction.c: abort search if isolation fails Mel Gorman
@ 2019-04-05 16:15 ` pr-tracker-bot
  2 siblings, 0 replies; 4+ messages in thread
From: pr-tracker-bot @ 2019-04-05 16:15 UTC (permalink / raw)
  To: Mel Gorman; +Cc: Linus Torvalds, Andrew Morton, Linux-MM, LKML, Mel Gorman

The pull request you sent on Fri,  5 Apr 2019 14:51:18 +0100:

> git://git.kernel.org/pub/scm/linux/kernel/git/mel/linux.git tags/mm-compaction-5.1-rc4

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/7f46774c6480174eb869a3c15167eafac467a6af

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2019-04-05 16:15 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-05 13:51 [GIT PULL] mm/compaction functional fixes for v5.1-rc4 Mel Gorman
2019-04-05 13:51 ` [PATCH 1/2] mm/compaction.c: correct zone boundary handling when resetting pageblock skip hints Mel Gorman
2019-04-05 13:51 ` [PATCH 2/2] mm/compaction.c: abort search if isolation fails Mel Gorman
2019-04-05 16:15 ` [GIT PULL] mm/compaction functional fixes for v5.1-rc4 pr-tracker-bot

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.