* [PATCH] mm: compaction: Check pfn_valid when entering a new MAX_ORDER_NR_PAGES block during isolation for migration
@ 2012-01-31 16:35 Mel Gorman
2012-01-31 20:40 ` Andrew Morton
0 siblings, 1 reply; 3+ messages in thread
From: Mel Gorman @ 2012-01-31 16:35 UTC (permalink / raw)
To: Andrew Morton; +Cc: Herbert van den Bergh, Michal Nazarewicz, Linux-MM, LKML
When isolating for migration, migration starts at the start of a zone
which is not necessarily pageblock aligned. Further, it stops isolating
when COMPACT_CLUSTER_MAX pages are isolated so migrate_pfn is generally
not aligned.
The problem is that pfn_valid is only called on the first PFN being
checked. Lets say we have a case like this
H = MAX_ORDER_NR_PAGES boundary
| = pageblock boundary
m = cc->migrate_pfn
f = cc->free_pfn
o = memory hole
H------|------H------|----m-Hoooooo|ooooooH-f----|------H
The migrate_pfn is just below a memory hole and the free scanner is
beyond the hole. When isolate_migratepages started, it scans from
migrate_pfn to migrate_pfn+pageblock_nr_pages which is now in a memory
hole. It checks pfn_valid() on the first PFN but then scans into the
hole where there are not necessarily valid struct pages.
This patch ensures that isolate_migratepages calls pfn_valid when
necessary.
Reported-and-tested-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Acked-by: Michal Nazarewicz <mina86@mina86.com>
Cc: stable@kernel.org
---
mm/compaction.c | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 899d956..edc1e26 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -313,6 +313,19 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
} else if (!locked)
spin_lock_irq(&zone->lru_lock);
+ /*
+ * migrate_pfn does not necessarily start aligned to a
+ * pageblock. Ensure that pfn_valid is called when moving
+ * into a new MAX_ORDER_NR_PAGES range in case of large
+ * memory holes within the zone
+ */
+ if ((low_pfn & (MAX_ORDER_NR_PAGES - 1)) == 0) {
+ if (!pfn_valid(low_pfn)) {
+ low_pfn += MAX_ORDER_NR_PAGES - 1;
+ continue;
+ }
+ }
+
if (!pfn_valid_within(low_pfn))
continue;
nr_scanned++;
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] mm: compaction: Check pfn_valid when entering a new MAX_ORDER_NR_PAGES block during isolation for migration
2012-01-31 16:35 [PATCH] mm: compaction: Check pfn_valid when entering a new MAX_ORDER_NR_PAGES block during isolation for migration Mel Gorman
@ 2012-01-31 20:40 ` Andrew Morton
2012-02-01 10:10 ` Mel Gorman
0 siblings, 1 reply; 3+ messages in thread
From: Andrew Morton @ 2012-01-31 20:40 UTC (permalink / raw)
To: Mel Gorman; +Cc: Herbert van den Bergh, Michal Nazarewicz, Linux-MM, LKML
On Tue, 31 Jan 2012 16:35:28 +0000
Mel Gorman <mgorman@suse.de> wrote:
> When isolating for migration, migration starts at the start of a zone
> which is not necessarily pageblock aligned. Further, it stops isolating
> when COMPACT_CLUSTER_MAX pages are isolated so migrate_pfn is generally
> not aligned.
>
> The problem is that pfn_valid is only called on the first PFN being
> checked. Lets say we have a case like this
>
> H = MAX_ORDER_NR_PAGES boundary
> | = pageblock boundary
> m = cc->migrate_pfn
> f = cc->free_pfn
> o = memory hole
>
> H------|------H------|----m-Hoooooo|ooooooH-f----|------H
>
> The migrate_pfn is just below a memory hole and the free scanner is
> beyond the hole. When isolate_migratepages started, it scans from
> migrate_pfn to migrate_pfn+pageblock_nr_pages which is now in a memory
> hole. It checks pfn_valid() on the first PFN but then scans into the
> hole where there are not necessarily valid struct pages.
>
> This patch ensures that isolate_migratepages calls pfn_valid when
> necessary.
>
> Reported-and-tested-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com>
> Signed-off-by: Mel Gorman <mgorman@suse.de>
> Acked-by: Michal Nazarewicz <mina86@mina86.com>
The changelog forgot to describe the user-visible effects of the bug.
> Cc: stable@kernel.org
So he (and others) will be confused.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] mm: compaction: Check pfn_valid when entering a new MAX_ORDER_NR_PAGES block during isolation for migration
2012-01-31 20:40 ` Andrew Morton
@ 2012-02-01 10:10 ` Mel Gorman
0 siblings, 0 replies; 3+ messages in thread
From: Mel Gorman @ 2012-02-01 10:10 UTC (permalink / raw)
To: Andrew Morton; +Cc: Herbert van den Bergh, Michal Nazarewicz, Linux-MM, LKML
On Tue, Jan 31, 2012 at 12:40:26PM -0800, Andrew Morton wrote:
> On Tue, 31 Jan 2012 16:35:28 +0000
> Mel Gorman <mgorman@suse.de> wrote:
>
> > When isolating for migration, migration starts at the start of a zone
> > which is not necessarily pageblock aligned. Further, it stops isolating
> > when COMPACT_CLUSTER_MAX pages are isolated so migrate_pfn is generally
> > not aligned.
> >
> > The problem is that pfn_valid is only called on the first PFN being
> > checked. Lets say we have a case like this
> >
> > H = MAX_ORDER_NR_PAGES boundary
> > | = pageblock boundary
> > m = cc->migrate_pfn
> > f = cc->free_pfn
> > o = memory hole
> >
> > H------|------H------|----m-Hoooooo|ooooooH-f----|------H
> >
> > The migrate_pfn is just below a memory hole and the free scanner is
> > beyond the hole. When isolate_migratepages started, it scans from
> > migrate_pfn to migrate_pfn+pageblock_nr_pages which is now in a memory
> > hole. It checks pfn_valid() on the first PFN but then scans into the
> > hole where there are not necessarily valid struct pages.
> >
> > This patch ensures that isolate_migratepages calls pfn_valid when
> > necessary.
> >
> > Reported-and-tested-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com>
> > Signed-off-by: Mel Gorman <mgorman@suse.de>
> > Acked-by: Michal Nazarewicz <mina86@mina86.com>
>
> The changelog forgot to describe the user-visible effects of the bug.
>
My bad. This changelog originally had a bugzilla number which I cut
out when reposting. Here is the same patch with a better changelog
=== CUT HERE ===
mm: compaction: Check pfn_valid when entering a new MAX_ORDER_NR_PAGES block during isolation for migration
When isolating for migration, migration starts at the start of a
zone which is not necessarily pageblock aligned. Further, it stops
isolating when COMPACT_CLUSTER_MAX pages are isolated so migrate_pfn
is generally not aligned. This allows isolate_migratepages() to call
pfn_to_page() on an invalid PFN which can result in a crash. This
was originally reported against a 3.0-based kernel with the following
trace in a crash dump.
PID: 9902 TASK: d47aecd0 CPU: 0 COMMAND: "memcg_process_s"
#0 [d72d3ad0] crash_kexec at c028cfdb
#1 [d72d3b24] oops_end at c05c5322
#2 [d72d3b38] __bad_area_nosemaphore at c0227e60
#3 [d72d3bec] bad_area at c0227fb6
#4 [d72d3c00] do_page_fault at c05c72ec
#5 [d72d3c80] error_code (via page_fault) at c05c47a4
EAX: 00000000 EBX: 000c0000 ECX: 00000001 EDX: 00000807 EBP: 000c0000
DS: 007b ESI: 00000001 ES: 007b EDI: f3000a80 GS: 6f50
CS: 0060 EIP: c030b15a ERR: ffffffff EFLAGS: 00010002
#6 [d72d3cb4] isolate_migratepages at c030b15a
#7 [d72d3d14] zone_watermark_ok at c02d26cb
#8 [d72d3d2c] compact_zone at c030b8de
#9 [d72d3d68] compact_zone_order at c030bba1
#10 [d72d3db4] try_to_compact_pages at c030bc84
#11 [d72d3ddc] __alloc_pages_direct_compact at c02d61e7
#12 [d72d3e08] __alloc_pages_slowpath at c02d66c7
#13 [d72d3e78] __alloc_pages_nodemask at c02d6a97
#14 [d72d3eb8] alloc_pages_vma at c030a845
#15 [d72d3ed4] do_huge_pmd_anonymous_page at c03178eb
#16 [d72d3f00] handle_mm_fault at c02f36c6
#17 [d72d3f30] do_page_fault at c05c70ed
#18 [d72d3fb0] error_code (via page_fault) at c05c47a4
EAX: b71ff000 EBX: 00000001 ECX: 00001600 EDX: 00000431
DS: 007b ESI: 08048950 ES: 007b EDI: bfaa3788
SS: 007b ESP: bfaa36e0 EBP: bfaa3828 GS: 6f50
CS: 0073 EIP: 080487c8 ERR: ffffffff EFLAGS: 00010202
It was also reported by Herbert van den Bergh against 3.1-based kernel
with the following snippit from the console log.
BUG: unable to handle kernel paging request at 01c00008
IP: [<c0522399>] isolate_migratepages+0x119/0x390
*pdpt = 000000002f7ce001 *pde = 0000000000000000
It is expected that it also affects 3.2.x and current mainline.
The problem is that pfn_valid is only called on the first PFN being
checked and that PFN is not necessarily aligned. Lets say we have a
case like this
H = MAX_ORDER_NR_PAGES boundary
| = pageblock boundary
m = cc->migrate_pfn
f = cc->free_pfn
o = memory hole
H------|------H------|----m-Hoooooo|ooooooH-f----|------H
The migrate_pfn is just below a memory hole and the free scanner is
beyond the hole. When isolate_migratepages started, it scans from
migrate_pfn to migrate_pfn+pageblock_nr_pages which is now in a memory
hole. It checks pfn_valid() on the first PFN but then scans into the
hole where there are not necessarily valid struct pages.
This patch ensures that isolate_migratepages calls pfn_valid when
necessary.
Reported-and-tested-by: Herbert van den Bergh <herbert.van.den.bergh@oracle.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Acked-by: Michal Nazarewicz <mina86@mina86.com>
Cc: stable <stable@vger.kernel.org>
---
mm/compaction.c | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 899d956..edc1e26 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -313,6 +313,19 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
} else if (!locked)
spin_lock_irq(&zone->lru_lock);
+ /*
+ * migrate_pfn does not necessarily start aligned to a
+ * pageblock. Ensure that pfn_valid is called when moving
+ * into a new MAX_ORDER_NR_PAGES range in case of large
+ * memory holes within the zone
+ */
+ if ((low_pfn & (MAX_ORDER_NR_PAGES - 1)) == 0) {
+ if (!pfn_valid(low_pfn)) {
+ low_pfn += MAX_ORDER_NR_PAGES - 1;
+ continue;
+ }
+ }
+
if (!pfn_valid_within(low_pfn))
continue;
nr_scanned++;
--
Mel Gorman
SUSE Labs
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2012-02-01 10:10 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-01-31 16:35 [PATCH] mm: compaction: Check pfn_valid when entering a new MAX_ORDER_NR_PAGES block during isolation for migration Mel Gorman
2012-01-31 20:40 ` Andrew Morton
2012-02-01 10:10 ` Mel Gorman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).