* [PATCH v2 0/5] compaction related commits
@ 2014-02-14 6:53 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:53 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
changes for v2
o include more experiment data in cover letter
o deal with vlastimil's comments mostly about commit description on 4/5
This patchset is related to the compaction.
patch 1 fixes contrary implementation of the purpose of compaction.
patch 2~4 are for optimization.
patch 5 is just for clean-up.
I tested this patchset with stress-highalloc benchmark on Mel's mmtest
and cannot find any regression in terms of success rate. And I find
much reduced(9%) elapsed time.
Below is the average result of 10 runs on my 4GB quad core system.
compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
compaction-fix+ has this patch series on top of compaction-base+.
Thanks.
stress-highalloc
3.13.0 3.13.0
compaction-base+ compaction-fix+
Success 1 14.10 15.00
Success 2 20.20 20.00
Success 3 68.30 73.40
3.13.0 3.13.0
compaction-base+ compaction-fix+
User 3486.02 3437.13
System 757.92 741.15
Elapsed 1638.52 1488.32
3.13.0 3.13.0
compaction-base+ compaction-fix+
Minor Faults 172591561 167116621
Major Faults 984 859
Swap Ins 743 653
Swap Outs 3657 3535
Direct pages scanned 129742 127344
Kswapd pages scanned 1852277 1817825
Kswapd pages reclaimed 1838000 1804212
Direct pages reclaimed 129719 127327
Kswapd efficiency 98% 98%
Kswapd velocity 1130.066 1221.296
Direct efficiency 99% 99%
Direct velocity 79.367 85.585
Percentage direct scans 6% 6%
Zone normal velocity 231.829 246.097
Zone dma32 velocity 972.589 1055.158
Zone dma velocity 5.015 5.626
Page writes by reclaim 6287 6534
Page writes file 2630 2999
Page writes anon 3657 3535
Page reclaim immediate 2187 2080
Sector Reads 2917808 2877336
Sector Writes 11477891 11206722
Page rescued immediate 0 0
Slabs scanned 2214118 2168524
Direct inode steals 12181 9788
Kswapd inode steals 144830 132109
Kswapd skipped wait 0 0
THP fault alloc 0 0
THP collapse alloc 0 0
THP splits 0 0
THP fault fallback 0 0
THP collapse fail 0 0
Compaction stalls 738 714
Compaction success 194 207
Compaction failures 543 507
Page migrate success 1806083 1464014
Page migrate failure 0 0
Compaction pages isolated 3873329 3162974
Compaction migrate scanned 74594862 59874420
Compaction free scanned 125888854 110868637
Compaction cost 2469 1998
Joonsoo Kim (5):
mm/compaction: disallow high-order page for migration target
mm/compaction: do not call suitable_migration_target() on every page
mm/compaction: change the timing to check to drop the spinlock
mm/compaction: check pageblock suitability once per pageblock
mm/compaction: clean-up code on success of ballon isolation
mm/compaction.c | 75 ++++++++++++++++++++++++++++---------------------------
1 file changed, 38 insertions(+), 37 deletions(-)
--
1.7.9.5
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v2 0/5] compaction related commits
@ 2014-02-14 6:53 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:53 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
changes for v2
o include more experiment data in cover letter
o deal with vlastimil's comments mostly about commit description on 4/5
This patchset is related to the compaction.
patch 1 fixes contrary implementation of the purpose of compaction.
patch 2~4 are for optimization.
patch 5 is just for clean-up.
I tested this patchset with stress-highalloc benchmark on Mel's mmtest
and cannot find any regression in terms of success rate. And I find
much reduced(9%) elapsed time.
Below is the average result of 10 runs on my 4GB quad core system.
compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
compaction-fix+ has this patch series on top of compaction-base+.
Thanks.
stress-highalloc
3.13.0 3.13.0
compaction-base+ compaction-fix+
Success 1 14.10 15.00
Success 2 20.20 20.00
Success 3 68.30 73.40
3.13.0 3.13.0
compaction-base+ compaction-fix+
User 3486.02 3437.13
System 757.92 741.15
Elapsed 1638.52 1488.32
3.13.0 3.13.0
compaction-base+ compaction-fix+
Minor Faults 172591561 167116621
Major Faults 984 859
Swap Ins 743 653
Swap Outs 3657 3535
Direct pages scanned 129742 127344
Kswapd pages scanned 1852277 1817825
Kswapd pages reclaimed 1838000 1804212
Direct pages reclaimed 129719 127327
Kswapd efficiency 98% 98%
Kswapd velocity 1130.066 1221.296
Direct efficiency 99% 99%
Direct velocity 79.367 85.585
Percentage direct scans 6% 6%
Zone normal velocity 231.829 246.097
Zone dma32 velocity 972.589 1055.158
Zone dma velocity 5.015 5.626
Page writes by reclaim 6287 6534
Page writes file 2630 2999
Page writes anon 3657 3535
Page reclaim immediate 2187 2080
Sector Reads 2917808 2877336
Sector Writes 11477891 11206722
Page rescued immediate 0 0
Slabs scanned 2214118 2168524
Direct inode steals 12181 9788
Kswapd inode steals 144830 132109
Kswapd skipped wait 0 0
THP fault alloc 0 0
THP collapse alloc 0 0
THP splits 0 0
THP fault fallback 0 0
THP collapse fail 0 0
Compaction stalls 738 714
Compaction success 194 207
Compaction failures 543 507
Page migrate success 1806083 1464014
Page migrate failure 0 0
Compaction pages isolated 3873329 3162974
Compaction migrate scanned 74594862 59874420
Compaction free scanned 125888854 110868637
Compaction cost 2469 1998
Joonsoo Kim (5):
mm/compaction: disallow high-order page for migration target
mm/compaction: do not call suitable_migration_target() on every page
mm/compaction: change the timing to check to drop the spinlock
mm/compaction: check pageblock suitability once per pageblock
mm/compaction: clean-up code on success of ballon isolation
mm/compaction.c | 75 ++++++++++++++++++++++++++++---------------------------
1 file changed, 38 insertions(+), 37 deletions(-)
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v2 1/5] mm/compaction: disallow high-order page for migration target
2014-02-14 6:53 ` Joonsoo Kim
@ 2014-02-14 6:53 ` Joonsoo Kim
-1 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:53 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
Purpose of compaction is to get a high order page. Currently, if we find
high-order page while searching migration target page, we break it to
order-0 pages and use them as migration target. It is contrary to purpose
of compaction, so disallow high-order page to be used for
migration target.
Additionally, clean-up logic in suitable_migration_target() to simplify
the code. There is no functional changes from this clean-up.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index 3a91a2e..bbe1260 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -217,21 +217,12 @@ static inline bool compact_trylock_irqsave(spinlock_t *lock,
/* Returns true if the page is within a block suitable for migration to */
static bool suitable_migration_target(struct page *page)
{
- int migratetype = get_pageblock_migratetype(page);
-
- /* Don't interfere with memory hot-remove or the min_free_kbytes blocks */
- if (migratetype == MIGRATE_RESERVE)
- return false;
-
- if (is_migrate_isolate(migratetype))
- return false;
-
- /* If the page is a large free page, then allow migration */
+ /* If the page is a large free page, then disallow migration */
if (PageBuddy(page) && page_order(page) >= pageblock_order)
- return true;
+ return false;
/* If the block is MIGRATE_MOVABLE or MIGRATE_CMA, allow migration */
- if (migrate_async_suitable(migratetype))
+ if (migrate_async_suitable(get_pageblock_migratetype(page)))
return true;
/* Otherwise skip the block */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 1/5] mm/compaction: disallow high-order page for migration target
@ 2014-02-14 6:53 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:53 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
Purpose of compaction is to get a high order page. Currently, if we find
high-order page while searching migration target page, we break it to
order-0 pages and use them as migration target. It is contrary to purpose
of compaction, so disallow high-order page to be used for
migration target.
Additionally, clean-up logic in suitable_migration_target() to simplify
the code. There is no functional changes from this clean-up.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index 3a91a2e..bbe1260 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -217,21 +217,12 @@ static inline bool compact_trylock_irqsave(spinlock_t *lock,
/* Returns true if the page is within a block suitable for migration to */
static bool suitable_migration_target(struct page *page)
{
- int migratetype = get_pageblock_migratetype(page);
-
- /* Don't interfere with memory hot-remove or the min_free_kbytes blocks */
- if (migratetype == MIGRATE_RESERVE)
- return false;
-
- if (is_migrate_isolate(migratetype))
- return false;
-
- /* If the page is a large free page, then allow migration */
+ /* If the page is a large free page, then disallow migration */
if (PageBuddy(page) && page_order(page) >= pageblock_order)
- return true;
+ return false;
/* If the block is MIGRATE_MOVABLE or MIGRATE_CMA, allow migration */
- if (migrate_async_suitable(migratetype))
+ if (migrate_async_suitable(get_pageblock_migratetype(page)))
return true;
/* Otherwise skip the block */
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 2/5] mm/compaction: do not call suitable_migration_target() on every page
2014-02-14 6:53 ` Joonsoo Kim
@ 2014-02-14 6:54 ` Joonsoo Kim
-1 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
suitable_migration_target() checks that pageblock is suitable for
migration target. In isolate_freepages_block(), it is called on every
page and this is inefficient. So make it called once per pageblock.
suitable_migration_target() also checks if page is highorder or not,
but it's criteria for highorder is pageblock order. So calling it once
within pageblock range has no problem.
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index bbe1260..0d821a2 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -245,6 +245,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
unsigned long nr_strict_required = end_pfn - blockpfn;
unsigned long flags;
bool locked = false;
+ bool checked_pageblock = false;
cursor = pfn_to_page(blockpfn);
@@ -275,8 +276,16 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
break;
/* Recheck this is a suitable migration target under lock */
- if (!strict && !suitable_migration_target(page))
- break;
+ if (!strict && !checked_pageblock) {
+ /*
+ * We need to check suitability of pageblock only once
+ * and this isolate_freepages_block() is called with
+ * pageblock range, so just check once is sufficient.
+ */
+ checked_pageblock = true;
+ if (!suitable_migration_target(page))
+ break;
+ }
/* Recheck this is a buddy page under lock */
if (!PageBuddy(page))
--
1.7.9.5
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 2/5] mm/compaction: do not call suitable_migration_target() on every page
@ 2014-02-14 6:54 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
suitable_migration_target() checks that pageblock is suitable for
migration target. In isolate_freepages_block(), it is called on every
page and this is inefficient. So make it called once per pageblock.
suitable_migration_target() also checks if page is highorder or not,
but it's criteria for highorder is pageblock order. So calling it once
within pageblock range has no problem.
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index bbe1260..0d821a2 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -245,6 +245,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
unsigned long nr_strict_required = end_pfn - blockpfn;
unsigned long flags;
bool locked = false;
+ bool checked_pageblock = false;
cursor = pfn_to_page(blockpfn);
@@ -275,8 +276,16 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
break;
/* Recheck this is a suitable migration target under lock */
- if (!strict && !suitable_migration_target(page))
- break;
+ if (!strict && !checked_pageblock) {
+ /*
+ * We need to check suitability of pageblock only once
+ * and this isolate_freepages_block() is called with
+ * pageblock range, so just check once is sufficient.
+ */
+ checked_pageblock = true;
+ if (!suitable_migration_target(page))
+ break;
+ }
/* Recheck this is a buddy page under lock */
if (!PageBuddy(page))
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 3/5] mm/compaction: change the timing to check to drop the spinlock
2014-02-14 6:53 ` Joonsoo Kim
@ 2014-02-14 6:54 ` Joonsoo Kim
-1 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
It is odd to drop the spinlock when we scan (SWAP_CLUSTER_MAX - 1) th pfn
page. This may results in below situation while isolating migratepage.
1. try isolate 0x0 ~ 0x200 pfn pages.
2. When low_pfn is 0x1ff, ((low_pfn+1) % SWAP_CLUSTER_MAX) == 0, so drop
the spinlock.
3. Then, to complete isolating, retry to aquire the lock.
I think that it is better to use SWAP_CLUSTER_MAX th pfn for checking
the criteria about dropping the lock. This has no harm 0x0 pfn, because,
at this time, locked variable would be false.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index 0d821a2..b1ba297 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -481,7 +481,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
cond_resched();
for (; low_pfn < end_pfn; low_pfn++) {
/* give a chance to irqs before checking need_resched() */
- if (locked && !((low_pfn+1) % SWAP_CLUSTER_MAX)) {
+ if (locked && !(low_pfn % SWAP_CLUSTER_MAX)) {
if (should_release_lock(&zone->lru_lock)) {
spin_unlock_irqrestore(&zone->lru_lock, flags);
locked = false;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 3/5] mm/compaction: change the timing to check to drop the spinlock
@ 2014-02-14 6:54 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
It is odd to drop the spinlock when we scan (SWAP_CLUSTER_MAX - 1) th pfn
page. This may results in below situation while isolating migratepage.
1. try isolate 0x0 ~ 0x200 pfn pages.
2. When low_pfn is 0x1ff, ((low_pfn+1) % SWAP_CLUSTER_MAX) == 0, so drop
the spinlock.
3. Then, to complete isolating, retry to aquire the lock.
I think that it is better to use SWAP_CLUSTER_MAX th pfn for checking
the criteria about dropping the lock. This has no harm 0x0 pfn, because,
at this time, locked variable would be false.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index 0d821a2..b1ba297 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -481,7 +481,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
cond_resched();
for (; low_pfn < end_pfn; low_pfn++) {
/* give a chance to irqs before checking need_resched() */
- if (locked && !((low_pfn+1) % SWAP_CLUSTER_MAX)) {
+ if (locked && !(low_pfn % SWAP_CLUSTER_MAX)) {
if (should_release_lock(&zone->lru_lock)) {
spin_unlock_irqrestore(&zone->lru_lock, flags);
locked = false;
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 4/5] mm/compaction: check pageblock suitability once per pageblock
2014-02-14 6:53 ` Joonsoo Kim
@ 2014-02-14 6:54 ` Joonsoo Kim
-1 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
isolation_suitable() and migrate_async_suitable() is used to be sure
that this pageblock range is fine to be migragted. It isn't needed to
call it on every page. Current code do well if not suitable, but, don't
do well when suitable.
1) It re-checks isolation_suitable() on each page of a pageblock that was
already estabilished as suitable.
2) It re-checks migrate_async_suitable() on each page of a pageblock that
was not entered through the next_pageblock: label, because
last_pageblock_nr is not otherwise updated.
This patch fixes situation by 1) calling isolation_suitable() only once
per pageblock and 2) always updating last_pageblock_nr to the pageblock
that was just checked.
Additionally, move PageBuddy() check after pageblock unit check,
since pageblock check is the first thing we should do and makes things
more simple.
[vbabka@suse.cz: rephrase commit description]
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index b1ba297..56536d3 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -520,26 +520,31 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
/* If isolation recently failed, do not retry */
pageblock_nr = low_pfn >> pageblock_order;
- if (!isolation_suitable(cc, page))
- goto next_pageblock;
+ if (last_pageblock_nr != pageblock_nr) {
+ int mt;
+
+ last_pageblock_nr = pageblock_nr;
+ if (!isolation_suitable(cc, page))
+ goto next_pageblock;
+
+ /*
+ * For async migration, also only scan in MOVABLE
+ * blocks. Async migration is optimistic to see if
+ * the minimum amount of work satisfies the allocation
+ */
+ mt = get_pageblock_migratetype(page);
+ if (!cc->sync && !migrate_async_suitable(mt)) {
+ cc->finished_update_migrate = true;
+ skipped_async_unsuitable = true;
+ goto next_pageblock;
+ }
+ }
/* Skip if free */
if (PageBuddy(page))
continue;
/*
- * For async migration, also only scan in MOVABLE blocks. Async
- * migration is optimistic to see if the minimum amount of work
- * satisfies the allocation
- */
- if (!cc->sync && last_pageblock_nr != pageblock_nr &&
- !migrate_async_suitable(get_pageblock_migratetype(page))) {
- cc->finished_update_migrate = true;
- skipped_async_unsuitable = true;
- goto next_pageblock;
- }
-
- /*
* Check may be lockless but that's ok as we recheck later.
* It's possible to migrate LRU pages and balloon pages
* Skip any other type of page
@@ -621,7 +626,6 @@ check_compact_cluster:
next_pageblock:
low_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages) - 1;
- last_pageblock_nr = pageblock_nr;
}
acct_isolated(zone, locked, cc);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 4/5] mm/compaction: check pageblock suitability once per pageblock
@ 2014-02-14 6:54 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
isolation_suitable() and migrate_async_suitable() is used to be sure
that this pageblock range is fine to be migragted. It isn't needed to
call it on every page. Current code do well if not suitable, but, don't
do well when suitable.
1) It re-checks isolation_suitable() on each page of a pageblock that was
already estabilished as suitable.
2) It re-checks migrate_async_suitable() on each page of a pageblock that
was not entered through the next_pageblock: label, because
last_pageblock_nr is not otherwise updated.
This patch fixes situation by 1) calling isolation_suitable() only once
per pageblock and 2) always updating last_pageblock_nr to the pageblock
that was just checked.
Additionally, move PageBuddy() check after pageblock unit check,
since pageblock check is the first thing we should do and makes things
more simple.
[vbabka@suse.cz: rephrase commit description]
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index b1ba297..56536d3 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -520,26 +520,31 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
/* If isolation recently failed, do not retry */
pageblock_nr = low_pfn >> pageblock_order;
- if (!isolation_suitable(cc, page))
- goto next_pageblock;
+ if (last_pageblock_nr != pageblock_nr) {
+ int mt;
+
+ last_pageblock_nr = pageblock_nr;
+ if (!isolation_suitable(cc, page))
+ goto next_pageblock;
+
+ /*
+ * For async migration, also only scan in MOVABLE
+ * blocks. Async migration is optimistic to see if
+ * the minimum amount of work satisfies the allocation
+ */
+ mt = get_pageblock_migratetype(page);
+ if (!cc->sync && !migrate_async_suitable(mt)) {
+ cc->finished_update_migrate = true;
+ skipped_async_unsuitable = true;
+ goto next_pageblock;
+ }
+ }
/* Skip if free */
if (PageBuddy(page))
continue;
/*
- * For async migration, also only scan in MOVABLE blocks. Async
- * migration is optimistic to see if the minimum amount of work
- * satisfies the allocation
- */
- if (!cc->sync && last_pageblock_nr != pageblock_nr &&
- !migrate_async_suitable(get_pageblock_migratetype(page))) {
- cc->finished_update_migrate = true;
- skipped_async_unsuitable = true;
- goto next_pageblock;
- }
-
- /*
* Check may be lockless but that's ok as we recheck later.
* It's possible to migrate LRU pages and balloon pages
* Skip any other type of page
@@ -621,7 +626,6 @@ check_compact_cluster:
next_pageblock:
low_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages) - 1;
- last_pageblock_nr = pageblock_nr;
}
acct_isolated(zone, locked, cc);
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 5/5] mm/compaction: clean-up code on success of ballon isolation
2014-02-14 6:53 ` Joonsoo Kim
@ 2014-02-14 6:54 ` Joonsoo Kim
-1 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
It is just for clean-up to reduce code size and improve readability.
There is no functional change.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index 56536d3..a1a9270 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -553,11 +553,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
if (unlikely(balloon_page_movable(page))) {
if (locked && balloon_page_isolate(page)) {
/* Successfully isolated */
- cc->finished_update_migrate = true;
- list_add(&page->lru, migratelist);
- cc->nr_migratepages++;
- nr_isolated++;
- goto check_compact_cluster;
+ goto isolate_success;
}
}
continue;
@@ -609,13 +605,14 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
VM_BUG_ON(PageTransCompound(page));
/* Successfully isolated */
- cc->finished_update_migrate = true;
del_page_from_lru_list(page, lruvec, page_lru(page));
+
+isolate_success:
+ cc->finished_update_migrate = true;
list_add(&page->lru, migratelist);
cc->nr_migratepages++;
nr_isolated++;
-check_compact_cluster:
/* Avoid isolating too much */
if (cc->nr_migratepages == COMPACT_CLUSTER_MAX) {
++low_pfn;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v2 5/5] mm/compaction: clean-up code on success of ballon isolation
@ 2014-02-14 6:54 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-02-14 6:54 UTC (permalink / raw)
To: Andrew Morton
Cc: Mel Gorman, Vlastimil Babka, Joonsoo Kim, Rik van Riel, linux-mm,
linux-kernel, Joonsoo Kim
It is just for clean-up to reduce code size and improve readability.
There is no functional change.
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
diff --git a/mm/compaction.c b/mm/compaction.c
index 56536d3..a1a9270 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -553,11 +553,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
if (unlikely(balloon_page_movable(page))) {
if (locked && balloon_page_isolate(page)) {
/* Successfully isolated */
- cc->finished_update_migrate = true;
- list_add(&page->lru, migratelist);
- cc->nr_migratepages++;
- nr_isolated++;
- goto check_compact_cluster;
+ goto isolate_success;
}
}
continue;
@@ -609,13 +605,14 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
VM_BUG_ON(PageTransCompound(page));
/* Successfully isolated */
- cc->finished_update_migrate = true;
del_page_from_lru_list(page, lruvec, page_lru(page));
+
+isolate_success:
+ cc->finished_update_migrate = true;
list_add(&page->lru, migratelist);
cc->nr_migratepages++;
nr_isolated++;
-check_compact_cluster:
/* Avoid isolating too much */
if (cc->nr_migratepages == COMPACT_CLUSTER_MAX) {
++low_pfn;
--
1.7.9.5
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v2 2/5] mm/compaction: do not call suitable_migration_target() on every page
2014-02-14 6:54 ` Joonsoo Kim
@ 2014-02-20 16:49 ` Vlastimil Babka
-1 siblings, 0 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-02-20 16:49 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Mel Gorman, Joonsoo Kim, Rik van Riel, linux-mm, linux-kernel
On 02/14/2014 07:54 AM, Joonsoo Kim wrote:
> suitable_migration_target() checks that pageblock is suitable for
> migration target. In isolate_freepages_block(), it is called on every
> page and this is inefficient. So make it called once per pageblock.
>
> suitable_migration_target() also checks if page is highorder or not,
> but it's criteria for highorder is pageblock order. So calling it once
> within pageblock range has no problem.
>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> diff --git a/mm/compaction.c b/mm/compaction.c
> index bbe1260..0d821a2 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -245,6 +245,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
> unsigned long nr_strict_required = end_pfn - blockpfn;
> unsigned long flags;
> bool locked = false;
> + bool checked_pageblock = false;
>
> cursor = pfn_to_page(blockpfn);
>
> @@ -275,8 +276,16 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
> break;
>
> /* Recheck this is a suitable migration target under lock */
> - if (!strict && !suitable_migration_target(page))
> - break;
> + if (!strict && !checked_pageblock) {
> + /*
> + * We need to check suitability of pageblock only once
> + * and this isolate_freepages_block() is called with
> + * pageblock range, so just check once is sufficient.
> + */
> + checked_pageblock = true;
> + if (!suitable_migration_target(page))
> + break;
> + }
>
> /* Recheck this is a buddy page under lock */
> if (!PageBuddy(page))
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 2/5] mm/compaction: do not call suitable_migration_target() on every page
@ 2014-02-20 16:49 ` Vlastimil Babka
0 siblings, 0 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-02-20 16:49 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Mel Gorman, Joonsoo Kim, Rik van Riel, linux-mm, linux-kernel
On 02/14/2014 07:54 AM, Joonsoo Kim wrote:
> suitable_migration_target() checks that pageblock is suitable for
> migration target. In isolate_freepages_block(), it is called on every
> page and this is inefficient. So make it called once per pageblock.
>
> suitable_migration_target() also checks if page is highorder or not,
> but it's criteria for highorder is pageblock order. So calling it once
> within pageblock range has no problem.
>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> diff --git a/mm/compaction.c b/mm/compaction.c
> index bbe1260..0d821a2 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -245,6 +245,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
> unsigned long nr_strict_required = end_pfn - blockpfn;
> unsigned long flags;
> bool locked = false;
> + bool checked_pageblock = false;
>
> cursor = pfn_to_page(blockpfn);
>
> @@ -275,8 +276,16 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
> break;
>
> /* Recheck this is a suitable migration target under lock */
> - if (!strict && !suitable_migration_target(page))
> - break;
> + if (!strict && !checked_pageblock) {
> + /*
> + * We need to check suitability of pageblock only once
> + * and this isolate_freepages_block() is called with
> + * pageblock range, so just check once is sufficient.
> + */
> + checked_pageblock = true;
> + if (!suitable_migration_target(page))
> + break;
> + }
>
> /* Recheck this is a buddy page under lock */
> if (!PageBuddy(page))
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/5] compaction related commits
2014-02-14 6:53 ` Joonsoo Kim
@ 2014-03-03 11:02 ` Vlastimil Babka
-1 siblings, 0 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-03-03 11:02 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Mel Gorman, Joonsoo Kim, Rik van Riel, linux-mm, linux-kernel
On 02/14/2014 07:53 AM, Joonsoo Kim wrote:
> changes for v2
> o include more experiment data in cover letter
> o deal with vlastimil's comments mostly about commit description on 4/5
>
> This patchset is related to the compaction.
>
> patch 1 fixes contrary implementation of the purpose of compaction.
> patch 2~4 are for optimization.
> patch 5 is just for clean-up.
>
> I tested this patchset with stress-highalloc benchmark on Mel's mmtest
> and cannot find any regression in terms of success rate. And I find
> much reduced(9%) elapsed time.
>
> Below is the average result of 10 runs on my 4GB quad core system.
>
> compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
> compaction-fix+ has this patch series on top of compaction-base+.
>
> Thanks.
>
>
> stress-highalloc
> 3.13.0 3.13.0
> compaction-base+ compaction-fix+
> Success 1 14.10 15.00
> Success 2 20.20 20.00
> Success 3 68.30 73.40
>
> 3.13.0 3.13.0
> compaction-base+ compaction-fix+
> User 3486.02 3437.13
> System 757.92 741.15
> Elapsed 1638.52 1488.32
>
> 3.13.0 3.13.0
> compaction-base+ compaction-fix+
> Minor Faults 172591561 167116621
> Major Faults 984 859
> Swap Ins 743 653
> Swap Outs 3657 3535
> Direct pages scanned 129742 127344
> Kswapd pages scanned 1852277 1817825
> Kswapd pages reclaimed 1838000 1804212
> Direct pages reclaimed 129719 127327
> Kswapd efficiency 98% 98%
> Kswapd velocity 1130.066 1221.296
> Direct efficiency 99% 99%
> Direct velocity 79.367 85.585
> Percentage direct scans 6% 6%
> Zone normal velocity 231.829 246.097
> Zone dma32 velocity 972.589 1055.158
> Zone dma velocity 5.015 5.626
> Page writes by reclaim 6287 6534
> Page writes file 2630 2999
> Page writes anon 3657 3535
> Page reclaim immediate 2187 2080
> Sector Reads 2917808 2877336
> Sector Writes 11477891 11206722
> Page rescued immediate 0 0
> Slabs scanned 2214118 2168524
> Direct inode steals 12181 9788
> Kswapd inode steals 144830 132109
> Kswapd skipped wait 0 0
> THP fault alloc 0 0
> THP collapse alloc 0 0
> THP splits 0 0
> THP fault fallback 0 0
> THP collapse fail 0 0
> Compaction stalls 738 714
> Compaction success 194 207
> Compaction failures 543 507
> Page migrate success 1806083 1464014
> Page migrate failure 0 0
> Compaction pages isolated 3873329 3162974
> Compaction migrate scanned 74594862 59874420
> Compaction free scanned 125888854 110868637
> Compaction cost 2469 1998
FWIW, I've let a machine run the series with individual patches applied
on 3.13 with my compaction patches, so 6 is the end of my series and 7-11 yours:
The average is of 10 runs (in case you wonder how that's done, the success rates are
calculated with a new R support that's pending Mel's merge; system time and vmstats
are currently a hack, but I hope to add R support for them as well, and maybe publish
to github or something if there's interest).
Interestingly, you have a much lower success rate and also much lower compaction cost
and, well, even the benchmark times. Wonder what difference in config or hw causes this.
You seem to have THP disabled, I enabled, but that would be weird to cause this.
Anyway, it seems nothing statistically significant happens here. But the patches make
sense so this is not an argument against them :)
3.13 3.13 3.13 3.13 3.13 3.13
6-nothp 7-nothp 8-nothp 9-nothp 10-nothp 11-nothp
Success 1 Min 30.00 ( 0.00%) 25.00 ( 16.67%) 46.00 (-53.33%) 41.00 (-36.67%) 25.00 ( 16.67%) 44.00 (-46.67%)
Success 1 Mean 45.50 ( 0.00%) 45.20 ( 0.66%) 47.30 ( -3.96%) 46.20 ( -1.54%) 44.60 ( 1.98%) 46.70 ( -2.64%)
Success 1 Max 52.00 ( 0.00%) 52.00 ( 0.00%) 50.00 ( 3.85%) 49.00 ( 5.77%) 53.00 ( -1.92%) 49.00 ( 5.77%)
Success 2 Min 36.00 ( 0.00%) 30.00 ( 16.67%) 46.00 (-27.78%) 42.00 (-16.67%) 25.00 ( 30.56%) 46.00 (-27.78%)
Success 2 Mean 47.70 ( 0.00%) 47.20 ( 1.05%) 49.00 ( -2.73%) 47.80 ( -0.21%) 45.60 ( 4.40%) 48.60 ( -1.89%)
Success 2 Max 56.00 ( 0.00%) 55.00 ( 1.79%) 52.00 ( 7.14%) 51.00 ( 8.93%) 53.00 ( 5.36%) 52.00 ( 7.14%)
Success 3 Min 84.00 ( 0.00%) 84.00 ( 0.00%) 84.00 ( 0.00%) 84.00 ( 0.00%) 83.00 ( 1.19%) 84.00 ( 0.00%)
Success 3 Mean 84.30 ( 0.00%) 85.00 ( -0.83%) 85.40 ( -1.30%) 84.90 ( -0.71%) 84.50 ( -0.24%) 84.80 ( -0.59%)
Success 3 Max 85.00 ( 0.00%) 86.00 ( -1.18%) 87.00 ( -2.35%) 86.00 ( -1.18%) 85.00 ( 0.00%) 85.00 ( 0.00%)
3.13 3.13 3.13 3.13 3.13 3.13
6-nothp 7-nothp 8-nothp 9-nothp 10-nothp 11-nothp
User 6059.77 6161.19 6037.71 6145.80 6157.28 6180.10
System 1034.15 1040.78 1034.66 1037.29 1034.39 1038.79
Elapsed 2132.04 2141.36 2149.86 2128.43 2110.11 2126.12
3.13 3.13 3.13 3.13 3.13 3.13
6-nothp 7-nothp 8-nothp 9-nothp 10-nothp 11-nothp
Minor Faults 252919751 253905255 252187591 253325723 252655233 253489596
Major Faults 637 618 626 619 619 622
Swap Ins 6 4 7 6 10 9
Swap Outs 322 404 339 334 393 367
Direct pages scanned 287033 285148 276390 279039 271468 285491
Kswapd pages scanned 1891892 1893049 1912150 1897328 1890594 1901914
Kswapd pages reclaimed 1889422 1890396 1909624 1894755 1888005 1899281
Direct pages reclaimed 286843 284965 276160 278852 271262 285257
Kswapd efficiency 99% 99% 99% 99% 99% 99%
Kswapd velocity 896.585 883.055 876.040 875.818 893.235 895.381
Direct efficiency 99% 99% 99% 99% 99% 99%
Direct velocity 136.028 133.014 126.626 128.806 128.259 134.403
Percentage direct scans 13% 13% 12% 12% 12% 13%
Zone normal velocity 321.232 316.821 310.244 310.490 320.010 315.868
Zone dma32 velocity 711.380 699.247 692.423 694.134 701.483 713.917
Zone dma velocity 0.000 0.000 0.000 0.000 0.000 0.000
Page writes by reclaim 340.600 440.800 381.300 377.300 429.400 452.400
Page writes file 17 36 41 42 35 84
Page writes anon 322 404 339 334 393 367
Page reclaim immediate 383 470 435 453 491 477
Sector Reads 2984072 3001944 3009073 2985139 2983261 2994914
Sector Writes 12175402 12179276 12145946 12116400 12108622 12113503
Page rescued immediate 0 0 0 0 0 0
Slabs scanned 1773696 1767654 1788134 1774720 1751065 1786291
Direct inode steals 15265 16402 16089 14913 14130 16178
Kswapd inode steals 52119 51160 51969 52261 50276 51807
Kswapd skipped wait 0 0 0 0 0 0
THP fault alloc 99 104 93 96 90 90
THP collapse alloc 559 560 592 585 531 608
THP splits 7 6 5 5 5 5
THP fault fallback 2 0 0 0 0 0
THP collapse fail 12 12 12 12 13 11
Compaction stalls 1528 1574 1613 1580 1507 1537
Compaction success 552 571 594 574 554 579
Compaction failures 975 1003 1018 1006 953 958
Page migrate success 3913572 4071734 4016156 3981713 3944844 3846041
Page migrate failure 0 0 0 0 0 0
Compaction pages isolated 8256377 8608215 8489839 8395767 8316197 8135089
Compaction migrate scanned 155216300 161068938 165093536 158926355 157504384 157996946
Compaction free scanned 348489189 367790502 363524078 358476708 354952212 340442016
Compaction cost 5305 5517 5485 5405 5355 5252
NUMA alloc hit 170245991 170939361 169815871 170557197 170160298 170719930
NUMA alloc miss 0 0 0 0 0 0
NUMA interleave hit 0 0 0 0 0 0
NUMA alloc local 170245991 170939361 169815871 170557197 170160298 170719930
NUMA page range updates 0 0 0 0 0 0
NUMA huge PMD updates 0 0 0 0 0 0
NUMA PTE updates 0 0 0 0 0 0
NUMA hint faults 0 0 0 0 0 0
NUMA hint local faults 0 0 0 0 0 0
NUMA hint local percent 100 100 100 100 100 100
NUMA pages migrated 0 0 0 0 0 0
AutoNUMA cost 0 0 0 0 0 0
>
> Joonsoo Kim (5):
> mm/compaction: disallow high-order page for migration target
> mm/compaction: do not call suitable_migration_target() on every page
> mm/compaction: change the timing to check to drop the spinlock
> mm/compaction: check pageblock suitability once per pageblock
> mm/compaction: clean-up code on success of ballon isolation
>
> mm/compaction.c | 75 ++++++++++++++++++++++++++++---------------------------
> 1 file changed, 38 insertions(+), 37 deletions(-)
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/5] compaction related commits
@ 2014-03-03 11:02 ` Vlastimil Babka
0 siblings, 0 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-03-03 11:02 UTC (permalink / raw)
To: Joonsoo Kim, Andrew Morton
Cc: Mel Gorman, Joonsoo Kim, Rik van Riel, linux-mm, linux-kernel
On 02/14/2014 07:53 AM, Joonsoo Kim wrote:
> changes for v2
> o include more experiment data in cover letter
> o deal with vlastimil's comments mostly about commit description on 4/5
>
> This patchset is related to the compaction.
>
> patch 1 fixes contrary implementation of the purpose of compaction.
> patch 2~4 are for optimization.
> patch 5 is just for clean-up.
>
> I tested this patchset with stress-highalloc benchmark on Mel's mmtest
> and cannot find any regression in terms of success rate. And I find
> much reduced(9%) elapsed time.
>
> Below is the average result of 10 runs on my 4GB quad core system.
>
> compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
> compaction-fix+ has this patch series on top of compaction-base+.
>
> Thanks.
>
>
> stress-highalloc
> 3.13.0 3.13.0
> compaction-base+ compaction-fix+
> Success 1 14.10 15.00
> Success 2 20.20 20.00
> Success 3 68.30 73.40
>
> 3.13.0 3.13.0
> compaction-base+ compaction-fix+
> User 3486.02 3437.13
> System 757.92 741.15
> Elapsed 1638.52 1488.32
>
> 3.13.0 3.13.0
> compaction-base+ compaction-fix+
> Minor Faults 172591561 167116621
> Major Faults 984 859
> Swap Ins 743 653
> Swap Outs 3657 3535
> Direct pages scanned 129742 127344
> Kswapd pages scanned 1852277 1817825
> Kswapd pages reclaimed 1838000 1804212
> Direct pages reclaimed 129719 127327
> Kswapd efficiency 98% 98%
> Kswapd velocity 1130.066 1221.296
> Direct efficiency 99% 99%
> Direct velocity 79.367 85.585
> Percentage direct scans 6% 6%
> Zone normal velocity 231.829 246.097
> Zone dma32 velocity 972.589 1055.158
> Zone dma velocity 5.015 5.626
> Page writes by reclaim 6287 6534
> Page writes file 2630 2999
> Page writes anon 3657 3535
> Page reclaim immediate 2187 2080
> Sector Reads 2917808 2877336
> Sector Writes 11477891 11206722
> Page rescued immediate 0 0
> Slabs scanned 2214118 2168524
> Direct inode steals 12181 9788
> Kswapd inode steals 144830 132109
> Kswapd skipped wait 0 0
> THP fault alloc 0 0
> THP collapse alloc 0 0
> THP splits 0 0
> THP fault fallback 0 0
> THP collapse fail 0 0
> Compaction stalls 738 714
> Compaction success 194 207
> Compaction failures 543 507
> Page migrate success 1806083 1464014
> Page migrate failure 0 0
> Compaction pages isolated 3873329 3162974
> Compaction migrate scanned 74594862 59874420
> Compaction free scanned 125888854 110868637
> Compaction cost 2469 1998
FWIW, I've let a machine run the series with individual patches applied
on 3.13 with my compaction patches, so 6 is the end of my series and 7-11 yours:
The average is of 10 runs (in case you wonder how that's done, the success rates are
calculated with a new R support that's pending Mel's merge; system time and vmstats
are currently a hack, but I hope to add R support for them as well, and maybe publish
to github or something if there's interest).
Interestingly, you have a much lower success rate and also much lower compaction cost
and, well, even the benchmark times. Wonder what difference in config or hw causes this.
You seem to have THP disabled, I enabled, but that would be weird to cause this.
Anyway, it seems nothing statistically significant happens here. But the patches make
sense so this is not an argument against them :)
3.13 3.13 3.13 3.13 3.13 3.13
6-nothp 7-nothp 8-nothp 9-nothp 10-nothp 11-nothp
Success 1 Min 30.00 ( 0.00%) 25.00 ( 16.67%) 46.00 (-53.33%) 41.00 (-36.67%) 25.00 ( 16.67%) 44.00 (-46.67%)
Success 1 Mean 45.50 ( 0.00%) 45.20 ( 0.66%) 47.30 ( -3.96%) 46.20 ( -1.54%) 44.60 ( 1.98%) 46.70 ( -2.64%)
Success 1 Max 52.00 ( 0.00%) 52.00 ( 0.00%) 50.00 ( 3.85%) 49.00 ( 5.77%) 53.00 ( -1.92%) 49.00 ( 5.77%)
Success 2 Min 36.00 ( 0.00%) 30.00 ( 16.67%) 46.00 (-27.78%) 42.00 (-16.67%) 25.00 ( 30.56%) 46.00 (-27.78%)
Success 2 Mean 47.70 ( 0.00%) 47.20 ( 1.05%) 49.00 ( -2.73%) 47.80 ( -0.21%) 45.60 ( 4.40%) 48.60 ( -1.89%)
Success 2 Max 56.00 ( 0.00%) 55.00 ( 1.79%) 52.00 ( 7.14%) 51.00 ( 8.93%) 53.00 ( 5.36%) 52.00 ( 7.14%)
Success 3 Min 84.00 ( 0.00%) 84.00 ( 0.00%) 84.00 ( 0.00%) 84.00 ( 0.00%) 83.00 ( 1.19%) 84.00 ( 0.00%)
Success 3 Mean 84.30 ( 0.00%) 85.00 ( -0.83%) 85.40 ( -1.30%) 84.90 ( -0.71%) 84.50 ( -0.24%) 84.80 ( -0.59%)
Success 3 Max 85.00 ( 0.00%) 86.00 ( -1.18%) 87.00 ( -2.35%) 86.00 ( -1.18%) 85.00 ( 0.00%) 85.00 ( 0.00%)
3.13 3.13 3.13 3.13 3.13 3.13
6-nothp 7-nothp 8-nothp 9-nothp 10-nothp 11-nothp
User 6059.77 6161.19 6037.71 6145.80 6157.28 6180.10
System 1034.15 1040.78 1034.66 1037.29 1034.39 1038.79
Elapsed 2132.04 2141.36 2149.86 2128.43 2110.11 2126.12
3.13 3.13 3.13 3.13 3.13 3.13
6-nothp 7-nothp 8-nothp 9-nothp 10-nothp 11-nothp
Minor Faults 252919751 253905255 252187591 253325723 252655233 253489596
Major Faults 637 618 626 619 619 622
Swap Ins 6 4 7 6 10 9
Swap Outs 322 404 339 334 393 367
Direct pages scanned 287033 285148 276390 279039 271468 285491
Kswapd pages scanned 1891892 1893049 1912150 1897328 1890594 1901914
Kswapd pages reclaimed 1889422 1890396 1909624 1894755 1888005 1899281
Direct pages reclaimed 286843 284965 276160 278852 271262 285257
Kswapd efficiency 99% 99% 99% 99% 99% 99%
Kswapd velocity 896.585 883.055 876.040 875.818 893.235 895.381
Direct efficiency 99% 99% 99% 99% 99% 99%
Direct velocity 136.028 133.014 126.626 128.806 128.259 134.403
Percentage direct scans 13% 13% 12% 12% 12% 13%
Zone normal velocity 321.232 316.821 310.244 310.490 320.010 315.868
Zone dma32 velocity 711.380 699.247 692.423 694.134 701.483 713.917
Zone dma velocity 0.000 0.000 0.000 0.000 0.000 0.000
Page writes by reclaim 340.600 440.800 381.300 377.300 429.400 452.400
Page writes file 17 36 41 42 35 84
Page writes anon 322 404 339 334 393 367
Page reclaim immediate 383 470 435 453 491 477
Sector Reads 2984072 3001944 3009073 2985139 2983261 2994914
Sector Writes 12175402 12179276 12145946 12116400 12108622 12113503
Page rescued immediate 0 0 0 0 0 0
Slabs scanned 1773696 1767654 1788134 1774720 1751065 1786291
Direct inode steals 15265 16402 16089 14913 14130 16178
Kswapd inode steals 52119 51160 51969 52261 50276 51807
Kswapd skipped wait 0 0 0 0 0 0
THP fault alloc 99 104 93 96 90 90
THP collapse alloc 559 560 592 585 531 608
THP splits 7 6 5 5 5 5
THP fault fallback 2 0 0 0 0 0
THP collapse fail 12 12 12 12 13 11
Compaction stalls 1528 1574 1613 1580 1507 1537
Compaction success 552 571 594 574 554 579
Compaction failures 975 1003 1018 1006 953 958
Page migrate success 3913572 4071734 4016156 3981713 3944844 3846041
Page migrate failure 0 0 0 0 0 0
Compaction pages isolated 8256377 8608215 8489839 8395767 8316197 8135089
Compaction migrate scanned 155216300 161068938 165093536 158926355 157504384 157996946
Compaction free scanned 348489189 367790502 363524078 358476708 354952212 340442016
Compaction cost 5305 5517 5485 5405 5355 5252
NUMA alloc hit 170245991 170939361 169815871 170557197 170160298 170719930
NUMA alloc miss 0 0 0 0 0 0
NUMA interleave hit 0 0 0 0 0 0
NUMA alloc local 170245991 170939361 169815871 170557197 170160298 170719930
NUMA page range updates 0 0 0 0 0 0
NUMA huge PMD updates 0 0 0 0 0 0
NUMA PTE updates 0 0 0 0 0 0
NUMA hint faults 0 0 0 0 0 0
NUMA hint local faults 0 0 0 0 0 0
NUMA hint local percent 100 100 100 100 100 100
NUMA pages migrated 0 0 0 0 0 0
AutoNUMA cost 0 0 0 0 0 0
>
> Joonsoo Kim (5):
> mm/compaction: disallow high-order page for migration target
> mm/compaction: do not call suitable_migration_target() on every page
> mm/compaction: change the timing to check to drop the spinlock
> mm/compaction: check pageblock suitability once per pageblock
> mm/compaction: clean-up code on success of ballon isolation
>
> mm/compaction.c | 75 ++++++++++++++++++++++++++++---------------------------
> 1 file changed, 38 insertions(+), 37 deletions(-)
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/5] compaction related commits
2014-03-03 11:02 ` Vlastimil Babka
@ 2014-03-04 0:23 ` Joonsoo Kim
-1 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-03-04 0:23 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, Mel Gorman, Rik van Riel, linux-mm, linux-kernel
On Mon, Mar 03, 2014 at 12:02:00PM +0100, Vlastimil Babka wrote:
> On 02/14/2014 07:53 AM, Joonsoo Kim wrote:
> > changes for v2
> > o include more experiment data in cover letter
> > o deal with vlastimil's comments mostly about commit description on 4/5
> >
> > This patchset is related to the compaction.
> >
> > patch 1 fixes contrary implementation of the purpose of compaction.
> > patch 2~4 are for optimization.
> > patch 5 is just for clean-up.
> >
> > I tested this patchset with stress-highalloc benchmark on Mel's mmtest
> > and cannot find any regression in terms of success rate. And I find
> > much reduced(9%) elapsed time.
> >
> > Below is the average result of 10 runs on my 4GB quad core system.
> >
> > compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
> > compaction-fix+ has this patch series on top of compaction-base+.
> >
> > Thanks.
> >
> >
> > stress-highalloc
> > 3.13.0 3.13.0
> > compaction-base+ compaction-fix+
> > Success 1 14.10 15.00
> > Success 2 20.20 20.00
> > Success 3 68.30 73.40
> >
> > 3.13.0 3.13.0
> > compaction-base+ compaction-fix+
> > User 3486.02 3437.13
> > System 757.92 741.15
> > Elapsed 1638.52 1488.32
> >
> > 3.13.0 3.13.0
> > compaction-base+ compaction-fix+
> > Minor Faults 172591561 167116621
> > Major Faults 984 859
> > Swap Ins 743 653
> > Swap Outs 3657 3535
> > Direct pages scanned 129742 127344
> > Kswapd pages scanned 1852277 1817825
> > Kswapd pages reclaimed 1838000 1804212
> > Direct pages reclaimed 129719 127327
> > Kswapd efficiency 98% 98%
> > Kswapd velocity 1130.066 1221.296
> > Direct efficiency 99% 99%
> > Direct velocity 79.367 85.585
> > Percentage direct scans 6% 6%
> > Zone normal velocity 231.829 246.097
> > Zone dma32 velocity 972.589 1055.158
> > Zone dma velocity 5.015 5.626
> > Page writes by reclaim 6287 6534
> > Page writes file 2630 2999
> > Page writes anon 3657 3535
> > Page reclaim immediate 2187 2080
> > Sector Reads 2917808 2877336
> > Sector Writes 11477891 11206722
> > Page rescued immediate 0 0
> > Slabs scanned 2214118 2168524
> > Direct inode steals 12181 9788
> > Kswapd inode steals 144830 132109
> > Kswapd skipped wait 0 0
> > THP fault alloc 0 0
> > THP collapse alloc 0 0
> > THP splits 0 0
> > THP fault fallback 0 0
> > THP collapse fail 0 0
> > Compaction stalls 738 714
> > Compaction success 194 207
> > Compaction failures 543 507
> > Page migrate success 1806083 1464014
> > Page migrate failure 0 0
> > Compaction pages isolated 3873329 3162974
> > Compaction migrate scanned 74594862 59874420
> > Compaction free scanned 125888854 110868637
> > Compaction cost 2469 1998
>
> FWIW, I've let a machine run the series with individual patches applied
> on 3.13 with my compaction patches, so 6 is the end of my series and 7-11 yours:
> The average is of 10 runs (in case you wonder how that's done, the success rates are
> calculated with a new R support that's pending Mel's merge; system time and vmstats
> are currently a hack, but I hope to add R support for them as well, and maybe publish
> to github or something if there's interest).
Good! I have an interest on it.
>
> Interestingly, you have a much lower success rate and also much lower compaction cost
> and, well, even the benchmark times. Wonder what difference in config or hw causes this.
> You seem to have THP disabled, I enabled, but that would be weird to cause this.
My 10 runs are continuous 10 runs without reboot. It makes compaction success
rate decline on every trial and therefore average result is so low than yours.
I heard that you did 10 runs because of large stdev, so I thought that continuous 10 runs
also can makes the result reliable. Therefore I decided this method although it is not
proper method to get the average. I had to notify about it. If it confuses you,
sorry about that.
Anyway, noticeable point of continuous 10 runs is that success rate decrease continuously
and significantly. I attach the rate of success 3 on every trial on below.
Base
% Success: 80
% Success: 60
% Success: 76
% Success: 74
% Success: 70
% Success: 68
% Success: 66
% Success: 65
% Success: 63
% Success: 61
Applied with my patches
% Success: 81
% Success: 78
% Success: 75
% Success: 74
% Success: 71
% Success: 72
% Success: 73
% Success: 70
% Success: 70
% Success: 70
It means that memory is fragmented continously. I didn't dig into this problem, but
it would be good subject to investigate.
Thanks.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/5] compaction related commits
@ 2014-03-04 0:23 ` Joonsoo Kim
0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-03-04 0:23 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, Mel Gorman, Rik van Riel, linux-mm, linux-kernel
On Mon, Mar 03, 2014 at 12:02:00PM +0100, Vlastimil Babka wrote:
> On 02/14/2014 07:53 AM, Joonsoo Kim wrote:
> > changes for v2
> > o include more experiment data in cover letter
> > o deal with vlastimil's comments mostly about commit description on 4/5
> >
> > This patchset is related to the compaction.
> >
> > patch 1 fixes contrary implementation of the purpose of compaction.
> > patch 2~4 are for optimization.
> > patch 5 is just for clean-up.
> >
> > I tested this patchset with stress-highalloc benchmark on Mel's mmtest
> > and cannot find any regression in terms of success rate. And I find
> > much reduced(9%) elapsed time.
> >
> > Below is the average result of 10 runs on my 4GB quad core system.
> >
> > compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
> > compaction-fix+ has this patch series on top of compaction-base+.
> >
> > Thanks.
> >
> >
> > stress-highalloc
> > 3.13.0 3.13.0
> > compaction-base+ compaction-fix+
> > Success 1 14.10 15.00
> > Success 2 20.20 20.00
> > Success 3 68.30 73.40
> >
> > 3.13.0 3.13.0
> > compaction-base+ compaction-fix+
> > User 3486.02 3437.13
> > System 757.92 741.15
> > Elapsed 1638.52 1488.32
> >
> > 3.13.0 3.13.0
> > compaction-base+ compaction-fix+
> > Minor Faults 172591561 167116621
> > Major Faults 984 859
> > Swap Ins 743 653
> > Swap Outs 3657 3535
> > Direct pages scanned 129742 127344
> > Kswapd pages scanned 1852277 1817825
> > Kswapd pages reclaimed 1838000 1804212
> > Direct pages reclaimed 129719 127327
> > Kswapd efficiency 98% 98%
> > Kswapd velocity 1130.066 1221.296
> > Direct efficiency 99% 99%
> > Direct velocity 79.367 85.585
> > Percentage direct scans 6% 6%
> > Zone normal velocity 231.829 246.097
> > Zone dma32 velocity 972.589 1055.158
> > Zone dma velocity 5.015 5.626
> > Page writes by reclaim 6287 6534
> > Page writes file 2630 2999
> > Page writes anon 3657 3535
> > Page reclaim immediate 2187 2080
> > Sector Reads 2917808 2877336
> > Sector Writes 11477891 11206722
> > Page rescued immediate 0 0
> > Slabs scanned 2214118 2168524
> > Direct inode steals 12181 9788
> > Kswapd inode steals 144830 132109
> > Kswapd skipped wait 0 0
> > THP fault alloc 0 0
> > THP collapse alloc 0 0
> > THP splits 0 0
> > THP fault fallback 0 0
> > THP collapse fail 0 0
> > Compaction stalls 738 714
> > Compaction success 194 207
> > Compaction failures 543 507
> > Page migrate success 1806083 1464014
> > Page migrate failure 0 0
> > Compaction pages isolated 3873329 3162974
> > Compaction migrate scanned 74594862 59874420
> > Compaction free scanned 125888854 110868637
> > Compaction cost 2469 1998
>
> FWIW, I've let a machine run the series with individual patches applied
> on 3.13 with my compaction patches, so 6 is the end of my series and 7-11 yours:
> The average is of 10 runs (in case you wonder how that's done, the success rates are
> calculated with a new R support that's pending Mel's merge; system time and vmstats
> are currently a hack, but I hope to add R support for them as well, and maybe publish
> to github or something if there's interest).
Good! I have an interest on it.
>
> Interestingly, you have a much lower success rate and also much lower compaction cost
> and, well, even the benchmark times. Wonder what difference in config or hw causes this.
> You seem to have THP disabled, I enabled, but that would be weird to cause this.
My 10 runs are continuous 10 runs without reboot. It makes compaction success
rate decline on every trial and therefore average result is so low than yours.
I heard that you did 10 runs because of large stdev, so I thought that continuous 10 runs
also can makes the result reliable. Therefore I decided this method although it is not
proper method to get the average. I had to notify about it. If it confuses you,
sorry about that.
Anyway, noticeable point of continuous 10 runs is that success rate decrease continuously
and significantly. I attach the rate of success 3 on every trial on below.
Base
% Success: 80
% Success: 60
% Success: 76
% Success: 74
% Success: 70
% Success: 68
% Success: 66
% Success: 65
% Success: 63
% Success: 61
Applied with my patches
% Success: 81
% Success: 78
% Success: 75
% Success: 74
% Success: 71
% Success: 72
% Success: 73
% Success: 70
% Success: 70
% Success: 70
It means that memory is fragmented continously. I didn't dig into this problem, but
it would be good subject to investigate.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/5] compaction related commits
2014-03-04 0:23 ` Joonsoo Kim
@ 2014-03-19 9:38 ` Vlastimil Babka
-1 siblings, 0 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-03-19 9:38 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Mel Gorman, Rik van Riel, linux-mm, linux-kernel
On 03/04/2014 01:23 AM, Joonsoo Kim wrote:
> On Mon, Mar 03, 2014 at 12:02:00PM +0100, Vlastimil Babka wrote:
>> On 02/14/2014 07:53 AM, Joonsoo Kim wrote:
>>> changes for v2
>>> o include more experiment data in cover letter
>>> o deal with vlastimil's comments mostly about commit description on 4/5
>>>
>>> This patchset is related to the compaction.
>>>
>>> patch 1 fixes contrary implementation of the purpose of compaction.
>>> patch 2~4 are for optimization.
>>> patch 5 is just for clean-up.
>>>
>>> I tested this patchset with stress-highalloc benchmark on Mel's mmtest
>>> and cannot find any regression in terms of success rate. And I find
>>> much reduced(9%) elapsed time.
>>>
>>> Below is the average result of 10 runs on my 4GB quad core system.
>>>
>>> compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
>>> compaction-fix+ has this patch series on top of compaction-base+.
>>>
>>> Thanks.
>>>
>>>
>>> stress-highalloc
>>> 3.13.0 3.13.0
>>> compaction-base+ compaction-fix+
>>> Success 1 14.10 15.00
>>> Success 2 20.20 20.00
>>> Success 3 68.30 73.40
>>>
>>> 3.13.0 3.13.0
>>> compaction-base+ compaction-fix+
>>> User 3486.02 3437.13
>>> System 757.92 741.15
>>> Elapsed 1638.52 1488.32
>>>
>>> 3.13.0 3.13.0
>>> compaction-base+ compaction-fix+
>>> Minor Faults 172591561 167116621
>>> Major Faults 984 859
>>> Swap Ins 743 653
>>> Swap Outs 3657 3535
>>> Direct pages scanned 129742 127344
>>> Kswapd pages scanned 1852277 1817825
>>> Kswapd pages reclaimed 1838000 1804212
>>> Direct pages reclaimed 129719 127327
>>> Kswapd efficiency 98% 98%
>>> Kswapd velocity 1130.066 1221.296
>>> Direct efficiency 99% 99%
>>> Direct velocity 79.367 85.585
>>> Percentage direct scans 6% 6%
>>> Zone normal velocity 231.829 246.097
>>> Zone dma32 velocity 972.589 1055.158
>>> Zone dma velocity 5.015 5.626
>>> Page writes by reclaim 6287 6534
>>> Page writes file 2630 2999
>>> Page writes anon 3657 3535
>>> Page reclaim immediate 2187 2080
>>> Sector Reads 2917808 2877336
>>> Sector Writes 11477891 11206722
>>> Page rescued immediate 0 0
>>> Slabs scanned 2214118 2168524
>>> Direct inode steals 12181 9788
>>> Kswapd inode steals 144830 132109
>>> Kswapd skipped wait 0 0
>>> THP fault alloc 0 0
>>> THP collapse alloc 0 0
>>> THP splits 0 0
>>> THP fault fallback 0 0
>>> THP collapse fail 0 0
>>> Compaction stalls 738 714
>>> Compaction success 194 207
>>> Compaction failures 543 507
>>> Page migrate success 1806083 1464014
>>> Page migrate failure 0 0
>>> Compaction pages isolated 3873329 3162974
>>> Compaction migrate scanned 74594862 59874420
>>> Compaction free scanned 125888854 110868637
>>> Compaction cost 2469 1998
>>
>> FWIW, I've let a machine run the series with individual patches applied
>> on 3.13 with my compaction patches, so 6 is the end of my series and 7-11 yours:
>> The average is of 10 runs (in case you wonder how that's done, the success rates are
>> calculated with a new R support that's pending Mel's merge; system time and vmstats
>> are currently a hack, but I hope to add R support for them as well, and maybe publish
>> to github or something if there's interest).
>
> Good! I have an interest on it.
Most of the support is now in mmtests' master branch on github. The system time
and vmstats is a hack with hardcoded number of iterations (now 10) that I attach
at the end of the mail.
You need R installed and run compare like this:
compare-kernels.sh --R --iterations 10
>
>>
>> Interestingly, you have a much lower success rate and also much lower compaction cost
>> and, well, even the benchmark times. Wonder what difference in config or hw causes this.
>> You seem to have THP disabled, I enabled, but that would be weird to cause this.
>
> My 10 runs are continuous 10 runs without reboot. It makes compaction success
> rate decline on every trial and therefore average result is so low than yours.
> I heard that you did 10 runs because of large stdev, so I thought that continuous 10 runs
> also can makes the result reliable. Therefore I decided this method although it is not
> proper method to get the average. I had to notify about it. If it confuses you,
> sorry about that.
>
> Anyway, noticeable point of continuous 10 runs is that success rate decrease continuously
> and significantly. I attach the rate of success 3 on every trial on below.
>
> Base
> % Success: 80
> % Success: 60
> % Success: 76
> % Success: 74
> % Success: 70
> % Success: 68
> % Success: 66
> % Success: 65
> % Success: 63
> % Success: 61
>
>
> Applied with my patches
> % Success: 81
> % Success: 78
> % Success: 75
> % Success: 74
> % Success: 71
> % Success: 72
> % Success: 73
> % Success: 70
> % Success: 70
> % Success: 70
>
> It means that memory is fragmented continously. I didn't dig into this problem, but
> it would be good subject to investigate.
You're right, I see that too. And the reduction of work done by compaction,
especially within first 3 iterations, is interestingly large.
The number of reclaimable and unmovable pageblocks indeed slightly grows,
although not as much as to translate directly into the success rates.
stress-highalloc
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
Success 1 49.00 ( 0.00%) 51.00 ( -4.08%) 51.00 ( -4.08%) 62.00 (-26.53%) 53.00 ( -8.16%) 57.00 (-16.33%) 55.00 (-12.24%) 58.00 (-18.37%) 58.00 (-18.37%) 46.00 ( 6.12%)
Success 2 55.00 ( 0.00%) 64.00 (-16.36%) 62.00 (-12.73%) 62.00 (-12.73%) 56.00 ( -1.82%) 57.00 ( -3.64%) 58.00 ( -5.45%) 59.00 ( -7.27%) 57.00 ( -3.64%) 55.00 ( 0.00%)
Success 3 85.00 ( 0.00%) 81.00 ( 4.71%) 77.00 ( 9.41%) 77.00 ( 9.41%) 75.00 ( 11.76%) 74.00 ( 12.94%) 74.00 ( 12.94%) 73.00 ( 14.12%) 73.00 ( 14.12%) 71.00 ( 16.47%)
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
User 5696.69 5713.20 5668.87 5412.19 5425.39 5357.67 5402.08 5345.72 5575.73 5473.66
System 1008.77 1028.72 1025.66 1015.87 1023.82 1023.45 1023.26 1019.52 1026.88 1028.05
Elapsed 2253.63 2285.87 2531.58 2198.46 2220.21 2199.75 2237.43 2266.97 2724.87 2288.90
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
Minor Faults 248584347 241444309 238931712 229339487 234134233 234113609 236172742 235955684 235274197 234072299
Major Faults 768 6273 5624 6146 7638 7038 7279 8475 6837 8948
Swap Ins 24 17320 12557 11956 31729 28434 29228 43494 29528 41212
Swap Outs 958 94106 106680 81672 147012 95169 133733 142124 133492 157440
Direct pages scanned 213760 42549 118178 22266 4818 3634 51636 23166 17319 40327
Kswapd pages scanned 2137836 3449877 3500913 3909294 3343227 3800324 3840356 3035274 3048595 3192582
Kswapd pages reclaimed 2134695 2231079 2536612 2238583 2232883 2219601 2180405 2324381 2331881 2276682
Direct pages reclaimed 213422 42304 117833 22134 4711 3562 51551 23088 17240 40281
Kswapd efficiency 99% 64% 72% 57% 66% 58% 56% 76% 76% 71%
Kswapd velocity 948.619 1509.218 1382.896 1778.197 1505.816 1727.616 1716.414 1338.912 1118.804 1394.811
Direct efficiency 99% 99% 99% 99% 97% 98% 99% 99% 99% 99%
Direct velocity 94.851 18.614 46.682 10.128 2.170 1.652 23.078 10.219 6.356 17.619
Percentage direct scans 9% 1% 3% 0% 0% 0% 1% 0% 0% 1%
Zone normal velocity 329.358 931.485 794.284 1165.576 922.453 1122.753 1146.195 711.711 594.119 801.484
Zone dma32 velocity 714.112 596.348 635.294 622.749 585.533 606.515 593.297 637.420 531.040 610.945
Zone dma velocity 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Page writes by reclaim 958.000 142306.000 147653.000 154673.000 168131.000 120184.000 157196.000 158862.000 155427.000 178478.000
Page writes file 0 48200 40973 73001 21119 25015 23463 16738 21935 21038
Page writes anon 958 94106 106680 81672 147012 95169 133733 142124 133492 157440
Page reclaim immediate 227 978397 717495 1393234 747156 1277758 1307521 315722 396171 528904
Sector Reads 3146168 3275924 4886280 3301644 3328564 3341776 3357484 3431016 3375072 3329768
Sector Writes 12552716 12767728 12971248 12339364 12829408 12581020 12810572 12822476 12836888 12906372
Page rescued immediate 0 0 0 0 0 0 0 0 0 0
Slabs scanned 1857664 2226176 2273280 2238464 2250752 2269184 2297856 2229248 2216960 2219008
Direct inode steals 10976 10215 17701 2688 19256 8041 3827 11226 10767 13599
Kswapd inode steals 58552 341409 352875 362913 406943 392956 454265 326993 310959 341106
Kswapd skipped wait 0 0 0 0 0 0 0 0 0 0
THP fault alloc 97 59 96 8 58 54 58 68 68 54
THP collapse alloc 655 416 616 396 396 393 402 386 396 387
THP splits 6 5 18 12 10 11 9 12 11 7
THP fault fallback 0 0 0 0 0 0 0 0 1 0
THP collapse fail 11 21 13 21 21 20 22 18 20 21
Compaction stalls 5851 6125 6925 5572 6183 6064 6224 6069 6196 6861
Compaction success 1163 1811 2144 1783 1747 1886 1833 1823 1781 1748
Compaction failures 3250 2200 2195 1800 1754 1421 1834 1521 1929 1753
Page migrate success 6110592 3508332 3093592 2740374 2091309 1400876 2208633 1516013 2223733 1699633
Page migrate failure 0 0 0 0 0 0 0 0 0 0
Compaction pages isolated 15360926 10185640 9900328 8228257 6719820 5195540 6677838 4841171 6708180 5479883
Compaction migrate scanned 349981387 300209536 297909273 286215718 268660759 256103243 285198850 241549363 287544164 261619750
Compaction free scanned 556386189 260105925 296420166 196203596 148614315 98672618 174033818 82384264 165998395 123011890
Compaction cost 9084 5936 5484 5004 4179 3345 4415 3356 4448 3699
NUMA alloc hit 167473642 162715095 160968619 154496168 157513861 157598611 159117516 158854501 158279702 157440309
NUMA alloc miss 0 0 0 0 0 0 0 0 0 0
NUMA interleave hit 0 0 0 0 0 0 0 0 0 0
NUMA alloc local 167473642 162715095 160968619 154496168 157513861 157598611 159117516 158854501 158279702 157440309
NUMA page range updates 0 0 0 0 0 0 0 0 0 0
NUMA huge PMD updates 0 0 0 0 0 0 0 0 0 0
NUMA PTE updates 0 0 0 0 0 0 0 0 0 0
NUMA hint faults 0 0 0 0 0 0 0 0 0 0
NUMA hint local faults 0 0 0 0 0 0 0 0 0 0
NUMA hint local percent 100 100 100 100 100 100 100 100 100 100
NUMA pages migrated 0 0 0 0 0 0 0 0 0 0
AutoNUMA cost 0 0 0 0 0 0 0 0 0 0
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
Mean sda-avgqz 62.27 64.24 67.82 68.47 64.52 74.36 68.65 66.98 52.45 61.41
Mean sda-await 356.96 384.46 396.37 426.02 384.43 447.38 409.69 384.68 315.29 357.33
Mean sda-r_await 43.62 44.58 40.66 49.54 45.40 47.44 48.68 41.98 35.12 39.42
Mean sda-w_await 705.15 813.62 1136.46 1029.17 834.44 1127.80 971.66 914.88 712.12 784.44
Max sda-avgqz 291.35 286.96 162.48 265.33 161.77 240.23 196.98 203.30 162.28 283.29
Max sda-await 1258.88 1680.93 2309.73 1566.46 1711.69 1948.61 1381.23 1986.32 1583.03 1737.79
Max sda-r_await 274.27 185.52 125.71 272.05 340.00 332.00 312.00 188.16 159.23 173.33
Max sda-w_await 8174.71 12488.89 13484.06 12931.84 9821.61 10462.28 9182.25 9573.30 12516.77 8842.26
> Thanks.
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
------8<------
commit 596bad5b6505b8b606c197a8ee2af8a74a7edbc6
Author: Vlastimil Babka <vbabka@suse.cz>
Date: Mon Dec 16 13:38:36 2013 +0100
HACK: iteration support for MonitorDuration and MonitorMmtestsvmstat
diff --git a/bin/lib/MMTests/MonitorDuration.pm b/bin/lib/MMTests/MonitorDuration.pm
index 1069e83..313eb0a 100644
--- a/bin/lib/MMTests/MonitorDuration.pm
+++ b/bin/lib/MMTests/MonitorDuration.pm
@@ -19,22 +19,33 @@ sub new() {
sub extractReport($$$) {
my ($self, $reportDir, $testName, $testBenchmark) = @_;
- my $file = "$reportDir/tests-timestamp-$testName";
+ my $file;
+ my $i;
+ my ($user, $system, $elapsed);
+ my $iterations = 10;
+
+ for ($i = 1; $i <= $iterations; $i++) {
+ $file = "$reportDir/$i/tests-timestamp-$testName";
open(INPUT, $file) || die("Failed to open $file\n");
while (<INPUT>) {
if ($_ =~ /^time \:\: $testBenchmark (.*)/) {
my $dummy;
- my ($user, $system, $elapsed);
+ my ($useri, $systemi, $elapsedi);
- ($user, $dummy,
- $system, $dummy,
- $elapsed, $dummy) = split(/\s/, $1);
+ ($useri, $dummy,
+ $systemi, $dummy,
+ $elapsedi, $dummy) = split(/\s/, $1);
- push @{$self->{_ResultData}}, [ "", $user, $system, $elapsed];
+ $user += $useri;
+ $system += $systemi;
+ $elapsed += $elapsedi;
}
}
close INPUT;
+ }
+
+ push @{$self->{_ResultData}}, [ "", $user / $iterations, $system / $iterations, $elapsed / $iterations];
}
1;
diff --git a/bin/lib/MMTests/MonitorMmtestsvmstat.pm b/bin/lib/MMTests/MonitorMmtestsvmstat.pm
index 7bd3a21..bec917b 100644
--- a/bin/lib/MMTests/MonitorMmtestsvmstat.pm
+++ b/bin/lib/MMTests/MonitorMmtestsvmstat.pm
@@ -167,8 +167,13 @@ sub extractReport($$$$) {
my $elapsed_time;
my %zones_seen;
- my $file = "$reportDir/tests-timestamp-$testName";
+ my $i;
+ my $iterations = 10;
+ my $file;
+
+ for ($i = 1; $i <= $iterations; $i++) {
+ $file = "$reportDir/$i/tests-timestamp-$testName";
open(INPUT, $file) || die("Failed to open $file\n");
while (<INPUT>) {
if ($_ =~ /^test begin \:\: $testBenchmark/) {
@@ -209,9 +214,9 @@ sub extractReport($$$$) {
my ($key, $value) = split(/\s/, $_);
if ($reading_before) {
- $vmstat_before{$key} = $value;
+ $vmstat_before{$key} += $value;
} elsif ($reading_after) {
- $vmstat_after{$key} = $value;
+ $vmstat_after{$key} += $value;
}
if ($key eq "pgmigrate_success") {
$new_compaction_stats = 1;
@@ -222,6 +227,12 @@ sub extractReport($$$$) {
}
}
close INPUT;
+ }
+
+ foreach my $key (sort keys %vmstat_before) {
+ $vmstat_before{$key} /= $iterations;
+ $vmstat_after{$key} /= $iterations;
+ }
# kswapd steal
foreach my $key ("kswapd_steal", "pgsteal_kswapd_dma", "pgsteal_kswapd_dma32",
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v2 0/5] compaction related commits
@ 2014-03-19 9:38 ` Vlastimil Babka
0 siblings, 0 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-03-19 9:38 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Mel Gorman, Rik van Riel, linux-mm, linux-kernel
On 03/04/2014 01:23 AM, Joonsoo Kim wrote:
> On Mon, Mar 03, 2014 at 12:02:00PM +0100, Vlastimil Babka wrote:
>> On 02/14/2014 07:53 AM, Joonsoo Kim wrote:
>>> changes for v2
>>> o include more experiment data in cover letter
>>> o deal with vlastimil's comments mostly about commit description on 4/5
>>>
>>> This patchset is related to the compaction.
>>>
>>> patch 1 fixes contrary implementation of the purpose of compaction.
>>> patch 2~4 are for optimization.
>>> patch 5 is just for clean-up.
>>>
>>> I tested this patchset with stress-highalloc benchmark on Mel's mmtest
>>> and cannot find any regression in terms of success rate. And I find
>>> much reduced(9%) elapsed time.
>>>
>>> Below is the average result of 10 runs on my 4GB quad core system.
>>>
>>> compaction-base+ is based on 3.13.0 with Vlastimil's recent fixes.
>>> compaction-fix+ has this patch series on top of compaction-base+.
>>>
>>> Thanks.
>>>
>>>
>>> stress-highalloc
>>> 3.13.0 3.13.0
>>> compaction-base+ compaction-fix+
>>> Success 1 14.10 15.00
>>> Success 2 20.20 20.00
>>> Success 3 68.30 73.40
>>>
>>> 3.13.0 3.13.0
>>> compaction-base+ compaction-fix+
>>> User 3486.02 3437.13
>>> System 757.92 741.15
>>> Elapsed 1638.52 1488.32
>>>
>>> 3.13.0 3.13.0
>>> compaction-base+ compaction-fix+
>>> Minor Faults 172591561 167116621
>>> Major Faults 984 859
>>> Swap Ins 743 653
>>> Swap Outs 3657 3535
>>> Direct pages scanned 129742 127344
>>> Kswapd pages scanned 1852277 1817825
>>> Kswapd pages reclaimed 1838000 1804212
>>> Direct pages reclaimed 129719 127327
>>> Kswapd efficiency 98% 98%
>>> Kswapd velocity 1130.066 1221.296
>>> Direct efficiency 99% 99%
>>> Direct velocity 79.367 85.585
>>> Percentage direct scans 6% 6%
>>> Zone normal velocity 231.829 246.097
>>> Zone dma32 velocity 972.589 1055.158
>>> Zone dma velocity 5.015 5.626
>>> Page writes by reclaim 6287 6534
>>> Page writes file 2630 2999
>>> Page writes anon 3657 3535
>>> Page reclaim immediate 2187 2080
>>> Sector Reads 2917808 2877336
>>> Sector Writes 11477891 11206722
>>> Page rescued immediate 0 0
>>> Slabs scanned 2214118 2168524
>>> Direct inode steals 12181 9788
>>> Kswapd inode steals 144830 132109
>>> Kswapd skipped wait 0 0
>>> THP fault alloc 0 0
>>> THP collapse alloc 0 0
>>> THP splits 0 0
>>> THP fault fallback 0 0
>>> THP collapse fail 0 0
>>> Compaction stalls 738 714
>>> Compaction success 194 207
>>> Compaction failures 543 507
>>> Page migrate success 1806083 1464014
>>> Page migrate failure 0 0
>>> Compaction pages isolated 3873329 3162974
>>> Compaction migrate scanned 74594862 59874420
>>> Compaction free scanned 125888854 110868637
>>> Compaction cost 2469 1998
>>
>> FWIW, I've let a machine run the series with individual patches applied
>> on 3.13 with my compaction patches, so 6 is the end of my series and 7-11 yours:
>> The average is of 10 runs (in case you wonder how that's done, the success rates are
>> calculated with a new R support that's pending Mel's merge; system time and vmstats
>> are currently a hack, but I hope to add R support for them as well, and maybe publish
>> to github or something if there's interest).
>
> Good! I have an interest on it.
Most of the support is now in mmtests' master branch on github. The system time
and vmstats is a hack with hardcoded number of iterations (now 10) that I attach
at the end of the mail.
You need R installed and run compare like this:
compare-kernels.sh --R --iterations 10
>
>>
>> Interestingly, you have a much lower success rate and also much lower compaction cost
>> and, well, even the benchmark times. Wonder what difference in config or hw causes this.
>> You seem to have THP disabled, I enabled, but that would be weird to cause this.
>
> My 10 runs are continuous 10 runs without reboot. It makes compaction success
> rate decline on every trial and therefore average result is so low than yours.
> I heard that you did 10 runs because of large stdev, so I thought that continuous 10 runs
> also can makes the result reliable. Therefore I decided this method although it is not
> proper method to get the average. I had to notify about it. If it confuses you,
> sorry about that.
>
> Anyway, noticeable point of continuous 10 runs is that success rate decrease continuously
> and significantly. I attach the rate of success 3 on every trial on below.
>
> Base
> % Success: 80
> % Success: 60
> % Success: 76
> % Success: 74
> % Success: 70
> % Success: 68
> % Success: 66
> % Success: 65
> % Success: 63
> % Success: 61
>
>
> Applied with my patches
> % Success: 81
> % Success: 78
> % Success: 75
> % Success: 74
> % Success: 71
> % Success: 72
> % Success: 73
> % Success: 70
> % Success: 70
> % Success: 70
>
> It means that memory is fragmented continously. I didn't dig into this problem, but
> it would be good subject to investigate.
You're right, I see that too. And the reduction of work done by compaction,
especially within first 3 iterations, is interestingly large.
The number of reclaimable and unmovable pageblocks indeed slightly grows,
although not as much as to translate directly into the success rates.
stress-highalloc
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
Success 1 49.00 ( 0.00%) 51.00 ( -4.08%) 51.00 ( -4.08%) 62.00 (-26.53%) 53.00 ( -8.16%) 57.00 (-16.33%) 55.00 (-12.24%) 58.00 (-18.37%) 58.00 (-18.37%) 46.00 ( 6.12%)
Success 2 55.00 ( 0.00%) 64.00 (-16.36%) 62.00 (-12.73%) 62.00 (-12.73%) 56.00 ( -1.82%) 57.00 ( -3.64%) 58.00 ( -5.45%) 59.00 ( -7.27%) 57.00 ( -3.64%) 55.00 ( 0.00%)
Success 3 85.00 ( 0.00%) 81.00 ( 4.71%) 77.00 ( 9.41%) 77.00 ( 9.41%) 75.00 ( 11.76%) 74.00 ( 12.94%) 74.00 ( 12.94%) 73.00 ( 14.12%) 73.00 ( 14.12%) 71.00 ( 16.47%)
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
User 5696.69 5713.20 5668.87 5412.19 5425.39 5357.67 5402.08 5345.72 5575.73 5473.66
System 1008.77 1028.72 1025.66 1015.87 1023.82 1023.45 1023.26 1019.52 1026.88 1028.05
Elapsed 2253.63 2285.87 2531.58 2198.46 2220.21 2199.75 2237.43 2266.97 2724.87 2288.90
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
Minor Faults 248584347 241444309 238931712 229339487 234134233 234113609 236172742 235955684 235274197 234072299
Major Faults 768 6273 5624 6146 7638 7038 7279 8475 6837 8948
Swap Ins 24 17320 12557 11956 31729 28434 29228 43494 29528 41212
Swap Outs 958 94106 106680 81672 147012 95169 133733 142124 133492 157440
Direct pages scanned 213760 42549 118178 22266 4818 3634 51636 23166 17319 40327
Kswapd pages scanned 2137836 3449877 3500913 3909294 3343227 3800324 3840356 3035274 3048595 3192582
Kswapd pages reclaimed 2134695 2231079 2536612 2238583 2232883 2219601 2180405 2324381 2331881 2276682
Direct pages reclaimed 213422 42304 117833 22134 4711 3562 51551 23088 17240 40281
Kswapd efficiency 99% 64% 72% 57% 66% 58% 56% 76% 76% 71%
Kswapd velocity 948.619 1509.218 1382.896 1778.197 1505.816 1727.616 1716.414 1338.912 1118.804 1394.811
Direct efficiency 99% 99% 99% 99% 97% 98% 99% 99% 99% 99%
Direct velocity 94.851 18.614 46.682 10.128 2.170 1.652 23.078 10.219 6.356 17.619
Percentage direct scans 9% 1% 3% 0% 0% 0% 1% 0% 0% 1%
Zone normal velocity 329.358 931.485 794.284 1165.576 922.453 1122.753 1146.195 711.711 594.119 801.484
Zone dma32 velocity 714.112 596.348 635.294 622.749 585.533 606.515 593.297 637.420 531.040 610.945
Zone dma velocity 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
Page writes by reclaim 958.000 142306.000 147653.000 154673.000 168131.000 120184.000 157196.000 158862.000 155427.000 178478.000
Page writes file 0 48200 40973 73001 21119 25015 23463 16738 21935 21038
Page writes anon 958 94106 106680 81672 147012 95169 133733 142124 133492 157440
Page reclaim immediate 227 978397 717495 1393234 747156 1277758 1307521 315722 396171 528904
Sector Reads 3146168 3275924 4886280 3301644 3328564 3341776 3357484 3431016 3375072 3329768
Sector Writes 12552716 12767728 12971248 12339364 12829408 12581020 12810572 12822476 12836888 12906372
Page rescued immediate 0 0 0 0 0 0 0 0 0 0
Slabs scanned 1857664 2226176 2273280 2238464 2250752 2269184 2297856 2229248 2216960 2219008
Direct inode steals 10976 10215 17701 2688 19256 8041 3827 11226 10767 13599
Kswapd inode steals 58552 341409 352875 362913 406943 392956 454265 326993 310959 341106
Kswapd skipped wait 0 0 0 0 0 0 0 0 0 0
THP fault alloc 97 59 96 8 58 54 58 68 68 54
THP collapse alloc 655 416 616 396 396 393 402 386 396 387
THP splits 6 5 18 12 10 11 9 12 11 7
THP fault fallback 0 0 0 0 0 0 0 0 1 0
THP collapse fail 11 21 13 21 21 20 22 18 20 21
Compaction stalls 5851 6125 6925 5572 6183 6064 6224 6069 6196 6861
Compaction success 1163 1811 2144 1783 1747 1886 1833 1823 1781 1748
Compaction failures 3250 2200 2195 1800 1754 1421 1834 1521 1929 1753
Page migrate success 6110592 3508332 3093592 2740374 2091309 1400876 2208633 1516013 2223733 1699633
Page migrate failure 0 0 0 0 0 0 0 0 0 0
Compaction pages isolated 15360926 10185640 9900328 8228257 6719820 5195540 6677838 4841171 6708180 5479883
Compaction migrate scanned 349981387 300209536 297909273 286215718 268660759 256103243 285198850 241549363 287544164 261619750
Compaction free scanned 556386189 260105925 296420166 196203596 148614315 98672618 174033818 82384264 165998395 123011890
Compaction cost 9084 5936 5484 5004 4179 3345 4415 3356 4448 3699
NUMA alloc hit 167473642 162715095 160968619 154496168 157513861 157598611 159117516 158854501 158279702 157440309
NUMA alloc miss 0 0 0 0 0 0 0 0 0 0
NUMA interleave hit 0 0 0 0 0 0 0 0 0 0
NUMA alloc local 167473642 162715095 160968619 154496168 157513861 157598611 159117516 158854501 158279702 157440309
NUMA page range updates 0 0 0 0 0 0 0 0 0 0
NUMA huge PMD updates 0 0 0 0 0 0 0 0 0 0
NUMA PTE updates 0 0 0 0 0 0 0 0 0 0
NUMA hint faults 0 0 0 0 0 0 0 0 0 0
NUMA hint local faults 0 0 0 0 0 0 0 0 0 0
NUMA hint local percent 100 100 100 100 100 100 100 100 100 100
NUMA pages migrated 0 0 0 0 0 0 0 0 0 0
AutoNUMA cost 0 0 0 0 0 0 0 0 0 0
test test test test test test test test test test
1 2 3 4 5 6 7 8 9 10
Mean sda-avgqz 62.27 64.24 67.82 68.47 64.52 74.36 68.65 66.98 52.45 61.41
Mean sda-await 356.96 384.46 396.37 426.02 384.43 447.38 409.69 384.68 315.29 357.33
Mean sda-r_await 43.62 44.58 40.66 49.54 45.40 47.44 48.68 41.98 35.12 39.42
Mean sda-w_await 705.15 813.62 1136.46 1029.17 834.44 1127.80 971.66 914.88 712.12 784.44
Max sda-avgqz 291.35 286.96 162.48 265.33 161.77 240.23 196.98 203.30 162.28 283.29
Max sda-await 1258.88 1680.93 2309.73 1566.46 1711.69 1948.61 1381.23 1986.32 1583.03 1737.79
Max sda-r_await 274.27 185.52 125.71 272.05 340.00 332.00 312.00 188.16 159.23 173.33
Max sda-w_await 8174.71 12488.89 13484.06 12931.84 9821.61 10462.28 9182.25 9573.30 12516.77 8842.26
> Thanks.
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>
------8<------
commit 596bad5b6505b8b606c197a8ee2af8a74a7edbc6
Author: Vlastimil Babka <vbabka@suse.cz>
Date: Mon Dec 16 13:38:36 2013 +0100
HACK: iteration support for MonitorDuration and MonitorMmtestsvmstat
diff --git a/bin/lib/MMTests/MonitorDuration.pm b/bin/lib/MMTests/MonitorDuration.pm
index 1069e83..313eb0a 100644
--- a/bin/lib/MMTests/MonitorDuration.pm
+++ b/bin/lib/MMTests/MonitorDuration.pm
@@ -19,22 +19,33 @@ sub new() {
sub extractReport($$$) {
my ($self, $reportDir, $testName, $testBenchmark) = @_;
- my $file = "$reportDir/tests-timestamp-$testName";
+ my $file;
+ my $i;
+ my ($user, $system, $elapsed);
+ my $iterations = 10;
+
+ for ($i = 1; $i <= $iterations; $i++) {
+ $file = "$reportDir/$i/tests-timestamp-$testName";
open(INPUT, $file) || die("Failed to open $file\n");
while (<INPUT>) {
if ($_ =~ /^time \:\: $testBenchmark (.*)/) {
my $dummy;
- my ($user, $system, $elapsed);
+ my ($useri, $systemi, $elapsedi);
- ($user, $dummy,
- $system, $dummy,
- $elapsed, $dummy) = split(/\s/, $1);
+ ($useri, $dummy,
+ $systemi, $dummy,
+ $elapsedi, $dummy) = split(/\s/, $1);
- push @{$self->{_ResultData}}, [ "", $user, $system, $elapsed];
+ $user += $useri;
+ $system += $systemi;
+ $elapsed += $elapsedi;
}
}
close INPUT;
+ }
+
+ push @{$self->{_ResultData}}, [ "", $user / $iterations, $system / $iterations, $elapsed / $iterations];
}
1;
diff --git a/bin/lib/MMTests/MonitorMmtestsvmstat.pm b/bin/lib/MMTests/MonitorMmtestsvmstat.pm
index 7bd3a21..bec917b 100644
--- a/bin/lib/MMTests/MonitorMmtestsvmstat.pm
+++ b/bin/lib/MMTests/MonitorMmtestsvmstat.pm
@@ -167,8 +167,13 @@ sub extractReport($$$$) {
my $elapsed_time;
my %zones_seen;
- my $file = "$reportDir/tests-timestamp-$testName";
+ my $i;
+ my $iterations = 10;
+ my $file;
+
+ for ($i = 1; $i <= $iterations; $i++) {
+ $file = "$reportDir/$i/tests-timestamp-$testName";
open(INPUT, $file) || die("Failed to open $file\n");
while (<INPUT>) {
if ($_ =~ /^test begin \:\: $testBenchmark/) {
@@ -209,9 +214,9 @@ sub extractReport($$$$) {
my ($key, $value) = split(/\s/, $_);
if ($reading_before) {
- $vmstat_before{$key} = $value;
+ $vmstat_before{$key} += $value;
} elsif ($reading_after) {
- $vmstat_after{$key} = $value;
+ $vmstat_after{$key} += $value;
}
if ($key eq "pgmigrate_success") {
$new_compaction_stats = 1;
@@ -222,6 +227,12 @@ sub extractReport($$$$) {
}
}
close INPUT;
+ }
+
+ foreach my $key (sort keys %vmstat_before) {
+ $vmstat_before{$key} /= $iterations;
+ $vmstat_after{$key} /= $iterations;
+ }
# kswapd steal
foreach my $key ("kswapd_steal", "pgsteal_kswapd_dma", "pgsteal_kswapd_dma32",
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 20+ messages in thread
end of thread, other threads:[~2014-03-19 9:38 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-14 6:53 [PATCH v2 0/5] compaction related commits Joonsoo Kim
2014-02-14 6:53 ` Joonsoo Kim
2014-02-14 6:53 ` [PATCH v2 1/5] mm/compaction: disallow high-order page for migration target Joonsoo Kim
2014-02-14 6:53 ` Joonsoo Kim
2014-02-14 6:54 ` [PATCH v2 2/5] mm/compaction: do not call suitable_migration_target() on every page Joonsoo Kim
2014-02-14 6:54 ` Joonsoo Kim
2014-02-20 16:49 ` Vlastimil Babka
2014-02-20 16:49 ` Vlastimil Babka
2014-02-14 6:54 ` [PATCH v2 3/5] mm/compaction: change the timing to check to drop the spinlock Joonsoo Kim
2014-02-14 6:54 ` Joonsoo Kim
2014-02-14 6:54 ` [PATCH v2 4/5] mm/compaction: check pageblock suitability once per pageblock Joonsoo Kim
2014-02-14 6:54 ` Joonsoo Kim
2014-02-14 6:54 ` [PATCH v2 5/5] mm/compaction: clean-up code on success of ballon isolation Joonsoo Kim
2014-02-14 6:54 ` Joonsoo Kim
2014-03-03 11:02 ` [PATCH v2 0/5] compaction related commits Vlastimil Babka
2014-03-03 11:02 ` Vlastimil Babka
2014-03-04 0:23 ` Joonsoo Kim
2014-03-04 0:23 ` Joonsoo Kim
2014-03-19 9:38 ` Vlastimil Babka
2014-03-19 9:38 ` Vlastimil Babka
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.