* [PATCH 0/3] CMA: Handle the issues of aggressively allocate the
@ 2014-12-25 9:43 ` Hui Zhu
0 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
I tried the Joonsoo's CMA patches [1] in my part and found that they works
better than mine [2] about handle LRU and other issues even if they
don't shrink the memory before cma_alloc. So I began to test it in my
part.
But my colleague Weixing found some issues around it. So we make 2 patches to
handle the issues.
And I merged cma_alloc_counter from [2] to cma_alloc work better.
This patchset is based on aa39477b5692611b91ac9455ae588738852b3f60 and [1].
[1] https://lkml.org/lkml/2014/5/28/64
[2] https://lkml.org/lkml/2014/10/15/623
Hui Zhu (3):
CMA: Fix the bug that CMA's page number is substructed twice
CMA: Fix the issue that nr_try_movable just count MIGRATE_MOVABLE memory
CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range
include/linux/cma.h | 2 +
include/linux/mmzone.h | 3 +
mm/cma.c | 6 +++
mm/page_alloc.c | 76 ++++++++++++++++++++++++++++++++++---------------
4 files changed, 65 insertions(+), 22 deletions(-)
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 0/3] CMA: Handle the issues of aggressively allocate the
@ 2014-12-25 9:43 ` Hui Zhu
0 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
I tried the Joonsoo's CMA patches [1] in my part and found that they works
better than mine [2] about handle LRU and other issues even if they
don't shrink the memory before cma_alloc. So I began to test it in my
part.
But my colleague Weixing found some issues around it. So we make 2 patches to
handle the issues.
And I merged cma_alloc_counter from [2] to cma_alloc work better.
This patchset is based on aa39477b5692611b91ac9455ae588738852b3f60 and [1].
[1] https://lkml.org/lkml/2014/5/28/64
[2] https://lkml.org/lkml/2014/10/15/623
Hui Zhu (3):
CMA: Fix the bug that CMA's page number is substructed twice
CMA: Fix the issue that nr_try_movable just count MIGRATE_MOVABLE memory
CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range
include/linux/cma.h | 2 +
include/linux/mmzone.h | 3 +
mm/cma.c | 6 +++
mm/page_alloc.c | 76 ++++++++++++++++++++++++++++++++++---------------
4 files changed, 65 insertions(+), 22 deletions(-)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice
2014-12-25 9:43 ` Hui Zhu
@ 2014-12-25 9:43 ` Hui Zhu
-1 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
In Joonsoo's CMA patch "CMA: always treat free cma pages as non-free on
watermark checking" [1], it changes __zone_watermark_ok to substruct CMA
pages number from free_pages if system use CMA:
if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
But after this part of code
for (o = 0; o < order; o++) {
/* At the next order, this order's pages become unavailable */
free_pages -= z->free_area[o].nr_free << o;
CMA memory in each order is part of z->free_area[o].nr_free, then the CMA
page number of this order is substructed twice. This bug will make
__zone_watermark_ok return more false.
This patch add cma_free_area to struct free_area that just record the number
of CMA pages. And add it back in the order loop to handle the substruct
twice issue.
[1] https://lkml.org/lkml/2014/5/28/110
Signed-off-by: Hui Zhu <zhuhui@xiaomi.com>
Signed-off-by: Weixing Liu <liuweixing@xiaomi.com>
---
include/linux/mmzone.h | 3 +++
mm/page_alloc.c | 29 ++++++++++++++++++++++++++++-
2 files changed, 31 insertions(+), 1 deletion(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index ee1ce1f..7ccad93 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -92,6 +92,9 @@ static inline int get_pfnblock_migratetype(struct page *page, unsigned long pfn)
struct free_area {
struct list_head free_list[MIGRATE_TYPES];
unsigned long nr_free;
+#ifdef CONFIG_CMA
+ unsigned long cma_nr_free;
+#endif
};
struct pglist_data;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1b6c82c..a8d9f03 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -650,6 +650,8 @@ static inline void __free_one_page(struct page *page,
} else {
list_del(&buddy->lru);
zone->free_area[order].nr_free--;
+ if (is_migrate_cma(migratetype))
+ zone->free_area[order].cma_nr_free--;
rmv_page_order(buddy);
}
combined_idx = buddy_idx & page_idx;
@@ -683,6 +685,8 @@ static inline void __free_one_page(struct page *page,
list_add(&page->lru, &zone->free_area[order].free_list[migratetype]);
out:
zone->free_area[order].nr_free++;
+ if (is_migrate_cma(migratetype))
+ zone->free_area[order].cma_nr_free++;
}
static inline int free_pages_check(struct page *page)
@@ -987,6 +991,8 @@ static inline void expand(struct zone *zone, struct page *page,
}
list_add(&page[size].lru, &area->free_list[migratetype]);
area->nr_free++;
+ if (is_migrate_cma(migratetype))
+ area->cma_nr_free++;
set_page_order(&page[size], high);
}
}
@@ -1070,6 +1076,8 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
list_del(&page->lru);
rmv_page_order(page);
area->nr_free--;
+ if (is_migrate_cma(migratetype))
+ area->cma_nr_free--;
expand(zone, page, order, current_order, area, migratetype);
set_freepage_migratetype(page, migratetype);
return page;
@@ -1258,6 +1266,8 @@ __rmqueue_fallback(struct zone *zone, unsigned int order, int start_migratetype)
page = list_entry(area->free_list[migratetype].next,
struct page, lru);
area->nr_free--;
+ if (is_migrate_cma(migratetype))
+ area->cma_nr_free--;
new_type = try_to_steal_freepages(zone, page,
start_migratetype,
@@ -1682,6 +1692,8 @@ int __isolate_free_page(struct page *page, unsigned int order)
/* Remove page from free list */
list_del(&page->lru);
zone->free_area[order].nr_free--;
+ if (is_migrate_cma(mt))
+ zone->free_area[order].cma_nr_free--;
rmv_page_order(page);
/* Set the pageblock if the isolated page is at least a pageblock */
@@ -1893,6 +1905,9 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
/* free_pages may go negative - that's OK */
long min = mark;
int o;
+#ifdef CONFIG_CMA
+ bool cma_is_subbed = false;
+#endif
free_pages -= (1 << order) - 1;
if (alloc_flags & ALLOC_HIGH)
@@ -1905,8 +1920,10 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
* unmovable/reclaimable allocation and they can suddenly
* vanish through CMA allocation
*/
- if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
+ if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages) {
free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
+ cma_is_subbed = true;
+ }
if (free_pages <= min + z->lowmem_reserve[classzone_idx])
return false;
@@ -1914,6 +1931,13 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
/* At the next order, this order's pages become unavailable */
free_pages -= z->free_area[o].nr_free << o;
+ /* If CMA's page number of this order was substructed as part
+ of "zone_page_state(z, NR_FREE_CMA_PAGES)", subtracting
+ "z->free_area[o].nr_free << o" substructed CMA's page
+ number of this order again. So add it back. */
+ if (IS_ENABLED(CONFIG_CMA) && cma_is_subbed)
+ free_pages += z->free_area[o].cma_nr_free << o;
+
/* Require fewer higher order pages to be free */
min >>= 1;
@@ -4318,6 +4342,7 @@ static void __meminit zone_init_free_lists(struct zone *zone)
for_each_migratetype_order(order, t) {
INIT_LIST_HEAD(&zone->free_area[order].free_list[t]);
zone->free_area[order].nr_free = 0;
+ zone->free_area[order].cma_nr_free = 0;
}
}
@@ -6691,6 +6716,8 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)
list_del(&page->lru);
rmv_page_order(page);
zone->free_area[order].nr_free--;
+ if (is_migrate_cma(get_pageblock_migratetype(page)))
+ zone->free_area[order].cma_nr_free--;
for (i = 0; i < (1 << order); i++)
SetPageReserved((page+i));
pfn += (1 << order);
--
1.9.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice
@ 2014-12-25 9:43 ` Hui Zhu
0 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
In Joonsoo's CMA patch "CMA: always treat free cma pages as non-free on
watermark checking" [1], it changes __zone_watermark_ok to substruct CMA
pages number from free_pages if system use CMA:
if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
But after this part of code
for (o = 0; o < order; o++) {
/* At the next order, this order's pages become unavailable */
free_pages -= z->free_area[o].nr_free << o;
CMA memory in each order is part of z->free_area[o].nr_free, then the CMA
page number of this order is substructed twice. This bug will make
__zone_watermark_ok return more false.
This patch add cma_free_area to struct free_area that just record the number
of CMA pages. And add it back in the order loop to handle the substruct
twice issue.
[1] https://lkml.org/lkml/2014/5/28/110
Signed-off-by: Hui Zhu <zhuhui@xiaomi.com>
Signed-off-by: Weixing Liu <liuweixing@xiaomi.com>
---
include/linux/mmzone.h | 3 +++
mm/page_alloc.c | 29 ++++++++++++++++++++++++++++-
2 files changed, 31 insertions(+), 1 deletion(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index ee1ce1f..7ccad93 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -92,6 +92,9 @@ static inline int get_pfnblock_migratetype(struct page *page, unsigned long pfn)
struct free_area {
struct list_head free_list[MIGRATE_TYPES];
unsigned long nr_free;
+#ifdef CONFIG_CMA
+ unsigned long cma_nr_free;
+#endif
};
struct pglist_data;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1b6c82c..a8d9f03 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -650,6 +650,8 @@ static inline void __free_one_page(struct page *page,
} else {
list_del(&buddy->lru);
zone->free_area[order].nr_free--;
+ if (is_migrate_cma(migratetype))
+ zone->free_area[order].cma_nr_free--;
rmv_page_order(buddy);
}
combined_idx = buddy_idx & page_idx;
@@ -683,6 +685,8 @@ static inline void __free_one_page(struct page *page,
list_add(&page->lru, &zone->free_area[order].free_list[migratetype]);
out:
zone->free_area[order].nr_free++;
+ if (is_migrate_cma(migratetype))
+ zone->free_area[order].cma_nr_free++;
}
static inline int free_pages_check(struct page *page)
@@ -987,6 +991,8 @@ static inline void expand(struct zone *zone, struct page *page,
}
list_add(&page[size].lru, &area->free_list[migratetype]);
area->nr_free++;
+ if (is_migrate_cma(migratetype))
+ area->cma_nr_free++;
set_page_order(&page[size], high);
}
}
@@ -1070,6 +1076,8 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
list_del(&page->lru);
rmv_page_order(page);
area->nr_free--;
+ if (is_migrate_cma(migratetype))
+ area->cma_nr_free--;
expand(zone, page, order, current_order, area, migratetype);
set_freepage_migratetype(page, migratetype);
return page;
@@ -1258,6 +1266,8 @@ __rmqueue_fallback(struct zone *zone, unsigned int order, int start_migratetype)
page = list_entry(area->free_list[migratetype].next,
struct page, lru);
area->nr_free--;
+ if (is_migrate_cma(migratetype))
+ area->cma_nr_free--;
new_type = try_to_steal_freepages(zone, page,
start_migratetype,
@@ -1682,6 +1692,8 @@ int __isolate_free_page(struct page *page, unsigned int order)
/* Remove page from free list */
list_del(&page->lru);
zone->free_area[order].nr_free--;
+ if (is_migrate_cma(mt))
+ zone->free_area[order].cma_nr_free--;
rmv_page_order(page);
/* Set the pageblock if the isolated page is at least a pageblock */
@@ -1893,6 +1905,9 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
/* free_pages may go negative - that's OK */
long min = mark;
int o;
+#ifdef CONFIG_CMA
+ bool cma_is_subbed = false;
+#endif
free_pages -= (1 << order) - 1;
if (alloc_flags & ALLOC_HIGH)
@@ -1905,8 +1920,10 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
* unmovable/reclaimable allocation and they can suddenly
* vanish through CMA allocation
*/
- if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
+ if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages) {
free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
+ cma_is_subbed = true;
+ }
if (free_pages <= min + z->lowmem_reserve[classzone_idx])
return false;
@@ -1914,6 +1931,13 @@ static bool __zone_watermark_ok(struct zone *z, unsigned int order,
/* At the next order, this order's pages become unavailable */
free_pages -= z->free_area[o].nr_free << o;
+ /* If CMA's page number of this order was substructed as part
+ of "zone_page_state(z, NR_FREE_CMA_PAGES)", subtracting
+ "z->free_area[o].nr_free << o" substructed CMA's page
+ number of this order again. So add it back. */
+ if (IS_ENABLED(CONFIG_CMA) && cma_is_subbed)
+ free_pages += z->free_area[o].cma_nr_free << o;
+
/* Require fewer higher order pages to be free */
min >>= 1;
@@ -4318,6 +4342,7 @@ static void __meminit zone_init_free_lists(struct zone *zone)
for_each_migratetype_order(order, t) {
INIT_LIST_HEAD(&zone->free_area[order].free_list[t]);
zone->free_area[order].nr_free = 0;
+ zone->free_area[order].cma_nr_free = 0;
}
}
@@ -6691,6 +6716,8 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)
list_del(&page->lru);
rmv_page_order(page);
zone->free_area[order].nr_free--;
+ if (is_migrate_cma(get_pageblock_migratetype(page)))
+ zone->free_area[order].cma_nr_free--;
for (i = 0; i < (1 << order); i++)
SetPageReserved((page+i));
pfn += (1 << order);
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 2/3] CMA: Fix the issue that nr_try_movable just count MIGRATE_MOVABLE memory
2014-12-25 9:43 ` Hui Zhu
@ 2014-12-25 9:43 ` Hui Zhu
-1 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
One of my plotform that use Joonsoo's CMA patch [1] has a device that
will alloc a lot of MIGRATE_UNMOVABLE memory when it works in a zone.
When this device works, the memory status of this zone is not OK. Most of
CMA is not allocated but most normal memory is allocated.
This issue is because in __rmqueue:
if (IS_ENABLED(CONFIG_CMA) &&
migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
page = __rmqueue_cma(zone, order);
Just allocated MIGRATE_MOVABLE will be record in nr_try_movable in function
__rmqueue_cma but not the others. This device allocated a lot of
MIGRATE_UNMOVABLE memory affect the behavior of this zone memory allocation.
This patch change __rmqueue to let nr_try_movable record all the memory
allocation of normal memory.
[1] https://lkml.org/lkml/2014/5/28/64
Signed-off-by: Hui Zhu <zhuhui@xiaomi.com>
Signed-off-by: Weixing Liu <liuweixing@xiaomi.com>
---
mm/page_alloc.c | 41 ++++++++++++++++++++---------------------
1 file changed, 20 insertions(+), 21 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a8d9f03..a5bbc38 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1301,28 +1301,23 @@ static struct page *__rmqueue_cma(struct zone *zone, unsigned int order)
{
struct page *page;
- if (zone->nr_try_movable > 0)
- goto alloc_movable;
+ if (zone->nr_try_cma <= 0) {
+ /* Reset counter */
+ zone->nr_try_movable = zone->max_try_movable;
+ zone->nr_try_cma = zone->max_try_cma;
- if (zone->nr_try_cma > 0) {
- /* Okay. Now, we can try to allocate the page from cma region */
- zone->nr_try_cma -= 1 << order;
- page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
-
- /* CMA pages can vanish through CMA allocation */
- if (unlikely(!page && order == 0))
- zone->nr_try_cma = 0;
-
- return page;
+ return NULL;
}
- /* Reset counter */
- zone->nr_try_movable = zone->max_try_movable;
- zone->nr_try_cma = zone->max_try_cma;
+ /* Okay. Now, we can try to allocate the page from cma region */
+ zone->nr_try_cma -= 1 << order;
+ page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
-alloc_movable:
- zone->nr_try_movable -= 1 << order;
- return NULL;
+ /* CMA pages can vanish through CMA allocation */
+ if (unlikely(!page && order == 0))
+ zone->nr_try_cma = 0;
+
+ return page;
}
#endif
@@ -1335,9 +1330,13 @@ static struct page *__rmqueue(struct zone *zone, unsigned int order,
{
struct page *page = NULL;
- if (IS_ENABLED(CONFIG_CMA) &&
- migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
- page = __rmqueue_cma(zone, order);
+ if (IS_ENABLED(CONFIG_CMA) && zone->managed_cma_pages) {
+ if (migratetype == MIGRATE_MOVABLE
+ && zone->nr_try_movable <= 0)
+ page = __rmqueue_cma(zone, order);
+ else
+ zone->nr_try_movable -= 1 << order;
+ }
retry_reserve:
if (!page)
--
1.9.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 2/3] CMA: Fix the issue that nr_try_movable just count MIGRATE_MOVABLE memory
@ 2014-12-25 9:43 ` Hui Zhu
0 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
One of my plotform that use Joonsoo's CMA patch [1] has a device that
will alloc a lot of MIGRATE_UNMOVABLE memory when it works in a zone.
When this device works, the memory status of this zone is not OK. Most of
CMA is not allocated but most normal memory is allocated.
This issue is because in __rmqueue:
if (IS_ENABLED(CONFIG_CMA) &&
migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
page = __rmqueue_cma(zone, order);
Just allocated MIGRATE_MOVABLE will be record in nr_try_movable in function
__rmqueue_cma but not the others. This device allocated a lot of
MIGRATE_UNMOVABLE memory affect the behavior of this zone memory allocation.
This patch change __rmqueue to let nr_try_movable record all the memory
allocation of normal memory.
[1] https://lkml.org/lkml/2014/5/28/64
Signed-off-by: Hui Zhu <zhuhui@xiaomi.com>
Signed-off-by: Weixing Liu <liuweixing@xiaomi.com>
---
mm/page_alloc.c | 41 ++++++++++++++++++++---------------------
1 file changed, 20 insertions(+), 21 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a8d9f03..a5bbc38 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1301,28 +1301,23 @@ static struct page *__rmqueue_cma(struct zone *zone, unsigned int order)
{
struct page *page;
- if (zone->nr_try_movable > 0)
- goto alloc_movable;
+ if (zone->nr_try_cma <= 0) {
+ /* Reset counter */
+ zone->nr_try_movable = zone->max_try_movable;
+ zone->nr_try_cma = zone->max_try_cma;
- if (zone->nr_try_cma > 0) {
- /* Okay. Now, we can try to allocate the page from cma region */
- zone->nr_try_cma -= 1 << order;
- page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
-
- /* CMA pages can vanish through CMA allocation */
- if (unlikely(!page && order == 0))
- zone->nr_try_cma = 0;
-
- return page;
+ return NULL;
}
- /* Reset counter */
- zone->nr_try_movable = zone->max_try_movable;
- zone->nr_try_cma = zone->max_try_cma;
+ /* Okay. Now, we can try to allocate the page from cma region */
+ zone->nr_try_cma -= 1 << order;
+ page = __rmqueue_smallest(zone, order, MIGRATE_CMA);
-alloc_movable:
- zone->nr_try_movable -= 1 << order;
- return NULL;
+ /* CMA pages can vanish through CMA allocation */
+ if (unlikely(!page && order == 0))
+ zone->nr_try_cma = 0;
+
+ return page;
}
#endif
@@ -1335,9 +1330,13 @@ static struct page *__rmqueue(struct zone *zone, unsigned int order,
{
struct page *page = NULL;
- if (IS_ENABLED(CONFIG_CMA) &&
- migratetype == MIGRATE_MOVABLE && zone->managed_cma_pages)
- page = __rmqueue_cma(zone, order);
+ if (IS_ENABLED(CONFIG_CMA) && zone->managed_cma_pages) {
+ if (migratetype == MIGRATE_MOVABLE
+ && zone->nr_try_movable <= 0)
+ page = __rmqueue_cma(zone, order);
+ else
+ zone->nr_try_movable -= 1 << order;
+ }
retry_reserve:
if (!page)
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 3/3] CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range
2014-12-25 9:43 ` Hui Zhu
@ 2014-12-25 9:43 ` Hui Zhu
-1 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
In [1], Joonsoo said that cma_alloc_counter is useless because pageblock
is isolated.
But if alloc_contig_range meet a busy range, it will undo_isolate_page_range
before goto try next range. At this time, __rmqueue_cma can begin allocd
CMA memory from the range.
So I add cma_alloc_counter let __rmqueue doesn't call __rmqueue_cma when
cma_alloc works.
[1] https://lkml.org/lkml/2014/10/24/26
Signed-off-by: Hui Zhu <zhuhui@xiaomi.com>
---
include/linux/cma.h | 2 ++
mm/cma.c | 6 ++++++
mm/page_alloc.c | 8 +++++++-
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/include/linux/cma.h b/include/linux/cma.h
index 9384ba6..155158f 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -26,6 +26,8 @@ extern int __init cma_declare_contiguous(phys_addr_t base,
extern int cma_init_reserved_mem(phys_addr_t base,
phys_addr_t size, int order_per_bit,
struct cma **res_cma);
+
+extern atomic_t cma_alloc_counter;
extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
extern bool cma_release(struct cma *cma, struct page *pages, int count);
#endif
diff --git a/mm/cma.c b/mm/cma.c
index 6707b5d..b63f6be 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -348,6 +348,8 @@ err:
return ret;
}
+atomic_t cma_alloc_counter = ATOMIC_INIT(0);
+
/**
* cma_alloc() - allocate pages from contiguous area
* @cma: Contiguous memory region for which the allocation is performed.
@@ -378,6 +380,8 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
bitmap_maxno = cma_bitmap_maxno(cma);
bitmap_count = cma_bitmap_pages_to_bits(cma, count);
+ atomic_inc(&cma_alloc_counter);
+
for (;;) {
mutex_lock(&cma->lock);
bitmap_no = bitmap_find_next_zero_area_off(cma->bitmap,
@@ -415,6 +419,8 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
start = bitmap_no + mask + 1;
}
+ atomic_dec(&cma_alloc_counter);
+
pr_debug("%s(): returned %p\n", __func__, page);
return page;
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a5bbc38..0622c4c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -66,6 +66,10 @@
#include <asm/div64.h>
#include "internal.h"
+#ifdef CONFIG_CMA
+#include <linux/cma.h>
+#endif
+
/* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */
static DEFINE_MUTEX(pcp_batch_high_lock);
#define MIN_PERCPU_PAGELIST_FRACTION (8)
@@ -1330,7 +1334,9 @@ static struct page *__rmqueue(struct zone *zone, unsigned int order,
{
struct page *page = NULL;
- if (IS_ENABLED(CONFIG_CMA) && zone->managed_cma_pages) {
+ if (IS_ENABLED(CONFIG_CMA)
+ && zone->managed_cma_pages
+ && atomic_read(&cma_alloc_counter) == 0) {
if (migratetype == MIGRATE_MOVABLE
&& zone->nr_try_movable <= 0)
page = __rmqueue_cma(zone, order);
--
1.9.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 3/3] CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range
@ 2014-12-25 9:43 ` Hui Zhu
0 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-25 9:43 UTC (permalink / raw)
To: m.szyprowski, mina86, akpm, iamjoonsoo.kim, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
Cc: teawater, Hui Zhu
In [1], Joonsoo said that cma_alloc_counter is useless because pageblock
is isolated.
But if alloc_contig_range meet a busy range, it will undo_isolate_page_range
before goto try next range. At this time, __rmqueue_cma can begin allocd
CMA memory from the range.
So I add cma_alloc_counter let __rmqueue doesn't call __rmqueue_cma when
cma_alloc works.
[1] https://lkml.org/lkml/2014/10/24/26
Signed-off-by: Hui Zhu <zhuhui@xiaomi.com>
---
include/linux/cma.h | 2 ++
mm/cma.c | 6 ++++++
mm/page_alloc.c | 8 +++++++-
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/include/linux/cma.h b/include/linux/cma.h
index 9384ba6..155158f 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -26,6 +26,8 @@ extern int __init cma_declare_contiguous(phys_addr_t base,
extern int cma_init_reserved_mem(phys_addr_t base,
phys_addr_t size, int order_per_bit,
struct cma **res_cma);
+
+extern atomic_t cma_alloc_counter;
extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
extern bool cma_release(struct cma *cma, struct page *pages, int count);
#endif
diff --git a/mm/cma.c b/mm/cma.c
index 6707b5d..b63f6be 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -348,6 +348,8 @@ err:
return ret;
}
+atomic_t cma_alloc_counter = ATOMIC_INIT(0);
+
/**
* cma_alloc() - allocate pages from contiguous area
* @cma: Contiguous memory region for which the allocation is performed.
@@ -378,6 +380,8 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
bitmap_maxno = cma_bitmap_maxno(cma);
bitmap_count = cma_bitmap_pages_to_bits(cma, count);
+ atomic_inc(&cma_alloc_counter);
+
for (;;) {
mutex_lock(&cma->lock);
bitmap_no = bitmap_find_next_zero_area_off(cma->bitmap,
@@ -415,6 +419,8 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
start = bitmap_no + mask + 1;
}
+ atomic_dec(&cma_alloc_counter);
+
pr_debug("%s(): returned %p\n", __func__, page);
return page;
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a5bbc38..0622c4c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -66,6 +66,10 @@
#include <asm/div64.h>
#include "internal.h"
+#ifdef CONFIG_CMA
+#include <linux/cma.h>
+#endif
+
/* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */
static DEFINE_MUTEX(pcp_batch_high_lock);
#define MIN_PERCPU_PAGELIST_FRACTION (8)
@@ -1330,7 +1334,9 @@ static struct page *__rmqueue(struct zone *zone, unsigned int order,
{
struct page *page = NULL;
- if (IS_ENABLED(CONFIG_CMA) && zone->managed_cma_pages) {
+ if (IS_ENABLED(CONFIG_CMA)
+ && zone->managed_cma_pages
+ && atomic_read(&cma_alloc_counter) == 0) {
if (migratetype == MIGRATE_MOVABLE
&& zone->nr_try_movable <= 0)
page = __rmqueue_cma(zone, order);
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice
2014-12-25 9:43 ` Hui Zhu
@ 2014-12-30 4:48 ` Joonsoo Kim
-1 siblings, 0 replies; 14+ messages in thread
From: Joonsoo Kim @ 2014-12-30 4:48 UTC (permalink / raw)
To: Hui Zhu
Cc: m.szyprowski, mina86, akpm, aneesh.kumar, pintu.k, weijie.yang,
mgorman, hannes, riel, vbabka, laurent.pinchart+renesas,
rientjes, sasha.levin, liuweixing, linux-kernel, linux-mm,
teawater
On Thu, Dec 25, 2014 at 05:43:26PM +0800, Hui Zhu wrote:
> In Joonsoo's CMA patch "CMA: always treat free cma pages as non-free on
> watermark checking" [1], it changes __zone_watermark_ok to substruct CMA
> pages number from free_pages if system use CMA:
> if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
> free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
Hello,
In fact, without that patch, watermark checking has a problem in current kernel.
If there is reserved CMA region, watermark check for high order
allocation is done loosly. See following thread.
https://lkml.org/lkml/2014/5/30/320
Your patch can fix this situation, so, how about submitting this patch
separately?
Thanks.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice
@ 2014-12-30 4:48 ` Joonsoo Kim
0 siblings, 0 replies; 14+ messages in thread
From: Joonsoo Kim @ 2014-12-30 4:48 UTC (permalink / raw)
To: Hui Zhu
Cc: m.szyprowski, mina86, akpm, aneesh.kumar, pintu.k, weijie.yang,
mgorman, hannes, riel, vbabka, laurent.pinchart+renesas,
rientjes, sasha.levin, liuweixing, linux-kernel, linux-mm,
teawater
On Thu, Dec 25, 2014 at 05:43:26PM +0800, Hui Zhu wrote:
> In Joonsoo's CMA patch "CMA: always treat free cma pages as non-free on
> watermark checking" [1], it changes __zone_watermark_ok to substruct CMA
> pages number from free_pages if system use CMA:
> if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
> free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
Hello,
In fact, without that patch, watermark checking has a problem in current kernel.
If there is reserved CMA region, watermark check for high order
allocation is done loosly. See following thread.
https://lkml.org/lkml/2014/5/30/320
Your patch can fix this situation, so, how about submitting this patch
separately?
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 3/3] CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range
2014-12-25 9:43 ` Hui Zhu
@ 2014-12-30 5:00 ` Joonsoo Kim
-1 siblings, 0 replies; 14+ messages in thread
From: Joonsoo Kim @ 2014-12-30 5:00 UTC (permalink / raw)
To: Hui Zhu
Cc: m.szyprowski, mina86, akpm, aneesh.kumar, pintu.k, weijie.yang,
mgorman, hannes, riel, vbabka, laurent.pinchart+renesas,
rientjes, sasha.levin, liuweixing, linux-kernel, linux-mm,
teawater
On Thu, Dec 25, 2014 at 05:43:28PM +0800, Hui Zhu wrote:
> In [1], Joonsoo said that cma_alloc_counter is useless because pageblock
> is isolated.
> But if alloc_contig_range meet a busy range, it will undo_isolate_page_range
> before goto try next range. At this time, __rmqueue_cma can begin allocd
> CMA memory from the range.
Is there any real issue from this?
When failed, we will quickly re-isolate pageblock for adjacent page
so there is no big problem I guess.
If there is real issue, how about doing start_isolation/undo_isolation
in cma_alloc()? It would reduce useless do/undo isolation due to
failed trial.
Thanks.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 3/3] CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range
@ 2014-12-30 5:00 ` Joonsoo Kim
0 siblings, 0 replies; 14+ messages in thread
From: Joonsoo Kim @ 2014-12-30 5:00 UTC (permalink / raw)
To: Hui Zhu
Cc: m.szyprowski, mina86, akpm, aneesh.kumar, pintu.k, weijie.yang,
mgorman, hannes, riel, vbabka, laurent.pinchart+renesas,
rientjes, sasha.levin, liuweixing, linux-kernel, linux-mm,
teawater
On Thu, Dec 25, 2014 at 05:43:28PM +0800, Hui Zhu wrote:
> In [1], Joonsoo said that cma_alloc_counter is useless because pageblock
> is isolated.
> But if alloc_contig_range meet a busy range, it will undo_isolate_page_range
> before goto try next range. At this time, __rmqueue_cma can begin allocd
> CMA memory from the range.
Is there any real issue from this?
When failed, we will quickly re-isolate pageblock for adjacent page
so there is no big problem I guess.
If there is real issue, how about doing start_isolation/undo_isolation
in cma_alloc()? It would reduce useless do/undo isolation due to
failed trial.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice
2014-12-30 4:48 ` Joonsoo Kim
@ 2014-12-30 10:02 ` Hui Zhu
-1 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-30 10:02 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Hui Zhu, m.szyprowski, mina86, Andrew Morton, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, Rik van Riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
On Tue, Dec 30, 2014 at 12:48 PM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Thu, Dec 25, 2014 at 05:43:26PM +0800, Hui Zhu wrote:
>> In Joonsoo's CMA patch "CMA: always treat free cma pages as non-free on
>> watermark checking" [1], it changes __zone_watermark_ok to substruct CMA
>> pages number from free_pages if system use CMA:
>> if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
>> free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
>
> Hello,
>
> In fact, without that patch, watermark checking has a problem in current kernel.
> If there is reserved CMA region, watermark check for high order
> allocation is done loosly. See following thread.
>
> https://lkml.org/lkml/2014/5/30/320
>
> Your patch can fix this situation, so, how about submitting this patch
> separately?
>
> Thanks.
>
Hi Joonsoo,
Thanks for your remind. I will post a separate patch for current kernel.
Thanks,
Hui
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice
@ 2014-12-30 10:02 ` Hui Zhu
0 siblings, 0 replies; 14+ messages in thread
From: Hui Zhu @ 2014-12-30 10:02 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Hui Zhu, m.szyprowski, mina86, Andrew Morton, aneesh.kumar,
pintu.k, weijie.yang, mgorman, hannes, Rik van Riel, vbabka,
laurent.pinchart+renesas, rientjes, sasha.levin, liuweixing,
linux-kernel, linux-mm
On Tue, Dec 30, 2014 at 12:48 PM, Joonsoo Kim <iamjoonsoo.kim@lge.com> wrote:
> On Thu, Dec 25, 2014 at 05:43:26PM +0800, Hui Zhu wrote:
>> In Joonsoo's CMA patch "CMA: always treat free cma pages as non-free on
>> watermark checking" [1], it changes __zone_watermark_ok to substruct CMA
>> pages number from free_pages if system use CMA:
>> if (IS_ENABLED(CONFIG_CMA) && z->managed_cma_pages)
>> free_pages -= zone_page_state(z, NR_FREE_CMA_PAGES);
>
> Hello,
>
> In fact, without that patch, watermark checking has a problem in current kernel.
> If there is reserved CMA region, watermark check for high order
> allocation is done loosly. See following thread.
>
> https://lkml.org/lkml/2014/5/30/320
>
> Your patch can fix this situation, so, how about submitting this patch
> separately?
>
> Thanks.
>
Hi Joonsoo,
Thanks for your remind. I will post a separate patch for current kernel.
Thanks,
Hui
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2014-12-30 10:02 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-12-25 9:43 [PATCH 0/3] CMA: Handle the issues of aggressively allocate the Hui Zhu
2014-12-25 9:43 ` Hui Zhu
2014-12-25 9:43 ` [PATCH 1/3] CMA: Fix the bug that CMA's page number is substructed twice Hui Zhu
2014-12-25 9:43 ` Hui Zhu
2014-12-30 4:48 ` Joonsoo Kim
2014-12-30 4:48 ` Joonsoo Kim
2014-12-30 10:02 ` Hui Zhu
2014-12-30 10:02 ` Hui Zhu
2014-12-25 9:43 ` [PATCH 2/3] CMA: Fix the issue that nr_try_movable just count MIGRATE_MOVABLE memory Hui Zhu
2014-12-25 9:43 ` Hui Zhu
2014-12-25 9:43 ` [PATCH 3/3] CMA: Add cma_alloc_counter to make cma_alloc work better if it meet busy range Hui Zhu
2014-12-25 9:43 ` Hui Zhu
2014-12-30 5:00 ` Joonsoo Kim
2014-12-30 5:00 ` Joonsoo Kim
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.