From: Marek Szyprowski <m.szyprowski@samsung.com>
To: linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-media@vger.kernel.org, linux-mm@kvack.org,
linaro-mm-sig@lists.linaro.org
Cc: Michal Nazarewicz <mina86@mina86.com>,
Marek Szyprowski <m.szyprowski@samsung.com>,
Kyungmin Park <kyungmin.park@samsung.com>,
Russell King <linux@arm.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
Daniel Walker <dwalker@codeaurora.org>,
Mel Gorman <mel@csn.ul.ie>, Arnd Bergmann <arnd@arndb.de>,
Jesse Barker <jesse.barker@linaro.org>,
Jonathan Corbet <corbet@lwn.net>,
Chunsang Jeong <chunsang.jeong@linaro.org>,
Dave Hansen <dave@linux.vnet.ibm.com>,
Benjamin Gaignard <benjamin.gaignard@linaro.org>,
Rob Clark <rob.clark@linaro.org>,
Ohad Ben-Cohen <ohad@wizery.com>,
Sandeep Patil <psandeep.s@gmail.com>
Subject: [PATCHv24 12/16] mm: trigger page reclaim in alloc_contig_range() to stabilise watermarks
Date: Tue, 03 Apr 2012 16:10:17 +0200 [thread overview]
Message-ID: <1333462221-3987-13-git-send-email-m.szyprowski@samsung.com> (raw)
In-Reply-To: <1333462221-3987-1-git-send-email-m.szyprowski@samsung.com>
alloc_contig_range() performs memory allocation so it also should keep
track on keeping the correct level of memory watermarks. This commit adds
a call to *_slowpath style reclaim to grab enough pages to make sure that
the final collection of contiguous pages from freelists will not starve
the system.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
CC: Michal Nazarewicz <mina86@mina86.com>
Tested-by: Rob Clark <rob.clark@linaro.org>
Tested-by: Ohad Ben-Cohen <ohad@wizery.com>
Tested-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
Tested-by: Robert Nelson <robertcnelson@gmail.com>
Tested-by: Barry Song <Baohua.Song@csr.com>
---
include/linux/mmzone.h | 9 +++++++
mm/page_alloc.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 69 insertions(+), 0 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 8c1335f..26f2040 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -63,8 +63,10 @@ enum {
#ifdef CONFIG_CMA
# define is_migrate_cma(migratetype) unlikely((migratetype) == MIGRATE_CMA)
+# define cma_wmark_pages(zone) zone->min_cma_pages
#else
# define is_migrate_cma(migratetype) false
+# define cma_wmark_pages(zone) 0
#endif
#define for_each_migratetype_order(order, type) \
@@ -371,6 +373,13 @@ struct zone {
/* see spanned/present_pages for more description */
seqlock_t span_seqlock;
#endif
+#ifdef CONFIG_CMA
+ /*
+ * CMA needs to increase watermark levels during the allocation
+ * process to make sure that the system is not starved.
+ */
+ unsigned long min_cma_pages;
+#endif
struct free_area free_area[MAX_ORDER];
#ifndef CONFIG_SPARSEMEM
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 5a90ada..17a5521 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5079,6 +5079,11 @@ static void __setup_per_zone_wmarks(void)
zone->watermark[WMARK_LOW] = min_wmark_pages(zone) + (tmp >> 2);
zone->watermark[WMARK_HIGH] = min_wmark_pages(zone) + (tmp >> 1);
+
+ zone->watermark[WMARK_MIN] += cma_wmark_pages(zone);
+ zone->watermark[WMARK_LOW] += cma_wmark_pages(zone);
+ zone->watermark[WMARK_HIGH] += cma_wmark_pages(zone);
+
setup_zone_migrate_reserve(zone);
spin_unlock_irqrestore(&zone->lock, flags);
}
@@ -5684,6 +5689,54 @@ static int __alloc_contig_migrate_range(unsigned long start, unsigned long end)
return ret > 0 ? 0 : ret;
}
+/*
+ * Update zone's cma pages counter used for watermark level calculation.
+ */
+static inline void __update_cma_watermarks(struct zone *zone, int count)
+{
+ unsigned long flags;
+ spin_lock_irqsave(&zone->lock, flags);
+ zone->min_cma_pages += count;
+ spin_unlock_irqrestore(&zone->lock, flags);
+ setup_per_zone_wmarks();
+}
+
+/*
+ * Trigger memory pressure bump to reclaim some pages in order to be able to
+ * allocate 'count' pages in single page units. Does similar work as
+ *__alloc_pages_slowpath() function.
+ */
+static int __reclaim_pages(struct zone *zone, gfp_t gfp_mask, int count)
+{
+ enum zone_type high_zoneidx = gfp_zone(gfp_mask);
+ struct zonelist *zonelist = node_zonelist(0, gfp_mask);
+ int did_some_progress = 0;
+ int order = 1;
+
+ /*
+ * Increase level of watermarks to force kswapd do his job
+ * to stabilise at new watermark level.
+ */
+ __update_cma_watermarks(zone, count);
+
+ /* Obey watermarks as if the page was being allocated */
+ while (!zone_watermark_ok(zone, 0, low_wmark_pages(zone), 0, 0)) {
+ wake_all_kswapd(order, zonelist, high_zoneidx, zone_idx(zone));
+
+ did_some_progress = __perform_reclaim(gfp_mask, order, zonelist,
+ NULL);
+ if (!did_some_progress) {
+ /* Exhausted what can be done so it's blamo time */
+ out_of_memory(zonelist, gfp_mask, order, NULL, false);
+ }
+ }
+
+ /* Restore original watermark levels. */
+ __update_cma_watermarks(zone, -count);
+
+ return count;
+}
+
/**
* alloc_contig_range() -- tries to allocate given range of pages
* @start: start PFN to allocate
@@ -5782,6 +5835,13 @@ int alloc_contig_range(unsigned long start, unsigned long end,
goto done;
}
+ /*
+ * Reclaim enough pages to make sure that contiguous allocation
+ * will not starve the system.
+ */
+ __reclaim_pages(zone, GFP_HIGHUSER_MOVABLE, end-start);
+
+ /* Grab isolated pages from freelists. */
outer_end = isolate_freepages_range(outer_start, end);
if (!outer_end) {
ret = -EBUSY;
--
1.7.1.569.g6f426
next prev parent reply other threads:[~2012-04-03 14:11 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-04-03 14:10 [PATCHv24 00/16] Contiguous Memory Allocator Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 01/16] mm: page_alloc: remove trailing whitespace Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 02/16] mm: compaction: introduce isolate_migratepages_range() Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 03/16] mm: compaction: introduce map_pages() Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 04/16] mm: compaction: introduce isolate_freepages_range() Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 05/16] mm: compaction: export some of the functions Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 06/16] mm: page_alloc: introduce alloc_contig_range() Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 07/16] mm: page_alloc: change fallbacks array handling Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 08/16] mm: mmzone: MIGRATE_CMA migration type added Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 09/16] mm: page_isolation: MIGRATE_CMA isolation functions added Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 10/16] mm: Serialize access to min_free_kbytes Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 11/16] mm: extract reclaim code from __alloc_pages_direct_reclaim() Marek Szyprowski
2012-04-03 14:10 ` Marek Szyprowski [this message]
2012-04-03 14:10 ` [PATCHv24 13/16] drivers: add Contiguous Memory Allocator Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 14/16] X86: integrate CMA with DMA-mapping subsystem Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 15/16] ARM: " Marek Szyprowski
2012-04-03 14:10 ` [PATCHv24 16/16] ARM: Samsung: use CMA for 2 memory banks for s5p-mfc device Marek Szyprowski
2012-04-10 13:41 ` [PATCHv24 00/16] Contiguous Memory Allocator Aaro Koskinen
2012-04-10 17:19 ` Sandeep Patil
2012-04-11 6:48 ` Marek Szyprowski
2012-04-12 12:39 ` cary.zou
2012-04-19 19:40 ` Andrew Morton
2012-04-20 12:22 ` Marek Szyprowski
2012-04-20 18:30 ` Kumar Gala
2012-04-20 18:31 ` Kumar Gala
2012-04-30 3:38 ` [Linaro-mm-sig] " Barry Song
2012-05-10 13:19 ` [PATCH] drivers: cma: don't fail if migration returns -EAGAIN Marek Szyprowski
2012-05-10 15:14 ` Minchan Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1333462221-3987-13-git-send-email-m.szyprowski@samsung.com \
--to=m.szyprowski@samsung.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=benjamin.gaignard@linaro.org \
--cc=chunsang.jeong@linaro.org \
--cc=corbet@lwn.net \
--cc=dave@linux.vnet.ibm.com \
--cc=dwalker@codeaurora.org \
--cc=jesse.barker@linaro.org \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kyungmin.park@samsung.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux@arm.linux.org.uk \
--cc=mel@csn.ul.ie \
--cc=mina86@mina86.com \
--cc=ohad@wizery.com \
--cc=psandeep.s@gmail.com \
--cc=rob.clark@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).