From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: Wu Fengguang <fengguang.wu@intel.com>
Cc: kosaki.motohiro@jp.fujitsu.com, Andreas Mohr <andi@lisas.de>,
Jens Axboe <axboe@kernel.dk>, Minchan Kim <minchan.kim@gmail.com>,
Linux Memory Management List <linux-mm@kvack.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Rik van Riel <riel@redhat.com>,
Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Subject: [PATCH] vmscan: page_check_references() check low order lumpy reclaim properly
Date: Fri, 16 Apr 2010 12:16:18 +0900 (JST) [thread overview]
Message-ID: <20100416115437.27AD.A69D9226@jp.fujitsu.com> (raw)
In-Reply-To: <20100415051911.GA17110@localhost>
> On Thu, Apr 15, 2010 at 12:55:30PM +0800, KOSAKI Motohiro wrote:
> > > On Thu, Apr 15, 2010 at 12:32:50PM +0800, KOSAKI Motohiro wrote:
> > > > > On Thu, Apr 15, 2010 at 11:31:52AM +0800, KOSAKI Motohiro wrote:
> > > > > > > > Many applications (this one and below) are stuck in
> > > > > > > > wait_on_page_writeback(). I guess this is why "heavy write to
> > > > > > > > irrelevant partition stalls the whole system". They are stuck on page
> > > > > > > > allocation. Your 512MB system memory is a bit tight, so reclaim
> > > > > > > > pressure is a bit high, which triggers the wait-on-writeback logic.
> > > > > > >
> > > > > > > I wonder if this hacking patch may help.
> > > > > > >
> > > > > > > When creating 300MB dirty file with dd, it is creating continuous
> > > > > > > region of hard-to-reclaim pages in the LRU list. priority can easily
> > > > > > > go low when irrelevant applications' direct reclaim run into these
> > > > > > > regions..
> > > > > >
> > > > > > Sorry I'm confused not. can you please tell us more detail explanation?
> > > > > > Why did lumpy reclaim cause OOM? lumpy reclaim might cause
> > > > > > direct reclaim slow down. but IIUC it's not cause OOM because OOM is
> > > > > > only occur when priority-0 reclaim failure.
> > > > >
> > > > > No I'm not talking OOM. Nor lumpy reclaim.
> > > > >
> > > > > I mean the direct reclaim can get stuck for long time, when we do
> > > > > wait_on_page_writeback() on lumpy_reclaim=1.
> > > > >
> > > > > > IO get stcking also prevent priority reach to 0.
> > > > >
> > > > > Sure. But we can wait for IO a bit later -- after scanning 1/64 LRU
> > > > > (the below patch) instead of the current 1/1024.
> > > > >
> > > > > In Andreas' case, 512MB/1024 = 512KB, this is way too low comparing to
> > > > > the 22MB writeback pages. There can easily be a continuous range of
> > > > > 512KB dirty/writeback pages in the LRU, which will trigger the wait
> > > > > logic.
> > > >
> > > > In my feeling from your explanation, we need auto adjustment mechanism
> > > > instead change default value for special machine. no?
> > >
> > > You mean the dumb DEF_PRIORITY/2 may be too large for a 1TB memory box?
> > >
> > > However for such boxes, whether it be DEF_PRIORITY-2 or DEF_PRIORITY/2
> > > shall be irrelevant: it's trivial anyway to reclaim an order-1 or
> > > order-2 page. In other word, lumpy_reclaim will hardly go 1. Do you
> > > think so?
> >
> > If my remember is correct, Its order-1 lumpy reclaim was introduced
> > for solving such big box + AIM7 workload made kernel stack (order-1 page)
> > allocation failure.
> >
> > Now, We are living on moore's law. so probably we need to pay attention
> > scalability always. today's big box is going to become desktop box after
> > 3-5 years.
> >
> > Probably, Lee know such problem than me. cc to him.
>
> In Andreas' trace, the processes are blocked in
> - do_fork: console-kit-d
> - __alloc_skb: x-terminal-em, konqueror
> - handle_mm_fault: tclsh
> - filemap_fault: ls
>
> I'm a bit confused by the last one, and wonder what's the typical
> gfp order of __alloc_skb().
Probably I've found one of reason of low order lumpy reclaim slow down.
Let's fix obvious bug at first!
============================================================
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Subject: [PATCH] vmscan: page_check_references() check low order lumpy reclaim properly
If vmscan is under lumpy reclaim mode, it have to ignore referenced bit
for making contenious free pages. but current page_check_references()
doesn't.
Fixes it.
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
mm/vmscan.c | 32 +++++++++++++++++---------------
1 files changed, 17 insertions(+), 15 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 3ff3311..13d9546 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -77,6 +77,8 @@ struct scan_control {
int order;
+ int lumpy_reclaim;
+
/* Which cgroup do we reclaim from */
struct mem_cgroup *mem_cgroup;
@@ -575,7 +577,7 @@ static enum page_references page_check_references(struct page *page,
referenced_page = TestClearPageReferenced(page);
/* Lumpy reclaim - ignore references */
- if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
+ if (sc->lumpy_reclaim)
return PAGEREF_RECLAIM;
/*
@@ -1130,7 +1132,6 @@ static unsigned long shrink_inactive_list(unsigned long max_scan,
unsigned long nr_scanned = 0;
unsigned long nr_reclaimed = 0;
struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
- int lumpy_reclaim = 0;
while (unlikely(too_many_isolated(zone, file, sc))) {
congestion_wait(BLK_RW_ASYNC, HZ/10);
@@ -1140,17 +1141,6 @@ static unsigned long shrink_inactive_list(unsigned long max_scan,
return SWAP_CLUSTER_MAX;
}
- /*
- * If we need a large contiguous chunk of memory, or have
- * trouble getting a small set of contiguous pages, we
- * will reclaim both active and inactive pages.
- *
- * We use the same threshold as pageout congestion_wait below.
- */
- if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
- lumpy_reclaim = 1;
- else if (sc->order && priority < DEF_PRIORITY - 2)
- lumpy_reclaim = 1;
pagevec_init(&pvec, 1);
@@ -1163,7 +1153,7 @@ static unsigned long shrink_inactive_list(unsigned long max_scan,
unsigned long nr_freed;
unsigned long nr_active;
unsigned int count[NR_LRU_LISTS] = { 0, };
- int mode = lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
+ int mode = sc->lumpy_reclaim ? ISOLATE_BOTH : ISOLATE_INACTIVE;
unsigned long nr_anon;
unsigned long nr_file;
@@ -1216,7 +1206,7 @@ static unsigned long shrink_inactive_list(unsigned long max_scan,
* but that should be acceptable to the caller
*/
if (nr_freed < nr_taken && !current_is_kswapd() &&
- lumpy_reclaim) {
+ sc->lumpy_reclaim) {
congestion_wait(BLK_RW_ASYNC, HZ/10);
/*
@@ -1655,6 +1645,18 @@ static void shrink_zone(int priority, struct zone *zone,
&reclaim_stat->nr_saved_scan[l]);
}
+ /*
+ * If we need a large contiguous chunk of memory, or have
+ * trouble getting a small set of contiguous pages, we
+ * will reclaim both active and inactive pages.
+ */
+ if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
+ sc->lumpy_reclaim = 1;
+ else if (sc->order && priority < DEF_PRIORITY - 2)
+ sc->lumpy_reclaim = 1;
+ else
+ sc->lumpy_reclaim = 0;
+
while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] ||
nr[LRU_INACTIVE_FILE]) {
for_each_evictable_lru(l) {
--
1.6.5.2
next prev parent reply other threads:[~2010-04-16 3:16 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-04-04 22:13 32GB SSD on USB1.1 P3/700 == ___HELL___ (2.6.34-rc3) Andreas Mohr
2010-04-04 23:31 ` Gábor Lénárt
2010-04-05 10:53 ` Andreas Mohr
2010-04-07 7:00 ` Wu Fengguang
2010-04-07 7:08 ` Wu Fengguang
2010-04-15 3:31 ` KOSAKI Motohiro
2010-04-15 4:19 ` Wu Fengguang
2010-04-15 4:32 ` KOSAKI Motohiro
2010-04-15 4:41 ` Wu Fengguang
2010-04-15 4:55 ` KOSAKI Motohiro
2010-04-15 5:19 ` Wu Fengguang
2010-04-16 3:16 ` KOSAKI Motohiro [this message]
2010-04-16 4:26 ` [PATCH] vmscan: page_check_references() check low order lumpy reclaim properly Minchan Kim
2010-04-16 5:33 ` KOSAKI Motohiro
2010-04-16 21:18 ` Andrew Morton
2010-05-13 2:54 ` KOSAKI Motohiro
2010-04-07 8:39 ` 32GB SSD on USB1.1 P3/700 == ___HELL___ (2.6.34-rc3) Minchan Kim
2010-04-07 8:52 ` Wu Fengguang
2010-04-07 11:17 ` Andreas Mohr
2010-04-08 19:46 ` Andreas Mohr
2010-04-08 20:12 ` Bill Davidsen
2010-04-08 20:35 ` Andreas Mohr
2010-04-08 22:01 ` Bill Davidsen
2010-04-09 15:56 ` Ben Gamari
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100416115437.27AD.A69D9226@jp.fujitsu.com \
--to=kosaki.motohiro@jp.fujitsu.com \
--cc=Lee.Schermerhorn@hp.com \
--cc=andi@lisas.de \
--cc=axboe@kernel.dk \
--cc=fengguang.wu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan.kim@gmail.com \
--cc=riel@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).