All of lore.kernel.org
 help / color / mirror / Atom feed
From: Johannes Weiner <hannes@cmpxchg.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>,
	Mel Gorman <mel@csn.ul.ie>, Rik van Riel <riel@redhat.com>,
	Michal Hocko <mhocko@suse.cz>,
	Kent Overstreet <kent.overstreet@gmail.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: [patch] vmscan: fix zone shrinking exit when scan work is done
Date: Wed, 9 Feb 2011 16:46:06 +0100	[thread overview]
Message-ID: <20110209154606.GJ27110@cmpxchg.org> (raw)

Hi,

I think this should fix the problem of processes getting stuck in
reclaim that has been reported several times.  Kent actually
single-stepped through this code and noted that it was never exiting
shrink_zone(), which really narrowed it down a lot, considering the
tons of nested loops from the allocator down to the list shrinking.

	Hannes

---
From: Johannes Weiner <hannes@cmpxchg.org>
Subject: vmscan: fix zone shrinking exit when scan work is done

'3e7d344 mm: vmscan: reclaim order-0 and use compaction instead of
lumpy reclaim' introduced an indefinite loop in shrink_zone().

It meant to break out of this loop when no pages had been reclaimed
and not a single page was even scanned.  The way it would detect the
latter is by taking a snapshot of sc->nr_scanned at the beginning of
the function and comparing it against the new sc->nr_scanned after the
scan loop.  But it would re-iterate without updating that snapshot,
looping forever if sc->nr_scanned changed at least once since
shrink_zone() was invoked.

This is not the sole condition that would exit that loop, but it
requires other processes to change the zone state, as the reclaimer
that is stuck obviously can not anymore.

This is only happening for higher-order allocations, where reclaim is
run back to back with compaction.

Reported-by: Michal Hocko <mhocko@suse.cz>
Reported-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 mm/vmscan.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 148c6e6..17497d0 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1882,12 +1882,12 @@ static void shrink_zone(int priority, struct zone *zone,
 	unsigned long nr[NR_LRU_LISTS];
 	unsigned long nr_to_scan;
 	enum lru_list l;
-	unsigned long nr_reclaimed;
+	unsigned long nr_reclaimed, nr_scanned;
 	unsigned long nr_to_reclaim = sc->nr_to_reclaim;
-	unsigned long nr_scanned = sc->nr_scanned;
 
 restart:
 	nr_reclaimed = 0;
+	nr_scanned = sc->nr_scanned;
 	get_scan_count(zone, sc, nr, priority);
 
 	while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] ||
-- 
1.7.4


WARNING: multiple messages have this Message-ID (diff)
From: Johannes Weiner <hannes@cmpxchg.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>,
	Mel Gorman <mel@csn.ul.ie>, Rik van Riel <riel@redhat.com>,
	Michal Hocko <mhocko@suse.cz>,
	Kent Overstreet <kent.overstreet@gmail.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: [patch] vmscan: fix zone shrinking exit when scan work is done
Date: Wed, 9 Feb 2011 16:46:06 +0100	[thread overview]
Message-ID: <20110209154606.GJ27110@cmpxchg.org> (raw)

Hi,

I think this should fix the problem of processes getting stuck in
reclaim that has been reported several times.  Kent actually
single-stepped through this code and noted that it was never exiting
shrink_zone(), which really narrowed it down a lot, considering the
tons of nested loops from the allocator down to the list shrinking.

	Hannes

---
From: Johannes Weiner <hannes@cmpxchg.org>
Subject: vmscan: fix zone shrinking exit when scan work is done

'3e7d344 mm: vmscan: reclaim order-0 and use compaction instead of
lumpy reclaim' introduced an indefinite loop in shrink_zone().

It meant to break out of this loop when no pages had been reclaimed
and not a single page was even scanned.  The way it would detect the
latter is by taking a snapshot of sc->nr_scanned at the beginning of
the function and comparing it against the new sc->nr_scanned after the
scan loop.  But it would re-iterate without updating that snapshot,
looping forever if sc->nr_scanned changed at least once since
shrink_zone() was invoked.

This is not the sole condition that would exit that loop, but it
requires other processes to change the zone state, as the reclaimer
that is stuck obviously can not anymore.

This is only happening for higher-order allocations, where reclaim is
run back to back with compaction.

Reported-by: Michal Hocko <mhocko@suse.cz>
Reported-by: Kent Overstreet <kent.overstreet@gmail.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
 mm/vmscan.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 148c6e6..17497d0 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1882,12 +1882,12 @@ static void shrink_zone(int priority, struct zone *zone,
 	unsigned long nr[NR_LRU_LISTS];
 	unsigned long nr_to_scan;
 	enum lru_list l;
-	unsigned long nr_reclaimed;
+	unsigned long nr_reclaimed, nr_scanned;
 	unsigned long nr_to_reclaim = sc->nr_to_reclaim;
-	unsigned long nr_scanned = sc->nr_scanned;
 
 restart:
 	nr_reclaimed = 0;
+	nr_scanned = sc->nr_scanned;
 	get_scan_count(zone, sc, nr, priority);
 
 	while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] ||
-- 
1.7.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

             reply	other threads:[~2011-02-09 15:46 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-02-09 15:46 Johannes Weiner [this message]
2011-02-09 15:46 ` [patch] vmscan: fix zone shrinking exit when scan work is done Johannes Weiner
2011-02-09 15:54 ` Kent Overstreet
2011-02-09 15:54   ` Kent Overstreet
2011-02-09 16:46 ` Mel Gorman
2011-02-09 16:46   ` Mel Gorman
2011-02-09 18:28   ` Andrea Arcangeli
2011-02-09 18:28     ` Andrea Arcangeli
2011-02-09 20:05     ` Andrew Morton
2011-02-09 20:05       ` Andrew Morton
2011-02-10 10:21     ` Mel Gorman
2011-02-10 10:21       ` Mel Gorman
2011-02-10 10:41       ` Michal Hocko
2011-02-10 10:41         ` Michal Hocko
2011-02-10 12:48       ` Andrea Arcangeli
2011-02-10 12:48         ` Andrea Arcangeli
2011-02-10 13:33         ` Mel Gorman
2011-02-10 13:33           ` Mel Gorman
2011-02-10 14:14           ` Andrea Arcangeli
2011-02-10 14:14             ` Andrea Arcangeli
2011-02-10 14:58             ` Mel Gorman
2011-02-10 14:58               ` Mel Gorman
2011-02-16  9:50               ` [PATCH] mm: vmscan: Stop reclaim/compaction earlier due to insufficient progress if !__GFP_REPEAT Mel Gorman
2011-02-16  9:50                 ` Mel Gorman
2011-02-16 10:13                 ` Andrea Arcangeli
2011-02-16 10:13                   ` Andrea Arcangeli
2011-02-16 11:22                   ` Mel Gorman
2011-02-16 11:22                     ` Mel Gorman
2011-02-16 14:44                     ` Andrea Arcangeli
2011-02-16 14:44                       ` Andrea Arcangeli
2011-02-16 12:03                 ` Andrea Arcangeli
2011-02-16 12:03                   ` Andrea Arcangeli
2011-02-16 12:14                 ` Rik van Riel
2011-02-16 12:14                   ` Rik van Riel
2011-02-16 12:38                 ` Johannes Weiner
2011-02-16 12:38                   ` Johannes Weiner
2011-02-16 23:26                 ` Minchan Kim
2011-02-16 23:26                   ` Minchan Kim
2011-02-17 22:22                 ` Andrew Morton
2011-02-17 22:22                   ` Andrew Morton
2011-02-18 12:22                   ` Mel Gorman
2011-02-18 12:22                     ` Mel Gorman
2011-02-10  4:04 ` [patch] vmscan: fix zone shrinking exit when scan work is done Minchan Kim
2011-02-10  4:04   ` Minchan Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110209154606.GJ27110@cmpxchg.org \
    --to=hannes@cmpxchg.org \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=kent.overstreet@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=mhocko@suse.cz \
    --cc=riel@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.