From: Mel Gorman <mgorman@suse.de> To: Stable <stable@vger.kernel.org> Cc: Linux-MM <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org>, Mel Gorman <mgorman@suse.de> Subject: [PATCH 14/34] mm: zone_reclaim: make isolate_lru_page() filter-aware Date: Mon, 23 Jul 2012 14:38:27 +0100 [thread overview] Message-ID: <1343050727-3045-15-git-send-email-mgorman@suse.de> (raw) In-Reply-To: <1343050727-3045-1-git-send-email-mgorman@suse.de> From: Minchan Kim <minchan.kim@gmail.com> commit f80c0673610e36ae29d63e3297175e22f70dde5f upstream. Stable note: Not tracked in Bugzilla. THP and compaction disrupt the LRU list leading to poor reclaim decisions which has a variable performance impact. In __zone_reclaim case, we don't want to shrink mapped page. Nonetheless, we have isolated mapped page and re-add it into LRU's head. It's unnecessary CPU overhead and makes LRU churning. Of course, when we isolate the page, the page might be mapped but when we try to migrate the page, the page would be not mapped. So it could be migrated. But race is rare and although it happens, it's no big deal. Signed-off-by: Minchan Kim <minchan.kim@gmail.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Reviewed-by: Michal Hocko <mhocko@suse.cz> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Mel Gorman <mgorman@suse.de> --- include/linux/mmzone.h | 2 ++ mm/vmscan.c | 20 ++++++++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 632107e..951ed81 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -164,6 +164,8 @@ static inline int is_unevictable_lru(enum lru_list l) #define ISOLATE_ACTIVE ((__force isolate_mode_t)0x2) /* Isolate clean file */ #define ISOLATE_CLEAN ((__force isolate_mode_t)0x4) +/* Isolate unmapped file */ +#define ISOLATE_UNMAPPED ((__force isolate_mode_t)0x8) /* LRU Isolation modes. */ typedef unsigned __bitwise__ isolate_mode_t; diff --git a/mm/vmscan.c b/mm/vmscan.c index 032f35e..9aa75e9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1048,6 +1048,9 @@ int __isolate_lru_page(struct page *page, isolate_mode_t mode, int file) if ((mode & ISOLATE_CLEAN) && (PageDirty(page) || PageWriteback(page))) return ret; + if ((mode & ISOLATE_UNMAPPED) && page_mapped(page)) + return ret; + if (likely(get_page_unless_zero(page))) { /* * Be careful not to clear PageLRU until after we're @@ -1471,6 +1474,12 @@ shrink_inactive_list(unsigned long nr_to_scan, struct zone *zone, reclaim_mode |= ISOLATE_ACTIVE; lru_add_drain(); + + if (!sc->may_unmap) + reclaim_mode |= ISOLATE_UNMAPPED; + if (!sc->may_writepage) + reclaim_mode |= ISOLATE_CLEAN; + spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { @@ -1588,19 +1597,26 @@ static void shrink_active_list(unsigned long nr_pages, struct zone *zone, struct page *page; struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc); unsigned long nr_rotated = 0; + isolate_mode_t reclaim_mode = ISOLATE_ACTIVE; lru_add_drain(); + + if (!sc->may_unmap) + reclaim_mode |= ISOLATE_UNMAPPED; + if (!sc->may_writepage) + reclaim_mode |= ISOLATE_CLEAN; + spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { nr_taken = isolate_pages_global(nr_pages, &l_hold, &pgscanned, sc->order, - ISOLATE_ACTIVE, zone, + reclaim_mode, zone, 1, file); zone->pages_scanned += pgscanned; } else { nr_taken = mem_cgroup_isolate_pages(nr_pages, &l_hold, &pgscanned, sc->order, - ISOLATE_ACTIVE, zone, + reclaim_mode, zone, sc->mem_cgroup, 1, file); /* * mem_cgroup_isolate_pages() keeps track of -- 1.7.9.2
WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de> To: Stable <stable@vger.kernel.org> Cc: Linux-MM <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org>, Mel Gorman <mgorman@suse.de> Subject: [PATCH 14/34] mm: zone_reclaim: make isolate_lru_page() filter-aware Date: Mon, 23 Jul 2012 14:38:27 +0100 [thread overview] Message-ID: <1343050727-3045-15-git-send-email-mgorman@suse.de> (raw) In-Reply-To: <1343050727-3045-1-git-send-email-mgorman@suse.de> From: Minchan Kim <minchan.kim@gmail.com> commit f80c0673610e36ae29d63e3297175e22f70dde5f upstream. Stable note: Not tracked in Bugzilla. THP and compaction disrupt the LRU list leading to poor reclaim decisions which has a variable performance impact. In __zone_reclaim case, we don't want to shrink mapped page. Nonetheless, we have isolated mapped page and re-add it into LRU's head. It's unnecessary CPU overhead and makes LRU churning. Of course, when we isolate the page, the page might be mapped but when we try to migrate the page, the page would be not mapped. So it could be migrated. But race is rare and although it happens, it's no big deal. Signed-off-by: Minchan Kim <minchan.kim@gmail.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Reviewed-by: Michal Hocko <mhocko@suse.cz> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Mel Gorman <mgorman@suse.de> --- include/linux/mmzone.h | 2 ++ mm/vmscan.c | 20 ++++++++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 632107e..951ed81 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -164,6 +164,8 @@ static inline int is_unevictable_lru(enum lru_list l) #define ISOLATE_ACTIVE ((__force isolate_mode_t)0x2) /* Isolate clean file */ #define ISOLATE_CLEAN ((__force isolate_mode_t)0x4) +/* Isolate unmapped file */ +#define ISOLATE_UNMAPPED ((__force isolate_mode_t)0x8) /* LRU Isolation modes. */ typedef unsigned __bitwise__ isolate_mode_t; diff --git a/mm/vmscan.c b/mm/vmscan.c index 032f35e..9aa75e9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1048,6 +1048,9 @@ int __isolate_lru_page(struct page *page, isolate_mode_t mode, int file) if ((mode & ISOLATE_CLEAN) && (PageDirty(page) || PageWriteback(page))) return ret; + if ((mode & ISOLATE_UNMAPPED) && page_mapped(page)) + return ret; + if (likely(get_page_unless_zero(page))) { /* * Be careful not to clear PageLRU until after we're @@ -1471,6 +1474,12 @@ shrink_inactive_list(unsigned long nr_to_scan, struct zone *zone, reclaim_mode |= ISOLATE_ACTIVE; lru_add_drain(); + + if (!sc->may_unmap) + reclaim_mode |= ISOLATE_UNMAPPED; + if (!sc->may_writepage) + reclaim_mode |= ISOLATE_CLEAN; + spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { @@ -1588,19 +1597,26 @@ static void shrink_active_list(unsigned long nr_pages, struct zone *zone, struct page *page; struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc); unsigned long nr_rotated = 0; + isolate_mode_t reclaim_mode = ISOLATE_ACTIVE; lru_add_drain(); + + if (!sc->may_unmap) + reclaim_mode |= ISOLATE_UNMAPPED; + if (!sc->may_writepage) + reclaim_mode |= ISOLATE_CLEAN; + spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { nr_taken = isolate_pages_global(nr_pages, &l_hold, &pgscanned, sc->order, - ISOLATE_ACTIVE, zone, + reclaim_mode, zone, 1, file); zone->pages_scanned += pgscanned; } else { nr_taken = mem_cgroup_isolate_pages(nr_pages, &l_hold, &pgscanned, sc->order, - ISOLATE_ACTIVE, zone, + reclaim_mode, zone, sc->mem_cgroup, 1, file); /* * mem_cgroup_isolate_pages() keeps track of -- 1.7.9.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2012-07-23 13:45 UTC|newest] Thread overview: 119+ messages / expand[flat|nested] mbox.gz Atom feed top 2012-07-23 13:38 [PATCH 00/34] Memory management performance backports for -stable V2 Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 01/34] mm: vmstat: cache align vm_stat Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 02/34] mm: memory hotplug: Check if pages are correctly reserved on a per-section basis Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 03/34] mm: Reduce the amount of work done when updating min_free_kbytes Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-24 22:47 ` Greg KH 2012-07-24 22:47 ` Greg KH 2012-07-25 7:57 ` Mel Gorman 2012-07-25 7:57 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 04/34] mm: vmscan: fix force-scanning small targets without swap Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 05/34] vmscan: clear ZONE_CONGESTED for zone with good watermark Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 06/34] vmscan: add shrink_slab tracepoints Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 07/34] vmscan: shrinker->nr updates race and go wrong Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 08/34] vmscan: reduce wind up shrinker->nr when shrinker can't do work Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 09/34] mm: limit direct reclaim for higher order allocations Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 10/34] mm: Abort reclaim/compaction if compaction can proceed Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 11/34] mm: compaction: trivial clean up in acct_isolated() Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 12/34] mm: change isolate mode from #define to bitwise type Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 13/34] mm: compaction: make isolate_lru_page() filter-aware Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` Mel Gorman [this message] 2012-07-23 13:38 ` [PATCH 14/34] mm: zone_reclaim: " Mel Gorman 2012-07-23 13:38 ` [PATCH 15/34] mm: migration: clean up unmap_and_move() Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-25 15:45 ` Greg KH 2012-07-25 15:45 ` Greg KH 2012-07-25 16:04 ` Mel Gorman 2012-07-25 16:04 ` Mel Gorman 2012-07-25 18:03 ` Greg KH 2012-07-25 18:03 ` Greg KH 2012-07-23 13:38 ` [PATCH 16/34] mm: compaction: Allow compaction to isolate dirty pages Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-25 15:47 ` Greg KH 2012-07-25 15:47 ` Greg KH 2012-07-25 16:07 ` Mel Gorman 2012-07-25 16:07 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 17/34] mm: compaction: Determine if dirty pages can be migrated without blocking within ->migratepage Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 18/34] mm: page allocator: Do not call direct reclaim for THP allocations while compaction is deferred Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 19/34] mm: compaction: make isolate_lru_page() filter-aware again Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 20/34] kswapd: avoid unnecessary rebalance after an unsuccessful balancing Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 21/34] kswapd: assign new_order and new_classzone_idx after wakeup in sleeping Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 22/34] mm: compaction: Introduce sync-light migration for use by compaction Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 23/34] mm: vmscan: When reclaiming for compaction, ensure there are sufficient free pages available Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 24/34] mm: vmscan: Do not OOM if aborting reclaim to start compaction Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 25/34] mm: vmscan: Check if reclaim should really abort even if compaction_ready() is true for one zone Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-25 19:51 ` Greg KH 2012-07-25 19:51 ` Greg KH 2012-07-23 13:38 ` [PATCH 26/34] vmscan: promote shared file mapped pages Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 27/34] vmscan: activate executable pages after first usage Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 28/34] mm/vmscan.c: consider swap space when deciding whether to continue reclaim Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 29/34] mm: test PageSwapBacked in lumpy reclaim Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 30/34] mm: vmscan: Do not force kswapd to scan small targets Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-25 19:59 ` Greg KH 2012-07-25 19:59 ` Greg KH 2012-07-25 21:35 ` Mel Gorman 2012-07-25 21:35 ` Mel Gorman 2012-07-25 21:44 ` Greg KH 2012-07-25 21:44 ` Greg KH 2012-07-23 13:38 ` [PATCH 31/34] cpusets: avoid looping when storing to mems_allowed if one node remains set Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 32/34] cpusets: stall when updating mems_allowed for mempolicy or disjoint nodemask Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 33/34] cpuset: mm: Reduce large amounts of memory barrier related damage v3 Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-23 13:38 ` [PATCH 34/34] mm/hugetlb: fix warning in alloc_huge_page/dequeue_huge_page_vma Mel Gorman 2012-07-23 13:38 ` Mel Gorman 2012-07-24 5:58 ` [PATCH 00/34] Memory management performance backports for -stable V2 Mike Galbraith 2012-07-24 5:58 ` Mike Galbraith 2012-07-24 8:10 ` Mel Gorman 2012-07-24 8:10 ` Mel Gorman 2012-07-24 13:18 ` Hillf Danton 2012-07-24 13:18 ` Hillf Danton 2012-07-24 13:27 ` Mel Gorman 2012-07-24 13:27 ` Mel Gorman 2012-07-24 13:34 ` Hillf Danton 2012-07-24 13:34 ` Hillf Danton 2012-07-24 13:53 ` Mel Gorman 2012-07-24 13:53 ` Mel Gorman 2012-07-24 14:11 ` Hillf Danton 2012-07-24 14:11 ` Hillf Danton 2012-07-24 13:52 ` Mike Galbraith 2012-07-24 13:52 ` Mike Galbraith 2012-07-24 14:18 ` Hillf Danton 2012-07-24 14:18 ` Hillf Danton 2012-07-24 14:41 ` Mike Galbraith 2012-07-24 14:41 ` Mike Galbraith 2012-07-25 22:30 ` Greg KH 2012-07-25 22:30 ` Greg KH 2012-07-25 22:48 ` Mel Gorman 2012-07-25 22:48 ` Mel Gorman 2012-07-30 1:13 ` Ben Hutchings -- strict thread matches above, loose matches on Subject: below -- 2012-07-19 14:36 [PATCH 00/34] Memory management performance backports for -stable Mel Gorman 2012-07-19 14:36 ` [PATCH 14/34] mm: zone_reclaim: make isolate_lru_page() filter-aware Mel Gorman 2012-07-19 14:36 ` Mel Gorman
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1343050727-3045-15-git-send-email-mgorman@suse.de \ --to=mgorman@suse.de \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=stable@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.