From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755490AbaFDAaL (ORCPT ); Tue, 3 Jun 2014 20:30:11 -0400 Received: from mail-ie0-f175.google.com ([209.85.223.175]:64433 "EHLO mail-ie0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755422AbaFDAaE (ORCPT ); Tue, 3 Jun 2014 20:30:04 -0400 Date: Tue, 3 Jun 2014 17:30:01 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Andrew Morton cc: Vlastimil Babka , Mel Gorman , Joonsoo Kim , Hugh Dickins , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Michal Nazarewicz , Rik van Riel Subject: [patch -mm 3/3] mm, compaction: avoid compacting memory for thp if pageblock cannot become free In-Reply-To: Message-ID: References: <1399904111-23520-1-git-send-email-vbabka@suse.cz> <1400233673-11477-1-git-send-email-vbabka@suse.cz> <537DB0E5.40602@suse.cz> <537DE799.3040400@suse.cz> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org It's pointless to migrate pages within a pageblock if the entire pageblock will not become free for a thp allocation. If we encounter a page that cannot be migrated and a direct compactor other than khugepaged is trying to allocate a hugepage for thp, then skip the entire pageblock and avoid migrating pages needlessly. Signed-off-by: David Rientjes --- mm/compaction.c | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c --- a/mm/compaction.c +++ b/mm/compaction.c @@ -537,12 +537,12 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, if ((low_pfn & (MAX_ORDER_NR_PAGES - 1)) == 0) { if (!pfn_valid(low_pfn)) { low_pfn += MAX_ORDER_NR_PAGES - 1; - continue; + goto next; } } if (!pfn_valid_within(low_pfn)) - continue; + goto next; nr_scanned++; /* @@ -553,7 +553,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, */ page = pfn_to_page(low_pfn); if (page_zone(page) != zone) - continue; + goto next; if (!valid_page) valid_page = page; @@ -599,7 +599,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, goto isolate_success; } } - continue; + goto next; } /* @@ -616,7 +616,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, if (!locked) goto next_pageblock; low_pfn += (1 << compound_order(page)) - 1; - continue; + goto next; } /* @@ -626,7 +626,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, */ if (!page_mapping(page) && page_count(page) > page_mapcount(page)) - continue; + goto next; /* Check if it is ok to still hold the lock */ locked = compact_checklock_irqsave(&zone->lru_lock, &flags, @@ -636,17 +636,17 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, /* Recheck PageLRU and PageTransHuge under lock */ if (!PageLRU(page)) - continue; + goto next; if (PageTransHuge(page)) { low_pfn += (1 << compound_order(page)) - 1; - continue; + goto next; } lruvec = mem_cgroup_page_lruvec(page, zone); /* Try isolate the page */ if (__isolate_lru_page(page, mode) != 0) - continue; + goto next; VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -669,6 +669,24 @@ isolate_success: next_pageblock: low_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages) - 1; + +next: + /* + * It is too expensive for compaction to migrate pages from a + * pageblock for thp page faults unless the entire pageblock + * will become free. + */ + if ((cc->gfp_mask & __GFP_NO_KSWAPD) && + !(current->flags & PF_KTHREAD)) { + if (locked) { + spin_unlock_irqrestore(&zone->lru_lock, flags); + locked = false; + } + putback_movable_pages(migratelist); + cc->nr_migratepages = 0; + nr_isolated = 0; + low_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages) - 1; + } } acct_isolated(zone, locked, cc); @@ -880,7 +898,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone, cc->migrate_pfn = low_pfn; - return ISOLATE_SUCCESS; + return cc->nr_migratepages ? ISOLATE_SUCCESS : ISOLATE_NONE; } static int compact_finished(struct zone *zone, struct compact_control *cc, @@ -1055,9 +1073,6 @@ static int compact_zone(struct zone *zone, struct compact_control *cc) ; } - if (!cc->nr_migratepages) - continue; - err = migrate_pages(&cc->migratepages, compaction_alloc, compaction_free, (unsigned long)cc, cc->mode, MR_COMPACTION); From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ig0-f179.google.com (mail-ig0-f179.google.com [209.85.213.179]) by kanga.kvack.org (Postfix) with ESMTP id 49F796B0036 for ; Tue, 3 Jun 2014 20:30:04 -0400 (EDT) Received: by mail-ig0-f179.google.com with SMTP id hn18so433273igb.6 for ; Tue, 03 Jun 2014 17:30:04 -0700 (PDT) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [2607:f8b0:4001:c05::22c]) by mx.google.com with ESMTPS id a16si1598903icp.24.2014.06.03.17.30.03 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 03 Jun 2014 17:30:03 -0700 (PDT) Received: by mail-ig0-f172.google.com with SMTP id l13so1118272iga.5 for ; Tue, 03 Jun 2014 17:30:03 -0700 (PDT) Date: Tue, 3 Jun 2014 17:30:01 -0700 (PDT) From: David Rientjes Subject: [patch -mm 3/3] mm, compaction: avoid compacting memory for thp if pageblock cannot become free In-Reply-To: Message-ID: References: <1399904111-23520-1-git-send-email-vbabka@suse.cz> <1400233673-11477-1-git-send-email-vbabka@suse.cz> <537DB0E5.40602@suse.cz> <537DE799.3040400@suse.cz> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Joonsoo Kim , Hugh Dickins , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Michal Nazarewicz , Rik van Riel It's pointless to migrate pages within a pageblock if the entire pageblock will not become free for a thp allocation. If we encounter a page that cannot be migrated and a direct compactor other than khugepaged is trying to allocate a hugepage for thp, then skip the entire pageblock and avoid migrating pages needlessly. Signed-off-by: David Rientjes --- mm/compaction.c | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c --- a/mm/compaction.c +++ b/mm/compaction.c @@ -537,12 +537,12 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, if ((low_pfn & (MAX_ORDER_NR_PAGES - 1)) == 0) { if (!pfn_valid(low_pfn)) { low_pfn += MAX_ORDER_NR_PAGES - 1; - continue; + goto next; } } if (!pfn_valid_within(low_pfn)) - continue; + goto next; nr_scanned++; /* @@ -553,7 +553,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, */ page = pfn_to_page(low_pfn); if (page_zone(page) != zone) - continue; + goto next; if (!valid_page) valid_page = page; @@ -599,7 +599,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, goto isolate_success; } } - continue; + goto next; } /* @@ -616,7 +616,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, if (!locked) goto next_pageblock; low_pfn += (1 << compound_order(page)) - 1; - continue; + goto next; } /* @@ -626,7 +626,7 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, */ if (!page_mapping(page) && page_count(page) > page_mapcount(page)) - continue; + goto next; /* Check if it is ok to still hold the lock */ locked = compact_checklock_irqsave(&zone->lru_lock, &flags, @@ -636,17 +636,17 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc, /* Recheck PageLRU and PageTransHuge under lock */ if (!PageLRU(page)) - continue; + goto next; if (PageTransHuge(page)) { low_pfn += (1 << compound_order(page)) - 1; - continue; + goto next; } lruvec = mem_cgroup_page_lruvec(page, zone); /* Try isolate the page */ if (__isolate_lru_page(page, mode) != 0) - continue; + goto next; VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -669,6 +669,24 @@ isolate_success: next_pageblock: low_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages) - 1; + +next: + /* + * It is too expensive for compaction to migrate pages from a + * pageblock for thp page faults unless the entire pageblock + * will become free. + */ + if ((cc->gfp_mask & __GFP_NO_KSWAPD) && + !(current->flags & PF_KTHREAD)) { + if (locked) { + spin_unlock_irqrestore(&zone->lru_lock, flags); + locked = false; + } + putback_movable_pages(migratelist); + cc->nr_migratepages = 0; + nr_isolated = 0; + low_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages) - 1; + } } acct_isolated(zone, locked, cc); @@ -880,7 +898,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone, cc->migrate_pfn = low_pfn; - return ISOLATE_SUCCESS; + return cc->nr_migratepages ? ISOLATE_SUCCESS : ISOLATE_NONE; } static int compact_finished(struct zone *zone, struct compact_control *cc, @@ -1055,9 +1073,6 @@ static int compact_zone(struct zone *zone, struct compact_control *cc) ; } - if (!cc->nr_migratepages) - continue; - err = migrate_pages(&cc->migratepages, compaction_alloc, compaction_free, (unsigned long)cc, cc->mode, MR_COMPACTION); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org