From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751910AbaFDWCm (ORCPT ); Wed, 4 Jun 2014 18:02:42 -0400 Received: from mail-ig0-f169.google.com ([209.85.213.169]:33584 "EHLO mail-ig0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751122AbaFDWCl (ORCPT ); Wed, 4 Jun 2014 18:02:41 -0400 Date: Wed, 4 Jun 2014 15:02:38 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Mel Gorman cc: Andrew Morton , Vlastimil Babka , Joonsoo Kim , Hugh Dickins , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Michal Nazarewicz , Rik van Riel Subject: Re: [patch -mm 3/3] mm, compaction: avoid compacting memory for thp if pageblock cannot become free In-Reply-To: <20140604110411.GK10819@suse.de> Message-ID: References: <1399904111-23520-1-git-send-email-vbabka@suse.cz> <1400233673-11477-1-git-send-email-vbabka@suse.cz> <537DB0E5.40602@suse.cz> <537DE799.3040400@suse.cz> <20140604110411.GK10819@suse.de> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 4 Jun 2014, Mel Gorman wrote: > > It's pointless to migrate pages within a pageblock if the entire pageblock will > > not become free for a thp allocation. > > > > If we encounter a page that cannot be migrated and a direct compactor other than > > khugepaged is trying to allocate a hugepage for thp, then skip the entire > > pageblock and avoid migrating pages needlessly. > > > > It's not completely pointless. A movable page may be placed within an > unmovable pageblock due to insufficient free memory or a pageblock changed > type. When this happens then partial migration moves the movable page > of out of the unmovable block. Future unmovable allocations can then be > placed with other unmovable pages instead of falling back to other blocks > and degrading fragmentation over time. > Sorry, this should say that it's pointless when doing a HPAGE_PMD_ORDER allocation and we're calling direct compaction for thp. While the result may be that there will be less external fragmentation in the longrun, I don't think it's appropriate to do this at fault. We keep a running tracker of how long it takes to fault 64MB of anonymous memory with thp enabled in one of our production cells. For an instance that took 1.58225s in fault (not including the mmap() time or munmap() time), here are the compaction stats: Before: compact_blocks_moved 508932592 compact_pages_moved 93068626 compact_pagemigrate_failed 199708939 compact_stall 7014989 compact_fail 6977371 compact_success 37617 After: compact_blocks_moved 508938635 compact_pages_moved 93068667 compact_pagemigrate_failed 199712677 compact_stall 7015029 compact_fail 6977411 compact_success 37617 Not one of the compaction stalls resulted in a thp page being allocated, probably because the number of pages actually migrated is very low. The delta here is 6043 pageblocks scanned over 40 compaction calls, 41 pages _total_ being successfully migrated and 3738 pages total being isolated but unsuccessfully migrated. Those statistics are horrible. We scan approximately 151 pageblocks per compaction stall needlessly in this case and, on average, migrate a single page but isolate and fail to migrate 93 pages. I believe my patch would reduce this pointless migration when an entire pageblock will not be freed in the thp fault path. I do need to factor in Vlastimil's feedback concerning the PageBuddy order, but I think this is generally the right approach for thp fault. Additionally, I need to figure out why those 3738 pages are isolated but fail to migrate and it doesn't seem to be because of any race. Perhaps there will be a chance to do something similar to what I did in commit 119d6d59dcc0 ("mm, compaction: avoid isolating pinned pages") to avoid even considering certain checks, but I'll have to identify the source(s) of these failures first. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ig0-f169.google.com (mail-ig0-f169.google.com [209.85.213.169]) by kanga.kvack.org (Postfix) with ESMTP id A36586B007B for ; Wed, 4 Jun 2014 18:02:41 -0400 (EDT) Received: by mail-ig0-f169.google.com with SMTP id a13so2433411igq.2 for ; Wed, 04 Jun 2014 15:02:41 -0700 (PDT) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [2607:f8b0:4001:c03::233]) by mx.google.com with ESMTPS id qg5si12549576igb.7.2014.06.04.15.02.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 04 Jun 2014 15:02:41 -0700 (PDT) Received: by mail-ie0-f179.google.com with SMTP id rd18so135192iec.10 for ; Wed, 04 Jun 2014 15:02:40 -0700 (PDT) Date: Wed, 4 Jun 2014 15:02:38 -0700 (PDT) From: David Rientjes Subject: Re: [patch -mm 3/3] mm, compaction: avoid compacting memory for thp if pageblock cannot become free In-Reply-To: <20140604110411.GK10819@suse.de> Message-ID: References: <1399904111-23520-1-git-send-email-vbabka@suse.cz> <1400233673-11477-1-git-send-email-vbabka@suse.cz> <537DB0E5.40602@suse.cz> <537DE799.3040400@suse.cz> <20140604110411.GK10819@suse.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Mel Gorman Cc: Andrew Morton , Vlastimil Babka , Joonsoo Kim , Hugh Dickins , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Michal Nazarewicz , Rik van Riel On Wed, 4 Jun 2014, Mel Gorman wrote: > > It's pointless to migrate pages within a pageblock if the entire pageblock will > > not become free for a thp allocation. > > > > If we encounter a page that cannot be migrated and a direct compactor other than > > khugepaged is trying to allocate a hugepage for thp, then skip the entire > > pageblock and avoid migrating pages needlessly. > > > > It's not completely pointless. A movable page may be placed within an > unmovable pageblock due to insufficient free memory or a pageblock changed > type. When this happens then partial migration moves the movable page > of out of the unmovable block. Future unmovable allocations can then be > placed with other unmovable pages instead of falling back to other blocks > and degrading fragmentation over time. > Sorry, this should say that it's pointless when doing a HPAGE_PMD_ORDER allocation and we're calling direct compaction for thp. While the result may be that there will be less external fragmentation in the longrun, I don't think it's appropriate to do this at fault. We keep a running tracker of how long it takes to fault 64MB of anonymous memory with thp enabled in one of our production cells. For an instance that took 1.58225s in fault (not including the mmap() time or munmap() time), here are the compaction stats: Before: compact_blocks_moved 508932592 compact_pages_moved 93068626 compact_pagemigrate_failed 199708939 compact_stall 7014989 compact_fail 6977371 compact_success 37617 After: compact_blocks_moved 508938635 compact_pages_moved 93068667 compact_pagemigrate_failed 199712677 compact_stall 7015029 compact_fail 6977411 compact_success 37617 Not one of the compaction stalls resulted in a thp page being allocated, probably because the number of pages actually migrated is very low. The delta here is 6043 pageblocks scanned over 40 compaction calls, 41 pages _total_ being successfully migrated and 3738 pages total being isolated but unsuccessfully migrated. Those statistics are horrible. We scan approximately 151 pageblocks per compaction stall needlessly in this case and, on average, migrate a single page but isolate and fail to migrate 93 pages. I believe my patch would reduce this pointless migration when an entire pageblock will not be freed in the thp fault path. I do need to factor in Vlastimil's feedback concerning the PageBuddy order, but I think this is generally the right approach for thp fault. Additionally, I need to figure out why those 3738 pages are isolated but fail to migrate and it doesn't seem to be because of any race. Perhaps there will be a chance to do something similar to what I did in commit 119d6d59dcc0 ("mm, compaction: avoid isolating pinned pages") to avoid even considering certain checks, but I'll have to identify the source(s) of these failures first. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org