From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752816AbcD0OGP (ORCPT ); Wed, 27 Apr 2016 10:06:15 -0400 Received: from mx2.suse.de ([195.135.220.15]:36824 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752437AbcD0OGO (ORCPT ); Wed, 27 Apr 2016 10:06:14 -0400 Subject: Re: [PATCH 28/28] mm, page_alloc: Defer debugging checks of pages allocated from the PCP To: Mel Gorman , Andrew Morton References: <1460710760-32601-1-git-send-email-mgorman@techsingularity.net> <1460711275-1130-1-git-send-email-mgorman@techsingularity.net> <1460711275-1130-16-git-send-email-mgorman@techsingularity.net> Cc: Jesper Dangaard Brouer , Linux-MM , LKML From: Vlastimil Babka Message-ID: <5720C753.2000804@suse.cz> Date: Wed, 27 Apr 2016 16:06:11 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <1460711275-1130-16-git-send-email-mgorman@techsingularity.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/15/2016 11:07 AM, Mel Gorman wrote: > Every page allocated checks a number of page fields for validity. This > catches corruption bugs of pages that are already freed but it is expensive. > This patch weakens the debugging check by checking PCP pages only when > the PCP lists are being refilled. All compound pages are checked. This > potentially avoids debugging checks entirely if the PCP lists are never > emptied and refilled so some corruption issues may be missed. Full checking > requires DEBUG_VM. > > With the two deferred debugging patches applied, the impact to a page > allocator microbenchmark is > > 4.6.0-rc3 4.6.0-rc3 > inline-v3r6 deferalloc-v3r7 > Min alloc-odr0-1 344.00 ( 0.00%) 317.00 ( 7.85%) > Min alloc-odr0-2 248.00 ( 0.00%) 231.00 ( 6.85%) > Min alloc-odr0-4 209.00 ( 0.00%) 192.00 ( 8.13%) > Min alloc-odr0-8 181.00 ( 0.00%) 166.00 ( 8.29%) > Min alloc-odr0-16 168.00 ( 0.00%) 154.00 ( 8.33%) > Min alloc-odr0-32 161.00 ( 0.00%) 148.00 ( 8.07%) > Min alloc-odr0-64 158.00 ( 0.00%) 145.00 ( 8.23%) > Min alloc-odr0-128 156.00 ( 0.00%) 143.00 ( 8.33%) > Min alloc-odr0-256 168.00 ( 0.00%) 154.00 ( 8.33%) > Min alloc-odr0-512 178.00 ( 0.00%) 167.00 ( 6.18%) > Min alloc-odr0-1024 186.00 ( 0.00%) 174.00 ( 6.45%) > Min alloc-odr0-2048 192.00 ( 0.00%) 180.00 ( 6.25%) > Min alloc-odr0-4096 198.00 ( 0.00%) 184.00 ( 7.07%) > Min alloc-odr0-8192 200.00 ( 0.00%) 188.00 ( 6.00%) > Min alloc-odr0-16384 201.00 ( 0.00%) 188.00 ( 6.47%) > Min free-odr0-1 189.00 ( 0.00%) 180.00 ( 4.76%) > Min free-odr0-2 132.00 ( 0.00%) 126.00 ( 4.55%) > Min free-odr0-4 104.00 ( 0.00%) 99.00 ( 4.81%) > Min free-odr0-8 90.00 ( 0.00%) 85.00 ( 5.56%) > Min free-odr0-16 84.00 ( 0.00%) 80.00 ( 4.76%) > Min free-odr0-32 80.00 ( 0.00%) 76.00 ( 5.00%) > Min free-odr0-64 78.00 ( 0.00%) 74.00 ( 5.13%) > Min free-odr0-128 77.00 ( 0.00%) 73.00 ( 5.19%) > Min free-odr0-256 94.00 ( 0.00%) 91.00 ( 3.19%) > Min free-odr0-512 108.00 ( 0.00%) 112.00 ( -3.70%) > Min free-odr0-1024 115.00 ( 0.00%) 118.00 ( -2.61%) > Min free-odr0-2048 120.00 ( 0.00%) 125.00 ( -4.17%) > Min free-odr0-4096 123.00 ( 0.00%) 129.00 ( -4.88%) > Min free-odr0-8192 126.00 ( 0.00%) 130.00 ( -3.17%) > Min free-odr0-16384 126.00 ( 0.00%) 131.00 ( -3.97%) > > Note that the free paths for large numbers of pages is impacted as the > debugging cost gets shifted into that path when the page data is no longer > necessarily cache-hot. > > Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka Unlike the free path, there are no duplications here, which is nice. Some un-inlining of bad page check should still work here though imho: >>From afdefd87f2d8d07cba4bd2a2f3531dc8bb0b7a19 Mon Sep 17 00:00:00 2001 From: Vlastimil Babka Date: Wed, 27 Apr 2016 15:47:29 +0200 Subject: [PATCH] mm, page_alloc: uninline the bad page part of check_new_page() Bad pages should be rare so the code handling them doesn't need to be inline for performance reasons. Put it to separate function which returns void. This also assumes that the initial page_expected_state() result will match the result of the thorough check, i.e. the page doesn't become "good" in the meanwhile. This matches the same expectations already in place in free_pages_check(). !DEBUG_VM bloat-o-meter: add/remove: 1/0 grow/shrink: 0/1 up/down: 134/-274 (-140) function old new delta check_new_page_bad - 134 +134 get_page_from_freelist 3468 3194 -274 Signed-off-by: Vlastimil Babka --- mm/page_alloc.c | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2b3aefdfcaa2..755ec9465d8a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1648,19 +1648,11 @@ static inline void expand(struct zone *zone, struct page *page, } } -/* - * This page is about to be returned from the page allocator - */ -static inline int check_new_page(struct page *page) +static void check_new_page_bad(struct page *page) { - const char *bad_reason; - unsigned long bad_flags; + const char *bad_reason = NULL; + unsigned long bad_flags = 0; - if (page_expected_state(page, PAGE_FLAGS_CHECK_AT_PREP|__PG_HWPOISON)) - return 0; - - bad_reason = NULL; - bad_flags = 0; if (unlikely(atomic_read(&page->_mapcount) != -1)) bad_reason = "nonzero mapcount"; if (unlikely(page->mapping != NULL)) @@ -1679,11 +1671,20 @@ static inline int check_new_page(struct page *page) if (unlikely(page->mem_cgroup)) bad_reason = "page still charged to cgroup"; #endif - if (unlikely(bad_reason)) { - bad_page(page, bad_reason, bad_flags); - return 1; - } - return 0; + bad_page(page, bad_reason, bad_flags); +} + +/* + * This page is about to be returned from the page allocator + */ +static inline int check_new_page(struct page *page) +{ + if (likely(page_expected_state(page, + PAGE_FLAGS_CHECK_AT_PREP|__PG_HWPOISON))) + return 0; + + check_new_page_bad(page); + return 1; } static inline bool free_pages_prezeroed(bool poisoned) -- 2.8.1