From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96E3EC4361B for ; Tue, 15 Dec 2020 03:11:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 456532250F for ; Tue, 15 Dec 2020 03:11:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 456532250F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D7FE68D004C; Mon, 14 Dec 2020 22:11:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D30188D001C; Mon, 14 Dec 2020 22:11:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C25CE8D004C; Mon, 14 Dec 2020 22:11:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id A9FD28D001C for ; Mon, 14 Dec 2020 22:11:21 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 811F58249980 for ; Tue, 15 Dec 2020 03:11:21 +0000 (UTC) X-FDA: 77594040762.23.fang99_5811c1627420 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 56A8237609 for ; Tue, 15 Dec 2020 03:11:21 +0000 (UTC) X-HE-Tag: fang99_5811c1627420 X-Filterd-Recvd-Size: 4877 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Dec 2020 03:11:20 +0000 (UTC) Date: Mon, 14 Dec 2020 19:11:19 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608001879; bh=2V0aWHd+HpvbqUbigniTNnRjaO/TilulSY1cerdprDg=; h=From:To:Subject:In-Reply-To:From; b=EbNsu40OczZTiXu5332A1Wmyg5q5fADzYvLPf25cXmAfVNjoHA/9T4FCu84zHDHG7 CCAeVw3iyPWuz9FKJPR6Jz9GiR6k3iMtRL0tf+GzYtjKEvnmwrcVkpbbDnCP0j4bN1 +8kESd8qJtM4amGNBNQO/wyP2uDkadhSIWh6L0Eo= From: Andrew Morton To: akpm@linux-foundation.org, linf@wangsu.com, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 135/200] init/main: fix broken buffer_init when DEFERRED_STRUCT_PAGE_INIT set Message-ID: <20201215031119.60LGvc0BJ%akpm@linux-foundation.org> In-Reply-To: <20201214190237.a17b70ae14f129e2dca3d204@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Lin Feng Subject: init/main: fix broken buffer_init when DEFERRED_STRUCT_PAGE_INIT set In the booting phase if CONFIG_DEFERRED_STRUCT_PAGE_INIT is set, we have following callchain: start_kernel ... mm_init mem_init memblock_free_all reset_all_zones_managed_pages free_low_memory_core_early ... buffer_init nr_free_buffer_pages zone->managed_pages ... rest_init kernel_init kernel_init_freeable page_alloc_init_late kthread_run(deferred_init_memmap, NODE_DATA(nid), "pgdatinit%d", nid); wait_for_completion(&pgdat_init_all_done_comp); ... files_maxfiles_init It's clear that buffer_init depends on zone->managed_pages, but it's reset in reset_all_zones_managed_pages after that pages are readded into zone->managed_pages, but when buffer_init runs this process is half done and most of them will finally be added till deferred_init_memmap done. In large memory couting of nr_free_buffer_pages drifts too much, also drifting from kernels to kernels on same hardware. Fix is simple, it delays buffer_init run till deferred_init_memmap all done. But as corrected by this patch, max_buffer_heads becomes very large, the value is roughly as many as 4 times of totalram_pages, formula: max_buffer_heads = nrpages * (10%) * (PAGE_SIZE / sizeof(struct buffer_head)); Say in a 64GB memory box we have 16777216 pages, then max_buffer_heads turns out to be roughly 67,108,864. In common cases, should a buffer_head be mapped to one page/block(4KB)? So max_buffer_heads never exceeds totalram_pages. IMO it's likely to make buffer_heads_over_limit bool value alwasy false, then make codes 'if (buffer_heads_over_limit)' test in vmscan unnecessary. So this patch will change the original behavior related to buffer_heads_over_limit in vmscan since we used a half done value of zone->managed_pages before, or should we use a smaller factor(<10%) in previous formula. akpm: I think this is OK - the max_buffer_heads code is only needed on highmem machines, to prevent ZONE_NORMAL from being consumed by large amounts of buffer_heads attached to highmem pagecache. This problem will not occur on 64-bit machines, so this feature's non-functionality on such machines is a feature, not a bug. Link: https://lkml.kernel.org/r/20201123110500.103523-1-linf@wangsu.com Signed-off-by: Lin Feng Acked-by: Vlastimil Babka Cc: Mel Gorman Signed-off-by: Andrew Morton --- init/main.c | 2 -- mm/page_alloc.c | 3 +++ 2 files changed, 3 insertions(+), 2 deletions(-) --- a/init/main.c~init-main-fix-broken-buffer_init-when-deferred_struct_page_init-set +++ a/init/main.c @@ -58,7 +58,6 @@ #include #include #include -#include #include #include #include @@ -1036,7 +1035,6 @@ asmlinkage __visible void __init __no_sa fork_init(); proc_caches_init(); uts_ns_init(); - buffer_init(); key_init(); security_init(); dbg_late_init(); --- a/mm/page_alloc.c~init-main-fix-broken-buffer_init-when-deferred_struct_page_init-set +++ a/mm/page_alloc.c @@ -71,6 +71,7 @@ #include #include #include +#include #include #include @@ -2113,6 +2114,8 @@ void __init page_alloc_init_late(void) files_maxfiles_init(); #endif + buffer_init(); + /* Discard memblock private memory */ memblock_discard(); _