From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F33A4C433E0 for ; Mon, 11 Jan 2021 19:40:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A567F22BEF for ; Mon, 11 Jan 2021 19:40:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A567F22BEF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B9AFC6B0250; Mon, 11 Jan 2021 14:40:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E6E66B02C5; Mon, 11 Jan 2021 14:40:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D66A6B01F6; Mon, 11 Jan 2021 14:40:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id 3A1766B02F4 for ; Mon, 11 Jan 2021 14:40:39 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 086F3180AD815 for ; Mon, 11 Jan 2021 19:40:39 +0000 (UTC) X-FDA: 77694511398.01.skin93_22008422750f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id EC3391004912D for ; Mon, 11 Jan 2021 19:40:38 +0000 (UTC) X-HE-Tag: skin93_22008422750f X-Filterd-Recvd-Size: 8774 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 Jan 2021 19:40:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0222922CAF; Mon, 11 Jan 2021 19:40:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1610394037; bh=QSSqxmMBYWOWvJ0gjPg24CkQDuQmjj5I8Cxdpfz7z78=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=snw6TIlWtpt0LTtlJWFZ0EyO6BYKcfUdhrdJVY/KLihtswohPUvOUGzZLhHoTO6w9 yycm607VeTmcWBBbHXJChTAoWbpj72WJll6divf4NoohfRdBqJPBIoE3PVDnFpJFkL 7FnnrZYAFMMf8vx0LZ9dvzrIGbstliX3DsZtfnSdkzXsOR+ha1D8tpi59dp3tnCkSv fBgRHJIqZbVnb+JRy/L+FQ2aR+XipLreSEkwqTV10CnI3rLtupIB/tUogCD70EgWtT 0dfYyEY1s/Y1PQ8OjmrDzI3Ht/NDejo4QuOAWpbp5NS9Ul80i38WHbGdKXaGV3/Mg8 Ck5oIM3+bYIkg== From: Mike Rapoport To: Andrew Morton Cc: Andrea Arcangeli , Baoquan He , Borislav Petkov , David Hildenbrand , "H. Peter Anvin" , Ingo Molnar , Mel Gorman , Michal Hocko , Mike Rapoport , Mike Rapoport , Qian Cai , Thomas Gleixner , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Subject: [PATCH v3 2/2] mm: fix initialization of struct page for holes in memory layout Date: Mon, 11 Jan 2021 21:40:17 +0200 Message-Id: <20210111194017.22696-3-rppt@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20210111194017.22696-1-rppt@kernel.org> References: <20210111194017.22696-1-rppt@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport There could be struct pages that are not backed by actual physical memory= . This can happen when the actual memory bank is not a multiple of SECTION_SIZE or when an architecture does not register memory holes reserved by the firmware as memblock.memory. Such pages are currently initialized using init_unavailable_mem() functio= n that iterates through PFNs in holes in memblock.memory and if there is a struct page corresponding to a PFN, the fields if this page are set to default values and the page is marked as Reserved. init_unavailable_mem() does not take into account zone and node the page belongs to and sets both zone and node links in struct page to zero. On a system that has firmware reserved holes in a zone above ZONE_DMA, fo= r instance in a configuration below: # grep -A1 E820 /proc/iomem 7a17b000-7a216fff : Unknown E820 type 7a217000-7bffffff : System RAM unset zone link in struct page will trigger VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); because there are pages in both ZONE_DMA32 and ZONE_DMA (unset zone link = in struct page) in the same pageblock. Update init_unavailable_mem() to use zone constraints defined by an architecture to properly setup the zone link and use node ID of the adjacent range in memblock.memory to set the node link. Fixes: 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions rath= er that check each PFN") Reported-by: Andrea Arcangeli Signed-off-by: Mike Rapoport --- mm/page_alloc.c | 84 +++++++++++++++++++++++++++++-------------------- 1 file changed, 50 insertions(+), 34 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index bdbec4c98173..0b56c3ca354e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7077,23 +7077,26 @@ void __init free_area_init_memoryless_node(int ni= d) * Initialize all valid struct pages in the range [spfn, epfn) and mark = them * PageReserved(). Return the number of struct pages that were initializ= ed. */ -static u64 __init init_unavailable_range(unsigned long spfn, unsigned lo= ng epfn) +static u64 __init init_unavailable_range(unsigned long spfn, unsigned lo= ng epfn, + int zone, int nid) { - unsigned long pfn; + unsigned long pfn, zone_spfn, zone_epfn; u64 pgcnt =3D 0; =20 + zone_spfn =3D arch_zone_lowest_possible_pfn[zone]; + zone_epfn =3D arch_zone_highest_possible_pfn[zone]; + + spfn =3D clamp(spfn, zone_spfn, zone_epfn); + epfn =3D clamp(epfn, zone_spfn, zone_epfn); + for (pfn =3D spfn; pfn < epfn; pfn++) { if (!pfn_valid(ALIGN_DOWN(pfn, pageblock_nr_pages))) { pfn =3D ALIGN_DOWN(pfn, pageblock_nr_pages) + pageblock_nr_pages - 1; continue; } - /* - * Use a fake node/zone (0) for now. Some of these pages - * (in memblock.reserved but not in memblock.memory) will - * get re-initialized via reserve_bootmem_region() later. - */ - __init_single_page(pfn_to_page(pfn), pfn, 0, 0); + + __init_single_page(pfn_to_page(pfn), pfn, zone, nid); __SetPageReserved(pfn_to_page(pfn)); pgcnt++; } @@ -7102,51 +7105,64 @@ static u64 __init init_unavailable_range(unsigned= long spfn, unsigned long epfn) } =20 /* - * Only struct pages that are backed by physical memory are zeroed and - * initialized by going through __init_single_page(). But, there are som= e - * struct pages which are reserved in memblock allocator and their field= s - * may be accessed (for example page_to_pfn() on some configuration acce= sses - * flags). We must explicitly initialize those struct pages. + * Only struct pages that correspond to ranges defined by memblock.memor= y + * are zeroed and initialized by going through __init_single_page() duri= ng + * memmap_init(). + * + * But, there could be struct pages that correspond to holes in + * memblock.memory. This can happen because of the following reasons: + * - phyiscal memory bank size is not necessarily the exact multiple of = the + * arbitrary section size + * - early reserved memory may not be listed in memblock.memory + * - memory layouts defined with memmap=3D kernel parameter may not alig= n + * nicely with memmap sections * - * This function also addresses a similar issue where struct pages are l= eft - * uninitialized because the physical address range is not covered by - * memblock.memory or memblock.reserved. That could happen when memblock - * layout is manually configured via memmap=3D, or when the highest phys= ical - * address (max_pfn) does not end on a section boundary. + * Explicitly initialize those struct pages so that: + * - PG_Reserved is set + * - zone link is set accorging to the architecture constrains + * - node is set to node id of the next populated region except for the + * trailing hole where last node id is used */ -static void __init init_unavailable_mem(void) +static void __init init_zone_unavailable_mem(int zone) { - phys_addr_t start, end; - u64 i, pgcnt; - phys_addr_t next =3D 0; + unsigned long start, end; + int i, nid; + u64 pgcnt; + unsigned long next =3D 0; =20 /* - * Loop through unavailable ranges not covered by memblock.memory. + * Loop through holes in memblock.memory and initialize struct + * pages corresponding to these holes */ pgcnt =3D 0; - for_each_mem_range(i, &start, &end) { + for_each_mem_pfn_range(i, MAX_NUMNODES, &start, &end, &nid) { if (next < start) - pgcnt +=3D init_unavailable_range(PFN_DOWN(next), - PFN_UP(start)); + pgcnt +=3D init_unavailable_range(next, start, zone, nid); next =3D end; } =20 /* - * Early sections always have a fully populated memmap for the whole - * section - see pfn_valid(). If the last section has holes at the - * end and that section is marked "online", the memmap will be - * considered initialized. Make sure that memmap has a well defined - * state. + * Last section may surpass the actual end of memory (e.g. we can + * have 1Gb section and 512Mb of RAM pouplated). + * Make sure that memmap has a well defined state in this case. */ - pgcnt +=3D init_unavailable_range(PFN_DOWN(next), - round_up(max_pfn, PAGES_PER_SECTION)); + end =3D round_up(max_pfn, PAGES_PER_SECTION); + pgcnt +=3D init_unavailable_range(next, end, zone, nid); =20 /* * Struct pages that do not have backing memory. This could be because * firmware is using some of this memory, or for some other reasons. */ if (pgcnt) - pr_info("Zeroed struct page in unavailable ranges: %lld pages", pgcnt)= ; + pr_info("Zone %s: zeroed struct page in unavailable ranges: %lld pages= ", zone_names[zone], pgcnt); +} + +static void __init init_unavailable_mem(void) +{ + int zone; + + for (zone =3D 0; zone < ZONE_MOVABLE; zone++) + init_zone_unavailable_mem(zone); } #else static inline void __init init_unavailable_mem(void) --=20 2.28.0