Re: [PATCH v6 1/1] mm/page_alloc.c: refactor initialization of struct page for holes in memory layout

From: David Hildenbrand <david@redhat.com>
To: Mike Rapoport <rppt@kernel.org>
Cc: "Andrew Morton" <akpm@linux-foundation.org>,
	"Andrea Arcangeli" <aarcange@redhat.com>,
	"Baoquan He" <bhe@redhat.com>, "Borislav Petkov" <bp@alien8.de>,
	"Chris Wilson" <chris@chris-wilson.co.uk>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Linus Torvalds" <torvalds@linux-foundation.org>,
	"Łukasz Majczak" <lma@semihalf.com>,
	"Mel Gorman" <mgorman@suse.de>,
	"Michal Hocko" <mhocko@kernel.org>,
	"Mike Rapoport" <rppt@linux.ibm.com>, "Qian Cai" <cai@lca.pw>,
	"Sarvela, Tomi P" <tomi.p.sarvela@intel.com>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Vlastimil Babka" <vbabka@suse.cz>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	stable@vger.kernel.org, x86@kernel.org
Subject: Re: [PATCH v6 1/1] mm/page_alloc.c: refactor initialization of struct page for holes in memory layout
Date: Tue, 23 Feb 2021 10:49:44 +0100	[thread overview]
Message-ID: <aafa291d-ced4-2e4d-f9c4-be99e9394c0c@redhat.com> (raw)
In-Reply-To: <20210223094802.GI1447004@kernel.org>

On 23.02.21 10:48, Mike Rapoport wrote:
> On Tue, Feb 23, 2021 at 09:04:19AM +0100, David Hildenbrand wrote:
>> On 22.02.21 11:57, Mike Rapoport wrote:
>>> From: Mike Rapoport <rppt@linux.ibm.com>
>>>
>>> There could be struct pages that are not backed by actual physical memory.
>>> This can happen when the actual memory bank is not a multiple of
>>> SECTION_SIZE or when an architecture does not register memory holes
>>> reserved by the firmware as memblock.memory.
>>>
>>> Such pages are currently initialized using init_unavailable_mem() function
>>> that iterates through PFNs in holes in memblock.memory and if there is a
>>> struct page corresponding to a PFN, the fields of this page are set to
>>> default values and it is marked as Reserved.
>>>
>>> init_unavailable_mem() does not take into account zone and node the page
>>> belongs to and sets both zone and node links in struct page to zero.
>>>
>>> Before commit 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions
>>> rather that check each PFN") the holes inside a zone were re-initialized
>>> during memmap_init() and got their zone/node links right. However, after
>>> that commit nothing updates the struct pages representing such holes.
>>>
>>> On a system that has firmware reserved holes in a zone above ZONE_DMA, for
>>> instance in a configuration below:
>>>
>>> 	# grep -A1 E820 /proc/iomem
>>> 	7a17b000-7a216fff : Unknown E820 type
>>> 	7a217000-7bffffff : System RAM
>>>
>>> unset zone link in struct page will trigger
>>>
>>> 	VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);
>>>
>>> because there are pages in both ZONE_DMA32 and ZONE_DMA (unset zone link
>>> in struct page) in the same pageblock.
>>>
>>> Interleave initialization of the unavailable pages with the normal
>>> initialization of memory map, so that zone and node information will be
>>> properly set on struct pages that are not backed by the actual memory.
>>>
>>> With this change the pages for holes inside a zone will get proper
>>> zone/node links and the pages that are not spanned by any node will get
>>> links to the adjacent zone/node.
>>
>> Does this include pages in the last section has handled by ...
>> ...
>>> -	/*
>>> -	 * Early sections always have a fully populated memmap for the whole
>>> -	 * section - see pfn_valid(). If the last section has holes at the
>>> -	 * end and that section is marked "online", the memmap will be
>>> -	 * considered initialized. Make sure that memmap has a well defined
>>> -	 * state.
>>> -	 */
>>> -	pgcnt += init_unavailable_range(PFN_DOWN(next),
>>> -					round_up(max_pfn, PAGES_PER_SECTION));
>>> -
>>
>> ^ this code?
>>
>> Or how is that case handled now?
> 
> Hmm, now it's clamped to node_end_pfn/zone_end_pfn, so in your funny example with
> 
>      -object memory-backend-ram,id=bmem0,size=4160M \
>      -object memory-backend-ram,id=bmem1,size=4032M \
> 
> this is not handled :(
> 
> But it will be handled with this on top:
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 29bbd08b8e63..6c9b490f5a8b 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -6350,9 +6350,12 @@ void __meminit __weak memmap_init_zone(struct zone *zone)
>   		hole_pfn = end_pfn;
>   	}
>   
> -	if (hole_pfn < zone_end_pfn)
> -		pgcnt += init_unavailable_range(hole_pfn, zone_end_pfn,
> +#ifdef CONFIG_SPARSEMEM
> +	end_pfn = round_up(zone_end_pfn, PAGES_PER_SECTION);
> +	if (hole_pfn < end_pfn)
> +		pgcnt += init_unavailable_range(hole_pfn, end_pfn,
>   						zone_id, nid);
> +#endif
>   
>   	if (pgcnt)
>   		pr_info("  %s zone: %lld pages in unavailable ranges\n",
> 

Also, just wondering, will PFN 0 still get initialized?

-- 
Thanks,

David / dhildenb