Re: [PATCH v5 1/2] memory_hotplug: Free pages as higher order

From: Michal Hocko <mhocko@kernel.org>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: vatsa@codeaurora.org, linux-mm@kvack.org,
	Arun KS <arunks@codeaurora.org>,
	getarunks@gmail.com, kys@microsoft.com,
	boris.ostrovsky@oracle.com, sthemmin@microsoft.com,
	xen-devel@lists.xenproject.org, haiyangz@microsoft.com,
	yasu.isimatu@gmail.com, dan.j.williams@intel.com,
	akpm@linux-foundation.org, osalvador@suse.de, jgross@suse.com,
	aaron.lu@intel.com, malat@debian.org, gregkh@linuxfoundation.org,
	linux-kernel@vger.kernel.org, vinmenon@codeaurora.org,
	jrdr.linux@gmail.com, devel@linuxdriverproject.org,
	iamjoonsoo.kim@lge.com, mgorman@techsingularity.net,
	kirill.shutemov@linux.intel.com
Subject: Re: [PATCH v5 1/2] memory_hotplug: Free pages as higher order
Date: Thu, 11 Oct 2018 10:15:10 +0200	[thread overview]
Message-ID: <20181011081510.GR5873__49925.1101784923$1539245642$gmane$org@dhcp22.suse.cz> (raw)
In-Reply-To: <97d8db4c-f117-8216-5f48-d5991692c867@suse.cz>

On Thu 11-10-18 10:07:02, Vlastimil Babka wrote:
> On 10/10/18 6:56 PM, Arun KS wrote:
> > On 2018-10-10 21:00, Vlastimil Babka wrote:
> >> On 10/5/18 10:10 AM, Arun KS wrote:
> >>> When free pages are done with higher order, time spend on
> >>> coalescing pages by buddy allocator can be reduced. With
> >>> section size of 256MB, hot add latency of a single section
> >>> shows improvement from 50-60 ms to less than 1 ms, hence
> >>> improving the hot add latency by 60%. Modify external
> >>> providers of online callback to align with the change.
> >>>
> >>> Signed-off-by: Arun KS <arunks@codeaurora.org>
> >>
> >> [...]
> >>
> >>> @@ -655,26 +655,44 @@ void __online_page_free(struct page *page)
> >>>  }
> >>>  EXPORT_SYMBOL_GPL(__online_page_free);
> >>>
> >>> -static void generic_online_page(struct page *page)
> >>> +static int generic_online_page(struct page *page, unsigned int order)
> >>>  {
> >>> -	__online_page_set_limits(page);
> >>
> >> This is now not called anymore, although the xen/hv variants still do
> >> it. The function seems empty these days, maybe remove it as a followup
> >> cleanup?
> >>
> >>> -	__online_page_increment_counters(page);
> >>> -	__online_page_free(page);
> >>> +	__free_pages_core(page, order);
> >>> +	totalram_pages += (1UL << order);
> >>> +#ifdef CONFIG_HIGHMEM
> >>> +	if (PageHighMem(page))
> >>> +		totalhigh_pages += (1UL << order);
> >>> +#endif
> >>
> >> __online_page_increment_counters() would have used
> >> adjust_managed_page_count() which would do the changes under
> >> managed_page_count_lock. Are we safe without the lock? If yes, there
> >> should perhaps be a comment explaining why.
> > 
> > Looks unsafe without managed_page_count_lock. I think better have a 
> > similar implementation of free_boot_core() in memory_hotplug.c like we 
> > had in version 1 of patch. And use adjust_managed_page_count() instead 
> > of page_zone(page)->managed_pages += nr_pages;
> > 
> > https://lore.kernel.org/patchwork/patch/989445/
> 
> Looks like deferred_free_range() has the same problem calling
> __free_pages_core() to adjust zone->managed_pages.

deferred initialization has one thread per node AFAIR so we cannot race
on managed_pages updates. Well, unless some of the mentioned can run
that early which I dunno.

> __free_pages_bootmem() is OK because at that point the system is still
> single-threaded?
> Could be solved by moving that out of __free_pages_core().
> 
> But do we care about readers potentially seeing a store tear? If yes
> then maybe these counters should be converted to atomics...

I wanted to suggest that already but I have no idea whether the lock
instructions would cause more overhead.
-- 
Michal Hocko
SUSE Labs

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel