From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 129DFC56201 for ; Wed, 11 Nov 2020 09:28:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7F67E2072C for ; Wed, 11 Nov 2020 09:28:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F67E2072C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D437B6B007B; Wed, 11 Nov 2020 04:28:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE1B76B0074; Wed, 11 Nov 2020 04:28:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0E496B007B; Wed, 11 Nov 2020 04:28:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 5E4826B0074 for ; Wed, 11 Nov 2020 04:28:26 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F3BE6879E for ; Wed, 11 Nov 2020 09:28:25 +0000 (UTC) X-FDA: 77471611770.03.lead83_191470d272fc Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id C281F28A4E9 for ; Wed, 11 Nov 2020 09:28:25 +0000 (UTC) X-HE-Tag: lead83_191470d272fc X-Filterd-Recvd-Size: 5323 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Nov 2020 09:28:25 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id CAF00AD29; Wed, 11 Nov 2020 09:28:22 +0000 (UTC) From: Vlastimil Babka To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Michal Hocko , Pavel Tatashin , David Hildenbrand , Oscar Salvador , Joonsoo Kim , Vlastimil Babka , Michal Hocko Subject: [PATCH v3 6/7] mm, page_alloc: move draining pcplists to page isolation users Date: Wed, 11 Nov 2020 10:28:11 +0100 Message-Id: <20201111092812.11329-7-vbabka@suse.cz> X-Mailer: git-send-email 2.29.1 In-Reply-To: <20201111092812.11329-1-vbabka@suse.cz> References: <20201111092812.11329-1-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, pcplists are drained during set_migratetype_isolate() which me= ans once per pageblock processed start_isolate_page_range(). This is somewhat wasteful. Moreover, the callers might need different guarantees, and the draining is currently prone to races and does not guarantee that no page from isolated pageblock will end up on the pcplist after the drain. Better guarantees are added by later patches and require explicit actions by page isolation users that need them. Thus it makes sense to move the current imperfect draining to the callers also as a preparation step. Suggested-by: David Hildenbrand Suggested-by: Pavel Tatashin Signed-off-by: Vlastimil Babka Reviewed-by: David Hildenbrand Reviewed-by: Oscar Salvador Acked-by: Michal Hocko --- mm/memory_hotplug.c | 11 ++++++----- mm/page_alloc.c | 2 ++ mm/page_isolation.c | 10 +++++----- 3 files changed, 13 insertions(+), 10 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 41c62295292b..3c494ab0d075 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1500,6 +1500,8 @@ int __ref offline_pages(unsigned long start_pfn, un= signed long nr_pages) goto failed_removal; } =20 + drain_all_pages(zone); + arg.start_pfn =3D start_pfn; arg.nr_pages =3D nr_pages; node_states_check_changes_offline(nr_pages, zone, &arg); @@ -1550,11 +1552,10 @@ int __ref offline_pages(unsigned long start_pfn, = unsigned long nr_pages) } =20 /* - * per-cpu pages are drained in start_isolate_page_range, but if - * there are still pages that are not free, make sure that we - * drain again, because when we isolated range we might - * have raced with another thread that was adding pages to pcp - * list. + * per-cpu pages are drained after start_isolate_page_range, but + * if there are still pages that are not free, make sure that we + * drain again, because when we isolated range we might have + * raced with another thread that was adding pages to pcp list. * * Forward progress should be still guaranteed because * pages on the pcp list can only belong to MOVABLE_ZONE diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 25bc9bb77696..2ec3e1e27169 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8539,6 +8539,8 @@ int alloc_contig_range(unsigned long start, unsigne= d long end, if (ret) return ret; =20 + drain_all_pages(cc.zone); + /* * In case of -EBUSY, we'd like to know which page causes problem. * So, just fall through. test_pages_isolated() has a tracepoint diff --git a/mm/page_isolation.c b/mm/page_isolation.c index abbf42214485..feab446d1982 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -49,7 +49,6 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ =20 __mod_zone_freepage_state(zone, -nr_pages, mt); spin_unlock_irqrestore(&zone->lock, flags); - drain_all_pages(zone); return 0; } =20 @@ -172,11 +171,12 @@ __first_valid_page(unsigned long pfn, unsigned long= nr_pages) * * Please note that there is no strong synchronization with the page all= ocator * either. Pages might be freed while their page blocks are marked ISOLA= TED. - * In some cases pages might still end up on pcp lists and that would al= low + * A call to drain_all_pages() after isolation can flush most of them. H= owever + * in some cases pages might still end up on pcp lists and that would al= low * for their allocation even when they are in fact isolated already. Dep= ending - * on how strong of a guarantee the caller needs drain_all_pages might b= e needed - * (e.g. __offline_pages will need to call it after check for isolated r= ange for - * a next retry). + * on how strong of a guarantee the caller needs, further drain_all_page= s() + * might be needed (e.g. __offline_pages will need to call it after chec= k for + * isolated range for a next retry). * * Return: 0 on success and -EBUSY if any part of range cannot be isolat= ed. */ --=20 2.29.1