From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D282DC2BB40 for ; Tue, 15 Dec 2020 03:10:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 84F6622509 for ; Tue, 15 Dec 2020 03:10:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84F6622509 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 25D418D0040; Mon, 14 Dec 2020 22:10:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 20D4C8D001C; Mon, 14 Dec 2020 22:10:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 153BD8D0040; Mon, 14 Dec 2020 22:10:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id F22D18D001C for ; Mon, 14 Dec 2020 22:10:58 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BF6921F06 for ; Tue, 15 Dec 2020 03:10:58 +0000 (UTC) X-FDA: 77594039796.06.note92_1c10ec927420 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 9C620100347BE for ; Tue, 15 Dec 2020 03:10:58 +0000 (UTC) X-HE-Tag: note92_1c10ec927420 X-Filterd-Recvd-Size: 5281 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Dec 2020 03:10:58 +0000 (UTC) Date: Mon, 14 Dec 2020 19:10:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608001857; bh=AqOkpiopoB86G+6duY3JvqamyH8hA/GBsKhH70AsAZM=; h=From:To:Subject:In-Reply-To:From; b=YzVFotE7fGvyGkH3KKIMzyKdE/tPfRhrdUCbW/7+V2MRZjucms0COoXEBG2YIqB7a y8GcmCs9+K7mE0H/vr7WlFkHWLrom938EmOyVa5Rs3fuGHUvbSiVXd+OAd/c3mAaBM SU2xST60R6J9Gxrs0P03HXODGW4lre8pB/eoRw5A= From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 128/200] mm, page_alloc: move draining pcplists to page isolation users Message-ID: <20201215031056.ZUBbvjltF%akpm@linux-foundation.org> In-Reply-To: <20201214190237.a17b70ae14f129e2dca3d204@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, page_alloc: move draining pcplists to page isolation users Currently, pcplists are drained during set_migratetype_isolate() which means once per pageblock processed start_isolate_page_range(). This is somewhat wasteful. Moreover, the callers might need different guarantees, and the draining is currently prone to races and does not guarantee that no page from isolated pageblock will end up on the pcplist after the drain. Better guarantees are added by later patches and require explicit actions by page isolation users that need them. Thus it makes sense to move the current imperfect draining to the callers also as a preparation step. Link: https://lkml.kernel.org/r/20201111092812.11329-7-vbabka@suse.cz Suggested-by: David Hildenbrand Suggested-by: Pavel Tatashin Signed-off-by: Vlastimil Babka Reviewed-by: David Hildenbrand Reviewed-by: Oscar Salvador Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 11 ++++++----- mm/page_alloc.c | 2 ++ mm/page_isolation.c | 10 +++++----- 3 files changed, 13 insertions(+), 10 deletions(-) --- a/mm/memory_hotplug.c~mm-page_alloc-move-draining-pcplists-to-page-isolation-users +++ a/mm/memory_hotplug.c @@ -1500,6 +1500,8 @@ int __ref offline_pages(unsigned long st goto failed_removal; } + drain_all_pages(zone); + arg.start_pfn = start_pfn; arg.nr_pages = nr_pages; node_states_check_changes_offline(nr_pages, zone, &arg); @@ -1550,11 +1552,10 @@ int __ref offline_pages(unsigned long st } /* - * per-cpu pages are drained in start_isolate_page_range, but if - * there are still pages that are not free, make sure that we - * drain again, because when we isolated range we might - * have raced with another thread that was adding pages to pcp - * list. + * per-cpu pages are drained after start_isolate_page_range, but + * if there are still pages that are not free, make sure that we + * drain again, because when we isolated range we might have + * raced with another thread that was adding pages to pcp list. * * Forward progress should be still guaranteed because * pages on the pcp list can only belong to MOVABLE_ZONE --- a/mm/page_alloc.c~mm-page_alloc-move-draining-pcplists-to-page-isolation-users +++ a/mm/page_alloc.c @@ -8528,6 +8528,8 @@ int alloc_contig_range(unsigned long sta if (ret) return ret; + drain_all_pages(cc.zone); + /* * In case of -EBUSY, we'd like to know which page causes problem. * So, just fall through. test_pages_isolated() has a tracepoint --- a/mm/page_isolation.c~mm-page_alloc-move-draining-pcplists-to-page-isolation-users +++ a/mm/page_isolation.c @@ -49,7 +49,6 @@ static int set_migratetype_isolate(struc __mod_zone_freepage_state(zone, -nr_pages, mt); spin_unlock_irqrestore(&zone->lock, flags); - drain_all_pages(zone); return 0; } @@ -172,11 +171,12 @@ __first_valid_page(unsigned long pfn, un * * Please note that there is no strong synchronization with the page allocator * either. Pages might be freed while their page blocks are marked ISOLATED. - * In some cases pages might still end up on pcp lists and that would allow + * A call to drain_all_pages() after isolation can flush most of them. However + * in some cases pages might still end up on pcp lists and that would allow * for their allocation even when they are in fact isolated already. Depending - * on how strong of a guarantee the caller needs drain_all_pages might be needed - * (e.g. __offline_pages will need to call it after check for isolated range for - * a next retry). + * on how strong of a guarantee the caller needs, further drain_all_pages() + * might be needed (e.g. __offline_pages will need to call it after check for + * isolated range for a next retry). * * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ _