From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 279DBC433B4 for ; Fri, 21 May 2021 10:29:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ACB23613CE for ; Fri, 21 May 2021 10:29:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ACB23613CE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=techsingularity.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4CF5B8E002B; Fri, 21 May 2021 06:29:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 45E688E0022; Fri, 21 May 2021 06:29:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D3508E002B; Fri, 21 May 2021 06:29:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0151.hostedemail.com [216.40.44.151]) by kanga.kvack.org (Postfix) with ESMTP id EA2358E0022 for ; Fri, 21 May 2021 06:29:00 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 06118180AD815 for ; Fri, 21 May 2021 10:28:59 +0000 (UTC) X-FDA: 78164865198.01.D122680 Received: from outbound-smtp26.blacknight.com (outbound-smtp26.blacknight.com [81.17.249.194]) by imf16.hostedemail.com (Postfix) with ESMTP id 8BF3C801910F for ; Fri, 21 May 2021 10:28:56 +0000 (UTC) Received: from mail.blacknight.com (pemlinmail01.blacknight.ie [81.17.254.10]) by outbound-smtp26.blacknight.com (Postfix) with ESMTPS id 0D865CAC4D for ; Fri, 21 May 2021 11:28:57 +0100 (IST) Received: (qmail 22629 invoked from network); 21 May 2021 10:28:56 -0000 Received: from unknown (HELO stampy.112glenside.lan) (mgorman@techsingularity.net@[84.203.23.168]) by 81.17.254.9 with ESMTPA; 21 May 2021 10:28:56 -0000 From: Mel Gorman To: Linux-MM Cc: Dave Hansen , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Nicholas Piggin , LKML , Mel Gorman Subject: [PATCH 2/6] mm/page_alloc: Disassociate the pcp->high from pcp->batch Date: Fri, 21 May 2021 11:28:22 +0100 Message-Id: <20210521102826.28552-3-mgorman@techsingularity.net> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210521102826.28552-1-mgorman@techsingularity.net> References: <20210521102826.28552-1-mgorman@techsingularity.net> MIME-Version: 1.0 Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.194 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 8BF3C801910F X-Stat-Signature: hbukkyyeebh5ndt55rcfjt1yqiigaq5f X-HE-Tag: 1621592936-660579 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The pcp high watermark is based on the batch size but there is no relationship between them other than it is convenient to use early in boot. This patch takes the first step and bases pcp->high on the zone low watermark split across the number of CPUs local to a zone while the batch size remains the same to avoid increasing allocation latencies. The inten= t behind the default pcp->high is "set the number of PCP pages such that if they are all full that background reclaim is not started prematurely". Note that in this patch the pcp->high values are adjusted after memory hotplug events, min_free_kbytes adjustments and watermark scale factor adjustments but not CPU hotplug events. On a test KVM instance; Before grep -E "high:|batch" /proc/zoneinfo | tail -2 high: 378 batch: 63 After grep -E "high:|batch" /proc/zoneinfo | tail -2 high: 649 batch: 63 Signed-off-by: Mel Gorman --- mm/page_alloc.c | 53 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a48f305f0381..bf5cdc466e6c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2163,14 +2163,6 @@ void __init page_alloc_init_late(void) /* Block until all are initialised */ wait_for_completion(&pgdat_init_all_done_comp); =20 - /* - * The number of managed pages has changed due to the initialisation - * so the pcpu batch and high limits needs to be updated or the limits - * will be artificially small. - */ - for_each_populated_zone(zone) - zone_pcp_update(zone); - /* * We initialized the rest of the deferred pages. Permanently disable * on-demand struct page initialization. @@ -6594,13 +6586,12 @@ static int zone_batchsize(struct zone *zone) int batch; =20 /* - * The per-cpu-pages pools are set to around 1000th of the - * size of the zone. + * The number of pages to batch allocate is either 0.1% + * of the zone or 1MB, whichever is smaller. The batch + * size is striking a balance between allocation latency + * and zone lock contention. */ - batch =3D zone_managed_pages(zone) / 1024; - /* But no more than a meg. */ - if (batch * PAGE_SIZE > 1024 * 1024) - batch =3D (1024 * 1024) / PAGE_SIZE; + batch =3D min(zone_managed_pages(zone) >> 10, (1024 * 1024) / PAGE_SIZE= ); batch /=3D 4; /* We effectively *=3D 4 below */ if (batch < 1) batch =3D 1; @@ -6637,6 +6628,27 @@ static int zone_batchsize(struct zone *zone) #endif } =20 +static int zone_highsize(struct zone *zone) +{ +#ifdef CONFIG_MMU + int high; + int nr_local_cpus; + + /* + * The high value of the pcp is based on the zone low watermark + * when reclaim is potentially active spread across the online + * CPUs local to a zone. Note that early in boot that CPUs may + * not be online yet. + */ + nr_local_cpus =3D max(1U, cpumask_weight(cpumask_of_node(zone_to_nid(zo= ne)))); + high =3D low_wmark_pages(zone) / nr_local_cpus; + + return high; +#else + return 0; +#endif +} + /* * pcp->high and pcp->batch values are related and generally batch is lo= wer * than high. They are also related to pcp->count such that count is low= er @@ -6698,11 +6710,10 @@ static void __zone_set_pageset_high_and_batch(str= uct zone *zone, unsigned long h */ static void zone_set_pageset_high_and_batch(struct zone *zone) { - unsigned long new_high, new_batch; + int new_high, new_batch; =20 - new_batch =3D zone_batchsize(zone); - new_high =3D 6 * new_batch; - new_batch =3D max(1UL, 1 * new_batch); + new_batch =3D max(1, zone_batchsize(zone)); + new_high =3D zone_highsize(zone); =20 if (zone->pageset_high =3D=3D new_high && zone->pageset_batch =3D=3D new_batch) @@ -8170,6 +8181,12 @@ static void __setup_per_zone_wmarks(void) zone->_watermark[WMARK_LOW] =3D min_wmark_pages(zone) + tmp; zone->_watermark[WMARK_HIGH] =3D min_wmark_pages(zone) + tmp * 2; =20 + /* + * The watermark size have changed so update the pcpu batch + * and high limits or the limits may be inappropriate. + */ + zone_set_pageset_high_and_batch(zone); + spin_unlock_irqrestore(&zone->lock, flags); } =20 --=20 2.26.2