From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 965CFC4361B for ; Sun, 6 Dec 2020 10:18:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3F4BA22D2C for ; Sun, 6 Dec 2020 10:18:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F4BA22D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D87828D0003; Sun, 6 Dec 2020 05:18:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D38196B0074; Sun, 6 Dec 2020 05:18:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C276B8D0003; Sun, 6 Dec 2020 05:18:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id AD7246B0073 for ; Sun, 6 Dec 2020 05:18:24 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7C7941EE6 for ; Sun, 6 Dec 2020 10:18:24 +0000 (UTC) X-FDA: 77562457728.03.car43_480aaf5273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 4201028A4EB for ; Sun, 6 Dec 2020 10:18:24 +0000 (UTC) X-HE-Tag: car43_480aaf5273d4 X-Filterd-Recvd-Size: 8777 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:23 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id t6so37852plq.1 for ; Sun, 06 Dec 2020 02:18:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=0MtFDJWkKOQQMDcBAWzMx6gdC5YkqHBAIYhp+BQWqaLCRwKSjpNgXt0q+GE5YDnhq7 zZlx16g/g5Hty2Kbo00woChd1R2HTbXvLREFGr+dgsJNg5wqsoVf732VjIl8t0tx/guG xKJjwyDtJ23mJ3u2LiGrZt8158SzBOEHqu1EONODMauwN44TsNCh+5mOLcjf6KWCRj+Q PpDhsHo4azjOMj55FDu9gE2qP5mrbjysGxlYeo4D7bHzSkfdcbJN0sB1YfNpHorDM+rk KTXv8MboLM+aP9XEdWHdigIwbkC+YMT4rLQKIz5VS8XacdkjzRDyPoEiVJiUVpjvTk70 B/IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=KwObw7VvRTSXZRvGTjAD7PM6qufoINlZJzxiEPAPJW+g0w0xkg7J2gevFVRTSPi21w ogIZom4V19wl8ht76vuLikPM/Lt4A0g25ozB4dyndiIJPrzP6hemgskTChGIjGH8r8ZA xD88sp8GVsYK4AhlBCcwP1HxxzZ9AZMWMVnlhh+HTwK6nr8ZRw6jIJOiSD3IyQLYA/mE 1x9c9Swt4OUBY9RRscDTMUqCEjPazvtnAzqY7B3cIabwi5K9I3N92PMkpklzbHAQ9KWY IV3ijN7A/91LiuIAHwZcflbRTHwD1eY/tkrqFHm57cDQNLRhUGNdNE/kFESG9ADCHDuK bmXw== X-Gm-Message-State: AOAM5327J6Md89QdQ5GZlY8/X1r3voIwq93V73p99e+I64nQTeeK8K8K lefMCZhrt4w2g5mQ8mgmiXOZQQ== X-Google-Smtp-Source: ABdhPJz5DNYCSJVhDue++Ah/epNUxyRIwmJ9vSLViSgMOWI7q8z87Dx9FtBFYEBh1GWmEzjkBxvqzQ== X-Received: by 2002:a17:902:b582:b029:d6:6008:264d with SMTP id a2-20020a170902b582b02900d66008264dmr11050550pls.80.1607249902812; Sun, 06 Dec 2020 02:18:22 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:22 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 10/12] mm: memcontrol: scale stat_threshold for byted-sized vmstat Date: Sun, 6 Dec 2020 18:14:49 +0800 Message-Id: <20201206101451.14706-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some vmstat counters are being accounted in bytes not pages, so the stat_threshold should also scale to bytes. The vmstat counters are already long type for memcg (can reference to struct lruvec_stat). For the global per-node vmstat counters also can scale to long. But the maximum vmstat threshold is 125, so the type of s32 is enough. Signed-off-by: Muchun Song --- include/linux/mmzone.h | 17 ++++++----------- include/linux/vmstat.h | 1 - mm/vmstat.c | 24 +++++++++++++----------- 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 1f9c83778629..d53328551225 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -216,17 +216,12 @@ enum node_stat_item { */ static __always_inline bool vmstat_item_in_bytes(int idx) { - /* - * Global and per-node slab counters track slab pages. - * It's expected that changes are multiples of PAGE_SIZE. - * Internally values are stored in pages. - * - * Per-memcg and per-lruvec counters track memory, consumed - * by individual slab objects. These counters are actually - * byte-precise. - */ return (idx =3D=3D NR_SLAB_RECLAIMABLE_B || - idx =3D=3D NR_SLAB_UNRECLAIMABLE_B); + idx =3D=3D NR_SLAB_UNRECLAIMABLE_B || +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + idx =3D=3D NR_KERNEL_SCS_B || +#endif + idx =3D=3D NR_KERNEL_STACK_B); } =20 /* @@ -340,7 +335,7 @@ struct per_cpu_pageset { =20 struct per_cpu_nodestat { s8 stat_threshold; - s8 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; + s32 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; }; =20 #endif /* !__GENERATING_BOUNDS.H */ diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index fd1a3d5d4926..afd84dc2398c 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -211,7 +211,6 @@ static inline unsigned long global_node_page_state(en= um node_stat_item item) { long x =3D atomic_long_read(&vm_node_stat[item]); =20 - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); #ifdef CONFIG_SMP if (x < 0) x =3D 0; diff --git a/mm/vmstat.c b/mm/vmstat.c index 7fb0c7cb9516..25751b1d8e2e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -341,13 +341,15 @@ void __mod_node_page_state(struct pglist_data *pgda= t, enum node_stat_item item, long delta) { struct per_cpu_nodestat __percpu *pcp =3D pgdat->per_cpu_nodestats; - s8 __percpu *p =3D pcp->vm_node_stat_diff + item; + s32 __percpu *p =3D pcp->vm_node_stat_diff + item; long x; long t; =20 x =3D delta + __this_cpu_read(*p); =20 t =3D __this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<=3D PAGE_SHIFT; =20 if (unlikely(abs(x) > t)) { node_page_state_add(x, pgdat, item); @@ -399,15 +401,15 @@ void __inc_zone_state(struct zone *zone, enum zone_= stat_item item) void __inc_node_state(struct pglist_data *pgdat, enum node_stat_item ite= m) { struct per_cpu_nodestat __percpu *pcp =3D pgdat->per_cpu_nodestats; - s8 __percpu *p =3D pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p =3D pcp->vm_node_stat_diff + item; + s32 v, t; =20 VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); =20 v =3D __this_cpu_inc_return(*p); t =3D __this_cpu_read(pcp->stat_threshold); if (unlikely(v > t)) { - s8 overstep =3D t >> 1; + s32 overstep =3D t >> 1; =20 node_page_state_add(v + overstep, pgdat, item); __this_cpu_write(*p, -overstep); @@ -445,8 +447,8 @@ void __dec_zone_state(struct zone *zone, enum zone_st= at_item item) void __dec_node_state(struct pglist_data *pgdat, enum node_stat_item ite= m) { struct per_cpu_nodestat __percpu *pcp =3D pgdat->per_cpu_nodestats; - s8 __percpu *p =3D pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p =3D pcp->vm_node_stat_diff + item; + s32 v, t; =20 VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); =20 @@ -546,7 +548,7 @@ static inline void mod_node_state(struct pglist_data = *pgdat, enum node_stat_item item, int delta, int overstep_mode) { struct per_cpu_nodestat __percpu *pcp =3D pgdat->per_cpu_nodestats; - s8 __percpu *p =3D pcp->vm_node_stat_diff + item; + s32 __percpu *p =3D pcp->vm_node_stat_diff + item; long o, n, t, z; =20 do { @@ -563,6 +565,8 @@ static inline void mod_node_state(struct pglist_data = *pgdat, * for all cpus in a node. */ t =3D this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<=3D PAGE_SHIFT; =20 o =3D this_cpu_read(*p); n =3D delta + o; @@ -829,7 +833,7 @@ static int refresh_cpu_vm_stats(bool do_pagesets) struct per_cpu_nodestat __percpu *p =3D pgdat->per_cpu_nodestats; =20 for (i =3D 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - int v; + s32 v; =20 v =3D this_cpu_xchg(p->vm_node_stat_diff[i], 0); if (v) { @@ -899,7 +903,7 @@ void cpu_vm_stats_fold(int cpu) =20 for (i =3D 0; i < NR_VM_NODE_STAT_ITEMS; i++) if (p->vm_node_stat_diff[i]) { - int v; + s32 v; =20 v =3D p->vm_node_stat_diff[i]; p->vm_node_stat_diff[i] =3D 0; @@ -1017,8 +1021,6 @@ unsigned long node_page_state(struct pglist_data *p= gdat, { long x =3D atomic_long_read(&pgdat->vm_stat[item]); =20 - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - #ifdef CONFIG_SMP if (x < 0) x =3D 0; --=20 2.11.0