From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 751A9C0018C for ; Mon, 7 Dec 2020 12:57:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EEF0F23403 for ; Mon, 7 Dec 2020 12:57:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EEF0F23403 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F9858D0002; Mon, 7 Dec 2020 07:57:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A9DD8D0001; Mon, 7 Dec 2020 07:57:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C0D38D0002; Mon, 7 Dec 2020 07:57:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id 33FC98D0001 for ; Mon, 7 Dec 2020 07:57:37 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E11841EE6 for ; Mon, 7 Dec 2020 12:57:36 +0000 (UTC) X-FDA: 77566487712.10.maid71_5404901273de Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id C318B16A4A4 for ; Mon, 7 Dec 2020 12:57:36 +0000 (UTC) X-HE-Tag: maid71_5404901273de X-Filterd-Recvd-Size: 6053 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Mon, 7 Dec 2020 12:57:35 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id e23so8791362pgk.12 for ; Mon, 07 Dec 2020 04:57:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=c6NT57stNU0XDGX2I8AiHgmrG8IdQvHx6UdW5r4PrTU=; b=ynKCNHc0hoYaubBLO2yot1cMk5pT7ZlOB4fRj7Htb16hVwAmW8M9gY7MDYOXTJ95Mb SrG+v40Amfbh0/fUq/cdWLsgjIx9Fjl2t7frw9gYAOHxgcBw5ZASltdxl5jGsphNwYMm yZVBFXQYa8FPmhii3kQk31G8hY1Eai+HHbJxW9mTnS4+MfE1cYom490tgrRsNNtwL05+ SLVFqFAlt+DBWc/iocnNSuFe+y3Iukt+GiQi6pKXhGLUtUhCSTP2UyXB+wpYwAxrO/tP h1ycyed2NkO78WuCIjfe7C5hdLFCS9qMFNL4B6dlNSDsClUxbq0ud/F4vMEYuq4UCO4L 5Edg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=c6NT57stNU0XDGX2I8AiHgmrG8IdQvHx6UdW5r4PrTU=; b=VtOvAqMQvXfCr3bAc3l+M4v5MpHjBI+1V/3g2YU8x9UgOohv7rCLLYp9crKSTyZ/fW 6c5h2XYu6en9Qorkgrh7iErkNOhCcn6Du4811fVCK91u/HNGUm8rSlhKE09n07FeERDo 5BCW183Slq5Ewro7vx0HDZbljk3U9lUh71Gf2UKqX0VtpTEqTo3CQ6OoA5voi5/gL3lE wxSDUEYWEMBVGGuiCcHel2SyYFDLdSGSXHf9RcJhQ2i553EjeZAqU1gUEIZYRLSVAB6b 8+5zgljQewXWuL3TTFkso4Dr/avaCwVBSY8X1KRYbutDpPgL3WQWtGkWQRHSyO/ovaE+ dwpw== X-Gm-Message-State: AOAM531H9WQilg0wl5yHbNuBIpOlixotkBegI9cidyUSGYSNWu8OuA0u kZNoqbQnHKAARB+5NwudvyO6tPrL9U3BpgxFK5mAWA== X-Google-Smtp-Source: ABdhPJxDduBK16GT+JoiOw4qerDvuVoZO1uD0cVugIWmA5+CwHWZMgSKCcVsK8E8JIhFH40nZdxIwhaVgrAAvNgRcYE= X-Received: by 2002:a63:1203:: with SMTP id h3mr14738079pgl.273.1607345854770; Mon, 07 Dec 2020 04:57:34 -0800 (PST) MIME-Version: 1.0 References: <20201206085639.12627-1-songmuchun@bytedance.com> <20201207123605.GH25569@dhcp22.suse.cz> In-Reply-To: <20201207123605.GH25569@dhcp22.suse.cz> From: Muchun Song Date: Mon, 7 Dec 2020 20:56:58 +0800 Message-ID: Subject: Re: [External] Re: [PATCH] mm: memcontrol: optimize per-lruvec stats counter memory usage To: Michal Hocko Cc: Johannes Weiner , Vladimir Davydov , Andrew Morton , Shakeel Butt , Roman Gushchin , Stephen Rothwell , alexander.h.duyck@linux.intel.com, Chris Down , Yafang Shao , richard.weiyang@gmail.com, LKML , Cgroups , Linux Memory Management List Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Dec 7, 2020 at 8:36 PM Michal Hocko wrote: > > On Sun 06-12-20 16:56:39, Muchun Song wrote: > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32 > > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat > > to optimize memory usage. > > How much savings are we talking about here? I am not deeply familiar > with the pcp allocator but can it compact smaller data types much > better? It is a percpu struct. The size of struct lruvec_stat is 304(tested on the linux-5.5). So we can save 304 / 2 * nproc bytes per memcg where nproc is the number of the possible CPU. If we have n memory cgroup in the system. Finally, we can save (152 * nproc * n) bytes. In some configurations, nproc here may be 512. And if we have a lot of dying cgroup. The n can be 100, 000 (I once saw it on my server). It can not be smaller. Because the vmstat threshold is 32, and we have some vmstat counters whose unit is byte. So the max bytes is 32 * 4k which is 131072. The s16 can't hold these. Thanks. > > > Signed-off-by: Muchun Song > > --- > > include/linux/memcontrol.h | 6 +++++- > > mm/memcontrol.c | 2 +- > > 2 files changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index f9a496c4eac7..34cf119976b1 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -92,6 +92,10 @@ struct lruvec_stat { > > long count[NR_VM_NODE_STAT_ITEMS]; > > }; > > > > +struct per_cpu_lruvec_stat { > > + s32 count[NR_VM_NODE_STAT_ITEMS]; > > +}; > > + > > /* > > * Bitmap of shrinker::id corresponding to memcg-aware shrinkers, > > * which have elements charged to this memcg. > > @@ -111,7 +115,7 @@ struct mem_cgroup_per_node { > > struct lruvec_stat __percpu *lruvec_stat_local; > > > > /* Subtree VM stats (batched updates) */ > > - struct lruvec_stat __percpu *lruvec_stat_cpu; > > + struct per_cpu_lruvec_stat __percpu *lruvec_stat_cpu; > > atomic_long_t lruvec_stat[NR_VM_NODE_STAT_ITEMS]; > > > > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > index 49fbcf003bf5..c874ea37b05d 100644 > > --- a/mm/memcontrol.c > > +++ b/mm/memcontrol.c > > @@ -5184,7 +5184,7 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) > > return 1; > > } > > > > - pn->lruvec_stat_cpu = alloc_percpu_gfp(struct lruvec_stat, > > + pn->lruvec_stat_cpu = alloc_percpu_gfp(struct per_cpu_lruvec_stat, > > GFP_KERNEL_ACCOUNT); > > if (!pn->lruvec_stat_cpu) { > > free_percpu(pn->lruvec_stat_local); > > -- > > 2.11.0 > > -- > Michal Hocko > SUSE Labs -- Yours, Muchun