From: Muchun Song <songmuchun@bytedance.com> To: Shakeel Butt <shakeelb@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Rafael J. Wysocki" <rafael@kernel.org>, Alexey Dobriyan <adobriyan@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, Johannes Weiner <hannes@cmpxchg.org>, Michal Hocko <mhocko@kernel.org>, Vladimir Davydov <vdavydov.dev@gmail.com>, Hugh Dickins <hughd@google.com>, Roman Gushchin <guro@fb.com>, Sami Tolvanen <samitolvanen@google.com>, Feng Tang <feng.tang@intel.com>, Neil Brown <neilb@suse.de>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Randy Dunlap <rdunlap@infradead.org>, LKML <linux-kernel@vger.kernel.org>, linux-fsdevel <linux-fsdevel@vger.kernel.org>, Linux MM <linux-mm@kvack.org>, Cgroups <cgroups@vger.kernel.org> Subject: Re: [External] Re: [PATCH v5 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Thu, 24 Dec 2020 10:45:58 +0800 [thread overview] Message-ID: <CAMZfGtVQvD4o-nVdCqNBjWtjDzxcfqme9xMH9ar=C=_sMyDm+g@mail.gmail.com> (raw) In-Reply-To: <CALvZod7kMhb7k6rDZj18JTE=RMji-SinJmfdcPbN9PUL9Off_w@mail.gmail.com> On Thu, Dec 24, 2020 at 6:08 AM Shakeel Butt <shakeelb@google.com> wrote: > > On Wed, Dec 16, 2020 at 7:45 PM Muchun Song <songmuchun@bytedance.com> wrote: > > > > Currently we use struct per_cpu_nodestat to cache the vmstat > > counters, which leads to inaccurate statistics expecially THP > > *especially Thanks. > > > vmstat counters. In the systems with hundreads of processors > > *hundreds Thanks. > > > it can be GBs of memory. For example, for a 96 CPUs system, > > the threshold is the maximum number of 125. And the per cpu > > counters can cache 23.4375 GB in total. > > > > The THP page is already a form of batched addition (it will > > add 512 worth of memory in one go) so skipping the batching > > seems like sensible. Although every THP stats update overflows > > the per-cpu counter, resorting to atomic global updates. But > > it can make the statistics more accuracy for the THP vmstat > > counters. > > > > So we convert the NR_ANON_THPS account to pages. This patch > > is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs > > on compound page arrival"). Doing this also can make the unit > > of vmstat counters more unified. Finally, the unit of the vmstat > > counters are pages, kB and bytes. The B/KB suffix can tell us > > that the unit is bytes or kB. The rest which is without suffix > > are pages. > > > > Signed-off-by: Muchun Song <songmuchun@bytedance.com> > > I agree with the motivation behind this patch but I would like to see > some performance numbers in the commit message. We might agree to pay > the price but at least we will know what exactly that cost is. Do you have any recommendations about benchmarks? I can do a test. Thanks very much. -- Yours, Muchun
WARNING: multiple messages have this Message-ID (diff)
From: Muchun Song <songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org> To: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Cc: Greg Kroah-Hartman <gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>, "Rafael J. Wysocki" <rafael-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Alexey Dobriyan <adobriyan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>, Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Vladimir Davydov <vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>, Sami Tolvanen <samitolvanen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Feng Tang <feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>, Neil Brown <neilb-l3A5Bk7waGM@public.gmane.org>, Joonsoo Kim <iamjoonsoo.kim-Hm3cg6mZ9cc@public.gmane.org>, Randy Dunlap <rdunlap-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, linux-fsdevel <linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>, Cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> Subject: Re: [External] Re: [PATCH v5 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Thu, 24 Dec 2020 10:45:58 +0800 [thread overview] Message-ID: <CAMZfGtVQvD4o-nVdCqNBjWtjDzxcfqme9xMH9ar=C=_sMyDm+g@mail.gmail.com> (raw) In-Reply-To: <CALvZod7kMhb7k6rDZj18JTE=RMji-SinJmfdcPbN9PUL9Off_w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> On Thu, Dec 24, 2020 at 6:08 AM Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote: > > On Wed, Dec 16, 2020 at 7:45 PM Muchun Song <songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org> wrote: > > > > Currently we use struct per_cpu_nodestat to cache the vmstat > > counters, which leads to inaccurate statistics expecially THP > > *especially Thanks. > > > vmstat counters. In the systems with hundreads of processors > > *hundreds Thanks. > > > it can be GBs of memory. For example, for a 96 CPUs system, > > the threshold is the maximum number of 125. And the per cpu > > counters can cache 23.4375 GB in total. > > > > The THP page is already a form of batched addition (it will > > add 512 worth of memory in one go) so skipping the batching > > seems like sensible. Although every THP stats update overflows > > the per-cpu counter, resorting to atomic global updates. But > > it can make the statistics more accuracy for the THP vmstat > > counters. > > > > So we convert the NR_ANON_THPS account to pages. This patch > > is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs > > on compound page arrival"). Doing this also can make the unit > > of vmstat counters more unified. Finally, the unit of the vmstat > > counters are pages, kB and bytes. The B/KB suffix can tell us > > that the unit is bytes or kB. The rest which is without suffix > > are pages. > > > > Signed-off-by: Muchun Song <songmuchun-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org> > > I agree with the motivation behind this patch but I would like to see > some performance numbers in the commit message. We might agree to pay > the price but at least we will know what exactly that cost is. Do you have any recommendations about benchmarks? I can do a test. Thanks very much. -- Yours, Muchun
next prev parent reply other threads:[~2020-12-24 2:47 UTC|newest] Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-17 3:43 [PATCH v5 0/7] Convert all THP vmstat counters to pages Muchun Song 2020-12-17 3:43 ` [PATCH v5 1/7] mm: memcontrol: fix NR_ANON_THPS accounting in charge moving Muchun Song 2020-12-23 21:01 ` Shakeel Butt 2020-12-23 21:01 ` Shakeel Butt 2020-12-17 3:43 ` [PATCH v5 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Muchun Song 2020-12-23 22:08 ` Shakeel Butt 2020-12-23 22:08 ` Shakeel Butt 2020-12-24 2:45 ` Muchun Song [this message] 2020-12-24 2:45 ` [External] " Muchun Song 2020-12-24 2:45 ` Muchun Song 2020-12-17 3:43 ` [PATCH v5 3/7] mm: memcontrol: convert NR_FILE_THPS " Muchun Song 2020-12-17 3:43 ` [PATCH v5 4/7] mm: memcontrol: convert NR_SHMEM_THPS " Muchun Song 2020-12-17 3:43 ` Muchun Song 2020-12-17 3:43 ` [PATCH v5 5/7] mm: memcontrol: convert NR_SHMEM_PMDMAPPED " Muchun Song 2020-12-17 3:43 ` [PATCH v5 6/7] mm: memcontrol: convert NR_FILE_PMDMAPPED " Muchun Song 2020-12-17 3:43 ` Muchun Song 2020-12-17 3:43 ` [PATCH v5 7/7] mm: memcontrol: make the slab calculation consistent Muchun Song 2020-12-23 21:21 ` Shakeel Butt 2020-12-23 21:21 ` Shakeel Butt 2020-12-23 21:21 ` Shakeel Butt 2020-12-24 2:43 ` [External] " Muchun Song 2020-12-24 2:43 ` Muchun Song 2020-12-24 2:43 ` Muchun Song
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAMZfGtVQvD4o-nVdCqNBjWtjDzxcfqme9xMH9ar=C=_sMyDm+g@mail.gmail.com' \ --to=songmuchun@bytedance.com \ --cc=adobriyan@gmail.com \ --cc=akpm@linux-foundation.org \ --cc=cgroups@vger.kernel.org \ --cc=feng.tang@intel.com \ --cc=gregkh@linuxfoundation.org \ --cc=guro@fb.com \ --cc=hannes@cmpxchg.org \ --cc=hughd@google.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mhocko@kernel.org \ --cc=neilb@suse.de \ --cc=rafael@kernel.org \ --cc=rdunlap@infradead.org \ --cc=samitolvanen@google.com \ --cc=shakeelb@google.com \ --cc=vdavydov.dev@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.