From: Shakeel Butt <shakeelb@google.com> To: Johannes Weiner <hannes@cmpxchg.org> Cc: Mel Gorman <mgorman@suse.de>, Roman Gushchin <guro@fb.com>, Michal Hocko <mhocko@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Yafang Shao <laoar.shao@gmail.com>, Linux MM <linux-mm@kvack.org>, Cgroups <cgroups@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org> Subject: Re: [PATCH] memcg: expose root cgroup's memory.stat Date: Sat, 9 May 2020 07:06:38 -0700 [thread overview] Message-ID: <CALvZod5VHHUV+_AXs4+5sLOPGyxm709kQ1q=uHMPVxW8pwXZ=g@mail.gmail.com> (raw) In-Reply-To: <20200508214405.GA226164@cmpxchg.org> On Fri, May 8, 2020 at 2:44 PM Johannes Weiner <hannes@cmpxchg.org> wrote: > > On Fri, May 08, 2020 at 10:06:30AM -0700, Shakeel Butt wrote: > > One way to measure the efficiency of memory reclaim is to look at the > > ratio (pgscan+pfrefill)/pgsteal. However at the moment these stats are > > not updated consistently at the system level and the ratio of these are > > not very meaningful. The pgsteal and pgscan are updated for only global > > reclaim while pgrefill gets updated for global as well as cgroup > > reclaim. > > > > Please note that this difference is only for system level vmstats. The > > cgroup stats returned by memory.stat are actually consistent. The > > cgroup's pgsteal contains number of reclaimed pages for global as well > > as cgroup reclaim. So, one way to get the system level stats is to get > > these stats from root's memory.stat, so, expose memory.stat for the root > > cgroup. > > > > from Johannes Weiner: > > There are subtle differences between /proc/vmstat and > > memory.stat, and cgroup-aware code that wants to watch the full > > hierarchy currently has to know about these intricacies and > > translate semantics back and forth. > > > > Generally having the fully recursive memory.stat at the root > > level could help a broader range of usecases. > > The changelog begs the question why we don't just "fix" the > system-level stats. It may be useful to include the conclusions from > that discussion, and why there is value in keeping the stats this way. > Right. Andrew, can you please add the following para to the changelog? Why not fix the stats by including both the global and cgroup reclaim activity instead of exposing root cgroup's memory.stat? The reason is the benefit of having metrics exposing the activity that happens purely due to machine capacity rather than localized activity that happens due to the limits throughout the cgroup tree. Additionally there are userspace tools like sysstat(sar) which reads these stats to inform about the system level reclaim activity. So, we should not break such use-cases. > > Signed-off-by: Shakeel Butt <shakeelb@google.com> > > Suggested-by: Johannes Weiner <hannes@cmpxchg.org> > > Acked-by: Johannes Weiner <hannes@cmpxchg.org> Thanks a lot.
WARNING: multiple messages have this Message-ID (diff)
From: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> To: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> Cc: Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>, Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>, Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, Yafang Shao <laoar.shao-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>, Cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> Subject: Re: [PATCH] memcg: expose root cgroup's memory.stat Date: Sat, 9 May 2020 07:06:38 -0700 [thread overview] Message-ID: <CALvZod5VHHUV+_AXs4+5sLOPGyxm709kQ1q=uHMPVxW8pwXZ=g@mail.gmail.com> (raw) In-Reply-To: <20200508214405.GA226164-druUgvl0LCNAfugRpC6u6w@public.gmane.org> On Fri, May 8, 2020 at 2:44 PM Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> wrote: > > On Fri, May 08, 2020 at 10:06:30AM -0700, Shakeel Butt wrote: > > One way to measure the efficiency of memory reclaim is to look at the > > ratio (pgscan+pfrefill)/pgsteal. However at the moment these stats are > > not updated consistently at the system level and the ratio of these are > > not very meaningful. The pgsteal and pgscan are updated for only global > > reclaim while pgrefill gets updated for global as well as cgroup > > reclaim. > > > > Please note that this difference is only for system level vmstats. The > > cgroup stats returned by memory.stat are actually consistent. The > > cgroup's pgsteal contains number of reclaimed pages for global as well > > as cgroup reclaim. So, one way to get the system level stats is to get > > these stats from root's memory.stat, so, expose memory.stat for the root > > cgroup. > > > > from Johannes Weiner: > > There are subtle differences between /proc/vmstat and > > memory.stat, and cgroup-aware code that wants to watch the full > > hierarchy currently has to know about these intricacies and > > translate semantics back and forth. > > > > Generally having the fully recursive memory.stat at the root > > level could help a broader range of usecases. > > The changelog begs the question why we don't just "fix" the > system-level stats. It may be useful to include the conclusions from > that discussion, and why there is value in keeping the stats this way. > Right. Andrew, can you please add the following para to the changelog? Why not fix the stats by including both the global and cgroup reclaim activity instead of exposing root cgroup's memory.stat? The reason is the benefit of having metrics exposing the activity that happens purely due to machine capacity rather than localized activity that happens due to the limits throughout the cgroup tree. Additionally there are userspace tools like sysstat(sar) which reads these stats to inform about the system level reclaim activity. So, we should not break such use-cases. > > Signed-off-by: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> > > Suggested-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> > > Acked-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> Thanks a lot.
next prev parent reply other threads:[~2020-05-09 14:06 UTC|newest] Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-05-08 17:06 [PATCH] memcg: expose root cgroup's memory.stat Shakeel Butt 2020-05-08 17:06 ` Shakeel Butt 2020-05-08 21:44 ` Johannes Weiner 2020-05-08 21:44 ` Johannes Weiner 2020-05-09 14:06 ` Shakeel Butt [this message] 2020-05-09 14:06 ` Shakeel Butt 2020-05-09 14:06 ` Shakeel Butt 2020-05-09 14:43 ` Yafang Shao 2020-05-09 14:43 ` Yafang Shao 2020-05-09 14:43 ` Yafang Shao 2020-05-15 8:29 ` Michal Hocko 2020-05-15 8:29 ` Michal Hocko 2020-05-15 13:24 ` Johannes Weiner 2020-05-15 13:44 ` Shakeel Butt 2020-05-15 13:44 ` Shakeel Butt 2020-05-15 15:00 ` Roman Gushchin 2020-05-15 15:00 ` Roman Gushchin 2020-05-15 17:49 ` Shakeel Butt 2020-05-15 17:49 ` Shakeel Butt 2020-05-15 18:09 ` Johannes Weiner 2020-05-15 18:09 ` Johannes Weiner 2020-05-16 0:13 ` Shakeel Butt 2020-05-16 0:13 ` Shakeel Butt 2020-05-15 18:09 ` Roman Gushchin 2020-05-16 0:06 ` Shakeel Butt 2020-05-16 0:06 ` Shakeel Butt 2020-05-16 1:42 ` Chris Down 2020-05-16 1:42 ` Chris Down
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CALvZod5VHHUV+_AXs4+5sLOPGyxm709kQ1q=uHMPVxW8pwXZ=g@mail.gmail.com' \ --to=shakeelb@google.com \ --cc=akpm@linux-foundation.org \ --cc=cgroups@vger.kernel.org \ --cc=guro@fb.com \ --cc=hannes@cmpxchg.org \ --cc=laoar.shao@gmail.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@suse.de \ --cc=mhocko@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.