From: David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
To: Chris Down <chris-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org>
Cc: Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
Yang Shi <shy828301-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
Yang Shi
<yang.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>,
Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>,
Greg Thelen <gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
Vladimir Davydov
<vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org
Subject: Re: [patch] mm, memcg: provide a stat to describe reclaimable memory
Date: Wed, 15 Jul 2020 11:02:17 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.23.453.2007151046320.2788464@chino.kir.corp.google.com> (raw)
In-Reply-To: <20200715131048.GA176092-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org>
On Wed, 15 Jul 2020, Chris Down wrote:
> Hi David,
>
> I'm somewhat against adding more metrics which try to approximate availability
> of memory when we already know it not to generally manifest very well in
> practice, especially since this *is* calculable by userspace (albeit with some
> knowledge of mm internals). Users and applications often vastly overestimate
> the reliability of these metrics, especially since they heavily depend on
> transient page states and whatever reclaim efficacy happens to be achieved at
> the time there is demand.
>
Hi Chris,
With the proposed anon_reclaimable, do you have any reliability concerns?
This would be the amount of lazy freeable memory and memory that can be
uncharged if compound pages from the deferred split queue are split under
memory pressure. It seems to be a very precise value (as slab_reclaimable
already in memory.stat is), so I'm not sure why there is a reliability
concern. Maybe you can elaborate?
Today, this information is indeed possible to calculate from userspace.
The idea is to present this information that will be backwards compatible,
however, as the kernel implementation changes. When lazy freeable memory
was added, for instance, userspace likely would not have preemptively been
doing an "active_file + inactive_file - file" calculation to factor that
in as reclaimable anon :)
> What do you intend to do with these metrics and how do you envisage other
> users should use them? Is it not possible to rework the strategy to use
> pressure information and/or workingset pressurisation instead?
>
Previously, users would interpret their anon usage as non reclaimable if
swap is disabled and now that value can include a *lot* of easily
reclaimable memory. Our users would also carefully monitor their current
memcg usage and/or anon usage to detect abnormalities without concern for
what is reclaimable, especially for things like deferred split queues that
was purely a kernel implementation change. Memcg usage and anon usag then
becomes wildly different between kernel versions and our users alert on
that abnormality.
The example I gave earlier in the thread showed how dramatically different
memory.current is before and after the introduction of deferred split
queues. Userspace sees ballooning memcg usage and alerts on it (suspects
a memory leak, for example) when in reality this is purely reclaimable
memory under pressure and is the result of a kernel implementation detail.
We plan on factoring this information in when determining what the actual
amount of memory that can and cannot be reclaimed from a memcg hierarchy
at any given time.
next prev parent reply other threads:[~2020-07-15 18:02 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-15 3:18 [patch] mm, memcg: provide a stat to describe reclaimable memory David Rientjes
2020-07-15 3:18 ` David Rientjes
2020-07-15 7:00 ` David Rientjes
2020-07-15 7:00 ` David Rientjes
2020-07-15 7:15 ` SeongJae Park
2020-07-15 7:15 ` SeongJae Park
2020-07-15 17:33 ` David Rientjes
2020-07-15 17:33 ` David Rientjes
2020-07-16 20:58 ` [patch] mm, memcg: provide an anon_reclaimable stat David Rientjes
2020-07-16 20:58 ` David Rientjes
2020-07-16 21:07 ` Shakeel Butt
2020-07-16 21:07 ` Shakeel Butt
2020-07-16 21:28 ` David Rientjes
2020-07-16 21:28 ` David Rientjes
2020-07-17 1:37 ` Shakeel Butt
2020-07-17 1:37 ` Shakeel Butt
2020-07-17 8:34 ` Michal Hocko
2020-07-17 8:34 ` Michal Hocko
2020-07-17 14:39 ` Johannes Weiner
2020-07-17 14:39 ` Johannes Weiner
2020-07-15 13:10 ` [patch] mm, memcg: provide a stat to describe reclaimable memory Chris Down
2020-07-15 13:10 ` Chris Down
[not found] ` <20200715131048.GA176092-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org>
2020-07-15 18:02 ` David Rientjes [this message]
2020-07-17 12:17 ` Chris Down
2020-07-17 12:17 ` Chris Down
2020-07-17 19:37 ` David Rientjes
2020-07-17 19:37 ` David Rientjes
2020-07-20 7:37 ` Michal Hocko
2020-07-20 7:37 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.23.453.2007151046320.2788464@chino.kir.corp.google.com \
--to=rientjes-hpiqsd4aklfqt0dzr+alfa@public.gmane.org \
--cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
--cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=chris-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org \
--cc=gthelen-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=guro-b10kYP2dOMg@public.gmane.org \
--cc=hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org \
--cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
--cc=mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
--cc=shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=shy828301-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
--cc=vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
--cc=yang.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.