From: Yang Shi <yang.shi@linux.alibaba.com>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: mhocko@suse.com, shakeelb@google.com, akpm@linux-foundation.org,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining
Date: Wed, 9 Jan 2019 12:36:11 -0800 [thread overview]
Message-ID: <d92912c7-511e-2ab5-39a6-38af3209fcaf@linux.alibaba.com> (raw)
In-Reply-To: <20190109193247.GA16319@cmpxchg.org>
On 1/9/19 11:32 AM, Johannes Weiner wrote:
> On Thu, Jan 10, 2019 at 03:14:40AM +0800, Yang Shi wrote:
>> We have some usecases which create and remove memcgs very frequently,
>> and the tasks in the memcg may just access the files which are unlikely
>> accessed by anyone else. So, we prefer force_empty the memcg before
>> rmdir'ing it to reclaim the page cache so that they don't get
>> accumulated to incur unnecessary memory pressure. Since the memory
>> pressure may incur direct reclaim to harm some latency sensitive
>> applications.
> We have kswapd for exactly this purpose. Can you lay out more details
> on why that is not good enough, especially in conjunction with tuning
> the watermark_scale_factor etc.?
watermark_scale_factor does help out for some workloads in general.
However, memcgs might be created then do memory allocation faster than
kswapd in some our workloads. And, the tune may work for one kind
machine or workload, but may not work for others. But, we may have
different kind workloads (for example, latency-sensitive and batch jobs)
run on the same machine, so it is kind of hard for us to guarantee all
the workloads work well together by relying on kswapd and
watermark_scale_factor only.
And, we know the page cache access pattern would be one-off for some
memcgs, and those page caches are unlikely shared by others, so why not
just drop them when the memcg is offlined. Reclaiming those cold page
caches earlier would also improve the efficiency of memcg creation for
long run.
>
> We've been pretty adamant that users shouldn't use drop_caches for
> performance for example, and that the need to do this usually is
> indicative of a problem or suboptimal tuning in the VM subsystem.
>
> How is this different?
IMHO, that depends on the usecases and workloads. As I mentioned above,
if we know some page caches from some memcgs are referenced one-off and
unlikely shared, why just keep them around to increase memory pressure?
Thanks,
Yang
next prev parent reply other threads:[~2019-01-09 20:41 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-09 19:14 [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining Yang Shi
2019-01-09 19:14 ` [v3 PATCH 1/5] doc: memcontrol: fix the obsolete content about force empty Yang Shi
2019-01-09 19:14 ` [v3 PATCH 2/5] mm: memcontrol: add may_swap parameter to mem_cgroup_force_empty() Yang Shi
2019-01-09 19:14 ` [v3 PATCH 3/5] mm: memcontrol: introduce wipe_on_offline interface Yang Shi
2019-01-09 19:14 ` [v3 PATCH 4/5] mm: memcontrol: bring force_empty into default hierarchy Yang Shi
2019-01-09 19:14 ` [v3 PATCH 5/5] doc: memcontrol: add description for wipe_on_offline Yang Shi
2019-01-10 12:00 ` William Kucharski
2019-01-09 19:32 ` [RFC v3 PATCH 0/5] mm: memcontrol: do memory reclaim when offlining Johannes Weiner
2019-01-09 20:36 ` Yang Shi [this message]
2019-01-09 21:23 ` Johannes Weiner
2019-01-09 22:09 ` Yang Shi
2019-01-09 22:51 ` Johannes Weiner
2019-01-10 1:47 ` Yang Shi
2019-01-14 19:01 ` Johannes Weiner
2019-01-17 22:55 ` Yang Shi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d92912c7-511e-2ab5-39a6-38af3209fcaf@linux.alibaba.com \
--to=yang.shi@linux.alibaba.com \
--cc=akpm@linux-foundation.org \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=shakeelb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).