From: Jianlin Lv <iecedge@gmail.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: tj@kernel.org, lizefan.x@bytedance.com, hannes@cmpxchg.org,
corbet@lwn.net, mhocko@kernel.org, roman.gushchin@linux.dev,
shakeelb@google.com, muchun.song@linux.dev,
akpm@linux-foundation.org, yosryahmed@google.com,
willy@infradead.org, linmiaohe@huawei.com,
wangkefeng.wang@huawei.com, laoar.shao@gmail.com,
yuzhao@google.com, wuyun.abel@bytedance.com, david@redhat.com,
peterx@redhat.com, vishal.moola@gmail.com, hughd@google.com,
cgroups@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
jianlv@ebay.com
Subject: Re: [PATCH] memcg: add interface to force disable swap
Date: Sun, 8 Oct 2023 15:52:04 +0800 [thread overview]
Message-ID: <CAFA-uR9ymPTktMbi96cb+smjQHB4Y=8SQfAqmsqTbniGbkGTLA@mail.gmail.com> (raw)
In-Reply-To: <87mswtkj8x.fsf@yhuang6-desk2.ccr.corp.intel.com>
On Sun, Oct 8, 2023 at 9:17 AM Huang, Ying <ying.huang@intel.com> wrote:
>
> Jianlin Lv <iecedge@gmail.com> writes:
>
> > From: Jianlin Lv <iecedge@gmail.com>
> >
> > Global reclaim will swap even if swappiness is set to 0.
>
> Why? Can you elaborate the situation?
We reproduced the issue of pages being swapped out even when swappiness is
set to 0 in the production environment through the following test program.
Not sure whether this program can reproduce the issue in any environment.
From the implementation of the get_scan_count code, it can be seen that,
based on the current runtime situation, memory reclamation will choose a
scanning method (SCAN_ANON/SCAN_FILE/SCAN_FRACT) to determine how
aggressively the anon and file LRU are scanned. However, this introduces
uncertainty.
For the JVM issue at hand, we expect deterministic SCAN_FILE scan to avoid
swapping out anon pages.
code::
#!/usr/bin/env python
import mmap
import os
import sys
def write_files():
count = 1
if not os.path.isdir(WRITE_DIR):
os.mkdir(WRITE_DIR)
while True:
_, i = divmod(count, 6000)
file = "{}/{}_{}.txt".format(WRITE_DIR, WRITE_FILE, i)
with open(file, 'w') as f:
# Write 100 MB to a file
num_chars = 100 * 1024 * 1024
f.write('0' * num_chars)
count = count + 1
def create_read_file():
with open(READ_FILE, 'wb') as f:
num_chars = 10000 * 1024 * 1024
f.write(b'0' * num_chars)
def read_file():
with open(READ_FILE, mode="r", encoding="utf8") as f:
mm = mmap.mmap(f.fileno(), length=0, access=mmap.ACCESS_READ)
text = mm.read()
write_files()
WRITE_FILE = "file"
WRITE_DIR = "/tmp/rm_rf_me"
READ_FILE="/tmp/10g_file_delete"
if not os.path.isfile(READ_FILE):
create_read_file()
read_file()
Jianlin
>
> > In particular
> > case, users wish to be able to completely disable swap for specific
> > processes. One scenario is that if JVM memory pages falls into swap,
> > the performance will noticeably reduce and the GC pauses tend to increase
> > to levels not tolerable by most applications.
> > If it's possible to only disable swap out for specific processes, it can
> > address the JVM GC pauses issues, and at the same time, memory reclaim
> > pressure is also manageable.
> >
> > This patch adds "memory.swap_force_disable" control file to support disable
> > swap for non-root cgroup. When process is associated with a cgroup,
> > 'echo 1 > memory.swap_force_disable' will forbid anon pages be swapped out.
> > This patch also adds read and write handler of the control file.
>
> --
> Best Regards,
> Huang, Ying
next prev parent reply other threads:[~2023-10-08 7:52 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-07 13:09 [PATCH] memcg: add interface to force disable swap Jianlin Lv
2023-10-08 1:14 ` Huang, Ying
2023-10-08 7:52 ` Jianlin Lv [this message]
2023-10-08 8:24 ` Huang, Ying
2023-10-08 9:34 ` Jianlin Lv
2023-10-09 5:58 ` Huang, Ying
2023-10-09 7:34 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFA-uR9ymPTktMbi96cb+smjQHB4Y=8SQfAqmsqTbniGbkGTLA@mail.gmail.com' \
--to=iecedge@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=david@redhat.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=jianlv@ebay.com \
--cc=laoar.shao@gmail.com \
--cc=linmiaohe@huawei.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lizefan.x@bytedance.com \
--cc=mhocko@kernel.org \
--cc=muchun.song@linux.dev \
--cc=peterx@redhat.com \
--cc=roman.gushchin@linux.dev \
--cc=shakeelb@google.com \
--cc=tj@kernel.org \
--cc=vishal.moola@gmail.com \
--cc=wangkefeng.wang@huawei.com \
--cc=willy@infradead.org \
--cc=wuyun.abel@bytedance.com \
--cc=ying.huang@intel.com \
--cc=yosryahmed@google.com \
--cc=yuzhao@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).