All of lore.kernel.org
 help / color / mirror / Atom feed
From: Shakeel Butt <shakeelb@google.com>
To: yulei zhang <yulei.kernel@gmail.com>
Cc: Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
	 Johannes Weiner <hannes@cmpxchg.org>,
	Christian Brauner <christian@brauner.io>,
	 Cgroups <cgroups@vger.kernel.org>,
	benbjiang@tencent.com,  Wanpeng Li <kernellwp@gmail.com>,
	Yulei Zhang <yuleixzhang@tencent.com>,
	 Linux MM <linux-mm@kvack.org>, Michal Hocko <mhocko@kernel.org>,
	Roman Gushchin <guro@fb.com>
Subject: Re: [RFC 0/7] Introduce memory allocation speed throttle in memcg
Date: Mon, 31 May 2021 11:20:46 -0700	[thread overview]
Message-ID: <CALvZod4FUEXuqs8mKiN-WdmF2eCi29=aRG-cOaUQEDCbHdXQRA@mail.gmail.com> (raw)
In-Reply-To: <CACZOiM3VhYyzCTx4FbW=FF8WB=X46xaV53abqOVL+eHQOs8Reg@mail.gmail.com>

On Mon, May 31, 2021 at 5:11 AM yulei zhang <yulei.kernel@gmail.com> wrote:
>
[...]
> > Can you please explain why memory.high is not good enough for your
> > use-case? You can orchestrate the memory.high limits in such a way
> > that those certain cgroups hit their memory.high limit before causing
> > the global reclaim. You might need to dynamically adjust the limits
> > based on other workloads or unaccounted memory.
> >
>
> Yep, dynamically adjust the memory.high limits can ease the memory pressure
> and postpone the global reclaim, but it can easily trigger the oom in
> the cgroups,

Can you please elaborate a bit more on this? The memory.high has a
strong throttling mechanism, so if you are observing memory.high being
ineffective then we need to fix that. Also can you please explain a
bit more on the specific of the workload which is able to escape
memory.high throttling e.g. the normal number of processes/threads in
the workload?


WARNING: multiple messages have this Message-ID (diff)
From: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
To: yulei zhang <yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	Zefan Li <lizefan.x-EC8Uxl6Npydl57MIdRCFDg@public.gmane.org>,
	Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
	Christian Brauner
	<christian-STijNZzMWpgWenYVfaLwtA@public.gmane.org>,
	Cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	benbjiang-1Nz4purKYjRBDgjK7y7TUQ@public.gmane.org,
	Wanpeng Li <kernellwp-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Yulei Zhang <yuleixzhang-1Nz4purKYjRBDgjK7y7TUQ@public.gmane.org>,
	Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>,
	Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>
Subject: Re: [RFC 0/7] Introduce memory allocation speed throttle in memcg
Date: Mon, 31 May 2021 11:20:46 -0700	[thread overview]
Message-ID: <CALvZod4FUEXuqs8mKiN-WdmF2eCi29=aRG-cOaUQEDCbHdXQRA@mail.gmail.com> (raw)
In-Reply-To: <CACZOiM3VhYyzCTx4FbW=FF8WB=X46xaV53abqOVL+eHQOs8Reg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On Mon, May 31, 2021 at 5:11 AM yulei zhang <yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
[...]
> > Can you please explain why memory.high is not good enough for your
> > use-case? You can orchestrate the memory.high limits in such a way
> > that those certain cgroups hit their memory.high limit before causing
> > the global reclaim. You might need to dynamically adjust the limits
> > based on other workloads or unaccounted memory.
> >
>
> Yep, dynamically adjust the memory.high limits can ease the memory pressure
> and postpone the global reclaim, but it can easily trigger the oom in
> the cgroups,

Can you please elaborate a bit more on this? The memory.high has a
strong throttling mechanism, so if you are observing memory.high being
ineffective then we need to fix that. Also can you please explain a
bit more on the specific of the workload which is able to escape
memory.high throttling e.g. the normal number of processes/threads in
the workload?

  reply	other threads:[~2021-05-31 18:21 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-26 16:17 [RFC 0/7] Introduce memory allocation speed throttle in memcg yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
     [not found] ` <cover.1622043596.git.yuleixzhang-1Nz4purKYjRBDgjK7y7TUQ@public.gmane.org>
2021-05-26 16:17   ` [RFC 1/7] mm: record total charge and max speed counter " yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 16:17   ` [RFC 2/7] mm: introduce alloc_bps to memcg for memory allocation speed throttle yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 16:18   ` [RFC 3/7] mm: memory allocation speed throttle setup in hierarchy yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 16:18   ` [RFC 4/7] mm: introduce slice analysis into memory speed throttle mechanism yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 16:18   ` [RFC 5/7] mm: introduce memory allocation speed throttle yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 16:18   ` [RFC 6/7] mm: record the numbers of memory allocation throttle yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 19:57     ` kernel test robot
2021-05-26 21:34     ` kernel test robot
2021-05-26 16:18   ` [RFC 7/7] mm: introduce mst low and min watermark yulei.kernel-Re5JQEeQqe8AvxtiuMwx3w
2021-05-26 20:52 ` [RFC 0/7] Introduce memory allocation speed throttle in memcg Shakeel Butt
2021-05-26 20:52   ` Shakeel Butt
2021-05-31 12:11   ` yulei zhang
2021-05-31 12:11     ` yulei zhang
2021-05-31 18:20     ` Shakeel Butt [this message]
2021-05-31 18:20       ` Shakeel Butt
2021-06-01 14:45     ` Chris Down
2021-06-01 14:45       ` Chris Down
2021-06-02  9:11       ` yulei zhang
2021-06-02  9:11         ` yulei zhang
2021-06-02 15:39         ` Shakeel Butt
2021-06-02 15:39           ` Shakeel Butt
2021-06-03 10:19           ` yulei zhang
2021-06-03 10:19             ` yulei zhang
2021-06-03 11:38             ` Chris Down
2021-06-03 11:38               ` Chris Down
2021-06-04 10:15               ` yulei zhang
2021-06-04 10:15                 ` yulei zhang
2021-06-04 11:51                 ` Chris Down
2021-06-04 11:51                   ` Chris Down

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALvZod4FUEXuqs8mKiN-WdmF2eCi29=aRG-cOaUQEDCbHdXQRA@mail.gmail.com' \
    --to=shakeelb@google.com \
    --cc=benbjiang@tencent.com \
    --cc=cgroups@vger.kernel.org \
    --cc=christian@brauner.io \
    --cc=guro@fb.com \
    --cc=hannes@cmpxchg.org \
    --cc=kernellwp@gmail.com \
    --cc=linux-mm@kvack.org \
    --cc=lizefan.x@bytedance.com \
    --cc=mhocko@kernel.org \
    --cc=tj@kernel.org \
    --cc=yulei.kernel@gmail.com \
    --cc=yuleixzhang@tencent.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.