From: Michal Hocko <mhocko@kernel.org> To: Tejun Heo <tj@kernel.org> Cc: Shakeel Butt <shakeelb@google.com>, Jakub Kicinski <kuba@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Linux MM <linux-mm@kvack.org>, Kernel Team <kernel-team@fb.com>, Johannes Weiner <hannes@cmpxchg.org>, Chris Down <chris@chrisdown.name>, Cgroups <cgroups@vger.kernel.org> Subject: Re: [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted Date: Tue, 21 Apr 2020 13:06:12 +0200 [thread overview] Message-ID: <20200421110612.GD27314@dhcp22.suse.cz> (raw) In-Reply-To: <20200420170650.GA169746@mtj.thefacebook.com> On Mon 20-04-20 13:06:50, Tejun Heo wrote: > Hello, > > On Mon, Apr 20, 2020 at 07:03:18PM +0200, Michal Hocko wrote: > > I have asked about the semantic of this know already and didn't really > > get any real answer. So how does swap.high fit into high limit semantic > > when it doesn't act as a limit. Considering that we cannot reclaim swap > > space I find this really hard to grasp. > > memory.high slow down is for the case when memory reclaim can't be depended > upon for throttling, right? This is the same. Swap can't be reclaimed so the > backpressure is applied by slowing down the source, the same way memory.high > does. Hmm, but the two differ quite considerably that we do not reclaim any swap which means that while no reclaimable memory at all is pretty much the corner case (essentially OOM) the no reclaimable swap is always in that state. So whenever you hit the high limit there is no other way then rely on userspace to unmap swap backed memory or increase the limit. Without that there is always throttling. The question also is what do you want to throttle in that case? Any swap backed allocation or swap based reclaim? The patch throttles any allocations unless I am misreading. This means that also any other !swap backed allocations get throttled as soon as the swap quota is reached. Is this really desirable behavior? I would find it quite surprising to say the least. I am also not sure about the isolation aspect. Because an external memory pressure might have pushed out memory to the swap and then the workload is throttled based on an external event. Compare that to the memory.high throttling which is not directly affected by the external pressure. There is also an aspect of non-determinism. There is no control over the file vs. swap backed reclaim decision for memcgs. That means that behavior is going to be very dependent on the internal implementation of the reclaim. More swapping is going to fill up swap quota quicker. > It fits together with memory.low in that it prevents runaway anon allocation > when swap can't be allocated anymore. It's addressing the same problem that > memory.high slowdown does. It's just a different vector. I suspect that the problem is more related to the swap being handled as a separate resource. And it is still not clear to me why it is easier for you to tune swap.high than memory.high. You have said that you do not want to set up memory.high because it is harder to tune but I do not see why swap is easier in this regards. Maybe it is just that the swap is almost never used so a bad estimate is much easier to tolerate and you really do care about runaways? -- Michal Hocko SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> To: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> Cc: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Jakub Kicinski <kuba-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>, Kernel Team <kernel-team-b10kYP2dOMg@public.gmane.org>, Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>, Chris Down <chris-6Bi1550iOqEnzZ6mRAm98g@public.gmane.org>, Cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> Subject: Re: [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted Date: Tue, 21 Apr 2020 13:06:12 +0200 [thread overview] Message-ID: <20200421110612.GD27314@dhcp22.suse.cz> (raw) In-Reply-To: <20200420170650.GA169746-146+VewaZzwNjtGbbfXrCEEOCMrvLtNR@public.gmane.org> On Mon 20-04-20 13:06:50, Tejun Heo wrote: > Hello, > > On Mon, Apr 20, 2020 at 07:03:18PM +0200, Michal Hocko wrote: > > I have asked about the semantic of this know already and didn't really > > get any real answer. So how does swap.high fit into high limit semantic > > when it doesn't act as a limit. Considering that we cannot reclaim swap > > space I find this really hard to grasp. > > memory.high slow down is for the case when memory reclaim can't be depended > upon for throttling, right? This is the same. Swap can't be reclaimed so the > backpressure is applied by slowing down the source, the same way memory.high > does. Hmm, but the two differ quite considerably that we do not reclaim any swap which means that while no reclaimable memory at all is pretty much the corner case (essentially OOM) the no reclaimable swap is always in that state. So whenever you hit the high limit there is no other way then rely on userspace to unmap swap backed memory or increase the limit. Without that there is always throttling. The question also is what do you want to throttle in that case? Any swap backed allocation or swap based reclaim? The patch throttles any allocations unless I am misreading. This means that also any other !swap backed allocations get throttled as soon as the swap quota is reached. Is this really desirable behavior? I would find it quite surprising to say the least. I am also not sure about the isolation aspect. Because an external memory pressure might have pushed out memory to the swap and then the workload is throttled based on an external event. Compare that to the memory.high throttling which is not directly affected by the external pressure. There is also an aspect of non-determinism. There is no control over the file vs. swap backed reclaim decision for memcgs. That means that behavior is going to be very dependent on the internal implementation of the reclaim. More swapping is going to fill up swap quota quicker. > It fits together with memory.low in that it prevents runaway anon allocation > when swap can't be allocated anymore. It's addressing the same problem that > memory.high slowdown does. It's just a different vector. I suspect that the problem is more related to the swap being handled as a separate resource. And it is still not clear to me why it is easier for you to tune swap.high than memory.high. You have said that you do not want to set up memory.high because it is harder to tune but I do not see why swap is easier in this regards. Maybe it is just that the swap is almost never used so a bad estimate is much easier to tolerate and you really do care about runaways? -- Michal Hocko SUSE Labs
next prev parent reply other threads:[~2020-04-21 11:06 UTC|newest] Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-04-17 1:06 [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted Jakub Kicinski 2020-04-17 1:06 ` Jakub Kicinski 2020-04-17 1:06 ` [PATCH 1/3] mm: prepare for swap over-high accounting and penalty calculation Jakub Kicinski 2020-04-17 1:06 ` Jakub Kicinski 2020-04-17 1:06 ` [PATCH 2/3] mm: move penalty delay clamping out of calculate_high_delay() Jakub Kicinski 2020-04-17 1:06 ` Jakub Kicinski 2020-04-17 1:06 ` [PATCH 3/3] mm: automatically penalize tasks with high swap use Jakub Kicinski 2020-04-17 1:06 ` Jakub Kicinski 2020-04-17 7:37 ` Michal Hocko 2020-04-17 7:37 ` Michal Hocko 2020-04-17 23:22 ` Jakub Kicinski 2020-04-17 23:22 ` Jakub Kicinski 2020-04-17 16:11 ` [PATCH 0/3] memcg: Slow down swap allocation as the available space gets depleted Shakeel Butt 2020-04-17 16:11 ` Shakeel Butt 2020-04-17 16:23 ` Tejun Heo 2020-04-17 16:23 ` Tejun Heo 2020-04-17 17:18 ` Shakeel Butt 2020-04-17 17:18 ` Shakeel Butt 2020-04-17 17:36 ` Tejun Heo 2020-04-17 17:36 ` Tejun Heo 2020-04-17 17:51 ` Shakeel Butt 2020-04-17 17:51 ` Shakeel Butt 2020-04-17 19:35 ` Tejun Heo 2020-04-17 19:35 ` Tejun Heo 2020-04-17 21:51 ` Shakeel Butt 2020-04-17 21:51 ` Shakeel Butt 2020-04-17 22:59 ` Tejun Heo 2020-04-17 22:59 ` Tejun Heo 2020-04-20 16:12 ` Shakeel Butt 2020-04-20 16:12 ` Shakeel Butt 2020-04-20 16:47 ` Tejun Heo 2020-04-20 16:47 ` Tejun Heo 2020-04-20 17:03 ` Michal Hocko 2020-04-20 17:03 ` Michal Hocko 2020-04-20 17:06 ` Tejun Heo 2020-04-20 17:06 ` Tejun Heo 2020-04-21 11:06 ` Michal Hocko [this message] 2020-04-21 11:06 ` Michal Hocko 2020-04-21 14:27 ` Johannes Weiner 2020-04-21 14:27 ` Johannes Weiner 2020-04-21 16:11 ` Michal Hocko 2020-04-21 16:11 ` Michal Hocko 2020-04-21 16:56 ` Johannes Weiner 2020-04-21 16:56 ` Johannes Weiner 2020-04-22 13:26 ` Michal Hocko 2020-04-22 13:26 ` Michal Hocko 2020-04-22 14:15 ` Johannes Weiner 2020-04-22 14:15 ` Johannes Weiner 2020-04-22 15:43 ` Michal Hocko 2020-04-22 15:43 ` Michal Hocko 2020-04-22 17:13 ` Johannes Weiner 2020-04-22 17:13 ` Johannes Weiner 2020-04-22 18:49 ` Michal Hocko 2020-04-22 18:49 ` Michal Hocko 2020-04-23 15:00 ` Johannes Weiner 2020-04-23 15:00 ` Johannes Weiner 2020-04-24 15:05 ` Michal Hocko 2020-04-24 15:05 ` Michal Hocko 2020-04-28 14:24 ` Johannes Weiner 2020-04-28 14:24 ` Johannes Weiner 2020-04-29 9:55 ` Michal Hocko 2020-04-29 9:55 ` Michal Hocko 2020-04-21 19:09 ` Shakeel Butt 2020-04-21 19:09 ` Shakeel Butt 2020-04-21 21:59 ` Johannes Weiner 2020-04-21 21:59 ` Johannes Weiner 2020-04-21 22:39 ` Shakeel Butt 2020-04-21 22:39 ` Shakeel Butt 2020-04-21 15:20 ` Tejun Heo 2020-04-21 15:20 ` Tejun Heo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200421110612.GD27314@dhcp22.suse.cz \ --to=mhocko@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=cgroups@vger.kernel.org \ --cc=chris@chrisdown.name \ --cc=hannes@cmpxchg.org \ --cc=kernel-team@fb.com \ --cc=kuba@kernel.org \ --cc=linux-mm@kvack.org \ --cc=shakeelb@google.com \ --cc=tj@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.