[PATCH 0/2] memcg oom: don't try to kill a process if there is no process

[PATCH 0/2] memcg oom: don't try to kill a process if there is no process

[Yafang Shao]

Recently Shakeel reported a issue which also confused me serveral months
earlier that it still try to kill a process even if there is no process.
If a memcg is not populated, it is useless to try to kill a process.

mem_cgroup_out_of_memory() is renamed to mem_cgroup_oom_kill() for better
understanding.

Yafang Shao (2):
  mm, memcg: better name mem_cgroup_out_of_memory()
  mm, memcg: don't try to kill a process if memcg is not populated

 mm/memcontrol.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

-- 
2.18.2

[PATCH 1/2] mm, memcg: better name mem_cgroup_out_of_memory()

[Yafang Shao]

Rename mem_cgroup_out_of_memory() to mem_cgroup_oom_kill() to indicate
that this function is used to try to kill a process.
With this change it will cooperate better with the oom events.
	function		memcg event
	mem_cgroup_oom()	oom
	mem_cgroup_oom_kill()	oom_kill

Cc: Shakeel Butt <shakeelb@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Roman Gushchin <guro@fb.com>
Cc: Greg Thelen <gthelen@google.com>
Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
---
 mm/memcontrol.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 5beea03dd58a..985edce98491 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1570,7 +1570,7 @@ unsigned long mem_cgroup_size(struct mem_cgroup *memcg)
 	return page_counter_read(&memcg->memory);
 }
 
-static bool mem_cgroup_out_of_memory(struct mem_cgroup *memcg, gfp_t gfp_mask,
+static bool mem_cgroup_oom_kill(struct mem_cgroup *memcg, gfp_t gfp_mask,
 				     int order)
 {
 	struct oom_control oc = {
@@ -1799,7 +1799,7 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
 	 * relying on the oom victim to make a forward progress and we can
 	 * invoke the oom killer here.
 	 *
-	 * Please note that mem_cgroup_out_of_memory might fail to find a
+	 * Please note that mem_cgroup_oom_kill might fail to find a
 	 * victim and then we have to bail out from the charge path.
 	 */
 	if (memcg->oom_kill_disable) {
@@ -1821,7 +1821,7 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
 		mem_cgroup_oom_notify(memcg);
 
 	mem_cgroup_unmark_under_oom(memcg);
-	if (mem_cgroup_out_of_memory(memcg, mask, order))
+	if (mem_cgroup_oom_kill(memcg, mask, order))
 		ret = OOM_SUCCESS;
 	else
 		ret = OOM_FAILED;
@@ -1879,7 +1879,7 @@ bool mem_cgroup_oom_synchronize(bool handle)
 	if (locked && !memcg->oom_kill_disable) {
 		mem_cgroup_unmark_under_oom(memcg);
 		finish_wait(&memcg_oom_waitq, &owait.wait);
-		mem_cgroup_out_of_memory(memcg, current->memcg_oom_gfp_mask,
+		mem_cgroup_oom_kill(memcg, current->memcg_oom_gfp_mask,
 					 current->memcg_oom_order);
 	} else {
 		schedule();
@@ -6102,7 +6102,7 @@ static ssize_t memory_max_write(struct kernfs_open_file *of,
 		}
 
 		memcg_memory_event(memcg, MEMCG_OOM);
-		if (!mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0))
+		if (!mem_cgroup_oom_kill(memcg, GFP_KERNEL, 0))
 			break;
 	}
 
-- 
2.18.2

[PATCH 2/2] mm, memcg: don't try to kill a process if memcg is not populated

[Yafang Shao]

Recently Shakeel reported a issue which also confused me serveral months
earlier. Bellow is his report -

Lowering memory.max can trigger an oom-kill if the reclaim does not
succeed. However if oom-killer does not find a process for killing, it
dumps a lot of warnings.

Deleting a memcg does not reclaim memory from it and the memory can
linger till there is a memory pressure. One normal way to proactively
reclaim such memory is to set memory.max to 0 just before deleting the
memcg. However if some of the memcg's memory is pinned by others, this
operation can trigger an oom-kill without any process and thus can log a
lot un-needed warnings. So, ignore all such warnings from memory.max.

[shakeelb@google.com: commit log above]

A better way to avoid this issue is to avoid trying to kill a process if
memcg is not populated.
Note that OOM is different with OOM kill. OOM is a status that the
system or memcg is out of memory, while OOM kill is a result that a
process inside this memcg is killed when this memcg is in OOM status.
That is the same reason why there're both MEMCG_OOM event and
MEMCG_OOM_KILL event. If we have already known that there's nothing to
kill, i.e. the memcg is not populated, then we don't need to have a try.

Cc: Shakeel Butt <shakeelb@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Roman Gushchin <guro@fb.com>
Cc: Greg Thelen <gthelen@google.com>
Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
---
 mm/memcontrol.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 985edce98491..29afe3df9d98 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6102,6 +6102,10 @@ static ssize_t memory_max_write(struct kernfs_open_file *of,
 		}
 
 		memcg_memory_event(memcg, MEMCG_OOM);
+
+		if (!cgroup_is_populated(memcg->css.cgroup))
+			break;
+
 		if (!mem_cgroup_oom_kill(memcg, GFP_KERNEL, 0))
 			break;
 	}
-- 
2.18.2

Re: [PATCH 1/2] mm, memcg: better name mem_cgroup_out_of_memory()

[Shakeel Butt]

On Sat, May 2, 2020 at 7:35 AM Yafang Shao <laoar.shao@gmail.com> wrote:
>
> Rename mem_cgroup_out_of_memory() to mem_cgroup_oom_kill() to indicate
> that this function is used to try to kill a process.
> With this change it will cooperate better with the oom events.
>         function                memcg event
>         mem_cgroup_oom()        oom
>         mem_cgroup_oom_kill()   oom_kill
>
> Cc: Shakeel Butt <shakeelb@google.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Michal Hocko <mhocko@kernel.org>
> Cc: Roman Gushchin <guro@fb.com>
> Cc: Greg Thelen <gthelen@google.com>
> Signed-off-by: Yafang Shao <laoar.shao@gmail.com>

Reviewed-by: Shakeel Butt <shakeelb@google.com>

> ---
>  mm/memcontrol.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 5beea03dd58a..985edce98491 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1570,7 +1570,7 @@ unsigned long mem_cgroup_size(struct mem_cgroup *memcg)
>         return page_counter_read(&memcg->memory);
>  }
>
> -static bool mem_cgroup_out_of_memory(struct mem_cgroup *memcg, gfp_t gfp_mask,
> +static bool mem_cgroup_oom_kill(struct mem_cgroup *memcg, gfp_t gfp_mask,
>                                      int order)
>  {
>         struct oom_control oc = {
> @@ -1799,7 +1799,7 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
>          * relying on the oom victim to make a forward progress and we can
>          * invoke the oom killer here.
>          *
> -        * Please note that mem_cgroup_out_of_memory might fail to find a
> +        * Please note that mem_cgroup_oom_kill might fail to find a
>          * victim and then we have to bail out from the charge path.
>          */
>         if (memcg->oom_kill_disable) {
> @@ -1821,7 +1821,7 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
>                 mem_cgroup_oom_notify(memcg);
>
>         mem_cgroup_unmark_under_oom(memcg);
> -       if (mem_cgroup_out_of_memory(memcg, mask, order))
> +       if (mem_cgroup_oom_kill(memcg, mask, order))
>                 ret = OOM_SUCCESS;
>         else
>                 ret = OOM_FAILED;
> @@ -1879,7 +1879,7 @@ bool mem_cgroup_oom_synchronize(bool handle)
>         if (locked && !memcg->oom_kill_disable) {
>                 mem_cgroup_unmark_under_oom(memcg);
>                 finish_wait(&memcg_oom_waitq, &owait.wait);
> -               mem_cgroup_out_of_memory(memcg, current->memcg_oom_gfp_mask,
> +               mem_cgroup_oom_kill(memcg, current->memcg_oom_gfp_mask,
>                                          current->memcg_oom_order);
>         } else {
>                 schedule();
> @@ -6102,7 +6102,7 @@ static ssize_t memory_max_write(struct kernfs_open_file *of,
>                 }
>
>                 memcg_memory_event(memcg, MEMCG_OOM);
> -               if (!mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0))
> +               if (!mem_cgroup_oom_kill(memcg, GFP_KERNEL, 0))
>                         break;
>         }
>
> --
> 2.18.2
>

Re: [PATCH 2/2] mm, memcg: don't try to kill a process if memcg is not populated

[Shakeel Butt]

On Sat, May 2, 2020 at 7:35 AM Yafang Shao <laoar.shao@gmail.com> wrote:
>
> Recently Shakeel reported a issue which also confused me serveral months
> earlier. Bellow is his report -
>
> Lowering memory.max can trigger an oom-kill if the reclaim does not
> succeed. However if oom-killer does not find a process for killing, it
> dumps a lot of warnings.
>
> Deleting a memcg does not reclaim memory from it and the memory can
> linger till there is a memory pressure. One normal way to proactively
> reclaim such memory is to set memory.max to 0 just before deleting the
> memcg. However if some of the memcg's memory is pinned by others, this
> operation can trigger an oom-kill without any process and thus can log a
> lot un-needed warnings. So, ignore all such warnings from memory.max.

lot *of* un-needed

>
> [shakeelb@google.com: commit log above]
>
> A better way to avoid this issue is to avoid trying to kill a process if
> memcg is not populated.
> Note that OOM is different with OOM kill.

different *from*

> OOM is a status that the
> system or memcg is out of memory, while OOM kill is a result that a
> process inside this memcg is killed when this memcg is in OOM status.
> That is the same reason why there're both MEMCG_OOM event and
> MEMCG_OOM_KILL event. If we have already known that there's nothing to
> kill, i.e. the memcg is not populated, then we don't need to have a try.

need to try

I think adding the discussion of memory.high is also useful in the
commit message. Basically why setting memory.max to 0 is better than
setting memory.high to 0 before deletion. The reason is remote
charging.

>
> Cc: Shakeel Butt <shakeelb@google.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Michal Hocko <mhocko@kernel.org>
> Cc: Roman Gushchin <guro@fb.com>
> Cc: Greg Thelen <gthelen@google.com>
> Signed-off-by: Yafang Shao <laoar.shao@gmail.com>

Reviewed-by: Shakeel Butt <shakeelb@google.com>

> ---
>  mm/memcontrol.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 985edce98491..29afe3df9d98 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -6102,6 +6102,10 @@ static ssize_t memory_max_write(struct kernfs_open_file *of,
>                 }
>
>                 memcg_memory_event(memcg, MEMCG_OOM);
> +
> +               if (!cgroup_is_populated(memcg->css.cgroup))
> +                       break;
> +
>                 if (!mem_cgroup_oom_kill(memcg, GFP_KERNEL, 0))
>                         break;
>         }
> --
> 2.18.2
>

Re: [PATCH 2/2] mm, memcg: don't try to kill a process if memcg is not populated

[Yafang Shao]

On Mon, May 4, 2020 at 7:00 AM Shakeel Butt <shakeelb@google.com> wrote:
>
> On Sat, May 2, 2020 at 7:35 AM Yafang Shao <laoar.shao@gmail.com> wrote:
> >
> > Recently Shakeel reported a issue which also confused me serveral months
> > earlier. Bellow is his report -
> >
> > Lowering memory.max can trigger an oom-kill if the reclaim does not
> > succeed. However if oom-killer does not find a process for killing, it
> > dumps a lot of warnings.
> >
> > Deleting a memcg does not reclaim memory from it and the memory can
> > linger till there is a memory pressure. One normal way to proactively
> > reclaim such memory is to set memory.max to 0 just before deleting the
> > memcg. However if some of the memcg's memory is pinned by others, this
> > operation can trigger an oom-kill without any process and thus can log a
> > lot un-needed warnings. So, ignore all such warnings from memory.max.
>
> lot *of* un-needed
>

Thanks.

> >
> > [shakeelb@google.com: commit log above]
> >
> > A better way to avoid this issue is to avoid trying to kill a process if
> > memcg is not populated.
> > Note that OOM is different with OOM kill.
>
> different *from*
>

Thanks

> > OOM is a status that the
> > system or memcg is out of memory, while OOM kill is a result that a
> > process inside this memcg is killed when this memcg is in OOM status.
> > That is the same reason why there're both MEMCG_OOM event and
> > MEMCG_OOM_KILL event. If we have already known that there's nothing to
> > kill, i.e. the memcg is not populated, then we don't need to have a try.
>
> need to try
>

Thanks

> I think adding the discussion of memory.high is also useful in the
> commit message. Basically why setting memory.max to 0 is better than
> setting memory.high to 0 before deletion. The reason is remote
> charging.
>

Sure, will add it.

> >
> > Cc: Shakeel Butt <shakeelb@google.com>
> > Cc: Johannes Weiner <hannes@cmpxchg.org>
> > Cc: Michal Hocko <mhocko@kernel.org>
> > Cc: Roman Gushchin <guro@fb.com>
> > Cc: Greg Thelen <gthelen@google.com>
> > Signed-off-by: Yafang Shao <laoar.shao@gmail.com>
>
> Reviewed-by: Shakeel Butt <shakeelb@google.com>
>
> > ---
> >  mm/memcontrol.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> > index 985edce98491..29afe3df9d98 100644
> > --- a/mm/memcontrol.c
> > +++ b/mm/memcontrol.c
> > @@ -6102,6 +6102,10 @@ static ssize_t memory_max_write(struct kernfs_open_file *of,
> >                 }
> >
> >                 memcg_memory_event(memcg, MEMCG_OOM);
> > +
> > +               if (!cgroup_is_populated(memcg->css.cgroup))
> > +                       break;
> > +
> >                 if (!mem_cgroup_oom_kill(memcg, GFP_KERNEL, 0))
> >                         break;
> >         }
> > --
> > 2.18.2
> >



-- 
Thanks
Yafang