All of lore.kernel.org
 help / color / mirror / Atom feed
* OOM notification for cgroupsv1 broken in 4.19
@ 2018-12-21 14:49 Burt Holzman
  2018-12-21 15:33   ` Michal Hocko
  2018-12-24  9:11   ` Michal Hocko
  0 siblings, 2 replies; 7+ messages in thread
From: Burt Holzman @ 2018-12-21 14:49 UTC (permalink / raw)
  To: hannes, vdavydov.dev, cgroups, linux-mm; +Cc: mhocko

Hi,

This patch: 29ef680ae7c21110af8e6416d84d8a72fc147b14
[PATCH] memcg, oom: move out_of_memory back to the charge path

has broken the eventfd notification for cgroups-v1. This is because 
mem_cgroup_oom_notify() is called only in mem_cgroup_oom_synchronize and 
not with the new, additional call to mem_cgroup_out_of_memory in the 
charge path.

- B

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: OOM notification for cgroupsv1 broken in 4.19
@ 2018-12-21 15:33   ` Michal Hocko
  0 siblings, 0 replies; 7+ messages in thread
From: Michal Hocko @ 2018-12-21 15:33 UTC (permalink / raw)
  To: Burt Holzman; +Cc: hannes, vdavydov.dev, cgroups, linux-mm

On Fri 21-12-18 14:49:38, Burt Holzman wrote:
> Hi,
> 
> This patch: 29ef680ae7c21110af8e6416d84d8a72fc147b14
> [PATCH] memcg, oom: move out_of_memory back to the charge path
> 
> has broken the eventfd notification for cgroups-v1. This is because 
> mem_cgroup_oom_notify() is called only in mem_cgroup_oom_synchronize and 
> not with the new, additional call to mem_cgroup_out_of_memory in the 
> charge path.

Yes, you are right and this is a clear regression. Does the following
patch fixes the issue for you? I am not super happy about the code
duplication but I wasn't able to separate this out from
mem_cgroup_oom_synchronize because that one has to handle the oom_killer
disabled case which is not the case in the charge path because we simply
back off and hand over to mem_cgroup_oom_synchronize in that case.
---
>From 51633f683173013741f4d0ab3e31bae575341c55 Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@suse.com>
Date: Fri, 21 Dec 2018 16:28:29 +0100
Subject: [PATCH] memcg, oom: notify on oom killer invocation from the charge
 path

Burt Holzman has noticed that memcg v1 doesn't notify about OOM events
via eventfd anymore. The reason is that 29ef680ae7c2 ("memcg, oom: move
out_of_memory back to the charge path") has moved the oom handling back
to the charge path. While doing so the notification was left behind in
mem_cgroup_oom_synchronize.

Fix the issue by replicating the oom hierarchy locking and the
notification.

Reported-by: Burt Holzman <burt@fnal.gov>
Fixes: 29ef680ae7c2 ("memcg, oom: move out_of_memory back to the charge path")
Cc: stable # 4.19+
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/memcontrol.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6e1469b80cb7..7e6bf74ddb1e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1666,6 +1666,9 @@ enum oom_status {
 
 static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
 {
+	enum oom_status ret;
+	bool locked;
+
 	if (order > PAGE_ALLOC_COSTLY_ORDER)
 		return OOM_SKIPPED;
 
@@ -1700,10 +1703,23 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
 		return OOM_ASYNC;
 	}
 
+	mem_cgroup_mark_under_oom(memcg);
+
+	locked = mem_cgroup_oom_trylock(memcg);
+
+	if (locked)
+		mem_cgroup_oom_notify(memcg);
+
+	mem_cgroup_unmark_under_oom(memcg);
 	if (mem_cgroup_out_of_memory(memcg, mask, order))
-		return OOM_SUCCESS;
+		ret = OOM_SUCCESS;
+	else
+		ret = OOM_FAILED;
 
-	return OOM_FAILED;
+	if (locked)
+		mem_cgroup_oom_unlock(memcg);
+
+	return ret;
 }
 
 /**
-- 
2.19.2

-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: OOM notification for cgroupsv1 broken in 4.19
@ 2018-12-21 15:33   ` Michal Hocko
  0 siblings, 0 replies; 7+ messages in thread
From: Michal Hocko @ 2018-12-21 15:33 UTC (permalink / raw)
  To: Burt Holzman; +Cc: hannes, vdavydov.dev, cgroups, linux-mm

On Fri 21-12-18 14:49:38, Burt Holzman wrote:
> Hi,
> 
> This patch: 29ef680ae7c21110af8e6416d84d8a72fc147b14
> [PATCH] memcg, oom: move out_of_memory back to the charge path
> 
> has broken the eventfd notification for cgroups-v1. This is because 
> mem_cgroup_oom_notify() is called only in mem_cgroup_oom_synchronize and 
> not with the new, additional call to mem_cgroup_out_of_memory in the 
> charge path.

Yes, you are right and this is a clear regression. Does the following
patch fixes the issue for you? I am not super happy about the code
duplication but I wasn't able to separate this out from
mem_cgroup_oom_synchronize because that one has to handle the oom_killer
disabled case which is not the case in the charge path because we simply
back off and hand over to mem_cgroup_oom_synchronize in that case.
---
From 51633f683173013741f4d0ab3e31bae575341c55 Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@suse.com>
Date: Fri, 21 Dec 2018 16:28:29 +0100
Subject: [PATCH] memcg, oom: notify on oom killer invocation from the charge
 path

Burt Holzman has noticed that memcg v1 doesn't notify about OOM events
via eventfd anymore. The reason is that 29ef680ae7c2 ("memcg, oom: move
out_of_memory back to the charge path") has moved the oom handling back
to the charge path. While doing so the notification was left behind in
mem_cgroup_oom_synchronize.

Fix the issue by replicating the oom hierarchy locking and the
notification.

Reported-by: Burt Holzman <burt@fnal.gov>
Fixes: 29ef680ae7c2 ("memcg, oom: move out_of_memory back to the charge path")
Cc: stable # 4.19+
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
 mm/memcontrol.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6e1469b80cb7..7e6bf74ddb1e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1666,6 +1666,9 @@ enum oom_status {
 
 static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
 {
+	enum oom_status ret;
+	bool locked;
+
 	if (order > PAGE_ALLOC_COSTLY_ORDER)
 		return OOM_SKIPPED;
 
@@ -1700,10 +1703,23 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
 		return OOM_ASYNC;
 	}
 
+	mem_cgroup_mark_under_oom(memcg);
+
+	locked = mem_cgroup_oom_trylock(memcg);
+
+	if (locked)
+		mem_cgroup_oom_notify(memcg);
+
+	mem_cgroup_unmark_under_oom(memcg);
 	if (mem_cgroup_out_of_memory(memcg, mask, order))
-		return OOM_SUCCESS;
+		ret = OOM_SUCCESS;
+	else
+		ret = OOM_FAILED;
 
-	return OOM_FAILED;
+	if (locked)
+		mem_cgroup_oom_unlock(memcg);
+
+	return ret;
 }
 
 /**
-- 
2.19.2

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: OOM notification for cgroupsv1 broken in 4.19
  2018-12-21 15:33   ` Michal Hocko
  (?)
@ 2018-12-21 17:33   ` Burt Holzman
  -1 siblings, 0 replies; 7+ messages in thread
From: Burt Holzman @ 2018-12-21 17:33 UTC (permalink / raw)
  To: Michal Hocko; +Cc: hannes, vdavydov.dev, cgroups, linux-mm


> On Dec 21, 2018, at 9:33 AM, Michal Hocko <mhocko@kernel.org> wrote:
> 
> On Fri 21-12-18 14:49:38, Burt Holzman wrote:
>> Hi,
>> 
>> This patch: 29ef680ae7c21110af8e6416d84d8a72fc147b14
>> [PATCH] memcg, oom: move out_of_memory back to the charge path
>> 
>> has broken the eventfd notification for cgroups-v1. This is because 
>> mem_cgroup_oom_notify() is called only in mem_cgroup_oom_synchronize and 
>> not with the new, additional call to mem_cgroup_out_of_memory in the 
>> charge path.
> 
> Yes, you are right and this is a clear regression. Does the following
> patch fixes the issue for you? I am not super happy about the code
> duplication but I wasn't able to separate this out from
> mem_cgroup_oom_synchronize because that one has to handle the oom_killer
> disabled case which is not the case in the charge path because we simply
> back off and hand over to mem_cgroup_oom_synchronize in that case.

Hi Michal,

Thanks for the quick response & patch. I can confirm that with this patch the notification is working for my sample test case.

- B

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: OOM notification for cgroupsv1 broken in 4.19
  2018-12-21 15:33   ` Michal Hocko
  (?)
  (?)
@ 2018-12-21 19:02   ` Johannes Weiner
  -1 siblings, 0 replies; 7+ messages in thread
From: Johannes Weiner @ 2018-12-21 19:02 UTC (permalink / raw)
  To: Michal Hocko; +Cc: Burt Holzman, vdavydov.dev, cgroups, linux-mm

On Fri, Dec 21, 2018 at 04:33:02PM +0100, Michal Hocko wrote:
> From 51633f683173013741f4d0ab3e31bae575341c55 Mon Sep 17 00:00:00 2001
> From: Michal Hocko <mhocko@suse.com>
> Date: Fri, 21 Dec 2018 16:28:29 +0100
> Subject: [PATCH] memcg, oom: notify on oom killer invocation from the charge
>  path
> 
> Burt Holzman has noticed that memcg v1 doesn't notify about OOM events
> via eventfd anymore. The reason is that 29ef680ae7c2 ("memcg, oom: move
> out_of_memory back to the charge path") has moved the oom handling back
> to the charge path. While doing so the notification was left behind in
> mem_cgroup_oom_synchronize.
> 
> Fix the issue by replicating the oom hierarchy locking and the
> notification.
> 
> Reported-by: Burt Holzman <burt@fnal.gov>
> Fixes: 29ef680ae7c2 ("memcg, oom: move out_of_memory back to the charge path")
> Cc: stable # 4.19+
> Signed-off-by: Michal Hocko <mhocko@suse.com>

Looks good to me. The async side really does too much other stuff to
cleanly share code between them, so I don't mind separate code even if
it means they both have to do the mark, lock, notify dance.

Acked-by: Johannes Weiner <hannes@cmpxchg.org>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH] memcg, oom: notify on oom killer invocation from the charge path
  2018-12-21 14:49 OOM notification for cgroupsv1 broken in 4.19 Burt Holzman
@ 2018-12-24  9:11   ` Michal Hocko
  2018-12-24  9:11   ` Michal Hocko
  1 sibling, 0 replies; 7+ messages in thread
From: Michal Hocko @ 2018-12-24  9:11 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Johannes Weiner, Vladimir Davydov, Burt Holzman,
	cgroups mailinglist, linux-mm, LKML, Michal Hocko, Stable tree

From: Michal Hocko <mhocko@suse.com>

Burt Holzman has noticed that memcg v1 doesn't notify about OOM events
via eventfd anymore. The reason is that 29ef680ae7c2 ("memcg, oom: move
out_of_memory back to the charge path") has moved the oom handling back
to the charge path. While doing so the notification was left behind in
mem_cgroup_oom_synchronize.

Fix the issue by replicating the oom hierarchy locking and the
notification.

Reported-by: Burt Holzman <burt@fnal.gov>
Fixes: 29ef680ae7c2 ("memcg, oom: move out_of_memory back to the charge path")
Cc: stable # 4.19+
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
Hi Andrew,
I forgot to CC you on the patch sent as a reply to the original bug
report [1] so I am reposting with Ack from Johannes. Burt has confirmed
this is resolving the regression for him [2]. 4.20 is out but I have
marked the patch for stable so it should hit both 4.19 and 4.20.

[1] http://lkml.kernel.org/r/20181221153302.GB6410@dhcp22.suse.cz
[2] http://lkml.kernel.org/r/96D4815C-420F-41B7-B1E9-A741E7523596@services.fnal.gov

 mm/memcontrol.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6e1469b80cb7..7e6bf74ddb1e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1666,6 +1666,9 @@ enum oom_status {
 
 static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
 {
+	enum oom_status ret;
+	bool locked;
+
 	if (order > PAGE_ALLOC_COSTLY_ORDER)
 		return OOM_SKIPPED;
 
@@ -1700,10 +1703,23 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
 		return OOM_ASYNC;
 	}
 
+	mem_cgroup_mark_under_oom(memcg);
+
+	locked = mem_cgroup_oom_trylock(memcg);
+
+	if (locked)
+		mem_cgroup_oom_notify(memcg);
+
+	mem_cgroup_unmark_under_oom(memcg);
 	if (mem_cgroup_out_of_memory(memcg, mask, order))
-		return OOM_SUCCESS;
+		ret = OOM_SUCCESS;
+	else
+		ret = OOM_FAILED;
 
-	return OOM_FAILED;
+	if (locked)
+		mem_cgroup_oom_unlock(memcg);
+
+	return ret;
 }
 
 /**
-- 
2.19.2


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH] memcg, oom: notify on oom killer invocation from the charge path
@ 2018-12-24  9:11   ` Michal Hocko
  0 siblings, 0 replies; 7+ messages in thread
From: Michal Hocko @ 2018-12-24  9:11 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Johannes Weiner, Vladimir Davydov, Burt Holzman,
	cgroups mailinglist, linux-mm, LKML, Michal Hocko, Stable tree

From: Michal Hocko <mhocko@suse.com>

Burt Holzman has noticed that memcg v1 doesn't notify about OOM events
via eventfd anymore. The reason is that 29ef680ae7c2 ("memcg, oom: move
out_of_memory back to the charge path") has moved the oom handling back
to the charge path. While doing so the notification was left behind in
mem_cgroup_oom_synchronize.

Fix the issue by replicating the oom hierarchy locking and the
notification.

Reported-by: Burt Holzman <burt@fnal.gov>
Fixes: 29ef680ae7c2 ("memcg, oom: move out_of_memory back to the charge path")
Cc: stable # 4.19+
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
Hi Andrew,
I forgot to CC you on the patch sent as a reply to the original bug
report [1] so I am reposting with Ack from Johannes. Burt has confirmed
this is resolving the regression for him [2]. 4.20 is out but I have
marked the patch for stable so it should hit both 4.19 and 4.20.

[1] http://lkml.kernel.org/r/20181221153302.GB6410@dhcp22.suse.cz
[2] http://lkml.kernel.org/r/96D4815C-420F-41B7-B1E9-A741E7523596@services.fnal.gov

 mm/memcontrol.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6e1469b80cb7..7e6bf74ddb1e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1666,6 +1666,9 @@ enum oom_status {
 
 static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
 {
+	enum oom_status ret;
+	bool locked;
+
 	if (order > PAGE_ALLOC_COSTLY_ORDER)
 		return OOM_SKIPPED;
 
@@ -1700,10 +1703,23 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
 		return OOM_ASYNC;
 	}
 
+	mem_cgroup_mark_under_oom(memcg);
+
+	locked = mem_cgroup_oom_trylock(memcg);
+
+	if (locked)
+		mem_cgroup_oom_notify(memcg);
+
+	mem_cgroup_unmark_under_oom(memcg);
 	if (mem_cgroup_out_of_memory(memcg, mask, order))
-		return OOM_SUCCESS;
+		ret = OOM_SUCCESS;
+	else
+		ret = OOM_FAILED;
 
-	return OOM_FAILED;
+	if (locked)
+		mem_cgroup_oom_unlock(memcg);
+
+	return ret;
 }
 
 /**
-- 
2.19.2

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2018-12-24  9:11 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-21 14:49 OOM notification for cgroupsv1 broken in 4.19 Burt Holzman
2018-12-21 15:33 ` Michal Hocko
2018-12-21 15:33   ` Michal Hocko
2018-12-21 17:33   ` Burt Holzman
2018-12-21 19:02   ` Johannes Weiner
2018-12-24  9:11 ` [PATCH] memcg, oom: notify on oom killer invocation from the charge path Michal Hocko
2018-12-24  9:11   ` Michal Hocko

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.