From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id 1FEFF6B0038 for ; Sat, 1 Apr 2017 23:53:12 -0400 (EDT) Received: by mail-pf0-f199.google.com with SMTP id s29so25610062pfg.21 for ; Sat, 01 Apr 2017 20:53:12 -0700 (PDT) Received: from www262.sakura.ne.jp (www262.sakura.ne.jp. [2001:e42:101:1:202:181:97:72]) by mx.google.com with ESMTPS id o1si9935706pgn.247.2017.04.01.20.53.10 for (version=TLS1 cipher=AES128-SHA bits=128/128); Sat, 01 Apr 2017 20:53:11 -0700 (PDT) Subject: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message From: Tetsuo Handa Message-Id: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> Date: Sun, 2 Apr 2017 12:52:55 +0900 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-linux-mm@kvack.org List-ID: To: vdavydov@virtuozzo.com, hannes@cmpxchg.org, mhocko@kernel.org, rientjes@google.com Cc: linux-mm@kvack.org I noticed that SysRq-f prints "sysrq: OOM request ignored because killer is disabled" when no process was selected (rather than when oom killer was disabled). This message was not printed until Linux 4.8 because commit 7c5f64f84483bd13 ("mm: oom: deduplicate victim selection code for memcg and global oom") changed from "return true;" to "return !!oc->chosen;" when is_sysrq_oom(oc) is true. Is this what we meant? [ 713.805315] sysrq: SysRq : Manual OOM execution [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB [ 714.004805] sysrq: SysRq : Manual OOM execution [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB [ 714.189310] sysrq: SysRq : Manual OOM execution [ 714.193425] sysrq: OOM request ignored because killer is disabled [ 714.381313] sysrq: SysRq : Manual OOM execution [ 714.385158] sysrq: OOM request ignored because killer is disabled [ 714.573320] sysrq: SysRq : Manual OOM execution [ 714.576988] sysrq: OOM request ignored because killer is disabled -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f197.google.com (mail-wr0-f197.google.com [209.85.128.197]) by kanga.kvack.org (Postfix) with ESMTP id 259576B0038 for ; Mon, 3 Apr 2017 04:38:21 -0400 (EDT) Received: by mail-wr0-f197.google.com with SMTP id 34so22991489wrb.20 for ; Mon, 03 Apr 2017 01:38:21 -0700 (PDT) Received: from mx2.suse.de (mx2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id c133si14340276wme.67.2017.04.03.01.38.19 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 03 Apr 2017 01:38:19 -0700 (PDT) Date: Mon, 3 Apr 2017 10:38:01 +0200 From: Michal Hocko Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message Message-ID: <20170403083800.GF24661@dhcp22.suse.cz> References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> Sender: owner-linux-mm@kvack.org List-ID: To: Tetsuo Handa Cc: vdavydov@virtuozzo.com, hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org On Sun 02-04-17 12:52:55, Tetsuo Handa wrote: > I noticed that SysRq-f prints > > "sysrq: OOM request ignored because killer is disabled" > > when no process was selected (rather than when oom killer was disabled). > This message was not printed until Linux 4.8 because commit 7c5f64f84483bd13 > ("mm: oom: deduplicate victim selection code for memcg and global oom") changed > from "return true;" to "return !!oc->chosen;" when is_sysrq_oom(oc) is true. > > Is this what we meant? > > [ 713.805315] sysrq: SysRq : Manual OOM execution > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > [ 714.004805] sysrq: SysRq : Manual OOM execution > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > [ 714.189310] sysrq: SysRq : Manual OOM execution > [ 714.193425] sysrq: OOM request ignored because killer is disabled > [ 714.381313] sysrq: SysRq : Manual OOM execution > [ 714.385158] sysrq: OOM request ignored because killer is disabled > [ 714.573320] sysrq: SysRq : Manual OOM execution > [ 714.576988] sysrq: OOM request ignored because killer is disabled So, what about this? --- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f197.google.com (mail-wr0-f197.google.com [209.85.128.197]) by kanga.kvack.org (Postfix) with ESMTP id 2B98D6B0038 for ; Mon, 3 Apr 2017 05:11:57 -0400 (EDT) Received: by mail-wr0-f197.google.com with SMTP id e11so23164102wra.0 for ; Mon, 03 Apr 2017 02:11:57 -0700 (PDT) Received: from mx2.suse.de (mx2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id 61si12666901wrl.157.2017.04.03.02.11.55 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 03 Apr 2017 02:11:55 -0700 (PDT) Date: Mon, 3 Apr 2017 11:11:53 +0200 From: Michal Hocko Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message Message-ID: <20170403091153.GH24661@dhcp22.suse.cz> References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> <20170403083800.GF24661@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170403083800.GF24661@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org List-ID: To: Tetsuo Handa Cc: hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org, Vladimir Davydov [Fixup Vladimir email address] On Mon 03-04-17 10:38:00, Michal Hocko wrote: > On Sun 02-04-17 12:52:55, Tetsuo Handa wrote: > > I noticed that SysRq-f prints > > > > "sysrq: OOM request ignored because killer is disabled" > > > > when no process was selected (rather than when oom killer was disabled). > > This message was not printed until Linux 4.8 because commit 7c5f64f84483bd13 > > ("mm: oom: deduplicate victim selection code for memcg and global oom") changed > > from "return true;" to "return !!oc->chosen;" when is_sysrq_oom(oc) is true. > > > > Is this what we meant? > > > > [ 713.805315] sysrq: SysRq : Manual OOM execution > > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > > [ 714.004805] sysrq: SysRq : Manual OOM execution > > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > > [ 714.189310] sysrq: SysRq : Manual OOM execution > > [ 714.193425] sysrq: OOM request ignored because killer is disabled > > [ 714.381313] sysrq: SysRq : Manual OOM execution > > [ 714.385158] sysrq: OOM request ignored because killer is disabled > > [ 714.573320] sysrq: SysRq : Manual OOM execution > > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > So, what about this? > --- > From 6721932dba5b5143be0fa8110450231038af4238 Mon Sep 17 00:00:00 2001 > From: Michal Hocko > Date: Mon, 3 Apr 2017 10:30:14 +0200 > Subject: [PATCH] oom: improve oom disable handling > > Tetsuo has reported that sysrq triggered OOM killer will print a > misleading information when no tasks are selected: > > [ 713.805315] sysrq: SysRq : Manual OOM execution > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > [ 714.004805] sysrq: SysRq : Manual OOM execution > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > [ 714.189310] sysrq: SysRq : Manual OOM execution > [ 714.193425] sysrq: OOM request ignored because killer is disabled > [ 714.381313] sysrq: SysRq : Manual OOM execution > [ 714.385158] sysrq: OOM request ignored because killer is disabled > [ 714.573320] sysrq: SysRq : Manual OOM execution > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > The real reason is that there are no eligible tasks for the OOM killer > to select but since 7c5f64f84483bd13 ("mm: oom: deduplicate victim > selection code for memcg and global oom") the semantic of out_of_memory > has changed without updating moom_callback. > > This patch updates moom_callback to tell that no task was eligible > which is the case for both oom killer disabled and no eligible tasks. > In order to help distinguish first case from the second add printk to > both oom_killer_{enable,disable}. This information is useful on its own > because it might help debugging potential memory allocation failures. > > Fixes: 7c5f64f84483bd13 ("mm: oom: deduplicate victim selection code for memcg and global oom") > Reported-by: Tetsuo Handa > Signed-off-by: Michal Hocko > --- > drivers/tty/sysrq.c | 2 +- > mm/oom_kill.c | 2 ++ > 2 files changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c > index 71136742e606..a91f58dc2cb6 100644 > --- a/drivers/tty/sysrq.c > +++ b/drivers/tty/sysrq.c > @@ -370,7 +370,7 @@ static void moom_callback(struct work_struct *ignored) > > mutex_lock(&oom_lock); > if (!out_of_memory(&oc)) > - pr_info("OOM request ignored because killer is disabled\n"); > + pr_info("OOM request ignored. No task eligible\n"); > mutex_unlock(&oom_lock); > } > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > index 51c091849dcb..ad2b112cdf3e 100644 > --- a/mm/oom_kill.c > +++ b/mm/oom_kill.c > @@ -682,6 +682,7 @@ void exit_oom_victim(void) > void oom_killer_enable(void) > { > oom_killer_disabled = false; > + pr_info("OOM killer enabled.\n"); > } > > /** > @@ -718,6 +719,7 @@ bool oom_killer_disable(signed long timeout) > oom_killer_enable(); > return false; > } > + pr_info("OOM killer disabled.\n"); > > return true; > } > -- > 2.11.0 > > > -- > Michal Hocko > SUSE Labs -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f70.google.com (mail-pg0-f70.google.com [74.125.83.70]) by kanga.kvack.org (Postfix) with ESMTP id 33CBB6B0038 for ; Mon, 3 Apr 2017 06:10:55 -0400 (EDT) Received: by mail-pg0-f70.google.com with SMTP id n5so137568667pgd.19 for ; Mon, 03 Apr 2017 03:10:55 -0700 (PDT) Received: from www262.sakura.ne.jp (www262.sakura.ne.jp. [2001:e42:101:1:202:181:97:72]) by mx.google.com with ESMTPS id f1si13840811pln.331.2017.04.03.03.10.45 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 03 Apr 2017 03:10:50 -0700 (PDT) Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message From: Tetsuo Handa References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> <20170403083800.GF24661@dhcp22.suse.cz> <20170403091153.GH24661@dhcp22.suse.cz> In-Reply-To: <20170403091153.GH24661@dhcp22.suse.cz> Message-Id: <201704031910.GGH56210.QVFFOSHLOMFtJO@I-love.SAKURA.ne.jp> Date: Mon, 3 Apr 2017 19:10:30 +0900 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-linux-mm@kvack.org List-ID: To: mhocko@kernel.org Cc: hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org, vdavydov.dev@gmail.com Michal Hocko wrote: > [Fixup Vladimir email address] > > On Mon 03-04-17 10:38:00, Michal Hocko wrote: > > On Sun 02-04-17 12:52:55, Tetsuo Handa wrote: > > > I noticed that SysRq-f prints > > > > > > "sysrq: OOM request ignored because killer is disabled" > > > > > > when no process was selected (rather than when oom killer was disabled). > > > This message was not printed until Linux 4.8 because commit 7c5f64f84483bd13 > > > ("mm: oom: deduplicate victim selection code for memcg and global oom") changed > > > from "return true;" to "return !!oc->chosen;" when is_sysrq_oom(oc) is true. > > > > > > Is this what we meant? > > > > > > [ 713.805315] sysrq: SysRq : Manual OOM execution > > > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > > > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > > > [ 714.004805] sysrq: SysRq : Manual OOM execution > > > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > > > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > > > [ 714.189310] sysrq: SysRq : Manual OOM execution > > > [ 714.193425] sysrq: OOM request ignored because killer is disabled > > > [ 714.381313] sysrq: SysRq : Manual OOM execution > > > [ 714.385158] sysrq: OOM request ignored because killer is disabled > > > [ 714.573320] sysrq: SysRq : Manual OOM execution > > > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > > > So, what about this? I thought below change in out_of_memory(). - return !!oc->chosen; + return oc->chosen || is_sysrq_oom(oc); You can take either approach. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f69.google.com (mail-lf0-f69.google.com [209.85.215.69]) by kanga.kvack.org (Postfix) with ESMTP id 239866B0397 for ; Mon, 3 Apr 2017 06:10:58 -0400 (EDT) Received: by mail-lf0-f69.google.com with SMTP id h89so26754756lfi.6 for ; Mon, 03 Apr 2017 03:10:58 -0700 (PDT) Received: from smtp62.i.mail.ru (smtp62.i.mail.ru. [217.69.128.42]) by mx.google.com with ESMTPS id 1si7362111ljp.49.2017.04.03.03.10.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Apr 2017 03:10:50 -0700 (PDT) Date: Mon, 3 Apr 2017 13:10:41 +0300 From: Vladimir Davydov Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message Message-ID: <20170403101041.GC29639@esperanza> References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> <20170403083800.GF24661@dhcp22.suse.cz> <20170403091153.GH24661@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170403091153.GH24661@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko Cc: Tetsuo Handa , hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org On Mon, Apr 03, 2017 at 11:11:53AM +0200, Michal Hocko wrote: > [Fixup Vladimir email address] > > On Mon 03-04-17 10:38:00, Michal Hocko wrote: > > On Sun 02-04-17 12:52:55, Tetsuo Handa wrote: > > > I noticed that SysRq-f prints > > > > > > "sysrq: OOM request ignored because killer is disabled" > > > > > > when no process was selected (rather than when oom killer was disabled). > > > This message was not printed until Linux 4.8 because commit 7c5f64f84483bd13 > > > ("mm: oom: deduplicate victim selection code for memcg and global oom") changed > > > from "return true;" to "return !!oc->chosen;" when is_sysrq_oom(oc) is true. > > > > > > Is this what we meant? No that was not intentional. > > > > > > [ 713.805315] sysrq: SysRq : Manual OOM execution > > > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > > > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > > > [ 714.004805] sysrq: SysRq : Manual OOM execution > > > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > > > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > > > [ 714.189310] sysrq: SysRq : Manual OOM execution > > > [ 714.193425] sysrq: OOM request ignored because killer is disabled > > > [ 714.381313] sysrq: SysRq : Manual OOM execution > > > [ 714.385158] sysrq: OOM request ignored because killer is disabled > > > [ 714.573320] sysrq: SysRq : Manual OOM execution > > > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > > > So, what about this? > > --- > > From 6721932dba5b5143be0fa8110450231038af4238 Mon Sep 17 00:00:00 2001 > > From: Michal Hocko > > Date: Mon, 3 Apr 2017 10:30:14 +0200 > > Subject: [PATCH] oom: improve oom disable handling > > > > Tetsuo has reported that sysrq triggered OOM killer will print a > > misleading information when no tasks are selected: > > > > [ 713.805315] sysrq: SysRq : Manual OOM execution > > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > > [ 714.004805] sysrq: SysRq : Manual OOM execution > > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > > [ 714.189310] sysrq: SysRq : Manual OOM execution > > [ 714.193425] sysrq: OOM request ignored because killer is disabled > > [ 714.381313] sysrq: SysRq : Manual OOM execution > > [ 714.385158] sysrq: OOM request ignored because killer is disabled > > [ 714.573320] sysrq: SysRq : Manual OOM execution > > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > > > The real reason is that there are no eligible tasks for the OOM killer > > to select but since 7c5f64f84483bd13 ("mm: oom: deduplicate victim > > selection code for memcg and global oom") the semantic of out_of_memory > > has changed without updating moom_callback. > > > > This patch updates moom_callback to tell that no task was eligible > > which is the case for both oom killer disabled and no eligible tasks. > > In order to help distinguish first case from the second add printk to > > both oom_killer_{enable,disable}. This information is useful on its own > > because it might help debugging potential memory allocation failures. I think this makes sense although personally I find the "No task eligible" message in case OOM killer is disabled manually a bit confusing: the thing is in order to find out why an OOM request failed you'll have to scan the full log, which might be unavailable. May be, we'd better just make out_of_memory() return true in case is_sysrq_oom() is true and no task was found, as it used to be. > > > > Fixes: 7c5f64f84483bd13 ("mm: oom: deduplicate victim selection code for memcg and global oom") > > Reported-by: Tetsuo Handa > > Signed-off-by: Michal Hocko > > --- > > drivers/tty/sysrq.c | 2 +- > > mm/oom_kill.c | 2 ++ > > 2 files changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c > > index 71136742e606..a91f58dc2cb6 100644 > > --- a/drivers/tty/sysrq.c > > +++ b/drivers/tty/sysrq.c > > @@ -370,7 +370,7 @@ static void moom_callback(struct work_struct *ignored) > > > > mutex_lock(&oom_lock); > > if (!out_of_memory(&oc)) > > - pr_info("OOM request ignored because killer is disabled\n"); > > + pr_info("OOM request ignored. No task eligible\n"); > > mutex_unlock(&oom_lock); > > } > > > > diff --git a/mm/oom_kill.c b/mm/oom_kill.c > > index 51c091849dcb..ad2b112cdf3e 100644 > > --- a/mm/oom_kill.c > > +++ b/mm/oom_kill.c > > @@ -682,6 +682,7 @@ void exit_oom_victim(void) > > void oom_killer_enable(void) > > { > > oom_killer_disabled = false; > > + pr_info("OOM killer enabled.\n"); > > } > > > > /** > > @@ -718,6 +719,7 @@ bool oom_killer_disable(signed long timeout) > > oom_killer_enable(); > > return false; > > } > > + pr_info("OOM killer disabled.\n"); > > > > return true; > > } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199]) by kanga.kvack.org (Postfix) with ESMTP id A66B06B0038 for ; Mon, 3 Apr 2017 06:20:34 -0400 (EDT) Received: by mail-wr0-f199.google.com with SMTP id o70so23400445wrb.11 for ; Mon, 03 Apr 2017 03:20:34 -0700 (PDT) Received: from mx2.suse.de (mx2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id q6si14596514wmg.122.2017.04.03.03.20.33 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 03 Apr 2017 03:20:33 -0700 (PDT) Date: Mon, 3 Apr 2017 12:20:30 +0200 From: Michal Hocko Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message Message-ID: <20170403102029.GJ24661@dhcp22.suse.cz> References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> <20170403083800.GF24661@dhcp22.suse.cz> <20170403091153.GH24661@dhcp22.suse.cz> <20170403101041.GC29639@esperanza> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170403101041.GC29639@esperanza> Sender: owner-linux-mm@kvack.org List-ID: To: Vladimir Davydov Cc: Tetsuo Handa , hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org On Mon 03-04-17 13:10:41, Vladimir Davydov wrote: > On Mon, Apr 03, 2017 at 11:11:53AM +0200, Michal Hocko wrote: > > [Fixup Vladimir email address] > > > > On Mon 03-04-17 10:38:00, Michal Hocko wrote: > > > On Sun 02-04-17 12:52:55, Tetsuo Handa wrote: > > > > I noticed that SysRq-f prints > > > > > > > > "sysrq: OOM request ignored because killer is disabled" > > > > > > > > when no process was selected (rather than when oom killer was disabled). > > > > This message was not printed until Linux 4.8 because commit 7c5f64f84483bd13 > > > > ("mm: oom: deduplicate victim selection code for memcg and global oom") changed > > > > from "return true;" to "return !!oc->chosen;" when is_sysrq_oom(oc) is true. > > > > > > > > Is this what we meant? > > No that was not intentional. > > > > > > > > > [ 713.805315] sysrq: SysRq : Manual OOM execution > > > > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > > > > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > > > > [ 714.004805] sysrq: SysRq : Manual OOM execution > > > > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > > > > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > > > > [ 714.189310] sysrq: SysRq : Manual OOM execution > > > > [ 714.193425] sysrq: OOM request ignored because killer is disabled > > > > [ 714.381313] sysrq: SysRq : Manual OOM execution > > > > [ 714.385158] sysrq: OOM request ignored because killer is disabled > > > > [ 714.573320] sysrq: SysRq : Manual OOM execution > > > > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > > > > > So, what about this? > > > --- > > > From 6721932dba5b5143be0fa8110450231038af4238 Mon Sep 17 00:00:00 2001 > > > From: Michal Hocko > > > Date: Mon, 3 Apr 2017 10:30:14 +0200 > > > Subject: [PATCH] oom: improve oom disable handling > > > > > > Tetsuo has reported that sysrq triggered OOM killer will print a > > > misleading information when no tasks are selected: > > > > > > [ 713.805315] sysrq: SysRq : Manual OOM execution > > > [ 713.808920] Out of memory: Kill process 4468 ((agetty)) score 0 or sacrifice child > > > [ 713.814913] Killed process 4468 ((agetty)) total-vm:43704kB, anon-rss:1760kB, file-rss:0kB, shmem-rss:0kB > > > [ 714.004805] sysrq: SysRq : Manual OOM execution > > > [ 714.005936] Out of memory: Kill process 4469 (systemd-cgroups) score 0 or sacrifice child > > > [ 714.008117] Killed process 4469 (systemd-cgroups) total-vm:10704kB, anon-rss:120kB, file-rss:0kB, shmem-rss:0kB > > > [ 714.189310] sysrq: SysRq : Manual OOM execution > > > [ 714.193425] sysrq: OOM request ignored because killer is disabled > > > [ 714.381313] sysrq: SysRq : Manual OOM execution > > > [ 714.385158] sysrq: OOM request ignored because killer is disabled > > > [ 714.573320] sysrq: SysRq : Manual OOM execution > > > [ 714.576988] sysrq: OOM request ignored because killer is disabled > > > > > > The real reason is that there are no eligible tasks for the OOM killer > > > to select but since 7c5f64f84483bd13 ("mm: oom: deduplicate victim > > > selection code for memcg and global oom") the semantic of out_of_memory > > > has changed without updating moom_callback. > > > > > > This patch updates moom_callback to tell that no task was eligible > > > which is the case for both oom killer disabled and no eligible tasks. > > > In order to help distinguish first case from the second add printk to > > > both oom_killer_{enable,disable}. This information is useful on its own > > > because it might help debugging potential memory allocation failures. > > I think this makes sense although personally I find the "No task > eligible" message in case OOM killer is disabled manually a bit > confusing: the thing is in order to find out why an OOM request > failed you'll have to scan the full log, which might be unavailable. > May be, we'd better just make out_of_memory() return true in case > is_sysrq_oom() is true and no task was found, as it used to be. Well, the thing is that the oom killer is disabled only during the PM suspend and I do not expect we would grow new users. And it is quite unlikely to invoke sysrq during that time. The OOM killer is disabled is unlikely to be too far in the past in that case. It is also a matter of fact that no tasks are eligible during that time period so the message is not misleading. I have considered is_sysrq_oom approach but I would rather not add yet another exception for that path, we have quite some of them already. Especially when the only point of that exception would be to control a log message. -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f198.google.com (mail-wr0-f198.google.com [209.85.128.198]) by kanga.kvack.org (Postfix) with ESMTP id 3FDFC6B0038 for ; Tue, 4 Apr 2017 09:23:54 -0400 (EDT) Received: by mail-wr0-f198.google.com with SMTP id g7so28541682wrd.16 for ; Tue, 04 Apr 2017 06:23:54 -0700 (PDT) Received: from mx2.suse.de (mx2.suse.de. [195.135.220.15]) by mx.google.com with ESMTPS id u203si8657027wmg.14.2017.04.04.06.23.52 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 04 Apr 2017 06:23:52 -0700 (PDT) Date: Tue, 4 Apr 2017 15:23:50 +0200 From: Michal Hocko Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message Message-ID: <20170404132349.GM15132@dhcp22.suse.cz> References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> <20170403083800.GF24661@dhcp22.suse.cz> <20170403091153.GH24661@dhcp22.suse.cz> <20170403101041.GC29639@esperanza> <20170403102029.GJ24661@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170403102029.GJ24661@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org List-ID: To: Vladimir Davydov Cc: Tetsuo Handa , hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org On Mon 03-04-17 12:20:29, Michal Hocko wrote: > On Mon 03-04-17 13:10:41, Vladimir Davydov wrote: > > On Mon, Apr 03, 2017 at 11:11:53AM +0200, Michal Hocko wrote: > > > [Fixup Vladimir email address] > > > > > > On Mon 03-04-17 10:38:00, Michal Hocko wrote: [...] > > > > The real reason is that there are no eligible tasks for the OOM killer > > > > to select but since 7c5f64f84483bd13 ("mm: oom: deduplicate victim > > > > selection code for memcg and global oom") the semantic of out_of_memory > > > > has changed without updating moom_callback. > > > > > > > > This patch updates moom_callback to tell that no task was eligible > > > > which is the case for both oom killer disabled and no eligible tasks. > > > > In order to help distinguish first case from the second add printk to > > > > both oom_killer_{enable,disable}. This information is useful on its own > > > > because it might help debugging potential memory allocation failures. > > > > I think this makes sense although personally I find the "No task > > eligible" message in case OOM killer is disabled manually a bit > > confusing: the thing is in order to find out why an OOM request > > failed you'll have to scan the full log, which might be unavailable. > > May be, we'd better just make out_of_memory() return true in case > > is_sysrq_oom() is true and no task was found, as it used to be. > > Well, the thing is that the oom killer is disabled only during the PM > suspend and I do not expect we would grow new users. And it is quite > unlikely to invoke sysrq during that time. The OOM killer is disabled is > unlikely to be too far in the past in that case. It is also a matter of > fact that no tasks are eligible during that time period so the message > is not misleading. I have considered is_sysrq_oom approach but I would > rather not add yet another exception for that path, we have quite some > of them already. Especially when the only point of that exception would > be to control a log message. Does this reasoning make sense to you? Can I post the patch to Andrew or you sill see strong reasons to tweak out_of_memory? -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f69.google.com (mail-lf0-f69.google.com [209.85.215.69]) by kanga.kvack.org (Postfix) with ESMTP id EA55C6B0038 for ; Tue, 4 Apr 2017 09:36:25 -0400 (EDT) Received: by mail-lf0-f69.google.com with SMTP id x137so32879158lff.3 for ; Tue, 04 Apr 2017 06:36:25 -0700 (PDT) Received: from smtp51.i.mail.ru (smtp51.i.mail.ru. [94.100.177.111]) by mx.google.com with ESMTPS id 77si9296002lfv.158.2017.04.04.06.36.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Apr 2017 06:36:23 -0700 (PDT) Date: Tue, 4 Apr 2017 16:36:17 +0300 From: Vladimir Davydov Subject: Re: oom: Bogus "sysrq: OOM request ignored because killer is disabled" message Message-ID: <20170404133616.GA8751@esperanza> References: <201704021252.GIF21549.QFFOFOMVJtHSLO@I-love.SAKURA.ne.jp> <20170403083800.GF24661@dhcp22.suse.cz> <20170403091153.GH24661@dhcp22.suse.cz> <20170403101041.GC29639@esperanza> <20170403102029.GJ24661@dhcp22.suse.cz> <20170404132349.GM15132@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170404132349.GM15132@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko Cc: Tetsuo Handa , hannes@cmpxchg.org, rientjes@google.com, linux-mm@kvack.org On Tue, Apr 04, 2017 at 03:23:50PM +0200, Michal Hocko wrote: > On Mon 03-04-17 12:20:29, Michal Hocko wrote: > > On Mon 03-04-17 13:10:41, Vladimir Davydov wrote: > > > On Mon, Apr 03, 2017 at 11:11:53AM +0200, Michal Hocko wrote: > > > > [Fixup Vladimir email address] > > > > > > > > On Mon 03-04-17 10:38:00, Michal Hocko wrote: > [...] > > > > > The real reason is that there are no eligible tasks for the OOM killer > > > > > to select but since 7c5f64f84483bd13 ("mm: oom: deduplicate victim > > > > > selection code for memcg and global oom") the semantic of out_of_memory > > > > > has changed without updating moom_callback. > > > > > > > > > > This patch updates moom_callback to tell that no task was eligible > > > > > which is the case for both oom killer disabled and no eligible tasks. > > > > > In order to help distinguish first case from the second add printk to > > > > > both oom_killer_{enable,disable}. This information is useful on its own > > > > > because it might help debugging potential memory allocation failures. > > > > > > I think this makes sense although personally I find the "No task > > > eligible" message in case OOM killer is disabled manually a bit > > > confusing: the thing is in order to find out why an OOM request > > > failed you'll have to scan the full log, which might be unavailable. > > > May be, we'd better just make out_of_memory() return true in case > > > is_sysrq_oom() is true and no task was found, as it used to be. > > > > Well, the thing is that the oom killer is disabled only during the PM > > suspend and I do not expect we would grow new users. And it is quite > > unlikely to invoke sysrq during that time. The OOM killer is disabled is > > unlikely to be too far in the past in that case. It is also a matter of > > fact that no tasks are eligible during that time period so the message > > is not misleading. I have considered is_sysrq_oom approach but I would > > rather not add yet another exception for that path, we have quite some > > of them already. Especially when the only point of that exception would > > be to control a log message. > > Does this reasoning make sense to you? Can I post the patch to Andrew or > you sill see strong reasons to tweak out_of_memory? I think your arguments are fair enough. I don't have any objections. Thanks. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org