From: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> To: mhocko@kernel.org, rientjes@google.com Cc: akpm@linux-foundation.org, mgorman@suse.de, penguin-kernel@i-love.sakura.ne.jp, torvalds@linux-foundation.org, oleg@redhat.com, hughd@google.com, andrea@kernel.org, riel@redhat.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/2] mm, oom: introduce oom reaper Date: Tue, 2 Feb 2016 20:48:05 +0900 [thread overview] Message-ID: <201602022048.GCJ04176.tOFFSVFHLMJOQO@I-love.SAKURA.ne.jp> (raw) In-Reply-To: <20160202085758.GE19910@dhcp22.suse.cz> Michal Hocko wrote: > > In this case, the oom reaper has ignored the next victim and doesn't do > > anything; the simple race has prevented it from zapping memory and does > > not reduce the livelock probability. > > > > This can be solved either by queueing mm's to reap or involving the oom > > reaper into the oom killer synchronization itself. > > as we have already discussed previously oom reaper is really tricky to > be called from the direct OOM context. I will go with queuing. > OK. But it is not easy to build a reliable OOM-reap queuing chain. I think that a dedicated kernel thread which does OOM-kill operation and OOM-reap operation will be expected. That will also handle the "sleeping for too long with oom_lock held after sending SIGKILL" problem. > > I'm baffled by any reference to "memcg oom heavy loads", I don't > > understand this paragraph, sorry. If a memcg is oom, we shouldn't be > > disrupting the global runqueue by running oom_reaper at a high priority. > > The disruption itself is not only in first wakeup but also in how long the > > reaper can run and when it is rescheduled: for a lot of memory this is > > potentially long. The reaper is best-effort, as the changelog indicates, > > and we shouldn't have a reliance on this high priority: oom kill exiting > > can't possibly be expected to be immediate. This high priority should be > > removed so memcg oom conditions are isolated and don't affect other loads. > > If this is a concern then I would be tempted to simply disable oom > reaper for memcg oom altogether. For me it is much more important that > the reaper, even though a best effort, is guaranteed to schedule if > something goes terribly wrong on the machine. I think that if something goes terribly wrong on the machine, a guarantee for scheduling the reaper will not help unless we build a reliable queuing chain. Building a reliable queuing chain will break some of assumptions provided by current behavior. For me, a guarantee for scheduling for next OOM-kill operation (with globally opening some or all of memory reserves) before building a reliable queuing chain is much more important. > But ohh well... I will queue up a patch to do this > on top. I plan to repost the full patchset shortly. Maybe we all agree with introducing OOM reaper without queuing, but I do want to see a guarantee for scheduling for next OOM-kill operation before trying to build a reliable queuing chain.
WARNING: multiple messages have this Message-ID (diff)
From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> To: mhocko@kernel.org, rientjes@google.com Cc: akpm@linux-foundation.org, mgorman@suse.de, penguin-kernel@i-love.sakura.ne.jp, torvalds@linux-foundation.org, oleg@redhat.com, hughd@google.com, andrea@kernel.org, riel@redhat.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/2] mm, oom: introduce oom reaper Date: Tue, 2 Feb 2016 20:48:05 +0900 [thread overview] Message-ID: <201602022048.GCJ04176.tOFFSVFHLMJOQO@I-love.SAKURA.ne.jp> (raw) In-Reply-To: <20160202085758.GE19910@dhcp22.suse.cz> Michal Hocko wrote: > > In this case, the oom reaper has ignored the next victim and doesn't do > > anything; the simple race has prevented it from zapping memory and does > > not reduce the livelock probability. > > > > This can be solved either by queueing mm's to reap or involving the oom > > reaper into the oom killer synchronization itself. > > as we have already discussed previously oom reaper is really tricky to > be called from the direct OOM context. I will go with queuing. > OK. But it is not easy to build a reliable OOM-reap queuing chain. I think that a dedicated kernel thread which does OOM-kill operation and OOM-reap operation will be expected. That will also handle the "sleeping for too long with oom_lock held after sending SIGKILL" problem. > > I'm baffled by any reference to "memcg oom heavy loads", I don't > > understand this paragraph, sorry. If a memcg is oom, we shouldn't be > > disrupting the global runqueue by running oom_reaper at a high priority. > > The disruption itself is not only in first wakeup but also in how long the > > reaper can run and when it is rescheduled: for a lot of memory this is > > potentially long. The reaper is best-effort, as the changelog indicates, > > and we shouldn't have a reliance on this high priority: oom kill exiting > > can't possibly be expected to be immediate. This high priority should be > > removed so memcg oom conditions are isolated and don't affect other loads. > > If this is a concern then I would be tempted to simply disable oom > reaper for memcg oom altogether. For me it is much more important that > the reaper, even though a best effort, is guaranteed to schedule if > something goes terribly wrong on the machine. I think that if something goes terribly wrong on the machine, a guarantee for scheduling the reaper will not help unless we build a reliable queuing chain. Building a reliable queuing chain will break some of assumptions provided by current behavior. For me, a guarantee for scheduling for next OOM-kill operation (with globally opening some or all of memory reserves) before building a reliable queuing chain is much more important. > But ohh well... I will queue up a patch to do this > on top. I plan to repost the full patchset shortly. Maybe we all agree with introducing OOM reaper without queuing, but I do want to see a guarantee for scheduling for next OOM-kill operation before trying to build a reliable queuing chain. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-02-02 11:48 UTC|newest] Thread overview: 108+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-01-06 15:42 [PATCH 0/2 -mm] oom reaper v4 Michal Hocko 2016-01-06 15:42 ` Michal Hocko 2016-01-06 15:42 ` [PATCH 1/2] mm, oom: introduce oom reaper Michal Hocko 2016-01-06 15:42 ` Michal Hocko 2016-01-07 11:23 ` Tetsuo Handa 2016-01-07 11:23 ` Tetsuo Handa 2016-01-07 12:30 ` Michal Hocko 2016-01-07 12:30 ` Michal Hocko 2016-01-11 22:54 ` Andrew Morton 2016-01-11 22:54 ` Andrew Morton 2016-01-12 8:16 ` Michal Hocko 2016-01-12 8:16 ` Michal Hocko 2016-01-28 1:28 ` David Rientjes 2016-01-28 1:28 ` David Rientjes 2016-01-28 21:42 ` Michal Hocko 2016-01-28 21:42 ` Michal Hocko 2016-02-02 3:02 ` David Rientjes 2016-02-02 3:02 ` David Rientjes 2016-02-02 8:57 ` Michal Hocko 2016-02-02 8:57 ` Michal Hocko 2016-02-02 11:48 ` Tetsuo Handa [this message] 2016-02-02 11:48 ` Tetsuo Handa 2016-02-02 22:55 ` David Rientjes 2016-02-02 22:55 ` David Rientjes 2016-02-02 22:51 ` David Rientjes 2016-02-02 22:51 ` David Rientjes 2016-02-03 10:31 ` Tetsuo Handa 2016-02-03 10:31 ` Tetsuo Handa 2016-01-06 15:42 ` [PATCH 2/2] oom reaper: handle anonymous mlocked pages Michal Hocko 2016-01-06 15:42 ` Michal Hocko 2016-01-07 8:14 ` Michal Hocko 2016-01-07 8:14 ` Michal Hocko 2016-01-11 12:42 ` [PATCH 3/2] oom: clear TIF_MEMDIE after oom_reaper managed to unmap the address space Michal Hocko 2016-01-11 12:42 ` Michal Hocko 2016-01-11 16:52 ` Johannes Weiner 2016-01-11 16:52 ` Johannes Weiner 2016-01-11 17:46 ` Michal Hocko 2016-01-11 17:46 ` Michal Hocko 2016-02-15 10:58 ` Tetsuo Handa 2016-02-15 10:58 ` Tetsuo Handa 2016-01-18 4:35 ` Tetsuo Handa 2016-01-18 4:35 ` Tetsuo Handa 2016-01-18 10:22 ` Tetsuo Handa 2016-01-18 10:22 ` Tetsuo Handa 2016-01-26 16:38 ` Michal Hocko 2016-01-26 16:38 ` Michal Hocko 2016-01-28 11:24 ` Tetsuo Handa 2016-01-28 11:24 ` Tetsuo Handa 2016-01-28 21:51 ` Michal Hocko 2016-01-28 21:51 ` Michal Hocko 2016-01-28 22:26 ` Tetsuo Handa 2016-01-28 22:26 ` Tetsuo Handa 2016-01-28 22:36 ` Michal Hocko 2016-01-28 22:36 ` Michal Hocko 2016-01-28 22:33 ` Michal Hocko 2016-01-28 22:33 ` Michal Hocko -- strict thread matches above, loose matches on Subject: below -- 2015-12-15 18:36 [PATCH 1/2] mm, oom: introduce oom reaper Michal Hocko 2015-12-15 18:36 ` Michal Hocko 2015-12-17 0:50 ` Andrew Morton 2015-12-17 0:50 ` Andrew Morton 2015-12-17 13:02 ` Michal Hocko 2015-12-17 13:02 ` Michal Hocko 2015-12-17 19:55 ` Linus Torvalds 2015-12-17 19:55 ` Linus Torvalds 2015-12-17 20:00 ` Andrew Morton 2015-12-17 20:00 ` Andrew Morton 2015-12-18 11:54 ` Michal Hocko 2015-12-18 11:54 ` Michal Hocko 2015-12-18 21:14 ` Andrew Morton 2015-12-18 21:14 ` Andrew Morton 2015-12-21 8:38 ` Michal Hocko 2015-12-21 8:38 ` Michal Hocko 2015-12-17 21:13 ` Andrew Morton 2015-12-17 21:13 ` Andrew Morton 2015-12-18 12:11 ` Michal Hocko 2015-12-18 12:11 ` Michal Hocko 2015-12-18 12:10 ` Tetsuo Handa 2015-12-18 12:10 ` Tetsuo Handa 2015-12-20 7:14 ` Tetsuo Handa 2015-12-20 7:14 ` Tetsuo Handa 2015-12-18 0:15 ` Andrew Morton 2015-12-18 0:15 ` Andrew Morton 2015-12-18 11:48 ` Michal Hocko 2015-12-18 11:48 ` Michal Hocko 2015-12-21 20:38 ` Paul Gortmaker 2015-12-21 20:38 ` Paul Gortmaker 2016-01-06 9:10 ` Michal Hocko 2016-01-06 9:10 ` Michal Hocko 2016-01-06 14:26 ` Paul Gortmaker 2016-01-06 14:26 ` Paul Gortmaker 2016-01-06 15:00 ` Michal Hocko 2016-01-06 15:00 ` Michal Hocko 2015-12-23 23:00 ` Ross Zwisler 2015-12-23 23:00 ` Ross Zwisler 2015-12-24 9:47 ` Michal Hocko 2015-12-24 9:47 ` Michal Hocko 2015-12-24 11:06 ` Tetsuo Handa 2015-12-24 11:06 ` Tetsuo Handa 2015-12-24 20:39 ` Ross Zwisler 2015-12-24 20:39 ` Ross Zwisler 2015-12-25 11:41 ` Michal Hocko 2015-12-25 11:41 ` Michal Hocko 2015-12-24 20:44 ` Ross Zwisler 2015-12-24 20:44 ` Ross Zwisler 2015-12-25 11:35 ` Michal Hocko 2015-12-25 11:35 ` Michal Hocko 2015-12-25 11:44 ` Michal Hocko 2015-12-25 11:44 ` Michal Hocko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=201602022048.GCJ04176.tOFFSVFHLMJOQO@I-love.SAKURA.ne.jp \ --to=penguin-kernel@i-love.sakura.ne.jp \ --cc=akpm@linux-foundation.org \ --cc=andrea@kernel.org \ --cc=hughd@google.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@suse.de \ --cc=mhocko@kernel.org \ --cc=oleg@redhat.com \ --cc=riel@redhat.com \ --cc=rientjes@google.com \ --cc=torvalds@linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.