From: Michal Hocko <email@example.com> To: Tetsuo Handa <firstname.lastname@example.org> Cc: David Rientjes <email@example.com>, Roman Gushchin <firstname.lastname@example.org>, email@example.com, Andrew Morton <firstname.lastname@example.org>, LKML <email@example.com>, Linus Torvalds <firstname.lastname@example.org> Subject: Re: [RFC PATCH v2 0/3] oom: rework oom_reaper vs. exit_mmap handoff Date: Thu, 15 Nov 2018 12:36:53 +0100 [thread overview] Message-ID: <20181115113653.GO23831@dhcp22.suse.cz> (raw) In-Reply-To: <email@example.com> On Thu 15-11-18 18:54:15, Tetsuo Handa wrote: > On 2018/11/14 19:16, Michal Hocko wrote: > > On Wed 14-11-18 18:46:13, Tetsuo Handa wrote: > > [...] > > > There is always an invisible lock called "scheduling priority". You can't > > > leave the MMF_OOM_SKIP to the exit path. Your approach is not ready for > > > handling the worst case. > > > > And that problem is all over the memory reclaim. You can get starved > > to death and block other resources. And the memory reclaim is not the > > only one. > > I think that it is a manner for kernel developers that no thread keeps > consuming CPU resources forever. In the kernel world, doing > > while (1); > > is not permitted. Likewise, doing > > for (i = 0; i < very_large_value; i++) > do_something_which_does_not_yield_CPU_to_others(); There is nothing like that proposed in this series. > has to be avoided, in order to avoid lockup problems. We are required to > yield CPU to others when we are waiting for somebody else to make progress. > It is the page allocator who is refusing to yield CPU to those who need CPU. And we do that in the reclaim path. > Since the OOM reaper kernel thread "has normal priority" and "can run on any > CPU", the possibility of failing to run is lower than an OOM victim thread > which "has idle priority" and "can run on only limited CPU". You are trying > to add a dependency on such thread, and I'm saying that adding a dependency > on such thread increases possibility of lockup. Sigh. No, this is not the case. All this patch series does is that we hand over to the exiting task once it doesn't block on any locks anymore. If the thread is low priority then it is quite likely that the oom reaper is done by the time the victim even reaches the exit path. > Yes, even the OOM reaper kernel thread might fail to run if all CPUs were > busy with realtime threads waiting for the OOM reaper kernel thread to make > progress. In that case, we had better stop relying on asynchronous memory > reclaim, and switch to direct OOM reaping by allocating threads. > > But what I demonstrated is that > > /* > * the exit path is guaranteed to finish the memory tear down > * without any unbound blocking at this stage so make it clear > * to the oom_reaper > */ > > becomes a lie even when only one CPU was busy with realtime threads waiting > for an idle thread to make progress. If the page allocator stops telling a > lie that "an OOM victim is making progress on behalf of me", we can avoid > the lockup. OK, I stopped reading right here. This discussion is pointless. Once you busy loop all CPUs you are screwed. Are you going to blame a filesystem that no progress can be made if a code path holding an important lock is preemempted by high priority stuff a no further progress can be made? This is just ridiculous. What you are arguing here is not fixable with the current upstream kernel. Even your so beloved timeout based solution doesn't cope with that because oom reaper can be preempted for unbound amount of time. Your argument just doens't make much sense in the context of the current kernel. Full stop. -- Michal Hocko SUSE Labs
next prev parent reply other threads:[~2018-11-15 11:36 UTC|newest] Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-10-25 8:24 Michal Hocko 2018-10-25 8:24 ` [RFC PATCH v2 1/3] mm, oom: rework mmap_exit vs. oom_reaper synchronization Michal Hocko 2018-10-25 8:24 ` [RFC PATCH v2 2/3] mm, oom: keep retrying the oom_reap operation as long as there is substantial memory left Michal Hocko 2018-10-25 8:24 ` [RFC PATCH v2 3/3] mm, oom: hand over MMF_OOM_SKIP to exit path if it is guranteed to finish Michal Hocko 2018-10-30 4:45 ` Tetsuo Handa 2018-10-30 6:31 ` Michal Hocko 2018-10-30 9:47 ` Tetsuo Handa 2018-10-30 11:39 ` Michal Hocko 2018-10-30 12:02 ` Tetsuo Handa 2018-10-30 12:10 ` Michal Hocko 2018-10-30 13:57 ` Tetsuo Handa 2018-10-30 14:23 ` Michal Hocko 2018-11-08 9:32 ` [RFC PATCH v2 0/3] oom: rework oom_reaper vs. exit_mmap handoff Michal Hocko 2018-11-14 9:46 ` Tetsuo Handa 2018-11-14 10:16 ` Michal Hocko 2018-11-15 9:54 ` Tetsuo Handa 2018-11-15 11:36 ` Michal Hocko [this message] 2018-11-16 10:06 ` Tetsuo Handa
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181115113653.GO23831@dhcp22.suse.cz \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [RFC PATCH v2 0/3] oom: rework oom_reaper vs. exit_mmap handoff' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).