From: Joel Fernandes <firstname.lastname@example.org>
To: Michal Hocko <email@example.com>
Cc: Suren Baghdasaryan <firstname.lastname@example.org>,
Andrew Morton <email@example.com>,
David Rientjes <firstname.lastname@example.org>,
Matthew Wilcox <email@example.com>,
firstname.lastname@example.org, email@example.com, firstname.lastname@example.org,
Johannes Weiner <email@example.com>,
firstname.lastname@example.org, Christian Brauner <email@example.com>,
Minchan Kim <firstname.lastname@example.org>,
Tim Murray <email@example.com>,
Daniel Colascione <firstname.lastname@example.org>,
Jann Horn <email@example.com>,
"open list:MEMORY MANAGEMENT" <firstname.lastname@example.org>,
"Cc: Android Kernel" <email@example.com>
Subject: Re: [RFC 0/2] opportunistic memory reclaim of a killed process
Date: Thu, 11 Apr 2019 17:11:00 -0400 [thread overview]
Message-ID: <20190411211100.GB130334@google.com> (raw)
On Thu, Apr 11, 2019 at 10:11:51PM +0200, Michal Hocko wrote:
> On Thu 11-04-19 15:14:30, Joel Fernandes wrote:
> > On Thu, Apr 11, 2019 at 08:12:43PM +0200, Michal Hocko wrote:
> > > On Thu 11-04-19 12:18:33, Joel Fernandes wrote:
> > > > On Thu, Apr 11, 2019 at 6:51 AM Michal Hocko <firstname.lastname@example.org> wrote:
> > > > >
> > > > > On Wed 10-04-19 18:43:51, Suren Baghdasaryan wrote:
> > > > > [...]
> > > > > > Proposed solution uses existing oom-reaper thread to increase memory
> > > > > > reclaim rate of a killed process and to make this rate more deterministic.
> > > > > > By no means the proposed solution is considered the best and was chosen
> > > > > > because it was simple to implement and allowed for test data collection.
> > > > > > The downside of this solution is that it requires additional “expedite”
> > > > > > hint for something which has to be fast in all cases. Would be great to
> > > > > > find a way that does not require additional hints.
> > > > >
> > > > > I have to say I do not like this much. It is abusing an implementation
> > > > > detail of the OOM implementation and makes it an official API. Also
> > > > > there are some non trivial assumptions to be fullfilled to use the
> > > > > current oom_reaper. First of all all the process groups that share the
> > > > > address space have to be killed. How do you want to guarantee/implement
> > > > > that with a simply kill to a thread/process group?
> > > >
> > > > Will task_will_free_mem() not bail out in such cases because of
> > > > process_shares_mm() returning true?
> > >
> > > I am not really sure I understand your question. task_will_free_mem is
> > > just a shortcut to not kill anything if the current process or a victim
> > > is already dying and likely to free memory without killing or spamming
> > > the log. My concern is that this patch allows to invoke the reaper
> > Got it.
> > > without guaranteeing the same. So it can only be an optimistic attempt
> > > and then I am wondering how reasonable of an interface this really is.
> > > Userspace send the signal and has no way to find out whether the async
> > > reaping has been scheduled or not.
> > Could you clarify more what you're asking to guarantee? I cannot picture it.
> > If you mean guaranteeing that "a task is dying anyway and will free its
> > memory on its own", we are calling task_will_free_mem() to check that before
> > invoking the oom reaper.
> No, I am talking about the API aspect. Say you kall kill with the flag
> to make the async address space tear down. Now you cannot really
> guarantee that this is safe to do because the target task might
> clone(CLONE_VM) at any time. So this will be known only once the signal
> is sent, but the calling process has no way to find out. So the caller
> has no way to know what is the actual result of the requested operation.
> That is a poor API in my book.
> > Could you clarify what is the draback if OOM reaper is invoked in parallel to
> > an exiting task which will free its memory soon? It looks like the OOM reaper
> > is taking all the locks necessary (mmap_sem) in particular and is unmapping
> > pages. It seemed to me to be safe, but I am missing what are the main draw
> > backs of this - other than the intereference with core dump. One could be
> > presumably scalability since the since OOM reaper could be bottlenecked by
> > freeing memory on behalf of potentially several dying tasks.
> oom_reaper or any other kernel thread doing the same is a mere
> implementation detail I think. The oom killer doesn't really need the
> oom_reaper to act swiftly because it is there to act as a last resort if
> the oom victim cannot terminate on its own. If you want to offer an
> user space API then you can assume users will like to use it and expect
> a certain behavior but what that is? E.g. what if there are thousands of
> tasks killed this way? Do we care that some of them will not get the
> async treatment? If yes why do we need an API to control that at all?
> Am I more clear now?
Yes, your concerns are more clear now. We will think more about this and your
other responses, thanks a lot.
next prev parent reply other threads:[~2019-04-11 21:11 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-11 1:43 [RFC 0/2] opportunistic memory reclaim of a killed process Suren Baghdasaryan
2019-04-11 1:43 ` [RFC 1/2] mm: oom: expose expedite_reclaim to use oom_reaper outside of oom_kill.c Suren Baghdasaryan
2019-04-25 21:12 ` Tetsuo Handa
2019-04-25 21:56 ` Suren Baghdasaryan
2019-04-11 1:43 ` [RFC 2/2] signal: extend pidfd_send_signal() to allow expedited process killing Suren Baghdasaryan
2019-04-11 10:30 ` Christian Brauner
2019-04-11 10:34 ` Christian Brauner
2019-04-11 15:18 ` Suren Baghdasaryan
2019-04-11 15:23 ` Suren Baghdasaryan
2019-04-11 16:25 ` Daniel Colascione
2019-04-11 15:33 ` Matthew Wilcox
2019-04-11 17:05 ` Johannes Weiner
2019-04-11 17:09 ` Suren Baghdasaryan
2019-04-11 17:33 ` Daniel Colascione
2019-04-11 17:36 ` Matthew Wilcox
2019-04-11 17:47 ` Daniel Colascione
2019-04-12 6:49 ` Michal Hocko
2019-04-12 14:15 ` Suren Baghdasaryan
2019-04-12 14:20 ` Daniel Colascione
2019-04-12 21:03 ` Matthew Wilcox
2019-04-11 17:52 ` Suren Baghdasaryan
2019-04-11 21:45 ` Roman Gushchin
2019-04-11 21:59 ` Suren Baghdasaryan
2019-04-12 6:53 ` Michal Hocko
2019-04-12 14:10 ` Suren Baghdasaryan
2019-04-12 14:14 ` Daniel Colascione
2019-04-12 15:30 ` Daniel Colascione
2019-04-25 16:09 ` Suren Baghdasaryan
2019-04-11 10:51 ` [RFC 0/2] opportunistic memory reclaim of a killed process Michal Hocko
2019-04-11 16:18 ` Joel Fernandes
2019-04-11 18:12 ` Michal Hocko
2019-04-11 19:14 ` Joel Fernandes
2019-04-11 20:11 ` Michal Hocko
2019-04-11 21:11 ` Joel Fernandes [this message]
2019-04-11 16:20 ` Sandeep Patil
2019-04-11 16:47 ` Suren Baghdasaryan
2019-04-11 18:19 ` Michal Hocko
2019-04-11 19:56 ` Suren Baghdasaryan
2019-04-11 20:17 ` Michal Hocko
2019-04-11 17:19 ` Johannes Weiner
2019-04-11 11:51 ` [Lsf-pc] " Rik van Riel
2019-04-11 12:16 ` Michal Hocko
2019-04-11 16:54 ` Suren Baghdasaryan
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).