From: Michal Hocko <firstname.lastname@example.org>
To: Joel Fernandes <email@example.com>
Cc: Suren Baghdasaryan <firstname.lastname@example.org>,
Andrew Morton <email@example.com>,
David Rientjes <firstname.lastname@example.org>,
Matthew Wilcox <email@example.com>,
firstname.lastname@example.org, email@example.com, firstname.lastname@example.org,
Johannes Weiner <email@example.com>,
firstname.lastname@example.org, Christian Brauner <email@example.com>,
Minchan Kim <firstname.lastname@example.org>,
Tim Murray <email@example.com>,
Daniel Colascione <firstname.lastname@example.org>,
Jann Horn <email@example.com>,
"open list:MEMORY MANAGEMENT" <firstname.lastname@example.org>,
"Cc: Android Kernel" <email@example.com>
Subject: Re: [RFC 0/2] opportunistic memory reclaim of a killed process
Date: Thu, 11 Apr 2019 22:11:51 +0200 [thread overview]
Message-ID: <20190411201151.GA4743@dhcp22.suse.cz> (raw)
On Thu 11-04-19 15:14:30, Joel Fernandes wrote:
> On Thu, Apr 11, 2019 at 08:12:43PM +0200, Michal Hocko wrote:
> > On Thu 11-04-19 12:18:33, Joel Fernandes wrote:
> > > On Thu, Apr 11, 2019 at 6:51 AM Michal Hocko <firstname.lastname@example.org> wrote:
> > > >
> > > > On Wed 10-04-19 18:43:51, Suren Baghdasaryan wrote:
> > > > [...]
> > > > > Proposed solution uses existing oom-reaper thread to increase memory
> > > > > reclaim rate of a killed process and to make this rate more deterministic.
> > > > > By no means the proposed solution is considered the best and was chosen
> > > > > because it was simple to implement and allowed for test data collection.
> > > > > The downside of this solution is that it requires additional “expedite”
> > > > > hint for something which has to be fast in all cases. Would be great to
> > > > > find a way that does not require additional hints.
> > > >
> > > > I have to say I do not like this much. It is abusing an implementation
> > > > detail of the OOM implementation and makes it an official API. Also
> > > > there are some non trivial assumptions to be fullfilled to use the
> > > > current oom_reaper. First of all all the process groups that share the
> > > > address space have to be killed. How do you want to guarantee/implement
> > > > that with a simply kill to a thread/process group?
> > >
> > > Will task_will_free_mem() not bail out in such cases because of
> > > process_shares_mm() returning true?
> > I am not really sure I understand your question. task_will_free_mem is
> > just a shortcut to not kill anything if the current process or a victim
> > is already dying and likely to free memory without killing or spamming
> > the log. My concern is that this patch allows to invoke the reaper
> Got it.
> > without guaranteeing the same. So it can only be an optimistic attempt
> > and then I am wondering how reasonable of an interface this really is.
> > Userspace send the signal and has no way to find out whether the async
> > reaping has been scheduled or not.
> Could you clarify more what you're asking to guarantee? I cannot picture it.
> If you mean guaranteeing that "a task is dying anyway and will free its
> memory on its own", we are calling task_will_free_mem() to check that before
> invoking the oom reaper.
No, I am talking about the API aspect. Say you kall kill with the flag
to make the async address space tear down. Now you cannot really
guarantee that this is safe to do because the target task might
clone(CLONE_VM) at any time. So this will be known only once the signal
is sent, but the calling process has no way to find out. So the caller
has no way to know what is the actual result of the requested operation.
That is a poor API in my book.
> Could you clarify what is the draback if OOM reaper is invoked in parallel to
> an exiting task which will free its memory soon? It looks like the OOM reaper
> is taking all the locks necessary (mmap_sem) in particular and is unmapping
> pages. It seemed to me to be safe, but I am missing what are the main draw
> backs of this - other than the intereference with core dump. One could be
> presumably scalability since the since OOM reaper could be bottlenecked by
> freeing memory on behalf of potentially several dying tasks.
oom_reaper or any other kernel thread doing the same is a mere
implementation detail I think. The oom killer doesn't really need the
oom_reaper to act swiftly because it is there to act as a last resort if
the oom victim cannot terminate on its own. If you want to offer an
user space API then you can assume users will like to use it and expect
a certain behavior but what that is? E.g. what if there are thousands of
tasks killed this way? Do we care that some of them will not get the
async treatment? If yes why do we need an API to control that at all?
Am I more clear now?
next prev parent reply other threads:[~2019-04-11 20:11 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-11 1:43 [RFC 0/2] opportunistic memory reclaim of a killed process Suren Baghdasaryan
2019-04-11 1:43 ` [RFC 1/2] mm: oom: expose expedite_reclaim to use oom_reaper outside of oom_kill.c Suren Baghdasaryan
2019-04-25 21:12 ` Tetsuo Handa
2019-04-25 21:56 ` Suren Baghdasaryan
2019-04-11 1:43 ` [RFC 2/2] signal: extend pidfd_send_signal() to allow expedited process killing Suren Baghdasaryan
2019-04-11 10:30 ` Christian Brauner
2019-04-11 10:34 ` Christian Brauner
2019-04-11 15:18 ` Suren Baghdasaryan
2019-04-11 15:23 ` Suren Baghdasaryan
2019-04-11 16:25 ` Daniel Colascione
2019-04-11 15:33 ` Matthew Wilcox
2019-04-11 17:05 ` Johannes Weiner
2019-04-11 17:09 ` Suren Baghdasaryan
2019-04-11 17:33 ` Daniel Colascione
2019-04-11 17:36 ` Matthew Wilcox
2019-04-11 17:47 ` Daniel Colascione
2019-04-12 6:49 ` Michal Hocko
2019-04-12 14:15 ` Suren Baghdasaryan
2019-04-12 14:20 ` Daniel Colascione
2019-04-12 21:03 ` Matthew Wilcox
2019-04-11 17:52 ` Suren Baghdasaryan
2019-04-11 21:45 ` Roman Gushchin
2019-04-11 21:59 ` Suren Baghdasaryan
2019-04-12 6:53 ` Michal Hocko
2019-04-12 14:10 ` Suren Baghdasaryan
2019-04-12 14:14 ` Daniel Colascione
2019-04-12 15:30 ` Daniel Colascione
2019-04-25 16:09 ` Suren Baghdasaryan
2019-04-11 10:51 ` [RFC 0/2] opportunistic memory reclaim of a killed process Michal Hocko
2019-04-11 16:18 ` Joel Fernandes
2019-04-11 18:12 ` Michal Hocko
2019-04-11 19:14 ` Joel Fernandes
2019-04-11 20:11 ` Michal Hocko [this message]
2019-04-11 21:11 ` Joel Fernandes
2019-04-11 16:20 ` Sandeep Patil
2019-04-11 16:47 ` Suren Baghdasaryan
2019-04-11 18:19 ` Michal Hocko
2019-04-11 19:56 ` Suren Baghdasaryan
2019-04-11 20:17 ` Michal Hocko
2019-04-11 17:19 ` Johannes Weiner
2019-04-11 11:51 ` [Lsf-pc] " Rik van Riel
2019-04-11 12:16 ` Michal Hocko
2019-04-11 16:54 ` Suren Baghdasaryan
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).