From: Michal Hocko <mhocko@kernel.org>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Greg Thelen <gthelen@google.com>,
Shakeel Butt <shakeelb@google.com>,
Alexander Viro <viro@zeniv.linux.org.uk>,
Vladimir Davydov <vdavydov.dev@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Linux MM <linux-mm@kvack.org>,
linux-fsdevel@vger.kernel.org,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] fs, mm: account filp and names caches to kmemcg
Date: Fri, 13 Oct 2017 08:35:55 +0200 [thread overview]
Message-ID: <20171013063555.pa7uco43mod7vrkn@dhcp22.suse.cz> (raw)
In-Reply-To: <20171012190312.GA5075@cmpxchg.org>
On Thu 12-10-17 15:03:12, Johannes Weiner wrote:
> On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote:
[...]
> > And we will simply mark the victim MMF_OOM_SKIP and hide it from the oom
> > killer if we fail to get the mmap_sem after several attempts. This will
> > allow to find a new victim. So we shouldn't deadlock.
>
> It's less likely to deadlock, but not exactly deadlock-free. There
> might not BE any other mm's holding significant amounts of memory.
true, try_charge would have to return with failure when out_of_memory
returns with false of course.
> > > What do you mean by 'v2'?
> >
> > cgroup v2 because the legacy memcg allowed sync wait for the oom killer
> > and that would be a bigger problem from a deep callchains for obevious
> > reasons.
>
> Actually, the async oom killing code isn't dependent on cgroup
> version. cgroup1 doesn't wait inside the charge context, either.
Sorry, I was just not clear. What I meant to say, would couldn't make v1
wait inside the try_charge path because async oom killing wouldn't help
for the oom disabled case (aka user space oom handling).
> > > > > > c) Overcharge kmem to oom memcg and queue an async memcg limit checker,
> > > > > > which will oom kill if needed.
> > > > >
> > > > > This makes the most sense to me. Architecturally, I imagine this would
> > > > > look like b), with an OOM handler at the point of return to userspace,
> > > > > except that we'd overcharge instead of retrying the syscall.
> > > >
> > > > I do not think we should break the hard limit semantic if possible. We
> > > > can currently allow that for allocations which are very short term (oom
> > > > victims) or too important to fail but allowing that for kmem charges in
> > > > general sounds like too easy to runaway.
> > >
> > > I'm not sure there is a convenient way out of this.
> > >
> > > If we want to respect the hard limit AND guarantee allocation success,
> > > the OOM killer has to free memory reliably - which it doesn't. But if
> > > it did, we could also break the limit temporarily and have the OOM
> > > killer replenish the pool before that userspace app can continue. The
> > > allocation wouldn't have to be short-lived, since memory is fungible.
> >
> > If we can guarantee the oom killer is started then we can allow temporal
> > access to reserves which is already implemented even for memcg. The
> > thing is we do not invoke the oom killer...
>
> You lost me here. Which reserves?
>
> All I'm saying is that, when the syscall-context fails to charge, we
> should do mem_cgroup_oom() to set up the async OOM killer, let the
> charge succeed over the hard limit - since the OOM killer will most
> likely get us back below the limit - then mem_cgroup_oom_synchronize()
> before the syscall returns to userspace.
OK, then we are on the same page now. Your initial wording didn't
mention async OOM killer. This makes more sense. Although I would argue
that we can retry the charge as long as out_of_memory finds a victim.
This would return ENOMEM to the pathological cases where no victims
could be found.
--
Michal Hocko
SUSE Labs
next prev parent reply other threads:[~2017-10-13 6:35 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-05 22:21 [PATCH] fs, mm: account filp and names caches to kmemcg Shakeel Butt
2017-10-06 7:59 ` Michal Hocko
2017-10-06 19:33 ` Shakeel Butt
2017-10-09 6:24 ` Michal Hocko
2017-10-09 17:52 ` Greg Thelen
2017-10-09 18:04 ` Michal Hocko
2017-10-09 18:17 ` Michal Hocko
2017-10-10 9:10 ` Michal Hocko
2017-10-10 22:21 ` Shakeel Butt
2017-10-11 9:09 ` Michal Hocko
2017-10-09 20:26 ` Johannes Weiner
2017-10-10 9:14 ` Michal Hocko
2017-10-10 14:17 ` Johannes Weiner
2017-10-10 14:24 ` Michal Hocko
2017-10-12 19:03 ` Johannes Weiner
2017-10-12 23:57 ` Greg Thelen
2017-10-13 6:51 ` Michal Hocko
2017-10-13 6:35 ` Michal Hocko [this message]
2017-10-13 7:00 ` Michal Hocko
2017-10-13 15:24 ` Michal Hocko
2017-10-24 12:18 ` Michal Hocko
2017-10-24 17:54 ` Johannes Weiner
2017-10-24 16:06 ` Johannes Weiner
2017-10-24 16:22 ` Michal Hocko
2017-10-24 17:23 ` Johannes Weiner
2017-10-24 17:55 ` Michal Hocko
2017-10-24 18:58 ` Johannes Weiner
2017-10-24 20:15 ` Michal Hocko
2017-10-25 6:51 ` Greg Thelen
2017-10-25 7:15 ` Michal Hocko
2017-10-25 13:11 ` Johannes Weiner
2017-10-25 14:12 ` Michal Hocko
2017-10-25 16:44 ` Johannes Weiner
2017-10-25 17:29 ` Michal Hocko
2017-10-25 18:11 ` Johannes Weiner
2017-10-25 19:00 ` Michal Hocko
2017-10-25 21:13 ` Johannes Weiner
2017-10-25 22:49 ` Greg Thelen
2017-10-26 7:49 ` Michal Hocko
2017-10-26 12:45 ` Tetsuo Handa
2017-10-26 14:31 ` Johannes Weiner
2017-10-26 19:56 ` Greg Thelen
2017-10-27 8:20 ` Michal Hocko
2017-10-27 20:50 ` Shakeel Butt
2017-10-30 8:29 ` Michal Hocko
2017-10-30 19:28 ` Shakeel Butt
2017-10-31 8:00 ` Michal Hocko
2017-10-31 16:49 ` Johannes Weiner
2017-10-31 18:50 ` Michal Hocko
2017-10-24 15:45 ` Johannes Weiner
2017-10-24 16:30 ` Michal Hocko
2017-10-10 23:32 ` Al Viro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171013063555.pa7uco43mod7vrkn@dhcp22.suse.cz \
--to=mhocko@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=gthelen@google.com \
--cc=hannes@cmpxchg.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=shakeelb@google.com \
--cc=vdavydov.dev@gmail.com \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).