From: Michal Hocko <mhocko@kernel.org>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Nils Holland <nholland@tisys.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Chris Mason <clm@fb.com>, David Sterba <dsterba@suse.cz>,
linux-btrfs@vger.kernel.org
Subject: Re: [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically
Date: Fri, 16 Dec 2016 23:12:02 +0100 [thread overview]
Message-ID: <20161216221202.GE7645@dhcp22.suse.cz> (raw)
In-Reply-To: <20161216173151.GA23182@cmpxchg.org>
On Fri 16-12-16 12:31:51, Johannes Weiner wrote:
> On Fri, Dec 16, 2016 at 04:58:08PM +0100, Michal Hocko wrote:
> > @@ -1013,7 +1013,7 @@ bool out_of_memory(struct oom_control *oc)
> > * make sure exclude 0 mask - all other users should have at least
> > * ___GFP_DIRECT_RECLAIM to get here.
> > */
> > - if (oc->gfp_mask && !(oc->gfp_mask & (__GFP_FS|__GFP_NOFAIL)))
> > + if (oc->gfp_mask && !(oc->gfp_mask & __GFP_FS))
> > return true;
>
> This makes sense, we should go back to what we had here. Because it's
> not that the reported OOMs are premature - there is genuinely no more
> memory reclaimable from the allocating context - but that this class
> of allocations should never invoke the OOM killer in the first place.
agreed, at least not with the current implementtion. If we had a proper
accounting where we know that the memory pinned by the fs is not really
there then we could invoke the oom killer and be safe
> > @@ -3737,6 +3752,16 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
> > */
> > WARN_ON_ONCE(order > PAGE_ALLOC_COSTLY_ORDER);
> >
> > + /*
> > + * Help non-failing allocations by giving them access to memory
> > + * reserves but do not use ALLOC_NO_WATERMARKS because this
> > + * could deplete whole memory reserves which would just make
> > + * the situation worse
> > + */
> > + page = __alloc_pages_cpuset_fallback(gfp_mask, order, ALLOC_HARDER, ac);
> > + if (page)
> > + goto got_pg;
> > +
>
> But this should be a separate patch, IMO.
>
> Do we observe GFP_NOFS lockups when we don't do this?
this is hard to tell but considering users like grow_dev_page we can get
stuck with a very slow progress I believe. Those allocations could see
some help.
> Don't we risk
> premature exhaustion of the memory reserves, and it's better to wait
> for other reclaimers to make some progress instead?
waiting for other reclaimers would be preferable but we should at least
give these some priority, which is what ALLOC_HARDER should help with.
> Should we give
> reserve access to all GFP_NOFS allocations, or just the ones from a
> reclaim/cleaning context?
I would focus only for those which are important enough. Which are those
is a harder question. But certainly those with GFP_NOFAIL are important
enough.
> All that should go into the changelog of a separate allocation booster
> patch, I think.
The reason I did both in the same patch is to address the concern about
potential lockups when NOFS|NOFAIL cannot make any progress. I've chosen
ALLOC_HARDER to give the minimum portion of the reserves so that we do
not risk other high priority users to be blocked out but still help a
bit at least and prevent from starvation when other reclaimers are
faster to consume the reclaimed memory.
I can extend the changelog of course but I believe that having both
changes together makes some sense. NOFS|NOFAIL allocations are not all
that rare and sometimes we really depend on them making a further
progress.
--
Michal Hocko
SUSE Labs
next prev parent reply other threads:[~2016-12-16 22:12 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-15 22:57 OOM: Better, but still there on 4.9 Nils Holland
2016-12-16 7:39 ` Michal Hocko
2016-12-16 15:58 ` OOM: Better, but still there on Michal Hocko
2016-12-16 15:58 ` [PATCH 1/2] mm: consolidate GFP_NOFAIL checks in the allocator slowpath Michal Hocko
2016-12-16 15:58 ` [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically Michal Hocko
2016-12-16 17:31 ` Johannes Weiner
2016-12-16 22:12 ` Michal Hocko [this message]
2016-12-17 11:17 ` Tetsuo Handa
2016-12-18 16:37 ` Michal Hocko
2016-12-16 18:47 ` OOM: Better, but still there on Nils Holland
2016-12-17 0:02 ` Michal Hocko
2016-12-17 12:59 ` Nils Holland
2016-12-17 14:44 ` Tetsuo Handa
2016-12-17 17:11 ` Nils Holland
2016-12-17 21:06 ` Nils Holland
2016-12-18 5:14 ` Tetsuo Handa
2016-12-19 13:45 ` Michal Hocko
2016-12-20 2:08 ` Nils Holland
2016-12-21 7:36 ` Michal Hocko
2016-12-21 11:00 ` Tetsuo Handa
2016-12-21 11:16 ` Michal Hocko
2016-12-21 14:04 ` Chris Mason
2016-12-22 10:10 ` Nils Holland
2016-12-22 10:27 ` Michal Hocko
2016-12-22 10:35 ` Nils Holland
2016-12-22 10:46 ` Tetsuo Handa
2016-12-22 19:17 ` Michal Hocko
2016-12-22 21:46 ` Nils Holland
2016-12-23 10:51 ` Michal Hocko
2016-12-23 12:18 ` Nils Holland
2016-12-23 12:57 ` Michal Hocko
2016-12-23 14:47 ` [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Michal Hocko
2016-12-23 22:26 ` Nils Holland
2016-12-26 12:48 ` Michal Hocko
2016-12-26 18:57 ` Nils Holland
2016-12-27 8:08 ` Michal Hocko
2016-12-27 11:23 ` Nils Holland
2016-12-27 11:27 ` Michal Hocko
2016-12-27 15:55 ` Michal Hocko
2016-12-27 16:28 ` [PATCH] mm, vmscan: consider eligible zones in get_scan_count kbuild test robot
2016-12-28 8:51 ` Michal Hocko
2016-12-27 19:33 ` [RFC PATCH] mm, memcg: fix (Re: OOM: Better, but still there on) Nils Holland
2016-12-28 8:57 ` Michal Hocko
2016-12-29 1:20 ` Minchan Kim
2016-12-29 9:04 ` Michal Hocko
2016-12-30 2:05 ` Minchan Kim
2016-12-30 10:40 ` Michal Hocko
2016-12-29 0:31 ` Minchan Kim
2016-12-29 0:48 ` Minchan Kim
2016-12-29 8:52 ` Michal Hocko
2016-12-30 10:19 ` Mel Gorman
2016-12-30 11:05 ` Michal Hocko
2016-12-30 12:43 ` Mel Gorman
2016-12-25 22:25 ` [lkp-developer] [mm, memcg] d18e2b2aca: WARNING:at_mm/memcontrol.c:#mem_cgroup_update_lru_size kernel test robot
2016-12-26 12:26 ` Michal Hocko
2016-12-26 12:50 ` Michal Hocko
2016-12-18 0:28 ` OOM: Better, but still there on Xin Zhou
2016-12-16 18:15 ` OOM: Better, but still there on 4.9 Chris Mason
2016-12-16 22:14 ` Michal Hocko
2016-12-16 22:47 ` Chris Mason
2016-12-16 23:31 ` Michal Hocko
2016-12-16 19:50 ` Chris Mason
-- strict thread matches above, loose matches on Subject: below --
2016-12-01 15:25 [PATCH 0/2] GFP_NOFAIL cleanups Michal Hocko
2016-12-01 15:25 ` [PATCH 2/2] mm, oom: do not enfore OOM killer for __GFP_NOFAIL automatically Michal Hocko
2016-12-02 7:23 ` Vlastimil Babka
2016-12-05 13:45 ` Tetsuo Handa
2016-12-05 14:10 ` Michal Hocko
2016-12-06 8:27 ` Michal Hocko
2016-12-06 10:38 ` Tetsuo Handa
2016-12-06 11:03 ` Vlastimil Babka
2016-12-06 19:25 ` Michal Hocko
2016-12-06 19:22 ` Michal Hocko
2016-12-08 12:53 ` Tetsuo Handa
2016-12-08 13:47 ` Michal Hocko
2016-12-11 11:23 ` Tetsuo Handa
2016-12-11 13:53 ` Tetsuo Handa
2016-12-12 8:52 ` Michal Hocko
2016-12-12 8:48 ` Michal Hocko
2016-12-14 10:34 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161216221202.GE7645@dhcp22.suse.cz \
--to=mhocko@kernel.org \
--cc=clm@fb.com \
--cc=dsterba@suse.cz \
--cc=hannes@cmpxchg.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=nholland@tisys.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).