From: Marc MERLIN <marc@merlins.org> To: Michal Hocko <mhocko@kernel.org> Cc: Vlastimil Babka <vbabka@suse.cz>, Linus Torvalds <torvalds@linux-foundation.org>, linux-mm <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Tejun Heo <tj@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org> Subject: Re: 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free Date: Tue, 29 Nov 2016 08:15:29 -0800 [thread overview] Message-ID: <20161129161529.wyvuxd3fpsxitag7@merlins.org> (raw) In-Reply-To: <20161128072315.GC14788@dhcp22.suse.cz> On Mon, Nov 28, 2016 at 08:23:15AM +0100, Michal Hocko wrote: > Marc, could you try this patch please? I think it should be pretty clear > it should help you but running it through your use case would be more > than welcome before I ask Greg to take this to the 4.8 stable tree. > > Thanks! > > On Wed 23-11-16 07:34:10, Michal Hocko wrote: > [...] > > commit b2ccdcb731b666aa28f86483656c39c5e53828c7 > > Author: Michal Hocko <mhocko@suse.com> > > Date: Wed Nov 23 07:26:30 2016 +0100 > > > > mm, oom: stop pre-mature high-order OOM killer invocations > > > > 31e49bfda184 ("mm, oom: protect !costly allocations some more for > > !CONFIG_COMPACTION") was an attempt to reduce chances of pre-mature OOM > > killer invocation for high order requests. It seemed to work for most > > users just fine but it is far from bullet proof and obviously not > > sufficient for Marc who has reported pre-mature OOM killer invocations > > with 4.8 based kernels. 4.9 will all the compaction improvements seems > > to be behaving much better but that would be too intrusive to backport > > to 4.8 stable kernels. Instead this patch simply never declares OOM for > > !costly high order requests. We rely on order-0 requests to do that in > > case we are really out of memory. Order-0 requests are much more common > > and so a risk of a livelock without any way forward is highly unlikely. > > > > Reported-by: Marc MERLIN <marc@merlins.org> > > Signed-off-by: Michal Hocko <mhocko@suse.com> Tested-by: Marc MERLIN <marc@merlins.org> Marc > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index a2214c64ed3c..7401e996009a 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -3161,6 +3161,16 @@ should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_fla > > if (!order || order > PAGE_ALLOC_COSTLY_ORDER) > > return false; > > > > +#ifdef CONFIG_COMPACTION > > + /* > > + * This is a gross workaround to compensate a lack of reliable compaction > > + * operation. We cannot simply go OOM with the current state of the compaction > > + * code because this can lead to pre mature OOM declaration. > > + */ > > + if (order <= PAGE_ALLOC_COSTLY_ORDER) > > + return true; > > +#endif > > + > > /* > > * There are setups with compaction disabled which would prefer to loop > > * inside the allocator rather than hit the oom killer prematurely. > > -- > > Michal Hocko > > SUSE Labs > > -- > Michal Hocko > SUSE Labs > -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
WARNING: multiple messages have this Message-ID (diff)
From: Marc MERLIN <marc@merlins.org> To: Michal Hocko <mhocko@kernel.org> Cc: Vlastimil Babka <vbabka@suse.cz>, Linus Torvalds <torvalds@linux-foundation.org>, linux-mm <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Tejun Heo <tj@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org> Subject: Re: 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free Date: Tue, 29 Nov 2016 08:15:29 -0800 [thread overview] Message-ID: <20161129161529.wyvuxd3fpsxitag7@merlins.org> (raw) In-Reply-To: <20161128072315.GC14788@dhcp22.suse.cz> On Mon, Nov 28, 2016 at 08:23:15AM +0100, Michal Hocko wrote: > Marc, could you try this patch please? I think it should be pretty clear > it should help you but running it through your use case would be more > than welcome before I ask Greg to take this to the 4.8 stable tree. > > Thanks! > > On Wed 23-11-16 07:34:10, Michal Hocko wrote: > [...] > > commit b2ccdcb731b666aa28f86483656c39c5e53828c7 > > Author: Michal Hocko <mhocko@suse.com> > > Date: Wed Nov 23 07:26:30 2016 +0100 > > > > mm, oom: stop pre-mature high-order OOM killer invocations > > > > 31e49bfda184 ("mm, oom: protect !costly allocations some more for > > !CONFIG_COMPACTION") was an attempt to reduce chances of pre-mature OOM > > killer invocation for high order requests. It seemed to work for most > > users just fine but it is far from bullet proof and obviously not > > sufficient for Marc who has reported pre-mature OOM killer invocations > > with 4.8 based kernels. 4.9 will all the compaction improvements seems > > to be behaving much better but that would be too intrusive to backport > > to 4.8 stable kernels. Instead this patch simply never declares OOM for > > !costly high order requests. We rely on order-0 requests to do that in > > case we are really out of memory. Order-0 requests are much more common > > and so a risk of a livelock without any way forward is highly unlikely. > > > > Reported-by: Marc MERLIN <marc@merlins.org> > > Signed-off-by: Michal Hocko <mhocko@suse.com> Tested-by: Marc MERLIN <marc@merlins.org> Marc > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index a2214c64ed3c..7401e996009a 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -3161,6 +3161,16 @@ should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_fla > > if (!order || order > PAGE_ALLOC_COSTLY_ORDER) > > return false; > > > > +#ifdef CONFIG_COMPACTION > > + /* > > + * This is a gross workaround to compensate a lack of reliable compaction > > + * operation. We cannot simply go OOM with the current state of the compaction > > + * code because this can lead to pre mature OOM declaration. > > + */ > > + if (order <= PAGE_ALLOC_COSTLY_ORDER) > > + return true; > > +#endif > > + > > /* > > * There are setups with compaction disabled which would prefer to loop > > * inside the allocator rather than hit the oom killer prematurely. > > -- > > Michal Hocko > > SUSE Labs > > -- > Michal Hocko > SUSE Labs > -- "A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | PGP 1024R/763BE901 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-11-29 16:16 UTC|newest] Thread overview: 113+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-11-21 15:43 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free Marc MERLIN 2016-11-21 16:30 ` Marc MERLIN 2016-11-21 21:50 ` Vlastimil Babka 2016-11-21 21:50 ` Vlastimil Babka 2016-11-21 21:56 ` Marc MERLIN 2016-11-21 21:56 ` Marc MERLIN 2016-11-21 23:03 ` [PATCH] block,blkcg: use __GFP_NOWARN for best-effort allocations in blkcg Tejun Heo 2016-11-21 23:03 ` Tejun Heo 2016-11-22 15:47 ` Vlastimil Babka 2016-11-22 15:47 ` Vlastimil Babka 2016-11-22 16:48 ` Tejun Heo 2016-11-22 16:48 ` Tejun Heo 2016-11-22 22:13 ` Linus Torvalds 2016-11-22 22:13 ` Linus Torvalds 2016-11-23 8:50 ` Vlastimil Babka 2016-11-23 8:50 ` Vlastimil Babka 2016-11-28 17:19 ` Tejun Heo 2016-11-28 17:19 ` Tejun Heo 2016-11-29 7:25 ` Michal Hocko 2016-11-29 7:25 ` Michal Hocko 2016-11-29 16:38 ` Tejun Heo 2016-11-29 16:38 ` Tejun Heo 2016-11-29 16:57 ` Vlastimil Babka 2016-11-29 16:57 ` Vlastimil Babka 2016-11-29 17:13 ` Michal Hocko 2016-11-29 17:13 ` Michal Hocko 2016-11-29 17:17 ` Linus Torvalds 2016-11-29 17:17 ` Linus Torvalds 2016-11-29 17:28 ` Michal Hocko 2016-11-29 17:28 ` Michal Hocko 2016-11-29 17:48 ` Linus Torvalds 2016-11-29 17:48 ` Linus Torvalds 2016-11-22 16:00 ` Jens Axboe 2016-11-22 16:00 ` Jens Axboe 2016-11-22 16:06 ` 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free Marc MERLIN 2016-11-22 16:06 ` Marc MERLIN 2016-11-22 16:14 ` Vlastimil Babka 2016-11-22 16:14 ` Vlastimil Babka 2016-11-22 16:25 ` Michal Hocko 2016-11-22 16:25 ` Michal Hocko 2016-11-22 16:47 ` Marc MERLIN 2016-11-22 16:47 ` Marc MERLIN 2016-11-22 16:38 ` Greg Kroah-Hartman 2016-11-22 16:38 ` Greg Kroah-Hartman 2016-11-29 16:25 ` Michal Hocko 2016-11-29 16:25 ` Michal Hocko 2016-11-29 16:25 ` Michal Hocko 2016-11-29 16:43 ` Patch "mm, oom: stop pre-mature high-order OOM killer invocations" has been added to the 4.8-stable tree gregkh 2016-11-29 16:43 ` 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free Greg Kroah-Hartman 2016-11-29 16:43 ` Greg Kroah-Hartman 2016-11-22 19:38 ` Linus Torvalds 2016-11-22 19:38 ` Linus Torvalds 2016-11-23 6:34 ` Michal Hocko 2016-11-23 6:34 ` Michal Hocko 2016-11-23 6:53 ` Hillf Danton 2016-11-23 6:53 ` Hillf Danton 2016-11-23 7:00 ` Michal Hocko 2016-11-23 7:00 ` Michal Hocko 2016-11-23 9:18 ` Vlastimil Babka 2016-11-23 9:18 ` Vlastimil Babka 2016-11-28 7:23 ` Michal Hocko 2016-11-28 7:23 ` Michal Hocko 2016-11-28 20:55 ` Marc MERLIN 2016-11-29 15:55 ` Marc MERLIN 2016-11-29 15:55 ` Marc MERLIN 2016-11-29 16:07 ` Michal Hocko 2016-11-29 16:07 ` Michal Hocko 2016-11-29 16:34 ` Marc MERLIN 2016-11-29 16:34 ` Marc MERLIN 2016-11-29 17:07 ` Linus Torvalds 2016-11-29 17:07 ` Linus Torvalds 2016-11-29 17:40 ` Marc MERLIN 2016-11-29 17:40 ` Marc MERLIN 2016-11-29 18:01 ` Linus Torvalds 2016-11-29 18:01 ` Linus Torvalds 2016-11-30 17:47 ` Marc MERLIN 2016-11-30 17:47 ` Marc MERLIN 2016-11-30 18:14 ` Linus Torvalds 2016-11-30 18:21 ` Marc MERLIN 2016-11-30 18:21 ` Marc MERLIN 2016-11-30 18:27 ` Jens Axboe 2016-11-30 18:27 ` Jens Axboe 2016-11-30 20:30 ` Tejun Heo 2016-11-30 20:30 ` Tejun Heo 2016-12-01 13:50 ` Kent Overstreet 2016-12-01 13:50 ` Kent Overstreet 2016-12-01 18:16 ` Linus Torvalds 2016-12-01 18:16 ` Linus Torvalds 2016-12-01 18:30 ` Jens Axboe 2016-12-01 18:30 ` Jens Axboe 2016-12-01 18:37 ` Linus Torvalds 2016-12-01 18:37 ` Linus Torvalds 2016-12-01 18:46 ` Jens Axboe 2016-12-01 18:46 ` Jens Axboe 2016-11-29 20:11 ` Holger Hoffstätte 2016-11-29 23:01 ` Marc MERLIN 2016-11-29 23:01 ` Marc MERLIN 2016-11-30 13:58 ` Tetsuo Handa 2016-11-30 13:58 ` Tetsuo Handa 2017-05-02 4:12 ` Marc MERLIN 2017-05-02 4:12 ` Marc MERLIN 2017-05-02 7:44 ` Michal Hocko 2017-05-02 7:44 ` Michal Hocko 2017-05-02 14:15 ` Marc MERLIN 2017-05-02 14:15 ` Marc MERLIN 2017-05-02 10:44 ` Tetsuo Handa 2017-05-02 10:44 ` Tetsuo Handa 2016-11-29 16:15 ` Marc MERLIN [this message] 2016-11-29 16:15 ` Marc MERLIN 2016-11-22 21:46 ` Simon Kirby 2016-11-22 21:46 ` Simon Kirby 2016-11-28 8:06 ` Vlastimil Babka 2016-11-28 8:06 ` Vlastimil Babka
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20161129161529.wyvuxd3fpsxitag7@merlins.org \ --to=marc@merlins.org \ --cc=gregkh@linuxfoundation.org \ --cc=iamjoonsoo.kim@lge.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mhocko@kernel.org \ --cc=tj@kernel.org \ --cc=torvalds@linux-foundation.org \ --cc=vbabka@suse.cz \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.