All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mikulas Patocka <mpatocka@redhat.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: rientjes@google.com, mhocko@kernel.org, okozina@redhat.com,
	jmarchan@redhat.com, skozina@redhat.com, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, dm-devel@redhat.com
Subject: Re: System freezes after OOM
Date: Thu, 14 Jul 2016 08:29:32 -0400 (EDT)	[thread overview]
Message-ID: <alpine.LRH.2.02.1607140827330.15554@file01.intranet.prod.int.rdu2.redhat.com> (raw)
In-Reply-To: <201607142001.BJD07258.SMOHFOJVtLFOQF@I-love.SAKURA.ne.jp>



On Thu, 14 Jul 2016, Tetsuo Handa wrote:

> Michal Hocko wrote:
> > OK, this is the part I have missed. I didn't realize that the swapout
> > path, which is indeed PF_MEMALLOC, can get down to blk code which uses
> > mempools. A quick code travers shows that at least
> > 	make_request_fn = blk_queue_bio
> > 	blk_queue_bio
> > 	  get_request
> > 	    __get_request
> > 
> > might do that. And in that case I agree that the above mentioned patch
> > has unintentional side effects and should be re-evaluated. David, what
> > do you think? An obvious fixup would be considering TIF_MEMDIE in
> > mempool_alloc explicitly.
> 
> TIF_MEMDIE is racy. Since the OOM killer sets TIF_MEMDIE on only one thread,
> there is no guarantee that TIF_MEMDIE is set to the thread which is looping
> inside mempool_alloc().

If the device mapper subsystem is not returning objects to the mempool, it 
should be investigated as a bug in the device mapper.

There is no need to add workarounds to mempool_alloc to work around that 
bug.

Mikulas

> And since __GFP_NORETRY is used (regardless of
> f9054c70d28bc214), out_of_memory() is not called via __alloc_pages_may_oom().
> This means that the thread which is looping inside mempool_alloc() can't
> get TIF_MEMDIE unless TIF_MEMDIE is set by the OOM killer.
> 
> Maybe set __GFP_NOMEMALLOC by default at mempool_alloc() and remove it
> at mempool_alloc() when fatal_signal_pending() is true? But that behavior
> can OOM-kill somebody else when current was not OOM-killed. Sigh...
> 
> David Rientjes wrote:
> > On Wed, 13 Jul 2016, Mikulas Patocka wrote:
> > 
> > > What are the real problems that f9054c70d28bc214b2857cf8db8269f4f45a5e23 
> > > tries to fix?
> > > 
> > 
> > It prevents the whole system from livelocking due to an oom killed process 
> > stalling forever waiting for mempool_alloc() to return.  No other threads 
> > may be oom killed while waiting for it to exit.
> 
> Is that concern still valid? We have the OOM reaper for CONFIG_MMU=y case.
> 

WARNING: multiple messages have this Message-ID (diff)
From: Mikulas Patocka <mpatocka@redhat.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: rientjes@google.com, mhocko@kernel.org, okozina@redhat.com,
	jmarchan@redhat.com, skozina@redhat.com, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, dm-devel@redhat.com
Subject: Re: System freezes after OOM
Date: Thu, 14 Jul 2016 08:29:32 -0400 (EDT)	[thread overview]
Message-ID: <alpine.LRH.2.02.1607140827330.15554@file01.intranet.prod.int.rdu2.redhat.com> (raw)
In-Reply-To: <201607142001.BJD07258.SMOHFOJVtLFOQF@I-love.SAKURA.ne.jp>



On Thu, 14 Jul 2016, Tetsuo Handa wrote:

> Michal Hocko wrote:
> > OK, this is the part I have missed. I didn't realize that the swapout
> > path, which is indeed PF_MEMALLOC, can get down to blk code which uses
> > mempools. A quick code travers shows that at least
> > 	make_request_fn = blk_queue_bio
> > 	blk_queue_bio
> > 	  get_request
> > 	    __get_request
> > 
> > might do that. And in that case I agree that the above mentioned patch
> > has unintentional side effects and should be re-evaluated. David, what
> > do you think? An obvious fixup would be considering TIF_MEMDIE in
> > mempool_alloc explicitly.
> 
> TIF_MEMDIE is racy. Since the OOM killer sets TIF_MEMDIE on only one thread,
> there is no guarantee that TIF_MEMDIE is set to the thread which is looping
> inside mempool_alloc().

If the device mapper subsystem is not returning objects to the mempool, it 
should be investigated as a bug in the device mapper.

There is no need to add workarounds to mempool_alloc to work around that 
bug.

Mikulas

> And since __GFP_NORETRY is used (regardless of
> f9054c70d28bc214), out_of_memory() is not called via __alloc_pages_may_oom().
> This means that the thread which is looping inside mempool_alloc() can't
> get TIF_MEMDIE unless TIF_MEMDIE is set by the OOM killer.
> 
> Maybe set __GFP_NOMEMALLOC by default at mempool_alloc() and remove it
> at mempool_alloc() when fatal_signal_pending() is true? But that behavior
> can OOM-kill somebody else when current was not OOM-killed. Sigh...
> 
> David Rientjes wrote:
> > On Wed, 13 Jul 2016, Mikulas Patocka wrote:
> > 
> > > What are the real problems that f9054c70d28bc214b2857cf8db8269f4f45a5e23 
> > > tries to fix?
> > > 
> > 
> > It prevents the whole system from livelocking due to an oom killed process 
> > stalling forever waiting for mempool_alloc() to return.  No other threads 
> > may be oom killed while waiting for it to exit.
> 
> Is that concern still valid? We have the OOM reaper for CONFIG_MMU=y case.
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2016-07-14 12:29 UTC|newest]

Thread overview: 136+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <57837CEE.1010609@redhat.com>
     [not found] ` <f80dc690-7e71-26b2-59a2-5a1557d26713@redhat.com>
     [not found]   ` <9be09452-de7f-d8be-fd5d-4a80d1cd1ba3@redhat.com>
2016-07-11 15:43     ` System freezes after OOM Mikulas Patocka
2016-07-11 15:43       ` Mikulas Patocka
2016-07-12  6:49       ` Michal Hocko
2016-07-12  6:49         ` Michal Hocko
2016-07-12 23:44         ` Mikulas Patocka
2016-07-12 23:44         ` Mikulas Patocka
2016-07-12 23:44           ` Mikulas Patocka
2016-07-13  8:35           ` Jerome Marchand
2016-07-13  8:35           ` Jerome Marchand
2016-07-13 11:14             ` Michal Hocko
2016-07-13 11:14             ` Michal Hocko
2016-07-13 11:14               ` Michal Hocko
2016-07-13 14:21               ` Mikulas Patocka
2016-07-13 14:21                 ` Mikulas Patocka
2016-07-13 11:10           ` Michal Hocko
2016-07-13 11:10           ` Michal Hocko
2016-07-13 11:10             ` Michal Hocko
2016-07-13 12:50             ` Michal Hocko
2016-07-13 12:50               ` Michal Hocko
2016-07-13 13:44               ` Milan Broz
2016-07-13 13:44                 ` Milan Broz
2016-07-13 15:21                 ` Mikulas Patocka
2016-07-13 15:21                   ` Mikulas Patocka
2016-07-14  9:09                   ` Michal Hocko
2016-07-14  9:09                     ` Michal Hocko
2016-07-14  9:46                     ` Milan Broz
2016-07-14  9:46                       ` Milan Broz
2016-07-13 12:50             ` Michal Hocko
2016-07-13 15:02             ` Mikulas Patocka
2016-07-13 15:02             ` Mikulas Patocka
2016-07-13 15:02               ` Mikulas Patocka
2016-07-14 10:51               ` [dm-devel] " Ondrej Kozina
2016-07-14 10:51                 ` Ondrej Kozina
2016-07-14 12:51               ` Michal Hocko
2016-07-14 12:51                 ` Michal Hocko
2016-07-14 14:00                 ` Mikulas Patocka
2016-07-14 14:00                   ` Mikulas Patocka
2016-07-14 14:59                   ` Michal Hocko
2016-07-14 14:59                     ` Michal Hocko
2016-07-14 15:25                     ` Ondrej Kozina
2016-07-14 15:25                       ` Ondrej Kozina
2016-07-14 17:35                     ` Mikulas Patocka
2016-07-14 17:35                       ` Mikulas Patocka
2016-07-15  8:35                       ` Michal Hocko
2016-07-15  8:35                         ` Michal Hocko
2016-07-15 12:11                         ` Mikulas Patocka
2016-07-15 12:11                           ` Mikulas Patocka
2016-07-15 12:22                           ` Michal Hocko
2016-07-15 12:22                             ` Michal Hocko
2016-07-15 17:02                             ` Mikulas Patocka
2016-07-15 17:02                               ` Mikulas Patocka
2016-07-18  7:22                               ` Michal Hocko
2016-07-18  7:22                                 ` Michal Hocko
2016-07-14 14:08                 ` Ondrej Kozina
2016-07-14 14:08                   ` Ondrej Kozina
2016-07-14 14:08                   ` Ondrej Kozina
2016-07-14 15:31                   ` Michal Hocko
2016-07-14 15:31                     ` Michal Hocko
2016-07-14 17:07                     ` Ondrej Kozina
2016-07-14 17:07                       ` Ondrej Kozina
2016-07-14 17:36                       ` Michal Hocko
2016-07-14 17:36                         ` Michal Hocko
2016-07-14 17:39                         ` Michal Hocko
2016-07-14 17:39                           ` Michal Hocko
2016-07-15 11:42                       ` Tetsuo Handa
2016-07-15 11:42                         ` Tetsuo Handa
2016-07-13 13:19           ` Tetsuo Handa
2016-07-13 13:19             ` Tetsuo Handa
2016-07-13 13:39             ` Michal Hocko
2016-07-13 13:39               ` Michal Hocko
2016-07-13 14:18               ` Mikulas Patocka
2016-07-13 14:18               ` Mikulas Patocka
2016-07-13 14:18                 ` Mikulas Patocka
2016-07-13 14:56                 ` Michal Hocko
2016-07-13 14:56                   ` Michal Hocko
2016-07-13 15:11                   ` Mikulas Patocka
2016-07-13 15:11                     ` Mikulas Patocka
2016-07-13 23:53                     ` David Rientjes
2016-07-13 23:53                       ` David Rientjes
2016-07-14 11:01                       ` Tetsuo Handa
2016-07-14 11:01                         ` Tetsuo Handa
2016-07-14 12:29                         ` Mikulas Patocka [this message]
2016-07-14 12:29                           ` Mikulas Patocka
2016-07-14 20:26                         ` David Rientjes
2016-07-14 20:26                           ` David Rientjes
2016-07-14 21:40                           ` Tetsuo Handa
2016-07-14 21:40                             ` Tetsuo Handa
2016-07-14 22:04                             ` David Rientjes
2016-07-14 22:04                               ` David Rientjes
2016-07-15 11:25                           ` Mikulas Patocka
2016-07-15 11:25                             ` Mikulas Patocka
2016-07-15 21:21                             ` David Rientjes
2016-07-15 21:21                               ` David Rientjes
2016-07-14 11:01                       ` Tetsuo Handa
2016-07-14 12:27                       ` Mikulas Patocka
2016-07-14 12:27                         ` Mikulas Patocka
2016-07-14 20:22                         ` David Rientjes
2016-07-14 20:22                           ` David Rientjes
2016-07-15 11:21                           ` Mikulas Patocka
2016-07-15 11:21                             ` Mikulas Patocka
2016-07-15 21:25                             ` David Rientjes
2016-07-15 21:25                               ` David Rientjes
2016-07-15 21:39                               ` Mikulas Patocka
2016-07-15 21:39                                 ` Mikulas Patocka
2016-07-15 21:58                                 ` David Rientjes
2016-07-15 21:58                                   ` David Rientjes
2016-07-15 23:53                                   ` Mikulas Patocka
2016-07-15 23:53                                     ` Mikulas Patocka
2016-07-18 15:14                             ` Johannes Weiner
2016-07-18 15:14                               ` Johannes Weiner
2016-07-14 15:29                       ` Michal Hocko
2016-07-14 15:29                         ` Michal Hocko
2016-07-14 20:38                         ` David Rientjes
2016-07-14 20:38                           ` David Rientjes
2016-07-15  7:22                           ` Michal Hocko
2016-07-15  7:22                             ` Michal Hocko
2016-07-15  8:23                             ` Michal Hocko
2016-07-15  8:23                               ` Michal Hocko
2016-07-15 12:00                             ` Mikulas Patocka
2016-07-15 12:00                               ` Mikulas Patocka
2016-07-15 21:47                             ` David Rientjes
2016-07-15 21:47                               ` David Rientjes
2016-07-18  7:39                               ` Michal Hocko
2016-07-18  7:39                                 ` Michal Hocko
2016-07-18 21:03                                 ` David Rientjes
2016-07-18 21:03                                   ` David Rientjes
2016-07-13 23:53                     ` David Rientjes
2016-07-13 15:11                   ` Mikulas Patocka
2016-07-13 14:56                 ` Michal Hocko
2016-07-13 13:39             ` Michal Hocko
2016-07-14  0:01             ` David Rientjes
2016-07-14  0:01               ` David Rientjes
2016-07-14  0:01             ` David Rientjes
2016-07-13 13:19           ` Tetsuo Handa
2016-07-12  6:49       ` Michal Hocko
2016-07-11 15:43     ` Mikulas Patocka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.LRH.2.02.1607140827330.15554@file01.intranet.prod.int.rdu2.redhat.com \
    --to=mpatocka@redhat.com \
    --cc=dm-devel@redhat.com \
    --cc=jmarchan@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=okozina@redhat.com \
    --cc=penguin-kernel@I-love.SAKURA.ne.jp \
    --cc=rientjes@google.com \
    --cc=skozina@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.