linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.cz>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Ralf-Peter Rohbeck <Ralf-Peter.Rohbeck@quantum.com>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: OOM killer changes
Date: Mon, 15 Aug 2016 17:01:23 +0200	[thread overview]
Message-ID: <20160815150123.GG3360@dhcp22.suse.cz> (raw)
In-Reply-To: <d1f63745-b9e3-b699-8a5a-08f06c72b392@suse.cz>

On Mon 15-08-16 11:16:36, Vlastimil Babka wrote:
> On 08/15/2016 06:48 AM, Ralf-Peter Rohbeck wrote:
> > On 02.08.2016 12:25, Ralf-Peter Rohbeck wrote:
> >>
> > Took me a little longer than expected due to work. The failure wouldn't 
> > happen for a while and so I started a couple of scripts and let them 
> > run. When I checked today the server didn't respond on the network and 
> > sure enough it had killed everything. This is with 4.7.0 with the config 
> > based on Debian 4.7-rc7.
> > 
> > trace_pipe got a little big (5GB) so I uploaded the logs to 
> > https://filebin.net/box0wycfouvhl6sr/OOM_4.7.0.tar.bz2. before_btrfs is 
> > before the btrfs filesystems were mounted.
> > I did run a btrfs balance because it creates IO load and I needed to 
> > balance anyway. Maybe that's what caused it?
> 
> pgmigrate_success        46738962
> pgmigrate_fail          135649772
> compact_migrate_scanned 309726659
> compact_free_scanned   9715615169
> compact_isolated        229689596
> compact_stall 4777
> compact_fail 3068
> compact_success 1709
> compact_daemon_wake 207834
> 
> The migration failures are quite enormous. Very quick analysis of the
> trace seems to confirm that these are mostly "real", as opposed to result
> of failure to isolate free pages for migration targets, although the free
> scanner spent a lot of time:
> 
> > grep "nr_failed=32" -B1 trace_pipe.log | grep isolate_freepages.*nr_taken=0 | wc -l
> 3246
> 
> So is it one of the cases where fs is unable to migrate dirty/writeback pages?

It smells that way. Now we should find out why and what can we do about
that. I suspect that try_to_release_page is not able to release the page
for migration. Btrfs doesn't seem to have migratepage for page cache
pages so it should go via fallback_migrate_page.

The following diff should tell us whether this is really the case. Just
open trace_pipe and see whether this path really triggered.
---
diff --git a/mm/migrate.c b/mm/migrate.c
index 72c09dea6526..120e2e5fcbea 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -729,8 +729,10 @@ static int fallback_migrate_page(struct address_space *mapping,
 	 * We must have no buffers or drop them.
 	 */
 	if (page_has_private(page) &&
-	    !try_to_release_page(page, GFP_KERNEL))
+	    !try_to_release_page(page, GFP_KERNEL)) {
+		trace_printk("try_to_release_page failed for a_ops:%pS\n", page->a_ops);
 		return -EAGAIN;
+	}
 
 	return migrate_page(mapping, newpage, page, mode);
 }
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2016-08-15 15:01 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <d8f3adcc-3607-1ef6-9ec5-82b2e125eef2@quantum.com>
2016-08-01  6:16 ` OOM killer changes Michal Hocko
     [not found]   ` <b1a39756-a0b5-1900-6575-d6e1f502cb26@Quantum.com>
     [not found]     ` <20160801182358.GB31957@dhcp22.suse.cz>
     [not found]       ` <30dbabc4-585c-55a5-9f3a-4e243c28356a@Quantum.com>
2016-08-01 19:26         ` Michal Hocko
2016-08-01 19:35           ` Ralf-Peter Rohbeck
2016-08-01 19:43             ` Michal Hocko
2016-08-01 19:52               ` Ralf-Peter Rohbeck
2016-08-01 20:09                 ` Michal Hocko
2016-08-01 20:16                   ` Ralf-Peter Rohbeck
2016-08-01 20:26                     ` Michal Hocko
2016-08-01 21:14                       ` Ralf-Peter Rohbeck
2016-08-01 21:27                         ` Ralf-Peter Rohbeck
2016-08-02  7:10                           ` Michal Hocko
2016-08-02 19:25                             ` Ralf-Peter Rohbeck
2016-08-15  4:48                               ` Ralf-Peter Rohbeck
2016-08-15  9:16                                 ` Vlastimil Babka
2016-08-15 15:01                                   ` Michal Hocko [this message]
2016-08-15 18:42                                     ` Ralf-Peter Rohbeck
2016-08-16  7:32                                       ` Michal Hocko
2016-08-16  7:43                                         ` Michal Hocko
2016-08-17  9:14                                           ` Ralf-Peter Rohbeck
2016-08-17  9:23                                             ` Vlastimil Babka
2016-08-17  9:28                                               ` Ralf-Peter Rohbeck
2016-08-17  9:33                                                 ` Michal Hocko
2016-08-17 23:37                                                   ` Ralf-Peter Rohbeck
2016-08-18  6:57                                                     ` Vlastimil Babka
2016-08-18 20:01                                                       ` Ralf-Peter Rohbeck
2016-08-18 20:12                                                         ` Vlastimil Babka
2016-08-19  2:42                                                           ` Ralf-Peter Rohbeck
2016-08-19  6:27                                                             ` Vlastimil Babka
2016-08-19  7:33                                                               ` Michal Hocko
2016-08-19  7:47                                                                 ` Vlastimil Babka
2016-08-19  8:26                                                                   ` Michal Hocko
2016-08-24 18:13                                                                     ` Ralf-Peter Rohbeck
2016-08-25  7:22                                                                       ` Michal Hocko
2016-08-25 20:35                                                                         ` Ralf-Peter Rohbeck
2016-08-26  8:35                                                                           ` Michal Hocko
2016-09-06 11:09                                                                             ` Vlastimil Babka
2016-08-23  5:02                                                               ` Joonsoo Kim
2016-08-23  7:45                                                                 ` Michal Hocko
2016-08-17  0:26                                         ` Ralf-Peter Rohbeck
2016-08-17  7:43                                           ` Vlastimil Babka
2016-08-16  3:12                                   ` Joonsoo Kim
2016-08-16  7:44                                     ` Vlastimil Babka
2016-08-17  4:48                                     ` Ralf-Peter Rohbeck
2016-08-17  7:56                                       ` Vlastimil Babka
2016-08-17  8:16                                         ` Joonsoo Kim
2016-08-17  9:21                                           ` Ralf-Peter Rohbeck
2016-08-17  9:11                                         ` Ralf-Peter Rohbeck
2016-08-17  9:20                                           ` Vlastimil Babka
2016-08-02  7:11           ` Vlastimil Babka
2016-08-02  9:02           ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160815150123.GG3360@dhcp22.suse.cz \
    --to=mhocko@suse.cz \
    --cc=Ralf-Peter.Rohbeck@quantum.com \
    --cc=linux-mm@kvack.org \
    --cc=vbabka@suse.cz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).