All of lore.kernel.org
 help / color / mirror / Atom feed
From: Colin Ian King <colin.king@ubuntu.com>
To: Mel Gorman <mgorman@suse.de>
Cc: James Bottomley <James.Bottomley@suse.de>,
	Mel Gorman <mgorman@novell.com>, Jan Kara <jack@suse.cz>,
	Chris Mason <chris.mason@oracle.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	linux-ext4 <linux-ext4@vger.kernel.org>
Subject: Re: [BUG] fatal hang untarring 90GB file, possibly writeback related.
Date: Tue, 10 May 2011 07:37:42 +0200	[thread overview]
Message-ID: <1305005862.1937.2.camel@hpmini> (raw)
In-Reply-To: <20110506154444.GG6591@suse.de>

On Fri, 2011-05-06 at 16:44 +0100, Mel Gorman wrote:
> On Fri, May 06, 2011 at 08:42:24AM +0100, Mel Gorman wrote:
> > 
> > 1. High-order allocations? You machine is using i915 and RPC, something
> >    neither of my test machine uses. i915 is potentially a source for
> >    high-order allocations. I'm attaching a perl script. Please run it as
> >    ./watch-highorder.pl --output /tmp/highorders.txt
> >    while you are running tar. When kswapd is running for about 30
> >    seconds, interrupt it with ctrl+c twice in quick succession and
> >    post /tmp/highorders.txt
> > 
> 
> Colin send me this information for his test case at least and I see
> 
> 11932 instances order=1 normal gfp_flags=GFP_NOWARN|GFP_NORETRY|GFP_COMP|GFP_NOMEMALLOC
>  => alloc_pages_current+0xa5/0x110 <ffffffff81149ef5>
>  => new_slab+0x1f5/0x290 <ffffffff81153645>
>  => __slab_alloc+0x262/0x390 <ffffffff81155192>
>  => kmem_cache_alloc+0x115/0x120 <ffffffff81155ab5>
>  => mempool_alloc_slab+0x15/0x20 <ffffffff8110e705>
>  => mempool_alloc+0x59/0x140 <ffffffff8110ea49>
>  => bio_alloc_bioset+0x3e/0xf0 <ffffffff811976ae>
>  => bio_alloc+0x15/0x30 <ffffffff81197805>
> 
> Colin and James: Did you happen to switch from SLAB to SLUB between
> 2.6.37 and 2.6.38? My own tests were against SLAB which might be why I
> didn't see the problem. Am restarting the tests with SLUB.

So I tested with SLAB instead of SLUB and I reliably ran my copy test
for 4+ hours with several hundred iterations of the test.  (Apologies
for taking time to respond, but I was travelling).
> 



WARNING: multiple messages have this Message-ID (diff)
From: Colin Ian King <colin.king@ubuntu.com>
To: Mel Gorman <mgorman@suse.de>
Cc: James Bottomley <James.Bottomley@suse.de>,
	Mel Gorman <mgorman@novell.com>,  Jan Kara <jack@suse.cz>,
	Chris Mason <chris.mason@oracle.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	linux-ext4 <linux-ext4@vger.kernel.org>
Subject: Re: [BUG] fatal hang untarring 90GB file, possibly writeback related.
Date: Tue, 10 May 2011 07:37:42 +0200	[thread overview]
Message-ID: <1305005862.1937.2.camel@hpmini> (raw)
In-Reply-To: <20110506154444.GG6591@suse.de>

On Fri, 2011-05-06 at 16:44 +0100, Mel Gorman wrote:
> On Fri, May 06, 2011 at 08:42:24AM +0100, Mel Gorman wrote:
> > 
> > 1. High-order allocations? You machine is using i915 and RPC, something
> >    neither of my test machine uses. i915 is potentially a source for
> >    high-order allocations. I'm attaching a perl script. Please run it as
> >    ./watch-highorder.pl --output /tmp/highorders.txt
> >    while you are running tar. When kswapd is running for about 30
> >    seconds, interrupt it with ctrl+c twice in quick succession and
> >    post /tmp/highorders.txt
> > 
> 
> Colin send me this information for his test case at least and I see
> 
> 11932 instances order=1 normal gfp_flags=GFP_NOWARN|GFP_NORETRY|GFP_COMP|GFP_NOMEMALLOC
>  => alloc_pages_current+0xa5/0x110 <ffffffff81149ef5>
>  => new_slab+0x1f5/0x290 <ffffffff81153645>
>  => __slab_alloc+0x262/0x390 <ffffffff81155192>
>  => kmem_cache_alloc+0x115/0x120 <ffffffff81155ab5>
>  => mempool_alloc_slab+0x15/0x20 <ffffffff8110e705>
>  => mempool_alloc+0x59/0x140 <ffffffff8110ea49>
>  => bio_alloc_bioset+0x3e/0xf0 <ffffffff811976ae>
>  => bio_alloc+0x15/0x30 <ffffffff81197805>
> 
> Colin and James: Did you happen to switch from SLAB to SLUB between
> 2.6.37 and 2.6.38? My own tests were against SLAB which might be why I
> didn't see the problem. Am restarting the tests with SLUB.

So I tested with SLAB instead of SLUB and I reliably ran my copy test
for 4+ hours with several hundred iterations of the test.  (Apologies
for taking time to respond, but I was travelling).
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Colin Ian King <colin.king@ubuntu.com>
To: Mel Gorman <mgorman@suse.de>
Cc: James Bottomley <James.Bottomley@suse.de>,
	Mel Gorman <mgorman@novell.com>, Jan Kara <jack@suse.cz>,
	Chris Mason <chris.mason@oracle.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	linux-ext4 <linux-ext4@vger.kernel.org>
Subject: Re: [BUG] fatal hang untarring 90GB file, possibly writeback related.
Date: Tue, 10 May 2011 07:37:42 +0200	[thread overview]
Message-ID: <1305005862.1937.2.camel@hpmini> (raw)
In-Reply-To: <20110506154444.GG6591@suse.de>

On Fri, 2011-05-06 at 16:44 +0100, Mel Gorman wrote:
> On Fri, May 06, 2011 at 08:42:24AM +0100, Mel Gorman wrote:
> > 
> > 1. High-order allocations? You machine is using i915 and RPC, something
> >    neither of my test machine uses. i915 is potentially a source for
> >    high-order allocations. I'm attaching a perl script. Please run it as
> >    ./watch-highorder.pl --output /tmp/highorders.txt
> >    while you are running tar. When kswapd is running for about 30
> >    seconds, interrupt it with ctrl+c twice in quick succession and
> >    post /tmp/highorders.txt
> > 
> 
> Colin send me this information for his test case at least and I see
> 
> 11932 instances order=1 normal gfp_flags=GFP_NOWARN|GFP_NORETRY|GFP_COMP|GFP_NOMEMALLOC
>  => alloc_pages_current+0xa5/0x110 <ffffffff81149ef5>
>  => new_slab+0x1f5/0x290 <ffffffff81153645>
>  => __slab_alloc+0x262/0x390 <ffffffff81155192>
>  => kmem_cache_alloc+0x115/0x120 <ffffffff81155ab5>
>  => mempool_alloc_slab+0x15/0x20 <ffffffff8110e705>
>  => mempool_alloc+0x59/0x140 <ffffffff8110ea49>
>  => bio_alloc_bioset+0x3e/0xf0 <ffffffff811976ae>
>  => bio_alloc+0x15/0x30 <ffffffff81197805>
> 
> Colin and James: Did you happen to switch from SLAB to SLUB between
> 2.6.37 and 2.6.38? My own tests were against SLAB which might be why I
> didn't see the problem. Am restarting the tests with SLUB.

So I tested with SLAB instead of SLUB and I reliably ran my copy test
for 4+ hours with several hundred iterations of the test.  (Apologies
for taking time to respond, but I was travelling).
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2011-05-10  5:37 UTC|newest]

Thread overview: 138+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-04-27 16:09 [BUG] fatal hang untarring 90GB file, possibly writeback related James Bottomley
2011-04-27 16:09 ` James Bottomley
2011-04-27 16:33 ` Chris Mason
2011-04-27 16:33   ` Chris Mason
2011-04-27 16:50   ` James Bottomley
2011-04-27 16:50     ` James Bottomley
2011-04-27 16:50     ` James Bottomley
2011-04-27 16:54     ` Chris Mason
2011-04-27 16:54       ` Chris Mason
2011-04-27 17:21       ` James Bottomley
2011-04-27 17:21         ` James Bottomley
2011-04-27 17:21         ` James Bottomley
2011-04-27 17:34         ` Chris Mason
2011-04-27 17:34           ` Chris Mason
2011-04-27 17:50           ` James Bottomley
2011-04-27 17:50             ` James Bottomley
2011-04-27 18:25             ` Colin Ian King
2011-04-27 18:25               ` Colin Ian King
2011-04-28 15:57               ` James Bottomley
2011-04-28 15:57                 ` James Bottomley
2011-04-27 20:05             ` James Bottomley
2011-04-27 20:05               ` James Bottomley
2011-04-28 11:36               ` Colin Ian King
2011-04-28 11:36                 ` Colin Ian King
2011-04-28 12:29                 ` Chris Mason
2011-04-28 12:29                   ` Chris Mason
2011-04-28 13:42                   ` Colin Ian King
2011-04-28 13:42                     ` Colin Ian King
2011-04-28 13:45                     ` Chris Mason
2011-04-28 13:45                       ` Chris Mason
2011-04-28 14:01                       ` Colin Ian King
2011-04-28 14:04                         ` Chris Mason
2011-04-28 14:04                           ` Chris Mason
2011-04-28 15:23                           ` Colin Ian King
2011-04-28 14:25                         ` Jan Kara
2011-04-28 14:25                           ` Jan Kara
2011-04-28 14:33                           ` Jan Kara
2011-04-28 14:33                             ` Jan Kara
2011-04-28 14:58                             ` Colin Ian King
2011-04-28 22:40                               ` Jan Kara
2011-04-28 22:40                                 ` Jan Kara
2011-04-28 22:44                                 ` James Bottomley
2011-04-28 22:44                                   ` James Bottomley
2011-05-03 18:55                                 ` Colin Ian King
2011-05-03 18:55                                   ` Colin Ian King
2011-04-28 16:11                             ` James Bottomley
2011-04-28 16:11                               ` James Bottomley
2011-04-28 14:49                   ` James Bottomley
2011-04-28 14:49                     ` James Bottomley
2011-04-28 13:52                 ` Jan Kara
2011-04-28 13:52                   ` Jan Kara
2011-04-28 14:07                   ` Mel Gorman
2011-04-28 14:07                     ` Mel Gorman
2011-04-28 14:25                     ` James Bottomley
2011-04-28 14:25                       ` James Bottomley
2011-04-28 15:08                       ` Mel Gorman
2011-04-28 15:08                         ` Mel Gorman
2011-04-28 16:01                         ` James Bottomley
2011-04-28 16:01                           ` James Bottomley
2011-04-28 16:50                           ` James Bottomley
2011-04-28 16:50                             ` James Bottomley
2011-04-28 16:56                             ` James Bottomley
2011-04-28 16:56                               ` James Bottomley
2011-04-28 17:18                               ` Mel Gorman
2011-04-28 17:18                                 ` Mel Gorman
2011-04-28 18:30                                 ` James Bottomley
2011-04-28 18:30                                   ` James Bottomley
2011-04-28 19:21                                   ` Mel Gorman
2011-04-28 19:21                                     ` Mel Gorman
2011-04-28 19:59                                     ` James Bottomley
2011-04-28 19:59                                       ` James Bottomley
2011-04-28 20:27                                       ` Mel Gorman
2011-04-28 20:27                                         ` Mel Gorman
2011-04-29 15:02                                         ` James Bottomley
2011-04-29 15:02                                           ` James Bottomley
2011-04-28 21:12                                       ` James Bottomley
2011-04-28 21:12                                         ` James Bottomley
2011-04-28 22:43                                         ` James Bottomley
2011-04-28 22:43                                           ` James Bottomley
2011-05-03  9:13                                           ` Mel Gorman
2011-05-03  9:13                                             ` Mel Gorman
2011-05-03 14:13                                             ` James Bottomley
2011-05-03 14:13                                               ` James Bottomley
2011-05-03 14:22                                               ` James Bottomley
2011-05-06  7:42                                                 ` Mel Gorman
2011-05-06  7:42                                                   ` Mel Gorman
2011-05-06  8:07                                                   ` Mel Gorman
2011-05-09 18:16                                                     ` James Bottomley
2011-05-09 18:16                                                       ` James Bottomley
2011-05-10 10:21                                                       ` Mel Gorman
2011-05-10 10:21                                                         ` Mel Gorman
2011-05-10 10:33                                                         ` Pekka Enberg
2011-05-10 10:33                                                           ` Pekka Enberg
2011-05-10 14:01                                                         ` James Bottomley
2011-05-10 14:01                                                           ` James Bottomley
2011-05-10 14:35                                                           ` Mel Gorman
2011-05-10 14:35                                                             ` Mel Gorman
2011-05-10 15:29                                                             ` James Bottomley
2011-05-10 15:29                                                               ` James Bottomley
2011-05-10 15:57                                                               ` James Bottomley
2011-05-10 15:57                                                                 ` James Bottomley
2011-05-10 17:05                                                                 ` James Bottomley
2011-05-10 17:05                                                                   ` James Bottomley
2011-05-10 17:17                                                                   ` Mel Gorman
2011-05-10 17:17                                                                     ` Mel Gorman
2011-05-10 17:29                                                                     ` James Bottomley
2011-05-10 17:29                                                                       ` James Bottomley
2011-05-10 21:08                                                               ` Raghavendra D Prabhu
2011-05-11  9:16                                                                 ` Mel Gorman
2011-05-11  9:16                                                                   ` Mel Gorman
2011-05-06 11:42                                                   ` Mel Gorman
2011-05-06 11:42                                                     ` Mel Gorman
2011-05-06 15:44                                                   ` Mel Gorman
2011-05-06 15:44                                                     ` Mel Gorman
2011-05-06 19:14                                                     ` James Bottomley
2011-05-06 19:14                                                       ` James Bottomley
2011-05-06 19:37                                                       ` Mel Gorman
2011-05-06 19:37                                                         ` Mel Gorman
2011-05-10  5:37                                                     ` Colin Ian King [this message]
2011-05-10  5:37                                                       ` Colin Ian King
2011-05-10  5:37                                                       ` Colin Ian King
2011-05-06 15:58                                                   ` James Bottomley
2011-05-03  9:54                                 ` Colin Ian King
2011-05-03  9:54                                   ` Colin Ian King
2011-04-28 17:10                         ` Colin Ian King
2011-04-28 17:10                           ` Colin Ian King
2011-04-28  0:37         ` Dave Chinner
2011-04-28  0:37           ` Dave Chinner
2011-04-29 10:23         ` Sedat Dilek
2011-04-29 10:23           ` Sedat Dilek
2011-04-29 15:37           ` James Bottomley
2011-04-29 15:37             ` James Bottomley
2011-04-29 16:31             ` James Bottomley
2011-04-29 16:31               ` James Bottomley
2011-04-29 18:02               ` James Bottomley
2011-04-29 18:02                 ` James Bottomley
2011-05-02 20:04                 ` James Bottomley
2011-05-02 20:04                   ` James Bottomley

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1305005862.1937.2.camel@hpmini \
    --to=colin.king@ubuntu.com \
    --cc=James.Bottomley@suse.de \
    --cc=chris.mason@oracle.com \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@novell.com \
    --cc=mgorman@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.