* [GIT PULL] Btrfs deadlock fix
@ 2014-11-23 15:14 Chris Mason
2014-11-24 0:23 ` Benjamin Herrenschmidt
2014-11-24 17:39 ` David Sterba
0 siblings, 2 replies; 7+ messages in thread
From: Chris Mason @ 2014-11-23 15:14 UTC (permalink / raw)
To: torvalds; +Cc: linux-btrfs, linux-kernel
Hi Linus,
My for-linus branch:
git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus
Has a fix for a long standing deadlock that we've been trying to nail
down for a while. It ended up being a bad interaction with the fair
reader/writer locks and the order btrfs reacquires locks in the btree.
Chris Mason (1) commits (+25/-15):
btrfs: fix lockups from btrfs_clear_path_blocking
Total: (1) commits (+25/-15)
fs/btrfs/ctree.c | 14 ++------------
fs/btrfs/locking.c | 24 +++++++++++++++++++++---
fs/btrfs/locking.h | 2 ++
3 files changed, 25 insertions(+), 15 deletions(-)
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix
2014-11-23 15:14 [GIT PULL] Btrfs deadlock fix Chris Mason
@ 2014-11-24 0:23 ` Benjamin Herrenschmidt
2014-11-24 0:27 ` Chris Mason
2014-11-24 17:39 ` David Sterba
1 sibling, 1 reply; 7+ messages in thread
From: Benjamin Herrenschmidt @ 2014-11-24 0:23 UTC (permalink / raw)
To: Chris Mason; +Cc: torvalds, linux-btrfs, linux-kernel
On Sun, 2014-11-23 at 10:14 -0500, Chris Mason wrote:
> Hi Linus,
>
> My for-linus branch:
Not sure if this is related, but with -rc4, on this ppc64el box, when
grub updates (it starts trying to mount everything under the sun with
every filesystem), the box hangs with:
[ 177.344522] INFO: rcu_sched detected stalls on CPUs/tasks: { 8}
(detected by 9, t=25883 jiffies, g=3642, c=3641, q=5206)
With xmon I can catch it with this backtrace:
[link register ] d00000000ecec3f8 test_check_exists+0x1f8/0x280 [btrfs]
[c000000f0cc63930] d00000000ecec2b0 test_check_exists+0xb0/0x280 [btrfs] (unreliable)
[c000000f0cc63990] d00000000ed2a4b4 btrfs_test_free_space_cache+0x774/0x1150 [btrfs]
[c000000f0cc63a20] d00000000ed30f04 init_btrfs_fs+0x11c/0x210 [btrfs]
[c000000f0cc63a90] c00000000000b4dc do_one_initcall+0x12c/0x280
[c000000f0cc63b60] c0000000001628b8 load_module+0x2088/0x2970
[c000000f0cc63d50] c0000000001633d0 SyS_finit_module+0xc0/0x120
[c000000f0cc63e30] c00000000000927c syscall_exit+0x0/0x7c
--- Exception: c01 (System Call) at 00003fff88697184
Note that afaik there isn't any btrfs partition on any disk on that
machine... well not that I know of :)
Cheers,
Ben.
> git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus
>
> Has a fix for a long standing deadlock that we've been trying to nail
> down for a while. It ended up being a bad interaction with the fair
> reader/writer locks and the order btrfs reacquires locks in the btree.
>
> Chris Mason (1) commits (+25/-15):
> btrfs: fix lockups from btrfs_clear_path_blocking
>
> Total: (1) commits (+25/-15)
>
> fs/btrfs/ctree.c | 14 ++------------
> fs/btrfs/locking.c | 24 +++++++++++++++++++++---
> fs/btrfs/locking.h | 2 ++
> 3 files changed, 25 insertions(+), 15 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix
2014-11-24 0:23 ` Benjamin Herrenschmidt
@ 2014-11-24 0:27 ` Chris Mason
2014-11-24 3:28 ` Benjamin Herrenschmidt
0 siblings, 1 reply; 7+ messages in thread
From: Chris Mason @ 2014-11-24 0:27 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: torvalds, linux-btrfs, linux-kernel
On Sun, Nov 23, 2014 at 7:23 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Sun, 2014-11-23 at 10:14 -0500, Chris Mason wrote:
>> Hi Linus,
>>
>> My for-linus branch:
>
> Not sure if this is related, but with -rc4, on this ppc64el box, when
> grub updates (it starts trying to mount everything under the sun with
> every filesystem), the box hangs with:
>
> [ 177.344522] INFO: rcu_sched detected stalls on CPUs/tasks: { 8}
> (detected by 9, t=25883 jiffies, g=3642, c=3641, q=5206)
>
> With xmon I can catch it with this backtrace:
>
> [link register ] d00000000ecec3f8 test_check_exists+0x1f8/0x280
> [btrfs]
> [c000000f0cc63930] d00000000ecec2b0 test_check_exists+0xb0/0x280
> [btrfs] (unreliable)
> [c000000f0cc63990] d00000000ed2a4b4
> btrfs_test_free_space_cache+0x774/0x1150 [btrfs]
> [c000000f0cc63a20] d00000000ed30f04 init_btrfs_fs+0x11c/0x210 [btrfs]
> [c000000f0cc63a90] c00000000000b4dc do_one_initcall+0x12c/0x280
> [c000000f0cc63b60] c0000000001628b8 load_module+0x2088/0x2970
> [c000000f0cc63d50] c0000000001633d0 SyS_finit_module+0xc0/0x120
> [c000000f0cc63e30] c00000000000927c syscall_exit+0x0/0x7c
> --- Exception: c01 (System Call) at 00003fff88697184
>
> Note that afaik there isn't any btrfs partition on any disk on that
> machine... well not that I know of :)
This should be from CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y. I do test with
this on here, but I'll look it over for ppc related fun.
The deadlock fix from this pull shouldn't be related. Just to make
sure I read your email right, this stall isn't new with rc6 is it?
-chris
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix
2014-11-24 0:27 ` Chris Mason
@ 2014-11-24 3:28 ` Benjamin Herrenschmidt
0 siblings, 0 replies; 7+ messages in thread
From: Benjamin Herrenschmidt @ 2014-11-24 3:28 UTC (permalink / raw)
To: Chris Mason; +Cc: torvalds, linux-btrfs, linux-kernel
On Sun, 2014-11-23 at 19:27 -0500, Chris Mason wrote:
> This should be from CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y. I do test
> with this on here, but I'll look it over for ppc related fun.
>
> The deadlock fix from this pull shouldn't be related. Just to make
> sure I read your email right, this stall isn't new with rc6 is it?
No, I observed it with -rc4 which is the first 3.18 I tried on this
specific machine. I can try to bisect but it will take a while, it
didn't happen afaik with 3.17
Cheers,
Ben.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix
2014-11-23 15:14 [GIT PULL] Btrfs deadlock fix Chris Mason
2014-11-24 0:23 ` Benjamin Herrenschmidt
@ 2014-11-24 17:39 ` David Sterba
2014-11-24 18:01 ` Chris Mason
1 sibling, 1 reply; 7+ messages in thread
From: David Sterba @ 2014-11-24 17:39 UTC (permalink / raw)
To: Chris Mason, torvalds, linux-btrfs, linux-kernel
Hi Chris,
I thought the fix for the scrub/replace deadlock would be included in
this pull, I can reproduce it on each run of xfstests with 3.18-rc.
btrfs: fix dead lock while running replace and defrag concurrently
https://patchwork.kernel.org/patch/5264531/
I've retested it again including this pull and still deadlocks reliably
at btrfs/070.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix
2014-11-24 17:39 ` David Sterba
@ 2014-11-24 18:01 ` Chris Mason
2014-11-24 18:27 ` David Sterba
0 siblings, 1 reply; 7+ messages in thread
From: Chris Mason @ 2014-11-24 18:01 UTC (permalink / raw)
To: dsterba; +Cc: torvalds, linux-btrfs, linux-kernel
On Mon, Nov 24, 2014 at 12:39 PM, David Sterba <dsterba@suse.cz> wrote:
> Hi Chris,
>
> I thought the fix for the scrub/replace deadlock would be included in
> this pull, I can reproduce it on each run of xfstests with 3.18-rc.
>
> btrfs: fix dead lock while running replace and defrag concurrently
> https://patchwork.kernel.org/patch/5264531/
>
> I've retested it again including this pull and still deadlocks
> reliably
> at btrfs/070.
This wasn't a new problem, so I had it queued for the merge window.
-chris
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix
2014-11-24 18:01 ` Chris Mason
@ 2014-11-24 18:27 ` David Sterba
0 siblings, 0 replies; 7+ messages in thread
From: David Sterba @ 2014-11-24 18:27 UTC (permalink / raw)
To: Chris Mason; +Cc: dsterba, torvalds, linux-btrfs, linux-kernel
On Mon, Nov 24, 2014 at 01:01:10PM -0500, Chris Mason wrote:
> > I've retested it again including this pull and still deadlocks
> > reliably
> > at btrfs/070.
>
> This wasn't a new problem, so I had it queued for the merge window.
Well, I don't remember seeing this problem with anything 3.17 based but
I can check. I know this does not mean that it was introduced in 3.18,
only that other changes exposed the problem, but this still IMHO
qualifies as a bug to be fixed within 3.18. Besides, it makes testing
3.18-rc slightly more annoying (either skip the test or manually pick
the patch after each pull).
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2014-11-24 18:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-23 15:14 [GIT PULL] Btrfs deadlock fix Chris Mason
2014-11-24 0:23 ` Benjamin Herrenschmidt
2014-11-24 0:27 ` Chris Mason
2014-11-24 3:28 ` Benjamin Herrenschmidt
2014-11-24 17:39 ` David Sterba
2014-11-24 18:01 ` Chris Mason
2014-11-24 18:27 ` David Sterba
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.