* [GIT PULL] Btrfs deadlock fix @ 2014-11-23 15:14 Chris Mason 2014-11-24 0:23 ` Benjamin Herrenschmidt 2014-11-24 17:39 ` David Sterba 0 siblings, 2 replies; 7+ messages in thread From: Chris Mason @ 2014-11-23 15:14 UTC (permalink / raw) To: torvalds; +Cc: linux-btrfs, linux-kernel Hi Linus, My for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus Has a fix for a long standing deadlock that we've been trying to nail down for a while. It ended up being a bad interaction with the fair reader/writer locks and the order btrfs reacquires locks in the btree. Chris Mason (1) commits (+25/-15): btrfs: fix lockups from btrfs_clear_path_blocking Total: (1) commits (+25/-15) fs/btrfs/ctree.c | 14 ++------------ fs/btrfs/locking.c | 24 +++++++++++++++++++++--- fs/btrfs/locking.h | 2 ++ 3 files changed, 25 insertions(+), 15 deletions(-) ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix 2014-11-23 15:14 [GIT PULL] Btrfs deadlock fix Chris Mason @ 2014-11-24 0:23 ` Benjamin Herrenschmidt 2014-11-24 0:27 ` Chris Mason 2014-11-24 17:39 ` David Sterba 1 sibling, 1 reply; 7+ messages in thread From: Benjamin Herrenschmidt @ 2014-11-24 0:23 UTC (permalink / raw) To: Chris Mason; +Cc: torvalds, linux-btrfs, linux-kernel On Sun, 2014-11-23 at 10:14 -0500, Chris Mason wrote: > Hi Linus, > > My for-linus branch: Not sure if this is related, but with -rc4, on this ppc64el box, when grub updates (it starts trying to mount everything under the sun with every filesystem), the box hangs with: [ 177.344522] INFO: rcu_sched detected stalls on CPUs/tasks: { 8} (detected by 9, t=25883 jiffies, g=3642, c=3641, q=5206) With xmon I can catch it with this backtrace: [link register ] d00000000ecec3f8 test_check_exists+0x1f8/0x280 [btrfs] [c000000f0cc63930] d00000000ecec2b0 test_check_exists+0xb0/0x280 [btrfs] (unreliable) [c000000f0cc63990] d00000000ed2a4b4 btrfs_test_free_space_cache+0x774/0x1150 [btrfs] [c000000f0cc63a20] d00000000ed30f04 init_btrfs_fs+0x11c/0x210 [btrfs] [c000000f0cc63a90] c00000000000b4dc do_one_initcall+0x12c/0x280 [c000000f0cc63b60] c0000000001628b8 load_module+0x2088/0x2970 [c000000f0cc63d50] c0000000001633d0 SyS_finit_module+0xc0/0x120 [c000000f0cc63e30] c00000000000927c syscall_exit+0x0/0x7c --- Exception: c01 (System Call) at 00003fff88697184 Note that afaik there isn't any btrfs partition on any disk on that machine... well not that I know of :) Cheers, Ben. > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus > > Has a fix for a long standing deadlock that we've been trying to nail > down for a while. It ended up being a bad interaction with the fair > reader/writer locks and the order btrfs reacquires locks in the btree. > > Chris Mason (1) commits (+25/-15): > btrfs: fix lockups from btrfs_clear_path_blocking > > Total: (1) commits (+25/-15) > > fs/btrfs/ctree.c | 14 ++------------ > fs/btrfs/locking.c | 24 +++++++++++++++++++++--- > fs/btrfs/locking.h | 2 ++ > 3 files changed, 25 insertions(+), 15 deletions(-) > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix 2014-11-24 0:23 ` Benjamin Herrenschmidt @ 2014-11-24 0:27 ` Chris Mason 2014-11-24 3:28 ` Benjamin Herrenschmidt 0 siblings, 1 reply; 7+ messages in thread From: Chris Mason @ 2014-11-24 0:27 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: torvalds, linux-btrfs, linux-kernel On Sun, Nov 23, 2014 at 7:23 PM, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote: > On Sun, 2014-11-23 at 10:14 -0500, Chris Mason wrote: >> Hi Linus, >> >> My for-linus branch: > > Not sure if this is related, but with -rc4, on this ppc64el box, when > grub updates (it starts trying to mount everything under the sun with > every filesystem), the box hangs with: > > [ 177.344522] INFO: rcu_sched detected stalls on CPUs/tasks: { 8} > (detected by 9, t=25883 jiffies, g=3642, c=3641, q=5206) > > With xmon I can catch it with this backtrace: > > [link register ] d00000000ecec3f8 test_check_exists+0x1f8/0x280 > [btrfs] > [c000000f0cc63930] d00000000ecec2b0 test_check_exists+0xb0/0x280 > [btrfs] (unreliable) > [c000000f0cc63990] d00000000ed2a4b4 > btrfs_test_free_space_cache+0x774/0x1150 [btrfs] > [c000000f0cc63a20] d00000000ed30f04 init_btrfs_fs+0x11c/0x210 [btrfs] > [c000000f0cc63a90] c00000000000b4dc do_one_initcall+0x12c/0x280 > [c000000f0cc63b60] c0000000001628b8 load_module+0x2088/0x2970 > [c000000f0cc63d50] c0000000001633d0 SyS_finit_module+0xc0/0x120 > [c000000f0cc63e30] c00000000000927c syscall_exit+0x0/0x7c > --- Exception: c01 (System Call) at 00003fff88697184 > > Note that afaik there isn't any btrfs partition on any disk on that > machine... well not that I know of :) This should be from CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y. I do test with this on here, but I'll look it over for ppc related fun. The deadlock fix from this pull shouldn't be related. Just to make sure I read your email right, this stall isn't new with rc6 is it? -chris ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix 2014-11-24 0:27 ` Chris Mason @ 2014-11-24 3:28 ` Benjamin Herrenschmidt 0 siblings, 0 replies; 7+ messages in thread From: Benjamin Herrenschmidt @ 2014-11-24 3:28 UTC (permalink / raw) To: Chris Mason; +Cc: torvalds, linux-btrfs, linux-kernel On Sun, 2014-11-23 at 19:27 -0500, Chris Mason wrote: > This should be from CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y. I do test > with this on here, but I'll look it over for ppc related fun. > > The deadlock fix from this pull shouldn't be related. Just to make > sure I read your email right, this stall isn't new with rc6 is it? No, I observed it with -rc4 which is the first 3.18 I tried on this specific machine. I can try to bisect but it will take a while, it didn't happen afaik with 3.17 Cheers, Ben. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix 2014-11-23 15:14 [GIT PULL] Btrfs deadlock fix Chris Mason 2014-11-24 0:23 ` Benjamin Herrenschmidt @ 2014-11-24 17:39 ` David Sterba 2014-11-24 18:01 ` Chris Mason 1 sibling, 1 reply; 7+ messages in thread From: David Sterba @ 2014-11-24 17:39 UTC (permalink / raw) To: Chris Mason, torvalds, linux-btrfs, linux-kernel Hi Chris, I thought the fix for the scrub/replace deadlock would be included in this pull, I can reproduce it on each run of xfstests with 3.18-rc. btrfs: fix dead lock while running replace and defrag concurrently https://patchwork.kernel.org/patch/5264531/ I've retested it again including this pull and still deadlocks reliably at btrfs/070. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix 2014-11-24 17:39 ` David Sterba @ 2014-11-24 18:01 ` Chris Mason 2014-11-24 18:27 ` David Sterba 0 siblings, 1 reply; 7+ messages in thread From: Chris Mason @ 2014-11-24 18:01 UTC (permalink / raw) To: dsterba; +Cc: torvalds, linux-btrfs, linux-kernel On Mon, Nov 24, 2014 at 12:39 PM, David Sterba <dsterba@suse.cz> wrote: > Hi Chris, > > I thought the fix for the scrub/replace deadlock would be included in > this pull, I can reproduce it on each run of xfstests with 3.18-rc. > > btrfs: fix dead lock while running replace and defrag concurrently > https://patchwork.kernel.org/patch/5264531/ > > I've retested it again including this pull and still deadlocks > reliably > at btrfs/070. This wasn't a new problem, so I had it queued for the merge window. -chris ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [GIT PULL] Btrfs deadlock fix 2014-11-24 18:01 ` Chris Mason @ 2014-11-24 18:27 ` David Sterba 0 siblings, 0 replies; 7+ messages in thread From: David Sterba @ 2014-11-24 18:27 UTC (permalink / raw) To: Chris Mason; +Cc: dsterba, torvalds, linux-btrfs, linux-kernel On Mon, Nov 24, 2014 at 01:01:10PM -0500, Chris Mason wrote: > > I've retested it again including this pull and still deadlocks > > reliably > > at btrfs/070. > > This wasn't a new problem, so I had it queued for the merge window. Well, I don't remember seeing this problem with anything 3.17 based but I can check. I know this does not mean that it was introduced in 3.18, only that other changes exposed the problem, but this still IMHO qualifies as a bug to be fixed within 3.18. Besides, it makes testing 3.18-rc slightly more annoying (either skip the test or manually pick the patch after each pull). ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2014-11-24 18:27 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-11-23 15:14 [GIT PULL] Btrfs deadlock fix Chris Mason 2014-11-24 0:23 ` Benjamin Herrenschmidt 2014-11-24 0:27 ` Chris Mason 2014-11-24 3:28 ` Benjamin Herrenschmidt 2014-11-24 17:39 ` David Sterba 2014-11-24 18:01 ` Chris Mason 2014-11-24 18:27 ` David Sterba
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.