Re: applications hang on a btrfs spanning two partitions

From: Chris Murphy <lists@colorremedies.com>
To: Florian Stecker <m19@florianstecker.de>
Cc: Nikolay Borisov <nborisov@suse.com>,
	Btrfs BTRFS <linux-btrfs@vger.kernel.org>
Subject: Re: applications hang on a btrfs spanning two partitions
Date: Fri, 11 Jan 2019 19:12:36 -0700	[thread overview]
Message-ID: <CAJCQCtQF0DU+yAH33iSXu2NBk7TCWCUZ4STC2TExW3+dJpK-Hw@mail.gmail.com> (raw)
In-Reply-To: <7b4fa64b-e981-52c5-4597-40b5ea2df5b3@florianstecker.de>

On Wed, Jan 9, 2019 at 1:10 PM Florian Stecker <m19@florianstecker.de> wrote:
>
>
>
> On 1/9/19 11:03 AM, Nikolay Borisov wrote:
> >
> >
> > On 9.01.19 г. 11:16 ч., Florian Stecker wrote:
> >>>
> >>> Provide output of echo w > /proc/sysrq-trigger when the hang occurs
> >>> otherwise it's hard to figure what's going on.
> >>>
> >>
> >> Here's one, again in gajim. This time, fdatasync() took "only" 2 seconds:
> >>
> >> [42481.243491] sysrq: SysRq : Show Blocked State
> >> [42481.243494]   task                        PC stack   pid father
> >> [42481.243566] gajim           D    0 15778  15774 0x00000083
> >> [42481.243569] Call Trace:
> >> [42481.243575]  ? __schedule+0x29b/0x8b0
> >> [42481.243576]  ? bit_wait+0x50/0x50
> >> [42481.243578]  schedule+0x32/0x90
> >> [42481.243580]  io_schedule+0x12/0x40
> >> [42481.243582]  bit_wait_io+0xd/0x50
> >> [42481.243583]  __wait_on_bit+0x6c/0x80
> >> [42481.243585]  out_of_line_wait_on_bit+0x91/0xb0
> >> [42481.243587]  ? init_wait_var_entry+0x40/0x40
> >> [42481.243605]  write_all_supers+0x418/0xa70 [btrfs]
> >> [42481.243622]  btrfs_sync_log+0x695/0x910 [btrfs]
> >> [42481.243625]  ? _raw_spin_lock_irqsave+0x25/0x50
> >> [42481.243641]  ? btrfs_log_dentry_safe+0x54/0x70 [btrfs]
> >> [42481.243655]  btrfs_sync_file+0x3a9/0x3d0 [btrfs]
> >> [42481.243659]  do_fsync+0x38/0x70
> >> [42481.243661]  __x64_sys_fdatasync+0x13/0x20
> >> [42481.243663]  do_syscall_64+0x5b/0x170
> >> [42481.243666]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >> [42481.243667] RIP: 0033:0x7fd4022f873f
> >> [42481.243671] Code: Bad RIP value.
> >> [42481.243672] RSP: 002b:00007ffd3710a300 EFLAGS: 00000293 ORIG_RAX:
> >> 000000000000004b
> >> [42481.243674] RAX: ffffffffffffffda RBX: 0000000000000019 RCX:
> >> 00007fd4022f873f
> >> [42481.243675] RDX: 0000000000000000 RSI: 0000000000000002 RDI:
> >> 0000000000000019
> >> [42481.243675] RBP: 0000000000000000 R08: 000055d8d8649f68 R09:
> >> 00007ffd3710a320
> >> [42481.243676] R10: 0000000000013000 R11: 0000000000000293 R12:
> >> 0000000000000000
> >> [42481.243677] R13: 0000000000000000 R14: 000055d8d8363fa0 R15:
> >> 000055d8d8613040
> >
> > This shows that IO was send to disk to write the supper blocks following
> > an fsync and it's waiting for IO to finish. This seems like a problem in
> > the storage layer, i.e IOs being stuck. Check your dmesg for any error.
> There are no IO errors in dmesg. Also, I never had any problems with
> this disk, SMART reports nothing, and also btrfs dev stats and btrfs
> scrub say everything's ok.

What do you get for:
mount | grep btrfs
btrfs insp dump-s -f /dev/sda8

I ran in this same configuration for a long time, maybe 5 months, and
never ran into this problem. But it was with much older kernel,
perhaps circa 4.8 era.

-- 
Chris Murphy