* [PATCH] f2fs: do checkpoint in kill_sb
@ 2018-07-07 1:20 Jaegeuk Kim
2018-07-09 15:04 ` [f2fs-dev] " Chao Yu
0 siblings, 1 reply; 6+ messages in thread
From: Jaegeuk Kim @ 2018-07-07 1:20 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel; +Cc: Jaegeuk Kim
When unmounting f2fs in force mode, we can get it stuck by io_schedule()
by some pending IOs in meta_inode.
io_schedule+0xd/0x30
wait_on_page_bit_common+0xc6/0x130
__filemap_fdatawait_range+0xbd/0x100
filemap_fdatawait_keep_errors+0x15/0x40
sync_inodes_sb+0x1cf/0x240
sync_filesystem+0x52/0x90
generic_shutdown_super+0x1d/0x110
ceph_kill_sb+0x28/0x80 [ceph]
deactivate_locked_super+0x35/0x60
cleanup_mnt+0x36/0x70
task_work_run+0x79/0xa0
exit_to_usermode_loop+0x62/0x70
do_syscall_64+0xdb/0xf0
entry_SYSCALL_64_after_hwframe+0x44/0xa9
0xffffffffffffffff
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
---
fs/f2fs/super.c | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 3995e926ba3a..1dc6809fac38 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -3089,9 +3089,19 @@ static struct dentry *f2fs_mount(struct file_system_type *fs_type, int flags,
static void kill_f2fs_super(struct super_block *sb)
{
if (sb->s_root) {
- set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
- f2fs_stop_gc_thread(F2FS_SB(sb));
- f2fs_stop_discard_thread(F2FS_SB(sb));
+ struct f2fs_sb_info *sbi = F2FS_SB(sb);
+
+ set_sbi_flag(sbi, SBI_IS_CLOSE);
+ f2fs_stop_gc_thread(sbi);
+ f2fs_stop_discard_thread(sbi);
+
+ if (is_sbi_flag_set(sbi, SBI_IS_DIRTY) ||
+ !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) {
+ struct cp_control cpc = {
+ .reason = CP_UMOUNT,
+ };
+ f2fs_write_checkpoint(sbi, &cpc);
+ }
}
kill_block_super(sb);
}
--
2.17.0.441.gb46fe60e1d-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: do checkpoint in kill_sb
2018-07-07 1:20 [PATCH] f2fs: do checkpoint in kill_sb Jaegeuk Kim
@ 2018-07-09 15:04 ` Chao Yu
2018-07-09 20:36 ` Jaegeuk Kim
0 siblings, 1 reply; 6+ messages in thread
From: Chao Yu @ 2018-07-09 15:04 UTC (permalink / raw)
To: Jaegeuk Kim, linux-kernel, linux-f2fs-devel
On 2018/7/7 9:20, Jaegeuk Kim wrote:
> When unmounting f2fs in force mode, we can get it stuck by io_schedule()
force mode means force shutdown?
> by some pending IOs in meta_inode.
>
> io_schedule+0xd/0x30
> wait_on_page_bit_common+0xc6/0x130
Looks like a deadlock here? Does this mean we forget to submit cached bio in
somewhere?
Thanks,
> __filemap_fdatawait_range+0xbd/0x100
> filemap_fdatawait_keep_errors+0x15/0x40
> sync_inodes_sb+0x1cf/0x240
> sync_filesystem+0x52/0x90
> generic_shutdown_super+0x1d/0x110
> ceph_kill_sb+0x28/0x80 [ceph]
> deactivate_locked_super+0x35/0x60
> cleanup_mnt+0x36/0x70
> task_work_run+0x79/0xa0
> exit_to_usermode_loop+0x62/0x70
> do_syscall_64+0xdb/0xf0
> entry_SYSCALL_64_after_hwframe+0x44/0xa9
> 0xffffffffffffffff
>
> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> ---
> fs/f2fs/super.c | 16 +++++++++++++---
> 1 file changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> index 3995e926ba3a..1dc6809fac38 100644
> --- a/fs/f2fs/super.c
> +++ b/fs/f2fs/super.c
> @@ -3089,9 +3089,19 @@ static struct dentry *f2fs_mount(struct file_system_type *fs_type, int flags,
> static void kill_f2fs_super(struct super_block *sb)
> {
> if (sb->s_root) {
> - set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
> - f2fs_stop_gc_thread(F2FS_SB(sb));
> - f2fs_stop_discard_thread(F2FS_SB(sb));
> + struct f2fs_sb_info *sbi = F2FS_SB(sb);
> +
> + set_sbi_flag(sbi, SBI_IS_CLOSE);
> + f2fs_stop_gc_thread(sbi);
> + f2fs_stop_discard_thread(sbi);
> +
> + if (is_sbi_flag_set(sbi, SBI_IS_DIRTY) ||
> + !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) {
> + struct cp_control cpc = {
> + .reason = CP_UMOUNT,
> + };
> + f2fs_write_checkpoint(sbi, &cpc);
> + }
> }
> kill_block_super(sb);
> }
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: do checkpoint in kill_sb
2018-07-09 15:04 ` [f2fs-dev] " Chao Yu
@ 2018-07-09 20:36 ` Jaegeuk Kim
2018-07-10 6:09 ` Chao Yu
0 siblings, 1 reply; 6+ messages in thread
From: Jaegeuk Kim @ 2018-07-09 20:36 UTC (permalink / raw)
To: Chao Yu; +Cc: linux-kernel, linux-f2fs-devel
On 07/09, Chao Yu wrote:
> On 2018/7/7 9:20, Jaegeuk Kim wrote:
> > When unmounting f2fs in force mode, we can get it stuck by io_schedule()
>
> force mode means force shutdown?
Yes.
>
> > by some pending IOs in meta_inode.
> >
> > io_schedule+0xd/0x30
> > wait_on_page_bit_common+0xc6/0x130
>
> Looks like a deadlock here? Does this mean we forget to submit cached bio in
> somewhere?
I'm hitting this randomly, when running all the testcases in xfstests.
I can't find the missing flow, and only get this during kill_sb.
>
> Thanks,
>
> > __filemap_fdatawait_range+0xbd/0x100
> > filemap_fdatawait_keep_errors+0x15/0x40
> > sync_inodes_sb+0x1cf/0x240
> > sync_filesystem+0x52/0x90
> > generic_shutdown_super+0x1d/0x110
> > ceph_kill_sb+0x28/0x80 [ceph]
> > deactivate_locked_super+0x35/0x60
> > cleanup_mnt+0x36/0x70
> > task_work_run+0x79/0xa0
> > exit_to_usermode_loop+0x62/0x70
> > do_syscall_64+0xdb/0xf0
> > entry_SYSCALL_64_after_hwframe+0x44/0xa9
> > 0xffffffffffffffff
> >
> > Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> > ---
> > fs/f2fs/super.c | 16 +++++++++++++---
> > 1 file changed, 13 insertions(+), 3 deletions(-)
> >
> > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> > index 3995e926ba3a..1dc6809fac38 100644
> > --- a/fs/f2fs/super.c
> > +++ b/fs/f2fs/super.c
> > @@ -3089,9 +3089,19 @@ static struct dentry *f2fs_mount(struct file_system_type *fs_type, int flags,
> > static void kill_f2fs_super(struct super_block *sb)
> > {
> > if (sb->s_root) {
> > - set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
> > - f2fs_stop_gc_thread(F2FS_SB(sb));
> > - f2fs_stop_discard_thread(F2FS_SB(sb));
> > + struct f2fs_sb_info *sbi = F2FS_SB(sb);
> > +
> > + set_sbi_flag(sbi, SBI_IS_CLOSE);
> > + f2fs_stop_gc_thread(sbi);
> > + f2fs_stop_discard_thread(sbi);
> > +
> > + if (is_sbi_flag_set(sbi, SBI_IS_DIRTY) ||
> > + !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) {
> > + struct cp_control cpc = {
> > + .reason = CP_UMOUNT,
> > + };
> > + f2fs_write_checkpoint(sbi, &cpc);
> > + }
> > }
> > kill_block_super(sb);
> > }
> >
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: do checkpoint in kill_sb
2018-07-09 20:36 ` Jaegeuk Kim
@ 2018-07-10 6:09 ` Chao Yu
2018-07-12 7:34 ` Jaegeuk Kim
0 siblings, 1 reply; 6+ messages in thread
From: Chao Yu @ 2018-07-10 6:09 UTC (permalink / raw)
To: Jaegeuk Kim, Chao Yu; +Cc: linux-kernel, linux-f2fs-devel
On 2018/7/10 4:36, Jaegeuk Kim wrote:
> On 07/09, Chao Yu wrote:
>> On 2018/7/7 9:20, Jaegeuk Kim wrote:
>>> When unmounting f2fs in force mode, we can get it stuck by io_schedule()
>>
>> force mode means force shutdown?
>
> Yes.
>
>>
>>> by some pending IOs in meta_inode.
>>>
>>> io_schedule+0xd/0x30
>>> wait_on_page_bit_common+0xc6/0x130
>>
>> Looks like a deadlock here? Does this mean we forget to submit cached bio in
>> somewhere?
>
> I'm hitting this randomly, when running all the testcases in xfstests.
> I can't find the missing flow, and only get this during kill_sb.
I guess we need this before returning from ->writepage{,s}:
if (is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN))
f2fs_submit_merged_write()
How about trying this instead?
Thanks,
>
>>
>> Thanks,
>>
>>> __filemap_fdatawait_range+0xbd/0x100
>>> filemap_fdatawait_keep_errors+0x15/0x40
>>> sync_inodes_sb+0x1cf/0x240
>>> sync_filesystem+0x52/0x90
>>> generic_shutdown_super+0x1d/0x110
>>> ceph_kill_sb+0x28/0x80 [ceph]
>>> deactivate_locked_super+0x35/0x60
>>> cleanup_mnt+0x36/0x70
>>> task_work_run+0x79/0xa0
>>> exit_to_usermode_loop+0x62/0x70
>>> do_syscall_64+0xdb/0xf0
>>> entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>> 0xffffffffffffffff
>>>
>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
>>> ---
>>> fs/f2fs/super.c | 16 +++++++++++++---
>>> 1 file changed, 13 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
>>> index 3995e926ba3a..1dc6809fac38 100644
>>> --- a/fs/f2fs/super.c
>>> +++ b/fs/f2fs/super.c
>>> @@ -3089,9 +3089,19 @@ static struct dentry *f2fs_mount(struct file_system_type *fs_type, int flags,
>>> static void kill_f2fs_super(struct super_block *sb)
>>> {
>>> if (sb->s_root) {
>>> - set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
>>> - f2fs_stop_gc_thread(F2FS_SB(sb));
>>> - f2fs_stop_discard_thread(F2FS_SB(sb));
>>> + struct f2fs_sb_info *sbi = F2FS_SB(sb);
>>> +
>>> + set_sbi_flag(sbi, SBI_IS_CLOSE);
>>> + f2fs_stop_gc_thread(sbi);
>>> + f2fs_stop_discard_thread(sbi);
>>> +
>>> + if (is_sbi_flag_set(sbi, SBI_IS_DIRTY) ||
>>> + !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) {
>>> + struct cp_control cpc = {
>>> + .reason = CP_UMOUNT,
>>> + };
>>> + f2fs_write_checkpoint(sbi, &cpc);
>>> + }
>>> }
>>> kill_block_super(sb);
>>> }
>>>
>
> .
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: do checkpoint in kill_sb
2018-07-10 6:09 ` Chao Yu
@ 2018-07-12 7:34 ` Jaegeuk Kim
2018-07-12 8:23 ` Chao Yu
0 siblings, 1 reply; 6+ messages in thread
From: Jaegeuk Kim @ 2018-07-12 7:34 UTC (permalink / raw)
To: Chao Yu; +Cc: Chao Yu, linux-kernel, linux-f2fs-devel
On 07/10, Chao Yu wrote:
> On 2018/7/10 4:36, Jaegeuk Kim wrote:
> > On 07/09, Chao Yu wrote:
> >> On 2018/7/7 9:20, Jaegeuk Kim wrote:
> >>> When unmounting f2fs in force mode, we can get it stuck by io_schedule()
> >>
> >> force mode means force shutdown?
> >
> > Yes.
> >
> >>
> >>> by some pending IOs in meta_inode.
> >>>
> >>> io_schedule+0xd/0x30
> >>> wait_on_page_bit_common+0xc6/0x130
> >>
> >> Looks like a deadlock here? Does this mean we forget to submit cached bio in
> >> somewhere?
> >
> > I'm hitting this randomly, when running all the testcases in xfstests.
> > I can't find the missing flow, and only get this during kill_sb.
>
> I guess we need this before returning from ->writepage{,s}:
>
> if (is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN))
> f2fs_submit_merged_write()
>
> How about trying this instead?
No, it's not shutdown case unfortunately.
>
> Thanks,
>
> >
> >>
> >> Thanks,
> >>
> >>> __filemap_fdatawait_range+0xbd/0x100
> >>> filemap_fdatawait_keep_errors+0x15/0x40
> >>> sync_inodes_sb+0x1cf/0x240
> >>> sync_filesystem+0x52/0x90
> >>> generic_shutdown_super+0x1d/0x110
> >>> ceph_kill_sb+0x28/0x80 [ceph]
> >>> deactivate_locked_super+0x35/0x60
> >>> cleanup_mnt+0x36/0x70
> >>> task_work_run+0x79/0xa0
> >>> exit_to_usermode_loop+0x62/0x70
> >>> do_syscall_64+0xdb/0xf0
> >>> entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >>> 0xffffffffffffffff
> >>>
> >>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> >>> ---
> >>> fs/f2fs/super.c | 16 +++++++++++++---
> >>> 1 file changed, 13 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> >>> index 3995e926ba3a..1dc6809fac38 100644
> >>> --- a/fs/f2fs/super.c
> >>> +++ b/fs/f2fs/super.c
> >>> @@ -3089,9 +3089,19 @@ static struct dentry *f2fs_mount(struct file_system_type *fs_type, int flags,
> >>> static void kill_f2fs_super(struct super_block *sb)
> >>> {
> >>> if (sb->s_root) {
> >>> - set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
> >>> - f2fs_stop_gc_thread(F2FS_SB(sb));
> >>> - f2fs_stop_discard_thread(F2FS_SB(sb));
> >>> + struct f2fs_sb_info *sbi = F2FS_SB(sb);
> >>> +
> >>> + set_sbi_flag(sbi, SBI_IS_CLOSE);
> >>> + f2fs_stop_gc_thread(sbi);
> >>> + f2fs_stop_discard_thread(sbi);
> >>> +
> >>> + if (is_sbi_flag_set(sbi, SBI_IS_DIRTY) ||
> >>> + !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) {
> >>> + struct cp_control cpc = {
> >>> + .reason = CP_UMOUNT,
> >>> + };
> >>> + f2fs_write_checkpoint(sbi, &cpc);
> >>> + }
> >>> }
> >>> kill_block_super(sb);
> >>> }
> >>>
> >
> > .
> >
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: do checkpoint in kill_sb
2018-07-12 7:34 ` Jaegeuk Kim
@ 2018-07-12 8:23 ` Chao Yu
0 siblings, 0 replies; 6+ messages in thread
From: Chao Yu @ 2018-07-12 8:23 UTC (permalink / raw)
To: Jaegeuk Kim; +Cc: Chao Yu, linux-kernel, linux-f2fs-devel
On 2018/7/12 15:34, Jaegeuk Kim wrote:
> On 07/10, Chao Yu wrote:
>> On 2018/7/10 4:36, Jaegeuk Kim wrote:
>>> On 07/09, Chao Yu wrote:
>>>> On 2018/7/7 9:20, Jaegeuk Kim wrote:
>>>>> When unmounting f2fs in force mode, we can get it stuck by io_schedule()
>>>>
>>>> force mode means force shutdown?
>>>
>>> Yes.
>>>
>>>>
>>>>> by some pending IOs in meta_inode.
>>>>>
>>>>> io_schedule+0xd/0x30
>>>>> wait_on_page_bit_common+0xc6/0x130
>>>>
>>>> Looks like a deadlock here? Does this mean we forget to submit cached bio in
>>>> somewhere?
>>>
>>> I'm hitting this randomly, when running all the testcases in xfstests.
>>> I can't find the missing flow, and only get this during kill_sb.
>>
>> I guess we need this before returning from ->writepage{,s}:
>>
>> if (is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN))
>> f2fs_submit_merged_write()
>>
>> How about trying this instead?
>
> No, it's not shutdown case unfortunately.
Still I didn't understand why this patch can fix the issue... :(
Thanks,
>
>>
>> Thanks,
>>
>>>
>>>>
>>>> Thanks,
>>>>
>>>>> __filemap_fdatawait_range+0xbd/0x100
>>>>> filemap_fdatawait_keep_errors+0x15/0x40
>>>>> sync_inodes_sb+0x1cf/0x240
>>>>> sync_filesystem+0x52/0x90
>>>>> generic_shutdown_super+0x1d/0x110
>>>>> ceph_kill_sb+0x28/0x80 [ceph]
>>>>> deactivate_locked_super+0x35/0x60
>>>>> cleanup_mnt+0x36/0x70
>>>>> task_work_run+0x79/0xa0
>>>>> exit_to_usermode_loop+0x62/0x70
>>>>> do_syscall_64+0xdb/0xf0
>>>>> entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>> 0xffffffffffffffff
>>>>>
>>>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
>>>>> ---
>>>>> fs/f2fs/super.c | 16 +++++++++++++---
>>>>> 1 file changed, 13 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
>>>>> index 3995e926ba3a..1dc6809fac38 100644
>>>>> --- a/fs/f2fs/super.c
>>>>> +++ b/fs/f2fs/super.c
>>>>> @@ -3089,9 +3089,19 @@ static struct dentry *f2fs_mount(struct file_system_type *fs_type, int flags,
>>>>> static void kill_f2fs_super(struct super_block *sb)
>>>>> {
>>>>> if (sb->s_root) {
>>>>> - set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
>>>>> - f2fs_stop_gc_thread(F2FS_SB(sb));
>>>>> - f2fs_stop_discard_thread(F2FS_SB(sb));
>>>>> + struct f2fs_sb_info *sbi = F2FS_SB(sb);
>>>>> +
>>>>> + set_sbi_flag(sbi, SBI_IS_CLOSE);
>>>>> + f2fs_stop_gc_thread(sbi);
>>>>> + f2fs_stop_discard_thread(sbi);
>>>>> +
>>>>> + if (is_sbi_flag_set(sbi, SBI_IS_DIRTY) ||
>>>>> + !is_set_ckpt_flags(sbi, CP_UMOUNT_FLAG)) {
>>>>> + struct cp_control cpc = {
>>>>> + .reason = CP_UMOUNT,
>>>>> + };
>>>>> + f2fs_write_checkpoint(sbi, &cpc);
>>>>> + }
>>>>> }
>>>>> kill_block_super(sb);
>>>>> }
>>>>>
>>>
>>> .
>>>
>
> .
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-07-12 8:23 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-07 1:20 [PATCH] f2fs: do checkpoint in kill_sb Jaegeuk Kim
2018-07-09 15:04 ` [f2fs-dev] " Chao Yu
2018-07-09 20:36 ` Jaegeuk Kim
2018-07-10 6:09 ` Chao Yu
2018-07-12 7:34 ` Jaegeuk Kim
2018-07-12 8:23 ` Chao Yu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).