[PATCH] f2fs: avoid hungtask problem caused by f2fs_balance_fs

[PATCH] f2fs: avoid hungtask problem caused by f2fs_balance_fs

[Yunlei He]

I came across a hungtask problem as below:

INFO: task 962 blocked for more than 120 s.

Call trace:
[<ffffffc000085a24>] __switch_to+0x74/0x8c
[<ffffffc000dd1434>] __schedule+0x3b4/0x8f4
[<ffffffc000dd1998>] schedule+0x24/0x68
[<ffffffc000dd1e80>] schedule_preempt_disabled+0x10/0x24
[<ffffffc000dd04b4>] __mutex_lock_slowpath+0x154/0x26c
[<ffffffc000dd0610>] mutex_lock+0x44/0x64
[<ffffffc0003471d0>] f2fs_balance_fs+0x78/0x94
[<ffffffc00032ca00>] f2fs_create+0x30/0x1d4
[<ffffffc0001a5e40>] vfs_create+0xc8/0xf8
[<ffffffc0001a7a34>] do_last.isra.43+0x608/0xc44
[<ffffffc0001a8118>] path_openat.isra.44+0xa8/0x4b0
[<ffffffc0001a92dc>] do_filp_open+0x2c/0x98
[<ffffffc000198da0>] do_sys_open+0x104/0x1c8
[<ffffffc000198e98>] SyS_openat+0xc/0x18

Besides, there are also lots of other operations were blocked
by gc_mutex like here.

The comments said we should do GC or end up with checkpoint, if there
are so many dirty dir/node pages without enough free segments. But should
all theads wait for gc if the condition is not satisfied? Maybe if someone
is gcing, they can by pass it.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
---
 fs/f2fs/segment.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 6f16b39..48ce7ab 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -345,8 +345,8 @@ void f2fs_balance_fs(struct f2fs_sb_info *sbi, bool need)
 	 * We should do GC or end up with checkpoint, if there are so many dirty
 	 * dir/node pages without enough free segments.
 	 */
-	if (has_not_enough_free_secs(sbi, 0)) {
-		mutex_lock(&sbi->gc_mutex);
+	if (has_not_enough_free_secs(sbi, 0) &&
+					mutex_trylock(&sbi->gc_mutex)) {
 		f2fs_gc(sbi, false);
 	}
 }
-- 
1.9.1


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140

Re: [PATCH] f2fs: avoid hungtask problem caused by f2fs_balance_fs

[Chao Yu]

Hi Yunlei,

On 2016/2/26 15:13, Yunlei He wrote:
> I came across a hungtask problem as below:
> 
> INFO: task 962 blocked for more than 120 s.
> 
> Call trace:
> [<ffffffc000085a24>] __switch_to+0x74/0x8c
> [<ffffffc000dd1434>] __schedule+0x3b4/0x8f4
> [<ffffffc000dd1998>] schedule+0x24/0x68
> [<ffffffc000dd1e80>] schedule_preempt_disabled+0x10/0x24
> [<ffffffc000dd04b4>] __mutex_lock_slowpath+0x154/0x26c
> [<ffffffc000dd0610>] mutex_lock+0x44/0x64
> [<ffffffc0003471d0>] f2fs_balance_fs+0x78/0x94
> [<ffffffc00032ca00>] f2fs_create+0x30/0x1d4
> [<ffffffc0001a5e40>] vfs_create+0xc8/0xf8
> [<ffffffc0001a7a34>] do_last.isra.43+0x608/0xc44
> [<ffffffc0001a8118>] path_openat.isra.44+0xa8/0x4b0
> [<ffffffc0001a92dc>] do_filp_open+0x2c/0x98
> [<ffffffc000198da0>] do_sys_open+0x104/0x1c8
> [<ffffffc000198e98>] SyS_openat+0xc/0x18
> 
> Besides, there are also lots of other operations were blocked
> by gc_mutex like here.
> 
> The comments said we should do GC or end up with checkpoint, if there
> are so many dirty dir/node pages without enough free segments. But should
> all theads wait for gc if the condition is not satisfied? Maybe if someone
> is gcing, they can by pass it.

I don't think so, once there is one thread goes into GC flow, if we don't block
all other foreground gc threads here, these concurrent threads may generate more
dirty node/dentry pages, which can easily lead sync_node_pages in GC or
checkpoint runs out of free sections.

Thanks,

> 
> Signed-off-by: Yunlei He <heyunlei@huawei.com>
> ---
>  fs/f2fs/segment.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
> index 6f16b39..48ce7ab 100644
> --- a/fs/f2fs/segment.c
> +++ b/fs/f2fs/segment.c
> @@ -345,8 +345,8 @@ void f2fs_balance_fs(struct f2fs_sb_info *sbi, bool need)
>  	 * We should do GC or end up with checkpoint, if there are so many dirty
>  	 * dir/node pages without enough free segments.
>  	 */
> -	if (has_not_enough_free_secs(sbi, 0)) {
> -		mutex_lock(&sbi->gc_mutex);
> +	if (has_not_enough_free_secs(sbi, 0) &&
> +					mutex_trylock(&sbi->gc_mutex)) {
>  		f2fs_gc(sbi, false);
>  	}
>  }
> 

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140