From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755676AbdKJCNp (ORCPT ); Thu, 9 Nov 2017 21:13:45 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:10480 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755528AbdKJCNn (ORCPT ); Thu, 9 Nov 2017 21:13:43 -0500 Subject: Re: [PATCH v2] f2fs: add bug_on when f2fs_gc even fails to get one victim To: Jaegeuk Kim CC: Chao Yu , , , , , , , References: <1507901500-162168-1-git-send-email-yunlong.song@huawei.com> <20171103034402.GC11335@jaegeuk-macbookpro.roam.corp.google.com> <3b26badb-c34e-3c2d-74a2-26ecef1f64ff@huawei.com> <20171107023835.GG88544@jaegeuk-macbookpro.roam.corp.google.com> <20171107024058.GH88544@jaegeuk-macbookpro.roam.corp.google.com> <20171107032636.GB92599@jaegeuk-macbookpro.roam.corp.google.com> <7433d584-b1e0-76e9-a6e4-8da1218dd71d@huawei.com> <19eec8fd-e941-770a-9477-6ad2d266d475@huawei.com> <20171109175915.GC55844@jaegeuk-macbookpro.roam.corp.google.com> From: Yunlong Song Message-ID: <67f53739-ccc5-59be-9770-85d3ab76ed07@huawei.com> Date: Fri, 10 Nov 2017 10:11:15 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20171109175915.GC55844@jaegeuk-macbookpro.roam.corp.google.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Originating-IP: [10.111.220.140] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020204.5A050B47.00D7,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 919335ccba5f1f7aaf58c8efa76eea8f Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Agree. On 2017/11/10 1:59, Jaegeuk Kim wrote: > On 11/08, Yunlong Song wrote: >> So we should use f2fs_bug_on(sbi, !total_freed && !sync && gc_type == >> FG_GC); > f2fs_bug_on(sbi, !total_freed && has_not_enough_free_secs(sbi, 0, 0)); ? > >> On 2017/11/7 14:56, Chao Yu wrote: >>> On 2017/11/7 12:01, Yunlong Song wrote: >>>> Sorry, misunderstanding, because I think when sync == true, FG_GC does not >>>> check has_not_enough_free_secs, so maybe it does not have to do any gc >>>> at all. >>>> For example, if there are 100 segments for f2fs, and 20 segments are full or >>>> valid blocks over fggc_threshold, then it is correct to fail in get victim. >>>> >>>> >>>> On 2017/11/7 11:26, Jaegeuk Kim wrote: >>>>> On 11/07, Yunlong Song wrote: >>>>>> Because I find that some out-of-free problem is caused by the failure >>>>>> of get victim target. For example, chao has pointed out that he has >>>>>> found out a bug when adding this bug_on last week. >>>>> That's NOT what I asked. Why not checking FG_GC all the time like this? >>>>> >>>>> f2fs_bug_on(sbi, !total_freed && gc_type == FG_GC); >>> ioctl(F2FS_IOC_GARBAGE_COLLECT, &1) will simply trigger this bug_on, so we >>> have to check the conditon only when we run out-of-free-space? >>> >>> Thanks, >>> >>>>>> On 2017/11/7 10:40, Jaegeuk Kim wrote: >>>>>>> On 11/06, Jaegeuk Kim wrote: >>>>>>>> On 11/06, Yunlong Song wrote: >>>>>>>>> Agree. >>>>>>>>> >>>>>>>>> On 2017/11/3 11:44, Jaegeuk Kim wrote: >>>>>>>>>> On 10/13, Yunlong Song wrote: >>>>>>>>>>> This can help us to debug on some corner case. >>>>>>>>>>> >>>>>>>>>>> Signed-off-by: Yunlong Song >>>>>>>>>>> Signed-off-by: Chao Yu >>>>>>>>>>> --- >>>>>>>>>>> fs/f2fs/gc.c | 6 +++++- >>>>>>>>>>> 1 file changed, 5 insertions(+), 1 deletion(-) >>>>>>>>>>> >>>>>>>>>>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>>>>>>>>>> index 197ebf4..2b03202 100644 >>>>>>>>>>> --- a/fs/f2fs/gc.c >>>>>>>>>>> +++ b/fs/f2fs/gc.c >>>>>>>>>>> @@ -986,6 +986,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>>>>>>>>>> .ilist = LIST_HEAD_INIT(gc_list.ilist), >>>>>>>>>>> .iroot = RADIX_TREE_INIT(GFP_NOFS), >>>>>>>>>>> }; >>>>>>>>>>> + bool need_fggc = false; >>>>>>>>>>> trace_f2fs_gc_begin(sbi->sb, sync, background, >>>>>>>>>>> get_pages(sbi, F2FS_DIRTY_NODES), >>>>>>>>>>> @@ -1018,8 +1019,10 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>>>>>>>>>> if (ret) >>>>>>>>>>> goto stop; >>>>>>>>>>> } >>>>>>>>>>> - if (has_not_enough_free_secs(sbi, 0, 0)) >>>>>>>>>>> + if (has_not_enough_free_secs(sbi, 0, 0)) { >>>>>>>>>>> gc_type = FG_GC; >>>>>>>>>>> + need_fggc = true; >>>>>>>>>>> + } >>>>>>>>>>> } >>>>>>>>>>> /* f2fs_balance_fs doesn't need to do BG_GC in critical path. */ >>>>>>>>>>> @@ -1028,6 +1031,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>>>>>>>>>> goto stop; >>>>>>>>>>> } >>>>>>>>>>> if (!__get_victim(sbi, &segno, gc_type)) { >>>>>>>>>>> + f2fs_bug_on(sbi, !total_freed && need_fggc); >>>>>>>>>> Just like this? >>>>>>>>> That's OK. >>>>>>>> I'm not quite sure whether this is really a bug_on case. >>>>>>>> Let me make it WARN_ON() for debugging purpose first. >>>>>>> BTW, why is this the special case where BG_GC detects FG_GC? >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>>>> f2fs_bug_on(sbi, !total_freed && !sync && gc_type == FG_GC); >>>>>>>>>> >>>>>>>>>>> ret = -ENODATA; >>>>>>>>>>> goto stop; >>>>>>>>>>> } >>>>>>>>>>> -- >>>>>>>>>>> 1.8.5.2 >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Thanks, >>>>>>>>> Yunlong Song >>>>>>>>> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> Yunlong Song >>>>>> >>>>> . >>>>> >>> . >>> >> -- >> Thanks, >> Yunlong Song >> > . > -- Thanks, Yunlong Song From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yunlong Song Subject: Re: [PATCH v2] f2fs: add bug_on when f2fs_gc even fails to get one victim Date: Fri, 10 Nov 2017 10:11:15 +0800 Message-ID: <67f53739-ccc5-59be-9770-85d3ab76ed07@huawei.com> References: <1507901500-162168-1-git-send-email-yunlong.song@huawei.com> <20171103034402.GC11335@jaegeuk-macbookpro.roam.corp.google.com> <3b26badb-c34e-3c2d-74a2-26ecef1f64ff@huawei.com> <20171107023835.GG88544@jaegeuk-macbookpro.roam.corp.google.com> <20171107024058.GH88544@jaegeuk-macbookpro.roam.corp.google.com> <20171107032636.GB92599@jaegeuk-macbookpro.roam.corp.google.com> <7433d584-b1e0-76e9-a6e4-8da1218dd71d@huawei.com> <19eec8fd-e941-770a-9477-6ad2d266d475@huawei.com> <20171109175915.GC55844@jaegeuk-macbookpro.roam.corp.google.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20171109175915.GC55844@jaegeuk-macbookpro.roam.corp.google.com> Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org To: Jaegeuk Kim Cc: Chao Yu , chao@kernel.org, yunlong.song@icloud.com, miaoxie@huawei.com, bintian.wang@huawei.com, linux-fsdevel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org List-Id: linux-f2fs-devel.lists.sourceforge.net Agree. On 2017/11/10 1:59, Jaegeuk Kim wrote: > On 11/08, Yunlong Song wrote: >> So we should use f2fs_bug_on(sbi, !total_freed && !sync && gc_type == >> FG_GC); > f2fs_bug_on(sbi, !total_freed && has_not_enough_free_secs(sbi, 0, 0)); ? > >> On 2017/11/7 14:56, Chao Yu wrote: >>> On 2017/11/7 12:01, Yunlong Song wrote: >>>> Sorry, misunderstanding, because I think when sync == true, FG_GC does not >>>> check has_not_enough_free_secs, so maybe it does not have to do any gc >>>> at all. >>>> For example, if there are 100 segments for f2fs, and 20 segments are full or >>>> valid blocks over fggc_threshold, then it is correct to fail in get victim. >>>> >>>> >>>> On 2017/11/7 11:26, Jaegeuk Kim wrote: >>>>> On 11/07, Yunlong Song wrote: >>>>>> Because I find that some out-of-free problem is caused by the failure >>>>>> of get victim target. For example, chao has pointed out that he has >>>>>> found out a bug when adding this bug_on last week. >>>>> That's NOT what I asked. Why not checking FG_GC all the time like this? >>>>> >>>>> f2fs_bug_on(sbi, !total_freed && gc_type == FG_GC); >>> ioctl(F2FS_IOC_GARBAGE_COLLECT, &1) will simply trigger this bug_on, so we >>> have to check the conditon only when we run out-of-free-space? >>> >>> Thanks, >>> >>>>>> On 2017/11/7 10:40, Jaegeuk Kim wrote: >>>>>>> On 11/06, Jaegeuk Kim wrote: >>>>>>>> On 11/06, Yunlong Song wrote: >>>>>>>>> Agree. >>>>>>>>> >>>>>>>>> On 2017/11/3 11:44, Jaegeuk Kim wrote: >>>>>>>>>> On 10/13, Yunlong Song wrote: >>>>>>>>>>> This can help us to debug on some corner case. >>>>>>>>>>> >>>>>>>>>>> Signed-off-by: Yunlong Song >>>>>>>>>>> Signed-off-by: Chao Yu >>>>>>>>>>> --- >>>>>>>>>>> fs/f2fs/gc.c | 6 +++++- >>>>>>>>>>> 1 file changed, 5 insertions(+), 1 deletion(-) >>>>>>>>>>> >>>>>>>>>>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>>>>>>>>>> index 197ebf4..2b03202 100644 >>>>>>>>>>> --- a/fs/f2fs/gc.c >>>>>>>>>>> +++ b/fs/f2fs/gc.c >>>>>>>>>>> @@ -986,6 +986,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>>>>>>>>>> .ilist = LIST_HEAD_INIT(gc_list.ilist), >>>>>>>>>>> .iroot = RADIX_TREE_INIT(GFP_NOFS), >>>>>>>>>>> }; >>>>>>>>>>> + bool need_fggc = false; >>>>>>>>>>> trace_f2fs_gc_begin(sbi->sb, sync, background, >>>>>>>>>>> get_pages(sbi, F2FS_DIRTY_NODES), >>>>>>>>>>> @@ -1018,8 +1019,10 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>>>>>>>>>> if (ret) >>>>>>>>>>> goto stop; >>>>>>>>>>> } >>>>>>>>>>> - if (has_not_enough_free_secs(sbi, 0, 0)) >>>>>>>>>>> + if (has_not_enough_free_secs(sbi, 0, 0)) { >>>>>>>>>>> gc_type = FG_GC; >>>>>>>>>>> + need_fggc = true; >>>>>>>>>>> + } >>>>>>>>>>> } >>>>>>>>>>> /* f2fs_balance_fs doesn't need to do BG_GC in critical path. */ >>>>>>>>>>> @@ -1028,6 +1031,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>>>>>>>>>> goto stop; >>>>>>>>>>> } >>>>>>>>>>> if (!__get_victim(sbi, &segno, gc_type)) { >>>>>>>>>>> + f2fs_bug_on(sbi, !total_freed && need_fggc); >>>>>>>>>> Just like this? >>>>>>>>> That's OK. >>>>>>>> I'm not quite sure whether this is really a bug_on case. >>>>>>>> Let me make it WARN_ON() for debugging purpose first. >>>>>>> BTW, why is this the special case where BG_GC detects FG_GC? >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>>>> f2fs_bug_on(sbi, !total_freed && !sync && gc_type == FG_GC); >>>>>>>>>> >>>>>>>>>>> ret = -ENODATA; >>>>>>>>>>> goto stop; >>>>>>>>>>> } >>>>>>>>>>> -- >>>>>>>>>>> 1.8.5.2 >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Thanks, >>>>>>>>> Yunlong Song >>>>>>>>> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> Yunlong Song >>>>>> >>>>> . >>>>> >>> . >>> >> -- >> Thanks, >> Yunlong Song >> > . > -- Thanks, Yunlong Song