Re: [f2fs-dev] Question about f2fs UDC(userdata checkpointing)

Re: [f2fs-dev] Question about f2fs UDC(userdata checkpointing)

[Chao Yu]

Hello,

On 2019/10/28 14:37, 王矛 wrote:
> Hi Chao,
> 
> Sorry to bother you again with last question about this topic. :)
> 
> "Actually, APP can update data during CP disabling, unless there is no enough
> free space"
> So we don't prevent app update data during CP disabling, but we prevent the
> dirty node/data information to be synced into storage, right?
> 
> From code i can see below sync operation will return directly if find CP is
> disabled:
>         f2fs_sync_fs()
>         f2fs_do_sync_file()
>             if (unlikely(is_sbi_flag_set(sbi, SBI_CP_DISABLED)))
>                   return 0;
> 
> My question is, i didn't find any code to check CP is disabled in the Wribe-Back
> procedure.
> As you know, WB procedure would be executed periodically and flush dirty data
> into storage.
> Thus if WB is not prevented during CP disabled, data still be updated onto
> flash,  this is out of expectation of UDC, is this right?

We don't need to take care of WB, if CP and fsync is disabled, after sudden
power-cut, we will rollback to previous CP, which will not contain writebacked
data/node.

Thanks,

> 
> Thanks,
> Mao
> 
> 
> At 2019-10-24 21:15:20, "王矛" <spearmao@126.com> wrote:
> 
>     Chao,
> 
>     Very appreciated for your kind and prompt reply. :)
> 
>     Let me try to spend more time on the f2fs UDC related code and get back
>     later if I have further questions.
> 
>     Thanks!
>     Mao
> 
>     At 2019-10-23 17:42:03, "Chao Yu" <yuchao0@huawei.com> wrote:
>     >Hello,
>     >
>     >On 2019/10/23 16:35, 王矛 wrote:
>     >
>     >[snip]
>     >
>     >> When CP is disabled, I saw many f2fs operations are prevented(listed above).
>     >> 
>     >> If so, during the period CP disabled, app can’t do any change to f2fs? 
>     >
>     >Please Cc f2fs mailing list for any f2fs question.
>     >
>     >Actually, APP can update data during CP disabling, unless there is no enough
>     >free space, the logic was indicated by below codes:
>     >
>     >static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi)
>     >{
>     >	if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED)))
>     >		return true;
>     >	if (likely(!has_not_enough_free_secs(sbi, 0, 0)))
>     >		return true;
>     >	return false;
>     >}
>     >
>     >static int f2fs_mknod(struct inode *dir, struct dentry *dentry,
>     >				umode_t mode, dev_t rdev)
>     >{
>     >	struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
>     >	struct inode *inode;
>     >	int err = 0;
>     >
>     >	if (unlikely(f2fs_cp_error(sbi)))
>     >		return -EIO;
>     >	if (!f2fs_is_checkpoint_ready(sbi))
>     >		return -ENOSPC;
>     >...
>     >}
>     >
>     >Once all data was updated by Android, we can terminate CP disabling status, and
>     >trigger a checkpoint to persist all previous updates.
>     >
>     >Thanks,
>     >
>     >> 
>     >> If yes, how it implemented the checkpoint function?
>     >> 
>     >> Maybe the patch I
>     >> found(https://sourceforge.net/p/linux-f2fs/mailman/message/36425511/) is not the
>     >> whole implementation of this feaute….
>     >> 
>     >>  
>     >> 
>     >> 
>     >> Thanks,
>     >> 
>     >> Mao
>     >> 
>     >> 
>     >> 
>     >>  
>     >> 
> 
> 
> 
>      
> 
> 
> 
>  
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] F2fs mount cost long time due to waiting f2fs gc with f2fs checkpoint disabled

[Chao Yu]

Hello,

Sorry for the delay reply.

On 2019/11/21 18:48, 王矛 wrote:
> Hi Chao,
> 
> Recently I encounter an issue that f2fs mount cost quite a long time(the longest time i met is almost 4min to mount f2fs completely).
> Below is a typical log:
> 01-24 18:42:09.825     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:42:15.369     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:42:20.952     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:42:26.510     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:42:32.288     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:42:37.843     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:42:43.476     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> ...
> 01-24 18:44:48.873     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:44:54.606     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:45:23.337     0     0 I init    : [libfs_mgr]Retrying mount (source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=-1(11): Try again
> 01-24 18:45:29.065     0     0 W F2FS-fs (mmcblk0p80): Start checkpoint disabled!
> 01-24 18:45:29.077     0     0 I F2FS-fs (mmcblk0p80): Mounted with checkpoint version = 249f83f
> 01-24 18:45:29.082     0     0 I init    : [libfs_mgr]__mount(source=/dev/block/bootdevice/by-name/userdata,target=/data,type=f2fs)=0: Try again
> 01-24 18:45:29.082     0     0 I init    : [libfs_mgr]/data is file encrypted
> 01-24 18:45:29.082     0     0 I init    : [libfs_mgr]Successfully mounted /data with file system type 'f2fs.: Try again
> 
> The related patch is: https://android-review.googlesource.com/c/platform/system/core/+/875538
> <https://android-review.googlesource.com/c/platform/system/core/+/875538>
> My question is:
> a) Is there any reason that f2fs gc must be done within 5s in the procedure of disable f2fs checkpoint?
>    code related:  f2fs_disable_checkpoint(sbi):
>           while (!f2fs_time_over(sbi, DISABLE_TIME))  //5s
>                  f2fs_gc(sbi, true, false, NULL_SEGNO);

I think we need to use this timeout mechanism to avoid mount being hanged too long
time if GC need many time to handle fragment migration.

> b) Why there is a condition test as below in f2fs_disable_cp_again()?

IIUC，we need to consider extreme case, e.g. if all fragmented segment's type is
data, and there is no free segment, then we may fail to find any free space for
newly written node, vice versa.

So during mount, we are trying to trigger GC to migrate fragmented block as much as
possible, to get enough free segment for later data/node written.

>      if (holes[DATA] > ovp || holes[NODE] > ovp)
>             return -EAGAIN;  <<<<---- mount will return here
>     My this issue is the code return in above code and fs_mgr will mount userdata again and again.
> 
> Can u kindly help to comment my above questions? 
> And is there is any solution to avoid or improve this scenario since if this happen, will impact the end user experience(end user may think phone get corrupted...). 

Checkpoint disabling is only used for upgrading of data partition, user can bear that
as upgrading usually takes long time.

Thanks,

> 
> Thanks,
> Mao
> 
> 
>  
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel