[PATCH] f2fs: set file as cold when file defragmentation

[PATCH] f2fs: set file as cold when file defragmentation

[Daejun Park]

In file defragmentation by ioctl, all data blocks in the file are
re-written out-of-place. File defragmentation implies user will not update
and mostly read the file. So before the defragmentation, we set file
temperature as cold for better block allocation.

Signed-off-by: Daejun Park <daejun7.park@samsung.com>
---
 fs/f2fs/file.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index d697c8900fa7..dcac965a05fe 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
 	map.m_len = pg_end - pg_start;
 	total = 0;
 
+	if (!file_is_cold(inode))
+		file_set_cold(inode);
+
 	while (map.m_lblk < pg_end) {
 		pgoff_t idx;
 		int cnt = 0;
-- 
2.25.1

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Chao Yu]

On 2021/4/29 14:20, Daejun Park wrote:
> In file defragmentation by ioctl, all data blocks in the file are
> re-written out-of-place. File defragmentation implies user will not update
> and mostly read the file. So before the defragmentation, we set file
> temperature as cold for better block allocation.

I don't think all fragmented files are cold, e.g. db file.

We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
"system.advise" -v value) to indicate this file is a hot/cold file, so my
suggestion is after file defragmentation, if you think this file is cold, and
use setxattr() to set it as cold.

Thanks,

> 
> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
> ---
>   fs/f2fs/file.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
> index d697c8900fa7..dcac965a05fe 100644
> --- a/fs/f2fs/file.c
> +++ b/fs/f2fs/file.c
> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>   	map.m_len = pg_end - pg_start;
>   	total = 0;
>   
> +	if (!file_is_cold(inode))
> +		file_set_cold(inode);
> +
>   	while (map.m_lblk < pg_end) {
>   		pgoff_t idx;
>   		int cnt = 0;
> 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Jaegeuk Kim]

On 05/06, Chao Yu wrote:
> On 2021/4/29 14:20, Daejun Park wrote:
> > In file defragmentation by ioctl, all data blocks in the file are
> > re-written out-of-place. File defragmentation implies user will not update
> > and mostly read the file. So before the defragmentation, we set file
> > temperature as cold for better block allocation.
> 
> I don't think all fragmented files are cold, e.g. db file.

I have a bit different opinion. I think one example would be users want to
defragment a file, when the they want to get higher read bandwidth for
usually multimedia files. That's likely to be cold files. Moreover, I don't
think they want to defragment db files which will be fragmented soon?

> 
> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
> "system.advise" -v value) to indicate this file is a hot/cold file, so my
> suggestion is after file defragmentation, if you think this file is cold, and
> use setxattr() to set it as cold.
> 
> Thanks,
> 
> > 
> > Signed-off-by: Daejun Park <daejun7.park@samsung.com>
> > ---
> >   fs/f2fs/file.c | 3 +++
> >   1 file changed, 3 insertions(+)
> > 
> > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
> > index d697c8900fa7..dcac965a05fe 100644
> > --- a/fs/f2fs/file.c
> > +++ b/fs/f2fs/file.c
> > @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
> >   	map.m_len = pg_end - pg_start;
> >   	total = 0;
> > +	if (!file_is_cold(inode))
> > +		file_set_cold(inode);
> > +
> >   	while (map.m_lblk < pg_end) {
> >   		pgoff_t idx;
> >   		int cnt = 0;
> > 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Chao Yu]

On 2021/5/6 12:46, Jaegeuk Kim wrote:
> On 05/06, Chao Yu wrote:
>> On 2021/4/29 14:20, Daejun Park wrote:
>>> In file defragmentation by ioctl, all data blocks in the file are
>>> re-written out-of-place. File defragmentation implies user will not update
>>> and mostly read the file. So before the defragmentation, we set file
>>> temperature as cold for better block allocation.
>>
>> I don't think all fragmented files are cold, e.g. db file.
> 
> I have a bit different opinion. I think one example would be users want to
> defragment a file, when the they want to get higher read bandwidth for

Multimedia file was already defined as cold file now via default extension
list?

> usually multimedia files. That's likely to be cold files. Moreover, I don't
> think they want to defragment db files which will be fragmented soon?

I guess like db files have less update but more access?

Thanks,

> 
>>
>> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>> "system.advise" -v value) to indicate this file is a hot/cold file, so my
>> suggestion is after file defragmentation, if you think this file is cold, and
>> use setxattr() to set it as cold.
>>
>> Thanks,
>>
>>>
>>> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
>>> ---
>>>    fs/f2fs/file.c | 3 +++
>>>    1 file changed, 3 insertions(+)
>>>
>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>>> index d697c8900fa7..dcac965a05fe 100644
>>> --- a/fs/f2fs/file.c
>>> +++ b/fs/f2fs/file.c
>>> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>>>    	map.m_len = pg_end - pg_start;
>>>    	total = 0;
>>> +	if (!file_is_cold(inode))
>>> +		file_set_cold(inode);
>>> +
>>>    	while (map.m_lblk < pg_end) {
>>>    		pgoff_t idx;
>>>    		int cnt = 0;
>>>
> .
> 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Jaegeuk Kim]

On 05/06, Chao Yu wrote:
> On 2021/5/6 12:46, Jaegeuk Kim wrote:
> > On 05/06, Chao Yu wrote:
> > > On 2021/4/29 14:20, Daejun Park wrote:
> > > > In file defragmentation by ioctl, all data blocks in the file are
> > > > re-written out-of-place. File defragmentation implies user will not update
> > > > and mostly read the file. So before the defragmentation, we set file
> > > > temperature as cold for better block allocation.
> > > 
> > > I don't think all fragmented files are cold, e.g. db file.
> > 
> > I have a bit different opinion. I think one example would be users want to
> > defragment a file, when the they want to get higher read bandwidth for
> 
> Multimedia file was already defined as cold file now via default extension
> list?

I just gave an example. And default is default.

> 
> > usually multimedia files. That's likely to be cold files. Moreover, I don't
> > think they want to defragment db files which will be fragmented soon?
> 
> I guess like db files have less update but more access?

I think both, and we set it as hot.

> 
> Thanks,
> 
> > 
> > > 
> > > We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
> > > "system.advise" -v value) to indicate this file is a hot/cold file, so my
> > > suggestion is after file defragmentation, if you think this file is cold, and
> > > use setxattr() to set it as cold.
> > > 
> > > Thanks,
> > > 
> > > > 
> > > > Signed-off-by: Daejun Park <daejun7.park@samsung.com>
> > > > ---
> > > >    fs/f2fs/file.c | 3 +++
> > > >    1 file changed, 3 insertions(+)
> > > > 
> > > > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
> > > > index d697c8900fa7..dcac965a05fe 100644
> > > > --- a/fs/f2fs/file.c
> > > > +++ b/fs/f2fs/file.c
> > > > @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
> > > >    	map.m_len = pg_end - pg_start;
> > > >    	total = 0;
> > > > +	if (!file_is_cold(inode))
> > > > +		file_set_cold(inode);
> > > > +
> > > >    	while (map.m_lblk < pg_end) {
> > > >    		pgoff_t idx;
> > > >    		int cnt = 0;
> > > > 
> > .
> > 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Chao Yu]

On 2021/5/10 22:47, Jaegeuk Kim wrote:
> On 05/06, Chao Yu wrote:
>> On 2021/5/6 12:46, Jaegeuk Kim wrote:
>>> On 05/06, Chao Yu wrote:
>>>> On 2021/4/29 14:20, Daejun Park wrote:
>>>>> In file defragmentation by ioctl, all data blocks in the file are
>>>>> re-written out-of-place. File defragmentation implies user will not update
>>>>> and mostly read the file. So before the defragmentation, we set file
>>>>> temperature as cold for better block allocation.
>>>>
>>>> I don't think all fragmented files are cold, e.g. db file.
>>>
>>> I have a bit different opinion. I think one example would be users want to
>>> defragment a file, when the they want to get higher read bandwidth for
>>
>> Multimedia file was already defined as cold file now via default extension
>> list?
> 
> I just gave an example. And default is default.
> 
>>
>>> usually multimedia files. That's likely to be cold files. Moreover, I don't
>>> think they want to defragment db files which will be fragmented soon?
>>
>> I guess like db files have less update but more access?
> 
> I think both, and we set it as hot.

Then hot and cold bit will set to the same db file after defragmentation?

Thanks,

> 
>>
>> Thanks,
>>
>>>
>>>>
>>>> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>>>> "system.advise" -v value) to indicate this file is a hot/cold file, so my
>>>> suggestion is after file defragmentation, if you think this file is cold, and
>>>> use setxattr() to set it as cold.
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
>>>>> ---
>>>>>     fs/f2fs/file.c | 3 +++
>>>>>     1 file changed, 3 insertions(+)
>>>>>
>>>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>>>>> index d697c8900fa7..dcac965a05fe 100644
>>>>> --- a/fs/f2fs/file.c
>>>>> +++ b/fs/f2fs/file.c
>>>>> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>>>>>     	map.m_len = pg_end - pg_start;
>>>>>     	total = 0;
>>>>> +	if (!file_is_cold(inode))
>>>>> +		file_set_cold(inode);
>>>>> +
>>>>>     	while (map.m_lblk < pg_end) {
>>>>>     		pgoff_t idx;
>>>>>     		int cnt = 0;
>>>>>
>>> .
>>>
> .
> 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Jaegeuk Kim]

On 05/11, Chao Yu wrote:
> On 2021/5/10 22:47, Jaegeuk Kim wrote:
> > On 05/06, Chao Yu wrote:
> > > On 2021/5/6 12:46, Jaegeuk Kim wrote:
> > > > On 05/06, Chao Yu wrote:
> > > > > On 2021/4/29 14:20, Daejun Park wrote:
> > > > > > In file defragmentation by ioctl, all data blocks in the file are
> > > > > > re-written out-of-place. File defragmentation implies user will not update
> > > > > > and mostly read the file. So before the defragmentation, we set file
> > > > > > temperature as cold for better block allocation.
> > > > > 
> > > > > I don't think all fragmented files are cold, e.g. db file.
> > > > 
> > > > I have a bit different opinion. I think one example would be users want to
> > > > defragment a file, when the they want to get higher read bandwidth for
> > > 
> > > Multimedia file was already defined as cold file now via default extension
> > > list?
> > 
> > I just gave an example. And default is default.
> > 
> > > 
> > > > usually multimedia files. That's likely to be cold files. Moreover, I don't
> > > > think they want to defragment db files which will be fragmented soon?
> > > 
> > > I guess like db files have less update but more access?
> > 
> > I think both, and we set it as hot.
> 
> Then hot and cold bit will set to the same db file after defragmentation?

Do you set cold bit to db files? I mean, generally db is not cold, but hot.

> 
> Thanks,
> 
> > 
> > > 
> > > Thanks,
> > > 
> > > > 
> > > > > 
> > > > > We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
> > > > > "system.advise" -v value) to indicate this file is a hot/cold file, so my
> > > > > suggestion is after file defragmentation, if you think this file is cold, and
> > > > > use setxattr() to set it as cold.
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > > 
> > > > > > Signed-off-by: Daejun Park <daejun7.park@samsung.com>
> > > > > > ---
> > > > > >     fs/f2fs/file.c | 3 +++
> > > > > >     1 file changed, 3 insertions(+)
> > > > > > 
> > > > > > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
> > > > > > index d697c8900fa7..dcac965a05fe 100644
> > > > > > --- a/fs/f2fs/file.c
> > > > > > +++ b/fs/f2fs/file.c
> > > > > > @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
> > > > > >     	map.m_len = pg_end - pg_start;
> > > > > >     	total = 0;
> > > > > > +	if (!file_is_cold(inode))
> > > > > > +		file_set_cold(inode);
> > > > > > +
> > > > > >     	while (map.m_lblk < pg_end) {
> > > > > >     		pgoff_t idx;
> > > > > >     		int cnt = 0;
> > > > > > 
> > > > .
> > > > 
> > .
> > 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Chao Yu]

On 2021/5/11 13:09, Jaegeuk Kim wrote:
> On 05/11, Chao Yu wrote:
>> On 2021/5/10 22:47, Jaegeuk Kim wrote:
>>> On 05/06, Chao Yu wrote:
>>>> On 2021/5/6 12:46, Jaegeuk Kim wrote:
>>>>> On 05/06, Chao Yu wrote:
>>>>>> On 2021/4/29 14:20, Daejun Park wrote:
>>>>>>> In file defragmentation by ioctl, all data blocks in the file are
>>>>>>> re-written out-of-place. File defragmentation implies user will not update
>>>>>>> and mostly read the file. So before the defragmentation, we set file
>>>>>>> temperature as cold for better block allocation.
>>>>>>
>>>>>> I don't think all fragmented files are cold, e.g. db file.
>>>>>
>>>>> I have a bit different opinion. I think one example would be users want to
>>>>> defragment a file, when the they want to get higher read bandwidth for
>>>>
>>>> Multimedia file was already defined as cold file now via default extension
>>>> list?
>>>
>>> I just gave an example. And default is default.
>>>
>>>>
>>>>> usually multimedia files. That's likely to be cold files. Moreover, I don't
>>>>> think they want to defragment db files which will be fragmented soon?
>>>>
>>>> I guess like db files have less update but more access?
>>>
>>> I think both, and we set it as hot.
>>
>> Then hot and cold bit will set to the same db file after defragmentation?
> 
> Do you set cold bit to db files? I mean, generally db is not cold, but hot.

I never set cold bit to db files, I mean if we defragment db file which
has less update and more access, db file may have bot hot and cold bit.

To Daejun, may I ask that is Samsung planning to use this defragment ioctl
in products? what's the user scenario?

Thanks,

> 
>>
>> Thanks,
>>
>>>
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>>>
>>>>>> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>>>>>> "system.advise" -v value) to indicate this file is a hot/cold file, so my
>>>>>> suggestion is after file defragmentation, if you think this file is cold, and
>>>>>> use setxattr() to set it as cold.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>>
>>>>>>> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
>>>>>>> ---
>>>>>>>      fs/f2fs/file.c | 3 +++
>>>>>>>      1 file changed, 3 insertions(+)
>>>>>>>
>>>>>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>>>>>>> index d697c8900fa7..dcac965a05fe 100644
>>>>>>> --- a/fs/f2fs/file.c
>>>>>>> +++ b/fs/f2fs/file.c
>>>>>>> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>>>>>>>      	map.m_len = pg_end - pg_start;
>>>>>>>      	total = 0;
>>>>>>> +	if (!file_is_cold(inode))
>>>>>>> +		file_set_cold(inode);
>>>>>>> +
>>>>>>>      	while (map.m_lblk < pg_end) {
>>>>>>>      		pgoff_t idx;
>>>>>>>      		int cnt = 0;
>>>>>>>
>>>>> .
>>>>>
>>> .
>>>
> .
> 

RE: Re: [PATCH] f2fs: set file as cold when file defragmentation

[Daejun Park]

>On 2021/5/11 13:09, Jaegeuk Kim wrote:
>> On 05/11, Chao Yu wrote:
>>> On 2021/5/10 22:47, Jaegeuk Kim wrote:
>>>> On 05/06, Chao Yu wrote:
>>>>> On 2021/5/6 12:46, Jaegeuk Kim wrote:
>>>>>> On 05/06, Chao Yu wrote:
>>>>>>> On 2021/4/29 14:20, Daejun Park wrote:
>>>>>>>> In file defragmentation by ioctl, all data blocks in the file are
>>>>>>>> re-written out-of-place. File defragmentation implies user will not update
>>>>>>>> and mostly read the file. So before the defragmentation, we set file
>>>>>>>> temperature as cold for better block allocation.
>>>>>>>
>>>>>>> I don't think all fragmented files are cold, e.g. db file.
>>>>>>
>>>>>> I have a bit different opinion. I think one example would be users want to
>>>>>> defragment a file, when the they want to get higher read bandwidth for
>>>>>
>>>>> Multimedia file was already defined as cold file now via default extension
>>>>> list?
>>>>
>>>> I just gave an example. And default is default.
>>>>
>>>>>
>>>>>> usually multimedia files. That's likely to be cold files. Moreover, I don't
>>>>>> think they want to defragment db files which will be fragmented soon?
>>>>>
>>>>> I guess like db files have less update but more access?
>>>>
>>>> I think both, and we set it as hot.
>>>
>>> Then hot and cold bit will set to the same db file after defragmentation?
>> 
>> Do you set cold bit to db files? I mean, generally db is not cold, but hot.
> 
>I never set cold bit to db files, I mean if we defragment db file which
>has less update and more access, db file may have bot hot and cold bit.
> 
>To Daejun, may I ask that is Samsung planning to use this defragment ioctl
>in products? what's the user scenario?

It is just my idea for defragmentation, not Samsung.
I think the user will call the defrag ioctl for the files that have been updated.

On the other hand, I think FS should be able to support defrag file even 
when in-place update is applied. What do you think?

Thanks,
Daejun
> 
>Thanks,
> 
>> 
>>>
>>> Thanks,
>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>>
>>>>>>>
>>>>>>> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>>>>>>> "system.advise" -v value) to indicate this file is a hot/cold file, so my
>>>>>>> suggestion is after file defragmentation, if you think this file is cold, and
>>>>>>> use setxattr() to set it as cold.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>>>
>>>>>>>> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
>>>>>>>> ---
>>>>>>>>      fs/f2fs/file.c | 3 +++
>>>>>>>>      1 file changed, 3 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>>>>>>>> index d697c8900fa7..dcac965a05fe 100644
>>>>>>>> --- a/fs/f2fs/file.c
>>>>>>>> +++ b/fs/f2fs/file.c
>>>>>>>> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>>>>>>>>              map.m_len = pg_end - pg_start;
>>>>>>>>              total = 0;
>>>>>>>> +        if (!file_is_cold(inode))
>>>>>>>> +                file_set_cold(inode);
>>>>>>>> +
>>>>>>>>              while (map.m_lblk < pg_end) {
>>>>>>>>                      pgoff_t idx;
>>>>>>>>                      int cnt = 0;
>>>>>>>>
>>>>>> .
>>>>>>
>>>> .
>>>>
>> .
>> 
> 
> 
>  

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Chao Yu]

On 2021/5/11 14:41, Daejun Park wrote:
>> On 2021/5/11 13:09, Jaegeuk Kim wrote:
>>> On 05/11, Chao Yu wrote:
>>>> On 2021/5/10 22:47, Jaegeuk Kim wrote:
>>>>> On 05/06, Chao Yu wrote:
>>>>>> On 2021/5/6 12:46, Jaegeuk Kim wrote:
>>>>>>> On 05/06, Chao Yu wrote:
>>>>>>>> On 2021/4/29 14:20, Daejun Park wrote:
>>>>>>>>> In file defragmentation by ioctl, all data blocks in the file are
>>>>>>>>> re-written out-of-place. File defragmentation implies user will not update
>>>>>>>>> and mostly read the file. So before the defragmentation, we set file
>>>>>>>>> temperature as cold for better block allocation.
>>>>>>>>
>>>>>>>> I don't think all fragmented files are cold, e.g. db file.
>>>>>>>
>>>>>>> I have a bit different opinion. I think one example would be users want to
>>>>>>> defragment a file, when the they want to get higher read bandwidth for
>>>>>>
>>>>>> Multimedia file was already defined as cold file now via default extension
>>>>>> list?
>>>>>
>>>>> I just gave an example. And default is default.
>>>>>
>>>>>>
>>>>>>> usually multimedia files. That's likely to be cold files. Moreover, I don't
>>>>>>> think they want to defragment db files which will be fragmented soon?
>>>>>>
>>>>>> I guess like db files have less update but more access?
>>>>>
>>>>> I think both, and we set it as hot.
>>>>
>>>> Then hot and cold bit will set to the same db file after defragmentation?
>>>
>>> Do you set cold bit to db files? I mean, generally db is not cold, but hot.
>>
>> I never set cold bit to db files, I mean if we defragment db file which
>> has less update and more access, db file may have bot hot and cold bit.
>>
>> To Daejun, may I ask that is Samsung planning to use this defragment ioctl
>> in products? what's the user scenario?
> 
> It is just my idea for defragmentation, not Samsung.

Alright,

> I think the user will call the defrag ioctl for the files that have been updated.

Sadly, I don't see there is any user of this defragment interface since it was
been introduced... so I really don't know the real use scenario of this interface
now.

> 
> On the other hand, I think FS should be able to support defrag file even
> when in-place update is applied. What do you think?

bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
{
	if (f2fs_is_pinned_file(inode))
		return true;

	/* if this is cold file, we should overwrite to avoid fragmentation */
	if (file_is_cold(inode))
		return true;

If cold bit was set, later rewrite in defragment interface can only trigger
IPU due to above IPU policy check, so after this interface, file is still
fragmented... what's the difference compared to just setting cold bit via
setxattr?

And if user know that he will trigger less update and more read in the file,
why not just calling setxattr("system.advise", cold_bit) to set the file as
cold before it becomes fragmented, e.g. at the time of file creation?

Thanks,

> 
> Thanks,
> Daejun
>>
>> Thanks,
>>
>>>
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>>>>>>>> "system.advise" -v value) to indicate this file is a hot/cold file, so my
>>>>>>>> suggestion is after file defragmentation, if you think this file is cold, and
>>>>>>>> use setxattr() to set it as cold.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
>>>>>>>>> ---
>>>>>>>>>       fs/f2fs/file.c | 3 +++
>>>>>>>>>       1 file changed, 3 insertions(+)
>>>>>>>>>
>>>>>>>>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>>>>>>>>> index d697c8900fa7..dcac965a05fe 100644
>>>>>>>>> --- a/fs/f2fs/file.c
>>>>>>>>> +++ b/fs/f2fs/file.c
>>>>>>>>> @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>>>>>>>>>               map.m_len = pg_end - pg_start;
>>>>>>>>>               total = 0;
>>>>>>>>> +        if (!file_is_cold(inode))
>>>>>>>>> +                file_set_cold(inode);
>>>>>>>>> +
>>>>>>>>>               while (map.m_lblk < pg_end) {
>>>>>>>>>                       pgoff_t idx;
>>>>>>>>>                       int cnt = 0;
>>>>>>>>>
>>>>>>> .
>>>>>>>
>>>>> .
>>>>>
>>> .
>>>
>>
>>
>>   
> .
> 

Re: [PATCH] f2fs: set file as cold when file defragmentation

[Jaegeuk Kim]

On 05/11, Chao Yu wrote:
> On 2021/5/11 14:41, Daejun Park wrote:
> > > On 2021/5/11 13:09, Jaegeuk Kim wrote:
> > > > On 05/11, Chao Yu wrote:
> > > > > On 2021/5/10 22:47, Jaegeuk Kim wrote:
> > > > > > On 05/06, Chao Yu wrote:
> > > > > > > On 2021/5/6 12:46, Jaegeuk Kim wrote:
> > > > > > > > On 05/06, Chao Yu wrote:
> > > > > > > > > On 2021/4/29 14:20, Daejun Park wrote:
> > > > > > > > > > In file defragmentation by ioctl, all data blocks in the file are
> > > > > > > > > > re-written out-of-place. File defragmentation implies user will not update
> > > > > > > > > > and mostly read the file. So before the defragmentation, we set file
> > > > > > > > > > temperature as cold for better block allocation.
> > > > > > > > > 
> > > > > > > > > I don't think all fragmented files are cold, e.g. db file.
> > > > > > > > 
> > > > > > > > I have a bit different opinion. I think one example would be users want to
> > > > > > > > defragment a file, when the they want to get higher read bandwidth for
> > > > > > > 
> > > > > > > Multimedia file was already defined as cold file now via default extension
> > > > > > > list?
> > > > > > 
> > > > > > I just gave an example. And default is default.
> > > > > > 
> > > > > > > 
> > > > > > > > usually multimedia files. That's likely to be cold files. Moreover, I don't
> > > > > > > > think they want to defragment db files which will be fragmented soon?
> > > > > > > 
> > > > > > > I guess like db files have less update but more access?
> > > > > > 
> > > > > > I think both, and we set it as hot.
> > > > > 
> > > > > Then hot and cold bit will set to the same db file after defragmentation?
> > > > 
> > > > Do you set cold bit to db files? I mean, generally db is not cold, but hot.
> > > 
> > > I never set cold bit to db files, I mean if we defragment db file which
> > > has less update and more access, db file may have bot hot and cold bit.
> > > 
> > > To Daejun, may I ask that is Samsung planning to use this defragment ioctl
> > > in products? what's the user scenario?
> > 
> > It is just my idea for defragmentation, not Samsung.
> 
> Alright,
> 
> > I think the user will call the defrag ioctl for the files that have been updated.
> 
> Sadly, I don't see there is any user of this defragment interface since it was
> been introduced... so I really don't know the real use scenario of this interface
> now.
> 
> > 
> > On the other hand, I think FS should be able to support defrag file even
> > when in-place update is applied. What do you think?
> 
> bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
> {
> 	if (f2fs_is_pinned_file(inode))
> 		return true;
> 
> 	/* if this is cold file, we should overwrite to avoid fragmentation */
> 	if (file_is_cold(inode))
> 		return true;
> 
> If cold bit was set, later rewrite in defragment interface can only trigger
> IPU due to above IPU policy check, so after this interface, file is still
> fragmented... what's the difference compared to just setting cold bit via
> setxattr?
> 
> And if user know that he will trigger less update and more read in the file,
> why not just calling setxattr("system.advise", cold_bit) to set the file as
> cold before it becomes fragmented, e.g. at the time of file creation?

yea, actually user can set it whatever they want after defragment. :P

> 
> Thanks,
> 
> > 
> > Thanks,
> > Daejun
> > > 
> > > Thanks,
> > > 
> > > > 
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > Thanks,
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
> > > > > > > > > "system.advise" -v value) to indicate this file is a hot/cold file, so my
> > > > > > > > > suggestion is after file defragmentation, if you think this file is cold, and
> > > > > > > > > use setxattr() to set it as cold.
> > > > > > > > > 
> > > > > > > > > Thanks,
> > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Signed-off-by: Daejun Park <daejun7.park@samsung.com>
> > > > > > > > > > ---
> > > > > > > > > >       fs/f2fs/file.c | 3 +++
> > > > > > > > > >       1 file changed, 3 insertions(+)
> > > > > > > > > > 
> > > > > > > > > > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
> > > > > > > > > > index d697c8900fa7..dcac965a05fe 100644
> > > > > > > > > > --- a/fs/f2fs/file.c
> > > > > > > > > > +++ b/fs/f2fs/file.c
> > > > > > > > > > @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
> > > > > > > > > >               map.m_len = pg_end - pg_start;
> > > > > > > > > >               total = 0;
> > > > > > > > > > +        if (!file_is_cold(inode))
> > > > > > > > > > +                file_set_cold(inode);
> > > > > > > > > > +
> > > > > > > > > >               while (map.m_lblk < pg_end) {
> > > > > > > > > >                       pgoff_t idx;
> > > > > > > > > >                       int cnt = 0;
> > > > > > > > > > 
> > > > > > > > .
> > > > > > > > 
> > > > > > .
> > > > > > 
> > > > .
> > > > 
> > > 
> > > 
> > .
> > 

RE: Re: [PATCH] f2fs: set file as cold when file defragmentation

[Daejun Park]

>On 05/11, Chao Yu wrote:
>> On 2021/5/11 14:41, Daejun Park wrote:
>> > > On 2021/5/11 13:09, Jaegeuk Kim wrote:
>> > > > On 05/11, Chao Yu wrote:
>> > > > > On 2021/5/10 22:47, Jaegeuk Kim wrote:
>> > > > > > On 05/06, Chao Yu wrote:
>> > > > > > > On 2021/5/6 12:46, Jaegeuk Kim wrote:
>> > > > > > > > On 05/06, Chao Yu wrote:
>> > > > > > > > > On 2021/4/29 14:20, Daejun Park wrote:
>> > > > > > > > > > In file defragmentation by ioctl, all data blocks in the file are
>> > > > > > > > > > re-written out-of-place. File defragmentation implies user will not update
>> > > > > > > > > > and mostly read the file. So before the defragmentation, we set file
>> > > > > > > > > > temperature as cold for better block allocation.
>> > > > > > > > > 
>> > > > > > > > > I don't think all fragmented files are cold, e.g. db file.
>> > > > > > > > 
>> > > > > > > > I have a bit different opinion. I think one example would be users want to
>> > > > > > > > defragment a file, when the they want to get higher read bandwidth for
>> > > > > > > 
>> > > > > > > Multimedia file was already defined as cold file now via default extension
>> > > > > > > list?
>> > > > > > 
>> > > > > > I just gave an example. And default is default.
>> > > > > > 
>> > > > > > > 
>> > > > > > > > usually multimedia files. That's likely to be cold files. Moreover, I don't
>> > > > > > > > think they want to defragment db files which will be fragmented soon?
>> > > > > > > 
>> > > > > > > I guess like db files have less update but more access?
>> > > > > > 
>> > > > > > I think both, and we set it as hot.
>> > > > > 
>> > > > > Then hot and cold bit will set to the same db file after defragmentation?
>> > > > 
>> > > > Do you set cold bit to db files? I mean, generally db is not cold, but hot.
>> > > 
>> > > I never set cold bit to db files, I mean if we defragment db file which
>> > > has less update and more access, db file may have bot hot and cold bit.
>> > > 
>> > > To Daejun, may I ask that is Samsung planning to use this defragment ioctl
>> > > in products? what's the user scenario?
>> > 
>> > It is just my idea for defragmentation, not Samsung.
>> 
>> Alright,
>> 
>> > I think the user will call the defrag ioctl for the files that have been updated.
>> 
>> Sadly, I don't see there is any user of this defragment interface since it was
>> been introduced... so I really don't know the real use scenario of this interface
>> now.
>> 
>> > 
>> > On the other hand, I think FS should be able to support defrag file even
>> > when in-place update is applied. What do you think?
>> 
>> bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
>> {
>>         if (f2fs_is_pinned_file(inode))
>>                 return true;
>> 
>>         /* if this is cold file, we should overwrite to avoid fragmentation */
>>         if (file_is_cold(inode))
>>                 return true;
>> 
>> If cold bit was set, later rewrite in defragment interface can only trigger
>> IPU due to above IPU policy check, so after this interface, file is still
>> fragmented... what's the difference compared to just setting cold bit via
>> setxattr?
>> 
>> And if user know that he will trigger less update and more read in the file,
>> why not just calling setxattr("system.advise", cold_bit) to set the file as
>> cold before it becomes fragmented, e.g. at the time of file creation?
> 
>yea, actually user can set it whatever they want after defragment. :P

I agree Chao yu's opinion so I drop this patch.

Thanks,
Daejun

> 
>> 
>> Thanks,
>> 
>> > 
>> > Thanks,
>> > Daejun
>> > > 
>> > > Thanks,
>> > > 
>> > > > 
>> > > > > 
>> > > > > Thanks,
>> > > > > 
>> > > > > > 
>> > > > > > > 
>> > > > > > > Thanks,
>> > > > > > > 
>> > > > > > > > 
>> > > > > > > > > 
>> > > > > > > > > We have separated interface (via f2fs_xattr_advise_handler, e.g. setfattr -n
>> > > > > > > > > "system.advise" -v value) to indicate this file is a hot/cold file, so my
>> > > > > > > > > suggestion is after file defragmentation, if you think this file is cold, and
>> > > > > > > > > use setxattr() to set it as cold.
>> > > > > > > > > 
>> > > > > > > > > Thanks,
>> > > > > > > > > 
>> > > > > > > > > > 
>> > > > > > > > > > Signed-off-by: Daejun Park <daejun7.park@samsung.com>
>> > > > > > > > > > ---
>> > > > > > > > > >       fs/f2fs/file.c | 3 +++
>> > > > > > > > > >       1 file changed, 3 insertions(+)
>> > > > > > > > > > 
>> > > > > > > > > > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>> > > > > > > > > > index d697c8900fa7..dcac965a05fe 100644
>> > > > > > > > > > --- a/fs/f2fs/file.c
>> > > > > > > > > > +++ b/fs/f2fs/file.c
>> > > > > > > > > > @@ -2669,6 +2669,9 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
>> > > > > > > > > >               map.m_len = pg_end - pg_start;
>> > > > > > > > > >               total = 0;
>> > > > > > > > > > +        if (!file_is_cold(inode))
>> > > > > > > > > > +                file_set_cold(inode);
>> > > > > > > > > > +
>> > > > > > > > > >               while (map.m_lblk < pg_end) {
>> > > > > > > > > >                       pgoff_t idx;
>> > > > > > > > > >                       int cnt = 0;
>> > > > > > > > > > 
>> > > > > > > > .
>> > > > > > > > 
>> > > > > > .
>> > > > > > 
>> > > > .
>> > > > 
>> > > 
>> > > 
>> > .
>> > 
> 
> 
>