[PATCH] f2fs: include non-compressed blocks in compr_written_block

[PATCH] f2fs: include non-compressed blocks in compr_written_block

[Daeho Jeong]

From: Daeho Jeong <daehojeong@google.com>

Need to include non-compressed blocks in compr_written_block to
estimate average compression ratio more accurately.

Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
 fs/f2fs/compress.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index c1bf9ad4c220..9b663eaf4805 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -1530,6 +1530,7 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
 	if (cluster_may_compress(cc)) {
 		err = f2fs_compress_pages(cc);
 		if (err == -EAGAIN) {
+			add_compr_block_stat(cc->inode, cc->cluster_size);
 			goto write;
 		} else if (err) {
 			f2fs_put_rpages_wbc(cc, wbc, true, 1);
-- 
2.33.0.882.g93a45727a2-goog

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Chao Yu]

On 2021/10/7 1:49, Daeho Jeong wrote:
> From: Daeho Jeong <daehojeong@google.com>
> 
> Need to include non-compressed blocks in compr_written_block to
> estimate average compression ratio more accurately.
> 
> Signed-off-by: Daeho Jeong <daehojeong@google.com>
> ---
>   fs/f2fs/compress.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
> index c1bf9ad4c220..9b663eaf4805 100644
> --- a/fs/f2fs/compress.c
> +++ b/fs/f2fs/compress.c
> @@ -1530,6 +1530,7 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
>   	if (cluster_may_compress(cc)) {
>   		err = f2fs_compress_pages(cc);
>   		if (err == -EAGAIN) {
> +			add_compr_block_stat(cc->inode, cc->cluster_size);

Shouldn't we relocate this after 'write' label?

One more concern, it looks we've changed what compr_block stat indicated,
literally, the block we account should be compressed..., how about accounting
it by introducing .presist_blocks, used_blocks or occupied_blocks.... thoughts?

Thanks,

>   			goto write;
>   		} else if (err) {
>   			f2fs_put_rpages_wbc(cc, wbc, true, 1);
> 

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Daeho Jeong]

> > diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
> > index c1bf9ad4c220..9b663eaf4805 100644
> > --- a/fs/f2fs/compress.c
> > +++ b/fs/f2fs/compress.c
> > @@ -1530,6 +1530,7 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
> >       if (cluster_may_compress(cc)) {
> >               err = f2fs_compress_pages(cc);
> >               if (err == -EAGAIN) {
> > +                     add_compr_block_stat(cc->inode, cc->cluster_size);
>
> Shouldn't we relocate this after 'write' label?
>
> One more concern, it looks we've changed what compr_block stat indicated,
> literally, the block we account should be compressed..., how about accounting
> it by introducing .presist_blocks, used_blocks or occupied_blocks.... thoughts?
>

What I wanted to add here is just one case in which compression was
tried, but couldn't save any block, so gave up.
If we put this below the "write" label, we will count blocks, even if
the file is turned off for compression in "user-controlled
compression" mode.
Like the commit comment says, I want to estimate the overall compression rate.
But, if we include every other compression disabled condition, it
won't work like that.

Thanks,

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Chao Yu]

On 2021/10/12 0:02, Daeho Jeong wrote:
>>> diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
>>> index c1bf9ad4c220..9b663eaf4805 100644
>>> --- a/fs/f2fs/compress.c
>>> +++ b/fs/f2fs/compress.c
>>> @@ -1530,6 +1530,7 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
>>>        if (cluster_may_compress(cc)) {
>>>                err = f2fs_compress_pages(cc);
>>>                if (err == -EAGAIN) {
>>> +                     add_compr_block_stat(cc->inode, cc->cluster_size);
>>
>> Shouldn't we relocate this after 'write' label?
>>
>> One more concern, it looks we've changed what compr_block stat indicated,
>> literally, the block we account should be compressed..., how about accounting
>> it by introducing .presist_blocks, used_blocks or occupied_blocks.... thoughts?
>>
> 
> What I wanted to add here is just one case in which compression was
> tried, but couldn't save any block, so gave up.
> If we put this below the "write" label, we will count blocks, even if
> the file is turned off for compression in "user-controlled
> compression" mode.
> Like the commit comment says, I want to estimate the overall compression rate.
> But, if we include every other compression disabled condition, it
> won't work like that.

Got it, thanks for the explanation.

Any thoughts about renaming compr_block? since some blocks accounted in
.compr_block weren't compressed blocks.

Thanks,

> 
> Thanks,
> 

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Daeho Jeong]

Sorry, many parts of userspace already rely on these names.
I wrote that compr_written_blocks shows the block count written after
compression since mount.
So, the count of blocks written as original data after compression
because of no gain would not be an exception.

Thanks,


On Wed, Oct 13, 2021 at 7:17 AM Chao Yu <chao@kernel.org> wrote:
>
> On 2021/10/12 0:02, Daeho Jeong wrote:
> >>> diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
> >>> index c1bf9ad4c220..9b663eaf4805 100644
> >>> --- a/fs/f2fs/compress.c
> >>> +++ b/fs/f2fs/compress.c
> >>> @@ -1530,6 +1530,7 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
> >>>        if (cluster_may_compress(cc)) {
> >>>                err = f2fs_compress_pages(cc);
> >>>                if (err == -EAGAIN) {
> >>> +                     add_compr_block_stat(cc->inode, cc->cluster_size);
> >>
> >> Shouldn't we relocate this after 'write' label?
> >>
> >> One more concern, it looks we've changed what compr_block stat indicated,
> >> literally, the block we account should be compressed..., how about accounting
> >> it by introducing .presist_blocks, used_blocks or occupied_blocks.... thoughts?
> >>
> >
> > What I wanted to add here is just one case in which compression was
> > tried, but couldn't save any block, so gave up.
> > If we put this below the "write" label, we will count blocks, even if
> > the file is turned off for compression in "user-controlled
> > compression" mode.
> > Like the commit comment says, I want to estimate the overall compression rate.
> > But, if we include every other compression disabled condition, it
> > won't work like that.
>
> Got it, thanks for the explanation.
>
> Any thoughts about renaming compr_block? since some blocks accounted in
> .compr_block weren't compressed blocks.
>
> Thanks,
>
> >
> > Thanks,
> >

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Chao Yu]

On 2021/10/14 1:52, Daeho Jeong wrote:
> Sorry, many parts of userspace already rely on these names.
> I wrote that compr_written_blocks shows the block count written after
> compression since mount.
> So, the count of blocks written as original data after compression
> because of no gain would not be an exception.

Okay, shouldn't we Cc stable mailing list for this patch? otherwise
userspace tool may get different stat number with the same compressed
file in different kernel?

Thanks,

> 
> Thanks,
> 
> 
> On Wed, Oct 13, 2021 at 7:17 AM Chao Yu <chao@kernel.org> wrote:
>>
>> On 2021/10/12 0:02, Daeho Jeong wrote:
>>>>> diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
>>>>> index c1bf9ad4c220..9b663eaf4805 100644
>>>>> --- a/fs/f2fs/compress.c
>>>>> +++ b/fs/f2fs/compress.c
>>>>> @@ -1530,6 +1530,7 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
>>>>>         if (cluster_may_compress(cc)) {
>>>>>                 err = f2fs_compress_pages(cc);
>>>>>                 if (err == -EAGAIN) {
>>>>> +                     add_compr_block_stat(cc->inode, cc->cluster_size);
>>>>
>>>> Shouldn't we relocate this after 'write' label?
>>>>
>>>> One more concern, it looks we've changed what compr_block stat indicated,
>>>> literally, the block we account should be compressed..., how about accounting
>>>> it by introducing .presist_blocks, used_blocks or occupied_blocks.... thoughts?
>>>>
>>>
>>> What I wanted to add here is just one case in which compression was
>>> tried, but couldn't save any block, so gave up.
>>> If we put this below the "write" label, we will count blocks, even if
>>> the file is turned off for compression in "user-controlled
>>> compression" mode.
>>> Like the commit comment says, I want to estimate the overall compression rate.
>>> But, if we include every other compression disabled condition, it
>>> won't work like that.
>>
>> Got it, thanks for the explanation.
>>
>> Any thoughts about renaming compr_block? since some blocks accounted in
>> .compr_block weren't compressed blocks.
>>
>> Thanks,
>>
>>>
>>> Thanks,
>>>

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Daeho Jeong]

> Okay, shouldn't we Cc stable mailing list for this patch? otherwise
> userspace tool may get different stat number with the same compressed
> file in different kernel?
>

Oh, could you let me know what the address of the "stable" mailing list is?

Thanks,

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Jaegeuk Kim]

On 10/21, Daeho Jeong wrote:
> > Okay, shouldn't we Cc stable mailing list for this patch? otherwise
> > userspace tool may get different stat number with the same compressed
> > file in different kernel?
> >
> 
> Oh, could you let me know what the address of the "stable" mailing list is?

Added Fixes and -stable. :)

> 
> Thanks,
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] f2fs: include non-compressed blocks in compr_written_block

[Chao Yu]

On 2021/10/22 5:18, Jaegeuk Kim wrote:
> On 10/21, Daeho Jeong wrote:
>>> Okay, shouldn't we Cc stable mailing list for this patch? otherwise
>>> userspace tool may get different stat number with the same compressed
>>> file in different kernel?
>>>
>>
>> Oh, could you let me know what the address of the "stable" mailing list is?
> 
> Added Fixes and -stable. :)

Reviewed-by: Chao Yu <chao@kernel.org>

Thanks,

> 
>>
>> Thanks,
>>
>>
>> _______________________________________________
>> Linux-f2fs-devel mailing list
>> Linux-f2fs-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel