[f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Park Ju Hyung]

A lot of Linux users have big, cold "zip" files.

Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
---
 mkfs/f2fs_format.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index 9402619..37d82c3 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -66,6 +66,7 @@ const char *media_ext_lists[] = {
 	"webp",
 
 	/* archives */
+	"zip",
 	"jar",
 	"deb",
 	"iso",
-- 
2.22.0



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH 2/2] mkfs.f2fs: add VM disk files to hot data types

[Park Ju Hyung]

Similar to .db files, these are randomly updated extremely frequently.

Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
---
 mkfs/f2fs_format.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index 37d82c3..1c08e3e 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -94,6 +94,11 @@ const char *media_ext_lists[] = {
 
 const char *hot_ext_lists[] = {
 	"db",
+
+	/* Virtual machines */
+	"vmdk", // VMware or VirtualBox
+	"vdi", // VirtualBox
+	"qcow2", // QEMU
 	NULL
 };
 
-- 
2.22.0



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Chao Yu]

Hi Ju Hyung,

On 2019/8/13 6:52, Park Ju Hyung wrote:
> A lot of Linux users have big, cold "zip" files.

In android, as I see, most zip file is small-sized log type, and will be removed
after a roll-back, such as:

time1: create log1.zip
time2: create log2.zip
time3: create log3.zip
time4: remove log1.zip, rename log2.zip -> log1.zip; rename log3.zip ->
log2.zip; create log3.zip

I suggest we can keep zip type in android as warm type with IPU mode to avoid
fragmentation caused by small holes in cold area. In linux distro, I agreed to
treat zip as cold type.

So what about configuring zip type in f2fs_init_configuration() under '#ifndef
WITH_ANDROID'?

What do you think?

Thanks,

> 
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> ---
>  mkfs/f2fs_format.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 9402619..37d82c3 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -66,6 +66,7 @@ const char *media_ext_lists[] = {
>  	"webp",
>  
>  	/* archives */
> +	"zip",
>  	"jar",
>  	"deb",
>  	"iso",
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 2/2] mkfs.f2fs: add VM disk files to hot data types

[Chao Yu]

On 2019/8/13 6:52, Park Ju Hyung wrote:
> Similar to .db files, these are randomly updated extremely frequently.

It looks android doesn't need this, how about adding them under "#ifndef
WITH_ANDROID"?

Thanks,

> 
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> ---
>  mkfs/f2fs_format.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 37d82c3..1c08e3e 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -94,6 +94,11 @@ const char *media_ext_lists[] = {
>  
>  const char *hot_ext_lists[] = {
>  	"db",
> +
> +	/* Virtual machines */
> +	"vmdk", // VMware or VirtualBox
> +	"vdi", // VirtualBox
> +	"qcow2", // QEMU
>  	NULL
>  };
>  
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Chao Yu]

On 2019/8/14 9:47, Chao Yu wrote:
> Hi Ju Hyung,
> 
> On 2019/8/13 6:52, Park Ju Hyung wrote:
>> A lot of Linux users have big, cold "zip" files.
> 
> In android, as I see, most zip file is small-sized log type, and will be removed
> after a roll-back, such as:
> 
> time1: create log1.zip
> time2: create log2.zip
> time3: create log3.zip
> time4: remove log1.zip, rename log2.zip -> log1.zip; rename log3.zip ->
> log2.zip; create log3.zip
> 
> I suggest we can keep zip type in android as warm type with IPU mode to avoid
> fragmentation caused by small holes in cold area. In linux distro, I agreed to
> treat zip as cold type.
> 
> So what about configuring zip type in f2fs_init_configuration() under '#ifndef

It seems it doesn't have to configure this in f2fs_init_configuration(), just
adding #ifndef WITH_ANDROID will be fine.

> WITH_ANDROID'?
> 
> What do you think?
> 
> Thanks,
> 
>>
>> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
>> ---
>>  mkfs/f2fs_format.c | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
>> index 9402619..37d82c3 100644
>> --- a/mkfs/f2fs_format.c
>> +++ b/mkfs/f2fs_format.c
>> @@ -66,6 +66,7 @@ const char *media_ext_lists[] = {
>>  	"webp",
>>  
>>  	/* archives */
>> +	"zip",
>>  	"jar",
>>  	"deb",
>>  	"iso",
>>
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> .
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Ju Hyung Park]

Hi Chao,

On Wed, Aug 14, 2019 at 10:47 AM Chao Yu <yuchao0@huawei.com> wrote:
> In android, as I see, most zip file is small-sized log type, and will be removed
> after a roll-back, such as:
>
> time1: create log1.zip
> time2: create log2.zip
> time3: create log3.zip
> time4: remove log1.zip, rename log2.zip -> log1.zip; rename log3.zip ->
> log2.zip; create log3.zip
>
> I suggest we can keep zip type in android as warm type with IPU mode to avoid
> fragmentation caused by small holes in cold area. In linux distro, I agreed to
> treat zip as cold type.

I actually thought your original suggestion of adding "zip" was to
handle big zip files under /sdcard(/data/media).

The one case you've mentioned will be entirely dependent on which apps
user's using.
In case of mine, I don't have any zip files under /data that's
seemingly used for logs.

Thanks,


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 2/2] mkfs.f2fs: add VM disk files to hot data types

[Ju Hyung Park]

Hi Chao,

You're right and Android will never use those types.

But then again, what's the point of separating the list?
I haven't encountered an Android user or an OEM that wants to
customize this list by passing an argument to mkfs.f2fs.

If an OEM want to customize this list, directly modifying the code for
mkfs.f2fs sounds better anyways.

Thanks.

On Wed, Aug 14, 2019 at 10:51 AM Chao Yu <yuchao0@huawei.com> wrote:
>
> On 2019/8/13 6:52, Park Ju Hyung wrote:
> > Similar to .db files, these are randomly updated extremely frequently.
>
> It looks android doesn't need this, how about adding them under "#ifndef
> WITH_ANDROID"?
>
> Thanks,
>
> >
> > Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> > ---
> >  mkfs/f2fs_format.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> > index 37d82c3..1c08e3e 100644
> > --- a/mkfs/f2fs_format.c
> > +++ b/mkfs/f2fs_format.c
> > @@ -94,6 +94,11 @@ const char *media_ext_lists[] = {
> >
> >  const char *hot_ext_lists[] = {
> >       "db",
> > +
> > +     /* Virtual machines */
> > +     "vmdk", // VMware or VirtualBox
> > +     "vdi", // VirtualBox
> > +     "qcow2", // QEMU
> >       NULL
> >  };
> >
> >


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Chao Yu]

Hi Ju Hyung,

On 2019/8/14 17:20, Ju Hyung Park wrote:
> Hi Chao,
> 
> On Wed, Aug 14, 2019 at 10:47 AM Chao Yu <yuchao0@huawei.com> wrote:
>> In android, as I see, most zip file is small-sized log type, and will be removed
>> after a roll-back, such as:
>>
>> time1: create log1.zip
>> time2: create log2.zip
>> time3: create log3.zip
>> time4: remove log1.zip, rename log2.zip -> log1.zip; rename log3.zip ->
>> log2.zip; create log3.zip
>>
>> I suggest we can keep zip type in android as warm type with IPU mode to avoid
>> fragmentation caused by small holes in cold area. In linux distro, I agreed to
>> treat zip as cold type.
> 
> I actually thought your original suggestion of adding "zip" was to
> handle big zip files under /sdcard(/data/media).
> 
> The one case you've mentioned will be entirely dependent on which apps
> user's using.

Yeah, actually, now I didn't see much large zip file in my external storage,
most of them are xxxKB or xMB, some of them looks very old tho. :)

> In case of mine, I don't have any zip files under /data that's
> seemingly used for logs.

Huawei cell phone has log file with .zip/.gz type locating in /data/log/... can
you search that directory?

If we relocate them into cold area, they will make holes in cold area crazily,
as they have small size and will be created/unlinked frequently.

Thanks,

> 
> Thanks,
> .
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Ju Hyung Park]

Hi Chao,

/data/log isn't even here.
I think it must be specific to EMUI.

Thanks.

On Wed, Aug 14, 2019 at 6:48 PM Chao Yu <yuchao0@huawei.com> wrote:
>
> Hi Ju Hyung,
>
> On 2019/8/14 17:20, Ju Hyung Park wrote:
> > Hi Chao,
> >
> > On Wed, Aug 14, 2019 at 10:47 AM Chao Yu <yuchao0@huawei.com> wrote:
> >> In android, as I see, most zip file is small-sized log type, and will be removed
> >> after a roll-back, such as:
> >>
> >> time1: create log1.zip
> >> time2: create log2.zip
> >> time3: create log3.zip
> >> time4: remove log1.zip, rename log2.zip -> log1.zip; rename log3.zip ->
> >> log2.zip; create log3.zip
> >>
> >> I suggest we can keep zip type in android as warm type with IPU mode to avoid
> >> fragmentation caused by small holes in cold area. In linux distro, I agreed to
> >> treat zip as cold type.
> >
> > I actually thought your original suggestion of adding "zip" was to
> > handle big zip files under /sdcard(/data/media).
> >
> > The one case you've mentioned will be entirely dependent on which apps
> > user's using.
>
> Yeah, actually, now I didn't see much large zip file in my external storage,
> most of them are xxxKB or xMB, some of them looks very old tho. :)
>
> > In case of mine, I don't have any zip files under /data that's
> > seemingly used for logs.
>
> Huawei cell phone has log file with .zip/.gz type locating in /data/log/... can
> you search that directory?
>
> If we relocate them into cold area, they will make holes in cold area crazily,
> as they have small size and will be created/unlinked frequently.
>
> Thanks,
>
> >
> > Thanks,
> > .
> >


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 1/2] mkfs.f2fs: add "zip" to cold data types

[Chao Yu]

Hi Ju Hyung,

On 2019/8/14 22:10, Ju Hyung Park wrote:
> Hi Chao,
> 
> /data/log isn't even here.
> I think it must be specific to EMUI.

Alright, at least in EMUI, it looks not proper to me to set cold flag on zip
file according to result of hot/cold separation investigation recently.

IMO, it will be more common to store large stable zip/rar file in Linux
desktop/server or Window OS...

Thanks,

> 
> Thanks.
> 
> On Wed, Aug 14, 2019 at 6:48 PM Chao Yu <yuchao0@huawei.com> wrote:
>>
>> Hi Ju Hyung,
>>
>> On 2019/8/14 17:20, Ju Hyung Park wrote:
>>> Hi Chao,
>>>
>>> On Wed, Aug 14, 2019 at 10:47 AM Chao Yu <yuchao0@huawei.com> wrote:
>>>> In android, as I see, most zip file is small-sized log type, and will be removed
>>>> after a roll-back, such as:
>>>>
>>>> time1: create log1.zip
>>>> time2: create log2.zip
>>>> time3: create log3.zip
>>>> time4: remove log1.zip, rename log2.zip -> log1.zip; rename log3.zip ->
>>>> log2.zip; create log3.zip
>>>>
>>>> I suggest we can keep zip type in android as warm type with IPU mode to avoid
>>>> fragmentation caused by small holes in cold area. In linux distro, I agreed to
>>>> treat zip as cold type.
>>>
>>> I actually thought your original suggestion of adding "zip" was to
>>> handle big zip files under /sdcard(/data/media).
>>>
>>> The one case you've mentioned will be entirely dependent on which apps
>>> user's using.
>>
>> Yeah, actually, now I didn't see much large zip file in my external storage,
>> most of them are xxxKB or xMB, some of them looks very old tho. :)
>>
>>> In case of mine, I don't have any zip files under /data that's
>>> seemingly used for logs.
>>
>> Huawei cell phone has log file with .zip/.gz type locating in /data/log/... can
>> you search that directory?
>>
>> If we relocate them into cold area, they will make holes in cold area crazily,
>> as they have small size and will be created/unlinked frequently.
>>
>> Thanks,
>>
>>>
>>> Thanks,
>>> .
>>>
> .
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 2/2] mkfs.f2fs: add VM disk files to hot data types

[Chao Yu]

Hi Ju Hyung,

On 2019/8/14 17:22, Ju Hyung Park wrote:
> Hi Chao,
> 
> You're right and Android will never use those types.
> 
> But then again, what's the point of separating the list?

I'm just thinking to use extension list space more efficiently to avoid free
slot exhaustion.

And in android, let f2fs_create() always to compare nonexistent extension name
is unneeded, right?

> I haven't encountered an Android user or an OEM that wants to
> customize this list by passing an argument to mkfs.f2fs.

That sounds sad for this interface's low use frequency, except Google and
Huawei, it looks f2fs needs more stable users first, now vendor like Huawei,
just use default list (had upstreamed .db as hot type as default yet). :(

> 
> If an OEM want to customize this list, directly modifying the code for
> mkfs.f2fs sounds better anyways.

Yes, also we support adding/deleting extensions via sys node.

Thanks,

> 
> Thanks.
> 
> On Wed, Aug 14, 2019 at 10:51 AM Chao Yu <yuchao0@huawei.com> wrote:
>>
>> On 2019/8/13 6:52, Park Ju Hyung wrote:
>>> Similar to .db files, these are randomly updated extremely frequently.
>>
>> It looks android doesn't need this, how about adding them under "#ifndef
>> WITH_ANDROID"?
>>
>> Thanks,
>>
>>>
>>> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
>>> ---
>>>  mkfs/f2fs_format.c | 5 +++++
>>>  1 file changed, 5 insertions(+)
>>>
>>> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
>>> index 37d82c3..1c08e3e 100644
>>> --- a/mkfs/f2fs_format.c
>>> +++ b/mkfs/f2fs_format.c
>>> @@ -94,6 +94,11 @@ const char *media_ext_lists[] = {
>>>
>>>  const char *hot_ext_lists[] = {
>>>       "db",
>>> +
>>> +     /* Virtual machines */
>>> +     "vmdk", // VMware or VirtualBox
>>> +     "vdi", // VirtualBox
>>> +     "qcow2", // QEMU
>>>       NULL
>>>  };
>>>
>>>
> .
> 


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel