[PATCH 1/2] mkfs.f2fs: make extensions list easier to read

[PATCH 1/2] mkfs.f2fs: make extensions list easier to read

[Park Ju Hyung]

Reorder the list and categorize.
No functional change introduced.

Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
---
 mkfs/f2fs_format.c | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index a2685cb..0ae0df3 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -38,19 +38,13 @@ struct f2fs_checkpoint *cp;
 static unsigned int quotatype_bits = 0;
 
 const char *media_ext_lists[] = {
-	"jpg",
-	"gif",
-	"png",
+	/* video */
 	"avi",
 	"divx",
-	"m4a",
 	"m4v",
 	"m4p",
 	"mp4",
-	"mp3",
-	"3gp",
 	"wmv",
-	"wma",
 	"mpeg",
 	"mkv",
 	"mov",
@@ -59,19 +53,35 @@ const char *media_ext_lists[] = {
 	"wmx",
 	"svi",
 	"wvx",
-	"wv",
 	"wm",
 	"mpg",
 	"mpe",
 	"rm",
+	"video",
+
+	/* audio */
+	"m4a",
+	"mp3",
+	"3gp",
+	"wma",
+	"wv",
 	"ogg",
 	"opus",
 	"flac",
+
+	/* image */
 	"jpeg",
-	"video",
-	"apk",	/* for android system */
-	"so",	/* for android system */
+	"jpg",
+	"gif",
+	"png",
+
+	/* other */
 	"exe",
+
+	/* android */
+	"apk",
+	"so",
+
 	NULL
 };
 
-- 
2.21.0

[PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Park Ju Hyung]

Following extensions are removed:
 - divx: deprecated video format and it's usually wrapped with avi
 - asf: deprecated streaming format
 - asx: redirecting file to asf(small)
 - wmx: redirecting file to wma/wmv(small)
 - rm: deprecated media container
 - video: unused
 - wv: unpopular audio format from 1998

The extensions list is limited to 64 and those don't deserve to be
on this space-precious list.

Common prefixes are introduced and are checked with
https://en.wikipedia.org/wiki/List_of_filename_extensions
to avoid treating possible hot files as cold:
 - mp: covers mp3, mp4, mpeg, mpg
 - wm: covers wma, wmb, wmv
 - og: covers oga, ogg, ogm, ogv
 - jp: covers jpg, jpeg, jp2

Following extensions are added:
 - webm: extremely popular free media container format from Google
         VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
 - wav: uncompressed audio format, commonly used with voice recorders
 - svg: vector image format commonly used in web
 - webp: free lossy image format commonly used in web
 - jar: Java archive file
 - deb: Debian software package
 - iso: disk image file
 - gz: gzip compressed file, unable to randomly update
 - xz: xz compressed file, unable to randomly update
 - zst: zstd compressed file, unable to randomly update
 - pdf: PDF document
 - pyc: Python bytecode automatically generated when
        executing python to run .py files
 - ttc, ttf: font files
 - cnt: image alias files commonly used in Android apps
 - exo: EXO player's cache files, commonly used in Android's YouTube app
 - odex, vdex: Android RunTime files found in /data/app/*/oat

Total entries on the list changed from 34 to 36.

Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
---
 mkfs/f2fs_format.c | 50 ++++++++++++++++++++++++++--------------------
 1 file changed, 28 insertions(+), 22 deletions(-)

diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index 0ae0df3..4560611 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -38,48 +38,54 @@ struct f2fs_checkpoint *cp;
 static unsigned int quotatype_bits = 0;
 
 const char *media_ext_lists[] = {
+	/* common prefix */
+	"mp", // Covers mp3, mp4, mpeg, mpg
+	"wm", // Covers wma, wmb, wmv
+	"og", // Covers oga, ogg, ogm, ogv
+	"jp", // Covers jpg, jpeg, jp2
+
 	/* video */
 	"avi",
-	"divx",
 	"m4v",
 	"m4p",
-	"mp4",
-	"wmv",
-	"mpeg",
 	"mkv",
 	"mov",
-	"asx",
-	"asf",
-	"wmx",
-	"svi",
-	"wvx",
-	"wm",
-	"mpg",
-	"mpe",
-	"rm",
-	"video",
+	"webm",
 
 	/* audio */
+	"wav",
 	"m4a",
-	"mp3",
 	"3gp",
-	"wma",
-	"wv",
-	"ogg",
 	"opus",
 	"flac",
 
 	/* image */
-	"jpeg",
-	"jpg",
 	"gif",
 	"png",
-
-	/* other */
+	"svg",
+	"webp",
+
+	/* archives */
+	"jar",
+	"deb",
+	"iso",
+	"gz",
+	"xz",
+	"zst",
+
+	/* others */
+	"pdf",
+	"pyc", // Python bytecode
+	"ttc",
+	"ttf",
 	"exe",
 
 	/* android */
 	"apk",
+	"cnt", // Image alias
+	"exo", // YouTube
+	"odex", // Android RunTime
+	"vdex", // Android RunTime
 	"so",
 
 	NULL
-- 
2.21.0

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Ju Hyung Park]

Hi Jaegeuk, Chao.

This is a revival from a patchset I sent about a year ago.

The original patchset's list was about 50 entries and we talked about
leaving too small room for the users to customize.
I made much more conservative choices here to made the new one to be
36(from 34), while still being very practical and relevant on both
regular Linux distros and Android.
I hope the explanation written in the commit message is enough to persuade you.

Thanks.


On Tue, Apr 16, 2019 at 3:44 PM Park Ju Hyung <qkrwngud825@gmail.com> wrote:
>
> Following extensions are removed:
>  - divx: deprecated video format and it's usually wrapped with avi
>  - asf: deprecated streaming format
>  - asx: redirecting file to asf(small)
>  - wmx: redirecting file to wma/wmv(small)
>  - rm: deprecated media container
>  - video: unused
>  - wv: unpopular audio format from 1998
>
> The extensions list is limited to 64 and those don't deserve to be
> on this space-precious list.
>
> Common prefixes are introduced and are checked with
> https://en.wikipedia.org/wiki/List_of_filename_extensions
> to avoid treating possible hot files as cold:
>  - mp: covers mp3, mp4, mpeg, mpg
>  - wm: covers wma, wmb, wmv
>  - og: covers oga, ogg, ogm, ogv
>  - jp: covers jpg, jpeg, jp2
>
> Following extensions are added:
>  - webm: extremely popular free media container format from Google
>          VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
>  - wav: uncompressed audio format, commonly used with voice recorders
>  - svg: vector image format commonly used in web
>  - webp: free lossy image format commonly used in web
>  - jar: Java archive file
>  - deb: Debian software package
>  - iso: disk image file
>  - gz: gzip compressed file, unable to randomly update
>  - xz: xz compressed file, unable to randomly update
>  - zst: zstd compressed file, unable to randomly update
>  - pdf: PDF document
>  - pyc: Python bytecode automatically generated when
>         executing python to run .py files
>  - ttc, ttf: font files
>  - cnt: image alias files commonly used in Android apps
>  - exo: EXO player's cache files, commonly used in Android's YouTube app
>  - odex, vdex: Android RunTime files found in /data/app/*/oat
>
> Total entries on the list changed from 34 to 36.
>
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> ---
>  mkfs/f2fs_format.c | 50 ++++++++++++++++++++++++++--------------------
>  1 file changed, 28 insertions(+), 22 deletions(-)
>
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 0ae0df3..4560611 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -38,48 +38,54 @@ struct f2fs_checkpoint *cp;
>  static unsigned int quotatype_bits = 0;
>
>  const char *media_ext_lists[] = {
> +       /* common prefix */
> +       "mp", // Covers mp3, mp4, mpeg, mpg
> +       "wm", // Covers wma, wmb, wmv
> +       "og", // Covers oga, ogg, ogm, ogv
> +       "jp", // Covers jpg, jpeg, jp2
> +
>         /* video */
>         "avi",
> -       "divx",
>         "m4v",
>         "m4p",
> -       "mp4",
> -       "wmv",
> -       "mpeg",
>         "mkv",
>         "mov",
> -       "asx",
> -       "asf",
> -       "wmx",
> -       "svi",
> -       "wvx",
> -       "wm",
> -       "mpg",
> -       "mpe",
> -       "rm",
> -       "video",
> +       "webm",
>
>         /* audio */
> +       "wav",
>         "m4a",
> -       "mp3",
>         "3gp",
> -       "wma",
> -       "wv",
> -       "ogg",
>         "opus",
>         "flac",
>
>         /* image */
> -       "jpeg",
> -       "jpg",
>         "gif",
>         "png",
> -
> -       /* other */
> +       "svg",
> +       "webp",
> +
> +       /* archives */
> +       "jar",
> +       "deb",
> +       "iso",
> +       "gz",
> +       "xz",
> +       "zst",
> +
> +       /* others */
> +       "pdf",
> +       "pyc", // Python bytecode
> +       "ttc",
> +       "ttf",
>         "exe",
>
>         /* android */
>         "apk",
> +       "cnt", // Image alias
> +       "exo", // YouTube
> +       "odex", // Android RunTime
> +       "vdex", // Android RunTime
>         "so",
>
>         NULL
> --
> 2.21.0
>

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Jaegeuk Kim]

Hi Ju Hyung,

On 04/16, Ju Hyung Park wrote:
> Hi Jaegeuk, Chao.
> 
> This is a revival from a patchset I sent about a year ago.
> 
> The original patchset's list was about 50 entries and we talked about
> leaving too small room for the users to customize.
> I made much more conservative choices here to made the new one to be
> 36(from 34), while still being very practical and relevant on both
> regular Linux distros and Android.
> I hope the explanation written in the commit message is enough to persuade you.

Thank you for heads up. Taking a look at two patches, I think it'd be fine to
merge them into single patch. Let me know, if you have any concerns.

Thanks,

> 
> Thanks.
> 
> 
> On Tue, Apr 16, 2019 at 3:44 PM Park Ju Hyung <qkrwngud825@gmail.com> wrote:
> >
> > Following extensions are removed:
> >  - divx: deprecated video format and it's usually wrapped with avi
> >  - asf: deprecated streaming format
> >  - asx: redirecting file to asf(small)
> >  - wmx: redirecting file to wma/wmv(small)
> >  - rm: deprecated media container
> >  - video: unused
> >  - wv: unpopular audio format from 1998
> >
> > The extensions list is limited to 64 and those don't deserve to be
> > on this space-precious list.
> >
> > Common prefixes are introduced and are checked with
> > https://en.wikipedia.org/wiki/List_of_filename_extensions
> > to avoid treating possible hot files as cold:
> >  - mp: covers mp3, mp4, mpeg, mpg
> >  - wm: covers wma, wmb, wmv
> >  - og: covers oga, ogg, ogm, ogv
> >  - jp: covers jpg, jpeg, jp2
> >
> > Following extensions are added:
> >  - webm: extremely popular free media container format from Google
> >          VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
> >  - wav: uncompressed audio format, commonly used with voice recorders
> >  - svg: vector image format commonly used in web
> >  - webp: free lossy image format commonly used in web
> >  - jar: Java archive file
> >  - deb: Debian software package
> >  - iso: disk image file
> >  - gz: gzip compressed file, unable to randomly update
> >  - xz: xz compressed file, unable to randomly update
> >  - zst: zstd compressed file, unable to randomly update
> >  - pdf: PDF document
> >  - pyc: Python bytecode automatically generated when
> >         executing python to run .py files
> >  - ttc, ttf: font files
> >  - cnt: image alias files commonly used in Android apps
> >  - exo: EXO player's cache files, commonly used in Android's YouTube app
> >  - odex, vdex: Android RunTime files found in /data/app/*/oat
> >
> > Total entries on the list changed from 34 to 36.
> >
> > Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> > ---
> >  mkfs/f2fs_format.c | 50 ++++++++++++++++++++++++++--------------------
> >  1 file changed, 28 insertions(+), 22 deletions(-)
> >
> > diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> > index 0ae0df3..4560611 100644
> > --- a/mkfs/f2fs_format.c
> > +++ b/mkfs/f2fs_format.c
> > @@ -38,48 +38,54 @@ struct f2fs_checkpoint *cp;
> >  static unsigned int quotatype_bits = 0;
> >
> >  const char *media_ext_lists[] = {
> > +       /* common prefix */
> > +       "mp", // Covers mp3, mp4, mpeg, mpg
> > +       "wm", // Covers wma, wmb, wmv
> > +       "og", // Covers oga, ogg, ogm, ogv
> > +       "jp", // Covers jpg, jpeg, jp2
> > +
> >         /* video */
> >         "avi",
> > -       "divx",
> >         "m4v",
> >         "m4p",
> > -       "mp4",
> > -       "wmv",
> > -       "mpeg",
> >         "mkv",
> >         "mov",
> > -       "asx",
> > -       "asf",
> > -       "wmx",
> > -       "svi",
> > -       "wvx",
> > -       "wm",
> > -       "mpg",
> > -       "mpe",
> > -       "rm",
> > -       "video",
> > +       "webm",
> >
> >         /* audio */
> > +       "wav",
> >         "m4a",
> > -       "mp3",
> >         "3gp",
> > -       "wma",
> > -       "wv",
> > -       "ogg",
> >         "opus",
> >         "flac",
> >
> >         /* image */
> > -       "jpeg",
> > -       "jpg",
> >         "gif",
> >         "png",
> > -
> > -       /* other */
> > +       "svg",
> > +       "webp",
> > +
> > +       /* archives */
> > +       "jar",
> > +       "deb",
> > +       "iso",
> > +       "gz",
> > +       "xz",
> > +       "zst",
> > +
> > +       /* others */
> > +       "pdf",
> > +       "pyc", // Python bytecode
> > +       "ttc",
> > +       "ttf",
> >         "exe",
> >
> >         /* android */
> >         "apk",
> > +       "cnt", // Image alias
> > +       "exo", // YouTube
> > +       "odex", // Android RunTime
> > +       "vdex", // Android RunTime
> >         "so",
> >
> >         NULL
> > --
> > 2.21.0
> >

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Ju Hyung Park]

Hi Jaegeuk,

On Wed, Apr 17, 2019 at 5:39 AM Jaegeuk Kim <jaegeuk@kernel.org> wrote:
> Thank you for heads up. Taking a look at two patches, I think it'd be fine to
> merge them into single patch. Let me know, if you have any concerns.

I'd prefer not merging it as a reorder was done prior to this patch.

If someone was looking at the extensions list closely, they'll have a
hard time understanding which extensions really changed.

But I believe this is a nitpick at this point. I see you already
merged the patches to dev-test, feel free to leave it as-is.

Thanks.

Re: [PATCH 1/2] mkfs.f2fs: make extensions list easier to read

[Chao Yu]

On 2019/4/16 14:43, Park Ju Hyung wrote:
> Reorder the list and categorize.
> No functional change introduced.
> 
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Chao Yu]

On 2019/4/16 14:43, Park Ju Hyung wrote:
> Following extensions are removed:
>  - divx: deprecated video format and it's usually wrapped with avi
>  - asf: deprecated streaming format
>  - asx: redirecting file to asf(small)
>  - wmx: redirecting file to wma/wmv(small)
>  - rm: deprecated media container
>  - video: unused
>  - wv: unpopular audio format from 1998
> 
> The extensions list is limited to 64 and those don't deserve to be
> on this space-precious list.
> 
> Common prefixes are introduced and are checked with
> https://en.wikipedia.org/wiki/List_of_filename_extensions
> to avoid treating possible hot files as cold:
>  - mp: covers mp3, mp4, mpeg, mpg
>  - wm: covers wma, wmb, wmv
>  - og: covers oga, ogg, ogm, ogv
>  - jp: covers jpg, jpeg, jp2
> 
> Following extensions are added:
>  - webm: extremely popular free media container format from Google
>          VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
>  - wav: uncompressed audio format, commonly used with voice recorders
>  - svg: vector image format commonly used in web
>  - webp: free lossy image format commonly used in web
>  - jar: Java archive file
>  - deb: Debian software package
>  - iso: disk image file
>  - gz: gzip compressed file, unable to randomly update
>  - xz: xz compressed file, unable to randomly update
>  - zst: zstd compressed file, unable to randomly update
>  - pdf: PDF document
>  - pyc: Python bytecode automatically generated when
>         executing python to run .py files
>  - ttc, ttf: font files
>  - cnt: image alias files commonly used in Android apps
>  - exo: EXO player's cache files, commonly used in Android's YouTube app
>  - odex, vdex: Android RunTime files found in /data/app/*/oat
> 
> Total entries on the list changed from 34 to 36.
> 
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> ---
>  mkfs/f2fs_format.c | 50 ++++++++++++++++++++++++++--------------------
>  1 file changed, 28 insertions(+), 22 deletions(-)
> 
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 0ae0df3..4560611 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -38,48 +38,54 @@ struct f2fs_checkpoint *cp;
>  static unsigned int quotatype_bits = 0;
>  
>  const char *media_ext_lists[] = {
> +	/* common prefix */
> +	"mp", // Covers mp3, mp4, mpeg, mpg
> +	"wm", // Covers wma, wmb, wmv
> +	"og", // Covers oga, ogg, ogm, ogv
> +	"jp", // Covers jpg, jpeg, jp2

Actually, kernel will compare each character of extension in list with file's
extension, rather than prefix, could you confirm that? :)

Thanks,

> +
>  	/* video */
>  	"avi",
> -	"divx",
>  	"m4v",
>  	"m4p",
> -	"mp4",
> -	"wmv",
> -	"mpeg",
>  	"mkv",
>  	"mov",
> -	"asx",
> -	"asf",
> -	"wmx",
> -	"svi",
> -	"wvx",
> -	"wm",
> -	"mpg",
> -	"mpe",
> -	"rm",
> -	"video",
> +	"webm",
>  
>  	/* audio */
> +	"wav",
>  	"m4a",
> -	"mp3",
>  	"3gp",
> -	"wma",
> -	"wv",
> -	"ogg",
>  	"opus",
>  	"flac",
>  
>  	/* image */
> -	"jpeg",
> -	"jpg",
>  	"gif",
>  	"png",
> -
> -	/* other */
> +	"svg",
> +	"webp",
> +
> +	/* archives */
> +	"jar",
> +	"deb",
> +	"iso",
> +	"gz",
> +	"xz",
> +	"zst",
> +
> +	/* others */
> +	"pdf",
> +	"pyc", // Python bytecode
> +	"ttc",
> +	"ttf",
>  	"exe",
>  
>  	/* android */
>  	"apk",
> +	"cnt", // Image alias
> +	"exo", // YouTube
> +	"odex", // Android RunTime
> +	"vdex", // Android RunTime
>  	"so",
>  
>  	NULL
> 

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Ju Hyung Park]

Hi Chao,

On Wed, Apr 17, 2019 at 6:41 PM Chao Yu <yuchao0@huawei.com> wrote:
> Actually, kernel will compare each character of extension in list with file's
> extension, rather than prefix, could you confirm that? :)

Just wrote a sample C program with namei.c's is_extension_exist() to
confirm this indeed works as intended.

Output:
Checking against "jp"
jpg: false
abc.jpg: true
abc.jpeg: true
abc.jpg.tmp: true
abc.jpeg.tmp: true
abc.jgp: false

Source:

#include <stdio.h>
#include <stddef.h>
#include <string.h>

static int is_extension_exist(const unsigned char *s, const char *sub)
{
    size_t slen = strlen(s);
    size_t sublen = strlen(sub);
    int i;

    /*
     * filename format of multimedia file should be defined as:
     * "filename + '.' + extension + (optional: '.' + temp extension)".
     */
    if (slen < sublen + 2)
        return 0;

    for (i = 1; i < slen - sublen; i++) {
        if (s[i] != '.')
            continue;
        if (!strncasecmp(s + i + 1, sub, sublen))
            return 1;
    }

    return 0;
}

static const char *list[] = {
    "jpg",
    "abc.jpg",
    "abc.jpeg",
    "abc.jpg.tmp",
    "abc.jpeg.tmp",
    "abc.jgp",
    NULL
};

#define CHECK "jp"

int main() {
    printf("Checking against \"%s\"\n", CHECK);

    for (int i = 0; list[i]; i++) {
        if (is_extension_exist(list[i], CHECK))
            printf("%s: true\n", list[i]);
        else
            printf("%s: false\n", list[i]);
    }
}

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Chao Yu]

Hi Ju Hyung,

On 2019/4/17 17:54, Ju Hyung Park wrote:
> Hi Chao,
> 
> On Wed, Apr 17, 2019 at 6:41 PM Chao Yu <yuchao0@huawei.com> wrote:
>> Actually, kernel will compare each character of extension in list with file's
>> extension, rather than prefix, could you confirm that? :)>
> Just wrote a sample C program with namei.c's is_extension_exist() to
> confirm this indeed works as intended.

Thanks for your confirmation.

Sorry for my mistake, I just recall that we added .db in hot file extension list
so that .db-journal, .db-wal, .db-shm can also be covered.

Maybe we have faced some risk that there is cold extension which is the prefix
of hot file's extension, but I guess this is really rare or even not exist.

Anyway I think we can afford that risk since extension itself is rough method
for judging the hot/cold type of file, it may not always be 100% correct.

So, it looks okay to me to use their common prefix. ;)

Thanks,

> 
> Output:
> Checking against "jp"
> jpg: false
> abc.jpg: true
> abc.jpeg: true
> abc.jpg.tmp: true
> abc.jpeg.tmp: true
> abc.jgp: false
> 
> Source:
> 
> #include <stdio.h>
> #include <stddef.h>
> #include <string.h>
> 
> static int is_extension_exist(const unsigned char *s, const char *sub)
> {
>     size_t slen = strlen(s);
>     size_t sublen = strlen(sub);
>     int i;
> 
>     /*
>      * filename format of multimedia file should be defined as:
>      * "filename + '.' + extension + (optional: '.' + temp extension)".
>      */
>     if (slen < sublen + 2)
>         return 0;
> 
>     for (i = 1; i < slen - sublen; i++) {
>         if (s[i] != '.')
>             continue;
>         if (!strncasecmp(s + i + 1, sub, sublen))
>             return 1;
>     }
> 
>     return 0;
> }
> 
> static const char *list[] = {
>     "jpg",
>     "abc.jpg",
>     "abc.jpeg",
>     "abc.jpg.tmp",
>     "abc.jpeg.tmp",
>     "abc.jgp",
>     NULL
> };
> 
> #define CHECK "jp"
> 
> int main() {
>     printf("Checking against \"%s\"\n", CHECK);
> 
>     for (int i = 0; list[i]; i++) {
>         if (is_extension_exist(list[i], CHECK))
>             printf("%s: true\n", list[i]);
>         else
>             printf("%s: false\n", list[i]);
>     }
> }
> .
> 

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Chao Yu]

On 2019/4/16 14:43, Park Ju Hyung wrote:
> Following extensions are removed:
>  - divx: deprecated video format and it's usually wrapped with avi
>  - asf: deprecated streaming format
>  - asx: redirecting file to asf(small)
>  - wmx: redirecting file to wma/wmv(small)
>  - rm: deprecated media container
>  - video: unused
>  - wv: unpopular audio format from 1998
> 
> The extensions list is limited to 64 and those don't deserve to be
> on this space-precious list.
> 
> Common prefixes are introduced and are checked with
> https://en.wikipedia.org/wiki/List_of_filename_extensions
> to avoid treating possible hot files as cold:
>  - mp: covers mp3, mp4, mpeg, mpg
>  - wm: covers wma, wmb, wmv
>  - og: covers oga, ogg, ogm, ogv
>  - jp: covers jpg, jpeg, jp2
> 
> Following extensions are added:
>  - webm: extremely popular free media container format from Google
>          VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
>  - wav: uncompressed audio format, commonly used with voice recorders
>  - svg: vector image format commonly used in web
>  - webp: free lossy image format commonly used in web
>  - jar: Java archive file
>  - deb: Debian software package
>  - iso: disk image file
>  - gz: gzip compressed file, unable to randomly update
>  - xz: xz compressed file, unable to randomly update
>  - zst: zstd compressed file, unable to randomly update
>  - pdf: PDF document
>  - pyc: Python bytecode automatically generated when
>         executing python to run .py files
>  - ttc, ttf: font files
>  - cnt: image alias files commonly used in Android apps
>  - exo: EXO player's cache files, commonly used in Android's YouTube app
>  - odex, vdex: Android RunTime files found in /data/app/*/oat
> 
> Total entries on the list changed from 34 to 36.
> 
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Chao Yu]

On 2019/4/16 14:43, Park Ju Hyung wrote:
> Following extensions are removed:
>  - divx: deprecated video format and it's usually wrapped with avi
>  - asf: deprecated streaming format
>  - asx: redirecting file to asf(small)
>  - wmx: redirecting file to wma/wmv(small)
>  - rm: deprecated media container
>  - video: unused
>  - wv: unpopular audio format from 1998
> 
> The extensions list is limited to 64 and those don't deserve to be
> on this space-precious list.
> 
> Common prefixes are introduced and are checked with
> https://en.wikipedia.org/wiki/List_of_filename_extensions
> to avoid treating possible hot files as cold:
>  - mp: covers mp3, mp4, mpeg, mpg
>  - wm: covers wma, wmb, wmv
>  - og: covers oga, ogg, ogm, ogv
>  - jp: covers jpg, jpeg, jp2
> 
> Following extensions are added:
>  - webm: extremely popular free media container format from Google
>          VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
>  - wav: uncompressed audio format, commonly used with voice recorders
>  - svg: vector image format commonly used in web
>  - webp: free lossy image format commonly used in web
>  - jar: Java archive file
>  - deb: Debian software package
>  - iso: disk image file
>  - gz: gzip compressed file, unable to randomly update
>  - xz: xz compressed file, unable to randomly update
>  - zst: zstd compressed file, unable to randomly update
>  - pdf: PDF document
>  - pyc: Python bytecode automatically generated when
>         executing python to run .py files
>  - ttc, ttf: font files
>  - cnt: image alias files commonly used in Android apps
>  - exo: EXO player's cache files, commonly used in Android's YouTube app
>  - odex, vdex: Android RunTime files found in /data/app/*/oat
> 
> Total entries on the list changed from 34 to 36.
> 
> Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
> ---
>  mkfs/f2fs_format.c | 50 ++++++++++++++++++++++++++--------------------
>  1 file changed, 28 insertions(+), 22 deletions(-)
> 
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 0ae0df3..4560611 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -38,48 +38,54 @@ struct f2fs_checkpoint *cp;
>  static unsigned int quotatype_bits = 0;
>  
>  const char *media_ext_lists[] = {
> +	/* common prefix */
> +	"mp", // Covers mp3, mp4, mpeg, mpg
> +	"wm", // Covers wma, wmb, wmv
> +	"og", // Covers oga, ogg, ogm, ogv
> +	"jp", // Covers jpg, jpeg, jp2
> +
>  	/* video */
>  	"avi",
> -	"divx",
>  	"m4v",
>  	"m4p",
> -	"mp4",
> -	"wmv",
> -	"mpeg",
>  	"mkv",
>  	"mov",
> -	"asx",
> -	"asf",
> -	"wmx",
> -	"svi",
> -	"wvx",
> -	"wm",
> -	"mpg",
> -	"mpe",
> -	"rm",
> -	"video",
> +	"webm",
>  
>  	/* audio */
> +	"wav",
>  	"m4a",
> -	"mp3",
>  	"3gp",
> -	"wma",
> -	"wv",
> -	"ogg",
>  	"opus",
>  	"flac",
>  
>  	/* image */
> -	"jpeg",
> -	"jpg",
>  	"gif",
>  	"png",
> -
> -	/* other */
> +	"svg",
> +	"webp",
> +
> +	/* archives */
> +	"jar",
> +	"deb",
> +	"iso",
> +	"gz",
> +	"xz",
> +	"zst",

How about adding below extensions:

	"zip",
	"bin",
	"dat",
	"txt",

Thanks,

> +
> +	/* others */
> +	"pdf",
> +	"pyc", // Python bytecode
> +	"ttc",
> +	"ttf",
>  	"exe",
>  
>  	/* android */
>  	"apk",
> +	"cnt", // Image alias
> +	"exo", // YouTube
> +	"odex", // Android RunTime
> +	"vdex", // Android RunTime
>  	"so",
>  
>  	NULL
> 

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Ju Hyung Park]

Hi Chao,

On Tue, May 28, 2019 at 7:19 PM Chao Yu <yuchao0@huawei.com> wrote:
> How about adding below extensions:
>
>         "zip",
>         "bin",
>         "dat",
>         "txt",

zip is capable of random updates. I didn't add bz2 for the same reason.
But I do agree that most users won't be constantly updating zip files.

I personally use my Android device with zip treated as cold, but I'm
not sure if it makes good sense to make it as the default that's
supposed to run under various scenarios.

How much different is the random write performance from cold to hot?

But I'm against the idea of adding the rest 3 extensions.
"bin" and "dat" is way too generic. You wouldn't know if a program
happens to heavily update files named .bin/.dat.

For txt, it won't be uncommon for a user to update it frequently.
Moreover, most txt size is pretty small anyways.

And finally, circling back to your original concern, we should be more
careful adding extensions as there's a limit.

Thanks.

Re: [PATCH 2/2] mkfs.f2fs: make the default extensions list much more sensical

[Chao Yu]

Hi Ju Hyung,

On 2019/5/28 18:28, Ju Hyung Park wrote:
> Hi Chao,
> 
> On Tue, May 28, 2019 at 7:19 PM Chao Yu <yuchao0@huawei.com> wrote:
>> How about adding below extensions:
>>
>>         "zip",
>>         "bin",
>>         "dat",
>>         "txt",
> 
> zip is capable of random updates. I didn't add bz2 for the same reason.
> But I do agree that most users won't be constantly updating zip files.

Yup, I think the most possible case is using zip/bz2 to pack log files or for
some downloaded resources, which should be write-once file.

> 
> I personally use my Android device with zip treated as cold, but I'm
> not sure if it makes good sense to make it as the default that's
> supposed to run under various scenarios.
> 
> How much different is the random write performance from cold to hot?

We use different write policies: trigger OPU for hot data mostly, and IPU for
cold data, so performance is quite relating to the test scenario.

> 
> But I'm against the idea of adding the rest 3 extensions.
> "bin" and "dat" is way too generic. You wouldn't know if a program
> happens to heavily update files named .bin/.dat.
> 
> For txt, it won't be uncommon for a user to update it frequently.
> Moreover, most txt size is pretty small anyways.

I think your concern is right, those extensions are too common and it needs to
consider various platforms instead of just android. So, let's leave those three
extensions.

Thanks,

> 
> And finally, circling back to your original concern, we should be more
> careful adding extensions as there's a limit.>
> Thanks.
> .
>