[PATCH v2 1/3] erofs-utils: README: trivial updates

[PATCH v2 1/3] erofs-utils: README: trivial updates

[Gao Xiang]

- update erofs naming to EROFS;
- add some words about compression levels;
- add more description about legacy EROFS images.

Link: https://lore.kernel.org/r/20210522055057.25004-1-xiang@kernel.org
Signed-off-by: Gao Xiang <xiang@kernel.org>
---
 README | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/README b/README
index b57550b2a09e..bcf30e11aa52 100644
--- a/README
+++ b/README
@@ -1,7 +1,7 @@
 erofs-utils
 ===========
 
-erofs-utils includes user-space tools for erofs filesystem.
+erofs-utils includes user-space tools for EROFS filesystem.
 Currently mkfs.erofs and erofsfuse (experimental) are available.
 
 Dependencies & build
@@ -50,7 +50,7 @@ as well.
 mkfs.erofs
 ----------
 
-two main kinds of erofs images can be generated: (un)compressed.
+two main kinds of EROFS images can be generated: (un)compressed.
 
  - For uncompressed images, there will be none of compression
    files in these images. However, it can decide whether the tail
@@ -61,8 +61,8 @@ two main kinds of erofs images can be generated: (un)compressed.
    saved with compression. If not, fallback to an uncompressed
    file.
 
-How to generate erofs images
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+How to generate EROFS images (Linux 5.3+)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Currently lz4 and lz4hc are available for compression, e.g.
  $ mkfs.erofs -zlz4hc foo.erofs.img foo/
@@ -70,17 +70,24 @@ Currently lz4 and lz4hc are available for compression, e.g.
 Or leave all files uncompressed as an option:
  $ mkfs.erofs foo.erofs.img foo/
 
-How to generate legacy erofs images
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+In addition, you could specify a higher compression level to get a
+(slightly) better compression ratio than the default level, e.g.
+ $ mkfs.erofs -zlz4hc,12 foo.erofs.img foo/
+
+How to generate legacy EROFS images (Linux 4.19+)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Decompression inplace and compacted indexes have been introduced in
 linux-5.3, which are not forward-compatible with older kernels.
 
-In order to generate _legacy_ erofs images for old kernels,
+In order to generate _legacy_ EROFS images for old kernels,
 consider adding "-E legacy-compress" to the command line, e.g.
 
  $ mkfs.erofs -E legacy-compress -zlz4hc foo.erofs.img foo/
 
+For Linux kernel >= 5.3, legacy EROFS images are _NOT recommended_
+due to runtime performance loss compared with non-legacy images.
+
 Obsoleted erofs.mkfs
 ~~~~~~~~~~~~~~~~~~~~
 
@@ -94,7 +101,7 @@ PLEASE NOTE: This version is highly _NOT recommended_ now.
 erofsfuse (experimental, unstable)
 ----------------------------------
 
-erofsfuse is introduced to support erofs format for various platforms
+erofsfuse is introduced to support EROFS format for various platforms
 (including older linux kernels) and new on-disk features iteration.
 It can also be used as an unpacking tool for unprivileged users.
 
@@ -120,7 +127,7 @@ and build it manually:
 
 erofsfuse binary will be generated under fuse folder.
 
-How to mount an erofs image with erofsfuse
+How to mount an EROFS image with erofsfuse
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 As the other FUSE implementations, it's quite simple to mount with
@@ -139,7 +146,7 @@ To unmount an erofsfuse mountpoint as a non-root user:
 Contribution
 ------------
 
-erofs-utils is under GPLv2+ as a part of erofs project,
+erofs-utils is under GPLv2+ as a part of EROFS filesystem project,
 feel free to send patches or feedback to us.
 
 To:
@@ -155,7 +162,7 @@ Cc:
 Comments
 --------
 
-[1] According to the erofs on-disk format, the tail block of files
+[1] According to the EROFS on-disk format, the tail block of files
     could be inlined aggressively with its metadata in order to reduce
     the I/O overhead and save the storage space (called tail-packing).
 
-- 
2.20.1

[PATCH v2 2/3] erofs-utils: README: big pcluster feature update

[Gao Xiang]

Add some description for end users to know its usage.

Link: https://lore.kernel.org/r/20210522055057.25004-2-xiang@kernel.org
Signed-off-by: Gao Xiang <xiang@kernel.org>
---
 README | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/README b/README
index bcf30e11aa52..af9cdf11c78a 100644
--- a/README
+++ b/README
@@ -74,6 +74,24 @@ In addition, you could specify a higher compression level to get a
 (slightly) better compression ratio than the default level, e.g.
  $ mkfs.erofs -zlz4hc,12 foo.erofs.img foo/
 
+How to generate EROFS big pcluster images (Linux 5.13+)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to get much better compression ratios (thus better sequential
+read performance for common storage devices), big pluster feature has
+been introduced since linux-5.13, which is not forward-compatible with
+old kernels.
+
+In details, -C is used to specify the maximum size of each big pcluster
+in bytes, e.g.
+ $ mkfs.erofs -zlz4hc -C65536 foo.erofs.img foo/
+
+So in that case, pcluster size can be 64KiB at most.
+
+Note that large pcluster size can cause bad random performance, so
+please evaluate carefully in advance. Or make your own per-(sub)file
+compression strategies according to file access patterns if needed.
+
 How to generate legacy EROFS images (Linux 4.19+)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.20.1

[PATCH v2 3/3] erofs-utils: support limit max decompressed extent size

[Gao Xiang]

Currently, some data pattern (e.g. zeroed data) could cause very
high CR with EROFS. However, for such extreme high C/R compressed
pclusters, the kernel side hasn't optimize such cases yet (such as
decompressing such pclusters fully in advance.)

Alternatively, let's add support to limit each decompressed extent
size for now.

Signed-off-by: Gao Xiang <xiang@kernel.org>
---
 include/erofs/config.h |  1 +
 lib/compress.c         |  2 +-
 lib/config.c           |  1 +
 man/mkfs.erofs.1       |  3 +++
 mkfs/main.c            | 11 +++++++++++
 5 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/include/erofs/config.h b/include/erofs/config.h
index 21bd25e886e6..d140a735bd49 100644
--- a/include/erofs/config.h
+++ b/include/erofs/config.h
@@ -58,6 +58,7 @@ struct erofs_configure {
 	int c_inline_xattr_tolerance;
 
 	u32 c_physical_clusterblks;
+	u32 c_max_decompressed_extent_bytes;
 	u64 c_unix_timestamp;
 	u32 c_uid, c_gid;
 #ifdef WITH_ANDROID
diff --git a/lib/compress.c b/lib/compress.c
index 1b847ce27c2f..2093bfd68b71 100644
--- a/lib/compress.c
+++ b/lib/compress.c
@@ -184,7 +184,7 @@ static int vle_compress_one(struct erofs_inode *inode,
 			}
 		}
 
-		count = len;
+		count = min(len, cfg.c_max_decompressed_extent_bytes);
 		ret = erofs_compress_destsize(h, compressionlevel,
 					      ctx->queue + ctx->head,
 					      &count, dst, pclustersize);
diff --git a/lib/config.c b/lib/config.c
index bc0afa284807..99fcf498f178 100644
--- a/lib/config.c
+++ b/lib/config.c
@@ -27,6 +27,7 @@ void erofs_init_configure(void)
 	cfg.c_uid = -1;
 	cfg.c_gid = -1;
 	cfg.c_physical_clusterblks = 1;
+	cfg.c_max_decompressed_extent_bytes = -1;
 }
 
 void erofs_show_config(void)
diff --git a/man/mkfs.erofs.1 b/man/mkfs.erofs.1
index 4f2e43565799..d164fa51fe17 100644
--- a/man/mkfs.erofs.1
+++ b/man/mkfs.erofs.1
@@ -85,6 +85,9 @@ Make all files owned by root.
 .TP
 .B \-\-help
 Display this help and exit.
+.TP
+.B \-\-max-extent-bytes #
+Specify maximum decompressed extent size # in bytes.
 .SH AUTHOR
 This version of \fBmkfs.erofs\fR is written by Li Guifu <blucerlee@gmail.com>,
 Miao Xie <miaoxie@huawei.com> and Gao Xiang <xiang@kernel.org> with
diff --git a/mkfs/main.c b/mkfs/main.c
index b2a4cba1d2f5..e476189f0731 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -42,6 +42,7 @@ static struct option long_options[] = {
 #ifndef NDEBUG
 	{"random-pclusterblks", no_argument, NULL, 8},
 #endif
+	{"max-extent-bytes", required_argument, NULL, 9},
 #ifdef WITH_ANDROID
 	{"mount-point", required_argument, NULL, 10},
 	{"product-out", required_argument, NULL, 11},
@@ -85,6 +86,7 @@ static void usage(void)
 	      " --force-gid=#         set all file gids to # (# = GID)\n"
 	      " --all-root            make all files owned by root\n"
 	      " --help                display this help and exit\n"
+	      " --max-extent-bytes=#  set maximum decompressed extent size # in bytes\n"
 #ifndef NDEBUG
 	      " --random-pclusterblks randomize pclusterblks for big pcluster (debugging only)\n"
 #endif
@@ -268,6 +270,15 @@ static int mkfs_parse_options_cfg(int argc, char *argv[])
 			cfg.c_random_pclusterblks = true;
 			break;
 #endif
+		case 9:
+			cfg.c_max_decompressed_extent_bytes =
+				strtoul(optarg, &endptr, 0);
+			if (*endptr != '\0') {
+				erofs_err("invalid maximum uncompressed extent size %s",
+					  optarg);
+				return -EINVAL;
+			}
+			break;
 #ifdef WITH_ANDROID
 		case 10:
 			cfg.mount_point = optarg;
-- 
2.20.1