linux-erofs.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
From: Gao Xiang <xiang@kernel.org>
To: linux-erofs@lists.ozlabs.org
Cc: Gao Xiang <xiang@kernel.org>
Subject: [PATCH v2 3/3] erofs-utils: support limit max decompressed extent size
Date: Mon, 31 May 2021 22:31:17 +0800	[thread overview]
Message-ID: <20210531143117.6327-3-xiang@kernel.org> (raw)
In-Reply-To: <20210531143117.6327-1-xiang@kernel.org>

Currently, some data pattern (e.g. zeroed data) could cause very
high CR with EROFS. However, for such extreme high C/R compressed
pclusters, the kernel side hasn't optimize such cases yet (such as
decompressing such pclusters fully in advance.)

Alternatively, let's add support to limit each decompressed extent
size for now.

Signed-off-by: Gao Xiang <xiang@kernel.org>
---
 include/erofs/config.h |  1 +
 lib/compress.c         |  2 +-
 lib/config.c           |  1 +
 man/mkfs.erofs.1       |  3 +++
 mkfs/main.c            | 11 +++++++++++
 5 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/include/erofs/config.h b/include/erofs/config.h
index 21bd25e886e6..d140a735bd49 100644
--- a/include/erofs/config.h
+++ b/include/erofs/config.h
@@ -58,6 +58,7 @@ struct erofs_configure {
 	int c_inline_xattr_tolerance;
 
 	u32 c_physical_clusterblks;
+	u32 c_max_decompressed_extent_bytes;
 	u64 c_unix_timestamp;
 	u32 c_uid, c_gid;
 #ifdef WITH_ANDROID
diff --git a/lib/compress.c b/lib/compress.c
index 1b847ce27c2f..2093bfd68b71 100644
--- a/lib/compress.c
+++ b/lib/compress.c
@@ -184,7 +184,7 @@ static int vle_compress_one(struct erofs_inode *inode,
 			}
 		}
 
-		count = len;
+		count = min(len, cfg.c_max_decompressed_extent_bytes);
 		ret = erofs_compress_destsize(h, compressionlevel,
 					      ctx->queue + ctx->head,
 					      &count, dst, pclustersize);
diff --git a/lib/config.c b/lib/config.c
index bc0afa284807..99fcf498f178 100644
--- a/lib/config.c
+++ b/lib/config.c
@@ -27,6 +27,7 @@ void erofs_init_configure(void)
 	cfg.c_uid = -1;
 	cfg.c_gid = -1;
 	cfg.c_physical_clusterblks = 1;
+	cfg.c_max_decompressed_extent_bytes = -1;
 }
 
 void erofs_show_config(void)
diff --git a/man/mkfs.erofs.1 b/man/mkfs.erofs.1
index 4f2e43565799..d164fa51fe17 100644
--- a/man/mkfs.erofs.1
+++ b/man/mkfs.erofs.1
@@ -85,6 +85,9 @@ Make all files owned by root.
 .TP
 .B \-\-help
 Display this help and exit.
+.TP
+.B \-\-max-extent-bytes #
+Specify maximum decompressed extent size # in bytes.
 .SH AUTHOR
 This version of \fBmkfs.erofs\fR is written by Li Guifu <blucerlee@gmail.com>,
 Miao Xie <miaoxie@huawei.com> and Gao Xiang <xiang@kernel.org> with
diff --git a/mkfs/main.c b/mkfs/main.c
index b2a4cba1d2f5..e476189f0731 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -42,6 +42,7 @@ static struct option long_options[] = {
 #ifndef NDEBUG
 	{"random-pclusterblks", no_argument, NULL, 8},
 #endif
+	{"max-extent-bytes", required_argument, NULL, 9},
 #ifdef WITH_ANDROID
 	{"mount-point", required_argument, NULL, 10},
 	{"product-out", required_argument, NULL, 11},
@@ -85,6 +86,7 @@ static void usage(void)
 	      " --force-gid=#         set all file gids to # (# = GID)\n"
 	      " --all-root            make all files owned by root\n"
 	      " --help                display this help and exit\n"
+	      " --max-extent-bytes=#  set maximum decompressed extent size # in bytes\n"
 #ifndef NDEBUG
 	      " --random-pclusterblks randomize pclusterblks for big pcluster (debugging only)\n"
 #endif
@@ -268,6 +270,15 @@ static int mkfs_parse_options_cfg(int argc, char *argv[])
 			cfg.c_random_pclusterblks = true;
 			break;
 #endif
+		case 9:
+			cfg.c_max_decompressed_extent_bytes =
+				strtoul(optarg, &endptr, 0);
+			if (*endptr != '\0') {
+				erofs_err("invalid maximum uncompressed extent size %s",
+					  optarg);
+				return -EINVAL;
+			}
+			break;
 #ifdef WITH_ANDROID
 		case 10:
 			cfg.mount_point = optarg;
-- 
2.20.1


      parent reply	other threads:[~2021-05-31 14:31 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-31 14:31 [PATCH v2 1/3] erofs-utils: README: trivial updates Gao Xiang
2021-05-31 14:31 ` [PATCH v2 2/3] erofs-utils: README: big pcluster feature update Gao Xiang
2021-05-31 14:31 ` Gao Xiang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210531143117.6327-3-xiang@kernel.org \
    --to=xiang@kernel.org \
    --cc=linux-erofs@lists.ozlabs.org \
    --subject='Re: [PATCH v2 3/3] erofs-utils: support limit max decompressed extent size' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).