From: Huang Jianan via Linux-erofs <linux-erofs@lists.ozlabs.org>
To: linux-erofs@lists.ozlabs.org
Cc: guoweichao@oppo.com, zhangshiming@oppo.com
Subject: [PATCH] erofs-utils: support adjust lz4 history window size
Date: Thu, 18 Feb 2021 20:00:09 +0800 [thread overview]
Message-ID: <20210218120009.17212-1-huangjianan@oppo.com> (raw)
lz4 uses LZ4_DISTANCE_MAX to record history preservation. When
using rolling decompression, a block with a higher compression
ratio will cause a larger memory allocation (up to 64k). It may
cause a large resource burden in extreme cases on devices with
small memory and a large number of concurrent IOs. So appropriately
reducing this value can improve performance.
Decreasing this value will reduce the compression ratio (except
when input_size <LZ4_DISTANCE_MAX). But considering that erofs
currently only supports 4k output, reducing this value will not
significantly reduce the compression benefits.
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
---
include/erofs/internal.h | 3 +++
include/erofs_fs.h | 3 ++-
lib/compressor_lz4.c | 5 +++++
lib/compressor_lz4hc.c | 3 +++
mkfs/main.c | 1 +
5 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index ac5b270..7a2489e 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -79,6 +79,9 @@ struct erofs_sb_info {
u64 inos;
u8 uuid[16];
+
+ /* compression algorithm specific parameters */
+ u16 compr_alg;
};
/* global sbi */
diff --git a/include/erofs_fs.h b/include/erofs_fs.h
index a69f179..8cb24da 100644
--- a/include/erofs_fs.h
+++ b/include/erofs_fs.h
@@ -41,7 +41,8 @@ struct erofs_super_block {
__u8 uuid[16]; /* 128-bit uuid for volume */
__u8 volume_name[16]; /* volume name */
__le32 feature_incompat;
- __u8 reserved2[44];
+ __le16 compr_alg; /* compression algorithm specific parameters */
+ __u8 reserved2[42];
};
/*
diff --git a/lib/compressor_lz4.c b/lib/compressor_lz4.c
index 8540a0d..db8d3d2 100644
--- a/lib/compressor_lz4.c
+++ b/lib/compressor_lz4.c
@@ -10,6 +10,10 @@
#include "erofs/internal.h"
#include "compressor.h"
+#ifndef LZ4_DISTANCE_MAX /* history window size */
+#define LZ4_DISTANCE_MAX 65535 /* set to maximum value by default */
+#endif
+
static int lz4_compress_destsize(struct erofs_compress *c,
int compression_level,
void *src, unsigned int *srcsize,
@@ -32,6 +36,7 @@ static int compressor_lz4_exit(struct erofs_compress *c)
static int compressor_lz4_init(struct erofs_compress *c)
{
c->alg = &erofs_compressor_lz4;
+ sbi.compr_alg = LZ4_DISTANCE_MAX;
return 0;
}
diff --git a/lib/compressor_lz4hc.c b/lib/compressor_lz4hc.c
index 6680563..38436f9 100644
--- a/lib/compressor_lz4hc.c
+++ b/lib/compressor_lz4hc.c
@@ -44,6 +44,9 @@ static int compressor_lz4hc_init(struct erofs_compress *c)
c->private_data = LZ4_createStreamHC();
if (!c->private_data)
return -ENOMEM;
+
+ sbi.compr_alg = LZ4_DISTANCE_MAX;
+
return 0;
}
diff --git a/mkfs/main.c b/mkfs/main.c
index d9c4c7f..c2ed261 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -299,6 +299,7 @@ int erofs_mkfs_update_super_block(struct erofs_buffer_head *bh,
.feature_incompat = cpu_to_le32(sbi.feature_incompat),
.feature_compat = cpu_to_le32(sbi.feature_compat &
~EROFS_FEATURE_COMPAT_SB_CHKSUM),
+ .compr_alg = cpu_to_le16(sbi.compr_alg),
};
const unsigned int sb_blksize =
round_up(EROFS_SUPER_END, EROFS_BLKSIZ);
--
2.25.1
reply other threads:[~2021-02-18 12:00 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210218120009.17212-1-huangjianan@oppo.com \
--to=linux-erofs@lists.ozlabs.org \
--cc=guoweichao@oppo.com \
--cc=huangjianan@oppo.com \
--cc=zhangshiming@oppo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).