From: Gao Xiang via Linux-erofs <linux-erofs@lists.ozlabs.org>
To: linux-erofs@lists.ozlabs.org
Subject: [RFC PATCH v0 0/3] erofs-utils: support multiple block compression
Date: Wed, 30 Dec 2020 16:47:25 +0800 [thread overview]
Message-ID: <20201230084728.813-1-hsiangkao@aol.com> (raw)
In-Reply-To: 20201230084728.813-1-hsiangkao.ref@aol.com
From: Gao Xiang <hsiangkao@aol.com>
Hi folks,
This is the first RFC patch of multiple block compression (including
erofsfuse) after I carefully think over the on-disk design to support
multiblock in-place decompression.
Compression ratio results (POC, lz4hc, lz4-1.9.3, not final result):
1000000000 enwik9
621211648 enwik9_4k.squashfs.img
557858816 enwik9_4k.erofs.img
556191744 enwik9_8k.squashfs.img
502661120 enwik9_16k.squashfs.img
500723712 enwik9_8k.erofs.img
458784768 enwik9_32k.squashfs.img
453971968 enwik9_16k.erofs.img
422318080 enwik9_64k.squashfs.img
416686080 enwik9_32k.erofs.img
398204928 enwik9_128k.squashfs.img
395276288 enwik9_64k.erofs.img
TODO:
- support compact indexes for multiple block compression **;
- support multithread compression (keep compressed data in memory);
- carefully design kernel optimized paths to maximize runtime performance;
- widely testing.
If you think that'd be useful for your products and you also have interest
in development, feel free to follow that as well since I don't have abundant
free time so the progress might be somewhat slow (I tend to finish them all before
the next LTS).
Thanks,
Gao Xiang
Gao Xiang (3):
erofs-utils: add -C# for the maximum size of pclusters
erofs-utils: mkfs: support multiple block compression
erofs-utils: fuse: support multiple block compression
include/erofs/config.h | 2 ++
include/erofs/internal.h | 1 +
include/erofs_fs.h | 19 ++++++++---
lib/compress.c | 70 ++++++++++++++++++++++++--------------
lib/config.c | 1 +
lib/data.c | 4 +--
lib/zmap.c | 72 ++++++++++++++++++++++++++++++++++++----
mkfs/main.c | 14 +++++++-
8 files changed, 146 insertions(+), 37 deletions(-)
--
2.24.0
next parent reply other threads:[~2020-12-30 8:48 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20201230084728.813-1-hsiangkao.ref@aol.com>
2020-12-30 8:47 ` Gao Xiang via Linux-erofs [this message]
2020-12-30 8:47 ` [RFC PATCH v0 1/3] erofs-utils: add -C# for the maximum size of pclusters Gao Xiang via Linux-erofs
2020-12-30 8:47 ` [RFC PATCH v0 2/3] erofs-utils: mkfs: support multiple block compression Gao Xiang via Linux-erofs
2020-12-30 8:47 ` [RFC PATCH v0 3/3] erofs-utils: fuse: " Gao Xiang via Linux-erofs
2020-12-30 10:12 ` [RFC PATCH v0 0/3] erofs-utils: " Gao Xiang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201230084728.813-1-hsiangkao@aol.com \
--to=linux-erofs@lists.ozlabs.org \
--cc=hsiangkao@aol.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).