From: Yue Hu <zbestahu@gmail.com>
To: linux-erofs@lists.ozlabs.org
Cc: Yue Hu <huyue2@coolpad.com>,
zhangwen@coolpad.com, shaojunjun@coolpad.com, zbestahu@163.com
Subject: [RFC PATCH 0/3] erofs-utils: compressed fragments feature
Date: Mon, 11 Jul 2022 17:09:55 +0800 [thread overview]
Message-ID: <cover.1657528899.git.huyue2@coolpad.com> (raw)
In order to achieve greater compression ratio, let's introduce
compressed fragments feature which can merge tail of per-file or the
whole files into one special inode to reach the target.
And we can also set pcluster size to fragments inode for different
compression requirments.
In this patchset, we also improve the uncompressed data layout of
compressed files. Just write it from 'clusterofs' instead of 0 since it
can benefit from in-place I/O. For now, it only goes with fragments.
The main idea above is from Xiang.
Here is some test data of Linux 5.10.87 source code under Ubuntu 18.04:
linux-5.10.87 (erofs, uncompressed) 1.1G
linux-5.10.87 (erofs, lz4hc,12 4k fragments,4k) 304M
linux-5.10.87 (erofs, lz4hc,12 8k fragments,8k) 271M
linux-5.10.87 (erofs, lz4hc,12 16k fragments,16k) 245M
linux-5.10.87 (erofs, lz4hc,12 32k fragments,32k) 228M
linux-5.10.87 (erofs, lz4hc,12 64k fragments,64k) 220M
linux-5.10.87 (erofs, lz4hc,12 4k vanilla) 396M
linux-5.10.87 (erofs, lz4hc,12 8k vanilla) 376M
linux-5.10.87 (erofs, lz4hc,12 16k vanilla) 364M
linux-5.10.87 (erofs, lz4hc,12 32k vanilla) 359M
linux-5.10.87 (erofs, lz4hc,12 64k vanilla) 358M
Usage:
mkfs.erofs -zlz4hc,12 -C65536 -Efragments,65536 foo.erofs.img foo/
Yue Hu (3):
erofs-utils: lib: add support for fragments data decompression
erofs-utils: lib: support on-disk offset for shifted decompression
erofs-utils: introduce compressed fragments support
include/erofs/compress.h | 2 +-
include/erofs/config.h | 3 +-
include/erofs/decompress.h | 3 ++
include/erofs/fragments.h | 25 ++++++++++
include/erofs/inode.h | 2 +
include/erofs/internal.h | 9 ++++
include/erofs_fs.h | 20 ++++++--
lib/Makefile.am | 4 +-
lib/compress.c | 94 ++++++++++++++++++++++++++++----------
lib/data.c | 33 ++++++++++++-
lib/decompress.c | 10 +++-
lib/fragments.c | 77 +++++++++++++++++++++++++++++++
lib/inode.c | 50 +++++++++++++-------
lib/super.c | 1 +
lib/zmap.c | 14 ++++++
mkfs/main.c | 63 ++++++++++++++++++++++---
16 files changed, 352 insertions(+), 58 deletions(-)
create mode 100644 include/erofs/fragments.h
create mode 100644 lib/fragments.c
--
2.17.1
next reply other threads:[~2022-07-11 9:10 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-11 9:09 Yue Hu [this message]
[not found] ` <cover.1657530420.git.huyue2@coolpad.com>
2022-07-11 9:09 ` [RFC PATCH 1/3] erofs-utils: lib: add support for fragments data decompression Yue Hu
2022-07-11 9:09 ` [RFC PATCH 2/3] erofs-utils: lib: support on-disk offset for shifted decompression Yue Hu
2022-07-11 9:09 ` [RFC PATCH 3/3] erofs-utils: introduce compressed fragments support Yue Hu
2022-07-11 10:02 ` [RFC PATCH 0/3] erofs-utils: compressed fragments feature Gao Xiang
2022-07-11 10:14 ` Yue Hu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1657528899.git.huyue2@coolpad.com \
--to=zbestahu@gmail.com \
--cc=huyue2@coolpad.com \
--cc=linux-erofs@lists.ozlabs.org \
--cc=shaojunjun@coolpad.com \
--cc=zbestahu@163.com \
--cc=zhangwen@coolpad.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.