From: Gao Xiang <hsiangkao@linux.alibaba.com>
To: linux-erofs@lists.ozlabs.org
Cc: Gao Xiang <hsiangkao@linux.alibaba.com>
Subject: [PATCH 2/3] erofs-utils: improve documentation for upcoming 1.6
Date: Thu, 9 Mar 2023 19:26:29 +0800 [thread overview]
Message-ID: <20230309112630.74230-2-hsiangkao@linux.alibaba.com> (raw)
In-Reply-To: <20230309112630.74230-1-hsiangkao@linux.alibaba.com>
Refine README as well as add docs/INSTALL.md and docs/PERFORMANCE.md.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
README | 241 ++++++++++++++++----------------------------
docs/INSTALL.md | 71 +++++++++++++
docs/PERFORMANCE.md | 182 +++++++++++++++++++++++++++++++++
3 files changed, 340 insertions(+), 154 deletions(-)
create mode 100644 docs/INSTALL.md
create mode 100644 docs/PERFORMANCE.md
diff --git a/README b/README
index 92b3128..6474ed1 100644
--- a/README
+++ b/README
@@ -1,7 +1,7 @@
erofs-utils
===========
-userspace tools for EROFS filesystem, currently including:
+Userspace tools for EROFS filesystem, currently including:
mkfs.erofs filesystem formatter
erofsfuse FUSE daemon alternative
@@ -9,76 +9,60 @@ userspace tools for EROFS filesystem, currently including:
fsck.erofs filesystem compatibility & consistency checker as well
as extractor
-Dependencies & build
---------------------
- lz4 1.8.0+ for lz4 enabled [2], lz4 1.9.3+ highly recommended [4][5].
- XZ Utils 5.3.2alpha [6] or later versions for MicroLZMA enabled.
-
- libfuse 2.6+ for erofsfuse enabled as a plus.
-
-How to build with lz4-1.9.0 or above
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To build, you can run the following commands in order:
-
-::
-
- $ ./autogen.sh
- $ ./configure
- $ make
-
-mkfs.erofs binary will be generated under mkfs folder.
+EROFS filesystem overview
+-------------------------
-* For lz4 < 1.9.2, there are some stability issues about
- LZ4_compress_destSize(). (lz4hc isn't impacted) [3].
+EROFS filesystem stands for Enhanced Read-Only File System. It aims to
+form a generic read-only filesystem solution for various read-only use
+cases instead of just focusing on storage space saving without
+considering any side effects of runtime performance.
-** For lz4 = 1.9.2, there is a noticeable regression about
- LZ4_decompress_safe_partial() [5], which impacts erofsfuse
- functionality for legacy images (without 0PADDING).
+Typically EROFS could be considered in the following use scenarios:
+ - Firmwares in performance-sensitive systems, such as system
+ partitions of Android smartphones;
-How to build with lz4-1.8.0~1.8.3
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ - Mountable immutable images such as container images for effective
+ metadata & data access compared with tar, cpio or other local
+ filesystems (e.g. ext4, XFS, btrfs, etc.)
-For these old lz4 versions, lz4hc algorithm cannot be supported
-without lz4-static installed due to LZ4_compress_HC_destSize()
-unstable api usage, which means lz4 will only be available if
-lz4-static isn't found.
+ - FSDAX-enabled rootfs for secure containers (Linux 5.15+);
-On Fedora, lz4-static can be installed by using:
+ - Live CDs which need a set of files with another high-performance
+ algorithm to optimize startup time; others files for archival
+ purposes only are not needed;
- yum install lz4-static.x86_64
+ - and more.
-However, it's still not recommended using those versions directly
-since there are serious bugs in these compressors, see [2] [3] [4]
-as well.
+Note that all EROFS metadata is uncompressed by design, so that you
+could take EROFS as a drop-in read-only replacement of ext4, XFS,
+btrfs, etc. without any compression-based dependencies and EROFS can
+bring more effective filesystem accesses to users with reduced
+metadata.
-How to build with liblzma
-~~~~~~~~~~~~~~~~~~~~~~~~~
+For more details of EROFS filesystem itself, please refer to:
+https://www.kernel.org/doc/html/next/filesystems/erofs.html
-In order to enable LZMA support, build with the following commands:
- $ ./configure --enable-lzma
- $ make
+For more details on how to build erofs-utils, see `docs/INSTALL.md`.
-Additionally, you could specify liblzma build paths with:
- --with-liblzma-incdir and --with-liblzma-libdir
+For more details about filesystem performance, see
+`docs/PERFORMANCE.md`.
mkfs.erofs
----------
-two main kinds of EROFS images can be generated: (un)compressed.
+Two main kinds of EROFS images can be generated: (un)compressed images.
- - For uncompressed images, there will be none of compression
- files in these images. However, it can decide whether the tail
- block of a file should be inlined or not properly [1].
+ - For uncompressed images, there will be none of compresssed files in
+ these images. However, it can decide whether the tail block of a
+ file should be inlined or not properly [1].
- - For compressed images, it'll try to use specific algorithms
- first for each regular file and see if storage space can be
- saved with compression. If not, fallback to an uncompressed
- file.
+ - For compressed images, it'll try to use the given algorithms first
+ for each regular file and see if storage space can be saved with
+ compression. If not, fallback to an uncompressed file.
-How to generate EROFS images (lz4 for Linux 5.3+, lzma for Linux 5.16+)
+How to generate EROFS images (LZ4 for Linux 5.3+, LZMA for Linux 5.16+)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Currently lz4(hc) and lzma are available for compression, e.g.
@@ -113,12 +97,55 @@ Note that large pcluster size can cause bad random performance, so
please evaluate carefully in advance. Or make your own per-(sub)file
compression strategies according to file access patterns if needed.
+How to generate EROFS images with multiple algorithms (Linux 5.16+)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's possible to generate an EROFS image with files in different
+algorithms due to various purposes. For example, LZMA for archival
+purposes and LZ4 for runtime purposes.
+
+In order to use alternative algorithms, just specify two or more
+compressing configurations together separated by ':' like below:
+ -zlzma:lz4hc,12:lzma,9 -C32768
+
+Although mkfs still choose the first one by default, you could try to
+write a compress-hints file like below:
+ 4096 1 .*\.so$
+ 32768 2 .*\.txt$
+ 4096 sbin/.*$
+ 16384 0 .*
+
+and specify with `--compress-hints=` so that ".so" files will use
+"lz4hc,12" compression with 4k pclusters, ".txt" files will use
+"lzma,9" compression with 32k pclusters, files under "/sbin" will use
+the default "lzma" compression with 4k plusters and other files will
+use "lzma" compression with 16k pclusters.
+
+Note that the largest pcluster size should be specified with the "-C"
+option (here 32k pcluster size), otherwise all larger pclusters will be
+limited.
+
+How to generate well-compressed EROFS images
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Even if EROFS is not designed for such purposes in the beginning, it
+could still produce some smaller images (not always) compared to other
+approaches with better performance (see `docs/PERFORMANCE.md`). In
+order to build well-compressed EROFS images, try the following options:
+ -C1048576 (5.13+)
+ -Eztailpacking (5.16+)
+ -Efragments / -Eall-fragments ( 6.1+);
+ -Ededupe ( 6.1+).
+
+Also EROFS uses lz4hc level 9 by default, whereas some other approaches
+use lz4hc level 12 by default. So please explicitly specify
+`-zlz4hc,12 ` for comparison purposes.
+
How to generate legacy EROFS images (Linux 4.19+)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Decompression inplace and compacted indexes have been introduced in
-Linux upstream v5.3, which are not forward-compatible with older
-kernels.
+Linux v5.3, which are not forward-compatible with older kernels.
In order to generate _legacy_ EROFS images for old kernels,
consider adding "-E legacy-compress" to the command line, e.g.
@@ -153,21 +180,6 @@ significant I/O overhead, double caching, etc.)
Therefore, NEVER use it if performance is the top concern.
-Note that extended attributes and ACLs aren't implemented yet due to
-the current Android use case vs limited time. If you are interested,
-contribution is, as always, welcome.
-
-How to build erofsfuse
-~~~~~~~~~~~~~~~~~~~~~~
-
-It's disabled by default as an experimental feature for now due to
-the extra libfuse dependency, to enable and build it manually:
-
- $ ./configure --enable-fuse
- $ make
-
-erofsfuse binary will be generated under fuse folder.
-
How to mount an EROFS image with erofsfuse
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -190,36 +202,8 @@ dump.erofs and fsck.erofs
dump.erofs and fsck.erofs are used to analyze, check, and extract
EROFS filesystems. Note that extended attributes and ACLs are still
-unsupported when extracting images with fsck.erofs.
-
-Container images
-----------------
-
-EROFS filesystem is well-suitably used for container images with
-advanced features like chunk-based files, multi-devices (blobs)
-and new fscache backend for lazy pulling and cache management, etc.
-
-For example, CNCF Dragonfly Nydus image service [7] introduces an
-(EROFS-compatible) RAFS v6 image format to overcome flaws of the
-current OCIv1 tgz images so that:
-
- - Images can be downloaded on demand in chunks aka lazy pulling with
- new fscache backend (5.19+) or userspace block devices (5.16+);
-
- - Finer chunk-based content-addressable data deduplication to minimize
- storage, transmission and memory footprints;
-
- - Merged filesystem tree to remove all metadata of intermediate layers
- as an option;
-
- - (e)stargz, zstd::chunked and other formats can be converted and run
- on the fly;
-
- - and more.
-
-Apart from Dragonfly Nydus, a native user daemon is planned to be added
-to erofs-utils to parse EROFS, (e)stargz and zstd::chunked images from
-network too as a real part of EROFS filesystem project.
+unsupported when extracting images with fsck.erofs. If you are
+interested, contribution is, as always, welcome.
Contribution
@@ -233,58 +217,7 @@ patches or feedback to:
Comments
--------
-[1] According to the EROFS on-disk format, the tail block of files
- could be inlined aggressively with its metadata in order to reduce
- the I/O overhead and save the storage space (called tail-packing).
-
-[2] There was a bug until lz4-1.8.3, which can crash erofs-utils
- randomly. Fortunately bugfix by our colleague Qiuyang Sun was
- merged in lz4-1.9.0.
-
- For more details, please refer to
- https://github.com/lz4/lz4/commit/660d21272e4c8a0f49db5fc1e6853f08713dff82
-
-[3] There were many bugfixes merged into lz4-1.9.2 for
- LZ4_compress_destSize(), and I once ran into some crashs due to
- those issues. * Again lz4hc is not affected. *
-
- [LZ4_compress_destSize] Allow 2 more bytes of match length
- https://github.com/lz4/lz4/commit/690009e2c2f9e5dcb0d40e7c0c40610ce6006eda
-
- [LZ4_compress_destSize] Fix rare data corruption bug
- https://github.com/lz4/lz4/commit/6bc6f836a18d1f8fd05c8fc2b42f1d800bc25de1
-
- [LZ4_compress_destSize] Fix overflow condition
- https://github.com/lz4/lz4/commit/13a2d9e34ffc4170720ce417c73e396d0ac1471a
-
- [LZ4_compress_destSize] Fix off-by-one error in fix
- https://github.com/lz4/lz4/commit/7c32101c655d93b61fc212dcd512b87119dd7333
-
- [LZ4_compress_destSize] Fix off-by-one error
- https://github.com/lz4/lz4/commit/d7cad81093cd805110291f84d64d385557d0ffba
-
- since upstream lz4 doesn't have stable branch for old versions, it's
- preferred to use latest upstream lz4 library (although some regressions
- could happen since new features are also introduced to latest upstream
- version as well) or backport all stable bugfixes to old stable versions,
- e.g. our unofficial lz4 fork: https://github.com/erofs/lz4
-
-[4] LZ4HC didn't compress long zeroed buffer properly with
- LZ4_compress_HC_destSize()
- https://github.com/lz4/lz4/issues/784
-
- which has been resolved in
- https://github.com/lz4/lz4/commit/e7fe105ac6ed02019d34731d2ba3aceb11b51bb1
-
- and already included in lz4-1.9.3, see:
- https://github.com/lz4/lz4/releases/tag/v1.9.3
-
-[5] LZ4_decompress_safe_partial is broken in 1.9.2
- https://github.com/lz4/lz4/issues/783
-
- which is also resolved in lz4-1.9.3.
-
-[6] https://tukaani.org/xz/xz-5.3.2alpha.tar.xz
-
-[7] https://nydus.dev
- https://github.com/dragonflyoss/image-service
+[1] According to the EROFS on-disk format, the tail blocks of files
+ could be inlined aggressively with their metadata (called
+ tail-packing) in order to minimize the extra I/Os and the storage
+ space.
diff --git a/docs/INSTALL.md b/docs/INSTALL.md
new file mode 100644
index 0000000..2e818da
--- /dev/null
+++ b/docs/INSTALL.md
@@ -0,0 +1,71 @@
+This document describes how to configure and build erofs-utils from
+source.
+
+See the [README](../README) file in the top level directory about
+the brief overview of erofs-utils.
+
+## Dependencies & build
+
+LZ4 1.9.3+ for LZ4(HC) enabled [^1].
+
+[XZ Utils 5.3.2alpha+](https://tukaani.org/xz/xz-5.3.2alpha.tar.gz) for
+LZMA enabled, [XZ Utils 5.4+](https://tukaani.org/xz/xz-5.4.1.tar.gz)
+highly recommended.
+
+libfuse 2.6+ for erofsfuse enabled.
+
+[^1]: It's not recommended to use LZ4 versions under 1.9.3 since
+unexpected crashes could make trouble to end users due to broken
+LZ4_compress_destSize() (fixed in v1.9.2),
+[LZ4_compress_HC_destSize()](https://github.com/lz4/lz4/commit/660d21272e4c8a0f49db5fc1e6853f08713dff82) or
+[LZ4_decompress_safe_partial()](https://github.com/lz4/lz4/issues/783).
+
+## How to build with LZ4
+
+To build, the following commands can be used in order:
+
+``` sh
+$ ./autogen.sh
+$ ./configure
+$ make
+```
+
+`mkfs.erofs`, `dump.erofs` and `fsck.erofs` binaries will be
+generated under the corresponding folders.
+
+## How to build with liblzma
+
+In order to enable LZMA support, build with the following commands:
+
+``` sh
+$ ./configure --enable-lzma
+$ make
+```
+
+Additionally, you could specify liblzma target paths with
+`--with-liblzma-incdir` and `--with-liblzma-libdir` manually.
+
+## How to build erofsfuse
+
+It's disabled by default as an experimental feature for now due
+to the extra libfuse dependency, to enable and build it manually:
+
+``` sh
+$ ./configure --enable-fuse
+$ make
+```
+
+`erofsfuse` binary will be generated under `fuse` folder.
+
+## How to install erofs-utils manually
+
+Use the following command to install erofs-utils binaries:
+
+``` sh
+# make install
+```
+
+By default, `make install` will install all the files in
+`/usr/local/bin`, `/usr/local/lib` etc. You can specify an
+installation prefix other than `/usr/local` using `--prefix`,
+for instance `--prefix=$HOME`.
diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md
new file mode 100644
index 0000000..202f2fe
--- /dev/null
+++ b/docs/PERFORMANCE.md
@@ -0,0 +1,182 @@
+# Test setup
+
+Processor: x86_64, Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz * 2 vcores
+
+Storage: Cloud disk, 3000 IOPS upper limit
+
+OS Kernel: Linux 6.2
+
+Softwares: LZ4 1.9.3, erofs-utils 1.6, squashes-tools 4.5.1
+
+Disclaimer: Test results could be varied from different hardware and/or data patterns. Therefore, the following results are ONLY for reference.
+
+# Benchmark on multiple files
+
+[rootfs of Debian docker image](https://github.com/debuerreotype/docker-debian-artifacts/blob/dist-amd64/bullseye/rootfs.tar.xz?raw=true) is used as the dataset, which contains 7000+ files and directories.
+Note that that dataset can be replaced regularly, and the SHA1 of the snapshot "rootfs.tar.xz" used here is "aee9b01a530078dbef8f08521bfcabe65b244955".
+
+## Image size
+
+| Size | Filesystem | Cluster size | Build options |
+|-----------|------------|--------------|-----------------------------------------------------------|
+| 124669952 | erofs | uncompressed | -T0 [^1] |
+| 124522496 | squashfs | uncompressed | -noD -noI -noX -noF -no-xattrs -all-time 0 -no-duplicates |
+| 73601024 | squashfs | 4096 | -b 4096 -comp lz4 -Xhc -no-xattrs -all-time 0 |
+| 73121792 | erofs | 4096 | -zlz4hc,12 [^2] -C4096 -Efragments -T0 |
+| 67162112 | squashfs | 16384 | -b 16384 -comp lz4 -Xhc -no-xattrs -all-time 0 |
+| 65478656 | erofs | 16384 | -zlz4hc,12 -C16384 -Efragments -T0 |
+| 61456384 | squashfs | 65536 | -b 65536 -comp lz4 -Xhc -no-xattrs -all-time 0 |
+| 59834368 | erofs | 65536 | -zlz4hc,12 -C65536 -Efragments -T0 |
+| 59150336 | squashfs | 131072 | -b 131072 -comp lz4 -Xhc -no-xattrs -all-time 0 |
+| 58515456 | erofs | 131072 | -zlz4hc,12 -C131072 -Efragments -T0 |
+
+[^1]: Forcely reset all timestamps to match squashfs on-disk basic inodes for now.
+[^2]: Because squashfs uses level 12 for LZ4HC by default.
+
+## Sequential data access
+
+hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "tar cf - . | cat > /dev/null"
+
+| Filesystem | Cluster size | Time |
+|------------|--------------|---------------------------------|
+| squashfs | 4096 | 10.257 s ± 0.031 s |
+| erofs | uncompressed | 1.111 s ± 0.022 s |
+| squashfs | uncompressed | 1.034 s ± 0.020 s |
+| squashfs | 131072 | 941.3 ms ± 7.5 ms |
+| erofs | 4096 | 848.1 ms ± 17.8 ms |
+| erofs | 131072 | 724.2 ms ± 11.0 ms |
+
+## Sequential metadata access
+
+hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "tar cf /dev/null ."
+
+| Filesystem | Cluster size | Time |
+|------------|--------------|---------------------------------|
+| erofs | uncompressed | 419.6 ms ± 8.2 ms |
+| squashfs | 4096 | 142.5 ms ± 5.4 ms |
+| squashfs | uncompressed | 129.2 ms ± 3.9 ms |
+| squashfs | 131072 | 125.4 ms ± 4.0 ms |
+| erofs | 4096 | 75.5 ms ± 3.5 ms |
+| erofs | 131072 | 65.8 ms ± 3.6 ms |
+
+[ Note that erofs-utils currently doesn't perform quite well for such cases due to metadata arrangement when building. It will be fixed in the later versions. ]
+
+## Small random data access (~7%)
+
+find mnt -type f -printf "%p\n" | sort -R | head -n 500 > list.txt
+hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "cat list.txt | xargs cat > /dev/null"
+
+| Filesystem | Cluster size | Time |
+|------------|--------------|---------------------------------|
+| squashfs | 4096 | 1.386 s ± 0.032 s |
+| squashfs | uncompressed | 1.083 s ± 0.044 s |
+| squashfs | 131072 | 1.067 s ± 0.046 s |
+| erofs | 4096 | 249.6 ms ± 6.5 ms |
+| erofs | uncompressed | 237.8 ms ± 6.3 ms |
+| erofs | 131072 | 189.6 ms ± 7.8 ms |
+
+
+## Small random metadata access (~7%)
+
+find mnt -type f -printf "%p\n" | sort -R | head -n 500 > list.txt
+hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "cat list.txt | xargs stat"
+
+| Filesystem | Cluster size | Time |
+|------------|--------------|---------------------------------|
+| squashfs | 4096 | 817.0 ms ± 34.5 ms |
+| squashfs | 131072 | 801.0 ms ± 40.1 ms |
+| squashfs | uncompressed | 741.3 ms ± 18.2 ms |
+| erofs | uncompressed | 197.8 ms ± 4.1 ms |
+| erofs | 4096 | 63.1 ms ± 2.0 ms |
+| erofs | 131072 | 60.7 ms ± 3.6 ms |
+
+## Full random data access (~100%)
+
+find mnt -type f -printf "%p\n" | sort -R > list.txt
+hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "cat list.txt | xargs cat > /dev/null"
+
+| Filesystem | Cluster size | Time |
+|------------|--------------|---------------------------------|
+| squashfs | 4096 | 20.668 s ± 0.040 s |
+| squashfs | uncompressed | 12.543 s ± 0.041 s |
+| squashfs | 131072 | 11.753 s ± 0.412 s |
+| erofs | uncompressed | 1.493 s ± 0.023 s |
+| erofs | 4096 | 1.223 s ± 0.013 s |
+| erofs | 131072 | 598.2 ms ± 6.6 ms |
+
+## Full random metadata access (~100%)
+
+find mnt -type f -printf "%p\n" | sort -R > list.txt
+hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "cat list.txt | xargs stat"
+
+| Filesystem | Cluster size | Time |
+|------------|--------------|---------------------------------|
+| squashfs | 131072 | 9.212 s ± 0.467 s |
+| squashfs | 4096 | 8.905 s ± 0.147 s |
+| squashfs | uncompressed | 7.961 s ± 0.045 s |
+| erofs | 4096 | 661.2 ms ± 14.9 ms |
+| erofs | uncompressed | 125.8 ms ± 6.6 ms |
+| erofs | 131072 | 119.6 ms ± 5.5 ms |
+
+
+# FIO benchmark on a single large file
+
+`silesia.tar` (203M) is used to benchmark, which could be generated from unzipping [silesia.zip](http://mattmahoney.net/dc/silesia.zip) and tar.
+
+## Image size
+
+| Size | Filesystem | Cluster size | Build options |
+|-----------|------------|--------------|-----------------------------------------------------------|
+| 114339840 | squashfs | 4096 | -b 4096 -comp lz4 -Xhc -no-xattrs |
+| 104972288 | erofs | 4096 | -zlz4hc,12 -C4096 |
+| 98033664 | squashfs | 16384 | -b 4096 -comp lz4 -Xhc -no-xattrs |
+| 89571328 | erofs | 16384 | -zlz4hc,12 -C16384 |
+| 85143552 | squashfs | 65536 | -b 65536 -comp lz4 -Xhc -no-xattrs |
+| 81211392 | squashfs | 131072 | -b 131072 -comp lz4 -Xhc -no-xattrs |
+| 80519168 | erofs | 65536 | -zlz4hc,12 -C65536 |
+| 78888960 | erofs | 131072 | -zlz4hc,12 -C131072 |
+
+## Sequential I/Os
+
+fio -filename=silesia.tar -bs=4k -rw=read -name=job1
+
+| Filesystem | Cluster size | Bandwidth |
+|------------|--------------|-----------|
+| erofs | 65536 | 624 MiB/s |
+| erofs | 16384 | 600 MiB/s |
+| erofs | 4096 | 569 MiB/s |
+| erofs | 131072 | 535 MiB/s |
+| squashfs | 131072 | 236 MiB/s |
+| squashfs | 65536 | 157 MiB/s |
+| squashfs | 16384 | 55.2MiB/s |
+| squashfs | 4096 | 12.5MiB/s |
+
+## Full Random I/Os
+
+fio -filename=silesia.tar -bs=4k -rw=randread -name=job1
+
+| Filesystem | Cluster size | Bandwidth |
+|------------|--------------|-----------|
+| erofs | 131072 | 242 MiB/s |
+| squashfs | 131072 | 232 MiB/s |
+| erofs | 65536 | 198 MiB/s |
+| squashfs | 65536 | 150 MiB/s |
+| erofs | 16384 | 96.4MiB/s |
+| squashfs | 16384 | 49.5MiB/s |
+| erofs | 4096 | 33.7MiB/s |
+| squashfs | 4096 | 6817KiB/s |
+
+## Small Random I/Os (5%)
+
+fio -filename=mnt/silesia.tar -bs=4k -rw=randread --io_size=10m -name=job1
+
+| Filesystem | Cluster size | Bandwidth |
+|------------|--------------|-----------|
+| erofs | 131072 | 19.2MiB/s |
+| erofs | 65536 | 16.9MiB/s |
+| squashfs | 131072 | 15.1MiB/s |
+| erofs | 16384 | 14.7MiB/s |
+| squashfs | 65536 | 13.8MiB/s |
+| erofs | 4096 | 13.0MiB/s |
+| squashfs | 16384 | 11.7MiB/s |
+| squashfs | 4096 | 4376KiB/s |
--
2.24.4
next prev parent reply other threads:[~2023-03-09 11:26 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-09 11:26 [PATCH 1/3] erofs-utils: optimize dedupe matching Gao Xiang
2023-03-09 11:26 ` Gao Xiang [this message]
2023-03-09 11:26 ` [PATCH 3/3] erofs-utils: lib: fix errors when building xattrs Gao Xiang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230309112630.74230-2-hsiangkao@linux.alibaba.com \
--to=hsiangkao@linux.alibaba.com \
--cc=linux-erofs@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.