All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] erofs-utils: update README
@ 2021-11-21 11:25 Gao Xiang
  0 siblings, 0 replies; 2+ messages in thread
From: Gao Xiang @ 2021-11-21 11:25 UTC (permalink / raw)
  To: linux-erofs

update README for the upcoming erofs-utils v1.4.

Signed-off-by: Gao Xiang <xiang@kernel.org>
---
 README | 55 +++++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 14 deletions(-)

diff --git a/README b/README
index 333dcb4..8a47a55 100644
--- a/README
+++ b/README
@@ -2,7 +2,8 @@ erofs-utils
 ===========
 
 erofs-utils includes user-space tools for EROFS filesystem.
-Currently mkfs.erofs and erofsfuse (experimental) are available.
+Currently mkfs.erofs, (experimental) erofsfuse, dump.erofs, fsck.erofs
+are available.
 
 Dependencies & build
 --------------------
@@ -12,8 +13,8 @@ Dependencies & build
 
  libfuse 2.6+ for erofsfuse enabled as a plus.
 
-How to build for lz4-1.9.0 or above
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+How to build with lz4-1.9.0 or above
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 To build, you can run the following commands in order:
 
@@ -32,8 +33,8 @@ mkfs.erofs binary will be generated under mkfs folder.
    LZ4_decompress_safe_partial() [5], which impacts erofsfuse
    functionality for legacy images (without 0PADDING).
 
-How to build for lz4-1.8.0~1.8.3
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+How to build with lz4-1.8.0~1.8.3
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 For these old lz4 versions, lz4hc algorithm cannot be supported
 without lz4-static installed due to LZ4_compress_HC_destSize()
@@ -48,6 +49,16 @@ However, it's still not recommended using those versions directly
 since there are serious bugs in these compressors, see [2] [3] [4]
 as well.
 
+How to build with liblzma
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to enable LZMA support, build with the following commands:
+	$ ./configure --enable-lzma
+	$ make
+
+Additionally, you could specify liblzma build paths with:
+	--with-liblzma-incdir and --with-liblzma-libdir
+
 mkfs.erofs
 ----------
 
@@ -57,15 +68,15 @@ two main kinds of EROFS images can be generated: (un)compressed.
    files in these images. However, it can decide whether the tail
    block of a file should be inlined or not properly [1].
 
- - For compressed images, it will try to use lz4(hc) algorithm
+ - For compressed images, it'll try to use specific algorithms
    first for each regular file and see if storage space can be
    saved with compression. If not, fallback to an uncompressed
    file.
 
-How to generate EROFS images (Linux 5.3+)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+How to generate EROFS images (lz4 for Linux 5.3+, lzma for Linux 5.16+)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Currently lz4 and lz4hc are available for compression, e.g.
+Currently lz4(hc) and lzma are available for compression, e.g.
  $ mkfs.erofs -zlz4hc foo.erofs.img foo/
 
 Or leave all files uncompressed as an option:
@@ -75,6 +86,10 @@ In addition, you could specify a higher compression level to get a
 (slightly) better compression ratio than the default level, e.g.
  $ mkfs.erofs -zlz4hc,12 foo.erofs.img foo/
 
+Note that all compressors are still single-threaded for now, thus it
+could take more time on the multiprocessor platform. Multi-threaded
+approach is already in our TODO list.
+
 How to generate EROFS big pcluster images (Linux 5.13+)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -97,7 +112,8 @@ How to generate legacy EROFS images (Linux 4.19+)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Decompression inplace and compacted indexes have been introduced in
-linux-5.3, which are not forward-compatible with older kernels.
+Linux upstream v5.3, which are not forward-compatible with older
+kernels.
 
 In order to generate _legacy_ EROFS images for old kernels,
 consider adding "-E legacy-compress" to the command line, e.g.
@@ -117,8 +133,8 @@ git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b obsoleted
 
 PLEASE NOTE: This version is highly _NOT recommended_ now.
 
-erofsfuse (experimental, unstable)
-----------------------------------
+erofsfuse (experimental)
+------------------------
 
 erofsfuse is introduced to support EROFS format for various platforms
 (including older linux kernels) and new on-disk features iteration.
@@ -138,8 +154,8 @@ as always, welcome.
 How to build erofsfuse
 ~~~~~~~~~~~~~~~~~~~~~~
 
-It's disabled by default as an experimental feature for now, to enable
-and build it manually:
+It's disabled by default as an experimental feature for now due to
+the extra libfuse dependency, to enable and build it manually:
 
 	$ ./configure --enable-fuse
 	$ make
@@ -162,6 +178,17 @@ To debug erofsfuse (also automatically run in foreground):
 To unmount an erofsfuse mountpoint as a non-root user:
  $ fusermount -u foo/
 
+dump.erofs and fsck.erofs (experimental)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+dump.erofs and fsck.erofs are two new experimental tools to analyse
+and check EROFS file systems.
+
+They are still incomplete and actively under development by the
+community. But you could check them out if needed in advance.
+
+Report, feedback and/or contribution are welcomed.
+
 Contribution
 ------------
 
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* [PATCH] erofs-utils: update README
@ 2022-06-11 11:36 Gao Xiang
  0 siblings, 0 replies; 2+ messages in thread
From: Gao Xiang @ 2022-06-11 11:36 UTC (permalink / raw)
  To: linux-erofs

update README for the upcoming erofs-utils v1.5, such as:

 - Image extraction with fsck.erofs;

 - Container image use cases.

Signed-off-by: Gao Xiang <xiang@kernel.org>
---
 README | 71 ++++++++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 54 insertions(+), 17 deletions(-)

diff --git a/README b/README
index aadd880..92b3128 100644
--- a/README
+++ b/README
@@ -1,9 +1,13 @@
 erofs-utils
 ===========
 
-erofs-utils includes user-space tools for EROFS filesystem.
-Currently mkfs.erofs, (experimental) erofsfuse, dump.erofs, fsck.erofs
-are available.
+userspace tools for EROFS filesystem, currently including:
+
+  mkfs.erofs    filesystem formatter
+  erofsfuse     FUSE daemon alternative
+  dump.erofs    filesystem analyzer
+  fsck.erofs    filesystem compatibility & consistency checker as well
+                as extractor
 
 Dependencies & build
 --------------------
@@ -59,6 +63,7 @@ In order to enable LZMA support, build with the following commands:
 Additionally, you could specify liblzma build paths with:
 	--with-liblzma-incdir and --with-liblzma-libdir
 
+
 mkfs.erofs
 ----------
 
@@ -133,8 +138,9 @@ git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b obsoleted
 
 PLEASE NOTE: This version is highly _NOT recommended_ now.
 
-erofsfuse (experimental)
-------------------------
+
+erofsfuse
+---------
 
 erofsfuse is introduced to support EROFS format for various platforms
 (including older linux kernels) and new on-disk features iteration.
@@ -147,9 +153,9 @@ significant I/O overhead, double caching, etc.)
 
 Therefore, NEVER use it if performance is the top concern.
 
-Note that xattr & ACL aren't implemented yet due to the current Android
-use-case vs limited time. If you have some interest, contribution is,
-as always, welcome.
+Note that extended attributes and ACLs aren't implemented yet due to
+the current Android use case vs limited time. If you are interested,
+contribution is, as always, welcome.
 
 How to build erofsfuse
 ~~~~~~~~~~~~~~~~~~~~~~
@@ -178,24 +184,52 @@ To debug erofsfuse (also automatically run in foreground):
 To unmount an erofsfuse mountpoint as a non-root user:
  $ fusermount -u foo/
 
-dump.erofs and fsck.erofs (experimental)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-dump.erofs and fsck.erofs are two new experimental tools to analyse
-and check EROFS file systems.
+dump.erofs and fsck.erofs
+-------------------------
+
+dump.erofs and fsck.erofs are used to analyze, check, and extract
+EROFS filesystems. Note that extended attributes and ACLs are still
+unsupported when extracting images with fsck.erofs.
+
+Container images
+----------------
+
+EROFS filesystem is well-suitably used for container images with
+advanced features like chunk-based files, multi-devices (blobs)
+and new fscache backend for lazy pulling and cache management, etc.
+
+For example, CNCF Dragonfly Nydus image service [7] introduces an
+(EROFS-compatible) RAFS v6 image format to overcome flaws of the
+current OCIv1 tgz images so that:
 
-They are still incomplete and actively under development by the
-community. But you could check them out if needed in advance.
+ - Images can be downloaded on demand in chunks aka lazy pulling with
+   new fscache backend (5.19+) or userspace block devices (5.16+);
+
+ - Finer chunk-based content-addressable data deduplication to minimize
+   storage, transmission and memory footprints;
+
+ - Merged filesystem tree to remove all metadata of intermediate layers
+   as an option;
+
+ - (e)stargz, zstd::chunked and other formats can be converted and run
+   on the fly;
+
+ - and more.
+
+Apart from Dragonfly Nydus, a native user daemon is planned to be added
+to erofs-utils to parse EROFS, (e)stargz and zstd::chunked images from
+network too as a real part of EROFS filesystem project.
 
-Report, feedback and/or contribution are welcomed.
 
 Contribution
 ------------
 
-erofs-utils is under GPLv2+ as a part of EROFS filesystem project,
-feel free to send patches or feedback to:
+erofs-utils is a part of EROFS filesystem project, feel free to send
+patches or feedback to:
   linux-erofs mailing list   <linux-erofs@lists.ozlabs.org>
 
+
 Comments
 --------
 
@@ -251,3 +285,6 @@ Comments
     which is also resolved in lz4-1.9.3.
 
 [6] https://tukaani.org/xz/xz-5.3.2alpha.tar.xz
+
+[7] https://nydus.dev
+    https://github.com/dragonflyoss/image-service
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-06-11 11:37 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-21 11:25 [PATCH] erofs-utils: update README Gao Xiang
2022-06-11 11:36 Gao Xiang

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.