* [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation
[not found] <20201114182517.9738-1-hsiangkao.ref@aol.com>
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 01/12] " Gao Xiang via Linux-erofs
` (9 more replies)
0 siblings, 10 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Hi,
v3: https://lore.kernel.org/r/20201102155558.1995-1-hsiangkao@aol.com
v2: https://lore.kernel.org/r/20201024130959.23720-1-hsiangkao@aol.com
background & v1:
https://lore.kernel.org/r/20201017051621.7810-1-hsiangkao@aol.com
All main logic has been cleaned up. Send out delta patches this round.
As the next step, will form the formal patchset for this simple
erofsfuse feature.
Thanks,
Gao Xiang
Gao Xiang (9):
erofs-utils: fuse: clean up path walking
erofs: clean up compress data read
erofs-utils: fuse: get rid of erofs_vnode
erofs-utils: fuse: move namei.c to lib/
erofs-utils: fuse: kill read.c
erofs-utils: fuse: clean up readdir
erofs-utils: fuse: rename readir.c to dir.c
erofs-utils: fuse: cleanup main.c
erofs-utils: fuse: fix up configure.ac / Makefile.am
Huang Jianan (2):
erofs-utils: fuse: add special file support
erofs-utils: fuse: add compressed file support
Li Guifu (1):
erofs-utils: introduce fuse implementation
Makefile.am | 4 +
README | 28 ++-
configure.ac | 22 +-
fuse/Makefile.am | 10 +
fuse/dir.c | 104 ++++++++++
fuse/main.c | 222 ++++++++++++++++++++
include/erofs/decompress.h | 35 ++++
include/erofs/defs.h | 16 ++
include/erofs/internal.h | 87 +++++++-
include/erofs/io.h | 1 +
include/erofs_fs.h | 4 +
lib/Makefile.am | 4 +-
lib/data.c | 206 ++++++++++++++++++
lib/decompress.c | 87 ++++++++
lib/io.c | 16 ++
lib/namei.c | 205 ++++++++++++++++++
lib/zmap.c | 415 +++++++++++++++++++++++++++++++++++++
17 files changed, 1461 insertions(+), 5 deletions(-)
create mode 100644 fuse/Makefile.am
create mode 100644 fuse/dir.c
create mode 100644 fuse/main.c
create mode 100644 include/erofs/decompress.h
create mode 100644 lib/data.c
create mode 100644 lib/decompress.c
create mode 100644 lib/namei.c
create mode 100644 lib/zmap.c
--
2.24.0
^ permalink raw reply [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 01/12] erofs-utils: introduce fuse implementation
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 02/12] erofs-utils: fuse: add special file support Gao Xiang via Linux-erofs
` (8 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs; +Cc: Guo Weichao
From: Li Guifu <blucerlee@gmail.com>
Let's add erofsfuse approach, and benefits are:
- images can be supported on various platforms;
- new unpack tool can be developed based on this;
- new on-disk features can be iterated, verified effectively.
This commit only aims at reading a regular file.
Other file (e.g. compressed file) support is out of scope for now.
Signed-off-by: Li Guifu <blucerlee@gmail.com>
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
Makefile.am | 2 +-
README | 28 ++++-
configure.ac | 3 +-
fuse/Makefile.am | 14 +++
fuse/main.c | 203 +++++++++++++++++++++++++++++++++++
fuse/namei.c | 222 +++++++++++++++++++++++++++++++++++++++
fuse/namei.h | 17 +++
fuse/read.c | 72 +++++++++++++
fuse/read.h | 16 +++
fuse/readir.c | 121 +++++++++++++++++++++
fuse/readir.h | 17 +++
include/erofs/defs.h | 3 +
include/erofs/internal.h | 83 +++++++++++++++
include/erofs/io.h | 1 +
lib/Makefile.am | 4 +-
lib/data.c | 117 +++++++++++++++++++++
lib/io.c | 16 +++
17 files changed, 934 insertions(+), 5 deletions(-)
create mode 100644 fuse/Makefile.am
create mode 100644 fuse/main.c
create mode 100644 fuse/namei.c
create mode 100644 fuse/namei.h
create mode 100644 fuse/read.c
create mode 100644 fuse/read.h
create mode 100644 fuse/readir.c
create mode 100644 fuse/readir.h
create mode 100644 lib/data.c
diff --git a/Makefile.am b/Makefile.am
index 1d20577068c5..24f4a7b3d5ad 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -3,4 +3,4 @@
ACLOCAL_AMFLAGS = -I m4
-SUBDIRS = man lib mkfs
+SUBDIRS = man lib mkfs fuse
diff --git a/README b/README
index 5addd6b80e04..870858c48b7d 100644
--- a/README
+++ b/README
@@ -2,7 +2,8 @@ erofs-utils
===========
erofs-utils includes user-space tools for erofs filesystem images.
-Currently only mkfs.erofs is available.
+One is mkfs.erofs to create a image, the other is erofsfuse which
+is used to mount a image on a directory
mkfs.erofs
----------
@@ -95,6 +96,31 @@ It may still be useful since new erofs-utils has not been widely used in
commercial products. However, if that happens, please report bug to us
as well.
+erofs-utils: erofsfuse
+----------------------
+erofsfuse mount a erofs filesystem image created by mkfs.erofs to a directory, and then
+It can be listed, read files and so on.
+
+Dependencies
+~~~~~~~~~~~~
+FUSE library version: 2.9.7
+fusermount version: 2.9.7
+using FUSE kernel interface version 7.19
+
+How to installed fuse
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+sudo apt-get update
+sudo apt-get install -y fuse libfuse-dev
+
+How to build erofsfuse
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ $ ./autogen.sh
+ $ ./configure
+ $ make
+
+erofsfuse binary will be generated under fuse folder.
+
Contribution
------------
diff --git a/configure.ac b/configure.ac
index bff1e293789a..e6d2e5fd3702 100644
--- a/configure.ac
+++ b/configure.ac
@@ -247,6 +247,7 @@ AC_SUBST([liblz4_LIBS])
AC_CONFIG_FILES([Makefile
man/Makefile
lib/Makefile
- mkfs/Makefile])
+ mkfs/Makefile
+ fuse/Makefile])
AC_OUTPUT
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
new file mode 100644
index 000000000000..0c1a24763f59
--- /dev/null
+++ b/fuse/Makefile.am
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0+
+# Makefile.am
+
+AUTOMAKE_OPTIONS = foreign
+bin_PROGRAMS = erofsfuse
+erofsfuse_SOURCES = main.c namei.c read.c readir.c
+erofsfuse_CFLAGS = -Wall -Werror \
+ -I$(top_srcdir)/include \
+ $(shell pkg-config fuse --cflags) \
+ -DFUSE_USE_VERSION=26 \
+ -std=gnu99
+LDFLAGS += $(shell pkg-config fuse --libs)
+erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl
+
diff --git a/fuse/main.c b/fuse/main.c
new file mode 100644
index 000000000000..5121e8325f32
--- /dev/null
+++ b/fuse/main.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/fuse/main.c
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <execinfo.h>
+#include <signal.h>
+#include <stddef.h>
+
+#include "erofs/print.h"
+#include "namei.h"
+#include "read.h"
+#include "readir.h"
+#include "erofs/io.h"
+
+/* XXX: after liberofs is linked in, it should be removed */
+struct erofs_configure cfg;
+
+enum {
+ EROFS_OPT_HELP,
+ EROFS_OPT_VER,
+};
+
+struct options {
+ const char *disk;
+ const char *mount;
+ const char *logfile;
+ unsigned int debug_lvl;
+};
+static struct options fusecfg;
+
+#define OPTION(t, p) { t, offsetof(struct options, p), 1 }
+
+static const struct fuse_opt option_spec[] = {
+ OPTION("--log=%s", logfile),
+ OPTION("--dbg=%u", debug_lvl),
+ FUSE_OPT_KEY("-h", EROFS_OPT_HELP),
+ FUSE_OPT_KEY("-v", EROFS_OPT_VER),
+ FUSE_OPT_END
+};
+
+static void usage(void)
+{
+ fprintf(stderr, "\terofsfuse [options] <image> <mountpoint>\n");
+ fprintf(stderr, "\t --log=<file> output log file\n");
+ fprintf(stderr, "\t --dbg=<level> log debug level 0 ~ 7\n");
+ fprintf(stderr, "\t -h show help\n");
+ fprintf(stderr, "\t -v show version\n");
+ exit(1);
+}
+
+static void dump_cfg(void)
+{
+ fprintf(stderr, "\tdisk :%s\n", fusecfg.disk);
+ fprintf(stderr, "\tmount:%s\n", fusecfg.mount);
+ fprintf(stderr, "\tdebug_lvl:%u\n", fusecfg.debug_lvl);
+ fprintf(stderr, "\tlogfile :%s\n", fusecfg.logfile);
+}
+
+static int optional_opt_func(void *data, const char *arg, int key,
+ struct fuse_args *outargs)
+{
+ UNUSED(data);
+ UNUSED(outargs);
+
+ switch (key) {
+ case FUSE_OPT_KEY_OPT:
+ return 1;
+
+ case FUSE_OPT_KEY_NONOPT:
+ if (!fusecfg.disk) {
+ fusecfg.disk = strdup(arg);
+ return 0;
+ } else if (!fusecfg.mount)
+ fusecfg.mount = strdup(arg);
+
+ return 1;
+ case EROFS_OPT_HELP:
+ usage();
+ break;
+
+ case EROFS_OPT_VER:
+ fprintf(stderr, "EROFS FUSE VERSION v 1.0.0\n");
+ exit(0);
+ }
+
+ return 1;
+}
+
+static void signal_handle_sigsegv(int signal)
+{
+ void *array[10];
+ size_t nptrs;
+ char **strings;
+ size_t i;
+
+ UNUSED(signal);
+ erofs_dbg("========================================");
+ erofs_dbg("Segmentation Fault. Starting backtrace:");
+ nptrs = backtrace(array, 10);
+ strings = backtrace_symbols(array, nptrs);
+ if (strings) {
+ for (i = 0; i < nptrs; i++)
+ erofs_dbg("%s", strings[i]);
+ free(strings);
+ }
+ erofs_dbg("========================================");
+
+ abort();
+}
+
+void *erofs_init(struct fuse_conn_info *info)
+{
+ erofs_info("Using FUSE protocol %d.%d", info->proto_major, info->proto_minor);
+ return NULL;
+}
+
+int erofs_open(const char *path, struct fuse_file_info *fi)
+{
+ erofs_info("open path=%s", path);
+
+ if ((fi->flags & O_ACCMODE) != O_RDONLY)
+ return -EACCES;
+
+ return 0;
+}
+
+int erofs_getattr(const char *path, struct stat *stbuf)
+{
+ struct erofs_vnode v;
+ int ret;
+
+ erofs_dbg("getattr(%s)", path);
+ memset(&v, 0, sizeof(v));
+ ret = erofs_iget_by_path(path, &v);
+ if (ret)
+ return -ENOENT;
+
+ stbuf->st_mode = v.i_mode;
+ stbuf->st_nlink = v.i_nlink;
+ stbuf->st_size = v.i_size;
+ stbuf->st_blocks = stbuf->st_size / EROFS_BLKSIZ;
+ stbuf->st_uid = v.i_uid;
+ stbuf->st_gid = v.i_gid;
+ stbuf->st_atime = sbi.build_time;
+ stbuf->st_mtime = sbi.build_time;
+ stbuf->st_ctime = sbi.build_time;
+ return 0;
+}
+
+static struct fuse_operations erofs_ops = {
+ .getattr = erofs_getattr,
+ .readdir = erofs_readdir,
+ .open = erofs_open,
+ .read = erofs_read,
+ .init = erofs_init,
+};
+
+int main(int argc, char *argv[])
+{
+ int ret = EXIT_FAILURE;
+ struct fuse_args args = FUSE_ARGS_INIT(argc, argv);
+
+ if (signal(SIGSEGV, signal_handle_sigsegv) == SIG_ERR) {
+ fprintf(stderr, "Failed to initialize signals\n");
+ return EXIT_FAILURE;
+ }
+
+ /* Parse options */
+ if (fuse_opt_parse(&args, &fusecfg, option_spec, optional_opt_func) < 0)
+ return 1;
+
+ dump_cfg();
+
+ cfg.c_dbg_lvl = fusecfg.debug_lvl;
+
+ if (dev_open_ro(fusecfg.disk) < 0) {
+ fprintf(stderr, "Failed to open disk:%s\n", fusecfg.disk);
+ goto exit;
+ }
+
+ if (erofs_read_superblock()) {
+ fprintf(stderr, "Failed to read erofs super block\n");
+ goto exit_dev;
+ }
+
+ erofs_info("fuse start");
+
+ ret = fuse_main(args.argc, args.argv, &erofs_ops, NULL);
+
+ erofs_info("fuse done ret=%d", ret);
+
+exit_dev:
+ dev_close();
+exit:
+ fuse_opt_free_args(&args);
+ return ret;
+}
+
diff --git a/fuse/namei.c b/fuse/namei.c
new file mode 100644
index 000000000000..cd747ad1be56
--- /dev/null
+++ b/fuse/namei.c
@@ -0,0 +1,222 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/fuse/namei.c
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#include "namei.h"
+#include <linux/kdev_t.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <errno.h>
+#include <sys/stat.h>
+
+#include "erofs/defs.h"
+#include "erofs/print.h"
+#include "erofs/io.h"
+
+#define IS_PATH_SEPARATOR(__c) ((__c) == '/')
+#define MINORBITS 20
+#define MINORMASK ((1U << MINORBITS) - 1)
+#define DT_UNKNOWN 0
+
+static const char *skip_trailing_backslash(const char *path)
+{
+ while (IS_PATH_SEPARATOR(*path))
+ path++;
+ return path;
+}
+
+static uint8_t get_path_token_len(const char *path)
+{
+ uint8_t len = 0;
+
+ while (path[len] != '/' && path[len])
+ len++;
+ return len;
+}
+
+int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
+{
+ int ret;
+ char buf[EROFS_BLKSIZ];
+ struct erofs_inode_compact *v1;
+ const erofs_off_t addr = iloc(nid);
+ const size_t size = EROFS_BLKSIZ - erofs_blkoff(addr);
+
+ ret = dev_read(buf, addr, size);
+ if (ret < 0)
+ return -EIO;
+
+ v1 = (struct erofs_inode_compact *)buf;
+ vi->datalayout = __inode_data_mapping(le16_to_cpu(v1->i_format));
+ vi->inode_isize = sizeof(struct erofs_inode_compact);
+ vi->xattr_isize = erofs_xattr_ibody_size(v1->i_xattr_icount);
+ vi->i_size = le32_to_cpu(v1->i_size);
+ vi->i_mode = le16_to_cpu(v1->i_mode);
+ vi->i_uid = le16_to_cpu(v1->i_uid);
+ vi->i_gid = le16_to_cpu(v1->i_gid);
+ vi->i_nlink = le16_to_cpu(v1->i_nlink);
+ vi->nid = nid;
+
+ switch (vi->i_mode & S_IFMT) {
+ case S_IFBLK:
+ case S_IFCHR:
+ /* fixme: add special devices support
+ * vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
+ */
+ break;
+ case S_IFIFO:
+ case S_IFSOCK:
+ /*fixme: vi->i_rdev = 0; */
+ break;
+ case S_IFREG:
+ case S_IFLNK:
+ case S_IFDIR:
+ vi->raw_blkaddr = le32_to_cpu(v1->i_u.raw_blkaddr);
+ break;
+ default:
+ return -EIO;
+ }
+
+ return 0;
+}
+
+
+struct erofs_dirent *find_target_dirent(erofs_nid_t pnid,
+ void *dentry_blk,
+ const char *name, unsigned int len,
+ unsigned int nameoff,
+ unsigned int maxsize)
+{
+ struct erofs_dirent *de = dentry_blk;
+ const struct erofs_dirent *end = dentry_blk + nameoff;
+
+ while (de < end) {
+ const char *de_name;
+ unsigned int de_namelen;
+
+ nameoff = le16_to_cpu(de->nameoff);
+ de_name = (char *)dentry_blk + nameoff;
+
+ /* the last dirent in the block? */
+ if (de + 1 >= end)
+ de_namelen = strnlen(de_name, maxsize - nameoff);
+ else
+ de_namelen = le16_to_cpu(de[1].nameoff) - nameoff;
+
+ /* a corrupted entry is found */
+ if (nameoff + de_namelen > maxsize ||
+ de_namelen > EROFS_NAME_LEN) {
+ erofs_err("bogus dirent @ nid %llu", pnid | 0ULL);
+ DBG_BUGON(1);
+ return ERR_PTR(-EFSCORRUPTED);
+ }
+
+ if (len == de_namelen && !memcmp(de_name, name, de_namelen))
+ return de;
+ ++de;
+ }
+ return NULL;
+}
+
+int erofs_namei(erofs_nid_t *nid,
+ const char *name, unsigned int len)
+{
+ int ret;
+ char buf[EROFS_BLKSIZ];
+ struct erofs_vnode v;
+
+ ret = erofs_iget_by_nid(*nid, &v);
+ if (ret)
+ return ret;
+
+ {
+ unsigned int offset = 0;
+
+ struct erofs_inode tmp = {
+ .u = {
+ .i_blkaddr = v.raw_blkaddr,
+ },
+ .nid = v.nid,
+ .i_size = v.i_size,
+ .datalayout = v.datalayout,
+ .inode_isize = v.inode_isize,
+ .xattr_isize = v.xattr_isize,
+ };
+
+ while (offset < v.i_size) {
+ int maxsize = min(v.i_size - offset, EROFS_BLKSIZ);
+ struct erofs_dirent *de = (void *)buf;
+ unsigned int nameoff;
+
+ ret = erofs_read_raw_data(&tmp, buf, offset, maxsize);
+ if (ret)
+ return ret;
+
+ nameoff = le16_to_cpu(de->nameoff);
+
+ if (nameoff < sizeof(struct erofs_dirent) ||
+ nameoff >= PAGE_SIZE) {
+ erofs_err("invalid de[0].nameoff %u @ nid %llu",
+ nameoff, *nid | 0ULL);
+ return -EFSCORRUPTED;
+ }
+
+ de = find_target_dirent(*nid, buf, name, len,
+ nameoff, maxsize);
+ if (IS_ERR(de))
+ return PTR_ERR(de);
+
+ if (de) {
+ *nid = le64_to_cpu(de->nid);
+ return 0;
+ }
+ offset += maxsize;
+ }
+ }
+ return -ENOENT;
+}
+
+extern struct dcache_entry root_entry;
+int walk_path(const char *_path, erofs_nid_t *out_nid)
+{
+ int ret;
+ erofs_nid_t nid = sbi.root_nid;
+ const char *path = _path;
+
+ for (;;) {
+ uint8_t path_len;
+
+ path = skip_trailing_backslash(path);
+ path_len = get_path_token_len(path);
+ if (path_len == 0)
+ break;
+
+ ret = erofs_namei(&nid, path, path_len);
+ if (ret)
+ return ret;
+
+ path += path_len;
+ }
+
+ erofs_dbg("find path = %s nid=%llu", _path, nid | 0ULL);
+
+ *out_nid = nid;
+ return 0;
+
+}
+
+int erofs_iget_by_path(const char *path, struct erofs_vnode *v)
+{
+ int ret;
+ erofs_nid_t nid;
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ return erofs_iget_by_nid(nid, v);
+}
+
diff --git a/fuse/namei.h b/fuse/namei.h
new file mode 100644
index 000000000000..2625ec58d434
--- /dev/null
+++ b/fuse/namei.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-utils/fuse/inode.h
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#ifndef __INODE_H
+#define __INODE_H
+
+#include "erofs/internal.h"
+#include "erofs_fs.h"
+
+int erofs_iget_by_path(const char *path, struct erofs_vnode *v);
+int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *v);
+int walk_path(const char *path, erofs_nid_t *out_nid);
+
+#endif
diff --git a/fuse/read.c b/fuse/read.c
new file mode 100644
index 000000000000..446e0837cbc4
--- /dev/null
+++ b/fuse/read.c
@@ -0,0 +1,72 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/fuse/read.c
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#include "read.h"
+#include <errno.h>
+#include <linux/fs.h>
+#include <sys/stat.h>
+#include <string.h>
+
+#include "erofs/defs.h"
+#include "erofs/internal.h"
+#include "erofs/print.h"
+#include "namei.h"
+#include "erofs/io.h"
+
+size_t erofs_read_data_wrapper(struct erofs_vnode *vnode, char *buffer,
+ size_t size, off_t offset)
+{
+ struct erofs_inode tmp = {
+ .u = {
+ .i_blkaddr = vnode->raw_blkaddr,
+ },
+ .nid = vnode->nid,
+ .i_size = vnode->i_size,
+ .datalayout = vnode->datalayout,
+ .inode_isize = vnode->inode_isize,
+ .xattr_isize = vnode->xattr_isize,
+ };
+
+ int ret = erofs_read_raw_data(&tmp, buffer, offset, size);
+ if (ret)
+ return ret;
+
+ erofs_info("nid:%llu size=%zd done", (unsigned long long)vnode->nid, size);
+ return size;
+}
+
+int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
+ struct fuse_file_info *fi)
+{
+ int ret;
+ erofs_nid_t nid;
+ struct erofs_vnode v;
+
+ UNUSED(fi);
+ erofs_info("path:%s size=%zd offset=%llu", path, size, (long long)offset);
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ ret = erofs_iget_by_nid(nid, &v);
+ if (ret)
+ return ret;
+
+ erofs_info("path:%s nid=%llu mode=%u", path, (unsigned long long)nid, v.datalayout);
+ switch (v.datalayout) {
+ case EROFS_INODE_FLAT_PLAIN:
+ case EROFS_INODE_FLAT_INLINE:
+ return erofs_read_data_wrapper(&v, buffer, size, offset);
+
+ case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
+ case EROFS_INODE_FLAT_COMPRESSION:
+ /* Fixme: */
+ default:
+ return -EINVAL;
+ }
+}
+
diff --git a/fuse/read.h b/fuse/read.h
new file mode 100644
index 000000000000..3f4af1495510
--- /dev/null
+++ b/fuse/read.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-utils/fuse/read.h
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#ifndef __EROFS_READ_H
+#define __EROFS_READ_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+
+int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
+ struct fuse_file_info *fi);
+
+#endif
diff --git a/fuse/readir.c b/fuse/readir.c
new file mode 100644
index 000000000000..5281c8b80e59
--- /dev/null
+++ b/fuse/readir.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/fuse/readir.c
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#include "readir.h"
+#include <errno.h>
+#include <linux/fs.h>
+#include <sys/stat.h>
+
+#include "erofs/defs.h"
+#include "erofs/internal.h"
+#include "erofs_fs.h"
+#include "namei.h"
+#include "erofs/io.h"
+#include "erofs/print.h"
+
+erofs_nid_t split_entry(char *entry, off_t ofs, char *end, char *name,
+ uint32_t dirend)
+{
+ struct erofs_dirent *de = (struct erofs_dirent *)(entry + ofs);
+ uint16_t nameoff = le16_to_cpu(de->nameoff);
+ const char *de_name = entry + nameoff;
+ uint32_t de_namelen;
+
+ if ((void *)(de + 1) >= (void *)end)
+ de_namelen = strnlen(de_name, dirend - nameoff);
+ else
+ de_namelen = le16_to_cpu(de[1].nameoff) - nameoff;
+
+ memcpy(name, de_name, de_namelen);
+ name[de_namelen] = '\0';
+ return le64_to_cpu(de->nid);
+}
+
+int fill_dir(char *entrybuf, fuse_fill_dir_t filler, void *buf,
+ uint32_t dirend)
+{
+ uint32_t ofs;
+ uint16_t nameoff;
+ char *end;
+ char name[EROFS_BLKSIZ];
+
+ nameoff = le16_to_cpu(((struct erofs_dirent *)entrybuf)->nameoff);
+ end = entrybuf + nameoff;
+ ofs = 0;
+ while (ofs < nameoff) {
+ (void)split_entry(entrybuf, ofs, end, name, dirend);
+ filler(buf, name, NULL, 0);
+ ofs += sizeof(struct erofs_dirent);
+ }
+
+ return 0;
+}
+
+/** Function to add an entry in a readdir() operation
+ *
+ * @param buf the buffer passed to the readdir() operation
+ * @param name the file name of the directory entry
+ * @param stat file attributes, can be NULL
+ * @param off offset of the next entry or zero
+ * @return 1 if buffer is full, zero otherwise
+ */
+int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi)
+{
+ int ret;
+ erofs_nid_t nid;
+ struct erofs_vnode v;
+ char dirsbuf[EROFS_BLKSIZ];
+ uint32_t dir_nr, dir_off, nr_cnt;
+
+ erofs_dbg("readdir:%s offset=%llu", path, (long long)offset);
+ UNUSED(fi);
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ erofs_dbg("path=%s nid = %llu", path, (unsigned long long)nid);
+ ret = erofs_iget_by_nid(nid, &v);
+ if (ret)
+ return ret;
+
+ if (!S_ISDIR(v.i_mode))
+ return -ENOTDIR;
+
+ if (!v.i_size)
+ return 0;
+
+ dir_nr = erofs_blknr(v.i_size);
+ dir_off = erofs_blkoff(v.i_size);
+ nr_cnt = 0;
+
+ erofs_dbg("dir_size=%u dir_nr = %u dir_off=%u", v.i_size, dir_nr, dir_off);
+
+ while (nr_cnt < dir_nr) {
+ memset(dirsbuf, 0, sizeof(dirsbuf));
+ ret = blk_read(dirsbuf, v.raw_blkaddr + nr_cnt, 1);
+ if (ret < 0)
+ return -EIO;
+ fill_dir(dirsbuf, filler, buf, EROFS_BLKSIZ);
+ ++nr_cnt;
+ }
+
+ if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
+ off_t addr;
+
+ addr = iloc(nid) + v.inode_isize + v.xattr_isize;
+
+ memset(dirsbuf, 0, sizeof(dirsbuf));
+ ret = dev_read(dirsbuf, addr, dir_off);
+ if (ret < 0)
+ return -EIO;
+ fill_dir(dirsbuf, filler, buf, dir_off);
+ }
+
+ return 0;
+}
+
diff --git a/fuse/readir.h b/fuse/readir.h
new file mode 100644
index 000000000000..ee2ab8bdd0f0
--- /dev/null
+++ b/fuse/readir.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-utils/fuse/readir.h
+ *
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+#ifndef __EROFS_READDIR_H
+#define __EROFS_READDIR_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+
+int erofs_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi);
+
+
+#endif
diff --git a/include/erofs/defs.h b/include/erofs/defs.h
index 8dee661ab9f0..e97de36aa04b 100644
--- a/include/erofs/defs.h
+++ b/include/erofs/defs.h
@@ -24,6 +24,9 @@
#ifdef HAVE_LINUX_TYPES_H
#include <linux/types.h>
#endif
+#ifndef UNUSED
+#define UNUSED(_X) ((void) _X)
+#endif
/*
* container_of - cast a member of a structure out to the containing structure
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index cabb2faa0072..f9e757316efe 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -55,10 +55,14 @@ typedef u32 erofs_blk_t;
#define blknr_to_addr(nr) ((erofs_off_t)(nr) * EROFS_BLKSIZ)
#define BLK_ROUND_UP(addr) DIV_ROUND_UP(addr, EROFS_BLKSIZ)
+#define IS_SLOT_ALIGN(__ADDR) (((__ADDR) % (EROFS_SLOTSIZE)) ? 0 : 1)
+#define IS_BLK_ALIGN(__ADDR) (((__ADDR) % (EROFS_BLKSIZ)) ? 0 : 1)
struct erofs_buffer_head;
struct erofs_sb_info {
+ u64 blocks;
+
erofs_blk_t meta_blkaddr;
erofs_blk_t xattr_blkaddr;
@@ -66,12 +70,25 @@ struct erofs_sb_info {
u32 feature_incompat;
u64 build_time;
u32 build_time_nsec;
+
+ unsigned char islotbits;
+
+ /* what we really care is nid, rather than ino.. */
+ erofs_nid_t root_nid;
+ /* used for statfs, f_files - f_favail */
+ u64 inos;
+
u8 uuid[16];
};
/* global sbi */
extern struct erofs_sb_info sbi;
+static inline erofs_off_t iloc(erofs_nid_t nid)
+{
+ return blknr_to_addr(sbi.meta_blkaddr) + (nid << sbi.islotbits);
+}
+
#define EROFS_FEATURE_FUNCS(name, compat, feature) \
static inline bool erofs_sb_has_##name(void) \
{ \
@@ -132,11 +149,46 @@ struct erofs_inode {
#endif
};
+struct erofs_vnode {
+ uint8_t datalayout;
+
+ uint32_t i_size;
+ /* inline size in bytes */
+ uint16_t inode_isize;
+ uint16_t xattr_isize;
+
+ uint16_t xattr_shared_count;
+ char *xattr_shared_xattrs;
+
+ erofs_blk_t raw_blkaddr;
+ erofs_nid_t nid;
+ uint32_t i_ino;
+
+ uint16_t i_mode;
+ uint16_t i_uid;
+ uint16_t i_gid;
+ uint16_t i_nlink;
+
+ /* if file is inline read inline data witch inode */
+ char *idata;
+};
+
static inline bool is_inode_layout_compression(struct erofs_inode *inode)
{
return erofs_inode_is_data_compressed(inode->datalayout);
}
+#define __inode_advise(x, bit, bits) \
+ (((x) >> (bit)) & ((1 << (bits)) - 1))
+
+#define __inode_version(advise) \
+ __inode_advise(advise, EROFS_I_VERSION_BIT, \
+ EROFS_I_VERSION_BITS)
+
+#define __inode_data_mapping(advise) \
+ __inode_advise(advise, EROFS_I_DATALAYOUT_BIT,\
+ EROFS_I_DATALAYOUT_BITS)
+
#define IS_ROOT(x) ((x) == (x)->i_parent)
struct erofs_dentry {
@@ -169,5 +221,36 @@ static inline const char *erofs_strerror(int err)
return msg;
}
+enum {
+ BH_Meta,
+ BH_Mapped,
+ BH_Zipped,
+ BH_FullMapped,
+};
+
+/* Has a disk mapping */
+#define EROFS_MAP_MAPPED (1 << BH_Mapped)
+/* Located in metadata (could be copied from bd_inode) */
+#define EROFS_MAP_META (1 << BH_Meta)
+
+struct erofs_map_blocks {
+ char mpage[EROFS_BLKSIZ];
+
+ erofs_off_t m_pa, m_la;
+ u64 m_plen, m_llen;
+
+ unsigned int m_flags;
+ erofs_blk_t index;
+};
+
+/* super.c */
+int erofs_read_superblock(void);
+
+/* data.c */
+int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
+ erofs_off_t offset, erofs_off_t size);
+
+#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
+
#endif
diff --git a/include/erofs/io.h b/include/erofs/io.h
index a23de64541c6..557424578ece 100644
--- a/include/erofs/io.h
+++ b/include/erofs/io.h
@@ -17,6 +17,7 @@
#endif
int dev_open(const char *devname);
+int dev_open_ro(const char *dev);
void dev_close(void);
int dev_write(const void *buf, u64 offset, size_t len);
int dev_read(void *buf, u64 offset, size_t len);
diff --git a/lib/Makefile.am b/lib/Makefile.am
index e4b51e65f053..22bd73255d04 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -2,8 +2,8 @@
# Makefile.am
noinst_LTLIBRARIES = liberofs.la
-liberofs_la_SOURCES = config.c io.c cache.c inode.c xattr.c \
- compress.c compressor.c exclude.c
+liberofs_la_SOURCES = config.c io.c cache.c super.c inode.c xattr.c exclude.c \
+ data.c compress.c compressor.c
liberofs_la_CFLAGS = -Wall -Werror -I$(top_srcdir)/include
if ENABLE_LZ4
liberofs_la_CFLAGS += ${LZ4_CFLAGS}
diff --git a/lib/data.c b/lib/data.c
new file mode 100644
index 000000000000..56b208513980
--- /dev/null
+++ b/lib/data.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/lib/data.c
+ *
+ * Copyright (C) 2020 Gao Xiang <hsiangkao@aol.com>
+ */
+#include "erofs/print.h"
+#include "erofs/internal.h"
+#include "erofs/io.h"
+#include "erofs/trace.h"
+
+static int erofs_map_blocks_flatmode(struct erofs_inode *inode,
+ struct erofs_map_blocks *map,
+ int flags)
+{
+ int err = 0;
+ erofs_blk_t nblocks, lastblk;
+ u64 offset = map->m_la;
+ struct erofs_inode *vi = inode;
+ bool tailendpacking = (vi->datalayout == EROFS_INODE_FLAT_INLINE);
+
+ trace_erofs_map_blocks_flatmode_enter(inode, map, flags);
+
+ nblocks = DIV_ROUND_UP(inode->i_size, PAGE_SIZE);
+ lastblk = nblocks - tailendpacking;
+
+ if (offset >= inode->i_size) {
+ /* leave out-of-bound access unmapped */
+ map->m_flags = 0;
+ goto out;
+ }
+
+ /* there is no hole in flatmode */
+ map->m_flags = EROFS_MAP_MAPPED;
+
+ if (offset < blknr_to_addr(lastblk)) {
+ map->m_pa = blknr_to_addr(vi->u.i_blkaddr) + map->m_la;
+ map->m_plen = blknr_to_addr(lastblk) - offset;
+ } else if (tailendpacking) {
+ /* 2 - inode inline B: inode, [xattrs], inline last blk... */
+ map->m_pa = iloc(vi->nid) + vi->inode_isize +
+ vi->xattr_isize + erofs_blkoff(map->m_la);
+ map->m_plen = inode->i_size - offset;
+
+ /* inline data should be located in one meta block */
+ if (erofs_blkoff(map->m_pa) + map->m_plen > PAGE_SIZE) {
+ erofs_err("inline data cross block boundary @ nid %" PRIu64,
+ vi->nid);
+ DBG_BUGON(1);
+ err = -EFSCORRUPTED;
+ goto err_out;
+ }
+
+ map->m_flags |= EROFS_MAP_META;
+ } else {
+ erofs_err("internal error @ nid: %" PRIu64 " (size %llu), m_la 0x%" PRIx64,
+ vi->nid, (unsigned long long)inode->i_size, map->m_la);
+ DBG_BUGON(1);
+ err = -EIO;
+ goto err_out;
+ }
+
+out:
+ map->m_llen = map->m_plen;
+
+err_out:
+ trace_erofs_map_blocks_flatmode_exit(inode, map, flags, 0);
+ return err;
+}
+
+int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
+ erofs_off_t offset, erofs_off_t size)
+{
+ struct erofs_map_blocks map = {
+ .index = UINT_MAX,
+ };
+ int ret;
+ erofs_off_t ptr = offset;
+
+ while (ptr < offset + size) {
+ erofs_off_t eend;
+
+ map.m_la = ptr;
+ ret = erofs_map_blocks_flatmode(inode, &map, 0);
+ if (ret)
+ return ret;
+
+ DBG_BUGON(map.m_plen != map.m_llen);
+
+ if (!(map.m_flags & EROFS_MAP_MAPPED)) {
+ if (!map.m_llen) {
+ ptr = offset + size;
+ continue;
+ }
+ ptr = map.m_la + map.m_llen;
+ continue;
+ }
+
+ /* trim extent */
+ eend = min(offset + size, map.m_la + map.m_llen);
+ DBG_BUGON(ptr < map.m_la);
+
+ if (ptr > map.m_la) {
+ map.m_pa += ptr - map.m_la;
+ map.m_la = ptr;
+ }
+
+ ret = dev_read(buffer + ptr - offset,
+ map.m_pa, eend - map.m_la);
+ if (ret < 0)
+ return -EIO;
+
+ ptr = eend;
+ }
+ return 0;
+}
+
diff --git a/lib/io.c b/lib/io.c
index 4f5d9a6edaa4..d835f34da50f 100644
--- a/lib/io.c
+++ b/lib/io.c
@@ -108,6 +108,22 @@ int dev_open(const char *dev)
return 0;
}
+/* XXX: temporary soluation. Disk I/O implementation needs to be refactored. */
+int dev_open_ro(const char *dev)
+{
+ int fd = open(dev, O_RDONLY | O_BINARY);
+
+ if (fd < 0) {
+ erofs_err("failed to open(%s).", dev);
+ return -errno;
+ }
+
+ erofs_devfd = fd;
+ erofs_devname = dev;
+ erofs_devsz = INT64_MAX;
+ return 0;
+}
+
u64 dev_length(void)
{
return erofs_devsz;
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 02/12] erofs-utils: fuse: add special file support
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 01/12] " Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 03/12] erofs-utils: fuse: add compressed " Gao Xiang via Linux-erofs
` (7 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs; +Cc: Guo Weichao
From: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/main.c | 2 ++
fuse/namei.c | 14 ++++++++++----
fuse/read.c | 26 ++++++++++++++++++++++++++
fuse/read.h | 1 +
include/erofs/internal.h | 1 +
5 files changed, 40 insertions(+), 4 deletions(-)
diff --git a/fuse/main.c b/fuse/main.c
index 5121e8325f32..563b2c378952 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -146,6 +146,7 @@ int erofs_getattr(const char *path, struct stat *stbuf)
stbuf->st_blocks = stbuf->st_size / EROFS_BLKSIZ;
stbuf->st_uid = v.i_uid;
stbuf->st_gid = v.i_gid;
+ stbuf->st_rdev = v.i_rdev;
stbuf->st_atime = sbi.build_time;
stbuf->st_mtime = sbi.build_time;
stbuf->st_ctime = sbi.build_time;
@@ -153,6 +154,7 @@ int erofs_getattr(const char *path, struct stat *stbuf)
}
static struct fuse_operations erofs_ops = {
+ .readlink = erofs_readlink,
.getattr = erofs_getattr,
.readdir = erofs_readdir,
.open = erofs_open,
diff --git a/fuse/namei.c b/fuse/namei.c
index cd747ad1be56..e79e77d1e3c9 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -11,6 +11,7 @@
#include <stdio.h>
#include <errno.h>
#include <sys/stat.h>
+#include <sys/sysmacros.h>
#include "erofs/defs.h"
#include "erofs/print.h"
@@ -37,6 +38,13 @@ static uint8_t get_path_token_len(const char *path)
return len;
}
+static inline dev_t new_decode_dev(u32 dev)
+{
+ unsigned major = (dev & 0xfff00) >> 8;
+ unsigned minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);
+ return makedev(major, minor);
+}
+
int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
{
int ret;
@@ -63,13 +71,11 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
switch (vi->i_mode & S_IFMT) {
case S_IFBLK:
case S_IFCHR:
- /* fixme: add special devices support
- * vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
- */
+ vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
break;
case S_IFIFO:
case S_IFSOCK:
- /*fixme: vi->i_rdev = 0; */
+ vi->i_rdev = 0;
break;
case S_IFREG:
case S_IFLNK:
diff --git a/fuse/read.c b/fuse/read.c
index 446e0837cbc4..a55c0f2f78cd 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -70,3 +70,29 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
}
}
+int erofs_readlink(const char *path, char *buffer, size_t size)
+{
+ int ret;
+ erofs_nid_t nid;
+ size_t lnksz;
+ struct erofs_vnode v;
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ ret = erofs_iget_by_nid(nid, &v);
+ if (ret)
+ return ret;
+
+ lnksz = min((size_t)v.i_size, size - 1);
+
+ ret = erofs_read(path, buffer, lnksz, 0, NULL);
+ buffer[lnksz] = '\0';
+ if (ret != (int)lnksz)
+ return ret;
+
+ erofs_info("path:%s link=%s size=%llu", path, buffer, (unsigned long long)lnksz);
+ return 0;
+}
+
diff --git a/fuse/read.h b/fuse/read.h
index 3f4af1495510..e901c607dc91 100644
--- a/fuse/read.h
+++ b/fuse/read.h
@@ -12,5 +12,6 @@
int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
struct fuse_file_info *fi);
+int erofs_readlink(const char *path, char *buffer, size_t size);
#endif
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index f9e757316efe..77fa8d82c746 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -168,6 +168,7 @@ struct erofs_vnode {
uint16_t i_uid;
uint16_t i_gid;
uint16_t i_nlink;
+ uint32_t i_rdev;
/* if file is inline read inline data witch inode */
char *idata;
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 03/12] erofs-utils: fuse: add compressed file support
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 01/12] " Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 02/12] erofs-utils: fuse: add special file support Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 04/12] erofs-utils: fuse: clean up path walking Gao Xiang via Linux-erofs
` (6 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs; +Cc: Guo Weichao
From: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/Makefile.am | 4 +-
fuse/namei.c | 11 +-
fuse/read.c | 77 ++++++-
fuse/zmap.c | 417 +++++++++++++++++++++++++++++++++++++
include/erofs/decompress.h | 35 ++++
include/erofs/defs.h | 13 ++
include/erofs/internal.h | 22 +-
include/erofs_fs.h | 4 +
lib/Makefile.am | 2 +-
lib/decompress.c | 87 ++++++++
10 files changed, 666 insertions(+), 6 deletions(-)
create mode 100644 fuse/zmap.c
create mode 100644 include/erofs/decompress.h
create mode 100644 lib/decompress.c
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index 0c1a24763f59..84e5f834d6a4 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -3,12 +3,12 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
-erofsfuse_SOURCES = main.c namei.c read.c readir.c
+erofsfuse_SOURCES = main.c namei.c read.c readir.c zmap.c
erofsfuse_CFLAGS = -Wall -Werror \
-I$(top_srcdir)/include \
$(shell pkg-config fuse --cflags) \
-DFUSE_USE_VERSION=26 \
-std=gnu99
LDFLAGS += $(shell pkg-config fuse --libs)
-erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl
+erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl ${LZ4_LIBS}
diff --git a/fuse/namei.c b/fuse/namei.c
index e79e77d1e3c9..5ee3f8d2a4b6 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -47,7 +47,7 @@ static inline dev_t new_decode_dev(u32 dev)
int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
{
- int ret;
+ int ret, ifmt;
char buf[EROFS_BLKSIZ];
struct erofs_inode_compact *v1;
const erofs_off_t addr = iloc(nid);
@@ -58,6 +58,11 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
return -EIO;
v1 = (struct erofs_inode_compact *)buf;
+ /* fixme: support extended inode */
+ ifmt = le16_to_cpu(v1->i_format);
+ if (__inode_version(ifmt) != EROFS_INODE_LAYOUT_COMPACT)
+ return -EOPNOTSUPP;
+
vi->datalayout = __inode_data_mapping(le16_to_cpu(v1->i_format));
vi->inode_isize = sizeof(struct erofs_inode_compact);
vi->xattr_isize = erofs_xattr_ibody_size(v1->i_xattr_icount);
@@ -86,6 +91,10 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
return -EIO;
}
+ vi->z_inited = false;
+ if (erofs_inode_is_data_compressed(vi->datalayout))
+ z_erofs_fill_inode(vi);
+
return 0;
}
diff --git a/fuse/read.c b/fuse/read.c
index a55c0f2f78cd..10a26d84c37c 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -3,6 +3,7 @@
* erofs-utils/fuse/read.c
*
* Created by Li Guifu <blucerlee@gmail.com>
+ * Compression support by Huang Jianan <huangjianan@oppo.com>
*/
#include "read.h"
#include <errno.h>
@@ -15,6 +16,7 @@
#include "erofs/print.h"
#include "namei.h"
#include "erofs/io.h"
+#include "erofs/decompress.h"
size_t erofs_read_data_wrapper(struct erofs_vnode *vnode, char *buffer,
size_t size, off_t offset)
@@ -38,6 +40,78 @@ size_t erofs_read_data_wrapper(struct erofs_vnode *vnode, char *buffer,
return size;
}
+size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
+ erofs_off_t size, erofs_off_t offset)
+{
+ int ret;
+ erofs_off_t end, length, skip;
+ struct erofs_map_blocks map = {
+ .index = UINT_MAX,
+ };
+ bool partial;
+ unsigned int algorithmformat;
+ char raw[EROFS_BLKSIZ];
+
+ end = offset + size;
+ while (end > offset) {
+ map.m_la = end - 1;
+
+ ret = z_erofs_map_blocks_iter(vnode, &map);
+ if (ret)
+ return ret;
+
+ if (!(map.m_flags & EROFS_MAP_MAPPED)) {
+ end = map.m_la;
+ continue;
+ }
+
+ ret = dev_read(raw, map.m_pa, EROFS_BLKSIZ);
+ if (ret < 0)
+ return -EIO;
+
+ algorithmformat = map.m_flags & EROFS_MAP_ZIPPED ?
+ Z_EROFS_COMPRESSION_LZ4 :
+ Z_EROFS_COMPRESSION_SHIFTED;
+
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
+ length = end - map.m_la;
+ partial = true;
+ } else {
+ DBG_BUGON(end != map.m_la + map.m_llen);
+ length = map.m_llen;
+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
+ }
+
+ if (map.m_la < offset) {
+ skip = offset - map.m_la;
+ end = offset;
+ } else {
+ skip = 0;
+ end = map.m_la;
+ }
+
+ ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
+ .in = raw,
+ .out = buffer + end - offset,
+ .decodedskip = skip,
+ .inputsize = map.m_plen,
+ .decodedlength = length,
+ .alg = algorithmformat,
+ .partial_decoding = partial
+ });
+ if (ret < 0)
+ return ret;
+ }
+
+ erofs_info("nid:%llu size=%zd offset=%llu done",
+ (unsigned long long)vnode->nid, size, (long long)offset);
+ return size;
+}
+
int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
struct fuse_file_info *fi)
{
@@ -64,7 +138,8 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
case EROFS_INODE_FLAT_COMPRESSION:
- /* Fixme: */
+ return erofs_read_data_compression(&v, buffer, size, offset);
+
default:
return -EINVAL;
}
diff --git a/fuse/zmap.c b/fuse/zmap.c
new file mode 100644
index 000000000000..ba5c457278a8
--- /dev/null
+++ b/fuse/zmap.c
@@ -0,0 +1,417 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/fuse/zmap.c
+ *
+ * (a large amount of code was adapted from Linux kernel. )
+ *
+ * Copyright (C) 2018-2019 HUAWEI, Inc.
+ * https://www.huawei.com/
+ * Created by Gao Xiang <gaoxiang25@huawei.com>
+ * Modified by Huang Jianan <huangjianan@oppo.com>
+ */
+#include "erofs/io.h"
+#include "erofs/print.h"
+
+int z_erofs_fill_inode(struct erofs_vnode *vi)
+{
+ if (vi->datalayout == EROFS_INODE_FLAT_COMPRESSION_LEGACY) {
+ vi->z_advise = 0;
+ vi->z_algorithmtype[0] = 0;
+ vi->z_algorithmtype[1] = 0;
+ vi->z_logical_clusterbits = LOG_BLOCK_SIZE;
+ vi->z_physical_clusterbits[0] = vi->z_logical_clusterbits;
+ vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits;
+ vi->z_inited = true;
+ }
+
+ return 0;
+}
+
+static int z_erofs_fill_inode_lazy(struct erofs_vnode *vi)
+{
+ int ret;
+ erofs_off_t pos;
+ struct z_erofs_map_header *h;
+ char buf[8];
+
+ if (vi->z_inited)
+ return 0;
+
+ DBG_BUGON(vi->datalayout == EROFS_INODE_FLAT_COMPRESSION_LEGACY);
+
+ pos = round_up(iloc(vi->nid) + vi->inode_isize + vi->xattr_isize, 8);
+
+ ret = dev_read(buf, pos, 8);
+ if (ret < 0)
+ return -EIO;
+
+ h = (struct z_erofs_map_header *)buf;
+ vi->z_advise = le16_to_cpu(h->h_advise);
+ vi->z_algorithmtype[0] = h->h_algorithmtype & 15;
+ vi->z_algorithmtype[1] = h->h_algorithmtype >> 4;
+
+ if (vi->z_algorithmtype[0] >= Z_EROFS_COMPRESSION_MAX) {
+ erofs_err("unknown compression format %u for nid %llu",
+ vi->z_algorithmtype[0], (unsigned long long)vi->nid);
+ return -EOPNOTSUPP;
+ }
+
+ vi->z_logical_clusterbits = LOG_BLOCK_SIZE + (h->h_clusterbits & 7);
+ vi->z_physical_clusterbits[0] = vi->z_logical_clusterbits +
+ ((h->h_clusterbits >> 3) & 3);
+
+ if (vi->z_physical_clusterbits[0] != LOG_BLOCK_SIZE) {
+ erofs_err("unsupported physical clusterbits %u for nid %llu",
+ vi->z_physical_clusterbits[0], (unsigned long long)vi->nid);
+ return -EOPNOTSUPP;
+ }
+
+ vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits +
+ ((h->h_clusterbits >> 5) & 7);
+ vi->z_inited = true;
+
+ return 0;
+}
+
+struct z_erofs_maprecorder {
+ struct erofs_vnode *vnode;
+ struct erofs_map_blocks *map;
+ void *kaddr;
+
+ unsigned long lcn;
+ /* compression extent information gathered */
+ u8 type;
+ u16 clusterofs;
+ u16 delta[2];
+ erofs_blk_t pblk;
+};
+
+static int z_erofs_reload_indexes(struct z_erofs_maprecorder *m,
+ erofs_blk_t eblk)
+{
+ int ret;
+ struct erofs_map_blocks *const map = m->map;
+ char *mpage = map->mpage;
+
+ if (map->index == eblk)
+ return 0;
+
+ ret = blk_read(mpage, eblk, 1);
+ if (ret < 0)
+ return -EIO;
+
+ map->index = eblk;
+
+ return 0;
+}
+
+static int legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m,
+ unsigned long lcn)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ const erofs_off_t ibase = iloc(vi->nid);
+ const erofs_off_t pos =
+ Z_EROFS_VLE_LEGACY_INDEX_ALIGN(ibase + vi->inode_isize +
+ vi->xattr_isize) +
+ lcn * sizeof(struct z_erofs_vle_decompressed_index);
+ struct z_erofs_vle_decompressed_index *di;
+ unsigned int advise, type;
+ int err;
+
+ err = z_erofs_reload_indexes(m, erofs_blknr(pos));
+ if (err)
+ return err;
+
+ m->lcn = lcn;
+ di = m->kaddr + erofs_blkoff(pos);
+
+ advise = le16_to_cpu(di->di_advise);
+ type = (advise >> Z_EROFS_VLE_DI_CLUSTER_TYPE_BIT) &
+ ((1 << Z_EROFS_VLE_DI_CLUSTER_TYPE_BITS) - 1);
+ switch (type) {
+ case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD:
+ m->clusterofs = 1 << vi->z_logical_clusterbits;
+ m->delta[0] = le16_to_cpu(di->di_u.delta[0]);
+ m->delta[1] = le16_to_cpu(di->di_u.delta[1]);
+ break;
+ case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
+ m->clusterofs = le16_to_cpu(di->di_clusterofs);
+ m->pblk = le32_to_cpu(di->di_u.blkaddr);
+ break;
+ default:
+ DBG_BUGON(1);
+ return -EOPNOTSUPP;
+ }
+ m->type = type;
+ return 0;
+}
+
+static unsigned int decode_compactedbits(unsigned int lobits,
+ unsigned int lomask,
+ u8 *in, unsigned int pos, u8 *type)
+{
+ const unsigned int v = get_unaligned_le32(in + pos / 8) >> (pos & 7);
+ const unsigned int lo = v & lomask;
+
+ *type = (v >> lobits) & 3;
+ return lo;
+}
+
+static int unpack_compacted_index(struct z_erofs_maprecorder *m,
+ unsigned int amortizedshift,
+ unsigned int eofs)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ const unsigned int lomask = (1 << lclusterbits) - 1;
+ unsigned int vcnt, base, lo, encodebits, nblk;
+ int i;
+ u8 *in, type;
+
+ if (1 << amortizedshift == 4)
+ vcnt = 2;
+ else if (1 << amortizedshift == 2 && lclusterbits == 12)
+ vcnt = 16;
+ else
+ return -EOPNOTSUPP;
+
+ encodebits = ((vcnt << amortizedshift) - sizeof(__le32)) * 8 / vcnt;
+ base = round_down(eofs, vcnt << amortizedshift);
+ in = m->kaddr + base;
+
+ i = (eofs - base) >> amortizedshift;
+
+ lo = decode_compactedbits(lclusterbits, lomask,
+ in, encodebits * i, &type);
+ m->type = type;
+ if (type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) {
+ m->clusterofs = 1 << lclusterbits;
+ if (i + 1 != (int)vcnt) {
+ m->delta[0] = lo;
+ return 0;
+ }
+ /*
+ * since the last lcluster in the pack is special,
+ * of which lo saves delta[1] rather than delta[0].
+ * Hence, get delta[0] by the previous lcluster indirectly.
+ */
+ lo = decode_compactedbits(lclusterbits, lomask,
+ in, encodebits * (i - 1), &type);
+ if (type != Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD)
+ lo = 0;
+ m->delta[0] = lo + 1;
+ return 0;
+ }
+ m->clusterofs = lo;
+ m->delta[0] = 0;
+ /* figout out blkaddr (pblk) for HEAD lclusters */
+ nblk = 1;
+ while (i > 0) {
+ --i;
+ lo = decode_compactedbits(lclusterbits, lomask,
+ in, encodebits * i, &type);
+ if (type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD)
+ i -= lo;
+
+ if (i >= 0)
+ ++nblk;
+ }
+ in += (vcnt << amortizedshift) - sizeof(__le32);
+ m->pblk = le32_to_cpu(*(__le32 *)in) + nblk;
+ return 0;
+}
+
+static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
+ unsigned long lcn)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ const erofs_off_t ebase = round_up(iloc(vi->nid) + vi->inode_isize +
+ vi->xattr_isize, 8) +
+ sizeof(struct z_erofs_map_header);
+ const unsigned int totalidx = DIV_ROUND_UP(vi->i_size, EROFS_BLKSIZ);
+ unsigned int compacted_4b_initial, compacted_2b;
+ unsigned int amortizedshift;
+ erofs_off_t pos;
+ int err;
+
+ if (lclusterbits != 12)
+ return -EOPNOTSUPP;
+
+ if (lcn >= totalidx)
+ return -EINVAL;
+
+ m->lcn = lcn;
+ /* used to align to 32-byte (compacted_2b) alignment */
+ compacted_4b_initial = (32 - ebase % 32) / 4;
+ if (compacted_4b_initial == 32 / 4)
+ compacted_4b_initial = 0;
+
+ if (vi->z_advise & Z_EROFS_ADVISE_COMPACTED_2B)
+ compacted_2b = rounddown(totalidx - compacted_4b_initial, 16);
+ else
+ compacted_2b = 0;
+
+ pos = ebase;
+ if (lcn < compacted_4b_initial) {
+ amortizedshift = 2;
+ goto out;
+ }
+ pos += compacted_4b_initial * 4;
+ lcn -= compacted_4b_initial;
+
+ if (lcn < compacted_2b) {
+ amortizedshift = 1;
+ goto out;
+ }
+ pos += compacted_2b * 2;
+ lcn -= compacted_2b;
+ amortizedshift = 2;
+out:
+ pos += lcn * (1 << amortizedshift);
+ err = z_erofs_reload_indexes(m, erofs_blknr(pos));
+ if (err)
+ return err;
+ return unpack_compacted_index(m, amortizedshift, erofs_blkoff(pos));
+}
+
+static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
+ unsigned int lcn)
+{
+ const unsigned int datamode = m->vnode->datalayout;
+
+ if (datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY)
+ return legacy_load_cluster_from_disk(m, lcn);
+
+ if (datamode == EROFS_INODE_FLAT_COMPRESSION)
+ return compacted_load_cluster_from_disk(m, lcn);
+
+ return -EINVAL;
+}
+
+static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
+ unsigned int lookback_distance)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ struct erofs_map_blocks *const map = m->map;
+ const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ unsigned long lcn = m->lcn;
+ int err;
+
+ if (lcn < lookback_distance) {
+ erofs_err("bogus lookback distance @ nid %llu",
+ (unsigned long long)vi->nid);
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
+ }
+
+ /* load extent head logical cluster if needed */
+ lcn -= lookback_distance;
+ err = z_erofs_load_cluster_from_disk(m, lcn);
+ if (err)
+ return err;
+
+ switch (m->type) {
+ case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD:
+ if (!m->delta[0]) {
+ erofs_err("invalid lookback distance 0 @ nid %llu",
+ (unsigned long long)vi->nid);
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
+ }
+ return z_erofs_extent_lookback(m, m->delta[0]);
+ case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
+ map->m_flags &= ~EROFS_MAP_ZIPPED;
+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
+ map->m_la = (lcn << lclusterbits) | m->clusterofs;
+ break;
+ default:
+ erofs_err("unknown type %u @ lcn %lu of nid %llu",
+ m->type, lcn, (unsigned long long)vi->nid);
+ DBG_BUGON(1);
+ return -EOPNOTSUPP;
+ }
+ return 0;
+}
+
+int z_erofs_map_blocks_iter(struct erofs_vnode *vi,
+ struct erofs_map_blocks *map)
+{
+ struct z_erofs_maprecorder m = {
+ .vnode = vi,
+ .map = map,
+ .kaddr = map->mpage,
+ };
+ int err = 0;
+ unsigned int lclusterbits, endoff;
+ unsigned long long ofs, end;
+
+ /* when trying to read beyond EOF, leave it unmapped */
+ if (map->m_la >= vi->i_size) {
+ map->m_llen = map->m_la + 1 - vi->i_size;
+ map->m_la = vi->i_size;
+ map->m_flags = 0;
+ goto out;
+ }
+
+ err = z_erofs_fill_inode_lazy(vi);
+ if (err)
+ goto out;
+
+ lclusterbits = vi->z_logical_clusterbits;
+ ofs = map->m_la;
+ m.lcn = ofs >> lclusterbits;
+ endoff = ofs & ((1 << lclusterbits) - 1);
+
+ err = z_erofs_load_cluster_from_disk(&m, m.lcn);
+ if (err)
+ goto out;
+
+ map->m_flags = EROFS_MAP_ZIPPED; /* by default, compressed */
+ end = (m.lcn + 1ULL) << lclusterbits;
+ switch (m.type) {
+ case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
+ if (endoff >= m.clusterofs)
+ map->m_flags &= ~EROFS_MAP_ZIPPED;
+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
+ if (endoff >= m.clusterofs) {
+ map->m_la = (m.lcn << lclusterbits) | m.clusterofs;
+ break;
+ }
+ /* m.lcn should be >= 1 if endoff < m.clusterofs */
+ if (!m.lcn) {
+ erofs_err("invalid logical cluster 0 at nid %llu",
+ (unsigned long long)vi->nid);
+ err = -EFSCORRUPTED;
+ goto out;
+ }
+ end = (m.lcn << lclusterbits) | m.clusterofs;
+ map->m_flags |= EROFS_MAP_FULL_MAPPED;
+ m.delta[0] = 1;
+ case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD:
+ /* get the correspoinding first chunk */
+ err = z_erofs_extent_lookback(&m, m.delta[0]);
+ if (err)
+ goto out;
+ break;
+ default:
+ erofs_err("unknown type %u @ offset %llu of nid %llu",
+ m.type, ofs, (unsigned long long)vi->nid);
+ err = -EOPNOTSUPP;
+ goto out;
+ }
+
+ map->m_llen = end - map->m_la;
+ map->m_plen = 1 << lclusterbits;
+ map->m_pa = blknr_to_addr(m.pblk);
+ map->m_flags |= EROFS_MAP_MAPPED;
+
+out:
+ erofs_dbg("m_la %" PRIu64 " m_pa %" PRIu64 " m_llen %" PRIu64 " m_plen %" PRIu64 " m_flags 0%o",
+ map->m_la, map->m_pa,
+ map->m_llen, map->m_plen, map->m_flags);
+
+ DBG_BUGON(err < 0 && err != -ENOMEM);
+ return err;
+}
diff --git a/include/erofs/decompress.h b/include/erofs/decompress.h
new file mode 100644
index 000000000000..beaac359b21f
--- /dev/null
+++ b/include/erofs/decompress.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-utils/include/erofs/decompress.h
+ *
+ * Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd.
+ * Created by Huang Jianan <huangjianan@oppo.com>
+ */
+#ifndef __EROFS_DECOMPRESS_H
+#define __EROFS_DECOMPRESS_H
+
+#include "internal.h"
+
+enum {
+ Z_EROFS_COMPRESSION_SHIFTED = Z_EROFS_COMPRESSION_MAX,
+ Z_EROFS_COMPRESSION_RUNTIME_MAX
+};
+
+struct z_erofs_decompress_req {
+ char *in, *out;
+
+ /*
+ * initial decompressed bytes that need to be skipped
+ * when finally copying to output buffer
+ */
+ unsigned int decodedskip;
+ unsigned int inputsize, decodedlength;
+
+ /* indicate the algorithm will be used for decompression */
+ unsigned int alg;
+ bool partial_decoding;
+};
+
+int z_erofs_decompress(struct z_erofs_decompress_req *rq);
+
+#endif
diff --git a/include/erofs/defs.h b/include/erofs/defs.h
index e97de36aa04b..c42ca401a8ee 100644
--- a/include/erofs/defs.h
+++ b/include/erofs/defs.h
@@ -173,5 +173,18 @@ typedef int64_t s64;
#define __maybe_unused __attribute__((__unused__))
#endif
+struct __una_u32 { u32 x; } __packed;
+
+static inline u32 __get_unaligned_cpu32(const void *p)
+{
+ const struct __una_u32 *ptr = (const struct __una_u32 *)p;
+ return ptr->x;
+}
+
+static inline u32 get_unaligned_le32(const void *p)
+{
+ return __get_unaligned_cpu32((const u8 *)p);
+}
+
#endif
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index 77fa8d82c746..fc28d82490a1 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -36,6 +36,8 @@ typedef unsigned short umode_t;
#error incompatible PAGE_SIZE is already defined
#endif
+#define PAGE_MASK (~(PAGE_SIZE-1))
+
#define LOG_BLOCK_SIZE (12)
#define EROFS_BLKSIZ (1U << LOG_BLOCK_SIZE)
@@ -160,7 +162,15 @@ struct erofs_vnode {
uint16_t xattr_shared_count;
char *xattr_shared_xattrs;
- erofs_blk_t raw_blkaddr;
+ union {
+ erofs_blk_t raw_blkaddr;
+ struct {
+ uint16_t z_advise;
+ uint8_t z_algorithmtype[2];
+ uint8_t z_logical_clusterbits;
+ uint8_t z_physical_clusterbits[2];
+ };
+ };
erofs_nid_t nid;
uint32_t i_ino;
@@ -170,6 +180,7 @@ struct erofs_vnode {
uint16_t i_nlink;
uint32_t i_rdev;
+ bool z_inited;
/* if file is inline read inline data witch inode */
char *idata;
};
@@ -233,6 +244,10 @@ enum {
#define EROFS_MAP_MAPPED (1 << BH_Mapped)
/* Located in metadata (could be copied from bd_inode) */
#define EROFS_MAP_META (1 << BH_Meta)
+/* The extent has been compressed */
+#define EROFS_MAP_ZIPPED (1 << BH_Zipped)
+/* The length of extent is full */
+#define EROFS_MAP_FULL_MAPPED (1 << BH_FullMapped)
struct erofs_map_blocks {
char mpage[EROFS_BLKSIZ];
@@ -251,6 +266,11 @@ int erofs_read_superblock(void);
int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
erofs_off_t offset, erofs_off_t size);
+/* zmap.c */
+int z_erofs_fill_inode(struct erofs_vnode *vi);
+int z_erofs_map_blocks_iter(struct erofs_vnode *vi,
+ struct erofs_map_blocks *map);
+
#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
#endif
diff --git a/include/erofs_fs.h b/include/erofs_fs.h
index 4cd79f01d820..a69f179a51a5 100644
--- a/include/erofs_fs.h
+++ b/include/erofs_fs.h
@@ -279,6 +279,10 @@ struct z_erofs_vle_decompressed_index {
} di_u;
};
+#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN(size) \
+ (round_up(size, sizeof(struct z_erofs_vle_decompressed_index)) + \
+ sizeof(struct z_erofs_map_header) + Z_EROFS_VLE_LEGACY_HEADER_PADDING)
+
#define Z_EROFS_VLE_EXTENT_ALIGN(size) round_up(size, \
sizeof(struct z_erofs_vle_decompressed_index))
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 22bd73255d04..487c4944479d 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -3,7 +3,7 @@
noinst_LTLIBRARIES = liberofs.la
liberofs_la_SOURCES = config.c io.c cache.c super.c inode.c xattr.c exclude.c \
- data.c compress.c compressor.c
+ data.c compress.c compressor.c decompress.c
liberofs_la_CFLAGS = -Wall -Werror -I$(top_srcdir)/include
if ENABLE_LZ4
liberofs_la_CFLAGS += ${LZ4_CFLAGS}
diff --git a/lib/decompress.c b/lib/decompress.c
new file mode 100644
index 000000000000..870b85430dd1
--- /dev/null
+++ b/lib/decompress.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-utils/lib/decompress.c
+ *
+ * Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd.
+ * Created by Huang Jianan <huangjianan@oppo.com>
+ */
+#include <stdlib.h>
+#include <lz4.h>
+
+#include "erofs/decompress.h"
+#include "erofs/err.h"
+
+#ifdef LZ4_ENABLED
+static int z_erofs_decompress_lz4(struct z_erofs_decompress_req *rq)
+{
+ int ret = 0;
+ char *dest = rq->out;
+ char *src = rq->in;
+ char *buff = NULL;
+ bool support_0padding = false;
+ unsigned int inputmargin = 0;
+
+ if (erofs_sb_has_lz4_0padding()) {
+ support_0padding = true;
+
+ while (!src[inputmargin & ~PAGE_MASK])
+ if (!(++inputmargin & ~PAGE_MASK))
+ break;
+
+ if (inputmargin >= rq->inputsize)
+ return -EIO;
+ }
+
+ if (rq->decodedskip) {
+ buff = malloc(rq->decodedlength);
+ if (!buff)
+ return -ENOMEM;
+ dest = buff;
+ }
+
+ if (rq->partial_decoding || !support_0padding)
+ ret = LZ4_decompress_safe_partial(src + inputmargin, dest,
+ rq->inputsize - inputmargin,
+ rq->decodedlength, rq->decodedlength);
+ else
+ ret = LZ4_decompress_safe(src + inputmargin, dest,
+ rq->inputsize - inputmargin,
+ rq->decodedlength);
+
+ if (ret != (int)rq->decodedlength) {
+ ret = -EIO;
+ goto out;
+ }
+
+ if (rq->decodedskip)
+ memcpy(rq->out, dest + rq->decodedskip,
+ rq->decodedlength - rq->decodedskip);
+
+out:
+ if (buff)
+ free(buff);
+
+ return ret;
+}
+#endif
+
+int z_erofs_decompress(struct z_erofs_decompress_req *rq)
+{
+ if (rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
+ if (rq->inputsize != EROFS_BLKSIZ)
+ return -EFSCORRUPTED;
+
+ DBG_BUGON(rq->decodedlength > EROFS_BLKSIZ);
+ DBG_BUGON(rq->decodedlength < rq->decodedskip);
+
+ memcpy(rq->out, rq->in + rq->decodedskip,
+ rq->decodedlength - rq->decodedskip);
+ return 0;
+ }
+
+#ifdef LZ4_ENABLED
+ if (rq->alg == Z_EROFS_COMPRESSION_LZ4)
+ return z_erofs_decompress_lz4(rq);
+#endif
+ return -EOPNOTSUPP;
+}
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 04/12] erofs-utils: fuse: clean up path walking
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (2 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 03/12] erofs-utils: fuse: add compressed " Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 05/12] erofs: clean up compress data read Gao Xiang via Linux-erofs
` (5 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/namei.c | 82 +++++++++++++++++++++------------------------------
fuse/namei.h | 2 --
fuse/read.c | 16 ++--------
fuse/readir.c | 10 ++-----
4 files changed, 39 insertions(+), 71 deletions(-)
diff --git a/fuse/namei.c b/fuse/namei.c
index 5ee3f8d2a4b6..37cf549cd2a6 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -17,27 +17,6 @@
#include "erofs/print.h"
#include "erofs/io.h"
-#define IS_PATH_SEPARATOR(__c) ((__c) == '/')
-#define MINORBITS 20
-#define MINORMASK ((1U << MINORBITS) - 1)
-#define DT_UNKNOWN 0
-
-static const char *skip_trailing_backslash(const char *path)
-{
- while (IS_PATH_SEPARATOR(*path))
- path++;
- return path;
-}
-
-static uint8_t get_path_token_len(const char *path)
-{
- uint8_t len = 0;
-
- while (path[len] != '/' && path[len])
- len++;
- return len;
-}
-
static inline dev_t new_decode_dev(u32 dev)
{
unsigned major = (dev & 0xfff00) >> 8;
@@ -45,7 +24,7 @@ static inline dev_t new_decode_dev(u32 dev)
return makedev(major, minor);
}
-int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
+static int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
{
int ret, ifmt;
char buf[EROFS_BLKSIZ];
@@ -136,14 +115,20 @@ struct erofs_dirent *find_target_dirent(erofs_nid_t pnid,
return NULL;
}
-int erofs_namei(erofs_nid_t *nid,
+struct nameidata {
+ erofs_nid_t nid;
+ unsigned int ftype;
+};
+
+int erofs_namei(struct nameidata *nd,
const char *name, unsigned int len)
{
+ erofs_nid_t nid = nd->nid;
int ret;
char buf[EROFS_BLKSIZ];
struct erofs_vnode v;
- ret = erofs_iget_by_nid(*nid, &v);
+ ret = erofs_iget_by_nid(nid, &v);
if (ret)
return ret;
@@ -154,7 +139,7 @@ int erofs_namei(erofs_nid_t *nid,
.u = {
.i_blkaddr = v.raw_blkaddr,
},
- .nid = v.nid,
+ .nid = nid,
.i_size = v.i_size,
.datalayout = v.datalayout,
.inode_isize = v.inode_isize,
@@ -175,17 +160,17 @@ int erofs_namei(erofs_nid_t *nid,
if (nameoff < sizeof(struct erofs_dirent) ||
nameoff >= PAGE_SIZE) {
erofs_err("invalid de[0].nameoff %u @ nid %llu",
- nameoff, *nid | 0ULL);
+ nameoff, nid | 0ULL);
return -EFSCORRUPTED;
}
- de = find_target_dirent(*nid, buf, name, len,
+ de = find_target_dirent(nid, buf, name, len,
nameoff, maxsize);
if (IS_ERR(de))
return PTR_ERR(de);
if (de) {
- *nid = le64_to_cpu(de->nid);
+ nd->nid = le64_to_cpu(de->nid);
return 0;
}
offset += maxsize;
@@ -194,44 +179,43 @@ int erofs_namei(erofs_nid_t *nid,
return -ENOENT;
}
-extern struct dcache_entry root_entry;
-int walk_path(const char *_path, erofs_nid_t *out_nid)
+static int link_path_walk(const char *name, struct nameidata *nd)
{
- int ret;
- erofs_nid_t nid = sbi.root_nid;
- const char *path = _path;
+ nd->nid = sbi.root_nid;
+
+ while (*name == '/')
+ name++;
- for (;;) {
- uint8_t path_len;
+ /* At this point we know we have a real path component. */
+ while (*name != '\0') {
+ const char *p = name;
+ int ret;
- path = skip_trailing_backslash(path);
- path_len = get_path_token_len(path);
- if (path_len == 0)
- break;
+ do {
+ ++p;
+ } while (*p != '\0' && *p != '/');
- ret = erofs_namei(&nid, path, path_len);
+ DBG_BUGON(p <= name);
+ ret = erofs_namei(nd, name, p - name);
if (ret)
return ret;
- path += path_len;
+ name = p;
+ /* Skip until no more slashes. */
+ for (name = p; *name == '/'; ++name);
}
-
- erofs_dbg("find path = %s nid=%llu", _path, nid | 0ULL);
-
- *out_nid = nid;
return 0;
-
}
int erofs_iget_by_path(const char *path, struct erofs_vnode *v)
{
int ret;
- erofs_nid_t nid;
+ struct nameidata nd;
- ret = walk_path(path, &nid);
+ ret = link_path_walk(path, &nd);
if (ret)
return ret;
- return erofs_iget_by_nid(nid, v);
+ return erofs_iget_by_nid(nd.nid, v);
}
diff --git a/fuse/namei.h b/fuse/namei.h
index 2625ec58d434..bd5adfda2969 100644
--- a/fuse/namei.h
+++ b/fuse/namei.h
@@ -11,7 +11,5 @@
#include "erofs_fs.h"
int erofs_iget_by_path(const char *path, struct erofs_vnode *v);
-int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *v);
-int walk_path(const char *path, erofs_nid_t *out_nid);
#endif
diff --git a/fuse/read.c b/fuse/read.c
index 10a26d84c37c..aa5221a60d4e 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -116,21 +116,16 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
struct fuse_file_info *fi)
{
int ret;
- erofs_nid_t nid;
struct erofs_vnode v;
UNUSED(fi);
erofs_info("path:%s size=%zd offset=%llu", path, size, (long long)offset);
- ret = walk_path(path, &nid);
+ ret = erofs_iget_by_path(path, &v);
if (ret)
return ret;
- ret = erofs_iget_by_nid(nid, &v);
- if (ret)
- return ret;
-
- erofs_info("path:%s nid=%llu mode=%u", path, (unsigned long long)nid, v.datalayout);
+ erofs_info("path:%s nid=%llu mode=%u", path, v.nid | 0ULL, v.datalayout);
switch (v.datalayout) {
case EROFS_INODE_FLAT_PLAIN:
case EROFS_INODE_FLAT_INLINE:
@@ -148,15 +143,10 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
int erofs_readlink(const char *path, char *buffer, size_t size)
{
int ret;
- erofs_nid_t nid;
size_t lnksz;
struct erofs_vnode v;
- ret = walk_path(path, &nid);
- if (ret)
- return ret;
-
- ret = erofs_iget_by_nid(nid, &v);
+ ret = erofs_iget_by_path(path, &v);
if (ret)
return ret;
diff --git a/fuse/readir.c b/fuse/readir.c
index 5281c8b80e59..1d28016a8900 100644
--- a/fuse/readir.c
+++ b/fuse/readir.c
@@ -66,7 +66,6 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
off_t offset, struct fuse_file_info *fi)
{
int ret;
- erofs_nid_t nid;
struct erofs_vnode v;
char dirsbuf[EROFS_BLKSIZ];
uint32_t dir_nr, dir_off, nr_cnt;
@@ -74,14 +73,11 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
erofs_dbg("readdir:%s offset=%llu", path, (long long)offset);
UNUSED(fi);
- ret = walk_path(path, &nid);
+ ret = erofs_iget_by_path(path, &v);
if (ret)
return ret;
- erofs_dbg("path=%s nid = %llu", path, (unsigned long long)nid);
- ret = erofs_iget_by_nid(nid, &v);
- if (ret)
- return ret;
+ erofs_dbg("path=%s nid = %llu", path, v.nid | 0ULL);
if (!S_ISDIR(v.i_mode))
return -ENOTDIR;
@@ -107,7 +103,7 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
off_t addr;
- addr = iloc(nid) + v.inode_isize + v.xattr_isize;
+ addr = iloc(v.nid) + v.inode_isize + v.xattr_isize;
memset(dirsbuf, 0, sizeof(dirsbuf));
ret = dev_read(dirsbuf, addr, dir_off);
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 05/12] erofs: clean up compress data read
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (3 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 04/12] erofs-utils: fuse: clean up path walking Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 06/12] erofs-utils: fuse: get rid of erofs_vnode Gao Xiang via Linux-erofs
` (4 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/Makefile.am | 2 +-
fuse/namei.c | 17 ++++++--
fuse/read.c | 87 +++++++++++-----------------------------
include/erofs/internal.h | 22 ++++++++--
lib/Makefile.am | 2 +-
lib/data.c | 73 +++++++++++++++++++++++++++++++++
{fuse => lib}/zmap.c | 36 ++++++++---------
7 files changed, 147 insertions(+), 92 deletions(-)
rename {fuse => lib}/zmap.c (94%)
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index 84e5f834d6a4..d6e6d60cbfdc 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -3,7 +3,7 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
-erofsfuse_SOURCES = main.c namei.c read.c readir.c zmap.c
+erofsfuse_SOURCES = main.c namei.c read.c readir.c
erofsfuse_CFLAGS = -Wall -Werror \
-I$(top_srcdir)/include \
$(shell pkg-config fuse --cflags) \
diff --git a/fuse/namei.c b/fuse/namei.c
index 37cf549cd2a6..fd5ae7bfc410 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -71,9 +71,20 @@ static int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
}
vi->z_inited = false;
- if (erofs_inode_is_data_compressed(vi->datalayout))
- z_erofs_fill_inode(vi);
-
+ if (erofs_inode_is_data_compressed(vi->datalayout)) {
+ struct erofs_inode ei = { .datalayout = vi->datalayout };
+
+ z_erofs_fill_inode(&ei);
+
+ /* XXX: will be dropped after erofs_vnode is removed */
+ vi->z_advise = ei.z_advise;
+ vi->z_algorithmtype[0] = ei.z_algorithmtype[0];
+ vi->z_algorithmtype[1] = ei.z_algorithmtype[1];
+ vi->z_logical_clusterbits = ei.z_logical_clusterbits;
+ vi->z_physical_clusterbits[0] = ei.z_physical_clusterbits[0];
+ vi->z_physical_clusterbits[1] = ei.z_physical_clusterbits[1];
+ vi->z_inited = (ei.flags != 0);
+ }
return 0;
}
diff --git a/fuse/read.c b/fuse/read.c
index aa5221a60d4e..21fbd2eea662 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -43,72 +43,31 @@ size_t erofs_read_data_wrapper(struct erofs_vnode *vnode, char *buffer,
size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
erofs_off_t size, erofs_off_t offset)
{
- int ret;
- erofs_off_t end, length, skip;
- struct erofs_map_blocks map = {
- .index = UINT_MAX,
+ struct erofs_inode tmp = {
+ .nid = vnode->nid,
+ .i_size = vnode->i_size,
+ .datalayout = vnode->datalayout,
+ .inode_isize = vnode->inode_isize,
+ .xattr_isize = vnode->xattr_isize,
+ .z_advise = vnode->z_advise,
+ .z_algorithmtype = {
+ [0] = vnode->z_algorithmtype[0],
+ [1] = vnode->z_algorithmtype[1],
+ },
+ .z_logical_clusterbits = vnode->z_logical_clusterbits,
+ .z_physical_clusterbits = {
+ [0] = vnode->z_physical_clusterbits[0],
+ [1] = vnode->z_physical_clusterbits[1],
+ },
};
- bool partial;
- unsigned int algorithmformat;
- char raw[EROFS_BLKSIZ];
-
- end = offset + size;
- while (end > offset) {
- map.m_la = end - 1;
-
- ret = z_erofs_map_blocks_iter(vnode, &map);
- if (ret)
- return ret;
-
- if (!(map.m_flags & EROFS_MAP_MAPPED)) {
- end = map.m_la;
- continue;
- }
-
- ret = dev_read(raw, map.m_pa, EROFS_BLKSIZ);
- if (ret < 0)
- return -EIO;
-
- algorithmformat = map.m_flags & EROFS_MAP_ZIPPED ?
- Z_EROFS_COMPRESSION_LZ4 :
- Z_EROFS_COMPRESSION_SHIFTED;
-
- /*
- * trim to the needed size if the returned extent is quite
- * larger than requested, and set up partial flag as well.
- */
- if (end < map.m_la + map.m_llen) {
- length = end - map.m_la;
- partial = true;
- } else {
- DBG_BUGON(end != map.m_la + map.m_llen);
- length = map.m_llen;
- partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
- }
-
- if (map.m_la < offset) {
- skip = offset - map.m_la;
- end = offset;
- } else {
- skip = 0;
- end = map.m_la;
- }
-
- ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
- .in = raw,
- .out = buffer + end - offset,
- .decodedskip = skip,
- .inputsize = map.m_plen,
- .decodedlength = length,
- .alg = algorithmformat,
- .partial_decoding = partial
- });
- if (ret < 0)
- return ret;
- }
- erofs_info("nid:%llu size=%zd offset=%llu done",
- (unsigned long long)vnode->nid, size, (long long)offset);
+ if (vnode->z_inited)
+ tmp.flags |= EROFS_I_Z_INITED;
+
+ int ret = z_erofs_read_data(&tmp, buffer, offset, size);
+ if (ret)
+ return ret;
+
return size;
}
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index fc28d82490a1..98e1263fa19c 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -108,9 +108,13 @@ static inline void erofs_sb_clear_##name(void) \
EROFS_FEATURE_FUNCS(lz4_0padding, incompat, INCOMPAT_LZ4_0PADDING)
EROFS_FEATURE_FUNCS(sb_chksum, compat, COMPAT_SB_CHKSUM)
+#define EROFS_I_EA_INITED (1 << 0)
+#define EROFS_I_Z_INITED (1 << 1)
+
struct erofs_inode {
struct list_head i_hash, i_subdirs, i_xattrs;
+ unsigned int flags;
unsigned int i_count;
struct erofs_inode *i_parent;
@@ -145,7 +149,16 @@ struct erofs_inode {
struct erofs_buffer_head *bh_inline, *bh_data;
void *idata;
- void *compressmeta;
+
+ union {
+ void *compressmeta;
+ struct {
+ uint16_t z_advise;
+ uint8_t z_algorithmtype[2];
+ uint8_t z_logical_clusterbits;
+ uint8_t z_physical_clusterbits[2];
+ };
+ };
#ifdef WITH_ANDROID
uint64_t capabilities;
#endif
@@ -265,10 +278,11 @@ int erofs_read_superblock(void);
/* data.c */
int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
erofs_off_t offset, erofs_off_t size);
-
+int z_erofs_read_data(struct erofs_inode *inode, char *buffer,
+ erofs_off_t size, erofs_off_t offset);
/* zmap.c */
-int z_erofs_fill_inode(struct erofs_vnode *vi);
-int z_erofs_map_blocks_iter(struct erofs_vnode *vi,
+int z_erofs_fill_inode(struct erofs_inode *vi);
+int z_erofs_map_blocks_iter(struct erofs_inode *vi,
struct erofs_map_blocks *map);
#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 487c4944479d..7d9446b3cbcf 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -3,7 +3,7 @@
noinst_LTLIBRARIES = liberofs.la
liberofs_la_SOURCES = config.c io.c cache.c super.c inode.c xattr.c exclude.c \
- data.c compress.c compressor.c decompress.c
+ data.c compress.c compressor.c zmap.c decompress.c
liberofs_la_CFLAGS = -Wall -Werror -I$(top_srcdir)/include
if ENABLE_LZ4
liberofs_la_CFLAGS += ${LZ4_CFLAGS}
diff --git a/lib/data.c b/lib/data.c
index 56b208513980..62fd057185ee 100644
--- a/lib/data.c
+++ b/lib/data.c
@@ -3,11 +3,13 @@
* erofs-utils/lib/data.c
*
* Copyright (C) 2020 Gao Xiang <hsiangkao@aol.com>
+ * Compression support by Huang Jianan <huangjianan@oppo.com>
*/
#include "erofs/print.h"
#include "erofs/internal.h"
#include "erofs/io.h"
#include "erofs/trace.h"
+#include "erofs/decompress.h"
static int erofs_map_blocks_flatmode(struct erofs_inode *inode,
struct erofs_map_blocks *map,
@@ -115,3 +117,74 @@ int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
return 0;
}
+int z_erofs_read_data(struct erofs_inode *inode, char *buffer,
+ erofs_off_t offset, erofs_off_t size)
+{
+ int ret;
+ erofs_off_t end, length, skip;
+ struct erofs_map_blocks map = {
+ .index = UINT_MAX,
+ };
+ bool partial;
+ unsigned int algorithmformat;
+ char raw[EROFS_BLKSIZ];
+
+ end = offset + size;
+ while (end > offset) {
+ map.m_la = end - 1;
+
+ ret = z_erofs_map_blocks_iter(inode, &map);
+ if (ret)
+ return ret;
+
+ if (!(map.m_flags & EROFS_MAP_MAPPED)) {
+ end = map.m_la;
+ continue;
+ }
+
+ ret = dev_read(raw, map.m_pa, EROFS_BLKSIZ);
+ if (ret < 0)
+ return -EIO;
+
+ algorithmformat = map.m_flags & EROFS_MAP_ZIPPED ?
+ Z_EROFS_COMPRESSION_LZ4 :
+ Z_EROFS_COMPRESSION_SHIFTED;
+
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
+ length = end - map.m_la;
+ partial = true;
+ } else {
+ DBG_BUGON(end != map.m_la + map.m_llen);
+ length = map.m_llen;
+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
+ }
+
+ if (map.m_la < offset) {
+ skip = offset - map.m_la;
+ end = offset;
+ } else {
+ skip = 0;
+ end = map.m_la;
+ }
+
+ ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
+ .in = raw,
+ .out = buffer + end - offset,
+ .decodedskip = skip,
+ .inputsize = map.m_plen,
+ .decodedlength = length,
+ .alg = algorithmformat,
+ .partial_decoding = partial
+ });
+ if (ret < 0)
+ return ret;
+ }
+ erofs_dbg("nid:%llu size=%zd offset=%llu done",
+ inode->nid | 0ULL, size, (long long)offset);
+ return 0;
+}
+
diff --git a/fuse/zmap.c b/lib/zmap.c
similarity index 94%
rename from fuse/zmap.c
rename to lib/zmap.c
index ba5c457278a8..ee63de74cab2 100644
--- a/fuse/zmap.c
+++ b/lib/zmap.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-utils/fuse/zmap.c
+ * erofs-utils/lib/zmap.c
*
* (a large amount of code was adapted from Linux kernel. )
*
@@ -12,7 +12,7 @@
#include "erofs/io.h"
#include "erofs/print.h"
-int z_erofs_fill_inode(struct erofs_vnode *vi)
+int z_erofs_fill_inode(struct erofs_inode *vi)
{
if (vi->datalayout == EROFS_INODE_FLAT_COMPRESSION_LEGACY) {
vi->z_advise = 0;
@@ -21,27 +21,26 @@ int z_erofs_fill_inode(struct erofs_vnode *vi)
vi->z_logical_clusterbits = LOG_BLOCK_SIZE;
vi->z_physical_clusterbits[0] = vi->z_logical_clusterbits;
vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits;
- vi->z_inited = true;
- }
+ vi->flags |= EROFS_I_Z_INITED;
+ }
return 0;
}
-static int z_erofs_fill_inode_lazy(struct erofs_vnode *vi)
+static int z_erofs_fill_inode_lazy(struct erofs_inode *vi)
{
int ret;
erofs_off_t pos;
struct z_erofs_map_header *h;
- char buf[8];
+ char buf[sizeof(struct z_erofs_map_header)];
- if (vi->z_inited)
+ if (vi->flags & EROFS_I_Z_INITED)
return 0;
DBG_BUGON(vi->datalayout == EROFS_INODE_FLAT_COMPRESSION_LEGACY);
-
pos = round_up(iloc(vi->nid) + vi->inode_isize + vi->xattr_isize, 8);
- ret = dev_read(buf, pos, 8);
+ ret = dev_read(buf, pos, sizeof(buf));
if (ret < 0)
return -EIO;
@@ -68,13 +67,12 @@ static int z_erofs_fill_inode_lazy(struct erofs_vnode *vi)
vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits +
((h->h_clusterbits >> 5) & 7);
- vi->z_inited = true;
-
+ vi->flags |= EROFS_I_Z_INITED;
return 0;
}
struct z_erofs_maprecorder {
- struct erofs_vnode *vnode;
+ struct erofs_inode *inode;
struct erofs_map_blocks *map;
void *kaddr;
@@ -108,7 +106,7 @@ static int z_erofs_reload_indexes(struct z_erofs_maprecorder *m,
static int legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m,
unsigned long lcn)
{
- struct erofs_vnode *const vi = m->vnode;
+ struct erofs_inode *const vi = m->inode;
const erofs_off_t ibase = iloc(vi->nid);
const erofs_off_t pos =
Z_EROFS_VLE_LEGACY_INDEX_ALIGN(ibase + vi->inode_isize +
@@ -162,7 +160,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
unsigned int amortizedshift,
unsigned int eofs)
{
- struct erofs_vnode *const vi = m->vnode;
+ struct erofs_inode *const vi = m->inode;
const unsigned int lclusterbits = vi->z_logical_clusterbits;
const unsigned int lomask = (1 << lclusterbits) - 1;
unsigned int vcnt, base, lo, encodebits, nblk;
@@ -225,7 +223,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
unsigned long lcn)
{
- struct erofs_vnode *const vi = m->vnode;
+ struct erofs_inode *const vi = m->inode;
const unsigned int lclusterbits = vi->z_logical_clusterbits;
const erofs_off_t ebase = round_up(iloc(vi->nid) + vi->inode_isize +
vi->xattr_isize, 8) +
@@ -279,7 +277,7 @@ out:
static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
unsigned int lcn)
{
- const unsigned int datamode = m->vnode->datalayout;
+ const unsigned int datamode = m->inode->datalayout;
if (datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY)
return legacy_load_cluster_from_disk(m, lcn);
@@ -293,7 +291,7 @@ static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
unsigned int lookback_distance)
{
- struct erofs_vnode *const vi = m->vnode;
+ struct erofs_inode *const vi = m->inode;
struct erofs_map_blocks *const map = m->map;
const unsigned int lclusterbits = vi->z_logical_clusterbits;
unsigned long lcn = m->lcn;
@@ -335,11 +333,11 @@ static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
return 0;
}
-int z_erofs_map_blocks_iter(struct erofs_vnode *vi,
+int z_erofs_map_blocks_iter(struct erofs_inode *vi,
struct erofs_map_blocks *map)
{
struct z_erofs_maprecorder m = {
- .vnode = vi,
+ .inode = vi,
.map = map,
.kaddr = map->mpage,
};
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 06/12] erofs-utils: fuse: get rid of erofs_vnode
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (4 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 05/12] erofs: clean up compress data read Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 07/12] erofs-utils: fuse: move namei.c to lib/ Gao Xiang via Linux-erofs
` (3 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/main.c | 7 ++-
fuse/namei.c | 110 +++++++++++++++------------------------
fuse/namei.h | 3 +-
fuse/read.c | 86 +++++-------------------------
fuse/readir.c | 9 ++--
include/erofs/internal.h | 34 ------------
6 files changed, 63 insertions(+), 186 deletions(-)
diff --git a/fuse/main.c b/fuse/main.c
index 563b2c378952..6176e836c2f1 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -131,12 +131,11 @@ int erofs_open(const char *path, struct fuse_file_info *fi)
int erofs_getattr(const char *path, struct stat *stbuf)
{
- struct erofs_vnode v;
+ struct erofs_inode v = { 0 };
int ret;
erofs_dbg("getattr(%s)", path);
- memset(&v, 0, sizeof(v));
- ret = erofs_iget_by_path(path, &v);
+ ret = erofs_ilookup(path, &v);
if (ret)
return -ENOENT;
@@ -146,7 +145,7 @@ int erofs_getattr(const char *path, struct stat *stbuf)
stbuf->st_blocks = stbuf->st_size / EROFS_BLKSIZ;
stbuf->st_uid = v.i_uid;
stbuf->st_gid = v.i_gid;
- stbuf->st_rdev = v.i_rdev;
+ stbuf->st_rdev = v.u.i_rdev;
stbuf->st_atime = sbi.build_time;
stbuf->st_mtime = sbi.build_time;
stbuf->st_ctime = sbi.build_time;
diff --git a/fuse/namei.c b/fuse/namei.c
index fd5ae7bfc410..326ea85809bb 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -13,7 +13,6 @@
#include <sys/stat.h>
#include <sys/sysmacros.h>
-#include "erofs/defs.h"
#include "erofs/print.h"
#include "erofs/io.h"
@@ -24,12 +23,12 @@ static inline dev_t new_decode_dev(u32 dev)
return makedev(major, minor);
}
-static int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
+static int erofs_read_inode_from_disk(struct erofs_inode *vi)
{
int ret, ifmt;
char buf[EROFS_BLKSIZ];
struct erofs_inode_compact *v1;
- const erofs_off_t addr = iloc(nid);
+ const erofs_off_t addr = iloc(vi->nid);
const size_t size = EROFS_BLKSIZ - erofs_blkoff(addr);
ret = dev_read(buf, addr, size);
@@ -50,41 +49,28 @@ static int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
vi->i_uid = le16_to_cpu(v1->i_uid);
vi->i_gid = le16_to_cpu(v1->i_gid);
vi->i_nlink = le16_to_cpu(v1->i_nlink);
- vi->nid = nid;
switch (vi->i_mode & S_IFMT) {
case S_IFBLK:
case S_IFCHR:
- vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
+ vi->u.i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
break;
case S_IFIFO:
case S_IFSOCK:
- vi->i_rdev = 0;
+ vi->u.i_rdev = 0;
break;
case S_IFREG:
case S_IFLNK:
case S_IFDIR:
- vi->raw_blkaddr = le32_to_cpu(v1->i_u.raw_blkaddr);
+ vi->u.i_blkaddr = le32_to_cpu(v1->i_u.raw_blkaddr);
break;
default:
return -EIO;
}
- vi->z_inited = false;
- if (erofs_inode_is_data_compressed(vi->datalayout)) {
- struct erofs_inode ei = { .datalayout = vi->datalayout };
-
- z_erofs_fill_inode(&ei);
-
- /* XXX: will be dropped after erofs_vnode is removed */
- vi->z_advise = ei.z_advise;
- vi->z_algorithmtype[0] = ei.z_algorithmtype[0];
- vi->z_algorithmtype[1] = ei.z_algorithmtype[1];
- vi->z_logical_clusterbits = ei.z_logical_clusterbits;
- vi->z_physical_clusterbits[0] = ei.z_physical_clusterbits[0];
- vi->z_physical_clusterbits[1] = ei.z_physical_clusterbits[1];
- vi->z_inited = (ei.flags != 0);
- }
+ vi->flags = 0;
+ if (erofs_inode_is_data_compressed(vi->datalayout))
+ z_erofs_fill_inode(vi);
return 0;
}
@@ -137,55 +123,42 @@ int erofs_namei(struct nameidata *nd,
erofs_nid_t nid = nd->nid;
int ret;
char buf[EROFS_BLKSIZ];
- struct erofs_vnode v;
+ struct erofs_inode vi = { .nid = nid };
+ erofs_off_t offset;
- ret = erofs_iget_by_nid(nid, &v);
+ ret = erofs_read_inode_from_disk(&vi);
if (ret)
return ret;
- {
- unsigned int offset = 0;
-
- struct erofs_inode tmp = {
- .u = {
- .i_blkaddr = v.raw_blkaddr,
- },
- .nid = nid,
- .i_size = v.i_size,
- .datalayout = v.datalayout,
- .inode_isize = v.inode_isize,
- .xattr_isize = v.xattr_isize,
- };
-
- while (offset < v.i_size) {
- int maxsize = min(v.i_size - offset, EROFS_BLKSIZ);
- struct erofs_dirent *de = (void *)buf;
- unsigned int nameoff;
-
- ret = erofs_read_raw_data(&tmp, buf, offset, maxsize);
- if (ret)
- return ret;
-
- nameoff = le16_to_cpu(de->nameoff);
-
- if (nameoff < sizeof(struct erofs_dirent) ||
- nameoff >= PAGE_SIZE) {
- erofs_err("invalid de[0].nameoff %u @ nid %llu",
- nameoff, nid | 0ULL);
- return -EFSCORRUPTED;
- }
-
- de = find_target_dirent(nid, buf, name, len,
- nameoff, maxsize);
- if (IS_ERR(de))
- return PTR_ERR(de);
-
- if (de) {
- nd->nid = le64_to_cpu(de->nid);
- return 0;
- }
- offset += maxsize;
+ offset = 0;
+ while (offset < vi.i_size) {
+ erofs_off_t maxsize = min_t(erofs_off_t,
+ vi.i_size - offset, EROFS_BLKSIZ);
+ struct erofs_dirent *de = (void *)buf;
+ unsigned int nameoff;
+
+ ret = erofs_read_raw_data(&vi, buf, offset, maxsize);
+ if (ret)
+ return ret;
+
+ nameoff = le16_to_cpu(de->nameoff);
+ if (nameoff < sizeof(struct erofs_dirent) ||
+ nameoff >= PAGE_SIZE) {
+ erofs_err("invalid de[0].nameoff %u @ nid %llu",
+ nameoff, nid | 0ULL);
+ return -EFSCORRUPTED;
+ }
+
+ de = find_target_dirent(nid, buf, name, len,
+ nameoff, maxsize);
+ if (IS_ERR(de))
+ return PTR_ERR(de);
+
+ if (de) {
+ nd->nid = le64_to_cpu(de->nid);
+ return 0;
}
+ offset += maxsize;
}
return -ENOENT;
}
@@ -218,7 +191,7 @@ static int link_path_walk(const char *name, struct nameidata *nd)
return 0;
}
-int erofs_iget_by_path(const char *path, struct erofs_vnode *v)
+int erofs_ilookup(const char *path, struct erofs_inode *vi)
{
int ret;
struct nameidata nd;
@@ -227,6 +200,7 @@ int erofs_iget_by_path(const char *path, struct erofs_vnode *v)
if (ret)
return ret;
- return erofs_iget_by_nid(nd.nid, v);
+ vi->nid = nd.nid;
+ return erofs_read_inode_from_disk(vi);
}
diff --git a/fuse/namei.h b/fuse/namei.h
index bd5adfda2969..730caf0085f7 100644
--- a/fuse/namei.h
+++ b/fuse/namei.h
@@ -8,8 +8,7 @@
#define __INODE_H
#include "erofs/internal.h"
-#include "erofs_fs.h"
-int erofs_iget_by_path(const char *path, struct erofs_vnode *v);
+int erofs_ilookup(const char *path, struct erofs_inode *vi);
#endif
diff --git a/fuse/read.c b/fuse/read.c
index 21fbd2eea662..4e0058c01e81 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -18,105 +18,43 @@
#include "erofs/io.h"
#include "erofs/decompress.h"
-size_t erofs_read_data_wrapper(struct erofs_vnode *vnode, char *buffer,
- size_t size, off_t offset)
-{
- struct erofs_inode tmp = {
- .u = {
- .i_blkaddr = vnode->raw_blkaddr,
- },
- .nid = vnode->nid,
- .i_size = vnode->i_size,
- .datalayout = vnode->datalayout,
- .inode_isize = vnode->inode_isize,
- .xattr_isize = vnode->xattr_isize,
- };
-
- int ret = erofs_read_raw_data(&tmp, buffer, offset, size);
- if (ret)
- return ret;
-
- erofs_info("nid:%llu size=%zd done", (unsigned long long)vnode->nid, size);
- return size;
-}
-
-size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
- erofs_off_t size, erofs_off_t offset)
-{
- struct erofs_inode tmp = {
- .nid = vnode->nid,
- .i_size = vnode->i_size,
- .datalayout = vnode->datalayout,
- .inode_isize = vnode->inode_isize,
- .xattr_isize = vnode->xattr_isize,
- .z_advise = vnode->z_advise,
- .z_algorithmtype = {
- [0] = vnode->z_algorithmtype[0],
- [1] = vnode->z_algorithmtype[1],
- },
- .z_logical_clusterbits = vnode->z_logical_clusterbits,
- .z_physical_clusterbits = {
- [0] = vnode->z_physical_clusterbits[0],
- [1] = vnode->z_physical_clusterbits[1],
- },
- };
-
- if (vnode->z_inited)
- tmp.flags |= EROFS_I_Z_INITED;
-
- int ret = z_erofs_read_data(&tmp, buffer, offset, size);
- if (ret)
- return ret;
-
- return size;
-}
-
int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
struct fuse_file_info *fi)
{
int ret;
- struct erofs_vnode v;
+ struct erofs_inode vi;
UNUSED(fi);
erofs_info("path:%s size=%zd offset=%llu", path, size, (long long)offset);
- ret = erofs_iget_by_path(path, &v);
+ ret = erofs_ilookup(path, &vi);
if (ret)
return ret;
- erofs_info("path:%s nid=%llu mode=%u", path, v.nid | 0ULL, v.datalayout);
- switch (v.datalayout) {
+ erofs_info("path:%s nid=%llu mode=%u", path, vi.nid | 0ULL, vi.datalayout);
+ switch (vi.datalayout) {
case EROFS_INODE_FLAT_PLAIN:
case EROFS_INODE_FLAT_INLINE:
- return erofs_read_data_wrapper(&v, buffer, size, offset);
-
+ ret = erofs_read_raw_data(&vi, buffer, offset, size);
+ break;
case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
case EROFS_INODE_FLAT_COMPRESSION:
- return erofs_read_data_compression(&v, buffer, size, offset);
+ ret = z_erofs_read_data(&vi, buffer, offset, size);
+ break;
default:
return -EINVAL;
}
+
+ return ret ? ret : size;
}
int erofs_readlink(const char *path, char *buffer, size_t size)
{
- int ret;
- size_t lnksz;
- struct erofs_vnode v;
+ int ret = erofs_read(path, buffer, size, 0, NULL);
- ret = erofs_iget_by_path(path, &v);
- if (ret)
+ if (ret < 0)
return ret;
-
- lnksz = min((size_t)v.i_size, size - 1);
-
- ret = erofs_read(path, buffer, lnksz, 0, NULL);
- buffer[lnksz] = '\0';
- if (ret != (int)lnksz)
- return ret;
-
- erofs_info("path:%s link=%s size=%llu", path, buffer, (unsigned long long)lnksz);
return 0;
}
diff --git a/fuse/readir.c b/fuse/readir.c
index 1d28016a8900..510aa7ebaf11 100644
--- a/fuse/readir.c
+++ b/fuse/readir.c
@@ -66,14 +66,14 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
off_t offset, struct fuse_file_info *fi)
{
int ret;
- struct erofs_vnode v;
+ struct erofs_inode v;
char dirsbuf[EROFS_BLKSIZ];
uint32_t dir_nr, dir_off, nr_cnt;
erofs_dbg("readdir:%s offset=%llu", path, (long long)offset);
UNUSED(fi);
- ret = erofs_iget_by_path(path, &v);
+ ret = erofs_ilookup(path, &v);
if (ret)
return ret;
@@ -89,11 +89,12 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
dir_off = erofs_blkoff(v.i_size);
nr_cnt = 0;
- erofs_dbg("dir_size=%u dir_nr = %u dir_off=%u", v.i_size, dir_nr, dir_off);
+ erofs_dbg("dir_size=%llu dir_nr = %u dir_off=%u",
+ v.i_size | 0ULL, dir_nr, dir_off);
while (nr_cnt < dir_nr) {
memset(dirsbuf, 0, sizeof(dirsbuf));
- ret = blk_read(dirsbuf, v.raw_blkaddr + nr_cnt, 1);
+ ret = blk_read(dirsbuf, v.u.i_blkaddr + nr_cnt, 1);
if (ret < 0)
return -EIO;
fill_dir(dirsbuf, filler, buf, EROFS_BLKSIZ);
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index 98e1263fa19c..573ebfc298b5 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -164,40 +164,6 @@ struct erofs_inode {
#endif
};
-struct erofs_vnode {
- uint8_t datalayout;
-
- uint32_t i_size;
- /* inline size in bytes */
- uint16_t inode_isize;
- uint16_t xattr_isize;
-
- uint16_t xattr_shared_count;
- char *xattr_shared_xattrs;
-
- union {
- erofs_blk_t raw_blkaddr;
- struct {
- uint16_t z_advise;
- uint8_t z_algorithmtype[2];
- uint8_t z_logical_clusterbits;
- uint8_t z_physical_clusterbits[2];
- };
- };
- erofs_nid_t nid;
- uint32_t i_ino;
-
- uint16_t i_mode;
- uint16_t i_uid;
- uint16_t i_gid;
- uint16_t i_nlink;
- uint32_t i_rdev;
-
- bool z_inited;
- /* if file is inline read inline data witch inode */
- char *idata;
-};
-
static inline bool is_inode_layout_compression(struct erofs_inode *inode)
{
return erofs_inode_is_data_compressed(inode->datalayout);
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 07/12] erofs-utils: fuse: move namei.c to lib/
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (5 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 06/12] erofs-utils: fuse: get rid of erofs_vnode Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 08/12] erofs-utils: fuse: kill read.c Gao Xiang via Linux-erofs
` (2 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/Makefile.am | 2 +-
fuse/main.c | 1 -
fuse/namei.h | 14 --------------
fuse/read.c | 1 -
fuse/readir.c | 1 -
include/erofs/internal.h | 3 +++
lib/Makefile.am | 2 +-
{fuse => lib}/namei.c | 3 +--
8 files changed, 6 insertions(+), 21 deletions(-)
delete mode 100644 fuse/namei.h
rename {fuse => lib}/namei.c (98%)
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index d6e6d60cbfdc..f37069ff7f12 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -3,7 +3,7 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
-erofsfuse_SOURCES = main.c namei.c read.c readir.c
+erofsfuse_SOURCES = main.c read.c readir.c
erofsfuse_CFLAGS = -Wall -Werror \
-I$(top_srcdir)/include \
$(shell pkg-config fuse --cflags) \
diff --git a/fuse/main.c b/fuse/main.c
index 6176e836c2f1..fee90154a251 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -12,7 +12,6 @@
#include <stddef.h>
#include "erofs/print.h"
-#include "namei.h"
#include "read.h"
#include "readir.h"
#include "erofs/io.h"
diff --git a/fuse/namei.h b/fuse/namei.h
deleted file mode 100644
index 730caf0085f7..000000000000
--- a/fuse/namei.h
+++ /dev/null
@@ -1,14 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0+ */
-/*
- * erofs-utils/fuse/inode.h
- *
- * Created by Li Guifu <blucerlee@gmail.com>
- */
-#ifndef __INODE_H
-#define __INODE_H
-
-#include "erofs/internal.h"
-
-int erofs_ilookup(const char *path, struct erofs_inode *vi);
-
-#endif
diff --git a/fuse/read.c b/fuse/read.c
index 4e0058c01e81..2ef979ddba63 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -14,7 +14,6 @@
#include "erofs/defs.h"
#include "erofs/internal.h"
#include "erofs/print.h"
-#include "namei.h"
#include "erofs/io.h"
#include "erofs/decompress.h"
diff --git a/fuse/readir.c b/fuse/readir.c
index 510aa7ebaf11..a405dd702d84 100644
--- a/fuse/readir.c
+++ b/fuse/readir.c
@@ -12,7 +12,6 @@
#include "erofs/defs.h"
#include "erofs/internal.h"
#include "erofs_fs.h"
-#include "namei.h"
#include "erofs/io.h"
#include "erofs/print.h"
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index 573ebfc298b5..7357ed75e3f8 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -241,6 +241,9 @@ struct erofs_map_blocks {
/* super.c */
int erofs_read_superblock(void);
+/* namei.c */
+int erofs_ilookup(const char *path, struct erofs_inode *vi);
+
/* data.c */
int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
erofs_off_t offset, erofs_off_t size);
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 7d9446b3cbcf..f21dc35eda51 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -3,7 +3,7 @@
noinst_LTLIBRARIES = liberofs.la
liberofs_la_SOURCES = config.c io.c cache.c super.c inode.c xattr.c exclude.c \
- data.c compress.c compressor.c zmap.c decompress.c
+ namei.c data.c compress.c compressor.c zmap.c decompress.c
liberofs_la_CFLAGS = -Wall -Werror -I$(top_srcdir)/include
if ENABLE_LZ4
liberofs_la_CFLAGS += ${LZ4_CFLAGS}
diff --git a/fuse/namei.c b/lib/namei.c
similarity index 98%
rename from fuse/namei.c
rename to lib/namei.c
index 326ea85809bb..2e024d88d93e 100644
--- a/fuse/namei.c
+++ b/lib/namei.c
@@ -1,10 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-utils/fuse/namei.c
+ * erofs-utils/lib/namei.c
*
* Created by Li Guifu <blucerlee@gmail.com>
*/
-#include "namei.h"
#include <linux/kdev_t.h>
#include <sys/types.h>
#include <unistd.h>
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 08/12] erofs-utils: fuse: kill read.c
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (6 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 07/12] erofs-utils: fuse: move namei.c to lib/ Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 09/12] erofs-utils: fuse: clean up readdir Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 10/12] erofs-utils: fuse: rename readir.c to dir.c Gao Xiang via Linux-erofs
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/Makefile.am | 2 +-
fuse/main.c | 34 +++++++++++++++++++++--
fuse/read.c | 59 ----------------------------------------
fuse/read.h | 17 ------------
include/erofs/internal.h | 6 ++--
lib/data.c | 24 +++++++++++++---
lib/namei.c | 2 +-
7 files changed, 55 insertions(+), 89 deletions(-)
delete mode 100644 fuse/read.c
delete mode 100644 fuse/read.h
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index f37069ff7f12..21a1ee975141 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -3,7 +3,7 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
-erofsfuse_SOURCES = main.c read.c readir.c
+erofsfuse_SOURCES = main.c readir.c
erofsfuse_CFLAGS = -Wall -Werror \
-I$(top_srcdir)/include \
$(shell pkg-config fuse --cflags) \
diff --git a/fuse/main.c b/fuse/main.c
index fee90154a251..9ac8149c88d9 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -12,7 +12,6 @@
#include <stddef.h>
#include "erofs/print.h"
-#include "read.h"
#include "readir.h"
#include "erofs/io.h"
@@ -151,12 +150,41 @@ int erofs_getattr(const char *path, struct stat *stbuf)
return 0;
}
+static int erofsfuse_read(const char *path, char *buffer,
+ size_t size, off_t offset,
+ struct fuse_file_info *fi)
+{
+ int ret;
+ struct erofs_inode vi;
+
+ UNUSED(fi);
+ erofs_info("path:%s size=%zd offset=%llu", path, size, (long long)offset);
+
+ ret = erofs_ilookup(path, &vi);
+ if (ret)
+ return ret;
+
+ ret = erofs_pread(&vi, buffer, size, offset);
+ if (ret)
+ return ret;
+ return size;
+}
+
+static int erofsfuse_readlink(const char *path, char *buffer, size_t size)
+{
+ int ret = erofsfuse_read(path, buffer, size, 0, NULL);
+
+ if (ret < 0)
+ return ret;
+ return 0;
+}
+
static struct fuse_operations erofs_ops = {
- .readlink = erofs_readlink,
+ .readlink = erofsfuse_readlink,
.getattr = erofs_getattr,
.readdir = erofs_readdir,
.open = erofs_open,
- .read = erofs_read,
+ .read = erofsfuse_read,
.init = erofs_init,
};
diff --git a/fuse/read.c b/fuse/read.c
deleted file mode 100644
index 2ef979ddba63..000000000000
--- a/fuse/read.c
+++ /dev/null
@@ -1,59 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0+
-/*
- * erofs-utils/fuse/read.c
- *
- * Created by Li Guifu <blucerlee@gmail.com>
- * Compression support by Huang Jianan <huangjianan@oppo.com>
- */
-#include "read.h"
-#include <errno.h>
-#include <linux/fs.h>
-#include <sys/stat.h>
-#include <string.h>
-
-#include "erofs/defs.h"
-#include "erofs/internal.h"
-#include "erofs/print.h"
-#include "erofs/io.h"
-#include "erofs/decompress.h"
-
-int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
- struct fuse_file_info *fi)
-{
- int ret;
- struct erofs_inode vi;
-
- UNUSED(fi);
- erofs_info("path:%s size=%zd offset=%llu", path, size, (long long)offset);
-
- ret = erofs_ilookup(path, &vi);
- if (ret)
- return ret;
-
- erofs_info("path:%s nid=%llu mode=%u", path, vi.nid | 0ULL, vi.datalayout);
- switch (vi.datalayout) {
- case EROFS_INODE_FLAT_PLAIN:
- case EROFS_INODE_FLAT_INLINE:
- ret = erofs_read_raw_data(&vi, buffer, offset, size);
- break;
- case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
- case EROFS_INODE_FLAT_COMPRESSION:
- ret = z_erofs_read_data(&vi, buffer, offset, size);
- break;
-
- default:
- return -EINVAL;
- }
-
- return ret ? ret : size;
-}
-
-int erofs_readlink(const char *path, char *buffer, size_t size)
-{
- int ret = erofs_read(path, buffer, size, 0, NULL);
-
- if (ret < 0)
- return ret;
- return 0;
-}
-
diff --git a/fuse/read.h b/fuse/read.h
deleted file mode 100644
index e901c607dc91..000000000000
--- a/fuse/read.h
+++ /dev/null
@@ -1,17 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0+ */
-/*
- * erofs-utils/fuse/read.h
- *
- * Created by Li Guifu <blucerlee@gmail.com>
- */
-#ifndef __EROFS_READ_H
-#define __EROFS_READ_H
-
-#include <fuse.h>
-#include <fuse_opt.h>
-
-int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
- struct fuse_file_info *fi);
-int erofs_readlink(const char *path, char *buffer, size_t size);
-
-#endif
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index 7357ed75e3f8..13420a8e7733 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -245,10 +245,8 @@ int erofs_read_superblock(void);
int erofs_ilookup(const char *path, struct erofs_inode *vi);
/* data.c */
-int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
- erofs_off_t offset, erofs_off_t size);
-int z_erofs_read_data(struct erofs_inode *inode, char *buffer,
- erofs_off_t size, erofs_off_t offset);
+int erofs_pread(struct erofs_inode *inode, char *buf,
+ erofs_off_t count, erofs_off_t offset);
/* zmap.c */
int z_erofs_fill_inode(struct erofs_inode *vi);
int z_erofs_map_blocks_iter(struct erofs_inode *vi,
diff --git a/lib/data.c b/lib/data.c
index 62fd057185ee..34811e49512f 100644
--- a/lib/data.c
+++ b/lib/data.c
@@ -70,8 +70,8 @@ err_out:
return err;
}
-int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
- erofs_off_t offset, erofs_off_t size)
+static int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
+ erofs_off_t size, erofs_off_t offset)
{
struct erofs_map_blocks map = {
.index = UINT_MAX,
@@ -117,8 +117,8 @@ int erofs_read_raw_data(struct erofs_inode *inode, char *buffer,
return 0;
}
-int z_erofs_read_data(struct erofs_inode *inode, char *buffer,
- erofs_off_t offset, erofs_off_t size)
+static int z_erofs_read_data(struct erofs_inode *inode, char *buffer,
+ erofs_off_t size, erofs_off_t offset)
{
int ret;
erofs_off_t end, length, skip;
@@ -188,3 +188,19 @@ int z_erofs_read_data(struct erofs_inode *inode, char *buffer,
return 0;
}
+int erofs_pread(struct erofs_inode *inode, char *buf,
+ erofs_off_t count, erofs_off_t offset)
+{
+ switch (inode->datalayout) {
+ case EROFS_INODE_FLAT_PLAIN:
+ case EROFS_INODE_FLAT_INLINE:
+ return erofs_read_raw_data(inode, buf, count, offset);
+ case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
+ case EROFS_INODE_FLAT_COMPRESSION:
+ return z_erofs_read_data(inode, buf, count, offset);
+ default:
+ break;
+ }
+ return -EINVAL;
+}
+
diff --git a/lib/namei.c b/lib/namei.c
index 2e024d88d93e..4e6aceb90df1 100644
--- a/lib/namei.c
+++ b/lib/namei.c
@@ -136,7 +136,7 @@ int erofs_namei(struct nameidata *nd,
struct erofs_dirent *de = (void *)buf;
unsigned int nameoff;
- ret = erofs_read_raw_data(&vi, buf, offset, maxsize);
+ ret = erofs_pread(&vi, buf, maxsize, offset);
if (ret)
return ret;
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 09/12] erofs-utils: fuse: clean up readdir
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (7 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 08/12] erofs-utils: fuse: kill read.c Gao Xiang via Linux-erofs
@ 2020-11-14 18:25 ` Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 10/12] erofs-utils: fuse: rename readir.c to dir.c Gao Xiang via Linux-erofs
9 siblings, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:25 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/main.c | 2 +-
fuse/readir.c | 155 +++++++++++++++++++++++---------------------------
fuse/readir.h | 4 +-
3 files changed, 74 insertions(+), 87 deletions(-)
diff --git a/fuse/main.c b/fuse/main.c
index 9ac8149c88d9..a4b7a3692c19 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -182,7 +182,7 @@ static int erofsfuse_readlink(const char *path, char *buffer, size_t size)
static struct fuse_operations erofs_ops = {
.readlink = erofsfuse_readlink,
.getattr = erofs_getattr,
- .readdir = erofs_readdir,
+ .readdir = erofsfuse_readdir,
.open = erofs_open,
.read = erofsfuse_read,
.init = erofs_init,
diff --git a/fuse/readir.c b/fuse/readir.c
index a405dd702d84..2e90c95c38aa 100644
--- a/fuse/readir.c
+++ b/fuse/readir.c
@@ -4,114 +4,101 @@
*
* Created by Li Guifu <blucerlee@gmail.com>
*/
-#include "readir.h"
-#include <errno.h>
-#include <linux/fs.h>
-#include <sys/stat.h>
+#include <fuse.h>
+#include <fuse_opt.h>
-#include "erofs/defs.h"
#include "erofs/internal.h"
-#include "erofs_fs.h"
-#include "erofs/io.h"
#include "erofs/print.h"
-erofs_nid_t split_entry(char *entry, off_t ofs, char *end, char *name,
- uint32_t dirend)
+static int erofs_fill_dentries(struct erofs_inode *dir,
+ fuse_fill_dir_t filler, void *buf,
+ void *dblk, unsigned int nameoff,
+ unsigned int maxsize)
{
- struct erofs_dirent *de = (struct erofs_dirent *)(entry + ofs);
- uint16_t nameoff = le16_to_cpu(de->nameoff);
- const char *de_name = entry + nameoff;
- uint32_t de_namelen;
-
- if ((void *)(de + 1) >= (void *)end)
- de_namelen = strnlen(de_name, dirend - nameoff);
- else
- de_namelen = le16_to_cpu(de[1].nameoff) - nameoff;
-
- memcpy(name, de_name, de_namelen);
- name[de_namelen] = '\0';
- return le64_to_cpu(de->nid);
-}
-
-int fill_dir(char *entrybuf, fuse_fill_dir_t filler, void *buf,
- uint32_t dirend)
-{
- uint32_t ofs;
- uint16_t nameoff;
- char *end;
- char name[EROFS_BLKSIZ];
-
- nameoff = le16_to_cpu(((struct erofs_dirent *)entrybuf)->nameoff);
- end = entrybuf + nameoff;
- ofs = 0;
- while (ofs < nameoff) {
- (void)split_entry(entrybuf, ofs, end, name, dirend);
- filler(buf, name, NULL, 0);
- ofs += sizeof(struct erofs_dirent);
+ struct erofs_dirent *de = dblk;
+ const struct erofs_dirent *end = dblk + nameoff;
+ char namebuf[EROFS_NAME_LEN];
+
+ while (de < end) {
+ const char *de_name;
+ unsigned int de_namelen;
+
+ nameoff = le16_to_cpu(de->nameoff);
+ de_name = (char *)dblk + nameoff;
+
+ /* the last dirent in the block? */
+ if (de + 1 >= end)
+ de_namelen = strnlen(de_name, maxsize - nameoff);
+ else
+ de_namelen = le16_to_cpu(de[1].nameoff) - nameoff;
+
+ /* a corrupted entry is found */
+ if (nameoff + de_namelen > maxsize ||
+ de_namelen > EROFS_NAME_LEN) {
+ erofs_err("bogus dirent @ nid %llu", dir->nid | 0ULL);
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
+ }
+
+ memcpy(namebuf, de_name, de_namelen);
+ namebuf[de_namelen] = '\0';
+
+ filler(buf, namebuf, NULL, 0);
+ ++de;
}
-
return 0;
}
-/** Function to add an entry in a readdir() operation
- *
- * @param buf the buffer passed to the readdir() operation
- * @param name the file name of the directory entry
- * @param stat file attributes, can be NULL
- * @param off offset of the next entry or zero
- * @return 1 if buffer is full, zero otherwise
- */
-int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
- off_t offset, struct fuse_file_info *fi)
+int erofsfuse_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi)
{
int ret;
- struct erofs_inode v;
- char dirsbuf[EROFS_BLKSIZ];
- uint32_t dir_nr, dir_off, nr_cnt;
+ struct erofs_inode dir;
+ char dblk[EROFS_BLKSIZ];
+ erofs_off_t pos;
erofs_dbg("readdir:%s offset=%llu", path, (long long)offset);
UNUSED(fi);
- ret = erofs_ilookup(path, &v);
+ ret = erofs_ilookup(path, &dir);
if (ret)
return ret;
- erofs_dbg("path=%s nid = %llu", path, v.nid | 0ULL);
+ erofs_dbg("path=%s nid = %llu", path, dir.nid | 0ULL);
- if (!S_ISDIR(v.i_mode))
+ if (!S_ISDIR(dir.i_mode))
return -ENOTDIR;
- if (!v.i_size)
+ if (!dir.i_size)
return 0;
- dir_nr = erofs_blknr(v.i_size);
- dir_off = erofs_blkoff(v.i_size);
- nr_cnt = 0;
-
- erofs_dbg("dir_size=%llu dir_nr = %u dir_off=%u",
- v.i_size | 0ULL, dir_nr, dir_off);
-
- while (nr_cnt < dir_nr) {
- memset(dirsbuf, 0, sizeof(dirsbuf));
- ret = blk_read(dirsbuf, v.u.i_blkaddr + nr_cnt, 1);
- if (ret < 0)
- return -EIO;
- fill_dir(dirsbuf, filler, buf, EROFS_BLKSIZ);
- ++nr_cnt;
- }
-
- if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
- off_t addr;
-
- addr = iloc(v.nid) + v.inode_isize + v.xattr_isize;
-
- memset(dirsbuf, 0, sizeof(dirsbuf));
- ret = dev_read(dirsbuf, addr, dir_off);
- if (ret < 0)
- return -EIO;
- fill_dir(dirsbuf, filler, buf, dir_off);
+ pos = 0;
+ while (pos < dir.i_size) {
+ unsigned int nameoff, maxsize;
+ struct erofs_dirent *de;
+
+ maxsize = min_t(unsigned int, EROFS_BLKSIZ,
+ dir.i_size - pos);
+ ret = erofs_pread(&dir, dblk, maxsize, pos);
+ if (ret)
+ return ret;
+
+ de = (struct erofs_dirent *)dblk;
+ nameoff = le16_to_cpu(de->nameoff);
+ if (nameoff < sizeof(struct erofs_dirent) ||
+ nameoff >= PAGE_SIZE) {
+ erofs_err("invalid de[0].nameoff %u @ nid %llu",
+ nameoff, dir.nid | 0ULL);
+ ret = -EFSCORRUPTED;
+ break;
+ }
+
+ ret = erofs_fill_dentries(&dir, filler, buf,
+ dblk, nameoff, maxsize);
+ if (ret)
+ break;
+ pos += maxsize;
}
-
return 0;
}
diff --git a/fuse/readir.h b/fuse/readir.h
index ee2ab8bdd0f0..16b878fe9f29 100644
--- a/fuse/readir.h
+++ b/fuse/readir.h
@@ -10,8 +10,8 @@
#include <fuse.h>
#include <fuse_opt.h>
-int erofs_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
- off_t offset, struct fuse_file_info *fi);
+int erofsfuse_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi);
#endif
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 10/12] erofs-utils: fuse: rename readir.c to dir.c
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (8 preceding siblings ...)
2020-11-14 18:25 ` [WIP] [PATCH v4 09/12] erofs-utils: fuse: clean up readdir Gao Xiang via Linux-erofs
@ 2020-11-14 18:27 ` Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 11/12] erofs-utils: fuse: cleanup main.c Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 12/12] erofs-utils: fuse: fix up configure.ac / Makefile.am Gao Xiang via Linux-erofs
9 siblings, 2 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:27 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/Makefile.am | 2 +-
fuse/{readir.c => dir.c} | 2 +-
fuse/main.c | 7 ++++++-
fuse/readir.h | 17 -----------------
4 files changed, 8 insertions(+), 20 deletions(-)
rename fuse/{readir.c => dir.c} (98%)
delete mode 100644 fuse/readir.h
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index 21a1ee975141..4ed4215b9936 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -3,7 +3,7 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
-erofsfuse_SOURCES = main.c readir.c
+erofsfuse_SOURCES = dir.c main.c
erofsfuse_CFLAGS = -Wall -Werror \
-I$(top_srcdir)/include \
$(shell pkg-config fuse --cflags) \
diff --git a/fuse/readir.c b/fuse/dir.c
similarity index 98%
rename from fuse/readir.c
rename to fuse/dir.c
index 2e90c95c38aa..f6c1295f52bd 100644
--- a/fuse/readir.c
+++ b/fuse/dir.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-utils/fuse/readir.c
+ * erofs-utils/fuse/dir.c
*
* Created by Li Guifu <blucerlee@gmail.com>
*/
diff --git a/fuse/main.c b/fuse/main.c
index a4b7a3692c19..236d54635acb 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -11,10 +11,15 @@
#include <signal.h>
#include <stddef.h>
+#include <fuse.h>
+#include <fuse_opt.h>
+
#include "erofs/print.h"
-#include "readir.h"
#include "erofs/io.h"
+int erofsfuse_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi);
+
/* XXX: after liberofs is linked in, it should be removed */
struct erofs_configure cfg;
diff --git a/fuse/readir.h b/fuse/readir.h
deleted file mode 100644
index 16b878fe9f29..000000000000
--- a/fuse/readir.h
+++ /dev/null
@@ -1,17 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0+ */
-/*
- * erofs-utils/fuse/readir.h
- *
- * Created by Li Guifu <blucerlee@gmail.com>
- */
-#ifndef __EROFS_READDIR_H
-#define __EROFS_READDIR_H
-
-#include <fuse.h>
-#include <fuse_opt.h>
-
-int erofsfuse_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
- off_t offset, struct fuse_file_info *fi);
-
-
-#endif
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 11/12] erofs-utils: fuse: cleanup main.c
2020-11-14 18:27 ` [WIP] [PATCH v4 10/12] erofs-utils: fuse: rename readir.c to dir.c Gao Xiang via Linux-erofs
@ 2020-11-14 18:27 ` Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 12/12] erofs-utils: fuse: fix up configure.ac / Makefile.am Gao Xiang via Linux-erofs
1 sibling, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:27 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/main.c | 240 +++++++++++++++++++++++++---------------------------
1 file changed, 113 insertions(+), 127 deletions(-)
diff --git a/fuse/main.c b/fuse/main.c
index 236d54635acb..7beaba8a9bc2 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -4,125 +4,28 @@
*
* Created by Li Guifu <blucerlee@gmail.com>
*/
-#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <execinfo.h>
#include <signal.h>
-#include <stddef.h>
-
+#include <libgen.h>
#include <fuse.h>
#include <fuse_opt.h>
+#include "erofs/config.h"
#include "erofs/print.h"
#include "erofs/io.h"
int erofsfuse_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
off_t offset, struct fuse_file_info *fi);
-/* XXX: after liberofs is linked in, it should be removed */
-struct erofs_configure cfg;
-
-enum {
- EROFS_OPT_HELP,
- EROFS_OPT_VER,
-};
-
-struct options {
- const char *disk;
- const char *mount;
- const char *logfile;
- unsigned int debug_lvl;
-};
-static struct options fusecfg;
-
-#define OPTION(t, p) { t, offsetof(struct options, p), 1 }
-
-static const struct fuse_opt option_spec[] = {
- OPTION("--log=%s", logfile),
- OPTION("--dbg=%u", debug_lvl),
- FUSE_OPT_KEY("-h", EROFS_OPT_HELP),
- FUSE_OPT_KEY("-v", EROFS_OPT_VER),
- FUSE_OPT_END
-};
-
-static void usage(void)
-{
- fprintf(stderr, "\terofsfuse [options] <image> <mountpoint>\n");
- fprintf(stderr, "\t --log=<file> output log file\n");
- fprintf(stderr, "\t --dbg=<level> log debug level 0 ~ 7\n");
- fprintf(stderr, "\t -h show help\n");
- fprintf(stderr, "\t -v show version\n");
- exit(1);
-}
-
-static void dump_cfg(void)
-{
- fprintf(stderr, "\tdisk :%s\n", fusecfg.disk);
- fprintf(stderr, "\tmount:%s\n", fusecfg.mount);
- fprintf(stderr, "\tdebug_lvl:%u\n", fusecfg.debug_lvl);
- fprintf(stderr, "\tlogfile :%s\n", fusecfg.logfile);
-}
-
-static int optional_opt_func(void *data, const char *arg, int key,
- struct fuse_args *outargs)
-{
- UNUSED(data);
- UNUSED(outargs);
-
- switch (key) {
- case FUSE_OPT_KEY_OPT:
- return 1;
-
- case FUSE_OPT_KEY_NONOPT:
- if (!fusecfg.disk) {
- fusecfg.disk = strdup(arg);
- return 0;
- } else if (!fusecfg.mount)
- fusecfg.mount = strdup(arg);
-
- return 1;
- case EROFS_OPT_HELP:
- usage();
- break;
-
- case EROFS_OPT_VER:
- fprintf(stderr, "EROFS FUSE VERSION v 1.0.0\n");
- exit(0);
- }
-
- return 1;
-}
-
-static void signal_handle_sigsegv(int signal)
-{
- void *array[10];
- size_t nptrs;
- char **strings;
- size_t i;
-
- UNUSED(signal);
- erofs_dbg("========================================");
- erofs_dbg("Segmentation Fault. Starting backtrace:");
- nptrs = backtrace(array, 10);
- strings = backtrace_symbols(array, nptrs);
- if (strings) {
- for (i = 0; i < nptrs; i++)
- erofs_dbg("%s", strings[i]);
- free(strings);
- }
- erofs_dbg("========================================");
-
- abort();
-}
-
-void *erofs_init(struct fuse_conn_info *info)
+static void *erofsfuse_init(struct fuse_conn_info *info)
{
erofs_info("Using FUSE protocol %d.%d", info->proto_major, info->proto_minor);
return NULL;
}
-int erofs_open(const char *path, struct fuse_file_info *fi)
+static int erofsfuse_open(const char *path, struct fuse_file_info *fi)
{
erofs_info("open path=%s", path);
@@ -132,7 +35,7 @@ int erofs_open(const char *path, struct fuse_file_info *fi)
return 0;
}
-int erofs_getattr(const char *path, struct stat *stbuf)
+static int erofsfuse_getattr(const char *path, struct stat *stbuf)
{
struct erofs_inode v = { 0 };
int ret;
@@ -186,51 +89,134 @@ static int erofsfuse_readlink(const char *path, char *buffer, size_t size)
static struct fuse_operations erofs_ops = {
.readlink = erofsfuse_readlink,
- .getattr = erofs_getattr,
+ .getattr = erofsfuse_getattr,
.readdir = erofsfuse_readdir,
- .open = erofs_open,
+ .open = erofsfuse_open,
.read = erofsfuse_read,
- .init = erofs_init,
+ .init = erofsfuse_init,
};
+static struct options {
+ const char *disk;
+ const char *mount;
+ unsigned int debug_lvl;
+ bool show_help;
+} fusecfg;
+
+#define OPTION(t, p) \
+ { t, offsetof(struct options, p), 1 }
+static const struct fuse_opt option_spec[] = {
+ OPTION("-d %u", debug_lvl),
+ OPTION("-h", show_help),
+ OPTION("--help", show_help),
+ FUSE_OPT_END
+};
+
+#define OPTION(t, p) { t, offsetof(struct options, p), 1 }
+
+static void usage(void)
+{
+ fprintf(stderr, "\terofsfuse [options] <image> <mountpoint>\n");
+ fprintf(stderr, "\t --log=<file> output log file\n");
+ fprintf(stderr, "\t --dbg=<level> log debug level 0 ~ 7\n");
+ fprintf(stderr, "\t -h show help\n");
+ fprintf(stderr, "\t -v show version\n");
+ exit(EXIT_FAILURE);
+}
+
+static void erofsfuse_dumpcfg(void)
+{
+ erofs_info("disk: %s", fusecfg.disk);
+ erofs_info("mountpoint: %s", fusecfg.mount);
+ erofs_info("debug level: %u", cfg.c_dbg_lvl);
+}
+
+static int optional_opt_func(void *data, const char *arg, int key,
+ struct fuse_args *outargs)
+{
+ switch (key) {
+ case FUSE_OPT_KEY_NONOPT:
+ if (!fusecfg.disk) {
+ fusecfg.disk = strdup(arg);
+ return 0;
+ }
+ if (!fusecfg.mount)
+ fusecfg.mount = strdup(arg);
+ case FUSE_OPT_KEY_OPT:
+ break;
+ default:
+ DBG_BUGON(1);
+ break;
+ }
+ return 1;
+}
+
+static void signal_handle_sigsegv(int signal)
+{
+ void *array[10];
+ size_t nptrs;
+ char **strings;
+ size_t i;
+
+ UNUSED(signal);
+ erofs_dbg("========================================");
+ erofs_dbg("Segmentation Fault. Starting backtrace:");
+ nptrs = backtrace(array, 10);
+ strings = backtrace_symbols(array, nptrs);
+ if (strings) {
+ for (i = 0; i < nptrs; i++)
+ erofs_dbg("%s", strings[i]);
+ free(strings);
+ }
+ erofs_dbg("========================================");
+
+ abort();
+}
+
+
int main(int argc, char *argv[])
{
- int ret = EXIT_FAILURE;
+ int ret;
struct fuse_args args = FUSE_ARGS_INIT(argc, argv);
+ erofs_init_configure();
+ fprintf(stderr, "%s %s\n", basename(argv[0]), cfg.c_version);
+
if (signal(SIGSEGV, signal_handle_sigsegv) == SIG_ERR) {
- fprintf(stderr, "Failed to initialize signals\n");
- return EXIT_FAILURE;
+ fprintf(stderr, "failed to initialize signals\n");
+ ret = -errno;
+ goto err;
}
- /* Parse options */
- if (fuse_opt_parse(&args, &fusecfg, option_spec, optional_opt_func) < 0)
- return 1;
-
- dump_cfg();
+ /* parse options */
+ ret = fuse_opt_parse(&args, &fusecfg, option_spec, optional_opt_func);
+ if (ret)
+ goto err;
+ if (fusecfg.show_help)
+ usage();
cfg.c_dbg_lvl = fusecfg.debug_lvl;
- if (dev_open_ro(fusecfg.disk) < 0) {
- fprintf(stderr, "Failed to open disk:%s\n", fusecfg.disk);
- goto exit;
+ erofsfuse_dumpcfg();
+ ret = dev_open_ro(fusecfg.disk);
+ if (ret) {
+ fprintf(stderr, "failed to open: %s\n", fusecfg.disk);
+ goto err_fuse_free_args;
}
- if (erofs_read_superblock()) {
- fprintf(stderr, "Failed to read erofs super block\n");
- goto exit_dev;
+ ret = erofs_read_superblock();
+ if (ret) {
+ fprintf(stderr, "failed to read erofs super block\n");
+ goto err_dev_close;
}
- erofs_info("fuse start");
-
ret = fuse_main(args.argc, args.argv, &erofs_ops, NULL);
-
- erofs_info("fuse done ret=%d", ret);
-
-exit_dev:
+err_dev_close:
dev_close();
-exit:
+err_fuse_free_args:
fuse_opt_free_args(&args);
- return ret;
+err:
+ erofs_exit_configure();
+ return ret ? EXIT_FAILURE : EXIT_SUCCESS;
}
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [WIP] [PATCH v4 12/12] erofs-utils: fuse: fix up configure.ac / Makefile.am
2020-11-14 18:27 ` [WIP] [PATCH v4 10/12] erofs-utils: fuse: rename readir.c to dir.c Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 11/12] erofs-utils: fuse: cleanup main.c Gao Xiang via Linux-erofs
@ 2020-11-14 18:27 ` Gao Xiang via Linux-erofs
1 sibling, 0 replies; 13+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-11-14 18:27 UTC (permalink / raw)
To: linux-erofs
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
Makefile.am | 6 +++++-
configure.ac | 19 +++++++++++++++++++
fuse/Makefile.am | 10 +++-------
3 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/Makefile.am b/Makefile.am
index 24f4a7b3d5ad..b804aa90efa9 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -3,4 +3,8 @@
ACLOCAL_AMFLAGS = -I m4
-SUBDIRS = man lib mkfs fuse
+SUBDIRS = man lib mkfs
+if ENABLE_FUSE
+SUBDIRS += fuse
+endif
+
diff --git a/configure.ac b/configure.ac
index e6d2e5fd3702..80d38d0ef952 100644
--- a/configure.ac
+++ b/configure.ac
@@ -63,6 +63,10 @@ AC_ARG_ENABLE(lz4,
[AS_HELP_STRING([--disable-lz4], [disable LZ4 compression support @<:@default=enabled@:>@])],
[enable_lz4="$enableval"], [enable_lz4="yes"])
+AC_ARG_ENABLE(fuse,
+ [AS_HELP_STRING([--disable-fuse], [disable erofsfuse @<:@default=enabled@:>@])],
+ [enable_fuse="$enableval"], [enable_fuse="yes"])
+
AC_ARG_WITH(uuid,
[AS_HELP_STRING([--without-uuid],
[Ignore presence of libuuid and disable uuid support @<:@default=enabled@:>@])])
@@ -183,6 +187,20 @@ AS_IF([test "x$with_selinux" != "xno"], [
LIBS="${saved_LIBS}"
CPPFLAGS="${saved_CPPFLAGS}"], [have_selinux="no"])
+# Configure fuse
+AS_IF([test "x$enable_fuse" != "xno"], [
+ PKG_CHECK_MODULES([libfuse], [fuse >= 2.6.0])
+ # Paranoia: don't trust the result reported by pkgconfig before trying out
+ saved_LIBS="$LIBS"
+ saved_CPPFLAGS=${CPPFLAGS}
+ CPPFLAGS="${libfuse_CFLAGS} ${CPPFLAGS}"
+ LIBS="${libfuse_LIBS} $LIBS"
+ AC_CHECK_LIB(fuse, fuse_main, [
+ have_fuse="yes" ], [
+ AC_MSG_ERROR([libfuse >= 2.6.0 doesn't work properly])])
+ LIBS="${saved_LIBS}"
+ CPPFLAGS="${saved_CPPFLAGS}"], [have_fuse="no"])
+
# Configure lz4
test -z $LZ4_LIBS && LZ4_LIBS='-llz4'
@@ -218,6 +236,7 @@ fi
# Set up needed symbols, conditionals and compiler/linker flags
AM_CONDITIONAL([ENABLE_LZ4], [test "x${have_lz4}" = "xyes"])
AM_CONDITIONAL([ENABLE_LZ4HC], [test "x${have_lz4hc}" = "xyes"])
+AM_CONDITIONAL([ENABLE_FUSE], [test "x${have_fuse}" = "xyes"])
if test "x$have_uuid" = "xyes"; then
AC_DEFINE([HAVE_LIBUUID], 1, [Define to 1 if libuuid is found])
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index 4ed4215b9936..6636ee1aec69 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -4,11 +4,7 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
erofsfuse_SOURCES = dir.c main.c
-erofsfuse_CFLAGS = -Wall -Werror \
- -I$(top_srcdir)/include \
- $(shell pkg-config fuse --cflags) \
- -DFUSE_USE_VERSION=26 \
- -std=gnu99
-LDFLAGS += $(shell pkg-config fuse --libs)
-erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl ${LZ4_LIBS}
+erofsfuse_CFLAGS = -Wall -Werror -I$(top_srcdir)/include
+erofsfuse_CFLAGS += -DFUSE_USE_VERSION=26 ${libfuse_CFLAGS}
+erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la ${libfuse_LIBS} ${LZ4_LIBS}
--
2.24.0
^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2020-11-14 18:28 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20201114182517.9738-1-hsiangkao.ref@aol.com>
2020-11-14 18:25 ` [WIP] [PATCH v4 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 01/12] " Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 02/12] erofs-utils: fuse: add special file support Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 03/12] erofs-utils: fuse: add compressed " Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 04/12] erofs-utils: fuse: clean up path walking Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 05/12] erofs: clean up compress data read Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 06/12] erofs-utils: fuse: get rid of erofs_vnode Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 07/12] erofs-utils: fuse: move namei.c to lib/ Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 08/12] erofs-utils: fuse: kill read.c Gao Xiang via Linux-erofs
2020-11-14 18:25 ` [WIP] [PATCH v4 09/12] erofs-utils: fuse: clean up readdir Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 10/12] erofs-utils: fuse: rename readir.c to dir.c Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 11/12] erofs-utils: fuse: cleanup main.c Gao Xiang via Linux-erofs
2020-11-14 18:27 ` [WIP] [PATCH v4 12/12] erofs-utils: fuse: fix up configure.ac / Makefile.am Gao Xiang via Linux-erofs
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.