* [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation
[not found] <20201017051621.7810-1-hsiangkao.ref@aol.com>
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 01/12] " Gao Xiang via Linux-erofs
` (11 more replies)
0 siblings, 12 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
Hi all,
This patchset is based on the original patch
https://lore.kernel.org/r/20190818135923.27444-1-blucerlee@gmail.com
and several fixes / compression support by Huang Jianan
https://lore.kernel.org/r/20201015133959.61007-1-huangjianan@oppo.com
with my additional 9 incremental patches to refactor my current
development status, which will be folded into original patches
in the next WIP version since the fuse approach hasn't been
merged into dev branch yet.
To summarize the benefits of erofsfuse, I think it would be:
- erofs images can be supported on various platforms;
- an independent unpack tool can be developed based on this;
- new on-disk feature can be iterated, verified effectively.
Any feedback or/and follow-on development/cleanup is welcomed.
(we still have a lot to do for the entire erofsfuse codebase...)
Thanks,
Gao Xiang
Gao Xiang (9):
erofs-utils: fuse: adjust larger extent handling
erofs-utils: fuse: use proper expression about inode size
erofs-utils: fuse: drop ofs_out
erofs-utils: fuse: refuse a undefined shifted cluster behavior
erofs-utils: fuse: drop z_erofs_shifted_transform()
erofs-utils: fuse: rename ofs_head and outputsize
erofs-utils: fuse: cleanup erofs_read_data_compression()
erofs-utils: fuse: move up mpage in struct erofs_map_blocks
erofs-utils: fuse: fix up source headers
Huang Jianan (2):
erofs-utils: fuse: support read special file
erofs-utils: fuse: support read compressed file
Li Guifu (1):
erofs-utils: introduce fuse implementation
Makefile.am | 2 +-
README | 28 ++-
configure.ac | 3 +-
fuse/Makefile.am | 18 ++
fuse/decompress.c | 84 ++++++++
fuse/decompress.h | 42 ++++
fuse/dentry.c | 129 ++++++++++++
fuse/dentry.h | 43 ++++
fuse/disk_io.c | 72 +++++++
fuse/disk_io.h | 21 ++
fuse/getattr.c | 65 ++++++
fuse/getattr.h | 15 ++
fuse/init.c | 118 +++++++++++
fuse/init.h | 24 +++
fuse/logging.c | 81 ++++++++
fuse/logging.h | 55 ++++++
fuse/main.c | 171 ++++++++++++++++
fuse/namei.c | 242 +++++++++++++++++++++++
fuse/namei.h | 22 +++
fuse/open.c | 22 +++
fuse/open.h | 15 ++
fuse/read.c | 214 ++++++++++++++++++++
fuse/read.h | 17 ++
fuse/readir.c | 123 ++++++++++++
fuse/readir.h | 17 ++
fuse/zmap.c | 417 +++++++++++++++++++++++++++++++++++++++
include/erofs/defs.h | 16 ++
include/erofs/internal.h | 79 ++++++++
include/erofs_fs.h | 4 +
29 files changed, 2156 insertions(+), 3 deletions(-)
create mode 100644 fuse/Makefile.am
create mode 100644 fuse/decompress.c
create mode 100644 fuse/decompress.h
create mode 100644 fuse/dentry.c
create mode 100644 fuse/dentry.h
create mode 100644 fuse/disk_io.c
create mode 100644 fuse/disk_io.h
create mode 100644 fuse/getattr.c
create mode 100644 fuse/getattr.h
create mode 100644 fuse/init.c
create mode 100644 fuse/init.h
create mode 100644 fuse/logging.c
create mode 100644 fuse/logging.h
create mode 100644 fuse/main.c
create mode 100644 fuse/namei.c
create mode 100644 fuse/namei.h
create mode 100644 fuse/open.c
create mode 100644 fuse/open.h
create mode 100644 fuse/read.c
create mode 100644 fuse/read.h
create mode 100644 fuse/readir.c
create mode 100644 fuse/readir.h
create mode 100644 fuse/zmap.c
--
2.24.0
^ permalink raw reply [flat|nested] 18+ messages in thread
* [WIP] [PATCH 01/12] erofs-utils: introduce fuse implementation
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 02/12] erofs-utils: fuse: support read special file Gao Xiang via Linux-erofs
` (10 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
From: Li Guifu <blucerlee@gmail.com>
Let's add erofsfuse approach, and benefits are:
- images can be supported on various platforms;
- new unpack tool can be developed based on this;
- new on-disk feature can be iterated, verified effectively.
This commit only aims at reading a regular file. Other file (e.g.
compressed file) support is out of scope for now.
Signed-off-by: Li Guifu <blucerlee@gmail.com>
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
Makefile.am | 2 +-
README | 28 ++++-
configure.ac | 3 +-
fuse/Makefile.am | 14 +++
fuse/dentry.c | 129 ++++++++++++++++++++++
fuse/dentry.h | 42 ++++++++
fuse/disk_io.c | 72 +++++++++++++
fuse/disk_io.h | 21 ++++
fuse/getattr.c | 64 +++++++++++
fuse/getattr.h | 15 +++
fuse/init.c | 98 +++++++++++++++++
fuse/init.h | 22 ++++
fuse/logging.c | 81 ++++++++++++++
fuse/logging.h | 55 ++++++++++
fuse/main.c | 170 +++++++++++++++++++++++++++++
fuse/namei.c | 227 +++++++++++++++++++++++++++++++++++++++
fuse/namei.h | 22 ++++
fuse/open.c | 22 ++++
fuse/open.h | 15 +++
fuse/read.c | 114 ++++++++++++++++++++
fuse/read.h | 16 +++
fuse/readir.c | 123 +++++++++++++++++++++
fuse/readir.h | 17 +++
include/erofs/defs.h | 3 +
include/erofs/internal.h | 37 +++++++
25 files changed, 1409 insertions(+), 3 deletions(-)
create mode 100644 fuse/Makefile.am
create mode 100644 fuse/dentry.c
create mode 100644 fuse/dentry.h
create mode 100644 fuse/disk_io.c
create mode 100644 fuse/disk_io.h
create mode 100644 fuse/getattr.c
create mode 100644 fuse/getattr.h
create mode 100644 fuse/init.c
create mode 100644 fuse/init.h
create mode 100644 fuse/logging.c
create mode 100644 fuse/logging.h
create mode 100644 fuse/main.c
create mode 100644 fuse/namei.c
create mode 100644 fuse/namei.h
create mode 100644 fuse/open.c
create mode 100644 fuse/open.h
create mode 100644 fuse/read.c
create mode 100644 fuse/read.h
create mode 100644 fuse/readir.c
create mode 100644 fuse/readir.h
diff --git a/Makefile.am b/Makefile.am
index 1d20577068c5..24f4a7b3d5ad 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -3,4 +3,4 @@
ACLOCAL_AMFLAGS = -I m4
-SUBDIRS = man lib mkfs
+SUBDIRS = man lib mkfs fuse
diff --git a/README b/README
index 5addd6b80e04..870858c48b7d 100644
--- a/README
+++ b/README
@@ -2,7 +2,8 @@ erofs-utils
===========
erofs-utils includes user-space tools for erofs filesystem images.
-Currently only mkfs.erofs is available.
+One is mkfs.erofs to create a image, the other is erofsfuse which
+is used to mount a image on a directory
mkfs.erofs
----------
@@ -95,6 +96,31 @@ It may still be useful since new erofs-utils has not been widely used in
commercial products. However, if that happens, please report bug to us
as well.
+erofs-utils: erofsfuse
+----------------------
+erofsfuse mount a erofs filesystem image created by mkfs.erofs to a directory, and then
+It can be listed, read files and so on.
+
+Dependencies
+~~~~~~~~~~~~
+FUSE library version: 2.9.7
+fusermount version: 2.9.7
+using FUSE kernel interface version 7.19
+
+How to installed fuse
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+sudo apt-get update
+sudo apt-get install -y fuse libfuse-dev
+
+How to build erofsfuse
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ $ ./autogen.sh
+ $ ./configure
+ $ make
+
+erofsfuse binary will be generated under fuse folder.
+
Contribution
------------
diff --git a/configure.ac b/configure.ac
index 0f40a840cf4f..4194a77f1e36 100644
--- a/configure.ac
+++ b/configure.ac
@@ -245,6 +245,7 @@ fi
AC_CONFIG_FILES([Makefile
man/Makefile
lib/Makefile
- mkfs/Makefile])
+ mkfs/Makefile
+ fuse/Makefile])
AC_OUTPUT
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
new file mode 100644
index 000000000000..fffd67a53fe1
--- /dev/null
+++ b/fuse/Makefile.am
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0+
+# Makefile.am
+
+AUTOMAKE_OPTIONS = foreign
+bin_PROGRAMS = erofsfuse
+erofsfuse_SOURCES = main.c dentry.c getattr.c logging.c namei.c read.c disk_io.c init.c open.c readir.c
+erofsfuse_CFLAGS = -Wall -Werror -Wextra \
+ -I$(top_srcdir)/include \
+ $(shell pkg-config fuse --cflags) \
+ -DFUSE_USE_VERSION=26 \
+ -std=gnu99
+LDFLAGS += $(shell pkg-config fuse --libs)
+erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl
+
diff --git a/fuse/dentry.c b/fuse/dentry.c
new file mode 100644
index 000000000000..27192ecfd32e
--- /dev/null
+++ b/fuse/dentry.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\dentry.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "dentry.h"
+#include "erofs/internal.h"
+#include "logging.h"
+
+#define DCACHE_ENTRY_CALLOC() calloc(1, sizeof(struct dcache_entry))
+#define DCACHE_ENTRY_LIFE 8
+
+struct dcache_entry root_entry;
+
+int dcache_init_root(uint32_t nid)
+{
+ if (root_entry.nid)
+ return -1;
+
+ /* Root entry doesn't need most of the fields. Namely, it only uses the
+ * nid field and the subdirs pointer.
+ */
+ logi("Initializing root_entry dcache entry");
+ root_entry.nid = nid;
+ root_entry.subdirs = NULL;
+ root_entry.siblings = NULL;
+
+ return 0;
+}
+
+/* Inserts a node as a subdirs of a given parent. The parent is updated to
+ * point the newly inserted subdirs as the first subdirs. We return the new
+ * entry so that further entries can be inserted.
+ *
+ * [0] [0]
+ * / ==> \
+ * / ==> \
+ * .->[1]->[2]-. .->[1]->[3]->[2]-.
+ * `-----------麓 `----------------麓
+ */
+struct dcache_entry *dcache_insert(struct dcache_entry *parent,
+ const char *name, int namelen, uint32_t nid)
+{
+ struct dcache_entry *new_entry;
+
+ logd("Inserting %s,%d to dcache", name, namelen);
+
+ /* TODO: Deal with names that exceed the allocated size */
+ if (namelen + 1 > DCACHE_ENTRY_NAME_LEN)
+ return NULL;
+
+ if (parent == NULL)
+ parent = &root_entry;
+
+ new_entry = DCACHE_ENTRY_CALLOC();
+ if (!new_entry)
+ return NULL;
+
+ strncpy(new_entry->name, name, namelen);
+ new_entry->name[namelen] = 0;
+ new_entry->nid = nid;
+
+ if (!parent->subdirs) {
+ new_entry->siblings = new_entry;
+ parent->subdirs = new_entry;
+ } else {
+ new_entry->siblings = parent->subdirs->siblings;
+ parent->subdirs->siblings = new_entry;
+ parent->subdirs = new_entry;
+ }
+
+ return new_entry;
+}
+
+/* Lookup a cache entry for a given file name. Return value is a struct pointer
+ * that can be used to both obtain the nid number and insert further child
+ * entries.
+ * TODO: Prune entries by using the LRU counter
+ */
+struct dcache_entry *dcache_lookup(struct dcache_entry *parent,
+ const char *name, int namelen)
+{
+ struct dcache_entry *iter;
+
+ if (parent == NULL)
+ parent = &root_entry;
+
+ if (!parent->subdirs)
+ return NULL;
+
+ /* Iterate the list of siblings to see if there is any match */
+ iter = parent->subdirs;
+
+ do {
+ if (strncmp(iter->name, name, namelen) == 0 &&
+ iter->name[namelen] == 0) {
+ parent->subdirs = iter;
+
+ return iter;
+ }
+
+ iter = iter->siblings;
+ } while (iter != parent->subdirs);
+
+ return NULL;
+}
+
+struct dcache_entry *dcache_try_insert(struct dcache_entry *parent,
+ const char *name, int namelen, uint32_t nid)
+{
+ struct dcache_entry *d = dcache_lookup(parent, name, namelen);
+
+ if (d)
+ return d;
+
+ return dcache_insert(parent, name, namelen, nid);
+
+}
+erofs_nid_t dcache_get_nid(struct dcache_entry *entry)
+{
+ return entry ? entry->nid : root_entry.nid;
+}
+
+struct dcache_entry *dcache_root(void)
+{
+ return &root_entry;
+}
+
diff --git a/fuse/dentry.h b/fuse/dentry.h
new file mode 100644
index 000000000000..ee2144de8a89
--- /dev/null
+++ b/fuse/dentry.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\dentry.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef _EROFS_DENTRY_H
+#define _EROFS_DENTRY_H
+
+#include <stdint.h>
+#include "erofs/internal.h"
+
+#ifdef __64BITS
+#define DCACHE_ENTRY_NAME_LEN 40
+#else
+#define DCACHE_ENTRY_NAME_LEN 48
+#endif
+
+/* This struct declares a node of a k-tree. Every node has a pointer to one of
+ * the subdirs and a pointer (in a circular list fashion) to its siblings.
+ */
+
+struct dcache_entry {
+ struct dcache_entry *subdirs;
+ struct dcache_entry *siblings;
+ uint32_t nid;
+ uint16_t lru_count;
+ uint8_t user_count;
+ char name[DCACHE_ENTRY_NAME_LEN];
+};
+
+struct dcache_entry *dcache_insert(struct dcache_entry *parent,
+ const char *name, int namelen, uint32_t n);
+struct dcache_entry *dcache_lookup(struct dcache_entry *parent,
+ const char *name, int namelen);
+struct dcache_entry *dcache_try_insert(struct dcache_entry *parent,
+ const char *name, int namelen,
+ uint32_t nid);
+
+erofs_nid_t dcache_get_nid(struct dcache_entry *entry);
+int dcache_init_root(uint32_t n);
+#endif
diff --git a/fuse/disk_io.c b/fuse/disk_io.c
new file mode 100644
index 000000000000..72d351b17806
--- /dev/null
+++ b/fuse/disk_io.c
@@ -0,0 +1,72 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\disk_io.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#define _XOPEN_SOURCE 500
+#include "disk_io.h"
+
+#include <sys/types.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <errno.h>
+#include <pthread.h>
+#include <errno.h>
+
+#include "logging.h"
+
+#ifdef __FreeBSD__
+#include <string.h>
+#endif
+
+static const char *erofs_devname;
+static int erofs_devfd = -1;
+static pthread_mutex_t read_lock = PTHREAD_MUTEX_INITIALIZER;
+
+int dev_open(const char *path)
+{
+ int fd = open(path, O_RDONLY);
+
+ if (fd < 0)
+ return -errno;
+
+ erofs_devfd = fd;
+ erofs_devname = path;
+
+ return 0;
+}
+
+static inline int pread_wrapper(int fd, void *buf, size_t count, off_t offset)
+{
+ return pread(fd, buf, count, offset);
+}
+
+int dev_read(void *buf, size_t count, off_t offset)
+{
+ ssize_t pread_ret;
+ int lerrno;
+
+ ASSERT(erofs_devfd >= 0);
+
+ pthread_mutex_lock(&read_lock);
+ pread_ret = pread_wrapper(erofs_devfd, buf, count, offset);
+ lerrno = errno;
+ logd("Disk Read: offset[0x%jx] count[%zd] pread_ret=%zd %s",
+ offset, count, pread_ret, strerror(lerrno));
+ pthread_mutex_unlock(&read_lock);
+ if (count == 0)
+ logw("Read operation with 0 size");
+
+ ASSERT((size_t)pread_ret == count);
+
+ return pread_ret;
+}
+
+void dev_close(void)
+{
+ if (erofs_devfd >= 0) {
+ close(erofs_devfd);
+ erofs_devfd = -1;
+ }
+}
diff --git a/fuse/disk_io.h b/fuse/disk_io.h
new file mode 100644
index 000000000000..6b4bd3cce085
--- /dev/null
+++ b/fuse/disk_io.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\disk_io.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __DISK_IO_H
+#define __DISK_IO_H
+
+#include "erofs/defs.h"
+#include "erofs/internal.h"
+
+int dev_open(const char *path);
+void dev_close(void);
+int dev_read(void *buf, size_t count, off_t offset);
+
+static inline int dev_read_blk(void *buf, uint32_t nr)
+{
+ return dev_read(buf, EROFS_BLKSIZ, blknr_to_addr(nr));
+}
+#endif
diff --git a/fuse/getattr.c b/fuse/getattr.c
new file mode 100644
index 000000000000..542cf35f6989
--- /dev/null
+++ b/fuse/getattr.c
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\getattr.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "getattr.h"
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <errno.h>
+
+#include "erofs/defs.h"
+#include "erofs/internal.h"
+#include "erofs_fs.h"
+
+#include "logging.h"
+#include "namei.h"
+
+extern struct erofs_super_block super;
+
+/* GNU's definitions of the attributes
+ * (http://www.gnu.org/software/libc/manual/html_node/Attribute-Meanings.html):
+ * st_uid: The user ID of the file鈥檚 owner.
+ * st_gid: The group ID of the file.
+ * st_atime: This is the last access time for the file.
+ * st_mtime: This is the time of the last modification to the contents of the
+ * file.
+ * st_mode: Specifies the mode of the file. This includes file type information
+ * (see Testing File Type) and the file permission bits (see Permission
+ * Bits).
+ * st_nlink: The number of hard links to the file.This count keeps track of how
+ * many directories have entries for this file. If the count is ever
+ * decremented to zero, then the file itself is discarded as soon as
+ * no process still holds it open. Symbolic links are not counted in
+ * the total.
+ * st_size: This specifies the size of a regular file in bytes. For files that
+ * are really devices this field isn鈥檛 usually meaningful.For symbolic
+ * links this specifies the length of the file name the link refers to.
+ */
+int erofs_getattr(const char *path, struct stat *stbuf)
+{
+ struct erofs_vnode v;
+ int ret;
+
+ logd("getattr(%s)", path);
+ memset(&v, 0, sizeof(v));
+ ret = erofs_iget_by_path(path, &v);
+ if (ret)
+ return -ENOENT;
+
+ stbuf->st_mode = le16_to_cpu(v.i_mode);
+ stbuf->st_nlink = le16_to_cpu(v.i_nlink);
+ stbuf->st_size = le32_to_cpu(v.i_size);
+ stbuf->st_blocks = stbuf->st_size / EROFS_BLKSIZ;
+ stbuf->st_uid = le16_to_cpu(v.i_uid);
+ stbuf->st_gid = le16_to_cpu(v.i_gid);
+ stbuf->st_atime = super.build_time;
+ stbuf->st_mtime = super.build_time;
+ stbuf->st_ctime = super.build_time;
+
+ return 0;
+}
diff --git a/fuse/getattr.h b/fuse/getattr.h
new file mode 100644
index 000000000000..dbcff7c1a6e1
--- /dev/null
+++ b/fuse/getattr.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\getattr.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __EROFS_GETATTR_H
+#define __EROFS_GETATTR_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+
+int erofs_getattr(const char *path, struct stat *st);
+
+#endif
diff --git a/fuse/init.c b/fuse/init.c
new file mode 100644
index 000000000000..8198fa78da96
--- /dev/null
+++ b/fuse/init.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\init.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "init.h"
+#include <string.h>
+#include <asm-generic/errno-base.h>
+
+#include "namei.h"
+#include "disk_io.h"
+#include "logging.h"
+
+#define STR(_X) (#_X)
+#define SUPER_MEM(_X) (super._X)
+
+
+struct erofs_super_block super;
+static struct erofs_super_block *sbk = &super;
+
+int erofs_init_super(void)
+{
+ int ret;
+ char buf[EROFS_BLKSIZ];
+ struct erofs_super_block *sb;
+
+ memset(buf, 0, sizeof(buf));
+ ret = dev_read_blk(buf, 0);
+ if (ret != EROFS_BLKSIZ) {
+ logi("Failed to read super block ret=%d", ret);
+ return -EINVAL;
+ }
+
+ sb = (struct erofs_super_block *) (buf + BOOT_SECTOR_SIZE);
+ sbk->magic = le32_to_cpu(sb->magic);
+ if (sbk->magic != EROFS_SUPER_MAGIC_V1) {
+ logi("EROFS magic[0x%X] NOT matched to [0x%X] ",
+ super.magic, EROFS_SUPER_MAGIC_V1);
+ return -EINVAL;
+ }
+
+ sbk->checksum = le32_to_cpu(sb->checksum);
+ sbk->feature_compat = le32_to_cpu(sb->feature_compat);
+ sbk->blkszbits = sb->blkszbits;
+ ASSERT(sbk->blkszbits != 32);
+
+ sbk->inos = le64_to_cpu(sb->inos);
+ sbk->build_time = le64_to_cpu(sb->build_time);
+ sbk->build_time_nsec = le32_to_cpu(sb->build_time_nsec);
+ sbk->blocks = le32_to_cpu(sb->blocks);
+ sbk->meta_blkaddr = le32_to_cpu(sb->meta_blkaddr);
+ sbk->xattr_blkaddr = le32_to_cpu(sb->xattr_blkaddr);
+ memcpy(sbk->uuid, sb->uuid, 16);
+ memcpy(sbk->volume_name, sb->volume_name, 16);
+ sbk->root_nid = le16_to_cpu(sb->root_nid);
+
+ logp("%-15s:0x%X", STR(magic), SUPER_MEM(magic));
+ logp("%-15s:0x%X", STR(feature_compat), SUPER_MEM(feature_compat));
+ logp("%-15s:%u", STR(blkszbits), SUPER_MEM(blkszbits));
+ logp("%-15s:%u", STR(root_nid), SUPER_MEM(root_nid));
+ logp("%-15s:%ul", STR(inos), SUPER_MEM(inos));
+ logp("%-15s:%d", STR(meta_blkaddr), SUPER_MEM(meta_blkaddr));
+ logp("%-15s:%d", STR(xattr_blkaddr), SUPER_MEM(xattr_blkaddr));
+
+ return 0;
+}
+
+erofs_nid_t erofs_get_root_nid(void)
+{
+ return sbk->root_nid;
+}
+
+erofs_nid_t addr2nid(erofs_off_t addr)
+{
+ erofs_nid_t offset = (erofs_nid_t)sbk->meta_blkaddr * EROFS_BLKSIZ;
+
+ ASSERT(IS_SLOT_ALIGN(addr));
+ return (addr - offset) >> EROFS_ISLOTBITS;
+}
+
+erofs_off_t nid2addr(erofs_nid_t nid)
+{
+ erofs_off_t offset = (erofs_off_t)sbk->meta_blkaddr * EROFS_BLKSIZ;
+
+ return (nid << EROFS_ISLOTBITS) + offset;
+}
+
+void *erofs_init(struct fuse_conn_info *info)
+{
+ logi("Using FUSE protocol %d.%d", info->proto_major, info->proto_minor);
+
+ if (inode_init(erofs_get_root_nid()) != 0) {
+ loge("inode initialization failed")
+ ABORT();
+ }
+ return NULL;
+}
diff --git a/fuse/init.h b/fuse/init.h
new file mode 100644
index 000000000000..d7a97b5ee043
--- /dev/null
+++ b/fuse/init.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\init.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __EROFS_INIT_H
+#define __EROFS_INIT_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+#include "erofs/internal.h"
+
+#define BOOT_SECTOR_SIZE 0x400
+
+int erofs_init_super(void);
+erofs_nid_t erofs_get_root_nid(void);
+erofs_off_t nid2addr(erofs_nid_t nid);
+erofs_nid_t addr2nid(erofs_off_t addr);
+void *erofs_init(struct fuse_conn_info *info);
+
+#endif
diff --git a/fuse/logging.c b/fuse/logging.c
new file mode 100644
index 000000000000..2d1f1c77c2a4
--- /dev/null
+++ b/fuse/logging.c
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\logging.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "logging.h"
+
+#include <stdio.h>
+#include <stdarg.h>
+#include <pthread.h>
+
+static int loglevel = DEFAULT_LOG_LEVEL;
+static FILE *logfile;
+static const char * const loglevel_str[] = {
+ [LOG_EMERG] = "[emerg]",
+ [LOG_ALERT] = "[alert]",
+ [LOG_DUMP] = "[dump] ",
+ [LOG_ERR] = "[err] ",
+ [LOG_WARNING] = "[warn] ",
+ [LOG_NOTICE] = "[notic]",
+ [LOG_INFO] = "[info] ",
+ [LOG_DEBUG] = "[debug]",
+};
+
+void __LOG(int level, const char *func, int line, const char *format, ...)
+{
+ static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
+ va_list ap;
+ FILE *fp = logfile ? logfile : stdout;
+
+ if (!fp)
+ return;
+ if (level < 0 || level > loglevel)
+ return;
+
+ /* We have a lock here so different threads don interleave the log
+ * output
+ */
+ pthread_mutex_lock(&lock);
+ va_start(ap, format);
+ fprintf(fp, "%s", loglevel_str[level]);
+ if (func)
+ fprintf(fp, "%s", func);
+ if (line >= 0)
+ fprintf(fp, "(%d):", line);
+ vfprintf(fp, format, ap);
+ fprintf(fp, "\n");
+ va_end(ap);
+ fflush(fp);
+ pthread_mutex_unlock(&lock);
+}
+
+inline void logging_setlevel(int new_level)
+{
+ loglevel = new_level;
+}
+
+int logging_open(const char *path)
+{
+ if (path == NULL)
+ return 0;
+
+ logfile = fopen(path, "w");
+ if (logfile == NULL) {
+ perror("open");
+ return -1;
+ }
+
+ return 0;
+}
+
+void logging_close(void)
+{
+ if (logfile) {
+ fflush(logfile);
+ fclose(logfile);
+ logfile = NULL;
+ }
+}
+
diff --git a/fuse/logging.h b/fuse/logging.h
new file mode 100644
index 000000000000..7aa2eda405db
--- /dev/null
+++ b/fuse/logging.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\logging.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __LOGGING_H
+#define __LOGGING_H
+
+#include <assert.h>
+#include <stdlib.h>
+
+#define LOG_EMERG 0
+#define LOG_ALERT 1
+#define LOG_DUMP 2
+#define LOG_ERR 3
+#define LOG_WARNING 4
+#define LOG_NOTICE 5
+#define LOG_INFO 6
+#define LOG_DEBUG 7
+
+#define logem(...) __LOG(LOG_EMERG, __func__, __LINE__, ##__VA_ARGS__)
+#define loga(...) __LOG(LOG_ALERT, __func__, __LINE__, ##__VA_ARGS__)
+#define loge(...) __LOG(LOG_ERR, __func__, __LINE__, ##__VA_ARGS__)
+#define logw(...) __LOG(LOG_WARNING, __func__, __LINE__, ##__VA_ARGS__)
+#define logn(...) __LOG(LOG_NOTICE, __func__, __LINE__, ##__VA_ARGS__)
+#define logi(...) __LOG(LOG_INFO, __func__, __LINE__, ##__VA_ARGS__)
+#define logp(...) __LOG(LOG_DUMP, "", -1, ##__VA_ARGS__)
+#define logd(...) __LOG(LOG_DEBUG, __func__, __LINE__, ##__VA_ARGS__)
+
+#define DEFAULT_LOG_FILE "fuse.log"
+
+#ifdef _DEBUG
+#define DEFAULT_LOG_LEVEL LOG_DEBUG
+
+#define ASSERT(assertion) ({ \
+ if (!(assertion)) { \
+ logw("ASSERT FAIL: " #assertion); \
+ assert(assertion); \
+ } \
+})
+#define ABORT(_X) abort(_X)
+#else
+#define DEFAULT_LOG_LEVEL LOG_ERR
+#define ASSERT(assertion)
+#define ABORT(_X)
+#endif
+
+void __LOG(int level, const char *func, int line, const char *format, ...);
+void logging_setlevel(int new_level);
+int logging_open(const char *path);
+void logging_close(void);
+
+#endif
+
diff --git a/fuse/main.c b/fuse/main.c
new file mode 100644
index 000000000000..fa9795ef1615
--- /dev/null
+++ b/fuse/main.c
@@ -0,0 +1,170 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\main.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <execinfo.h>
+#include <signal.h>
+#include <stddef.h>
+
+#include "logging.h"
+#include "init.h"
+#include "read.h"
+#include "getattr.h"
+#include "open.h"
+#include "readir.h"
+#include "disk_io.h"
+
+enum {
+ EROFS_OPT_HELP,
+ EROFS_OPT_VER,
+};
+
+struct options {
+ const char *disk;
+ const char *mount;
+ const char *logfile;
+ unsigned int debug_lvl;
+};
+static struct options cfg;
+
+#define OPTION(t, p) { t, offsetof(struct options, p), 1 }
+
+static const struct fuse_opt option_spec[] = {
+ OPTION("--log=%s", logfile),
+ OPTION("--dbg=%u", debug_lvl),
+ FUSE_OPT_KEY("-h", EROFS_OPT_HELP),
+ FUSE_OPT_KEY("-v", EROFS_OPT_VER),
+ FUSE_OPT_END
+};
+
+static void usage(void)
+{
+ fprintf(stderr, "\terofsfuse [options] <image> <mountpoint>\n");
+ fprintf(stderr, "\t --log=<file> output log file\n");
+ fprintf(stderr, "\t --dbg=<level> log debug level 0 ~ 7\n");
+ fprintf(stderr, "\t -h show help\n");
+ fprintf(stderr, "\t -v show version\n");
+ exit(1);
+}
+
+static void dump_cfg(void)
+{
+ fprintf(stderr, "\tdisk :%s\n", cfg.disk);
+ fprintf(stderr, "\tmount:%s\n", cfg.mount);
+ fprintf(stderr, "\tdebug_lvl:%u\n", cfg.debug_lvl);
+ fprintf(stderr, "\tlogfile :%s\n", cfg.logfile);
+}
+
+static int optional_opt_func(void *data, const char *arg, int key,
+ struct fuse_args *outargs)
+{
+ UNUSED(data);
+ UNUSED(outargs);
+
+ switch (key) {
+ case FUSE_OPT_KEY_OPT:
+ return 1;
+
+ case FUSE_OPT_KEY_NONOPT:
+ if (!cfg.disk) {
+ cfg.disk = strdup(arg);
+ return 0;
+ } else if (!cfg.mount)
+ cfg.mount = strdup(arg);
+
+ return 1;
+ case EROFS_OPT_HELP:
+ usage();
+ break;
+
+ case EROFS_OPT_VER:
+ fprintf(stderr, "EROFS FUSE VERSION v 1.0.0\n");
+ exit(0);
+ }
+
+ return 1;
+}
+
+static void signal_handle_sigsegv(int signal)
+{
+ void *array[10];
+ size_t nptrs;
+ char **strings;
+ size_t i;
+
+ UNUSED(signal);
+ logd("========================================");
+ logd("Segmentation Fault. Starting backtrace:");
+ nptrs = backtrace(array, 10);
+ strings = backtrace_symbols(array, nptrs);
+ if (strings) {
+ for (i = 0; i < nptrs; i++)
+ logd("%s", strings[i]);
+ free(strings);
+ }
+ logd("========================================");
+
+ abort();
+}
+
+static struct fuse_operations erofs_ops = {
+ .getattr = erofs_getattr,
+ .readdir = erofs_readdir,
+ .open = erofs_open,
+ .read = erofs_read,
+ .init = erofs_init,
+};
+
+int main(int argc, char *argv[])
+{
+ int ret = EXIT_FAILURE;
+ struct fuse_args args = FUSE_ARGS_INIT(argc, argv);
+
+ if (signal(SIGSEGV, signal_handle_sigsegv) == SIG_ERR) {
+ fprintf(stderr, "Failed to initialize signals\n");
+ return EXIT_FAILURE;
+ }
+
+ /* Parse options */
+ if (fuse_opt_parse(&args, &cfg, option_spec, optional_opt_func) < 0)
+ return 1;
+
+ dump_cfg();
+
+ if (logging_open(cfg.logfile) < 0) {
+ fprintf(stderr, "Failed to initialize logging\n");
+ goto exit;
+ }
+
+ logging_setlevel(cfg.debug_lvl);
+
+ if (dev_open(cfg.disk) < 0) {
+ fprintf(stderr, "Failed to open disk:%s\n", cfg.disk);
+ goto exit_log;
+ }
+
+ if (erofs_init_super()) {
+ fprintf(stderr, "Failed to read erofs super block\n");
+ goto exit_dev;
+ }
+
+ logi("fuse start");
+
+ ret = fuse_main(args.argc, args.argv, &erofs_ops, NULL);
+
+ logi("fuse done ret=%d", ret);
+
+exit_dev:
+ dev_close();
+exit_log:
+ logging_close();
+exit:
+ fuse_opt_free_args(&args);
+ return ret;
+}
+
diff --git a/fuse/namei.c b/fuse/namei.c
new file mode 100644
index 000000000000..3503a8d56e15
--- /dev/null
+++ b/fuse/namei.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\namei.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "namei.h"
+#include <linux/kdev_t.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <errno.h>
+#include <sys/stat.h>
+
+#include "erofs/defs.h"
+#include "logging.h"
+#include "disk_io.h"
+#include "dentry.h"
+#include "init.h"
+
+#define IS_PATH_SEPARATOR(__c) ((__c) == '/')
+#define MINORBITS 20
+#define MINORMASK ((1U << MINORBITS) - 1)
+#define DT_UNKNOWN 0
+
+static const char *skip_trailing_backslash(const char *path)
+{
+ while (IS_PATH_SEPARATOR(*path))
+ path++;
+ return path;
+}
+
+static uint8_t get_path_token_len(const char *path)
+{
+ uint8_t len = 0;
+
+ while (path[len] != '/' && path[len])
+ len++;
+ return len;
+}
+
+int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
+{
+ int ret;
+ char buf[EROFS_BLKSIZ];
+ struct erofs_inode_compact *v1;
+ const erofs_off_t addr = nid2addr(nid);
+ const size_t size = EROFS_BLKSIZ - erofs_blkoff(addr);
+
+ ret = dev_read(buf, size, addr);
+ if (ret != (int)size)
+ return -EIO;
+
+ v1 = (struct erofs_inode_compact *)buf;
+ vi->datalayout = __inode_data_mapping(le16_to_cpu(v1->i_format));
+ vi->inode_isize = sizeof(struct erofs_inode_compact);
+ vi->xattr_isize = erofs_xattr_ibody_size(v1->i_xattr_icount);
+ vi->i_size = le32_to_cpu(v1->i_size);
+ vi->i_mode = le16_to_cpu(v1->i_mode);
+ vi->i_uid = le16_to_cpu(v1->i_uid);
+ vi->i_gid = le16_to_cpu(v1->i_gid);
+ vi->i_nlink = le16_to_cpu(v1->i_nlink);
+ vi->nid = nid;
+
+ switch (vi->i_mode & S_IFMT) {
+ case S_IFBLK:
+ case S_IFCHR:
+ /* fixme: add special devices support
+ * vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
+ */
+ break;
+ case S_IFIFO:
+ case S_IFSOCK:
+ /*fixme: vi->i_rdev = 0; */
+ break;
+ case S_IFREG:
+ case S_IFLNK:
+ case S_IFDIR:
+ vi->raw_blkaddr = le32_to_cpu(v1->i_u.raw_blkaddr);
+ break;
+ default:
+ return -EIO;
+ }
+
+ return 0;
+}
+
+/* dirent + name string */
+struct dcache_entry *list_name(const char *buf, struct dcache_entry *parent,
+ const char *name, unsigned int len,
+ uint32_t dirend)
+{
+ struct dcache_entry *entry = NULL;
+ struct erofs_dirent *ds, *de;
+
+ ds = (struct erofs_dirent *)buf;
+ de = (struct erofs_dirent *)(buf + le16_to_cpu(ds->nameoff));
+
+ while (ds < de) {
+ erofs_nid_t nid = le64_to_cpu(ds->nid);
+ uint16_t nameoff = le16_to_cpu(ds->nameoff);
+ char *d_name = (char *)(buf + nameoff);
+ uint16_t name_len = (ds + 1 >= de) ?
+ (uint16_t)strnlen(d_name, dirend - nameoff) :
+ le16_to_cpu(ds[1].nameoff) - nameoff;
+
+ #if defined(EROFS_DEBUG_ENTRY)
+ {
+ char debug[EROFS_BLKSIZ];
+
+ memcpy(debug, d_name, name_len);
+ debug[name_len] = '\0';
+ logi("list entry: %s nid=%u", debug, nid);
+ }
+ #endif
+
+ entry = dcache_try_insert(parent, d_name, name_len, nid);
+ if (len == name_len && !memcmp(name, d_name, name_len))
+ return entry;
+
+ entry = NULL;
+ ++ds;
+ }
+
+ return entry;
+}
+
+struct dcache_entry *disk_lookup(struct dcache_entry *parent, const char *name,
+ unsigned int name_len)
+{
+ int ret;
+ char buf[EROFS_BLKSIZ];
+ struct dcache_entry *entry = NULL;
+ struct erofs_vnode v;
+ uint32_t nr_cnt, dir_nr, dirsize, blkno;
+
+ ret = erofs_iget_by_nid(parent->nid, &v);
+ if (ret)
+ return NULL;
+
+ /* to check whether dirent is in the inline dirs */
+ blkno = v.raw_blkaddr;
+ dirsize = v.i_size;
+ dir_nr = erofs_blknr(dirsize);
+
+ nr_cnt = 0;
+ while (nr_cnt < dir_nr) {
+ if (dev_read_blk(buf, blkno + nr_cnt) != EROFS_BLKSIZ)
+ return NULL;
+
+ entry = list_name(buf, parent, name, name_len, EROFS_BLKSIZ);
+ if (entry)
+ goto next;
+
+ ++nr_cnt;
+ }
+
+ if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
+ uint32_t dir_off = erofs_blkoff(dirsize);
+ off_t dir_addr = nid2addr(dcache_get_nid(parent))
+ + sizeof(struct erofs_inode_compact) + v.xattr_isize;
+
+ memset(buf, 0, sizeof(buf));
+ ret = dev_read(buf, dir_off, dir_addr);
+ if (ret < 0 && (uint32_t)ret != dir_off)
+ return NULL;
+
+ entry = list_name(buf, parent, name, name_len, dir_off);
+ }
+next:
+ return entry;
+}
+
+extern struct dcache_entry root_entry;
+int walk_path(const char *_path, erofs_nid_t *out_nid)
+{
+ struct dcache_entry *next, *ret;
+ const char *path = _path;
+
+ ret = next = &root_entry;
+ for (;;) {
+ uint8_t path_len;
+
+ path = skip_trailing_backslash(path);
+ path_len = get_path_token_len(path);
+ ret = next;
+ if (path_len == 0)
+ break;
+
+ next = dcache_lookup(ret, path, path_len);
+ if (!next) {
+ next = disk_lookup(ret, path, path_len);
+ if (!next)
+ return -ENOENT;
+ }
+
+ path += path_len;
+ }
+
+ if (!ret)
+ return -ENOENT;
+ logd("find path = %s nid=%u", _path, ret->nid);
+
+ *out_nid = ret->nid;
+ return 0;
+
+}
+
+int erofs_iget_by_path(const char *path, struct erofs_vnode *v)
+{
+ int ret;
+ erofs_nid_t nid;
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ return erofs_iget_by_nid(nid, v);
+}
+
+int inode_init(erofs_nid_t root)
+{
+ dcache_init_root(root);
+
+ return 0;
+}
+
diff --git a/fuse/namei.h b/fuse/namei.h
new file mode 100644
index 000000000000..80e84d7220aa
--- /dev/null
+++ b/fuse/namei.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\inode.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __INODE_H
+#define __INODE_H
+
+#include "erofs/internal.h"
+#include "erofs_fs.h"
+
+int inode_init(erofs_nid_t root);
+struct dcache_entry *get_cached_dentry(struct dcache_entry **parent,
+ const char **path);
+int erofs_iget_by_path(const char *path, struct erofs_vnode *v);
+int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *v);
+struct dcache_entry *disk_lookup(struct dcache_entry *parent, const char *name,
+ unsigned int name_len);
+int walk_path(const char *path, erofs_nid_t *out_nid);
+
+#endif
diff --git a/fuse/open.c b/fuse/open.c
new file mode 100644
index 000000000000..9d2edca54d23
--- /dev/null
+++ b/fuse/open.c
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\open.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "open.h"
+#include <asm-generic/errno-base.h>
+#include <fuse.h>
+#include <fuse_opt.h>
+#include "logging.h"
+
+int erofs_open(const char *path, struct fuse_file_info *fi)
+{
+ logi("open path=%s", path);
+
+ if ((fi->flags & O_ACCMODE) != O_RDONLY)
+ return -EACCES;
+
+ return 0;
+}
+
diff --git a/fuse/open.h b/fuse/open.h
new file mode 100644
index 000000000000..d02c1f752204
--- /dev/null
+++ b/fuse/open.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\open.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __EROFS_OPEN_H
+#define __EROFS_OPEN_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+
+int erofs_open(const char *path, struct fuse_file_info *fi);
+
+#endif
diff --git a/fuse/read.c b/fuse/read.c
new file mode 100644
index 000000000000..ffe976e59a11
--- /dev/null
+++ b/fuse/read.c
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\read.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "read.h"
+#include <errno.h>
+#include <linux/fs.h>
+#include <sys/stat.h>
+#include <string.h>
+
+#include "erofs/defs.h"
+#include "erofs/internal.h"
+#include "logging.h"
+#include "namei.h"
+#include "disk_io.h"
+#include "init.h"
+
+size_t erofs_read_data(struct erofs_vnode *vnode, char *buffer,
+ size_t size, off_t offset)
+{
+ int ret;
+ size_t sum, rdsz = 0;
+ uint32_t addr = blknr_to_addr(vnode->raw_blkaddr) + offset;
+
+ sum = (offset + size) > vnode->i_size ?
+ (size_t)(vnode->i_size - offset) : size;
+ while (rdsz < sum) {
+ size_t count = min(EROFS_BLKSIZ, (uint32_t)(sum - rdsz));
+
+ ret = dev_read(buffer + rdsz, count, addr + rdsz);
+ if (ret < 0 || (size_t)ret != count)
+ return -EIO;
+ rdsz += count;
+ }
+
+ logi("nid:%u size=%zd offset=%llu realsize=%zd done",
+ vnode->nid, size, (long long)offset, rdsz);
+ return rdsz;
+
+}
+
+size_t erofs_read_data_inline(struct erofs_vnode *vnode, char *buffer,
+ size_t size, off_t offset)
+{
+ int ret;
+ size_t sum, suminline, rdsz = 0;
+ uint32_t addr = blknr_to_addr(vnode->raw_blkaddr) + offset;
+ uint32_t szblk = vnode->i_size - erofs_blkoff(vnode->i_size);
+
+ sum = (offset + size) > szblk ? (size_t)(szblk - offset) : size;
+ suminline = size - sum;
+
+ while (rdsz < sum) {
+ size_t count = min(EROFS_BLKSIZ, (uint32_t)(sum - rdsz));
+
+ ret = dev_read(buffer + rdsz, count, addr + rdsz);
+ if (ret < 0 || (uint32_t)ret != count)
+ return -EIO;
+ rdsz += count;
+ }
+
+ if (!suminline)
+ goto finished;
+
+ addr = nid2addr(vnode->nid) + sizeof(struct erofs_inode_compact)
+ + vnode->xattr_isize;
+ ret = dev_read(buffer + rdsz, suminline, addr);
+ if (ret < 0 || (size_t)ret != suminline)
+ return -EIO;
+ rdsz += suminline;
+
+finished:
+ logi("nid:%u size=%zd suminline=%u offset=%llu realsize=%zd done",
+ vnode->nid, size, suminline, (long long)offset, rdsz);
+ return rdsz;
+
+}
+
+
+int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
+ struct fuse_file_info *fi)
+{
+ int ret;
+ erofs_nid_t nid;
+ struct erofs_vnode v;
+
+ UNUSED(fi);
+ logi("path:%s size=%zd offset=%llu", path, size, (long long)offset);
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ ret = erofs_iget_by_nid(nid, &v);
+ if (ret)
+ return ret;
+
+ logi("path:%s nid=%llu mode=%u", path, nid, v.datalayout);
+ switch (v.datalayout) {
+ case EROFS_INODE_FLAT_PLAIN:
+ return erofs_read_data(&v, buffer, size, offset);
+
+ case EROFS_INODE_FLAT_INLINE:
+ return erofs_read_data_inline(&v, buffer, size, offset);
+
+ case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
+ case EROFS_INODE_FLAT_COMPRESSION:
+ /* Fixme: */
+ default:
+ return -EINVAL;
+ }
+}
diff --git a/fuse/read.h b/fuse/read.h
new file mode 100644
index 000000000000..f2fcebec9d07
--- /dev/null
+++ b/fuse/read.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\read.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __EROFS_READ_H
+#define __EROFS_READ_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+
+int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
+ struct fuse_file_info *fi);
+
+#endif
diff --git a/fuse/readir.c b/fuse/readir.c
new file mode 100644
index 000000000000..9589685c9122
--- /dev/null
+++ b/fuse/readir.c
@@ -0,0 +1,123 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * erofs-fuse\readir.c
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#include "readir.h"
+#include <errno.h>
+#include <linux/fs.h>
+#include <sys/stat.h>
+
+#include "erofs/defs.h"
+#include "erofs/internal.h"
+#include "erofs_fs.h"
+#include "namei.h"
+#include "disk_io.h"
+#include "logging.h"
+#include "init.h"
+
+erofs_nid_t split_entry(char *entry, off_t ofs, char *end, char *name,
+ uint32_t dirend)
+{
+ struct erofs_dirent *de = (struct erofs_dirent *)(entry + ofs);
+ uint16_t nameoff = le16_to_cpu(de->nameoff);
+ const char *de_name = entry + nameoff;
+ uint32_t de_namelen;
+
+ if ((void *)(de + 1) >= (void *)end)
+ de_namelen = strnlen(de_name, dirend - nameoff);
+ else
+ de_namelen = le16_to_cpu(de[1].nameoff) - nameoff;
+
+ memcpy(name, de_name, de_namelen);
+ name[de_namelen] = '\0';
+ return le64_to_cpu(de->nid);
+}
+
+int fill_dir(char *entrybuf, fuse_fill_dir_t filler, void *buf,
+ uint32_t dirend)
+{
+ uint32_t ofs;
+ uint16_t nameoff;
+ char *end;
+ char name[EROFS_BLKSIZ];
+
+ nameoff = le16_to_cpu(((struct erofs_dirent *)entrybuf)->nameoff);
+ end = entrybuf + nameoff;
+ ofs = 0;
+ while (ofs < nameoff) {
+ (void)split_entry(entrybuf, ofs, end, name, dirend);
+ filler(buf, name, NULL, 0);
+ ofs += sizeof(struct erofs_dirent);
+ }
+
+ return 0;
+}
+
+/** Function to add an entry in a readdir() operation
+ *
+ * @param buf the buffer passed to the readdir() operation
+ * @param name the file name of the directory entry
+ * @param stat file attributes, can be NULL
+ * @param off offset of the next entry or zero
+ * @return 1 if buffer is full, zero otherwise
+ */
+int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi)
+{
+ int ret;
+ erofs_nid_t nid;
+ struct erofs_vnode v;
+ char dirsbuf[EROFS_BLKSIZ];
+ uint32_t dir_nr, dir_off, nr_cnt;
+
+ logd("readdir:%s offset=%llu", path, (long long)offset);
+ UNUSED(fi);
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ logd("path=%s nid = %u", path, nid);
+ ret = erofs_iget_by_nid(nid, &v);
+ if (ret)
+ return ret;
+
+ if (!S_ISDIR(v.i_mode))
+ return -ENOTDIR;
+
+ if (!v.i_size)
+ return 0;
+
+ dir_nr = erofs_blknr(v.i_size);
+ dir_off = erofs_blkoff(v.i_size);
+ nr_cnt = 0;
+
+ logd("dir_size=%u dir_nr = %u dir_off=%u", v.i_size, dir_nr, dir_off);
+
+ while (nr_cnt < dir_nr) {
+ memset(dirsbuf, 0, sizeof(dirsbuf));
+ ret = dev_read_blk(dirsbuf, v.raw_blkaddr + nr_cnt);
+ if (ret != EROFS_BLKSIZ)
+ return -EIO;
+ fill_dir(dirsbuf, filler, buf, EROFS_BLKSIZ);
+ ++nr_cnt;
+ }
+
+ if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
+ off_t addr;
+
+ addr = nid2addr(nid) + sizeof(struct erofs_inode_compact)
+ + v.xattr_isize;
+
+ memset(dirsbuf, 0, sizeof(dirsbuf));
+ ret = dev_read(dirsbuf, dir_off, addr);
+ if (ret < 0 || (uint32_t)ret != dir_off)
+ return -EIO;
+ fill_dir(dirsbuf, filler, buf, dir_off);
+ }
+
+ return 0;
+}
+
diff --git a/fuse/readir.h b/fuse/readir.h
new file mode 100644
index 000000000000..21ab7a4e4e0b
--- /dev/null
+++ b/fuse/readir.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * erofs-fuse\readir.h
+ * Created by Li Guifu <blucerlee@gmail.com>
+ */
+
+#ifndef __EROFS_READDIR_H
+#define __EROFS_READDIR_H
+
+#include <fuse.h>
+#include <fuse_opt.h>
+
+int erofs_readdir(const char *path, void *buffer, fuse_fill_dir_t filler,
+ off_t offset, struct fuse_file_info *fi);
+
+
+#endif
diff --git a/include/erofs/defs.h b/include/erofs/defs.h
index 33320dbe8e3c..a9c769ec88ae 100644
--- a/include/erofs/defs.h
+++ b/include/erofs/defs.h
@@ -24,6 +24,9 @@
#ifdef HAVE_LINUX_TYPES_H
#include <linux/types.h>
#endif
+#ifndef UNUSED
+#define UNUSED(_X) ((void) _X)
+#endif
/*
* container_of - cast a member of a structure out to the containing structure
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index bc77c43719e8..cba3ce4695d4 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -55,6 +55,8 @@ typedef u32 erofs_blk_t;
#define blknr_to_addr(nr) ((erofs_off_t)(nr) * EROFS_BLKSIZ)
#define BLK_ROUND_UP(addr) DIV_ROUND_UP(addr, EROFS_BLKSIZ)
+#define IS_SLOT_ALIGN(__ADDR) (((__ADDR) % (EROFS_SLOTSIZE)) ? 0 : 1)
+#define IS_BLK_ALIGN(__ADDR) (((__ADDR) % (EROFS_BLKSIZ)) ? 0 : 1)
struct erofs_buffer_head;
@@ -132,11 +134,46 @@ struct erofs_inode {
#endif
};
+struct erofs_vnode {
+ uint8_t datalayout;
+
+ uint32_t i_size;
+ /* inline size in bytes */
+ uint16_t inode_isize;
+ uint16_t xattr_isize;
+
+ uint16_t xattr_shared_count;
+ char *xattr_shared_xattrs;
+
+ erofs_blk_t raw_blkaddr;
+ erofs_nid_t nid;
+ uint32_t i_ino;
+
+ uint16_t i_mode;
+ uint16_t i_uid;
+ uint16_t i_gid;
+ uint16_t i_nlink;
+
+ /* if file is inline read inline data witch inode */
+ char *idata;
+};
+
static inline bool is_inode_layout_compression(struct erofs_inode *inode)
{
return erofs_inode_is_data_compressed(inode->datalayout);
}
+#define __inode_advise(x, bit, bits) \
+ (((x) >> (bit)) & ((1 << (bits)) - 1))
+
+#define __inode_version(advise) \
+ __inode_advise(advise, EROFS_I_VERSION_BIT, \
+ EROFS_I_VERSION_BITS)
+
+#define __inode_data_mapping(advise) \
+ __inode_advise(advise, EROFS_I_DATALAYOUT_BIT,\
+ EROFS_I_DATALAYOUT_BITS)
+
#define IS_ROOT(x) ((x) == (x)->i_parent)
struct erofs_dentry {
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 02/12] erofs-utils: fuse: support read special file
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 01/12] " Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 03/12] erofs-utils: fuse: support read compressed file Gao Xiang via Linux-erofs
` (9 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
From: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/getattr.c | 1 +
fuse/main.c | 1 +
fuse/namei.c | 14 ++++++++++----
fuse/read.c | 26 ++++++++++++++++++++++++++
fuse/read.h | 1 +
include/erofs/internal.h | 1 +
6 files changed, 40 insertions(+), 4 deletions(-)
diff --git a/fuse/getattr.c b/fuse/getattr.c
index 542cf35f6989..d2134f486e19 100644
--- a/fuse/getattr.c
+++ b/fuse/getattr.c
@@ -56,6 +56,7 @@ int erofs_getattr(const char *path, struct stat *stbuf)
stbuf->st_blocks = stbuf->st_size / EROFS_BLKSIZ;
stbuf->st_uid = le16_to_cpu(v.i_uid);
stbuf->st_gid = le16_to_cpu(v.i_gid);
+ stbuf->st_rdev = le32_to_cpu(v.i_rdev);
stbuf->st_atime = super.build_time;
stbuf->st_mtime = super.build_time;
stbuf->st_ctime = super.build_time;
diff --git a/fuse/main.c b/fuse/main.c
index fa9795ef1615..884101374bcf 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -113,6 +113,7 @@ static void signal_handle_sigsegv(int signal)
}
static struct fuse_operations erofs_ops = {
+ .readlink = erofs_readlink,
.getattr = erofs_getattr,
.readdir = erofs_readdir,
.open = erofs_open,
diff --git a/fuse/namei.c b/fuse/namei.c
index 3503a8d56e15..7ed1168fc14f 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -11,6 +11,7 @@
#include <stdio.h>
#include <errno.h>
#include <sys/stat.h>
+#include <sys/sysmacros.h>
#include "erofs/defs.h"
#include "logging.h"
@@ -39,6 +40,13 @@ static uint8_t get_path_token_len(const char *path)
return len;
}
+static inline dev_t new_decode_dev(u32 dev)
+{
+ unsigned major = (dev & 0xfff00) >> 8;
+ unsigned minor = (dev & 0xff) | ((dev >> 12) & 0xfff00);
+ return makedev(major, minor);
+}
+
int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
{
int ret;
@@ -65,13 +73,11 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
switch (vi->i_mode & S_IFMT) {
case S_IFBLK:
case S_IFCHR:
- /* fixme: add special devices support
- * vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
- */
+ vi->i_rdev = new_decode_dev(le32_to_cpu(v1->i_u.rdev));
break;
case S_IFIFO:
case S_IFSOCK:
- /*fixme: vi->i_rdev = 0; */
+ vi->i_rdev = 0;
break;
case S_IFREG:
case S_IFLNK:
diff --git a/fuse/read.c b/fuse/read.c
index ffe976e59a11..3ce5c4f2911e 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -112,3 +112,29 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
return -EINVAL;
}
}
+
+int erofs_readlink(const char *path, char *buffer, size_t size)
+{
+ int ret;
+ erofs_nid_t nid;
+ size_t lnksz;
+ struct erofs_vnode v;
+
+ ret = walk_path(path, &nid);
+ if (ret)
+ return ret;
+
+ ret = erofs_iget_by_nid(nid, &v);
+ if (ret)
+ return ret;
+
+ lnksz = min((size_t)v.i_size, size - 1);
+
+ ret = erofs_read(path, buffer, lnksz, 0, NULL);
+ buffer[lnksz] = '\0';
+ if (ret != (int)lnksz)
+ return ret;
+
+ logi("path:%s link=%s size=%llu", path, buffer, lnksz);
+ return 0;
+}
\ No newline at end of file
diff --git a/fuse/read.h b/fuse/read.h
index f2fcebec9d07..89d4b4cd600c 100644
--- a/fuse/read.h
+++ b/fuse/read.h
@@ -12,5 +12,6 @@
int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
struct fuse_file_info *fi);
+int erofs_readlink(const char *path, char *buffer, size_t size);
#endif
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index cba3ce4695d4..47ad96d0488c 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -153,6 +153,7 @@ struct erofs_vnode {
uint16_t i_uid;
uint16_t i_gid;
uint16_t i_nlink;
+ uint32_t i_rdev;
/* if file is inline read inline data witch inode */
char *idata;
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 03/12] erofs-utils: fuse: support read compressed file
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 01/12] " Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 02/12] erofs-utils: fuse: support read special file Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling Gao Xiang via Linux-erofs
` (8 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
From: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Guo Weichao <guoweichao@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/Makefile.am | 8 +-
fuse/decompress.c | 86 ++++++++
fuse/decompress.h | 37 ++++
fuse/dentry.h | 5 +-
fuse/init.c | 22 ++-
fuse/init.h | 2 +
fuse/namei.c | 11 +-
fuse/read.c | 73 ++++++-
fuse/zmap.c | 416 +++++++++++++++++++++++++++++++++++++++
include/erofs/defs.h | 13 ++
include/erofs/internal.h | 43 +++-
include/erofs_fs.h | 4 +
12 files changed, 711 insertions(+), 9 deletions(-)
create mode 100644 fuse/decompress.c
create mode 100644 fuse/decompress.h
create mode 100644 fuse/zmap.c
diff --git a/fuse/Makefile.am b/fuse/Makefile.am
index fffd67a53fe1..052c7163dff7 100644
--- a/fuse/Makefile.am
+++ b/fuse/Makefile.am
@@ -3,12 +3,16 @@
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = erofsfuse
-erofsfuse_SOURCES = main.c dentry.c getattr.c logging.c namei.c read.c disk_io.c init.c open.c readir.c
+erofsfuse_SOURCES = main.c dentry.c getattr.c logging.c namei.c read.c disk_io.c init.c open.c readir.c zmap.c
+if ENABLE_LZ4
+erofsfuse_SOURCES += decompress.c
+endif
erofsfuse_CFLAGS = -Wall -Werror -Wextra \
+ -Wno-implicit-fallthrough \
-I$(top_srcdir)/include \
$(shell pkg-config fuse --cflags) \
-DFUSE_USE_VERSION=26 \
-std=gnu99
LDFLAGS += $(shell pkg-config fuse --libs)
-erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl
+erofsfuse_LDADD = $(top_builddir)/lib/liberofs.la -ldl ${LZ4_LIBS}
diff --git a/fuse/decompress.c b/fuse/decompress.c
new file mode 100644
index 000000000000..e2df3cea78f7
--- /dev/null
+++ b/fuse/decompress.c
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd.
+ * Created by Huang Jianan <huangjianan@oppo.com>
+ */
+
+#include <stdlib.h>
+#include <lz4.h>
+
+#include "erofs/internal.h"
+#include "erofs/err.h"
+#include "decompress.h"
+#include "logging.h"
+#include "init.h"
+
+static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)
+{
+ char *dest = rq->out + rq->ofs_out;
+ char *src = rq->in + rq->ofs_head;
+
+ memcpy(dest, src, rq->outputsize - rq->ofs_head);
+
+ return 0;
+}
+
+static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)
+{
+ int ret = 0;
+ char *dest = rq->out + rq->ofs_out;
+ char *src = rq->in;
+ char *buff = NULL;
+ bool support_0padding = false;
+ unsigned int inputmargin = 0;
+
+ if (sbk->feature_incompat & EROFS_FEATURE_INCOMPAT_LZ4_0PADDING) {
+ support_0padding = true;
+
+ while (!src[inputmargin & ~PAGE_MASK])
+ if (!(++inputmargin & ~PAGE_MASK))
+ break;
+
+ if (inputmargin >= rq->inputsize)
+ return -EIO;
+ }
+
+ if (rq->ofs_head) {
+ buff = malloc(rq->outputsize);
+ if (!buff)
+ return -ENOMEM;
+ dest = buff;
+ }
+
+ if (rq->partial_decoding || !support_0padding)
+ ret = LZ4_decompress_safe_partial(src + inputmargin, dest,
+ rq->inputsize - inputmargin,
+ rq->outputsize, rq->outputsize);
+ else
+ ret = LZ4_decompress_safe(src + inputmargin, dest,
+ rq->inputsize - inputmargin,
+ rq->outputsize);
+
+ if (ret != (int)rq->outputsize) {
+ ret = -EIO;
+ goto out;
+ }
+
+ if (rq->ofs_head) {
+ src = dest + rq->ofs_head;
+ dest = rq->out + rq->ofs_out;
+ memcpy(dest, src, rq->outputsize - rq->ofs_head);
+ }
+
+out:
+ if (buff)
+ free(buff);
+
+ return ret;
+}
+
+int z_erofs_decompress(struct z_erofs_decompress_req *rq)
+{
+ if (rq->alg == Z_EROFS_COMPRESSION_SHIFTED)
+ return z_erofs_shifted_transform(rq);
+
+ return z_erofs_decompress_generic(rq);
+}
diff --git a/fuse/decompress.h b/fuse/decompress.h
new file mode 100644
index 000000000000..cd395c3d6b3f
--- /dev/null
+++ b/fuse/decompress.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd.
+ * Created by Huang Jianan <huangjianan@oppo.com>
+ */
+
+#ifndef __EROFS_DECOMPRESS_H
+#define __EROFS_DECOMPRESS_H
+
+#include "erofs/internal.h"
+
+enum {
+ Z_EROFS_COMPRESSION_SHIFTED = Z_EROFS_COMPRESSION_MAX,
+ Z_EROFS_COMPRESSION_RUNTIME_MAX
+};
+
+struct z_erofs_decompress_req {
+ char *in, *out;
+
+ size_t ofs_out, ofs_head;
+ unsigned int inputsize, outputsize;
+
+ /* indicate the algorithm will be used for decompression */
+ unsigned int alg;
+ bool partial_decoding;
+};
+
+#ifdef LZ4_ENABLED
+int z_erofs_decompress(struct z_erofs_decompress_req *rq);
+#else
+int z_erofs_decompress(struct z_erofs_decompress_req *rq)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+
+#endif
diff --git a/fuse/dentry.h b/fuse/dentry.h
index ee2144de8a89..f89c50646fb5 100644
--- a/fuse/dentry.h
+++ b/fuse/dentry.h
@@ -10,10 +10,11 @@
#include <stdint.h>
#include "erofs/internal.h"
+/* fixme: Deal with names that exceed the allocated size */
#ifdef __64BITS
-#define DCACHE_ENTRY_NAME_LEN 40
+#define DCACHE_ENTRY_NAME_LEN EROFS_NAME_LEN
#else
-#define DCACHE_ENTRY_NAME_LEN 48
+#define DCACHE_ENTRY_NAME_LEN EROFS_NAME_LEN
#endif
/* This struct declares a node of a k-tree. Every node has a pointer to one of
diff --git a/fuse/init.c b/fuse/init.c
index 8198fa78da96..e9cc9f81d4c7 100644
--- a/fuse/init.c
+++ b/fuse/init.c
@@ -17,7 +17,23 @@
struct erofs_super_block super;
-static struct erofs_super_block *sbk = &super;
+struct erofs_super_block *sbk = &super;
+
+static bool check_layout_compatibility(struct erofs_super_block *sb,
+ struct erofs_super_block *dsb)
+{
+ const unsigned int feature = le32_to_cpu(dsb->feature_incompat);
+
+ sb->feature_incompat = feature;
+
+ /* check if current kernel meets all mandatory requirements */
+ if (feature & (~EROFS_ALL_FEATURE_INCOMPAT)) {
+ loge("unidentified incompatible feature %x, please upgrade kernel version",
+ feature & ~EROFS_ALL_FEATURE_INCOMPAT);
+ return false;
+ }
+ return true;
+}
int erofs_init_super(void)
{
@@ -40,6 +56,9 @@ int erofs_init_super(void)
return -EINVAL;
}
+ if (!check_layout_compatibility(sbk, sb))
+ return -EINVAL;
+
sbk->checksum = le32_to_cpu(sb->checksum);
sbk->feature_compat = le32_to_cpu(sb->feature_compat);
sbk->blkszbits = sb->blkszbits;
@@ -56,6 +75,7 @@ int erofs_init_super(void)
sbk->root_nid = le16_to_cpu(sb->root_nid);
logp("%-15s:0x%X", STR(magic), SUPER_MEM(magic));
+ logp("%-15s:0x%X", STR(feature_incompat), SUPER_MEM(feature_incompat));
logp("%-15s:0x%X", STR(feature_compat), SUPER_MEM(feature_compat));
logp("%-15s:%u", STR(blkszbits), SUPER_MEM(blkszbits));
logp("%-15s:%u", STR(root_nid), SUPER_MEM(root_nid));
diff --git a/fuse/init.h b/fuse/init.h
index d7a97b5ee043..3fc4eb548dda 100644
--- a/fuse/init.h
+++ b/fuse/init.h
@@ -13,6 +13,8 @@
#define BOOT_SECTOR_SIZE 0x400
+extern struct erofs_super_block *sbk;
+
int erofs_init_super(void);
erofs_nid_t erofs_get_root_nid(void);
erofs_off_t nid2addr(erofs_nid_t nid);
diff --git a/fuse/namei.c b/fuse/namei.c
index 7ed1168fc14f..510fcfda5a76 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -49,7 +49,7 @@ static inline dev_t new_decode_dev(u32 dev)
int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
{
- int ret;
+ int ret, ifmt;
char buf[EROFS_BLKSIZ];
struct erofs_inode_compact *v1;
const erofs_off_t addr = nid2addr(nid);
@@ -60,6 +60,11 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
return -EIO;
v1 = (struct erofs_inode_compact *)buf;
+ /* fixme: support extended inode */
+ ifmt = le16_to_cpu(v1->i_format);
+ if (__inode_version(ifmt) != EROFS_INODE_LAYOUT_COMPACT)
+ return -EOPNOTSUPP;
+
vi->datalayout = __inode_data_mapping(le16_to_cpu(v1->i_format));
vi->inode_isize = sizeof(struct erofs_inode_compact);
vi->xattr_isize = erofs_xattr_ibody_size(v1->i_xattr_icount);
@@ -88,6 +93,10 @@ int erofs_iget_by_nid(erofs_nid_t nid, struct erofs_vnode *vi)
return -EIO;
}
+ vi->z_inited = false;
+ if (erofs_inode_is_data_compressed(vi->datalayout))
+ z_erofs_fill_inode(vi);
+
return 0;
}
diff --git a/fuse/read.c b/fuse/read.c
index 3ce5c4f2911e..0d0e3b0fa468 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -16,6 +16,7 @@
#include "namei.h"
#include "disk_io.h"
#include "init.h"
+#include "decompress.h"
size_t erofs_read_data(struct erofs_vnode *vnode, char *buffer,
size_t size, off_t offset)
@@ -78,6 +79,73 @@ finished:
}
+size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
+ size_t size, off_t offset)
+{
+ int ret;
+ size_t end, count, ofs, sum = size;
+ struct erofs_map_blocks map = {
+ .index = UINT_MAX,
+ };
+ bool partial;
+ unsigned int algorithmformat;
+ char raw[EROFS_BLKSIZ];
+
+ while (sum) {
+ end = offset + sum;
+ map.m_la = end - 1;
+
+ ret = z_erofs_map_blocks_iter(vnode, &map);
+ if (ret)
+ return ret;
+
+ if (!(map.m_flags & EROFS_MAP_MAPPED)) {
+ sum -= map.m_llen;
+ continue;
+ }
+
+ ret = dev_read(raw, EROFS_BLKSIZ, map.m_pa);
+ if (ret < 0 || (size_t)ret != EROFS_BLKSIZ)
+ return -EIO;
+
+ algorithmformat = map.m_flags & EROFS_MAP_ZIPPED ?
+ Z_EROFS_COMPRESSION_LZ4 :
+ Z_EROFS_COMPRESSION_SHIFTED;
+
+ if (end >= map.m_la + map.m_llen) {
+ count = map.m_llen;
+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
+ } else {
+ count = end - map.m_la;
+ partial = true;
+ }
+
+ if ((off_t)map.m_la < offset) {
+ ofs = offset - map.m_la;
+ sum = 0;
+ } else {
+ ofs = 0;
+ sum -= count;
+ }
+
+ ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
+ .in = raw,
+ .out = buffer,
+ .ofs_out = sum,
+ .ofs_head = ofs,
+ .inputsize = EROFS_BLKSIZ,
+ .outputsize = count,
+ .alg = algorithmformat,
+ .partial_decoding = partial
+ });
+ if (ret < 0)
+ return ret;
+ }
+
+ logi("nid:%u size=%zd offset=%llu done",
+ vnode->nid, size, (long long)offset);
+ return size;
+}
int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
struct fuse_file_info *fi)
@@ -107,7 +175,8 @@ int erofs_read(const char *path, char *buffer, size_t size, off_t offset,
case EROFS_INODE_FLAT_COMPRESSION_LEGACY:
case EROFS_INODE_FLAT_COMPRESSION:
- /* Fixme: */
+ return erofs_read_data_compression(&v, buffer, size, offset);
+
default:
return -EINVAL;
}
@@ -137,4 +206,4 @@ int erofs_readlink(const char *path, char *buffer, size_t size)
logi("path:%s link=%s size=%llu", path, buffer, lnksz);
return 0;
-}
\ No newline at end of file
+}
diff --git a/fuse/zmap.c b/fuse/zmap.c
new file mode 100644
index 000000000000..022ca1b9e437
--- /dev/null
+++ b/fuse/zmap.c
@@ -0,0 +1,416 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Many parts of codes are copied from Linux kernel/fs/erofs.
+ *
+ * Copyright (C) 2018-2019 HUAWEI, Inc.
+ * https://www.huawei.com/
+ * Created by Gao Xiang <gaoxiang25@huawei.com>
+ * Modified by Huang Jianan <huangjianan@oppo.com>
+ */
+
+#include "init.h"
+#include "disk_io.h"
+#include "logging.h"
+
+int z_erofs_fill_inode(struct erofs_vnode *vi)
+{
+ if (vi->datalayout == EROFS_INODE_FLAT_COMPRESSION_LEGACY) {
+ vi->z_advise = 0;
+ vi->z_algorithmtype[0] = 0;
+ vi->z_algorithmtype[1] = 0;
+ vi->z_logical_clusterbits = LOG_BLOCK_SIZE;
+ vi->z_physical_clusterbits[0] = vi->z_logical_clusterbits;
+ vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits;
+ vi->z_inited = true;
+ }
+
+ return 0;
+}
+
+static int z_erofs_fill_inode_lazy(struct erofs_vnode *vi)
+{
+ int ret;
+ erofs_off_t pos;
+ struct z_erofs_map_header *h;
+ char buf[8];
+
+ if (vi->z_inited)
+ return 0;
+
+ DBG_BUGON(vi->datalayout == EROFS_INODE_FLAT_COMPRESSION_LEGACY);
+
+ pos = round_up(nid2addr(vi->nid) + vi->inode_isize + vi->xattr_isize, 8);
+
+ ret = dev_read(buf, 8, pos);
+ if (ret < 0 && (uint32_t)ret != 8)
+ return -EIO;
+
+ h = (struct z_erofs_map_header *)buf;
+ vi->z_advise = le16_to_cpu(h->h_advise);
+ vi->z_algorithmtype[0] = h->h_algorithmtype & 15;
+ vi->z_algorithmtype[1] = h->h_algorithmtype >> 4;
+
+ if (vi->z_algorithmtype[0] >= Z_EROFS_COMPRESSION_MAX) {
+ loge("unknown compression format %u for nid %llu",
+ vi->z_algorithmtype[0], vi->nid);
+ return -EOPNOTSUPP;
+ }
+
+ vi->z_logical_clusterbits = LOG_BLOCK_SIZE + (h->h_clusterbits & 7);
+ vi->z_physical_clusterbits[0] = vi->z_logical_clusterbits +
+ ((h->h_clusterbits >> 3) & 3);
+
+ if (vi->z_physical_clusterbits[0] != LOG_BLOCK_SIZE) {
+ loge("unsupported physical clusterbits %u for nid %llu",
+ vi->z_physical_clusterbits[0], vi->nid);
+ return -EOPNOTSUPP;
+ }
+
+ vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits +
+ ((h->h_clusterbits >> 5) & 7);
+ vi->z_inited = true;
+
+ return 0;
+}
+
+struct z_erofs_maprecorder {
+ struct erofs_vnode *vnode;
+ struct erofs_map_blocks *map;
+ void *kaddr;
+
+ unsigned long lcn;
+ /* compression extent information gathered */
+ u8 type;
+ u16 clusterofs;
+ u16 delta[2];
+ erofs_blk_t pblk;
+};
+
+static int z_erofs_reload_indexes(struct z_erofs_maprecorder *m,
+ erofs_blk_t eblk)
+{
+ int ret;
+ struct erofs_map_blocks *const map = m->map;
+ char *mpage = map->mpage;
+
+ if (map->index == eblk)
+ return 0;
+
+ ret = dev_read(mpage, EROFS_BLKSIZ, blknr_to_addr(eblk));
+ if (ret < 0 && (uint32_t)ret != EROFS_BLKSIZ)
+ return -EIO;
+
+ map->index = eblk;
+
+ return 0;
+}
+
+static int legacy_load_cluster_from_disk(struct z_erofs_maprecorder *m,
+ unsigned long lcn)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ const erofs_off_t ibase = nid2addr(vi->nid);
+ const erofs_off_t pos =
+ Z_EROFS_VLE_LEGACY_INDEX_ALIGN(ibase + vi->inode_isize +
+ vi->xattr_isize) +
+ lcn * sizeof(struct z_erofs_vle_decompressed_index);
+ struct z_erofs_vle_decompressed_index *di;
+ unsigned int advise, type;
+ int err;
+
+ err = z_erofs_reload_indexes(m, erofs_blknr(pos));
+ if (err)
+ return err;
+
+ m->lcn = lcn;
+ di = m->kaddr + erofs_blkoff(pos);
+
+ advise = le16_to_cpu(di->di_advise);
+ type = (advise >> Z_EROFS_VLE_DI_CLUSTER_TYPE_BIT) &
+ ((1 << Z_EROFS_VLE_DI_CLUSTER_TYPE_BITS) - 1);
+ switch (type) {
+ case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD:
+ m->clusterofs = 1 << vi->z_logical_clusterbits;
+ m->delta[0] = le16_to_cpu(di->di_u.delta[0]);
+ m->delta[1] = le16_to_cpu(di->di_u.delta[1]);
+ break;
+ case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
+ m->clusterofs = le16_to_cpu(di->di_clusterofs);
+ m->pblk = le32_to_cpu(di->di_u.blkaddr);
+ break;
+ default:
+ DBG_BUGON(1);
+ return -EOPNOTSUPP;
+ }
+ m->type = type;
+ return 0;
+}
+
+static unsigned int decode_compactedbits(unsigned int lobits,
+ unsigned int lomask,
+ u8 *in, unsigned int pos, u8 *type)
+{
+ const unsigned int v = get_unaligned_le32(in + pos / 8) >> (pos & 7);
+ const unsigned int lo = v & lomask;
+
+ *type = (v >> lobits) & 3;
+ return lo;
+}
+
+static int unpack_compacted_index(struct z_erofs_maprecorder *m,
+ unsigned int amortizedshift,
+ unsigned int eofs)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ const unsigned int lomask = (1 << lclusterbits) - 1;
+ unsigned int vcnt, base, lo, encodebits, nblk;
+ int i;
+ u8 *in, type;
+
+ if (1 << amortizedshift == 4)
+ vcnt = 2;
+ else if (1 << amortizedshift == 2 && lclusterbits == 12)
+ vcnt = 16;
+ else
+ return -EOPNOTSUPP;
+
+ encodebits = ((vcnt << amortizedshift) - sizeof(__le32)) * 8 / vcnt;
+ base = round_down(eofs, vcnt << amortizedshift);
+ in = m->kaddr + base;
+
+ i = (eofs - base) >> amortizedshift;
+
+ lo = decode_compactedbits(lclusterbits, lomask,
+ in, encodebits * i, &type);
+ m->type = type;
+ if (type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) {
+ m->clusterofs = 1 << lclusterbits;
+ if (i + 1 != (int)vcnt) {
+ m->delta[0] = lo;
+ return 0;
+ }
+ /*
+ * since the last lcluster in the pack is special,
+ * of which lo saves delta[1] rather than delta[0].
+ * Hence, get delta[0] by the previous lcluster indirectly.
+ */
+ lo = decode_compactedbits(lclusterbits, lomask,
+ in, encodebits * (i - 1), &type);
+ if (type != Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD)
+ lo = 0;
+ m->delta[0] = lo + 1;
+ return 0;
+ }
+ m->clusterofs = lo;
+ m->delta[0] = 0;
+ /* figout out blkaddr (pblk) for HEAD lclusters */
+ nblk = 1;
+ while (i > 0) {
+ --i;
+ lo = decode_compactedbits(lclusterbits, lomask,
+ in, encodebits * i, &type);
+ if (type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD)
+ i -= lo;
+
+ if (i >= 0)
+ ++nblk;
+ }
+ in += (vcnt << amortizedshift) - sizeof(__le32);
+ m->pblk = le32_to_cpu(*(__le32 *)in) + nblk;
+ return 0;
+}
+
+static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
+ unsigned long lcn)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ const erofs_off_t ebase = round_up(nid2addr(vi->nid) + vi->inode_isize +
+ vi->xattr_isize, 8) +
+ sizeof(struct z_erofs_map_header);
+ const unsigned int totalidx = DIV_ROUND_UP(vi->i_size, EROFS_BLKSIZ);
+ unsigned int compacted_4b_initial, compacted_2b;
+ unsigned int amortizedshift;
+ erofs_off_t pos;
+ int err;
+
+ if (lclusterbits != 12)
+ return -EOPNOTSUPP;
+
+ if (lcn >= totalidx)
+ return -EINVAL;
+
+ m->lcn = lcn;
+ /* used to align to 32-byte (compacted_2b) alignment */
+ compacted_4b_initial = (32 - ebase % 32) / 4;
+ if (compacted_4b_initial == 32 / 4)
+ compacted_4b_initial = 0;
+
+ if (vi->z_advise & Z_EROFS_ADVISE_COMPACTED_2B)
+ compacted_2b = rounddown(totalidx - compacted_4b_initial, 16);
+ else
+ compacted_2b = 0;
+
+ pos = ebase;
+ if (lcn < compacted_4b_initial) {
+ amortizedshift = 2;
+ goto out;
+ }
+ pos += compacted_4b_initial * 4;
+ lcn -= compacted_4b_initial;
+
+ if (lcn < compacted_2b) {
+ amortizedshift = 1;
+ goto out;
+ }
+ pos += compacted_2b * 2;
+ lcn -= compacted_2b;
+ amortizedshift = 2;
+out:
+ pos += lcn * (1 << amortizedshift);
+ err = z_erofs_reload_indexes(m, erofs_blknr(pos));
+ if (err)
+ return err;
+ return unpack_compacted_index(m, amortizedshift, erofs_blkoff(pos));
+}
+
+static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
+ unsigned int lcn)
+{
+ const unsigned int datamode = m->vnode->datalayout;
+
+ if (datamode == EROFS_INODE_FLAT_COMPRESSION_LEGACY)
+ return legacy_load_cluster_from_disk(m, lcn);
+
+ if (datamode == EROFS_INODE_FLAT_COMPRESSION)
+ return compacted_load_cluster_from_disk(m, lcn);
+
+ return -EINVAL;
+}
+
+static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
+ unsigned int lookback_distance)
+{
+ struct erofs_vnode *const vi = m->vnode;
+ struct erofs_map_blocks *const map = m->map;
+ const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ unsigned long lcn = m->lcn;
+ int err;
+
+ if (lcn < lookback_distance) {
+ loge("bogus lookback distance @ nid %llu", vi->nid);
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
+ }
+
+ /* load extent head logical cluster if needed */
+ lcn -= lookback_distance;
+ err = z_erofs_load_cluster_from_disk(m, lcn);
+ if (err)
+ return err;
+
+ switch (m->type) {
+ case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD:
+ if (!m->delta[0]) {
+ loge("invalid lookback distance 0 @ nid %llu",
+ vi->nid);
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
+ }
+ return z_erofs_extent_lookback(m, m->delta[0]);
+ case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
+ map->m_flags &= ~EROFS_MAP_ZIPPED;
+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
+ map->m_la = (lcn << lclusterbits) | m->clusterofs;
+ break;
+ default:
+ loge("unknown type %u @ lcn %lu of nid %llu",
+ m->type, lcn, vi->nid);
+ DBG_BUGON(1);
+ return -EOPNOTSUPP;
+ }
+ return 0;
+}
+
+int z_erofs_map_blocks_iter(struct erofs_vnode *vi,
+ struct erofs_map_blocks *map)
+{
+ struct z_erofs_maprecorder m = {
+ .vnode = vi,
+ .map = map,
+ .kaddr = map->mpage,
+ };
+ int err = 0;
+ unsigned int lclusterbits, endoff;
+ unsigned long long ofs, end;
+
+ /* when trying to read beyond EOF, leave it unmapped */
+ if (map->m_la >= vi->i_size) {
+ map->m_llen = map->m_la + 1 - vi->i_size;
+ map->m_la = vi->i_size;
+ map->m_flags = 0;
+ goto out;
+ }
+
+ err = z_erofs_fill_inode_lazy(vi);
+ if (err)
+ goto out;
+
+ lclusterbits = vi->z_logical_clusterbits;
+ ofs = map->m_la;
+ m.lcn = ofs >> lclusterbits;
+ endoff = ofs & ((1 << lclusterbits) - 1);
+
+ err = z_erofs_load_cluster_from_disk(&m, m.lcn);
+ if (err)
+ goto out;
+
+ map->m_flags = EROFS_MAP_ZIPPED; /* by default, compressed */
+ end = (m.lcn + 1ULL) << lclusterbits;
+ switch (m.type) {
+ case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN:
+ if (endoff >= m.clusterofs)
+ map->m_flags &= ~EROFS_MAP_ZIPPED;
+ case Z_EROFS_VLE_CLUSTER_TYPE_HEAD:
+ if (endoff >= m.clusterofs) {
+ map->m_la = (m.lcn << lclusterbits) | m.clusterofs;
+ break;
+ }
+ /* m.lcn should be >= 1 if endoff < m.clusterofs */
+ if (!m.lcn) {
+ loge("invalid logical cluster 0 at nid %llu",
+ vi->nid);
+ err = -EFSCORRUPTED;
+ goto out;
+ }
+ end = (m.lcn << lclusterbits) | m.clusterofs;
+ map->m_flags |= EROFS_MAP_FULL_MAPPED;
+ m.delta[0] = 1;
+ case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD:
+ /* get the correspoinding first chunk */
+ err = z_erofs_extent_lookback(&m, m.delta[0]);
+ if (err)
+ goto out;
+ break;
+ default:
+ loge("unknown type %u @ offset %llu of nid %llu",
+ m.type, ofs, vi->nid);
+ err = -EOPNOTSUPP;
+ goto out;
+ }
+
+ map->m_llen = end - map->m_la;
+ map->m_plen = 1 << lclusterbits;
+ map->m_pa = blknr_to_addr(m.pblk);
+ map->m_flags |= EROFS_MAP_MAPPED;
+
+out:
+ logd("m_la %llu m_pa %llu m_llen %llu m_plen %llu m_flags 0%o",
+ map->m_la, map->m_pa,
+ map->m_llen, map->m_plen, map->m_flags);
+
+ DBG_BUGON(err < 0 && err != -ENOMEM);
+ return err;
+}
diff --git a/include/erofs/defs.h b/include/erofs/defs.h
index a9c769ec88ae..06d29a9575da 100644
--- a/include/erofs/defs.h
+++ b/include/erofs/defs.h
@@ -173,5 +173,18 @@ typedef int64_t s64;
#define __maybe_unused __attribute__((__unused__))
#endif
+struct __una_u32 { u32 x; } __packed;
+
+static inline u32 __get_unaligned_cpu32(const void *p)
+{
+ const struct __una_u32 *ptr = (const struct __una_u32 *)p;
+ return ptr->x;
+}
+
+static inline u32 get_unaligned_le32(const void *p)
+{
+ return __get_unaligned_cpu32((const u8 *)p);
+}
+
#endif
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index 47ad96d0488c..5807b67637c8 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -36,6 +36,8 @@ typedef unsigned short umode_t;
#error incompatible PAGE_SIZE is already defined
#endif
+#define PAGE_MASK (~(PAGE_SIZE-1))
+
#define LOG_BLOCK_SIZE (12)
#define EROFS_BLKSIZ (1U << LOG_BLOCK_SIZE)
@@ -145,7 +147,15 @@ struct erofs_vnode {
uint16_t xattr_shared_count;
char *xattr_shared_xattrs;
- erofs_blk_t raw_blkaddr;
+ union {
+ erofs_blk_t raw_blkaddr;
+ struct {
+ uint16_t z_advise;
+ uint8_t z_algorithmtype[2];
+ uint8_t z_logical_clusterbits;
+ uint8_t z_physical_clusterbits[2];
+ };
+ };
erofs_nid_t nid;
uint32_t i_ino;
@@ -155,6 +165,7 @@ struct erofs_vnode {
uint16_t i_nlink;
uint32_t i_rdev;
+ bool z_inited;
/* if file is inline read inline data witch inode */
char *idata;
};
@@ -207,5 +218,35 @@ static inline const char *erofs_strerror(int err)
return msg;
}
+enum {
+ BH_Mapped ,
+ BH_Zipped ,
+ BH_FullMapped,
+};
+
+/* Has a disk mapping */
+#define EROFS_MAP_MAPPED (1 << BH_Mapped)
+/* The extent has been compressed */
+#define EROFS_MAP_ZIPPED (1 << BH_Zipped)
+/* The length of extent is full */
+#define EROFS_MAP_FULL_MAPPED (1 << BH_FullMapped)
+
+struct erofs_map_blocks {
+ erofs_off_t m_pa, m_la;
+ u64 m_plen, m_llen;
+
+ unsigned int m_flags;
+
+ char mpage[EROFS_BLKSIZ];
+ erofs_blk_t index;
+};
+
+/* zmap.c */
+int z_erofs_fill_inode(struct erofs_vnode *vi);
+int z_erofs_map_blocks_iter(struct erofs_vnode *vi,
+ struct erofs_map_blocks *map);
+
+#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
+
#endif
diff --git a/include/erofs_fs.h b/include/erofs_fs.h
index bcc4f0c630ad..0c70897d2113 100644
--- a/include/erofs_fs.h
+++ b/include/erofs_fs.h
@@ -279,6 +279,10 @@ struct z_erofs_vle_decompressed_index {
} di_u;
};
+#define Z_EROFS_VLE_LEGACY_INDEX_ALIGN(size) \
+ (round_up(size, sizeof(struct z_erofs_vle_decompressed_index)) + \
+ sizeof(struct z_erofs_map_header) + Z_EROFS_VLE_LEGACY_HEADER_PADDING)
+
#define Z_EROFS_VLE_EXTENT_ALIGN(size) round_up(size, \
sizeof(struct z_erofs_vle_decompressed_index))
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (2 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 03/12] erofs-utils: fuse: support read compressed file Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-19 3:13 ` 回复: " huangjianan
2020-10-19 3:37 ` 回复: " Huang Jianan
2020-10-17 5:16 ` [WIP] [PATCH 05/12] erofs-utils: fuse: use proper expression about inode size Gao Xiang via Linux-erofs
` (7 subsequent siblings)
11 siblings, 2 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
so more easy to understand.
[ let's fold in to the original patch. ]
Cc: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/read.c | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/fuse/read.c b/fuse/read.c
index 0d0e3b0fa468..dd44adaa1c40 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
Z_EROFS_COMPRESSION_LZ4 :
Z_EROFS_COMPRESSION_SHIFTED;
- if (end >= map.m_la + map.m_llen) {
- count = map.m_llen;
- partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
- } else {
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
count = end - map.m_la;
partial = true;
+ } else {
+ ASSERT(end == map.m_la + map_m_llen);
+ count = map.m_llen;
+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
}
if ((off_t)map.m_la < offset) {
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 05/12] erofs-utils: fuse: use proper expression about inode size
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (3 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 06/12] erofs-utils: fuse: drop ofs_out Gao Xiang via Linux-erofs
` (6 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
[ let's fold in to the original patch. ]
Cc: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/namei.c | 4 ++--
fuse/read.c | 2 +-
fuse/readir.c | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/fuse/namei.c b/fuse/namei.c
index 510fcfda5a76..0b71072027ce 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -172,8 +172,8 @@ struct dcache_entry *disk_lookup(struct dcache_entry *parent, const char *name,
if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
uint32_t dir_off = erofs_blkoff(dirsize);
- off_t dir_addr = nid2addr(dcache_get_nid(parent))
- + sizeof(struct erofs_inode_compact) + v.xattr_isize;
+ off_t dir_addr = nid2addr(dcache_get_nid(parent)) +
+ v.inode_isize + v.xattr_isize;
memset(buf, 0, sizeof(buf));
ret = dev_read(buf, dir_off, dir_addr);
diff --git a/fuse/read.c b/fuse/read.c
index dd44adaa1c40..3f2e49c390de 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -65,7 +65,7 @@ size_t erofs_read_data_inline(struct erofs_vnode *vnode, char *buffer,
if (!suminline)
goto finished;
- addr = nid2addr(vnode->nid) + sizeof(struct erofs_inode_compact)
+ addr = nid2addr(vnode->nid) + vnode->inode_isize +
+ vnode->xattr_isize;
ret = dev_read(buffer + rdsz, suminline, addr);
if (ret < 0 || (size_t)ret != suminline)
diff --git a/fuse/readir.c b/fuse/readir.c
index 9589685c9122..46ceb1d90a7f 100644
--- a/fuse/readir.c
+++ b/fuse/readir.c
@@ -108,7 +108,7 @@ int erofs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
if (v.datalayout == EROFS_INODE_FLAT_INLINE) {
off_t addr;
- addr = nid2addr(nid) + sizeof(struct erofs_inode_compact)
+ addr = nid2addr(nid) + v.inode_isize
+ v.xattr_isize;
memset(dirsbuf, 0, sizeof(dirsbuf));
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 06/12] erofs-utils: fuse: drop ofs_out
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (4 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 05/12] erofs-utils: fuse: use proper expression about inode size Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 07/12] erofs-utils: fuse: refuse a undefined shifted cluster behavior Gao Xiang via Linux-erofs
` (5 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
[ let's fold in to the original patch. ]
Cc: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/decompress.c | 7 +++----
fuse/decompress.h | 2 +-
fuse/read.c | 3 +--
3 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/fuse/decompress.c b/fuse/decompress.c
index e2df3cea78f7..4b7cf3211319 100644
--- a/fuse/decompress.c
+++ b/fuse/decompress.c
@@ -15,7 +15,7 @@
static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)
{
- char *dest = rq->out + rq->ofs_out;
+ char *dest = rq->out;
char *src = rq->in + rq->ofs_head;
memcpy(dest, src, rq->outputsize - rq->ofs_head);
@@ -26,7 +26,7 @@ static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)
static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)
{
int ret = 0;
- char *dest = rq->out + rq->ofs_out;
+ char *dest = rq->out;
char *src = rq->in;
char *buff = NULL;
bool support_0padding = false;
@@ -66,8 +66,7 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)
if (rq->ofs_head) {
src = dest + rq->ofs_head;
- dest = rq->out + rq->ofs_out;
- memcpy(dest, src, rq->outputsize - rq->ofs_head);
+ memcpy(rq->out, src, rq->outputsize - rq->ofs_head);
}
out:
diff --git a/fuse/decompress.h b/fuse/decompress.h
index cd395c3d6b3f..5a3e2356dc1b 100644
--- a/fuse/decompress.h
+++ b/fuse/decompress.h
@@ -17,7 +17,7 @@ enum {
struct z_erofs_decompress_req {
char *in, *out;
- size_t ofs_out, ofs_head;
+ size_t ofs_head;
unsigned int inputsize, outputsize;
/* indicate the algorithm will be used for decompression */
diff --git a/fuse/read.c b/fuse/read.c
index 3f2e49c390de..06ca08587e7c 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -135,8 +135,7 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
.in = raw,
- .out = buffer,
- .ofs_out = sum,
+ .out = buffer + sum,
.ofs_head = ofs,
.inputsize = EROFS_BLKSIZ,
.outputsize = count,
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 07/12] erofs-utils: fuse: refuse a undefined shifted cluster behavior
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (5 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 06/12] erofs-utils: fuse: drop ofs_out Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 08/12] erofs-utils: fuse: drop z_erofs_shifted_transform() Gao Xiang via Linux-erofs
` (4 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
[ let's fold in to the original patch. ]
Cc: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/decompress.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/fuse/decompress.c b/fuse/decompress.c
index 4b7cf3211319..4a9f8e7995c1 100644
--- a/fuse/decompress.c
+++ b/fuse/decompress.c
@@ -18,8 +18,11 @@ static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)
char *dest = rq->out;
char *src = rq->in + rq->ofs_head;
- memcpy(dest, src, rq->outputsize - rq->ofs_head);
+ if (rq->inputsize != EROFS_BLKSIZ)
+ return -EFSCORRUPTED;
+ DBG_BUGON(rq->outputsize > EROFS_BLKSIZ);
+ memcpy(dest, src, rq->outputsize - rq->ofs_head);
return 0;
}
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 08/12] erofs-utils: fuse: drop z_erofs_shifted_transform()
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (6 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 07/12] erofs-utils: fuse: refuse a undefined shifted cluster behavior Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 09/12] erofs-utils: fuse: rename ofs_head and outputsize Gao Xiang via Linux-erofs
` (3 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
clean up as well.
[ let's fold in to the original patch. ]
Cc: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/decompress.c | 26 +++++++++++---------------
1 file changed, 11 insertions(+), 15 deletions(-)
diff --git a/fuse/decompress.c b/fuse/decompress.c
index 4a9f8e7995c1..d3ee3677e9a3 100644
--- a/fuse/decompress.c
+++ b/fuse/decompress.c
@@ -13,19 +13,6 @@
#include "logging.h"
#include "init.h"
-static int z_erofs_shifted_transform(struct z_erofs_decompress_req *rq)
-{
- char *dest = rq->out;
- char *src = rq->in + rq->ofs_head;
-
- if (rq->inputsize != EROFS_BLKSIZ)
- return -EFSCORRUPTED;
-
- DBG_BUGON(rq->outputsize > EROFS_BLKSIZ);
- memcpy(dest, src, rq->outputsize - rq->ofs_head);
- return 0;
-}
-
static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)
{
int ret = 0;
@@ -81,8 +68,17 @@ out:
int z_erofs_decompress(struct z_erofs_decompress_req *rq)
{
- if (rq->alg == Z_EROFS_COMPRESSION_SHIFTED)
- return z_erofs_shifted_transform(rq);
+ if (rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
+ if (rq->inputsize != EROFS_BLKSIZ)
+ return -EFSCORRUPTED;
+
+ DBG_BUGON(rq->outputsize > EROFS_BLKSIZ);
+ DBG_BUGON(rq->outputsize < rq->ofs_head);
+
+ memcpy(rq->out, rq->in + rq->ofs_head,
+ rq->outputsize - rq->ofs_head);
+ return 0;
+ }
return z_erofs_decompress_generic(rq);
}
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 09/12] erofs-utils: fuse: rename ofs_head and outputsize
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (7 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 08/12] erofs-utils: fuse: drop z_erofs_shifted_transform() Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 10/12] erofs-utils: fuse: cleanup erofs_read_data_compression() Gao Xiang via Linux-erofs
` (2 subsequent siblings)
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
[ let's fold in to the original patch. ]
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/decompress.c | 27 +++++++++++++--------------
fuse/decompress.h | 8 ++++++--
fuse/read.c | 4 ++--
3 files changed, 21 insertions(+), 18 deletions(-)
diff --git a/fuse/decompress.c b/fuse/decompress.c
index d3ee3677e9a3..fc12852ac6b7 100644
--- a/fuse/decompress.c
+++ b/fuse/decompress.c
@@ -33,8 +33,8 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)
return -EIO;
}
- if (rq->ofs_head) {
- buff = malloc(rq->outputsize);
+ if (rq->decodedskip) {
+ buff = malloc(rq->decodedlength);
if (!buff)
return -ENOMEM;
dest = buff;
@@ -42,22 +42,21 @@ static int z_erofs_decompress_generic(struct z_erofs_decompress_req *rq)
if (rq->partial_decoding || !support_0padding)
ret = LZ4_decompress_safe_partial(src + inputmargin, dest,
- rq->inputsize - inputmargin,
- rq->outputsize, rq->outputsize);
+ rq->inputsize - inputmargin,
+ rq->decodedlength, rq->decodedlength);
else
ret = LZ4_decompress_safe(src + inputmargin, dest,
rq->inputsize - inputmargin,
- rq->outputsize);
+ rq->decodedlength);
- if (ret != (int)rq->outputsize) {
+ if (ret != (int)rq->decodedlength) {
ret = -EIO;
goto out;
}
- if (rq->ofs_head) {
- src = dest + rq->ofs_head;
- memcpy(rq->out, src, rq->outputsize - rq->ofs_head);
- }
+ if (rq->decodedskip)
+ memcpy(rq->out, dest + rq->decodedskip,
+ rq->decodedlength - rq->decodedskip);
out:
if (buff)
@@ -72,11 +71,11 @@ int z_erofs_decompress(struct z_erofs_decompress_req *rq)
if (rq->inputsize != EROFS_BLKSIZ)
return -EFSCORRUPTED;
- DBG_BUGON(rq->outputsize > EROFS_BLKSIZ);
- DBG_BUGON(rq->outputsize < rq->ofs_head);
+ DBG_BUGON(rq->decodedlength > EROFS_BLKSIZ);
+ DBG_BUGON(rq->decodedlength < rq->decodedskip);
- memcpy(rq->out, rq->in + rq->ofs_head,
- rq->outputsize - rq->ofs_head);
+ memcpy(rq->out, rq->in + rq->decodedskip,
+ rq->decodedlength - rq->decodedskip);
return 0;
}
diff --git a/fuse/decompress.h b/fuse/decompress.h
index 5a3e2356dc1b..7d436f18da86 100644
--- a/fuse/decompress.h
+++ b/fuse/decompress.h
@@ -17,8 +17,12 @@ enum {
struct z_erofs_decompress_req {
char *in, *out;
- size_t ofs_head;
- unsigned int inputsize, outputsize;
+ /*
+ * initial decompressed bytes that need to be skipped
+ * when finally copying to output buffer
+ */
+ unsigned int decodedskip;
+ unsigned int inputsize, decodedlength;
/* indicate the algorithm will be used for decompression */
unsigned int alg;
diff --git a/fuse/read.c b/fuse/read.c
index 06ca08587e7c..46be5cc64a90 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -136,9 +136,9 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
.in = raw,
.out = buffer + sum,
- .ofs_head = ofs,
+ .decodedskip = ofs,
.inputsize = EROFS_BLKSIZ,
- .outputsize = count,
+ .decodedlength = count,
.alg = algorithmformat,
.partial_decoding = partial
});
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 10/12] erofs-utils: fuse: cleanup erofs_read_data_compression()
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (8 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 09/12] erofs-utils: fuse: rename ofs_head and outputsize Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 11/12] erofs-utils: fuse: move up mpage in struct erofs_map_blocks Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 12/12] erofs-utils: fuse: fix up source headers Gao Xiang via Linux-erofs
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
[ let's fold in to the original patch. ]
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/read.c | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/fuse/read.c b/fuse/read.c
index 46be5cc64a90..f3aa628945e3 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -80,10 +80,10 @@ finished:
}
size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
- size_t size, off_t offset)
+ erofs_off_t size, erofs_off_t offset)
{
int ret;
- size_t end, count, ofs, sum = size;
+ erofs_off_t end, length, skip;
struct erofs_map_blocks map = {
.index = UINT_MAX,
};
@@ -91,8 +91,8 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
unsigned int algorithmformat;
char raw[EROFS_BLKSIZ];
- while (sum) {
- end = offset + sum;
+ end = offset + size;
+ while (end > offset) {
map.m_la = end - 1;
ret = z_erofs_map_blocks_iter(vnode, &map);
@@ -100,7 +100,7 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
return ret;
if (!(map.m_flags & EROFS_MAP_MAPPED)) {
- sum -= map.m_llen;
+ end = map.m_la;
continue;
}
@@ -117,28 +117,28 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
* larger than requested, and set up partial flag as well.
*/
if (end < map.m_la + map.m_llen) {
- count = end - map.m_la;
+ length = end - map.m_la;
partial = true;
} else {
ASSERT(end == map.m_la + map_m_llen);
- count = map.m_llen;
+ length = map.m_llen;
partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
}
- if ((off_t)map.m_la < offset) {
- ofs = offset - map.m_la;
- sum = 0;
+ if (map.m_la < offset) {
+ skip = offset - map.m_la;
+ end = offset;
} else {
- ofs = 0;
- sum -= count;
+ skip = 0;
+ end = map.m_la;
}
ret = z_erofs_decompress(&(struct z_erofs_decompress_req) {
.in = raw,
- .out = buffer + sum,
- .decodedskip = ofs,
- .inputsize = EROFS_BLKSIZ,
- .decodedlength = count,
+ .out = buffer + end - offset,
+ .decodedskip = skip,
+ .inputsize = map.m_plen,
+ .decodedlength = length,
.alg = algorithmformat,
.partial_decoding = partial
});
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 11/12] erofs-utils: fuse: move up mpage in struct erofs_map_blocks
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (9 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 10/12] erofs-utils: fuse: cleanup erofs_read_data_compression() Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 12/12] erofs-utils: fuse: fix up source headers Gao Xiang via Linux-erofs
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
[ let's fold in to the original patch. ]
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
include/erofs/internal.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/erofs/internal.h b/include/erofs/internal.h
index 5807b67637c8..1637b6749411 100644
--- a/include/erofs/internal.h
+++ b/include/erofs/internal.h
@@ -232,12 +232,12 @@ enum {
#define EROFS_MAP_FULL_MAPPED (1 << BH_FullMapped)
struct erofs_map_blocks {
+ char mpage[EROFS_BLKSIZ];
+
erofs_off_t m_pa, m_la;
u64 m_plen, m_llen;
unsigned int m_flags;
-
- char mpage[EROFS_BLKSIZ];
erofs_blk_t index;
};
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [WIP] [PATCH 12/12] erofs-utils: fuse: fix up source headers
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
` (10 preceding siblings ...)
2020-10-17 5:16 ` [WIP] [PATCH 11/12] erofs-utils: fuse: move up mpage in struct erofs_map_blocks Gao Xiang via Linux-erofs
@ 2020-10-17 5:16 ` Gao Xiang via Linux-erofs
11 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-17 5:16 UTC (permalink / raw)
To: linux-erofs; +Cc: Zhang Shiming, Guo Weichao
fix up weird paths and relicense zmap.c
[ let's fold in to the original patch. ]
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/decompress.c | 5 +++--
fuse/decompress.h | 3 ++-
fuse/dentry.c | 4 ++--
fuse/dentry.h | 4 ++--
fuse/disk_io.c | 4 ++--
fuse/disk_io.h | 4 ++--
fuse/getattr.c | 4 ++--
fuse/getattr.h | 4 ++--
fuse/init.c | 4 ++--
fuse/init.h | 4 ++--
fuse/logging.c | 4 ++--
fuse/logging.h | 4 ++--
fuse/main.c | 4 ++--
fuse/namei.c | 4 ++--
fuse/namei.h | 4 ++--
fuse/open.c | 4 ++--
fuse/open.h | 4 ++--
fuse/read.c | 5 +++--
fuse/read.h | 4 ++--
fuse/readir.c | 4 ++--
fuse/readir.h | 4 ++--
fuse/zmap.c | 7 ++++---
22 files changed, 48 insertions(+), 44 deletions(-)
diff --git a/fuse/decompress.c b/fuse/decompress.c
index fc12852ac6b7..f2aa84146946 100644
--- a/fuse/decompress.c
+++ b/fuse/decompress.c
@@ -1,9 +1,10 @@
-/* SPDX-License-Identifier: GPL-2.0+ */
+// SPDX-License-Identifier: GPL-2.0+
/*
+ * erofs-utils/fuse/decompress.c
+ *
* Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd.
* Created by Huang Jianan <huangjianan@oppo.com>
*/
-
#include <stdlib.h>
#include <lz4.h>
diff --git a/fuse/decompress.h b/fuse/decompress.h
index 7d436f18da86..508aabab1664 100644
--- a/fuse/decompress.h
+++ b/fuse/decompress.h
@@ -1,9 +1,10 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
+ * erofs-utils/fuse/decompress.h
+ *
* Copyright (C), 2008-2020, OPPO Mobile Comm Corp., Ltd.
* Created by Huang Jianan <huangjianan@oppo.com>
*/
-
#ifndef __EROFS_DECOMPRESS_H
#define __EROFS_DECOMPRESS_H
diff --git a/fuse/dentry.c b/fuse/dentry.c
index 27192ecfd32e..1ae37e3abc86 100644
--- a/fuse/dentry.c
+++ b/fuse/dentry.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\dentry.c
+ * erofs-utils/fuse/dentry.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "dentry.h"
#include "erofs/internal.h"
#include "logging.h"
diff --git a/fuse/dentry.h b/fuse/dentry.h
index f89c50646fb5..12f4cf6bafd9 100644
--- a/fuse/dentry.h
+++ b/fuse/dentry.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\dentry.h
+ * erofs-utils/fuse/dentry.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef _EROFS_DENTRY_H
#define _EROFS_DENTRY_H
diff --git a/fuse/disk_io.c b/fuse/disk_io.c
index 72d351b17806..3fc087699dc9 100644
--- a/fuse/disk_io.c
+++ b/fuse/disk_io.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\disk_io.c
+ * erofs-utils/fuse/disk_io.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#define _XOPEN_SOURCE 500
#include "disk_io.h"
diff --git a/fuse/disk_io.h b/fuse/disk_io.h
index 6b4bd3cce085..d2c3dd598bc0 100644
--- a/fuse/disk_io.h
+++ b/fuse/disk_io.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\disk_io.h
+ * erofs-utils/fuse/disk_io.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __DISK_IO_H
#define __DISK_IO_H
diff --git a/fuse/getattr.c b/fuse/getattr.c
index d2134f486e19..e5200ebeef1a 100644
--- a/fuse/getattr.c
+++ b/fuse/getattr.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\getattr.c
+ * erofs-utils/fuse/getattr.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "getattr.h"
#include <sys/types.h>
diff --git a/fuse/getattr.h b/fuse/getattr.h
index dbcff7c1a6e1..735529a91d5b 100644
--- a/fuse/getattr.h
+++ b/fuse/getattr.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\getattr.h
+ * erofs-utils/fuse/getattr.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __EROFS_GETATTR_H
#define __EROFS_GETATTR_H
diff --git a/fuse/init.c b/fuse/init.c
index e9cc9f81d4c7..48125c5791fa 100644
--- a/fuse/init.c
+++ b/fuse/init.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\init.c
+ * erofs-utils/fuse/init.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "init.h"
#include <string.h>
#include <asm-generic/errno-base.h>
diff --git a/fuse/init.h b/fuse/init.h
index 3fc4eb548dda..405a92913b4a 100644
--- a/fuse/init.h
+++ b/fuse/init.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\init.h
+ * erofs-utils/fuse/init.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __EROFS_INIT_H
#define __EROFS_INIT_H
diff --git a/fuse/logging.c b/fuse/logging.c
index 2d1f1c77c2a4..192f546b94ec 100644
--- a/fuse/logging.c
+++ b/fuse/logging.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\logging.c
+ * erofs-utils/fuse/logging.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "logging.h"
#include <stdio.h>
diff --git a/fuse/logging.h b/fuse/logging.h
index 7aa2eda405db..3aa77ab08107 100644
--- a/fuse/logging.h
+++ b/fuse/logging.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\logging.h
+ * erofs-utils/fuse/logging.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __LOGGING_H
#define __LOGGING_H
diff --git a/fuse/main.c b/fuse/main.c
index 884101374bcf..9c8169725fa7 100644
--- a/fuse/main.c
+++ b/fuse/main.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\main.c
+ * erofs-utils/fuse/main.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
diff --git a/fuse/namei.c b/fuse/namei.c
index 0b71072027ce..c33af4b04b45 100644
--- a/fuse/namei.c
+++ b/fuse/namei.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\namei.c
+ * erofs-utils/fuse/namei.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "namei.h"
#include <linux/kdev_t.h>
#include <sys/types.h>
diff --git a/fuse/namei.h b/fuse/namei.h
index 80e84d7220aa..1803a673daaf 100644
--- a/fuse/namei.h
+++ b/fuse/namei.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\inode.h
+ * erofs-utils/fuse/inode.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __INODE_H
#define __INODE_H
diff --git a/fuse/open.c b/fuse/open.c
index 9d2edca54d23..c219d3870000 100644
--- a/fuse/open.c
+++ b/fuse/open.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\open.c
+ * erofs-utils/fuse/open.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "open.h"
#include <asm-generic/errno-base.h>
#include <fuse.h>
diff --git a/fuse/open.h b/fuse/open.h
index d02c1f752204..dfc8b3cdd515 100644
--- a/fuse/open.h
+++ b/fuse/open.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\open.h
+ * erofs-utils/fuse/open.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __EROFS_OPEN_H
#define __EROFS_OPEN_H
diff --git a/fuse/read.c b/fuse/read.c
index f3aa628945e3..621f7d6ce6ea 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -1,9 +1,10 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\read.c
+ * erofs-utils/fuse/read.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
+ * Compression support by Huang Jianan <huangjianan@oppo.com>
*/
-
#include "read.h"
#include <errno.h>
#include <linux/fs.h>
diff --git a/fuse/read.h b/fuse/read.h
index 89d4b4cd600c..e901c607dc91 100644
--- a/fuse/read.h
+++ b/fuse/read.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\read.h
+ * erofs-utils/fuse/read.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __EROFS_READ_H
#define __EROFS_READ_H
diff --git a/fuse/readir.c b/fuse/readir.c
index 46ceb1d90a7f..f3dd0c42c6e2 100644
--- a/fuse/readir.c
+++ b/fuse/readir.c
@@ -1,9 +1,9 @@
// SPDX-License-Identifier: GPL-2.0+
/*
- * erofs-fuse\readir.c
+ * erofs-utils/fuse/readir.c
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#include "readir.h"
#include <errno.h>
#include <linux/fs.h>
diff --git a/fuse/readir.h b/fuse/readir.h
index 21ab7a4e4e0b..ee2ab8bdd0f0 100644
--- a/fuse/readir.h
+++ b/fuse/readir.h
@@ -1,9 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0+ */
/*
- * erofs-fuse\readir.h
+ * erofs-utils/fuse/readir.h
+ *
* Created by Li Guifu <blucerlee@gmail.com>
*/
-
#ifndef __EROFS_READDIR_H
#define __EROFS_READDIR_H
diff --git a/fuse/zmap.c b/fuse/zmap.c
index 022ca1b9e437..8ec0a7707fd6 100644
--- a/fuse/zmap.c
+++ b/fuse/zmap.c
@@ -1,13 +1,14 @@
-// SPDX-License-Identifier: GPL-2.0-only
+// SPDX-License-Identifier: GPL-2.0+
/*
- * Many parts of codes are copied from Linux kernel/fs/erofs.
+ * erofs-utils/fuse/zmap.c
+ *
+ * (a large amount of code was adapted from Linux kernel. )
*
* Copyright (C) 2018-2019 HUAWEI, Inc.
* https://www.huawei.com/
* Created by Gao Xiang <gaoxiang25@huawei.com>
* Modified by Huang Jianan <huangjianan@oppo.com>
*/
-
#include "init.h"
#include "disk_io.h"
#include "logging.h"
--
2.24.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* 回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
2020-10-17 5:16 ` [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling Gao Xiang via Linux-erofs
@ 2020-10-19 3:13 ` huangjianan
2020-10-19 3:32 ` Gao Xiang via Linux-erofs
2020-10-19 3:37 ` 回复: " Huang Jianan
1 sibling, 1 reply; 18+ messages in thread
From: huangjianan @ 2020-10-19 3:13 UTC (permalink / raw)
To: Gao Xiang; +Cc: guoweichao, linux-erofs, zhangshiming
[-- Attachment #1: Type: text/plain, Size: 2536 bytes --]
Hi, Gao Xiang，
________________________________
发件人： Gao Xiang<mailto:hsiangkao@aol.com>
发送时间： 2020-10-17 13:16
收件人： linux-erofs<mailto:linux-erofs@lists.ozlabs.org>
抄送： Huang Jianan<mailto:huangjianan@oppo.com>; Li Guifu<mailto:bluce.liguifu@huawei.com>; Li Guifu<mailto:bluce.lee@aliyun.com>; Chao Yu<mailto:chao@kernel.org>; Guo Weichao<mailto:guoweichao@oppo.com>; Zhang Shiming<mailto:zhangshiming@oppo.com>; Gao Xiang<mailto:hsiangkao@aol.com>
主题： [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
so more easy to understand.
[ let's fold in to the original patch. ]
Cc: Huang Jianan <huangjianan@oppo.com>
Signed-off-by: Gao Xiang <hsiangkao@aol.com>
---
fuse/read.c | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/fuse/read.c b/fuse/read.c
index 0d0e3b0fa468..dd44adaa1c40 100644
--- a/fuse/read.c
+++ b/fuse/read.c
@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
Z_EROFS_COMPRESSION_LZ4 :
Z_EROFS_COMPRESSION_SHIFTED;
- if (end >= map.m_la + map.m_llen) {
- count = map.m_llen;
- partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
- } else {
+ /*
+ * trim to the needed size if the returned extent is quite
+ * larger than requested, and set up partial flag as well.
+ */
+ if (end < map.m_la + map.m_llen) {
count = end - map.m_la;
partial = true;
+ } else {
+ ASSERT(end == map.m_la + map_m_llen);
I think you mean map.m_llen intesad of map_m_llen.
Besides, I don't understand why add ASSERT here.
I think this condition will be true if offset+size is exactly the end of a compressed block?
+ count = map.m_llen;
+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
}
if ((off_t)map.m_la < offset) {
--
2.24.0
Thanks,
Jianan
________________________________
OPPO
本电子邮件及其附件含有OPPO公司的保密信息，仅限于邮件指明的收件人使用（包含个人及群组）。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件，请立即以电子邮件通知发件人并删除本邮件及其附件。
This e-mail and its attachments contain confidential information from OPPO, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
[-- Attachment #2: Type: text/html, Size: 7286 bytes --]
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: 回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
2020-10-19 3:13 ` 回复: " huangjianan
@ 2020-10-19 3:32 ` Gao Xiang via Linux-erofs
2020-10-19 4:49 ` 回复: " Huang Jianan
0 siblings, 1 reply; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-19 3:32 UTC (permalink / raw)
To: huangjianan; +Cc: guoweichao, linux-erofs, zhangshiming
Hi Jianan,
On Mon, Oct 19, 2020 at 11:13:42AM +0800, huangjianan@oppo.com wrote:
> Hi, Gao Xiang锟斤拷
> ________________________________
>
> 锟斤拷锟斤拷锟剿ｏ拷 Gao Xiang<mailto:hsiangkao@aol.com>
> 锟斤拷锟斤拷时锟戒： 2020-10-17 13:16
> 锟秸硷拷锟剿ｏ拷 linux-erofs<mailto:linux-erofs@lists.ozlabs.org>
> 锟斤拷锟酵ｏ拷 Huang Jianan<mailto:huangjianan@oppo.com>; Li Guifu<mailto:bluce.liguifu@huawei.com>; Li Guifu<mailto:bluce.lee@aliyun.com>; Chao Yu<mailto:chao@kernel.org>; Guo Weichao<mailto:guoweichao@oppo.com>; Zhang Shiming<mailto:zhangshiming@oppo.com>; Gao Xiang<mailto:hsiangkao@aol.com>
> 锟斤拷锟解： [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
> so more easy to understand.
>
> [ let's fold in to the original patch. ]
> Cc: Huang Jianan <huangjianan@oppo.com>
> Signed-off-by: Gao Xiang <hsiangkao@aol.com>
> ---
> fuse/read.c | 13 +++++++++----
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/fuse/read.c b/fuse/read.c
> index 0d0e3b0fa468..dd44adaa1c40 100644
> --- a/fuse/read.c
> +++ b/fuse/read.c
> @@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
> Z_EROFS_COMPRESSION_LZ4 :
> Z_EROFS_COMPRESSION_SHIFTED;
> - if (end >= map.m_la + map.m_llen) {
> - count = map.m_llen;
> - partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
> - } else {
> + /*
> + * trim to the needed size if the returned extent is quite
> + * larger than requested, and set up partial flag as well.
> + */
> + if (end < map.m_la + map.m_llen) {
> count = end - map.m_la;
> partial = true;
> + } else {
> + ASSERT(end == map.m_la + map_m_llen);
>
> I think you mean map.m_llen intesad of map_m_llen.
> Besides, I don't understand why add ASSERT here.
> I think this condition will be true if offset+size is exactly the end of a compressed block?
Thanks for your question.
The idea is that we requested the extent with
map.m_la = end - 1;
ret = z_erofs_map_blocks_iter(vnode, &map);
if (ret)
return ret;
so the extent must include "end - 1", so
it's impossible that "end > map.m_la + map.m_llen"
(invalid return).
or the entire extent would be holed extent, anyway,
that is another extent rather than a data extent.
(BTW, the up-to-date commits is at
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=wip/experimental_fuse
kindly check out them as well :) )
Thanks,
Gao Xiang
>
> + count = map.m_llen;
> + partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
> }
> if ((off_t)map.m_la < offset) {
> --
> 2.24.0
>
> Thanks,
> Jianan
^ permalink raw reply [flat|nested] 18+ messages in thread
* 回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
2020-10-17 5:16 ` [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling Gao Xiang via Linux-erofs
2020-10-19 3:13 ` 回复: " huangjianan
@ 2020-10-19 3:37 ` Huang Jianan
2020-10-19 3:43 ` Gao Xiang via Linux-erofs
1 sibling, 1 reply; 18+ messages in thread
From: Huang Jianan @ 2020-10-19 3:37 UTC (permalink / raw)
To: Gao Xiang; +Cc: zhangshiming, linux-erofs, guoweichao
Hi, Gao Xiang,
Sorry for the format of the previous email.
>so more easy to understand.
>
>[ let's fold in to the original patch. ]
>Cc: Huang Jianan <huangjianan@oppo.com>
>Signed-off-by: Gao Xiang <hsiangkao@aol.com>
>---
> fuse/read.c | 13 +++++++++----
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
>diff --git a/fuse/read.c b/fuse/read.c
>index 0d0e3b0fa468..dd44adaa1c40 100644
>--- a/fuse/read.c
>+++ b/fuse/read.c
>@@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
> Z_EROFS_COMPRESSION_LZ4 :
> Z_EROFS_COMPRESSION_SHIFTED;
>
>- if (end >= map.m_la + map.m_llen) {
>- count = map.m_llen;
>- partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
>- } else {
>+ /*
>+ * trim to the needed size if the returned extent is quite
>+ * larger than requested, and set up partial flag as well.
>+ */
>+ if (end < map.m_la + map.m_llen) {
> count = end - map.m_la;
> partial = true;
>+ } else {
>+ ASSERT(end == map.m_la + map_m_llen);
I think you mean map.m_llen intesad of map_m_llen.
Besides, I don't understand why add ASSERT here.
I think this condition will be true if offset+size is exactly the end of a compressed block?
>+ count = map.m_llen;
>+ partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
> }
>
> if ((off_t)map.m_la < offset) {
>--
>2.24.0
>
Thanks,
Jianan
________________________________
OPPO
本电子邮件及其附件含有OPPO公司的保密信息，仅限于邮件指明的收件人使用（包含个人及群组）。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件，请立即以电子邮件通知发件人并删除本邮件及其附件。
This e-mail and its attachments contain confidential information from OPPO, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 回复: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
2020-10-19 3:37 ` 回复: " Huang Jianan
@ 2020-10-19 3:43 ` Gao Xiang via Linux-erofs
0 siblings, 0 replies; 18+ messages in thread
From: Gao Xiang via Linux-erofs @ 2020-10-19 3:43 UTC (permalink / raw)
To: Huang Jianan; +Cc: zhangshiming, linux-erofs, guoweichao
Hi Jianan,
On Mon, Oct 19, 2020 at 11:37:51AM +0800, Huang Jianan wrote:
> Hi, Gao Xiang,
> Sorry for the format of the previous email.
I've replied the previous email at
https://lore.kernel.org/r/20201019033251.GA29138@hsiangkao-HP-ZHAN-66-Pro-G1
As for the email format, I think you can give a try to use mutt or
Thunderbird (assume your company PC is in MS Windows) , which is
more widely used in open-source communities :)
Thanks,
Gao Xiang
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* 回复: Re: [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
2020-10-19 3:32 ` Gao Xiang via Linux-erofs
@ 2020-10-19 4:49 ` Huang Jianan
0 siblings, 0 replies; 18+ messages in thread
From: Huang Jianan @ 2020-10-19 4:49 UTC (permalink / raw)
To: Gao Xiang; +Cc: zhangshiming, linux-erofs, guoweichao
Hi, Gao Xiang,
Thanks for you answer, I understand now.
In addition, I sent a patch to fix the clerical error here.
It seems that the compilation won't be affected under the default configuration.
Thanks,
Jianan
--------------
Huang Jianan
>Hi Jianan,
>
>On Mon, Oct 19, 2020 at 11:13:42AM +0800, huangjianan@oppo.com wrote:
>> Hi, Gao Xiang锟斤拷
>> ________________________________
>>
>> 锟斤拷锟斤拷锟剿ｏ拷 Gao Xiang<mailto:hsiangkao@aol.com>
>> 锟斤拷锟斤拷时锟戒： 2020-10-17 13:16
>> 锟秸硷拷锟剿ｏ拷 linux-erofs<mailto:linux-erofs@lists.ozlabs.org>
>> 锟斤拷锟酵ｏ拷 Huang Jianan<mailto:huangjianan@oppo.com>; Li Guifu<mailto:bluce.liguifu@huawei.com>; Li Guifu<mailto:bluce.lee@aliyun.com>; Chao Yu<mailto:chao@kernel.org>; Guo Weichao<mailto:guoweichao@oppo.com>; Zhang Shiming<mailto:zhangshiming@oppo.com>; Gao Xiang<mailto:hsiangkao@aol.com>
>> 锟斤拷锟解： [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling
>> so more easy to understand.
>>
>> [ let's fold in to the original patch. ]
>> Cc: Huang Jianan <huangjianan@oppo.com>
>> Signed-off-by: Gao Xiang <hsiangkao@aol.com>
>> ---
>> fuse/read.c | 13 +++++++++----
>> 1 file changed, 9 insertions(+), 4 deletions(-)
>>
>> diff --git a/fuse/read.c b/fuse/read.c
>> index 0d0e3b0fa468..dd44adaa1c40 100644
>> --- a/fuse/read.c
>> +++ b/fuse/read.c
>> @@ -112,12 +112,17 @@ size_t erofs_read_data_compression(struct erofs_vnode *vnode, char *buffer,
>> Z_EROFS_COMPRESSION_LZ4 :
>> Z_EROFS_COMPRESSION_SHIFTED;
>> - if (end >= map.m_la + map.m_llen) {
>> - count = map.m_llen;
>> - partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
>> - } else {
>> + /*
>> + * trim to the needed size if the returned extent is quite
>> + * larger than requested, and set up partial flag as well.
>> + */
>> + if (end < map.m_la + map.m_llen) {
>> count = end - map.m_la;
>> partial = true;
>> + } else {
>> + ASSERT(end == map.m_la + map_m_llen);
>>
>> I think you mean map.m_llen intesad of map_m_llen.
>> Besides, I don't understand why add ASSERT here.
>> I think this condition will be true if offset+size is exactly the end of a compressed block?
>
>Thanks for your question.
>The idea is that we requested the extent with
>
> map.m_la = end - 1;
>
> ret = z_erofs_map_blocks_iter(vnode, &map);
> if (ret)
> return ret;
>
>so the extent must include "end - 1", so
>it's impossible that "end > map.m_la + map.m_llen"
>(invalid return).
>
>or the entire extent would be holed extent, anyway,
>that is another extent rather than a data extent.
>
>(BTW, the up-to-date commits is at
>https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=wip/experimental_fuse
>kindly check out them as well :) )
>
>Thanks,
>Gao Xiang
>
>
>>
>> + count = map.m_llen;
>> + partial = !(map.m_flags & EROFS_MAP_FULL_MAPPED);
>> }
>> if ((off_t)map.m_la < offset) {
>> --
>> 2.24.0
>>
>> Thanks,
>> Jianan
>
________________________________
OPPO
本电子邮件及其附件含有OPPO公司的保密信息，仅限于邮件指明的收件人使用（包含个人及群组）。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件，请立即以电子邮件通知发件人并删除本邮件及其附件。
This e-mail and its attachments contain confidential information from OPPO, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2020-10-19 4:50 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20201017051621.7810-1-hsiangkao.ref@aol.com>
2020-10-17 5:16 ` [WIP] [PATCH 00/12] erofs-utils: introduce fuse implementation Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 01/12] " Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 02/12] erofs-utils: fuse: support read special file Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 03/12] erofs-utils: fuse: support read compressed file Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 04/12] erofs-utils: fuse: adjust larger extent handling Gao Xiang via Linux-erofs
2020-10-19 3:13 ` 回复: " huangjianan
2020-10-19 3:32 ` Gao Xiang via Linux-erofs
2020-10-19 4:49 ` 回复: " Huang Jianan
2020-10-19 3:37 ` 回复: " Huang Jianan
2020-10-19 3:43 ` Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 05/12] erofs-utils: fuse: use proper expression about inode size Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 06/12] erofs-utils: fuse: drop ofs_out Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 07/12] erofs-utils: fuse: refuse a undefined shifted cluster behavior Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 08/12] erofs-utils: fuse: drop z_erofs_shifted_transform() Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 09/12] erofs-utils: fuse: rename ofs_head and outputsize Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 10/12] erofs-utils: fuse: cleanup erofs_read_data_compression() Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 11/12] erofs-utils: fuse: move up mpage in struct erofs_map_blocks Gao Xiang via Linux-erofs
2020-10-17 5:16 ` [WIP] [PATCH 12/12] erofs-utils: fuse: fix up source headers Gao Xiang via Linux-erofs
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).