From: Gao Xiang <gaoxiang25@huawei.com> To: Alexander Viro <viro@zeniv.linux.org.uk>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Andrew Morton <akpm@linux-foundation.org>, Stephen Rothwell <sfr@canb.auug.org.au>, Theodore Ts'o <tytso@mit.edu>, "David Sterba" <dsterba@suse.cz>, Amir Goldstein <amir73il@gmail.com>, "Christoph Hellwig" <hch@infradead.org>, "Darrick J . Wong" <darrick.wong@oracle.com>, Dave Chinner <david@fromorbit.com>, Jaegeuk Kim <jaegeuk@kernel.org>, "Jan Kara" <jack@suse.cz>, Linus Torvalds <torvalds@linux-foundation.org> Cc: <linux-fsdevel@vger.kernel.org>, <devel@driverdev.osuosl.org>, LKML <linux-kernel@vger.kernel.org>, <linux-erofs@lists.ozlabs.org>, Chao Yu <yuchao0@huawei.com>, Miao Xie <miaoxie@huawei.com>, Li Guifu <bluce.liguifu@huawei.com>, Fang Wei <fangwei1@huawei.com>, Gao Xiang <gaoxiang25@huawei.com> Subject: [PATCH v5 21/24] erofs: introduce LZ4 decompression inplace Date: Tue, 30 Jul 2019 15:14:10 +0800 [thread overview] Message-ID: <20190730071413.11871-22-gaoxiang25@huawei.com> (raw) In-Reply-To: <20190730071413.11871-1-gaoxiang25@huawei.com> compressed data will be usually loaded into last pages of the extent (the last page for 4k) for in-place decompression (more specifically, in-place IO), as ilustration below, start of compressed logical extent | end of this logical extent | | ______v___________________________v________ ... | page 6 | page 7 | page 8 | page 9 | ... |__________|__________|__________|__________| . ^ . ^ . |compressed| . | data | . . . |< dstsize >|<margin>| oend iend op ip Therefore, it's possible to do decompression inplace (thus no memcpy at all) if the margin is sufficient and safe enough [1], and it can be implemented only for fixed-size output compression compared with fixed-size input compression. No memcpy for most of in-place IO (about 99% of enwik9) after decompression inplace is implemented and sequential read will be improved of course (see the following patches for test results). [1] https://github.com/lz4/lz4/commit/b17f578a919b7e6b078cede2d52be29dd48c8e8c https://github.com/lz4/lz4/commit/5997e139f53169fa3a1c1b4418d2452a90b01602 Signed-off-by: Gao Xiang <gaoxiang25@huawei.com> --- fs/erofs/decompressor.c | 36 ++++++++++++++++++++++++++++++++---- fs/erofs/erofs_fs.h | 2 +- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c index 2523f6ab552b..e882a8ff0903 100644 --- a/fs/erofs/decompressor.c +++ b/fs/erofs/decompressor.c @@ -15,6 +15,9 @@ #endif #define LZ4_MAX_DISTANCE_PAGES (DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE) + 1) +#ifndef LZ4_DECOMPRESS_INPLACE_MARGIN +#define LZ4_DECOMPRESS_INPLACE_MARGIN(srcsize) (((srcsize) >> 8) + 32) +#endif struct z_erofs_decompressor { /* @@ -117,7 +120,7 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) { unsigned int inputmargin, inlen; u8 *src; - bool copied; + bool copied, support_0padding; int ret; if (rq->inputsize > PAGE_SIZE) @@ -125,13 +128,38 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) src = kmap_atomic(*rq->in); inputmargin = 0; + support_0padding = false; + + /* decompression inplace is only safe when 0padding is enabled */ + if (EROFS_SB(rq->sb)->requirements & EROFS_REQUIREMENT_LZ4_0PADDING) { + support_0padding = true; + + while (!src[inputmargin & ~PAGE_MASK]) + if (!(++inputmargin & ~PAGE_MASK)) + break; + + if (inputmargin >= rq->inputsize) { + kunmap_atomic(src); + return -EIO; + } + } copied = false; inlen = rq->inputsize - inputmargin; if (rq->inplace_io) { - src = generic_copy_inplace_data(rq, src, inputmargin); - inputmargin = 0; - copied = true; + const uint oend = (rq->pageofs_out + + rq->outputsize) & ~PAGE_MASK; + const uint nr = PAGE_ALIGN(rq->pageofs_out + + rq->outputsize) >> PAGE_SHIFT; + + if (rq->partial_decoding || !support_0padding || + rq->out[nr - 1] != rq->in[0] || + rq->inputsize - oend < + LZ4_DECOMPRESS_INPLACE_MARGIN(inlen)) { + src = generic_copy_inplace_data(rq, src, inputmargin); + inputmargin = 0; + copied = true; + } } ret = LZ4_decompress_safe_partial(src + inputmargin, out, diff --git a/fs/erofs/erofs_fs.h b/fs/erofs/erofs_fs.h index b307060dd220..25bda459f2e6 100644 --- a/fs/erofs/erofs_fs.h +++ b/fs/erofs/erofs_fs.h @@ -17,7 +17,7 @@ * incompatible with this kernel version. */ #define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001 -#define EROFS_ALL_REQUIREMENTS 0 +#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING struct erofs_super_block { /* 0 */__le32 magic; /* in the little endian */ -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: gaoxiang25@huawei.com (Gao Xiang) Subject: [PATCH v5 21/24] erofs: introduce LZ4 decompression inplace Date: Tue, 30 Jul 2019 15:14:10 +0800 [thread overview] Message-ID: <20190730071413.11871-22-gaoxiang25@huawei.com> (raw) In-Reply-To: <20190730071413.11871-1-gaoxiang25@huawei.com> compressed data will be usually loaded into last pages of the extent (the last page for 4k) for in-place decompression (more specifically, in-place IO), as ilustration below, start of compressed logical extent | end of this logical extent | | ______v___________________________v________ ... | page 6 | page 7 | page 8 | page 9 | ... |__________|__________|__________|__________| . ^ . ^ . |compressed| . | data | . . . |< dstsize >|<margin>| oend iend op ip Therefore, it's possible to do decompression inplace (thus no memcpy at all) if the margin is sufficient and safe enough [1], and it can be implemented only for fixed-size output compression compared with fixed-size input compression. No memcpy for most of in-place IO (about 99% of enwik9) after decompression inplace is implemented and sequential read will be improved of course (see the following patches for test results). [1] https://github.com/lz4/lz4/commit/b17f578a919b7e6b078cede2d52be29dd48c8e8c https://github.com/lz4/lz4/commit/5997e139f53169fa3a1c1b4418d2452a90b01602 Signed-off-by: Gao Xiang <gaoxiang25 at huawei.com> --- fs/erofs/decompressor.c | 36 ++++++++++++++++++++++++++++++++---- fs/erofs/erofs_fs.h | 2 +- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c index 2523f6ab552b..e882a8ff0903 100644 --- a/fs/erofs/decompressor.c +++ b/fs/erofs/decompressor.c @@ -15,6 +15,9 @@ #endif #define LZ4_MAX_DISTANCE_PAGES (DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE) + 1) +#ifndef LZ4_DECOMPRESS_INPLACE_MARGIN +#define LZ4_DECOMPRESS_INPLACE_MARGIN(srcsize) (((srcsize) >> 8) + 32) +#endif struct z_erofs_decompressor { /* @@ -117,7 +120,7 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) { unsigned int inputmargin, inlen; u8 *src; - bool copied; + bool copied, support_0padding; int ret; if (rq->inputsize > PAGE_SIZE) @@ -125,13 +128,38 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out) src = kmap_atomic(*rq->in); inputmargin = 0; + support_0padding = false; + + /* decompression inplace is only safe when 0padding is enabled */ + if (EROFS_SB(rq->sb)->requirements & EROFS_REQUIREMENT_LZ4_0PADDING) { + support_0padding = true; + + while (!src[inputmargin & ~PAGE_MASK]) + if (!(++inputmargin & ~PAGE_MASK)) + break; + + if (inputmargin >= rq->inputsize) { + kunmap_atomic(src); + return -EIO; + } + } copied = false; inlen = rq->inputsize - inputmargin; if (rq->inplace_io) { - src = generic_copy_inplace_data(rq, src, inputmargin); - inputmargin = 0; - copied = true; + const uint oend = (rq->pageofs_out + + rq->outputsize) & ~PAGE_MASK; + const uint nr = PAGE_ALIGN(rq->pageofs_out + + rq->outputsize) >> PAGE_SHIFT; + + if (rq->partial_decoding || !support_0padding || + rq->out[nr - 1] != rq->in[0] || + rq->inputsize - oend < + LZ4_DECOMPRESS_INPLACE_MARGIN(inlen)) { + src = generic_copy_inplace_data(rq, src, inputmargin); + inputmargin = 0; + copied = true; + } } ret = LZ4_decompress_safe_partial(src + inputmargin, out, diff --git a/fs/erofs/erofs_fs.h b/fs/erofs/erofs_fs.h index b307060dd220..25bda459f2e6 100644 --- a/fs/erofs/erofs_fs.h +++ b/fs/erofs/erofs_fs.h @@ -17,7 +17,7 @@ * incompatible with this kernel version. */ #define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001 -#define EROFS_ALL_REQUIREMENTS 0 +#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING struct erofs_super_block { /* 0 */__le32 magic; /* in the little endian */ -- 2.17.1
next prev parent reply other threads:[~2019-07-30 7:15 UTC|newest] Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-07-30 7:13 [PATCH v5 00/24] erofs: promote erofs from staging Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 01/24] erofs: add on-disk layout Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 02/24] erofs: add erofs in-memory stuffs Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 03/24] erofs: add super block operations Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 04/24] erofs: add raw address_space operations Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 05/24] erofs: add inode operations Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 06/24] erofs: support special inode Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 07/24] erofs: add directory operations Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 08/24] erofs: add namei functions Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 09/24] erofs: support tracepoint Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:13 ` [PATCH v5 10/24] erofs: update Kconfig and Makefile Gao Xiang 2019-07-30 7:13 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 11/24] erofs: introduce xattr & posixacl support Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 12/24] erofs: introduce tagged pointer Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-31 13:01 ` Jan Kara 2019-07-31 13:01 ` Jan Kara 2019-07-31 13:20 ` Gao Xiang 2019-07-31 13:20 ` Gao Xiang 2019-07-31 13:30 ` Gao Xiang 2019-07-31 13:30 ` Gao Xiang 2019-07-31 13:52 ` Gao Xiang 2019-07-31 13:52 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 13/24] erofs: add compression indexes support Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 14/24] erofs: introduce superblock registration Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 15/24] erofs: introduce erofs shrinker Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 16/24] erofs: introduce workstation for decompression Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 17/24] erofs: introduce per-CPU buffers implementation Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 18/24] erofs: introduce pagevec for decompression subsystem Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 19/24] erofs: add erofs_allocpage() Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 20/24] erofs: introduce generic decompression backend Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` Gao Xiang [this message] 2019-07-30 7:14 ` [PATCH v5 21/24] erofs: introduce LZ4 decompression inplace Gao Xiang 2019-07-30 7:14 ` [PATCH v5 22/24] erofs: introduce the decompression frontend Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 23/24] erofs: introduce cached decompression Gao Xiang 2019-07-30 7:14 ` Gao Xiang 2019-07-30 7:14 ` [PATCH v5 24/24] erofs: add document Gao Xiang 2019-07-30 7:14 ` Gao Xiang
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190730071413.11871-22-gaoxiang25@huawei.com \ --to=gaoxiang25@huawei.com \ --cc=akpm@linux-foundation.org \ --cc=amir73il@gmail.com \ --cc=bluce.liguifu@huawei.com \ --cc=darrick.wong@oracle.com \ --cc=david@fromorbit.com \ --cc=devel@driverdev.osuosl.org \ --cc=dsterba@suse.cz \ --cc=fangwei1@huawei.com \ --cc=gregkh@linuxfoundation.org \ --cc=hch@infradead.org \ --cc=jack@suse.cz \ --cc=jaegeuk@kernel.org \ --cc=linux-erofs@lists.ozlabs.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=miaoxie@huawei.com \ --cc=sfr@canb.auug.org.au \ --cc=torvalds@linux-foundation.org \ --cc=tytso@mit.edu \ --cc=viro@zeniv.linux.org.uk \ --cc=yuchao0@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.