From: Gao Xiang <hsiangkao@aol.com>
To: Chao Yu <yuchao0@huawei.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: devel@driverdev.osuosl.org, LKML <linux-kernel@vger.kernel.org>,
linux-fsdevel@vger.kernel.org, linux-erofs@lists.ozlabs.org,
Chao Yu <chao@kernel.org>, Miao Xie <miaoxie@huawei.com>,
Fang Wei <fangwei1@huawei.com>, Du Wei <weidu.du@huawei.com>,
Gao Xiang <gaoxiang25@huawei.com>
Subject: [PATCH v3 6/8] staging: erofs: introduce LZ4 decompression inplace
Date: Mon, 24 Jun 2019 15:22:56 +0800 [thread overview]
Message-ID: <20190624072258.28362-7-hsiangkao@aol.com> (raw)
In-Reply-To: <20190624072258.28362-1-hsiangkao@aol.com>
From: Gao Xiang <gaoxiang25@huawei.com>
compressed data will be usually loaded into last pages of
the extent (the last page for 4k) for in-place decompression
(more specifically, in-place IO), as ilustration below,
start of compressed logical extent
| end of this logical extent
| |
______v___________________________v________
... | page 6 | page 7 | page 8 | page 9 | ...
|__________|__________|__________|__________|
. ^ . ^
. |compressed|
. | data |
. . .
|< dstsize >|<margin>|
oend iend
op ip
Therefore, it's possible to do decompression inplace (thus no
memcpy at all) if the margin is sufficient and safe enough [1],
and it can be implemented only for fixed-size output compression
compared with fixed-size input compression.
No memcpy for most of in-place IO (about 99% of enwik9) after
decompression inplace is implemented and sequential read will
be improved of course (see the following patches for test results).
[1] https://github.com/lz4/lz4/commit/b17f578a919b7e6b078cede2d52be29dd48c8e8c
https://github.com/lz4/lz4/commit/5997e139f53169fa3a1c1b4418d2452a90b01602
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Gao Xiang <gaoxiang25@huawei.com>
---
drivers/staging/erofs/compress.h | 1 +
drivers/staging/erofs/decompressor.c | 36 ++++++++++++++++++++++++----
drivers/staging/erofs/erofs_fs.h | 3 ++-
3 files changed, 35 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/erofs/compress.h b/drivers/staging/erofs/compress.h
index ebeccb1f4eae..c43aa3374d28 100644
--- a/drivers/staging/erofs/compress.h
+++ b/drivers/staging/erofs/compress.h
@@ -17,6 +17,7 @@ enum {
};
struct z_erofs_decompress_req {
+ struct super_block *sb;
struct page **in, **out;
unsigned short pageofs_out;
diff --git a/drivers/staging/erofs/decompressor.c b/drivers/staging/erofs/decompressor.c
index df8fd68a338b..80f1f39719ba 100644
--- a/drivers/staging/erofs/decompressor.c
+++ b/drivers/staging/erofs/decompressor.c
@@ -14,6 +14,9 @@
#endif
#define LZ4_MAX_DISTANCE_PAGES DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE)
+#ifndef LZ4_DECOMPRESS_INPLACE_MARGIN
+#define LZ4_DECOMPRESS_INPLACE_MARGIN(srcsize) (((srcsize) >> 8) + 32)
+#endif
struct z_erofs_decompressor {
/*
@@ -112,7 +115,7 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
{
unsigned int inputmargin, inlen;
u8 *src;
- bool copied;
+ bool copied, support_0padding;
int ret;
if (rq->inputsize > PAGE_SIZE)
@@ -120,13 +123,38 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
src = kmap_atomic(*rq->in);
inputmargin = 0;
+ support_0padding = false;
+
+ /* decompression inplace is only safe when 0padding is enabled */
+ if (EROFS_SB(rq->sb)->requirements & EROFS_REQUIREMENT_LZ4_0PADDING) {
+ support_0padding = true;
+
+ while (!src[inputmargin & ~PAGE_MASK])
+ if (!(++inputmargin & ~PAGE_MASK))
+ break;
+
+ if (inputmargin >= rq->inputsize) {
+ kunmap_atomic(src);
+ return -EIO;
+ }
+ }
copied = false;
inlen = rq->inputsize - inputmargin;
if (rq->inplace_io) {
- src = generic_copy_inplace_data(rq, src, inputmargin);
- inputmargin = 0;
- copied = true;
+ const uint oend = (rq->pageofs_out +
+ rq->outputsize) & ~PAGE_MASK;
+ const uint nr = PAGE_ALIGN(rq->pageofs_out +
+ rq->outputsize) >> PAGE_SHIFT;
+
+ if (rq->partial_decoding || !support_0padding ||
+ rq->out[nr - 1] != rq->in[0] ||
+ rq->inputsize - oend <
+ LZ4_DECOMPRESS_INPLACE_MARGIN(inlen)) {
+ src = generic_copy_inplace_data(rq, src, inputmargin);
+ inputmargin = 0;
+ copied = true;
+ }
}
ret = LZ4_decompress_safe_partial(src + inputmargin, out,
diff --git a/drivers/staging/erofs/erofs_fs.h b/drivers/staging/erofs/erofs_fs.h
index 9a9aaf2d9fbb..9f61abb7c1ca 100644
--- a/drivers/staging/erofs/erofs_fs.h
+++ b/drivers/staging/erofs/erofs_fs.h
@@ -21,7 +21,8 @@
* Any bits that aren't in EROFS_ALL_REQUIREMENTS should be
* incompatible with this kernel version.
*/
-#define EROFS_ALL_REQUIREMENTS 0
+#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001
+#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING
struct erofs_super_block {
/* 0 */__le32 magic; /* in the little endian */
--
2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Gao Xiang <hsiangkao@aol.com>
To: Chao Yu <yuchao0@huawei.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: devel@driverdev.osuosl.org, linux-erofs@lists.ozlabs.org,
Chao Yu <chao@kernel.org>, LKML <linux-kernel@vger.kernel.org>,
Du Wei <weidu.du@huawei.com>, Fang Wei <fangwei1@huawei.com>,
linux-fsdevel@vger.kernel.org, Miao Xie <miaoxie@huawei.com>
Subject: [PATCH v3 6/8] staging: erofs: introduce LZ4 decompression inplace
Date: Mon, 24 Jun 2019 15:22:56 +0800 [thread overview]
Message-ID: <20190624072258.28362-7-hsiangkao@aol.com> (raw)
In-Reply-To: <20190624072258.28362-1-hsiangkao@aol.com>
From: Gao Xiang <gaoxiang25@huawei.com>
compressed data will be usually loaded into last pages of
the extent (the last page for 4k) for in-place decompression
(more specifically, in-place IO), as ilustration below,
start of compressed logical extent
| end of this logical extent
| |
______v___________________________v________
... | page 6 | page 7 | page 8 | page 9 | ...
|__________|__________|__________|__________|
. ^ . ^
. |compressed|
. | data |
. . .
|< dstsize >|<margin>|
oend iend
op ip
Therefore, it's possible to do decompression inplace (thus no
memcpy at all) if the margin is sufficient and safe enough [1],
and it can be implemented only for fixed-size output compression
compared with fixed-size input compression.
No memcpy for most of in-place IO (about 99% of enwik9) after
decompression inplace is implemented and sequential read will
be improved of course (see the following patches for test results).
[1] https://github.com/lz4/lz4/commit/b17f578a919b7e6b078cede2d52be29dd48c8e8c
https://github.com/lz4/lz4/commit/5997e139f53169fa3a1c1b4418d2452a90b01602
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Gao Xiang <gaoxiang25@huawei.com>
---
drivers/staging/erofs/compress.h | 1 +
drivers/staging/erofs/decompressor.c | 36 ++++++++++++++++++++++++----
drivers/staging/erofs/erofs_fs.h | 3 ++-
3 files changed, 35 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/erofs/compress.h b/drivers/staging/erofs/compress.h
index ebeccb1f4eae..c43aa3374d28 100644
--- a/drivers/staging/erofs/compress.h
+++ b/drivers/staging/erofs/compress.h
@@ -17,6 +17,7 @@ enum {
};
struct z_erofs_decompress_req {
+ struct super_block *sb;
struct page **in, **out;
unsigned short pageofs_out;
diff --git a/drivers/staging/erofs/decompressor.c b/drivers/staging/erofs/decompressor.c
index df8fd68a338b..80f1f39719ba 100644
--- a/drivers/staging/erofs/decompressor.c
+++ b/drivers/staging/erofs/decompressor.c
@@ -14,6 +14,9 @@
#endif
#define LZ4_MAX_DISTANCE_PAGES DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE)
+#ifndef LZ4_DECOMPRESS_INPLACE_MARGIN
+#define LZ4_DECOMPRESS_INPLACE_MARGIN(srcsize) (((srcsize) >> 8) + 32)
+#endif
struct z_erofs_decompressor {
/*
@@ -112,7 +115,7 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
{
unsigned int inputmargin, inlen;
u8 *src;
- bool copied;
+ bool copied, support_0padding;
int ret;
if (rq->inputsize > PAGE_SIZE)
@@ -120,13 +123,38 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
src = kmap_atomic(*rq->in);
inputmargin = 0;
+ support_0padding = false;
+
+ /* decompression inplace is only safe when 0padding is enabled */
+ if (EROFS_SB(rq->sb)->requirements & EROFS_REQUIREMENT_LZ4_0PADDING) {
+ support_0padding = true;
+
+ while (!src[inputmargin & ~PAGE_MASK])
+ if (!(++inputmargin & ~PAGE_MASK))
+ break;
+
+ if (inputmargin >= rq->inputsize) {
+ kunmap_atomic(src);
+ return -EIO;
+ }
+ }
copied = false;
inlen = rq->inputsize - inputmargin;
if (rq->inplace_io) {
- src = generic_copy_inplace_data(rq, src, inputmargin);
- inputmargin = 0;
- copied = true;
+ const uint oend = (rq->pageofs_out +
+ rq->outputsize) & ~PAGE_MASK;
+ const uint nr = PAGE_ALIGN(rq->pageofs_out +
+ rq->outputsize) >> PAGE_SHIFT;
+
+ if (rq->partial_decoding || !support_0padding ||
+ rq->out[nr - 1] != rq->in[0] ||
+ rq->inputsize - oend <
+ LZ4_DECOMPRESS_INPLACE_MARGIN(inlen)) {
+ src = generic_copy_inplace_data(rq, src, inputmargin);
+ inputmargin = 0;
+ copied = true;
+ }
}
ret = LZ4_decompress_safe_partial(src + inputmargin, out,
diff --git a/drivers/staging/erofs/erofs_fs.h b/drivers/staging/erofs/erofs_fs.h
index 9a9aaf2d9fbb..9f61abb7c1ca 100644
--- a/drivers/staging/erofs/erofs_fs.h
+++ b/drivers/staging/erofs/erofs_fs.h
@@ -21,7 +21,8 @@
* Any bits that aren't in EROFS_ALL_REQUIREMENTS should be
* incompatible with this kernel version.
*/
-#define EROFS_ALL_REQUIREMENTS 0
+#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001
+#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING
struct erofs_super_block {
/* 0 */__le32 magic; /* in the little endian */
--
2.17.1
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
WARNING: multiple messages have this Message-ID (diff)
From: hsiangkao@aol.com (Gao Xiang)
Subject: [PATCH v3 6/8] staging: erofs: introduce LZ4 decompression inplace
Date: Mon, 24 Jun 2019 15:22:56 +0800 [thread overview]
Message-ID: <20190624072258.28362-7-hsiangkao@aol.com> (raw)
In-Reply-To: <20190624072258.28362-1-hsiangkao@aol.com>
From: Gao Xiang <gaoxiang25@huawei.com>
compressed data will be usually loaded into last pages of
the extent (the last page for 4k) for in-place decompression
(more specifically, in-place IO), as ilustration below,
start of compressed logical extent
| end of this logical extent
| |
______v___________________________v________
... | page 6 | page 7 | page 8 | page 9 | ...
|__________|__________|__________|__________|
. ^ . ^
. |compressed|
. | data |
. . .
|< dstsize >|<margin>|
oend iend
op ip
Therefore, it's possible to do decompression inplace (thus no
memcpy at all) if the margin is sufficient and safe enough [1],
and it can be implemented only for fixed-size output compression
compared with fixed-size input compression.
No memcpy for most of in-place IO (about 99% of enwik9) after
decompression inplace is implemented and sequential read will
be improved of course (see the following patches for test results).
[1] https://github.com/lz4/lz4/commit/b17f578a919b7e6b078cede2d52be29dd48c8e8c
https://github.com/lz4/lz4/commit/5997e139f53169fa3a1c1b4418d2452a90b01602
Reviewed-by: Chao Yu <yuchao0 at huawei.com>
Signed-off-by: Gao Xiang <gaoxiang25 at huawei.com>
---
drivers/staging/erofs/compress.h | 1 +
drivers/staging/erofs/decompressor.c | 36 ++++++++++++++++++++++++----
drivers/staging/erofs/erofs_fs.h | 3 ++-
3 files changed, 35 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/erofs/compress.h b/drivers/staging/erofs/compress.h
index ebeccb1f4eae..c43aa3374d28 100644
--- a/drivers/staging/erofs/compress.h
+++ b/drivers/staging/erofs/compress.h
@@ -17,6 +17,7 @@ enum {
};
struct z_erofs_decompress_req {
+ struct super_block *sb;
struct page **in, **out;
unsigned short pageofs_out;
diff --git a/drivers/staging/erofs/decompressor.c b/drivers/staging/erofs/decompressor.c
index df8fd68a338b..80f1f39719ba 100644
--- a/drivers/staging/erofs/decompressor.c
+++ b/drivers/staging/erofs/decompressor.c
@@ -14,6 +14,9 @@
#endif
#define LZ4_MAX_DISTANCE_PAGES DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE)
+#ifndef LZ4_DECOMPRESS_INPLACE_MARGIN
+#define LZ4_DECOMPRESS_INPLACE_MARGIN(srcsize) (((srcsize) >> 8) + 32)
+#endif
struct z_erofs_decompressor {
/*
@@ -112,7 +115,7 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
{
unsigned int inputmargin, inlen;
u8 *src;
- bool copied;
+ bool copied, support_0padding;
int ret;
if (rq->inputsize > PAGE_SIZE)
@@ -120,13 +123,38 @@ static int lz4_decompress(struct z_erofs_decompress_req *rq, u8 *out)
src = kmap_atomic(*rq->in);
inputmargin = 0;
+ support_0padding = false;
+
+ /* decompression inplace is only safe when 0padding is enabled */
+ if (EROFS_SB(rq->sb)->requirements & EROFS_REQUIREMENT_LZ4_0PADDING) {
+ support_0padding = true;
+
+ while (!src[inputmargin & ~PAGE_MASK])
+ if (!(++inputmargin & ~PAGE_MASK))
+ break;
+
+ if (inputmargin >= rq->inputsize) {
+ kunmap_atomic(src);
+ return -EIO;
+ }
+ }
copied = false;
inlen = rq->inputsize - inputmargin;
if (rq->inplace_io) {
- src = generic_copy_inplace_data(rq, src, inputmargin);
- inputmargin = 0;
- copied = true;
+ const uint oend = (rq->pageofs_out +
+ rq->outputsize) & ~PAGE_MASK;
+ const uint nr = PAGE_ALIGN(rq->pageofs_out +
+ rq->outputsize) >> PAGE_SHIFT;
+
+ if (rq->partial_decoding || !support_0padding ||
+ rq->out[nr - 1] != rq->in[0] ||
+ rq->inputsize - oend <
+ LZ4_DECOMPRESS_INPLACE_MARGIN(inlen)) {
+ src = generic_copy_inplace_data(rq, src, inputmargin);
+ inputmargin = 0;
+ copied = true;
+ }
}
ret = LZ4_decompress_safe_partial(src + inputmargin, out,
diff --git a/drivers/staging/erofs/erofs_fs.h b/drivers/staging/erofs/erofs_fs.h
index 9a9aaf2d9fbb..9f61abb7c1ca 100644
--- a/drivers/staging/erofs/erofs_fs.h
+++ b/drivers/staging/erofs/erofs_fs.h
@@ -21,7 +21,8 @@
* Any bits that aren't in EROFS_ALL_REQUIREMENTS should be
* incompatible with this kernel version.
*/
-#define EROFS_ALL_REQUIREMENTS 0
+#define EROFS_REQUIREMENT_LZ4_0PADDING 0x00000001
+#define EROFS_ALL_REQUIREMENTS EROFS_REQUIREMENT_LZ4_0PADDING
struct erofs_super_block {
/* 0 */__le32 magic; /* in the little endian */
--
2.17.1
next prev parent reply other threads:[~2019-06-24 7:24 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-24 7:22 [PATCH v3 0/8] staging: erofs: decompression inplace approach Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` [PATCH v3 1/8] staging: erofs: add compacted ondisk compression indexes Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` [PATCH v3 2/8] staging: erofs: add compacted compression indexes support Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` [PATCH v3 3/8] staging: erofs: move per-CPU buffers implementation to utils.c Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:33 ` Chao Yu
2019-06-24 7:33 ` Chao Yu
2019-06-24 7:33 ` Chao Yu
2019-06-24 7:22 ` [PATCH v3 4/8] staging: erofs: move stagingpage operations to compress.h Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` [PATCH v3 5/8] staging: erofs: introduce generic decompression backend Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` Gao Xiang [this message]
2019-06-24 7:22 ` [PATCH v3 6/8] staging: erofs: introduce LZ4 decompression inplace Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` [PATCH v3 7/8] staging: erofs: switch to new decompression backend Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:22 ` [PATCH v3 8/8] staging: erofs: integrate decompression inplace Gao Xiang
2019-06-24 7:22 ` Gao Xiang
2019-06-24 7:34 ` [PATCH v3 0/8] staging: erofs: decompression inplace approach Chao Yu
2019-06-24 7:34 ` Chao Yu
2019-06-24 7:34 ` Chao Yu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190624072258.28362-7-hsiangkao@aol.com \
--to=hsiangkao@aol.com \
--cc=chao@kernel.org \
--cc=devel@driverdev.osuosl.org \
--cc=fangwei1@huawei.com \
--cc=gaoxiang25@huawei.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-erofs@lists.ozlabs.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=miaoxie@huawei.com \
--cc=weidu.du@huawei.com \
--cc=yuchao0@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.