* [PATCH 0/5] erofs: basic sub-page compressed data support
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Gao Xiang
Hi folks,
Recently, there are two new cases so that we need to add a preliminary
sub-page block support for compressed files;
- As Android folks requested, Android ecosystem itself is now switching
to 16k page size for their arm64 devices. They needs an option of
4k-block image compatibility on their new 16k devices;
- Some arm64 cloud servers use 64k page size for their optimized
workloads, but 4k-block EROFS container images need to be parsed too.
So this patchset mainly addresses the requirements above with a very
very simple approach as a start: just allocate short-lived temporary
buffers all the time to keep compressed data if sub-page blocks are
identified. In other words, no inplace/cache decompression for
the preliminary support.
This patchset survives EROFS stress test on my own testfarms.
Thanks,
Gao Xiang
Gao Xiang (5):
erofs: support I/O submission for sub-page compressed blocks
erofs: record `pclustersize` in bytes instead of pages
erofs: fix up compacted indexes for block size < 4096
erofs: refine z_erofs_transform_plain() for sub-page block support
erofs: enable sub-page compressed block support
fs/erofs/decompressor.c | 81 +++++++++------
fs/erofs/inode.c | 6 +-
fs/erofs/zdata.c | 224 ++++++++++++++++++----------------------
fs/erofs/zmap.c | 32 +++---
4 files changed, 169 insertions(+), 174 deletions(-)
--
2.39.3
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 0/5] erofs: basic sub-page compressed data support
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Gao Xiang, LKML
Hi folks,
Recently, there are two new cases so that we need to add a preliminary
sub-page block support for compressed files;
- As Android folks requested, Android ecosystem itself is now switching
to 16k page size for their arm64 devices. They needs an option of
4k-block image compatibility on their new 16k devices;
- Some arm64 cloud servers use 64k page size for their optimized
workloads, but 4k-block EROFS container images need to be parsed too.
So this patchset mainly addresses the requirements above with a very
very simple approach as a start: just allocate short-lived temporary
buffers all the time to keep compressed data if sub-page blocks are
identified. In other words, no inplace/cache decompression for
the preliminary support.
This patchset survives EROFS stress test on my own testfarms.
Thanks,
Gao Xiang
Gao Xiang (5):
erofs: support I/O submission for sub-page compressed blocks
erofs: record `pclustersize` in bytes instead of pages
erofs: fix up compacted indexes for block size < 4096
erofs: refine z_erofs_transform_plain() for sub-page block support
erofs: enable sub-page compressed block support
fs/erofs/decompressor.c | 81 +++++++++------
fs/erofs/inode.c | 6 +-
fs/erofs/zdata.c | 224 ++++++++++++++++++----------------------
fs/erofs/zmap.c | 32 +++---
4 files changed, 169 insertions(+), 174 deletions(-)
--
2.39.3
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 1/5] erofs: support I/O submission for sub-page compressed blocks
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-06 9:10 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Gao Xiang
Add a basic I/O submission path first to support sub-page blocks:
- Temporary short-lived pages will be used entirely;
- In-place I/O pages can be used partially, but compressed pages need
to be able to be mapped in contiguous virtual memory.
As a start, currently cache decompression is explicitly disabled for
sub-page blocks, which will be supported in the future.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/zdata.c | 156 ++++++++++++++++++++++-------------------------
1 file changed, 74 insertions(+), 82 deletions(-)
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index a33cd6757f98..421c0a88a0ca 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1435,86 +1435,85 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
z_erofs_decompressqueue_work(&io->u.work);
}
-static struct page *pickup_page_for_submission(struct z_erofs_pcluster *pcl,
- unsigned int nr,
- struct page **pagepool,
- struct address_space *mc)
+static void z_erofs_fill_bio_vec(struct bio_vec *bvec,
+ struct z_erofs_decompress_frontend *f,
+ struct z_erofs_pcluster *pcl,
+ unsigned int nr,
+ struct address_space *mc)
{
- const pgoff_t index = pcl->obj.index;
gfp_t gfp = mapping_gfp_mask(mc);
bool tocache = false;
-
+ struct z_erofs_bvec *zbv = pcl->compressed_bvecs + nr;
struct address_space *mapping;
- struct page *oldpage, *page;
- int justfound;
+ struct page *page, *oldpage;
+ int justfound, bs = i_blocksize(f->inode);
+ /* Except for inplace pages, the entire page can be used for I/Os */
+ bvec->bv_offset = 0;
+ bvec->bv_len = PAGE_SIZE;
repeat:
- page = READ_ONCE(pcl->compressed_bvecs[nr].page);
- oldpage = page;
-
- if (!page)
+ oldpage = READ_ONCE(zbv->page);
+ if (!oldpage)
goto out_allocpage;
- justfound = (unsigned long)page & 1UL;
- page = (struct page *)((unsigned long)page & ~1UL);
+ justfound = (unsigned long)oldpage & 1UL;
+ page = (struct page *)((unsigned long)oldpage & ~1UL);
+ bvec->bv_page = page;
+ DBG_BUGON(z_erofs_is_shortlived_page(page));
/*
- * preallocated cached pages, which is used to avoid direct reclaim
- * otherwise, it will go inplace I/O path instead.
+ * Handle preallocated cached pages. We tried to allocate such pages
+ * without triggering direct reclaim. If allocation failed, inplace
+ * file-backed pages will be used instead.
*/
if (page->private == Z_EROFS_PREALLOCATED_PAGE) {
- WRITE_ONCE(pcl->compressed_bvecs[nr].page, page);
set_page_private(page, 0);
+ WRITE_ONCE(zbv->page, page);
tocache = true;
goto out_tocache;
}
- mapping = READ_ONCE(page->mapping);
+ mapping = READ_ONCE(page->mapping);
/*
- * file-backed online pages in plcuster are all locked steady,
- * therefore it is impossible for `mapping' to be NULL.
+ * File-backed pages for inplace I/Os are all locked steady,
+ * therefore it is impossible for `mapping` to be NULL.
*/
- if (mapping && mapping != mc)
- /* ought to be unmanaged pages */
- goto out;
-
- /* directly return for shortlived page as well */
- if (z_erofs_is_shortlived_page(page))
- goto out;
+ if (mapping && mapping != mc) {
+ if (zbv->offset < 0)
+ bvec->bv_offset = round_up(-zbv->offset, bs);
+ bvec->bv_len = round_up(zbv->end, bs) - bvec->bv_offset;
+ return;
+ }
lock_page(page);
-
/* only true if page reclaim goes wrong, should never happen */
DBG_BUGON(justfound && PagePrivate(page));
- /* the page is still in manage cache */
+ /* the cached page is still in managed cache */
if (page->mapping == mc) {
- WRITE_ONCE(pcl->compressed_bvecs[nr].page, page);
-
+ WRITE_ONCE(zbv->page, page);
+ /*
+ * The cached page is still available but without a valid
+ * `->private` pcluster hint. Let's reconnect them.
+ */
if (!PagePrivate(page)) {
- /*
- * impossible to be !PagePrivate(page) for
- * the current restriction as well if
- * the page is already in compressed_bvecs[].
- */
DBG_BUGON(!justfound);
-
- justfound = 0;
- set_page_private(page, (unsigned long)pcl);
- SetPagePrivate(page);
+ /* compressed_bvecs[] already takes a ref */
+ attach_page_private(page, pcl);
+ put_page(page);
}
- /* no need to submit io if it is already up-to-date */
+ /* no need to submit if it is already up-to-date */
if (PageUptodate(page)) {
unlock_page(page);
- page = NULL;
+ bvec->bv_page = NULL;
}
- goto out;
+ return;
}
/*
- * the managed page has been truncated, it's unsafe to
- * reuse this one, let's allocate a new cache-managed page.
+ * It has been truncated, so it's unsafe to reuse this one. Let's
+ * allocate a new page for compressed data.
*/
DBG_BUGON(page->mapping);
DBG_BUGON(!justfound);
@@ -1523,25 +1522,23 @@ static struct page *pickup_page_for_submission(struct z_erofs_pcluster *pcl,
unlock_page(page);
put_page(page);
out_allocpage:
- page = erofs_allocpage(pagepool, gfp | __GFP_NOFAIL);
- if (oldpage != cmpxchg(&pcl->compressed_bvecs[nr].page,
- oldpage, page)) {
- erofs_pagepool_add(pagepool, page);
+ page = erofs_allocpage(&f->pagepool, gfp | __GFP_NOFAIL);
+ if (oldpage != cmpxchg(&zbv->page, oldpage, page)) {
+ erofs_pagepool_add(&f->pagepool, page);
cond_resched();
goto repeat;
}
+ bvec->bv_page = page;
out_tocache:
- if (!tocache || add_to_page_cache_lru(page, mc, index + nr, gfp)) {
- /* turn into temporary page if fails (1 ref) */
+ if (!tocache || bs != PAGE_SIZE ||
+ add_to_page_cache_lru(page, mc, pcl->obj.index + nr, gfp)) {
+ /* turn into a temporary shortlived page (1 ref) */
set_page_private(page, Z_EROFS_SHORTLIVED_PAGE);
- goto out;
+ return;
}
attach_page_private(page, pcl);
- /* drop a refcount added by allocpage (then we have 2 refs here) */
+ /* drop a refcount added by allocpage (then 2 refs in total here) */
put_page(page);
-
-out: /* the only exit (for tracing and debugging) */
- return page;
}
static struct z_erofs_decompressqueue *jobqueue_init(struct super_block *sb,
@@ -1596,7 +1593,7 @@ static void move_to_bypass_jobqueue(struct z_erofs_pcluster *pcl,
qtail[JQ_BYPASS] = &pcl->next;
}
-static void z_erofs_decompressqueue_endio(struct bio *bio)
+static void z_erofs_submissionqueue_endio(struct bio *bio)
{
struct z_erofs_decompressqueue *q = bio->bi_private;
blk_status_t err = bio->bi_status;
@@ -1608,7 +1605,6 @@ static void z_erofs_decompressqueue_endio(struct bio *bio)
DBG_BUGON(PageUptodate(page));
DBG_BUGON(z_erofs_page_is_invalidated(page));
-
if (erofs_page_is_managed(EROFS_SB(q->sb), page)) {
if (!err)
SetPageUptodate(page);
@@ -1631,17 +1627,14 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
struct z_erofs_decompressqueue *q[NR_JOBQUEUES];
z_erofs_next_pcluster_t owned_head = f->owned_head;
/* bio is NULL initially, so no need to initialize last_{index,bdev} */
- pgoff_t last_index;
+ erofs_off_t last_pa;
struct block_device *last_bdev;
unsigned int nr_bios = 0;
struct bio *bio = NULL;
unsigned long pflags;
int memstall = 0;
- /*
- * if managed cache is enabled, bypass jobqueue is needed,
- * no need to read from device for all pclusters in this queue.
- */
+ /* No need to read from device for pclusters in the bypass queue. */
q[JQ_BYPASS] = jobqueue_init(sb, fgq + JQ_BYPASS, NULL);
q[JQ_SUBMIT] = jobqueue_init(sb, fgq + JQ_SUBMIT, force_fg);
@@ -1654,7 +1647,8 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
do {
struct erofs_map_dev mdev;
struct z_erofs_pcluster *pcl;
- pgoff_t cur, end;
+ erofs_off_t cur, end;
+ struct bio_vec bvec;
unsigned int i = 0;
bool bypass = true;
@@ -1673,18 +1667,14 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
};
(void)erofs_map_dev(sb, &mdev);
- cur = erofs_blknr(sb, mdev.m_pa);
- end = cur + pcl->pclusterpages;
-
+ cur = mdev.m_pa;
+ end = cur + pcl->pclusterpages << PAGE_SHIFT;
do {
- struct page *page;
-
- page = pickup_page_for_submission(pcl, i++,
- &f->pagepool, mc);
- if (!page)
+ z_erofs_fill_bio_vec(&bvec, f, pcl, i++, mc);
+ if (!bvec.bv_page)
continue;
- if (bio && (cur != last_index + 1 ||
+ if (bio && (cur != last_pa ||
last_bdev != mdev.m_bdev)) {
submit_bio_retry:
submit_bio(bio);
@@ -1695,7 +1685,8 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
bio = NULL;
}
- if (unlikely(PageWorkingset(page)) && !memstall) {
+ if (unlikely(PageWorkingset(bvec.bv_page)) &&
+ !memstall) {
psi_memstall_enter(&pflags);
memstall = 1;
}
@@ -1703,23 +1694,24 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
if (!bio) {
bio = bio_alloc(mdev.m_bdev, BIO_MAX_VECS,
REQ_OP_READ, GFP_NOIO);
- bio->bi_end_io = z_erofs_decompressqueue_endio;
-
- last_bdev = mdev.m_bdev;
- bio->bi_iter.bi_sector = (sector_t)cur <<
- (sb->s_blocksize_bits - 9);
+ bio->bi_end_io = z_erofs_submissionqueue_endio;
+ bio->bi_iter.bi_sector = cur >> 9;
bio->bi_private = q[JQ_SUBMIT];
if (readahead)
bio->bi_opf |= REQ_RAHEAD;
++nr_bios;
+ last_bdev = mdev.m_bdev;
}
- if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE)
+ if (cur + bvec.bv_len > end)
+ bvec.bv_len = end - cur;
+ if (!bio_add_page(bio, bvec.bv_page, bvec.bv_len,
+ bvec.bv_offset))
goto submit_bio_retry;
- last_index = cur;
+ last_pa = cur + bvec.bv_len;
bypass = false;
- } while (++cur < end);
+ } while ((cur += bvec.bv_len) < end);
if (!bypass)
qtail[JQ_SUBMIT] = &pcl->next;
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 1/5] erofs: support I/O submission for sub-page compressed blocks
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Gao Xiang, LKML
Add a basic I/O submission path first to support sub-page blocks:
- Temporary short-lived pages will be used entirely;
- In-place I/O pages can be used partially, but compressed pages need
to be able to be mapped in contiguous virtual memory.
As a start, currently cache decompression is explicitly disabled for
sub-page blocks, which will be supported in the future.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/zdata.c | 156 ++++++++++++++++++++++-------------------------
1 file changed, 74 insertions(+), 82 deletions(-)
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index a33cd6757f98..421c0a88a0ca 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1435,86 +1435,85 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
z_erofs_decompressqueue_work(&io->u.work);
}
-static struct page *pickup_page_for_submission(struct z_erofs_pcluster *pcl,
- unsigned int nr,
- struct page **pagepool,
- struct address_space *mc)
+static void z_erofs_fill_bio_vec(struct bio_vec *bvec,
+ struct z_erofs_decompress_frontend *f,
+ struct z_erofs_pcluster *pcl,
+ unsigned int nr,
+ struct address_space *mc)
{
- const pgoff_t index = pcl->obj.index;
gfp_t gfp = mapping_gfp_mask(mc);
bool tocache = false;
-
+ struct z_erofs_bvec *zbv = pcl->compressed_bvecs + nr;
struct address_space *mapping;
- struct page *oldpage, *page;
- int justfound;
+ struct page *page, *oldpage;
+ int justfound, bs = i_blocksize(f->inode);
+ /* Except for inplace pages, the entire page can be used for I/Os */
+ bvec->bv_offset = 0;
+ bvec->bv_len = PAGE_SIZE;
repeat:
- page = READ_ONCE(pcl->compressed_bvecs[nr].page);
- oldpage = page;
-
- if (!page)
+ oldpage = READ_ONCE(zbv->page);
+ if (!oldpage)
goto out_allocpage;
- justfound = (unsigned long)page & 1UL;
- page = (struct page *)((unsigned long)page & ~1UL);
+ justfound = (unsigned long)oldpage & 1UL;
+ page = (struct page *)((unsigned long)oldpage & ~1UL);
+ bvec->bv_page = page;
+ DBG_BUGON(z_erofs_is_shortlived_page(page));
/*
- * preallocated cached pages, which is used to avoid direct reclaim
- * otherwise, it will go inplace I/O path instead.
+ * Handle preallocated cached pages. We tried to allocate such pages
+ * without triggering direct reclaim. If allocation failed, inplace
+ * file-backed pages will be used instead.
*/
if (page->private == Z_EROFS_PREALLOCATED_PAGE) {
- WRITE_ONCE(pcl->compressed_bvecs[nr].page, page);
set_page_private(page, 0);
+ WRITE_ONCE(zbv->page, page);
tocache = true;
goto out_tocache;
}
- mapping = READ_ONCE(page->mapping);
+ mapping = READ_ONCE(page->mapping);
/*
- * file-backed online pages in plcuster are all locked steady,
- * therefore it is impossible for `mapping' to be NULL.
+ * File-backed pages for inplace I/Os are all locked steady,
+ * therefore it is impossible for `mapping` to be NULL.
*/
- if (mapping && mapping != mc)
- /* ought to be unmanaged pages */
- goto out;
-
- /* directly return for shortlived page as well */
- if (z_erofs_is_shortlived_page(page))
- goto out;
+ if (mapping && mapping != mc) {
+ if (zbv->offset < 0)
+ bvec->bv_offset = round_up(-zbv->offset, bs);
+ bvec->bv_len = round_up(zbv->end, bs) - bvec->bv_offset;
+ return;
+ }
lock_page(page);
-
/* only true if page reclaim goes wrong, should never happen */
DBG_BUGON(justfound && PagePrivate(page));
- /* the page is still in manage cache */
+ /* the cached page is still in managed cache */
if (page->mapping == mc) {
- WRITE_ONCE(pcl->compressed_bvecs[nr].page, page);
-
+ WRITE_ONCE(zbv->page, page);
+ /*
+ * The cached page is still available but without a valid
+ * `->private` pcluster hint. Let's reconnect them.
+ */
if (!PagePrivate(page)) {
- /*
- * impossible to be !PagePrivate(page) for
- * the current restriction as well if
- * the page is already in compressed_bvecs[].
- */
DBG_BUGON(!justfound);
-
- justfound = 0;
- set_page_private(page, (unsigned long)pcl);
- SetPagePrivate(page);
+ /* compressed_bvecs[] already takes a ref */
+ attach_page_private(page, pcl);
+ put_page(page);
}
- /* no need to submit io if it is already up-to-date */
+ /* no need to submit if it is already up-to-date */
if (PageUptodate(page)) {
unlock_page(page);
- page = NULL;
+ bvec->bv_page = NULL;
}
- goto out;
+ return;
}
/*
- * the managed page has been truncated, it's unsafe to
- * reuse this one, let's allocate a new cache-managed page.
+ * It has been truncated, so it's unsafe to reuse this one. Let's
+ * allocate a new page for compressed data.
*/
DBG_BUGON(page->mapping);
DBG_BUGON(!justfound);
@@ -1523,25 +1522,23 @@ static struct page *pickup_page_for_submission(struct z_erofs_pcluster *pcl,
unlock_page(page);
put_page(page);
out_allocpage:
- page = erofs_allocpage(pagepool, gfp | __GFP_NOFAIL);
- if (oldpage != cmpxchg(&pcl->compressed_bvecs[nr].page,
- oldpage, page)) {
- erofs_pagepool_add(pagepool, page);
+ page = erofs_allocpage(&f->pagepool, gfp | __GFP_NOFAIL);
+ if (oldpage != cmpxchg(&zbv->page, oldpage, page)) {
+ erofs_pagepool_add(&f->pagepool, page);
cond_resched();
goto repeat;
}
+ bvec->bv_page = page;
out_tocache:
- if (!tocache || add_to_page_cache_lru(page, mc, index + nr, gfp)) {
- /* turn into temporary page if fails (1 ref) */
+ if (!tocache || bs != PAGE_SIZE ||
+ add_to_page_cache_lru(page, mc, pcl->obj.index + nr, gfp)) {
+ /* turn into a temporary shortlived page (1 ref) */
set_page_private(page, Z_EROFS_SHORTLIVED_PAGE);
- goto out;
+ return;
}
attach_page_private(page, pcl);
- /* drop a refcount added by allocpage (then we have 2 refs here) */
+ /* drop a refcount added by allocpage (then 2 refs in total here) */
put_page(page);
-
-out: /* the only exit (for tracing and debugging) */
- return page;
}
static struct z_erofs_decompressqueue *jobqueue_init(struct super_block *sb,
@@ -1596,7 +1593,7 @@ static void move_to_bypass_jobqueue(struct z_erofs_pcluster *pcl,
qtail[JQ_BYPASS] = &pcl->next;
}
-static void z_erofs_decompressqueue_endio(struct bio *bio)
+static void z_erofs_submissionqueue_endio(struct bio *bio)
{
struct z_erofs_decompressqueue *q = bio->bi_private;
blk_status_t err = bio->bi_status;
@@ -1608,7 +1605,6 @@ static void z_erofs_decompressqueue_endio(struct bio *bio)
DBG_BUGON(PageUptodate(page));
DBG_BUGON(z_erofs_page_is_invalidated(page));
-
if (erofs_page_is_managed(EROFS_SB(q->sb), page)) {
if (!err)
SetPageUptodate(page);
@@ -1631,17 +1627,14 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
struct z_erofs_decompressqueue *q[NR_JOBQUEUES];
z_erofs_next_pcluster_t owned_head = f->owned_head;
/* bio is NULL initially, so no need to initialize last_{index,bdev} */
- pgoff_t last_index;
+ erofs_off_t last_pa;
struct block_device *last_bdev;
unsigned int nr_bios = 0;
struct bio *bio = NULL;
unsigned long pflags;
int memstall = 0;
- /*
- * if managed cache is enabled, bypass jobqueue is needed,
- * no need to read from device for all pclusters in this queue.
- */
+ /* No need to read from device for pclusters in the bypass queue. */
q[JQ_BYPASS] = jobqueue_init(sb, fgq + JQ_BYPASS, NULL);
q[JQ_SUBMIT] = jobqueue_init(sb, fgq + JQ_SUBMIT, force_fg);
@@ -1654,7 +1647,8 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
do {
struct erofs_map_dev mdev;
struct z_erofs_pcluster *pcl;
- pgoff_t cur, end;
+ erofs_off_t cur, end;
+ struct bio_vec bvec;
unsigned int i = 0;
bool bypass = true;
@@ -1673,18 +1667,14 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
};
(void)erofs_map_dev(sb, &mdev);
- cur = erofs_blknr(sb, mdev.m_pa);
- end = cur + pcl->pclusterpages;
-
+ cur = mdev.m_pa;
+ end = cur + pcl->pclusterpages << PAGE_SHIFT;
do {
- struct page *page;
-
- page = pickup_page_for_submission(pcl, i++,
- &f->pagepool, mc);
- if (!page)
+ z_erofs_fill_bio_vec(&bvec, f, pcl, i++, mc);
+ if (!bvec.bv_page)
continue;
- if (bio && (cur != last_index + 1 ||
+ if (bio && (cur != last_pa ||
last_bdev != mdev.m_bdev)) {
submit_bio_retry:
submit_bio(bio);
@@ -1695,7 +1685,8 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
bio = NULL;
}
- if (unlikely(PageWorkingset(page)) && !memstall) {
+ if (unlikely(PageWorkingset(bvec.bv_page)) &&
+ !memstall) {
psi_memstall_enter(&pflags);
memstall = 1;
}
@@ -1703,23 +1694,24 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
if (!bio) {
bio = bio_alloc(mdev.m_bdev, BIO_MAX_VECS,
REQ_OP_READ, GFP_NOIO);
- bio->bi_end_io = z_erofs_decompressqueue_endio;
-
- last_bdev = mdev.m_bdev;
- bio->bi_iter.bi_sector = (sector_t)cur <<
- (sb->s_blocksize_bits - 9);
+ bio->bi_end_io = z_erofs_submissionqueue_endio;
+ bio->bi_iter.bi_sector = cur >> 9;
bio->bi_private = q[JQ_SUBMIT];
if (readahead)
bio->bi_opf |= REQ_RAHEAD;
++nr_bios;
+ last_bdev = mdev.m_bdev;
}
- if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE)
+ if (cur + bvec.bv_len > end)
+ bvec.bv_len = end - cur;
+ if (!bio_add_page(bio, bvec.bv_page, bvec.bv_len,
+ bvec.bv_offset))
goto submit_bio_retry;
- last_index = cur;
+ last_pa = cur + bvec.bv_len;
bypass = false;
- } while (++cur < end);
+ } while ((cur += bvec.bv_len) < end);
if (!bypass)
qtail[JQ_SUBMIT] = &pcl->next;
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 2/5] erofs: record `pclustersize` in bytes instead of pages
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-06 9:10 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Gao Xiang
Currently, compressed sizes are recorded in pages using `pclusterpages`,
However, for tailpacking pclusters, `tailpacking_size` is used instead.
This approach doesn't work when dealing with sub-page blocks. To address
this, let's switch them to the unified `pclustersize` in bytes.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/zdata.c | 64 ++++++++++++++++++++----------------------------
1 file changed, 26 insertions(+), 38 deletions(-)
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index 421c0a88a0ca..d02989466711 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -56,6 +56,9 @@ struct z_erofs_pcluster {
/* L: total number of bvecs */
unsigned int vcnt;
+ /* I: pcluster size (compressed size) in bytes */
+ unsigned int pclustersize;
+
/* I: page offset of start position of decompression */
unsigned short pageofs_out;
@@ -70,14 +73,6 @@ struct z_erofs_pcluster {
struct rcu_head rcu;
};
- union {
- /* I: physical cluster size in pages */
- unsigned short pclusterpages;
-
- /* I: tailpacking inline compressed size */
- unsigned short tailpacking_size;
- };
-
/* I: compression algorithm format */
unsigned char algorithmformat;
@@ -115,9 +110,7 @@ static inline bool z_erofs_is_inline_pcluster(struct z_erofs_pcluster *pcl)
static inline unsigned int z_erofs_pclusterpages(struct z_erofs_pcluster *pcl)
{
- if (z_erofs_is_inline_pcluster(pcl))
- return 1;
- return pcl->pclusterpages;
+ return PAGE_ALIGN(pcl->pclustersize) >> PAGE_SHIFT;
}
/*
@@ -298,12 +291,12 @@ static int z_erofs_create_pcluster_pool(void)
return 0;
}
-static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int nrpages)
+static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int size)
{
- int i;
+ unsigned int nrpages = PAGE_ALIGN(size) >> PAGE_SHIFT;
+ struct z_erofs_pcluster_slab *pcs = pcluster_pool;
- for (i = 0; i < ARRAY_SIZE(pcluster_pool); ++i) {
- struct z_erofs_pcluster_slab *pcs = pcluster_pool + i;
+ for (; pcs < pcluster_pool + ARRAY_SIZE(pcluster_pool); ++pcs) {
struct z_erofs_pcluster *pcl;
if (nrpages > pcs->maxpages)
@@ -312,7 +305,7 @@ static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int nrpages)
pcl = kmem_cache_zalloc(pcs->slab, GFP_NOFS);
if (!pcl)
return ERR_PTR(-ENOMEM);
- pcl->pclusterpages = nrpages;
+ pcl->pclustersize = size;
return pcl;
}
return ERR_PTR(-EINVAL);
@@ -559,6 +552,7 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
{
struct address_space *mc = MNGD_MAPPING(EROFS_I_SB(fe->inode));
struct z_erofs_pcluster *pcl = fe->pcl;
+ unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
bool shouldalloc = z_erofs_should_alloc_cache(fe);
bool standalone = true;
/*
@@ -572,10 +566,9 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
if (fe->mode < Z_EROFS_PCLUSTER_FOLLOWED)
return;
- for (i = 0; i < pcl->pclusterpages; ++i) {
- struct page *page;
+ for (i = 0; i < pclusterpages; ++i) {
+ struct page *page, *newpage;
void *t; /* mark pages just found for debugging */
- struct page *newpage = NULL;
/* the compressed page was loaded before */
if (READ_ONCE(pcl->compressed_bvecs[i].page))
@@ -585,6 +578,7 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
if (page) {
t = (void *)((unsigned long)page | 1);
+ newpage = NULL;
} else {
/* I/O is needed, no possible to decompress directly */
standalone = false;
@@ -592,9 +586,8 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
continue;
/*
- * try to use cached I/O if page allocation
- * succeeds or fallback to in-place I/O instead
- * to avoid any direct reclaim.
+ * Try cached I/O if allocation succeeds or fallback to
+ * in-place I/O instead to avoid any direct reclaim.
*/
newpage = erofs_allocpage(&fe->pagepool, gfp);
if (!newpage)
@@ -626,6 +619,7 @@ int erofs_try_to_free_all_cached_pages(struct erofs_sb_info *sbi,
{
struct z_erofs_pcluster *const pcl =
container_of(grp, struct z_erofs_pcluster, obj);
+ unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
int i;
DBG_BUGON(z_erofs_is_inline_pcluster(pcl));
@@ -633,7 +627,7 @@ int erofs_try_to_free_all_cached_pages(struct erofs_sb_info *sbi,
* refcount of workgroup is now freezed as 0,
* therefore no need to worry about available decompression users.
*/
- for (i = 0; i < pcl->pclusterpages; ++i) {
+ for (i = 0; i < pclusterpages; ++i) {
struct page *page = pcl->compressed_bvecs[i].page;
if (!page)
@@ -657,6 +651,7 @@ int erofs_try_to_free_all_cached_pages(struct erofs_sb_info *sbi,
static bool z_erofs_cache_release_folio(struct folio *folio, gfp_t gfp)
{
struct z_erofs_pcluster *pcl = folio_get_private(folio);
+ unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
bool ret;
int i;
@@ -669,7 +664,7 @@ static bool z_erofs_cache_release_folio(struct folio *folio, gfp_t gfp)
goto out;
DBG_BUGON(z_erofs_is_inline_pcluster(pcl));
- for (i = 0; i < pcl->pclusterpages; ++i) {
+ for (i = 0; i < pclusterpages; ++i) {
if (pcl->compressed_bvecs[i].page == &folio->page) {
WRITE_ONCE(pcl->compressed_bvecs[i].page, NULL);
ret = true;
@@ -778,20 +773,20 @@ static void z_erofs_try_to_claim_pcluster(struct z_erofs_decompress_frontend *f)
static int z_erofs_register_pcluster(struct z_erofs_decompress_frontend *fe)
{
struct erofs_map_blocks *map = &fe->map;
+ struct super_block *sb = fe->inode->i_sb;
bool ztailpacking = map->m_flags & EROFS_MAP_META;
struct z_erofs_pcluster *pcl;
struct erofs_workgroup *grp;
int err;
if (!(map->m_flags & EROFS_MAP_ENCODED) ||
- (!ztailpacking && !(map->m_pa >> PAGE_SHIFT))) {
+ (!ztailpacking && !erofs_blknr(sb, map->m_pa))) {
DBG_BUGON(1);
return -EFSCORRUPTED;
}
/* no available pcluster, let's allocate one */
- pcl = z_erofs_alloc_pcluster(ztailpacking ? 1 :
- map->m_plen >> PAGE_SHIFT);
+ pcl = z_erofs_alloc_pcluster(map->m_plen);
if (IS_ERR(pcl))
return PTR_ERR(pcl);
@@ -816,9 +811,8 @@ static int z_erofs_register_pcluster(struct z_erofs_decompress_frontend *fe)
if (ztailpacking) {
pcl->obj.index = 0; /* which indicates ztailpacking */
pcl->pageofs_in = erofs_blkoff(fe->inode->i_sb, map->m_pa);
- pcl->tailpacking_size = map->m_plen;
} else {
- pcl->obj.index = map->m_pa >> PAGE_SHIFT;
+ pcl->obj.index = erofs_blknr(sb, map->m_pa);
grp = erofs_insert_workgroup(fe->inode->i_sb, &pcl->obj);
if (IS_ERR(grp)) {
@@ -1244,8 +1238,7 @@ static int z_erofs_decompress_pcluster(struct z_erofs_decompress_backend *be,
unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
const struct z_erofs_decompressor *decompressor =
&erofs_decompressors[pcl->algorithmformat];
- unsigned int i, inputsize;
- int err2;
+ int i, err2;
struct page *page;
bool overlapped;
@@ -1282,18 +1275,13 @@ static int z_erofs_decompress_pcluster(struct z_erofs_decompress_backend *be,
if (err)
goto out;
- if (z_erofs_is_inline_pcluster(pcl))
- inputsize = pcl->tailpacking_size;
- else
- inputsize = pclusterpages * PAGE_SIZE;
-
err = decompressor->decompress(&(struct z_erofs_decompress_req) {
.sb = be->sb,
.in = be->compressed_pages,
.out = be->decompressed_pages,
.pageofs_in = pcl->pageofs_in,
.pageofs_out = pcl->pageofs_out,
- .inputsize = inputsize,
+ .inputsize = pcl->pclustersize,
.outputsize = pcl->length,
.alg = pcl->algorithmformat,
.inplace_io = overlapped,
@@ -1668,7 +1656,7 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
(void)erofs_map_dev(sb, &mdev);
cur = mdev.m_pa;
- end = cur + pcl->pclusterpages << PAGE_SHIFT;
+ end = cur + pcl->pclustersize;
do {
z_erofs_fill_bio_vec(&bvec, f, pcl, i++, mc);
if (!bvec.bv_page)
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 2/5] erofs: record `pclustersize` in bytes instead of pages
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Gao Xiang, LKML
Currently, compressed sizes are recorded in pages using `pclusterpages`,
However, for tailpacking pclusters, `tailpacking_size` is used instead.
This approach doesn't work when dealing with sub-page blocks. To address
this, let's switch them to the unified `pclustersize` in bytes.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/zdata.c | 64 ++++++++++++++++++++----------------------------
1 file changed, 26 insertions(+), 38 deletions(-)
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index 421c0a88a0ca..d02989466711 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -56,6 +56,9 @@ struct z_erofs_pcluster {
/* L: total number of bvecs */
unsigned int vcnt;
+ /* I: pcluster size (compressed size) in bytes */
+ unsigned int pclustersize;
+
/* I: page offset of start position of decompression */
unsigned short pageofs_out;
@@ -70,14 +73,6 @@ struct z_erofs_pcluster {
struct rcu_head rcu;
};
- union {
- /* I: physical cluster size in pages */
- unsigned short pclusterpages;
-
- /* I: tailpacking inline compressed size */
- unsigned short tailpacking_size;
- };
-
/* I: compression algorithm format */
unsigned char algorithmformat;
@@ -115,9 +110,7 @@ static inline bool z_erofs_is_inline_pcluster(struct z_erofs_pcluster *pcl)
static inline unsigned int z_erofs_pclusterpages(struct z_erofs_pcluster *pcl)
{
- if (z_erofs_is_inline_pcluster(pcl))
- return 1;
- return pcl->pclusterpages;
+ return PAGE_ALIGN(pcl->pclustersize) >> PAGE_SHIFT;
}
/*
@@ -298,12 +291,12 @@ static int z_erofs_create_pcluster_pool(void)
return 0;
}
-static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int nrpages)
+static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int size)
{
- int i;
+ unsigned int nrpages = PAGE_ALIGN(size) >> PAGE_SHIFT;
+ struct z_erofs_pcluster_slab *pcs = pcluster_pool;
- for (i = 0; i < ARRAY_SIZE(pcluster_pool); ++i) {
- struct z_erofs_pcluster_slab *pcs = pcluster_pool + i;
+ for (; pcs < pcluster_pool + ARRAY_SIZE(pcluster_pool); ++pcs) {
struct z_erofs_pcluster *pcl;
if (nrpages > pcs->maxpages)
@@ -312,7 +305,7 @@ static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int nrpages)
pcl = kmem_cache_zalloc(pcs->slab, GFP_NOFS);
if (!pcl)
return ERR_PTR(-ENOMEM);
- pcl->pclusterpages = nrpages;
+ pcl->pclustersize = size;
return pcl;
}
return ERR_PTR(-EINVAL);
@@ -559,6 +552,7 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
{
struct address_space *mc = MNGD_MAPPING(EROFS_I_SB(fe->inode));
struct z_erofs_pcluster *pcl = fe->pcl;
+ unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
bool shouldalloc = z_erofs_should_alloc_cache(fe);
bool standalone = true;
/*
@@ -572,10 +566,9 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
if (fe->mode < Z_EROFS_PCLUSTER_FOLLOWED)
return;
- for (i = 0; i < pcl->pclusterpages; ++i) {
- struct page *page;
+ for (i = 0; i < pclusterpages; ++i) {
+ struct page *page, *newpage;
void *t; /* mark pages just found for debugging */
- struct page *newpage = NULL;
/* the compressed page was loaded before */
if (READ_ONCE(pcl->compressed_bvecs[i].page))
@@ -585,6 +578,7 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
if (page) {
t = (void *)((unsigned long)page | 1);
+ newpage = NULL;
} else {
/* I/O is needed, no possible to decompress directly */
standalone = false;
@@ -592,9 +586,8 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
continue;
/*
- * try to use cached I/O if page allocation
- * succeeds or fallback to in-place I/O instead
- * to avoid any direct reclaim.
+ * Try cached I/O if allocation succeeds or fallback to
+ * in-place I/O instead to avoid any direct reclaim.
*/
newpage = erofs_allocpage(&fe->pagepool, gfp);
if (!newpage)
@@ -626,6 +619,7 @@ int erofs_try_to_free_all_cached_pages(struct erofs_sb_info *sbi,
{
struct z_erofs_pcluster *const pcl =
container_of(grp, struct z_erofs_pcluster, obj);
+ unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
int i;
DBG_BUGON(z_erofs_is_inline_pcluster(pcl));
@@ -633,7 +627,7 @@ int erofs_try_to_free_all_cached_pages(struct erofs_sb_info *sbi,
* refcount of workgroup is now freezed as 0,
* therefore no need to worry about available decompression users.
*/
- for (i = 0; i < pcl->pclusterpages; ++i) {
+ for (i = 0; i < pclusterpages; ++i) {
struct page *page = pcl->compressed_bvecs[i].page;
if (!page)
@@ -657,6 +651,7 @@ int erofs_try_to_free_all_cached_pages(struct erofs_sb_info *sbi,
static bool z_erofs_cache_release_folio(struct folio *folio, gfp_t gfp)
{
struct z_erofs_pcluster *pcl = folio_get_private(folio);
+ unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
bool ret;
int i;
@@ -669,7 +664,7 @@ static bool z_erofs_cache_release_folio(struct folio *folio, gfp_t gfp)
goto out;
DBG_BUGON(z_erofs_is_inline_pcluster(pcl));
- for (i = 0; i < pcl->pclusterpages; ++i) {
+ for (i = 0; i < pclusterpages; ++i) {
if (pcl->compressed_bvecs[i].page == &folio->page) {
WRITE_ONCE(pcl->compressed_bvecs[i].page, NULL);
ret = true;
@@ -778,20 +773,20 @@ static void z_erofs_try_to_claim_pcluster(struct z_erofs_decompress_frontend *f)
static int z_erofs_register_pcluster(struct z_erofs_decompress_frontend *fe)
{
struct erofs_map_blocks *map = &fe->map;
+ struct super_block *sb = fe->inode->i_sb;
bool ztailpacking = map->m_flags & EROFS_MAP_META;
struct z_erofs_pcluster *pcl;
struct erofs_workgroup *grp;
int err;
if (!(map->m_flags & EROFS_MAP_ENCODED) ||
- (!ztailpacking && !(map->m_pa >> PAGE_SHIFT))) {
+ (!ztailpacking && !erofs_blknr(sb, map->m_pa))) {
DBG_BUGON(1);
return -EFSCORRUPTED;
}
/* no available pcluster, let's allocate one */
- pcl = z_erofs_alloc_pcluster(ztailpacking ? 1 :
- map->m_plen >> PAGE_SHIFT);
+ pcl = z_erofs_alloc_pcluster(map->m_plen);
if (IS_ERR(pcl))
return PTR_ERR(pcl);
@@ -816,9 +811,8 @@ static int z_erofs_register_pcluster(struct z_erofs_decompress_frontend *fe)
if (ztailpacking) {
pcl->obj.index = 0; /* which indicates ztailpacking */
pcl->pageofs_in = erofs_blkoff(fe->inode->i_sb, map->m_pa);
- pcl->tailpacking_size = map->m_plen;
} else {
- pcl->obj.index = map->m_pa >> PAGE_SHIFT;
+ pcl->obj.index = erofs_blknr(sb, map->m_pa);
grp = erofs_insert_workgroup(fe->inode->i_sb, &pcl->obj);
if (IS_ERR(grp)) {
@@ -1244,8 +1238,7 @@ static int z_erofs_decompress_pcluster(struct z_erofs_decompress_backend *be,
unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
const struct z_erofs_decompressor *decompressor =
&erofs_decompressors[pcl->algorithmformat];
- unsigned int i, inputsize;
- int err2;
+ int i, err2;
struct page *page;
bool overlapped;
@@ -1282,18 +1275,13 @@ static int z_erofs_decompress_pcluster(struct z_erofs_decompress_backend *be,
if (err)
goto out;
- if (z_erofs_is_inline_pcluster(pcl))
- inputsize = pcl->tailpacking_size;
- else
- inputsize = pclusterpages * PAGE_SIZE;
-
err = decompressor->decompress(&(struct z_erofs_decompress_req) {
.sb = be->sb,
.in = be->compressed_pages,
.out = be->decompressed_pages,
.pageofs_in = pcl->pageofs_in,
.pageofs_out = pcl->pageofs_out,
- .inputsize = inputsize,
+ .inputsize = pcl->pclustersize,
.outputsize = pcl->length,
.alg = pcl->algorithmformat,
.inplace_io = overlapped,
@@ -1668,7 +1656,7 @@ static void z_erofs_submit_queue(struct z_erofs_decompress_frontend *f,
(void)erofs_map_dev(sb, &mdev);
cur = mdev.m_pa;
- end = cur + pcl->pclusterpages << PAGE_SHIFT;
+ end = cur + pcl->pclustersize;
do {
z_erofs_fill_bio_vec(&bvec, f, pcl, i++, mc);
if (!bvec.bv_page)
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 3/5] erofs: fix up compacted indexes for block size < 4096
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-06 9:10 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Gao Xiang
Previously, the block size always equaled to PAGE_SIZE, therefore
`lclusterbits` couldn't be less than 12.
Since sub-page compressed blocks are now considered, `lobits` for
a lcluster in each pack cannot always be `lclusterbits` as before.
Otherwise, there is no enough room for the special value
`Z_EROFS_LI_D0_CBLKCNT`.
To support smaller block sizes, `lobits` for each compacted lcluster is
now calculated as:
lobits = max(lclusterbits, ilog2(Z_EROFS_LI_D0_CBLKCNT) + 1)
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/zmap.c | 32 ++++++++++++++------------------
1 file changed, 14 insertions(+), 18 deletions(-)
diff --git a/fs/erofs/zmap.c b/fs/erofs/zmap.c
index 7b55111fd533..9753875e41cb 100644
--- a/fs/erofs/zmap.c
+++ b/fs/erofs/zmap.c
@@ -82,29 +82,26 @@ static int z_erofs_load_full_lcluster(struct z_erofs_maprecorder *m,
}
static unsigned int decode_compactedbits(unsigned int lobits,
- unsigned int lomask,
u8 *in, unsigned int pos, u8 *type)
{
const unsigned int v = get_unaligned_le32(in + pos / 8) >> (pos & 7);
- const unsigned int lo = v & lomask;
+ const unsigned int lo = v & ((1 << lobits) - 1);
*type = (v >> lobits) & 3;
return lo;
}
-static int get_compacted_la_distance(unsigned int lclusterbits,
+static int get_compacted_la_distance(unsigned int lobits,
unsigned int encodebits,
unsigned int vcnt, u8 *in, int i)
{
- const unsigned int lomask = (1 << lclusterbits) - 1;
unsigned int lo, d1 = 0;
u8 type;
DBG_BUGON(i >= vcnt);
do {
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in, encodebits * i, &type);
if (type != Z_EROFS_LCLUSTER_TYPE_NONHEAD)
return d1;
@@ -123,15 +120,14 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
{
struct erofs_inode *const vi = EROFS_I(m->inode);
const unsigned int lclusterbits = vi->z_logical_clusterbits;
- const unsigned int lomask = (1 << lclusterbits) - 1;
- unsigned int vcnt, base, lo, encodebits, nblk, eofs;
+ unsigned int vcnt, base, lo, lobits, encodebits, nblk, eofs;
int i;
u8 *in, type;
bool big_pcluster;
if (1 << amortizedshift == 4 && lclusterbits <= 14)
vcnt = 2;
- else if (1 << amortizedshift == 2 && lclusterbits == 12)
+ else if (1 << amortizedshift == 2 && lclusterbits <= 12)
vcnt = 16;
else
return -EOPNOTSUPP;
@@ -140,6 +136,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
m->nextpackoff = round_down(pos, vcnt << amortizedshift) +
(vcnt << amortizedshift);
big_pcluster = vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_1;
+ lobits = max(lclusterbits, ilog2(Z_EROFS_LI_D0_CBLKCNT) + 1U);
encodebits = ((vcnt << amortizedshift) - sizeof(__le32)) * 8 / vcnt;
eofs = erofs_blkoff(m->inode->i_sb, pos);
base = round_down(eofs, vcnt << amortizedshift);
@@ -147,15 +144,14 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
i = (eofs - base) >> amortizedshift;
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in, encodebits * i, &type);
m->type = type;
if (type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
m->clusterofs = 1 << lclusterbits;
/* figure out lookahead_distance: delta[1] if needed */
if (lookahead)
- m->delta[1] = get_compacted_la_distance(lclusterbits,
+ m->delta[1] = get_compacted_la_distance(lobits,
encodebits, vcnt, in, i);
if (lo & Z_EROFS_LI_D0_CBLKCNT) {
if (!big_pcluster) {
@@ -174,8 +170,8 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
* of which lo saves delta[1] rather than delta[0].
* Hence, get delta[0] by the previous lcluster indirectly.
*/
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * (i - 1), &type);
+ lo = decode_compactedbits(lobits, in,
+ encodebits * (i - 1), &type);
if (type != Z_EROFS_LCLUSTER_TYPE_NONHEAD)
lo = 0;
else if (lo & Z_EROFS_LI_D0_CBLKCNT)
@@ -190,8 +186,8 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
nblk = 1;
while (i > 0) {
--i;
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in,
+ encodebits * i, &type);
if (type == Z_EROFS_LCLUSTER_TYPE_NONHEAD)
i -= lo;
@@ -202,8 +198,8 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
nblk = 0;
while (i > 0) {
--i;
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in,
+ encodebits * i, &type);
if (type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
if (lo & Z_EROFS_LI_D0_CBLKCNT) {
--i;
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 3/5] erofs: fix up compacted indexes for block size < 4096
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Gao Xiang, LKML
Previously, the block size always equaled to PAGE_SIZE, therefore
`lclusterbits` couldn't be less than 12.
Since sub-page compressed blocks are now considered, `lobits` for
a lcluster in each pack cannot always be `lclusterbits` as before.
Otherwise, there is no enough room for the special value
`Z_EROFS_LI_D0_CBLKCNT`.
To support smaller block sizes, `lobits` for each compacted lcluster is
now calculated as:
lobits = max(lclusterbits, ilog2(Z_EROFS_LI_D0_CBLKCNT) + 1)
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/zmap.c | 32 ++++++++++++++------------------
1 file changed, 14 insertions(+), 18 deletions(-)
diff --git a/fs/erofs/zmap.c b/fs/erofs/zmap.c
index 7b55111fd533..9753875e41cb 100644
--- a/fs/erofs/zmap.c
+++ b/fs/erofs/zmap.c
@@ -82,29 +82,26 @@ static int z_erofs_load_full_lcluster(struct z_erofs_maprecorder *m,
}
static unsigned int decode_compactedbits(unsigned int lobits,
- unsigned int lomask,
u8 *in, unsigned int pos, u8 *type)
{
const unsigned int v = get_unaligned_le32(in + pos / 8) >> (pos & 7);
- const unsigned int lo = v & lomask;
+ const unsigned int lo = v & ((1 << lobits) - 1);
*type = (v >> lobits) & 3;
return lo;
}
-static int get_compacted_la_distance(unsigned int lclusterbits,
+static int get_compacted_la_distance(unsigned int lobits,
unsigned int encodebits,
unsigned int vcnt, u8 *in, int i)
{
- const unsigned int lomask = (1 << lclusterbits) - 1;
unsigned int lo, d1 = 0;
u8 type;
DBG_BUGON(i >= vcnt);
do {
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in, encodebits * i, &type);
if (type != Z_EROFS_LCLUSTER_TYPE_NONHEAD)
return d1;
@@ -123,15 +120,14 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
{
struct erofs_inode *const vi = EROFS_I(m->inode);
const unsigned int lclusterbits = vi->z_logical_clusterbits;
- const unsigned int lomask = (1 << lclusterbits) - 1;
- unsigned int vcnt, base, lo, encodebits, nblk, eofs;
+ unsigned int vcnt, base, lo, lobits, encodebits, nblk, eofs;
int i;
u8 *in, type;
bool big_pcluster;
if (1 << amortizedshift == 4 && lclusterbits <= 14)
vcnt = 2;
- else if (1 << amortizedshift == 2 && lclusterbits == 12)
+ else if (1 << amortizedshift == 2 && lclusterbits <= 12)
vcnt = 16;
else
return -EOPNOTSUPP;
@@ -140,6 +136,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
m->nextpackoff = round_down(pos, vcnt << amortizedshift) +
(vcnt << amortizedshift);
big_pcluster = vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_1;
+ lobits = max(lclusterbits, ilog2(Z_EROFS_LI_D0_CBLKCNT) + 1U);
encodebits = ((vcnt << amortizedshift) - sizeof(__le32)) * 8 / vcnt;
eofs = erofs_blkoff(m->inode->i_sb, pos);
base = round_down(eofs, vcnt << amortizedshift);
@@ -147,15 +144,14 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
i = (eofs - base) >> amortizedshift;
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in, encodebits * i, &type);
m->type = type;
if (type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
m->clusterofs = 1 << lclusterbits;
/* figure out lookahead_distance: delta[1] if needed */
if (lookahead)
- m->delta[1] = get_compacted_la_distance(lclusterbits,
+ m->delta[1] = get_compacted_la_distance(lobits,
encodebits, vcnt, in, i);
if (lo & Z_EROFS_LI_D0_CBLKCNT) {
if (!big_pcluster) {
@@ -174,8 +170,8 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
* of which lo saves delta[1] rather than delta[0].
* Hence, get delta[0] by the previous lcluster indirectly.
*/
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * (i - 1), &type);
+ lo = decode_compactedbits(lobits, in,
+ encodebits * (i - 1), &type);
if (type != Z_EROFS_LCLUSTER_TYPE_NONHEAD)
lo = 0;
else if (lo & Z_EROFS_LI_D0_CBLKCNT)
@@ -190,8 +186,8 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
nblk = 1;
while (i > 0) {
--i;
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in,
+ encodebits * i, &type);
if (type == Z_EROFS_LCLUSTER_TYPE_NONHEAD)
i -= lo;
@@ -202,8 +198,8 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
nblk = 0;
while (i > 0) {
--i;
- lo = decode_compactedbits(lclusterbits, lomask,
- in, encodebits * i, &type);
+ lo = decode_compactedbits(lobits, in,
+ encodebits * i, &type);
if (type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
if (lo & Z_EROFS_LI_D0_CBLKCNT) {
--i;
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-06 9:10 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Gao Xiang
Sub-page block support is still unusable even with previous commits if
interlaced PLAIN pclusters exist. Such pclusters can be found if the
fragment feature is enabled.
This commit tries to handle "the head part" of interlaced PLAIN
pclusters first: it was once explained in commit fdffc091e6f9 ("erofs:
support interlaced uncompressed data for compressed files").
It uses a unique way for both shifted and interlaced PLAIN pclusters.
As an added bonus, PLAIN pclusters larger than the block size is also
supported now for the upcoming large lclusters.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/decompressor.c | 81 ++++++++++++++++++++++++-----------------
1 file changed, 48 insertions(+), 33 deletions(-)
diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 021be5feb1bc..5ec11f5024b7 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -319,43 +319,58 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
struct page **pagepool)
{
- const unsigned int inpages = PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
- const unsigned int outpages =
+ const unsigned int nrpages_in =
+ PAGE_ALIGN(rq->pageofs_in + rq->inputsize) >> PAGE_SHIFT;
+ const unsigned int nrpages_out =
PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
- const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
- PAGE_SIZE - rq->pageofs_out);
- const unsigned int lefthalf = rq->outputsize - righthalf;
- const unsigned int interlaced_offset =
- rq->alg == Z_EROFS_COMPRESSION_SHIFTED ? 0 : rq->pageofs_out;
- u8 *src;
-
- if (outpages > 2 && rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
- DBG_BUGON(1);
- return -EFSCORRUPTED;
- }
-
- if (rq->out[0] == *rq->in) {
- DBG_BUGON(rq->pageofs_out);
- return 0;
+ const unsigned int bs = rq->sb->s_blocksize;
+ unsigned int cur = 0, ni = 0, no, pi, po, insz, cnt;
+ u8 *kin;
+
+ DBG_BUGON(rq->outputsize > rq->inputsize);
+ if (rq->alg == Z_EROFS_COMPRESSION_INTERLACED) {
+ cur = bs - (rq->pageofs_out & (bs - 1));
+ pi = (rq->pageofs_in + rq->inputsize - cur) & ~PAGE_MASK;
+ cur = min(cur, rq->outputsize);
+ if (cur && rq->out[0]) {
+ kin = kmap_local_page(rq->in[nrpages_in - 1]);
+ if (rq->out[0] == rq->in[nrpages_in - 1]) {
+ memmove(kin + rq->pageofs_out, kin + pi, cur);
+ flush_dcache_page(rq->out[0]);
+ } else {
+ memcpy_to_page(rq->out[0], rq->pageofs_out,
+ kin + pi, cur);
+ }
+ kunmap_local(kin);
+ }
+ rq->outputsize -= cur;
}
- src = kmap_local_page(rq->in[inpages - 1]) + rq->pageofs_in;
- if (rq->out[0])
- memcpy_to_page(rq->out[0], rq->pageofs_out,
- src + interlaced_offset, righthalf);
-
- if (outpages > inpages) {
- DBG_BUGON(!rq->out[outpages - 1]);
- if (rq->out[outpages - 1] != rq->in[inpages - 1]) {
- memcpy_to_page(rq->out[outpages - 1], 0, src +
- (interlaced_offset ? 0 : righthalf),
- lefthalf);
- } else if (!interlaced_offset) {
- memmove(src, src + righthalf, lefthalf);
- flush_dcache_page(rq->in[inpages - 1]);
- }
+ for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
+ insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
+ rq->outputsize -= insz;
+ if (!rq->in[ni])
+ continue;
+ kin = kmap_local_page(rq->in[ni]);
+ pi = 0;
+ do {
+ no = (rq->pageofs_out + cur + pi) >> PAGE_SHIFT;
+ po = (rq->pageofs_out + cur + pi) & ~PAGE_MASK;
+ DBG_BUGON(no >= nrpages_out);
+ cnt = min(insz - pi, PAGE_SIZE - po);
+ if (rq->out[no] == rq->in[ni]) {
+ memmove(kin + po,
+ kin + rq->pageofs_in + pi, cnt);
+ flush_dcache_page(rq->out[no]);
+ } else if (rq->out[no]) {
+ memcpy_to_page(rq->out[no], po,
+ kin + rq->pageofs_in + pi, cnt);
+ }
+ pi += cnt;
+ } while (pi < insz);
+ kunmap_local(kin);
}
- kunmap_local(src);
+ DBG_BUGON(ni > nrpages_in);
return 0;
}
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Gao Xiang, LKML
Sub-page block support is still unusable even with previous commits if
interlaced PLAIN pclusters exist. Such pclusters can be found if the
fragment feature is enabled.
This commit tries to handle "the head part" of interlaced PLAIN
pclusters first: it was once explained in commit fdffc091e6f9 ("erofs:
support interlaced uncompressed data for compressed files").
It uses a unique way for both shifted and interlaced PLAIN pclusters.
As an added bonus, PLAIN pclusters larger than the block size is also
supported now for the upcoming large lclusters.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/decompressor.c | 81 ++++++++++++++++++++++++-----------------
1 file changed, 48 insertions(+), 33 deletions(-)
diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 021be5feb1bc..5ec11f5024b7 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -319,43 +319,58 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
struct page **pagepool)
{
- const unsigned int inpages = PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
- const unsigned int outpages =
+ const unsigned int nrpages_in =
+ PAGE_ALIGN(rq->pageofs_in + rq->inputsize) >> PAGE_SHIFT;
+ const unsigned int nrpages_out =
PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
- const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
- PAGE_SIZE - rq->pageofs_out);
- const unsigned int lefthalf = rq->outputsize - righthalf;
- const unsigned int interlaced_offset =
- rq->alg == Z_EROFS_COMPRESSION_SHIFTED ? 0 : rq->pageofs_out;
- u8 *src;
-
- if (outpages > 2 && rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
- DBG_BUGON(1);
- return -EFSCORRUPTED;
- }
-
- if (rq->out[0] == *rq->in) {
- DBG_BUGON(rq->pageofs_out);
- return 0;
+ const unsigned int bs = rq->sb->s_blocksize;
+ unsigned int cur = 0, ni = 0, no, pi, po, insz, cnt;
+ u8 *kin;
+
+ DBG_BUGON(rq->outputsize > rq->inputsize);
+ if (rq->alg == Z_EROFS_COMPRESSION_INTERLACED) {
+ cur = bs - (rq->pageofs_out & (bs - 1));
+ pi = (rq->pageofs_in + rq->inputsize - cur) & ~PAGE_MASK;
+ cur = min(cur, rq->outputsize);
+ if (cur && rq->out[0]) {
+ kin = kmap_local_page(rq->in[nrpages_in - 1]);
+ if (rq->out[0] == rq->in[nrpages_in - 1]) {
+ memmove(kin + rq->pageofs_out, kin + pi, cur);
+ flush_dcache_page(rq->out[0]);
+ } else {
+ memcpy_to_page(rq->out[0], rq->pageofs_out,
+ kin + pi, cur);
+ }
+ kunmap_local(kin);
+ }
+ rq->outputsize -= cur;
}
- src = kmap_local_page(rq->in[inpages - 1]) + rq->pageofs_in;
- if (rq->out[0])
- memcpy_to_page(rq->out[0], rq->pageofs_out,
- src + interlaced_offset, righthalf);
-
- if (outpages > inpages) {
- DBG_BUGON(!rq->out[outpages - 1]);
- if (rq->out[outpages - 1] != rq->in[inpages - 1]) {
- memcpy_to_page(rq->out[outpages - 1], 0, src +
- (interlaced_offset ? 0 : righthalf),
- lefthalf);
- } else if (!interlaced_offset) {
- memmove(src, src + righthalf, lefthalf);
- flush_dcache_page(rq->in[inpages - 1]);
- }
+ for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
+ insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
+ rq->outputsize -= insz;
+ if (!rq->in[ni])
+ continue;
+ kin = kmap_local_page(rq->in[ni]);
+ pi = 0;
+ do {
+ no = (rq->pageofs_out + cur + pi) >> PAGE_SHIFT;
+ po = (rq->pageofs_out + cur + pi) & ~PAGE_MASK;
+ DBG_BUGON(no >= nrpages_out);
+ cnt = min(insz - pi, PAGE_SIZE - po);
+ if (rq->out[no] == rq->in[ni]) {
+ memmove(kin + po,
+ kin + rq->pageofs_in + pi, cnt);
+ flush_dcache_page(rq->out[no]);
+ } else if (rq->out[no]) {
+ memcpy_to_page(rq->out[no], po,
+ kin + rq->pageofs_in + pi, cnt);
+ }
+ pi += cnt;
+ } while (pi < insz);
+ kunmap_local(kin);
}
- kunmap_local(src);
+ DBG_BUGON(ni > nrpages_in);
return 0;
}
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 5/5] erofs: enable sub-page compressed block support
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-06 9:10 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Gao Xiang
Let's just disable cached decompression and inplace I/Os for partial
pages as a first step in order to enable sub-page block initial
support. In other words, currently it works primarily based on
temporary short-lived pages. Don't expect too much in terms of
performance.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/inode.c | 6 ++++--
fs/erofs/zdata.c | 6 ++++--
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/fs/erofs/inode.c b/fs/erofs/inode.c
index 14a79d3226ab..3d616dea55dc 100644
--- a/fs/erofs/inode.c
+++ b/fs/erofs/inode.c
@@ -259,8 +259,10 @@ static int erofs_fill_inode(struct inode *inode)
if (erofs_inode_is_data_compressed(vi->datalayout)) {
#ifdef CONFIG_EROFS_FS_ZIP
- if (!erofs_is_fscache_mode(inode->i_sb) &&
- inode->i_sb->s_blocksize_bits == PAGE_SHIFT) {
+ if (!erofs_is_fscache_mode(inode->i_sb)) {
+ DO_ONCE_LITE_IF(inode->i_sb->s_blocksize != PAGE_SIZE,
+ erofs_info, inode->i_sb,
+ "EXPERIMENTAL EROFS subpage compressed block support in use. Use at your own risk!");
inode->i_mapping->a_ops = &z_erofs_aops;
err = 0;
goto out_unlock;
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index d02989466711..a2c3e87d2f81 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -563,6 +563,8 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
__GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
unsigned int i;
+ if (i_blocksize(fe->inode) != PAGE_SIZE)
+ return;
if (fe->mode < Z_EROFS_PCLUSTER_FOLLOWED)
return;
@@ -967,12 +969,12 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe,
struct inode *const inode = fe->inode;
struct erofs_map_blocks *const map = &fe->map;
const loff_t offset = page_offset(page);
+ const unsigned int bs = i_blocksize(inode);
bool tight = true, exclusive;
unsigned int cur, end, len, split;
int err = 0;
z_erofs_onlinepage_init(page);
-
split = 0;
end = PAGE_SIZE;
repeat:
@@ -1021,7 +1023,7 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe,
* for inplace I/O or bvpage (should be processed in a strict order.)
*/
tight &= (fe->mode > Z_EROFS_PCLUSTER_FOLLOWED_NOINPLACE);
- exclusive = (!cur && ((split <= 1) || tight));
+ exclusive = (!cur && ((split <= 1) || (tight && bs == PAGE_SIZE)));
if (cur)
tight &= (fe->mode >= Z_EROFS_PCLUSTER_FOLLOWED);
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 5/5] erofs: enable sub-page compressed block support
@ 2023-12-06 9:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-06 9:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Gao Xiang, LKML
Let's just disable cached decompression and inplace I/Os for partial
pages as a first step in order to enable sub-page block initial
support. In other words, currently it works primarily based on
temporary short-lived pages. Don't expect too much in terms of
performance.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
---
fs/erofs/inode.c | 6 ++++--
fs/erofs/zdata.c | 6 ++++--
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/fs/erofs/inode.c b/fs/erofs/inode.c
index 14a79d3226ab..3d616dea55dc 100644
--- a/fs/erofs/inode.c
+++ b/fs/erofs/inode.c
@@ -259,8 +259,10 @@ static int erofs_fill_inode(struct inode *inode)
if (erofs_inode_is_data_compressed(vi->datalayout)) {
#ifdef CONFIG_EROFS_FS_ZIP
- if (!erofs_is_fscache_mode(inode->i_sb) &&
- inode->i_sb->s_blocksize_bits == PAGE_SHIFT) {
+ if (!erofs_is_fscache_mode(inode->i_sb)) {
+ DO_ONCE_LITE_IF(inode->i_sb->s_blocksize != PAGE_SIZE,
+ erofs_info, inode->i_sb,
+ "EXPERIMENTAL EROFS subpage compressed block support in use. Use at your own risk!");
inode->i_mapping->a_ops = &z_erofs_aops;
err = 0;
goto out_unlock;
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index d02989466711..a2c3e87d2f81 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -563,6 +563,8 @@ static void z_erofs_bind_cache(struct z_erofs_decompress_frontend *fe)
__GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
unsigned int i;
+ if (i_blocksize(fe->inode) != PAGE_SIZE)
+ return;
if (fe->mode < Z_EROFS_PCLUSTER_FOLLOWED)
return;
@@ -967,12 +969,12 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe,
struct inode *const inode = fe->inode;
struct erofs_map_blocks *const map = &fe->map;
const loff_t offset = page_offset(page);
+ const unsigned int bs = i_blocksize(inode);
bool tight = true, exclusive;
unsigned int cur, end, len, split;
int err = 0;
z_erofs_onlinepage_init(page);
-
split = 0;
end = PAGE_SIZE;
repeat:
@@ -1021,7 +1023,7 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe,
* for inplace I/O or bvpage (should be processed in a strict order.)
*/
tight &= (fe->mode > Z_EROFS_PCLUSTER_FOLLOWED_NOINPLACE);
- exclusive = (!cur && ((split <= 1) || tight));
+ exclusive = (!cur && ((split <= 1) || (tight && bs == PAGE_SIZE)));
if (cur)
tight &= (fe->mode >= Z_EROFS_PCLUSTER_FOLLOWED);
--
2.39.3
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-08 5:20 ` Yue Hu
-1 siblings, 0 replies; 25+ messages in thread
From: Yue Hu @ 2023-12-08 5:20 UTC (permalink / raw)
To: Gao Xiang; +Cc: linux-erofs, LKML, huyue2, zhangwen
On Wed, 6 Dec 2023 17:10:56 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> Sub-page block support is still unusable even with previous commits if
> interlaced PLAIN pclusters exist. Such pclusters can be found if the
> fragment feature is enabled.
>
> This commit tries to handle "the head part" of interlaced PLAIN
> pclusters first: it was once explained in commit fdffc091e6f9 ("erofs:
> support interlaced uncompressed data for compressed files").
>
> It uses a unique way for both shifted and interlaced PLAIN pclusters.
> As an added bonus, PLAIN pclusters larger than the block size is also
> supported now for the upcoming large lclusters.
>
> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
> ---
> fs/erofs/decompressor.c | 81 ++++++++++++++++++++++++-----------------
> 1 file changed, 48 insertions(+), 33 deletions(-)
>
> diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
> index 021be5feb1bc..5ec11f5024b7 100644
> --- a/fs/erofs/decompressor.c
> +++ b/fs/erofs/decompressor.c
> @@ -319,43 +319,58 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
> static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
> struct page **pagepool)
> {
> - const unsigned int inpages = PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
> - const unsigned int outpages =
> + const unsigned int nrpages_in =
> + PAGE_ALIGN(rq->pageofs_in + rq->inputsize) >> PAGE_SHIFT;
> + const unsigned int nrpages_out =
> PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
> - const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
> - PAGE_SIZE - rq->pageofs_out);
> - const unsigned int lefthalf = rq->outputsize - righthalf;
> - const unsigned int interlaced_offset =
> - rq->alg == Z_EROFS_COMPRESSION_SHIFTED ? 0 : rq->pageofs_out;
> - u8 *src;
> -
> - if (outpages > 2 && rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
> - DBG_BUGON(1);
> - return -EFSCORRUPTED;
> - }
> -
> - if (rq->out[0] == *rq->in) {
> - DBG_BUGON(rq->pageofs_out);
> - return 0;
> + const unsigned int bs = rq->sb->s_blocksize;
> + unsigned int cur = 0, ni = 0, no, pi, po, insz, cnt;
> + u8 *kin;
> +
> + DBG_BUGON(rq->outputsize > rq->inputsize);
> + if (rq->alg == Z_EROFS_COMPRESSION_INTERLACED) {
> + cur = bs - (rq->pageofs_out & (bs - 1));
> + pi = (rq->pageofs_in + rq->inputsize - cur) & ~PAGE_MASK;
> + cur = min(cur, rq->outputsize);
> + if (cur && rq->out[0]) {
> + kin = kmap_local_page(rq->in[nrpages_in - 1]);
> + if (rq->out[0] == rq->in[nrpages_in - 1]) {
> + memmove(kin + rq->pageofs_out, kin + pi, cur);
> + flush_dcache_page(rq->out[0]);
> + } else {
> + memcpy_to_page(rq->out[0], rq->pageofs_out,
> + kin + pi, cur);
> + }
> + kunmap_local(kin);
> + }
> + rq->outputsize -= cur;
> }
>
> - src = kmap_local_page(rq->in[inpages - 1]) + rq->pageofs_in;
> - if (rq->out[0])
> - memcpy_to_page(rq->out[0], rq->pageofs_out,
> - src + interlaced_offset, righthalf);
> -
> - if (outpages > inpages) {
> - DBG_BUGON(!rq->out[outpages - 1]);
> - if (rq->out[outpages - 1] != rq->in[inpages - 1]) {
> - memcpy_to_page(rq->out[outpages - 1], 0, src +
> - (interlaced_offset ? 0 : righthalf),
> - lefthalf);
> - } else if (!interlaced_offset) {
> - memmove(src, src + righthalf, lefthalf);
> - flush_dcache_page(rq->in[inpages - 1]);
> - }
> + for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
> + insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
min_t(unsigned int, ,)?
../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
(!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
> + rq->outputsize -= insz;
> + if (!rq->in[ni])
> + continue;
> + kin = kmap_local_page(rq->in[ni]);
> + pi = 0;
> + do {
> + no = (rq->pageofs_out + cur + pi) >> PAGE_SHIFT;
> + po = (rq->pageofs_out + cur + pi) & ~PAGE_MASK;
> + DBG_BUGON(no >= nrpages_out);
> + cnt = min(insz - pi, PAGE_SIZE - po);
ditto
> + if (rq->out[no] == rq->in[ni]) {
> + memmove(kin + po,
> + kin + rq->pageofs_in + pi, cnt);
> + flush_dcache_page(rq->out[no]);
> + } else if (rq->out[no]) {
> + memcpy_to_page(rq->out[no], po,
> + kin + rq->pageofs_in + pi, cnt);
> + }
> + pi += cnt;
> + } while (pi < insz);
> + kunmap_local(kin);
> }
> - kunmap_local(src);
> + DBG_BUGON(ni > nrpages_in);
> return 0;
> }
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
@ 2023-12-08 5:20 ` Yue Hu
0 siblings, 0 replies; 25+ messages in thread
From: Yue Hu @ 2023-12-08 5:20 UTC (permalink / raw)
To: Gao Xiang; +Cc: huyue2, linux-erofs, LKML, zhangwen
On Wed, 6 Dec 2023 17:10:56 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> Sub-page block support is still unusable even with previous commits if
> interlaced PLAIN pclusters exist. Such pclusters can be found if the
> fragment feature is enabled.
>
> This commit tries to handle "the head part" of interlaced PLAIN
> pclusters first: it was once explained in commit fdffc091e6f9 ("erofs:
> support interlaced uncompressed data for compressed files").
>
> It uses a unique way for both shifted and interlaced PLAIN pclusters.
> As an added bonus, PLAIN pclusters larger than the block size is also
> supported now for the upcoming large lclusters.
>
> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
> ---
> fs/erofs/decompressor.c | 81 ++++++++++++++++++++++++-----------------
> 1 file changed, 48 insertions(+), 33 deletions(-)
>
> diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
> index 021be5feb1bc..5ec11f5024b7 100644
> --- a/fs/erofs/decompressor.c
> +++ b/fs/erofs/decompressor.c
> @@ -319,43 +319,58 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
> static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
> struct page **pagepool)
> {
> - const unsigned int inpages = PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
> - const unsigned int outpages =
> + const unsigned int nrpages_in =
> + PAGE_ALIGN(rq->pageofs_in + rq->inputsize) >> PAGE_SHIFT;
> + const unsigned int nrpages_out =
> PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
> - const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
> - PAGE_SIZE - rq->pageofs_out);
> - const unsigned int lefthalf = rq->outputsize - righthalf;
> - const unsigned int interlaced_offset =
> - rq->alg == Z_EROFS_COMPRESSION_SHIFTED ? 0 : rq->pageofs_out;
> - u8 *src;
> -
> - if (outpages > 2 && rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
> - DBG_BUGON(1);
> - return -EFSCORRUPTED;
> - }
> -
> - if (rq->out[0] == *rq->in) {
> - DBG_BUGON(rq->pageofs_out);
> - return 0;
> + const unsigned int bs = rq->sb->s_blocksize;
> + unsigned int cur = 0, ni = 0, no, pi, po, insz, cnt;
> + u8 *kin;
> +
> + DBG_BUGON(rq->outputsize > rq->inputsize);
> + if (rq->alg == Z_EROFS_COMPRESSION_INTERLACED) {
> + cur = bs - (rq->pageofs_out & (bs - 1));
> + pi = (rq->pageofs_in + rq->inputsize - cur) & ~PAGE_MASK;
> + cur = min(cur, rq->outputsize);
> + if (cur && rq->out[0]) {
> + kin = kmap_local_page(rq->in[nrpages_in - 1]);
> + if (rq->out[0] == rq->in[nrpages_in - 1]) {
> + memmove(kin + rq->pageofs_out, kin + pi, cur);
> + flush_dcache_page(rq->out[0]);
> + } else {
> + memcpy_to_page(rq->out[0], rq->pageofs_out,
> + kin + pi, cur);
> + }
> + kunmap_local(kin);
> + }
> + rq->outputsize -= cur;
> }
>
> - src = kmap_local_page(rq->in[inpages - 1]) + rq->pageofs_in;
> - if (rq->out[0])
> - memcpy_to_page(rq->out[0], rq->pageofs_out,
> - src + interlaced_offset, righthalf);
> -
> - if (outpages > inpages) {
> - DBG_BUGON(!rq->out[outpages - 1]);
> - if (rq->out[outpages - 1] != rq->in[inpages - 1]) {
> - memcpy_to_page(rq->out[outpages - 1], 0, src +
> - (interlaced_offset ? 0 : righthalf),
> - lefthalf);
> - } else if (!interlaced_offset) {
> - memmove(src, src + righthalf, lefthalf);
> - flush_dcache_page(rq->in[inpages - 1]);
> - }
> + for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
> + insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
min_t(unsigned int, ,)?
../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
(!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
> + rq->outputsize -= insz;
> + if (!rq->in[ni])
> + continue;
> + kin = kmap_local_page(rq->in[ni]);
> + pi = 0;
> + do {
> + no = (rq->pageofs_out + cur + pi) >> PAGE_SHIFT;
> + po = (rq->pageofs_out + cur + pi) & ~PAGE_MASK;
> + DBG_BUGON(no >= nrpages_out);
> + cnt = min(insz - pi, PAGE_SIZE - po);
ditto
> + if (rq->out[no] == rq->in[ni]) {
> + memmove(kin + po,
> + kin + rq->pageofs_in + pi, cnt);
> + flush_dcache_page(rq->out[no]);
> + } else if (rq->out[no]) {
> + memcpy_to_page(rq->out[no], po,
> + kin + rq->pageofs_in + pi, cnt);
> + }
> + pi += cnt;
> + } while (pi < insz);
> + kunmap_local(kin);
> }
> - kunmap_local(src);
> + DBG_BUGON(ni > nrpages_in);
> return 0;
> }
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
2023-12-08 5:20 ` Yue Hu
@ 2023-12-08 7:34 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-08 7:34 UTC (permalink / raw)
To: Yue Hu; +Cc: linux-erofs, LKML, huyue2, zhangwen
On 2023/12/8 13:20, Yue Hu wrote:
> On Wed, 6 Dec 2023 17:10:56 +0800
> Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
>
>> Sub-page block support is still unusable even with previous commits if
>> interlaced PLAIN pclusters exist. Such pclusters can be found if the
>> fragment feature is enabled.
>>
>> This commit tries to handle "the head part" of interlaced PLAIN
>> pclusters first: it was once explained in commit fdffc091e6f9 ("erofs:
>> support interlaced uncompressed data for compressed files").
>>
>> It uses a unique way for both shifted and interlaced PLAIN pclusters.
>> As an added bonus, PLAIN pclusters larger than the block size is also
>> supported now for the upcoming large lclusters.
>>
>> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
>> ---
>> fs/erofs/decompressor.c | 81 ++++++++++++++++++++++++-----------------
>> 1 file changed, 48 insertions(+), 33 deletions(-)
>>
>> diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
>> index 021be5feb1bc..5ec11f5024b7 100644
>> --- a/fs/erofs/decompressor.c
>> +++ b/fs/erofs/decompressor.c
>> @@ -319,43 +319,58 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
>> static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
>> struct page **pagepool)
>> {
>> - const unsigned int inpages = PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
>> - const unsigned int outpages =
>> + const unsigned int nrpages_in =
>> + PAGE_ALIGN(rq->pageofs_in + rq->inputsize) >> PAGE_SHIFT;
>> + const unsigned int nrpages_out =
>> PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
>> - const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
>> - PAGE_SIZE - rq->pageofs_out);
>> - const unsigned int lefthalf = rq->outputsize - righthalf;
>> - const unsigned int interlaced_offset =
>> - rq->alg == Z_EROFS_COMPRESSION_SHIFTED ? 0 : rq->pageofs_out;
>> - u8 *src;
>> -
>> - if (outpages > 2 && rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
>> - DBG_BUGON(1);
>> - return -EFSCORRUPTED;
>> - }
>> -
>> - if (rq->out[0] == *rq->in) {
>> - DBG_BUGON(rq->pageofs_out);
>> - return 0;
>> + const unsigned int bs = rq->sb->s_blocksize;
>> + unsigned int cur = 0, ni = 0, no, pi, po, insz, cnt;
>> + u8 *kin;
>> +
>> + DBG_BUGON(rq->outputsize > rq->inputsize);
>> + if (rq->alg == Z_EROFS_COMPRESSION_INTERLACED) {
>> + cur = bs - (rq->pageofs_out & (bs - 1));
>> + pi = (rq->pageofs_in + rq->inputsize - cur) & ~PAGE_MASK;
>> + cur = min(cur, rq->outputsize);
>> + if (cur && rq->out[0]) {
>> + kin = kmap_local_page(rq->in[nrpages_in - 1]);
>> + if (rq->out[0] == rq->in[nrpages_in - 1]) {
>> + memmove(kin + rq->pageofs_out, kin + pi, cur);
>> + flush_dcache_page(rq->out[0]);
>> + } else {
>> + memcpy_to_page(rq->out[0], rq->pageofs_out,
>> + kin + pi, cur);
>> + }
>> + kunmap_local(kin);
>> + }
>> + rq->outputsize -= cur;
>> }
>>
>> - src = kmap_local_page(rq->in[inpages - 1]) + rq->pageofs_in;
>> - if (rq->out[0])
>> - memcpy_to_page(rq->out[0], rq->pageofs_out,
>> - src + interlaced_offset, righthalf);
>> -
>> - if (outpages > inpages) {
>> - DBG_BUGON(!rq->out[outpages - 1]);
>> - if (rq->out[outpages - 1] != rq->in[inpages - 1]) {
>> - memcpy_to_page(rq->out[outpages - 1], 0, src +
>> - (interlaced_offset ? 0 : righthalf),
>> - lefthalf);
>> - } else if (!interlaced_offset) {
>> - memmove(src, src + righthalf, lefthalf);
>> - flush_dcache_page(rq->in[inpages - 1]);
>> - }
>> + for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
>> + insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
>
> min_t(unsigned int, ,)?
>
> ../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
> (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
What compiler version are you using? I didn't find any error
and
https://lore.kernel.org/linux-erofs/202312080122.iCCXzSuE-lkp@intel.com
also didn't report this.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
@ 2023-12-08 7:34 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-08 7:34 UTC (permalink / raw)
To: Yue Hu; +Cc: huyue2, linux-erofs, LKML, zhangwen
On 2023/12/8 13:20, Yue Hu wrote:
> On Wed, 6 Dec 2023 17:10:56 +0800
> Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
>
>> Sub-page block support is still unusable even with previous commits if
>> interlaced PLAIN pclusters exist. Such pclusters can be found if the
>> fragment feature is enabled.
>>
>> This commit tries to handle "the head part" of interlaced PLAIN
>> pclusters first: it was once explained in commit fdffc091e6f9 ("erofs:
>> support interlaced uncompressed data for compressed files").
>>
>> It uses a unique way for both shifted and interlaced PLAIN pclusters.
>> As an added bonus, PLAIN pclusters larger than the block size is also
>> supported now for the upcoming large lclusters.
>>
>> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
>> ---
>> fs/erofs/decompressor.c | 81 ++++++++++++++++++++++++-----------------
>> 1 file changed, 48 insertions(+), 33 deletions(-)
>>
>> diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
>> index 021be5feb1bc..5ec11f5024b7 100644
>> --- a/fs/erofs/decompressor.c
>> +++ b/fs/erofs/decompressor.c
>> @@ -319,43 +319,58 @@ static int z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
>> static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
>> struct page **pagepool)
>> {
>> - const unsigned int inpages = PAGE_ALIGN(rq->inputsize) >> PAGE_SHIFT;
>> - const unsigned int outpages =
>> + const unsigned int nrpages_in =
>> + PAGE_ALIGN(rq->pageofs_in + rq->inputsize) >> PAGE_SHIFT;
>> + const unsigned int nrpages_out =
>> PAGE_ALIGN(rq->pageofs_out + rq->outputsize) >> PAGE_SHIFT;
>> - const unsigned int righthalf = min_t(unsigned int, rq->outputsize,
>> - PAGE_SIZE - rq->pageofs_out);
>> - const unsigned int lefthalf = rq->outputsize - righthalf;
>> - const unsigned int interlaced_offset =
>> - rq->alg == Z_EROFS_COMPRESSION_SHIFTED ? 0 : rq->pageofs_out;
>> - u8 *src;
>> -
>> - if (outpages > 2 && rq->alg == Z_EROFS_COMPRESSION_SHIFTED) {
>> - DBG_BUGON(1);
>> - return -EFSCORRUPTED;
>> - }
>> -
>> - if (rq->out[0] == *rq->in) {
>> - DBG_BUGON(rq->pageofs_out);
>> - return 0;
>> + const unsigned int bs = rq->sb->s_blocksize;
>> + unsigned int cur = 0, ni = 0, no, pi, po, insz, cnt;
>> + u8 *kin;
>> +
>> + DBG_BUGON(rq->outputsize > rq->inputsize);
>> + if (rq->alg == Z_EROFS_COMPRESSION_INTERLACED) {
>> + cur = bs - (rq->pageofs_out & (bs - 1));
>> + pi = (rq->pageofs_in + rq->inputsize - cur) & ~PAGE_MASK;
>> + cur = min(cur, rq->outputsize);
>> + if (cur && rq->out[0]) {
>> + kin = kmap_local_page(rq->in[nrpages_in - 1]);
>> + if (rq->out[0] == rq->in[nrpages_in - 1]) {
>> + memmove(kin + rq->pageofs_out, kin + pi, cur);
>> + flush_dcache_page(rq->out[0]);
>> + } else {
>> + memcpy_to_page(rq->out[0], rq->pageofs_out,
>> + kin + pi, cur);
>> + }
>> + kunmap_local(kin);
>> + }
>> + rq->outputsize -= cur;
>> }
>>
>> - src = kmap_local_page(rq->in[inpages - 1]) + rq->pageofs_in;
>> - if (rq->out[0])
>> - memcpy_to_page(rq->out[0], rq->pageofs_out,
>> - src + interlaced_offset, righthalf);
>> -
>> - if (outpages > inpages) {
>> - DBG_BUGON(!rq->out[outpages - 1]);
>> - if (rq->out[outpages - 1] != rq->in[inpages - 1]) {
>> - memcpy_to_page(rq->out[outpages - 1], 0, src +
>> - (interlaced_offset ? 0 : righthalf),
>> - lefthalf);
>> - } else if (!interlaced_offset) {
>> - memmove(src, src + righthalf, lefthalf);
>> - flush_dcache_page(rq->in[inpages - 1]);
>> - }
>> + for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
>> + insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
>
> min_t(unsigned int, ,)?
>
> ../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
> (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
What compiler version are you using? I didn't find any error
and
https://lore.kernel.org/linux-erofs/202312080122.iCCXzSuE-lkp@intel.com
also didn't report this.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
2023-12-08 7:34 ` Gao Xiang
@ 2023-12-08 7:44 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-08 7:44 UTC (permalink / raw)
To: Yue Hu; +Cc: linux-erofs, LKML, huyue2, zhangwen
On 2023/12/8 15:34, Gao Xiang wrote:
>
>
...
>> min_t(unsigned int, ,)?
>>
>> ../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
>> (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
>
> What compiler version are you using? I didn't find any error
> and
> https://lore.kernel.org/linux-erofs/202312080122.iCCXzSuE-lkp@intel.com
>
> also didn't report this.
Did you test against the latest kernel codebase?
>
> Thanks,
> Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
@ 2023-12-08 7:44 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-08 7:44 UTC (permalink / raw)
To: Yue Hu; +Cc: huyue2, linux-erofs, LKML, zhangwen
On 2023/12/8 15:34, Gao Xiang wrote:
>
>
...
>> min_t(unsigned int, ,)?
>>
>> ../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
>> (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
>
> What compiler version are you using? I didn't find any error
> and
> https://lore.kernel.org/linux-erofs/202312080122.iCCXzSuE-lkp@intel.com
>
> also didn't report this.
Did you test against the latest kernel codebase?
>
> Thanks,
> Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
2023-12-08 7:44 ` Gao Xiang
@ 2023-12-08 8:30 ` Yue Hu
-1 siblings, 0 replies; 25+ messages in thread
From: Yue Hu @ 2023-12-08 8:30 UTC (permalink / raw)
To: Gao Xiang; +Cc: linux-erofs, LKML, huyue2, zhangwen
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=GB18030, Size: 706 bytes --]
On Fri, 8 Dec 2023 15:44:33 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> On 2023/12/8 15:34, Gao Xiang wrote:
> >
> >
>
> ...
>
> >> min_t(unsigned int, ,)?
> >>
> >> ../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
> >> 0„20„2 (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
> >
> > What compiler version are you using? I didn't find any error
> > and
> > https://lore.kernel.org/linux-erofs/202312080122.iCCXzSuE-lkp@intel.com
> >
> > also didn't report this.
>
> Did you test against the latest kernel codebase?
Sorry, it's my testing environment issue, please ignore the noise.
Thanks.
>
> >
> > Thanks,
> > Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support
@ 2023-12-08 8:30 ` Yue Hu
0 siblings, 0 replies; 25+ messages in thread
From: Yue Hu @ 2023-12-08 8:30 UTC (permalink / raw)
To: Gao Xiang; +Cc: huyue2, linux-erofs, LKML, zhangwen
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=GB18030, Size: 706 bytes --]
On Fri, 8 Dec 2023 15:44:33 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> On 2023/12/8 15:34, Gao Xiang wrote:
> >
> >
>
> ...
>
> >> min_t(unsigned int, ,)?
> >>
> >> ../include/linux/minmax.h:21:28: error: comparison of distinct pointer types lacks a cast [-Werror]
> >> 0„20„2 (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1)))
> >
> > What compiler version are you using? I didn't find any error
> > and
> > https://lore.kernel.org/linux-erofs/202312080122.iCCXzSuE-lkp@intel.com
> >
> > also didn't report this.
>
> Did you test against the latest kernel codebase?
Sorry, it's my testing environment issue, please ignore the noise.
Thanks.
>
> >
> > Thanks,
> > Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 0/5] erofs: basic sub-page compressed data support
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-11 6:50 ` Yue Hu
-1 siblings, 0 replies; 25+ messages in thread
From: Yue Hu @ 2023-12-11 6:50 UTC (permalink / raw)
To: Gao Xiang; +Cc: linux-erofs, LKML, huyue2
On Wed, 6 Dec 2023 17:10:52 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> Hi folks,
>
> Recently, there are two new cases so that we need to add a preliminary
> sub-page block support for compressed files;
>
> - As Android folks requested, Android ecosystem itself is now switching
> to 16k page size for their arm64 devices. They needs an option of
> 4k-block image compatibility on their new 16k devices;
>
> - Some arm64 cloud servers use 64k page size for their optimized
> workloads, but 4k-block EROFS container images need to be parsed too.
>
> So this patchset mainly addresses the requirements above with a very
> very simple approach as a start: just allocate short-lived temporary
> buffers all the time to keep compressed data if sub-page blocks are
> identified. In other words, no inplace/cache decompression for
> the preliminary support.
>
> This patchset survives EROFS stress test on my own testfarms.
>
> Thanks,
> Gao Xiang
>
> Gao Xiang (5):
> erofs: support I/O submission for sub-page compressed blocks
> erofs: record `pclustersize` in bytes instead of pages
> erofs: fix up compacted indexes for block size < 4096
> erofs: refine z_erofs_transform_plain() for sub-page block support
> erofs: enable sub-page compressed block support
>
> fs/erofs/decompressor.c | 81 +++++++++------
> fs/erofs/inode.c | 6 +-
> fs/erofs/zdata.c | 224 ++++++++++++++++++----------------------
> fs/erofs/zmap.c | 32 +++---
> 4 files changed, 169 insertions(+), 174 deletions(-)
>
Reviewed-by: Yue Hu <huyue2@coolpad.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 0/5] erofs: basic sub-page compressed data support
@ 2023-12-11 6:50 ` Yue Hu
0 siblings, 0 replies; 25+ messages in thread
From: Yue Hu @ 2023-12-11 6:50 UTC (permalink / raw)
To: Gao Xiang; +Cc: huyue2, linux-erofs, LKML
On Wed, 6 Dec 2023 17:10:52 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> Hi folks,
>
> Recently, there are two new cases so that we need to add a preliminary
> sub-page block support for compressed files;
>
> - As Android folks requested, Android ecosystem itself is now switching
> to 16k page size for their arm64 devices. They needs an option of
> 4k-block image compatibility on their new 16k devices;
>
> - Some arm64 cloud servers use 64k page size for their optimized
> workloads, but 4k-block EROFS container images need to be parsed too.
>
> So this patchset mainly addresses the requirements above with a very
> very simple approach as a start: just allocate short-lived temporary
> buffers all the time to keep compressed data if sub-page blocks are
> identified. In other words, no inplace/cache decompression for
> the preliminary support.
>
> This patchset survives EROFS stress test on my own testfarms.
>
> Thanks,
> Gao Xiang
>
> Gao Xiang (5):
> erofs: support I/O submission for sub-page compressed blocks
> erofs: record `pclustersize` in bytes instead of pages
> erofs: fix up compacted indexes for block size < 4096
> erofs: refine z_erofs_transform_plain() for sub-page block support
> erofs: enable sub-page compressed block support
>
> fs/erofs/decompressor.c | 81 +++++++++------
> fs/erofs/inode.c | 6 +-
> fs/erofs/zdata.c | 224 ++++++++++++++++++----------------------
> fs/erofs/zmap.c | 32 +++---
> 4 files changed, 169 insertions(+), 174 deletions(-)
>
Reviewed-by: Yue Hu <huyue2@coolpad.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/5] erofs: support I/O submission for sub-page compressed blocks
2023-12-06 9:10 ` Gao Xiang
@ 2023-12-11 8:10 ` Gao Xiang
-1 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-11 8:10 UTC (permalink / raw)
To: linux-erofs; +Cc: LKML, dhavale, Yue Hu
On 2023/12/6 17:10, Gao Xiang wrote:
> Add a basic I/O submission path first to support sub-page blocks:
>
> - Temporary short-lived pages will be used entirely;
>
> - In-place I/O pages can be used partially, but compressed pages need
> to be able to be mapped in contiguous virtual memory.
>
> As a start, currently cache decompression is explicitly disabled for
> sub-page blocks, which will be supported in the future.
>
> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
...
> + cur = mdev.m_pa;
> + end = cur + pcl->pclusterpages << PAGE_SHIFT;
In this patch, here should be
end = cur + (pcl->pclusterpages << PAGE_SHIFT);
but since this line will be immediately updated in the next patch
as `end = cur + pcl->pclustersize;` so it will have no impact.
I've fixed it up in the development tree.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 1/5] erofs: support I/O submission for sub-page compressed blocks
@ 2023-12-11 8:10 ` Gao Xiang
0 siblings, 0 replies; 25+ messages in thread
From: Gao Xiang @ 2023-12-11 8:10 UTC (permalink / raw)
To: linux-erofs; +Cc: Yue Hu, LKML
On 2023/12/6 17:10, Gao Xiang wrote:
> Add a basic I/O submission path first to support sub-page blocks:
>
> - Temporary short-lived pages will be used entirely;
>
> - In-place I/O pages can be used partially, but compressed pages need
> to be able to be mapped in contiguous virtual memory.
>
> As a start, currently cache decompression is explicitly disabled for
> sub-page blocks, which will be supported in the future.
>
> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
...
> + cur = mdev.m_pa;
> + end = cur + pcl->pclusterpages << PAGE_SHIFT;
In this patch, here should be
end = cur + (pcl->pclusterpages << PAGE_SHIFT);
but since this line will be immediately updated in the next patch
as `end = cur + pcl->pclustersize;` so it will have no impact.
I've fixed it up in the development tree.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 0/5] erofs: basic sub-page compressed data support
2023-12-06 9:10 ` Gao Xiang
` (6 preceding siblings ...)
(?)
@ 2023-12-14 14:58 ` Chao Yu
-1 siblings, 0 replies; 25+ messages in thread
From: Chao Yu @ 2023-12-14 14:58 UTC (permalink / raw)
To: Gao Xiang, linux-erofs; +Cc: LKML
On 2023/12/6 17:10, Gao Xiang wrote:
> Hi folks,
>
> Recently, there are two new cases so that we need to add a preliminary
> sub-page block support for compressed files;
>
> - As Android folks requested, Android ecosystem itself is now switching
> to 16k page size for their arm64 devices. They needs an option of
> 4k-block image compatibility on their new 16k devices;
>
> - Some arm64 cloud servers use 64k page size for their optimized
> workloads, but 4k-block EROFS container images need to be parsed too.
>
> So this patchset mainly addresses the requirements above with a very
> very simple approach as a start: just allocate short-lived temporary
> buffers all the time to keep compressed data if sub-page blocks are
> identified. In other words, no inplace/cache decompression for
> the preliminary support.
>
> This patchset survives EROFS stress test on my own testfarms.
Looks good to me, feel free to add:
Reviewed-by: Chao Yu <chao@kernel.org>
Thanks,
>
> Thanks,
> Gao Xiang
>
> Gao Xiang (5):
> erofs: support I/O submission for sub-page compressed blocks
> erofs: record `pclustersize` in bytes instead of pages
> erofs: fix up compacted indexes for block size < 4096
> erofs: refine z_erofs_transform_plain() for sub-page block support
> erofs: enable sub-page compressed block support
>
> fs/erofs/decompressor.c | 81 +++++++++------
> fs/erofs/inode.c | 6 +-
> fs/erofs/zdata.c | 224 ++++++++++++++++++----------------------
> fs/erofs/zmap.c | 32 +++---
> 4 files changed, 169 insertions(+), 174 deletions(-)
>
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2023-12-14 14:58 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-06 9:10 [PATCH 0/5] erofs: basic sub-page compressed data support Gao Xiang
2023-12-06 9:10 ` Gao Xiang
2023-12-06 9:10 ` [PATCH 1/5] erofs: support I/O submission for sub-page compressed blocks Gao Xiang
2023-12-06 9:10 ` Gao Xiang
2023-12-11 8:10 ` Gao Xiang
2023-12-11 8:10 ` Gao Xiang
2023-12-06 9:10 ` [PATCH 2/5] erofs: record `pclustersize` in bytes instead of pages Gao Xiang
2023-12-06 9:10 ` Gao Xiang
2023-12-06 9:10 ` [PATCH 3/5] erofs: fix up compacted indexes for block size < 4096 Gao Xiang
2023-12-06 9:10 ` Gao Xiang
2023-12-06 9:10 ` [PATCH 4/5] erofs: refine z_erofs_transform_plain() for sub-page block support Gao Xiang
2023-12-06 9:10 ` Gao Xiang
2023-12-08 5:20 ` Yue Hu
2023-12-08 5:20 ` Yue Hu
2023-12-08 7:34 ` Gao Xiang
2023-12-08 7:34 ` Gao Xiang
2023-12-08 7:44 ` Gao Xiang
2023-12-08 7:44 ` Gao Xiang
2023-12-08 8:30 ` Yue Hu
2023-12-08 8:30 ` Yue Hu
2023-12-06 9:10 ` [PATCH 5/5] erofs: enable sub-page compressed " Gao Xiang
2023-12-06 9:10 ` Gao Xiang
2023-12-11 6:50 ` [PATCH 0/5] erofs: basic sub-page compressed data support Yue Hu
2023-12-11 6:50 ` Yue Hu
2023-12-14 14:58 ` Chao Yu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.