From: Qu Wenruo <wqu@suse.com>
To: linux-btrfs@vger.kernel.org
Subject: [PATCH 00/17] btrfs: add read-only support for subpage sector size
Date: Tue, 8 Sep 2020 15:52:13 +0800 [thread overview]
Message-ID: <20200908075230.86856-1-wqu@suse.com> (raw)
Patches can be fetched from github:
https://github.com/adam900710/linux/tree/subpage
Currently btrfs only allows to mount fs with sectorsize == PAGE_SIZE.
That means, for 64K page size system, they can only use 64K sector size
fs.
This brings a big compatible problem for btrfs.
This patch is going to slightly solve the problem by, allowing 64K
system to mount 4K sectorsize fs in read-only mode.
The main objective here, is to remove the blockage in the code base, and
pave the road to full RW mount support.
== What works ==
Existing regular page sized sector size support
Subpage read-only Mount (with all self tests and ASSERT)
Subpage metadata read (including all trees and inline extents, and csum checking)
Subpage uncompressed data read (with csum checking)
== What doesn't work ==
Read-write mount (see the subject)
Compressed data read
== Challenge we meet ==
The main problem is metadata, where we have several limitations:
- We always read the full page of a metadata
In subpage case, one full page can contain several tree blocks.
- We use page::private to point to extent buffer
This means we currently can only support one-page-to-one-extent-buffer
mapping.
For subpage size support, we need one-page-to-multiple-extent-buffer
mapping.
== Solutions ==
So here for the metadata part, we use the following methods to
workaround the problem:
- Introduce subpage_eb_mapping structure to do bitmap
Now for subpage, page::private points to a subpage_eb_mapping
structure, which has a bitmap to mapping one page to multiple extent
buffers.
- Still do full page read for metadata
This means, at read time, we're not reading just one extent buffer,
but possibly many more.
In that case, we first do tree block verification for the tree blocks
triggering the read, and mark the page uptodate.
For newly incoming tree block read, they will check if the tree block
is verified. If not verified, even if the page is uptodate, we still
need to check the extent buffer.
By this all subpage extent buffers are verified properly.
For data part, it's pretty simple, all existing infrastructure can be
easily converted to support subpage read, without any subpage specific
handing yet.
== Patchset structure ==
The structure of the patchset:
Patch 01~11: Preparation patches for metadata subpage read support.
These patches can be merged without problem, and work for
both regular and subpage case.
Patch 12~14: Patches for data subpage read support.
These patches works for both cases.
That means, patch 01~14 can be applied to current kernel, and shouldn't
affect any existing behavior.
Patch 15~17: Subpage metadata read specific patches.
These patches introduces the main part of the subpage
metadata read support.
The number of patches is the main reason I'm submitting them to the mail
list. As there are too many preparation patches already.
Qu Wenruo (17):
btrfs: extent-io-tests: remove invalid tests
btrfs: calculate inline extent buffer page size based on page size
btrfs: remove the open-code to read disk-key
btrfs: make btrfs_fs_info::buffer_radix to take sector size devided
values
btrfs: don't allow tree block to cross page boundary for subpage
support
btrfs: handle sectorsize < PAGE_SIZE case for extent buffer accessors
btrfs: make csum_tree_block() handle sectorsize smaller than page size
btrfs: refactor how we extract extent buffer from page for
alloc_extent_buffer()
btrfs: refactor btrfs_release_extent_buffer_pages()
btrfs: add assert_spin_locked() for attach_extent_buffer_page()
btrfs: extract the extent buffer verification from
btree_readpage_end_io_hook()
btrfs: remove the unnecessary parameter @start and @len for
check_data_csum()
btrfs: extent_io: only require sector size alignment for page read
btrfs: make btrfs_readpage_end_io_hook() follow sector size
btrfs: introduce subpage_eb_mapping for extent buffers
btrfs: handle extent buffer verification proper for subpage size
btrfs: allow RO mount of 4K sector size fs on 64K page system
fs/btrfs/ctree.c | 13 +-
fs/btrfs/ctree.h | 38 ++-
fs/btrfs/disk-io.c | 111 ++++---
fs/btrfs/disk-io.h | 1 +
fs/btrfs/extent_io.c | 538 +++++++++++++++++++++++++------
fs/btrfs/extent_io.h | 19 +-
fs/btrfs/inode.c | 40 ++-
fs/btrfs/struct-funcs.c | 18 +-
fs/btrfs/super.c | 7 +
fs/btrfs/tests/extent-io-tests.c | 26 +-
10 files changed, 633 insertions(+), 178 deletions(-)
--
2.28.0
next reply other threads:[~2020-09-08 7:52 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-08 7:52 Qu Wenruo [this message]
2020-09-08 7:52 ` [PATCH 01/17] btrfs: extent-io-tests: remove invalid tests Qu Wenruo
2020-09-09 12:26 ` Nikolay Borisov
2020-09-09 13:06 ` Qu Wenruo
2020-09-08 7:52 ` [PATCH 02/17] btrfs: calculate inline extent buffer page size based on page size Qu Wenruo
2020-09-11 9:56 ` Nikolay Borisov
2020-09-11 10:13 ` Qu Wenruo
2020-09-08 7:52 ` [PATCH 03/17] btrfs: remove the open-code to read disk-key Qu Wenruo
2020-09-11 10:07 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 04/17] btrfs: make btrfs_fs_info::buffer_radix to take sector size devided values Qu Wenruo
2020-09-11 10:11 ` Nikolay Borisov
2020-09-11 10:15 ` Qu Wenruo
2020-09-08 7:52 ` [PATCH 05/17] btrfs: don't allow tree block to cross page boundary for subpage support Qu Wenruo
2020-09-11 10:26 ` Nikolay Borisov
2020-09-11 11:36 ` Qu Wenruo
2020-09-11 12:08 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 06/17] btrfs: handle sectorsize < PAGE_SIZE case for extent buffer accessors Qu Wenruo
2020-09-08 7:52 ` [PATCH 07/17] btrfs: make csum_tree_block() handle sectorsize smaller than page size Qu Wenruo
2020-09-11 11:10 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 08/17] btrfs: refactor how we extract extent buffer from page for alloc_extent_buffer() Qu Wenruo
2020-09-11 11:14 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 09/17] btrfs: refactor btrfs_release_extent_buffer_pages() Qu Wenruo
2020-09-11 11:17 ` Nikolay Borisov
2020-09-11 11:39 ` Qu Wenruo
2020-09-08 7:52 ` [PATCH 10/17] btrfs: add assert_spin_locked() for attach_extent_buffer_page() Qu Wenruo
2020-09-11 11:22 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 11/17] btrfs: extract the extent buffer verification from btree_readpage_end_io_hook() Qu Wenruo
2020-09-11 13:00 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 12/17] btrfs: remove the unnecessary parameter @start and @len for check_data_csum() Qu Wenruo
2020-09-11 13:50 ` Nikolay Borisov
2020-09-08 7:52 ` [PATCH 13/17] btrfs: extent_io: only require sector size alignment for page read Qu Wenruo
2020-09-11 13:55 ` Nikolay Borisov
2020-09-15 1:54 ` Qu Wenruo
2020-09-08 7:52 ` [PATCH 14/17] btrfs: make btrfs_readpage_end_io_hook() follow sector size Qu Wenruo
2020-09-09 17:34 ` Goldwyn Rodrigues
2020-09-10 0:05 ` Qu Wenruo
2020-09-10 14:26 ` Goldwyn Rodrigues
2020-09-08 7:52 ` [PATCH 15/17] btrfs: introduce subpage_eb_mapping for extent buffers Qu Wenruo
2020-09-08 10:22 ` kernel test robot
2020-09-08 14:24 ` Dan Carpenter
2020-09-08 7:52 ` [PATCH 16/17] btrfs: handle extent buffer verification proper for subpage size Qu Wenruo
2020-09-08 7:52 ` [PATCH 17/17] btrfs: allow RO mount of 4K sector size fs on 64K page system Qu Wenruo
2020-09-08 8:03 ` [PATCH 00/17] btrfs: add read-only support for subpage sector size Qu Wenruo
2020-09-11 10:24 ` Qu Wenruo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200908075230.86856-1-wqu@suse.com \
--to=wqu@suse.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).