From: Chandan Rajendra <chandan@linux.vnet.ibm.com>
To: linux-btrfs@vger.kernel.org
Cc: Chandan Rajendra <chandan@linux.vnet.ibm.com>,
dsterba@suse.cz, clm@fb.com, jbacik@fb.com, chandan@mykolab.com,
aneesh.kumar@linux.vnet.ibm.com
Subject: [PATCH V16 00/18] Allow I/O on blocks whose size is less than page size
Date: Thu, 14 Apr 2016 14:02:37 +0530 [thread overview]
Message-ID: <1460622775-20723-1-git-send-email-chandan@linux.vnet.ibm.com> (raw)
Btrfs assumes block size to be the same as the machine's page
size. This would mean that a Btrfs instance created on a 4k page size
machine (e.g. x86) will not be mountable on machines with larger page
sizes (e.g. PPC64/AARCH64). This patchset aims to resolve this
incompatibility.
This patchset continues with the work posted previously at
http://thread.gmane.org/gmane.comp.file-systems.btrfs/53073.
I have reverted the upstream commit "btrfs: fix lockups from
btrfs_clear_path_blocking" (f82c458a2c3ffb94b431fc6ad791a79df1b3713e)
since this led to soft-lockups when the patch "Btrfs:
subpagesize-blocksize: Prevent writes to an extent buffer when
PG_writeback flag is set" is applied. During 2015's Vault Conference
Btrfs meetup, Chris Mason had suggested that he will write up a
suitable locking function to be used when writing dirty pages that map
metadata blocks. Until we have a suitable locking function available,
this patchset temporarily disables the commit
f82c458a2c3ffb94b431fc6ad791a79df1b3713e.
The commits for the Btrfs kernel module can be found at
https://github.com/chandanr/linux/tree/btrfs/subpagesize-blocksize.
To create a filesystem with block size < page size, a patched version
of the Btrfs-progs package is required. The corresponding fixes for
Btrfs-progs can be found at
https://github.com/chandanr/btrfs-progs/tree/btrfs/subpagesize-blocksize.
Fstests run status:
1. x86_64
- With 4k sectorsize, all the tests that succeed with the master
branch at git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git
branch also do so with the patches applied.
2. ppc64
- With 4k sectorsize, 16k nodesize and with "nospace_cache" mount
option, except for scrub and compression tests, all the tests
that succeed with the master branch at
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git
branch also do so with the patches applied.
- With 64k sectorsize & nodesize, all the tests that succeed with
the master branch at
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git
branch also do so with the patches applied.
TODO:
1. The selftests code needs to be fixed to work in subpage-blocksize
scenario.
2. I am planning to fix Scrub & Compression via a separate patchset.
Changes from V15:
1. The invocation of cleancache_get_page() in __do_readpage() assumed
blocksize to be same as PAGE_SIZE. We now invoke cleancache_get_page()
only if blocksize is same as PAGE_SIZE. Thanks to David Sterba for
pointing this out.
2. In __extent_writepage_io() we used to accumulate all the contiguous
dirty blocks within the page before submitting the file offset range
for I/O. In some cases this caused the bio to span across more than
a stripe. For example, With 4k block size, 64K stripe size
and 64K page size, assume
- All the blocks mapped by the page are contiguous on the logical
address space.
- The first block of the page is mapped to the second block of the
stripe.
In such a scenario, we would add all the blocks of the page to
bio. This would mean that we would overflow the stripe by one 4K
block. Hence this patchset removes the optimization and invokes
submit_extent_page() for every dirty 4K block.
3. The following patches are newly added:
- Btrfs: subpage-blocksize: __btrfs_lookup_bio_sums: Set offset
when moving to a new bio_vec
- Btrfs: subpage-blocksize: Make file extent relocate code subpage
blocksize aware
- Btrfs: btrfs_clone: Flush dirty blocks of a page that do not map
the clone range
Changes from V14:
1. Fix usage of cleancache_get_page() in __do_readpage().
In filesystems which support subpage-blocksize scenario, a page can
map one or more blocks. Hence cleancache_get_page() should be
invoked only when the page maps a non-hole extent and block size
being used is equal to the page size. Thanks to David Sterba for
pointing this out.
2. Replace page_read_complete() and page_write_complete() functions
with page_io_complete().
3. Provide more documentation (as part of both commit message and code
comments) about the usage of the per-page
btrfs_page_private->io_lock.
Changes from V13:
1. Enable dedup ioctl to work in subpagesize-blocksize scenario.
Changes from V12:
1. The logic in the function btrfs_punch_hole() has been fixed to
check for the presence of BLK_STATE_UPTODATE flags for blocks in
pages which partially map the file range being punched.
Changes from V11:
1. Addressed the review comments provided by Liu Bo for version V11.
2. Fixed file defragmentation code to work in subpagesize-blocksize
scenario.
3. Many "hard to reproduce" bugs were fixed.
Chandan Rajendra (18):
Btrfs: subpage-blocksize: Fix whole page read.
Btrfs: subpage-blocksize: Fix whole page write
Btrfs: subpage-blocksize: Make sure delalloc range intersects with the
locked page's range
Btrfs: subpage-blocksize: Define extent_buffer_head.
Btrfs: subpage-blocksize: Read tree blocks whose size is <
PAGE_CACHE_SIZE
Btrfs: subpage-blocksize: Write only dirty extent buffers belonging to
a page
Btrfs: subpage-blocksize: Allow mounting filesystems where sectorsize
< PAGE_SIZE
Btrfs: subpage-blocksize: Deal with partial ordered extent
allocations.
Btrfs: subpage-blocksize: Explicitly track I/O status of blocks of an
ordered extent.
Btrfs: subpage-blocksize: btrfs_punch_hole: Fix uptodate blocks check
Btrfs: subpage-blocksize: Prevent writes to an extent buffer when
PG_writeback flag is set
Revert "btrfs: fix lockups from btrfs_clear_path_blocking"
Btrfs: subpage-blocksize: Fix file defragmentation code
Btrfs: subpage-blocksize: extent_clear_unlock_delalloc: Prevent page
from being unlocked more than once
Btrfs: subpage-blocksize: Enable dedupe ioctl
Btrfs: btrfs_clone: Flush dirty blocks of a page that do not map the
clone range
Btrfs: subpage-blocksize: Make file extent relocate code subpage
blocksize aware
Btrfs: subpage-blocksize: __btrfs_lookup_bio_sums: Set offset when
moving to a new bio_vec
fs/btrfs/ctree.c | 37 +-
fs/btrfs/ctree.h | 6 +-
fs/btrfs/disk-io.c | 156 ++--
fs/btrfs/disk-io.h | 3 +
fs/btrfs/extent-tree.c | 17 +-
fs/btrfs/extent_io.c | 1611 +++++++++++++++++++++++++++++-------------
fs/btrfs/extent_io.h | 145 +++-
fs/btrfs/file-item.c | 7 +-
fs/btrfs/file.c | 82 ++-
fs/btrfs/inode.c | 491 +++++++++----
fs/btrfs/ioctl.c | 219 ++++--
fs/btrfs/locking.c | 24 +-
fs/btrfs/locking.h | 2 -
fs/btrfs/ordered-data.c | 19 +
fs/btrfs/ordered-data.h | 4 +
fs/btrfs/relocation.c | 70 +-
fs/btrfs/root-tree.c | 2 +-
fs/btrfs/volumes.c | 2 +-
include/trace/events/btrfs.h | 2 +-
19 files changed, 2050 insertions(+), 849 deletions(-)
--
2.1.0
next reply other threads:[~2016-04-14 8:33 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-04-14 8:32 Chandan Rajendra [this message]
2016-04-14 8:32 ` [PATCH V16 01/18] Btrfs: subpage-blocksize: Fix whole page read Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 02/18] Btrfs: subpage-blocksize: Fix whole page write Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 03/18] Btrfs: subpage-blocksize: Make sure delalloc range intersects with the locked page's range Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 04/18] Btrfs: subpage-blocksize: Define extent_buffer_head Chandan Rajendra
2016-04-14 9:57 ` kbuild test robot
2016-04-14 8:32 ` [PATCH V16 05/18] Btrfs: subpage-blocksize: Read tree blocks whose size is < PAGE_CACHE_SIZE Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 06/18] Btrfs: subpage-blocksize: Write only dirty extent buffers belonging to a page Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 07/18] Btrfs: subpage-blocksize: Allow mounting filesystems where sectorsize < PAGE_SIZE Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 08/18] Btrfs: subpage-blocksize: Deal with partial ordered extent allocations Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 09/18] Btrfs: subpage-blocksize: Explicitly track I/O status of blocks of an ordered extent Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 10/18] Btrfs: subpage-blocksize: btrfs_punch_hole: Fix uptodate blocks check Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 11/18] Btrfs: subpage-blocksize: Prevent writes to an extent buffer when PG_writeback flag is set Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 12/18] Revert "btrfs: fix lockups from btrfs_clear_path_blocking" Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 13/18] Btrfs: subpage-blocksize: Fix file defragmentation code Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 14/18] Btrfs: subpage-blocksize: extent_clear_unlock_delalloc: Prevent page from being unlocked more than once Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 15/18] Btrfs: subpage-blocksize: Enable dedupe ioctl Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 16/18] Btrfs: btrfs_clone: Flush dirty blocks of a page that do not map the clone range Chandan Rajendra
2016-04-14 9:36 ` kbuild test robot
2016-04-14 8:32 ` [PATCH V16 17/18] Btrfs: subpage-blocksize: Make file extent relocate code subpage blocksize aware Chandan Rajendra
2016-04-14 8:32 ` [PATCH V16 18/18] Btrfs: subpage-blocksize: __btrfs_lookup_bio_sums: Set offset when moving to a new bio_vec Chandan Rajendra
2016-04-14 11:30 ` [PATCH V16 00/18] Allow I/O on blocks whose size is less than page size Chandan Rajendra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1460622775-20723-1-git-send-email-chandan@linux.vnet.ibm.com \
--to=chandan@linux.vnet.ibm.com \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=chandan@mykolab.com \
--cc=clm@fb.com \
--cc=dsterba@suse.cz \
--cc=jbacik@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.