* [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O.
@ 2014-11-14 15:38 Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 01/17] Btrfs: subpagesize-blocksize: Get rid of whole page reads Chandan Rajendra
` (16 more replies)
0 siblings, 17 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
This patchset continues with the work posted earlier at
http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg38775.html.
Changes from V1:
1. Remove usage of bio_vec->bv_{len,offset} in end_bio_extent_readpage()
and end_bio_extent_writepage().
Changes from V2:
1. Get __extent_writepage() to write only the dirty blocks of a page.
2. Fix "page private not zero on page" warning message which is printed
when running xfstests.
Changes from V3:
1. Get "Hole punching" and "Extent preallocation" to work correctly in
subpagesize-blocksize scenario.
2. Get btrfs_page_mkwrite() to reserve space in sectorsized units.
Changes from V4:
1. V2's "Btrfs: subpagesize-blocksize: Get rid of whole page reads"
patch was incorrectly replaced with an older version when working
on V3 patches. Fix this.
2. Fix btrfs_endio_direct_read() to compute checksums for all possible
blocks in a page.
Changes from V5:
1. Rebased patchset on top of current btrfs-next tree (i.e. commit
8d875f95da43c6a8f18f77869f2ef26e9594fecc). This involved using
"immutable biovecs".
2. Deal with partially allocated ordered extents across a page.
3. Explicitly track I/O status of blocks of an ordered extent.
Changes from V6:
1. Fix softlockup issue that occured during unmounting a 4k blocksized
filesystem instance.
2. Track blocks of an ordered extent submitted for write I/O to avoid
I/O resubmission in certain scenarios.
Changes from V7:
1. Fix a softlockup issue that occured because the page corresponding
to the delalloc region did not exist. This bug was introduced by
the code added in btrfs_invalidatepage and related functions in V7
version.
Changes from V8:
1. In subpagesize-blocksize scenario, prevent writes to an extent
buffer when the corresponding page's PG_writeback flag is set. This
race condition was triggered when running xfstests' generic/083
test. With the new patch applied, I have run the complete xfstests
suite as well as run generic/083 multiple times on both 4k and 2k
block size setups. There were 2 non-related test failures that
occured rarely, but they were reproducible even when the patch was
not applied.
Xfstests' generic tests were run on an x86_64 machine with the patches
applied. The Btrfs kernel module was compiled without ACL support and
hence tests related to that were not run.
For 2k blocksize, the following xfstests' generic tests failed:
1. generic/091
2. generic/125
3. generic/251
4. generic/274
The following xfstests' generic tests failed for both 2k and 4k blocksize:
1. generic/224 (OOM)
This looks mostly an issue caused by non-btrfs code as the test failed
for the exact same reason when run on an ext4 filesystem instance.
2. generic/263
FALLOC_FL_ZERO_RANGE isn't supported by Btrfs. Hence the test fails.
The following is a list of known TODO items which will be implemented in
future revisions of this patchset:
1. Fix synchronization issues that occur in multi-processor setup.
2. Get Xfstests' generic tests to successfully run on both 4k and 2k
blocksizes.
2. Remove PAGE_CACHE_SIZE delalloc reservation in btrfs_writepage_fixup_worker().
3. Create separate slab caches for 'extent buffer head' and 'extent buffer'.
4. Add 'leak list' tracking for 'extent buffer' instances.
5. Rename EXTENT_BUFFER_TREE_REF and EXTENT_BUFFER_IN_TREE to
EXTENT_BUFFER_HEAD_TREE_REF and EXTENT_BUFFER_HEAD_IN_TREE respectively.
Chandan Rajendra (15):
Btrfs: subpagesize-blocksize: Get rid of whole page reads.
Btrfs: subpagesize-blocksize: Get rid of whole page writes.
Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release
extents aligned to block size.
Btrfs: subpagesize-blocksize: Read tree blocks whose size is
<PAGE_CACHE_SIZE.
Btrfs: subpagesize-blocksize: Write only dirty extent buffers
belonging to a page
Btrfs: subpagesize-blocksize: Compute and look up csums based on
sectorsized blocks.
Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty
blocks of a page.
Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units.
Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in
sectorsized units.
Btrfs: subpagesize-blocksize: Search for all ordered extents that
could span across a page.
Btrfs: subpagesize-blocksize: Deal with partial ordered extent
allocations.
Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of
an ordered extent.
Btrfs: subpagesize-blocksize: Revert commit
fc4adbff823f76577ece26dcb88bf6f8392dbd43.
Btrfs: subpagesize-blocksize: Track blocks of ordered extent submitted
for write I/O.
Btrfs: subpagesize-blocksize: Prevent writes to an extent buffer when
PG_writeback flag is set.
Chandra Seetharaman (2):
Btrfs: subpagesize-blocksize: Define extent_buffer_head.
Btrfs: subpagesize-blocksize: Allow mounting filesystems where
sectorsize != PAGE_SIZE
fs/btrfs/backref.c | 2 +-
fs/btrfs/btrfs_inode.h | 2 -
fs/btrfs/ctree.c | 22 +-
fs/btrfs/ctree.h | 8 +-
fs/btrfs/disk-io.c | 117 ++--
fs/btrfs/disk-io.h | 3 +
fs/btrfs/extent-tree.c | 18 +-
fs/btrfs/extent_io.c | 1253 +++++++++++++++++++++++++++++-------------
fs/btrfs/extent_io.h | 55 +-
fs/btrfs/file-item.c | 87 +--
fs/btrfs/file.c | 77 +--
fs/btrfs/inode.c | 580 ++++++++++++-------
fs/btrfs/ordered-data.c | 19 +
fs/btrfs/ordered-data.h | 8 +
fs/btrfs/volumes.c | 2 +-
include/trace/events/btrfs.h | 2 +-
16 files changed, 1517 insertions(+), 738 deletions(-)
--
2.1.0
^ permalink raw reply [flat|nested] 18+ messages in thread
* [RFC PATCH V9 01/17] Btrfs: subpagesize-blocksize: Get rid of whole page reads.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 02/17] Btrfs: subpagesize-blocksize: Get rid of whole page writes Chandan Rajendra
` (15 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
Based on original patch from Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
For the subpagesize-blocksize scenario, a page can contain multiple
blocks. This patch handles this case.
This patch also brings back check_page_locked() to reliably unlock pages in
readpage's end bio function.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 152 ++++++++++++++++++++-------------------------------
1 file changed, 58 insertions(+), 94 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index a389820..5d9cc68 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -1951,15 +1951,29 @@ int test_range_bit(struct extent_io_tree *tree, u64 start, u64 end,
* helper function to set a given page up to date if all the
* extents in the tree for that page are up to date
*/
-static void check_page_uptodate(struct extent_io_tree *tree, struct page *page)
+static void check_page_uptodate(struct extent_io_tree *tree, struct page *page,
+ struct extent_state *cached)
{
u64 start = page_offset(page);
u64 end = start + PAGE_CACHE_SIZE - 1;
- if (test_range_bit(tree, start, end, EXTENT_UPTODATE, 1, NULL))
+ if (test_range_bit(tree, start, end, EXTENT_UPTODATE, 1, cached))
SetPageUptodate(page);
}
/*
+ * helper function to unlock a page if all the extents in the tree
+ * for that page are unlocked
+ */
+static void check_page_locked(struct extent_io_tree *tree, struct page *page)
+{
+ u64 start = page_offset(page);
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+
+ if (!test_range_bit(tree, start, end, EXTENT_LOCKED, 0, NULL))
+ unlock_page(page);
+}
+
+/*
* When IO fails, either with EIO or csum verification fails, we
* try other mirrors that might have a good copy of the data. This
* io_failure_record is used to record state as we go through all the
@@ -2275,7 +2289,9 @@ static int bio_readpage_error(struct bio *failed_bio, u64 phy_offset,
* a) deliver good data to the caller
* b) correct the bad sectors on disk
*/
- if (failed_bio->bi_vcnt > 1) {
+ if ((failed_bio->bi_vcnt > 1)
+ || (failed_bio->bi_io_vec->bv_len
+ > BTRFS_I(inode)->root->sectorsize)) {
/*
* to fulfill b), we need to know the exact failing sectors, as
* we don't want to rewrite any more than the failed ones. thus,
@@ -2422,18 +2438,6 @@ static void end_bio_extent_writepage(struct bio *bio, int err)
bio_put(bio);
}
-static void
-endio_readpage_release_extent(struct extent_io_tree *tree, u64 start, u64 len,
- int uptodate)
-{
- struct extent_state *cached = NULL;
- u64 end = start + len - 1;
-
- if (uptodate && tree->track_uptodate)
- set_extent_uptodate(tree, start, end, &cached, GFP_ATOMIC);
- unlock_extent_cached(tree, start, end, &cached, GFP_ATOMIC);
-}
-
/*
* after a readpage IO is done, we need to:
* clear the uptodate bits on error
@@ -2450,13 +2454,12 @@ static void end_bio_extent_readpage(struct bio *bio, int err)
struct bio_vec *bvec;
int uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags);
struct btrfs_io_bio *io_bio = btrfs_io_bio(bio);
+ struct extent_state *cached = NULL;
struct extent_io_tree *tree;
u64 offset = 0;
u64 start;
u64 end;
- u64 len;
- u64 extent_start = 0;
- u64 extent_len = 0;
+ int nr_sectors;
int mirror;
int ret;
int i;
@@ -2467,54 +2470,31 @@ static void end_bio_extent_readpage(struct bio *bio, int err)
bio_for_each_segment_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
struct inode *inode = page->mapping->host;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
pr_debug("end_bio_extent_readpage: bi_sector=%llu, err=%d, "
"mirror=%lu\n", (u64)bio->bi_iter.bi_sector, err,
io_bio->mirror_num);
tree = &BTRFS_I(inode)->io_tree;
- /* We always issue full-page reads, but if some block
- * in a page fails to read, blk_update_request() will
- * advance bv_offset and adjust bv_len to compensate.
- * Print a warning for nonzero offsets, and an error
- * if they don't add up to a full page. */
- if (bvec->bv_offset || bvec->bv_len != PAGE_CACHE_SIZE) {
- if (bvec->bv_offset + bvec->bv_len != PAGE_CACHE_SIZE)
- btrfs_err(BTRFS_I(page->mapping->host)->root->fs_info,
- "partial page read in btrfs with offset %u and length %u",
- bvec->bv_offset, bvec->bv_len);
- else
- btrfs_info(BTRFS_I(page->mapping->host)->root->fs_info,
- "incomplete page read in btrfs with offset %u and "
- "length %u",
- bvec->bv_offset, bvec->bv_len);
- }
-
- start = page_offset(page);
- end = start + bvec->bv_offset + bvec->bv_len - 1;
- len = bvec->bv_len;
-
+ start = page_offset(page) + bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+ nr_sectors = bvec->bv_len >> inode->i_sb->s_blocksize_bits;
mirror = io_bio->mirror_num;
- if (likely(uptodate && tree->ops &&
- tree->ops->readpage_end_io_hook)) {
+
+next_block:
+ if (likely(uptodate)) {
ret = tree->ops->readpage_end_io_hook(io_bio, offset,
- page, start, end,
- mirror);
+ page, start,
+ start + root->sectorsize - 1,
+ mirror);
if (ret)
uptodate = 0;
else
clean_io_failure(start, page);
}
- if (likely(uptodate))
- goto readpage_ok;
-
- if (tree->ops && tree->ops->readpage_io_failed_hook) {
- ret = tree->ops->readpage_io_failed_hook(page, mirror);
- if (!ret && !err &&
- test_bit(BIO_UPTODATE, &bio->bi_flags))
- uptodate = 1;
- } else {
+ if (!uptodate) {
/*
* The generic bio_readpage_error handles errors the
* following way: If possible, new read requests are
@@ -2525,60 +2505,44 @@ static void end_bio_extent_readpage(struct bio *bio, int err)
* can't handle the error it will return -EIO and we
* remain responsible for that page.
*/
- ret = bio_readpage_error(bio, offset, page, start, end,
- mirror);
+ ret = bio_readpage_error(bio, offset, page,
+ start, start + root->sectorsize - 1,
+ mirror);
if (ret == 0) {
- uptodate =
- test_bit(BIO_UPTODATE, &bio->bi_flags);
+ uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags);
if (err)
uptodate = 0;
- continue;
+ offset += root->sectorsize;
+ if (--nr_sectors) {
+ start += root->sectorsize;
+ goto next_block;
+ } else {
+ continue;
+ }
}
}
-readpage_ok:
- if (likely(uptodate)) {
- loff_t i_size = i_size_read(inode);
- pgoff_t end_index = i_size >> PAGE_CACHE_SHIFT;
- unsigned offset;
-
- /* Zero out the end if this page straddles i_size */
- offset = i_size & (PAGE_CACHE_SIZE-1);
- if (page->index == end_index && offset)
- zero_user_segment(page, offset, PAGE_CACHE_SIZE);
- SetPageUptodate(page);
+
+ if (uptodate) {
+ set_extent_uptodate(tree, start,
+ start + root->sectorsize - 1,
+ &cached, GFP_ATOMIC);
+ check_page_uptodate(tree, page, cached);
} else {
ClearPageUptodate(page);
SetPageError(page);
}
- unlock_page(page);
- offset += len;
-
- if (unlikely(!uptodate)) {
- if (extent_len) {
- endio_readpage_release_extent(tree,
- extent_start,
- extent_len, 1);
- extent_start = 0;
- extent_len = 0;
- }
- endio_readpage_release_extent(tree, start,
- end - start + 1, 0);
- } else if (!extent_len) {
- extent_start = start;
- extent_len = end + 1 - start;
- } else if (extent_start + extent_len == start) {
- extent_len += end + 1 - start;
+
+ unlock_extent(tree, start, start + root->sectorsize - 1);
+ offset += root->sectorsize;
+ if (--nr_sectors) {
+ start += root->sectorsize;
+ goto next_block;
} else {
- endio_readpage_release_extent(tree, extent_start,
- extent_len, uptodate);
- extent_start = start;
- extent_len = end + 1 - start;
+ check_page_locked(tree, page);
+ continue;
}
}
- if (extent_len)
- endio_readpage_release_extent(tree, extent_start, extent_len,
- uptodate);
if (io_bio->end_io)
io_bio->end_io(io_bio, err);
bio_put(bio);
@@ -2918,7 +2882,7 @@ static int __do_readpage(struct extent_io_tree *tree,
/* the get_extent function already copied into the page */
if (test_range_bit(tree, cur, cur_end,
EXTENT_UPTODATE, 1, NULL)) {
- check_page_uptodate(tree, page);
+ check_page_uptodate(tree, page, NULL);
if (!parent_locked)
unlock_extent(tree, cur, cur + iosize - 1);
cur = cur + iosize;
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 02/17] Btrfs: subpagesize-blocksize: Get rid of whole page writes.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 01/17] Btrfs: subpagesize-blocksize: Get rid of whole page reads Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 03/17] Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release extents aligned to block size Chandan Rajendra
` (14 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
This commit brings back functions that set/clear EXTENT_WRITEBACK bits. These
are required to reliably clear PG_writeback page flag.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 47 +++++++++++++++++++++++++++--------------------
fs/btrfs/inode.c | 40 +++++++++++++++++++++++++++++++---------
2 files changed, 58 insertions(+), 29 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 5d9cc68..7229c4d 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -1300,6 +1300,20 @@ int clear_extent_uptodate(struct extent_io_tree *tree, u64 start, u64 end,
cached_state, mask);
}
+static int set_extent_writeback(struct extent_io_tree *tree, u64 start, u64 end,
+ struct extent_state **cached_state, gfp_t mask)
+{
+ return set_extent_bit(tree, start, end, EXTENT_WRITEBACK, NULL,
+ cached_state, mask);
+}
+
+static int clear_extent_writeback(struct extent_io_tree *tree, u64 start, u64 end,
+ struct extent_state **cached_state, gfp_t mask)
+{
+ return clear_extent_bit(tree, start, end, EXTENT_WRITEBACK, 1, 0,
+ cached_state, mask);
+}
+
/*
* either insert or lock state struct between start and end use mask to tell
* us if waiting is desired.
@@ -1406,6 +1420,7 @@ static int set_range_writeback(struct extent_io_tree *tree, u64 start, u64 end)
page_cache_release(page);
index++;
}
+ set_extent_writeback(tree, start, end, NULL, GFP_NOFS);
return 0;
}
@@ -2408,31 +2423,23 @@ static void end_bio_extent_writepage(struct bio *bio, int err)
bio_for_each_segment_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
+ struct inode *inode = page->mapping->host;
+ struct extent_io_tree *tree = &BTRFS_I(inode)->io_tree;
+ u64 page_start, page_end;
- /* We always issue full-page reads, but if some block
- * in a page fails to read, blk_update_request() will
- * advance bv_offset and adjust bv_len to compensate.
- * Print a warning for nonzero offsets, and an error
- * if they don't add up to a full page. */
- if (bvec->bv_offset || bvec->bv_len != PAGE_CACHE_SIZE) {
- if (bvec->bv_offset + bvec->bv_len != PAGE_CACHE_SIZE)
- btrfs_err(BTRFS_I(page->mapping->host)->root->fs_info,
- "partial page write in btrfs with offset %u and length %u",
- bvec->bv_offset, bvec->bv_len);
- else
- btrfs_info(BTRFS_I(page->mapping->host)->root->fs_info,
- "incomplete page write in btrfs with offset %u and "
- "length %u",
- bvec->bv_offset, bvec->bv_len);
- }
-
- start = page_offset(page);
- end = start + bvec->bv_offset + bvec->bv_len - 1;
+ start = page_offset(page) + bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
if (end_extent_writepage(page, err, start, end))
continue;
- end_page_writeback(page);
+ clear_extent_writeback(tree, start, end, NULL, GFP_ATOMIC);
+
+ page_start = page_offset(page);
+ page_end = page_offset(page) + PAGE_CACHE_SIZE - 1;
+ if (!test_range_bit(tree, page_start, page_end,
+ EXTENT_WRITEBACK, 0, NULL))
+ end_page_writeback(page);
}
bio_put(bio);
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 7309832..2ffb4df 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -2823,22 +2823,44 @@ static int btrfs_writepage_end_io_hook(struct page *page, u64 start, u64 end,
struct btrfs_root *root = BTRFS_I(inode)->root;
struct btrfs_ordered_extent *ordered_extent = NULL;
struct btrfs_workqueue *workers;
+ u64 ordered_start, ordered_end;
+ int done;
trace_btrfs_writepage_end_io_hook(page, start, end, uptodate);
ClearPagePrivate2(page);
- if (!btrfs_dec_test_ordered_pending(inode, &ordered_extent, start,
- end - start + 1, uptodate))
- return 0;
+loop:
+ ordered_extent = btrfs_lookup_ordered_range(inode, start,
+ end - start + 1);
+ if (!ordered_extent)
+ goto out;
- btrfs_init_work(&ordered_extent->work, finish_ordered_fn, NULL, NULL);
+ ordered_start = max_t(u64, start, ordered_extent->file_offset);
+ ordered_end = min_t(u64, end,
+ ordered_extent->file_offset + ordered_extent->len - 1);
- if (btrfs_is_free_space_inode(inode))
- workers = root->fs_info->endio_freespace_worker;
- else
- workers = root->fs_info->endio_write_workers;
- btrfs_queue_work(workers, &ordered_extent->work);
+ done = btrfs_dec_test_ordered_pending(inode, &ordered_extent,
+ ordered_start,
+ ordered_end - ordered_start + 1,
+ uptodate);
+ if (done) {
+ btrfs_init_work(&ordered_extent->work, finish_ordered_fn, NULL, NULL);
+
+ if (btrfs_is_free_space_inode(inode))
+ workers = root->fs_info->endio_freespace_worker;
+ else
+ workers = root->fs_info->endio_write_workers;
+ btrfs_queue_work(workers, &ordered_extent->work);
+ }
+
+ btrfs_put_ordered_extent(ordered_extent);
+
+ start = ordered_end + 1;
+
+ if (start < end)
+ goto loop;
+out:
return 0;
}
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 03/17] Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release extents aligned to block size.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 01/17] Btrfs: subpagesize-blocksize: Get rid of whole page reads Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 02/17] Btrfs: subpagesize-blocksize: Get rid of whole page writes Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 04/17] Btrfs: subpagesize-blocksize: Define extent_buffer_head Chandan Rajendra
` (13 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
Currently, the code reserves/releases extents in multiples of PAGE_CACHE_SIZE
units. Fix this.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/file.c | 32 ++++++++++++++++++++------------
1 file changed, 20 insertions(+), 12 deletions(-)
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index d3afac2..444819d 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -1366,18 +1366,21 @@ fail:
static noinline int
lock_and_cleanup_extent_if_need(struct inode *inode, struct page **pages,
size_t num_pages, loff_t pos,
+ size_t write_bytes,
u64 *lockstart, u64 *lockend,
struct extent_state **cached_state)
{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
u64 start_pos;
u64 last_pos;
int i;
int ret = 0;
- start_pos = pos & ~((u64)PAGE_CACHE_SIZE - 1);
- last_pos = start_pos + ((u64)num_pages << PAGE_CACHE_SHIFT) - 1;
+ start_pos = pos & ~((u64)root->sectorsize - 1);
+ last_pos = start_pos
+ + ALIGN(pos + write_bytes - start_pos, root->sectorsize) - 1;
- if (start_pos < inode->i_size) {
+ if (start_pos < inode->i_size) {
struct btrfs_ordered_extent *ordered;
lock_extent_bits(&BTRFS_I(inode)->io_tree,
start_pos, last_pos, 0, cached_state);
@@ -1494,6 +1497,7 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file,
while (iov_iter_count(i) > 0) {
size_t offset = pos & (PAGE_CACHE_SIZE - 1);
+ size_t sector_offset;
size_t write_bytes = min(iov_iter_count(i),
nrptrs * (size_t)PAGE_CACHE_SIZE -
offset);
@@ -1514,7 +1518,9 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file,
break;
}
- reserve_bytes = num_pages << PAGE_CACHE_SHIFT;
+ sector_offset = pos & (root->sectorsize - 1);
+ reserve_bytes = ALIGN(write_bytes + sector_offset, root->sectorsize);
+
ret = btrfs_check_data_free_space(inode, reserve_bytes);
if (ret == -ENOSPC &&
(BTRFS_I(inode)->flags & (BTRFS_INODE_NODATACOW |
@@ -1529,7 +1535,9 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file,
num_pages = (write_bytes + offset +
PAGE_CACHE_SIZE - 1) >>
PAGE_CACHE_SHIFT;
- reserve_bytes = num_pages << PAGE_CACHE_SHIFT;
+
+ reserve_bytes = ALIGN(write_bytes + sector_offset,
+ root->sectorsize);
ret = 0;
} else {
ret = -ENOSPC;
@@ -1564,8 +1572,8 @@ again:
break;
ret = lock_and_cleanup_extent_if_need(inode, pages, num_pages,
- pos, &lockstart, &lockend,
- &cached_state);
+ pos, write_bytes, &lockstart, &lockend,
+ &cached_state);
if (ret < 0) {
if (ret == -EAGAIN)
goto again;
@@ -1602,9 +1610,9 @@ again:
* we still have an outstanding extent for the chunk we actually
* managed to copy.
*/
- if (num_pages > dirty_pages) {
- release_bytes = (num_pages - dirty_pages) <<
- PAGE_CACHE_SHIFT;
+ if (write_bytes > copied) {
+ release_bytes = (write_bytes - copied)
+ & ~((u64)root->sectorsize - 1);
if (copied > 0) {
spin_lock(&BTRFS_I(inode)->lock);
BTRFS_I(inode)->outstanding_extents++;
@@ -1618,7 +1626,7 @@ again:
release_bytes);
}
- release_bytes = dirty_pages << PAGE_CACHE_SHIFT;
+ release_bytes = ALIGN(copied + sector_offset, root->sectorsize);
if (copied > 0)
ret = btrfs_dirty_pages(root, inode, pages,
@@ -1640,7 +1648,7 @@ again:
if (only_release_metadata && copied > 0) {
u64 lockstart = round_down(pos, root->sectorsize);
u64 lockend = lockstart +
- (dirty_pages << PAGE_CACHE_SHIFT) - 1;
+ ALIGN(copied, root->sectorsize) - 1;
set_extent_bit(&BTRFS_I(inode)->io_tree, lockstart,
lockend, EXTENT_NORESERVE, NULL,
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 04/17] Btrfs: subpagesize-blocksize: Define extent_buffer_head.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (2 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 03/17] Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release extents aligned to block size Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 05/17] Btrfs: subpagesize-blocksize: Read tree blocks whose size is <PAGE_CACHE_SIZE Chandan Rajendra
` (12 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandra Seetharaman, aneesh.kumar, linux-btrfs, chandan,
steve.capper, Chandan Rajendra
From: Chandra Seetharaman <sekharan@us.ibm.com>
In order to handle multiple extent buffers per page, first we need to create a
way to handle all the extent buffers that are attached to a page.
This patch creates a new data structure 'struct extent_buffer_head', and moves
fields that are common to all extent buffers in a page from 'struct extent
buffer' to 'struct extent_buffer_head'
Also, this patch moves EXTENT_BUFFER_TREE_REF, EXTENT_BUFFER_DUMMY and
EXTENT_BUFFER_IN_TREE flags from extent_buffer->ebflags to
extent_buffer_head->bflags.
Signed-off-by: Chandra Seetharaman <sekharan@us.ibm.com>
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/backref.c | 2 +-
fs/btrfs/ctree.c | 2 +-
fs/btrfs/ctree.h | 6 +-
fs/btrfs/disk-io.c | 46 ++++--
fs/btrfs/extent-tree.c | 6 +-
fs/btrfs/extent_io.c | 373 +++++++++++++++++++++++++++++--------------
fs/btrfs/extent_io.h | 47 ++++--
fs/btrfs/volumes.c | 2 +-
include/trace/events/btrfs.h | 2 +-
9 files changed, 328 insertions(+), 158 deletions(-)
diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c
index 54a201d..1d3d5d6 100644
--- a/fs/btrfs/backref.c
+++ b/fs/btrfs/backref.c
@@ -1305,7 +1305,7 @@ char *btrfs_ref_to_path(struct btrfs_root *fs_root, struct btrfs_path *path,
eb = path->nodes[0];
/* make sure we can use eb after releasing the path */
if (eb != eb_in) {
- atomic_inc(&eb->refs);
+ atomic_inc(&eb_head(eb)->refs);
btrfs_tree_read_lock(eb);
btrfs_set_lock_blocking_rw(eb, BTRFS_READ_LOCK);
}
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 44ee5d2..693b541 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -169,7 +169,7 @@ struct extent_buffer *btrfs_root_node(struct btrfs_root *root)
* the inc_not_zero dance and if it doesn't work then
* synchronize_rcu and try again.
*/
- if (atomic_inc_not_zero(&eb->refs)) {
+ if (atomic_inc_not_zero(&eb_head(eb)->refs)) {
rcu_read_unlock();
break;
}
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 8e29b61..5b7b7ca 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2215,14 +2215,16 @@ static inline void btrfs_set_token_##name(struct extent_buffer *eb, \
#define BTRFS_SETGET_HEADER_FUNCS(name, type, member, bits) \
static inline u##bits btrfs_##name(struct extent_buffer *eb) \
{ \
- type *p = page_address(eb->pages[0]); \
+ type *p = page_address(eb_head(eb)->pages[0]) + \
+ (eb->start & (PAGE_CACHE_SIZE -1)); \
u##bits res = le##bits##_to_cpu(p->member); \
return res; \
} \
static inline void btrfs_set_##name(struct extent_buffer *eb, \
u##bits val) \
{ \
- type *p = page_address(eb->pages[0]); \
+ type *p = page_address(eb_head(eb)->pages[0]) + \
+ (eb->start & (PAGE_CACHE_SIZE -1)); \
p->member = cpu_to_le##bits(val); \
}
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index d0ed9e6..3a79833 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1030,13 +1030,21 @@ static int btree_set_page_dirty(struct page *page)
{
#ifdef DEBUG
struct extent_buffer *eb;
+ int i, dirty = 0;
BUG_ON(!PagePrivate(page));
eb = (struct extent_buffer *)page->private;
BUG_ON(!eb);
- BUG_ON(!test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags));
- BUG_ON(!atomic_read(&eb->refs));
- btrfs_assert_tree_locked(eb);
+
+ do {
+ dirty = test_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags);
+ if (dirty)
+ break;
+ } while ((eb = eb->eb_next) != NULL);
+
+ BUG_ON(!dirty);
+ BUG_ON(!atomic_read(&(eb_head(eb)->refs)));
+ btrfs_assert_tree_locked(&ebh->eb);
#endif
return __set_page_dirty_nobuffers(page);
}
@@ -1080,7 +1088,7 @@ int reada_tree_block_flagged(struct btrfs_root *root, u64 bytenr, u32 blocksize,
if (!buf)
return 0;
- set_bit(EXTENT_BUFFER_READAHEAD, &buf->bflags);
+ set_bit(EXTENT_BUFFER_READAHEAD, &buf->ebflags);
ret = read_extent_buffer_pages(io_tree, buf, 0, WAIT_PAGE_LOCK,
btree_get_extent, mirror_num);
@@ -1089,7 +1097,7 @@ int reada_tree_block_flagged(struct btrfs_root *root, u64 bytenr, u32 blocksize,
return ret;
}
- if (test_bit(EXTENT_BUFFER_CORRUPT, &buf->bflags)) {
+ if (test_bit(EXTENT_BUFFER_CORRUPT, &buf->ebflags)) {
free_extent_buffer(buf);
return -EIO;
} else if (extent_buffer_uptodate(buf)) {
@@ -1120,14 +1128,16 @@ struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_root *root,
int btrfs_write_tree_block(struct extent_buffer *buf)
{
- return filemap_fdatawrite_range(buf->pages[0]->mapping, buf->start,
+ return filemap_fdatawrite_range(eb_head(buf)->pages[0]->mapping,
+ buf->start,
buf->start + buf->len - 1);
}
int btrfs_wait_tree_block_writeback(struct extent_buffer *buf)
{
- return filemap_fdatawait_range(buf->pages[0]->mapping,
- buf->start, buf->start + buf->len - 1);
+ return filemap_fdatawait_range(eb_head(buf)->pages[0]->mapping,
+ buf->start,
+ buf->start + buf->len - 1);
}
struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
@@ -1158,7 +1168,8 @@ void clean_tree_block(struct btrfs_trans_handle *trans, struct btrfs_root *root,
fs_info->running_transaction->transid) {
btrfs_assert_tree_locked(buf);
- if (test_and_clear_bit(EXTENT_BUFFER_DIRTY, &buf->bflags)) {
+ if (test_and_clear_bit(EXTENT_BUFFER_DIRTY,
+ &buf->ebflags)) {
__percpu_counter_add(&fs_info->dirty_metadata_bytes,
-buf->len,
fs_info->dirty_metadata_batch);
@@ -2648,7 +2659,8 @@ int open_ctree(struct super_block *sb,
btrfs_super_chunk_root(disk_super),
blocksize, generation);
if (!chunk_root->node ||
- !test_bit(EXTENT_BUFFER_UPTODATE, &chunk_root->node->bflags)) {
+ !test_bit(EXTENT_BUFFER_UPTODATE,
+ &chunk_root->node->ebflags)) {
printk(KERN_WARNING "BTRFS: failed to read chunk root on %s\n",
sb->s_id);
goto fail_tree_roots;
@@ -2687,7 +2699,8 @@ retry_root_backup:
btrfs_super_root(disk_super),
blocksize, generation);
if (!tree_root->node ||
- !test_bit(EXTENT_BUFFER_UPTODATE, &tree_root->node->bflags)) {
+ !test_bit(EXTENT_BUFFER_UPTODATE,
+ &tree_root->node->ebflags)) {
printk(KERN_WARNING "BTRFS: failed to read tree root on %s\n",
sb->s_id);
@@ -3713,7 +3726,7 @@ int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
int atomic)
{
int ret;
- struct inode *btree_inode = buf->pages[0]->mapping->host;
+ struct inode *btree_inode = eb_head(buf)->pages[0]->mapping->host;
ret = extent_buffer_uptodate(buf);
if (!ret)
@@ -3743,10 +3756,10 @@ void btrfs_mark_buffer_dirty(struct extent_buffer *buf)
* enabled. Normal people shouldn't be marking dummy buffers as dirty
* outside of the sanity tests.
*/
- if (unlikely(test_bit(EXTENT_BUFFER_DUMMY, &buf->bflags)))
+ if (unlikely(test_bit(EXTENT_BUFFER_DUMMY, &eb_head(buf)->bflags)))
return;
#endif
- root = BTRFS_I(buf->pages[0]->mapping->host)->root;
+ root = BTRFS_I(eb_head(buf)->pages[0]->mapping->host)->root;
btrfs_assert_tree_locked(buf);
if (transid != root->fs_info->generation)
WARN(1, KERN_CRIT "btrfs transid mismatch buffer %llu, "
@@ -3801,7 +3814,8 @@ void btrfs_btree_balance_dirty_nodelay(struct btrfs_root *root)
int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid)
{
- struct btrfs_root *root = BTRFS_I(buf->pages[0]->mapping->host)->root;
+ struct btrfs_root *root =
+ BTRFS_I(eb_head(buf)->pages[0]->mapping->host)->root;
return btree_read_extent_buffer_pages(root, buf, 0, parent_transid);
}
@@ -4011,7 +4025,7 @@ static int btrfs_destroy_marked_extents(struct btrfs_root *root,
wait_on_extent_buffer_writeback(eb);
if (test_and_clear_bit(EXTENT_BUFFER_DIRTY,
- &eb->bflags))
+ &eb->ebflags))
clear_extent_buffer_dirty(eb);
free_extent_buffer_stale(eb);
}
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 102ed31..fbcad82 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -6209,7 +6209,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
goto out;
}
- WARN_ON(test_bit(EXTENT_BUFFER_DIRTY, &buf->bflags));
+ WARN_ON(test_bit(EXTENT_BUFFER_DIRTY, &buf->ebflags));
btrfs_add_free_space(cache, buf->start, buf->len);
btrfs_update_reserved_bytes(cache, buf->len, RESERVE_FREE, 0);
@@ -6226,7 +6226,7 @@ out:
* Deleting the buffer, clear the corrupt flag since it doesn't matter
* anymore.
*/
- clear_bit(EXTENT_BUFFER_CORRUPT, &buf->bflags);
+ clear_bit(EXTENT_BUFFER_CORRUPT, &buf->ebflags);
btrfs_put_block_group(cache);
}
@@ -7212,7 +7212,7 @@ btrfs_init_new_buffer(struct btrfs_trans_handle *trans, struct btrfs_root *root,
btrfs_set_buffer_lockdep_class(root->root_key.objectid, buf, level);
btrfs_tree_lock(buf);
clean_tree_block(trans, root, buf);
- clear_bit(EXTENT_BUFFER_STALE, &buf->bflags);
+ clear_bit(EXTENT_BUFFER_STALE, &buf->ebflags);
btrfs_set_lock_blocking(buf);
btrfs_set_buffer_uptodate(buf);
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 7229c4d..7a923b7 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -56,6 +56,7 @@ void btrfs_leak_debug_check(void)
{
struct extent_state *state;
struct extent_buffer *eb;
+ struct extent_buffer_head *ebh;
while (!list_empty(&states)) {
state = list_entry(states.next, struct extent_state, leak_list);
@@ -68,12 +69,17 @@ void btrfs_leak_debug_check(void)
}
while (!list_empty(&buffers)) {
- eb = list_entry(buffers.next, struct extent_buffer, leak_list);
- printk(KERN_ERR "BTRFS: buffer leak start %llu len %lu "
- "refs %d\n",
- eb->start, eb->len, atomic_read(&eb->refs));
- list_del(&eb->leak_list);
- kmem_cache_free(extent_buffer_cache, eb);
+ ebh = list_entry(buffers.next, struct extent_buffer_head, leak_list);
+ printk(KERN_ERR "btrfs buffer leak ");
+
+ eb = &ebh->eb;
+ do {
+ printk(KERN_ERR "eb %p %llu:%lu ", eb, eb->start, eb->len);
+ } while ((eb = eb->eb_next) != NULL);
+
+ printk(KERN_ERR "refs %d\n", atomic_read(&ebh->refs));
+ list_del(&ebh->leak_list);
+ kmem_cache_free(extent_buffer_cache, ebh);
}
}
@@ -144,7 +150,7 @@ int __init extent_io_init(void)
return -ENOMEM;
extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer",
- sizeof(struct extent_buffer), 0,
+ sizeof(struct extent_buffer_head), 0,
SLAB_RECLAIM_ACCOUNT | SLAB_MEM_SPREAD, NULL);
if (!extent_buffer_cache)
goto free_state_cache;
@@ -3416,7 +3422,7 @@ static int eb_wait(void *word)
void wait_on_extent_buffer_writeback(struct extent_buffer *eb)
{
- wait_on_bit(&eb->bflags, EXTENT_BUFFER_WRITEBACK, eb_wait,
+ wait_on_bit(&eb->ebflags, EXTENT_BUFFER_WRITEBACK, eb_wait,
TASK_UNINTERRUPTIBLE);
}
@@ -4341,29 +4347,47 @@ out:
return ret;
}
-static void __free_extent_buffer(struct extent_buffer *eb)
+static void __free_extent_buffer(struct extent_buffer_head *ebh)
{
- btrfs_leak_debug_del(&eb->leak_list);
- kmem_cache_free(extent_buffer_cache, eb);
+ struct extent_buffer *eb, *next_eb;
+
+ btrfs_leak_debug_del(&ebh->leak_list);
+
+ eb = ebh->eb.eb_next;
+ while (eb) {
+ next_eb = eb->eb_next;
+ kfree(eb);
+ eb = next_eb;
+ }
+
+ kmem_cache_free(extent_buffer_cache, ebh);
}
int extent_buffer_under_io(struct extent_buffer *eb)
{
- return (atomic_read(&eb->io_pages) ||
- test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags) ||
- test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags));
+ struct extent_buffer_head *ebh = eb->ebh;
+ int dirty_or_writeback = 0;
+
+ for (eb = &ebh->eb; eb; eb = eb->eb_next) {
+ if (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags)
+ || test_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags))
+ dirty_or_writeback = 1;
+ }
+
+ return (atomic_read(&ebh->io_bvecs) || dirty_or_writeback);
}
/*
* Helper for releasing extent buffer page.
*/
static void btrfs_release_extent_buffer_page(struct extent_buffer *eb,
- unsigned long start_idx)
+ unsigned long start_idx)
{
unsigned long index;
unsigned long num_pages;
struct page *page;
- int mapped = !test_bit(EXTENT_BUFFER_DUMMY, &eb->bflags);
+ struct extent_buffer_head *ebh = eb_head(eb);
+ int mapped = !test_bit(EXTENT_BUFFER_DUMMY, &ebh->bflags);
BUG_ON(extent_buffer_under_io(eb));
@@ -4373,6 +4397,8 @@ static void btrfs_release_extent_buffer_page(struct extent_buffer *eb,
return;
do {
+ struct extent_buffer *e;
+
index--;
page = extent_buffer_page(eb, index);
if (page && mapped) {
@@ -4385,8 +4411,10 @@ static void btrfs_release_extent_buffer_page(struct extent_buffer *eb,
* this eb.
*/
if (PagePrivate(page) &&
- page->private == (unsigned long)eb) {
- BUG_ON(test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags));
+ page->private == (unsigned long)(&ebh->eb)) {
+ for (e = &ebh->eb; !e; e = e->eb_next)
+ BUG_ON(test_bit(EXTENT_BUFFER_DIRTY,
+ &e->ebflags));
BUG_ON(PageDirty(page));
BUG_ON(PageWriteback(page));
/*
@@ -4414,22 +4442,18 @@ static void btrfs_release_extent_buffer_page(struct extent_buffer *eb,
static inline void btrfs_release_extent_buffer(struct extent_buffer *eb)
{
btrfs_release_extent_buffer_page(eb, 0);
- __free_extent_buffer(eb);
+ __free_extent_buffer(eb_head(eb));
}
-static struct extent_buffer *
-__alloc_extent_buffer(struct btrfs_fs_info *fs_info, u64 start,
- unsigned long len, gfp_t mask)
+static void __init_extent_buffer(struct extent_buffer *eb,
+ struct extent_buffer_head *ebh,
+ u64 start,
+ unsigned long len)
{
- struct extent_buffer *eb = NULL;
-
- eb = kmem_cache_zalloc(extent_buffer_cache, mask);
- if (eb == NULL)
- return NULL;
eb->start = start;
eb->len = len;
- eb->fs_info = fs_info;
- eb->bflags = 0;
+ eb->ebh = ebh;
+ eb->eb_next = NULL;
rwlock_init(&eb->lock);
atomic_set(&eb->write_locks, 0);
atomic_set(&eb->read_locks, 0);
@@ -4440,12 +4464,26 @@ __alloc_extent_buffer(struct btrfs_fs_info *fs_info, u64 start,
eb->lock_nested = 0;
init_waitqueue_head(&eb->write_lock_wq);
init_waitqueue_head(&eb->read_lock_wq);
+}
+
+static struct extent_buffer *__alloc_extent_buffer(struct btrfs_fs_info *fs_info,
+ u64 start, unsigned long len,
+ gfp_t mask)
+{
+ struct extent_buffer_head *ebh = NULL;
+ struct extent_buffer *eb = NULL;
+ int i;
- btrfs_leak_debug_add(&eb->leak_list, &buffers);
+ ebh = kmem_cache_zalloc(extent_buffer_cache, mask);
+ if (ebh == NULL)
+ return NULL;
+ ebh->fs_info = fs_info;
+ ebh->bflags = 0;
+ btrfs_leak_debug_add(&ebh->leak_list, &buffers);
- spin_lock_init(&eb->refs_lock);
- atomic_set(&eb->refs, 1);
- atomic_set(&eb->io_pages, 0);
+ spin_lock_init(&ebh->refs_lock);
+ atomic_set(&ebh->refs, 1);
+ atomic_set(&ebh->io_bvecs, 0);
/*
* Sanity checks, currently the maximum is 64k covered by 16x 4k pages
@@ -4454,6 +4492,29 @@ __alloc_extent_buffer(struct btrfs_fs_info *fs_info, u64 start,
> MAX_INLINE_EXTENT_BUFFER_SIZE);
BUG_ON(len > MAX_INLINE_EXTENT_BUFFER_SIZE);
+ if (len < PAGE_CACHE_SIZE) {
+ struct extent_buffer *cur_eb, *prev_eb;
+ int ebs_per_page = PAGE_CACHE_SIZE / len;
+ u64 st = start & ~(PAGE_CACHE_SIZE - 1);
+
+ prev_eb = NULL;
+ cur_eb = &ebh->eb;
+ for (i = 0; i < ebs_per_page; i++, st += len) {
+ if (prev_eb) {
+ cur_eb = kzalloc(sizeof(*eb), mask);
+ prev_eb->eb_next = cur_eb;
+ }
+ __init_extent_buffer(cur_eb, ebh, st, len);
+ prev_eb = cur_eb;
+ if (st == start)
+ eb = cur_eb;
+ }
+ BUG_ON(!eb);
+ } else {
+ eb = &ebh->eb;
+ __init_extent_buffer(eb, ebh, start, len);
+ }
+
return eb;
}
@@ -4474,15 +4535,16 @@ struct extent_buffer *btrfs_clone_extent_buffer(struct extent_buffer *src)
btrfs_release_extent_buffer(new);
return NULL;
}
- attach_extent_buffer_page(new, p);
+ attach_extent_buffer_page(&(eb_head(new)->eb), p);
WARN_ON(PageDirty(p));
SetPageUptodate(p);
- new->pages[i] = p;
+ eb_head(new)->pages[i] = p;
}
+ set_bit(EXTENT_BUFFER_UPTODATE, &new->ebflags);
+ set_bit(EXTENT_BUFFER_DUMMY, &eb_head(new)->bflags);
+
copy_extent_buffer(new, src, 0, 0, src->len);
- set_bit(EXTENT_BUFFER_UPTODATE, &new->bflags);
- set_bit(EXTENT_BUFFER_DUMMY, &new->bflags);
return new;
}
@@ -4498,19 +4560,19 @@ struct extent_buffer *alloc_dummy_extent_buffer(u64 start, unsigned long len)
return NULL;
for (i = 0; i < num_pages; i++) {
- eb->pages[i] = alloc_page(GFP_NOFS);
- if (!eb->pages[i])
+ eb_head(eb)->pages[i] = alloc_page(GFP_NOFS);
+ if (!eb_head(eb)->pages[i])
goto err;
}
set_extent_buffer_uptodate(eb);
btrfs_set_header_nritems(eb, 0);
- set_bit(EXTENT_BUFFER_DUMMY, &eb->bflags);
+ set_bit(EXTENT_BUFFER_DUMMY, &eb_head(eb)->bflags);
return eb;
err:
for (; i > 0; i--)
- __free_page(eb->pages[i - 1]);
- __free_extent_buffer(eb);
+ __free_page(eb_head(eb)->pages[i - 1]);
+ __free_extent_buffer(eb_head(eb));
return NULL;
}
@@ -4537,14 +4599,15 @@ static void check_buffer_tree_ref(struct extent_buffer *eb)
* So bump the ref count first, then set the bit. If someone
* beat us to it, drop the ref we added.
*/
- refs = atomic_read(&eb->refs);
- if (refs >= 2 && test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags))
+ refs = atomic_read(&eb_head(eb)->refs);
+ if (refs >= 2 && test_bit(EXTENT_BUFFER_TREE_REF,
+ &eb_head(eb)->bflags))
return;
- spin_lock(&eb->refs_lock);
- if (!test_and_set_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags))
- atomic_inc(&eb->refs);
- spin_unlock(&eb->refs_lock);
+ spin_lock(&eb_head(eb)->refs_lock);
+ if (!test_and_set_bit(EXTENT_BUFFER_TREE_REF, &eb_head(eb)->bflags))
+ atomic_inc(&eb_head(eb)->refs);
+ spin_unlock(&eb_head(eb)->refs_lock);
}
static void mark_extent_buffer_accessed(struct extent_buffer *eb,
@@ -4565,15 +4628,24 @@ static void mark_extent_buffer_accessed(struct extent_buffer *eb,
struct extent_buffer *find_extent_buffer(struct btrfs_fs_info *fs_info,
u64 start)
{
+ struct extent_buffer_head *ebh;
struct extent_buffer *eb;
rcu_read_lock();
- eb = radix_tree_lookup(&fs_info->buffer_radix,
- start >> PAGE_CACHE_SHIFT);
- if (eb && atomic_inc_not_zero(&eb->refs)) {
+ ebh = radix_tree_lookup(&fs_info->buffer_radix,
+ start >> PAGE_CACHE_SHIFT);
+ if (ebh && atomic_inc_not_zero(&ebh->refs)) {
rcu_read_unlock();
- mark_extent_buffer_accessed(eb, NULL);
- return eb;
+
+ eb = &ebh->eb;
+ do {
+ if (eb->start == start) {
+ mark_extent_buffer_accessed(eb, NULL);
+ return eb;
+ }
+ } while ((eb = eb->eb_next) != NULL);
+
+ BUG();
}
rcu_read_unlock();
@@ -4633,7 +4705,7 @@ struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info,
unsigned long num_pages = num_extent_pages(start, len);
unsigned long i;
unsigned long index = start >> PAGE_CACHE_SHIFT;
- struct extent_buffer *eb;
+ struct extent_buffer *eb, *cur_eb;
struct extent_buffer *exists = NULL;
struct page *p;
struct address_space *mapping = fs_info->btree_inode->i_mapping;
@@ -4663,12 +4735,18 @@ struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info,
* overwrite page->private.
*/
exists = (struct extent_buffer *)p->private;
- if (atomic_inc_not_zero(&exists->refs)) {
+ if (atomic_inc_not_zero(&eb_head(exists)->refs)) {
spin_unlock(&mapping->private_lock);
unlock_page(p);
page_cache_release(p);
- mark_extent_buffer_accessed(exists, p);
- goto free_eb;
+ do {
+ if (exists->start == start) {
+ mark_extent_buffer_accessed(exists, p);
+ goto free_eb;
+ }
+ } while ((exists = exists->eb_next) != NULL);
+
+ BUG();
}
/*
@@ -4679,10 +4757,11 @@ struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info,
WARN_ON(PageDirty(p));
page_cache_release(p);
}
- attach_extent_buffer_page(eb, p);
+ attach_extent_buffer_page(&(eb_head(eb)->eb), p);
spin_unlock(&mapping->private_lock);
WARN_ON(PageDirty(p));
- eb->pages[i] = p;
+ mark_page_accessed(p);
+ eb_head(eb)->pages[i] = p;
if (!PageUptodate(p))
uptodate = 0;
@@ -4691,16 +4770,22 @@ struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info,
* and why we unlock later
*/
}
- if (uptodate)
- set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags);
+ if (uptodate) {
+ cur_eb = &(eb_head(eb)->eb);
+ do {
+ set_bit(EXTENT_BUFFER_UPTODATE, &cur_eb->ebflags);
+ } while ((cur_eb = cur_eb->eb_next) != NULL);
+ }
again:
ret = radix_tree_preload(GFP_NOFS & ~__GFP_HIGHMEM);
- if (ret)
+ if (ret) {
+ exists = NULL;
goto free_eb;
+ }
spin_lock(&fs_info->buffer_lock);
ret = radix_tree_insert(&fs_info->buffer_radix,
- start >> PAGE_CACHE_SHIFT, eb);
+ start >> PAGE_CACHE_SHIFT, eb_head(eb));
spin_unlock(&fs_info->buffer_lock);
radix_tree_preload_end();
if (ret == -EEXIST) {
@@ -4712,7 +4797,7 @@ again:
}
/* add one reference for the tree */
check_buffer_tree_ref(eb);
- set_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags);
+ set_bit(EXTENT_BUFFER_IN_TREE, &eb_head(eb)->bflags);
/*
* there is a race where release page may have
@@ -4723,108 +4808,125 @@ again:
* after the extent buffer is in the radix tree so
* it doesn't get lost
*/
- SetPageChecked(eb->pages[0]);
+ SetPageChecked(eb_head(eb)->pages[0]);
for (i = 1; i < num_pages; i++) {
p = extent_buffer_page(eb, i);
ClearPageChecked(p);
unlock_page(p);
}
- unlock_page(eb->pages[0]);
+ unlock_page(eb_head(eb)->pages[0]);
return eb;
free_eb:
for (i = 0; i < num_pages; i++) {
- if (eb->pages[i])
- unlock_page(eb->pages[i]);
+ if (eb_head(eb)->pages[i])
+ unlock_page(eb_head(eb)->pages[i]);
}
- WARN_ON(!atomic_dec_and_test(&eb->refs));
+ WARN_ON(!atomic_dec_and_test(&eb_head(eb)->refs));
btrfs_release_extent_buffer(eb);
return exists;
}
static inline void btrfs_release_extent_buffer_rcu(struct rcu_head *head)
{
- struct extent_buffer *eb =
- container_of(head, struct extent_buffer, rcu_head);
+ struct extent_buffer_head *ebh =
+ container_of(head, struct extent_buffer_head, rcu_head);
- __free_extent_buffer(eb);
+ __free_extent_buffer(ebh);
}
/* Expects to have eb->eb_lock already held */
-static int release_extent_buffer(struct extent_buffer *eb)
+static int release_extent_buffer(struct extent_buffer_head *ebh)
{
- WARN_ON(atomic_read(&eb->refs) == 0);
- if (atomic_dec_and_test(&eb->refs)) {
- if (test_and_clear_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags)) {
- struct btrfs_fs_info *fs_info = eb->fs_info;
+ WARN_ON(atomic_read(&ebh->refs) == 0);
+ if (atomic_dec_and_test(&ebh->refs)) {
+ if (test_and_clear_bit(EXTENT_BUFFER_IN_TREE, &ebh->bflags)) {
+ struct btrfs_fs_info *fs_info = ebh->fs_info;
- spin_unlock(&eb->refs_lock);
+ spin_unlock(&ebh->refs_lock);
spin_lock(&fs_info->buffer_lock);
radix_tree_delete(&fs_info->buffer_radix,
- eb->start >> PAGE_CACHE_SHIFT);
+ ebh->eb.start >> PAGE_CACHE_SHIFT);
spin_unlock(&fs_info->buffer_lock);
} else {
- spin_unlock(&eb->refs_lock);
+ spin_unlock(&ebh->refs_lock);
}
/* Should be safe to release our pages at this point */
- btrfs_release_extent_buffer_page(eb, 0);
- call_rcu(&eb->rcu_head, btrfs_release_extent_buffer_rcu);
+ btrfs_release_extent_buffer_page(&ebh->eb, 0);
+ call_rcu(&ebh->rcu_head, btrfs_release_extent_buffer_rcu);
return 1;
}
- spin_unlock(&eb->refs_lock);
+ spin_unlock(&ebh->refs_lock);
return 0;
}
void free_extent_buffer(struct extent_buffer *eb)
{
+ struct extent_buffer_head *ebh;
int refs;
int old;
if (!eb)
return;
+ ebh = eb_head(eb);
while (1) {
- refs = atomic_read(&eb->refs);
+ refs = atomic_read(&ebh->refs);
if (refs <= 3)
break;
- old = atomic_cmpxchg(&eb->refs, refs, refs - 1);
+ old = atomic_cmpxchg(&ebh->refs, refs, refs - 1);
if (old == refs)
return;
}
- spin_lock(&eb->refs_lock);
- if (atomic_read(&eb->refs) == 2 &&
- test_bit(EXTENT_BUFFER_DUMMY, &eb->bflags))
- atomic_dec(&eb->refs);
+ spin_lock(&ebh->refs_lock);
+ if (atomic_read(&ebh->refs) == 2 &&
+ test_bit(EXTENT_BUFFER_DUMMY, &ebh->bflags))
+ atomic_dec(&ebh->refs);
- if (atomic_read(&eb->refs) == 2 &&
- test_bit(EXTENT_BUFFER_STALE, &eb->bflags) &&
+ if (atomic_read(&ebh->refs) == 2 &&
+ test_bit(EXTENT_BUFFER_STALE, &eb->ebflags) &&
!extent_buffer_under_io(eb) &&
- test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags))
- atomic_dec(&eb->refs);
+ test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &ebh->bflags))
+ atomic_dec(&ebh->refs);
/*
* I know this is terrible, but it's temporary until we stop tracking
* the uptodate bits and such for the extent buffers.
*/
- release_extent_buffer(eb);
+ release_extent_buffer(ebh);
}
void free_extent_buffer_stale(struct extent_buffer *eb)
{
+ struct extent_buffer_head *ebh;
if (!eb)
return;
- spin_lock(&eb->refs_lock);
- set_bit(EXTENT_BUFFER_STALE, &eb->bflags);
+ ebh = eb_head(eb);
+ spin_lock(&ebh->refs_lock);
- if (atomic_read(&eb->refs) == 2 && !extent_buffer_under_io(eb) &&
- test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags))
- atomic_dec(&eb->refs);
- release_extent_buffer(eb);
+ set_bit(EXTENT_BUFFER_STALE, &eb->ebflags);
+ if (atomic_read(&ebh->refs) == 2 && !extent_buffer_under_io(eb) &&
+ test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &ebh->bflags))
+ atomic_dec(&ebh->refs);
+
+ release_extent_buffer(ebh);
+}
+
+static int page_ebs_clean(struct extent_buffer_head *ebh)
+{
+ struct extent_buffer *eb = &ebh->eb;;
+
+ do {
+ if (test_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags))
+ return 0;
+ } while ((eb = eb->eb_next) != NULL);
+
+ return 1;
}
void clear_extent_buffer_dirty(struct extent_buffer *eb)
@@ -4835,6 +4937,9 @@ void clear_extent_buffer_dirty(struct extent_buffer *eb)
num_pages = num_extent_pages(eb->start, eb->len);
+ if (eb->len < PAGE_CACHE_SIZE && !page_ebs_clean(eb_head(eb)))
+ return;
+
for (i = 0; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
if (!PageDirty(page))
@@ -4854,7 +4959,7 @@ void clear_extent_buffer_dirty(struct extent_buffer *eb)
ClearPageError(page);
unlock_page(page);
}
- WARN_ON(atomic_read(&eb->refs) == 0);
+ WARN_ON(atomic_read(&eb_head(eb)->refs) == 0);
}
int set_extent_buffer_dirty(struct extent_buffer *eb)
@@ -4865,11 +4970,11 @@ int set_extent_buffer_dirty(struct extent_buffer *eb)
check_buffer_tree_ref(eb);
- was_dirty = test_and_set_bit(EXTENT_BUFFER_DIRTY, &eb->bflags);
+ was_dirty = test_and_set_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags);
num_pages = num_extent_pages(eb->start, eb->len);
- WARN_ON(atomic_read(&eb->refs) == 0);
- WARN_ON(!test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags));
+ WARN_ON(atomic_read(&eb_head(eb)->refs) == 0);
+ WARN_ON(!test_bit(EXTENT_BUFFER_TREE_REF, &eb_head(eb)->bflags));
for (i = 0; i < num_pages; i++)
set_page_dirty(extent_buffer_page(eb, i));
@@ -4882,7 +4987,9 @@ int clear_extent_buffer_uptodate(struct extent_buffer *eb)
struct page *page;
unsigned long num_pages;
- clear_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags);
+ if (!eb || !eb_head(eb))
+ return 0;
+ clear_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags);
num_pages = num_extent_pages(eb->start, eb->len);
for (i = 0; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
@@ -4894,22 +5001,43 @@ int clear_extent_buffer_uptodate(struct extent_buffer *eb)
int set_extent_buffer_uptodate(struct extent_buffer *eb)
{
+ struct extent_buffer_head *ebh;
unsigned long i;
struct page *page;
unsigned long num_pages;
+ int uptodate;
- set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags);
- num_pages = num_extent_pages(eb->start, eb->len);
- for (i = 0; i < num_pages; i++) {
- page = extent_buffer_page(eb, i);
- SetPageUptodate(page);
+ ebh = eb->ebh;
+
+ set_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags);
+ if (eb->len < PAGE_CACHE_SIZE) {
+ eb = &(eb_head(eb)->eb);
+ uptodate = 1;
+ do {
+ if (!test_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags)) {
+ uptodate = 0;
+ break;
+ }
+ } while ((eb = eb->eb_next) != NULL);
+
+ if (uptodate) {
+ page = extent_buffer_page(&ebh->eb, 0);
+ SetPageUptodate(page);
+ }
+ } else {
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = 0; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ SetPageUptodate(page);
+ }
}
+
return 0;
}
int extent_buffer_uptodate(struct extent_buffer *eb)
{
- return test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags);
+ return test_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags);
}
int read_extent_buffer_pages(struct extent_io_tree *tree,
@@ -5163,12 +5291,12 @@ void write_extent_buffer(struct extent_buffer *eb, const void *srcv,
WARN_ON(start > eb->len);
WARN_ON(start + len > eb->start + eb->len);
+ WARN_ON(!test_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags));
offset = (start_offset + start) & (PAGE_CACHE_SIZE - 1);
while (len > 0) {
page = extent_buffer_page(eb, i);
- WARN_ON(!PageUptodate(page));
cur = min(len, PAGE_CACHE_SIZE - offset);
kaddr = page_address(page);
@@ -5196,9 +5324,10 @@ void memset_extent_buffer(struct extent_buffer *eb, char c,
offset = (start_offset + start) & (PAGE_CACHE_SIZE - 1);
+ WARN_ON(!test_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags));
+
while (len > 0) {
page = extent_buffer_page(eb, i);
- WARN_ON(!PageUptodate(page));
cur = min(len, PAGE_CACHE_SIZE - offset);
kaddr = page_address(page);
@@ -5227,9 +5356,10 @@ void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src,
offset = (start_offset + dst_offset) &
(PAGE_CACHE_SIZE - 1);
+ WARN_ON(!test_bit(EXTENT_BUFFER_UPTODATE, &dst->ebflags));
+
while (len > 0) {
page = extent_buffer_page(dst, i);
- WARN_ON(!PageUptodate(page));
cur = min(len, (unsigned long)(PAGE_CACHE_SIZE - offset));
@@ -5366,6 +5496,7 @@ void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset,
int try_release_extent_buffer(struct page *page)
{
+ struct extent_buffer_head *ebh;
struct extent_buffer *eb;
/*
@@ -5381,14 +5512,15 @@ int try_release_extent_buffer(struct page *page)
eb = (struct extent_buffer *)page->private;
BUG_ON(!eb);
+ ebh = eb->ebh;
/*
* This is a little awful but should be ok, we need to make sure that
* the eb doesn't disappear out from under us while we're looking at
* this page.
*/
- spin_lock(&eb->refs_lock);
- if (atomic_read(&eb->refs) != 1 || extent_buffer_under_io(eb)) {
- spin_unlock(&eb->refs_lock);
+ spin_lock(&ebh->refs_lock);
+ if (atomic_read(&ebh->refs) != 1 || extent_buffer_under_io(eb)) {
+ spin_unlock(&ebh->refs_lock);
spin_unlock(&page->mapping->private_lock);
return 0;
}
@@ -5398,10 +5530,11 @@ int try_release_extent_buffer(struct page *page)
* If tree ref isn't set then we know the ref on this eb is a real ref,
* so just return, this page will likely be freed soon anyway.
*/
- if (!test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) {
- spin_unlock(&eb->refs_lock);
+ if (!test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &ebh->bflags)) {
+ spin_unlock(&ebh->refs_lock);
return 0;
}
- return release_extent_buffer(eb);
+ return release_extent_buffer(ebh);
}
+
diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h
index ccc264e..840e9a0 100644
--- a/fs/btrfs/extent_io.h
+++ b/fs/btrfs/extent_io.h
@@ -123,19 +123,17 @@ struct extent_state {
#define INLINE_EXTENT_BUFFER_PAGES 16
#define MAX_INLINE_EXTENT_BUFFER_SIZE (INLINE_EXTENT_BUFFER_PAGES * PAGE_CACHE_SIZE)
+
+/* Forward declaration */
+struct extent_buffer_head;
+
struct extent_buffer {
u64 start;
unsigned long len;
- unsigned long map_start;
- unsigned long map_len;
- unsigned long bflags;
- struct btrfs_fs_info *fs_info;
- spinlock_t refs_lock;
- atomic_t refs;
- atomic_t io_pages;
+ unsigned long ebflags;
+ struct extent_buffer_head *ebh;
+ struct extent_buffer *eb_next;
int read_mirror;
- struct rcu_head rcu_head;
- pid_t lock_owner;
/* count of read lock holders on the extent buffer */
atomic_t write_locks;
@@ -146,6 +144,8 @@ struct extent_buffer {
atomic_t spinning_writers;
int lock_nested;
+ pid_t lock_owner;
+
/* protects write locks */
rwlock_t lock;
@@ -158,7 +158,20 @@ struct extent_buffer {
* to unlock
*/
wait_queue_head_t read_lock_wq;
+ wait_queue_head_t lock_wq;
+};
+
+struct extent_buffer_head {
+ unsigned long bflags;
+ struct btrfs_fs_info *fs_info;
+ spinlock_t refs_lock;
+ atomic_t refs;
+ atomic_t io_bvecs;
+ struct rcu_head rcu_head;
+
struct page *pages[INLINE_EXTENT_BUFFER_PAGES];
+
+ struct extent_buffer eb;
#ifdef CONFIG_BTRFS_DEBUG
struct list_head leak_list;
#endif
@@ -175,6 +188,14 @@ static inline int extent_compress_type(unsigned long bio_flags)
return bio_flags >> EXTENT_BIO_FLAG_SHIFT;
}
+/*
+ * return the extent_buffer_head that contains the extent buffer provided.
+ */
+static inline struct extent_buffer_head *eb_head(struct extent_buffer *eb)
+{
+ return eb->ebh;
+
+}
struct extent_map_tree;
typedef struct extent_map *(get_extent_t)(struct inode *inode,
@@ -286,15 +307,15 @@ static inline unsigned long num_extent_pages(u64 start, u64 len)
(start >> PAGE_CACHE_SHIFT);
}
-static inline struct page *extent_buffer_page(struct extent_buffer *eb,
- unsigned long i)
+static inline struct page *extent_buffer_page(
+ struct extent_buffer *eb, unsigned long i)
{
- return eb->pages[i];
+ return eb_head(eb)->pages[i];
}
static inline void extent_buffer_get(struct extent_buffer *eb)
{
- atomic_inc(&eb->refs);
+ atomic_inc(&eb_head(eb)->refs);
}
int memcmp_extent_buffer(struct extent_buffer *eb, const void *ptrv,
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6cb82f6..99bafcb 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -6060,7 +6060,7 @@ int btrfs_read_sys_array(struct btrfs_root *root)
* to silence the warning eg. on PowerPC 64.
*/
if (PAGE_CACHE_SIZE > BTRFS_SUPER_INFO_SIZE)
- SetPageUptodate(sb->pages[0]);
+ SetPageUptodate(eb_head(sb)->pages[0]);
write_extent_buffer(sb, super_copy, 0, BTRFS_SUPER_INFO_SIZE);
array_size = btrfs_super_sys_array_size(super_copy);
diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h
index 4ee4e30..2dc966e 100644
--- a/include/trace/events/btrfs.h
+++ b/include/trace/events/btrfs.h
@@ -697,7 +697,7 @@ TRACE_EVENT(btrfs_cow_block,
TP_fast_assign(
__entry->root_objectid = root->root_key.objectid;
__entry->buf_start = buf->start;
- __entry->refs = atomic_read(&buf->refs);
+ __entry->refs = atomic_read(&eb_head(buf)->refs);
__entry->cow_start = cow->start;
__entry->buf_level = btrfs_header_level(buf);
__entry->cow_level = btrfs_header_level(cow);
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 05/17] Btrfs: subpagesize-blocksize: Read tree blocks whose size is <PAGE_CACHE_SIZE.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (3 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 04/17] Btrfs: subpagesize-blocksize: Define extent_buffer_head Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 06/17] Btrfs: subpagesize-blocksize: Write only dirty extent buffers belonging to a page Chandan Rajendra
` (11 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In the case of subpagesize-blocksize, this patch makes it possible to read
only a single metadata block from the disk instead of all the metadata blocks
that map into a page.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/disk-io.c | 45 +++++++----------
fs/btrfs/disk-io.h | 3 ++
fs/btrfs/extent_io.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++-----
3 files changed, 148 insertions(+), 38 deletions(-)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 3a79833..20168e6 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -431,7 +431,7 @@ static int btree_read_extent_buffer_pages(struct btrfs_root *root,
int mirror_num = 0;
int failed_mirror = 0;
- clear_bit(EXTENT_BUFFER_CORRUPT, &eb->bflags);
+ clear_bit(EXTENT_BUFFER_CORRUPT, &eb->ebflags);
io_tree = &BTRFS_I(root->fs_info->btree_inode)->io_tree;
while (1) {
ret = read_extent_buffer_pages(io_tree, eb, start,
@@ -450,7 +450,7 @@ static int btree_read_extent_buffer_pages(struct btrfs_root *root,
* there is no reason to read the other copies, they won't be
* any less wrong.
*/
- if (test_bit(EXTENT_BUFFER_CORRUPT, &eb->bflags))
+ if (test_bit(EXTENT_BUFFER_CORRUPT, &eb->ebflags))
break;
num_copies = btrfs_num_copies(root->fs_info,
@@ -582,12 +582,13 @@ static noinline int check_leaf(struct btrfs_root *root,
return 0;
}
-static int btree_readpage_end_io_hook(struct btrfs_io_bio *io_bio,
- u64 phy_offset, struct page *page,
- u64 start, u64 end, int mirror)
+int verify_extent_buffer_read(struct btrfs_io_bio *io_bio,
+ struct page *page,
+ u64 start, u64 end, int mirror)
{
u64 found_start;
int found_level;
+ struct extent_buffer_head *ebh;
struct extent_buffer *eb;
struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
int ret = 0;
@@ -597,18 +598,26 @@ static int btree_readpage_end_io_hook(struct btrfs_io_bio *io_bio,
goto out;
eb = (struct extent_buffer *)page->private;
+ do {
+ if ((eb->start <= start) && (eb->start + eb->len - 1 > start))
+ break;
+ } while ((eb = eb->eb_next) != NULL);
+
+ BUG_ON(!eb);
+
+ ebh = eb_head(eb);
/* the pending IO might have been the only thing that kept this buffer
* in memory. Make sure we have a ref for all this other checks
*/
extent_buffer_get(eb);
- reads_done = atomic_dec_and_test(&eb->io_pages);
+ reads_done = atomic_dec_and_test(&ebh->io_bvecs);
if (!reads_done)
goto err;
eb->read_mirror = mirror;
- if (test_bit(EXTENT_BUFFER_IOERR, &eb->bflags)) {
+ if (test_bit(EXTENT_BUFFER_IOERR, &eb->ebflags)) {
ret = -EIO;
goto err;
}
@@ -650,7 +659,7 @@ static int btree_readpage_end_io_hook(struct btrfs_io_bio *io_bio,
* return -EIO.
*/
if (found_level == 0 && check_leaf(root, eb)) {
- set_bit(EXTENT_BUFFER_CORRUPT, &eb->bflags);
+ set_bit(EXTENT_BUFFER_CORRUPT, &eb->ebflags);
ret = -EIO;
}
@@ -658,7 +667,7 @@ static int btree_readpage_end_io_hook(struct btrfs_io_bio *io_bio,
set_extent_buffer_uptodate(eb);
err:
if (reads_done &&
- test_and_clear_bit(EXTENT_BUFFER_READAHEAD, &eb->bflags))
+ test_and_clear_bit(EXTENT_BUFFER_READAHEAD, &eb->ebflags))
btree_readahead_hook(root, eb, eb->start, ret);
if (ret) {
@@ -667,7 +676,7 @@ err:
* again, we have to make sure it has something
* to decrement
*/
- atomic_inc(&eb->io_pages);
+ atomic_inc(&eb_head(eb)->io_bvecs);
clear_extent_buffer_uptodate(eb);
}
free_extent_buffer(eb);
@@ -675,20 +684,6 @@ out:
return ret;
}
-static int btree_io_failed_hook(struct page *page, int failed_mirror)
-{
- struct extent_buffer *eb;
- struct btrfs_root *root = BTRFS_I(page->mapping->host)->root;
-
- eb = (struct extent_buffer *)page->private;
- set_bit(EXTENT_BUFFER_IOERR, &eb->bflags);
- eb->read_mirror = failed_mirror;
- atomic_dec(&eb->io_pages);
- if (test_and_clear_bit(EXTENT_BUFFER_READAHEAD, &eb->bflags))
- btree_readahead_hook(root, eb, eb->start, -EIO);
- return -EIO; /* we fixed nothing */
-}
-
static void end_workqueue_bio(struct bio *bio, int err)
{
struct end_io_wq *end_io_wq = bio->bi_private;
@@ -4156,8 +4151,6 @@ static int btrfs_cleanup_transaction(struct btrfs_root *root)
}
static struct extent_io_ops btree_extent_io_ops = {
- .readpage_end_io_hook = btree_readpage_end_io_hook,
- .readpage_io_failed_hook = btree_io_failed_hook,
.submit_bio_hook = btree_submit_bio_hook,
/* note we're sharing with inode.c for the merge bio hook */
.merge_bio_hook = btrfs_merge_bio_hook,
diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h
index 23ce3ce..482ed21 100644
--- a/fs/btrfs/disk-io.h
+++ b/fs/btrfs/disk-io.h
@@ -111,6 +111,9 @@ static inline void btrfs_put_fs_root(struct btrfs_root *root)
kfree(root);
}
+int verify_extent_buffer_read(struct btrfs_io_bio *io_bio,
+ struct page *page,
+ u64 start, u64 end, int mirror);
void btrfs_mark_buffer_dirty(struct extent_buffer *buf);
int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid,
int atomic);
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 7a923b7..bcf6412 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -14,6 +14,7 @@
#include "extent_io.h"
#include "extent_map.h"
#include "ctree.h"
+#include "disk-io.h"
#include "btrfs_inode.h"
#include "volumes.h"
#include "check-integrity.h"
@@ -2118,7 +2119,7 @@ int repair_eb_io_failure(struct btrfs_root *root, struct extent_buffer *eb,
for (i = 0; i < num_pages; i++) {
struct page *p = extent_buffer_page(eb, i);
- ret = repair_io_failure(root->fs_info, start, PAGE_CACHE_SIZE,
+ ret = repair_io_failure(root->fs_info, start, eb->len,
start, p, mirror_num);
if (ret)
break;
@@ -3497,6 +3498,93 @@ lock_extent_buffer_for_io(struct extent_buffer *eb,
return ret;
}
+static void end_bio_extent_buffer_readpage(struct bio *bio, int err)
+{
+ struct address_space *mapping = bio->bi_io_vec->bv_page->mapping;
+ struct extent_io_tree *tree = &BTRFS_I(mapping->host)->io_tree;
+ struct btrfs_io_bio *io_bio = btrfs_io_bio(bio);
+ struct extent_buffer *eb;
+ struct btrfs_root *root;
+ struct bio_vec *bvec;
+ struct page *page;
+ int uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags);
+ u64 start;
+ u64 end;
+ int mirror;
+ int ret;
+ int i;
+
+ if (err)
+ uptodate = 0;
+
+ bio_for_each_segment_all(bvec, bio, i) {
+ page = bvec->bv_page;
+ root = BTRFS_I(page->mapping->host)->root;
+
+ if (!page->private) {
+ SetPageUptodate(page);
+ goto unlock;
+ }
+
+ eb = (struct extent_buffer *)page->private;
+
+ start = page_offset(page) + bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+
+ do {
+ /*
+ read_extent_buffer_pages() does not start
+ I/O on PG_uptodate pages. Hence the bio may
+ map only part of the extent buffer.
+ */
+ if ((eb->start <= start) && (eb->start + eb->len - 1 > start))
+ break;
+ } while ((eb = eb->eb_next) != NULL);
+
+ BUG_ON(!eb);
+
+ mirror = io_bio->mirror_num;
+
+ if (uptodate) {
+ ret = verify_extent_buffer_read(io_bio, page, start,
+ end, mirror);
+ if (ret)
+ uptodate = 0;
+ }
+
+ if (!uptodate) {
+ set_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
+ eb->read_mirror = mirror;
+ atomic_dec(&eb_head(eb)->io_bvecs);
+ if (test_and_clear_bit(EXTENT_BUFFER_READAHEAD,
+ &eb->ebflags))
+ btree_readahead_hook(root, eb, eb->start,
+ -EIO);
+ ClearPageUptodate(page);
+ SetPageError(page);
+ goto unlock;
+ }
+
+unlock:
+ unlock_page(page);
+ }
+
+ /*
+ We don't need to add a check to see if
+ extent_io_tree->track_uptodate is set or not, Since
+ this function only deals with extent buffers.
+ */
+ bvec = bio->bi_io_vec;
+ start = page_offset(bvec->bv_page) + bvec->bv_offset;
+
+ bvec = bio->bi_io_vec + bio->bi_vcnt - 1;
+ end = page_offset(bvec->bv_page) + bvec->bv_offset + bvec->bv_len - 1;
+
+ unlock_extent(tree, start, end);
+
+ bio_put(bio);
+}
+
static void end_extent_buffer_writeback(struct extent_buffer *eb)
{
clear_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags);
@@ -5044,6 +5132,9 @@ int read_extent_buffer_pages(struct extent_io_tree *tree,
struct extent_buffer *eb, u64 start, int wait,
get_extent_t *get_extent, int mirror_num)
{
+ struct inode *inode = tree->mapping->host;
+ struct btrfs_fs_info *fs_info = BTRFS_I(inode)->root->fs_info;
+ struct extent_state *cached_state = NULL;
unsigned long i;
unsigned long start_i;
struct page *page;
@@ -5056,7 +5147,7 @@ int read_extent_buffer_pages(struct extent_io_tree *tree,
struct bio *bio = NULL;
unsigned long bio_flags = 0;
- if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags))
+ if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags))
return 0;
if (start) {
@@ -5084,21 +5175,37 @@ int read_extent_buffer_pages(struct extent_io_tree *tree,
}
if (all_uptodate) {
if (start_i == 0)
- set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags);
+ set_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags);
goto unlock_exit;
}
- clear_bit(EXTENT_BUFFER_IOERR, &eb->bflags);
+ clear_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
eb->read_mirror = 0;
- atomic_set(&eb->io_pages, num_reads);
+ atomic_set(&eb_head(eb)->io_bvecs, num_reads);
for (i = start_i; i < num_pages; i++) {
page = extent_buffer_page(eb, i);
if (!PageUptodate(page)) {
ClearPageError(page);
- err = __extent_read_full_page(tree, page,
- get_extent, &bio,
- mirror_num, &bio_flags,
- READ | REQ_META);
+ if (eb->len < PAGE_CACHE_SIZE) {
+ lock_extent_bits(tree, eb->start, eb->start + eb->len - 1, 0,
+ &cached_state);
+ err = submit_extent_page(READ | REQ_META, tree,
+ page, eb->start >> 9,
+ eb->len, eb->start - page_offset(page),
+ fs_info->fs_devices->latest_bdev,
+ &bio, -1, end_bio_extent_buffer_readpage,
+ mirror_num, bio_flags, bio_flags);
+ } else {
+ lock_extent_bits(tree, page_offset(page),
+ page_offset(page) + PAGE_CACHE_SIZE - 1,
+ 0, &cached_state);
+ err = submit_extent_page(READ | REQ_META, tree,
+ page, page_offset(page) >> 9,
+ PAGE_CACHE_SIZE, 0,
+ fs_info->fs_devices->latest_bdev,
+ &bio, -1, end_bio_extent_buffer_readpage,
+ mirror_num, bio_flags, bio_flags);
+ }
if (err)
ret = err;
} else {
@@ -5116,11 +5223,18 @@ int read_extent_buffer_pages(struct extent_io_tree *tree,
if (ret || wait != WAIT_COMPLETE)
return ret;
- for (i = start_i; i < num_pages; i++) {
- page = extent_buffer_page(eb, i);
+ if (eb->len < PAGE_CACHE_SIZE) {
+ page = extent_buffer_page(eb, 0);
wait_on_page_locked(page);
- if (!PageUptodate(page))
+ if (!extent_buffer_uptodate(eb))
ret = -EIO;
+ } else {
+ for (i = start_i; i < num_pages; i++) {
+ page = extent_buffer_page(eb, i);
+ wait_on_page_locked(page);
+ if (!PageUptodate(page))
+ ret = -EIO;
+ }
}
return ret;
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 06/17] Btrfs: subpagesize-blocksize: Write only dirty extent buffers belonging to a page
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (4 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 05/17] Btrfs: subpagesize-blocksize: Read tree blocks whose size is <PAGE_CACHE_SIZE Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 07/17] Btrfs: subpagesize-blocksize: Allow mounting filesystems where sectorsize != PAGE_SIZE Chandan Rajendra
` (10 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
For the subpagesize-blocksize scenario, This patch adds the ability to write a
single extent buffer to the disk.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/disk-io.c | 20 ++--
fs/btrfs/extent_io.c | 300 ++++++++++++++++++++++++++++++++++++++++-----------
2 files changed, 250 insertions(+), 70 deletions(-)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 20168e6..6c6e8bb 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -484,17 +484,23 @@ static int btree_read_extent_buffer_pages(struct btrfs_root *root,
static int csum_dirty_buffer(struct btrfs_root *root, struct page *page)
{
- u64 start = page_offset(page);
- u64 found_start;
struct extent_buffer *eb;
+ u64 found_start;
eb = (struct extent_buffer *)page->private;
- if (page != eb->pages[0])
+ if (page != eb_head(eb)->pages[0])
return 0;
- found_start = btrfs_header_bytenr(eb);
- if (WARN_ON(found_start != start || !PageUptodate(page)))
- return 0;
- csum_tree_block(root, eb, 0);
+ do {
+ if (!test_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags))
+ continue;
+ if (WARN_ON(!test_bit(EXTENT_BUFFER_UPTODATE, &eb->ebflags)))
+ continue;
+ found_start = btrfs_header_bytenr(eb);
+ if (WARN_ON(found_start != eb->start))
+ return 0;
+ csum_tree_block(root, eb, 0);
+ } while ((eb = eb->eb_next) != NULL);
+
return 0;
}
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index bcf6412..f9db1be 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -3427,33 +3427,54 @@ void wait_on_extent_buffer_writeback(struct extent_buffer *eb)
TASK_UNINTERRUPTIBLE);
}
-static noinline_for_stack int
-lock_extent_buffer_for_io(struct extent_buffer *eb,
- struct btrfs_fs_info *fs_info,
- struct extent_page_data *epd)
+static void lock_extent_buffer_pages(struct extent_buffer_head *ebh,
+ struct extent_page_data *epd)
{
+ struct extent_buffer *eb = &ebh->eb;
unsigned long i, num_pages;
- int flush = 0;
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = 0; i < num_pages; i++) {
+ struct page *p = extent_buffer_page(eb, i);
+
+ if (!trylock_page(p)) {
+ flush_write_bio(epd);
+ lock_page(p);
+ }
+ }
+
+ return;
+}
+
+static int noinline_for_stack
+lock_extent_buffer_for_io(struct extent_buffer *eb,
+ struct btrfs_fs_info *fs_info,
+ struct extent_page_data *epd)
+{
+ int dirty;
int ret = 0;
if (!btrfs_try_tree_write_lock(eb)) {
- flush = 1;
flush_write_bio(epd);
btrfs_tree_lock(eb);
}
- if (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)) {
+ if (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags)) {
+ dirty = test_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags);
btrfs_tree_unlock(eb);
- if (!epd->sync_io)
- return 0;
- if (!flush) {
- flush_write_bio(epd);
- flush = 1;
+ if (!epd->sync_io) {
+ if (!dirty)
+ return 1;
+ else
+ return 2;
}
+
+ flush_write_bio(epd);
+
while (1) {
wait_on_extent_buffer_writeback(eb);
btrfs_tree_lock(eb);
- if (!test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags))
+ if (!test_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags))
break;
btrfs_tree_unlock(eb);
}
@@ -3464,37 +3485,22 @@ lock_extent_buffer_for_io(struct extent_buffer *eb,
* under IO since we can end up having no IO bits set for a short period
* of time.
*/
- spin_lock(&eb->refs_lock);
- if (test_and_clear_bit(EXTENT_BUFFER_DIRTY, &eb->bflags)) {
- set_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags);
- spin_unlock(&eb->refs_lock);
+ spin_lock(&eb_head(eb)->refs_lock);
+ if (test_and_clear_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags)) {
+ set_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags);
+ spin_unlock(&eb_head(eb)->refs_lock);
btrfs_set_header_flag(eb, BTRFS_HEADER_FLAG_WRITTEN);
__percpu_counter_add(&fs_info->dirty_metadata_bytes,
-eb->len,
fs_info->dirty_metadata_batch);
- ret = 1;
+ ret = 0;
} else {
- spin_unlock(&eb->refs_lock);
+ spin_unlock(&eb_head(eb)->refs_lock);
+ ret = 1;
}
btrfs_tree_unlock(eb);
- if (!ret)
- return ret;
-
- num_pages = num_extent_pages(eb->start, eb->len);
- for (i = 0; i < num_pages; i++) {
- struct page *p = extent_buffer_page(eb, i);
-
- if (!trylock_page(p)) {
- if (!flush) {
- flush_write_bio(epd);
- flush = 1;
- }
- lock_page(p);
- }
- }
-
return ret;
}
@@ -3587,12 +3593,12 @@ unlock:
static void end_extent_buffer_writeback(struct extent_buffer *eb)
{
- clear_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags);
+ clear_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags);
smp_mb__after_atomic();
- wake_up_bit(&eb->bflags, EXTENT_BUFFER_WRITEBACK);
+ wake_up_bit(&eb->ebflags, EXTENT_BUFFER_WRITEBACK);
}
-static void end_bio_extent_buffer_writepage(struct bio *bio, int err)
+static void end_bio_subpagesize_blocksize_ebh_writepage(struct bio *bio, int err)
{
struct bio_vec *bvec;
struct extent_buffer *eb;
@@ -3600,13 +3606,54 @@ static void end_bio_extent_buffer_writepage(struct bio *bio, int err)
bio_for_each_segment_all(bvec, bio, i) {
struct page *page = bvec->bv_page;
+ u64 start, end;
eb = (struct extent_buffer *)page->private;
BUG_ON(!eb);
- done = atomic_dec_and_test(&eb->io_pages);
+ start = page_offset(page) + bvec->bv_offset;
+ end = start + bvec->bv_len - 1;
+
+ do {
+ if (!(eb->start >= start
+ && (eb->start + eb->len) <= (end + 1))) {
+ continue;
+ }
+
+ done = atomic_dec_and_test(&eb_head(eb)->io_bvecs);
- if (err || test_bit(EXTENT_BUFFER_IOERR, &eb->bflags)) {
- set_bit(EXTENT_BUFFER_IOERR, &eb->bflags);
+ if (err || test_bit(EXTENT_BUFFER_IOERR, &eb->ebflags)) {
+ set_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
+ ClearPageUptodate(page);
+ SetPageError(page);
+ }
+
+ end_extent_buffer_writeback(eb);
+
+ if (done)
+ end_page_writeback(page);
+
+ } while ((eb = eb->eb_next) != NULL);
+
+ }
+
+ bio_put(bio);
+}
+
+static void end_bio_regular_ebh_writepage(struct bio *bio, int err)
+{
+ struct extent_buffer *eb;
+ struct bio_vec *bvec;
+ int i, done;
+
+ bio_for_each_segment_all(bvec, bio, i) {
+ struct page *page = bvec->bv_page;
+
+ eb = (struct extent_buffer *)page->private;
+ BUG_ON(!eb);
+ done = atomic_dec_and_test(&eb_head(eb)->io_bvecs);
+
+ if (err || test_bit(EXTENT_BUFFER_IOERR, &eb->ebflags)) {
+ set_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
ClearPageUptodate(page);
SetPageError(page);
}
@@ -3622,22 +3669,25 @@ static void end_bio_extent_buffer_writepage(struct bio *bio, int err)
bio_put(bio);
}
-static noinline_for_stack int write_one_eb(struct extent_buffer *eb,
- struct btrfs_fs_info *fs_info,
- struct writeback_control *wbc,
- struct extent_page_data *epd)
+
+static noinline_for_stack int
+write_regular_ebh(struct extent_buffer_head *ebh,
+ struct btrfs_fs_info *fs_info,
+ struct writeback_control *wbc,
+ struct extent_page_data *epd)
{
struct block_device *bdev = fs_info->fs_devices->latest_bdev;
struct extent_io_tree *tree = &BTRFS_I(fs_info->btree_inode)->io_tree;
- u64 offset = eb->start;
+ struct extent_buffer *eb = &ebh->eb;
+ u64 offset = eb->start & ~(PAGE_CACHE_SIZE - 1);
unsigned long i, num_pages;
unsigned long bio_flags = 0;
int rw = (epd->sync_io ? WRITE_SYNC : WRITE) | REQ_META;
int ret = 0;
- clear_bit(EXTENT_BUFFER_IOERR, &eb->bflags);
+ clear_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
num_pages = num_extent_pages(eb->start, eb->len);
- atomic_set(&eb->io_pages, num_pages);
+ atomic_set(&eb_head(eb)->io_bvecs, num_pages);
if (btrfs_header_owner(eb) == BTRFS_TREE_LOG_OBJECTID)
bio_flags = EXTENT_BIO_TREE_LOG;
@@ -3648,13 +3698,14 @@ static noinline_for_stack int write_one_eb(struct extent_buffer *eb,
set_page_writeback(p);
ret = submit_extent_page(rw, tree, p, offset >> 9,
PAGE_CACHE_SIZE, 0, bdev, &epd->bio,
- -1, end_bio_extent_buffer_writepage,
+ -1, end_bio_regular_ebh_writepage,
0, epd->bio_flags, bio_flags);
epd->bio_flags = bio_flags;
if (ret) {
- set_bit(EXTENT_BUFFER_IOERR, &eb->bflags);
+ set_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
SetPageError(p);
- if (atomic_sub_and_test(num_pages - i, &eb->io_pages))
+ if (atomic_sub_and_test(num_pages - i,
+ &eb_head(eb)->io_bvecs))
end_extent_buffer_writeback(eb);
ret = -EIO;
break;
@@ -3674,12 +3725,85 @@ static noinline_for_stack int write_one_eb(struct extent_buffer *eb,
return ret;
}
+static int write_subpagesize_blocksize_ebh(struct extent_buffer_head *ebh,
+ struct btrfs_fs_info *fs_info,
+ struct writeback_control *wbc,
+ struct extent_page_data *epd,
+ unsigned long ebs_to_write)
+{
+ struct block_device *bdev = fs_info->fs_devices->latest_bdev;
+ struct extent_io_tree *tree = &BTRFS_I(fs_info->btree_inode)->io_tree;
+ struct extent_buffer *eb;
+ struct page *p;
+ u64 offset;
+ unsigned long i;
+ unsigned long bio_flags = 0;
+ int rw = (epd->sync_io ? WRITE_SYNC : WRITE) | REQ_META;
+ int ret = 0, err = 0;
+
+ eb = &ebh->eb;
+ p = extent_buffer_page(eb, 0);
+ clear_page_dirty_for_io(p);
+ set_page_writeback(p);
+ i = 0;
+ do {
+ if (!test_bit(i++, &ebs_to_write))
+ continue;
+
+ clear_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
+ atomic_inc(&eb_head(eb)->io_bvecs);
+
+ if (btrfs_header_owner(eb) == BTRFS_TREE_LOG_OBJECTID)
+ bio_flags = EXTENT_BIO_TREE_LOG;
+
+ offset = eb->start - page_offset(p);
+
+ ret = submit_extent_page(rw, tree, p, eb->start >> 9,
+ eb->len, offset,
+ bdev, &epd->bio, -1,
+ end_bio_subpagesize_blocksize_ebh_writepage,
+ 0, epd->bio_flags, bio_flags);
+ epd->bio_flags = bio_flags;
+ if (ret) {
+ set_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
+ SetPageError(p);
+ atomic_dec(&eb_head(eb)->io_bvecs);
+ end_extent_buffer_writeback(eb);
+ err = -EIO;
+ }
+ } while ((eb = eb->eb_next) != NULL);
+
+ if (!err) {
+ update_nr_written(p, wbc, 1);
+ }
+
+ unlock_page(p);
+
+ return ret;
+}
+
+static void redirty_extent_buffer_pages_for_writepage(struct extent_buffer *eb,
+ struct writeback_control *wbc)
+{
+ unsigned long i, num_pages;
+ struct page *p;
+
+ num_pages = num_extent_pages(eb->start, eb->len);
+ for (i = 0; i < num_pages; i++) {
+ p = extent_buffer_page(eb, i);
+ redirty_page_for_writepage(wbc, p);
+ }
+
+ return;
+}
+
int btree_write_cache_pages(struct address_space *mapping,
- struct writeback_control *wbc)
+ struct writeback_control *wbc)
{
struct extent_io_tree *tree = &BTRFS_I(mapping->host)->io_tree;
struct btrfs_fs_info *fs_info = BTRFS_I(mapping->host)->root->fs_info;
- struct extent_buffer *eb, *prev_eb = NULL;
+ struct extent_buffer *eb;
+ struct extent_buffer_head *ebh, *prev_ebh = NULL;
struct extent_page_data epd = {
.bio = NULL,
.tree = tree,
@@ -3690,6 +3814,7 @@ int btree_write_cache_pages(struct address_space *mapping,
int ret = 0;
int done = 0;
int nr_to_write_done = 0;
+ unsigned long ebs_to_write, dirty_ebs;
struct pagevec pvec;
int nr_pages;
pgoff_t index;
@@ -3716,7 +3841,7 @@ retry:
while (!done && !nr_to_write_done && (index <= end) &&
(nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1))) {
- unsigned i;
+ unsigned i, j;
scanned = 1;
for (i = 0; i < nr_pages; i++) {
@@ -3748,30 +3873,79 @@ retry:
continue;
}
- if (eb == prev_eb) {
+ ebh = eb_head(eb);
+ if (ebh == prev_ebh) {
spin_unlock(&mapping->private_lock);
continue;
}
- ret = atomic_inc_not_zero(&eb->refs);
+ ret = atomic_inc_not_zero(&ebh->refs);
spin_unlock(&mapping->private_lock);
if (!ret)
continue;
- prev_eb = eb;
- ret = lock_extent_buffer_for_io(eb, fs_info, &epd);
- if (!ret) {
- free_extent_buffer(eb);
+ prev_ebh = ebh;
+
+ j = 0;
+ ebs_to_write = dirty_ebs = 0;
+ eb = &ebh->eb;
+ do {
+ BUG_ON(j >= BITS_PER_LONG);
+
+ ret = lock_extent_buffer_for_io(eb, fs_info, &epd);
+ switch (ret) {
+ case 0:
+ /*
+ EXTENT_BUFFER_DIRTY was set and we were able to
+ clear it.
+ */
+ set_bit(j, &ebs_to_write);
+ break;
+ case 2:
+ /*
+ EXTENT_BUFFER_DIRTY was set, but we were unable
+ to clear EXTENT_BUFFER_WRITEBACK that was set
+ before we got the extent buffer locked.
+ */
+ set_bit(j, &dirty_ebs);
+ default:
+ /*
+ EXTENT_BUFFER_DIRTY wasn't set.
+ */
+ break;
+ }
+ ++j;
+ } while ((eb = eb->eb_next) != NULL);
+
+ ret = 0;
+
+ if (!ebs_to_write) {
+ free_extent_buffer(&ebh->eb);
continue;
}
- ret = write_one_eb(eb, fs_info, wbc, &epd);
+ /*
+ Now that we know that atleast one of the extent buffer
+ belonging to the extent buffer head must be written to
+ the disk, lock the extent_buffer_head's pages.
+ */
+ lock_extent_buffer_pages(ebh, &epd);
+
+ if (ebh->eb.len < PAGE_CACHE_SIZE) {
+ ret = write_subpagesize_blocksize_ebh(ebh, fs_info, wbc, &epd, ebs_to_write);
+ if (dirty_ebs) {
+ redirty_extent_buffer_pages_for_writepage(&ebh->eb, wbc);
+ }
+ } else {
+ ret = write_regular_ebh(ebh, fs_info, wbc, &epd);
+ }
+
if (ret) {
done = 1;
- free_extent_buffer(eb);
+ free_extent_buffer(&ebh->eb);
break;
}
- free_extent_buffer(eb);
+ free_extent_buffer(&ebh->eb);
/*
* the filesystem may choose to bump up nr_to_write.
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 07/17] Btrfs: subpagesize-blocksize: Allow mounting filesystems where sectorsize != PAGE_SIZE
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (5 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 06/17] Btrfs: subpagesize-blocksize: Write only dirty extent buffers belonging to a page Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 08/17] Btrfs: subpagesize-blocksize: Compute and look up csums based on sectorsized blocks Chandan Rajendra
` (9 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandra Seetharaman, aneesh.kumar, linux-btrfs, chandan,
steve.capper, Chandan Rajendra
From: Chandra Seetharaman <sekharan@us.ibm.com>
This patch allows mounting filesystems with blocksize smaller than the
PAGE_SIZE.
Signed-off-by: Chandra Seetharaman <sekharan@us.ibm.com>
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/disk-io.c | 6 ------
1 file changed, 6 deletions(-)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 6c6e8bb..2f3caaf 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2634,12 +2634,6 @@ int open_ctree(struct super_block *sb,
goto fail_sb_buffer;
}
- if (sectorsize != PAGE_SIZE) {
- printk(KERN_WARNING "BTRFS: Incompatible sector size(%lu) "
- "found on %s\n", (unsigned long)sectorsize, sb->s_id);
- goto fail_sb_buffer;
- }
-
mutex_lock(&fs_info->chunk_mutex);
ret = btrfs_read_sys_array(tree_root);
mutex_unlock(&fs_info->chunk_mutex);
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 08/17] Btrfs: subpagesize-blocksize: Compute and look up csums based on sectorsized blocks.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (6 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 07/17] Btrfs: subpagesize-blocksize: Allow mounting filesystems where sectorsize != PAGE_SIZE Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 09/17] Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty blocks of a page Chandan Rajendra
` (8 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
Checksums are applicable to sectorsize units. The current code uses
bio->bv_len units to compute and look up checksums. This works on machines
where sectorsize == PAGE_CACHE_SIZE. This patch makes the checksum
computation and look up code to work with sectorsize units.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/file-item.c | 87 ++++++++++++++++++++++++++++++++--------------------
fs/btrfs/inode.c | 53 +++++++++++++++++++++-----------
2 files changed, 89 insertions(+), 51 deletions(-)
diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c
index 54c84da..000418a 100644
--- a/fs/btrfs/file-item.c
+++ b/fs/btrfs/file-item.c
@@ -172,6 +172,7 @@ static int __btrfs_lookup_bio_sums(struct btrfs_root *root,
u64 item_start_offset = 0;
u64 item_last_offset = 0;
u64 disk_bytenr;
+ u64 page_bytes_left;
u32 diff;
int nblocks;
int bio_index = 0;
@@ -220,6 +221,8 @@ static int __btrfs_lookup_bio_sums(struct btrfs_root *root,
disk_bytenr = (u64)bio->bi_iter.bi_sector << 9;
if (dio)
offset = logical_offset;
+
+ page_bytes_left = bvec->bv_len;
while (bio_index < bio->bi_vcnt) {
if (!dio)
offset = page_offset(bvec->bv_page) + bvec->bv_offset;
@@ -243,7 +246,7 @@ static int __btrfs_lookup_bio_sums(struct btrfs_root *root,
if (BTRFS_I(inode)->root->root_key.objectid ==
BTRFS_DATA_RELOC_TREE_OBJECTID) {
set_extent_bits(io_tree, offset,
- offset + bvec->bv_len - 1,
+ offset + root->sectorsize - 1,
EXTENT_NODATASUM, GFP_NOFS);
} else {
btrfs_info(BTRFS_I(inode)->root->fs_info,
@@ -281,11 +284,17 @@ static int __btrfs_lookup_bio_sums(struct btrfs_root *root,
found:
csum += count * csum_size;
nblocks -= count;
- bio_index += count;
+
while (count--) {
- disk_bytenr += bvec->bv_len;
- offset += bvec->bv_len;
- bvec++;
+ disk_bytenr += root->sectorsize;
+ offset += root->sectorsize;
+ page_bytes_left -= root->sectorsize;
+ if (!page_bytes_left) {
+ bio_index++;
+ bvec++;
+ page_bytes_left = bvec->bv_len;
+ }
+
}
}
btrfs_free_path(path);
@@ -442,6 +451,8 @@ int btrfs_csum_one_bio(struct btrfs_root *root, struct inode *inode,
struct bio_vec *bvec = bio->bi_io_vec;
int bio_index = 0;
int index;
+ int nr_sectors;
+ int i;
unsigned long total_bytes = 0;
unsigned long this_sum_bytes = 0;
u64 offset;
@@ -469,41 +480,51 @@ int btrfs_csum_one_bio(struct btrfs_root *root, struct inode *inode,
if (!contig)
offset = page_offset(bvec->bv_page) + bvec->bv_offset;
- if (offset >= ordered->file_offset + ordered->len ||
- offset < ordered->file_offset) {
- unsigned long bytes_left;
- sums->len = this_sum_bytes;
- this_sum_bytes = 0;
- btrfs_add_ordered_sum(inode, ordered, sums);
- btrfs_put_ordered_extent(ordered);
+ data = kmap_atomic(bvec->bv_page);
- bytes_left = bio->bi_iter.bi_size - total_bytes;
- sums = kzalloc(btrfs_ordered_sum_size(root, bytes_left),
- GFP_NOFS);
- BUG_ON(!sums); /* -ENOMEM */
- sums->len = bytes_left;
- ordered = btrfs_lookup_ordered_extent(inode, offset);
- BUG_ON(!ordered); /* Logic error */
- sums->bytenr = ((u64)bio->bi_iter.bi_sector << 9) +
- total_bytes;
- index = 0;
+ nr_sectors = (bvec->bv_len + root->sectorsize - 1)
+ >> root->fs_info->sb->s_blocksize_bits;
+
+
+ for (i = 0; i < nr_sectors; i++) {
+ if (offset >= ordered->file_offset + ordered->len ||
+ offset < ordered->file_offset) {
+ unsigned long bytes_left;
+ sums->len = this_sum_bytes;
+ this_sum_bytes = 0;
+ btrfs_add_ordered_sum(inode, ordered, sums);
+ btrfs_put_ordered_extent(ordered);
+
+ bytes_left = bio->bi_iter.bi_size - total_bytes;
+
+ sums = kzalloc(btrfs_ordered_sum_size(root, bytes_left),
+ GFP_NOFS);
+ BUG_ON(!sums); /* -ENOMEM */
+ sums->len = bytes_left;
+ ordered = btrfs_lookup_ordered_extent(inode, offset);
+ BUG_ON(!ordered); /* Logic error */
+ sums->bytenr = ((u64)bio->bi_iter.bi_sector << 9) +
+ total_bytes;
+ index = 0;
+ }
+
+ sums->sums[index] = ~(u32)0;
+ sums->sums[index]
+ = btrfs_csum_data(data + bvec->bv_offset + (i * root->sectorsize),
+ sums->sums[index],
+ root->sectorsize);
+ btrfs_csum_final(sums->sums[index],
+ (char *)(sums->sums + index));
+ index++;
+ offset += root->sectorsize;
+ this_sum_bytes += root->sectorsize;
+ total_bytes += root->sectorsize;
}
- data = kmap_atomic(bvec->bv_page);
- sums->sums[index] = ~(u32)0;
- sums->sums[index] = btrfs_csum_data(data + bvec->bv_offset,
- sums->sums[index],
- bvec->bv_len);
kunmap_atomic(data);
- btrfs_csum_final(sums->sums[index],
- (char *)(sums->sums + index));
bio_index++;
- index++;
- total_bytes += bvec->bv_len;
- this_sum_bytes += bvec->bv_len;
- offset += bvec->bv_len;
bvec++;
}
this_sum_bytes = 0;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 2ffb4df..ae5d459 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7116,37 +7116,54 @@ static void btrfs_endio_direct_read(struct bio *bio, int err)
struct btrfs_root *root = BTRFS_I(inode)->root;
struct bio *dio_bio;
u32 *csums = (u32 *)dip->csum;
+ int index = 0;
+ int nr_sectors;
u64 start;
- int i;
+ int i, j;
+
+ if (err || !(BTRFS_I(inode)->flags & BTRFS_INODE_NODATASUM))
+ goto unlock;
start = dip->logical_offset;
bio_for_each_segment_all(bvec, bio, i) {
- if (!(BTRFS_I(inode)->flags & BTRFS_INODE_NODATASUM)) {
- struct page *page = bvec->bv_page;
- char *kaddr;
- u32 csum = ~(u32)0;
- unsigned long flags;
+ struct page *page = bvec->bv_page;
+ char *kaddr;
+ u32 csum;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ kaddr = kmap_atomic(page);
+
+ nr_sectors = bvec->bv_len >> inode->i_sb->s_blocksize_bits;
- local_irq_save(flags);
- kaddr = kmap_atomic(page);
- csum = btrfs_csum_data(kaddr + bvec->bv_offset,
- csum, bvec->bv_len);
+ for (j = 0; j < nr_sectors; j++) {
+ csum = ~(u32)0;
+ csum = btrfs_csum_data(kaddr + bvec->bv_offset
+ + (root->sectorsize * j),
+ csum, root->sectorsize);
btrfs_csum_final(csum, (char *)&csum);
- kunmap_atomic(kaddr);
- local_irq_restore(flags);
- flush_dcache_page(bvec->bv_page);
- if (csum != csums[i]) {
- btrfs_err(root->fs_info, "csum failed ino %llu off %llu csum %u expected csum %u",
- btrfs_ino(inode), start, csum,
- csums[i]);
+ if (csum != csums[index]) {
+ btrfs_err(root->fs_info,
+ "csum failed ino %llu off %llu csum %u expected csum %u",
+ btrfs_ino(inode), start, csum,
+ csums[index]);
err = -EIO;
+ break;
}
+
+ start += root->sectorsize;
+ index++;
}
- start += bvec->bv_len;
+ kunmap_atomic(kaddr);
+ local_irq_restore(flags);
+ flush_dcache_page(bvec->bv_page);
+ if (err)
+ break;
}
+unlock:
unlock_extent(&BTRFS_I(inode)->io_tree, dip->logical_offset,
dip->logical_offset + dip->bytes - 1);
dio_bio = dip->dio_bio;
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 09/17] Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty blocks of a page.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (7 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 08/17] Btrfs: subpagesize-blocksize: Compute and look up csums based on sectorsized blocks Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 10/17] Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units Chandan Rajendra
` (7 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
The code now loops across 'ordered extents' instead of 'extent maps' to figure
out the dirty blocks of the page to be submitted for a write operation.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 74 ++++++++++++++++++++--------------------------------
1 file changed, 29 insertions(+), 45 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index f9db1be..3c33944 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -3186,18 +3186,18 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
int write_flags, int *nr_ret)
{
struct extent_io_tree *tree = epd->tree;
+ struct btrfs_ordered_extent *ordered;
u64 start = page_offset(page);
u64 page_end = start + PAGE_CACHE_SIZE - 1;
u64 end;
u64 cur = start;
u64 extent_offset;
- u64 block_start;
+ u64 extent_end;
u64 iosize;
sector_t sector;
struct extent_state *cached_state = NULL;
- struct extent_map *em;
struct block_device *bdev;
- size_t pg_offset = 0;
+ size_t pg_offset;
size_t blocksize;
int ret = 0;
int nr = 0;
@@ -3237,59 +3237,46 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
blocksize = inode->i_sb->s_blocksize;
while (cur <= end) {
- u64 em_end;
if (cur >= i_size) {
if (tree->ops && tree->ops->writepage_end_io_hook)
tree->ops->writepage_end_io_hook(page, cur,
page_end, NULL, 1);
break;
}
- em = epd->get_extent(inode, page, pg_offset, cur,
- end - cur + 1, 1);
- if (IS_ERR_OR_NULL(em)) {
- SetPageError(page);
- ret = PTR_ERR_OR_ZERO(em);
- break;
- }
- extent_offset = cur - em->start;
- em_end = extent_map_end(em);
- BUG_ON(em_end <= cur);
+ ordered = btrfs_lookup_ordered_extent(inode, cur);
+ if (!ordered) {
+ cur += blocksize;
+ continue;
+ }
+
+ pg_offset = cur & (PAGE_CACHE_SIZE - 1);
+
+ extent_offset = cur - ordered->file_offset;
+ extent_end = ordered->file_offset + ordered->len;
+ extent_end = (extent_end < ordered->file_offset) ? -1 : extent_end;
+ BUG_ON(extent_end <= cur);
BUG_ON(end < cur);
- iosize = min(em_end - cur, end - cur + 1);
+ iosize = min(extent_end - cur, end - cur + 1);
iosize = ALIGN(iosize, blocksize);
- sector = (em->block_start + extent_offset) >> 9;
- bdev = em->bdev;
- block_start = em->block_start;
- compressed = test_bit(EXTENT_FLAG_COMPRESSED, &em->flags);
- free_extent_map(em);
- em = NULL;
+
+ sector = (ordered->start + extent_offset) >> 9;
+ bdev = BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
+ compressed = test_bit(BTRFS_ORDERED_COMPRESSED, &ordered->flags);
+ btrfs_put_ordered_extent(ordered);
+ ordered = NULL;
/*
* compressed and inline extents are written through other
* paths in the FS
*/
- if (compressed || block_start == EXTENT_MAP_HOLE ||
- block_start == EXTENT_MAP_INLINE) {
- /*
- * end_io notification does not happen here for
- * compressed extents
- */
- if (!compressed && tree->ops &&
- tree->ops->writepage_end_io_hook)
- tree->ops->writepage_end_io_hook(page, cur,
- cur + iosize - 1,
- NULL, 1);
- else if (compressed) {
- /* we don't want to end_page_writeback on
- * a compressed extent. this happens
- * elsewhere
- */
- nr++;
- }
-
+ if (compressed) {
+ /* we don't want to end_page_writeback on
+ * a compressed extent. this happens
+ * elsewhere
+ */
+ nr++;
cur += iosize;
- pg_offset += iosize;
continue;
}
@@ -3320,7 +3307,6 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
SetPageError(page);
}
cur = cur + iosize;
- pg_offset += iosize;
nr++;
}
done:
@@ -3348,7 +3334,7 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc,
u64 page_end = start + PAGE_CACHE_SIZE - 1;
int ret;
int nr = 0;
- size_t pg_offset = 0;
+ size_t pg_offset;
loff_t i_size = i_size_read(inode);
unsigned long end_index = i_size >> PAGE_CACHE_SHIFT;
int write_flags;
@@ -3383,8 +3369,6 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc,
flush_dcache_page(page);
}
- pg_offset = 0;
-
set_page_extent_mapped(page);
ret = writepage_delalloc(inode, page, wbc, epd, start, &nr_written);
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 10/17] Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (8 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 09/17] Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty blocks of a page Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 11/17] Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in " Chandan Rajendra
` (6 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
While at it, this commit changes btrfs_truncate_page() to truncate sectorsized
blocks instead of pages. Hence the function has been renamed to
btrfs_truncate_block().
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/ctree.h | 2 +-
fs/btrfs/file.c | 41 ++++++++++++++++++++++-------------------
fs/btrfs/inode.c | 48 +++++++++++++++++++++++++-----------------------
3 files changed, 48 insertions(+), 43 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 5b7b7ca..59779dc 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -3815,7 +3815,7 @@ int btrfs_unlink_subvol(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct inode *dir, u64 objectid,
const char *name, int name_len);
-int btrfs_truncate_page(struct inode *inode, loff_t from, loff_t len,
+int btrfs_truncate_block(struct inode *inode, loff_t from, loff_t len,
int front);
int btrfs_truncate_inode_items(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index 444819d..b1e0d27 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -2200,21 +2200,24 @@ static int btrfs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
u64 tail_len;
u64 orig_start = offset;
u64 cur_offset;
+ unsigned char blocksize_bits;
u64 min_size = btrfs_calc_trunc_metadata_size(root, 1);
u64 drop_end;
int ret = 0;
int err = 0;
int rsv_count;
- bool same_page;
+ bool same_block;
bool no_holes = btrfs_fs_incompat(root->fs_info, NO_HOLES);
u64 ino_size;
+ blocksize_bits = inode->i_sb->s_blocksize_bits;
+
ret = btrfs_wait_ordered_range(inode, offset, len);
if (ret)
return ret;
mutex_lock(&inode->i_mutex);
- ino_size = round_up(inode->i_size, PAGE_CACHE_SIZE);
+ ino_size = round_up(inode->i_size, root->sectorsize);
ret = find_first_non_hole(inode, &offset, &len);
if (ret < 0)
goto out_only_mutex;
@@ -2224,29 +2227,28 @@ static int btrfs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
goto out_only_mutex;
}
- lockstart = round_up(offset , BTRFS_I(inode)->root->sectorsize);
+ lockstart = round_up(offset, BTRFS_I(inode)->root->sectorsize);
lockend = round_down(offset + len,
BTRFS_I(inode)->root->sectorsize) - 1;
- same_page = ((offset >> PAGE_CACHE_SHIFT) ==
- ((offset + len - 1) >> PAGE_CACHE_SHIFT));
-
+ same_block = ((offset >> blocksize_bits)
+ == ((offset + len - 1) >> blocksize_bits));
/*
- * We needn't truncate any page which is beyond the end of the file
+ * We needn't truncate any block which is beyond the end of the file
* because we are sure there is no data there.
*/
/*
- * Only do this if we are in the same page and we aren't doing the
- * entire page.
+ * Only do this if we are in the same block and we aren't doing the
+ * entire block.
*/
- if (same_page && len < PAGE_CACHE_SIZE) {
+ if (same_block && len < root->sectorsize) {
if (offset < ino_size)
- ret = btrfs_truncate_page(inode, offset, len, 0);
+ ret = btrfs_truncate_block(inode, offset, len, 0);
goto out_only_mutex;
}
- /* zero back part of the first page */
+ /* zero back part of the first block */
if (offset < ino_size) {
- ret = btrfs_truncate_page(inode, offset, 0, 0);
+ ret = btrfs_truncate_block(inode, offset, 0, 0);
if (ret) {
mutex_unlock(&inode->i_mutex);
return ret;
@@ -2281,11 +2283,12 @@ static int btrfs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
if (!ret) {
/* zero the front end of the last page */
if (tail_start + tail_len < ino_size) {
- ret = btrfs_truncate_page(inode,
- tail_start + tail_len, 0, 1);
+ ret = btrfs_truncate_block(inode,
+ tail_start + tail_len,
+ 0, 1);
if (ret)
goto out_only_mutex;
- }
+ }
}
}
@@ -2506,10 +2509,10 @@ static long btrfs_fallocate(struct file *file, int mode,
} else {
/*
* If we are fallocating from the end of the file onward we
- * need to zero out the end of the page if i_size lands in the
- * middle of a page.
+ * need to zero out the end of the block if i_size lands in the
+ * middle of a block.
*/
- ret = btrfs_truncate_page(inode, inode->i_size, 0, 0);
+ ret = btrfs_truncate_block(inode, inode->i_size, 0, 0);
if (ret)
goto out;
}
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index ae5d459..7ad7d0f 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -4273,7 +4273,7 @@ error:
* This will find the page for the "from" offset and cow the page and zero the
* part we want to zero. This is used with truncate and hole punching.
*/
-int btrfs_truncate_page(struct inode *inode, loff_t from, loff_t len,
+int btrfs_truncate_block(struct inode *inode, loff_t from, loff_t len,
int front)
{
struct address_space *mapping = inode->i_mapping;
@@ -4284,30 +4284,30 @@ int btrfs_truncate_page(struct inode *inode, loff_t from, loff_t len,
char *kaddr;
u32 blocksize = root->sectorsize;
pgoff_t index = from >> PAGE_CACHE_SHIFT;
- unsigned offset = from & (PAGE_CACHE_SIZE-1);
+ unsigned offset = from & (blocksize - 1);
struct page *page;
gfp_t mask = btrfs_alloc_write_mask(mapping);
int ret = 0;
- u64 page_start;
- u64 page_end;
+ u64 block_start;
+ u64 block_end;
if ((offset & (blocksize - 1)) == 0 &&
(!len || ((len & (blocksize - 1)) == 0)))
goto out;
- ret = btrfs_delalloc_reserve_space(inode, PAGE_CACHE_SIZE);
+ ret = btrfs_delalloc_reserve_space(inode, blocksize);
if (ret)
goto out;
again:
page = find_or_create_page(mapping, index, mask);
if (!page) {
- btrfs_delalloc_release_space(inode, PAGE_CACHE_SIZE);
+ btrfs_delalloc_release_space(inode, blocksize);
ret = -ENOMEM;
goto out;
}
- page_start = page_offset(page);
- page_end = page_start + PAGE_CACHE_SIZE - 1;
+ block_start = round_down(from, blocksize);
+ block_end = block_start + blocksize - 1;
if (!PageUptodate(page)) {
ret = btrfs_readpage(NULL, page);
@@ -4324,12 +4324,12 @@ again:
}
wait_on_page_writeback(page);
- lock_extent_bits(io_tree, page_start, page_end, 0, &cached_state);
+ lock_extent_bits(io_tree, block_start, block_end, 0, &cached_state);
set_page_extent_mapped(page);
- ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ ordered = btrfs_lookup_ordered_extent(inode, block_start);
if (ordered) {
- unlock_extent_cached(io_tree, page_start, page_end,
+ unlock_extent_cached(io_tree, block_start, block_end,
&cached_state, GFP_NOFS);
unlock_page(page);
page_cache_release(page);
@@ -4338,38 +4338,40 @@ again:
goto again;
}
- clear_extent_bit(&BTRFS_I(inode)->io_tree, page_start, page_end,
+ clear_extent_bit(&BTRFS_I(inode)->io_tree, block_start, block_end,
EXTENT_DIRTY | EXTENT_DELALLOC |
EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG,
0, 0, &cached_state, GFP_NOFS);
- ret = btrfs_set_extent_delalloc(inode, page_start, page_end,
+ ret = btrfs_set_extent_delalloc(inode, block_start, block_end,
&cached_state);
if (ret) {
- unlock_extent_cached(io_tree, page_start, page_end,
+ unlock_extent_cached(io_tree, block_start, block_end,
&cached_state, GFP_NOFS);
goto out_unlock;
}
- if (offset != PAGE_CACHE_SIZE) {
+ if (offset != blocksize) {
if (!len)
- len = PAGE_CACHE_SIZE - offset;
+ len = blocksize - offset;
kaddr = kmap(page);
if (front)
- memset(kaddr, 0, offset);
+ memset(kaddr + (block_start - page_offset(page)),
+ 0, offset);
else
- memset(kaddr + offset, 0, len);
+ memset(kaddr + (block_start - page_offset(page)) + offset,
+ 0, len);
flush_dcache_page(page);
kunmap(page);
}
ClearPageChecked(page);
set_page_dirty(page);
- unlock_extent_cached(io_tree, page_start, page_end, &cached_state,
+ unlock_extent_cached(io_tree, block_start, block_end, &cached_state,
GFP_NOFS);
out_unlock:
if (ret)
- btrfs_delalloc_release_space(inode, PAGE_CACHE_SIZE);
+ btrfs_delalloc_release_space(inode, blocksize);
unlock_page(page);
page_cache_release(page);
out:
@@ -4440,11 +4442,11 @@ int btrfs_cont_expand(struct inode *inode, loff_t oldsize, loff_t size)
int err = 0;
/*
- * If our size started in the middle of a page we need to zero out the
- * rest of the page before we expand the i_size, otherwise we could
+ * If our size started in the middle of a block we need to zero out the
+ * rest of the block before we expand the i_size, otherwise we could
* expose stale data.
*/
- err = btrfs_truncate_page(inode, oldsize, 0, 0);
+ err = btrfs_truncate_block(inode, oldsize, 0, 0);
if (err)
return err;
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 11/17] Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in sectorsized units.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (9 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 10/17] Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 12/17] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page Chandan Rajendra
` (5 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In subpagesize-blocksize scenario, if i_size occurs in a block which is not
the last block in the page, then the space to be reserved should be calculated
appropriately.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/inode.c | 33 ++++++++++++++++++++++-----------
1 file changed, 22 insertions(+), 11 deletions(-)
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 7ad7d0f..23ce9ff 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7812,26 +7812,23 @@ int btrfs_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
loff_t size;
int ret;
int reserved = 0;
+ u64 delalloc_size;
u64 page_start;
u64 page_end;
sb_start_pagefault(inode->i_sb);
- ret = btrfs_delalloc_reserve_space(inode, PAGE_CACHE_SIZE);
- if (!ret) {
- ret = file_update_time(vma->vm_file);
- reserved = 1;
- }
+
+ ret = file_update_time(vma->vm_file);
if (ret) {
if (ret == -ENOMEM)
ret = VM_FAULT_OOM;
else /* -ENOSPC, -EIO, etc */
ret = VM_FAULT_SIGBUS;
- if (reserved)
- goto out;
- goto out_noreserve;
+ goto out;
}
ret = VM_FAULT_NOPAGE; /* make the VM retry the fault */
+
again:
lock_page(page);
size = i_size_read(inode);
@@ -7862,6 +7859,19 @@ again:
goto again;
}
+ if (page->index == ((size - 1) >> PAGE_CACHE_SHIFT))
+ delalloc_size = round_up(size - page_start, root->sectorsize);
+ else
+ delalloc_size = PAGE_CACHE_SIZE;
+
+ ret = btrfs_delalloc_reserve_space(inode, delalloc_size);
+ if (ret) {
+ /* -ENOSPC */
+ ret = VM_FAULT_SIGBUS;
+ goto out_unlock;
+ }
+ reserved = 1;
+
/*
* XXX - page_mkwrite gets called every time the page is dirtied, even
* if it was already dirty, so for space accounting reasons we need to
@@ -7874,7 +7884,8 @@ again:
EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG,
0, 0, &cached_state, GFP_NOFS);
- ret = btrfs_set_extent_delalloc(inode, page_start, page_end,
+ ret = btrfs_set_extent_delalloc(inode, page_start,
+ page_start + delalloc_size - 1,
&cached_state);
if (ret) {
unlock_extent_cached(io_tree, page_start, page_end,
@@ -7913,8 +7924,8 @@ out_unlock:
}
unlock_page(page);
out:
- btrfs_delalloc_release_space(inode, PAGE_CACHE_SIZE);
-out_noreserve:
+ if (reserved)
+ btrfs_delalloc_release_space(inode, delalloc_size);
sb_end_pagefault(inode->i_sb);
return ret;
}
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 12/17] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (10 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 11/17] Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in " Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 13/17] Btrfs: subpagesize-blocksize: Deal with partial ordered extent allocations Chandan Rajendra
` (4 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In subpagesize-blocksize scenario it is not sufficient to search using the
first byte of the page to make sure that there are no ordered extents
present across the page. Fix this.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 3 ++-
fs/btrfs/inode.c | 6 +++---
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 3c33944..8ea21c1 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -3027,7 +3027,8 @@ static int __extent_read_full_page(struct extent_io_tree *tree,
while (1) {
lock_extent(tree, start, end);
- ordered = btrfs_lookup_ordered_extent(inode, start);
+ ordered = btrfs_lookup_ordered_range(inode, start,
+ PAGE_CACHE_SIZE);
if (!ordered)
break;
unlock_extent(tree, start, end);
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 23ce9ff..91c5580 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -1821,7 +1821,7 @@ again:
if (PagePrivate2(page))
goto out;
- ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ ordered = btrfs_lookup_ordered_range(inode, page_start, PAGE_CACHE_SIZE);
if (ordered) {
unlock_extent_cached(&BTRFS_I(inode)->io_tree, page_start,
page_end, &cached_state, GFP_NOFS);
@@ -7724,7 +7724,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
if (!inode_evicting)
lock_extent_bits(tree, page_start, page_end, 0, &cached_state);
- ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ ordered = btrfs_lookup_ordered_range(inode, page_start, PAGE_CACHE_SIZE);
if (ordered) {
/*
* IO on this page will never be started, so we need
@@ -7849,7 +7849,7 @@ again:
* we can't set the delalloc bits if there are pending ordered
* extents. Drop our locks and wait for them to finish
*/
- ordered = btrfs_lookup_ordered_extent(inode, page_start);
+ ordered = btrfs_lookup_ordered_range(inode, page_start, page_end);
if (ordered) {
unlock_extent_cached(io_tree, page_start, page_end,
&cached_state, GFP_NOFS);
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 13/17] Btrfs: subpagesize-blocksize: Deal with partial ordered extent allocations.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (11 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 12/17] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 14/17] Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of an ordered extent Chandan Rajendra
` (3 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In subpagesize-blocksize scenario, extent allocations for only some of the
dirty blocks of a page can succeed, while allocation for rest of the blocks
can fail. This patch allows I/O against such partially allocated ordered
extents to be submitted.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 24 +++++++++++++-----------
fs/btrfs/extent_io.h | 1 +
fs/btrfs/inode.c | 39 +++++++++++++++++++++++++--------------
3 files changed, 39 insertions(+), 25 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 8ea21c1..ccd9e1c 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -1774,15 +1774,22 @@ int extent_clear_unlock_delalloc(struct inode *inode, u64 start, u64 end,
if (page_ops & PAGE_SET_PRIVATE2)
SetPagePrivate2(pages[i]);
+ if (page_ops & PAGE_SET_ERROR)
+ SetPageError(pages[i]);
+
if (pages[i] == locked_page) {
page_cache_release(pages[i]);
continue;
}
- if (page_ops & PAGE_CLEAR_DIRTY)
+
+ if ((page_ops & PAGE_CLEAR_DIRTY)
+ && !PagePrivate2(pages[i]))
clear_page_dirty_for_io(pages[i]);
- if (page_ops & PAGE_SET_WRITEBACK)
+ if ((page_ops & PAGE_SET_WRITEBACK)
+ && !PagePrivate2(pages[i]))
set_page_writeback(pages[i]);
- if (page_ops & PAGE_END_WRITEBACK)
+ if ((page_ops & PAGE_END_WRITEBACK)
+ && !PagePrivate2(pages[i]))
end_page_writeback(pages[i]);
if (page_ops & PAGE_UNLOCK)
unlock_page(pages[i]);
@@ -2403,7 +2410,7 @@ int end_extent_writepage(struct page *page, int err, u64 start, u64 end)
uptodate = 0;
}
- if (!uptodate) {
+ if (!uptodate || PageError(page)) {
ClearPageUptodate(page);
SetPageError(page);
ret = ret < 0 ? ret : -EIO;
@@ -3123,7 +3130,6 @@ static noinline_for_stack int writepage_delalloc(struct inode *inode,
nr_written);
/* File system has been set read-only */
if (ret) {
- SetPageError(page);
/* fill_delalloc should be return < 0 for error
* but just in case, we use > 0 here meaning the
* IO is started, so we don't want to return > 0
@@ -3332,7 +3338,6 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc,
struct inode *inode = page->mapping->host;
struct extent_page_data *epd = data;
u64 start = page_offset(page);
- u64 page_end = start + PAGE_CACHE_SIZE - 1;
int ret;
int nr = 0;
size_t pg_offset;
@@ -3375,7 +3380,7 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc,
ret = writepage_delalloc(inode, page, wbc, epd, start, &nr_written);
if (ret == 1)
goto done_unlocked;
- if (ret)
+ if (ret && !PagePrivate2(page))
goto done;
ret = __extent_writepage_io(inode, page, wbc, epd,
@@ -3389,10 +3394,7 @@ done:
set_page_writeback(page);
end_page_writeback(page);
}
- if (PageError(page)) {
- ret = ret < 0 ? ret : -EIO;
- end_extent_writepage(page, ret, start, page_end);
- }
+
unlock_page(page);
return ret;
diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h
index 840e9a0..04ffd5b 100644
--- a/fs/btrfs/extent_io.h
+++ b/fs/btrfs/extent_io.h
@@ -51,6 +51,7 @@
#define PAGE_SET_WRITEBACK (1 << 2)
#define PAGE_END_WRITEBACK (1 << 3)
#define PAGE_SET_PRIVATE2 (1 << 4)
+#define PAGE_SET_ERROR (1 << 5)
/*
* page->private values. Every page that is controlled by the extent
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 91c5580..4ed78dd 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -880,6 +880,8 @@ static noinline int cow_file_range(struct inode *inode,
struct btrfs_key ins;
struct extent_map *em;
struct extent_map_tree *em_tree = &BTRFS_I(inode)->extent_tree;
+ struct btrfs_ordered_extent *ordered;
+ unsigned long page_ops, extent_ops;
int ret = 0;
if (btrfs_is_free_space_inode(inode)) {
@@ -924,8 +926,6 @@ static noinline int cow_file_range(struct inode *inode,
btrfs_drop_extent_cache(inode, start, start + num_bytes - 1, 0);
while (disk_num_bytes > 0) {
- unsigned long op;
-
cur_alloc_size = disk_num_bytes;
ret = btrfs_reserve_extent(root, cur_alloc_size,
root->sectorsize, 0, alloc_hint,
@@ -971,14 +971,14 @@ static noinline int cow_file_range(struct inode *inode,
ret = btrfs_add_ordered_extent(inode, start, ins.objectid,
ram_size, cur_alloc_size, 0);
if (ret)
- goto out_reserve;
+ goto out_remove_extent_map;
if (root->root_key.objectid ==
BTRFS_DATA_RELOC_TREE_OBJECTID) {
ret = btrfs_reloc_clone_csums(inode, start,
cur_alloc_size);
if (ret)
- goto out_reserve;
+ goto out_remove_ordered_extent;
}
if (disk_num_bytes < cur_alloc_size)
@@ -991,13 +991,12 @@ static noinline int cow_file_range(struct inode *inode,
* Do set the Private2 bit so we know this page was properly
* setup for writepage
*/
- op = unlock ? PAGE_UNLOCK : 0;
- op |= PAGE_SET_PRIVATE2;
-
+ page_ops = unlock ? PAGE_UNLOCK : 0;
+ page_ops |= PAGE_SET_PRIVATE2;
+ extent_ops = EXTENT_LOCKED | EXTENT_DELALLOC;
extent_clear_unlock_delalloc(inode, start,
- start + ram_size - 1, locked_page,
- EXTENT_LOCKED | EXTENT_DELALLOC,
- op);
+ start + ram_size - 1, locked_page,
+ extent_ops, page_ops);
disk_num_bytes -= cur_alloc_size;
num_bytes -= cur_alloc_size;
alloc_hint = ins.objectid + ins.offset;
@@ -1006,14 +1005,26 @@ static noinline int cow_file_range(struct inode *inode,
out:
return ret;
+out_remove_ordered_extent:
+ ordered = btrfs_lookup_ordered_extent(inode, ins.objectid);
+ BUG_ON(!ordered);
+ btrfs_remove_ordered_extent(inode, ordered);
+
+out_remove_extent_map:
+ btrfs_drop_extent_cache(inode, start, start + ram_size - 1, 0);
+
out_reserve:
btrfs_free_reserved_extent(root, ins.objectid, ins.offset, 1);
+
out_unlock:
+ page_ops = unlock ? PAGE_UNLOCK : 0;
+ page_ops |= PAGE_CLEAR_DIRTY | PAGE_SET_WRITEBACK | PAGE_END_WRITEBACK
+ | PAGE_SET_ERROR;
+ extent_ops = EXTENT_LOCKED | EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING
+ | EXTENT_DEFRAG;
+
extent_clear_unlock_delalloc(inode, start, end, locked_page,
- EXTENT_LOCKED | EXTENT_DO_ACCOUNTING |
- EXTENT_DELALLOC | EXTENT_DEFRAG,
- PAGE_UNLOCK | PAGE_CLEAR_DIRTY |
- PAGE_SET_WRITEBACK | PAGE_END_WRITEBACK);
+ extent_ops, page_ops);
goto out;
}
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 14/17] Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of an ordered extent.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (12 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 13/17] Btrfs: subpagesize-blocksize: Deal with partial ordered extent allocations Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 15/17] Btrfs: subpagesize-blocksize: Revert commit fc4adbff823f76577ece26dcb88bf6f8392dbd43 Chandan Rajendra
` (2 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In subpagesize-blocksize scenario a page can have more than one block. So
in addition to PagePrivate2 flag, we would have to track the I/O status of
each block of a page to reliably mark the ordered extent as complete.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 19 +--
fs/btrfs/extent_io.h | 5 +-
fs/btrfs/inode.c | 338 +++++++++++++++++++++++++++++++++++-------------
fs/btrfs/ordered-data.c | 17 +++
fs/btrfs/ordered-data.h | 4 +
5 files changed, 285 insertions(+), 98 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index ccd9e1c..168252e 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -4277,11 +4277,10 @@ int extent_invalidatepage(struct extent_io_tree *tree,
* to drop the page.
*/
static int try_release_extent_state(struct extent_map_tree *map,
- struct extent_io_tree *tree,
- struct page *page, gfp_t mask)
+ struct extent_io_tree *tree,
+ struct page *page, u64 start, u64 end,
+ gfp_t mask)
{
- u64 start = page_offset(page);
- u64 end = start + PAGE_CACHE_SIZE - 1;
int ret = 1;
if (test_range_bit(tree, start, end,
@@ -4315,12 +4314,12 @@ static int try_release_extent_state(struct extent_map_tree *map,
* map records are removed
*/
int try_release_extent_mapping(struct extent_map_tree *map,
- struct extent_io_tree *tree, struct page *page,
- gfp_t mask)
+ struct extent_io_tree *tree, struct page *page,
+ u64 start, u64 end, gfp_t mask)
{
struct extent_map *em;
- u64 start = page_offset(page);
- u64 end = start + PAGE_CACHE_SIZE - 1;
+ u64 orig_start = start;
+ u64 orig_end = end;
if ((mask & __GFP_WAIT) &&
page->mapping->host->i_size > 16 * 1024 * 1024) {
@@ -4354,7 +4353,9 @@ int try_release_extent_mapping(struct extent_map_tree *map,
free_extent_map(em);
}
}
- return try_release_extent_state(map, tree, page, mask);
+ return try_release_extent_state(map, tree, page,
+ orig_start, orig_end,
+ mask);
}
/*
diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h
index 04ffd5b..ec12455 100644
--- a/fs/btrfs/extent_io.h
+++ b/fs/btrfs/extent_io.h
@@ -208,8 +208,9 @@ typedef struct extent_map *(get_extent_t)(struct inode *inode,
void extent_io_tree_init(struct extent_io_tree *tree,
struct address_space *mapping);
int try_release_extent_mapping(struct extent_map_tree *map,
- struct extent_io_tree *tree, struct page *page,
- gfp_t mask);
+ struct extent_io_tree *tree, struct page *page,
+ u64 start, u64 end,
+ gfp_t mask);
int try_release_extent_buffer(struct page *page);
int lock_extent(struct extent_io_tree *tree, u64 start, u64 end);
int lock_extent_bits(struct extent_io_tree *tree, u64 start, u64 end,
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 4ed78dd..e0dd338 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -2827,51 +2827,115 @@ static void finish_ordered_fn(struct btrfs_work *work)
btrfs_finish_ordered_io(ordered_extent);
}
-static int btrfs_writepage_end_io_hook(struct page *page, u64 start, u64 end,
+static void mark_blks_io_complete(struct btrfs_ordered_extent *ordered,
+ u64 blk, u64 nr_blks, int uptodate)
+{
+ struct inode *inode = ordered->inode;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_workqueue *workers;
+ int done;
+
+ while (nr_blks--) {
+ if (test_and_set_bit(blk, ordered->blocks_done)) {
+ blk++;
+ continue;
+ }
+
+ done = btrfs_dec_test_ordered_pending(inode, &ordered,
+ ordered->file_offset
+ + (blk << inode->i_sb->s_blocksize_bits),
+ root->sectorsize,
+ uptodate);
+ if (done) {
+ btrfs_init_work(&ordered->work, finish_ordered_fn,
+ NULL, NULL);
+
+ ordered->work.func = finish_ordered_fn;
+ ordered->work.flags = 0;
+
+ if (btrfs_is_free_space_inode(inode))
+ workers = root->fs_info->endio_freespace_worker;
+ else
+ workers = root->fs_info->endio_write_workers;
+
+ btrfs_queue_work(workers, &ordered->work);
+ }
+
+ blk++;
+ }
+}
+
+int btrfs_writepage_end_io_hook(struct page *page, u64 start, u64 end,
struct extent_state *state, int uptodate)
{
struct inode *inode = page->mapping->host;
struct btrfs_root *root = BTRFS_I(inode)->root;
struct btrfs_ordered_extent *ordered_extent = NULL;
- struct btrfs_workqueue *workers;
- u64 ordered_start, ordered_end;
- int done;
+ u64 blk, nr_blks;
+ int clear;
trace_btrfs_writepage_end_io_hook(page, start, end, uptodate);
- ClearPagePrivate2(page);
-loop:
- ordered_extent = btrfs_lookup_ordered_range(inode, start,
- end - start + 1);
- if (!ordered_extent)
- goto out;
+ while (start < end) {
+ ordered_extent = btrfs_lookup_ordered_extent(inode, start);
+ if (!ordered_extent) {
+ start += root->sectorsize;
+ continue;
+ }
- ordered_start = max_t(u64, start, ordered_extent->file_offset);
- ordered_end = min_t(u64, end,
- ordered_extent->file_offset + ordered_extent->len - 1);
+ blk = (start - ordered_extent->file_offset)
+ >> inode->i_sb->s_blocksize_bits;
- done = btrfs_dec_test_ordered_pending(inode, &ordered_extent,
- ordered_start,
- ordered_end - ordered_start + 1,
- uptodate);
- if (done) {
- btrfs_init_work(&ordered_extent->work, finish_ordered_fn, NULL, NULL);
+ nr_blks = (min(end, ordered_extent->file_offset + ordered_extent->len - 1)
+ + 1 - start) >> inode->i_sb->s_blocksize_bits;
- if (btrfs_is_free_space_inode(inode))
- workers = root->fs_info->endio_freespace_worker;
- else
- workers = root->fs_info->endio_write_workers;
+ BUG_ON(!nr_blks);
+
+ mark_blks_io_complete(ordered_extent, blk, nr_blks, uptodate);
- btrfs_queue_work(workers, &ordered_extent->work);
+ start = ordered_extent->file_offset + ordered_extent->len;
+
+ btrfs_put_ordered_extent(ordered_extent);
}
- btrfs_put_ordered_extent(ordered_extent);
+ start = page_offset(page);
+ end = start + PAGE_CACHE_SIZE - 1;
+ clear = 1;
+
+ while (start < end) {
+ ordered_extent = btrfs_lookup_ordered_extent(inode, start);
+ if (!ordered_extent) {
+ start += root->sectorsize;
+ continue;
+ }
- start = ordered_end + 1;
+ blk = (start - ordered_extent->file_offset)
+ >> inode->i_sb->s_blocksize_bits;
+ nr_blks = (min(end, ordered_extent->file_offset + ordered_extent->len - 1)
+ + 1 - start) >> inode->i_sb->s_blocksize_bits;
+
+ BUG_ON(!nr_blks);
+
+ while (nr_blks--) {
+ if (!test_bit(blk++, ordered_extent->blocks_done)) {
+ clear = 0;
+ break;
+ }
+ }
+
+ if (!clear) {
+ btrfs_put_ordered_extent(ordered_extent);
+ break;
+ }
+
+ start += ordered_extent->len;
+
+ btrfs_put_ordered_extent(ordered_extent);
+ }
+
+ if (clear)
+ ClearPagePrivate2(page);
- if (start < end)
- goto loop;
-out:
return 0;
}
@@ -7683,7 +7747,9 @@ btrfs_readpages(struct file *file, struct address_space *mapping,
return extent_readpages(tree, mapping, pages, nr_pages,
btrfs_get_extent);
}
-static int __btrfs_releasepage(struct page *page, gfp_t gfp_flags)
+
+static int __btrfs_releasepage(struct page *page, u64 start, u64 end,
+ gfp_t gfp_flags)
{
struct extent_io_tree *tree;
struct extent_map_tree *map;
@@ -7691,33 +7757,150 @@ static int __btrfs_releasepage(struct page *page, gfp_t gfp_flags)
tree = &BTRFS_I(page->mapping->host)->io_tree;
map = &BTRFS_I(page->mapping->host)->extent_tree;
- ret = try_release_extent_mapping(map, tree, page, gfp_flags);
- if (ret == 1) {
+ ret = try_release_extent_mapping(map, tree, page, start, end,
+ gfp_flags);
+ if ((ret == 1) && ((end - start + 1) == PAGE_CACHE_SIZE)) {
ClearPagePrivate(page);
set_page_private(page, 0);
page_cache_release(page);
+ } else {
+ ret = 0;
}
+
return ret;
}
static int btrfs_releasepage(struct page *page, gfp_t gfp_flags)
{
+ u64 start = page_offset(page);
+ u64 end = start + PAGE_CACHE_SIZE - 1;
+
if (PageWriteback(page) || PageDirty(page))
return 0;
- return __btrfs_releasepage(page, gfp_flags & GFP_NOFS);
+
+ return __btrfs_releasepage(page, start, end, gfp_flags & GFP_NOFS);
+}
+
+static void invalidate_ordered_extent_blocks(struct inode *inode,
+ struct btrfs_ordered_extent *ordered,
+ u64 locked_start, u64 locked_end,
+ u64 cur,
+ int inode_evicting)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct btrfs_ordered_inode_tree *ordered_tree;
+ struct extent_io_tree *tree;
+ u64 blk, blk_done, nr_blks;
+ u64 end;
+ u64 new_len;
+
+ tree = &BTRFS_I(inode)->io_tree;
+
+ end = min(locked_end, ordered->file_offset + ordered->len - 1);
+
+ if (!inode_evicting) {
+ clear_extent_bit(tree, cur, end,
+ EXTENT_DIRTY | EXTENT_DELALLOC |
+ EXTENT_DO_ACCOUNTING |
+ EXTENT_DEFRAG, 1, 0, NULL,
+ GFP_NOFS);
+ unlock_extent(tree, locked_start, locked_end);
+ }
+
+
+ ordered_tree = &BTRFS_I(inode)->ordered_tree;
+ spin_lock_irq(&ordered_tree->lock);
+ set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
+ new_len = cur - ordered->file_offset;
+ if (new_len < ordered->truncated_len)
+ ordered->truncated_len = new_len;
+
+ blk = (cur - ordered->file_offset) >> inode->i_sb->s_blocksize_bits;
+ nr_blks = (end + 1 - cur) >> inode->i_sb->s_blocksize_bits;
+
+ while (nr_blks--) {
+ blk_done = !test_and_set_bit(blk, ordered->blocks_done);
+ if (blk_done) {
+ spin_unlock_irq(&ordered_tree->lock);
+ if (btrfs_dec_test_ordered_pending(inode, &ordered,
+ ordered->file_offset + (blk << inode->i_sb->s_blocksize_bits),
+ root->sectorsize,
+ 1))
+ btrfs_finish_ordered_io(ordered);
+
+ spin_lock_irq(&ordered_tree->lock);
+ }
+ blk++;
+ }
+
+ spin_unlock_irq(&ordered_tree->lock);
+
+ if (!inode_evicting)
+ lock_extent_bits(tree, locked_start, locked_end, 0, NULL);
+}
+
+static int page_blocks_written(struct page *page)
+{
+ struct btrfs_ordered_extent *ordered;
+ struct btrfs_root *root;
+ struct inode *inode;
+ unsigned long outstanding_blk;
+ u64 page_start, page_end;
+ u64 blk, last_blk, nr_blks;
+ u64 cur;
+ u64 len;
+
+ inode = page->mapping->host;
+ root = BTRFS_I(inode)->root;
+
+ page_start = page_offset(page);
+ page_end = page_start + PAGE_CACHE_SIZE - 1;
+
+ cur = page_start;
+ while (cur < page_end) {
+ ordered = btrfs_lookup_ordered_extent(inode, cur);
+ if (!ordered) {
+ cur += root->sectorsize;
+ continue;
+ }
+
+ blk = (cur - ordered->file_offset)
+ >> inode->i_sb->s_blocksize_bits;
+ len = min(page_end, ordered->file_offset + ordered->len - 1)
+ - cur + 1;
+ nr_blks = len >> inode->i_sb->s_blocksize_bits;
+
+ last_blk = blk + nr_blks - 1;
+
+ outstanding_blk = find_next_zero_bit(ordered->blocks_done,
+ ordered->len >> inode->i_sb->s_blocksize_bits,
+ blk);
+ if (outstanding_blk <= last_blk) {
+ btrfs_put_ordered_extent(ordered);
+ return 0;
+ }
+
+ btrfs_put_ordered_extent(ordered);
+ cur += len;
+ }
+
+ return 1;
}
static void btrfs_invalidatepage(struct page *page, unsigned int offset,
- unsigned int length)
+ unsigned int length)
{
struct inode *inode = page->mapping->host;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
struct extent_io_tree *tree;
struct btrfs_ordered_extent *ordered;
- struct extent_state *cached_state = NULL;
- u64 page_start = page_offset(page);
- u64 page_end = page_start + PAGE_CACHE_SIZE - 1;
+ u64 start, end, cur;
+ u64 page_start, page_end;
int inode_evicting = inode->i_state & I_FREEING;
+ page_start = page_offset(page);
+ page_end = page_start + PAGE_CACHE_SIZE - 1;
+
/*
* we have the page locked, so new writeback can't start,
* and the dirty bit won't be cleared while we are here.
@@ -7728,73 +7911,54 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
wait_on_page_writeback(page);
tree = &BTRFS_I(inode)->io_tree;
- if (offset) {
+
+ start = round_up(offset, root->sectorsize);
+ end = round_down(offset + length, root->sectorsize) - 1;
+ if (end - start + 1 < root->sectorsize) {
btrfs_releasepage(page, GFP_NOFS);
return;
}
+ start = round_up(page_start + offset, root->sectorsize);
+ end = round_down(page_start + offset + length,
+ root->sectorsize) - 1;
+
if (!inode_evicting)
- lock_extent_bits(tree, page_start, page_end, 0, &cached_state);
- ordered = btrfs_lookup_ordered_range(inode, page_start, PAGE_CACHE_SIZE);
- if (ordered) {
- /*
- * IO on this page will never be started, so we need
- * to account for any ordered extents now
- */
- if (!inode_evicting)
- clear_extent_bit(tree, page_start, page_end,
- EXTENT_DIRTY | EXTENT_DELALLOC |
- EXTENT_LOCKED | EXTENT_DO_ACCOUNTING |
- EXTENT_DEFRAG, 1, 0, &cached_state,
- GFP_NOFS);
- /*
- * whoever cleared the private bit is responsible
- * for the finish_ordered_io
- */
- if (TestClearPagePrivate2(page)) {
- struct btrfs_ordered_inode_tree *tree;
- u64 new_len;
+ lock_extent_bits(tree, start, end, 0, NULL);
- tree = &BTRFS_I(inode)->ordered_tree;
+ cur = start;
+ while (cur < end) {
+ ordered = btrfs_lookup_ordered_extent(inode, cur);
+ if (!ordered) {
+ cur += root->sectorsize;
+ continue;
+ }
- spin_lock_irq(&tree->lock);
- set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
- new_len = page_start - ordered->file_offset;
- if (new_len < ordered->truncated_len)
- ordered->truncated_len = new_len;
- spin_unlock_irq(&tree->lock);
+ invalidate_ordered_extent_blocks(inode, ordered,
+ start, end, cur,
+ inode_evicting);
- if (btrfs_dec_test_ordered_pending(inode, &ordered,
- page_start,
- PAGE_CACHE_SIZE, 1))
- btrfs_finish_ordered_io(ordered);
- }
+ cur = min(end + 1, ordered->file_offset + ordered->len);
btrfs_put_ordered_extent(ordered);
- if (!inode_evicting) {
- cached_state = NULL;
- lock_extent_bits(tree, page_start, page_end, 0,
- &cached_state);
- }
}
- if (!inode_evicting) {
- clear_extent_bit(tree, page_start, page_end,
- EXTENT_LOCKED | EXTENT_DIRTY |
- EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING |
- EXTENT_DEFRAG, 1, 1,
- &cached_state, GFP_NOFS);
+ if (page_blocks_written(page))
+ ClearPagePrivate2(page);
- __btrfs_releasepage(page, GFP_NOFS);
+ if (!inode_evicting) {
+ clear_extent_bit(tree, start, end,
+ EXTENT_LOCKED | EXTENT_DIRTY |
+ EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING |
+ EXTENT_DEFRAG, 1, 1, NULL, GFP_NOFS);
}
- ClearPageChecked(page);
- if (PagePrivate(page)) {
- ClearPagePrivate(page);
- set_page_private(page, 0);
- page_cache_release(page);
+ if (!offset && length == PAGE_CACHE_SIZE) {
+ WARN_ON(!__btrfs_releasepage(page, start, end, GFP_NOFS));
+ ClearPageChecked(page);
}
}
+
/*
* btrfs_page_mkwrite() is not allowed to change the file size as it gets
* called from a page fault handler when a page is first dirtied. Hence we must
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 963895c..4d9832f 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -189,12 +189,25 @@ static int __btrfs_add_ordered_extent(struct inode *inode, u64 file_offset,
struct btrfs_ordered_inode_tree *tree;
struct rb_node *node;
struct btrfs_ordered_extent *entry;
+ u64 nr_longs;
tree = &BTRFS_I(inode)->ordered_tree;
entry = kmem_cache_zalloc(btrfs_ordered_extent_cache, GFP_NOFS);
if (!entry)
return -ENOMEM;
+ nr_longs = BITS_TO_LONGS(len >> inode->i_sb->s_blocksize_bits);
+ if (nr_longs == 1) {
+ entry->blocks_done = &entry->blocks_bitmap;
+ } else {
+ entry->blocks_done = kzalloc(nr_longs * sizeof(unsigned long),
+ GFP_NOFS);
+ if (!entry->blocks_done) {
+ kmem_cache_free(btrfs_ordered_extent_cache, entry);
+ return -ENOMEM;
+ }
+ }
+
entry->file_offset = file_offset;
entry->start = start;
entry->len = len;
@@ -541,6 +554,10 @@ void btrfs_put_ordered_extent(struct btrfs_ordered_extent *entry)
list_del(&sum->list);
kfree(sum);
}
+
+ if (entry->blocks_done != &entry->blocks_bitmap)
+ kfree(entry->blocks_done);
+
kmem_cache_free(btrfs_ordered_extent_cache, entry);
}
}
diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs/ordered-data.h
index d81a274..7de3b1e 100644
--- a/fs/btrfs/ordered-data.h
+++ b/fs/btrfs/ordered-data.h
@@ -135,6 +135,10 @@ struct btrfs_ordered_extent {
struct completion completion;
struct btrfs_work flush_work;
struct list_head work_list;
+
+ /* bitmap to track the blocks that have been written to disk */
+ unsigned long *blocks_done;
+ unsigned long blocks_bitmap;
};
/*
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 15/17] Btrfs: subpagesize-blocksize: Revert commit fc4adbff823f76577ece26dcb88bf6f8392dbd43.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (13 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 14/17] Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of an ordered extent Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 16/17] Btrfs: subpagesize-blocksize: Track blocks of ordered extent submitted for write I/O Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 17/17] Btrfs: subpagesize-blocksize: Prevent writes to an extent buffer when PG_writeback flag is set Chandan Rajendra
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In subpagesize-blocksize, we have multiple blocks in a page. Checking for
existence of a page in the page cache isn't a sufficient check, since we
could be truncating a subset of the blocks mapped by the page.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/btrfs_inode.h | 2 --
fs/btrfs/file.c | 4 ++-
fs/btrfs/inode.c | 77 +++-----------------------------------------------
3 files changed, 7 insertions(+), 76 deletions(-)
diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h
index 43527fd..50497bf 100644
--- a/fs/btrfs/btrfs_inode.h
+++ b/fs/btrfs/btrfs_inode.h
@@ -278,6 +278,4 @@ static inline void btrfs_inode_resume_unlocked_dio(struct inode *inode)
&BTRFS_I(inode)->runtime_flags);
}
-bool btrfs_page_exists_in_range(struct inode *inode, loff_t start, loff_t end);
-
#endif
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index b1e0d27..3707515 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -2314,7 +2314,9 @@ static int btrfs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
if ((!ordered ||
(ordered->file_offset + ordered->len <= lockstart ||
ordered->file_offset > lockend)) &&
- !btrfs_page_exists_in_range(inode, lockstart, lockend)) {
+ !test_range_bit(&BTRFS_I(inode)->io_tree, lockstart,
+ lockend, EXTENT_UPTODATE, 0,
+ cached_state)) {
if (ordered)
btrfs_put_ordered_extent(ordered);
break;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index e0dd338..b236417 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -6832,76 +6832,6 @@ out:
return ret;
}
-bool btrfs_page_exists_in_range(struct inode *inode, loff_t start, loff_t end)
-{
- struct radix_tree_root *root = &inode->i_mapping->page_tree;
- int found = false;
- void **pagep = NULL;
- struct page *page = NULL;
- int start_idx;
- int end_idx;
-
- start_idx = start >> PAGE_CACHE_SHIFT;
-
- /*
- * end is the last byte in the last page. end == start is legal
- */
- end_idx = end >> PAGE_CACHE_SHIFT;
-
- rcu_read_lock();
-
- /* Most of the code in this while loop is lifted from
- * find_get_page. It's been modified to begin searching from a
- * page and return just the first page found in that range. If the
- * found idx is less than or equal to the end idx then we know that
- * a page exists. If no pages are found or if those pages are
- * outside of the range then we're fine (yay!) */
- while (page == NULL &&
- radix_tree_gang_lookup_slot(root, &pagep, NULL, start_idx, 1)) {
- page = radix_tree_deref_slot(pagep);
- if (unlikely(!page))
- break;
-
- if (radix_tree_exception(page)) {
- if (radix_tree_deref_retry(page)) {
- page = NULL;
- continue;
- }
- /*
- * Otherwise, shmem/tmpfs must be storing a swap entry
- * here as an exceptional entry: so return it without
- * attempting to raise page count.
- */
- page = NULL;
- break; /* TODO: Is this relevant for this use case? */
- }
-
- if (!page_cache_get_speculative(page)) {
- page = NULL;
- continue;
- }
-
- /*
- * Has the page moved?
- * This is part of the lockless pagecache protocol. See
- * include/linux/pagemap.h for details.
- */
- if (unlikely(page != *pagep)) {
- page_cache_release(page);
- page = NULL;
- }
- }
-
- if (page) {
- if (page->index <= end_idx)
- found = true;
- page_cache_release(page);
- }
-
- rcu_read_unlock();
- return found;
-}
-
static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
struct extent_state **cached_state, int writing)
{
@@ -6926,9 +6856,10 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
* invalidate needs to happen so that reads after a write do not
* get stale data.
*/
- if (!ordered &&
- (!writing ||
- !btrfs_page_exists_in_range(inode, lockstart, lockend)))
+ if (!ordered && (!writing ||
+ !test_range_bit(&BTRFS_I(inode)->io_tree,
+ lockstart, lockend, EXTENT_UPTODATE, 0,
+ *cached_state)))
break;
unlock_extent_cached(&BTRFS_I(inode)->io_tree, lockstart, lockend,
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 16/17] Btrfs: subpagesize-blocksize: Track blocks of ordered extent submitted for write I/O.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (14 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 15/17] Btrfs: subpagesize-blocksize: Revert commit fc4adbff823f76577ece26dcb88bf6f8392dbd43 Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 17/17] Btrfs: subpagesize-blocksize: Prevent writes to an extent buffer when PG_writeback flag is set Chandan Rajendra
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In the subpagesize-blocksize scenario, the following command (with 4k as the
PAGE_SIZE and 2k as the block size) can cause false accounting of blocks of an
ordered extent that is written to disk:
$ xfs_io -f -c "pwrite 0 10240" \
-c "sync_range 0 4096" \
-c "sync_range 8192 2048" \
-c "pwrite 10240 2048" \
-c "sync_range 10240 2048" \
/mnt/btrfs/file.bin
To fix this, we would have to explicitly track the blocks of an ordered extent
that have already been submitted for write I/O.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/extent_io.c | 24 ++++++++++++++++++++++--
fs/btrfs/ordered-data.c | 4 +++-
fs/btrfs/ordered-data.h | 4 ++++
3 files changed, 29 insertions(+), 3 deletions(-)
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 168252e..3649c5d 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -3201,6 +3201,8 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
u64 extent_offset;
u64 extent_end;
u64 iosize;
+ u64 blk, nr_blks;
+ u64 blk_submitted;
sector_t sector;
struct extent_state *cached_state = NULL;
struct block_device *bdev;
@@ -3267,11 +3269,26 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
iosize = min(extent_end - cur, end - cur + 1);
iosize = ALIGN(iosize, blocksize);
+ blk = extent_offset >> inode->i_sb->s_blocksize_bits;
+ nr_blks = iosize >> inode->i_sb->s_blocksize_bits;
+
+ blk_submitted = find_next_bit(ordered->blocks_submitted,
+ ordered->len >> inode->i_sb->s_blocksize_bits,
+ blk);
+ if (blk_submitted < blk + nr_blks) {
+ if (blk_submitted == blk) {
+ cur += blocksize;
+ btrfs_put_ordered_extent(ordered);
+ continue;
+ }
+ iosize = (blk_submitted - blk)
+ << inode->i_sb->s_blocksize_bits;
+ nr_blks = iosize >> inode->i_sb->s_blocksize_bits;
+ }
+
sector = (ordered->start + extent_offset) >> 9;
bdev = BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev;
compressed = test_bit(BTRFS_ORDERED_COMPRESSED, &ordered->flags);
- btrfs_put_ordered_extent(ordered);
- ordered = NULL;
/*
* compressed and inline extents are written through other
@@ -3284,6 +3301,7 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
*/
nr++;
cur += iosize;
+ btrfs_put_ordered_extent(ordered);
continue;
}
@@ -3298,6 +3316,8 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode,
} else {
unsigned long max_nr = (i_size >> PAGE_CACHE_SHIFT) + 1;
+ bitmap_set(ordered->blocks_submitted, blk, nr_blks);
+ btrfs_put_ordered_extent(ordered);
set_range_writeback(tree, cur, cur + iosize - 1);
if (!PageWriteback(page)) {
btrfs_err(BTRFS_I(inode)->root->fs_info,
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 4d9832f..59b2544 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -199,13 +199,15 @@ static int __btrfs_add_ordered_extent(struct inode *inode, u64 file_offset,
nr_longs = BITS_TO_LONGS(len >> inode->i_sb->s_blocksize_bits);
if (nr_longs == 1) {
entry->blocks_done = &entry->blocks_bitmap;
+ entry->blocks_submitted = &entry->blocks_submitted_bitmap;
} else {
- entry->blocks_done = kzalloc(nr_longs * sizeof(unsigned long),
+ entry->blocks_done = kzalloc(2 * nr_longs * sizeof(unsigned long),
GFP_NOFS);
if (!entry->blocks_done) {
kmem_cache_free(btrfs_ordered_extent_cache, entry);
return -ENOMEM;
}
+ entry->blocks_submitted = entry->blocks_done + nr_longs;
}
entry->file_offset = file_offset;
diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs/ordered-data.h
index 7de3b1e..851914c 100644
--- a/fs/btrfs/ordered-data.h
+++ b/fs/btrfs/ordered-data.h
@@ -139,6 +139,10 @@ struct btrfs_ordered_extent {
/* bitmap to track the blocks that have been written to disk */
unsigned long *blocks_done;
unsigned long blocks_bitmap;
+
+ /* bitmap to track the blocks that have been submitted for write i/o */
+ unsigned long *blocks_submitted;
+ unsigned long blocks_submitted_bitmap;
};
/*
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [RFC PATCH V9 17/17] Btrfs: subpagesize-blocksize: Prevent writes to an extent buffer when PG_writeback flag is set.
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
` (15 preceding siblings ...)
2014-11-14 15:38 ` [RFC PATCH V9 16/17] Btrfs: subpagesize-blocksize: Track blocks of ordered extent submitted for write I/O Chandan Rajendra
@ 2014-11-14 15:38 ` Chandan Rajendra
16 siblings, 0 replies; 18+ messages in thread
From: Chandan Rajendra @ 2014-11-14 15:38 UTC (permalink / raw)
To: clm, jbacik, bo.li.liu, dsterba
Cc: Chandan Rajendra, aneesh.kumar, linux-btrfs, chandan, steve.capper
In non-subpagesize-blocksize scenario, BTRFS_HEADER_FLAG_WRITTEN flag prevents
Btrfs code from writing into an extent buffer whose pages are under
writeback. This facility isn't sufficient for achieving the same in
subpagesize-blocksize scenario, since we have more than one extent buffer
mapped to a page.
Hence this patch adds a new flag (i.e. EXTENT_BUFFER_HEAD_WRITEBACK) and
corresponding code to track the writeback status of the page and to prevent
writes to any of the extent buffers mapped to the page while writeback is
going on.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
---
fs/btrfs/ctree.c | 20 ++++++-
fs/btrfs/extent-tree.c | 12 ++++
fs/btrfs/extent_io.c | 153 +++++++++++++++++++++++++++++++++++++++----------
fs/btrfs/extent_io.h | 2 +
4 files changed, 157 insertions(+), 30 deletions(-)
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 693b541..75129da 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -1543,6 +1543,7 @@ noinline int btrfs_cow_block(struct btrfs_trans_handle *trans,
struct extent_buffer *parent, int parent_slot,
struct extent_buffer **cow_ret)
{
+ struct extent_buffer_head *ebh = eb_head(buf);
u64 search_start;
int ret;
@@ -1556,6 +1557,13 @@ noinline int btrfs_cow_block(struct btrfs_trans_handle *trans,
trans->transid, root->fs_info->generation);
if (!should_cow_block(trans, root, buf)) {
+ if (test_bit(EXTENT_BUFFER_HEAD_WRITEBACK, &ebh->bflags)) {
+ if (parent)
+ btrfs_set_lock_blocking(parent);
+ btrfs_set_lock_blocking(buf);
+ wait_on_bit(&ebh->bflags, EXTENT_BUFFER_HEAD_WRITEBACK,
+ eb_wait, TASK_UNINTERRUPTIBLE);
+ }
*cow_ret = buf;
return 0;
}
@@ -2687,6 +2695,7 @@ int btrfs_search_slot(struct btrfs_trans_handle *trans, struct btrfs_root
*root, struct btrfs_key *key, struct btrfs_path *p, int
ins_len, int cow)
{
+ struct extent_buffer_head *ebh;
struct extent_buffer *b;
int slot;
int ret;
@@ -2789,8 +2798,17 @@ again:
* then we don't want to set the path blocking,
* so we test it here
*/
- if (!should_cow_block(trans, root, b))
+ if (!should_cow_block(trans, root, b)) {
+ ebh = eb_head(b);
+ if (test_bit(EXTENT_BUFFER_HEAD_WRITEBACK,
+ &ebh->bflags)) {
+ btrfs_set_path_blocking(p);
+ wait_on_bit(&ebh->bflags,
+ EXTENT_BUFFER_HEAD_WRITEBACK,
+ eb_wait, TASK_UNINTERRUPTIBLE);
+ }
goto cow_done;
+ }
btrfs_set_path_blocking(p);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index fbcad82..fb5cc46 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -7203,14 +7203,26 @@ static struct extent_buffer *
btrfs_init_new_buffer(struct btrfs_trans_handle *trans, struct btrfs_root *root,
u64 bytenr, u32 blocksize, int level)
{
+ struct extent_buffer_head *ebh;
struct extent_buffer *buf;
buf = btrfs_find_create_tree_block(root, bytenr, blocksize);
if (!buf)
return ERR_PTR(-ENOMEM);
+
+ ebh = eb_head(buf);
btrfs_set_header_generation(buf, trans->transid);
btrfs_set_buffer_lockdep_class(root->root_key.objectid, buf, level);
btrfs_tree_lock(buf);
+
+ if (test_bit(EXTENT_BUFFER_HEAD_WRITEBACK,
+ &ebh->bflags)) {
+ btrfs_set_lock_blocking(buf);
+ wait_on_bit(&ebh->bflags,
+ EXTENT_BUFFER_HEAD_WRITEBACK,
+ eb_wait, TASK_UNINTERRUPTIBLE);
+ }
+
clean_tree_block(trans, root, buf);
clear_bit(EXTENT_BUFFER_STALE, &buf->ebflags);
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 3649c5d..ff51cfa 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -3422,7 +3422,7 @@ done_unlocked:
return 0;
}
-static int eb_wait(void *word)
+int eb_wait(void *word)
{
io_schedule();
return 0;
@@ -3434,6 +3434,52 @@ void wait_on_extent_buffer_writeback(struct extent_buffer *eb)
TASK_UNINTERRUPTIBLE);
}
+static void lock_extent_buffers(struct extent_buffer_head *ebh,
+ struct extent_page_data *epd)
+{
+ struct extent_buffer *locked_eb = NULL;
+ struct extent_buffer *eb;
+again:
+ eb = &ebh->eb;
+ do {
+ if (eb == locked_eb)
+ continue;
+
+ if (!btrfs_try_tree_write_lock(eb))
+ goto backoff;
+
+ } while ((eb = eb->eb_next) != NULL);
+
+ return;
+
+backoff:
+ if (locked_eb && (locked_eb->start > eb->start))
+ btrfs_tree_unlock(locked_eb);
+
+ locked_eb = eb;
+
+ eb = &ebh->eb;
+ while (eb != locked_eb) {
+ btrfs_tree_unlock(eb);
+ eb = eb->eb_next;
+ }
+
+ flush_write_bio(epd);
+
+ btrfs_tree_lock(locked_eb);
+
+ goto again;
+}
+
+static void unlock_extent_buffers(struct extent_buffer_head *ebh)
+{
+ struct extent_buffer *eb = &ebh->eb;
+
+ do {
+ btrfs_tree_unlock(eb);
+ } while ((eb = eb->eb_next) != NULL);
+}
+
static void lock_extent_buffer_pages(struct extent_buffer_head *ebh,
struct extent_page_data *epd)
{
@@ -3454,21 +3500,17 @@ static void lock_extent_buffer_pages(struct extent_buffer_head *ebh,
}
static int noinline_for_stack
-lock_extent_buffer_for_io(struct extent_buffer *eb,
+mark_extent_buffer_writeback(struct extent_buffer *eb,
struct btrfs_fs_info *fs_info,
struct extent_page_data *epd)
{
+ struct extent_buffer_head *ebh = eb_head(eb);
+ struct extent_buffer *cur;
int dirty;
int ret = 0;
- if (!btrfs_try_tree_write_lock(eb)) {
- flush_write_bio(epd);
- btrfs_tree_lock(eb);
- }
-
if (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags)) {
dirty = test_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags);
- btrfs_tree_unlock(eb);
if (!epd->sync_io) {
if (!dirty)
return 1;
@@ -3476,15 +3518,23 @@ lock_extent_buffer_for_io(struct extent_buffer *eb,
return 2;
}
+ cur = &ebh->eb;
+ do {
+ btrfs_set_lock_blocking(cur);
+ } while ((cur = cur->eb_next) != NULL);
+
flush_write_bio(epd);
while (1) {
wait_on_extent_buffer_writeback(eb);
- btrfs_tree_lock(eb);
if (!test_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags))
break;
- btrfs_tree_unlock(eb);
}
+
+ cur = &ebh->eb;
+ do {
+ btrfs_clear_lock_blocking(cur);
+ } while ((cur = cur->eb_next) != NULL);
}
/*
@@ -3492,22 +3542,20 @@ lock_extent_buffer_for_io(struct extent_buffer *eb,
* under IO since we can end up having no IO bits set for a short period
* of time.
*/
- spin_lock(&eb_head(eb)->refs_lock);
+ spin_lock(&ebh->refs_lock);
if (test_and_clear_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags)) {
set_bit(EXTENT_BUFFER_WRITEBACK, &eb->ebflags);
- spin_unlock(&eb_head(eb)->refs_lock);
+ spin_unlock(&ebh->refs_lock);
btrfs_set_header_flag(eb, BTRFS_HEADER_FLAG_WRITTEN);
__percpu_counter_add(&fs_info->dirty_metadata_bytes,
-eb->len,
fs_info->dirty_metadata_batch);
ret = 0;
} else {
- spin_unlock(&eb_head(eb)->refs_lock);
+ spin_unlock(&ebh->refs_lock);
ret = 1;
}
- btrfs_tree_unlock(eb);
-
return ret;
}
@@ -3607,8 +3655,8 @@ static void end_extent_buffer_writeback(struct extent_buffer *eb)
static void end_bio_subpagesize_blocksize_ebh_writepage(struct bio *bio, int err)
{
- struct bio_vec *bvec;
struct extent_buffer *eb;
+ struct bio_vec *bvec;
int i, done;
bio_for_each_segment_all(bvec, bio, i) {
@@ -3636,9 +3684,17 @@ static void end_bio_subpagesize_blocksize_ebh_writepage(struct bio *bio, int err
end_extent_buffer_writeback(eb);
- if (done)
+ if (done) {
+ struct extent_buffer_head *ebh = eb_head(eb);
+
end_page_writeback(page);
+ clear_bit(EXTENT_BUFFER_HEAD_WRITEBACK,
+ &ebh->bflags);
+ smp_mb__after_atomic();
+ wake_up_bit(&ebh->bflags,
+ EXTENT_BUFFER_HEAD_WRITEBACK);
+ }
} while ((eb = eb->eb_next) != NULL);
}
@@ -3648,6 +3704,7 @@ static void end_bio_subpagesize_blocksize_ebh_writepage(struct bio *bio, int err
static void end_bio_regular_ebh_writepage(struct bio *bio, int err)
{
+ struct extent_buffer_head *ebh;
struct extent_buffer *eb;
struct bio_vec *bvec;
int i, done;
@@ -3657,6 +3714,8 @@ static void end_bio_regular_ebh_writepage(struct bio *bio, int err)
eb = (struct extent_buffer *)page->private;
BUG_ON(!eb);
+ ebh = eb_head(eb);
+
done = atomic_dec_and_test(&eb_head(eb)->io_bvecs);
if (err || test_bit(EXTENT_BUFFER_IOERR, &eb->ebflags)) {
@@ -3671,6 +3730,10 @@ static void end_bio_regular_ebh_writepage(struct bio *bio, int err)
continue;
end_extent_buffer_writeback(eb);
+
+ clear_bit(EXTENT_BUFFER_HEAD_WRITEBACK, &ebh->bflags);
+ smp_mb__after_atomic();
+ wake_up_bit(&ebh->bflags, EXTENT_BUFFER_HEAD_WRITEBACK);
}
bio_put(bio);
@@ -3712,8 +3775,14 @@ write_regular_ebh(struct extent_buffer_head *ebh,
set_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
SetPageError(p);
if (atomic_sub_and_test(num_pages - i,
- &eb_head(eb)->io_bvecs))
+ &ebh->io_bvecs)) {
end_extent_buffer_writeback(eb);
+ clear_bit(EXTENT_BUFFER_HEAD_WRITEBACK,
+ &ebh->bflags);
+ smp_mb__after_atomic();
+ wake_up_bit(&ebh->bflags,
+ EXTENT_BUFFER_HEAD_WRITEBACK);
+ }
ret = -EIO;
break;
}
@@ -3746,6 +3815,7 @@ static int write_subpagesize_blocksize_ebh(struct extent_buffer_head *ebh,
unsigned long i;
unsigned long bio_flags = 0;
int rw = (epd->sync_io ? WRITE_SYNC : WRITE) | REQ_META;
+ int nr_eb_submitted = 0;
int ret = 0, err = 0;
eb = &ebh->eb;
@@ -3758,7 +3828,7 @@ static int write_subpagesize_blocksize_ebh(struct extent_buffer_head *ebh,
continue;
clear_bit(EXTENT_BUFFER_IOERR, &eb->ebflags);
- atomic_inc(&eb_head(eb)->io_bvecs);
+ atomic_inc(&ebh->io_bvecs);
if (btrfs_header_owner(eb) == BTRFS_TREE_LOG_OBJECTID)
bio_flags = EXTENT_BIO_TREE_LOG;
@@ -3777,6 +3847,8 @@ static int write_subpagesize_blocksize_ebh(struct extent_buffer_head *ebh,
atomic_dec(&eb_head(eb)->io_bvecs);
end_extent_buffer_writeback(eb);
err = -EIO;
+ } else {
+ ++nr_eb_submitted;
}
} while ((eb = eb->eb_next) != NULL);
@@ -3784,6 +3856,12 @@ static int write_subpagesize_blocksize_ebh(struct extent_buffer_head *ebh,
update_nr_written(p, wbc, 1);
}
+ if (!nr_eb_submitted) {
+ clear_bit(EXTENT_BUFFER_HEAD_WRITEBACK, &ebh->bflags);
+ smp_mb__after_atomic();
+ wake_up_bit(&ebh->bflags, EXTENT_BUFFER_HEAD_WRITEBACK);
+ }
+
unlock_page(p);
return ret;
@@ -3895,24 +3973,31 @@ retry:
j = 0;
ebs_to_write = dirty_ebs = 0;
+
+ lock_extent_buffers(ebh, &epd);
+
+ set_bit(EXTENT_BUFFER_HEAD_WRITEBACK, &ebh->bflags);
+
eb = &ebh->eb;
do {
BUG_ON(j >= BITS_PER_LONG);
- ret = lock_extent_buffer_for_io(eb, fs_info, &epd);
+ ret = mark_extent_buffer_writeback(eb, fs_info,
+ &epd);
switch (ret) {
case 0:
/*
- EXTENT_BUFFER_DIRTY was set and we were able to
- clear it.
+ EXTENT_BUFFER_DIRTY was set and we were
+ able to clear it.
*/
set_bit(j, &ebs_to_write);
break;
case 2:
/*
- EXTENT_BUFFER_DIRTY was set, but we were unable
- to clear EXTENT_BUFFER_WRITEBACK that was set
- before we got the extent buffer locked.
+ EXTENT_BUFFER_DIRTY was set, but we were
+ unable to clear EXTENT_BUFFER_WRITEBACK
+ that was set before we got the extent
+ buffer locked.
*/
set_bit(j, &dirty_ebs);
default:
@@ -3926,22 +4011,32 @@ retry:
ret = 0;
+ unlock_extent_buffers(ebh);
+
if (!ebs_to_write) {
+ clear_bit(EXTENT_BUFFER_HEAD_WRITEBACK,
+ &ebh->bflags);
+ smp_mb__after_atomic();
+ wake_up_bit(&ebh->bflags,
+ EXTENT_BUFFER_HEAD_WRITEBACK);
free_extent_buffer(&ebh->eb);
continue;
}
/*
- Now that we know that atleast one of the extent buffer
+ Now that we know that atleast one of the extent buffers
belonging to the extent buffer head must be written to
the disk, lock the extent_buffer_head's pages.
*/
lock_extent_buffer_pages(ebh, &epd);
if (ebh->eb.len < PAGE_CACHE_SIZE) {
- ret = write_subpagesize_blocksize_ebh(ebh, fs_info, wbc, &epd, ebs_to_write);
+ ret = write_subpagesize_blocksize_ebh(ebh, fs_info,
+ wbc, &epd,
+ ebs_to_write);
if (dirty_ebs) {
- redirty_extent_buffer_pages_for_writepage(&ebh->eb, wbc);
+ redirty_extent_buffer_pages_for_writepage(&ebh->eb,
+ wbc);
}
} else {
ret = write_regular_ebh(ebh, fs_info, wbc, &epd);
@@ -5189,7 +5284,7 @@ void free_extent_buffer_stale(struct extent_buffer *eb)
static int page_ebs_clean(struct extent_buffer_head *ebh)
{
- struct extent_buffer *eb = &ebh->eb;;
+ struct extent_buffer *eb = &ebh->eb;
do {
if (test_bit(EXTENT_BUFFER_DIRTY, &eb->ebflags))
diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h
index ec12455..c8f39ed 100644
--- a/fs/btrfs/extent_io.h
+++ b/fs/btrfs/extent_io.h
@@ -44,6 +44,7 @@
#define EXTENT_BUFFER_IOERR 8
#define EXTENT_BUFFER_DUMMY 9
#define EXTENT_BUFFER_IN_TREE 10
+#define EXTENT_BUFFER_HEAD_WRITEBACK 11
/* these are flags for extent_clear_unlock_delalloc */
#define PAGE_UNLOCK (1 << 0)
@@ -301,6 +302,7 @@ void free_extent_buffer_stale(struct extent_buffer *eb);
int read_extent_buffer_pages(struct extent_io_tree *tree,
struct extent_buffer *eb, u64 start, int wait,
get_extent_t *get_extent, int mirror_num);
+int eb_wait(void *word);
void wait_on_extent_buffer_writeback(struct extent_buffer *eb);
static inline unsigned long num_extent_pages(u64 start, u64 len)
--
2.1.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
end of thread, other threads:[~2014-11-14 15:48 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-14 15:38 [RFC PATCH V9 00/17] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 01/17] Btrfs: subpagesize-blocksize: Get rid of whole page reads Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 02/17] Btrfs: subpagesize-blocksize: Get rid of whole page writes Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 03/17] Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release extents aligned to block size Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 04/17] Btrfs: subpagesize-blocksize: Define extent_buffer_head Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 05/17] Btrfs: subpagesize-blocksize: Read tree blocks whose size is <PAGE_CACHE_SIZE Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 06/17] Btrfs: subpagesize-blocksize: Write only dirty extent buffers belonging to a page Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 07/17] Btrfs: subpagesize-blocksize: Allow mounting filesystems where sectorsize != PAGE_SIZE Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 08/17] Btrfs: subpagesize-blocksize: Compute and look up csums based on sectorsized blocks Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 09/17] Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty blocks of a page Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 10/17] Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 11/17] Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in " Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 12/17] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 13/17] Btrfs: subpagesize-blocksize: Deal with partial ordered extent allocations Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 14/17] Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of an ordered extent Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 15/17] Btrfs: subpagesize-blocksize: Revert commit fc4adbff823f76577ece26dcb88bf6f8392dbd43 Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 16/17] Btrfs: subpagesize-blocksize: Track blocks of ordered extent submitted for write I/O Chandan Rajendra
2014-11-14 15:38 ` [RFC PATCH V9 17/17] Btrfs: subpagesize-blocksize: Prevent writes to an extent buffer when PG_writeback flag is set Chandan Rajendra
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.