From: Qu Wenruo <wqu@suse.com>
To: linux-btrfs@vger.kernel.org
Subject: [PATCH v3 00/10] btrfs: defrag: rework to support sector perfect defrag
Date: Tue, 8 Jun 2021 10:59:17 +0800 [thread overview]
Message-ID: <20210608025927.119169-1-wqu@suse.com> (raw)
This branch is based on subpage RW branch, as the last patch needs to
enable defrag support for subpage cases.
But despite that one, all other patches can be applied on current
misc-next.
[BACKGROUND]
In subpage rw branch, we disable defrag completely due to the fact that
current code can only work on page basis.
This could lead to problems like btrfs/062 crash.
Thus this patchset will make defrag to work on both regular and subpage
sectorsize.
[SOLUTION]
To defrag a file range, what we do is pretty much like buffered write,
except we don't really write any new data to page cache, but just mark
the range dirty.
Then let later writeback to merge the range into a larger extent.
But current defrag code is working on per-page basis, not per-sector,
thus we have to refactor it a little to make it to work properly for
subpage.
This patch will separate the code into 3 layers:
Layer 0: btrfs_defrag_file()
The defrag entrace
Just do proper inode lock and split the file into
page aligned 256K clusters to defrag
Layer 1: defrag_one_cluster()
Will collect the initial targets file extents, and pass
each continuous target to defrag_one_range()
Layer 2: defrag_one_range()
Will prepare the needed page and extent locking.
Then re-check the range for real target list, as initial
target list is not consistent as it doesn't hage
page/extent locking to prevent hole punching.
Layer 3: defrag_one_locked_target()
The real work, to make the extent range defrag and
update involved page status
[BEHAVIOR CHANGE]
In the refactor, there is one behavior change:
- Defraged sector counter is based on the initial target list
This is mostly to avoid the paremters to be passed too deep into
defrag_one_locked_target().
Considering the accounting is not that important, we can afford some
difference.
[PATCH STRUTURE]
Patch 01~03: Small independent refactor to improve readability
Patch 04~08: Implement the more readable and subpage friendly defrag
Patch 09: Cleanup of old infrastruture
Patch 10: Enable defrag for subpage case
Now both regular sectorsize and subpage sectorsize can pass defrag test
group.
[CHANGELOG]
v2:
- Make sure we won't defrag hole
This is done by re-collect the target list after have page and extent
locked. So that we can have a consistent view of the extent map.
- Add a new layer to avoid variable naming bugs
Since we need to handle real target list inside defrag_one_range(),
and that function has parameters like "start" and "len", while inside
the loop we need things like "entry->start" and "entry->len", it has
already caused hard to debug bugs during development.
Thus introduce a new layer, defrag_one_ragen() to prepare pages/extent
lock then pass the entry to defrag_one_locked_target().
v3:
- Fix extent_state leak
Now we pass the @cached_state to defrag_one_locked_target() other
than allowing it to allocate a new one.
This can be reproduced by enabling "TEST_FS_MODULE_RELOAD" environment
variable for fstests and run "-g defrag" group.
- Fix a random hang in test cases like btrfs/062
Since defrag_one_range() will lock the extent range first, then
call defrag_collect_targets(), which will in turn call
defrag_lookup_extent() and lock extent range again.
This will cause a dead lock, and this only happens when the
extent range is smaller than the original locked range.
Thus sometimes the test can pass, but sometimes it can hang.
Fix it by teaching defrag_collect_targets() and defrag_lookup_extent()
to skip extent lock for certain call sites.
Thus this needs some loops for btrfs/062.
The hang possibility is around 1/2 ~ 1/3 when run in a loop.
Qu Wenruo (10):
btrfs: defrag: pass file_ra_state instead of file for
btrfs_defrag_file()
btrfs: defrag: extract the page preparation code into one helper
btrfs: defrag: replace hard coded PAGE_SIZE to sectorsize for
defrag_lookup_extent()
btrfs: defrag: introduce a new helper to collect target file extents
btrfs: defrag: introduce a helper to defrag a continuous prepared
range
btrfs: defrag: introduce a helper to defrag a range
btrfs: defrag: introduce a new helper to defrag one cluster
btrfs: defrag: use defrag_one_cluster() to implement
btrfs_defrag_file()
btrfs: defrag: remove the old infrastructure
btrfs: defrag: enable defrag for subpage case
fs/btrfs/ctree.h | 4 +-
fs/btrfs/ioctl.c | 921 ++++++++++++++++++++++-------------------------
2 files changed, 432 insertions(+), 493 deletions(-)
--
2.31.1
next reply other threads:[~2021-06-08 2:59 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-08 2:59 Qu Wenruo [this message]
2021-06-08 2:59 ` [PATCH v3 01/10] btrfs: defrag: pass file_ra_state instead of file for btrfs_defrag_file() Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 02/10] btrfs: defrag: extract the page preparation code into one helper Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 03/10] btrfs: defrag: replace hard coded PAGE_SIZE to sectorsize for defrag_lookup_extent() Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 04/10] btrfs: defrag: introduce a new helper to collect target file extents Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 05/10] btrfs: defrag: introduce a helper to defrag a continuous prepared range Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 06/10] btrfs: defrag: introduce a helper to defrag a range Qu Wenruo
2021-06-09 5:38 ` Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 07/10] btrfs: defrag: introduce a new helper to defrag one cluster Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 08/10] btrfs: defrag: use defrag_one_cluster() to implement btrfs_defrag_file() Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 09/10] btrfs: defrag: remove the old infrastructure Qu Wenruo
2021-06-08 2:59 ` [PATCH v3 10/10] btrfs: defrag: enable defrag for subpage case Qu Wenruo
2021-06-09 15:26 ` [PATCH v3 00/10] btrfs: defrag: rework to support sector perfect defrag David Sterba
2021-06-09 22:48 ` Neal Gompa
2021-06-09 23:02 ` David Sterba
2021-06-09 23:33 ` Qu Wenruo
2021-06-09 23:31 ` Qu Wenruo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210608025927.119169-1-wqu@suse.com \
--to=wqu@suse.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.