From: Dennis Zhou <dennis@kernel.org>
To: David Sterba <dsterba@suse.com>, Chris Mason <clm@fb.com>,
Josef Bacik <josef@toxicpanda.com>,
Omar Sandoval <osandov@osandov.com>
Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org,
Dennis Zhou <dennis@kernel.org>
Subject: [PATCH v5 00/22] btrfs: async discard support
Date: Mon, 9 Dec 2019 11:45:45 -0800 [thread overview]
Message-ID: <cover.1575919745.git.dennis@kernel.org> (raw)
Hello,
Dave reported that with async discard enabled, relocation fails [1].
This could be caused by two things. First, if we unpin extents, that
means we haven't fully discarded the block group and need to let async
discard revisit it. Second, relocation removes block_groups outside of
the normal. I fixed both issues and now it successfully passes xfstests
btrfs/003.
Changes in v5:
- Changed the rules so free space is always added as the right type
based on discard settings (see btrfs_add_free_space()), this removes
the need to pass around trim_state in unpin_extent_range().
- Handled relocation block group deletion (xfstests btrfs/003)
- When adding to the discard lists, make sure the work queue is active.
(made all additions go through either btrfs_discard_queue_work() or
btrfs_discard_check_filter()).
- Added 10 sec reuse timeout for fully empty block groups.
v4 is available here: [2].
This series is on top of btrfs-devel#misc-next fbc0468e42d2 + [3].
[1] https://lore.kernel.org/linux-btrfs/20191126215204.GP2734@twin.jikos.cz/
[2] https://lore.kernel.org/linux-btrfs/cover.1574709825.git.dennis@kernel.org/
[3] https://lore.kernel.org/linux-btrfs/20191209193846.18162-1-dennis@kernel.org/
Dennis Zhou (22):
bitmap: genericize percpu bitmap region iterators
btrfs: rename DISCARD opt to DISCARD_SYNC
btrfs: keep track of which extents have been discarded
btrfs: keep track of cleanliness of the bitmap
btrfs: add the beginning of async discard, discard workqueue
btrfs: handle empty block_group removal
btrfs: discard one region at a time in async discard
btrfs: add removal calls for sysfs debug/
btrfs: make UUID/debug have its own kobject
btrfs: add discard sysfs directory
btrfs: track discardable extents for async discard
btrfs: keep track of discardable_bytes
btrfs: calculate discard delay based on number of extents
btrfs: add bps discard rate limit
btrfs: limit max discard size for async discard
btrfs: make max async discard size tunable
btrfs: have multiple discard lists
btrfs: only keep track of data extents for async discard
btrfs: keep track of discard reuse stats
btrfs: add async discard header
btrfs: increase the metadata allowance for the free_space_cache
btrfs: make smaller extents more likely to go into bitmaps
fs/btrfs/Makefile | 2 +-
fs/btrfs/block-group.c | 88 ++++-
fs/btrfs/block-group.h | 30 ++
fs/btrfs/ctree.h | 52 ++-
fs/btrfs/discard.c | 684 ++++++++++++++++++++++++++++++++++++
fs/btrfs/discard.h | 42 +++
fs/btrfs/disk-io.c | 15 +-
fs/btrfs/extent-tree.c | 8 +-
fs/btrfs/free-space-cache.c | 611 +++++++++++++++++++++++++++-----
fs/btrfs/free-space-cache.h | 41 ++-
fs/btrfs/inode-map.c | 13 +-
fs/btrfs/inode.c | 2 +-
fs/btrfs/scrub.c | 7 +-
fs/btrfs/super.c | 39 +-
fs/btrfs/sysfs.c | 205 ++++++++++-
fs/btrfs/volumes.c | 7 +
include/linux/bitmap.h | 35 ++
mm/percpu.c | 61 +---
18 files changed, 1789 insertions(+), 153 deletions(-)
create mode 100644 fs/btrfs/discard.c
create mode 100644 fs/btrfs/discard.h
--
2.17.1
next reply other threads:[~2019-12-09 19:46 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-09 19:45 Dennis Zhou [this message]
2019-12-09 19:45 ` [PATCH 01/22] bitmap: genericize percpu bitmap region iterators Dennis Zhou
2019-12-09 19:45 ` [PATCH 02/22] btrfs: rename DISCARD opt to DISCARD_SYNC Dennis Zhou
2019-12-09 19:45 ` [PATCH 03/22] btrfs: keep track of which extents have been discarded Dennis Zhou
2019-12-09 19:45 ` [PATCH 04/22] btrfs: keep track of cleanliness of the bitmap Dennis Zhou
2019-12-09 19:45 ` [PATCH 05/22] btrfs: add the beginning of async discard, discard workqueue Dennis Zhou
2019-12-09 19:45 ` [PATCH 06/22] btrfs: handle empty block_group removal Dennis Zhou
2019-12-09 19:45 ` [PATCH 07/22] btrfs: discard one region at a time in async discard Dennis Zhou
2019-12-09 19:45 ` [PATCH 08/22] btrfs: add removal calls for sysfs debug/ Dennis Zhou
2019-12-09 19:45 ` [PATCH 09/22] btrfs: make UUID/debug have its own kobject Dennis Zhou
2019-12-09 19:45 ` [PATCH 10/22] btrfs: add discard sysfs directory Dennis Zhou
2019-12-09 19:45 ` [PATCH 11/22] btrfs: track discardable extents for async discard Dennis Zhou
2019-12-09 19:45 ` [PATCH 12/22] btrfs: keep track of discardable_bytes Dennis Zhou
2019-12-09 19:45 ` [PATCH 13/22] btrfs: calculate discard delay based on number of extents Dennis Zhou
2019-12-09 19:45 ` [PATCH 14/22] btrfs: add bps discard rate limit Dennis Zhou
2019-12-09 19:46 ` [PATCH 15/22] btrfs: limit max discard size for async discard Dennis Zhou
2019-12-09 19:46 ` [PATCH 16/22] btrfs: make max async discard size tunable Dennis Zhou
2019-12-09 19:46 ` [PATCH 17/22] btrfs: have multiple discard lists Dennis Zhou
2019-12-09 19:46 ` [PATCH 18/22] btrfs: only keep track of data extents for async discard Dennis Zhou
2019-12-09 19:46 ` [PATCH 19/22] btrfs: keep track of discard reuse stats Dennis Zhou
2019-12-09 19:46 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou
2019-12-09 19:46 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou
2019-12-09 19:46 ` [PATCH 22/22] btrfs: make smaller extents more likely to go into bitmaps Dennis Zhou
2019-12-10 14:04 ` [PATCH v5 00/22] btrfs: async discard support David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1575919745.git.dennis@kernel.org \
--to=dennis@kernel.org \
--cc=clm@fb.com \
--cc=dsterba@suse.com \
--cc=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=osandov@osandov.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).