From: Josef Bacik <josef@toxicpanda.com>
To: linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: [PATCH 00/15] btrfs-progs: extent tree v2 gc tree and metadata ref changes
Date: Wed, 15 Dec 2021 15:14:38 -0500 [thread overview]
Message-ID: <cover.1639598612.git.josef@toxicpanda.com> (raw)
Hello,
Here's the next set of btrfs-progs patches for the next set of extent tree v2
features, no more metadata reference tracking and support for a garbge
collection tree.
1. No more metadata reference tracking. For cowonly trees (global roots) we
never have more than one reference for a block, so the reference counting
adds a fair bit of overhead for these trees. This is especially noticeable
with the extent tree, we have to update the extent tree to include references
for every block, including blocks in the extent tree, which generates more
modifications to the extent tree. In practice there's a stable state so we
do not recursively use all the disk space describing the extent tree, however
it generates a certain amount of churn.
For FS tree's that can be shared we still want a way to know when we can free
blocks. That will be accomplished by drop trees in a future patchset and
described more thoroughly there.
2. Garbage collection tree. Free'ing objects in btrfs is a more complicated
operation than in other file systems. For truncate we can mete out the
removal of the inode items, but the cleaning up of checksums has to be done
in the transaction that the extent is freed. For large files this can be
quite expensive and cause latencies because we have to hold the transaction
open for these operations. Deferring this work to a GC tree will allow us to
better throttle these long running operations without causing latencies for
the rest of the system.
As alluded above, free'ing shared metadata extents via the drop trees will
be one of these longer running operations, having a GC tree in place will
allow us to do these heavier operations without requiring they be done in a
single transaction.
These patches have been tested with the corresponding kernel patches and pass
the xfstests that are able to be run. Thanks,
Josef
Josef Bacik (15):
btrfs-progs: extract out free extent accounting handling
btrfs-progs: check: skip owner ref check for extent tree v2
btrfs-progs: check: skip extent backref for metadata in extent tree v2
btrfs-progs: check: calculate normal flags for extent tree v2 metadata
btrfs-progs: check: make metadata ref counting extent tree v2 aware
btrfs-progs: check: update block group used properly for extent tree
v2
btrfs-progs: do not insert extent items for metadata for extent tree
v2
btrfs-progs: do not remove metadata backrefs for extent tree v2
btrfs-progs: repair: traverse tree blocks for extent tree v2
btrfs-progs: cache the block group with the free space tree if
possible
btrfs-progs: make btrfs_lookup_extent_info extent tree v2 aware
btrfs-progs: mkfs: don't populate extent tree with extent tree v2
btrfs-progs: add on-disk items and read support for the gc tree
btrfs-progs: mkfs: create the gc tree at mkfs time
btrfs-progs: deal with GC items in check
check/main.c | 98 ++++++++++++++-
check/mode-original.h | 1 +
common/repair.c | 6 +
kernel-shared/ctree.h | 6 +
kernel-shared/disk-io.c | 5 +
kernel-shared/extent-tree.c | 82 ++++++++----
kernel-shared/free-space-cache.c | 3 +
kernel-shared/print-tree.c | 4 +
mkfs/common.c | 207 +++++++++++++++++--------------
mkfs/main.c | 13 +-
10 files changed, 303 insertions(+), 122 deletions(-)
--
2.26.3
next reply other threads:[~2021-12-15 20:14 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-15 20:14 Josef Bacik [this message]
2021-12-15 20:14 ` [PATCH 01/15] btrfs-progs: extract out free extent accounting handling Josef Bacik
2021-12-15 20:14 ` [PATCH 02/15] btrfs-progs: check: skip owner ref check for extent tree v2 Josef Bacik
2021-12-15 20:14 ` [PATCH 03/15] btrfs-progs: check: skip extent backref for metadata in " Josef Bacik
2021-12-15 20:14 ` [PATCH 04/15] btrfs-progs: check: calculate normal flags for extent tree v2 metadata Josef Bacik
2021-12-15 20:14 ` [PATCH 05/15] btrfs-progs: check: make metadata ref counting extent tree v2 aware Josef Bacik
2021-12-15 20:14 ` [PATCH 06/15] btrfs-progs: check: update block group used properly for extent tree v2 Josef Bacik
2021-12-15 20:14 ` [PATCH 07/15] btrfs-progs: do not insert extent items for metadata " Josef Bacik
2021-12-15 20:14 ` [PATCH 08/15] btrfs-progs: do not remove metadata backrefs " Josef Bacik
2021-12-15 20:14 ` [PATCH 09/15] btrfs-progs: repair: traverse tree blocks " Josef Bacik
2021-12-15 20:14 ` [PATCH 10/15] btrfs-progs: cache the block group with the free space tree if possible Josef Bacik
2021-12-15 20:14 ` [PATCH 11/15] btrfs-progs: make btrfs_lookup_extent_info extent tree v2 aware Josef Bacik
2021-12-15 20:14 ` [PATCH 12/15] btrfs-progs: mkfs: don't populate extent tree with extent tree v2 Josef Bacik
2021-12-15 20:14 ` [PATCH 13/15] btrfs-progs: add on-disk items and read support for the gc tree Josef Bacik
2021-12-15 20:14 ` [PATCH 14/15] btrfs-progs: mkfs: create the gc tree at mkfs time Josef Bacik
2021-12-15 20:14 ` [PATCH 15/15] btrfs-progs: deal with GC items in check Josef Bacik
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1639598612.git.josef@toxicpanda.com \
--to=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).