From: "Darrick J. Wong" <djwong@kernel.org>
To: zlang@redhat.com, djwong@kernel.org
Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me
Subject: [PATCHSET v1.0 00/12] xfsprogs: shard the realtime section
Date: Fri, 30 Dec 2022 14:20:39 -0800 [thread overview]
Message-ID: <167243883943.739029.3041109696120604285.stgit@magnolia> (raw)
In-Reply-To: <Y69UsO7tDT3HcFri@magnolia>
Hi all,
Right now, the realtime section uses a single pair of metadata inodes to
store the free space information. This presents a scalability problem
since every thread trying to allocate or free rt extents have to lock
these files. It would be very useful if we could begin to tackle these
problems by sharding the realtime section, so create the notion of
realtime groups, which are similar to allocation groups on the data
section.
While we're at it, define a superblock to be stamped into the start of
each rt section. This enables utilities such as blkid to identify block
devices containing realtime sections, and helpfully avoids the situation
where a file extent can cross an rtgroup boundary.
The best advantage for rtgroups will become evident later when we get to
adding rmap and reflink to the realtime volume, since the geometry
constraints are the same for rt groups and AGs. Hence we can reuse all
that code directly.
If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy!
Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-groups
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-groups
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-groups
---
common/fuzzy | 33 +++++++++++++++----
common/populate | 12 ++++++-
common/xfs | 83 ++++++++++++++++++++++++++++++++++++++++-------
src/punch-alternating.c | 28 +++++++++++++++-
tests/xfs/114 | 4 ++
tests/xfs/122 | 2 +
tests/xfs/122.out | 8 +++++
tests/xfs/146 | 2 +
tests/xfs/185 | 2 +
tests/xfs/187 | 3 +-
tests/xfs/206 | 3 +-
tests/xfs/271 | 3 +-
tests/xfs/341 | 4 +-
tests/xfs/449 | 6 +++
tests/xfs/556 | 16 ++++++---
tests/xfs/800 | 2 +
tests/xfs/840 | 2 +
tests/xfs/841 | 2 +
18 files changed, 176 insertions(+), 39 deletions(-)
next prev parent reply other threads:[~2022-12-31 1:04 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-30 21:14 [NYE DELUGE 3/4] xfs: modernize the realtime volume Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET 0/1] fstests: test upgrading older features Darrick J. Wong
2022-12-30 22:20 ` [PATCH 1/1] xfs: test upgrading old features Darrick J. Wong
2023-03-06 15:56 ` Zorro Lang
2023-03-06 16:41 ` Darrick J. Wong
2023-03-06 16:54 ` Zorro Lang
2023-03-06 23:14 ` Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET v1.0 0/9] fstests: test XFS metadata directories Darrick J. Wong
2022-12-30 22:20 ` [PATCH 3/9] xfs/{030,033,178}: forcibly disable metadata directory trees Darrick J. Wong
2022-12-30 22:20 ` [PATCH 1/9] xfs/122: fix metadirino Darrick J. Wong
2022-12-30 22:20 ` [PATCH 2/9] various: fix finding metadata inode numbers when metadir is enabled Darrick J. Wong
2023-03-06 16:41 ` Zorro Lang
2022-12-30 22:20 ` [PATCH 4/9] common/repair: patch up repair sb inode value complaints Darrick J. Wong
2022-12-30 22:20 ` [PATCH 6/9] xfs/{050,144,153,299,330}: update quota reports to leave out metadir files Darrick J. Wong
2022-12-30 22:20 ` [PATCH 9/9] xfs: create fuzz tests for metadata directories Darrick J. Wong
2022-12-30 22:20 ` [PATCH 7/9] xfs/769: add metadir upgrade to test matrix Darrick J. Wong
2022-12-30 22:20 ` [PATCH 5/9] xfs/206: update for metadata directory support Darrick J. Wong
2022-12-30 22:20 ` [PATCH 8/9] xfs/509: adjust inumbers accounting for metadata directories Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET v1.0 0/4] fstests: support metadump to external devices Darrick J. Wong
2022-12-30 22:20 ` [PATCH 3/4] common/ext4: reformat external logs during mdrestore operations Darrick J. Wong
2022-12-30 22:20 ` [PATCH 2/4] common/xfs: wipe " Darrick J. Wong
2022-12-30 22:20 ` [PATCH 1/4] common/populate: refactor caching of metadumps to a helper Darrick J. Wong
2022-12-30 22:20 ` [PATCH 4/4] common/xfs: capture external logs during metadump/mdrestore Darrick J. Wong
2022-12-30 22:20 ` Darrick J. Wong [this message]
2022-12-30 22:20 ` [PATCH 03/12] xfs/206: update mkfs filtering for rt groups feature Darrick J. Wong
2022-12-30 22:20 ` [PATCH 01/12] xfs/122: update for rtgroups Darrick J. Wong
2022-12-30 22:20 ` [PATCH 02/12] punch-alternating: detect xfs realtime files with large allocation units Darrick J. Wong
2022-12-30 22:20 ` [PATCH 10/12] xfs/27[46],xfs/556: fix tests to deal with rtgroups output in bmap/fsmap commands Darrick J. Wong
2022-12-30 22:20 ` [PATCH 07/12] xfs/449: update test to know about xfs_db -R Darrick J. Wong
2022-12-30 22:20 ` [PATCH 11/12] common/xfs: capture realtime devices during metadump/mdrestore Darrick J. Wong
2022-12-30 22:20 ` [PATCH 05/12] common: filter rtgroups when we're disabling metadir Darrick J. Wong
2022-12-30 22:20 ` [PATCH 08/12] xfs/122: update for rtbitmap headers Darrick J. Wong
2022-12-30 22:20 ` [PATCH 06/12] xfs/185: update for rtgroups Darrick J. Wong
2022-12-30 22:20 ` [PATCH 04/12] common: pass the realtime device to xfs_db when possible Darrick J. Wong
2022-12-30 22:20 ` [PATCH 09/12] xfs/122: udpate test to pick up rtword/suminfo ondisk unions Darrick J. Wong
2022-12-30 22:20 ` [PATCH 12/12] common/fuzzy: adapt the scrub stress tests to support rtgroups Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET v1.0 00/13] fstests: fixes for realtime rmap Darrick J. Wong
2022-12-30 22:20 ` [PATCH 03/13] xfs: race fsstress with realtime rmap btree scrub and repair Darrick J. Wong
2022-12-30 22:20 ` [PATCH 04/13] xfs/769: add rtrmapbt upgrade to test matrix Darrick J. Wong
2022-12-30 22:20 ` [PATCH 05/13] xfs/122: update for rtgroups-based realtime rmap btrees Darrick J. Wong
2022-12-30 22:20 ` [PATCH 07/13] xfs/341: update test for rtgroup-based rmap Darrick J. Wong
2022-12-30 22:20 ` [PATCH 06/13] xfs: fix various problems with fsmap detecting the data device Darrick J. Wong
2022-12-30 22:20 ` [PATCH 01/13] xfs: fix tests that try to access the realtime rmap inode Darrick J. Wong
2022-12-30 22:20 ` [PATCH 02/13] fuzz: for fuzzing the rtrmapbt, find the path to the rt rmap btree file Darrick J. Wong
2022-12-30 22:20 ` [PATCH 09/13] xfs: skip tests if formatting small filesystem fails Darrick J. Wong
2022-12-30 22:20 ` [PATCH 10/13] xfs/443: use file allocation unit, not dbsize Darrick J. Wong
2022-12-30 22:20 ` [PATCH 12/13] populate: check that we created a realtime rmap btree of the given height Darrick J. Wong
2022-12-30 22:20 ` [PATCH 13/13] fuzzy: create missing fuzz tests for rt rmap btrees Darrick J. Wong
2022-12-30 22:20 ` [PATCH 11/13] populate: adjust rtrmap calculations for rtgroups Darrick J. Wong
2022-12-30 22:20 ` [PATCH 08/13] xfs/3{43,32}: adapt tests for rt extent size greater than 1 Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET v1.0 00/10] fstests: reflink on the realtime device Darrick J. Wong
2022-12-30 22:20 ` [PATCH 01/10] xfs/122: update fields for realtime reflink Darrick J. Wong
2022-12-30 22:20 ` [PATCH 02/10] common/populate: create realtime refcount btree Darrick J. Wong
2022-12-30 22:20 ` [PATCH 03/10] xfs: create fuzz tests for the " Darrick J. Wong
2022-12-30 22:20 ` [PATCH 06/10] xfs: race fsstress with realtime refcount btree scrub and repair Darrick J. Wong
2022-12-30 22:20 ` [PATCH 09/10] generic/331,xfs/240: support files that skip delayed allocation Darrick J. Wong
2022-12-30 22:20 ` [PATCH 05/10] xfs/243: don't run when realtime storage is the default Darrick J. Wong
2022-12-30 22:20 ` [PATCH 04/10] xfs/27[24]: adapt for checking files on the realtime volume Darrick J. Wong
2022-12-30 22:20 ` [PATCH 08/10] xfs/769: add rtreflink upgrade to test matrix Darrick J. Wong
2022-12-30 22:20 ` [PATCH 10/10] common/xfs: fix _xfs_get_file_block_size when rtinherit is set and no rt section Darrick J. Wong
2022-12-30 22:20 ` [PATCH 07/10] xfs: remove xfs/131 now that we allow reflink on realtime volumes Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET v1.0 0/4] fstests: reflink with large realtime extents Darrick J. Wong
2022-12-30 22:20 ` [PATCH 1/4] xfs: make sure that CoW will write around when rextsize > 1 Darrick J. Wong
2022-12-30 22:20 ` [PATCH 2/4] xfs: skip cowextsize hint fragmentation tests on realtime volumes Darrick J. Wong
2022-12-30 22:20 ` [PATCH 4/4] generic/303: avoid test failures on weird rt extent sizes Darrick J. Wong
2022-12-30 22:20 ` [PATCH 3/4] misc: add more congruent oplen testing Darrick J. Wong
2022-12-30 22:20 ` [PATCHSET v1.0 0/1] fstests: functional tests for rt quota Darrick J. Wong
2022-12-30 22:20 ` [PATCH 1/1] xfs: regression testing of quota on the realtime device Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=167243883943.739029.3041109696120604285.stgit@magnolia \
--to=djwong@kernel.org \
--cc=fstests@vger.kernel.org \
--cc=guan@eryu.me \
--cc=linux-xfs@vger.kernel.org \
--cc=zlang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).