* [NYE DELUGE 4/4] xfs: freespace defrag for online shrink
@ 2022-12-30 21:14 Darrick J. Wong
2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong
` (9 more replies)
0 siblings, 10 replies; 40+ messages in thread
From: Darrick J. Wong @ 2022-12-30 21:14 UTC (permalink / raw)
To: djwong; +Cc: xfs, fstests
Hi all,
This fourth patch deluge has two faces -- one way of looking at it is
that it is random odds and ends at the tail of my development tree. A
second interpretation is that it is necessary pieces for defragmenting
free space, which is a precursor for online shrink of XFS filesystems.
The kernel side isn't that exciting -- we export refcounting information
for space extents, and add a new fallocate mode for mapping exact
portions of free filesystem space into a file.
Userspace is where things get interesting! The free space defragmenter
is an iterative algorithm that assigns free space to a dummy file, and
then uses the GETFSMAP and GETFSREFCOUNTS information to target file
space extents in order of decreasing share counts. Once an extent has
been targeted, it uses reflinking to freeze the space, copies it
elsewhere, and uses FIDEDUPERANGE to remap existing file data until the
dummy file is the sole owner of the targetted space. If metadata are
involved, the defrag utility invokes online repair to rebuild the
metadata somewhere else.
When the defragmenter finishes, all the free space has been isolated to
the dummy file, which can be unlinked and closed if defragmentation was
the goal; or it could be passed to a shrinkfs operation.
NOTE: There's also an experimental vectorization interface for scrub.
Given how long it's likely to take to get to this fourth deluge, it
might make more sense to integrate with io_uring when that day comes.
As a warning, the patches will likely take several days to trickle in.
--D
^ permalink raw reply [flat|nested] 40+ messages in thread* [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: don't free EOF blocks on read close Darrick J. Wong ` (2 more replies) 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: vectorize scrub kernel calls Darrick J. Wong ` (8 subsequent siblings) 9 siblings, 3 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: Dave Chinner, linux-xfs Hi all, Here's a few patches mostly from Dave to make XFS more aggressive about keeping post-eof speculative preallocations when closing files. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=reduce-eofblocks-gc-on-close --- fs/xfs/xfs_bmap_util.c | 9 ++++++--- fs/xfs/xfs_file.c | 14 ++++++++++++-- fs/xfs/xfs_inode.c | 13 ++++++------- fs/xfs/xfs_inode.h | 2 +- 4 files changed, 25 insertions(+), 13 deletions(-) ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 2/3] xfs: don't free EOF blocks on read close 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 3/3] xfs: Don't free EOF blocks on close when extent size hints are set Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/3] xfs: only free posteof blocks on first close Darrick J. Wong 2 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: Dave Chinner, linux-xfs From: Dave Chinner <dchinner@redhat.com> When we have a workload that does open/read/close in parallel with other allocation, the file becomes rapidly fragmented. This is due to close() calling xfs_release() and removing the speculative preallocation beyond EOF. The existing open/*/close heuristic in xfs_release() does not catch this as a sync writer does not leave delayed allocation blocks allocated on the inode for later writeback that can be detected in xfs_release() and hence XFS_IDIRTY_RELEASE never gets set. In xfs_file_release(), we know more about the released file context, and so we need to communicate some of the details to xfs_release() so it can do the right thing here and skip EOF block truncation. This defers the EOF block cleanup for synchronous write contexts to the background EOF block cleaner which will clean up within a few minutes. Before: Test 1: sync write fragmentation counts /mnt/scratch/file.0: 919 /mnt/scratch/file.1: 916 /mnt/scratch/file.2: 919 /mnt/scratch/file.3: 920 /mnt/scratch/file.4: 920 /mnt/scratch/file.5: 921 /mnt/scratch/file.6: 916 /mnt/scratch/file.7: 918 After: Test 1: sync write fragmentation counts /mnt/scratch/file.0: 24 /mnt/scratch/file.1: 24 /mnt/scratch/file.2: 11 /mnt/scratch/file.3: 24 /mnt/scratch/file.4: 3 /mnt/scratch/file.5: 24 /mnt/scratch/file.6: 24 /mnt/scratch/file.7: 23 Signed-off-by: Dave Chinner <dchinner@redhat.com> [darrick: wordsmithing, fix commit message] Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/xfs_file.c | 14 ++++++++++++-- fs/xfs/xfs_inode.c | 9 +++++---- fs/xfs/xfs_inode.h | 2 +- 3 files changed, 18 insertions(+), 7 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index e172ca1b18df..87e836e1aeb3 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1381,12 +1381,22 @@ xfs_dir_open( return error; } +/* + * When we release the file, we don't want it to trim EOF blocks if it is a + * readonly context. This avoids open/read/close workloads from removing + * EOF blocks that other writers depend upon to reduce fragmentation. + */ STATIC int xfs_file_release( struct inode *inode, - struct file *filp) + struct file *file) { - return xfs_release(XFS_I(inode)); + bool free_eof_blocks = true; + + if ((file->f_mode & (FMODE_WRITE | FMODE_READ)) == FMODE_READ) + free_eof_blocks = false; + + return xfs_release(XFS_I(inode), free_eof_blocks); } STATIC int diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index f0e44c96b769..763f07867325 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -1311,10 +1311,11 @@ xfs_itruncate_extents_flags( int xfs_release( - xfs_inode_t *ip) + struct xfs_inode *ip, + bool want_free_eofblocks) { - xfs_mount_t *mp = ip->i_mount; - int error = 0; + struct xfs_mount *mp = ip->i_mount; + int error = 0; if (!S_ISREG(VFS_I(ip)->i_mode) || (VFS_I(ip)->i_mode == 0)) return 0; @@ -1356,7 +1357,7 @@ xfs_release( * another chance to drop them once the last reference to the inode is * dropped, so we'll never leak blocks permanently. */ - if (!xfs_ilock_nowait(ip, XFS_IOLOCK_EXCL)) + if (!want_free_eofblocks || !xfs_ilock_nowait(ip, XFS_IOLOCK_EXCL)) return 0; if (xfs_can_free_eofblocks(ip, false)) { diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 32a1d114dfaf..4ab0a63da367 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -493,7 +493,7 @@ enum layout_break_reason { #define XFS_INHERIT_GID(pip) \ (xfs_has_grpid((pip)->i_mount) || (VFS_I(pip)->i_mode & S_ISGID)) -int xfs_release(struct xfs_inode *ip); +int xfs_release(struct xfs_inode *ip, bool can_free_eofblocks); void xfs_inactive(struct xfs_inode *ip); int xfs_lookup(struct xfs_inode *dp, const struct xfs_name *name, struct xfs_inode **ipp, struct xfs_name *ci_name); ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 3/3] xfs: Don't free EOF blocks on close when extent size hints are set 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: don't free EOF blocks on read close Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/3] xfs: only free posteof blocks on first close Darrick J. Wong 2 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: Dave Chinner, linux-xfs From: Dave Chinner <david@fromorbit.com> When we have a workload that does open/write/close on files with extent size hints set in parallel with other allocation, the file becomes rapidly fragmented. This is due to close() calling xfs_release() and removing the preallocated extent beyond EOF. This occurs for both buffered and direct writes that append to files with extent size hints. The existing open/write/close hueristic in xfs_release() does not catch this as writes to files using extent size hints do not use delayed allocation and hence do not leave delayed allocation blocks allocated on the inode that can be detected in xfs_release(). Hence XFS_IDIRTY_RELEASE never gets set. In xfs_file_release(), we can tell whether the inode has extent size hints set and skip EOF block truncation. We add this check to xfs_can_free_eofblocks() so that we treat the post-EOF preallocated extent like intentional preallocation and so are persistent unless directly removed by userspace. Before: Test 2: Extent size hint fragmentation counts /mnt/scratch/file.0: 1002 /mnt/scratch/file.1: 1002 /mnt/scratch/file.2: 1002 /mnt/scratch/file.3: 1002 /mnt/scratch/file.4: 1002 /mnt/scratch/file.5: 1002 /mnt/scratch/file.6: 1002 /mnt/scratch/file.7: 1002 After: Test 2: Extent size hint fragmentation counts /mnt/scratch/file.0: 4 /mnt/scratch/file.1: 4 /mnt/scratch/file.2: 4 /mnt/scratch/file.3: 4 /mnt/scratch/file.4: 4 /mnt/scratch/file.5: 4 /mnt/scratch/file.6: 4 /mnt/scratch/file.7: 4 Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/xfs_bmap_util.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index a54ed26e1cc0..558951710404 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -710,12 +710,15 @@ xfs_can_free_eofblocks( return false; /* - * Do not free real preallocated or append-only files unless the file - * has delalloc blocks and we are forced to remove them. + * Do not free extent size hints, real preallocated or append-only files + * unless the file has delalloc blocks and we are forced to remove + * them. */ - if (ip->i_diflags & (XFS_DIFLAG_PREALLOC | XFS_DIFLAG_APPEND)) + if (xfs_get_extsz_hint(ip) || + (ip->i_diflags & (XFS_DIFLAG_PREALLOC | XFS_DIFLAG_APPEND))) { if (!force || ip->i_delayed_blks == 0) return false; + } /* * Do not try to free post-EOF blocks if EOF is beyond the end of the ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 1/3] xfs: only free posteof blocks on first close 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: don't free EOF blocks on read close Darrick J. Wong 2022-12-30 22:19 ` [PATCH 3/3] xfs: Don't free EOF blocks on close when extent size hints are set Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Certain workloads fragment files on XFS very badly, such as a software package that creates a number of threads, each of which repeatedly run the sequence: open a file, perform a synchronous write, and close the file, which defeats the speculative preallocation mechanism. We work around this problem by only deleting posteof blocks the /first/ time a file is closed to preserve the behavior that unpacking a tarball lays out files one after the other with no gaps. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/xfs_inode.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index d50cbd0eb260..f0e44c96b769 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -1381,9 +1381,7 @@ xfs_release( if (error) goto out_unlock; - /* delalloc blocks after truncation means it really is dirty */ - if (ip->i_delayed_blks) - xfs_iflags_set(ip, XFS_IDIRTY_RELEASE); + xfs_iflags_set(ip, XFS_IDIRTY_RELEASE); } out_unlock: ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/3] xfs: vectorize scrub kernel calls 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: whine to dmesg when we encounter errors Darrick J. Wong ` (2 more replies) 2022-12-30 22:19 ` [PATCHSET 0/1] xfs: report refcount information to userspace Darrick J. Wong ` (7 subsequent siblings) 9 siblings, 3 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs Hi all, Create a vectorized version of the metadata scrub and repair ioctl, and adapt xfs_scrub to use that. This is an experiment to measure overhead and to try refactoring xfs_scrub. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=vectorized-scrub xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=vectorized-scrub --- fs/xfs/libxfs/xfs_defer.c | 14 +++ fs/xfs/libxfs/xfs_fs.h | 37 +++++++++ fs/xfs/scrub/btree.c | 88 +++++++++++++++++++++ fs/xfs/scrub/common.c | 104 +++++++++++++++++++++++++ fs/xfs/scrub/common.h | 1 fs/xfs/scrub/dabtree.c | 24 ++++++ fs/xfs/scrub/inode.c | 4 + fs/xfs/scrub/scrub.c | 185 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.c | 22 +++++ fs/xfs/scrub/trace.h | 80 +++++++++++++++++++ fs/xfs/scrub/xfs_scrub.h | 2 fs/xfs/xfs_ioctl.c | 47 +++++++++++ fs/xfs/xfs_trace.h | 19 +++++ fs/xfs/xfs_trans.c | 3 + fs/xfs/xfs_trans.h | 7 ++ 15 files changed, 636 insertions(+), 1 deletion(-) ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 2/3] xfs: whine to dmesg when we encounter errors 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: vectorize scrub kernel calls Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/3] xfs: track deferred ops statistics Darrick J. Wong 2022-12-30 22:19 ` [PATCH 3/3] xfs: introduce vectored scrub mode Darrick J. Wong 2 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Forward everything scrub whines about to dmesg. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/scrub/btree.c | 88 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.c | 104 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/common.h | 1 fs/xfs/scrub/dabtree.c | 24 +++++++++++ fs/xfs/scrub/inode.c | 4 ++ fs/xfs/scrub/scrub.c | 37 +++++++++++++++++ fs/xfs/scrub/trace.c | 22 ++++++++++ fs/xfs/scrub/trace.h | 2 + 8 files changed, 282 insertions(+) diff --git a/fs/xfs/scrub/btree.c b/fs/xfs/scrub/btree.c index 24ea77e46ebd..a6b1d82383e8 100644 --- a/fs/xfs/scrub/btree.c +++ b/fs/xfs/scrub/btree.c @@ -11,6 +11,8 @@ #include "xfs_mount.h" #include "xfs_inode.h" #include "xfs_btree.h" +#include "xfs_log_format.h" +#include "xfs_ag.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/btree.h" @@ -18,6 +20,62 @@ /* btree scrubbing */ +/* Figure out which block the btree cursor was pointing to. */ +static inline xfs_fsblock_t +xchk_btree_cur_fsbno( + struct xfs_btree_cur *cur, + int level) +{ + if (level < cur->bc_nlevels && cur->bc_levels[level].bp) + return XFS_DADDR_TO_FSB(cur->bc_mp, + xfs_buf_daddr(cur->bc_levels[level].bp)); + else if (level == cur->bc_nlevels - 1 && + (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE)) + return XFS_INO_TO_FSB(cur->bc_mp, cur->bc_ino.ip->i_ino); + else if (!(cur->bc_flags & XFS_BTREE_LONG_PTRS)) + return XFS_AGB_TO_FSB(cur->bc_mp, cur->bc_ag.pag->pag_agno, 0); + return NULLFSBLOCK; +} + +static inline void +process_error_whine( + struct xfs_scrub *sc, + struct xfs_btree_cur *cur, + int level, + int *error, + __u32 errflag, + void *ret_ip) +{ + xfs_fsblock_t fsbno = xchk_btree_cur_fsbno(cur, level); + + if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) { + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s btnum %d level %d ptr %d agno 0x%x agbno 0x%x error %d errflag 0x%x ret_ip %pS", + cur->bc_ino.ip->i_ino, + cur->bc_ino.whichfork, + xchk_type_string(sc->sm->sm_type), + cur->bc_btnum, + level, + cur->bc_levels[level].ptr, + XFS_FSB_TO_AGNO(cur->bc_mp, fsbno), + XFS_FSB_TO_AGBNO(cur->bc_mp, fsbno), + *error, + errflag, + ret_ip); + return; + } + + xchk_whine(sc->mp, "type %s btnum %d level %d ptr %d agno 0x%x agbno 0x%x error %d errflag 0x%x ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + cur->bc_btnum, + level, + cur->bc_levels[level].ptr, + XFS_FSB_TO_AGNO(cur->bc_mp, fsbno), + XFS_FSB_TO_AGBNO(cur->bc_mp, fsbno), + *error, + errflag, + ret_ip); +} + /* * Check for btree operation errors. See the section about handling * operational errors in common.c. @@ -44,9 +102,13 @@ __xchk_btree_process_error( case -EFSCORRUPTED: /* Note the badness but don't abort. */ sc->sm->sm_flags |= errflag; + process_error_whine(sc, cur, level, error, errflag, ret_ip); *error = 0; fallthrough; default: + if (*error) + process_error_whine(sc, cur, level, error, errflag, + ret_ip); if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) trace_xchk_ifork_btree_op_error(sc, cur, level, *error, ret_ip); @@ -92,11 +154,37 @@ __xchk_btree_set_corrupt( sc->sm->sm_flags |= errflag; if (cur->bc_flags & XFS_BTREE_ROOT_IN_INODE) + { + xfs_fsblock_t fsbno = xchk_btree_cur_fsbno(cur, level); + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s btnum %d level %d ptr %d agno 0x%x agbno 0x%x errflag 0x%x ret_ip %pS", + cur->bc_ino.ip->i_ino, + cur->bc_ino.whichfork, + xchk_type_string(sc->sm->sm_type), + cur->bc_btnum, + level, + cur->bc_levels[level].ptr, + XFS_FSB_TO_AGNO(cur->bc_mp, fsbno), + XFS_FSB_TO_AGBNO(cur->bc_mp, fsbno), + errflag, + ret_ip); trace_xchk_ifork_btree_error(sc, cur, level, ret_ip); + } else + { + xfs_fsblock_t fsbno = xchk_btree_cur_fsbno(cur, level); + xchk_whine(sc->mp, "type %s btnum %d level %d ptr %d agno 0x%x agbno 0x%x errflag 0x%x ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + cur->bc_btnum, + level, + cur->bc_levels[level].ptr, + XFS_FSB_TO_AGNO(cur->bc_mp, fsbno), + XFS_FSB_TO_AGBNO(cur->bc_mp, fsbno), + errflag, + ret_ip); trace_xchk_btree_error(sc, cur, level, ret_ip); + } } void diff --git a/fs/xfs/scrub/common.c b/fs/xfs/scrub/common.c index a632d56f255f..2c6fd62874c6 100644 --- a/fs/xfs/scrub/common.c +++ b/fs/xfs/scrub/common.c @@ -105,9 +105,23 @@ __xchk_process_error( case -EFSCORRUPTED: /* Note the badness but don't abort. */ sc->sm->sm_flags |= errflag; + xchk_whine(sc->mp, "type %s agno 0x%x agbno 0x%x error %d errflag 0x%x ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + agno, + bno, + *error, + errflag, + ret_ip); *error = 0; fallthrough; default: + if (*error) + xchk_whine(sc->mp, "type %s agno 0x%x agbno 0x%x error %d ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + agno, + bno, + *error, + ret_ip); trace_xchk_op_error(sc, agno, bno, *error, ret_ip); break; } @@ -190,9 +204,25 @@ __xchk_fblock_process_error( case -EFSCORRUPTED: /* Note the badness but don't abort. */ sc->sm->sm_flags |= errflag; + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s offset %llu error %d errflag 0x%x ret_ip %pS", + sc->ip->i_ino, + whichfork, + xchk_type_string(sc->sm->sm_type), + offset, + *error, + errflag, + ret_ip); *error = 0; fallthrough; default: + if (*error) + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s offset %llu error %d ret_ip %pS", + sc->ip->i_ino, + whichfork, + xchk_type_string(sc->sm->sm_type), + offset, + *error, + ret_ip); trace_xchk_file_op_error(sc, whichfork, offset, *error, ret_ip); break; @@ -264,6 +294,8 @@ xchk_set_corrupt( struct xfs_scrub *sc) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + xchk_whine(sc->mp, "type %s ret_ip %pS", xchk_type_string(sc->sm->sm_type), + __return_address); trace_xchk_fs_error(sc, 0, __return_address); } @@ -275,6 +307,11 @@ xchk_block_set_corrupt( { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; trace_xchk_block_error(sc, xfs_buf_daddr(bp), __return_address); + xchk_whine(sc->mp, "type %s agno 0x%x agbno 0x%x ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + xfs_daddr_to_agno(sc->mp, xfs_buf_daddr(bp)), + xfs_daddr_to_agbno(sc->mp, xfs_buf_daddr(bp)), + __return_address); } #ifdef CONFIG_XFS_QUOTA @@ -286,6 +323,8 @@ xchk_qcheck_set_corrupt( xfs_dqid_t id) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + xchk_whine(sc->mp, "type %s dqtype %u id %u ret_ip %pS", + xchk_type_string(sc->sm->sm_type), dqtype, id, __return_address); trace_xchk_qcheck_error(sc, dqtype, id, __return_address); } #endif /* CONFIG_XFS_QUOTA */ @@ -298,6 +337,11 @@ xchk_block_xref_set_corrupt( { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_XCORRUPT; trace_xchk_block_error(sc, xfs_buf_daddr(bp), __return_address); + xchk_whine(sc->mp, "type %s agno 0x%x agbno 0x%x ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + xfs_daddr_to_agno(sc->mp, xfs_buf_daddr(bp)), + xfs_daddr_to_agbno(sc->mp, xfs_buf_daddr(bp)), + __return_address); } /* @@ -311,6 +355,8 @@ xchk_ino_set_corrupt( xfs_ino_t ino) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + xchk_whine(sc->mp, "ino 0x%llx type %s ret_ip %pS", + ino, xchk_type_string(sc->sm->sm_type), __return_address); trace_xchk_ino_error(sc, ino, __return_address); } @@ -321,6 +367,8 @@ xchk_ino_xref_set_corrupt( xfs_ino_t ino) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_XCORRUPT; + xchk_whine(sc->mp, "ino 0x%llx type %s ret_ip %pS", + ino, xchk_type_string(sc->sm->sm_type), __return_address); trace_xchk_ino_error(sc, ino, __return_address); } @@ -332,6 +380,12 @@ xchk_fblock_set_corrupt( xfs_fileoff_t offset) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s offset %llu ret_ip %pS", + sc->ip->i_ino, + whichfork, + xchk_type_string(sc->sm->sm_type), + offset, + __return_address); trace_xchk_fblock_error(sc, whichfork, offset, __return_address); } @@ -343,6 +397,12 @@ xchk_fblock_xref_set_corrupt( xfs_fileoff_t offset) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_XCORRUPT; + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s offset %llu ret_ip %pS", + sc->ip->i_ino, + whichfork, + xchk_type_string(sc->sm->sm_type), + offset, + __return_address); trace_xchk_fblock_error(sc, whichfork, offset, __return_address); } @@ -356,6 +416,8 @@ xchk_ino_set_warning( xfs_ino_t ino) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_WARNING; + xchk_whine(sc->mp, "ino 0x%llx type %s ret_ip %pS", + ino, xchk_type_string(sc->sm->sm_type), __return_address); trace_xchk_ino_warning(sc, ino, __return_address); } @@ -367,6 +429,12 @@ xchk_fblock_set_warning( xfs_fileoff_t offset) { sc->sm->sm_flags |= XFS_SCRUB_OFLAG_WARNING; + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s offset %llu ret_ip %pS", + sc->ip->i_ino, + whichfork, + xchk_type_string(sc->sm->sm_type), + offset, + __return_address); trace_xchk_fblock_warning(sc, whichfork, offset, __return_address); } @@ -1255,6 +1323,10 @@ xchk_iget_for_scrubbing( out_cancel: xchk_trans_cancel(sc); out_error: + xchk_whine(mp, "type %s agno 0x%x agbno 0x%x error %d ret_ip %pS", + xchk_type_string(sc->sm->sm_type), agno, + XFS_INO_TO_AGBNO(mp, sc->sm->sm_ino), error, + __return_address); trace_xchk_op_error(sc, agno, XFS_INO_TO_AGBNO(mp, sc->sm->sm_ino), error, __return_address); return error; @@ -1390,6 +1462,10 @@ xchk_should_check_xref( } sc->sm->sm_flags |= XFS_SCRUB_OFLAG_XFAIL; + xchk_whine(sc->mp, "type %s xref error %d ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + *error, + __return_address); trace_xchk_xref_error(sc, *error, __return_address); /* @@ -1421,6 +1497,11 @@ xchk_buffer_recheck( return; sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; trace_xchk_block_error(sc, xfs_buf_daddr(bp), fa); + xchk_whine(sc->mp, "type %s agno 0x%x agbno 0x%x ret_ip %pS", + xchk_type_string(sc->sm->sm_type), + xfs_daddr_to_agno(sc->mp, xfs_buf_daddr(bp)), + xfs_daddr_to_agbno(sc->mp, xfs_buf_daddr(bp)), + fa); } static inline int @@ -1587,3 +1668,26 @@ xchk_inode_count_blocks( *count = btblocks - 1; return 0; } + +/* Complain about failures... */ +void +xchk_whine( + const struct xfs_mount *mp, + const char *fmt, + ...) +{ + struct va_format vaf; + va_list args; + + va_start(args, fmt); + + vaf.fmt = fmt; + vaf.va = &args; + + printk(KERN_INFO "XFS (%s) %pS: %pV\n", mp->m_super->s_id, + __return_address, &vaf); + va_end(args); + + if (xfs_error_level >= XFS_ERRLEVEL_HIGH) + xfs_stack_trace(); +} diff --git a/fs/xfs/scrub/common.h b/fs/xfs/scrub/common.h index dd1b838a183f..10b124e4b02b 100644 --- a/fs/xfs/scrub/common.h +++ b/fs/xfs/scrub/common.h @@ -210,6 +210,7 @@ bool xchk_ilock_nowait(struct xfs_scrub *sc, unsigned int ilock_flags); void xchk_iunlock(struct xfs_scrub *sc, unsigned int ilock_flags); void xchk_buffer_recheck(struct xfs_scrub *sc, struct xfs_buf *bp); +void xchk_whine(const struct xfs_mount *mp, const char *fmt, ...); int xchk_iget(struct xfs_scrub *sc, xfs_ino_t inum, struct xfs_inode **ipp); int xchk_iget_agi(struct xfs_scrub *sc, xfs_ino_t inum, diff --git a/fs/xfs/scrub/dabtree.c b/fs/xfs/scrub/dabtree.c index 764f7dfd78b5..15c9bfcda0d3 100644 --- a/fs/xfs/scrub/dabtree.c +++ b/fs/xfs/scrub/dabtree.c @@ -47,9 +47,26 @@ xchk_da_process_error( case -EFSCORRUPTED: /* Note the badness but don't abort. */ sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s dablk 0x%llx error %d ret_ip %pS", + sc->ip->i_ino, + ds->dargs.whichfork, + xchk_type_string(sc->sm->sm_type), + xfs_dir2_da_to_db(ds->dargs.geo, + ds->state->path.blk[level].blkno), + *error, + __return_address); *error = 0; fallthrough; default: + if (*error) + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s dablk 0x%llx error %d ret_ip %pS", + sc->ip->i_ino, + ds->dargs.whichfork, + xchk_type_string(sc->sm->sm_type), + xfs_dir2_da_to_db(ds->dargs.geo, + ds->state->path.blk[level].blkno), + *error, + __return_address); trace_xchk_file_op_error(sc, ds->dargs.whichfork, xfs_dir2_da_to_db(ds->dargs.geo, ds->state->path.blk[level].blkno), @@ -72,6 +89,13 @@ xchk_da_set_corrupt( sc->sm->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + xchk_whine(sc->mp, "ino 0x%llx fork %d type %s dablk 0x%llx ret_ip %pS", + sc->ip->i_ino, + ds->dargs.whichfork, + xchk_type_string(sc->sm->sm_type), + xfs_dir2_da_to_db(ds->dargs.geo, + ds->state->path.blk[level].blkno), + __return_address); trace_xchk_fblock_error(sc, ds->dargs.whichfork, xfs_dir2_da_to_db(ds->dargs.geo, ds->state->path.blk[level].blkno), diff --git a/fs/xfs/scrub/inode.c b/fs/xfs/scrub/inode.c index 6a37973823d2..acd44858b2d0 100644 --- a/fs/xfs/scrub/inode.c +++ b/fs/xfs/scrub/inode.c @@ -187,6 +187,10 @@ xchk_setup_inode( out_cancel: xchk_trans_cancel(sc); out_error: + xchk_whine(mp, "type %s agno 0x%x agbno 0x%x error %d ret_ip %pS", + xchk_type_string(sc->sm->sm_type), agno, + XFS_INO_TO_AGBNO(mp, sc->sm->sm_ino), error, + __return_address); trace_xchk_op_error(sc, agno, XFS_INO_TO_AGBNO(mp, sc->sm->sm_ino), error, __return_address); return error; diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c index 2f60fd6b86a9..342a50248650 100644 --- a/fs/xfs/scrub/scrub.c +++ b/fs/xfs/scrub/scrub.c @@ -551,6 +551,42 @@ static inline void xchk_postmortem(struct xfs_scrub *sc) } #endif /* CONFIG_XFS_ONLINE_REPAIR */ +static inline void +repair_outcomes(struct xfs_scrub *sc, int error) +{ + struct xfs_scrub_metadata *sm = sc->sm; + const char *wut = NULL; + + if (sc->flags & XREP_ALREADY_FIXED) { + wut = "*** REPAIR SUCCESS"; + error = 0; + } else if (error == -EBUSY) { + wut = "??? FILESYSTEM BUSY"; + } else if (error == -EAGAIN) { + wut = "??? REPAIR DEFERRED"; + } else if (error == -ECANCELED) { + wut = "??? REPAIR CANCELLED"; + } else if (error == -EINTR) { + wut = "??? REPAIR INTERRUPTED"; + } else if (error != -EOPNOTSUPP && error != -ENOENT) { + wut = "!!! REPAIR FAILED"; + xfs_info(sc->mp, +"%s ino 0x%llx type %s agno 0x%x inum 0x%llx gen 0x%x flags 0x%x error %d", + wut, XFS_I(file_inode(sc->file))->i_ino, + xchk_type_string(sm->sm_type), sm->sm_agno, + sm->sm_ino, sm->sm_gen, sm->sm_flags, error); + return; + } else { + return; + } + + xfs_info_ratelimited(sc->mp, +"%s ino 0x%llx type %s agno 0x%x inum 0x%llx gen 0x%x flags 0x%x error %d", + wut, XFS_I(file_inode(sc->file))->i_ino, + xchk_type_string(sm->sm_type), sm->sm_agno, sm->sm_ino, + sm->sm_gen, sm->sm_flags, error); +} + /* Dispatch metadata scrubbing. */ int xfs_scrub_metadata( @@ -643,6 +679,7 @@ xfs_scrub_metadata( * already tried to fix it, then attempt a repair. */ error = xrep_attempt(sc); + repair_outcomes(sc, error); if (error == -EAGAIN) { /* * Either the repair function succeeded or it couldn't diff --git a/fs/xfs/scrub/trace.c b/fs/xfs/scrub/trace.c index 1bb868a54c06..f1a2b46a9355 100644 --- a/fs/xfs/scrub/trace.c +++ b/fs/xfs/scrub/trace.c @@ -62,3 +62,25 @@ xfbtree_ino( */ #define CREATE_TRACE_POINTS #include "scrub/trace.h" + +/* xchk_whine stuff */ +struct xchk_tstr { + unsigned int type; + const char *tag; +}; + +static const struct xchk_tstr xchk_tstr_tags[] = { XFS_SCRUB_TYPE_STRINGS }; + +const char * +xchk_type_string( + unsigned int type) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(xchk_tstr_tags); i++) { + if (xchk_tstr_tags[i].type == type) + return xchk_tstr_tags[i].tag; + } + + return "???"; +} diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h index 4d8e4b77cbbe..0e945f842732 100644 --- a/fs/xfs/scrub/trace.h +++ b/fs/xfs/scrub/trace.h @@ -115,6 +115,8 @@ TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RTREFCBT); { XFS_SCRUB_TYPE_RTRMAPBT, "rtrmapbt" }, \ { XFS_SCRUB_TYPE_RTREFCBT, "rtrefcountbt" } +const char *xchk_type_string(unsigned int type); + #define XFS_SCRUB_FLAG_STRINGS \ { XFS_SCRUB_IFLAG_REPAIR, "repair" }, \ { XFS_SCRUB_OFLAG_CORRUPT, "corrupt" }, \ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 1/3] xfs: track deferred ops statistics 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: vectorize scrub kernel calls Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: whine to dmesg when we encounter errors Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 3/3] xfs: introduce vectored scrub mode Darrick J. Wong 2 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Track some basic statistics on how hard we're pushing the defer ops. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/libxfs/xfs_defer.c | 14 ++++++++++++++ fs/xfs/xfs_trace.h | 19 +++++++++++++++++++ fs/xfs/xfs_trans.c | 3 +++ fs/xfs/xfs_trans.h | 7 +++++++ 4 files changed, 43 insertions(+) diff --git a/fs/xfs/libxfs/xfs_defer.c b/fs/xfs/libxfs/xfs_defer.c index 1aefb4c99e7b..1b13056f6199 100644 --- a/fs/xfs/libxfs/xfs_defer.c +++ b/fs/xfs/libxfs/xfs_defer.c @@ -509,6 +509,8 @@ xfs_defer_finish_one( /* Done with the dfp, free it. */ list_del(&dfp->dfp_list); kmem_cache_free(xfs_defer_pending_cache, dfp); + tp->t_dfops_nr--; + tp->t_dfops_finished++; out: if (ops->finish_cleanup) ops->finish_cleanup(tp, state, error); @@ -550,6 +552,9 @@ xfs_defer_finish_noroll( list_splice_init(&(*tp)->t_dfops, &dop_pending); + (*tp)->t_dfops_nr_max = max((*tp)->t_dfops_nr, + (*tp)->t_dfops_nr_max); + if (has_intents < 0) { error = has_intents; goto out_shutdown; @@ -580,6 +585,7 @@ xfs_defer_finish_noroll( xfs_force_shutdown((*tp)->t_mountp, SHUTDOWN_CORRUPT_INCORE); trace_xfs_defer_finish_error(*tp, error); xfs_defer_cancel_list((*tp)->t_mountp, &dop_pending); + (*tp)->t_dfops_nr = 0; xfs_defer_cancel(*tp); return error; } @@ -620,6 +626,7 @@ xfs_defer_cancel( trace_xfs_defer_cancel(tp, _RET_IP_); xfs_defer_cancel_list(mp, &tp->t_dfops); + tp->t_dfops_nr = 0; } /* Add an item for later deferred processing. */ @@ -656,6 +663,7 @@ xfs_defer_add( dfp->dfp_count = 0; INIT_LIST_HEAD(&dfp->dfp_work); list_add_tail(&dfp->dfp_list, &tp->t_dfops); + tp->t_dfops_nr++; } list_add_tail(li, &dfp->dfp_work); @@ -674,6 +682,12 @@ xfs_defer_move( struct xfs_trans *stp) { list_splice_init(&stp->t_dfops, &dtp->t_dfops); + dtp->t_dfops_nr += stp->t_dfops_nr; + dtp->t_dfops_nr_max = stp->t_dfops_nr_max; + dtp->t_dfops_finished = stp->t_dfops_finished; + stp->t_dfops_nr = 0; + stp->t_dfops_nr_max = 0; + stp->t_dfops_finished = 0; /* * Low free space mode was historically controlled by a dfops field. diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index 00716f112f4e..e1a94bfc8a13 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -2681,6 +2681,25 @@ TRACE_EVENT(xfs_btree_free_block, /* deferred ops */ struct xfs_defer_pending; +TRACE_EVENT(xfs_defer_stats, + TP_PROTO(struct xfs_trans *tp), + TP_ARGS(tp), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(unsigned int, max) + __field(unsigned int, finished) + ), + TP_fast_assign( + __entry->dev = tp->t_mountp->m_super->s_dev; + __entry->max = tp->t_dfops_nr_max; + __entry->finished = tp->t_dfops_finished; + ), + TP_printk("dev %d:%d max %u finished %u", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->max, + __entry->finished) +) + DECLARE_EVENT_CLASS(xfs_defer_class, TP_PROTO(struct xfs_trans *tp, unsigned long caller_ip), TP_ARGS(tp, caller_ip), diff --git a/fs/xfs/xfs_trans.c b/fs/xfs/xfs_trans.c index fd389a8582fd..3c293c563cae 100644 --- a/fs/xfs/xfs_trans.c +++ b/fs/xfs/xfs_trans.c @@ -70,6 +70,9 @@ xfs_trans_free( xfs_extent_busy_sort(&tp->t_busy); xfs_extent_busy_clear(tp->t_mountp, &tp->t_busy, false); + if (tp->t_dfops_finished > 0) + trace_xfs_defer_stats(tp); + trace_xfs_trans_free(tp, _RET_IP_); xfs_trans_clear_context(tp); if (!(tp->t_flags & XFS_TRANS_NO_WRITECOUNT)) diff --git a/fs/xfs/xfs_trans.h b/fs/xfs/xfs_trans.h index efa7eace0859..09df8cedee8d 100644 --- a/fs/xfs/xfs_trans.h +++ b/fs/xfs/xfs_trans.h @@ -155,6 +155,13 @@ typedef struct xfs_trans { struct list_head t_busy; /* list of busy extents */ struct list_head t_dfops; /* deferred operations */ unsigned long t_pflags; /* saved process flags state */ + + /* Count of deferred ops attached to transaction. */ + unsigned int t_dfops_nr; + /* Maximum t_dfops_nr seen in a loop. */ + unsigned int t_dfops_nr_max; + /* Number of dfops finished. */ + unsigned int t_dfops_finished; } xfs_trans_t; /* ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 3/3] xfs: introduce vectored scrub mode 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: vectorize scrub kernel calls Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: whine to dmesg when we encounter errors Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/3] xfs: track deferred ops statistics Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Introduce a variant on XFS_SCRUB_METADATA that allows for vectored mode. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/libxfs/xfs_fs.h | 37 ++++++++++++ fs/xfs/scrub/scrub.c | 148 ++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/trace.h | 78 ++++++++++++++++++++++++ fs/xfs/scrub/xfs_scrub.h | 2 + fs/xfs/xfs_ioctl.c | 47 +++++++++++++++ 5 files changed, 311 insertions(+), 1 deletion(-) diff --git a/fs/xfs/libxfs/xfs_fs.h b/fs/xfs/libxfs/xfs_fs.h index 453b08612256..067dd0b1315b 100644 --- a/fs/xfs/libxfs/xfs_fs.h +++ b/fs/xfs/libxfs/xfs_fs.h @@ -751,6 +751,15 @@ struct xfs_scrub_metadata { /* Number of scrub subcommands. */ #define XFS_SCRUB_TYPE_NR 32 +/* + * This special type code only applies to the vectored scrub implementation. + * + * If any of the previous scrub vectors recorded runtime errors or have + * sv_flags bits set that match the OFLAG bits in the barrier vector's + * sv_flags, set the barrier's sv_ret to -ECANCELED and return to userspace. + */ +#define XFS_SCRUB_TYPE_BARRIER (-1U) + /* i: Repair this metadata. */ #define XFS_SCRUB_IFLAG_REPAIR (1u << 0) @@ -795,6 +804,33 @@ struct xfs_scrub_metadata { XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) #define XFS_SCRUB_FLAGS_ALL (XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT) +struct xfs_scrub_vec { + __u32 sv_type; /* XFS_SCRUB_TYPE_* */ + __u32 sv_flags; /* XFS_SCRUB_FLAGS_* */ + __s32 sv_ret; /* 0 or a negative error code */ + __u32 sv_reserved; /* must be zero */ +}; + +/* Vectored metadata scrub control structure. */ +struct xfs_scrub_vec_head { + __u64 svh_ino; /* inode number. */ + __u32 svh_gen; /* inode generation. */ + __u32 svh_agno; /* ag number. */ + __u32 svh_flags; /* XFS_SCRUB_VEC_FLAGS_* */ + __u16 svh_rest_us; /* wait this much time between vector items */ + __u16 svh_nr; /* number of svh_vecs */ + + struct xfs_scrub_vec svh_vecs[0]; +}; + +#define XFS_SCRUB_VEC_FLAGS_ALL (0) + +static inline size_t sizeof_xfs_scrub_vec(unsigned int nr) +{ + return sizeof(struct xfs_scrub_vec_head) + + nr * sizeof(struct xfs_scrub_vec); +} + /* * ioctl limits */ @@ -839,6 +875,7 @@ struct xfs_scrub_metadata { #define XFS_IOC_FREE_EOFBLOCKS _IOR ('X', 58, struct xfs_fs_eofblocks) /* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata) +#define XFS_IOC_SCRUBV_METADATA _IOWR('X', 60, struct xfs_scrub_vec_head) #define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) #define XFS_IOC_RTGROUP_GEOMETRY _IOWR('X', 62, struct xfs_rtgroup_geometry) diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c index 342a50248650..fc2cfef68366 100644 --- a/fs/xfs/scrub/scrub.c +++ b/fs/xfs/scrub/scrub.c @@ -20,6 +20,7 @@ #include "xfs_rmap.h" #include "xfs_xchgrange.h" #include "xfs_swapext.h" +#include "xfs_icache.h" #include "scrub/scrub.h" #include "scrub/common.h" #include "scrub/trace.h" @@ -726,3 +727,150 @@ xfs_scrub_metadata( sc->flags |= XCHK_TRY_HARDER; goto retry_op; } + +/* Decide if there have been any scrub failures up to this point. */ +static inline bool +xfs_scrubv_previous_failures( + struct xfs_mount *mp, + struct xfs_scrub_vec_head *vhead, + struct xfs_scrub_vec *barrier_vec) +{ + struct xfs_scrub_vec *v; + __u32 failmask; + + failmask = barrier_vec->sv_flags & XFS_SCRUB_FLAGS_OUT; + + for (v = vhead->svh_vecs; v < barrier_vec; v++) { + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) + continue; + + /* + * Runtime errors count as a previous failure, except the ones + * used to ask userspace to retry. + */ + if (v->sv_ret && v->sv_ret != -EBUSY && v->sv_ret != -ENOENT && + v->sv_ret != -EUSERS) + return true; + + /* + * If any of the out-flags on the scrub vector match the mask + * that was set on the barrier vector, that's a previous fail. + */ + if (v->sv_flags & failmask) + return true; + } + + return false; +} + +/* Vectored scrub implementation to reduce ioctl calls. */ +int +xfs_scrubv_metadata( + struct file *file, + struct xfs_scrub_vec_head *vhead) +{ + struct xfs_inode *ip_in = XFS_I(file_inode(file)); + struct xfs_mount *mp = ip_in->i_mount; + struct xfs_inode *ip = NULL; + struct xfs_scrub_vec *v; + bool set_dontcache = false; + unsigned int i; + int error = 0; + + BUILD_BUG_ON(sizeof(struct xfs_scrub_vec_head) == + sizeof(struct xfs_scrub_metadata)); + BUILD_BUG_ON(XFS_IOC_SCRUB_METADATA == XFS_IOC_SCRUBV_METADATA); + + trace_xchk_scrubv_start(ip_in, vhead); + + if (vhead->svh_flags & ~XFS_SCRUB_VEC_FLAGS_ALL) + return -EINVAL; + for (i = 0, v = vhead->svh_vecs; i < vhead->svh_nr; i++, v++) { + if (v->sv_reserved) + return -EINVAL; + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER && + (v->sv_flags & ~XFS_SCRUB_FLAGS_OUT)) + return -EINVAL; + + /* + * If we detect at least one inode-type scrub, we might + * consider setting dontcache at the end. + */ + if (v->sv_type < XFS_SCRUB_TYPE_NR && + meta_scrub_ops[v->sv_type].type == ST_INODE) + set_dontcache = true; + + trace_xchk_scrubv_item(mp, vhead, v); + } + + /* + * If the caller provided us with a nonzero inode number that isn't the + * ioctl file, try to grab a reference to it to eliminate all further + * untrusted inode lookups. If we can't get the inode, let each scrub + * function try again. + */ + if (vhead->svh_ino != ip_in->i_ino) { + xfs_iget(mp, NULL, vhead->svh_ino, XFS_IGET_UNTRUSTED, 0, &ip); + if (ip && (VFS_I(ip)->i_generation != vhead->svh_gen || + (xfs_is_metadata_inode(ip) && + !S_ISDIR(VFS_I(ip)->i_mode)))) { + xfs_irele(ip); + ip = NULL; + } + } + if (!ip) { + if (!igrab(VFS_I(ip_in))) + return -EFSCORRUPTED; + ip = ip_in; + } + + /* Run all the scrubbers. */ + for (i = 0, v = vhead->svh_vecs; i < vhead->svh_nr; i++, v++) { + struct xfs_scrub_metadata sm = { + .sm_type = v->sv_type, + .sm_flags = v->sv_flags, + .sm_ino = vhead->svh_ino, + .sm_gen = vhead->svh_gen, + .sm_agno = vhead->svh_agno, + }; + + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) { + if (xfs_scrubv_previous_failures(mp, vhead, v)) { + v->sv_ret = -ECANCELED; + trace_xchk_scrubv_barrier_fail(mp, vhead, v); + break; + } + + continue; + } + + v->sv_ret = xfs_scrub_metadata(file, &sm); + v->sv_flags = sm.sm_flags; + + /* Leave the inode in memory if something's wrong with it. */ + if (xchk_needs_repair(&sm)) + set_dontcache = false; + + if (vhead->svh_rest_us) { + ktime_t expires; + + expires = ktime_add_ns(ktime_get(), + vhead->svh_rest_us * 1000); + set_current_state(TASK_KILLABLE); + schedule_hrtimeout(&expires, HRTIMER_MODE_ABS); + } + if (fatal_signal_pending(current)) { + error = -EINTR; + break; + } + } + + /* + * If we're holding the only reference to this inode and the scan was + * clean, mark it dontcache so that we don't pollute the cache. + */ + if (set_dontcache && atomic_read(&VFS_I(ip)->i_count) == 1) + d_mark_dontcache(VFS_I(ip)); + xfs_irele(ip); + return error; +} diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h index 0e945f842732..8767dd39b80c 100644 --- a/fs/xfs/scrub/trace.h +++ b/fs/xfs/scrub/trace.h @@ -80,6 +80,7 @@ TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RGSUPER); TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RGBITMAP); TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RTRMAPBT); TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RTREFCBT); +TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_BARRIER); #define XFS_SCRUB_TYPE_STRINGS \ { XFS_SCRUB_TYPE_PROBE, "probe" }, \ @@ -113,7 +114,8 @@ TRACE_DEFINE_ENUM(XFS_SCRUB_TYPE_RTREFCBT); { XFS_SCRUB_TYPE_RGSUPER, "rgsuper" }, \ { XFS_SCRUB_TYPE_RGBITMAP, "rgbitmap" }, \ { XFS_SCRUB_TYPE_RTRMAPBT, "rtrmapbt" }, \ - { XFS_SCRUB_TYPE_RTREFCBT, "rtrefcountbt" } + { XFS_SCRUB_TYPE_RTREFCBT, "rtrefcountbt" }, \ + { XFS_SCRUB_TYPE_BARRIER, "barrier" } const char *xchk_type_string(unsigned int type); @@ -213,6 +215,80 @@ DEFINE_EVENT(xchk_fshook_class, name, \ DEFINE_SCRUB_FSHOOK_EVENT(xchk_fshooks_enable); DEFINE_SCRUB_FSHOOK_EVENT(xchk_fshooks_disable); +DECLARE_EVENT_CLASS(xchk_vector_head_class, + TP_PROTO(struct xfs_inode *ip, struct xfs_scrub_vec_head *vhead), + TP_ARGS(ip, vhead), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(xfs_agnumber_t, agno) + __field(xfs_ino_t, inum) + __field(unsigned int, gen) + __field(unsigned int, flags) + __field(unsigned short, rest_us) + __field(unsigned short, nr_vecs) + ), + TP_fast_assign( + __entry->dev = ip->i_mount->m_super->s_dev; + __entry->ino = ip->i_ino; + __entry->agno = vhead->svh_agno; + __entry->inum = vhead->svh_ino; + __entry->gen = vhead->svh_gen; + __entry->flags = vhead->svh_flags; + __entry->rest_us = vhead->svh_rest_us; + __entry->nr_vecs = vhead->svh_nr; + ), + TP_printk("dev %d:%d ino 0x%llx agno 0x%x inum 0x%llx gen 0x%x flags 0x%x rest_us %u nr_vecs %u", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->ino, + __entry->agno, + __entry->inum, + __entry->gen, + __entry->flags, + __entry->rest_us, + __entry->nr_vecs) +) +#define DEFINE_SCRUBV_HEAD_EVENT(name) \ +DEFINE_EVENT(xchk_vector_head_class, name, \ + TP_PROTO(struct xfs_inode *ip, struct xfs_scrub_vec_head *vhead), \ + TP_ARGS(ip, vhead)) + +DEFINE_SCRUBV_HEAD_EVENT(xchk_scrubv_start); + +DECLARE_EVENT_CLASS(xchk_vector_class, + TP_PROTO(struct xfs_mount *mp, struct xfs_scrub_vec_head *vhead, + struct xfs_scrub_vec *v), + TP_ARGS(mp, vhead, v), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(unsigned int, vec_nr) + __field(unsigned int, vec_type) + __field(unsigned int, vec_flags) + __field(int, vec_ret) + ), + TP_fast_assign( + __entry->dev = mp->m_super->s_dev; + __entry->vec_nr = v - vhead->svh_vecs; + __entry->vec_type = v->sv_type; + __entry->vec_flags = v->sv_flags; + __entry->vec_ret = v->sv_ret; + ), + TP_printk("dev %d:%d vec[%u] type %s flags 0x%x ret %d", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->vec_nr, + __print_symbolic(__entry->vec_type, XFS_SCRUB_TYPE_STRINGS), + __entry->vec_flags, + __entry->vec_ret) +) +#define DEFINE_SCRUBV_EVENT(name) \ +DEFINE_EVENT(xchk_vector_class, name, \ + TP_PROTO(struct xfs_mount *mp, struct xfs_scrub_vec_head *vhead, \ + struct xfs_scrub_vec *v), \ + TP_ARGS(mp, vhead, v)) + +DEFINE_SCRUBV_EVENT(xchk_scrubv_barrier_fail); +DEFINE_SCRUBV_EVENT(xchk_scrubv_item); + TRACE_EVENT(xchk_op_error, TP_PROTO(struct xfs_scrub *sc, xfs_agnumber_t agno, xfs_agblock_t bno, int error, void *ret_ip), diff --git a/fs/xfs/scrub/xfs_scrub.h b/fs/xfs/scrub/xfs_scrub.h index 2ceae614ade8..bdf89242e6cd 100644 --- a/fs/xfs/scrub/xfs_scrub.h +++ b/fs/xfs/scrub/xfs_scrub.h @@ -8,8 +8,10 @@ #ifndef CONFIG_XFS_ONLINE_SCRUB # define xfs_scrub_metadata(file, sm) (-ENOTTY) +# define xfs_scrubv_metadata(file, vhead) (-ENOTTY) #else int xfs_scrub_metadata(struct file *file, struct xfs_scrub_metadata *sm); +int xfs_scrubv_metadata(struct file *file, struct xfs_scrub_vec_head *vhead); #endif /* CONFIG_XFS_ONLINE_SCRUB */ #endif /* __XFS_SCRUB_H__ */ diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c index abca384c86a4..47704a7854cf 100644 --- a/fs/xfs/xfs_ioctl.c +++ b/fs/xfs/xfs_ioctl.c @@ -1643,6 +1643,51 @@ xfs_ioc_scrub_metadata( return 0; } +STATIC int +xfs_ioc_scrubv_metadata( + struct file *filp, + void __user *arg) +{ + struct xfs_scrub_vec_head __user *uhead = arg; + struct xfs_scrub_vec_head head; + struct xfs_scrub_vec_head *vhead; + size_t bytes; + int error; + + if (!capable(CAP_SYS_ADMIN)) + return -EPERM; + + if (copy_from_user(&head, uhead, sizeof(head))) + return -EFAULT; + + bytes = sizeof_xfs_scrub_vec(head.svh_nr); + if (bytes > PAGE_SIZE) + return -ENOMEM; + vhead = kvmalloc(bytes, GFP_KERNEL | __GFP_RETRY_MAYFAIL); + if (!vhead) + return -ENOMEM; + memcpy(vhead, &head, sizeof(struct xfs_scrub_vec_head)); + + if (copy_from_user(&vhead->svh_vecs, &uhead->svh_vecs, + head.svh_nr * sizeof(struct xfs_scrub_vec))) { + error = -EFAULT; + goto err_free; + } + + error = xfs_scrubv_metadata(filp, vhead); + if (error) + goto err_free; + + if (copy_to_user(uhead, vhead, bytes)) { + error = -EFAULT; + goto err_free; + } + +err_free: + kvfree(vhead); + return error; +} + int xfs_ioc_swapext( struct xfs_swapext *sxp) @@ -1908,6 +1953,8 @@ xfs_file_ioctl( case FS_IOC_GETFSMAP: return xfs_ioc_getfsmap(ip, arg); + case XFS_IOC_SCRUBV_METADATA: + return xfs_ioc_scrubv_metadata(filp, arg); case XFS_IOC_SCRUB_METADATA: return xfs_ioc_scrub_metadata(filp, arg); ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/1] xfs: report refcount information to userspace 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: vectorize scrub kernel calls Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/1] xfs: export reference count " Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/2] xfs: defragment free space Darrick J. Wong ` (6 subsequent siblings) 9 siblings, 1 reply; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs Hi all, Create a new ioctl to report the number of owners of each disk block so that reflink-aware defraggers can make better decisions about which extents to target. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=report-refcounts xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=report-refcounts fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=report-refcounts --- fs/xfs/Makefile | 1 fs/xfs/xfs_fsrefs.c | 935 ++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_fsrefs.h | 34 + fs/xfs/xfs_ioctl.c | 135 +++++ fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 74 +++ include/uapi/linux/fsrefcounts.h | 96 ++++ 7 files changed, 1276 insertions(+) create mode 100644 fs/xfs/xfs_fsrefs.c create mode 100644 fs/xfs/xfs_fsrefs.h create mode 100644 include/uapi/linux/fsrefcounts.h ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 1/1] xfs: export reference count information to userspace 2022-12-30 22:19 ` [PATCHSET 0/1] xfs: report refcount information to userspace Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 0 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Export refcount info to userspace so we can prototype a sharing-aware defrag/fs rearranging tool. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/Makefile | 1 fs/xfs/xfs_fsrefs.c | 935 ++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_fsrefs.h | 34 + fs/xfs/xfs_ioctl.c | 135 +++++ fs/xfs/xfs_trace.c | 1 fs/xfs/xfs_trace.h | 74 +++ include/uapi/linux/fsrefcounts.h | 96 ++++ 7 files changed, 1276 insertions(+) create mode 100644 fs/xfs/xfs_fsrefs.c create mode 100644 fs/xfs/xfs_fsrefs.h create mode 100644 include/uapi/linux/fsrefcounts.h diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile index 2f84dff55b6e..168302373a9f 100644 --- a/fs/xfs/Makefile +++ b/fs/xfs/Makefile @@ -78,6 +78,7 @@ xfs-y += xfs_aops.o \ xfs_filestream.o \ xfs_fsmap.o \ xfs_fsops.o \ + xfs_fsrefs.o \ xfs_globals.o \ xfs_health.o \ xfs_icache.o \ diff --git a/fs/xfs/xfs_fsrefs.c b/fs/xfs/xfs_fsrefs.c new file mode 100644 index 000000000000..d35848bfbd86 --- /dev/null +++ b/fs/xfs/xfs_fsrefs.c @@ -0,0 +1,935 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_shared.h" +#include "xfs_format.h" +#include "xfs_log_format.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_inode.h" +#include "xfs_trans.h" +#include "xfs_btree.h" +#include "xfs_trace.h" +#include "xfs_alloc.h" +#include "xfs_bit.h" +#include <linux/fsrefcounts.h> +#include "xfs_fsrefs.h" +#include "xfs_refcount.h" +#include "xfs_refcount_btree.h" +#include "xfs_alloc_btree.h" +#include "xfs_rtalloc.h" +#include "xfs_rtrefcount_btree.h" +#include "xfs_ag.h" +#include "xfs_rtbitmap.h" +#include "xfs_rtgroup.h" + +/* getfsrefs query state */ +struct xfs_fsrefs_info { + struct xfs_fsrefs_head *head; + struct fsrefs *fsrefs_recs; /* mapping records */ + + struct xfs_btree_cur *refc_cur; /* refcount btree cursor */ + struct xfs_btree_cur *bno_cur; /* bnobt btree cursor */ + + struct xfs_buf *agf_bp; /* AGF, for refcount queries */ + struct xfs_perag *pag; /* perag structure */ + struct xfs_rtgroup *rtg; + + xfs_daddr_t next_daddr; /* next daddr we expect */ + /* daddr of low fsmap key when we're using the rtbitmap */ + xfs_daddr_t low_daddr; + + struct xfs_refcount_irec low; /* low refcount key */ + struct xfs_refcount_irec high; /* high refcount key */ + + u32 dev; /* device id */ + bool last; /* last extent? */ +}; + +/* Associate a device with a getfsrefs handler. */ +struct xfs_fsrefs_dev { + u32 dev; + int (*fn)(struct xfs_trans *tp, + const struct xfs_fsrefs *keys, + struct xfs_fsrefs_info *info); +}; + +/* Convert an xfs_fsrefs to an fsrefs. */ +static void +xfs_fsrefs_from_internal( + struct fsrefs *dest, + struct xfs_fsrefs *src) +{ + dest->fcr_device = src->fcr_device; + dest->fcr_flags = src->fcr_flags; + dest->fcr_physical = BBTOB(src->fcr_physical); + dest->fcr_owners = src->fcr_owners; + dest->fcr_length = BBTOB(src->fcr_length); + dest->fcr_reserved[0] = 0; + dest->fcr_reserved[1] = 0; + dest->fcr_reserved[2] = 0; + dest->fcr_reserved[3] = 0; +} + +/* Convert an fsrefs to an xfs_fsrefs. */ +void +xfs_fsrefs_to_internal( + struct xfs_fsrefs *dest, + struct fsrefs *src) +{ + dest->fcr_device = src->fcr_device; + dest->fcr_flags = src->fcr_flags; + dest->fcr_physical = BTOBBT(src->fcr_physical); + dest->fcr_owners = src->fcr_owners; + dest->fcr_length = BTOBBT(src->fcr_length); +} + +/* Compare two getfsrefs device handlers. */ +static int +xfs_fsrefs_dev_compare( + const void *p1, + const void *p2) +{ + const struct xfs_fsrefs_dev *d1 = p1; + const struct xfs_fsrefs_dev *d2 = p2; + + return d1->dev - d2->dev; +} + +static inline bool +xfs_fsrefs_rec_before_start( + struct xfs_fsrefs_info *info, + const struct xfs_refcount_irec *rec, + xfs_daddr_t rec_daddr) +{ + if (info->low_daddr != -1ULL) + return rec_daddr < info->low_daddr; + return rec->rc_startblock < info->low.rc_startblock; +} + +/* + * Format a refcount record for fsrefs, having translated rc_startblock into + * the appropriate daddr units. + */ +STATIC int +xfs_fsrefs_helper( + struct xfs_trans *tp, + struct xfs_fsrefs_info *info, + const struct xfs_refcount_irec *rec, + xfs_daddr_t rec_daddr, + xfs_daddr_t len_daddr) +{ + struct xfs_fsrefs fcr; + struct fsrefs *row; + struct xfs_mount *mp = tp->t_mountp; + + if (fatal_signal_pending(current)) + return -EINTR; + + if (len_daddr == 0) + len_daddr = XFS_FSB_TO_BB(mp, rec->rc_blockcount); + + /* + * Filter out records that start before our startpoint, if the + * caller requested that. + */ + if (xfs_fsrefs_rec_before_start(info, rec, rec_daddr)) + return 0; + + /* Are we just counting mappings? */ + if (info->head->fch_count == 0) { + if (info->head->fch_entries == UINT_MAX) + return -ECANCELED; + + info->head->fch_entries++; + return 0; + } + + /* Fill out the extent we found */ + if (info->head->fch_entries >= info->head->fch_count) + return -ECANCELED; + + if (info->pag) + trace_xfs_fsrefs_mapping(mp, info->dev, info->pag->pag_agno, + rec); + else if (info->rtg) + trace_xfs_fsrefs_mapping(mp, info->dev, info->rtg->rtg_rgno, + rec); + else + trace_xfs_fsrefs_mapping(mp, info->dev, NULLAGNUMBER, rec); + + fcr.fcr_device = info->dev; + fcr.fcr_flags = 0; + fcr.fcr_physical = rec_daddr; + fcr.fcr_owners = rec->rc_refcount; + fcr.fcr_length = len_daddr; + + trace_xfs_getfsrefs_mapping(mp, &fcr); + + row = &info->fsrefs_recs[info->head->fch_entries++]; + xfs_fsrefs_from_internal(row, &fcr); + return 0; +} + +/* Synthesize fsrefs records from free space data. */ +STATIC int +xfs_fsrefs_ddev_bnobt_helper( + struct xfs_btree_cur *cur, + const struct xfs_alloc_rec_incore *rec, + void *priv) +{ + struct xfs_refcount_irec irec; + struct xfs_mount *mp = cur->bc_mp; + struct xfs_fsrefs_info *info = priv; + xfs_agnumber_t next_agno; + xfs_agblock_t next_agbno; + xfs_daddr_t rec_daddr; + + /* + * Figure out if there's a gap between the last fsrefs record we + * emitted and this free extent. If there is, report the gap as a + * refcount==1 record. + */ + next_agno = xfs_daddr_to_agno(mp, info->next_daddr); + next_agbno = xfs_daddr_to_agbno(mp, info->next_daddr); + + ASSERT(next_agno >= cur->bc_ag.pag->pag_agno); + ASSERT(rec->ar_startblock >= next_agbno); + + /* + * If we've already moved on to the next AG, we don't have any fsrefs + * records to synthesize. + */ + if (next_agno > cur->bc_ag.pag->pag_agno) + return 0; + + info->next_daddr = XFS_AGB_TO_DADDR(mp, cur->bc_ag.pag->pag_agno, + rec->ar_startblock + rec->ar_blockcount); + + if (rec->ar_startblock == next_agbno) + return 0; + + /* Emit a record for the in-use space */ + irec.rc_startblock = next_agbno; + irec.rc_blockcount = rec->ar_startblock - next_agbno; + irec.rc_refcount = 1; + irec.rc_domain = XFS_REFC_DOMAIN_SHARED; + rec_daddr = XFS_AGB_TO_DADDR(mp, cur->bc_ag.pag->pag_agno, + irec.rc_startblock); + + return xfs_fsrefs_helper(cur->bc_tp, info, &irec, rec_daddr, 0); +} + +/* Emit records to fill a gap in the refcount btree with singly-owned blocks. */ +STATIC int +xfs_fsrefs_ddev_fill_refcount_gap( + struct xfs_trans *tp, + struct xfs_fsrefs_info *info, + xfs_agblock_t agbno) +{ + struct xfs_alloc_rec_incore low = {0}; + struct xfs_alloc_rec_incore high = {0}; + struct xfs_mount *mp = tp->t_mountp; + struct xfs_btree_cur *cur = info->bno_cur; + struct xfs_agf *agf; + int error; + + ASSERT(xfs_daddr_to_agno(mp, info->next_daddr) == + cur->bc_ag.pag->pag_agno); + + low.ar_startblock = xfs_daddr_to_agbno(mp, info->next_daddr); + if (low.ar_startblock >= agbno) + return 0; + + high.ar_startblock = agbno; + error = xfs_alloc_query_range(cur, &low, &high, + xfs_fsrefs_ddev_bnobt_helper, info); + if (error) + return error; + + /* + * Synthesize records for single-owner extents between the last + * fsrefcount record emitted and the end of the query range. + */ + agf = cur->bc_ag.agbp->b_addr; + low.ar_startblock = min_t(xfs_agblock_t, agbno, + be32_to_cpu(agf->agf_length)); + if (xfs_daddr_to_agbno(mp, info->next_daddr) > low.ar_startblock) + return 0; + + info->last = true; + return xfs_fsrefs_ddev_bnobt_helper(cur, &low, info); +} + +/* Transform a refcountbt irec into a fsrefs */ +STATIC int +xfs_fsrefs_ddev_helper( + struct xfs_btree_cur *cur, + const struct xfs_refcount_irec *rec, + void *priv) +{ + struct xfs_mount *mp = cur->bc_mp; + struct xfs_fsrefs_info *info = priv; + xfs_daddr_t rec_daddr; + int error; + + /* + * Stop once we get to the CoW staging extents; they're all shoved to + * the right side of the btree and were already covered by the bnobt + * scan. + */ + if (rec->rc_domain != XFS_REFC_DOMAIN_SHARED) + return -ECANCELED; + + /* Report on any gaps first */ + error = xfs_fsrefs_ddev_fill_refcount_gap(cur->bc_tp, info, + rec->rc_startblock); + if (error) + return error; + + rec_daddr = XFS_AGB_TO_DADDR(mp, cur->bc_ag.pag->pag_agno, + rec->rc_startblock); + info->next_daddr = XFS_AGB_TO_DADDR(mp, cur->bc_ag.pag->pag_agno, + rec->rc_startblock + rec->rc_blockcount); + + return xfs_fsrefs_helper(cur->bc_tp, info, rec, rec_daddr, 0); +} + +/* Execute a getfsrefs query against the regular data device. */ +STATIC int +xfs_fsrefs_ddev( + struct xfs_trans *tp, + const struct xfs_fsrefs *keys, + struct xfs_fsrefs_info *info) +{ + struct xfs_mount *mp = tp->t_mountp; + struct xfs_buf *agf_bp = NULL; + struct xfs_perag *pag = NULL; + xfs_fsblock_t start_fsb; + xfs_fsblock_t end_fsb; + xfs_agnumber_t start_ag; + xfs_agnumber_t end_ag; + xfs_agnumber_t agno; + uint64_t eofs; + int error = 0; + + eofs = XFS_FSB_TO_BB(mp, mp->m_sb.sb_dblocks); + if (keys[0].fcr_physical >= eofs) + return 0; + start_fsb = XFS_DADDR_TO_FSB(mp, keys[0].fcr_physical); + end_fsb = XFS_DADDR_TO_FSB(mp, min(eofs - 1, keys[1].fcr_physical)); + + info->refc_cur = info->bno_cur = NULL; + + /* + * Convert the fsrefs low/high keys to AG based keys. Initialize + * low to the fsrefs low key and max out the high key to the end + * of the AG. + */ + info->low.rc_startblock = XFS_FSB_TO_AGBNO(mp, start_fsb); + info->low.rc_blockcount = 0; + info->low.rc_refcount = 0; + info->low.rc_domain = XFS_REFC_DOMAIN_SHARED; + + info->high.rc_startblock = -1U; + info->high.rc_blockcount = 0; + info->high.rc_refcount = 0; + info->high.rc_domain = XFS_REFC_DOMAIN_SHARED; + + start_ag = XFS_FSB_TO_AGNO(mp, start_fsb); + end_ag = XFS_FSB_TO_AGNO(mp, end_fsb); + + /* Query each AG */ + agno = start_ag; + for_each_perag_range(mp, agno, end_ag, pag) { + /* + * Set the AG high key from the fsrefs high key if this + * is the last AG that we're querying. + */ + info->pag = pag; + if (pag->pag_agno == end_ag) + info->high.rc_startblock = XFS_FSB_TO_AGBNO(mp, + end_fsb); + + if (info->refc_cur) { + xfs_btree_del_cursor(info->refc_cur, XFS_BTREE_NOERROR); + info->refc_cur = NULL; + } + if (info->bno_cur) { + xfs_btree_del_cursor(info->bno_cur, XFS_BTREE_NOERROR); + info->bno_cur = NULL; + } + if (agf_bp) { + xfs_trans_brelse(tp, agf_bp); + agf_bp = NULL; + } + + error = xfs_alloc_read_agf(pag, tp, 0, &agf_bp); + if (error) + break; + + trace_xfs_fsrefs_low_key(mp, info->dev, pag->pag_agno, + &info->low); + trace_xfs_fsrefs_high_key(mp, info->dev, pag->pag_agno, + &info->high); + + info->bno_cur = xfs_allocbt_init_cursor(mp, tp, agf_bp, pag, + XFS_BTNUM_BNO); + + if (xfs_has_reflink(mp)) { + info->refc_cur = xfs_refcountbt_init_cursor(mp, tp, + agf_bp, pag); + + /* + * Fill the query with refcount records and synthesize + * singly-owned block records from free space data. + */ + error = xfs_refcount_query_range(info->refc_cur, + &info->low, &info->high, + xfs_fsrefs_ddev_helper, info); + if (error && error != -ECANCELED) + break; + } + + /* + * Synthesize refcount==1 records from the free space data + * between the end of the last fsrefs record reported and the + * end of the range. If we don't have refcount support, the + * starting point will be the start of the query range. + */ + error = xfs_fsrefs_ddev_fill_refcount_gap(tp, info, + info->high.rc_startblock); + if (error) + break; + + /* + * Set the AG low key to the start of the AG prior to + * moving on to the next AG. + */ + if (pag->pag_agno == start_ag) + info->low.rc_startblock = 0; + + info->pag = NULL; + } + + if (info->refc_cur) { + xfs_btree_del_cursor(info->refc_cur, error); + info->refc_cur = NULL; + } + if (info->bno_cur) { + xfs_btree_del_cursor(info->bno_cur, error); + info->bno_cur = NULL; + } + if (agf_bp) + xfs_trans_brelse(tp, agf_bp); + if (info->pag) { + xfs_perag_put(info->pag); + info->pag = NULL; + } else if (pag) { + /* loop termination case */ + xfs_perag_put(pag); + } + + return error; +} + +/* Execute a getfsrefs query against the log device. */ +STATIC int +xfs_fsrefs_logdev( + struct xfs_trans *tp, + const struct xfs_fsrefs *keys, + struct xfs_fsrefs_info *info) +{ + struct xfs_mount *mp = tp->t_mountp; + struct xfs_refcount_irec refc; + + /* Set up search keys */ + info->low.rc_startblock = XFS_BB_TO_FSBT(mp, keys[0].fcr_physical); + info->low.rc_blockcount = 0; + info->low.rc_refcount = 0; + + info->high.rc_startblock = -1U; + info->high.rc_blockcount = 0; + info->high.rc_refcount = 0; + + trace_xfs_fsrefs_low_key(mp, info->dev, 0, &info->low); + trace_xfs_fsrefs_high_key(mp, info->dev, 0, &info->high); + + if (keys[0].fcr_physical > 0) + return 0; + + /* Fabricate an refc entry for the external log device. */ + refc.rc_startblock = 0; + refc.rc_blockcount = mp->m_sb.sb_logblocks; + refc.rc_refcount = 1; + refc.rc_domain = XFS_REFC_DOMAIN_SHARED; + + return xfs_fsrefs_helper(tp, info, &refc, 0, 0); +} + +#ifdef CONFIG_XFS_RT +/* Synthesize fsrefs records from rtbitmap records. */ +STATIC int +xfs_fsrefs_rtdev_bitmap_helper( + struct xfs_mount *mp, + struct xfs_trans *tp, + const struct xfs_rtalloc_rec *rec, + void *priv) +{ + struct xfs_refcount_irec irec; + struct xfs_fsrefs_info *info = priv; + xfs_rtblock_t rt_startblock; + xfs_rtblock_t rec_rtlen; + xfs_rtblock_t next_rtbno; + xfs_daddr_t rec_daddr; + + /* + * Figure out if there's a gap between the last fsrefs record we + * emitted and this free extent. If there is, report the gap as a + * refcount==1 record. + */ + next_rtbno = XFS_BB_TO_FSBT(mp, info->next_daddr); + rt_startblock = xfs_rtx_to_rtb(mp, rec->ar_startext); + rec_rtlen = xfs_rtx_to_rtb(mp, rec->ar_extcount); + + ASSERT(rt_startblock >= next_rtbno); + + info->next_daddr = XFS_FSB_TO_BB(mp, rt_startblock + rec_rtlen); + + if (rt_startblock == next_rtbno) + return 0; + + /* Emit a record for the in-use space */ + irec.rc_startblock = next_rtbno; + irec.rc_blockcount = rt_startblock - next_rtbno; + irec.rc_refcount = 1; + irec.rc_domain = XFS_REFC_DOMAIN_SHARED; + rec_daddr = XFS_FSB_TO_BB(mp, next_rtbno); + + return xfs_fsrefs_helper(tp, info, &irec, rec_daddr, + XFS_FSB_TO_BB(mp, rt_startblock - next_rtbno)); +} + +/* Emit records to fill a gap in the refcount btree with singly-owned blocks. */ +STATIC int +xfs_fsrefs_rtdev_fill_refcount_gap( + struct xfs_trans *tp, + struct xfs_fsrefs_info *info, + xfs_rtblock_t next_rtbno) +{ + struct xfs_rtalloc_rec low = { 0 }; + struct xfs_rtalloc_rec high = { 0 }; + struct xfs_mount *mp = tp->t_mountp; + xfs_rtblock_t rtbno; + xfs_daddr_t rec_daddr; + xfs_extlen_t mod; + int error; + + /* + * Set up query parameters to return free extents covering the range we + * want. + */ + rtbno = XFS_BB_TO_FSBT(mp, info->next_daddr); + low.ar_startext = xfs_rtb_to_rtxt(mp, rtbno); + + high.ar_startext = xfs_rtb_to_rtx(mp, next_rtbno, &mod); + if (mod) + high.ar_startext++; + + error = xfs_rtalloc_query_range(mp, tp, &low, &high, + xfs_fsrefs_rtdev_bitmap_helper, info); + if (error) + return error; + + /* + * Synthesize records for single-owner extents between the last + * fsrefcount record emitted and the end of the query range. + */ + high.ar_startext = min(mp->m_sb.sb_rextents, high.ar_startext); + rec_daddr = XFS_FSB_TO_BB(mp, xfs_rtx_to_rtb(mp, high.ar_startext)); + if (info->next_daddr > rec_daddr) + return 0; + + info->last = true; + return xfs_fsrefs_rtdev_bitmap_helper(mp, tp, &high, info); +} + +/* Transform a absolute-startblock refcount (rtdev, logdev) into a fsrefs */ +STATIC int +xfs_fsrefs_rtdev_helper( + struct xfs_btree_cur *cur, + const struct xfs_refcount_irec *rec, + void *priv) +{ + struct xfs_mount *mp = cur->bc_mp; + struct xfs_fsrefs_info *info = priv; + xfs_rtblock_t rtbno; + int error; + + /* + * Stop once we get to the CoW staging extents; they're all shoved to + * the right side of the btree and were already covered by the rtbitmap + * scan. + */ + if (rec->rc_domain != XFS_REFC_DOMAIN_SHARED) + return -ECANCELED; + + /* Report on any gaps first */ + rtbno = xfs_rgbno_to_rtb(mp, cur->bc_ino.rtg->rtg_rgno, + rec->rc_startblock); + error = xfs_fsrefs_rtdev_fill_refcount_gap(cur->bc_tp, info, rtbno); + if (error) + return error; + + /* Report this refcount extent. */ + info->next_daddr = XFS_FSB_TO_BB(mp, rtbno + rec->rc_blockcount); + return xfs_fsrefs_helper(cur->bc_tp, info, rec, + XFS_FSB_TO_BB(mp, rtbno), 0); +} + +/* Execute a getfsrefs query against the realtime bitmap. */ +STATIC int +xfs_fsrefs_rtdev_rtbitmap( + struct xfs_trans *tp, + const struct xfs_fsrefs *keys, + struct xfs_fsrefs_info *info) +{ + struct xfs_mount *mp = tp->t_mountp; + xfs_rtblock_t start_fsb; + xfs_rtblock_t end_fsb; + uint64_t eofs; + int error; + + eofs = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rblocks); + if (keys[0].fcr_physical >= eofs) + return 0; + start_fsb = XFS_BB_TO_FSBT(mp, keys[0].fcr_physical); + end_fsb = XFS_BB_TO_FSB(mp, min(eofs - 1, keys[1].fcr_physical)); + + info->refc_cur = NULL; + + /* Set up search keys */ + info->low.rc_startblock = start_fsb; + info->low_daddr = XFS_FSB_TO_BB(mp, start_fsb); + info->low.rc_blockcount = 0; + info->low.rc_refcount = 0; + + info->high.rc_startblock = end_fsb; + info->high.rc_blockcount = 0; + info->high.rc_refcount = 0; + + trace_xfs_fsrefs_low_key(mp, info->dev, NULLAGNUMBER, &info->low); + trace_xfs_fsrefs_high_key(mp, info->dev, NULLAGNUMBER, &info->high); + + /* Synthesize refcount==1 records from the free space data. */ + xfs_rtbitmap_lock_shared(mp, XFS_RBMLOCK_BITMAP); + error = xfs_fsrefs_rtdev_fill_refcount_gap(tp, info, end_fsb); + xfs_rtbitmap_unlock_shared(mp, XFS_RBMLOCK_BITMAP); + return error; +} + +#define XFS_RTGLOCK_FSREFS (XFS_RTGLOCK_BITMAP_SHARED | \ + XFS_RTGLOCK_REFCOUNT) + +/* Execute a getfsrefs query against the realtime device. */ +STATIC int +xfs_fsrefs_rtdev( + struct xfs_trans *tp, + const struct xfs_fsrefs *keys, + struct xfs_fsrefs_info *info) +{ + struct xfs_mount *mp = tp->t_mountp; + struct xfs_rtgroup *rtg; + xfs_rtblock_t start_fsb; + xfs_rtblock_t end_fsb; + uint64_t eofs; + xfs_rgnumber_t start_rg, end_rg; + int error = 0; + + eofs = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rblocks); + if (keys[0].fcr_physical >= eofs) + return 0; + start_fsb = XFS_BB_TO_FSBT(mp, keys[0].fcr_physical); + end_fsb = XFS_BB_TO_FSB(mp, min(eofs - 1, keys[1].fcr_physical)); + + info->refc_cur = NULL; + + /* + * Convert the fsrefs low/high keys to rtgroup based keys. Initialize + * low to the fsrefs low key and max out the high key to the end of the + * rtgroup. + */ + info->low.rc_startblock = xfs_rtb_to_rgbno(mp, start_fsb, &start_rg); + info->low.rc_blockcount = 0; + info->low.rc_refcount = 0; + info->low.rc_domain = XFS_REFC_DOMAIN_SHARED; + + info->high.rc_startblock = -1U; + info->high.rc_blockcount = 0; + info->high.rc_refcount = 0; + info->high.rc_domain = XFS_REFC_DOMAIN_SHARED; + + end_rg = xfs_rtb_to_rgno(mp, end_fsb); + + for_each_rtgroup_range(mp, start_rg, end_rg, rtg) { + /* + * Set the rtgroup high key from the fsrefs high key if this + * is the last rtgroup that we're querying. + */ + info->rtg = rtg; + if (rtg->rtg_rgno == end_rg) { + xfs_rgnumber_t junk; + + info->high.rc_startblock = xfs_rtb_to_rgbno(mp, + end_fsb, &junk); + } + + if (info->refc_cur) { + xfs_rtgroup_unlock(info->refc_cur->bc_ino.rtg, + XFS_RTGLOCK_FSREFS); + xfs_btree_del_cursor(info->refc_cur, XFS_BTREE_NOERROR); + info->refc_cur = NULL; + } + + trace_xfs_fsrefs_low_key(mp, info->dev, rtg->rtg_rgno, + &info->low); + trace_xfs_fsrefs_high_key(mp, info->dev, rtg->rtg_rgno, + &info->high); + + xfs_rtgroup_lock(NULL, rtg, XFS_RTGLOCK_FSREFS); + info->refc_cur = xfs_rtrefcountbt_init_cursor(mp, tp, rtg, + rtg->rtg_refcountip); + + /* + * Fill the query with refcount records and synthesize + * singly-owned block records from free space data. + */ + error = xfs_refcount_query_range(info->refc_cur, + &info->low, &info->high, + xfs_fsrefs_rtdev_helper, info); + if (error && error != -ECANCELED) + break; + + /* + * Set the rtgroup low key to the start of the rtgroup prior to + * moving on to the next rtgroup. + */ + if (rtg->rtg_rgno == start_rg) + info->low.rc_startblock = 0; + + /* + * If this is the last rtgroup, report any gap at the end of it + * before we drop the reference to the perag when the loop + * terminates. + */ + if (rtg->rtg_rgno == end_rg) { + info->last = true; + error = xfs_fsrefs_rtdev_fill_refcount_gap(tp, info, + end_fsb); + if (error) + break; + } + info->rtg = NULL; + } + + if (info->refc_cur) { + xfs_rtgroup_unlock(info->refc_cur->bc_ino.rtg, + XFS_RTGLOCK_FSREFS); + xfs_btree_del_cursor(info->refc_cur, error); + info->refc_cur = NULL; + } + if (info->rtg) { + xfs_rtgroup_put(info->rtg); + info->rtg = NULL; + } else if (rtg) { + /* loop termination case */ + xfs_rtgroup_put(rtg); + } + + return error; +} +#endif + +/* Do we recognize the device? */ +STATIC bool +xfs_fsrefs_is_valid_device( + struct xfs_mount *mp, + struct xfs_fsrefs *fcr) +{ + if (fcr->fcr_device == 0 || fcr->fcr_device == UINT_MAX || + fcr->fcr_device == new_encode_dev(mp->m_ddev_targp->bt_dev)) + return true; + if (mp->m_logdev_targp && + fcr->fcr_device == new_encode_dev(mp->m_logdev_targp->bt_dev)) + return true; + if (mp->m_rtdev_targp && + fcr->fcr_device == new_encode_dev(mp->m_rtdev_targp->bt_dev)) + return true; + return false; +} + +/* Ensure that the low key is less than the high key. */ +STATIC bool +xfs_fsrefs_check_keys( + struct xfs_fsrefs *low_key, + struct xfs_fsrefs *high_key) +{ + if (low_key->fcr_device > high_key->fcr_device) + return false; + if (low_key->fcr_device < high_key->fcr_device) + return true; + + if (low_key->fcr_physical > high_key->fcr_physical) + return false; + if (low_key->fcr_physical < high_key->fcr_physical) + return true; + + return false; +} + +/* + * There are only two devices if we didn't configure RT devices at build time. + */ +#ifdef CONFIG_XFS_RT +#define XFS_GETFSREFS_DEVS 3 +#else +#define XFS_GETFSREFS_DEVS 2 +#endif /* CONFIG_XFS_RT */ + +/* + * Get filesystem's extent refcounts as described in head, and format for + * output. Fills in the supplied records array until there are no more reverse + * mappings to return or head.fch_entries == head.fch_count. In the second + * case, this function returns -ECANCELED to indicate that more records would + * have been returned. + * + * Key to Confusion + * ---------------- + * There are multiple levels of keys and counters at work here: + * xfs_fsrefs_head.fch_keys -- low and high fsrefs keys passed in; + * these reflect fs-wide sector addrs. + * dkeys -- fch_keys used to query each device; + * these are fch_keys but w/ the low key + * bumped up by fcr_length. + * xfs_fsrefs_info.next_daddr-- next disk addr we expect to see; this + * is how we detect gaps in the fsrefs + records and report them. + * xfs_fsrefs_info.low/high -- per-AG low/high keys computed from + * dkeys; used to query the metadata. + */ +int +xfs_getfsrefs( + struct xfs_mount *mp, + struct xfs_fsrefs_head *head, + struct fsrefs *fsrefs_recs) +{ + struct xfs_trans *tp = NULL; + struct xfs_fsrefs dkeys[2]; /* per-dev keys */ + struct xfs_fsrefs_dev handlers[XFS_GETFSREFS_DEVS]; + struct xfs_fsrefs_info info = { NULL }; + int i; + int error = 0; + + if (head->fch_iflags & ~FCH_IF_VALID) + return -EINVAL; + if (!xfs_fsrefs_is_valid_device(mp, &head->fch_keys[0]) || + !xfs_fsrefs_is_valid_device(mp, &head->fch_keys[1])) + return -EINVAL; + + head->fch_entries = 0; + + /* Set up our device handlers. */ + memset(handlers, 0, sizeof(handlers)); + handlers[0].dev = new_encode_dev(mp->m_ddev_targp->bt_dev); + handlers[0].fn = xfs_fsrefs_ddev; + if (mp->m_logdev_targp != mp->m_ddev_targp) { + handlers[1].dev = new_encode_dev(mp->m_logdev_targp->bt_dev); + handlers[1].fn = xfs_fsrefs_logdev; + } +#ifdef CONFIG_XFS_RT + if (mp->m_rtdev_targp) { + handlers[2].dev = new_encode_dev(mp->m_rtdev_targp->bt_dev); + if (xfs_has_rtreflink(mp)) + handlers[2].fn = xfs_fsrefs_rtdev; + else + handlers[2].fn = xfs_fsrefs_rtdev_rtbitmap; + } +#endif /* CONFIG_XFS_RT */ + + xfs_sort(handlers, XFS_GETFSREFS_DEVS, sizeof(struct xfs_fsrefs_dev), + xfs_fsrefs_dev_compare); + + /* + * To continue where we left off, we allow userspace to use the last + * mapping from a previous call as the low key of the next. This is + * identified by a non-zero length in the low key. We have to increment + * the low key in this scenario to ensure we don't return the same + * mapping again, and instead return the very next mapping. Bump the + * physical offset as there can be no other mapping for the same + * physical block range. + */ + dkeys[0] = head->fch_keys[0]; + dkeys[0].fcr_physical += dkeys[0].fcr_length; + dkeys[0].fcr_length = 0; + memset(&dkeys[1], 0xFF, sizeof(struct xfs_fsrefs)); + + if (!xfs_fsrefs_check_keys(dkeys, &head->fch_keys[1])) + return -EINVAL; + + info.next_daddr = head->fch_keys[0].fcr_physical + + head->fch_keys[0].fcr_length; + info.fsrefs_recs = fsrefs_recs; + info.head = head; + + /* For each device we support... */ + for (i = 0; i < XFS_GETFSREFS_DEVS; i++) { + /* Is this device within the range the user asked for? */ + if (!handlers[i].fn) + continue; + if (head->fch_keys[0].fcr_device > handlers[i].dev) + continue; + if (head->fch_keys[1].fcr_device < handlers[i].dev) + break; + + /* + * If this device number matches the high key, we have to pass + * the high key to the handler to limit the query results. If + * the device number exceeds the low key, zero out the low key + * so that we get everything from the beginning. + */ + if (handlers[i].dev == head->fch_keys[1].fcr_device) + dkeys[1] = head->fch_keys[1]; + if (handlers[i].dev > head->fch_keys[0].fcr_device) + memset(&dkeys[0], 0, sizeof(struct xfs_fsrefs)); + + /* + * Grab an empty transaction so that we can use its recursive + * buffer locking abilities to detect cycles in the refcountbt + * without deadlocking. + */ + error = xfs_trans_alloc_empty(mp, &tp); + if (error) + break; + + info.dev = handlers[i].dev; + info.last = false; + info.pag = NULL; + info.rtg = NULL; + info.low_daddr = -1ULL; + error = handlers[i].fn(tp, dkeys, &info); + if (error) + break; + xfs_trans_cancel(tp); + tp = NULL; + info.next_daddr = 0; + } + + if (tp) + xfs_trans_cancel(tp); + head->fch_oflags = FCH_OF_DEV_T; + return error; +} diff --git a/fs/xfs/xfs_fsrefs.h b/fs/xfs/xfs_fsrefs.h new file mode 100644 index 000000000000..fa74e1a84d92 --- /dev/null +++ b/fs/xfs/xfs_fsrefs.h @@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#ifndef __XFS_FSREFS_H__ +#define __XFS_FSREFS_H__ + +struct fsrefs; + +/* internal fsrefs representation */ +struct xfs_fsrefs { + dev_t fcr_device; /* device id */ + uint32_t fcr_flags; /* mapping flags */ + uint64_t fcr_physical; /* device offset of segment */ + uint64_t fcr_owners; /* number of owners */ + xfs_filblks_t fcr_length; /* length of segment, blocks */ +}; + +struct xfs_fsrefs_head { + uint32_t fch_iflags; /* control flags */ + uint32_t fch_oflags; /* output flags */ + unsigned int fch_count; /* # of entries in array incl. input */ + unsigned int fch_entries; /* # of entries filled in (output). */ + + struct xfs_fsrefs fch_keys[2]; /* low and high keys */ +}; + +void xfs_fsrefs_to_internal(struct xfs_fsrefs *dest, struct fsrefs *src); + +int xfs_getfsrefs(struct xfs_mount *mp, struct xfs_fsrefs_head *head, + struct fsrefs *out_recs); + +#endif /* __XFS_FSREFS_H__ */ diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c index fdd8fd35fc48..3eda06f93f5b 100644 --- a/fs/xfs/xfs_ioctl.c +++ b/fs/xfs/xfs_ioctl.c @@ -31,6 +31,8 @@ #include "xfs_btree.h" #include <linux/fsmap.h> #include "xfs_fsmap.h" +#include <linux/fsrefcounts.h> +#include "xfs_fsrefs.h" #include "scrub/xfs_scrub.h" #include "xfs_sb.h" #include "xfs_ag.h" @@ -1621,6 +1623,136 @@ xfs_ioc_getfsmap( return error; } +STATIC int +xfs_ioc_getfsrefcounts( + struct xfs_inode *ip, + struct fsrefs_head __user *arg) +{ + struct xfs_fsrefs_head xhead = {0}; + struct fsrefs_head head; + struct fsrefs *recs; + unsigned int count; + __u32 last_flags = 0; + bool done = false; + int error; + + if (copy_from_user(&head, arg, sizeof(struct fsrefs_head))) + return -EFAULT; + if (memchr_inv(head.fch_reserved, 0, sizeof(head.fch_reserved)) || + memchr_inv(head.fch_keys[0].fcr_reserved, 0, + sizeof(head.fch_keys[0].fcr_reserved)) || + memchr_inv(head.fch_keys[1].fcr_reserved, 0, + sizeof(head.fch_keys[1].fcr_reserved))) + return -EINVAL; + + /* + * Use an internal memory buffer so that we don't have to copy fsrefs + * data to userspace while holding locks. Start by trying to allocate + * up to 128k for the buffer, but fall back to a single page if needed. + */ + count = min_t(unsigned int, head.fch_count, + 131072 / sizeof(struct fsrefs)); + recs = kvzalloc(count * sizeof(struct fsrefs), GFP_KERNEL); + if (!recs) { + count = min_t(unsigned int, head.fch_count, + PAGE_SIZE / sizeof(struct fsrefs)); + recs = kvzalloc(count * sizeof(struct fsrefs), GFP_KERNEL); + if (!recs) + return -ENOMEM; + } + + xhead.fch_iflags = head.fch_iflags; + xfs_fsrefs_to_internal(&xhead.fch_keys[0], &head.fch_keys[0]); + xfs_fsrefs_to_internal(&xhead.fch_keys[1], &head.fch_keys[1]); + + trace_xfs_getfsrefs_low_key(ip->i_mount, &xhead.fch_keys[0]); + trace_xfs_getfsrefs_high_key(ip->i_mount, &xhead.fch_keys[1]); + + head.fch_entries = 0; + do { + struct fsrefs __user *user_recs; + struct fsrefs *last_rec; + + user_recs = &arg->fch_recs[head.fch_entries]; + xhead.fch_entries = 0; + xhead.fch_count = min_t(unsigned int, count, + head.fch_count - head.fch_entries); + + /* Run query, record how many entries we got. */ + error = xfs_getfsrefs(ip->i_mount, &xhead, recs); + switch (error) { + case 0: + /* + * There are no more records in the result set. Copy + * whatever we got to userspace and break out. + */ + done = true; + break; + case -ECANCELED: + /* + * The internal memory buffer is full. Copy whatever + * records we got to userspace and go again if we have + * not yet filled the userspace buffer. + */ + error = 0; + break; + default: + goto out_free; + } + head.fch_entries += xhead.fch_entries; + head.fch_oflags = xhead.fch_oflags; + + /* + * If the caller wanted a record count or there aren't any + * new records to return, we're done. + */ + if (head.fch_count == 0 || xhead.fch_entries == 0) + break; + + /* Copy all the records we got out to userspace. */ + if (copy_to_user(user_recs, recs, + xhead.fch_entries * sizeof(struct fsrefs))) { + error = -EFAULT; + goto out_free; + } + + /* Remember the last record flags we copied to userspace. */ + last_rec = &recs[xhead.fch_entries - 1]; + last_flags = last_rec->fcr_flags; + + /* Set up the low key for the next iteration. */ + xfs_fsrefs_to_internal(&xhead.fch_keys[0], last_rec); + trace_xfs_getfsrefs_low_key(ip->i_mount, &xhead.fch_keys[0]); + } while (!done && head.fch_entries < head.fch_count); + + /* + * If there are no more records in the query result set and we're not + * in counting mode, mark the last record returned with the LAST flag. + */ + if (done && head.fch_count > 0 && head.fch_entries > 0) { + struct fsrefs __user *user_rec; + + last_flags |= FCR_OF_LAST; + user_rec = &arg->fch_recs[head.fch_entries - 1]; + + if (copy_to_user(&user_rec->fcr_flags, &last_flags, + sizeof(last_flags))) { + error = -EFAULT; + goto out_free; + } + } + + /* copy back header */ + if (copy_to_user(arg, &head, sizeof(struct fsrefs_head))) { + error = -EFAULT; + goto out_free; + } + +out_free: + kmem_free(recs); + return error; +} + STATIC int xfs_ioc_scrub_metadata( struct file *file, @@ -1955,6 +2087,9 @@ xfs_file_ioctl( case FS_IOC_GETFSMAP: return xfs_ioc_getfsmap(ip, arg); + case FS_IOC_GETFSREFCOUNTS: + return xfs_ioc_getfsrefcounts(ip, arg); + case XFS_IOC_SCRUBV_METADATA: return xfs_ioc_scrubv_metadata(filp, arg); case XFS_IOC_SCRUB_METADATA: diff --git a/fs/xfs/xfs_trace.c b/fs/xfs/xfs_trace.c index 64f11a535763..948532a8274c 100644 --- a/fs/xfs/xfs_trace.c +++ b/fs/xfs/xfs_trace.c @@ -45,6 +45,7 @@ #include "xfs_rtgroup.h" #include "xfs_rmap.h" #include "xfs_refcount.h" +#include "xfs_fsrefs.h" static inline void xfs_rmapbt_crack_agno_opdev( diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index ba892e6a8233..b422a206edb5 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -92,6 +92,7 @@ struct xfs_rtgroup; struct xfs_extent_free_item; struct xfs_rmap_intent; struct xfs_refcount_intent; +struct xfs_fsrefs; #define XFS_ATTR_FILTER_FLAGS \ { XFS_ATTR_ROOT, "ROOT" }, \ @@ -4140,6 +4141,79 @@ DEFINE_GETFSMAP_EVENT(xfs_getfsmap_low_key); DEFINE_GETFSMAP_EVENT(xfs_getfsmap_high_key); DEFINE_GETFSMAP_EVENT(xfs_getfsmap_mapping); +/* fsrefs traces */ +DECLARE_EVENT_CLASS(xfs_fsrefs_class, + TP_PROTO(struct xfs_mount *mp, u32 keydev, xfs_agnumber_t agno, + const struct xfs_refcount_irec *refc), + TP_ARGS(mp, keydev, agno, refc), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(dev_t, keydev) + __field(xfs_agnumber_t, agno) + __field(xfs_agblock_t, bno) + __field(xfs_extlen_t, len) + __field(uint64_t, owners) + ), + TP_fast_assign( + __entry->dev = mp->m_super->s_dev; + __entry->keydev = new_decode_dev(keydev); + __entry->agno = agno; + __entry->bno = refc->rc_startblock; + __entry->len = refc->rc_blockcount; + __entry->owners = refc->rc_refcount; + ), + TP_printk("dev %d:%d keydev %d:%d agno 0x%x refcbno 0x%x fsbcount 0x%x owners %llu", + MAJOR(__entry->dev), MINOR(__entry->dev), + MAJOR(__entry->keydev), MINOR(__entry->keydev), + __entry->agno, + __entry->bno, + __entry->len, + __entry->owners) +) +#define DEFINE_FSREFS_EVENT(name) \ +DEFINE_EVENT(xfs_fsrefs_class, name, \ + TP_PROTO(struct xfs_mount *mp, u32 keydev, xfs_agnumber_t agno, \ + const struct xfs_refcount_irec *refc), \ + TP_ARGS(mp, keydev, agno, refc)) +DEFINE_FSREFS_EVENT(xfs_fsrefs_low_key); +DEFINE_FSREFS_EVENT(xfs_fsrefs_high_key); +DEFINE_FSREFS_EVENT(xfs_fsrefs_mapping); + +DECLARE_EVENT_CLASS(xfs_getfsrefs_class, + TP_PROTO(struct xfs_mount *mp, struct xfs_fsrefs *fsrefs), + TP_ARGS(mp, fsrefs), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(dev_t, keydev) + __field(xfs_daddr_t, block) + __field(xfs_daddr_t, len) + __field(uint64_t, owners) + __field(uint32_t, flags) + ), + TP_fast_assign( + __entry->dev = mp->m_super->s_dev; + __entry->keydev = new_decode_dev(fsrefs->fcr_device); + __entry->block = fsrefs->fcr_physical; + __entry->len = fsrefs->fcr_length; + __entry->owners = fsrefs->fcr_owners; + __entry->flags = fsrefs->fcr_flags; + ), + TP_printk("dev %d:%d keydev %d:%d daddr 0x%llx bbcount 0x%llx owners %llu flags 0x%x", + MAJOR(__entry->dev), MINOR(__entry->dev), + MAJOR(__entry->keydev), MINOR(__entry->keydev), + __entry->block, + __entry->len, + __entry->owners, + __entry->flags) +) +#define DEFINE_GETFSREFS_EVENT(name) \ +DEFINE_EVENT(xfs_getfsrefs_class, name, \ + TP_PROTO(struct xfs_mount *mp, struct xfs_fsrefs *fsrefs), \ + TP_ARGS(mp, fsrefs)) +DEFINE_GETFSREFS_EVENT(xfs_getfsrefs_low_key); +DEFINE_GETFSREFS_EVENT(xfs_getfsrefs_high_key); +DEFINE_GETFSREFS_EVENT(xfs_getfsrefs_mapping); + DECLARE_EVENT_CLASS(xfs_trans_resv_class, TP_PROTO(struct xfs_mount *mp, unsigned int type, struct xfs_trans_res *res), diff --git a/include/uapi/linux/fsrefcounts.h b/include/uapi/linux/fsrefcounts.h new file mode 100644 index 000000000000..edb02bf99599 --- /dev/null +++ b/include/uapi/linux/fsrefcounts.h @@ -0,0 +1,96 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * FS_IOC_GETFSREFCOUNTS ioctl infrastructure. + * + * Copyright (C) 2022 Oracle. All Rights Reserved. + * + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#ifndef _LINUX_FSREFCOUNTS_H +#define _LINUX_FSREFCOUNTS_H + +#include <linux/types.h> + +/* + * Structure for FS_IOC_GETFSREFCOUNTS. + * + * The memory layout for this call are the scalar values defined in + * struct fsrefs_head, followed by two struct fsrefs that describe the + * lower and upper bound of mappings to return, followed by an array of + * struct fsrefs mappings. + * + * fch_iflags control the output of the call, whereas fch_oflags report + * on the overall record output. fch_count should be set to the length + * of the fch_recs array, and fch_entries will be set to the number of + * entries filled out during each call. If fch_count is zero, the number + * of refcount mappings will be returned in fch_entries, though no + * mappings will be returned. fch_reserved must be set to zero. + * + * The two elements in the fch_keys array are used to constrain the + * output. The first element in the array should represent the lowest + * disk mapping ("low key") that the user wants to learn about. If this + * value is all zeroes, the filesystem will return the first entry it + * knows about. For a subsequent call, the contents of + * fsrefs_head.fch_recs[fsrefs_head.fch_count - 1] should be copied into + * fch_keys[0] to have the kernel start where it left off. + * + * The second element in the fch_keys array should represent the highest + * disk mapping ("high key") that the user wants to learn about. If this + * value is all ones, the filesystem will not stop until it runs out of + * mapping to return or runs out of space in fch_recs. + * + * fcr_device can be either a 32-bit cookie representing a device, or a + * 32-bit dev_t if the FCH_OF_DEV_T flag is set. fcr_physical and + * fcr_length are expressed in units of bytes. fcr_owners is the number + * of owners. + */ +struct fsrefs { + __u32 fcr_device; /* device id */ + __u32 fcr_flags; /* mapping flags */ + __u64 fcr_physical; /* device offset of segment */ + __u64 fcr_owners; /* number of owners */ + __u64 fcr_length; /* length of segment */ + __u64 fcr_reserved[4]; /* must be zero */ +}; + +struct fsrefs_head { + __u32 fch_iflags; /* control flags */ + __u32 fch_oflags; /* output flags */ + __u32 fch_count; /* # of entries in array incl. input */ + __u32 fch_entries; /* # of entries filled in (output). */ + __u64 fch_reserved[6]; /* must be zero */ + + struct fsrefs fch_keys[2]; /* low and high keys for the mapping search */ + struct fsrefs fch_recs[]; /* returned records */ +}; + +/* Size of an fsrefs_head with room for nr records. */ +static inline unsigned long long +fsrefs_sizeof( + unsigned int nr) +{ + return sizeof(struct fsrefs_head) + nr * sizeof(struct fsrefs); +} + +/* Start the next fsrefs query at the end of the current query results. */ +static inline void +fsrefs_advance( + struct fsrefs_head *head) +{ + head->fch_keys[0] = head->fch_recs[head->fch_entries - 1]; +} + +/* fch_iflags values - set by FS_IOC_GETFSREFCOUNTS caller in the header. */ +/* no flags defined yet */ +#define FCH_IF_VALID 0 + +/* fch_oflags values - returned in the header segment only. */ +#define FCH_OF_DEV_T 0x1 /* fcr_device values will be dev_t */ + +/* fcr_flags values - returned for each non-header segment */ +#define FCR_OF_LAST (1U << 0) /* segment is the last in the dataset */ + +/* XXX stealing XFS_IOC_GETBIOSIZE */ +#define FS_IOC_GETFSREFCOUNTS _IOWR('X', 47, struct fsrefs_head) + +#endif /* _LINUX_FSREFCOUNTS_H */ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/2] xfs: defragment free space 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (2 preceding siblings ...) 2022-12-30 22:19 ` [PATCHSET 0/1] xfs: report refcount information to userspace Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/2] xfs: fallocate free space into a file Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/2] xfs: capture the offset and length in fallocate tracepoints Darrick J. Wong 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (5 subsequent siblings) 9 siblings, 2 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs Hi all, These patches contain experimental code to enable userspace to defragment the free space in a filesystem. Two purposes are imagined for this functionality: clearing space at the end of a filesystem before shrinking it, and clearing free space in anticipation of making a large allocation. The first patch adds a new fallocate mode that allows userspace to allocate free space from the filesystem into a file. The goal here is to allow the filesystem shrink process to prevent allocation from a certain part of the filesystem while a free space defragmenter evacuates all the files from the doomed part of the filesystem. The second patch amends the online repair system to allow the sysadmin to forcibly rebuild metadata structures, even if they're not corrupt. Without adding an ioctl to move metadata btree blocks, this is the only way to dislodge metadata. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=defrag-freespace --- fs/open.c | 5 + fs/xfs/libxfs/xfs_alloc.c | 88 ++++++++++++ fs/xfs/libxfs/xfs_alloc.h | 4 + fs/xfs/libxfs/xfs_bmap.c | 150 ++++++++++++++++++++ fs/xfs/libxfs/xfs_bmap.h | 3 fs/xfs/xfs_bmap_util.c | 315 ++++++++++++++++++++++++++++++++++++++++++- fs/xfs/xfs_bmap_util.h | 7 + fs/xfs/xfs_file.c | 39 +++++ fs/xfs/xfs_rtalloc.c | 49 +++++++ fs/xfs/xfs_rtalloc.h | 12 ++ fs/xfs/xfs_trace.h | 72 +++++++++- include/linux/falloc.h | 3 include/uapi/linux/falloc.h | 8 + 13 files changed, 742 insertions(+), 13 deletions(-) ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 2/2] xfs: fallocate free space into a file 2022-12-30 22:19 ` [PATCHSET 0/2] xfs: defragment free space Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/2] xfs: capture the offset and length in fallocate tracepoints Darrick J. Wong 1 sibling, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Add a new fallocate mode to map free physical space into a file, at the same file offset as if the file were a sparse image of the physical device backing the filesystem. The intent here is to use this to prototype a free space defragmentation tool. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/open.c | 5 + fs/xfs/libxfs/xfs_alloc.c | 88 ++++++++++++ fs/xfs/libxfs/xfs_alloc.h | 4 + fs/xfs/libxfs/xfs_bmap.c | 150 +++++++++++++++++++++ fs/xfs/libxfs/xfs_bmap.h | 3 fs/xfs/xfs_bmap_util.c | 307 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_bmap_util.h | 7 + fs/xfs/xfs_file.c | 37 +++++ fs/xfs/xfs_rtalloc.c | 49 +++++++ fs/xfs/xfs_rtalloc.h | 12 ++ fs/xfs/xfs_trace.h | 62 +++++++++ include/linux/falloc.h | 3 include/uapi/linux/falloc.h | 8 + 13 files changed, 732 insertions(+), 3 deletions(-) diff --git a/fs/open.c b/fs/open.c index 82c1a28b3308..d9ae216c7f75 100644 --- a/fs/open.c +++ b/fs/open.c @@ -277,6 +277,11 @@ int vfs_fallocate(struct file *file, int mode, loff_t offset, loff_t len) (mode & ~(FALLOC_FL_UNSHARE_RANGE | FALLOC_FL_KEEP_SIZE))) return -EINVAL; + /* Mapping free space should only be used by itself. */ + if ((mode & FALLOC_FL_MAP_FREE_SPACE) && + (mode & ~FALLOC_FL_MAP_FREE_SPACE)) + return -EINVAL; + if (!(file->f_mode & FMODE_WRITE)) return -EBADF; diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c index 5d091789ff74..2e37212ad163 100644 --- a/fs/xfs/libxfs/xfs_alloc.c +++ b/fs/xfs/libxfs/xfs_alloc.c @@ -3705,3 +3705,91 @@ xfs_extfree_intent_destroy_cache(void) kmem_cache_destroy(xfs_extfree_item_cache); xfs_extfree_item_cache = NULL; } + +/* + * Find the next chunk of free space in @pag starting at @agbno and going no + * higher than @end_agbno. Set @agbno and @len to whatever free space we find, + * or to @end_agbno if we find no space. + */ +int +xfs_alloc_find_freesp( + struct xfs_trans *tp, + struct xfs_perag *pag, + xfs_agblock_t *agbno, + xfs_agblock_t end_agbno, + xfs_extlen_t *len) +{ + struct xfs_mount *mp = pag->pag_mount; + struct xfs_btree_cur *cur; + struct xfs_buf *agf_bp = NULL; + xfs_agblock_t found_agbno; + xfs_extlen_t found_len; + int found; + int error; + + trace_xfs_alloc_find_freesp(mp, pag->pag_agno, *agbno, + end_agbno - *agbno); + + error = xfs_alloc_read_agf(pag, tp, 0, &agf_bp); + if (error) + return error; + + cur = xfs_allocbt_init_cursor(mp, tp, agf_bp, pag, XFS_BTNUM_BNO); + + /* Try to find a free extent that starts before here. */ + error = xfs_alloc_lookup_le(cur, *agbno, 0, &found); + if (error) + goto out_cur; + if (found) { + error = xfs_alloc_get_rec(cur, &found_agbno, &found_len, + &found); + if (error) + goto out_cur; + if (XFS_IS_CORRUPT(mp, !found)) { + xfs_btree_mark_sick(cur); + error = -EFSCORRUPTED; + goto out_cur; + } + + if (found_agbno + found_len > *agbno) + goto found; + } + + /* Examine the next record if free extent not in range. */ + error = xfs_btree_increment(cur, 0, &found); + if (error) + goto out_cur; + if (!found) + goto next_ag; + + error = xfs_alloc_get_rec(cur, &found_agbno, &found_len, &found); + if (error) + goto out_cur; + if (XFS_IS_CORRUPT(mp, !found)) { + xfs_btree_mark_sick(cur); + error = -EFSCORRUPTED; + goto out_cur; + } + + if (found_agbno >= end_agbno) + goto next_ag; + +found: + /* Found something, so update the mapping. */ + trace_xfs_alloc_find_freesp_done(mp, pag->pag_agno, found_agbno, + found_len); + if (found_agbno < *agbno) { + found_len -= *agbno - found_agbno; + found_agbno = *agbno; + } + *len = found_len; + *agbno = found_agbno; + goto out_cur; +next_ag: + /* Found nothing, so advance the cursor beyond the end of the range. */ + *agbno = end_agbno; + *len = 0; +out_cur: + xfs_btree_del_cursor(cur, error); + return error; +} diff --git a/fs/xfs/libxfs/xfs_alloc.h b/fs/xfs/libxfs/xfs_alloc.h index cd7b26568a33..327c66f55780 100644 --- a/fs/xfs/libxfs/xfs_alloc.h +++ b/fs/xfs/libxfs/xfs_alloc.h @@ -268,4 +268,8 @@ extern struct kmem_cache *xfs_extfree_item_cache; int __init xfs_extfree_intent_init_cache(void); void xfs_extfree_intent_destroy_cache(void); +int xfs_alloc_find_freesp(struct xfs_trans *tp, struct xfs_perag *pag, + xfs_agblock_t *agbno, xfs_agblock_t end_agbno, + xfs_extlen_t *len); + #endif /* __XFS_ALLOC_H__ */ diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index 053d72063999..73a9a586b05d 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -39,6 +39,7 @@ #include "xfs_health.h" #include "xfs_symlink_remote.h" #include "xfs_inode_util.h" +#include "xfs_rtalloc.h" struct kmem_cache *xfs_bmap_intent_cache; @@ -6485,3 +6486,152 @@ xfs_get_cowextsz_hint( return XFS_DEFAULT_COWEXTSZ_HINT; return a; } + +static inline xfs_fileoff_t +xfs_fsblock_to_fileoff( + struct xfs_mount *mp, + xfs_fsblock_t fsbno) +{ + xfs_daddr_t daddr = XFS_FSB_TO_DADDR(mp, fsbno); + + return XFS_B_TO_FSB(mp, BBTOB(daddr)); +} + +/* + * Given a file and a free physical extent, map it into the file at the same + * offset if the file were a sparse image of the physical device. Set @mval to + * whatever mapping we added to the file. + */ +int +xfs_bmapi_freesp( + struct xfs_trans *tp, + struct xfs_inode *ip, + xfs_fsblock_t fsbno, + xfs_extlen_t len, + struct xfs_bmbt_irec *mval) +{ + struct xfs_bmbt_irec irec; + struct xfs_mount *mp = ip->i_mount; + xfs_fileoff_t startoff; + bool isrt = XFS_IS_REALTIME_INODE(ip); + int nimaps; + int error; + + trace_xfs_bmapi_freesp(ip, fsbno, len); + + error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK, + XFS_IEXT_ADD_NOSPLIT_CNT); + if (error) + return error; + + if (isrt) + startoff = fsbno; + else + startoff = xfs_fsblock_to_fileoff(mp, fsbno); + + /* Make sure the entire range is a hole. */ + nimaps = 1; + error = xfs_bmapi_read(ip, startoff, len, &irec, &nimaps, 0); + if (error) + return error; + + if (irec.br_startoff != startoff || + irec.br_startblock != HOLESTARTBLOCK || + irec.br_blockcount < len) + return -EINVAL; + + /* + * Allocate the physical extent. We should not have dropped the lock + * since the scan of the free space metadata, so this should work, + * though the length may be adjusted to play nicely with metadata space + * reservations. + */ + if (isrt) { + xfs_rtxnum_t rtx_in, rtx_out; + xfs_extlen_t rtxlen_in, rtxlen_out; + uint32_t mod; + + rtx_in = xfs_rtb_to_rtx(mp, fsbno, &mod); + if (mod) { + ASSERT(mod == 0); + return -EFSCORRUPTED; + } + + rtxlen_in = xfs_rtb_to_rtx(mp, len, &mod); + if (mod) { + ASSERT(mod == 0); + return -EFSCORRUPTED; + } + + error = xfs_rtallocate_extent(tp, rtx_in, 1, rtxlen_in, + &rtxlen_out, 0, 1, &rtx_out); + if (error) + return error; + if (rtx_out == NULLRTEXTNO) { + /* + * We were promised the space! In theory the aren't + * any reserve lists that would prevent us from getting + * the space. + */ + return -ENOSPC; + } + if (rtx_out != rtx_in) { + ASSERT(0); + xfs_bmap_mark_sick(ip, XFS_DATA_FORK); + return -EFSCORRUPTED; + } + mval->br_blockcount = rtxlen_out * mp->m_sb.sb_rextsize; + } else { + struct xfs_alloc_arg args = { + .mp = ip->i_mount, + .type = XFS_ALLOCTYPE_THIS_BNO, + .oinfo = XFS_RMAP_OINFO_SKIP_UPDATE, + .resv = XFS_AG_RESV_NONE, + .prod = 1, + .datatype = XFS_ALLOC_USERDATA, + .tp = tp, + .maxlen = len, + .minlen = 1, + .fsbno = fsbno, + }; + error = xfs_alloc_vextent(&args); + if (error) + return error; + if (args.fsbno == NULLFSBLOCK) { + /* + * We were promised the space, but failed to get it. + * This could be because the space is reserved for + * metadata expansion, or it could be because the AGFL + * fixup grabbed the first block we wanted. Either + * way, if the transaction is dirty we must commit it + * and tell the caller to try again. + */ + if (tp->t_flags & XFS_TRANS_DIRTY) + return -EAGAIN; + return -ENOSPC; + } + if (args.fsbno != fsbno) { + ASSERT(0); + xfs_bmap_mark_sick(ip, XFS_DATA_FORK); + return -EFSCORRUPTED; + } + mval->br_blockcount = args.len; + } + + /* Map extent into file, update quota. */ + mval->br_startblock = fsbno; + mval->br_startoff = startoff; + mval->br_state = XFS_EXT_UNWRITTEN; + + trace_xfs_bmapi_freesp_done(ip, mval); + + xfs_bmap_map_extent(tp, ip, XFS_DATA_FORK, mval); + if (isrt) + xfs_trans_mod_dquot_byino(tp, ip, XFS_TRANS_DQ_RTBCOUNT, + mval->br_blockcount); + else + xfs_trans_mod_dquot_byino(tp, ip, XFS_TRANS_DQ_BCOUNT, + mval->br_blockcount); + + return 0; +} diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h index 05097b1d5c7d..ef20a625762d 100644 --- a/fs/xfs/libxfs/xfs_bmap.h +++ b/fs/xfs/libxfs/xfs_bmap.h @@ -193,6 +193,9 @@ int xfs_bmapi_read(struct xfs_inode *ip, xfs_fileoff_t bno, int xfs_bmapi_write(struct xfs_trans *tp, struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t len, uint32_t flags, xfs_extlen_t total, struct xfs_bmbt_irec *mval, int *nmap); +int xfs_bmapi_freesp(struct xfs_trans *tp, struct xfs_inode *ip, + xfs_fsblock_t fsbno, xfs_extlen_t len, + struct xfs_bmbt_irec *mval); int __xfs_bunmapi(struct xfs_trans *tp, struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t *rlen, uint32_t flags, xfs_extnum_t nexts); diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index 8515190d2bc6..6b2ad693ecc6 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -30,6 +30,13 @@ #include "xfs_reflink.h" #include "xfs_swapext.h" #include "xfs_rtbitmap.h" +#include "xfs_rtrmap_btree.h" +#include "xfs_rtrefcount_btree.h" +#include "xfs_health.h" +#include "xfs_alloc_btree.h" +#include "xfs_rmap.h" +#include "xfs_ag.h" +#include "xfs_rtbitmap.h" /* Kernel only BMAP related definitions and functions */ @@ -1274,3 +1281,303 @@ xfs_insert_file_space( xfs_iunlock(ip, XFS_ILOCK_EXCL); return error; } + +/* + * Reserve space and quota to this transaction to map in as much free space + * as we can. Callers should set @len to the amount of space desired; this + * function will shorten that quantity if it can't get space. + */ +STATIC int +xfs_map_free_reserve_more( + struct xfs_trans *tp, + struct xfs_inode *ip, + xfs_extlen_t *len) +{ + struct xfs_mount *mp = ip->i_mount; + unsigned int dblocks; + unsigned int rblocks; + unsigned int min_len; + bool isrt = XFS_IS_REALTIME_INODE(ip); + int error; + + if (*len > XFS_MAX_BMBT_EXTLEN) + *len = XFS_MAX_BMBT_EXTLEN; + min_len = isrt ? mp->m_sb.sb_rextsize : 1; + +again: + if (isrt) { + dblocks = XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK); + rblocks = *len; + } else { + dblocks = XFS_DIOSTRAT_SPACE_RES(mp, *len); + rblocks = 0; + } + error = xfs_trans_reserve_more_inode(tp, ip, dblocks, rblocks, false); + if (error == -ENOSPC && *len > min_len) { + *len >>= 1; + goto again; + } + if (error) { + trace_xfs_map_free_reserve_more_fail(ip, error, _RET_IP_); + return error; + } + + return 0; +} + +/* Find a free extent in this AG and map it into the file. */ +STATIC int +xfs_map_free_extent( + struct xfs_inode *ip, + struct xfs_perag *pag, + xfs_agblock_t *cursor, + xfs_agblock_t end_agbno, + xfs_agblock_t *last_enospc_agbno) +{ + struct xfs_bmbt_irec irec; + struct xfs_mount *mp = ip->i_mount; + struct xfs_trans *tp; + xfs_off_t endpos; + xfs_fsblock_t fsbno; + xfs_extlen_t len; + int error; + + if (fatal_signal_pending(current)) + return -EINTR; + + error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, 0, 0, false, + &tp); + if (error) + return error; + + error = xfs_alloc_find_freesp(tp, pag, cursor, end_agbno, &len); + if (error) + goto out_cancel; + + /* Bail out if the cursor is beyond what we asked for. */ + if (*cursor >= end_agbno) + goto out_cancel; + + error = xfs_map_free_reserve_more(tp, ip, &len); + if (error) + goto out_cancel; + + fsbno = XFS_AGB_TO_FSB(mp, pag->pag_agno, *cursor); + do { + error = xfs_bmapi_freesp(tp, ip, fsbno, len, &irec); + if (error == -EAGAIN) { + /* Failed to map space but were told to try again. */ + error = xfs_trans_commit(tp); + goto out; + } + if (error != -ENOSPC) + break; + /* + * If we can't get the space, try asking for successively less + * space in case we're bumping up against per-AG metadata + * reservation limits... + */ + len >>= 1; + } while (len > 0); + if (error == -ENOSPC && *last_enospc_agbno != *cursor) { + /* + * ...but even that might not work if an AGFL fixup allocated + * the block at *cursor. The first time this happens, remember + * that we ran out of space here, and try again. + */ + *last_enospc_agbno = *cursor; + error = 0; + goto out_cancel; + } + if (error) + goto out_cancel; + + /* Update isize if needed. */ + endpos = XFS_FSB_TO_B(mp, irec.br_startoff + irec.br_blockcount); + if (endpos > i_size_read(VFS_I(ip))) { + i_size_write(VFS_I(ip), endpos); + ip->i_disk_size = endpos; + xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); + } + + error = xfs_trans_commit(tp); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + if (error) + return error; + + *cursor += irec.br_blockcount; + return 0; +out_cancel: + xfs_trans_cancel(tp); +out: + xfs_iunlock(ip, XFS_ILOCK_EXCL); + return error; +} + +/* + * Allocate all free physical space between off and len and map it to this + * regular non-realtime file. + */ +int +xfs_map_free_space( + struct xfs_inode *ip, + xfs_off_t off, + xfs_off_t len) +{ + struct xfs_mount *mp = ip->i_mount; + struct xfs_perag *pag = NULL; + xfs_daddr_t off_daddr = BTOBB(off); + xfs_daddr_t end_daddr = BTOBBT(off + len); + xfs_fsblock_t off_fsb = XFS_DADDR_TO_FSB(mp, off_daddr); + xfs_fsblock_t end_fsb = XFS_DADDR_TO_FSB(mp, end_daddr); + xfs_agnumber_t off_agno = XFS_FSB_TO_AGNO(mp, off_fsb); + xfs_agnumber_t end_agno = XFS_FSB_TO_AGNO(mp, end_fsb); + xfs_agnumber_t agno; + int error = 0; + + trace_xfs_map_free_space(ip, off, len); + + agno = off_agno; + for_each_perag_range(mp, agno, end_agno, pag) { + xfs_agblock_t off_agbno = 0; + xfs_agblock_t end_agbno; + xfs_agblock_t last_enospc_agbno = NULLAGBLOCK; + + end_agbno = xfs_ag_block_count(mp, pag->pag_agno); + + if (pag->pag_agno == off_agno) + off_agbno = XFS_FSB_TO_AGBNO(mp, off_fsb); + if (pag->pag_agno == end_agno) + end_agbno = XFS_FSB_TO_AGBNO(mp, end_fsb); + + while (off_agbno < end_agbno) { + error = xfs_map_free_extent(ip, pag, &off_agbno, + end_agbno, &last_enospc_agbno); + if (error) + goto out; + } + } + +out: + if (pag) + xfs_perag_put(pag); + if (error == -ENOSPC) + return 0; + return error; +} + +#ifdef CONFIG_XFS_RT +STATIC int +xfs_map_free_rt_extent( + struct xfs_inode *ip, + xfs_rtxnum_t *cursor, + xfs_rtxnum_t end_rtx) +{ + struct xfs_bmbt_irec irec; + struct xfs_mount *mp = ip->i_mount; + struct xfs_trans *tp; + xfs_off_t endpos; + xfs_rtblock_t rtbno; + xfs_rtxnum_t add; + xfs_rtxlen_t len_rtx; + xfs_extlen_t len; + uint32_t mod; + int error; + + if (fatal_signal_pending(current)) + return -EINTR; + + error = xfs_trans_alloc_inode(ip, &M_RES(mp)->tr_write, 0, 0, false, + &tp); + if (error) + return error; + + xfs_rtbitmap_lock(tp, mp); + error = xfs_rtalloc_find_freesp(tp, cursor, end_rtx, &len_rtx); + if (error) + goto out_cancel; + + /* + * If off_rtx is beyond the end of the rt device or is past what the + * user asked for, bail out. + */ + if (*cursor >= end_rtx) + goto out_cancel; + + len = xfs_rtx_to_rtb(mp, len_rtx); + error = xfs_map_free_reserve_more(tp, ip, &len); + if (error) + goto out_cancel; + + rtbno = xfs_rtx_to_rtb(mp, *cursor); + error = xfs_bmapi_freesp(tp, ip, rtbno, len, &irec); + if (error) + goto out_cancel; + + /* Update isize if needed. */ + endpos = XFS_FSB_TO_B(mp, irec.br_startoff + irec.br_blockcount); + if (endpos > i_size_read(VFS_I(ip))) { + i_size_write(VFS_I(ip), endpos); + ip->i_disk_size = endpos; + xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); + } + + error = xfs_trans_commit(tp); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + if (error) + return error; + + add = xfs_rtb_to_rtx(mp, irec.br_blockcount, &mod); + if (mod) + return -EFSCORRUPTED; + + *cursor += add; + return 0; +out_cancel: + xfs_trans_cancel(tp); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + return error; +} + +/* + * Allocate all free physical space between off and len and map it to this + * regular non-realtime file. + */ +int +xfs_map_free_rt_space( + struct xfs_inode *ip, + xfs_off_t off, + xfs_off_t len) +{ + struct xfs_mount *mp = ip->i_mount; + xfs_rtblock_t off_rtb = XFS_B_TO_FSB(mp, off); + xfs_rtblock_t end_rtb = XFS_B_TO_FSBT(mp, off + len); + xfs_rtxnum_t off_rtx; + xfs_rtxnum_t end_rtx; + uint32_t mod; + int error = 0; + + /* Compute rt extents from the input parameters. */ + off_rtx = xfs_rtb_to_rtx(mp, off_rtb, &mod); + if (mod) + off_rtx++; + end_rtx = xfs_rtb_to_rtxt(mp, end_rtb); + + if (off_rtx >= mp->m_sb.sb_rextents) + return 0; + if (end_rtx >= mp->m_sb.sb_rextents) + end_rtx = mp->m_sb.sb_rextents - 1; + + trace_xfs_map_free_rt_space(ip, off, len); + + while (off_rtx < end_rtx) { + error = xfs_map_free_rt_extent(ip, &off_rtx, end_rtx); + if (error) + break; + } + + if (error == -ENOSPC) + return 0; + return error; +} +#endif diff --git a/fs/xfs/xfs_bmap_util.h b/fs/xfs/xfs_bmap_util.h index 8eb7166aa9d4..4e11f0def12b 100644 --- a/fs/xfs/xfs_bmap_util.h +++ b/fs/xfs/xfs_bmap_util.h @@ -76,4 +76,11 @@ int xfs_bmap_count_blocks(struct xfs_trans *tp, struct xfs_inode *ip, int xfs_flush_unmap_range(struct xfs_inode *ip, xfs_off_t offset, xfs_off_t len); +int xfs_map_free_space(struct xfs_inode *ip, xfs_off_t off, xfs_off_t len); +#ifdef CONFIG_XFS_RT +int xfs_map_free_rt_space(struct xfs_inode *ip, xfs_off_t off, xfs_off_t len); +#else +# define xfs_map_free_rt_space(ip, off, len) (-EOPNOTSUPP) +#endif + #endif /* __XFS_BMAP_UTIL_H__ */ diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 449146f9af41..ec90900eeacd 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -960,7 +960,8 @@ static inline bool xfs_file_sync_writes(struct file *filp) #define XFS_FALLOC_FL_SUPPORTED \ (FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | \ FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_ZERO_RANGE | \ - FALLOC_FL_INSERT_RANGE | FALLOC_FL_UNSHARE_RANGE) + FALLOC_FL_INSERT_RANGE | FALLOC_FL_UNSHARE_RANGE | \ + FALLOC_FL_MAP_FREE_SPACE) STATIC long xfs_file_fallocate( @@ -1075,6 +1076,40 @@ xfs_file_fallocate( goto out_unlock; } do_file_insert = true; + } else if (mode & FALLOC_FL_MAP_FREE_SPACE) { + struct xfs_mount *mp = ip->i_mount; + xfs_off_t device_size; + + if (!capable(CAP_SYS_ADMIN)) { + error = -EPERM; + goto out_unlock; + } + + if (XFS_IS_REALTIME_INODE(ip)) + device_size = XFS_FSB_TO_B(mp, mp->m_sb.sb_rblocks); + else + device_size = XFS_FSB_TO_B(mp, mp->m_sb.sb_dblocks); + + /* + * Bail out now if we aren't allowed to make the file size the + * same length as the device. + */ + if (device_size > i_size_read(inode)) { + new_size = device_size; + error = inode_newsize_ok(inode, new_size); + if (error) + goto out_unlock; + } + + if (XFS_IS_REALTIME_INODE(ip)) + error = xfs_map_free_rt_space(ip, offset, len); + else + error = xfs_map_free_space(ip, offset, len); + if (error) { + if (error == -ECANCELED) + error = 0; + goto out_unlock; + } } else { if (!(mode & FALLOC_FL_KEEP_SIZE) && offset + len > i_size_read(inode)) { diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c index 4165899cdc96..ca78b274cf57 100644 --- a/fs/xfs/xfs_rtalloc.c +++ b/fs/xfs/xfs_rtalloc.c @@ -2290,3 +2290,52 @@ xfs_rtfile_convert_unwritten( xfs_trans_cancel(tp); return ret; } + +/* + * Find the next free realtime extent starting at @rtx and going no higher than + * @end_rtx. Set @rtx and @len_rtx to whatever free extents we find, or to + * @end_rtx if we find no space. + */ +int +xfs_rtalloc_find_freesp( + struct xfs_trans *tp, + xfs_rtxnum_t *rtx, + xfs_rtxnum_t end_rtx, + xfs_rtxlen_t *len_rtx) +{ + struct xfs_mount *mp = tp->t_mountp; + unsigned int max_rt_extlen; + int error; + + trace_xfs_rtalloc_find_freesp(mp, *rtx, end_rtx - *rtx); + + max_rt_extlen = xfs_rtb_to_rtxt(mp, XFS_MAX_BMBT_EXTLEN); + + while (*rtx < end_rtx) { + xfs_rtblock_t range_end_rtx; + int is_free = 0; + + /* Is the first block in the range free? */ + error = xfs_rtcheck_range(mp, tp, *rtx, 1, 1, &range_end_rtx, + &is_free); + if (error) + return error; + + /* Free or not, how many more rtx have the same status? */ + error = xfs_rtfind_forw(mp, tp, *rtx, end_rtx, + &range_end_rtx); + if (error) + return error; + + if (is_free) { + trace_xfs_rtalloc_find_freesp_done(mp, *rtx, *len_rtx); + *len_rtx = min_t(xfs_rtblock_t, max_rt_extlen, + range_end_rtx - *rtx + 1); + return 0; + } + + *rtx = range_end_rtx + 1; + } + + return 0; +} diff --git a/fs/xfs/xfs_rtalloc.h b/fs/xfs/xfs_rtalloc.h index 35737a09cdb9..3189e3b42012 100644 --- a/fs/xfs/xfs_rtalloc.h +++ b/fs/xfs/xfs_rtalloc.h @@ -89,8 +89,17 @@ int xfs_growfs_check_rtgeom(const struct xfs_mount *mp, xfs_rfsblock_t dblocks, xfs_rfsblock_t rblocks, xfs_agblock_t rextsize, xfs_rtblock_t rextents, xfs_extlen_t rbmblocks, uint8_t rextslog); + +int xfs_rtalloc_find_freesp(struct xfs_trans *tp, xfs_rtxnum_t *rtx, + xfs_rtxnum_t end_rtx, xfs_rtxlen_t *len_rtx); #else -# define xfs_rtallocate_extent(t,b,min,max,l,f,p,rb) (-ENOSYS) +static inline int +xfs_rtallocate_extent(struct xfs_trans *tp, xfs_rtxnum_t start, + xfs_rtxlen_t minlen, xfs_rtxlen_t maxlen, xfs_rtxlen_t *len, + int wasdel, xfs_rtxlen_t prod, xfs_rtxnum_t *rtblock) +{ + return -ENOSYS; +} # define xfs_rtpick_extent(m,t,l,rb) (-ENOSYS) # define xfs_growfs_rt(mp,in) (-ENOSYS) # define xfs_rtalloc_reinit_frextents(m) (0) @@ -113,6 +122,7 @@ xfs_rtmount_init( # define xfs_rt_resv_init(mp) (0) # define xfs_rtmount_dqattach(mp) (0) # define xfs_growfs_check_rtgeom(mp, d, r, rs, rx, rb, rl) (0) +# define xfs_rtalloc_find_freesp(tp, rtx, end_rtx, len_rtx) (-ENOSYS) #endif /* CONFIG_XFS_RT */ #endif /* __XFS_RTALLOC_H__ */ diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index 8a7d08228586..f6130b85d305 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -1610,6 +1610,10 @@ DEFINE_SIMPLE_IO_EVENT(xfs_free_file_space); DEFINE_SIMPLE_IO_EVENT(xfs_zero_file_space); DEFINE_SIMPLE_IO_EVENT(xfs_collapse_file_space); DEFINE_SIMPLE_IO_EVENT(xfs_insert_file_space); +#ifdef CONFIG_XFS_RT +DEFINE_SIMPLE_IO_EVENT(xfs_map_free_rt_space); +#endif /* CONFIG_XFS_RT */ +DEFINE_SIMPLE_IO_EVENT(xfs_map_free_space); DECLARE_EVENT_CLASS(xfs_itrunc_class, TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size), @@ -1699,6 +1703,31 @@ TRACE_EVENT(xfs_bunmap, ); +TRACE_EVENT(xfs_bmapi_freesp, + TP_PROTO(struct xfs_inode *ip, xfs_fileoff_t bno, xfs_extlen_t len), + TP_ARGS(ip, bno, len), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(xfs_ino_t, ino) + __field(xfs_fsize_t, size) + __field(xfs_fileoff_t, bno) + __field(xfs_extlen_t, len) + ), + TP_fast_assign( + __entry->dev = VFS_I(ip)->i_sb->s_dev; + __entry->ino = ip->i_ino; + __entry->size = ip->i_disk_size; + __entry->bno = bno; + __entry->len = len; + ), + TP_printk("dev %d:%d ino 0x%llx disize 0x%llx fileoff 0x%llx fsbcount 0x%x", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->ino, + __entry->size, + __entry->bno, + __entry->len) +); + DECLARE_EVENT_CLASS(xfs_extent_busy_class, TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, xfs_agblock_t agbno, xfs_extlen_t len), @@ -1731,6 +1760,8 @@ DEFINE_BUSY_EVENT(xfs_extent_busy_enomem); DEFINE_BUSY_EVENT(xfs_extent_busy_force); DEFINE_BUSY_EVENT(xfs_extent_busy_reuse); DEFINE_BUSY_EVENT(xfs_extent_busy_clear); +DEFINE_BUSY_EVENT(xfs_alloc_find_freesp); +DEFINE_BUSY_EVENT(xfs_alloc_find_freesp_done); TRACE_EVENT(xfs_extent_busy_trim, TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, @@ -1762,6 +1793,35 @@ TRACE_EVENT(xfs_extent_busy_trim, __entry->tlen) ); +#ifdef CONFIG_XFS_RT +DECLARE_EVENT_CLASS(xfs_rtextent_class, + TP_PROTO(struct xfs_mount *mp, xfs_rtxnum_t off_rtx, + xfs_rtxnum_t len_rtx), + TP_ARGS(mp, off_rtx, len_rtx), + TP_STRUCT__entry( + __field(dev_t, dev) + __field(xfs_rtxnum_t, off_rtx) + __field(xfs_rtxnum_t, len_rtx) + ), + TP_fast_assign( + __entry->dev = mp->m_super->s_dev; + __entry->off_rtx = off_rtx; + __entry->len_rtx = len_rtx; + ), + TP_printk("dev %d:%d rtx 0x%llx rtxcount 0x%llx", + MAJOR(__entry->dev), MINOR(__entry->dev), + __entry->off_rtx, + __entry->len_rtx) +); +#define DEFINE_RTEXTENT_EVENT(name) \ +DEFINE_EVENT(xfs_rtextent_class, name, \ + TP_PROTO(struct xfs_mount *mp, xfs_rtxnum_t off_rtx, \ + xfs_rtxnum_t len_rtx), \ + TP_ARGS(mp, off_rtx, len_rtx)) +DEFINE_RTEXTENT_EVENT(xfs_rtalloc_find_freesp); +DEFINE_RTEXTENT_EVENT(xfs_rtalloc_find_freesp_done); +#endif /* CONFIG_XFS_RT */ + DECLARE_EVENT_CLASS(xfs_agf_class, TP_PROTO(struct xfs_mount *mp, struct xfs_agf *agf, int flags, unsigned long caller_ip), @@ -3744,6 +3804,7 @@ DECLARE_EVENT_CLASS(xfs_inode_irec_class, DEFINE_EVENT(xfs_inode_irec_class, name, \ TP_PROTO(struct xfs_inode *ip, struct xfs_bmbt_irec *irec), \ TP_ARGS(ip, irec)) +DEFINE_INODE_IREC_EVENT(xfs_bmapi_freesp_done); /* inode iomap invalidation events */ DECLARE_EVENT_CLASS(xfs_wb_invalid_class, @@ -3878,6 +3939,7 @@ DEFINE_INODE_ERROR_EVENT(xfs_reflink_remap_blocks_error); DEFINE_INODE_ERROR_EVENT(xfs_reflink_remap_extent_error); DEFINE_INODE_IREC_EVENT(xfs_reflink_remap_extent_src); DEFINE_INODE_IREC_EVENT(xfs_reflink_remap_extent_dest); +DEFINE_INODE_ERROR_EVENT(xfs_map_free_reserve_more_fail); /* dedupe tracepoints */ DEFINE_DOUBLE_IO_EVENT(xfs_reflink_compare_extents); diff --git a/include/linux/falloc.h b/include/linux/falloc.h index f3f0b97b1675..b47aae9e487a 100644 --- a/include/linux/falloc.h +++ b/include/linux/falloc.h @@ -30,7 +30,8 @@ struct space_resv { FALLOC_FL_COLLAPSE_RANGE | \ FALLOC_FL_ZERO_RANGE | \ FALLOC_FL_INSERT_RANGE | \ - FALLOC_FL_UNSHARE_RANGE) + FALLOC_FL_UNSHARE_RANGE | \ + FALLOC_FL_MAP_FREE_SPACE) /* on ia32 l_start is on a 32-bit boundary */ #if defined(CONFIG_X86_64) diff --git a/include/uapi/linux/falloc.h b/include/uapi/linux/falloc.h index 51398fa57f6c..89bfbb02bc16 100644 --- a/include/uapi/linux/falloc.h +++ b/include/uapi/linux/falloc.h @@ -2,6 +2,14 @@ #ifndef _UAPI_FALLOC_H_ #define _UAPI_FALLOC_H_ +/* + * FALLOC_FL_MAP_FREE_SPACE maps all the free physical space in the + * filesystem into the file at the same offsets. This flag requires + * CAP_SYS_ADMIN, and cannot be used with any other flags. It probably + * only works on filesystems that are backed by physical media. + */ +#define FALLOC_FL_MAP_FREE_SPACE (1U << 30) + #define FALLOC_FL_KEEP_SIZE 0x01 /* default is extend size */ #define FALLOC_FL_PUNCH_HOLE 0x02 /* de-allocates range */ #define FALLOC_FL_NO_HIDE_STALE 0x04 /* reserved codepoint */ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 1/2] xfs: capture the offset and length in fallocate tracepoints 2022-12-30 22:19 ` [PATCHSET 0/2] xfs: defragment free space Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/2] xfs: fallocate free space into a file Darrick J. Wong @ 2022-12-30 22:19 ` Darrick J. Wong 1 sibling, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:19 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Change the class of the fallocate tracepoints to capture the offset and length of the requested operation. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/xfs_bmap_util.c | 8 ++++---- fs/xfs/xfs_file.c | 2 +- fs/xfs/xfs_trace.h | 10 +++++----- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index 558951710404..8515190d2bc6 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -828,7 +828,7 @@ xfs_alloc_file_space( xfs_bmbt_irec_t imaps[1], *imapp; int error; - trace_xfs_alloc_file_space(ip); + trace_xfs_alloc_file_space(ip, offset, len); if (xfs_is_shutdown(mp)) return -EIO; @@ -1012,7 +1012,7 @@ xfs_free_file_space( xfs_fileoff_t endoffset_fsb; int done = 0, error; - trace_xfs_free_file_space(ip); + trace_xfs_free_file_space(ip, offset, len); error = xfs_qm_dqattach(ip); if (error) @@ -1150,7 +1150,7 @@ xfs_collapse_file_space( ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); - trace_xfs_collapse_file_space(ip); + trace_xfs_collapse_file_space(ip, offset, len); error = xfs_free_file_space(ip, offset, len); if (error) @@ -1220,7 +1220,7 @@ xfs_insert_file_space( ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); - trace_xfs_insert_file_space(ip); + trace_xfs_insert_file_space(ip, offset, len); error = xfs_bmap_can_insert_extents(ip, stop_fsb, shift_fsb); if (error) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 87e836e1aeb3..449146f9af41 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1096,7 +1096,7 @@ xfs_file_fallocate( */ unsigned int blksize = i_blocksize(inode); - trace_xfs_zero_file_space(ip); + trace_xfs_zero_file_space(ip, offset, len); /* Unshare around the region to zero, if needed. */ if (xfs_inode_needs_cow_around(ip)) { diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h index b422a206edb5..8a7d08228586 100644 --- a/fs/xfs/xfs_trace.h +++ b/fs/xfs/xfs_trace.h @@ -817,11 +817,6 @@ DEFINE_INODE_EVENT(xfs_getattr); DEFINE_INODE_EVENT(xfs_setattr); DEFINE_INODE_EVENT(xfs_readlink); DEFINE_INODE_EVENT(xfs_inactive_symlink); -DEFINE_INODE_EVENT(xfs_alloc_file_space); -DEFINE_INODE_EVENT(xfs_free_file_space); -DEFINE_INODE_EVENT(xfs_zero_file_space); -DEFINE_INODE_EVENT(xfs_collapse_file_space); -DEFINE_INODE_EVENT(xfs_insert_file_space); DEFINE_INODE_EVENT(xfs_readdir); #ifdef CONFIG_XFS_POSIX_ACL DEFINE_INODE_EVENT(xfs_get_acl); @@ -1610,6 +1605,11 @@ DEFINE_SIMPLE_IO_EVENT(xfs_zero_eof); DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write); DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write_unwritten); DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write_append); +DEFINE_SIMPLE_IO_EVENT(xfs_alloc_file_space); +DEFINE_SIMPLE_IO_EVENT(xfs_free_file_space); +DEFINE_SIMPLE_IO_EVENT(xfs_zero_file_space); +DEFINE_SIMPLE_IO_EVENT(xfs_collapse_file_space); +DEFINE_SIMPLE_IO_EVENT(xfs_insert_file_space); DECLARE_EVENT_CLASS(xfs_itrunc_class, TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size), ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 00/11] xfs_scrub: vectorize kernel calls 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (3 preceding siblings ...) 2022-12-30 22:19 ` [PATCHSET 0/2] xfs: defragment free space Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 03/11] libfrog: support vectored scrub Darrick J. Wong ` (10 more replies) 2022-12-30 22:20 ` [PATCHSET 0/1] libxfs: report refcount information to userspace Darrick J. Wong ` (4 subsequent siblings) 9 siblings, 11 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs Hi all, Create a vectorized version of the metadata scrub and repair ioctl, and adapt xfs_scrub to use that. This is an experiment to measure overhead and to try refactoring xfs_scrub. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=vectorized-scrub xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=vectorized-scrub --- include/xfs_trans.h | 4 io/scrub.c | 485 +++++++++++++++++++++++++++++++++++++++++-------- libfrog/fsgeom.h | 6 + libfrog/scrub.c | 124 +++++++++++++ libfrog/scrub.h | 1 libxfs/xfs_defer.c | 14 + libxfs/xfs_fs.h | 37 ++++ man/man8/xfs_io.8 | 47 +++++ scrub/phase1.c | 2 scrub/phase2.c | 106 +++++++++-- scrub/phase3.c | 84 +++++++- scrub/repair.c | 347 ++++++++++++++++++++++------------- scrub/scrub.c | 353 ++++++++++++++++++++++++++---------- scrub/scrub.h | 19 ++ scrub/scrub_private.h | 62 ++++-- 15 files changed, 1344 insertions(+), 347 deletions(-) ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 03/11] libfrog: support vectored scrub 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 04/11] xfs_io: " Darrick J. Wong ` (9 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Enhance libfrog to support performing vectored metadata scrub. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- libfrog/fsgeom.h | 6 +++ libfrog/scrub.c | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ libfrog/scrub.h | 1 3 files changed, 131 insertions(+) diff --git a/libfrog/fsgeom.h b/libfrog/fsgeom.h index 6c6d6bb815a..0265a7c1684 100644 --- a/libfrog/fsgeom.h +++ b/libfrog/fsgeom.h @@ -58,6 +58,12 @@ struct xfs_fd { /* Only use FIEXCHANGE_RANGE for file data exchanges. */ #define XFROG_FLAG_FORCE_FIEXCHANGE (1 << 3) +/* Only use the older one-at-a-time scrub ioctl. */ +#define XFROG_FLAG_SCRUB_FORCE_SINGLE (1 << 4) + +/* Only use the vectored scrub ioctl. */ +#define XFROG_FLAG_SCRUB_FORCE_VECTOR (1 << 5) + /* Static initializers */ #define XFS_FD_INIT(_fd) { .fd = (_fd), } #define XFS_FD_INIT_EMPTY XFS_FD_INIT(-1) diff --git a/libfrog/scrub.c b/libfrog/scrub.c index c3cf5312f80..02b659ea2bd 100644 --- a/libfrog/scrub.c +++ b/libfrog/scrub.c @@ -186,3 +186,127 @@ xfrog_scrub_metadata( return 0; } + +/* Decide if there have been any scrub failures up to this point. */ +static inline int +xfrog_scrubv_previous_failures( + struct xfs_scrub_vec_head *vhead, + struct xfs_scrub_vec *barrier_vec) +{ + struct xfs_scrub_vec *v; + __u32 failmask; + + failmask = barrier_vec->sv_flags & XFS_SCRUB_FLAGS_OUT; + for (v = vhead->svh_vecs; v < barrier_vec; v++) { + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) + continue; + + /* + * Runtime errors count as a previous failure, except the ones + * used to ask userspace to retry. + */ + if (v->sv_ret && v->sv_ret != -EBUSY && v->sv_ret != -ENOENT && + v->sv_ret != -EUSERS) + return -ECANCELED; + + /* + * If any of the out-flags on the scrub vector match the mask + * that was set on the barrier vector, that's a previous fail. + */ + if (v->sv_flags & failmask) + return -ECANCELED; + } + + return 0; +} + +static int +xfrog_scrubv_fallback( + struct xfs_fd *xfd, + struct xfs_scrub_vec_head *vhead) +{ + struct xfs_scrub_vec *v; + unsigned int i; + + if (vhead->svh_flags & ~XFS_SCRUB_VEC_FLAGS_ALL) + return -EINVAL; + for (i = 0, v = vhead->svh_vecs; i < vhead->svh_nr; i++, v++) { + if (v->sv_reserved) + return -EINVAL; + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER && + (v->sv_flags & ~XFS_SCRUB_FLAGS_OUT)) + return -EINVAL; + } + + /* Run all the scrubbers. */ + for (i = 0, v = vhead->svh_vecs; i < vhead->svh_nr; i++, v++) { + struct xfs_scrub_metadata sm = { + .sm_type = v->sv_type, + .sm_flags = v->sv_flags, + .sm_ino = vhead->svh_ino, + .sm_gen = vhead->svh_gen, + .sm_agno = vhead->svh_agno, + }; + struct timespec tv; + + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) { + v->sv_ret = xfrog_scrubv_previous_failures(vhead, v); + if (v->sv_ret) + break; + continue; + } + + v->sv_ret = xfrog_scrub_metadata(xfd, &sm); + v->sv_flags = sm.sm_flags; + + if (vhead->svh_rest_us) { + tv.tv_sec = 0; + tv.tv_nsec = vhead->svh_rest_us * 1000; + nanosleep(&tv, NULL); + } + } + + return 0; +} + +/* Invoke the vectored scrub ioctl. */ +static int +xfrog_scrubv_call( + struct xfs_fd *xfd, + struct xfs_scrub_vec_head *vhead) +{ + int ret; + + ret = ioctl(xfd->fd, XFS_IOC_SCRUBV_METADATA, vhead); + if (ret) + return -errno; + + return 0; +} + +/* Invoke the vectored scrub ioctl. Returns zero or negative error code. */ +int +xfrog_scrubv_metadata( + struct xfs_fd *xfd, + struct xfs_scrub_vec_head *vhead) +{ + int error = 0; + + if (xfd->flags & XFROG_FLAG_SCRUB_FORCE_SINGLE) + goto try_single; + + error = xfrog_scrubv_call(xfd, vhead); + if (error == 0 || (xfd->flags & XFROG_FLAG_SCRUB_FORCE_VECTOR)) + return error; + + /* If the vectored scrub ioctl wasn't found, force single mode. */ + switch (error) { + case -EOPNOTSUPP: + case -ENOTTY: + xfd->flags |= XFROG_FLAG_SCRUB_FORCE_SINGLE; + break; + } + +try_single: + return xfrog_scrubv_fallback(xfd, vhead); +} diff --git a/libfrog/scrub.h b/libfrog/scrub.h index 7155e6a9b0e..2f5ca2b1317 100644 --- a/libfrog/scrub.h +++ b/libfrog/scrub.h @@ -28,5 +28,6 @@ struct xfrog_scrub_descr { extern const struct xfrog_scrub_descr xfrog_scrubbers[XFS_SCRUB_TYPE_NR]; int xfrog_scrub_metadata(struct xfs_fd *xfd, struct xfs_scrub_metadata *meta); +int xfrog_scrubv_metadata(struct xfs_fd *xfd, struct xfs_scrub_vec_head *vhead); #endif /* __LIBFROG_SCRUB_H__ */ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 04/11] xfs_io: support vectored scrub 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong 2022-12-30 22:20 ` [PATCH 03/11] libfrog: support vectored scrub Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 01/11] xfs: track deferred ops statistics Darrick J. Wong ` (8 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Create a new scrubv command to xfs_io to support the vectored scrub ioctl. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- io/scrub.c | 485 ++++++++++++++++++++++++++++++++++++++++++++--------- man/man8/xfs_io.8 | 47 +++++ 2 files changed, 451 insertions(+), 81 deletions(-) diff --git a/io/scrub.c b/io/scrub.c index d764a5a997b..117855f8c7a 100644 --- a/io/scrub.c +++ b/io/scrub.c @@ -12,10 +12,13 @@ #include "libfrog/paths.h" #include "libfrog/fsgeom.h" #include "libfrog/scrub.h" +#include "libfrog/logging.h" #include "io.h" +#include "list.h" static struct cmdinfo scrub_cmd; static struct cmdinfo repair_cmd; +static const struct cmdinfo scrubv_cmd; static void scrub_help(void) @@ -42,54 +45,61 @@ scrub_help(void) } static void -scrub_ioctl( - int fd, - int type, - uint64_t control, - uint32_t control2, - uint32_t flags) +report_scrub_outcome( + uint32_t flags) { - struct xfs_scrub_metadata meta; - const struct xfrog_scrub_descr *sc; - int error; - - sc = &xfrog_scrubbers[type]; - memset(&meta, 0, sizeof(meta)); - meta.sm_type = type; - switch (sc->group) { - case XFROG_SCRUB_GROUP_AGHEADER: - case XFROG_SCRUB_GROUP_PERAG: - case XFROG_SCRUB_GROUP_RTGROUP: - meta.sm_agno = control; - break; - case XFROG_SCRUB_GROUP_INODE: - meta.sm_ino = control; - meta.sm_gen = control2; - break; - case XFROG_SCRUB_GROUP_NONE: - case XFROG_SCRUB_GROUP_METAFILES: - case XFROG_SCRUB_GROUP_SUMMARY: - case XFROG_SCRUB_GROUP_ISCAN: - /* no control parameters */ - break; - } - meta.sm_flags = flags; - - error = ioctl(fd, XFS_IOC_SCRUB_METADATA, &meta); - if (error) - perror("scrub"); - if (meta.sm_flags & XFS_SCRUB_OFLAG_CORRUPT) + if (flags & XFS_SCRUB_OFLAG_CORRUPT) printf(_("Corruption detected.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_PREEN) + if (flags & XFS_SCRUB_OFLAG_PREEN) printf(_("Optimization possible.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_XFAIL) + if (flags & XFS_SCRUB_OFLAG_XFAIL) printf(_("Cross-referencing failed.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_XCORRUPT) + if (flags & XFS_SCRUB_OFLAG_XCORRUPT) printf(_("Corruption detected during cross-referencing.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE) + if (flags & XFS_SCRUB_OFLAG_INCOMPLETE) printf(_("Scan was not complete.\n")); } +static void +scrub_ioctl( + int fd, + int type, + uint64_t control, + uint32_t control2, + uint32_t flags) +{ + struct xfs_scrub_metadata meta; + const struct xfrog_scrub_descr *sc; + int error; + + sc = &xfrog_scrubbers[type]; + memset(&meta, 0, sizeof(meta)); + meta.sm_type = type; + switch (sc->group) { + case XFROG_SCRUB_GROUP_AGHEADER: + case XFROG_SCRUB_GROUP_PERAG: + case XFROG_SCRUB_GROUP_RTGROUP: + meta.sm_agno = control; + break; + case XFROG_SCRUB_GROUP_INODE: + meta.sm_ino = control; + meta.sm_gen = control2; + break; + case XFROG_SCRUB_GROUP_NONE: + case XFROG_SCRUB_GROUP_METAFILES: + case XFROG_SCRUB_GROUP_SUMMARY: + case XFROG_SCRUB_GROUP_ISCAN: + /* no control parameters */ + break; + } + meta.sm_flags = flags; + + error = ioctl(fd, XFS_IOC_SCRUB_METADATA, &meta); + if (error) + perror("scrub"); + report_scrub_outcome(meta.sm_flags); +} + static int parse_args( int argc, @@ -223,6 +233,7 @@ scrub_init(void) scrub_cmd.help = scrub_help; add_command(&scrub_cmd); + add_command(&scrubv_cmd); } static void @@ -252,56 +263,63 @@ repair_help(void) } static void -repair_ioctl( - int fd, - int type, - uint64_t control, - uint32_t control2, - uint32_t flags) +report_repair_outcome( + uint32_t flags) { - struct xfs_scrub_metadata meta; - const struct xfrog_scrub_descr *sc; - int error; - - sc = &xfrog_scrubbers[type]; - memset(&meta, 0, sizeof(meta)); - meta.sm_type = type; - switch (sc->group) { - case XFROG_SCRUB_GROUP_AGHEADER: - case XFROG_SCRUB_GROUP_PERAG: - case XFROG_SCRUB_GROUP_RTGROUP: - meta.sm_agno = control; - break; - case XFROG_SCRUB_GROUP_INODE: - meta.sm_ino = control; - meta.sm_gen = control2; - break; - case XFROG_SCRUB_GROUP_NONE: - case XFROG_SCRUB_GROUP_METAFILES: - case XFROG_SCRUB_GROUP_SUMMARY: - case XFROG_SCRUB_GROUP_ISCAN: - /* no control parameters */ - break; - } - meta.sm_flags = flags | XFS_SCRUB_IFLAG_REPAIR; - - error = ioctl(fd, XFS_IOC_SCRUB_METADATA, &meta); - if (error) - perror("repair"); - if (meta.sm_flags & XFS_SCRUB_OFLAG_CORRUPT) + if (flags & XFS_SCRUB_OFLAG_CORRUPT) printf(_("Corruption remains.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_PREEN) + if (flags & XFS_SCRUB_OFLAG_PREEN) printf(_("Optimization possible.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_XFAIL) + if (flags & XFS_SCRUB_OFLAG_XFAIL) printf(_("Cross-referencing failed.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_XCORRUPT) + if (flags & XFS_SCRUB_OFLAG_XCORRUPT) printf(_("Corruption still detected during cross-referencing.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE) + if (flags & XFS_SCRUB_OFLAG_INCOMPLETE) printf(_("Repair was not complete.\n")); - if (meta.sm_flags & XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) + if (flags & XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) printf(_("Metadata did not need repair or optimization.\n")); } +static void +repair_ioctl( + int fd, + int type, + uint64_t control, + uint32_t control2, + uint32_t flags) +{ + struct xfs_scrub_metadata meta; + const struct xfrog_scrub_descr *sc; + int error; + + sc = &xfrog_scrubbers[type]; + memset(&meta, 0, sizeof(meta)); + meta.sm_type = type; + switch (sc->group) { + case XFROG_SCRUB_GROUP_AGHEADER: + case XFROG_SCRUB_GROUP_PERAG: + case XFROG_SCRUB_GROUP_RTGROUP: + meta.sm_agno = control; + break; + case XFROG_SCRUB_GROUP_INODE: + meta.sm_ino = control; + meta.sm_gen = control2; + break; + case XFROG_SCRUB_GROUP_NONE: + case XFROG_SCRUB_GROUP_METAFILES: + case XFROG_SCRUB_GROUP_SUMMARY: + case XFROG_SCRUB_GROUP_ISCAN: + /* no control parameters */ + break; + } + meta.sm_flags = flags | XFS_SCRUB_IFLAG_REPAIR; + + error = ioctl(fd, XFS_IOC_SCRUB_METADATA, &meta); + if (error) + perror("repair"); + report_repair_outcome(meta.sm_flags); +} + static int repair_f( int argc, @@ -327,3 +345,308 @@ repair_init(void) add_command(&repair_cmd); } + +static void +scrubv_help(void) +{ + printf(_( +"\n" +" Scrubs pieces of XFS filesystem metadata. The first argument is the group\n" +" of metadata to examine. If the group is 'ag', the second parameter should\n" +" be the AG number. If the group is 'inode', the second and third parameters\n" +" should be the inode number and generation number to act upon; if these are\n" +" omitted, the scrub is performed on the open file. If the group is 'fs',\n" +" 'summary', or 'probe', there are no other parameters.\n" +"\n" +" Flags are -d for debug, and -r to allow repairs.\n" +" -b NN will insert a scrub barrier after every NN scrubs, and -m sets the\n" +" desired corruption mask in all barriers. -w pauses for some microseconds\n" +" after each scrub call.\n" +"\n" +" Example:\n" +" 'scrubv ag 3' - scrub all metadata in AG 3.\n" +" 'scrubv ag 3 -b 2 -m 0x4' - scrub all metadata in AG 3, and use barriers\n" +" every third scrub to exit early if there are optimizations.\n" +" 'scrubv fs' - scrub all non-AG non-file metadata.\n" +" 'scrubv inode' - scrub all metadata for the open file.\n" +" 'scrubv inode 128 13525' - scrub all metadata for inode 128 gen 13525.\n" +" 'scrubv probe' - check for presence of online scrub.\n" +" 'scrubv summary' - scrub all summary metadata.\n")); +} + +/* Fill out the scrub vectors for a group of scrubber (ag, ino, fs, summary) */ +static void +scrubv_fill_group( + struct xfs_scrub_vec_head *vhead, + int barrier_interval, + __u32 barrier_mask, + enum xfrog_scrub_group group) +{ + const struct xfrog_scrub_descr *d; + unsigned int i; + + for (i = 0, d = xfrog_scrubbers; i < XFS_SCRUB_TYPE_NR; i++, d++) { + if (d->group != group) + continue; + vhead->svh_vecs[vhead->svh_nr++].sv_type = i; + + if (barrier_interval && + vhead->svh_nr % (barrier_interval + 1) == 0) { + struct xfs_scrub_vec *v; + + v = &vhead->svh_vecs[vhead->svh_nr++]; + v->sv_flags = barrier_mask; + v->sv_type = XFS_SCRUB_TYPE_BARRIER; + } + } +} + +/* Declare a structure big enough to handle all scrub types + barriers */ +struct scrubv_head { + struct xfs_scrub_vec_head head; + struct xfs_scrub_vec __vecs[XFS_SCRUB_TYPE_NR * 2]; +}; + + +static int +scrubv_f( + int argc, + char **argv) +{ + struct scrubv_head bighead = { }; + struct xfs_fd xfd = XFS_FD_INIT(file->fd); + struct xfs_scrub_vec_head *vhead = &bighead.head; + struct xfs_scrub_vec *v; + char *p; + uint32_t flags = 0; + __u32 barrier_mask = XFS_SCRUB_OFLAG_CORRUPT; + enum xfrog_scrub_group group; + bool debug = false; + int version = -1; + int barrier_interval = 0; + int rest_us = 0; + int c; + int error; + + while ((c = getopt(argc, argv, "b:dm:rv:w:")) != EOF) { + switch (c) { + case 'b': + barrier_interval = atoi(optarg); + if (barrier_interval < 0) { + printf( + _("Negative barrier interval makes no sense.\n")); + return 0; + } + break; + case 'd': + debug = true; + break; + case 'm': + barrier_mask = strtoul(optarg, NULL, 0); + break; + case 'r': + flags |= XFS_SCRUB_IFLAG_REPAIR; + break; + case 'v': + version = atoi(optarg); + if (version != 0 && version != 1) { + printf(_("API version must be 0 or 1.\n")); + return 0; + } + break; + case 'w': + rest_us = atoi(optarg); + if (rest_us < 0) { + printf(_("Rest time must be positive.\n")); + return 0; + } + break; + default: + scrubv_help(); + return 0; + } + } + if (optind > argc - 1) { + scrubv_help(); + return 0; + } + + if ((flags & XFS_SCRUB_IFLAG_REPAIR) && !expert) { + printf(_("Repair flag requires expert mode.\n")); + return 1; + } + + vhead->svh_rest_us = rest_us; + for (c = 0, v = vhead->svh_vecs; c < vhead->svh_nr; c++, v++) + v->sv_flags = flags; + + /* Extract group and domain information from cmdline. */ + if (!strcmp(argv[optind], "probe")) + group = XFROG_SCRUB_GROUP_NONE; + else if (!strcmp(argv[optind], "agheader")) + group = XFROG_SCRUB_GROUP_AGHEADER; + else if (!strcmp(argv[optind], "ag")) + group = XFROG_SCRUB_GROUP_PERAG; + else if (!strcmp(argv[optind], "metafiles")) + group = XFROG_SCRUB_GROUP_METAFILES; + else if (!strcmp(argv[optind], "inode")) + group = XFROG_SCRUB_GROUP_INODE; + else if (!strcmp(argv[optind], "iscan")) + group = XFROG_SCRUB_GROUP_ISCAN; + else if (!strcmp(argv[optind], "summary")) + group = XFROG_SCRUB_GROUP_SUMMARY; + else if (!strcmp(argv[optind], "rtgroup")) + group = XFROG_SCRUB_GROUP_RTGROUP; + else { + printf(_("Unknown group '%s'.\n"), argv[optind]); + scrubv_help(); + return 0; + } + optind++; + + switch (group) { + case XFROG_SCRUB_GROUP_INODE: + if (optind == argc) { + vhead->svh_ino = 0; + vhead->svh_gen = 0; + } else if (optind == argc - 2) { + vhead->svh_ino = strtoull(argv[optind], &p, 0); + if (*p != '\0') { + fprintf(stderr, + _("Bad inode number '%s'.\n"), + argv[optind]); + return 0; + } + vhead->svh_gen = strtoul(argv[optind + 1], &p, 0); + if (*p != '\0') { + fprintf(stderr, + _("Bad generation number '%s'.\n"), + argv[optind + 1]); + return 0; + } + } else { + fprintf(stderr, + _("Must specify inode number and generation.\n")); + return 0; + } + break; + case XFROG_SCRUB_GROUP_AGHEADER: + case XFROG_SCRUB_GROUP_PERAG: + if (optind != argc - 1) { + fprintf(stderr, + _("Must specify one AG number.\n")); + return 0; + } + vhead->svh_agno = strtoul(argv[optind], &p, 0); + if (*p != '\0') { + fprintf(stderr, + _("Bad AG number '%s'.\n"), argv[optind]); + return 0; + } + break; + case XFROG_SCRUB_GROUP_METAFILES: + case XFROG_SCRUB_GROUP_SUMMARY: + case XFROG_SCRUB_GROUP_ISCAN: + case XFROG_SCRUB_GROUP_NONE: + if (optind != argc) { + fprintf(stderr, + _("No parameters allowed.\n")); + return 0; + } + break; + case XFROG_SCRUB_GROUP_RTGROUP: + if (optind != argc - 1) { + fprintf(stderr, + _("Must specify one rtgroup number.\n")); + return 0; + } + vhead->svh_agno = strtoul(argv[optind], &p, 0); + if (*p != '\0') { + fprintf(stderr, + _("Bad rtgroup number '%s'.\n"), argv[optind]); + return 0; + } + break; + default: + ASSERT(0); + break; + } + scrubv_fill_group(vhead, barrier_interval, barrier_mask, group); + assert(vhead->svh_nr < ARRAY_SIZE(bighead.__vecs)); + + error = -xfd_prepare_geometry(&xfd); + if (error) { + xfrog_perror(error, "xfd_prepare_geometry"); + exitcode = 1; + return 0; + } + + switch (version) { + case 0: + xfd.flags |= XFROG_FLAG_SCRUB_FORCE_SINGLE; + break; + case 1: + xfd.flags |= XFROG_FLAG_SCRUB_FORCE_VECTOR; + break; + default: + break; + } + + error = -xfrog_scrubv_metadata(&xfd, vhead); + if (error) { + xfrog_perror(error, "xfrog_scrub_many"); + exitcode = 1; + return 0; + } + + /* Figure out what happened. */ + for (c = 0, v = vhead->svh_vecs; debug && c < vhead->svh_nr; c++, v++) { + const char *type; + + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) + type = _("barrier"); + else + type = _(xfrog_scrubbers[v->sv_type].descr); + printf(_("[%02u] %-25s: flags 0x%x ret %d\n"), c, type, + v->sv_flags, v->sv_ret); + } + + /* Figure out what happened. */ + for (c = 0, v = vhead->svh_vecs; c < vhead->svh_nr; c++, v++) { + /* Report barrier failures. */ + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) { + if (v->sv_ret) { + printf(_("barrier: FAILED\n")); + break; + } + continue; + } + + printf("%s: ", _(xfrog_scrubbers[v->sv_type].descr)); + switch (v->sv_ret) { + case 0: + break; + default: + printf("%s\n", strerror(-v->sv_ret)); + continue; + } + if (!(v->sv_flags & XFS_SCRUB_FLAGS_OUT)) + printf(_("OK.\n")); + else if (v->sv_flags & XFS_SCRUB_IFLAG_REPAIR) + report_repair_outcome(v->sv_flags); + else + report_scrub_outcome(v->sv_flags); + } + + return 0; +} + +static const struct cmdinfo scrubv_cmd = { + .name = "scrubv", + .cfunc = scrubv_f, + .argmin = 1, + .argmax = -1, + .flags = CMD_NOMAP_OK, + .oneline = N_("vectored metadata scrub"), + .help = scrubv_help, +}; diff --git a/man/man8/xfs_io.8 b/man/man8/xfs_io.8 index 16768275b5c..92458e8a787 100644 --- a/man/man8/xfs_io.8 +++ b/man/man8/xfs_io.8 @@ -1420,6 +1420,53 @@ inode number and generation number are specified. .RE .PD .TP +.BI "scrubv [ \-b NN ] [ \-d ] [ \-f ] [ \-r ] [ \-v NN ] [ \-w ms ] " group " [ " agnumber " | " "ino" " " "gen" " ]" +Scrub a bunch of internal XFS filesystem metadata. +The +.BI group +parameter specifies which group of metadata to scrub. +Valid groups are +.IR ag ", " agheader ", " inode ", " iscan ", " metafiles ", " probe ", " rtgroup ", or " summary . + +For +.BR ag " and " agheader +metadata, one AG number must be specified. +For +.B inode +metadata, the scrub is applied to the open file unless the +inode number and generation number are specified. +For +.B rtgroup +metadata, one rt group number must be specified. + +.RS 1.0i +.PD 0 +.TP +.BI "\-b " NN +Inject scrub barriers into the vector stream at the given interval. +Barriers abort vector processing if any previous scrub function found +corruption. +.TP +.BI \-d +Enables debug mode. +.TP +.BI \-f +Permit the kernel to freeze the filesystem in order to scrub or repair. +.TP +.BI \-r +Repair metadata if corruptions are found. +This option requires expert mode. +.TP +.BI "\-v " NN +Force a particular API version. +0 selects XFS_SCRUB_METADATA (one-by-one). +1 selects XFS_SCRUBV_METADATA (vectored). +.TP +.BI "\-w " us +Wait the given number of microseconds between each scrub function. +.RE +.PD +.TP .BI "repair " type " [ " agnumber " | " "ino" " " "gen" " ]" Repair internal XFS filesystem metadata. The .BI type ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 01/11] xfs: track deferred ops statistics 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong 2022-12-30 22:20 ` [PATCH 03/11] libfrog: support vectored scrub Darrick J. Wong 2022-12-30 22:20 ` [PATCH 04/11] xfs_io: " Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 02/11] xfs: introduce vectored scrub mode Darrick J. Wong ` (7 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Track some basic statistics on how hard we're pushing the defer ops. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- include/xfs_trans.h | 4 ++++ libxfs/xfs_defer.c | 14 ++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/xfs_trans.h b/include/xfs_trans.h index 0ecf0a95560..9b12791ff4e 100644 --- a/include/xfs_trans.h +++ b/include/xfs_trans.h @@ -75,6 +75,10 @@ typedef struct xfs_trans { long t_frextents_delta;/* superblock freextents chg*/ struct list_head t_items; /* log item descriptors */ struct list_head t_dfops; /* deferred operations */ + + unsigned int t_dfops_nr; + unsigned int t_dfops_nr_max; + unsigned int t_dfops_finished; } xfs_trans_t; void xfs_trans_init(struct xfs_mount *); diff --git a/libxfs/xfs_defer.c b/libxfs/xfs_defer.c index 1d8bf2f6f65..2ca03674e2d 100644 --- a/libxfs/xfs_defer.c +++ b/libxfs/xfs_defer.c @@ -504,6 +504,8 @@ xfs_defer_finish_one( /* Done with the dfp, free it. */ list_del(&dfp->dfp_list); kmem_cache_free(xfs_defer_pending_cache, dfp); + tp->t_dfops_nr--; + tp->t_dfops_finished++; out: if (ops->finish_cleanup) ops->finish_cleanup(tp, state, error); @@ -545,6 +547,9 @@ xfs_defer_finish_noroll( list_splice_init(&(*tp)->t_dfops, &dop_pending); + (*tp)->t_dfops_nr_max = max((*tp)->t_dfops_nr, + (*tp)->t_dfops_nr_max); + if (has_intents < 0) { error = has_intents; goto out_shutdown; @@ -575,6 +580,7 @@ xfs_defer_finish_noroll( xfs_force_shutdown((*tp)->t_mountp, SHUTDOWN_CORRUPT_INCORE); trace_xfs_defer_finish_error(*tp, error); xfs_defer_cancel_list((*tp)->t_mountp, &dop_pending); + (*tp)->t_dfops_nr = 0; xfs_defer_cancel(*tp); return error; } @@ -615,6 +621,7 @@ xfs_defer_cancel( trace_xfs_defer_cancel(tp, _RET_IP_); xfs_defer_cancel_list(mp, &tp->t_dfops); + tp->t_dfops_nr = 0; } /* Add an item for later deferred processing. */ @@ -651,6 +658,7 @@ xfs_defer_add( dfp->dfp_count = 0; INIT_LIST_HEAD(&dfp->dfp_work); list_add_tail(&dfp->dfp_list, &tp->t_dfops); + tp->t_dfops_nr++; } list_add_tail(li, &dfp->dfp_work); @@ -669,6 +677,12 @@ xfs_defer_move( struct xfs_trans *stp) { list_splice_init(&stp->t_dfops, &dtp->t_dfops); + dtp->t_dfops_nr += stp->t_dfops_nr; + dtp->t_dfops_nr_max = stp->t_dfops_nr_max; + dtp->t_dfops_finished = stp->t_dfops_finished; + stp->t_dfops_nr = 0; + stp->t_dfops_nr_max = 0; + stp->t_dfops_finished = 0; /* * Low free space mode was historically controlled by a dfops field. ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 02/11] xfs: introduce vectored scrub mode 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (2 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 01/11] xfs: track deferred ops statistics Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 08/11] xfs_scrub: vectorize scrub calls Darrick J. Wong ` (6 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Introduce a variant on XFS_SCRUB_METADATA that allows for vectored mode. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- libxfs/xfs_fs.h | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/libxfs/xfs_fs.h b/libxfs/xfs_fs.h index 453b0861225..067dd0b1315 100644 --- a/libxfs/xfs_fs.h +++ b/libxfs/xfs_fs.h @@ -751,6 +751,15 @@ struct xfs_scrub_metadata { /* Number of scrub subcommands. */ #define XFS_SCRUB_TYPE_NR 32 +/* + * This special type code only applies to the vectored scrub implementation. + * + * If any of the previous scrub vectors recorded runtime errors or have + * sv_flags bits set that match the OFLAG bits in the barrier vector's + * sv_flags, set the barrier's sv_ret to -ECANCELED and return to userspace. + */ +#define XFS_SCRUB_TYPE_BARRIER (-1U) + /* i: Repair this metadata. */ #define XFS_SCRUB_IFLAG_REPAIR (1u << 0) @@ -795,6 +804,33 @@ struct xfs_scrub_metadata { XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) #define XFS_SCRUB_FLAGS_ALL (XFS_SCRUB_FLAGS_IN | XFS_SCRUB_FLAGS_OUT) +struct xfs_scrub_vec { + __u32 sv_type; /* XFS_SCRUB_TYPE_* */ + __u32 sv_flags; /* XFS_SCRUB_FLAGS_* */ + __s32 sv_ret; /* 0 or a negative error code */ + __u32 sv_reserved; /* must be zero */ +}; + +/* Vectored metadata scrub control structure. */ +struct xfs_scrub_vec_head { + __u64 svh_ino; /* inode number. */ + __u32 svh_gen; /* inode generation. */ + __u32 svh_agno; /* ag number. */ + __u32 svh_flags; /* XFS_SCRUB_VEC_FLAGS_* */ + __u16 svh_rest_us; /* wait this much time between vector items */ + __u16 svh_nr; /* number of svh_vecs */ + + struct xfs_scrub_vec svh_vecs[0]; +}; + +#define XFS_SCRUB_VEC_FLAGS_ALL (0) + +static inline size_t sizeof_xfs_scrub_vec(unsigned int nr) +{ + return sizeof(struct xfs_scrub_vec_head) + + nr * sizeof(struct xfs_scrub_vec); +} + /* * ioctl limits */ @@ -839,6 +875,7 @@ struct xfs_scrub_metadata { #define XFS_IOC_FREE_EOFBLOCKS _IOR ('X', 58, struct xfs_fs_eofblocks) /* XFS_IOC_GETFSMAP ------ hoisted 59 */ #define XFS_IOC_SCRUB_METADATA _IOWR('X', 60, struct xfs_scrub_metadata) +#define XFS_IOC_SCRUBV_METADATA _IOWR('X', 60, struct xfs_scrub_vec_head) #define XFS_IOC_AG_GEOMETRY _IOWR('X', 61, struct xfs_ag_geometry) #define XFS_IOC_RTGROUP_GEOMETRY _IOWR('X', 62, struct xfs_rtgroup_geometry) ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 08/11] xfs_scrub: vectorize scrub calls 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (3 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 02/11] xfs: introduce vectored scrub mode Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 09/11] xfs_scrub: vectorize repair calls Darrick J. Wong ` (5 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Use the new vectorized kernel scrub calls to reduce the overhead of checking metadata. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/phase1.c | 2 scrub/scrub.c | 265 +++++++++++++++++++++++++++++++++++-------------- scrub/scrub.h | 2 scrub/scrub_private.h | 21 ++++ 4 files changed, 216 insertions(+), 74 deletions(-) diff --git a/scrub/phase1.c b/scrub/phase1.c index 7b9caa4258c..85da1b7a7d1 100644 --- a/scrub/phase1.c +++ b/scrub/phase1.c @@ -216,6 +216,8 @@ _("Kernel metadata scrubbing facility is not available.")); return ECANCELED; } + check_scrubv(ctx); + /* * Normally, callers are required to pass -n if the provided path is a * readonly filesystem or the kernel wasn't built with online repair diff --git a/scrub/scrub.c b/scrub/scrub.c index 37cc97cdfda..95c798acd0a 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -22,11 +22,42 @@ #include "descr.h" #include "scrub_private.h" -static int scrub_epilogue(struct scrub_ctx *ctx, struct descr *dsc, - struct scrub_item *sri, struct xfs_scrub_vec *vec); - /* Online scrub and repair wrappers. */ +/* Describe the current state of a vectored scrub. */ +static int +format_scrubv_descr( + struct scrub_ctx *ctx, + char *buf, + size_t buflen, + void *where) +{ + struct scrubv_head *bh = where; + struct xfs_scrub_vec_head *vhead = &bh->head; + struct xfs_scrub_vec *v = bh->head.svh_vecs + bh->i; + const struct xfrog_scrub_descr *sc = &xfrog_scrubbers[v->sv_type]; + + switch (sc->group) { + case XFROG_SCRUB_GROUP_AGHEADER: + case XFROG_SCRUB_GROUP_PERAG: + return snprintf(buf, buflen, _("AG %u %s"), vhead->svh_agno, + _(sc->descr)); + case XFROG_SCRUB_GROUP_INODE: + return scrub_render_ino_descr(ctx, buf, buflen, + vhead->svh_ino, vhead->svh_gen, "%s", + _(sc->descr)); + case XFROG_SCRUB_GROUP_METAFILES: + case XFROG_SCRUB_GROUP_SUMMARY: + case XFROG_SCRUB_GROUP_ISCAN: + case XFROG_SCRUB_GROUP_NONE: + return snprintf(buf, buflen, _("%s"), _(sc->descr)); + case XFROG_SCRUB_GROUP_RTGROUP: + return snprintf(buf, buflen, _("rtgroup %u %s"), + vhead->svh_agno, _(sc->descr)); + } + return -1; +} + /* Format a scrub description. */ int format_scrub_descr( @@ -83,52 +114,6 @@ scrub_warn_incomplete_scrub( _("Cross-referencing failed.")); } -/* Do a read-only check of some metadata. */ -static int -xfs_check_metadata( - struct scrub_ctx *ctx, - struct xfs_fd *xfdp, - unsigned int scrub_type, - struct scrub_item *sri) -{ - DEFINE_DESCR(dsc, ctx, format_scrub_descr); - struct xfs_scrub_metadata meta = { }; - struct xfs_scrub_vec vec; - enum xfrog_scrub_group group; - - background_sleep(); - - group = xfrog_scrubbers[scrub_type].group; - meta.sm_type = scrub_type; - switch (group) { - case XFROG_SCRUB_GROUP_AGHEADER: - case XFROG_SCRUB_GROUP_PERAG: - case XFROG_SCRUB_GROUP_RTGROUP: - meta.sm_agno = sri->sri_agno; - break; - case XFROG_SCRUB_GROUP_METAFILES: - case XFROG_SCRUB_GROUP_SUMMARY: - case XFROG_SCRUB_GROUP_ISCAN: - case XFROG_SCRUB_GROUP_NONE: - break; - case XFROG_SCRUB_GROUP_INODE: - meta.sm_ino = sri->sri_ino; - meta.sm_gen = sri->sri_gen; - break; - } - - assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); - assert(scrub_type < XFS_SCRUB_TYPE_NR); - descr_set(&dsc, &meta); - - dbg_printf("check %s flags %xh\n", descr_render(&dsc), meta.sm_flags); - - vec.sv_ret = xfrog_scrub_metadata(xfdp, &meta); - vec.sv_type = scrub_type; - vec.sv_flags = meta.sm_flags; - return scrub_epilogue(ctx, &dsc, sri, &vec); -} - /* * Update all internal state after a scrub ioctl call. * Returns 0 for success, or ECANCELED to abort the program. @@ -260,6 +245,88 @@ _("Optimization is possible.")); return 0; } +/* Fill out the scrub vector header. */ +void +scrub_item_to_vhead( + struct scrubv_head *bighead, + const struct scrub_item *sri) +{ + struct xfs_scrub_vec_head *vhead = &bighead->head; + + if (bg_mode > 1) + vhead->svh_rest_us = bg_mode - 1; + if (sri->sri_agno != -1) + vhead->svh_agno = sri->sri_agno; + if (sri->sri_ino != -1ULL) { + vhead->svh_ino = sri->sri_ino; + vhead->svh_gen = sri->sri_gen; + } +} + +/* Add a scrubber to the scrub vector. */ +void +scrub_vhead_add( + struct scrubv_head *bighead, + const struct scrub_item *sri, + unsigned int scrub_type) +{ + struct xfs_scrub_vec_head *vhead = &bighead->head; + struct xfs_scrub_vec *v; + + v = &vhead->svh_vecs[vhead->svh_nr++]; + v->sv_type = scrub_type; + bighead->i = v - vhead->svh_vecs; +} + +/* Do a read-only check of some metadata. */ +static int +scrub_call_kernel( + struct scrub_ctx *ctx, + struct xfs_fd *xfdp, + struct scrub_item *sri) +{ + DEFINE_DESCR(dsc, ctx, format_scrubv_descr); + struct scrubv_head bh = { }; + struct xfs_scrub_vec *v; + unsigned int scrub_type; + int error; + + assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); + + scrub_item_to_vhead(&bh, sri); + descr_set(&dsc, &bh); + + foreach_scrub_type(scrub_type) { + if (!(sri->sri_state[scrub_type] & SCRUB_ITEM_NEEDSCHECK)) + continue; + scrub_vhead_add(&bh, sri, scrub_type); + + dbg_printf("check %s flags %xh tries %u\n", descr_render(&dsc), + sri->sri_state[scrub_type], + sri->sri_tries[scrub_type]); + } + + error = -xfrog_scrubv_metadata(xfdp, &bh.head); + if (error) + return error; + + foreach_bighead_vec(&bh, v) { + error = scrub_epilogue(ctx, &dsc, sri, v); + if (error) + return error; + + /* + * Progress is counted by the inode for inode metadata; for + * everything else, it's counted for each scrub call. + */ + if (!(sri->sri_state[v->sv_type] & SCRUB_ITEM_NEEDSCHECK) && + sri->sri_ino == -1ULL) + progress_add(1); + } + + return 0; +} + /* Bulk-notify user about things that could be optimized. */ void scrub_report_preen_triggers( @@ -295,6 +362,37 @@ scrub_item_schedule_group( } } +/* Decide if we call the kernel again to finish scrub/repair activity. */ +static inline bool +scrub_item_call_kernel_again_future( + struct scrub_item *sri, + uint8_t work_mask, + const struct scrub_item *old) +{ + unsigned int scrub_type; + unsigned int nr = 0; + + /* If there's nothing to do, we're done. */ + foreach_scrub_type(scrub_type) { + if (sri->sri_state[scrub_type] & work_mask) + nr++; + } + if (!nr) + return false; + + foreach_scrub_type(scrub_type) { + uint8_t statex = sri->sri_state[scrub_type] ^ + old->sri_state[scrub_type]; + + if (statex & work_mask) + return true; + if (sri->sri_tries[scrub_type] != old->sri_tries[scrub_type]) + return true; + } + + return false; +} + /* Decide if we call the kernel again to finish scrub/repair activity. */ bool scrub_item_call_kernel_again( @@ -323,6 +421,29 @@ scrub_item_call_kernel_again( return false; } +/* + * For each scrub item whose state matches the state_flags, set up the item + * state for a kernel call. Returns true if any work was scheduled. + */ +bool +scrub_item_schedule_work( + struct scrub_item *sri, + uint8_t state_flags) +{ + unsigned int scrub_type; + unsigned int nr = 0; + + foreach_scrub_type(scrub_type) { + if (!(sri->sri_state[scrub_type] & state_flags)) + continue; + + sri->sri_tries[scrub_type] = SCRUB_ITEM_MAX_RETRIES; + nr++; + } + + return nr > 0; +} + /* Run all the incomplete scans on this scrub principal. */ int scrub_item_check_file( @@ -333,8 +454,10 @@ scrub_item_check_file( struct xfs_fd xfd; struct scrub_item old_sri; struct xfs_fd *xfdp = &ctx->mnt; - unsigned int scrub_type; - int error; + int error = 0; + + if (!scrub_item_schedule_work(sri, SCRUB_ITEM_NEEDSCHECK)) + return 0; /* * If the caller passed us a file descriptor for a scrub, use it @@ -347,31 +470,15 @@ scrub_item_check_file( xfdp = &xfd; } - foreach_scrub_type(scrub_type) { - if (!(sri->sri_state[scrub_type] & SCRUB_ITEM_NEEDSCHECK)) - continue; - - sri->sri_tries[scrub_type] = SCRUB_ITEM_MAX_RETRIES; - do { - memcpy(&old_sri, sri, sizeof(old_sri)); - error = xfs_check_metadata(ctx, xfdp, scrub_type, sri); - if (error) - return error; - } while (scrub_item_call_kernel_again(sri, scrub_type, - SCRUB_ITEM_NEEDSCHECK, &old_sri)); - - /* - * Progress is counted by the inode for inode metadata; for - * everything else, it's counted for each scrub call. - */ - if (sri->sri_ino == -1ULL) - progress_add(1); - + do { + memcpy(&old_sri, sri, sizeof(old_sri)); + error = scrub_call_kernel(ctx, xfdp, sri); if (error) - break; - } + return error; + } while (scrub_item_call_kernel_again_future(sri, SCRUB_ITEM_NEEDSCHECK, + &old_sri)); - return error; + return 0; } /* How many items do we have to check? */ @@ -566,3 +673,13 @@ can_force_rebuild( return __scrub_test(ctx, XFS_SCRUB_TYPE_PROBE, XFS_SCRUB_IFLAG_REPAIR | XFS_SCRUB_IFLAG_FORCE_REBUILD); } + +void +check_scrubv( + struct scrub_ctx *ctx) +{ + struct xfs_scrub_vec_head head = { }; + + /* We set the fallback flag if this doesn't work. */ + xfrog_scrubv_metadata(&ctx->mnt, &head); +} diff --git a/scrub/scrub.h b/scrub/scrub.h index b7e6173f8fa..0db94da5281 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -147,6 +147,8 @@ bool can_scrub_parent(struct scrub_ctx *ctx); bool can_repair(struct scrub_ctx *ctx); bool can_force_rebuild(struct scrub_ctx *ctx); +void check_scrubv(struct scrub_ctx *ctx); + int scrub_file(struct scrub_ctx *ctx, int fd, const struct xfs_bulkstat *bstat, unsigned int type, struct scrub_item *sri); diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index 383bc17a567..1059c197fa2 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -8,6 +8,26 @@ /* Shared code between scrub.c and repair.c. */ +/* + * Declare a structure big enough to handle all scrub types + barriers, and + * an iteration pointer. So far we only need two barriers. + */ +struct scrubv_head { + struct xfs_scrub_vec_head head; + struct xfs_scrub_vec __vecs[XFS_SCRUB_TYPE_NR + 2]; + unsigned int i; +}; + +#define foreach_bighead_vec(bh, v) \ + for ((bh)->i = 0, (v) = (bh)->head.svh_vecs; \ + (bh)->i < (bh)->head.svh_nr; \ + (bh)->i++, (v)++) + +void scrub_item_to_vhead(struct scrubv_head *bighead, + const struct scrub_item *sri); +void scrub_vhead_add(struct scrubv_head *bighead, const struct scrub_item *sri, + unsigned int scrub_type); + int format_scrub_descr(struct scrub_ctx *ctx, char *buf, size_t buflen, void *where); @@ -104,5 +124,6 @@ scrub_item_schedule_retry(struct scrub_item *sri, unsigned int scrub_type) bool scrub_item_call_kernel_again(struct scrub_item *sri, unsigned int scrub_type, uint8_t work_mask, const struct scrub_item *old); +bool scrub_item_schedule_work(struct scrub_item *sri, uint8_t state_flags); #endif /* XFS_SCRUB_SCRUB_PRIVATE_H_ */ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 09/11] xfs_scrub: vectorize repair calls 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (4 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 08/11] xfs_scrub: vectorize scrub calls Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 07/11] xfs_scrub: convert scrub and repair epilogues to use xfs_scrub_vec Darrick J. Wong ` (4 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Use the new vectorized scrub kernel calls to reduce the overhead of performing repairs. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/repair.c | 268 +++++++++++++++++++++++++++---------------------- scrub/scrub.c | 82 +++------------ scrub/scrub_private.h | 7 + 3 files changed, 166 insertions(+), 191 deletions(-) diff --git a/scrub/repair.c b/scrub/repair.c index 7c4fc6143f0..8a6263c675e 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -20,11 +20,6 @@ #include "descr.h" #include "scrub_private.h" -static int repair_epilogue(struct scrub_ctx *ctx, struct descr *dsc, - struct scrub_item *sri, unsigned int repair_flags, - const struct xfs_scrub_vec *oldm, - const struct xfs_scrub_vec *meta); - /* General repair routines. */ /* @@ -91,65 +86,14 @@ repair_want_service_downgrade( return false; } -/* Repair some metadata. */ -static int -xfs_repair_metadata( - struct scrub_ctx *ctx, - struct xfs_fd *xfdp, - unsigned int scrub_type, - struct scrub_item *sri, - unsigned int repair_flags) +static inline void +restore_oldvec( + struct xfs_scrub_vec *oldvec, + const struct scrub_item *sri, + unsigned int scrub_type) { - struct xfs_scrub_metadata meta = { 0 }; - struct xfs_scrub_vec oldm, vec; - DEFINE_DESCR(dsc, ctx, format_scrub_descr); - bool repair_only; - - /* - * If the caller boosted the priority of this scrub type on behalf of a - * higher level repair by setting IFLAG_REPAIR, turn off REPAIR_ONLY. - */ - repair_only = (repair_flags & XRM_REPAIR_ONLY) && - scrub_item_type_boosted(sri, scrub_type); - - assert(scrub_type < XFS_SCRUB_TYPE_NR); - assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); - meta.sm_type = scrub_type; - meta.sm_flags = XFS_SCRUB_IFLAG_REPAIR; - if (use_force_rebuild) - meta.sm_flags |= XFS_SCRUB_IFLAG_FORCE_REBUILD; - switch (xfrog_scrubbers[scrub_type].group) { - case XFROG_SCRUB_GROUP_AGHEADER: - case XFROG_SCRUB_GROUP_PERAG: - case XFROG_SCRUB_GROUP_RTGROUP: - meta.sm_agno = sri->sri_agno; - break; - case XFROG_SCRUB_GROUP_INODE: - meta.sm_ino = sri->sri_ino; - meta.sm_gen = sri->sri_gen; - break; - default: - break; - } - - vec.sv_type = scrub_type; - vec.sv_flags = sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY; - memcpy(&oldm, &vec, sizeof(struct xfs_scrub_vec)); - if (!is_corrupt(&vec) && repair_only) - return 0; - - descr_set(&dsc, &meta); - - if (needs_repair(&vec)) - str_info(ctx, descr_render(&dsc), _("Attempting repair.")); - else if (debug || verbose) - str_info(ctx, descr_render(&dsc), - _("Attempting optimization.")); - - vec.sv_ret = xfrog_scrub_metadata(xfdp, &meta); - vec.sv_flags = meta.sm_flags; - - return repair_epilogue(ctx, &dsc, sri, repair_flags, &oldm, &vec); + oldvec->sv_type = scrub_type; + oldvec->sv_flags = sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY; } static int @@ -158,12 +102,15 @@ repair_epilogue( struct descr *dsc, struct scrub_item *sri, unsigned int repair_flags, - const struct xfs_scrub_vec *oldm, const struct xfs_scrub_vec *meta) { + struct xfs_scrub_vec oldv; + struct xfs_scrub_vec *oldm = &oldv; unsigned int scrub_type = meta->sv_type; int error = -meta->sv_ret; + restore_oldvec(oldm, sri, meta->sv_type); + switch (error) { case 0: /* No operational errors encountered. */ @@ -297,6 +244,132 @@ _("Read-only filesystem; cannot make changes.")); return 0; } +/* Decide if the dependent scrub types of the given scrub type are ok. */ +static bool +repair_item_dependencies_ok( + const struct scrub_item *sri, + unsigned int scrub_type) +{ + unsigned int dep_mask = repair_deps[scrub_type]; + unsigned int b; + + for (b = 0; dep_mask && b < XFS_SCRUB_TYPE_NR; b++, dep_mask >>= 1) { + if (!(dep_mask & 1)) + continue; + /* + * If this lower level object also needs repair, we can't fix + * the higher level item. + */ + if (sri->sri_state[b] & SCRUB_ITEM_NEEDSREPAIR) + return false; + } + + return true; +} + +/* Decide if we want to repair a particular type of metadata. */ +static bool +can_repair_now( + const struct scrub_item *sri, + unsigned int scrub_type, + __u32 repair_mask, + unsigned int repair_flags) +{ + struct xfs_scrub_vec oldvec; + bool repair_only; + + /* Do we even need to repair this thing? */ + if (!(sri->sri_state[scrub_type] & repair_mask)) + return false; + + restore_oldvec(&oldvec, sri, scrub_type); + + /* + * If the caller boosted the priority of this scrub type on behalf of a + * higher level repair by setting IFLAG_REPAIR, ignore REPAIR_ONLY. + */ + repair_only = (repair_flags & XRM_REPAIR_ONLY) && + !(sri->sri_state[scrub_type] & SCRUB_ITEM_BOOST_REPAIR); + if (!is_corrupt(&oldvec) && repair_only) + return false; + + /* + * Don't try to repair higher level items if their lower-level + * dependencies haven't been verified, unless this is our last chance + * to fix things without complaint. + */ + if (!(repair_flags & XRM_FINAL_WARNING) && + !repair_item_dependencies_ok(sri, scrub_type)) + return false; + + return true; +} + +/* + * Repair some metadata. + * + * Returns 0 for success (or repair item deferral), or ECANCELED to abort the + * program. + */ +static int +repair_call_kernel( + struct scrub_ctx *ctx, + struct xfs_fd *xfdp, + struct scrub_item *sri, + __u32 repair_mask, + unsigned int repair_flags) +{ + DEFINE_DESCR(dsc, ctx, format_scrubv_descr); + struct scrubv_head bh = { }; + struct xfs_scrub_vec *v; + unsigned int scrub_type; + int error; + + assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); + + scrub_item_to_vhead(&bh, sri); + descr_set(&dsc, &bh); + + foreach_scrub_type(scrub_type) { + if (scrub_excessive_errors(ctx)) + return ECANCELED; + + if (!can_repair_now(sri, scrub_type, repair_mask, + repair_flags)) + continue; + + scrub_vhead_add(&bh, sri, scrub_type, true); + + if (sri->sri_state[scrub_type] & SCRUB_ITEM_NEEDSREPAIR) + str_info(ctx, descr_render(&dsc), + _("Attempting repair.")); + else if (debug || verbose) + str_info(ctx, descr_render(&dsc), + _("Attempting optimization.")); + + dbg_printf("repair %s flags %xh tries %u\n", descr_render(&dsc), + sri->sri_state[scrub_type], + sri->sri_tries[scrub_type]); + } + + error = -xfrog_scrubv_metadata(xfdp, &bh.head); + if (error) + return error; + + foreach_bighead_vec(&bh, v) { + error = repair_epilogue(ctx, &dsc, sri, repair_flags, v); + if (error) + return error; + + /* Maybe update progress if we fixed the problem. */ + if (!(repair_flags & XRM_NOPROGRESS) && + !(sri->sri_state[v->sv_type] & SCRUB_ITEM_REPAIR_ANY)) + progress_add(1); + } + + return 0; +} + /* * Prioritize action items in order of how long we can wait. * @@ -636,29 +709,6 @@ action_list_process( return ret; } -/* Decide if the dependent scrub types of the given scrub type are ok. */ -static bool -repair_item_dependencies_ok( - const struct scrub_item *sri, - unsigned int scrub_type) -{ - unsigned int dep_mask = repair_deps[scrub_type]; - unsigned int b; - - for (b = 0; dep_mask && b < XFS_SCRUB_TYPE_NR; b++, dep_mask >>= 1) { - if (!(dep_mask & 1)) - continue; - /* - * If this lower level object also needs repair, we can't fix - * the higher level item. - */ - if (sri->sri_state[b] & SCRUB_ITEM_NEEDSREPAIR) - return false; - } - - return true; -} - /* * For a given filesystem object, perform all repairs of a given class * (corrupt, xcorrupt, xfail, preen) if the repair item says it's needed. @@ -674,13 +724,14 @@ repair_item_class( struct xfs_fd xfd; struct scrub_item old_sri; struct xfs_fd *xfdp = &ctx->mnt; - unsigned int scrub_type; int error = 0; if (ctx->mode == SCRUB_MODE_DRY_RUN) return 0; if (ctx->mode == SCRUB_MODE_PREEN && !(repair_mask & SCRUB_ITEM_PREEN)) return 0; + if (!scrub_item_schedule_work(sri, repair_mask)) + return 0; /* * If the caller passed us a file descriptor for a scrub, use it @@ -693,39 +744,14 @@ repair_item_class( xfdp = &xfd; } - foreach_scrub_type(scrub_type) { - if (scrub_excessive_errors(ctx)) - return ECANCELED; - - if (!(sri->sri_state[scrub_type] & repair_mask)) - continue; - - /* - * Don't try to repair higher level items if their lower-level - * dependencies haven't been verified, unless this is our last - * chance to fix things without complaint. - */ - if (!(flags & XRM_FINAL_WARNING) && - !repair_item_dependencies_ok(sri, scrub_type)) - continue; - - sri->sri_tries[scrub_type] = SCRUB_ITEM_MAX_RETRIES; - do { - memcpy(&old_sri, sri, sizeof(old_sri)); - error = xfs_repair_metadata(ctx, xfdp, scrub_type, sri, - flags); - if (error) - return error; - } while (scrub_item_call_kernel_again(sri, scrub_type, - repair_mask, &old_sri)); - - /* Maybe update progress if we fixed the problem. */ - if (!(flags & XRM_NOPROGRESS) && - !(sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY)) - progress_add(1); - } - - return error; + do { + memcpy(&old_sri, sri, sizeof(struct scrub_item)); + error = repair_call_kernel(ctx, xfdp, sri, repair_mask, flags); + if (error) + return error; + } while (scrub_item_call_kernel_again(sri, repair_mask, &old_sri)); + + return 0; } /* diff --git a/scrub/scrub.c b/scrub/scrub.c index 95c798acd0a..76d4fa87931 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -25,7 +25,7 @@ /* Online scrub and repair wrappers. */ /* Describe the current state of a vectored scrub. */ -static int +int format_scrubv_descr( struct scrub_ctx *ctx, char *buf, @@ -58,38 +58,6 @@ format_scrubv_descr( return -1; } -/* Format a scrub description. */ -int -format_scrub_descr( - struct scrub_ctx *ctx, - char *buf, - size_t buflen, - void *where) -{ - struct xfs_scrub_metadata *meta = where; - const struct xfrog_scrub_descr *sc = &xfrog_scrubbers[meta->sm_type]; - - switch (sc->group) { - case XFROG_SCRUB_GROUP_AGHEADER: - case XFROG_SCRUB_GROUP_PERAG: - return snprintf(buf, buflen, _("AG %u %s"), meta->sm_agno, - _(sc->descr)); - case XFROG_SCRUB_GROUP_INODE: - return scrub_render_ino_descr(ctx, buf, buflen, - meta->sm_ino, meta->sm_gen, "%s", - _(sc->descr)); - case XFROG_SCRUB_GROUP_METAFILES: - case XFROG_SCRUB_GROUP_SUMMARY: - case XFROG_SCRUB_GROUP_ISCAN: - case XFROG_SCRUB_GROUP_NONE: - return snprintf(buf, buflen, _("%s"), _(sc->descr)); - case XFROG_SCRUB_GROUP_RTGROUP: - return snprintf(buf, buflen, _("rtgroup %u %s"), meta->sm_agno, - _(sc->descr)); - } - return -1; -} - /* Warn about strange circumstances after scrub. */ void scrub_warn_incomplete_scrub( @@ -268,13 +236,18 @@ void scrub_vhead_add( struct scrubv_head *bighead, const struct scrub_item *sri, - unsigned int scrub_type) + unsigned int scrub_type, + bool repair) { struct xfs_scrub_vec_head *vhead = &bighead->head; struct xfs_scrub_vec *v; v = &vhead->svh_vecs[vhead->svh_nr++]; v->sv_type = scrub_type; + if (repair) + v->sv_flags |= XFS_SCRUB_IFLAG_REPAIR; + if (repair && use_force_rebuild) + v->sv_flags |= XFS_SCRUB_IFLAG_FORCE_REBUILD; bighead->i = v - vhead->svh_vecs; } @@ -299,7 +272,7 @@ scrub_call_kernel( foreach_scrub_type(scrub_type) { if (!(sri->sri_state[scrub_type] & SCRUB_ITEM_NEEDSCHECK)) continue; - scrub_vhead_add(&bh, sri, scrub_type); + scrub_vhead_add(&bh, sri, scrub_type, false); dbg_printf("check %s flags %xh tries %u\n", descr_render(&dsc), sri->sri_state[scrub_type], @@ -363,8 +336,8 @@ scrub_item_schedule_group( } /* Decide if we call the kernel again to finish scrub/repair activity. */ -static inline bool -scrub_item_call_kernel_again_future( +bool +scrub_item_call_kernel_again( struct scrub_item *sri, uint8_t work_mask, const struct scrub_item *old) @@ -380,6 +353,11 @@ scrub_item_call_kernel_again_future( if (!nr) return false; + /* + * We are willing to go again if the last call had any effect on the + * state of the scrub item that the caller cares about or if the kernel + * asked us to try again. + */ foreach_scrub_type(scrub_type) { uint8_t statex = sri->sri_state[scrub_type] ^ old->sri_state[scrub_type]; @@ -393,34 +371,6 @@ scrub_item_call_kernel_again_future( return false; } -/* Decide if we call the kernel again to finish scrub/repair activity. */ -bool -scrub_item_call_kernel_again( - struct scrub_item *sri, - unsigned int scrub_type, - uint8_t work_mask, - const struct scrub_item *old) -{ - uint8_t statex; - - /* If there's nothing to do, we're done. */ - if (!(sri->sri_state[scrub_type] & work_mask)) - return false; - - /* - * We are willing to go again if the last call had any effect on the - * state of the scrub item that the caller cares about, if the freeze - * flag got set, or if the kernel asked us to try again... - */ - statex = sri->sri_state[scrub_type] ^ old->sri_state[scrub_type]; - if (statex & work_mask) - return true; - if (sri->sri_tries[scrub_type] != old->sri_tries[scrub_type]) - return true; - - return false; -} - /* * For each scrub item whose state matches the state_flags, set up the item * state for a kernel call. Returns true if any work was scheduled. @@ -475,7 +425,7 @@ scrub_item_check_file( error = scrub_call_kernel(ctx, xfdp, sri); if (error) return error; - } while (scrub_item_call_kernel_again_future(sri, SCRUB_ITEM_NEEDSCHECK, + } while (scrub_item_call_kernel_again(sri, SCRUB_ITEM_NEEDSCHECK, &old_sri)); return 0; diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index 1059c197fa2..8daf28c26ee 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -26,9 +26,9 @@ struct scrubv_head { void scrub_item_to_vhead(struct scrubv_head *bighead, const struct scrub_item *sri); void scrub_vhead_add(struct scrubv_head *bighead, const struct scrub_item *sri, - unsigned int scrub_type); + unsigned int scrub_type, bool repair); -int format_scrub_descr(struct scrub_ctx *ctx, char *buf, size_t buflen, +int format_scrubv_descr(struct scrub_ctx *ctx, char *buf, size_t buflen, void *where); /* Predicates for scrub flag state. */ @@ -121,8 +121,7 @@ scrub_item_schedule_retry(struct scrub_item *sri, unsigned int scrub_type) return true; } -bool scrub_item_call_kernel_again(struct scrub_item *sri, - unsigned int scrub_type, uint8_t work_mask, +bool scrub_item_call_kernel_again(struct scrub_item *sri, uint8_t work_mask, const struct scrub_item *old); bool scrub_item_schedule_work(struct scrub_item *sri, uint8_t state_flags); ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 07/11] xfs_scrub: convert scrub and repair epilogues to use xfs_scrub_vec 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (5 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 09/11] xfs_scrub: vectorize repair calls Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 11/11] xfs_scrub: try spot repairs of metadata items to make scrub progress Darrick J. Wong ` (3 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Convert the scrub and repair epilogue code to pass around xfs_scrub_vecs as we prepare for vectorized operation. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/repair.c | 35 ++++++++++++++++++----------------- scrub/scrub.c | 27 ++++++++++++++------------- scrub/scrub_private.h | 34 +++++++++++++++++----------------- 3 files changed, 49 insertions(+), 47 deletions(-) diff --git a/scrub/repair.c b/scrub/repair.c index 4d8552cf9d0..7c4fc6143f0 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -22,8 +22,8 @@ static int repair_epilogue(struct scrub_ctx *ctx, struct descr *dsc, struct scrub_item *sri, unsigned int repair_flags, - struct xfs_scrub_metadata *oldm, - struct xfs_scrub_metadata *meta, int error); + const struct xfs_scrub_vec *oldm, + const struct xfs_scrub_vec *meta); /* General repair routines. */ @@ -101,10 +101,9 @@ xfs_repair_metadata( unsigned int repair_flags) { struct xfs_scrub_metadata meta = { 0 }; - struct xfs_scrub_metadata oldm; + struct xfs_scrub_vec oldm, vec; DEFINE_DESCR(dsc, ctx, format_scrub_descr); bool repair_only; - int error; /* * If the caller boosted the priority of this scrub type on behalf of a @@ -133,22 +132,24 @@ xfs_repair_metadata( break; } - if (!is_corrupt(&meta) && repair_only) + vec.sv_type = scrub_type; + vec.sv_flags = sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY; + memcpy(&oldm, &vec, sizeof(struct xfs_scrub_vec)); + if (!is_corrupt(&vec) && repair_only) return 0; - memcpy(&oldm, &meta, sizeof(oldm)); - oldm.sm_flags = sri->sri_state[scrub_type] & SCRUB_ITEM_REPAIR_ANY; - descr_set(&dsc, &oldm); + descr_set(&dsc, &meta); - if (needs_repair(&oldm)) + if (needs_repair(&vec)) str_info(ctx, descr_render(&dsc), _("Attempting repair.")); else if (debug || verbose) str_info(ctx, descr_render(&dsc), _("Attempting optimization.")); - error = -xfrog_scrub_metadata(xfdp, &meta); - return repair_epilogue(ctx, &dsc, sri, repair_flags, &oldm, &meta, - error); + vec.sv_ret = xfrog_scrub_metadata(xfdp, &meta); + vec.sv_flags = meta.sm_flags; + + return repair_epilogue(ctx, &dsc, sri, repair_flags, &oldm, &vec); } static int @@ -157,11 +158,11 @@ repair_epilogue( struct descr *dsc, struct scrub_item *sri, unsigned int repair_flags, - struct xfs_scrub_metadata *oldm, - struct xfs_scrub_metadata *meta, - int error) + const struct xfs_scrub_vec *oldm, + const struct xfs_scrub_vec *meta) { - unsigned int scrub_type = meta->sm_type; + unsigned int scrub_type = meta->sv_type; + int error = -meta->sv_ret; switch (error) { case 0: @@ -278,7 +279,7 @@ _("Read-only filesystem; cannot make changes.")); _("Seems correct but cross-referencing failed; will keep checking.")); return 0; } - } else if (meta->sm_flags & XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) { + } else if (meta->sv_flags & XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) { if (verbose) str_info(ctx, descr_render(dsc), _("No modification needed.")); diff --git a/scrub/scrub.c b/scrub/scrub.c index b102a457cc2..37cc97cdfda 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -23,8 +23,7 @@ #include "scrub_private.h" static int scrub_epilogue(struct scrub_ctx *ctx, struct descr *dsc, - struct scrub_item *sri, struct xfs_scrub_metadata *meta, - int error); + struct scrub_item *sri, struct xfs_scrub_vec *vec); /* Online scrub and repair wrappers. */ @@ -65,7 +64,7 @@ void scrub_warn_incomplete_scrub( struct scrub_ctx *ctx, struct descr *dsc, - struct xfs_scrub_metadata *meta) + const struct xfs_scrub_vec *meta) { if (is_incomplete(meta)) str_info(ctx, descr_render(dsc), _("Check incomplete.")); @@ -94,8 +93,8 @@ xfs_check_metadata( { DEFINE_DESCR(dsc, ctx, format_scrub_descr); struct xfs_scrub_metadata meta = { }; + struct xfs_scrub_vec vec; enum xfrog_scrub_group group; - int error; background_sleep(); @@ -124,8 +123,10 @@ xfs_check_metadata( dbg_printf("check %s flags %xh\n", descr_render(&dsc), meta.sm_flags); - error = -xfrog_scrub_metadata(xfdp, &meta); - return scrub_epilogue(ctx, &dsc, sri, &meta, error); + vec.sv_ret = xfrog_scrub_metadata(xfdp, &meta); + vec.sv_type = scrub_type; + vec.sv_flags = meta.sm_flags; + return scrub_epilogue(ctx, &dsc, sri, &vec); } /* @@ -137,11 +138,11 @@ scrub_epilogue( struct scrub_ctx *ctx, struct descr *dsc, struct scrub_item *sri, - struct xfs_scrub_metadata *meta, - int error) + struct xfs_scrub_vec *meta) { - unsigned int scrub_type = meta->sm_type; + unsigned int scrub_type = meta->sv_type; enum xfrog_scrub_group group; + int error = -meta->sv_ret; group = xfrog_scrubbers[scrub_type].group; @@ -150,7 +151,7 @@ scrub_epilogue( /* No operational errors encountered. */ if (!sri->sri_revalidate && debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) - meta->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + meta->sv_flags |= XFS_SCRUB_OFLAG_CORRUPT; break; case ENOENT: /* Metadata not present, just skip it. */ @@ -211,7 +212,7 @@ _("Repairs are required.")); } /* Schedule repairs. */ - scrub_item_save_state(sri, scrub_type, meta->sm_flags); + scrub_item_save_state(sri, scrub_type, meta->sv_flags); return 0; } @@ -238,7 +239,7 @@ _("Optimization is possible.")); } /* Schedule optimizations. */ - scrub_item_save_state(sri, scrub_type, meta->sm_flags); + scrub_item_save_state(sri, scrub_type, meta->sv_flags); return 0; } @@ -250,7 +251,7 @@ _("Optimization is possible.")); * deem it completely consistent at some point. */ if (xref_failed(meta) && ctx->mode == SCRUB_MODE_REPAIR) { - scrub_item_save_state(sri, scrub_type, meta->sm_flags); + scrub_item_save_state(sri, scrub_type, meta->sv_flags); return 0; } diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index c60ea555885..383bc17a567 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -13,40 +13,40 @@ int format_scrub_descr(struct scrub_ctx *ctx, char *buf, size_t buflen, /* Predicates for scrub flag state. */ -static inline bool is_corrupt(struct xfs_scrub_metadata *sm) +static inline bool is_corrupt(const struct xfs_scrub_vec *sv) { - return sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT; + return sv->sv_flags & XFS_SCRUB_OFLAG_CORRUPT; } -static inline bool is_unoptimized(struct xfs_scrub_metadata *sm) +static inline bool is_unoptimized(const struct xfs_scrub_vec *sv) { - return sm->sm_flags & XFS_SCRUB_OFLAG_PREEN; + return sv->sv_flags & XFS_SCRUB_OFLAG_PREEN; } -static inline bool xref_failed(struct xfs_scrub_metadata *sm) +static inline bool xref_failed(const struct xfs_scrub_vec *sv) { - return sm->sm_flags & XFS_SCRUB_OFLAG_XFAIL; + return sv->sv_flags & XFS_SCRUB_OFLAG_XFAIL; } -static inline bool xref_disagrees(struct xfs_scrub_metadata *sm) +static inline bool xref_disagrees(const struct xfs_scrub_vec *sv) { - return sm->sm_flags & XFS_SCRUB_OFLAG_XCORRUPT; + return sv->sv_flags & XFS_SCRUB_OFLAG_XCORRUPT; } -static inline bool is_incomplete(struct xfs_scrub_metadata *sm) +static inline bool is_incomplete(const struct xfs_scrub_vec *sv) { - return sm->sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE; + return sv->sv_flags & XFS_SCRUB_OFLAG_INCOMPLETE; } -static inline bool is_suspicious(struct xfs_scrub_metadata *sm) +static inline bool is_suspicious(const struct xfs_scrub_vec *sv) { - return sm->sm_flags & XFS_SCRUB_OFLAG_WARNING; + return sv->sv_flags & XFS_SCRUB_OFLAG_WARNING; } /* Should we fix it? */ -static inline bool needs_repair(struct xfs_scrub_metadata *sm) +static inline bool needs_repair(const struct xfs_scrub_vec *sv) { - return is_corrupt(sm) || xref_disagrees(sm); + return is_corrupt(sv) || xref_disagrees(sv); } /* @@ -54,13 +54,13 @@ static inline bool needs_repair(struct xfs_scrub_metadata *sm) * scan/repair; or if there were cross-referencing problems but the object was * not obviously corrupt. */ -static inline bool want_retry(struct xfs_scrub_metadata *sm) +static inline bool want_retry(const struct xfs_scrub_vec *sv) { - return is_incomplete(sm) || (xref_disagrees(sm) && !is_corrupt(sm)); + return is_incomplete(sv) || (xref_disagrees(sv) && !is_corrupt(sv)); } void scrub_warn_incomplete_scrub(struct scrub_ctx *ctx, struct descr *dsc, - struct xfs_scrub_metadata *meta); + const struct xfs_scrub_vec *meta); /* Scrub item functions */ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 11/11] xfs_scrub: try spot repairs of metadata items to make scrub progress 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (6 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 07/11] xfs_scrub: convert scrub and repair epilogues to use xfs_scrub_vec Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 06/11] xfs_scrub: split the repair epilogue code into a separate function Darrick J. Wong ` (2 subsequent siblings) 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Now that we've enabled scrub dependency barriers, it's possible that a scrub_item_check call will return with some of the scrub items still in NEEDSCHECK state. If, for example, scrub type B depends on scrub type A being clean and A is not clean, B will still be in NEEDSCHECK state. In order to make as much scanning progress as possible during phase 2 and phase 3, allow ourselves to try some spot repairs in the hopes that it will enable us to make progress towards at least scanning the whole metadata item. If we can't make any forward progress, we'll queue the scrub item for repair in phase 4, which means that anything still in in NEEDSCHECK state becomes CORRUPT state. (At worst, the NEEDSCHECK item will actually be clean by phase 4, and xfs_scrub will report that it didn't need any work after all.) Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/phase2.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ scrub/phase3.c | 71 +++++++++++++++++++++++++++++++++++++++++++- scrub/repair.c | 15 +++++++++ 3 files changed, 176 insertions(+), 1 deletion(-) diff --git a/scrub/phase2.c b/scrub/phase2.c index e4c7d32d75e..e2d46cba640 100644 --- a/scrub/phase2.c +++ b/scrub/phase2.c @@ -77,6 +77,53 @@ defer_fs_repair( return 0; } +/* + * If we couldn't check all the scheduled metadata items, try performing spot + * repairs until we check everything or stop making forward progress. + */ +static int +repair_and_scrub_loop( + struct scrub_ctx *ctx, + struct scrub_item *sri, + const char *descr, + bool *defer) +{ + unsigned int to_check; + int ret; + + *defer = false; + if (ctx->mode != SCRUB_MODE_REPAIR) + return 0; + + to_check = scrub_item_count_needscheck(sri); + while (to_check > 0) { + unsigned int nr; + + ret = repair_item_corruption(ctx, sri); + if (ret) + return ret; + + ret = scrub_item_check(ctx, sri); + if (ret) + return ret; + + nr = scrub_item_count_needscheck(sri); + if (nr == to_check) { + /* + * We cannot make forward scanning progress with this + * metadata, so defer the rest until phase 4. + */ + str_info(ctx, descr, + _("Unable to make forward checking progress; will try again in phase 4.")); + *defer = true; + return 0; + } + to_check = nr; + } + + return 0; +} + /* Scrub each AG's metadata btrees. */ static void scan_ag_metadata( @@ -90,6 +137,7 @@ scan_ag_metadata( struct scan_ctl *sctl = arg; char descr[DESCR_BUFSZ]; unsigned int difficulty; + bool defer_repairs; int ret; if (sctl->aborted) @@ -105,10 +153,22 @@ scan_ag_metadata( scrub_item_schedule_group(&sri, XFROG_SCRUB_GROUP_AGHEADER); scrub_item_schedule_group(&sri, XFROG_SCRUB_GROUP_PERAG); + /* + * Try to check all of the AG metadata items that we just scheduled. + * If we return with some types still needing a check, try repairing + * any damaged metadata that we've found so far, and try again. Abort + * if we stop making forward progress. + */ ret = scrub_item_check(ctx, &sri); if (ret) goto err; + ret = repair_and_scrub_loop(ctx, &sri, descr, &defer_repairs); + if (ret) + goto err; + if (defer_repairs) + goto defer; + /* * Figure out if we need to perform early fixing. The only * reason we need to do this is if the inobt is broken, which @@ -125,6 +185,7 @@ scan_ag_metadata( if (ret) goto err; +defer: /* Everything else gets fixed during phase 4. */ ret = defer_fs_repair(ctx, &sri); if (ret) @@ -145,11 +206,18 @@ scan_metafile( struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; unsigned int difficulty; + bool defer_repairs; int ret; if (sctl->aborted) goto out; + /* + * Try to check all of the metadata files that we just scheduled. If + * we return with some types still needing a check, try repairing any + * damaged metadata that we've found so far, and try again. Abort if + * we stop making forward progress. + */ scrub_item_init_fs(&sri); scrub_item_schedule(&sri, type); ret = scrub_item_check(ctx, &sri); @@ -158,10 +226,20 @@ scan_metafile( goto out; } + ret = repair_and_scrub_loop(ctx, &sri, xfrog_scrubbers[type].descr, + &defer_repairs); + if (ret) { + sctl->aborted = true; + goto out; + } + if (defer_repairs) + goto defer; + /* Complain about metadata corruptions that might not be fixable. */ difficulty = repair_item_difficulty(&sri); warn_repair_difficulties(ctx, difficulty, xfrog_scrubbers[type].descr); +defer: ret = defer_fs_repair(ctx, &sri); if (ret) { sctl->aborted = true; @@ -188,6 +266,7 @@ scan_rtgroup_metadata( struct scrub_ctx *ctx = (struct scrub_ctx *)wq->wq_ctx; struct scan_ctl *sctl = arg; char descr[DESCR_BUFSZ]; + bool defer_repairs; int ret; if (sctl->aborted) @@ -196,6 +275,12 @@ scan_rtgroup_metadata( scrub_item_init_rtgroup(&sri, rgno); snprintf(descr, DESCR_BUFSZ, _("rtgroup %u"), rgno); + /* + * Try to check all of the rtgroup metadata items that we just + * scheduled. If we return with some types still needing a check, try + * repairing any damaged metadata that we've found so far, and try + * again. Abort if we stop making forward progress. + */ scrub_item_schedule_group(&sri, XFROG_SCRUB_GROUP_RTGROUP); ret = scrub_item_check(ctx, &sri); if (ret) { @@ -203,6 +288,12 @@ scan_rtgroup_metadata( goto out; } + ret = repair_and_scrub_loop(ctx, &sri, descr, &defer_repairs); + if (ret) { + sctl->aborted = true; + goto out; + } + /* Everything else gets fixed during phase 4. */ ret = defer_fs_repair(ctx, &sri); if (ret) { diff --git a/scrub/phase3.c b/scrub/phase3.c index 14fff96ff77..43495b3b746 100644 --- a/scrub/phase3.c +++ b/scrub/phase3.c @@ -99,6 +99,58 @@ try_inode_repair( return repair_file_corruption(ictx->ctx, sri, fd); } +/* + * If we couldn't check all the scheduled file metadata items, try performing + * spot repairs until we check everything or stop making forward progress. + */ +static int +repair_and_scrub_inode_loop( + struct scrub_ctx *ctx, + struct xfs_bulkstat *bstat, + int fd, + struct scrub_item *sri, + bool *defer) +{ + unsigned int to_check; + int error; + + *defer = false; + if (ctx->mode != SCRUB_MODE_REPAIR) + return 0; + + to_check = scrub_item_count_needscheck(sri); + while (to_check > 0) { + unsigned int nr; + + error = repair_file_corruption(ctx, sri, fd); + if (error) + return error; + + error = scrub_item_check_file(ctx, sri, fd); + if (error) + return error; + + nr = scrub_item_count_needscheck(sri); + if (nr == to_check) { + char descr[DESCR_BUFSZ]; + + /* + * We cannot make forward scanning progress with this + * inode, so defer the rest until phase 4. + */ + scrub_render_ino_descr(ctx, descr, DESCR_BUFSZ, + bstat->bs_ino, bstat->bs_gen, NULL); + str_info(ctx, descr, + _("Unable to make forward checking progress; will try again in phase 4.")); + *defer = true; + return 0; + } + to_check = nr; + } + + return 0; +} + /* Verify the contents, xattrs, and extent maps of an inode. */ static int scrub_inode( @@ -160,11 +212,28 @@ scrub_inode( scrub_item_schedule(&sri, XFS_SCRUB_TYPE_XATTR); scrub_item_schedule(&sri, XFS_SCRUB_TYPE_PARENT); - /* Try to check and repair the file while it's open. */ + /* + * Try to check all of the metadata items that we just scheduled. If + * we return with some types still needing a check and the space + * metadata isn't also in need of repairs, try repairing any damaged + * file metadata that we've found so far, and try checking the file + * again. Worst case, defer the repairs and the checks to phase 4 if + * we can't make any progress on anything. + */ error = scrub_item_check_file(ctx, &sri, fd); if (error) goto out; + if (!ictx->always_defer_repairs) { + bool defer_repairs; + + error = repair_and_scrub_inode_loop(ctx, bstat, fd, &sri, + &defer_repairs); + if (error || defer_repairs) + goto out; + } + + /* Try to repair the file while it's open. */ error = try_inode_repair(ictx, &sri, fd); if (error) goto out; diff --git a/scrub/repair.c b/scrub/repair.c index 91259feb758..bf843522993 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -849,6 +849,7 @@ repair_item_to_action_item( struct action_item **aitemp) { struct action_item *aitem; + unsigned int scrub_type; if (repair_item_count_needsrepair(sri) == 0) return 0; @@ -864,6 +865,20 @@ repair_item_to_action_item( INIT_LIST_HEAD(&aitem->list); memcpy(&aitem->sri, sri, sizeof(struct scrub_item)); + /* + * If the scrub item indicates that there is unchecked metadata, assume + * that the scrub type checker depends on something that couldn't be + * fixed. Mark that type as corrupt so that phase 4 will try it again. + */ + foreach_scrub_type(scrub_type) { + __u8 *state = aitem->sri.sri_state; + + if (state[scrub_type] & SCRUB_ITEM_NEEDSCHECK) { + state[scrub_type] &= ~SCRUB_ITEM_NEEDSCHECK; + state[scrub_type] |= SCRUB_ITEM_CORRUPT; + } + } + *aitemp = aitem; return 0; } ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 06/11] xfs_scrub: split the repair epilogue code into a separate function 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (7 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 11/11] xfs_scrub: try spot repairs of metadata items to make scrub progress Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 05/11] xfs_scrub: split the scrub " Darrick J. Wong 2022-12-30 22:20 ` [PATCH 10/11] xfs_scrub: use scrub barriers to reduce kernel calls Darrick J. Wong 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Move all the code that updates the internal state in response to a repair ioctl() call completion into a separate function. This will help with vectorizing repair calls later on. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/repair.c | 61 ++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 20 deletions(-) diff --git a/scrub/repair.c b/scrub/repair.c index cd652dc85a1..4d8552cf9d0 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -20,6 +20,11 @@ #include "descr.h" #include "scrub_private.h" +static int repair_epilogue(struct scrub_ctx *ctx, struct descr *dsc, + struct scrub_item *sri, unsigned int repair_flags, + struct xfs_scrub_metadata *oldm, + struct xfs_scrub_metadata *meta, int error); + /* General repair routines. */ /* @@ -142,6 +147,22 @@ xfs_repair_metadata( _("Attempting optimization.")); error = -xfrog_scrub_metadata(xfdp, &meta); + return repair_epilogue(ctx, &dsc, sri, repair_flags, &oldm, &meta, + error); +} + +static int +repair_epilogue( + struct scrub_ctx *ctx, + struct descr *dsc, + struct scrub_item *sri, + unsigned int repair_flags, + struct xfs_scrub_metadata *oldm, + struct xfs_scrub_metadata *meta, + int error) +{ + unsigned int scrub_type = meta->sm_type; + switch (error) { case 0: /* No operational errors encountered. */ @@ -150,12 +171,12 @@ xfs_repair_metadata( case EBUSY: /* Filesystem is busy, try again later. */ if (debug || verbose) - str_info(ctx, descr_render(&dsc), + str_info(ctx, descr_render(dsc), _("Filesystem is busy, deferring repair.")); return 0; case ESHUTDOWN: /* Filesystem is already shut down, abort. */ - str_error(ctx, descr_render(&dsc), + str_error(ctx, descr_render(dsc), _("Filesystem is shut down, aborting.")); return ECANCELED; case ENOTTY: @@ -174,7 +195,7 @@ _("Filesystem is shut down, aborting.")); * If we forced repairs or this is a preen, don't * error out if the kernel doesn't know how to fix. */ - if (is_unoptimized(&oldm) || + if (is_unoptimized(oldm) || debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) { scrub_item_clean_state(sri, scrub_type); return 0; @@ -182,14 +203,14 @@ _("Filesystem is shut down, aborting.")); fallthrough; case EINVAL: /* Kernel doesn't know how to repair this? */ - str_corrupt(ctx, descr_render(&dsc), + str_corrupt(ctx, descr_render(dsc), _("Don't know how to fix; offline repair required.")); scrub_item_clean_state(sri, scrub_type); return 0; case EROFS: /* Read-only filesystem, can't fix. */ - if (verbose || debug || needs_repair(&oldm)) - str_error(ctx, descr_render(&dsc), + if (verbose || debug || needs_repair(oldm)) + str_error(ctx, descr_render(dsc), _("Read-only filesystem; cannot make changes.")); return ECANCELED; case ENOENT: @@ -199,7 +220,7 @@ _("Read-only filesystem; cannot make changes.")); case ENOMEM: case ENOSPC: /* Don't care if preen fails due to low resources. */ - if (is_unoptimized(&oldm) && !needs_repair(&oldm)) { + if (is_unoptimized(oldm) && !needs_repair(oldm)) { scrub_item_clean_state(sri, scrub_type); return 0; } @@ -214,7 +235,7 @@ _("Read-only filesystem; cannot make changes.")); */ if (!(repair_flags & XRM_FINAL_WARNING)) return 0; - str_liberror(ctx, error, descr_render(&dsc)); + str_liberror(ctx, error, descr_render(dsc)); scrub_item_clean_state(sri, scrub_type); return 0; } @@ -225,12 +246,12 @@ _("Read-only filesystem; cannot make changes.")); * the repair again, just in case the fs was busy. Only retry so many * times. */ - if (want_retry(&meta) && scrub_item_schedule_retry(sri, scrub_type)) + if (want_retry(meta) && scrub_item_schedule_retry(sri, scrub_type)) return 0; if (repair_flags & XRM_FINAL_WARNING) - scrub_warn_incomplete_scrub(ctx, &dsc, &meta); - if (needs_repair(&meta) || is_incomplete(&meta)) { + scrub_warn_incomplete_scrub(ctx, dsc, meta); + if (needs_repair(meta) || is_incomplete(meta)) { /* * Still broken; if we've been told not to complain then we * just requeue this and try again later. Otherwise we @@ -238,9 +259,9 @@ _("Read-only filesystem; cannot make changes.")); */ if (!(repair_flags & XRM_FINAL_WARNING)) return 0; - str_corrupt(ctx, descr_render(&dsc), + str_corrupt(ctx, descr_render(dsc), _("Repair unsuccessful; offline repair required.")); - } else if (xref_failed(&meta)) { + } else if (xref_failed(meta)) { /* * This metadata object itself looks ok, but we still noticed * inconsistencies when comparing it with the other filesystem @@ -249,25 +270,25 @@ _("Read-only filesystem; cannot make changes.")); * reverify the cross-referencing as repairs progress. */ if (repair_flags & XRM_FINAL_WARNING) { - str_info(ctx, descr_render(&dsc), + str_info(ctx, descr_render(dsc), _("Seems correct but cross-referencing failed; offline repair recommended.")); } else { if (verbose) - str_info(ctx, descr_render(&dsc), + str_info(ctx, descr_render(dsc), _("Seems correct but cross-referencing failed; will keep checking.")); return 0; } - } else if (meta.sm_flags & XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) { + } else if (meta->sm_flags & XFS_SCRUB_OFLAG_NO_REPAIR_NEEDED) { if (verbose) - str_info(ctx, descr_render(&dsc), + str_info(ctx, descr_render(dsc), _("No modification needed.")); } else { /* Clean operation, no corruption detected. */ - if (needs_repair(&oldm)) - record_repair(ctx, descr_render(&dsc), + if (needs_repair(oldm)) + record_repair(ctx, descr_render(dsc), _("Repairs successful.")); else - record_preen(ctx, descr_render(&dsc), + record_preen(ctx, descr_render(dsc), _("Optimization successful.")); } ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 05/11] xfs_scrub: split the scrub epilogue code into a separate function 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (8 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 06/11] xfs_scrub: split the repair epilogue code into a separate function Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 10/11] xfs_scrub: use scrub barriers to reduce kernel calls Darrick J. Wong 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Move all the code that updates the internal state in response to a scrub ioctl() call completion into a separate function. This will help with vectorizing scrub calls later on. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/scrub.c | 52 ++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/scrub/scrub.c b/scrub/scrub.c index a6d5ec056c8..b102a457cc2 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -22,6 +22,10 @@ #include "descr.h" #include "scrub_private.h" +static int scrub_epilogue(struct scrub_ctx *ctx, struct descr *dsc, + struct scrub_item *sri, struct xfs_scrub_metadata *meta, + int error); + /* Online scrub and repair wrappers. */ /* Format a scrub description. */ @@ -121,12 +125,32 @@ xfs_check_metadata( dbg_printf("check %s flags %xh\n", descr_render(&dsc), meta.sm_flags); error = -xfrog_scrub_metadata(xfdp, &meta); + return scrub_epilogue(ctx, &dsc, sri, &meta, error); +} + +/* + * Update all internal state after a scrub ioctl call. + * Returns 0 for success, or ECANCELED to abort the program. + */ +static int +scrub_epilogue( + struct scrub_ctx *ctx, + struct descr *dsc, + struct scrub_item *sri, + struct xfs_scrub_metadata *meta, + int error) +{ + unsigned int scrub_type = meta->sm_type; + enum xfrog_scrub_group group; + + group = xfrog_scrubbers[scrub_type].group; + switch (error) { case 0: /* No operational errors encountered. */ if (!sri->sri_revalidate && debug_tweak_on("XFS_SCRUB_FORCE_REPAIR")) - meta.sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; + meta->sm_flags |= XFS_SCRUB_OFLAG_CORRUPT; break; case ENOENT: /* Metadata not present, just skip it. */ @@ -134,13 +158,13 @@ xfs_check_metadata( return 0; case ESHUTDOWN: /* FS already crashed, give up. */ - str_error(ctx, descr_render(&dsc), + str_error(ctx, descr_render(dsc), _("Filesystem is shut down, aborting.")); return ECANCELED; case EIO: case ENOMEM: /* Abort on I/O errors or insufficient memory. */ - str_liberror(ctx, error, descr_render(&dsc)); + str_liberror(ctx, error, descr_render(dsc)); return ECANCELED; case EDEADLOCK: case EBUSY: @@ -156,7 +180,7 @@ _("Filesystem is shut down, aborting.")); return 0; default: /* Operational error. Log it and move on. */ - str_liberror(ctx, error, descr_render(&dsc)); + str_liberror(ctx, error, descr_render(dsc)); scrub_item_clean_state(sri, scrub_type); return 0; } @@ -167,27 +191,27 @@ _("Filesystem is shut down, aborting.")); * we'll try the scan again, just in case the fs was busy. * Only retry so many times. */ - if (want_retry(&meta) && scrub_item_schedule_retry(sri, scrub_type)) + if (want_retry(meta) && scrub_item_schedule_retry(sri, scrub_type)) return 0; /* Complain about incomplete or suspicious metadata. */ - scrub_warn_incomplete_scrub(ctx, &dsc, &meta); + scrub_warn_incomplete_scrub(ctx, dsc, meta); /* * If we need repairs or there were discrepancies, schedule a * repair if desired, otherwise complain. */ - if (is_corrupt(&meta) || xref_disagrees(&meta)) { + if (is_corrupt(meta) || xref_disagrees(meta)) { if (ctx->mode != SCRUB_MODE_REPAIR) { /* Dry-run mode, so log an error and forget it. */ - str_corrupt(ctx, descr_render(&dsc), + str_corrupt(ctx, descr_render(dsc), _("Repairs are required.")); scrub_item_clean_state(sri, scrub_type); return 0; } /* Schedule repairs. */ - scrub_item_save_state(sri, scrub_type, meta.sm_flags); + scrub_item_save_state(sri, scrub_type, meta->sm_flags); return 0; } @@ -195,12 +219,12 @@ _("Repairs are required.")); * If we could optimize, schedule a repair if desired, * otherwise complain. */ - if (is_unoptimized(&meta)) { + if (is_unoptimized(meta)) { if (ctx->mode == SCRUB_MODE_DRY_RUN) { /* Dry-run mode, so log an error and forget it. */ if (group != XFROG_SCRUB_GROUP_INODE) { /* AG or FS metadata, always warn. */ - str_info(ctx, descr_render(&dsc), + str_info(ctx, descr_render(dsc), _("Optimization is possible.")); } else if (!ctx->preen_triggers[scrub_type]) { /* File metadata, only warn once per type. */ @@ -214,7 +238,7 @@ _("Optimization is possible.")); } /* Schedule optimizations. */ - scrub_item_save_state(sri, scrub_type, meta.sm_flags); + scrub_item_save_state(sri, scrub_type, meta->sm_flags); return 0; } @@ -225,8 +249,8 @@ _("Optimization is possible.")); * re-examine the object as repairs progress to see if the kernel will * deem it completely consistent at some point. */ - if (xref_failed(&meta) && ctx->mode == SCRUB_MODE_REPAIR) { - scrub_item_save_state(sri, scrub_type, meta.sm_flags); + if (xref_failed(meta) && ctx->mode == SCRUB_MODE_REPAIR) { + scrub_item_save_state(sri, scrub_type, meta->sm_flags); return 0; } ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 10/11] xfs_scrub: use scrub barriers to reduce kernel calls 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (9 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 05/11] xfs_scrub: split the scrub " Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 10 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Use scrub barriers so that we can submit a single scrub request for a bunch of things, and have the kernel stop midway through if it finds anything broken. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- scrub/phase2.c | 15 ++------- scrub/phase3.c | 17 +--------- scrub/repair.c | 32 ++++++++++++++++++- scrub/scrub.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++- scrub/scrub.h | 17 ++++++++++ scrub/scrub_private.h | 4 ++ 6 files changed, 134 insertions(+), 32 deletions(-) diff --git a/scrub/phase2.c b/scrub/phase2.c index a224af11ed4..e4c7d32d75e 100644 --- a/scrub/phase2.c +++ b/scrub/phase2.c @@ -99,21 +99,12 @@ scan_ag_metadata( snprintf(descr, DESCR_BUFSZ, _("AG %u"), agno); /* - * First we scrub and fix the AG headers, because we need - * them to work well enough to check the AG btrees. + * First we scrub and fix the AG headers, because we need them to work + * well enough to check the AG btrees. Then scrub the AG btrees. */ scrub_item_schedule_group(&sri, XFROG_SCRUB_GROUP_AGHEADER); - ret = scrub_item_check(ctx, &sri); - if (ret) - goto err; - - /* Repair header damage. */ - ret = repair_item_corruption(ctx, &sri); - if (ret) - goto err; - - /* Now scrub the AG btrees. */ scrub_item_schedule_group(&sri, XFROG_SCRUB_GROUP_PERAG); + ret = scrub_item_check(ctx, &sri); if (ret) goto err; diff --git a/scrub/phase3.c b/scrub/phase3.c index 56a4385a408..14fff96ff77 100644 --- a/scrub/phase3.c +++ b/scrub/phase3.c @@ -145,25 +145,11 @@ scrub_inode( /* Scrub the inode. */ scrub_item_schedule(&sri, XFS_SCRUB_TYPE_INODE); - error = scrub_item_check_file(ctx, &sri, fd); - if (error) - goto out; - - error = try_inode_repair(ictx, &sri, fd); - if (error) - goto out; /* Scrub all block mappings. */ scrub_item_schedule(&sri, XFS_SCRUB_TYPE_BMBTD); scrub_item_schedule(&sri, XFS_SCRUB_TYPE_BMBTA); scrub_item_schedule(&sri, XFS_SCRUB_TYPE_BMBTC); - error = scrub_item_check_file(ctx, &sri, fd); - if (error) - goto out; - - error = try_inode_repair(ictx, &sri, fd); - if (error) - goto out; /* Check everything accessible via file mapping. */ if (S_ISLNK(bstat->bs_mode)) @@ -173,11 +159,12 @@ scrub_inode( scrub_item_schedule(&sri, XFS_SCRUB_TYPE_XATTR); scrub_item_schedule(&sri, XFS_SCRUB_TYPE_PARENT); + + /* Try to check and repair the file while it's open. */ error = scrub_item_check_file(ctx, &sri, fd); if (error) goto out; - /* Try to repair the file while it's open. */ error = try_inode_repair(ictx, &sri, fd); if (error) goto out; diff --git a/scrub/repair.c b/scrub/repair.c index 8a6263c675e..91259feb758 100644 --- a/scrub/repair.c +++ b/scrub/repair.c @@ -323,6 +323,7 @@ repair_call_kernel( struct scrubv_head bh = { }; struct xfs_scrub_vec *v; unsigned int scrub_type; + bool need_barrier = false; int error; assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); @@ -338,6 +339,11 @@ repair_call_kernel( repair_flags)) continue; + if (need_barrier) { + scrub_vhead_add_barrier(&bh); + need_barrier = false; + } + scrub_vhead_add(&bh, sri, scrub_type, true); if (sri->sri_state[scrub_type] & SCRUB_ITEM_NEEDSREPAIR) @@ -350,6 +356,17 @@ repair_call_kernel( dbg_printf("repair %s flags %xh tries %u\n", descr_render(&dsc), sri->sri_state[scrub_type], sri->sri_tries[scrub_type]); + + /* + * One of the other scrub types depends on this one. Set us up + * to add a repair barrier if we decide to schedule a repair + * after this one. If the UNFIXED flag is set, that means this + * is our last chance to fix things, so we skip the barriers + * just let everything run. + */ + if (!(repair_flags & XRM_FINAL_WARNING) && + (sri->sri_state[scrub_type] & SCRUB_ITEM_BARRIER)) + need_barrier = true; } error = -xfrog_scrubv_metadata(xfdp, &bh.head); @@ -357,6 +374,16 @@ repair_call_kernel( return error; foreach_bighead_vec(&bh, v) { + /* Deal with barriers separately. */ + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) { + /* -ECANCELED means the kernel stopped here. */ + if (v->sv_ret == -ECANCELED) + return 0; + if (v->sv_ret) + return -v->sv_ret; + continue; + } + error = repair_epilogue(ctx, &dsc, sri, repair_flags, v); if (error) return error; @@ -445,7 +472,8 @@ repair_item_boost_priorities( * bits are left untouched to force a rescan in phase 4. */ #define MUSTFIX_STATES (SCRUB_ITEM_CORRUPT | \ - SCRUB_ITEM_BOOST_REPAIR) + SCRUB_ITEM_BOOST_REPAIR | \ + SCRUB_ITEM_BARRIER) /* * Figure out which AG metadata must be fixed before we can move on * to the inode scan. @@ -730,7 +758,7 @@ repair_item_class( return 0; if (ctx->mode == SCRUB_MODE_PREEN && !(repair_mask & SCRUB_ITEM_PREEN)) return 0; - if (!scrub_item_schedule_work(sri, repair_mask)) + if (!scrub_item_schedule_work(sri, repair_mask, repair_deps)) return 0; /* diff --git a/scrub/scrub.c b/scrub/scrub.c index 76d4fa87931..6031e2b1991 100644 --- a/scrub/scrub.c +++ b/scrub/scrub.c @@ -24,6 +24,35 @@ /* Online scrub and repair wrappers. */ +/* + * Bitmap showing the correctness dependencies between scrub types for scrubs. + * Dependencies cannot cross scrub groups. + */ +#define DEP(x) (1U << (x)) +static const unsigned int scrub_deps[XFS_SCRUB_TYPE_NR] = { + [XFS_SCRUB_TYPE_AGF] = DEP(XFS_SCRUB_TYPE_SB), + [XFS_SCRUB_TYPE_AGFL] = DEP(XFS_SCRUB_TYPE_SB) | + DEP(XFS_SCRUB_TYPE_AGF), + [XFS_SCRUB_TYPE_AGI] = DEP(XFS_SCRUB_TYPE_SB), + [XFS_SCRUB_TYPE_BNOBT] = DEP(XFS_SCRUB_TYPE_AGF), + [XFS_SCRUB_TYPE_CNTBT] = DEP(XFS_SCRUB_TYPE_AGF), + [XFS_SCRUB_TYPE_INOBT] = DEP(XFS_SCRUB_TYPE_AGI), + [XFS_SCRUB_TYPE_FINOBT] = DEP(XFS_SCRUB_TYPE_AGI), + [XFS_SCRUB_TYPE_RMAPBT] = DEP(XFS_SCRUB_TYPE_AGF), + [XFS_SCRUB_TYPE_REFCNTBT] = DEP(XFS_SCRUB_TYPE_AGF), + [XFS_SCRUB_TYPE_BMBTD] = DEP(XFS_SCRUB_TYPE_INODE), + [XFS_SCRUB_TYPE_BMBTA] = DEP(XFS_SCRUB_TYPE_INODE), + [XFS_SCRUB_TYPE_BMBTC] = DEP(XFS_SCRUB_TYPE_INODE), + [XFS_SCRUB_TYPE_DIR] = DEP(XFS_SCRUB_TYPE_BMBTD), + [XFS_SCRUB_TYPE_XATTR] = DEP(XFS_SCRUB_TYPE_BMBTA), + [XFS_SCRUB_TYPE_SYMLINK] = DEP(XFS_SCRUB_TYPE_BMBTD), + [XFS_SCRUB_TYPE_PARENT] = DEP(XFS_SCRUB_TYPE_BMBTD), + [XFS_SCRUB_TYPE_QUOTACHECK] = DEP(XFS_SCRUB_TYPE_UQUOTA) | + DEP(XFS_SCRUB_TYPE_GQUOTA) | + DEP(XFS_SCRUB_TYPE_PQUOTA), +}; +#undef DEP + /* Describe the current state of a vectored scrub. */ int format_scrubv_descr( @@ -251,6 +280,21 @@ scrub_vhead_add( bighead->i = v - vhead->svh_vecs; } +/* Add a barrier to the scrub vector. */ +void +scrub_vhead_add_barrier( + struct scrubv_head *bighead) +{ + struct xfs_scrub_vec_head *vhead = &bighead->head; + struct xfs_scrub_vec *v; + + v = &vhead->svh_vecs[vhead->svh_nr++]; + v->sv_type = XFS_SCRUB_TYPE_BARRIER; + v->sv_flags = XFS_SCRUB_OFLAG_CORRUPT | XFS_SCRUB_OFLAG_XFAIL | + XFS_SCRUB_OFLAG_XCORRUPT | XFS_SCRUB_OFLAG_INCOMPLETE; + bighead->i = v - vhead->svh_vecs; +} + /* Do a read-only check of some metadata. */ static int scrub_call_kernel( @@ -262,6 +306,7 @@ scrub_call_kernel( struct scrubv_head bh = { }; struct xfs_scrub_vec *v; unsigned int scrub_type; + bool need_barrier = false; int error; assert(!debug_tweak_on("XFS_SCRUB_NO_KERNEL")); @@ -272,8 +317,17 @@ scrub_call_kernel( foreach_scrub_type(scrub_type) { if (!(sri->sri_state[scrub_type] & SCRUB_ITEM_NEEDSCHECK)) continue; + + if (need_barrier) { + scrub_vhead_add_barrier(&bh); + need_barrier = false; + } + scrub_vhead_add(&bh, sri, scrub_type, false); + if (sri->sri_state[scrub_type] & SCRUB_ITEM_BARRIER) + need_barrier = true; + dbg_printf("check %s flags %xh tries %u\n", descr_render(&dsc), sri->sri_state[scrub_type], sri->sri_tries[scrub_type]); @@ -284,6 +338,16 @@ scrub_call_kernel( return error; foreach_bighead_vec(&bh, v) { + /* Deal with barriers separately. */ + if (v->sv_type == XFS_SCRUB_TYPE_BARRIER) { + /* -ECANCELED means the kernel stopped here. */ + if (v->sv_ret == -ECANCELED) + return 0; + if (v->sv_ret) + return -v->sv_ret; + continue; + } + error = scrub_epilogue(ctx, &dsc, sri, v); if (error) return error; @@ -378,15 +442,25 @@ scrub_item_call_kernel_again( bool scrub_item_schedule_work( struct scrub_item *sri, - uint8_t state_flags) + uint8_t state_flags, + const unsigned int *schedule_deps) { unsigned int scrub_type; unsigned int nr = 0; foreach_scrub_type(scrub_type) { + unsigned int j; + + sri->sri_state[scrub_type] &= ~SCRUB_ITEM_BARRIER; + if (!(sri->sri_state[scrub_type] & state_flags)) continue; + foreach_scrub_type(j) { + if (schedule_deps[scrub_type] & (1U << j)) + sri->sri_state[j] |= SCRUB_ITEM_BARRIER; + } + sri->sri_tries[scrub_type] = SCRUB_ITEM_MAX_RETRIES; nr++; } @@ -406,7 +480,7 @@ scrub_item_check_file( struct xfs_fd *xfdp = &ctx->mnt; int error = 0; - if (!scrub_item_schedule_work(sri, SCRUB_ITEM_NEEDSCHECK)) + if (!scrub_item_schedule_work(sri, SCRUB_ITEM_NEEDSCHECK, scrub_deps)) return 0; /* @@ -630,6 +704,9 @@ check_scrubv( { struct xfs_scrub_vec_head head = { }; + if (debug_tweak_on("XFS_SCRUB_FORCE_SINGLE")) + ctx->mnt.flags |= XFROG_FLAG_SCRUB_FORCE_SINGLE; + /* We set the fallback flag if this doesn't work. */ xfrog_scrubv_metadata(&ctx->mnt, &head); } diff --git a/scrub/scrub.h b/scrub/scrub.h index 0db94da5281..69a37ad7bfa 100644 --- a/scrub/scrub.h +++ b/scrub/scrub.h @@ -30,6 +30,9 @@ enum xfrog_scrub_group; /* This scrub type needs to be checked. */ #define SCRUB_ITEM_NEEDSCHECK (1 << 5) +/* Scrub barrier. */ +#define SCRUB_ITEM_BARRIER (1 << 6) + /* All of the state flags that we need to prioritize repair work. */ #define SCRUB_ITEM_REPAIR_ANY (SCRUB_ITEM_CORRUPT | \ SCRUB_ITEM_PREEN | \ @@ -135,6 +138,20 @@ scrub_item_check(struct scrub_ctx *ctx, struct scrub_item *sri) return scrub_item_check_file(ctx, sri, -1); } +/* Count the number of metadata objects still needing a scrub. */ +static inline unsigned int +scrub_item_count_needscheck( + const struct scrub_item *sri) +{ + unsigned int ret = 0; + unsigned int i; + + foreach_scrub_type(i) + if (sri->sri_state[i] & SCRUB_ITEM_NEEDSCHECK) + ret++; + return ret; +} + void scrub_report_preen_triggers(struct scrub_ctx *ctx); bool can_scrub_fs_metadata(struct scrub_ctx *ctx); diff --git a/scrub/scrub_private.h b/scrub/scrub_private.h index 8daf28c26ee..b21a6ca62ba 100644 --- a/scrub/scrub_private.h +++ b/scrub/scrub_private.h @@ -27,6 +27,7 @@ void scrub_item_to_vhead(struct scrubv_head *bighead, const struct scrub_item *sri); void scrub_vhead_add(struct scrubv_head *bighead, const struct scrub_item *sri, unsigned int scrub_type, bool repair); +void scrub_vhead_add_barrier(struct scrubv_head *bighead); int format_scrubv_descr(struct scrub_ctx *ctx, char *buf, size_t buflen, void *where); @@ -123,6 +124,7 @@ scrub_item_schedule_retry(struct scrub_item *sri, unsigned int scrub_type) bool scrub_item_call_kernel_again(struct scrub_item *sri, uint8_t work_mask, const struct scrub_item *old); -bool scrub_item_schedule_work(struct scrub_item *sri, uint8_t state_flags); +bool scrub_item_schedule_work(struct scrub_item *sri, uint8_t state_flags, + const unsigned int *schedule_deps); #endif /* XFS_SCRUB_SCRUB_PRIVATE_H_ */ ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/1] libxfs: report refcount information to userspace 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (4 preceding siblings ...) 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 1/1] xfs_io: dump reference count information Darrick J. Wong 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong ` (3 subsequent siblings) 9 siblings, 1 reply; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs Hi all, Create a new ioctl to report the number of owners of each disk block so that reflink-aware defraggers can make better decisions about which extents to target. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=report-refcounts xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=report-refcounts fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=report-refcounts --- configure.ac | 1 include/builddefs.in | 4 io/Makefile | 5 + io/fsrefcounts.c | 478 +++++++++++++++++++++++++++++++++++++++++++++++++ io/init.c | 1 io/io.h | 1 libfrog/fsrefcounts.h | 100 ++++++++++ m4/package_libcdev.m4 | 19 ++ man/man8/xfs_io.8 | 86 +++++++++ 9 files changed, 695 insertions(+) create mode 100644 io/fsrefcounts.c create mode 100644 libfrog/fsrefcounts.h ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 1/1] xfs_io: dump reference count information 2022-12-30 22:20 ` [PATCHSET 0/1] libxfs: report refcount information to userspace Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 0 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Dump refcount info from the kernel so we can prototype a sharing-aware defrag/fs rearranging tool. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- configure.ac | 1 include/builddefs.in | 4 io/Makefile | 5 + io/fsrefcounts.c | 478 +++++++++++++++++++++++++++++++++++++++++++++++++ io/init.c | 1 io/io.h | 1 libfrog/fsrefcounts.h | 100 ++++++++++ m4/package_libcdev.m4 | 19 ++ man/man8/xfs_io.8 | 86 +++++++++ 9 files changed, 695 insertions(+) create mode 100644 io/fsrefcounts.c create mode 100644 libfrog/fsrefcounts.h diff --git a/configure.ac b/configure.ac index f4f1563da8b..18e783a9180 100644 --- a/configure.ac +++ b/configure.ac @@ -188,6 +188,7 @@ AC_HAVE_MREMAP AC_NEED_INTERNAL_FSXATTR AC_NEED_INTERNAL_FSCRYPT_ADD_KEY_ARG AC_HAVE_GETFSMAP +AC_HAVE_GETFSREFCOUNTS AC_HAVE_STATFS_FLAGS AC_HAVE_MAP_SYNC AC_HAVE_DEVMAPPER diff --git a/include/builddefs.in b/include/builddefs.in index 50ebb9f75d8..bf7d340ceb7 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -114,6 +114,7 @@ HAVE_MREMAP = @have_mremap@ NEED_INTERNAL_FSXATTR = @need_internal_fsxattr@ NEED_INTERNAL_FSCRYPT_ADD_KEY_ARG = @need_internal_fscrypt_add_key_arg@ HAVE_GETFSMAP = @have_getfsmap@ +HAVE_GETFSREFCOUNTS = @have_getfsrefcounts@ HAVE_STATFS_FLAGS = @have_statfs_flags@ HAVE_MAP_SYNC = @have_map_sync@ HAVE_DEVMAPPER = @have_devmapper@ @@ -165,6 +166,9 @@ endif ifeq ($(HAVE_GETFSMAP),yes) PCFLAGS+= -DHAVE_GETFSMAP endif +ifeq ($(HAVE_GETFSREFCOUNTS),yes) +PCFLAGS+= -DHAVE_GETFSREFCOUNTS +endif ifeq ($(HAVE_FALLOCATE),yes) PCFLAGS += -DHAVE_FALLOCATE endif diff --git a/io/Makefile b/io/Makefile index 2b7748bfc13..c9e224c415a 100644 --- a/io/Makefile +++ b/io/Makefile @@ -116,6 +116,11 @@ ifeq ($(HAVE_FIEXCHANGE),yes) LCFLAGS += -DHAVE_FIEXCHANGE endif +# On linux we get fsrefcounts from the system or define it ourselves +# so include this unconditionally. If this reverts to only +# the autoconf check w/o local definition, test HAVE_GETFSREFCOUNTS +CFILES += fsrefcounts.c + default: depend $(LTCOMMAND) include $(BUILDRULES) diff --git a/io/fsrefcounts.c b/io/fsrefcounts.c new file mode 100644 index 00000000000..930f8639fdc --- /dev/null +++ b/io/fsrefcounts.c @@ -0,0 +1,478 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#include "platform_defs.h" +#include "command.h" +#include "init.h" +#include "libfrog/paths.h" +#include "io.h" +#include "input.h" +#include "libfrog/fsgeom.h" +#include "libfrog/fsrefcounts.h" + +static cmdinfo_t fsrefcounts_cmd; +static dev_t xfs_data_dev; + +static void +fsrefcounts_help(void) +{ + printf(_( +"\n" +" Prints extent owner counts for the filesystem hosting the current file" +"\n" +" fsrefcounts prints the number of owners of disk blocks used by the whole\n" +" filesystem. When possible, owner and offset information will be included\n" +" in the space report.\n" +"\n" +" By default, each line of the listing takes the following form:\n" +" extent: major:minor [startblock..endblock]: owner startoffset..endoffset length\n" +" All the file offsets and disk blocks are in units of 512-byte blocks.\n" +" -d -- query only the data device (default).\n" +" -l -- query only the log device.\n" +" -r -- query only the realtime device.\n" +" -n -- query n extents at a time.\n" +" -o -- only print extents with at least this many owners (default 1).\n" +" -O -- only print extents with no more than this many owners (default 2^64-1).\n" +" -m -- output machine-readable format.\n" +" -v -- Verbose information, show AG and offsets. Show flags legend on 2nd -v\n" +"\n" +"The optional start and end arguments require one of -d, -l, or -r to be set.\n" +"\n")); +} + +static void +dump_refcounts( + unsigned long long *nr, + const unsigned long long min_owners, + const unsigned long long max_owners, + struct fsrefs_head *head) +{ + unsigned long long i; + struct fsrefs *p; + + for (i = 0, p = head->fch_recs; i < head->fch_entries; i++, p++) { + if (p->fcr_owners < min_owners || p->fcr_owners > max_owners) + continue; + printf("\t%llu: %u:%u [%lld..%lld]: ", i + (*nr), + major(p->fcr_device), minor(p->fcr_device), + (long long)BTOBBT(p->fcr_physical), + (long long)BTOBBT(p->fcr_physical + p->fcr_length - 1)); + printf(_("%llu %lld\n"), + (unsigned long long)p->fcr_owners, + (long long)BTOBBT(p->fcr_length)); + } + + (*nr) += head->fch_entries; +} + +static void +dump_refcounts_machine( + unsigned long long *nr, + const unsigned long long min_owners, + const unsigned long long max_owners, + struct fsrefs_head *head) +{ + unsigned long long i; + struct fsrefs *p; + + if (*nr == 0) + printf(_("EXT,MAJOR,MINOR,PSTART,PEND,OWNERS,LENGTH\n")); + for (i = 0, p = head->fch_recs; i < head->fch_entries; i++, p++) { + if (p->fcr_owners < min_owners || p->fcr_owners > max_owners) + continue; + printf("%llu,%u,%u,%lld,%lld,", i + (*nr), + major(p->fcr_device), minor(p->fcr_device), + (long long)BTOBBT(p->fcr_physical), + (long long)BTOBBT(p->fcr_physical + p->fcr_length - 1)); + printf("%llu,%lld\n", + (unsigned long long)p->fcr_owners, + (long long)BTOBBT(p->fcr_length)); + } + + (*nr) += head->fch_entries; +} + +/* + * Verbose mode displays: + * extent: major:minor [startblock..endblock]: owners \ + * ag# (agoffset..agendoffset) totalbbs flags + */ +#define MINRANGE_WIDTH 16 +#define MINAG_WIDTH 2 +#define MINTOT_WIDTH 5 +#define NFLG 4 /* count of flags */ +#define FLG_NULL 00000 /* Null flag */ +#define FLG_BSU 01000 /* Not on begin of stripe unit */ +#define FLG_ESU 00100 /* Not on end of stripe unit */ +#define FLG_BSW 00010 /* Not on begin of stripe width */ +#define FLG_ESW 00001 /* Not on end of stripe width */ +static void +dump_refcounts_verbose( + unsigned long long *nr, + const unsigned long long min_owners, + const unsigned long long max_owners, + struct fsrefs_head *head, + bool *dumped_flags, + struct xfs_fsop_geom *fsgeo) +{ + unsigned long long i; + struct fsrefs *p; + int agno; + off64_t agoff, bperag; + int boff_w, aoff_w, tot_w, agno_w, own_w; + int nr_w, dev_w; + char bbuf[40], abuf[40], obuf[40]; + char nbuf[40], dbuf[40], gbuf[40]; + int sunit, swidth; + int flg = 0; + + boff_w = aoff_w = own_w = MINRANGE_WIDTH; + dev_w = 3; + nr_w = 4; + tot_w = MINTOT_WIDTH; + bperag = (off64_t)fsgeo->agblocks * + (off64_t)fsgeo->blocksize; + sunit = (fsgeo->sunit * fsgeo->blocksize); + swidth = (fsgeo->swidth * fsgeo->blocksize); + + /* + * Go through the extents and figure out the width + * needed for all columns. + */ + for (i = 0, p = head->fch_recs; i < head->fch_entries; i++, p++) { + if (p->fcr_owners < min_owners || p->fcr_owners > max_owners) + continue; + if (sunit && + (p->fcr_physical % sunit != 0 || + ((p->fcr_physical + p->fcr_length) % sunit) != 0 || + p->fcr_physical % swidth != 0 || + ((p->fcr_physical + p->fcr_length) % swidth) != 0)) + flg = 1; + if (flg) + *dumped_flags = true; + snprintf(nbuf, sizeof(nbuf), "%llu", (*nr) + i); + nr_w = max(nr_w, strlen(nbuf)); + if (head->fch_oflags & FCH_OF_DEV_T) + snprintf(dbuf, sizeof(dbuf), "%u:%u", + major(p->fcr_device), + minor(p->fcr_device)); + else + snprintf(dbuf, sizeof(dbuf), "0x%x", p->fcr_device); + dev_w = max(dev_w, strlen(dbuf)); + snprintf(bbuf, sizeof(bbuf), "[%lld..%lld]:", + (long long)BTOBBT(p->fcr_physical), + (long long)BTOBBT(p->fcr_physical + p->fcr_length - 1)); + boff_w = max(boff_w, strlen(bbuf)); + snprintf(obuf, sizeof(obuf), "%llu", + (unsigned long long)p->fcr_owners); + own_w = max(own_w, strlen(obuf)); + if (p->fcr_device == xfs_data_dev) { + agno = p->fcr_physical / bperag; + agoff = p->fcr_physical - (agno * bperag); + snprintf(abuf, sizeof(abuf), + "(%lld..%lld)", + (long long)BTOBBT(agoff), + (long long)BTOBBT(agoff + p->fcr_length - 1)); + } else + abuf[0] = 0; + aoff_w = max(aoff_w, strlen(abuf)); + tot_w = max(tot_w, + numlen(BTOBBT(p->fcr_length), 10)); + } + agno_w = max(MINAG_WIDTH, numlen(fsgeo->agcount, 10)); + if (*nr == 0) + printf("%*s: %-*s %-*s %-*s %*s %-*s %*s%s\n", + nr_w, _("EXT"), + dev_w, _("DEV"), + boff_w, _("BLOCK-RANGE"), + own_w, _("OWNERS"), + agno_w, _("AG"), + aoff_w, _("AG-OFFSET"), + tot_w, _("TOTAL"), + flg ? _(" FLAGS") : ""); + for (i = 0, p = head->fch_recs; i < head->fch_entries; i++, p++) { + if (p->fcr_owners < min_owners || p->fcr_owners > max_owners) + continue; + flg = FLG_NULL; + /* + * If striping enabled, determine if extent starts/ends + * on a stripe unit boundary. + */ + if (sunit) { + if (p->fcr_physical % sunit != 0) + flg |= FLG_BSU; + if (((p->fcr_physical + + p->fcr_length ) % sunit ) != 0) + flg |= FLG_ESU; + if (p->fcr_physical % swidth != 0) + flg |= FLG_BSW; + if (((p->fcr_physical + + p->fcr_length ) % swidth ) != 0) + flg |= FLG_ESW; + } + if (head->fch_oflags & FCH_OF_DEV_T) + snprintf(dbuf, sizeof(dbuf), "%u:%u", + major(p->fcr_device), + minor(p->fcr_device)); + else + snprintf(dbuf, sizeof(dbuf), "0x%x", p->fcr_device); + snprintf(bbuf, sizeof(bbuf), "[%lld..%lld]:", + (long long)BTOBBT(p->fcr_physical), + (long long)BTOBBT(p->fcr_physical + p->fcr_length - 1)); + snprintf(obuf, sizeof(obuf), "%llu", + (unsigned long long)p->fcr_owners); + if (p->fcr_device == xfs_data_dev) { + agno = p->fcr_physical / bperag; + agoff = p->fcr_physical - (agno * bperag); + snprintf(abuf, sizeof(abuf), + "(%lld..%lld)", + (long long)BTOBBT(agoff), + (long long)BTOBBT(agoff + p->fcr_length - 1)); + snprintf(gbuf, sizeof(gbuf), + "%lld", + (long long)agno); + } else { + abuf[0] = 0; + gbuf[0] = 0; + } + printf("%*llu: %-*s %-*s %-*s %-*s %-*s %*lld", + nr_w, (*nr) + i, dev_w, dbuf, boff_w, bbuf, own_w, + obuf, agno_w, gbuf, aoff_w, abuf, tot_w, + (long long)BTOBBT(p->fcr_length)); + if (flg == FLG_NULL) + printf("\n"); + else + printf(" %-*.*o\n", NFLG, NFLG, flg); + } + + (*nr) += head->fch_entries; +} + +static void +dump_verbose_key(void) +{ + printf(_(" FLAG Values:\n")); + printf(_(" %*.*o Doesn't begin on stripe unit\n"), + NFLG+1, NFLG+1, FLG_BSU); + printf(_(" %*.*o Doesn't end on stripe unit\n"), + NFLG+1, NFLG+1, FLG_ESU); + printf(_(" %*.*o Doesn't begin on stripe width\n"), + NFLG+1, NFLG+1, FLG_BSW); + printf(_(" %*.*o Doesn't end on stripe width\n"), + NFLG+1, NFLG+1, FLG_ESW); +} + +static int +fsrefcounts_f( + int argc, + char **argv) +{ + struct fsrefs *p; + struct fsrefs_head *head; + struct fsrefs *l, *h; + struct xfs_fsop_geom fsgeo; + long long start = 0; + long long end = -1; + unsigned long long min_owners = 1; + unsigned long long max_owners = ULLONG_MAX; + int map_size; + int nflag = 0; + int vflag = 0; + int mflag = 0; + int i = 0; + int c; + unsigned long long nr = 0; + size_t fsblocksize, fssectsize; + struct fs_path *fs; + static bool tab_init; + bool dumped_flags = false; + int dflag, lflag, rflag; + + init_cvtnum(&fsblocksize, &fssectsize); + + dflag = lflag = rflag = 0; + while ((c = getopt(argc, argv, "dlmn:o:O:rv")) != EOF) { + switch (c) { + case 'd': /* data device */ + dflag = 1; + break; + case 'l': /* log device */ + lflag = 1; + break; + case 'm': /* machine readable format */ + mflag++; + break; + case 'n': /* number of extents specified */ + nflag = cvt_u32(optarg, 10); + if (errno) + return command_usage(&fsrefcounts_cmd); + break; + case 'o': /* minimum owners */ + min_owners = cvt_u64(optarg, 10); + if (errno) + return command_usage(&fsrefcounts_cmd); + if (min_owners < 1) { + fprintf(stderr, + _("min_owners must be greater than zero.\n")); + exitcode = 1; + return 0; + } + break; + case 'O': /* maximum owners */ + max_owners = cvt_u64(optarg, 10); + if (errno) + return command_usage(&fsrefcounts_cmd); + if (max_owners < 1) { + fprintf(stderr, + _("max_owners must be greater than zero.\n")); + exitcode = 1; + return 0; + } + break; + case 'r': /* rt device */ + rflag = 1; + break; + case 'v': /* Verbose output */ + vflag++; + break; + default: + exitcode = 1; + return command_usage(&fsrefcounts_cmd); + } + } + + if ((dflag + lflag + rflag > 1) || (mflag > 0 && vflag > 0) || + (argc > optind && dflag + lflag + rflag == 0)) { + exitcode = 1; + return command_usage(&fsrefcounts_cmd); + } + + if (argc > optind) { + start = cvtnum(fsblocksize, fssectsize, argv[optind]); + if (start < 0) { + fprintf(stderr, + _("Bad refcount start_bblock %s.\n"), + argv[optind]); + exitcode = 1; + return 0; + } + start <<= BBSHIFT; + } + + if (argc > optind + 1) { + end = cvtnum(fsblocksize, fssectsize, argv[optind + 1]); + if (end < 0) { + fprintf(stderr, + _("Bad refcount end_bblock %s.\n"), + argv[optind + 1]); + exitcode = 1; + return 0; + } + end <<= BBSHIFT; + } + + if (vflag) { + c = -xfrog_geometry(file->fd, &fsgeo); + if (c) { + fprintf(stderr, + _("%s: can't get geometry [\"%s\"]: %s\n"), + progname, file->name, strerror(c)); + exitcode = 1; + return 0; + } + } + + map_size = nflag ? nflag : 131072 / sizeof(struct fsrefs); + head = malloc(fsrefs_sizeof(map_size)); + if (head == NULL) { + fprintf(stderr, _("%s: malloc of %llu bytes failed.\n"), + progname, + (unsigned long long)fsrefs_sizeof(map_size)); + exitcode = 1; + return 0; + } + + memset(head, 0, sizeof(*head)); + l = head->fch_keys; + h = head->fch_keys + 1; + if (dflag) { + l->fcr_device = h->fcr_device = file->fs_path.fs_datadev; + } else if (lflag) { + l->fcr_device = h->fcr_device = file->fs_path.fs_logdev; + } else if (rflag) { + l->fcr_device = h->fcr_device = file->fs_path.fs_rtdev; + } else { + l->fcr_device = 0; + h->fcr_device = UINT_MAX; + } + l->fcr_physical = start; + h->fcr_physical = end; + h->fcr_owners = ULLONG_MAX; + h->fcr_flags = UINT_MAX; + + /* + * If this is an XFS filesystem, remember the data device. + * (We report AG number/block for data device extents on XFS). + */ + if (!tab_init) { + fs_table_initialise(0, NULL, 0, NULL); + tab_init = true; + } + fs = fs_table_lookup(file->name, FS_MOUNT_POINT); + xfs_data_dev = fs ? fs->fs_datadev : 0; + + head->fch_count = map_size; + do { + /* Get some extents */ + i = ioctl(file->fd, FS_IOC_GETFSREFCOUNTS, head); + if (i < 0) { + fprintf(stderr, _("%s: xfsctl(XFS_IOC_GETFSREFCOUNTS)" + " iflags=0x%x [\"%s\"]: %s\n"), + progname, head->fch_iflags, file->name, + strerror(errno)); + free(head); + exitcode = 1; + return 0; + } + + if (head->fch_entries == 0) + break; + + if (vflag) + dump_refcounts_verbose(&nr, min_owners, max_owners, + head, &dumped_flags, &fsgeo); + else if (mflag) + dump_refcounts_machine(&nr, min_owners, max_owners, + head); + else + dump_refcounts(&nr, min_owners, max_owners, head); + + p = &head->fch_recs[head->fch_entries - 1]; + if (p->fcr_flags & FCR_OF_LAST) + break; + fsrefs_advance(head); + } while (true); + + if (dumped_flags) + dump_verbose_key(); + + free(head); + return 0; +} + +void +fsrefcounts_init(void) +{ + fsrefcounts_cmd.name = "fsrefcounts"; + fsrefcounts_cmd.cfunc = fsrefcounts_f; + fsrefcounts_cmd.argmin = 0; + fsrefcounts_cmd.argmax = -1; + fsrefcounts_cmd.flags = CMD_NOMAP_OK | CMD_FLAG_FOREIGN_OK; + fsrefcounts_cmd.args = _("[-d|-l|-r] [-m|-v] [-n nx] [start] [end]"); + fsrefcounts_cmd.oneline = _("print filesystem owner counts for a range of blocks"); + fsrefcounts_cmd.help = fsrefcounts_help; + + add_command(&fsrefcounts_cmd); +} diff --git a/io/init.c b/io/init.c index 78d7d04e7a6..771091412d0 100644 --- a/io/init.c +++ b/io/init.c @@ -58,6 +58,7 @@ init_commands(void) flink_init(); freeze_init(); fsmap_init(); + fsrefcounts_init(); fsync_init(); getrusage_init(); help_init(); diff --git a/io/io.h b/io/io.h index 77bedf5159d..de4ef6077f2 100644 --- a/io/io.h +++ b/io/io.h @@ -190,3 +190,4 @@ extern void crc32cselftest_init(void); extern void bulkstat_init(void); extern void atomicupdate_init(void); extern void aginfo_init(void); +extern void fsrefcounts_init(void); diff --git a/libfrog/fsrefcounts.h b/libfrog/fsrefcounts.h new file mode 100644 index 00000000000..b9057b90ff9 --- /dev/null +++ b/libfrog/fsrefcounts.h @@ -0,0 +1,100 @@ +#ifdef HAVE_GETFSREFCOUNTS +# include <linux/fsrefcounts.h> +#endif + +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * FS_IOC_GETFSREFCOUNTS ioctl infrastructure. + * + * Copyright (C) 2022 Oracle. All Rights Reserved. + * + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#ifndef _LINUX_FSREFCOUNTS_H +#define _LINUX_FSREFCOUNTS_H + +#include <linux/types.h> + +/* + * Structure for FS_IOC_GETFSREFCOUNTS. + * + * The memory layout for this call are the scalar values defined in + * struct fsrefs_head, followed by two struct fsrefs that describe the + * lower and upper bound of mappings to return, followed by an array of + * struct fsrefs mappings. + * + * fch_iflags control the output of the call, whereas fch_oflags report + * on the overall record output. fch_count should be set to the length + * of the fch_recs array, and fch_entries will be set to the number of + * entries filled out during each call. If fch_count is zero, the number + * of refcount mappings will be returned in fch_entries, though no + * mappings will be returned. fch_reserved must be set to zero. + * + * The two elements in the fch_keys array are used to constrain the + * output. The first element in the array should represent the lowest + * disk mapping ("low key") that the user wants to learn about. If this + * value is all zeroes, the filesystem will return the first entry it + * knows about. For a subsequent call, the contents of + * fsrefs_head.fch_recs[fsrefs_head.fch_count - 1] should be copied into + * fch_keys[0] to have the kernel start where it left off. + * + * The second element in the fch_keys array should represent the highest + * disk mapping ("high key") that the user wants to learn about. If this + * value is all ones, the filesystem will not stop until it runs out of + * mapping to return or runs out of space in fch_recs. + * + * fcr_device can be either a 32-bit cookie representing a device, or a + * 32-bit dev_t if the FCH_OF_DEV_T flag is set. fcr_physical and + * fcr_length are expressed in units of bytes. fcr_owners is the number + * of owners. + */ +struct fsrefs { + __u32 fcr_device; /* device id */ + __u32 fcr_flags; /* mapping flags */ + __u64 fcr_physical; /* device offset of segment */ + __u64 fcr_owners; /* number of owners */ + __u64 fcr_length; /* length of segment */ + __u64 fcr_reserved[4]; /* must be zero */ +}; + +struct fsrefs_head { + __u32 fch_iflags; /* control flags */ + __u32 fch_oflags; /* output flags */ + __u32 fch_count; /* # of entries in array incl. input */ + __u32 fch_entries; /* # of entries filled in (output). */ + __u64 fch_reserved[6]; /* must be zero */ + + struct fsrefs fch_keys[2]; /* low and high keys for the mapping search */ + struct fsrefs fch_recs[]; /* returned records */ +}; + +/* Size of an fsrefs_head with room for nr records. */ +static inline unsigned long long +fsrefs_sizeof( + unsigned int nr) +{ + return sizeof(struct fsrefs_head) + nr * sizeof(struct fsrefs); +} + +/* Start the next fsrefs query at the end of the current query results. */ +static inline void +fsrefs_advance( + struct fsrefs_head *head) +{ + head->fch_keys[0] = head->fch_recs[head->fch_entries - 1]; +} + +/* fch_iflags values - set by FS_IOC_GETFSREFCOUNTS caller in the header. */ +/* no flags defined yet */ +#define FCH_IF_VALID 0 + +/* fch_oflags values - returned in the header segment only. */ +#define FCH_OF_DEV_T 0x1 /* fcr_device values will be dev_t */ + +/* fcr_flags values - returned for each non-header segment */ +#define FCR_OF_LAST (1U << 0) /* segment is the last in the dataset */ + +/* XXX stealing XFS_IOC_GETBIOSIZE */ +#define FS_IOC_GETFSREFCOUNTS _IOWR('X', 47, struct fsrefs_head) + +#endif /* _LINUX_FSREFCOUNTS_H */ diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index 062730a1f06..5293dd1aec2 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -347,6 +347,25 @@ struct fsmap_head fh; AC_SUBST(have_getfsmap) ]) +# +# Check if we have a FS_IOC_GETFSREFCOUNTS ioctl (Linux) +# +AC_DEFUN([AC_HAVE_GETFSREFCOUNTS], + [ AC_MSG_CHECKING([for GETFSREFCOUNTS]) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[ +#define _GNU_SOURCE +#include <sys/syscall.h> +#include <unistd.h> +#include <linux/fs.h> +#include <linux/fsrefcounts.h> + ]], [[ + unsigned long x = FS_IOC_GETFSREFCOUNTS; + struct fsrefs_head fh; + ]])],[have_getfsrefcounts=yes + AC_MSG_RESULT(yes)],[AC_MSG_RESULT(no)]) + AC_SUBST(have_getfsrefcounts) + ]) + AC_DEFUN([AC_HAVE_STATFS_FLAGS], [ AC_CHECK_TYPE(struct statfs, diff --git a/man/man8/xfs_io.8 b/man/man8/xfs_io.8 index df56080b8b8..ece778fc76c 100644 --- a/man/man8/xfs_io.8 +++ b/man/man8/xfs_io.8 @@ -1593,6 +1593,92 @@ flag. .RE .PD .TP +.BI "fsrefcounts [ \-d | \-l | \-r ] [ \-m | \-v ] [ \-n " nx " ] [ \-o " min_owners " ] [ \-O " max_owners " ] [ " start " ] [ " end " ] +Prints the number of owners of disk extents used by the filesystem hosting the +current file. +The listing does not include free blocks. +Each line of the listings takes the following form: +.PP +.RS +.IR extent ": " major ":" minor " [" startblock .. endblock "]: " owners " " length +.PP +All blocks, offsets, and lengths are specified in units of 512-byte +blocks, no matter what the filesystem's block size is. +The optional +.I start +and +.I end +arguments can be used to constrain the output to a particular range of +disk blocks. +If these two options are specified, exactly one of +.BR "-d" ", " "-l" ", or " "-r" +must also be set. +.RE +.RS 1.0i +.PD 0 +.TP +.BI \-d +Display only extents from the data device. +This option only applies for XFS filesystems. +.TP +.BI \-l +Display only extents from the external log device. +This option only applies to XFS filesystems. +.TP +.BI \-r +Display only extents from the realtime device. +This option only applies to XFS filesystems. +.TP +.BI \-m +Display results in a machine readable format (CSV). +This option is not compatible with the +.B \-v +flag. +The columns of the output are: extent number, device major, device minor, +physical start, physical end, number of owners, length. +The start, end, and length numbers are provided in units of 512b. + +.TP +.BI \-n " num_extents" +If this option is given, +.B fsrefcounts +obtains the extent list of the file in groups of +.I num_extents +extents. +In the absence of +.BR "-n" ", " "fsrefcounts" +queries the system for extents in groups of 131,072 records. +.TP +.BI \-o " min_owners" +Only print extents having at least this many owners. +This argument must be in the range 1 to 2^64-1. +The default value is 1. +.TP +.BI \-O " max_owners" +Only print extents having this many or fewer owners. +This argument must be in the range 1 to 2^64-1. +There is no limit by default. +.TP +.B \-v +Shows verbose information. +When this flag is specified, additional AG specific information is +appended to each line in the following form: +.IP +.RS 1.2i +.IR agno " (" startagblock .. endagblock ") " nblocks " " flags +.RE +.IP +A second +.B \-v +option will print out the +.I flags +legend. +This option is not compatible with the +.B \-m +flag. +.RE +.PD +.TP .BI "aginfo [ \-a " agno " ] [ \-o " nr " ]" Show information about or update the state of allocation groups. .RE ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/5] xfsprogs: defragment free space 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (5 preceding siblings ...) 2022-12-30 22:20 ` [PATCHSET 0/1] libxfs: report refcount information to userspace Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 1/5] xfs: fallocate free space into a file Darrick J. Wong ` (4 more replies) 2022-12-30 22:21 ` [PATCHSET 0/1] xfs_scrub: vectorize kernel calls Darrick J. Wong ` (2 subsequent siblings) 9 siblings, 5 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs Hi all, These patches contain experimental code to enable userspace to defragment the free space in a filesystem. Two purposes are imagined for this functionality: clearing space at the end of a filesystem before shrinking it, and clearing free space in anticipation of making a large allocation. The first patch adds a new fallocate mode that allows userspace to allocate free space from the filesystem into a file. The goal here is to allow the filesystem shrink process to prevent allocation from a certain part of the filesystem while a free space defragmenter evacuates all the files from the doomed part of the filesystem. The second patch amends the online repair system to allow the sysadmin to forcibly rebuild metadata structures, even if they're not corrupt. Without adding an ioctl to move metadata btree blocks, this is the only way to dislodge metadata. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=defrag-freespace --- Makefile | 2 db/agfl.c | 297 +++++ include/xfs_trace.h | 4 io/prealloc.c | 37 + libfrog/Makefile | 6 libfrog/clearspace.c | 2940 ++++++++++++++++++++++++++++++++++++++++++++++ libfrog/clearspace.h | 72 + libxfs/libxfs_api_defs.h | 3 libxfs/libxfs_priv.h | 10 libxfs/xfs_alloc.c | 88 + libxfs/xfs_alloc.h | 4 libxfs/xfs_bmap.c | 149 ++ libxfs/xfs_bmap.h | 3 man/man8/xfs_db.8 | 11 man/man8/xfs_io.8 | 8 man/man8/xfs_spaceman.8 | 17 spaceman/Makefile | 6 spaceman/clearfree.c | 164 +++ spaceman/init.c | 1 spaceman/space.h | 2 20 files changed, 3815 insertions(+), 9 deletions(-) create mode 100644 libfrog/clearspace.c create mode 100644 libfrog/clearspace.h create mode 100644 spaceman/clearfree.c ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 1/5] xfs: fallocate free space into a file 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 2/5] xfs_io: support using fallocate to map free space Darrick J. Wong ` (3 subsequent siblings) 4 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Add a new fallocate mode to map free physical space into a file, at the same file offset as if the file were a sparse image of the physical device backing the filesystem. The intent here is to use this to prototype a free space defragmentation tool. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- include/xfs_trace.h | 4 + libxfs/libxfs_priv.h | 10 +++ libxfs/xfs_alloc.c | 88 ++++++++++++++++++++++++++++++ libxfs/xfs_alloc.h | 4 + libxfs/xfs_bmap.c | 149 ++++++++++++++++++++++++++++++++++++++++++++++++++ libxfs/xfs_bmap.h | 3 + 6 files changed, 258 insertions(+) diff --git a/include/xfs_trace.h b/include/xfs_trace.h index 3b305e4d16a..aa000fad856 100644 --- a/include/xfs_trace.h +++ b/include/xfs_trace.h @@ -26,6 +26,8 @@ #define trace_xfs_alloc_exact_done(a) ((void) 0) #define trace_xfs_alloc_exact_notfound(a) ((void) 0) #define trace_xfs_alloc_exact_error(a) ((void) 0) +#define trace_xfs_alloc_find_freesp(...) ((void) 0) +#define trace_xfs_alloc_find_freesp_done(...) ((void) 0) #define trace_xfs_alloc_near_first(a) ((void) 0) #define trace_xfs_alloc_near_greater(a) ((void) 0) #define trace_xfs_alloc_near_lesser(a) ((void) 0) @@ -189,6 +191,8 @@ #define trace_xfs_bmap_pre_update(a,b,c,d) ((void) 0) #define trace_xfs_bmap_post_update(a,b,c,d) ((void) 0) +#define trace_xfs_bmapi_freesp(...) ((void) 0) +#define trace_xfs_bmapi_freesp_done(...) ((void) 0) #define trace_xfs_bunmap(a,b,c,d,e) ((void) 0) #define trace_xfs_read_extent(a,b,c,d) ((void) 0) diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h index b5b956dea11..fa8ddd5b1aa 100644 --- a/libxfs/libxfs_priv.h +++ b/libxfs/libxfs_priv.h @@ -502,6 +502,16 @@ void __xfs_buf_mark_corrupt(struct xfs_buf *bp, xfs_failaddr_t fa); #define xfs_filestream_lookup_ag(ip) (0) #define xfs_filestream_new_ag(ip,ag) (0) +struct xfs_trans; + +static inline int +xfs_rtallocate_extent(struct xfs_trans *tp, xfs_rtblock_t bno, + xfs_extlen_t minlen, xfs_extlen_t maxlen, xfs_extlen_t *len, + int wasdel, xfs_extlen_t prod, xfs_rtblock_t *rtblock) +{ + return -EOPNOTSUPP; +} + #define xfs_trans_inode_buf(tp, bp) ((void) 0) /* quota bits */ diff --git a/libxfs/xfs_alloc.c b/libxfs/xfs_alloc.c index 43b63462374..6099054046a 100644 --- a/libxfs/xfs_alloc.c +++ b/libxfs/xfs_alloc.c @@ -3701,3 +3701,91 @@ xfs_extfree_intent_destroy_cache(void) kmem_cache_destroy(xfs_extfree_item_cache); xfs_extfree_item_cache = NULL; } + +/* + * Find the next chunk of free space in @pag starting at @agbno and going no + * higher than @end_agbno. Set @agbno and @len to whatever free space we find, + * or to @end_agbno if we find no space. + */ +int +xfs_alloc_find_freesp( + struct xfs_trans *tp, + struct xfs_perag *pag, + xfs_agblock_t *agbno, + xfs_agblock_t end_agbno, + xfs_extlen_t *len) +{ + struct xfs_mount *mp = pag->pag_mount; + struct xfs_btree_cur *cur; + struct xfs_buf *agf_bp = NULL; + xfs_agblock_t found_agbno; + xfs_extlen_t found_len; + int found; + int error; + + trace_xfs_alloc_find_freesp(mp, pag->pag_agno, *agbno, + end_agbno - *agbno); + + error = xfs_alloc_read_agf(pag, tp, 0, &agf_bp); + if (error) + return error; + + cur = xfs_allocbt_init_cursor(mp, tp, agf_bp, pag, XFS_BTNUM_BNO); + + /* Try to find a free extent that starts before here. */ + error = xfs_alloc_lookup_le(cur, *agbno, 0, &found); + if (error) + goto out_cur; + if (found) { + error = xfs_alloc_get_rec(cur, &found_agbno, &found_len, + &found); + if (error) + goto out_cur; + if (XFS_IS_CORRUPT(mp, !found)) { + xfs_btree_mark_sick(cur); + error = -EFSCORRUPTED; + goto out_cur; + } + + if (found_agbno + found_len > *agbno) + goto found; + } + + /* Examine the next record if free extent not in range. */ + error = xfs_btree_increment(cur, 0, &found); + if (error) + goto out_cur; + if (!found) + goto next_ag; + + error = xfs_alloc_get_rec(cur, &found_agbno, &found_len, &found); + if (error) + goto out_cur; + if (XFS_IS_CORRUPT(mp, !found)) { + xfs_btree_mark_sick(cur); + error = -EFSCORRUPTED; + goto out_cur; + } + + if (found_agbno >= end_agbno) + goto next_ag; + +found: + /* Found something, so update the mapping. */ + trace_xfs_alloc_find_freesp_done(mp, pag->pag_agno, found_agbno, + found_len); + if (found_agbno < *agbno) { + found_len -= *agbno - found_agbno; + found_agbno = *agbno; + } + *len = found_len; + *agbno = found_agbno; + goto out_cur; +next_ag: + /* Found nothing, so advance the cursor beyond the end of the range. */ + *agbno = end_agbno; + *len = 0; +out_cur: + xfs_btree_del_cursor(cur, error); + return error; +} diff --git a/libxfs/xfs_alloc.h b/libxfs/xfs_alloc.h index cd7b26568a3..327c66f5578 100644 --- a/libxfs/xfs_alloc.h +++ b/libxfs/xfs_alloc.h @@ -268,4 +268,8 @@ extern struct kmem_cache *xfs_extfree_item_cache; int __init xfs_extfree_intent_init_cache(void); void xfs_extfree_intent_destroy_cache(void); +int xfs_alloc_find_freesp(struct xfs_trans *tp, struct xfs_perag *pag, + xfs_agblock_t *agbno, xfs_agblock_t end_agbno, + xfs_extlen_t *len); + #endif /* __XFS_ALLOC_H__ */ diff --git a/libxfs/xfs_bmap.c b/libxfs/xfs_bmap.c index b8fe093f0f3..b0f19bae120 100644 --- a/libxfs/xfs_bmap.c +++ b/libxfs/xfs_bmap.c @@ -6479,3 +6479,152 @@ xfs_get_cowextsz_hint( return XFS_DEFAULT_COWEXTSZ_HINT; return a; } + +static inline xfs_fileoff_t +xfs_fsblock_to_fileoff( + struct xfs_mount *mp, + xfs_fsblock_t fsbno) +{ + xfs_daddr_t daddr = XFS_FSB_TO_DADDR(mp, fsbno); + + return XFS_B_TO_FSB(mp, BBTOB(daddr)); +} + +/* + * Given a file and a free physical extent, map it into the file at the same + * offset if the file were a sparse image of the physical device. Set @mval to + * whatever mapping we added to the file. + */ +int +xfs_bmapi_freesp( + struct xfs_trans *tp, + struct xfs_inode *ip, + xfs_fsblock_t fsbno, + xfs_extlen_t len, + struct xfs_bmbt_irec *mval) +{ + struct xfs_bmbt_irec irec; + struct xfs_mount *mp = ip->i_mount; + xfs_fileoff_t startoff; + bool isrt = XFS_IS_REALTIME_INODE(ip); + int nimaps; + int error; + + trace_xfs_bmapi_freesp(ip, fsbno, len); + + error = xfs_iext_count_may_overflow(ip, XFS_DATA_FORK, + XFS_IEXT_ADD_NOSPLIT_CNT); + if (error) + return error; + + if (isrt) + startoff = fsbno; + else + startoff = xfs_fsblock_to_fileoff(mp, fsbno); + + /* Make sure the entire range is a hole. */ + nimaps = 1; + error = xfs_bmapi_read(ip, startoff, len, &irec, &nimaps, 0); + if (error) + return error; + + if (irec.br_startoff != startoff || + irec.br_startblock != HOLESTARTBLOCK || + irec.br_blockcount < len) + return -EINVAL; + + /* + * Allocate the physical extent. We should not have dropped the lock + * since the scan of the free space metadata, so this should work, + * though the length may be adjusted to play nicely with metadata space + * reservations. + */ + if (isrt) { + xfs_rtxnum_t rtx_in, rtx_out; + xfs_extlen_t rtxlen_in, rtxlen_out; + uint32_t mod; + + rtx_in = xfs_rtb_to_rtx(mp, fsbno, &mod); + if (mod) { + ASSERT(mod == 0); + return -EFSCORRUPTED; + } + + rtxlen_in = xfs_rtb_to_rtx(mp, len, &mod); + if (mod) { + ASSERT(mod == 0); + return -EFSCORRUPTED; + } + + error = xfs_rtallocate_extent(tp, rtx_in, 1, rtxlen_in, + &rtxlen_out, 0, 1, &rtx_out); + if (error) + return error; + if (rtx_out == NULLRTEXTNO) { + /* + * We were promised the space! In theory the aren't + * any reserve lists that would prevent us from getting + * the space. + */ + return -ENOSPC; + } + if (rtx_out != rtx_in) { + ASSERT(0); + xfs_bmap_mark_sick(ip, XFS_DATA_FORK); + return -EFSCORRUPTED; + } + mval->br_blockcount = rtxlen_out * mp->m_sb.sb_rextsize; + } else { + struct xfs_alloc_arg args = { + .mp = ip->i_mount, + .type = XFS_ALLOCTYPE_THIS_BNO, + .oinfo = XFS_RMAP_OINFO_SKIP_UPDATE, + .resv = XFS_AG_RESV_NONE, + .prod = 1, + .datatype = XFS_ALLOC_USERDATA, + .tp = tp, + .maxlen = len, + .minlen = 1, + .fsbno = fsbno, + }; + error = xfs_alloc_vextent(&args); + if (error) + return error; + if (args.fsbno == NULLFSBLOCK) { + /* + * We were promised the space, but failed to get it. + * This could be because the space is reserved for + * metadata expansion, or it could be because the AGFL + * fixup grabbed the first block we wanted. Either + * way, if the transaction is dirty we must commit it + * and tell the caller to try again. + */ + if (tp->t_flags & XFS_TRANS_DIRTY) + return -EAGAIN; + return -ENOSPC; + } + if (args.fsbno != fsbno) { + ASSERT(0); + xfs_bmap_mark_sick(ip, XFS_DATA_FORK); + return -EFSCORRUPTED; + } + mval->br_blockcount = args.len; + } + + /* Map extent into file, update quota. */ + mval->br_startblock = fsbno; + mval->br_startoff = startoff; + mval->br_state = XFS_EXT_UNWRITTEN; + + trace_xfs_bmapi_freesp_done(ip, mval); + + xfs_bmap_map_extent(tp, ip, XFS_DATA_FORK, mval); + if (isrt) + xfs_trans_mod_dquot_byino(tp, ip, XFS_TRANS_DQ_RTBCOUNT, + mval->br_blockcount); + else + xfs_trans_mod_dquot_byino(tp, ip, XFS_TRANS_DQ_BCOUNT, + mval->br_blockcount); + + return 0; +} diff --git a/libxfs/xfs_bmap.h b/libxfs/xfs_bmap.h index 05097b1d5c7..ef20a625762 100644 --- a/libxfs/xfs_bmap.h +++ b/libxfs/xfs_bmap.h @@ -193,6 +193,9 @@ int xfs_bmapi_read(struct xfs_inode *ip, xfs_fileoff_t bno, int xfs_bmapi_write(struct xfs_trans *tp, struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t len, uint32_t flags, xfs_extlen_t total, struct xfs_bmbt_irec *mval, int *nmap); +int xfs_bmapi_freesp(struct xfs_trans *tp, struct xfs_inode *ip, + xfs_fsblock_t fsbno, xfs_extlen_t len, + struct xfs_bmbt_irec *mval); int __xfs_bunmapi(struct xfs_trans *tp, struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t *rlen, uint32_t flags, xfs_extnum_t nexts); ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 2/5] xfs_io: support using fallocate to map free space 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong 2022-12-30 22:20 ` [PATCH 1/5] xfs: fallocate free space into a file Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 4/5] xfs_spaceman: implement clearing " Darrick J. Wong ` (2 subsequent siblings) 4 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Support FALLOC_FL_MAP_FREE_SPACE as a fallocate mode. This is experimental code to see if we can build a free space defragmenter out of this. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- io/prealloc.c | 37 +++++++++++++++++++++++++++++++++++++ man/man8/xfs_io.8 | 8 +++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/io/prealloc.c b/io/prealloc.c index 5805897a4a0..02de16651c1 100644 --- a/io/prealloc.c +++ b/io/prealloc.c @@ -32,6 +32,10 @@ #define FALLOC_FL_UNSHARE_RANGE 0x40 #endif +#ifndef FALLOC_FL_MAP_FREE_SPACE +#define FALLOC_FL_MAP_FREE_SPACE (1U << 30) +#endif + static cmdinfo_t allocsp_cmd; static cmdinfo_t freesp_cmd; static cmdinfo_t resvsp_cmd; @@ -44,6 +48,7 @@ static cmdinfo_t fcollapse_cmd; static cmdinfo_t finsert_cmd; static cmdinfo_t fzero_cmd; static cmdinfo_t funshare_cmd; +static cmdinfo_t fmapfree_cmd; #endif static int @@ -381,6 +386,28 @@ funshare_f( } return 0; } + +static int +fmapfree_f( + int argc, + char **argv) +{ + xfs_flock64_t segment; + int mode = FALLOC_FL_MAP_FREE_SPACE; + int index = 1; + + if (!offset_length(argv[index], argv[index + 1], &segment)) { + exitcode = 1; + return 0; + } + + if (fallocate(file->fd, mode, segment.l_start, segment.l_len)) { + perror("fallocate"); + exitcode = 1; + return 0; + } + return 0; +} #endif /* HAVE_FALLOCATE */ void @@ -496,5 +523,15 @@ prealloc_init(void) funshare_cmd.oneline = _("unshares shared blocks within the range"); add_command(&funshare_cmd); + + fmapfree_cmd.name = "fmapfree"; + fmapfree_cmd.cfunc = fmapfree_f; + fmapfree_cmd.argmin = 2; + fmapfree_cmd.argmax = 2; + fmapfree_cmd.flags = CMD_NOMAP_OK | CMD_FOREIGN_OK; + fmapfree_cmd.args = _("off len"); + fmapfree_cmd.oneline = + _("maps free space into a file"); + add_command(&fmapfree_cmd); #endif /* HAVE_FALLOCATE */ } diff --git a/man/man8/xfs_io.8 b/man/man8/xfs_io.8 index ece778fc76c..b10523325ee 100644 --- a/man/man8/xfs_io.8 +++ b/man/man8/xfs_io.8 @@ -513,8 +513,14 @@ Call fallocate with FALLOC_FL_INSERT_RANGE flag as described in the .BR fallocate (2) manual page to create the hole by shifting data blocks. .TP +.BI fmapfree " offset length" +Maps free physical space into the file by calling fallocate with +the FALLOC_FL_MAP_FREE_SPACE flag as described in the +.BR fallocate (2) +manual page. +.TP .BI fpunch " offset length" -Punches (de-allocates) blocks in the file by calling fallocate with +Punches (de-allocates) blocks in the file by calling fallocate with the FALLOC_FL_PUNCH_HOLE flag as described in the .BR fallocate (2) manual page. ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 4/5] xfs_spaceman: implement clearing free space 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong 2022-12-30 22:20 ` [PATCH 1/5] xfs: fallocate free space into a file Darrick J. Wong 2022-12-30 22:20 ` [PATCH 2/5] xfs_io: support using fallocate to map free space Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 3/5] xfs_db: get and put blocks on the AGFL Darrick J. Wong 2022-12-30 22:20 ` [PATCH 5/5] xfs_spaceman: defragment free space with normal files Darrick J. Wong 4 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> First attempt at evacuating all the used blocks from part of a filesystem. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- Makefile | 2 libfrog/Makefile | 6 libfrog/clearspace.c | 2723 +++++++++++++++++++++++++++++++++++++++++++++++ libfrog/clearspace.h | 72 + man/man8/xfs_spaceman.8 | 17 spaceman/Makefile | 6 spaceman/clearfree.c | 164 +++ spaceman/init.c | 1 spaceman/space.h | 2 9 files changed, 2989 insertions(+), 4 deletions(-) create mode 100644 libfrog/clearspace.c create mode 100644 libfrog/clearspace.h create mode 100644 spaceman/clearfree.c diff --git a/Makefile b/Makefile index 0edc2700933..c7c31444446 100644 --- a/Makefile +++ b/Makefile @@ -105,7 +105,7 @@ quota: libxcmd repair: libxlog libxcmd copy: libxlog mkfs: libxcmd -spaceman: libxcmd +spaceman: libhandle libxcmd scrub: libhandle libxcmd rtcp: libfrog diff --git a/libfrog/Makefile b/libfrog/Makefile index 04aecf1abf1..4031f07e422 100644 --- a/libfrog/Makefile +++ b/libfrog/Makefile @@ -61,6 +61,12 @@ ifeq ($(HAVE_FIEXCHANGE),yes) LCFLAGS += -DHAVE_FIEXCHANGE endif +ifeq ($(HAVE_GETFSMAP),yes) +CFILES+=clearspace.c +HFILES+=clearspace.h +endif + + LDIRT = gen_crc32table crc32table.h crc32selftest default: crc32selftest ltdepend $(LTLIBRARY) diff --git a/libfrog/clearspace.c b/libfrog/clearspace.c new file mode 100644 index 00000000000..601257022b8 --- /dev/null +++ b/libfrog/clearspace.c @@ -0,0 +1,2723 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#include "xfs.h" +#include <linux/fsmap.h> +#include "paths.h" +#include "fsgeom.h" +#include "fsrefcounts.h" +#include "logging.h" +#include "bulkstat.h" +#include "bitmap.h" +#include "fiexchange.h" +#include "file_exchange.h" +#include "clearspace.h" +#include "handle.h" + +/* VFS helpers */ + +#ifndef FALLOC_FL_MAP_FREE_SPACE +#define FALLOC_FL_MAP_FREE_SPACE 0x8000 +#endif + +/* Remap the file range described by @fcr into fd, or return an errno. */ +static inline int +clonerange(int fd, struct file_clone_range *fcr) +{ + int ret; + + ret = ioctl(fd, FICLONERANGE, fcr); + if (ret) + return errno; + + return 0; +} + +/* Exchange the file ranges described by @xchg into fd, or return an errno. */ +static inline int +exchangerange(int fd, struct file_xchg_range *xchg) +{ + int ret; + + ret = ioctl(fd, FIEXCHANGE_RANGE, xchg); + if (ret) + return errno; + + return 0; +} + +/* + * Deduplicate part of fd into the file range described by fdr. If the + * operation succeeded, we set @same to whether or not we deduped the data and + * return zero. If not, return an errno. + */ +static inline int +deduperange(int fd, struct file_dedupe_range *fdr, bool *same) +{ + struct file_dedupe_range_info *info = &fdr->info[0]; + int ret; + + assert(fdr->dest_count == 1); + *same = false; + + ret = ioctl(fd, FIDEDUPERANGE, fdr); + if (ret) + return errno; + + if (info->status < 0) + return -info->status; + + if (info->status == FILE_DEDUPE_RANGE_DIFFERS) + return 0; + + /* The kernel should never dedupe more than it was asked. */ + assert(fdr->src_length >= info->bytes_deduped); + + *same = true; + return 0; +} + +/* Space clearing operation control */ + +#define QUERY_BATCH_SIZE 1024 + +struct clearspace_tgt { + unsigned long long start; + unsigned long long length; + unsigned long long owners; + unsigned long long prio; + unsigned long long evacuated; + bool try_again; +}; + +struct clearspace_req { + struct xfs_fd *xfd; + + /* all the blocks that we've tried to clear */ + struct bitmap *visited; + + /* stat buffer of the open file */ + struct stat statbuf; + struct stat temp_statbuf; + struct stat space_statbuf; + + /* handle to this filesystem */ + void *fshandle; + size_t fshandle_sz; + + /* physical storage that we want to clear */ + unsigned long long start; + unsigned long long length; + dev_t dev; + + /* convenience variable */ + bool realtime:1; + bool use_reflink:1; + bool can_evac_metadata:1; + + /* + * The "space capture" file. Each extent in this file must be mapped + * to the same byte offset as the byte address of the physical space. + */ + int space_fd; + + /* work file for migrating file data */ + int work_fd; + + /* preallocated buffers for queries */ + struct getbmapx *bhead; + struct fsmap_head *mhead; + struct fsrefs_head *rhead; + + /* buffer for copying data */ + char *buf; + + /* buffer for deduping data */ + struct file_dedupe_range *fdr; + + /* tracing mask and indent level */ + unsigned int trace_mask; + unsigned int trace_indent; +}; + +static inline bool +csp_is_internal_owner( + const struct clearspace_req *req, + unsigned long long owner) +{ + return owner == req->temp_statbuf.st_ino || + owner == req->space_statbuf.st_ino; +} + +/* Debugging stuff */ + +static const struct csp_errstr { + unsigned int mask; + const char *tag; +} errtags[] = { + { CSP_TRACE_FREEZE, "freeze" }, + { CSP_TRACE_GRAB, "grab" }, + { CSP_TRACE_PREP, "prep" }, + { CSP_TRACE_TARGET, "target" }, + { CSP_TRACE_DEDUPE, "dedupe" }, + { CSP_TRACE_FIEXCHANGE, "fiexchange" }, + { CSP_TRACE_XREBUILD, "rebuild" }, + { CSP_TRACE_EFFICACY, "efficacy" }, + { CSP_TRACE_SETUP, "setup" }, + { CSP_TRACE_DUMPFILE, "dumpfile" }, + { CSP_TRACE_BITMAP, "bitmap" }, + + /* prioritize high level functions over low level queries for tagging */ + { CSP_TRACE_FSMAP, "fsmap" }, + { CSP_TRACE_FSREFS, "fsrefs" }, + { CSP_TRACE_BMAPX, "bmapx" }, + { CSP_TRACE_FALLOC, "falloc" }, + { CSP_TRACE_STATUS, "status" }, + { 0, NULL }, +}; + +static void +csp_debug( + struct clearspace_req *req, + unsigned int mask, + const char *func, + int line, + const char *format, + ...) +{ + const struct csp_errstr *et = errtags; + bool debug = (req->trace_mask & ~CSP_TRACE_STATUS); + int indent = req->trace_indent; + va_list args; + + if ((req->trace_mask & mask) != mask) + return; + + if (debug) { + while (indent > 0) { + fprintf(stderr, " "); + indent--; + } + + for (; et->tag; et++) { + if (et->mask & mask) { + fprintf(stderr, "%s: ", et->tag); + break; + } + } + } + + va_start(args, format); + vfprintf(stderr, format, args); + va_end(args); + + if (debug) + fprintf(stderr, " (line %d)\n", line); + else + fprintf(stderr, "\n"); + fflush(stderr); +} + +#define trace_freeze(req, format, ...) \ + csp_debug((req), CSP_TRACE_FREEZE, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_grabfree(req, format, ...) \ + csp_debug((req), CSP_TRACE_GRAB, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_fsmap(req, format, ...) \ + csp_debug((req), CSP_TRACE_FSMAP, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_fsmap_rec(req, mask, mrec) \ + while (!csp_is_internal_owner((req), (mrec)->fmr_owner)) { \ + csp_debug((req), (mask) | CSP_TRACE_FSMAP, __func__, __LINE__, \ +"fsmap phys 0x%llx owner 0x%llx offset 0x%llx bytecount 0x%llx flags 0x%x", \ + (unsigned long long)(mrec)->fmr_physical, \ + (unsigned long long)(mrec)->fmr_owner, \ + (unsigned long long)(mrec)->fmr_offset, \ + (unsigned long long)(mrec)->fmr_length, \ + (mrec)->fmr_flags); \ + break; \ + } + +#define trace_fsrefs(req, format, ...) \ + csp_debug((req), CSP_TRACE_FSREFS, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_fsrefs_rec(req, mask, rrec) \ + csp_debug((req), (mask) | CSP_TRACE_FSREFS, __func__, __LINE__, \ +"fsref phys 0x%llx bytecount 0x%llx owners %llu flags 0x%x", \ + (unsigned long long)(rrec)->fcr_physical, \ + (unsigned long long)(rrec)->fcr_length, \ + (unsigned long long)(rrec)->fcr_owners, \ + (rrec)->fcr_flags) + +#define trace_bmapx(req, format, ...) \ + csp_debug((req), CSP_TRACE_BMAPX, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_bmapx_rec(req, mask, brec) \ + csp_debug((req), (mask) | CSP_TRACE_BMAPX, __func__, __LINE__, \ +"bmapx pos 0x%llx bytecount 0x%llx phys 0x%llx flags 0x%x", \ + (unsigned long long)BBTOB((brec)->bmv_offset), \ + (unsigned long long)BBTOB((brec)->bmv_length), \ + (unsigned long long)BBTOB((brec)->bmv_block), \ + (brec)->bmv_oflags) + +#define trace_prep(req, format, ...) \ + csp_debug((req), CSP_TRACE_PREP, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_target(req, format, ...) \ + csp_debug((req), CSP_TRACE_TARGET, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_dedupe(req, format, ...) \ + csp_debug((req), CSP_TRACE_DEDUPE, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_falloc(req, format, ...) \ + csp_debug((req), CSP_TRACE_FALLOC, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_fiexchange(req, format, ...) \ + csp_debug((req), CSP_TRACE_FIEXCHANGE, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_xrebuild(req, format, ...) \ + csp_debug((req), CSP_TRACE_XREBUILD, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_setup(req, format, ...) \ + csp_debug((req), CSP_TRACE_SETUP, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_status(req, format, ...) \ + csp_debug((req), CSP_TRACE_STATUS, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_dumpfile(req, format, ...) \ + csp_debug((req), CSP_TRACE_DUMPFILE, __func__, __LINE__, format, __VA_ARGS__) + +#define trace_bitmap(req, format, ...) \ + csp_debug((req), CSP_TRACE_BITMAP, __func__, __LINE__, format, __VA_ARGS__) + +/* VFS Iteration helpers */ + +static inline void +start_spacefd_iter(struct clearspace_req *req) +{ + req->trace_indent++; +} + +static inline void +end_spacefd_iter(struct clearspace_req *req) +{ + req->trace_indent--; +} + +/* + * Iterate each hole in the space-capture file. Returns 1 if holepos/length + * has been set to a hole; 0 if there aren't any holes left, or -1 for error. + */ +static inline int +spacefd_hole_iter( + const struct clearspace_req *req, + loff_t *holepos, + loff_t *length) +{ + loff_t end = req->start + req->length; + loff_t h; + loff_t d; + + if (*length == 0) + d = req->start; + else + d = *holepos + *length; + if (d >= end) + return 0; + + h = lseek(req->space_fd, d, SEEK_HOLE); + if (h < 0) { + perror(_("finding start of hole in space capture file")); + return h; + } + if (h >= end) + return 0; + + d = lseek(req->space_fd, h, SEEK_DATA); + if (d < 0 && errno == ENXIO) + d = end; + if (d < 0) { + perror(_("finding end of hole in space capture file")); + return d; + } + if (d > end) + d = end; + + *holepos = h; + *length = d - h; + return 1; +} + +/* + * Iterate each written region in the space-capture file. Returns 1 if + * datapos/length have been set to a data area; 0 if there isn't any data left, + * or -1 for error. + */ +static int +spacefd_data_iter( + const struct clearspace_req *req, + loff_t *datapos, + loff_t *length) +{ + loff_t end = req->start + req->length; + loff_t d; + loff_t h; + + if (*length == 0) + h = req->start; + else + h = *datapos + *length; + if (h >= end) + return 0; + + d = lseek(req->space_fd, h, SEEK_DATA); + if (d < 0 && errno == ENXIO) + return 0; + if (d < 0) { + perror(_("finding start of data in space capture file")); + return d; + } + if (d >= end) + return 0; + + h = lseek(req->space_fd, d, SEEK_HOLE); + if (h < 0) { + perror(_("finding end of data in space capture file")); + return h; + } + if (h > end) + h = end; + + *datapos = d; + *length = h - d; + return 1; +} + +/* Filesystem space usage queries */ + +/* Allocate the structures needed for a fsmap query. */ +static void +start_fsmap_query( + struct clearspace_req *req, + dev_t dev, + unsigned long long physical, + unsigned long long length) +{ + struct fsmap_head *mhead = req->mhead; + + assert(req->mhead->fmh_count == 0); + memset(mhead, 0, sizeof(struct fsmap_head)); + mhead->fmh_count = QUERY_BATCH_SIZE; + mhead->fmh_keys[0].fmr_device = dev; + mhead->fmh_keys[0].fmr_physical = physical; + mhead->fmh_keys[1].fmr_device = dev; + mhead->fmh_keys[1].fmr_physical = physical + length; + mhead->fmh_keys[1].fmr_owner = ULLONG_MAX; + mhead->fmh_keys[1].fmr_flags = UINT_MAX; + mhead->fmh_keys[1].fmr_offset = ULLONG_MAX; + + trace_fsmap(req, "dev %u:%u physical 0x%llx bytecount 0x%llx highkey 0x%llx", + major(dev), minor(dev), + (unsigned long long)physical, + (unsigned long long)length, + (unsigned long long)mhead->fmh_keys[1].fmr_physical); + req->trace_indent++; +} + +static inline void +end_fsmap_query( + struct clearspace_req *req) +{ + req->trace_indent--; + req->mhead->fmh_count = 0; +} + +/* Set us up for the next run_fsmap_query, or return false. */ +static inline bool +advance_fsmap_cursor(struct fsmap_head *mhead) +{ + struct fsmap *mrec; + + mrec = &mhead->fmh_recs[mhead->fmh_entries - 1]; + if (mrec->fmr_flags & FMR_OF_LAST) + return false; + + fsmap_advance(mhead); + return true; +} + +/* + * Run a GETFSMAP query. Returns 1 if there are rows, 0 if there are no rows, + * or -1 for error. + */ +static inline int +run_fsmap_query( + struct clearspace_req *req) +{ + struct fsmap_head *mhead = req->mhead; + int ret; + + if (mhead->fmh_entries > 0 && !advance_fsmap_cursor(mhead)) + return 0; + + trace_fsmap(req, + "ioctl dev %u:%u physical 0x%llx length 0x%llx highkey 0x%llx", + major(mhead->fmh_keys[0].fmr_device), + minor(mhead->fmh_keys[0].fmr_device), + (unsigned long long)mhead->fmh_keys[0].fmr_physical, + (unsigned long long)mhead->fmh_keys[0].fmr_length, + (unsigned long long)mhead->fmh_keys[1].fmr_physical); + + ret = ioctl(req->xfd->fd, FS_IOC_GETFSMAP, mhead); + if (ret) { + perror(_("querying fsmap data")); + return -1; + } + + if (!(mhead->fmh_oflags & FMH_OF_DEV_T)) { + fprintf(stderr, _("fsmap does not return dev_t.\n")); + return -1; + } + + if (mhead->fmh_entries == 0) + return 0; + + return 1; +} + +#define for_each_fsmap_row(req, rec) \ + for ((rec) = (req)->mhead->fmh_recs; \ + (rec) < (req)->mhead->fmh_recs + (req)->mhead->fmh_entries; \ + (rec)++) + +/* Allocate the structures needed for a fsrefcounts query. */ +static void +start_fsrefs_query( + struct clearspace_req *req, + dev_t dev, + unsigned long long physical, + unsigned long long length) +{ + struct fsrefs_head *rhead = req->rhead; + + assert(req->rhead->fch_count == 0); + memset(rhead, 0, sizeof(struct fsrefs_head)); + rhead->fch_count = QUERY_BATCH_SIZE; + rhead->fch_keys[0].fcr_device = dev; + rhead->fch_keys[0].fcr_physical = physical; + rhead->fch_keys[1].fcr_device = dev; + rhead->fch_keys[1].fcr_physical = physical + length; + rhead->fch_keys[1].fcr_owners = ULLONG_MAX; + rhead->fch_keys[1].fcr_flags = UINT_MAX; + + trace_fsrefs(req, "dev %u:%u physical 0x%llx bytecount 0x%llx highkey 0x%llx", + major(dev), minor(dev), + (unsigned long long)physical, + (unsigned long long)length, + (unsigned long long)rhead->fch_keys[1].fcr_physical); + req->trace_indent++; +} + +static inline void +end_fsrefs_query( + struct clearspace_req *req) +{ + req->trace_indent--; + req->rhead->fch_count = 0; +} + +/* Set us up for the next run_fsrefs_query, or return false. */ +static inline bool +advance_fsrefs_query(struct fsrefs_head *rhead) +{ + struct fsrefs *rrec; + + rrec = &rhead->fch_recs[rhead->fch_entries - 1]; + if (rrec->fcr_flags & FCR_OF_LAST) + return false; + + fsrefs_advance(rhead); + return true; +} + +/* + * Run a GETFSREFCOUNTS query. Returns 1 if there are rows, 0 if there are + * no rows, or -1 for error. + */ +static inline int +run_fsrefs_query( + struct clearspace_req *req) +{ + struct fsrefs_head *rhead = req->rhead; + int ret; + + if (rhead->fch_entries > 0 && !advance_fsrefs_query(rhead)) + return 0; + + trace_fsrefs(req, + "ioctl dev %u:%u physical 0x%llx length 0x%llx highkey 0x%llx", + major(rhead->fch_keys[0].fcr_device), + minor(rhead->fch_keys[0].fcr_device), + (unsigned long long)rhead->fch_keys[0].fcr_physical, + (unsigned long long)rhead->fch_keys[0].fcr_length, + (unsigned long long)rhead->fch_keys[1].fcr_physical); + + ret = ioctl(req->xfd->fd, FS_IOC_GETFSREFCOUNTS, rhead); + if (ret) { + perror(_("querying refcount data")); + abort(); + return -1; + } + + if (!(rhead->fch_oflags & FCH_OF_DEV_T)) { + fprintf(stderr, _("fsrefcounts does not return dev_t.\n")); + return -1; + } + + if (rhead->fch_entries == 0) + return 0; + + return 1; +} + +#define for_each_fsref_row(req, rec) \ + for ((rec) = (req)->rhead->fch_recs; \ + (rec) < (req)->rhead->fch_recs + (req)->rhead->fch_entries; \ + (rec)++) + +/* Allocate the structures needed for a bmapx query. */ +static void +start_bmapx_query( + struct clearspace_req *req, + unsigned int fork, + unsigned long long pos, + unsigned long long length) +{ + struct getbmapx *bhead = req->bhead; + + assert(fork == BMV_IF_ATTRFORK || fork == BMV_IF_COWFORK || !fork); + assert(req->bhead->bmv_count == 0); + + memset(bhead, 0, sizeof(struct getbmapx)); + bhead[0].bmv_offset = BTOBB(pos); + bhead[0].bmv_length = BTOBB(length); + bhead[0].bmv_count = QUERY_BATCH_SIZE + 1; + bhead[0].bmv_iflags = fork | BMV_IF_PREALLOC | BMV_IF_DELALLOC; + + trace_bmapx(req, "%s pos 0x%llx bytecount 0x%llx", + fork == BMV_IF_COWFORK ? "cow" : fork == BMV_IF_ATTRFORK ? "attr" : "data", + (unsigned long long)BBTOB(bhead[0].bmv_offset), + (unsigned long long)BBTOB(bhead[0].bmv_length)); + req->trace_indent++; +} + +static inline void +end_bmapx_query( + struct clearspace_req *req) +{ + req->trace_indent--; + req->bhead->bmv_count = 0; +} + +/* Set us up for the next run_bmapx_query, or return false. */ +static inline bool +advance_bmapx_query(struct getbmapx *bhead) +{ + struct getbmapx *brec; + unsigned long long next_offset; + unsigned long long end = bhead->bmv_offset + bhead->bmv_length; + + brec = &bhead[bhead->bmv_entries]; + if (brec->bmv_oflags & BMV_OF_LAST) + return false; + + next_offset = brec->bmv_offset + brec->bmv_length; + if (next_offset > end) + return false; + + bhead->bmv_offset = next_offset; + bhead->bmv_length = end - next_offset; + return true; +} + +/* + * Run a GETBMAPX query. Returns 1 if there are rows, 0 if there are no rows, + * or -1 for error. + */ +static inline int +run_bmapx_query( + struct clearspace_req *req, + int fd) +{ + struct getbmapx *bhead = req->bhead; + unsigned int fork; + int ret; + + if (bhead->bmv_entries > 0 && !advance_bmapx_query(bhead)) + return 0; + + fork = bhead[0].bmv_iflags & (BMV_IF_COWFORK | BMV_IF_ATTRFORK); + trace_bmapx(req, "ioctl %s pos 0x%llx bytecount 0x%llx", + fork == BMV_IF_COWFORK ? "cow" : fork == BMV_IF_ATTRFORK ? "attr" : "data", + (unsigned long long)BBTOB(bhead[0].bmv_offset), + (unsigned long long)BBTOB(bhead[0].bmv_length)); + + ret = ioctl(fd, XFS_IOC_GETBMAPX, bhead); + if (ret) { + perror(_("querying bmapx data")); + return -1; + } + + if (bhead->bmv_entries == 0) + return 0; + + return 1; +} + +#define for_each_bmapx_row(req, rec) \ + for ((rec) = (req)->bhead + 1; \ + (rec) < (req)->bhead + 1 + (req)->bhead->bmv_entries; \ + (rec)++) + +static inline void +csp_dump_bmapx_row( + struct clearspace_req *req, + unsigned int nr, + const struct getbmapx *brec) +{ + if (brec->bmv_block == -1) { + trace_dumpfile(req, "[%u]: pos 0x%llx len 0x%llx hole", + nr, + (unsigned long long)BBTOB(brec->bmv_offset), + (unsigned long long)BBTOB(brec->bmv_length)); + return; + } + + if (brec->bmv_block == -2) { + trace_dumpfile(req, "[%u]: pos 0x%llx len 0x%llx delalloc", + nr, + (unsigned long long)BBTOB(brec->bmv_offset), + (unsigned long long)BBTOB(brec->bmv_length)); + return; + } + + trace_dumpfile(req, "[%u]: pos 0x%llx len 0x%llx phys 0x%llx flags 0x%x", + nr, + (unsigned long long)BBTOB(brec->bmv_offset), + (unsigned long long)BBTOB(brec->bmv_length), + (unsigned long long)BBTOB(brec->bmv_block), + brec->bmv_oflags); +} + +static inline void +csp_dump_bmapx( + struct clearspace_req *req, + int fd, + unsigned int indent, + const char *tag) +{ + unsigned int nr; + int ret; + + trace_dumpfile(req, "DUMP BMAP OF DATA FORK %s", tag); + start_bmapx_query(req, 0, req->start, req->length); + nr = 0; + while ((ret = run_bmapx_query(req, fd)) > 0) { + struct getbmapx *brec; + + for_each_bmapx_row(req, brec) { + csp_dump_bmapx_row(req, nr++, brec); + if (nr > 10) + goto dump_cow; + } + } + +dump_cow: + end_bmapx_query(req); + trace_dumpfile(req, "DUMP BMAP OF COW FORK %s", tag); + start_bmapx_query(req, BMV_IF_COWFORK, req->start, req->length); + nr = 0; + while ((ret = run_bmapx_query(req, fd)) > 0) { + struct getbmapx *brec; + + for_each_bmapx_row(req, brec) { + csp_dump_bmapx_row(req, nr++, brec); + if (nr > 10) + goto dump_attr; + } + } + +dump_attr: + end_bmapx_query(req); + trace_dumpfile(req, "DUMP BMAP OF ATTR FORK %s", tag); + start_bmapx_query(req, BMV_IF_ATTRFORK, req->start, req->length); + nr = 0; + while ((ret = run_bmapx_query(req, fd)) > 0) { + struct getbmapx *brec; + + for_each_bmapx_row(req, brec) { + csp_dump_bmapx_row(req, nr++, brec); + if (nr > 10) + goto stop; + } + } + +stop: + end_bmapx_query(req); + trace_dumpfile(req, "DONE DUMPING %s", tag); +} + +/* Return the first bmapx for the given file range. */ +static int +bmapx_one( + struct clearspace_req *req, + int fd, + unsigned long long pos, + unsigned long long length, + struct getbmapx *brec) +{ + struct getbmapx bhead[2]; + int ret; + + memset(bhead, 0, sizeof(struct getbmapx) * 2); + bhead[0].bmv_offset = BTOBB(pos); + bhead[0].bmv_length = BTOBB(length); + bhead[0].bmv_count = 2; + bhead[0].bmv_iflags = BMV_IF_PREALLOC | BMV_IF_DELALLOC; + + ret = ioctl(fd, XFS_IOC_GETBMAPX, bhead); + if (ret) { + perror(_("simple bmapx query")); + return -1; + } + + if (bhead->bmv_entries > 0) { + memcpy(brec, &bhead[1], sizeof(struct getbmapx)); + return 0; + } + + memset(brec, 0, sizeof(struct getbmapx)); + brec->bmv_offset = pos; + brec->bmv_block = -1; /* hole */ + brec->bmv_length = length; + return 0; +} + +/* Constrain space map records. */ +static void +__trim_fsmap( + uint64_t start, + uint64_t length, + struct fsmap *fsmap) +{ + unsigned long long delta, end; + bool need_off; + + need_off = (fsmap->fmr_flags & (FMR_OF_EXTENT_MAP | + FMR_OF_SPECIAL_OWNER)); + + if (fsmap->fmr_physical < start) { + delta = start - fsmap->fmr_physical; + fsmap->fmr_physical = start; + fsmap->fmr_length -= delta; + if (need_off) + fsmap->fmr_offset += delta; + } + + end = fsmap->fmr_physical + fsmap->fmr_length; + if (end > start + length) { + delta = end - (start + length); + fsmap->fmr_length -= delta; + } +} + +static inline void +trim_target_fsmap(const struct clearspace_tgt *tgt, struct fsmap *fsmap) +{ + return __trim_fsmap(tgt->start, tgt->length, fsmap); +} + +static inline void +trim_request_fsmap(const struct clearspace_req *req, struct fsmap *fsmap) +{ + return __trim_fsmap(req->start, req->length, fsmap); +} + +/* Actual space clearing code */ + +/* + * Map all the free space in the region that we're clearing to the space + * catcher file. + */ +static int +csp_grab_free_space( + struct clearspace_req *req) +{ + int ret; + + trace_grabfree(req, "start 0x%llx length 0x%llx", + (unsigned long long)req->start, + (unsigned long long)req->length); + + ret = fallocate(req->space_fd, FALLOC_FL_MAP_FREE_SPACE, req->start, + req->length); + if (ret) { + perror(_("map free space to space capture file")); + return -1; + } + + return 0; +} + +/* + * Rank a refcount record. We prefer to tackle highly shared and longer + * extents first. + */ +static inline unsigned long long +csp_space_prio( + const struct xfs_fsop_geom *g, + const struct fsrefs *p) +{ + unsigned long long blocks = p->fcr_length / g->blocksize; + unsigned long long ret = blocks * p->fcr_owners; + + if (ret < blocks || ret < p->fcr_owners) + return UINT64_MAX; + return ret; +} + +/* Make the current refcount record the clearing target if desirable. */ +static void +csp_adjust_target( + struct clearspace_req *req, + struct clearspace_tgt *target, + const struct fsrefs *rec, + unsigned long long prio) +{ + if (prio < target->prio) + return; + if (prio == target->prio && + rec->fcr_length <= target->length) + return; + + /* Ignore results that go beyond the end of what we wanted. */ + if (rec->fcr_physical >= req->start + req->length) + return; + + /* Ignore regions that we already tried to clear. */ + if (bitmap_test(req->visited, rec->fcr_physical, rec->fcr_length)) + return; + + trace_target(req, + "set target, prio 0x%llx -> 0x%llx phys 0x%llx bytecount 0x%llx", + target->prio, prio, + (unsigned long long)rec->fcr_physical, + (unsigned long long)rec->fcr_length); + + target->start = rec->fcr_physical; + target->length = rec->fcr_length; + target->owners = rec->fcr_owners; + target->prio = prio; +} + +/* + * Decide if this refcount record maps to extents that are sufficiently + * interesting to target. + */ +static int +csp_evaluate_refcount( + struct clearspace_req *req, + const struct fsrefs *rrec, + struct clearspace_tgt *target) +{ + const struct xfs_fsop_geom *fsgeom = &req->xfd->fsgeom; + unsigned long long prio = csp_space_prio(fsgeom, rrec); + int ret; + + if (rrec->fcr_device != req->dev) + return 0; + + if (prio < target->prio) + return 0; + + /* + * XFS only supports sharing data blocks. If there's more than one + * owner, we know that we can easily move the blocks. + */ + if (rrec->fcr_owners > 1) { + csp_adjust_target(req, target, rrec, prio); + return 0; + } + + /* + * Otherwise, this extent has single owners. Walk the fsmap records to + * figure out if they're movable or not. + */ + start_fsmap_query(req, rrec->fcr_device, rrec->fcr_physical, + rrec->fcr_length); + while ((ret = run_fsmap_query(req)) > 0) { + struct fsmap *mrec; + uint64_t next_phys = 0; + + for_each_fsmap_row(req, mrec) { + struct fsrefs fake_rec = { }; + + trace_fsmap_rec(req, CSP_TRACE_TARGET, mrec); + + if (mrec->fmr_device != rrec->fcr_device) + continue; + if (mrec->fmr_flags & FMR_OF_SPECIAL_OWNER) + continue; + if (csp_is_internal_owner(req, mrec->fmr_owner)) + continue; + + /* + * If the space has become shared since the fsrefs + * query, just skip this record. We might come back to + * it in a later iteration. + */ + if (mrec->fmr_physical < next_phys) + continue; + + /* Fake enough of a fsrefs to calculate the priority. */ + fake_rec.fcr_physical = mrec->fmr_physical; + fake_rec.fcr_length = mrec->fmr_length; + fake_rec.fcr_owners = 1; + prio = csp_space_prio(fsgeom, &fake_rec); + + /* Target unwritten extents first; they're cheap. */ + if (mrec->fmr_flags & FMR_OF_PREALLOC) + prio |= (1ULL << 63); + + csp_adjust_target(req, target, &fake_rec, prio); + + next_phys = mrec->fmr_physical + mrec->fmr_length; + } + } + end_fsmap_query(req); + + return ret; +} + +/* + * Given a range of storage to search, find the most appealing target for space + * clearing. If nothing suitable is found, the target will be zeroed. + */ +static int +csp_find_target( + struct clearspace_req *req, + struct clearspace_tgt *target) +{ + int ret; + + memset(target, 0, sizeof(struct clearspace_tgt)); + + start_fsrefs_query(req, req->dev, req->start, req->length); + while ((ret = run_fsrefs_query(req)) > 0) { + struct fsrefs *rrec; + + for_each_fsref_row(req, rrec) { + trace_fsrefs_rec(req, CSP_TRACE_TARGET, rrec); + ret = csp_evaluate_refcount(req, rrec, target); + if (ret) { + end_fsrefs_query(req); + return ret; + } + } + } + end_fsrefs_query(req); + + if (target->length != 0) { + /* + * Mark this extent visited so that we won't try again this + * round. + */ + trace_bitmap(req, "set filedata start 0x%llx length 0x%llx", + target->start, target->length); + ret = bitmap_set(req->visited, target->start, target->length); + if (ret) { + perror(_("marking file extent visited")); + return ret; + } + } + + return 0; +} + +/* Try to evacuate blocks by using online repair. */ +static int +csp_evac_file_metadata( + struct clearspace_req *req, + struct clearspace_tgt *target, + const struct fsmap *mrec, + int fd, + const struct xfs_bulkstat *bulkstat) +{ + struct xfs_scrub_metadata scrub = { + .sm_type = XFS_SCRUB_TYPE_PROBE, + .sm_flags = XFS_SCRUB_IFLAG_REPAIR | + XFS_SCRUB_IFLAG_FORCE_REBUILD, + }; + struct xfs_fd *xfd = req->xfd; + int ret; + + trace_xrebuild(req, + "ino 0x%llx pos 0x%llx bytecount 0x%llx phys 0x%llx flags 0x%llx", + (unsigned long long)mrec->fmr_owner, + (unsigned long long)mrec->fmr_offset, + (unsigned long long)mrec->fmr_physical, + (unsigned long long)mrec->fmr_length, + (unsigned long long)mrec->fmr_flags); + + if (fd == -1) { + scrub.sm_ino = mrec->fmr_owner; + scrub.sm_gen = bulkstat->bs_gen; + fd = xfd->fd; + } + + if (mrec->fmr_flags & FMR_OF_ATTR_FORK) { + if (mrec->fmr_flags & FMR_OF_EXTENT_MAP) + scrub.sm_type = XFS_SCRUB_TYPE_BMBTA; + else + scrub.sm_type = XFS_SCRUB_TYPE_XATTR; + } else if (mrec->fmr_flags & FMR_OF_EXTENT_MAP) { + scrub.sm_type = XFS_SCRUB_TYPE_BMBTD; + } else if (S_ISLNK(bulkstat->bs_mode)) { + scrub.sm_type = XFS_SCRUB_TYPE_SYMLINK; + } else if (S_ISDIR(bulkstat->bs_mode)) { + scrub.sm_type = XFS_SCRUB_TYPE_DIR; + } + + if (scrub.sm_type == XFS_SCRUB_TYPE_PROBE) + return 0; + + trace_xrebuild(req, "ino 0x%llx gen 0x%x type %u", + (unsigned long long)mrec->fmr_owner, + (unsigned int)bulkstat->bs_gen, + (unsigned int)scrub.sm_type); + + ret = ioctl(fd, XFS_IOC_SCRUB_METADATA, &scrub); + if (ret) { + fprintf(stderr, + _("evacuating inode 0x%llx metadata type %u: %s\n"), + (unsigned long long)mrec->fmr_owner, + scrub.sm_type, strerror(errno)); + return -1; + } + + target->evacuated++; + return 0; +} + +/* + * Open an inode via handle. Returns a file descriptor, -2 if the file is + * gone, or -1 on error. + */ +static int +csp_open_by_handle( + struct clearspace_req *req, + int oflags, + uint64_t ino, + uint32_t gen) +{ + struct xfs_handle handle = { }; + struct xfs_fsop_handlereq hreq = { + .oflags = oflags | O_NOATIME | O_NOFOLLOW | + O_NOCTTY | O_LARGEFILE, + .ihandle = &handle, + .ihandlen = sizeof(handle), + }; + int ret; + + memcpy(&handle.ha_fsid, req->fshandle, sizeof(handle.ha_fsid)); + handle.ha_fid.fid_len = sizeof(xfs_fid_t) - + sizeof(handle.ha_fid.fid_len); + handle.ha_fid.fid_pad = 0; + handle.ha_fid.fid_ino = ino; + handle.ha_fid.fid_gen = gen; + + /* + * Since we extracted the fshandle from the open file instead of using + * path_to_fshandle, the fsid cache doesn't know about the fshandle. + * Construct the open by handle request manually. + */ + ret = ioctl(req->xfd->fd, XFS_IOC_OPEN_BY_HANDLE, &hreq); + if (ret < 0) { + if (errno == ENOENT || errno == EINVAL) + return -2; + + fprintf(stderr, _("open inode 0x%llx: %s\n"), + (unsigned long long)ino, + strerror(errno)); + return -1; + } + + return ret; +} + +/* + * Open a file for evacuation. Returns a positive errno on error; a fd in @fd + * if the caller is supposed to do something; or @fd == -1 if there's nothing + * further to do. + */ +static int +csp_evac_open( + struct clearspace_req *req, + struct clearspace_tgt *target, + const struct fsmap *mrec, + struct xfs_bulkstat *bulkstat, + int oflags, + int *fd) +{ + struct xfs_bulkstat __bs; + int target_fd; + int ret; + + *fd = -1; + + if (csp_is_internal_owner(req, mrec->fmr_owner) || + (mrec->fmr_flags & FMR_OF_SPECIAL_OWNER)) + goto nothing_to_do; + + if (bulkstat == NULL) + bulkstat = &__bs; + + /* + * Snapshot this file so that we can perform a fresh-only exchange. + * For other types of files we just skip to the evacuation step. + */ + ret = -xfrog_bulkstat_single(req->xfd, mrec->fmr_owner, 0, bulkstat); + if (ret) { + if (ret == ENOENT || ret == EINVAL) + goto nothing_to_do; + + fprintf(stderr, _("bulkstat inode 0x%llx: %s\n"), + (unsigned long long)mrec->fmr_owner, + strerror(ret)); + return ret; + } + + /* + * If we get stats for a different inode, the file may have been freed + * out from under us and there's nothing to do. + */ + if (bulkstat->bs_ino != mrec->fmr_owner) + goto nothing_to_do; + + /* + * We're only allowed to open regular files and directories via handle + * so jump to online rebuild for all other file types. + */ + if (!S_ISREG(bulkstat->bs_mode) && !S_ISDIR(bulkstat->bs_mode)) + return csp_evac_file_metadata(req, target, mrec, -1, + bulkstat); + + if (S_ISDIR(bulkstat->bs_mode)) + oflags = O_RDONLY; + + target_fd = csp_open_by_handle(req, oflags, mrec->fmr_owner, + bulkstat->bs_gen); + if (target_fd == -2) + goto nothing_to_do; + if (target_fd < 0) + return -target_fd; + + /* + * Exchange only works for regular file data blocks. If that isn't the + * case, our only recourse is online rebuild. + */ + if (S_ISDIR(bulkstat->bs_mode) || + (mrec->fmr_flags & (FMR_OF_ATTR_FORK | FMR_OF_EXTENT_MAP))) { + int ret2; + + ret = csp_evac_file_metadata(req, target, mrec, target_fd, + bulkstat); + ret2 = close(target_fd); + if (!ret && ret2) + ret = ret2; + return ret; + } + + *fd = target_fd; + return 0; + +nothing_to_do: + target->try_again = true; + return 0; +} + +/* Unshare the space in the work file that we're using for deduplication. */ +static int +csp_unshare_workfile( + struct clearspace_req *req, + unsigned long long start, + unsigned long long length) +{ + int ret; + + trace_falloc(req, "funshare workfd pos 0x%llx bytecount 0x%llx", + start, length); + + ret = fallocate(req->work_fd, FALLOC_FL_UNSHARE_RANGE, start, length); + if (ret) { + perror(_("unsharing work file")); + return ret; + } + + ret = fsync(req->work_fd); + if (ret) { + perror(_("syncing work file")); + return ret; + } + + /* Make sure we didn't get any space within the clearing range. */ + start_bmapx_query(req, 0, start, length); + while ((ret = run_bmapx_query(req, req->work_fd)) > 0) { + struct getbmapx *brec; + + for_each_bmapx_row(req, brec) { + unsigned long long p, l; + + trace_bmapx_rec(req, CSP_TRACE_FALLOC, brec); + p = BBTOB(brec->bmv_block); + l = BBTOB(brec->bmv_length); + + if (p + l < req->start || p >= req->start + req->length) + continue; + + trace_prep(req, + "workfd has extent inside clearing range, phys 0x%llx fsbcount 0x%llx", + p, l); + end_bmapx_query(req); + return -1; + } + } + end_bmapx_query(req); + + return 0; +} + +/* Try to deduplicate every block in the fdr request, if we can. */ +static int +csp_evac_dedupe_loop( + struct clearspace_req *req, + struct clearspace_tgt *target, + unsigned long long ino, + int max_reqlen) +{ + struct file_dedupe_range *fdr = req->fdr; + struct file_dedupe_range_info *info = &fdr->info[0]; + loff_t last_unshare_off = -1; + int ret; + + while (fdr->src_length > 0) { + struct getbmapx brec; + bool same; + unsigned int old_reqlen = fdr->src_length; + + if (max_reqlen && fdr->src_length > max_reqlen) + fdr->src_length = max_reqlen; + + trace_dedupe(req, "ino 0x%llx pos 0x%llx bytecount 0x%llx", + ino, + (unsigned long long)info->dest_offset, + (unsigned long long)fdr->src_length); + + ret = bmapx_one(req, req->work_fd, fdr->src_offset, + fdr->src_length, &brec); + if (ret) + return ret; + + trace_dedupe(req, "workfd pos 0x%llx phys 0x%llx", + (unsigned long long)fdr->src_offset, + (unsigned long long)BBTOB(brec.bmv_block)); + + ret = deduperange(req->work_fd, fdr, &same); + if (ret == ENOSPC && last_unshare_off < fdr->src_offset) { + req->trace_indent++; + trace_dedupe(req, "funshare workfd at phys 0x%llx", + (unsigned long long)fdr->src_offset); + /* + * If we ran out of space, it's possible that we have + * reached the maximum sharing factor of the blocks in + * the work file. Try unsharing the range of the work + * file to get a singly-owned range and loop again. + */ + ret = csp_unshare_workfile(req, fdr->src_offset, + fdr->src_length); + req->trace_indent--; + if (ret) + return ret; + + ret = fsync(req->work_fd); + if (ret) { + perror(_("sync after unshare work file")); + return ret; + } + + last_unshare_off = fdr->src_offset; + fdr->src_length = old_reqlen; + continue; + } + if (ret) { + fprintf(stderr, _("evacuating inode 0x%llx: %s\n"), + ino, strerror(ret)); + return ret; + } + + if (same) { + req->trace_indent++; + trace_dedupe(req, + "evacuated ino 0x%llx pos 0x%llx bytecount 0x%llx", + ino, + (unsigned long long)info->dest_offset, + (unsigned long long)info->bytes_deduped); + req->trace_indent--; + + target->evacuated++; + } else { + req->trace_indent++; + trace_dedupe(req, + "failed evac ino 0x%llx pos 0x%llx bytecount 0x%llx", + ino, + (unsigned long long)info->dest_offset, + (unsigned long long)fdr->src_length); + req->trace_indent--; + + target->try_again = true; + + /* + * If we aren't single-stepping the deduplication, + * stop early so that the caller goes into single-step + * mode. + */ + if (!max_reqlen) { + fdr->src_length = old_reqlen; + return 0; + } + + /* Contents changed, move on to the next block. */ + info->bytes_deduped = fdr->src_length; + } + fdr->src_length = old_reqlen; + + fdr->src_offset += info->bytes_deduped; + info->dest_offset += info->bytes_deduped; + fdr->src_length -= info->bytes_deduped; + } + + return 0; +} + +/* + * Evacuate one fsmapping by using dedupe to remap data stored in the target + * range to a copy stored in the work file. + */ +static int +csp_evac_dedupe_fsmap( + struct clearspace_req *req, + struct clearspace_tgt *target, + const struct fsmap *mrec) +{ + struct file_dedupe_range *fdr = req->fdr; + struct file_dedupe_range_info *info = &fdr->info[0]; + bool can_single_step; + int target_fd; + int ret, ret2; + + if (mrec->fmr_device != req->dev) { + fprintf(stderr, _("wrong fsmap device in results.\n")); + return -1; + } + + ret = csp_evac_open(req, target, mrec, NULL, O_RDONLY, &target_fd); + if (ret || target_fd < 0) + return ret; + + /* + * Use dedupe to try to shift the target file's mappings to use the + * copy of the data that's in the work file. + */ + fdr->src_offset = mrec->fmr_physical; + fdr->src_length = mrec->fmr_length; + fdr->dest_count = 1; + info->dest_fd = target_fd; + info->dest_offset = mrec->fmr_offset; + + can_single_step = mrec->fmr_length > req->xfd->fsgeom.blocksize; + + /* First we try to do the entire thing all at once. */ + ret = csp_evac_dedupe_loop(req, target, mrec->fmr_owner, 0); + if (ret) + goto out_fd; + + /* If there's any work left, try again one block at a time. */ + if (can_single_step && fdr->src_length > 0) { + ret = csp_evac_dedupe_loop(req, target, mrec->fmr_owner, + req->xfd->fsgeom.blocksize); + if (ret) + goto out_fd; + } + +out_fd: + ret2 = close(target_fd); + if (!ret && ret2) + ret = ret2; + return ret; +} + +/* Use deduplication to remap data extents away from where we're clearing. */ +static int +csp_evac_dedupe( + struct clearspace_req *req, + struct clearspace_tgt *target) +{ + int ret; + + start_fsmap_query(req, req->dev, target->start, target->length); + while ((ret = run_fsmap_query(req)) > 0) { + struct fsmap *mrec; + + for_each_fsmap_row(req, mrec) { + trace_fsmap_rec(req, CSP_TRACE_DEDUPE, mrec); + trim_target_fsmap(target, mrec); + + req->trace_indent++; + ret = csp_evac_dedupe_fsmap(req, target, mrec); + req->trace_indent--; + if (ret) + goto out; + + ret = csp_grab_free_space(req); + if (ret) + goto out; + } + } + +out: + end_fsmap_query(req); + if (ret) + trace_dedupe(req, "ret %d", ret); + return ret; +} + +#define BUFFERCOPY_BUFSZ 65536 + +/* + * Use a memory buffer to copy part of src_fd to dst_fd, or return an errno. */ +static int +csp_buffercopy( + struct clearspace_req *req, + int src_fd, + loff_t src_off, + int dst_fd, + loff_t dst_off, + loff_t len) +{ + int ret = 0; + + while (len > 0) { + size_t count = min(BUFFERCOPY_BUFSZ, len); + ssize_t bytes_read, bytes_written; + + bytes_read = pread(src_fd, req->buf, count, src_off); + if (bytes_read < 0) { + ret = errno; + break; + } + + bytes_written = pwrite(dst_fd, req->buf, bytes_read, dst_off); + if (bytes_written < 0) { + ret = errno; + break; + } + + src_off += bytes_written; + dst_off += bytes_written; + len -= bytes_written; + } + + return ret; +} + +/* + * Prepare the work file to assist in evacuating file data by copying the + * contents of the frozen space into the work file. + */ +static int +csp_prepare_for_dedupe( + struct clearspace_req *req) +{ + struct file_clone_range fcr; + struct stat statbuf; + loff_t datapos = 0; + loff_t length = 0; + int ret; + + ret = fstat(req->space_fd, &statbuf); + if (ret) { + perror(_("space capture file")); + return ret; + } + + ret = ftruncate(req->work_fd, 0); + if (ret) { + perror(_("truncate work file")); + return ret; + } + + ret = ftruncate(req->work_fd, statbuf.st_size); + if (ret) { + perror(_("reset work file")); + return ret; + } + + /* Make a working copy of the frozen file data. */ + start_spacefd_iter(req); + while ((ret = spacefd_data_iter(req, &datapos, &length)) > 0) { + trace_prep(req, "clone spacefd data 0x%llx length 0x%llx", + (long long)datapos, (long long)length); + + fcr.src_fd = req->space_fd; + fcr.src_offset = datapos; + fcr.src_length = length; + fcr.dest_offset = datapos; + + ret = clonerange(req->work_fd, &fcr); + if (ret == ENOSPC) { + req->trace_indent++; + trace_prep(req, + "falling back to buffered copy at 0x%llx", + (long long)datapos); + req->trace_indent--; + ret = csp_buffercopy(req, req->space_fd, datapos, + req->work_fd, datapos, length); + } + if (ret) { + perror( + _("copying space capture file contents to work file")); + return ret; + } + } + end_spacefd_iter(req); + if (ret < 0) + return ret; + + /* + * Unshare the work file so that it contains an identical copy of the + * contents of the space capture file but mapped to different blocks. + * This is key to using dedupe to migrate file space away from the + * requested region. + */ + req->trace_indent++; + ret = csp_unshare_workfile(req, req->start, req->length); + req->trace_indent--; + return ret; +} + +/* + * Evacuate one fsmapping by using dedupe to remap data stored in the target + * range to a copy stored in the work file. + */ +static int +csp_evac_exchange_fsmap( + struct clearspace_req *req, + struct clearspace_tgt *target, + const struct fsmap *mrec) +{ + struct xfs_bulkstat bulkstat; + struct file_xchg_range xchg = { }; + struct getbmapx brec; + int target_fd; + int ret, ret2; + + if (mrec->fmr_device != req->dev) { + fprintf(stderr, _("wrong fsmap device in results.\n")); + return -1; + } + + ret = csp_evac_open(req, target, mrec, &bulkstat, O_RDWR, &target_fd); + if (ret || target_fd < 0) + return ret; + + ret = ftruncate(req->work_fd, 0); + if (ret) { + perror(_("truncating work file")); + goto out_fd; + } + + /* + * Copy the data from the original file to the work file. We assume + * that the work file will end up with different data blocks and that + * they're outside of the requested range. + */ + ret = csp_buffercopy(req, target_fd, mrec->fmr_offset, req->work_fd, + mrec->fmr_offset, mrec->fmr_length); + if (ret) { + fprintf(stderr, _("copying target file to work file: %s\n"), + strerror(ret)); + goto out_fd; + } + + ret = fsync(req->work_fd); + if (ret) { + perror(_("flush work file for fiexchange")); + goto out_fd; + } + + ret = bmapx_one(req, req->work_fd, mrec->fmr_physical, + mrec->fmr_length, &brec); + if (ret) + return ret; + + trace_fiexchange(req, "workfd pos 0x%llx phys 0x%llx", + (unsigned long long)mrec->fmr_physical, + (unsigned long long)BBTOB(brec.bmv_block)); + + /* + * Exchange the mappings, with the freshness check enabled. This + * should result in the target file being switched to new blocks unless + * it has changed, in which case we bounce out and find a new target. + */ + xfrog_file_exchange_prep(NULL, FILE_XCHG_RANGE_NONATOMIC, + mrec->fmr_offset, req->work_fd, mrec->fmr_offset, + mrec->fmr_length, &xchg); + xfrog_file_exchange_require_file2_fresh(&xchg, &bulkstat); + ret = exchangerange(target_fd, &xchg); + if (ret) { + if (ret == EBUSY) { + req->trace_indent++; + trace_fiexchange(req, + "failed evac ino 0x%llx pos 0x%llx bytecount 0x%llx", + bulkstat.bs_ino, + (unsigned long long)mrec->fmr_offset, + (unsigned long long)mrec->fmr_length); + req->trace_indent--; + target->try_again = true; + } else { + fprintf(stderr, + _("exchanging target and work file contents: %s\n"), + strerror(ret)); + } + goto out_fd; + } + + req->trace_indent++; + trace_fiexchange(req, + "evacuated ino 0x%llx pos 0x%llx bytecount 0x%llx", + bulkstat.bs_ino, + (unsigned long long)mrec->fmr_offset, + (unsigned long long)mrec->fmr_length); + req->trace_indent--; + target->evacuated++; + +out_fd: + ret2 = close(target_fd); + if (!ret && ret2) + ret = ret2; + return ret; +} + +/* + * Try to evacuate all data blocks in the target region by copying the contents + * to a new file and exchanging the extents. + */ +static int +csp_evac_exchange( + struct clearspace_req *req, + struct clearspace_tgt *target) +{ + int ret; + + start_fsmap_query(req, req->dev, target->start, target->length); + while ((ret = run_fsmap_query(req)) > 0) { + struct fsmap *mrec; + + for_each_fsmap_row(req, mrec) { + trace_fsmap_rec(req, CSP_TRACE_FIEXCHANGE, mrec); + trim_target_fsmap(target, mrec); + + req->trace_indent++; + ret = csp_evac_exchange_fsmap(req, target, mrec); + req->trace_indent--; + if (ret) + goto out; + + ret = csp_grab_free_space(req); + if (ret) + goto out; + } + } +out: + end_fsmap_query(req); + if (ret) + trace_fiexchange(req, "ret %d", ret); + return ret; +} + +/* Try to evacuate blocks by using online repair to rebuild AG metadata. */ +static int +csp_evac_ag_metadata( + struct clearspace_req *req, + struct clearspace_tgt *target, + uint32_t agno, + uint32_t mask) +{ + struct xfs_scrub_metadata scrub = { + .sm_flags = XFS_SCRUB_IFLAG_REPAIR | + XFS_SCRUB_IFLAG_FORCE_REBUILD, + }; + unsigned int i; + int ret; + + trace_xrebuild(req, "agno 0x%x mask 0x%x", + (unsigned int)agno, + (unsigned int)mask); + + for (i = XFS_SCRUB_TYPE_AGFL; i < XFS_SCRUB_TYPE_REFCNTBT; i++) { + + if (!(mask & (1U << i))) + continue; + + scrub.sm_type = i; + + req->trace_indent++; + trace_xrebuild(req, "agno %u type %u", + (unsigned int)agno, + (unsigned int)scrub.sm_type); + req->trace_indent--; + + ret = ioctl(req->xfd->fd, XFS_IOC_SCRUB_METADATA, &scrub); + if (ret) { + if (errno == ENOENT || errno == ENOSPC) + continue; + fprintf(stderr, _("rebuilding ag %u type %u: %s\n"), + (unsigned int)agno, scrub.sm_type, + strerror(errno)); + return -1; + } + + target->evacuated++; + + ret = csp_grab_free_space(req); + if (ret) + return ret; + } + + return 0; +} + +/* Compute a scrub mask for a fsmap special owner. */ +static uint32_t +fsmap_owner_to_scrub_mask(__u64 owner) +{ + switch (owner) { + case XFS_FMR_OWN_FREE: + case XFS_FMR_OWN_UNKNOWN: + case XFS_FMR_OWN_FS: + case XFS_FMR_OWN_LOG: + /* can't move these */ + return 0; + case XFS_FMR_OWN_AG: + return (1U << XFS_SCRUB_TYPE_BNOBT) | + (1U << XFS_SCRUB_TYPE_CNTBT) | + (1U << XFS_SCRUB_TYPE_AGFL) | + (1U << XFS_SCRUB_TYPE_RMAPBT); + case XFS_FMR_OWN_INOBT: + return (1U << XFS_SCRUB_TYPE_INOBT) | + (1U << XFS_SCRUB_TYPE_FINOBT); + case XFS_FMR_OWN_REFC: + return (1U << XFS_SCRUB_TYPE_REFCNTBT); + case XFS_FMR_OWN_INODES: + case XFS_FMR_OWN_COW: + /* don't know how to get rid of these */ + return 0; + case XFS_FMR_OWN_DEFECTIVE: + /* good, get rid of it */ + return 0; + default: + return 0; + } +} + +/* Try to clear all per-AG metadata from the requested range. */ +static int +csp_evac_fs_metadata( + struct clearspace_req *req, + struct clearspace_tgt *target, + bool *cleared_anything) +{ + uint32_t curr_agno = -1U; + uint32_t curr_mask = 0; + int ret = 0; + + if (req->realtime) + return 0; + + start_fsmap_query(req, req->dev, target->start, target->length); + while ((ret = run_fsmap_query(req)) > 0) { + struct fsmap *mrec; + + for_each_fsmap_row(req, mrec) { + uint64_t daddr; + uint32_t agno; + uint32_t mask; + + if (mrec->fmr_device != req->dev) + continue; + if (!(mrec->fmr_flags & FMR_OF_SPECIAL_OWNER)) + continue; + + /* Ignore regions that we already tried to clear. */ + if (bitmap_test(req->visited, mrec->fmr_physical, + mrec->fmr_length)) + continue; + + mask = fsmap_owner_to_scrub_mask(mrec->fmr_owner); + if (!mask) + continue; + + trace_fsmap_rec(req, CSP_TRACE_XREBUILD, mrec); + + daddr = BTOBB(mrec->fmr_physical); + agno = cvt_daddr_to_agno(req->xfd, daddr); + + trace_xrebuild(req, + "agno 0x%x -> 0x%x mask 0x%x owner %lld", + curr_agno, agno, curr_mask, + (unsigned long long)mrec->fmr_owner); + + if (curr_agno == -1U) { + curr_agno = agno; + } else if (curr_agno != agno) { + ret = csp_evac_ag_metadata(req, target, + curr_agno, curr_mask); + if (ret) + goto out; + + *cleared_anything = true; + curr_agno = agno; + curr_mask = 0; + } + + /* Put this on the list and try to clear it once. */ + curr_mask |= mask; + ret = bitmap_set(req->visited, mrec->fmr_physical, + mrec->fmr_length); + if (ret) { + perror(_("marking metadata extent visited")); + goto out; + } + } + } + + if (curr_agno != -1U && curr_mask != 0) { + ret = csp_evac_ag_metadata(req, target, curr_agno, curr_mask); + if (ret) + goto out; + *cleared_anything = true; + } + + if (*cleared_anything) + trace_bitmap(req, "set metadata start 0x%llx length 0x%llx", + target->start, target->length); + +out: + end_fsmap_query(req); + if (ret) + trace_xrebuild(req, "ret %d", ret); + return ret; +} + +/* + * Check that at least the start of the mapping was frozen into the work file + * at the correct offset. Set @len to the number of bytes that were frozen. + * Returns -1 for error, zero if written extents are waiting to be mapped into + * the space capture file, or 1 if there's nothing to transfer to the space + * capture file. + */ +static int +csp_freeze_check_attempt( + struct clearspace_req *req, + const struct fsmap *mrec, + unsigned long long *len) +{ + struct getbmapx brec; + int ret; + + *len = 0; + + ret = bmapx_one(req, req->work_fd, mrec->fmr_physical, + mrec->fmr_length, &brec); + if (ret) + return ret; + + trace_freeze(req, + "does workfd pos 0x%llx len 0x%llx map to phys 0x%llx len 0x%llx?", + (unsigned long long)mrec->fmr_physical, + (unsigned long long)mrec->fmr_length, + (unsigned long long)BBTOB(brec.bmv_block), + (unsigned long long)BBTOB(brec.bmv_length)); + + /* freeze of an unwritten extent punches a hole in the work file. */ + if ((mrec->fmr_flags & FMR_OF_PREALLOC) && brec.bmv_block == -1) { + *len = BBTOB(brec.bmv_length); + return 1; + } + + /* + * freeze of a written extent must result in the same physical space + * being mapped into the work file. + */ + if (!(mrec->fmr_flags & FMR_OF_PREALLOC) && + BBTOB(brec.bmv_block) == mrec->fmr_physical) { + *len = BBTOB(brec.bmv_length); + return 0; + } + + /* + * We didn't find what we were looking for, which implies that the + * mapping changed out from under us. Punch out everything that could + * have been mapped into the work file. Set @len to zero and return so + * that we try again with the next mapping. + */ + + trace_falloc(req, "fpunch workfd pos 0x%llx bytecount 0x%llx", + (unsigned long long)mrec->fmr_physical, + (unsigned long long)mrec->fmr_length); + + ret = fallocate(req->work_fd, + FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, + mrec->fmr_physical, mrec->fmr_length); + if (ret) { + perror(_("resetting work file after failed freeze")); + return ret; + } + + return 1; +} + +/* + * Open a file to try to freeze whatever data is in the requested range. + * + * Returns nonzero on error. Returns zero and a file descriptor in @fd if the + * caller is supposed to do something; or returns zero and @fd == -1 if there's + * nothing to freeze. + */ +static int +csp_freeze_open( + struct clearspace_req *req, + const struct fsmap *mrec, + int *fd) +{ + struct xfs_bulkstat bulkstat; + int target_fd; + int ret; + + *fd = -1; + + ret = -xfrog_bulkstat_single(req->xfd, mrec->fmr_owner, 0, &bulkstat); + if (ret) { + if (ret == ENOENT || ret == EINVAL) + return 0; + + fprintf(stderr, _("bulkstat inode 0x%llx: %s\n"), + (unsigned long long)mrec->fmr_owner, + strerror(errno)); + return ret; + } + + /* + * If we get stats for a different inode, the file may have been freed + * out from under us and there's nothing to do. + */ + if (bulkstat.bs_ino != mrec->fmr_owner) + return 0; + + /* Skip anything we can't freeze. */ + if (!S_ISREG(bulkstat.bs_mode) && !S_ISDIR(bulkstat.bs_mode)) + return 0; + + target_fd = csp_open_by_handle(req, O_RDONLY, mrec->fmr_owner, + bulkstat.bs_gen); + if (target_fd == -2) + return 0; + if (target_fd < 0) + return target_fd; + + /* + * Skip mappings for directories, xattr data, and block mapping btree + * blocks. We still have to close the file though. + */ + if (S_ISDIR(bulkstat.bs_mode) || + (mrec->fmr_flags & (FMR_OF_ATTR_FORK | FMR_OF_EXTENT_MAP))) { + return close(target_fd); + } + + *fd = target_fd; + return 0; +} + +/* + * Given a fsmap, try to reflink the physical space into the space capture + * file. + */ +static int +csp_freeze_req_fsmap( + struct clearspace_req *req, + unsigned long long *cursor, + const struct fsmap *mrec) +{ + struct fsmap short_mrec; + struct file_clone_range fcr = { }; + unsigned long long frozen_len; + int src_fd; + int ret, ret2; + + if (mrec->fmr_device != req->dev) { + fprintf(stderr, _("wrong fsmap device in results.\n")); + return -1; + } + + /* Ignore mappings for our secret files. */ + if (csp_is_internal_owner(req, mrec->fmr_owner)) + return 0; + + /* Ignore mappings before the cursor. */ + if (mrec->fmr_physical + mrec->fmr_length < *cursor) + return 0; + + /* Jump past mappings for metadata. */ + if (mrec->fmr_flags & FMR_OF_SPECIAL_OWNER) + goto skip; + + /* + * Open this file so that we can try to freeze its data blocks. + * For other types of files we just skip to the evacuation step. + */ + ret = csp_freeze_open(req, mrec, &src_fd); + if (ret) + return ret; + if (src_fd < 0) + goto skip; + + /* + * If the cursor is in the middle of this mapping, increase the start + * of the mapping to start at the cursor. + */ + if (mrec->fmr_physical < *cursor) { + unsigned long long delta = *cursor - mrec->fmr_physical; + + short_mrec = *mrec; + short_mrec.fmr_physical = *cursor; + short_mrec.fmr_offset += delta; + short_mrec.fmr_length -= delta; + + mrec = &short_mrec; + } + + req->trace_indent++; + if (mrec->fmr_length == 0) { + trace_freeze(req, "skipping zero-length freeze", 0); + goto out_fd; + } + + /* + * Reflink the mapping from the source file into the work file. If we + * can't do that, we're sunk. If the mapping is unwritten, we'll leave + * a hole in the work file. + */ + fcr.src_fd = src_fd; + fcr.src_offset = mrec->fmr_offset; + fcr.src_length = mrec->fmr_length; + fcr.dest_offset = mrec->fmr_physical; + + trace_freeze(req, "freeze to workfd pos 0x%llx", + (unsigned long long)fcr.dest_offset); + + ret = clonerange(req->work_fd, &fcr); + if (ret) { + fprintf(stderr, _("freezing space to work file: %s\n"), + strerror(ret)); + goto out_fd; + } + + req->trace_indent++; + ret = csp_freeze_check_attempt(req, mrec, &frozen_len); + req->trace_indent--; + if (ret < 0) + goto out_fd; + if (ret == 1) { + ret = 0; + goto advance; + } + + /* + * We've frozen the mapping by reflinking it into the work file and + * confirmed that the work file has the space we wanted. Now we need + * to map the same extent into the space capture file. If reflink + * fails because we're out of space, fall back to FIEXCHANGE. The end + * goal is to populate the space capture file; we don't care about + * the contents of the work file. + */ + fcr.src_fd = req->work_fd; + fcr.src_offset = mrec->fmr_physical; + fcr.dest_offset = mrec->fmr_physical; + fcr.src_length = frozen_len; + + trace_freeze(req, "link phys 0x%llx len 0x%llx to spacefd", + (unsigned long long)mrec->fmr_physical, + (unsigned long long)mrec->fmr_length); + + ret = clonerange(req->space_fd, &fcr); + if (ret == ENOSPC) { + struct file_xchg_range xchg; + + xfrog_file_exchange_prep(NULL, FILE_XCHG_RANGE_NONATOMIC, + mrec->fmr_physical, req->work_fd, + mrec->fmr_physical, frozen_len, &xchg); + ret = exchangerange(req->space_fd, &xchg); + } + if (ret) { + fprintf(stderr, _("freezing space to space capture file: %s\n"), + strerror(ret)); + goto out_fd; + } + +advance: + *cursor += frozen_len; +out_fd: + ret2 = close(src_fd); + if (!ret && ret2) + ret = ret2; + req->trace_indent--; + if (ret) + trace_freeze(req, "ret %d", ret); + return ret; +skip: + *cursor += mrec->fmr_length; + return 0; +} + +/* + * Try to freeze all the space in the requested range against overwrites. + * + * For each file data fsmap within each hole in the part of the space capture + * file corresponding to the requested range, try to reflink the space into the + * space capture file so that any subsequent writes to the original owner are + * CoW and nobody else can allocate the space. If we cannot use reflink to + * freeze all the space, we cannot proceed with the clearing. + */ +static int +csp_freeze_req_range( + struct clearspace_req *req) +{ + unsigned long long cursor = req->start; + loff_t holepos = 0; + loff_t length = 0; + int ret; + + ret = ftruncate(req->space_fd, req->start + req->length); + if (ret) { + perror(_("setting up space capture file")); + return ret; + } + + if (!req->use_reflink) + return 0; + + start_spacefd_iter(req); + while ((ret = spacefd_hole_iter(req, &holepos, &length)) > 0) { + trace_freeze(req, "spacefd hole 0x%llx length 0x%llx", + (long long)holepos, (long long)length); + + start_fsmap_query(req, req->dev, holepos, length); + while ((ret = run_fsmap_query(req)) > 0) { + struct fsmap *mrec; + + for_each_fsmap_row(req, mrec) { + trace_fsmap_rec(req, CSP_TRACE_FREEZE, mrec); + trim_request_fsmap(req, mrec); + ret = csp_freeze_req_fsmap(req, &cursor, mrec); + if (ret) { + end_fsmap_query(req); + goto out; + } + } + } + end_fsmap_query(req); + } +out: + end_spacefd_iter(req); + return ret; +} + +/* + * Dump all speculative preallocations, COW staging blocks, and inactive inodes + * to try to free up as much space as we can. + */ +static int +csp_collect_garbage( + struct clearspace_req *req) +{ + struct xfs_fs_eofblocks eofb = { + .eof_version = XFS_EOFBLOCKS_VERSION, + .eof_flags = XFS_EOF_FLAGS_SYNC, + }; + int ret; + + ret = ioctl(req->xfd->fd, XFS_IOC_FREE_EOFBLOCKS, &eofb); + if (ret) { + perror(_("xfs garbage collector")); + return -1; + } + + return 0; +} + +/* Set up the target to clear all metadata from the given range. */ +static inline void +csp_target_metadata( + struct clearspace_req *req, + struct clearspace_tgt *target) +{ + target->start = req->start; + target->length = req->length; + target->prio = 0; + target->evacuated = 0; + target->owners = 0; + target->try_again = false; +} + +/* + * Loop through the space to find the most appealing part of the device to + * clear, then try to evacuate everything within. + */ +int +clearspace_run( + struct clearspace_req *req) +{ + struct clearspace_tgt target; + const struct csp_errstr *es; + bool cleared_anything; + int ret; + + if (req->trace_mask) { + fprintf(stderr, "debug flags 0x%x:", req->trace_mask); + for (es = errtags; es->tag; es++) { + if (req->trace_mask & es->mask) + fprintf(stderr, " %s", es->tag); + } + fprintf(stderr, "\n"); + } + + req->trace_indent = 0; + trace_status(req, + _("Clearing dev %u:%u physical 0x%llx bytecount 0x%llx."), + major(req->dev), minor(req->dev), + req->start, req->length); + + if (req->trace_mask & ~CSP_TRACE_STATUS) + trace_status(req, "reflink? %d evac_metadata? %d", + req->use_reflink, req->can_evac_metadata); + + ret = bitmap_alloc(&req->visited); + if (ret) { + perror(_("allocating visited bitmap")); + return ret; + } + + /* + * Empty out CoW forks and speculative post-EOF preallocations before + * starting the clearing process. This may be somewhat overkill. + */ + ret = syncfs(req->xfd->fd); + if (ret) { + perror(_("syncing filesystem")); + goto out_bitmap; + } + + ret = csp_collect_garbage(req); + if (ret) + goto out_bitmap; + + /* + * Try to freeze as much of the requested range as we can, grab the + * free space in that range, and run freeze again to pick up anything + * that may have been allocated while all that was going on. + */ + ret = csp_freeze_req_range(req); + if (ret) + goto out_bitmap; + + ret = csp_grab_free_space(req); + if (ret) + goto out_bitmap; + + ret = csp_freeze_req_range(req); + if (ret) + goto out_bitmap; + + /* + * If reflink is enabled, our strategy is to dedupe to free blocks in + * the area that we're clearing without making any user-visible changes + * to the file contents. For all the written file data blocks in area + * we're clearing, make an identical copy in the work file that is + * backed by blocks that are not in the clearing area. + */ + if (req->use_reflink) { + ret = csp_prepare_for_dedupe(req); + if (ret) + goto out_bitmap; + } + + /* Evacuate as many file blocks as we can. */ + do { + ret = csp_find_target(req, &target); + if (ret) + goto out_bitmap; + + if (target.length == 0) + break; + + trace_target(req, + "phys 0x%llx len 0x%llx owners 0x%llx prio 0x%llx", + target.start, target.length, + target.owners, target.prio); + + if (req->use_reflink) + ret = csp_evac_dedupe(req, &target); + else + ret = csp_evac_exchange(req, &target); + if (ret) + goto out_bitmap; + + trace_status(req, _("Evacuated %llu file items."), + target.evacuated); + } while (target.evacuated > 0 || target.try_again); + + if (!req->can_evac_metadata) + goto out_bitmap; + + /* Evacuate as many AG metadata blocks as we can. */ + do { + csp_target_metadata(req, &target); + + ret = csp_evac_fs_metadata(req, &target, &cleared_anything); + if (ret) + goto out_bitmap; + + trace_status(req, "evacuated %llu metadata items", + target.evacuated); + } while (target.evacuated > 0 && cleared_anything); + +out_bitmap: + bitmap_free(&req->visited); + return ret; +} + +/* How much space did we actually clear? */ +int +clearspace_efficacy( + struct clearspace_req *req, + unsigned long long *cleared_bytes) +{ + unsigned long long cleared = 0; + int ret; + + start_bmapx_query(req, 0, req->start, req->length); + while ((ret = run_bmapx_query(req, req->space_fd)) > 0) { + struct getbmapx *brec; + + for_each_bmapx_row(req, brec) { + if (brec->bmv_block == -1) + continue; + + trace_bmapx_rec(req, CSP_TRACE_EFFICACY, brec); + + if (brec->bmv_offset != brec->bmv_block) { + fprintf(stderr, + _("space capture file mapped incorrectly\n")); + end_bmapx_query(req); + return -1; + } + cleared += BBTOB(brec->bmv_length); + } + } + end_bmapx_query(req); + if (ret) + return ret; + + *cleared_bytes = cleared; + return 0; +} + +/* + * Create a temporary file on the same volume (data/rt) that we're trying to + * clear free space on. + */ +static int +csp_open_tempfile( + struct clearspace_req *req, + struct stat *statbuf) +{ + struct fsxattr fsx; + int fd, ret; + + fd = openat(req->xfd->fd, ".", O_TMPFILE | O_RDWR | O_EXCL, 0600); + if (fd < 0) { + perror(_("opening temp file")); + return -1; + } + + /* Make sure we got the same filesystem as the open file. */ + ret = fstat(fd, statbuf); + if (ret) { + perror(_("stat temp file")); + goto fail; + } + if (statbuf->st_dev != req->statbuf.st_dev) { + fprintf(stderr, + _("Cannot create temp file on same fs as open file.\n")); + goto fail; + } + + /* Ensure this file targets the correct data/rt device. */ + ret = ioctl(fd, FS_IOC_FSGETXATTR, &fsx); + if (ret) { + perror(_("FSGETXATTR temp file")); + goto fail; + } + + if (!!(fsx.fsx_xflags & FS_XFLAG_REALTIME) != req->realtime) { + if (req->realtime) + fsx.fsx_xflags |= FS_XFLAG_REALTIME; + else + fsx.fsx_xflags &= ~FS_XFLAG_REALTIME; + + ret = ioctl(fd, FS_IOC_FSSETXATTR, &fsx); + if (ret) { + perror(_("FSSETXATTR temp file")); + goto fail; + } + } + + trace_setup(req, "opening temp inode 0x%llx as fd %d", + (unsigned long long)statbuf->st_ino, fd); + + return fd; +fail: + close(fd); + return -1; +} + +/* Extract fshandle from the open file. */ +static int +csp_install_file( + struct clearspace_req *req, + struct xfs_fd *xfd) +{ + void *handle; + size_t handle_sz; + int ret; + + ret = fstat(xfd->fd, &req->statbuf); + if (ret) + return ret; + + if (!S_ISDIR(req->statbuf.st_mode)) { + errno = -ENOTDIR; + return -1; + } + + ret = fd_to_handle(xfd->fd, &handle, &handle_sz); + if (ret) + return ret; + + ret = handle_to_fshandle(handle, handle_sz, &req->fshandle, + &req->fshandle_sz); + if (ret) + return ret; + + free_handle(handle, handle_sz); + req->xfd = xfd; + return 0; +} + +/* Decide if we can use online repair to evacuate metadata. */ +static void +csp_detect_evac_metadata( + struct clearspace_req *req) +{ + struct xfs_scrub_metadata scrub = { + .sm_type = XFS_SCRUB_TYPE_PROBE, + .sm_flags = XFS_SCRUB_IFLAG_REPAIR | + XFS_SCRUB_IFLAG_FORCE_REBUILD, + }; + int ret; + + ret = ioctl(req->xfd->fd, XFS_IOC_SCRUB_METADATA, &scrub); + if (ret) + return; + + /* + * We'll try to evacuate metadata if the probe works. This doesn't + * guarantee success; it merely means that the kernel call exists. + */ + req->can_evac_metadata = true; +} + +/* Detect FALLOC_FL_MAP_FREE; this is critical for grabbing free space! */ +static int +csp_detect_fallocate_map_free( + struct clearspace_req *req) +{ + int ret; + + /* + * A single-byte fallocate request will succeed without doing anything + * to the filesystem. + */ + ret = fallocate(req->work_fd, FALLOC_FL_MAP_FREE_SPACE, 0, 1); + if (!ret) + return 0; + + if (errno == EOPNOTSUPP) { + fprintf(stderr, + _("Filesystem does not support FALLOC_FL_MAP_FREE_SPACE\n")); + return -1; + } + + perror(_("test FALLOC_FL_MAP_FREE_SPACE on work file")); + return -1; +} + +/* + * Assemble operation information to clear the physical space in part of a + * filesystem. + */ +int +clearspace_init( + struct clearspace_req **reqp, + const struct clearspace_init *attrs) +{ + struct clearspace_req *req; + int ret; + + req = calloc(1, sizeof(struct clearspace_req)); + if (!req) { + perror(_("malloc clearspace")); + return -1; + } + + req->work_fd = -1; + req->space_fd = -1; + req->trace_mask = attrs->trace_mask; + + req->realtime = attrs->is_realtime; + req->dev = attrs->dev; + req->start = attrs->start; + req->length = attrs->length; + + ret = csp_install_file(req, attrs->xfd); + if (ret) { + perror(attrs->fname); + goto fail; + } + + csp_detect_evac_metadata(req); + + req->work_fd = csp_open_tempfile(req, &req->temp_statbuf); + if (req->work_fd < 0) + goto fail; + + req->space_fd = csp_open_tempfile(req, &req->space_statbuf); + if (req->space_fd < 0) + goto fail; + + ret = csp_detect_fallocate_map_free(req); + if (ret) + goto fail; + + req->mhead = calloc(1, fsmap_sizeof(QUERY_BATCH_SIZE)); + if (!req->mhead) { + perror(_("opening fs mapping query")); + goto fail; + } + + req->rhead = calloc(1, fsrefs_sizeof(QUERY_BATCH_SIZE)); + if (!req->rhead) { + perror(_("opening refcount query")); + goto fail; + } + + req->bhead = calloc(QUERY_BATCH_SIZE + 1, sizeof(struct getbmapx)); + if (!req->bhead) { + perror(_("opening file mapping query")); + goto fail; + } + + req->buf = malloc(BUFFERCOPY_BUFSZ); + if (!req->buf) { + perror(_("allocating file copy buffer")); + goto fail; + } + + req->fdr = calloc(1, sizeof(struct file_dedupe_range) + + sizeof(struct file_dedupe_range_info)); + if (!req->fdr) { + perror(_("allocating dedupe control buffer")); + goto fail; + } + + req->use_reflink = req->xfd->fsgeom.flags & XFS_FSOP_GEOM_FLAGS_REFLINK; + + *reqp = req; + return 0; +fail: + clearspace_free(&req); + return -1; +} + +/* Free all resources associated with a space clearing request. */ +int +clearspace_free( + struct clearspace_req **reqp) +{ + struct clearspace_req *req = *reqp; + int ret = 0; + + if (!req) + return 0; + + *reqp = NULL; + free(req->fdr); + free(req->buf); + free(req->bhead); + free(req->rhead); + free(req->mhead); + + if (req->space_fd >= 0) { + ret = close(req->space_fd); + if (ret) + perror(_("closing space capture file")); + } + + if (req->work_fd >= 0) { + int ret2 = close(req->work_fd); + + if (ret2) { + perror(_("closing work file")); + if (!ret && ret2) + ret = ret2; + } + } + + if (req->fshandle) + free_handle(req->fshandle, req->fshandle_sz); + free(req); + return ret; +} diff --git a/libfrog/clearspace.h b/libfrog/clearspace.h new file mode 100644 index 00000000000..4b12c9b0475 --- /dev/null +++ b/libfrog/clearspace.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2022 Oracle. Inc. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#ifndef __LIBFROG_CLEARSPACE_H__ +#define __LIBFROG_CLEARSPACE_H__ + +struct clearspace_req; + +struct clearspace_init { + /* Open file and its pathname */ + struct xfs_fd *xfd; + const char *fname; + + /* Which device do we want? */ + bool is_realtime; + dev_t dev; + + /* Range of device to clear. */ + unsigned long long start; + unsigned long long length; + + unsigned int trace_mask; +}; + +int clearspace_init(struct clearspace_req **reqp, + const struct clearspace_init *init); +int clearspace_free(struct clearspace_req **reqp); + +int clearspace_run(struct clearspace_req *req); + +int clearspace_efficacy(struct clearspace_req *req, + unsigned long long *cleared_bytes); + +/* Debugging levels */ + +#define CSP_TRACE_FREEZE (0x01) +#define CSP_TRACE_GRAB (0x02) +#define CSP_TRACE_FSMAP (0x04) +#define CSP_TRACE_FSREFS (0x08) +#define CSP_TRACE_BMAPX (0x10) +#define CSP_TRACE_PREP (0x20) +#define CSP_TRACE_TARGET (0x40) +#define CSP_TRACE_DEDUPE (0x80) +#define CSP_TRACE_FALLOC (0x100) +#define CSP_TRACE_FIEXCHANGE (0x200) +#define CSP_TRACE_XREBUILD (0x400) +#define CSP_TRACE_EFFICACY (0x800) +#define CSP_TRACE_SETUP (0x1000) +#define CSP_TRACE_STATUS (0x2000) +#define CSP_TRACE_DUMPFILE (0x4000) +#define CSP_TRACE_BITMAP (0x8000) + +#define CSP_TRACE_ALL (CSP_TRACE_FREEZE | \ + CSP_TRACE_GRAB | \ + CSP_TRACE_FSMAP | \ + CSP_TRACE_FSREFS | \ + CSP_TRACE_BMAPX | \ + CSP_TRACE_PREP | \ + CSP_TRACE_TARGET | \ + CSP_TRACE_DEDUPE | \ + CSP_TRACE_FALLOC | \ + CSP_TRACE_FIEXCHANGE | \ + CSP_TRACE_XREBUILD | \ + CSP_TRACE_EFFICACY | \ + CSP_TRACE_SETUP | \ + CSP_TRACE_STATUS | \ + CSP_TRACE_DUMPFILE | \ + CSP_TRACE_BITMAP) + +#endif /* __LIBFROG_CLEARSPACE_H__ */ diff --git a/man/man8/xfs_spaceman.8 b/man/man8/xfs_spaceman.8 index 837fc497f27..7c11953d16b 100644 --- a/man/man8/xfs_spaceman.8 +++ b/man/man8/xfs_spaceman.8 @@ -25,6 +25,23 @@ then the program exits. .SH COMMANDS .TP +.BI "clearfree [ \-n nr ] [ \-r ] [ \-v mask ] " start " " length +Try to clear the specified physical range in the filesystem. +The +.B start +and +.B length +arguments must be given in units of bytes. +If the +.B -n +option is given, run the clearing algorithm this many times. +If the +.B -r +option is given, clear the realtime device. +If the +.B -v +option is given, print what's happening every step of the way. +.TP .BI "freesp [ \-dgrs ] [-a agno]... [ \-b | \-e bsize | \-h bsize | \-m factor ]" With no arguments, .B freesp diff --git a/spaceman/Makefile b/spaceman/Makefile index 1f048d54a4d..75df4ce86c2 100644 --- a/spaceman/Makefile +++ b/spaceman/Makefile @@ -10,8 +10,8 @@ HFILES = init.h space.h CFILES = info.c init.c file.c health.c prealloc.c trim.c LSRCFILES = xfs_info.sh -LLDLIBS = $(LIBXCMD) $(LIBFROG) -LTDEPENDENCIES = $(LIBXCMD) $(LIBFROG) +LLDLIBS = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) +LTDEPENDENCIES = $(LIBHANDLE) $(LIBXCMD) $(LIBFROG) LLDFLAGS = -static ifeq ($(ENABLE_EDITLINE),yes) @@ -19,7 +19,7 @@ LLDLIBS += $(LIBEDITLINE) $(LIBTERMCAP) endif ifeq ($(HAVE_GETFSMAP),yes) -CFILES += freesp.c +CFILES += freesp.c clearfree.c endif default: depend $(LTCOMMAND) diff --git a/spaceman/clearfree.c b/spaceman/clearfree.c new file mode 100644 index 00000000000..e944a07b887 --- /dev/null +++ b/spaceman/clearfree.c @@ -0,0 +1,164 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#include "platform_defs.h" +#include "command.h" +#include "init.h" +#include "libfrog/paths.h" +#include "input.h" +#include "libfrog/fsgeom.h" +#include "libfrog/clearspace.h" +#include "handle.h" +#include "space.h" + +static void +clearfree_help(void) +{ + printf(_( +"Evacuate the contents of the given range of physical storage in the filesystem" +"\n" +" -n -- Run the space clearing algorithm this many times.\n" +" -r -- clear space on the realtime device.\n" +" -v -- verbosity level, or \"all\" to print everything.\n" +"\n" +"The start and length arguments are required, and must be specified in units\n" +"of bytes.\n" +"\n")); +} + +static int +clearfree_f( + int argc, + char **argv) +{ + struct clearspace_init attrs = { + .xfd = &file->xfd, + .fname = file->name, + }; + struct clearspace_req *req = NULL; + unsigned long long cleared; + unsigned long arg; + long long lnum; + unsigned int i, nr = 1; + int c, ret; + + while ((c = getopt(argc, argv, "n:rv:")) != EOF) { + switch (c) { + case 'n': + errno = 0; + arg = strtoul(optarg, NULL, 0); + if (errno) { + perror(optarg); + return 1; + } + if (arg > UINT_MAX) + arg = UINT_MAX; + nr = arg; + break; + case 'r': /* rt device */ + attrs.is_realtime = true; + break; + case 'v': /* Verbose output */ + if (!strcmp(optarg, "all")) { + attrs.trace_mask = CSP_TRACE_ALL; + } else { + errno = 0; + attrs.trace_mask = strtoul(optarg, NULL, 0); + if (errno) { + perror(optarg); + return 1; + } + } + break; + default: + exitcode = 1; + clearfree_help(); + return 0; + } + } + + if (attrs.trace_mask) + attrs.trace_mask |= CSP_TRACE_STATUS; + + if (argc != optind + 2) { + clearfree_help(); + goto fail; + } + + if (attrs.is_realtime) { + if (file->xfd.fsgeom.rtblocks == 0) { + fprintf(stderr, _("No realtime volume present.\n")); + goto fail; + } + attrs.dev = file->fs_path.fs_rtdev; + } else { + attrs.dev = file->fs_path.fs_datadev; + } + + lnum = cvtnum(file->xfd.fsgeom.blocksize, file->xfd.fsgeom.sectsize, + argv[optind]); + if (lnum < 0) { + fprintf(stderr, _("Bad clearfree start sector %s.\n"), + argv[optind]); + goto fail; + } + attrs.start = lnum; + + lnum = cvtnum(file->xfd.fsgeom.blocksize, file->xfd.fsgeom.sectsize, + argv[optind + 1]); + if (lnum < 0) { + fprintf(stderr, _("Bad clearfree length %s.\n"), + argv[optind + 1]); + goto fail; + } + attrs.length = lnum; + + ret = clearspace_init(&req, &attrs); + if (ret) + goto fail; + + for (i = 0; i < nr; i++) { + ret = clearspace_run(req); + if (ret) + goto fail; + } + + ret = clearspace_efficacy(req, &cleared); + if (ret) + goto fail; + + printf(_("Cleared 0x%llx bytes (%.1f%%) from 0x%llx to 0x%llx.\n"), + cleared, 100.0 * cleared / attrs.length, attrs.start, + attrs.start + attrs.length); + + ret = clearspace_free(&req); + if (ret) + goto fail; + + fshandle_destroy(); + return 0; +fail: + fshandle_destroy(); + exitcode = 1; + return 1; +} + +static struct cmdinfo clearfree_cmd = { + .name = "clearfree", + .cfunc = clearfree_f, + .argmin = 0, + .argmax = -1, + .flags = CMD_FLAG_ONESHOT, + .args = "[-n runs] [-r] [-v mask] start length", + .help = clearfree_help, +}; + +void +clearfree_init(void) +{ + clearfree_cmd.oneline = _("clear free space in the filesystem"); + + add_command(&clearfree_cmd); +} diff --git a/spaceman/init.c b/spaceman/init.c index cf1ff3cbb0e..bce62dec47f 100644 --- a/spaceman/init.c +++ b/spaceman/init.c @@ -35,6 +35,7 @@ init_commands(void) trim_init(); freesp_init(); health_init(); + clearfree_init(); } static int diff --git a/spaceman/space.h b/spaceman/space.h index 723209edd99..be4a7426ebf 100644 --- a/spaceman/space.h +++ b/spaceman/space.h @@ -28,8 +28,10 @@ extern void quit_init(void); extern void trim_init(void); #ifdef HAVE_GETFSMAP extern void freesp_init(void); +extern void clearfree_init(void); #else # define freesp_init() do { } while (0) +# define clearfree_init() do { } while(0) #endif extern void info_init(void); extern void health_init(void); ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 3/5] xfs_db: get and put blocks on the AGFL 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong ` (2 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 4/5] xfs_spaceman: implement clearing " Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 2022-12-30 22:20 ` [PATCH 5/5] xfs_spaceman: defragment free space with normal files Darrick J. Wong 4 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Add a new xfs_db command to let people add and remove blocks from an AGFL. This isn't really related to rmap btree reconstruction, other than enabling debugging code to mess around with the AGFL to exercise various odd scenarios. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- db/agfl.c | 297 ++++++++++++++++++++++++++++++++++++++++++++++ libxfs/libxfs_api_defs.h | 3 man/man8/xfs_db.8 | 11 ++ 3 files changed, 307 insertions(+), 4 deletions(-) diff --git a/db/agfl.c b/db/agfl.c index f0f3f21a64d..804653e40b5 100644 --- a/db/agfl.c +++ b/db/agfl.c @@ -15,13 +15,14 @@ #include "output.h" #include "init.h" #include "agfl.h" +#include "libfrog/bitmap.h" static int agfl_bno_size(void *obj, int startoff); static int agfl_f(int argc, char **argv); static void agfl_help(void); static const cmdinfo_t agfl_cmd = - { "agfl", NULL, agfl_f, 0, 1, 1, N_("[agno]"), + { "agfl", NULL, agfl_f, 0, -1, 1, N_("[agno] [-g nr] [-p nr]"), N_("set address to agfl block"), agfl_help }; const field_t agfl_hfld[] = { { @@ -77,10 +78,280 @@ agfl_help(void) " for each allocation group. This acts as a reserved pool of space\n" " separate from the general filesystem freespace (not used for user data).\n" "\n" +" -g quantity\tRemove this many blocks from the AGFL.\n" +" -p quantity\tAdd this many blocks to the AGFL.\n" +"\n" )); } +struct dump_info { + struct xfs_perag *pag; + bool leak; +}; + +/* Return blocks freed from the AGFL to the free space btrees. */ +static int +free_grabbed( + uint64_t start, + uint64_t length, + void *data) +{ + struct dump_info *di = data; + struct xfs_perag *pag = di->pag; + struct xfs_mount *mp = pag->pag_mount; + struct xfs_trans *tp; + struct xfs_buf *agf_bp; + int error; + + error = -libxfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, 0, 0, 0, + &tp); + if (error) + return error; + + error = -libxfs_alloc_read_agf(pag, tp, 0, &agf_bp); + if (error) + goto out_cancel; + + error = -libxfs_free_extent(tp, pag, start, length, &XFS_RMAP_OINFO_AG, + XFS_AG_RESV_AGFL); + if (error) + goto out_cancel; + + return -libxfs_trans_commit(tp); + +out_cancel: + libxfs_trans_cancel(tp); + return error; +} + +/* Report blocks freed from the AGFL. */ +static int +dump_grabbed( + uint64_t start, + uint64_t length, + void *data) +{ + struct dump_info *di = data; + const char *fmt; + + if (length == 1) + fmt = di->leak ? _("agfl %u: leaked agbno %u\n") : + _("agfl %u: removed agbno %u\n"); + else + fmt = di->leak ? _("agfl %u: leaked agbno %u-%u\n") : + _("agfl %u: removed agbno %u-%u\n"); + + printf(fmt, di->pag->pag_agno, (unsigned int)start, + (unsigned int)(start + length - 1)); + return 0; +} + +/* Remove blocks from the AGFL. */ +static int +agfl_get( + struct xfs_perag *pag, + int quantity) +{ + struct dump_info di = { + .pag = pag, + .leak = quantity < 0, + }; + struct xfs_agf *agf; + struct xfs_buf *agf_bp; + struct xfs_trans *tp; + struct bitmap *grabbed; + const unsigned int agfl_size = libxfs_agfl_size(pag->pag_mount); + unsigned int i; + int error; + + if (!quantity) + return 0; + + if (di.leak) + quantity = -quantity; + quantity = min(quantity, agfl_size); + + error = bitmap_alloc(&grabbed); + if (error) + goto out; + + error = -libxfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, quantity, 0, + 0, &tp); + if (error) + goto out_bitmap; + + error = -libxfs_alloc_read_agf(pag, tp, 0, &agf_bp); + if (error) + goto out_cancel; + + agf = agf_bp->b_addr; + quantity = min(quantity, be32_to_cpu(agf->agf_flcount)); + + for (i = 0; i < quantity; i++) { + xfs_agblock_t agbno; + + error = -libxfs_alloc_get_freelist(pag, tp, agf_bp, &agbno, 0); + if (error) + goto out_cancel; + + if (agbno == NULLAGBLOCK) { + error = ENOSPC; + goto out_cancel; + } + + error = bitmap_set(grabbed, agbno, 1); + if (error) + goto out_cancel; + } + + error = -libxfs_trans_commit(tp); + if (error) + goto out_bitmap; + + error = bitmap_iterate(grabbed, dump_grabbed, &di); + if (error) + goto out_bitmap; + + if (!di.leak) { + error = bitmap_iterate(grabbed, free_grabbed, &di); + if (error) + goto out_bitmap; + } + + bitmap_free(&grabbed); + return 0; + +out_cancel: + libxfs_trans_cancel(tp); +out_bitmap: + bitmap_free(&grabbed); +out: + if (error) + printf(_("agfl %u: %s\n"), pag->pag_agno, strerror(error)); + return error; +} + +/* Add blocks to the AGFL. */ +static int +agfl_put( + struct xfs_perag *pag, + int quantity) +{ + struct xfs_alloc_arg targs = { + .mp = pag->pag_mount, + .alignment = 1, + .minlen = 1, + .prod = 1, + .type = XFS_ALLOCTYPE_NEAR_BNO, + .resv = XFS_AG_RESV_AGFL, + .oinfo = XFS_RMAP_OINFO_AG, + }; + struct xfs_buf *agfl_bp; + struct xfs_agf *agf; + struct xfs_trans *tp; + const unsigned int agfl_size = libxfs_agfl_size(pag->pag_mount); + unsigned int i; + bool eoag = quantity < 0; + int error; + + if (!quantity) + return 0; + + if (eoag) + quantity = -quantity; + quantity = min(quantity, agfl_size); + + error = -libxfs_trans_alloc(mp, &M_RES(mp)->tr_itruncate, quantity, 0, + 0, &tp); + if (error) + return error; + targs.tp = tp; + + error = -libxfs_alloc_read_agf(pag, tp, 0, &targs.agbp); + if (error) + goto out_cancel; + + agf = targs.agbp->b_addr; + targs.maxlen = min(quantity, agfl_size - be32_to_cpu(agf->agf_flcount)); + + if (eoag) + targs.fsbno = XFS_AGB_TO_FSB(pag->pag_mount, pag->pag_agno, + be32_to_cpu(agf->agf_length) - 1); + else + targs.fsbno = XFS_AGB_TO_FSB(pag->pag_mount, pag->pag_agno, 0); + + error = -libxfs_alloc_read_agfl(pag, tp, &agfl_bp); + if (error) + goto out_cancel; + + error = -libxfs_alloc_vextent(&targs); + if (error) + goto out_cancel; + + if (targs.agbno == NULLAGBLOCK) { + error = ENOSPC; + goto out_cancel; + } + + for (i = 0; i < targs.len; i++) { + error = -libxfs_alloc_put_freelist(pag, tp, targs.agbp, + agfl_bp, targs.agbno + i, 0); + if (error) + goto out_cancel; + } + + if (i == 1) + printf(_("agfl %u: added agbno %u\n"), pag->pag_agno, + targs.agbno); + else if (i > 1) + printf(_("agfl %u: added agbno %u-%u\n"), pag->pag_agno, + targs.agbno, targs.agbno + i - 1); + + error = -libxfs_trans_commit(tp); + if (error) + goto out; + + return 0; + +out_cancel: + libxfs_trans_cancel(tp); +out: + if (error) + printf(_("agfl %u: %s\n"), pag->pag_agno, strerror(error)); + return error; +} + +static void +agfl_adjust( + struct xfs_mount *mp, + xfs_agnumber_t agno, + int gblocks, + int pblocks) +{ + struct xfs_perag *pag; + int error; + + if (!expert_mode) { + printf(_("AGFL get/put only supported in expert mode.\n")); + exitcode = 1; + return; + } + + pag = libxfs_perag_get(mp, agno); + + error = agfl_get(pag, gblocks); + if (error) + goto out_pag; + + error = agfl_put(pag, pblocks); + +out_pag: + libxfs_perag_put(pag); + if (error) + exitcode = 1; +} + static int agfl_f( int argc, @@ -88,9 +359,25 @@ agfl_f( { xfs_agnumber_t agno; char *p; + int c; + int gblocks = 0, pblocks = 0; - if (argc > 1) { - agno = (xfs_agnumber_t)strtoul(argv[1], &p, 0); + while ((c = getopt(argc, argv, "g:p:")) != -1) { + switch (c) { + case 'g': + gblocks = atoi(optarg); + break; + case 'p': + pblocks = atoi(optarg); + break; + default: + agfl_help(); + return 1; + } + } + + if (argc > optind) { + agno = (xfs_agnumber_t)strtoul(argv[optind], &p, 0); if (*p != '\0' || agno >= mp->m_sb.sb_agcount) { dbprintf(_("bad allocation group number %s\n"), argv[1]); return 0; @@ -98,6 +385,10 @@ agfl_f( cur_agno = agno; } else if (cur_agno == NULLAGNUMBER) cur_agno = 0; + + if (gblocks || pblocks) + agfl_adjust(mp, cur_agno, gblocks, pblocks); + ASSERT(typtab[TYP_AGFL].typnm == TYP_AGFL); set_cur(&typtab[TYP_AGFL], XFS_AG_DADDR(mp, cur_agno, XFS_AGFL_DADDR(mp)), diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h index 63607cf2b94..d4bc59f3bd1 100644 --- a/libxfs/libxfs_api_defs.h +++ b/libxfs/libxfs_api_defs.h @@ -29,8 +29,11 @@ #define xfs_allocbt_maxrecs libxfs_allocbt_maxrecs #define xfs_allocbt_stage_cursor libxfs_allocbt_stage_cursor #define xfs_alloc_fix_freelist libxfs_alloc_fix_freelist +#define xfs_alloc_get_freelist libxfs_alloc_get_freelist #define xfs_alloc_min_freelist libxfs_alloc_min_freelist +#define xfs_alloc_put_freelist libxfs_alloc_put_freelist #define xfs_alloc_read_agf libxfs_alloc_read_agf +#define xfs_alloc_read_agfl libxfs_alloc_read_agfl #define xfs_alloc_vextent libxfs_alloc_vextent #define xfs_attr_get libxfs_attr_get diff --git a/man/man8/xfs_db.8 b/man/man8/xfs_db.8 index a694c8ed916..b0ae2abea57 100644 --- a/man/man8/xfs_db.8 +++ b/man/man8/xfs_db.8 @@ -182,10 +182,19 @@ Set current address to the AGF block for allocation group .IR agno . If no argument is given, use the current allocation group. .TP -.BI "agfl [" agno ] +.BI "agfl [" agno "] [\-g " " quantity" "] [\-p " quantity ] Set current address to the AGFL block for allocation group .IR agno . If no argument is given, use the current allocation group. +If the +.B -g +option is specified with a positive quantity, remove that many blocks from the +AGFL and put them in the free space btrees. +If the quantity is negative, remove the blocks and leak them. +If the +.B -p +option is specified, add that many blocks to the AGFL. +If the quantity is negative, the blocks are selected from the end of the AG. .TP .BI "agi [" agno ] Set current address to the AGI block for allocation group ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH 5/5] xfs_spaceman: defragment free space with normal files 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong ` (3 preceding siblings ...) 2022-12-30 22:20 ` [PATCH 3/5] xfs_db: get and put blocks on the AGFL Darrick J. Wong @ 2022-12-30 22:20 ` Darrick J. Wong 4 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:20 UTC (permalink / raw) To: cem, djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- libfrog/clearspace.c | 377 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 297 insertions(+), 80 deletions(-) diff --git a/libfrog/clearspace.c b/libfrog/clearspace.c index 601257022b8..452cd13db45 100644 --- a/libfrog/clearspace.c +++ b/libfrog/clearspace.c @@ -1362,6 +1362,17 @@ csp_evac_dedupe_loop( fdr->src_length = old_reqlen; continue; } + if (ret == EINVAL) { + /* + * If we can't dedupe get the block, it's possible that + * src_fd was punched or truncated out from under us. + * Treat this the same way we would if the contents + * didn't match. + */ + trace_dedupe(req, "cannot evac space, moving on", 0); + same = false; + ret = 0; + } if (ret) { fprintf(stderr, _("evacuating inode 0x%llx: %s\n"), ino, strerror(ret)); @@ -1939,8 +1950,14 @@ csp_evac_fs_metadata( * the space capture file, or 1 if there's nothing to transfer to the space * capture file. */ -static int -csp_freeze_check_attempt( +enum freeze_outcome { + FREEZE_FAILED = -1, + FREEZE_DONE, + FREEZE_SKIP, +}; + +static enum freeze_outcome +csp_freeze_check_outcome( struct clearspace_req *req, const struct fsmap *mrec, unsigned long long *len) @@ -1950,13 +1967,12 @@ csp_freeze_check_attempt( *len = 0; - ret = bmapx_one(req, req->work_fd, mrec->fmr_physical, - mrec->fmr_length, &brec); + ret = bmapx_one(req, req->work_fd, 0, mrec->fmr_length, &brec); if (ret) - return ret; + return FREEZE_FAILED; trace_freeze(req, - "does workfd pos 0x%llx len 0x%llx map to phys 0x%llx len 0x%llx?", + "check if workfd pos 0x0 phys 0x%llx len 0x%llx maps to phys 0x%llx len 0x%llx", (unsigned long long)mrec->fmr_physical, (unsigned long long)mrec->fmr_length, (unsigned long long)BBTOB(brec.bmv_block), @@ -1964,8 +1980,8 @@ csp_freeze_check_attempt( /* freeze of an unwritten extent punches a hole in the work file. */ if ((mrec->fmr_flags & FMR_OF_PREALLOC) && brec.bmv_block == -1) { - *len = BBTOB(brec.bmv_length); - return 1; + *len = min(mrec->fmr_length, BBTOB(brec.bmv_length)); + return FREEZE_SKIP; } /* @@ -1974,8 +1990,8 @@ csp_freeze_check_attempt( */ if (!(mrec->fmr_flags & FMR_OF_PREALLOC) && BBTOB(brec.bmv_block) == mrec->fmr_physical) { - *len = BBTOB(brec.bmv_length); - return 0; + *len = min(mrec->fmr_length, BBTOB(brec.bmv_length)); + return FREEZE_DONE; } /* @@ -1984,20 +2000,15 @@ csp_freeze_check_attempt( * have been mapped into the work file. Set @len to zero and return so * that we try again with the next mapping. */ + trace_falloc(req, "reset workfd isize 0x0", 0); - trace_falloc(req, "fpunch workfd pos 0x%llx bytecount 0x%llx", - (unsigned long long)mrec->fmr_physical, - (unsigned long long)mrec->fmr_length); - - ret = fallocate(req->work_fd, - FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, - mrec->fmr_physical, mrec->fmr_length); + ret = ftruncate(req->work_fd, 0); if (ret) { perror(_("resetting work file after failed freeze")); - return ret; + return FREEZE_FAILED; } - return 1; + return FREEZE_SKIP; } /* @@ -2014,6 +2025,7 @@ csp_freeze_open( int *fd) { struct xfs_bulkstat bulkstat; + int oflags = O_RDWR; int target_fd; int ret; @@ -2041,7 +2053,10 @@ csp_freeze_open( if (!S_ISREG(bulkstat.bs_mode) && !S_ISDIR(bulkstat.bs_mode)) return 0; - target_fd = csp_open_by_handle(req, O_RDONLY, mrec->fmr_owner, + if (S_ISDIR(bulkstat.bs_mode)) + oflags = O_RDONLY; + + target_fd = csp_open_by_handle(req, oflags, mrec->fmr_owner, bulkstat.bs_gen); if (target_fd == -2) return 0; @@ -2061,6 +2076,122 @@ csp_freeze_open( return 0; } +static inline uint64_t rounddown_64(uint64_t x, uint64_t y) +{ + return (x / y) * y; +} + +/* + * Deal with a frozen extent containing a partially written EOF block. Either + * we use funshare to get src_fd to release the block, or we reduce the length + * of the frozen extent by one block. + */ +static int +csp_freeze_unaligned_eofblock( + struct clearspace_req *req, + int src_fd, + const struct fsmap *mrec, + unsigned long long *frozen_len) +{ + struct getbmapx brec; + struct stat statbuf; + loff_t work_offset, length; + int ret; + + ret = fstat(req->work_fd, &statbuf); + if (ret) { + perror(_("statting work file")); + return ret; + } + + /* + * The frozen extent is less than the size of the work file, which + * means that we're already block aligned. + */ + if (*frozen_len <= statbuf.st_size) + return 0; + + /* The frozen extent does not contain a partially written EOF block. */ + if (statbuf.st_size % statbuf.st_blksize == 0) + return 0; + + /* + * Unshare what we think is a partially written EOF block of the + * original file, to try to force it to release that block. + */ + work_offset = rounddown_64(statbuf.st_size, statbuf.st_blksize); + length = statbuf.st_size - work_offset; + + trace_freeze(req, + "unaligned eofblock 0x%llx work_size 0x%llx blksize 0x%x work_offset 0x%llx work_length 0x%llx", + *frozen_len, statbuf.st_size, statbuf.st_blksize, + work_offset, length); + + ret = fallocate(src_fd, FALLOC_FL_UNSHARE_RANGE, + mrec->fmr_offset + work_offset, length); + if (ret) { + perror(_("unsharing original file")); + return ret; + } + + ret = fsync(src_fd); + if (ret) { + perror(_("flushing original file")); + return ret; + } + + ret = bmapx_one(req, req->work_fd, work_offset, length, &brec); + if (ret) + return ret; + + if (BBTOB(brec.bmv_block) != mrec->fmr_physical + work_offset) { + fprintf(stderr, + _("work file offset 0x%llx maps to phys 0x%llx, expected 0x%llx"), + (unsigned long long)work_offset, + (unsigned long long)BBTOB(brec.bmv_block), + (unsigned long long)mrec->fmr_physical); + return -1; + } + + /* + * If the block is still shared, there must be other owners of this + * block. Round down the frozen length and we'll come back to it + * eventually. + */ + if (brec.bmv_oflags & BMV_OF_SHARED) { + *frozen_len = work_offset; + return 0; + } + + /* + * Not shared anymore, so increase the size of the file to the next + * block boundary so that we can reflink it into the space capture + * file. + */ + ret = ftruncate(req->work_fd, + BBTOB(brec.bmv_length) + BBTOB(brec.bmv_offset)); + if (ret) { + perror(_("expanding work file")); + return ret; + } + + /* Double-check that we didn't lose the block. */ + ret = bmapx_one(req, req->work_fd, work_offset, length, &brec); + if (ret) + return ret; + + if (BBTOB(brec.bmv_block) != mrec->fmr_physical + work_offset) { + fprintf(stderr, + _("work file offset 0x%llx maps to phys 0x%llx, should be 0x%llx"), + (unsigned long long)work_offset, + (unsigned long long)BBTOB(brec.bmv_block), + (unsigned long long)mrec->fmr_physical); + return -1; + } + + return 0; +} + /* * Given a fsmap, try to reflink the physical space into the space capture * file. @@ -2074,6 +2205,7 @@ csp_freeze_req_fsmap( struct fsmap short_mrec; struct file_clone_range fcr = { }; unsigned long long frozen_len; + enum freeze_outcome outcome; int src_fd; int ret, ret2; @@ -2126,33 +2258,86 @@ csp_freeze_req_fsmap( } /* - * Reflink the mapping from the source file into the work file. If we + * Reflink the mapping from the source file into the empty work file so + * that a write will be written elsewhere. The only way to reflink a + * partially written EOF block is if the kernel can reset the work file + * size so that the post-EOF part of the block remains post-EOF. If we * can't do that, we're sunk. If the mapping is unwritten, we'll leave * a hole in the work file. */ + ret = ftruncate(req->work_fd, 0); + if (ret) { + perror(_("truncating work file for freeze")); + goto out_fd; + } + fcr.src_fd = src_fd; fcr.src_offset = mrec->fmr_offset; fcr.src_length = mrec->fmr_length; - fcr.dest_offset = mrec->fmr_physical; + fcr.dest_offset = 0; - trace_freeze(req, "freeze to workfd pos 0x%llx", - (unsigned long long)fcr.dest_offset); + trace_freeze(req, + "reflink ino 0x%llx offset 0x%llx bytecount 0x%llx into workfd", + (unsigned long long)mrec->fmr_owner, + (unsigned long long)fcr.src_offset, + (unsigned long long)fcr.src_length); ret = clonerange(req->work_fd, &fcr); - if (ret) { - fprintf(stderr, _("freezing space to work file: %s\n"), - strerror(ret)); - goto out_fd; + if (ret == EINVAL) { + /* + * If that didn't work, try reflinking to EOF and picking out + * whatever pieces we want. + */ + fcr.src_length = 0; + + trace_freeze(req, + "reflink ino 0x%llx offset 0x%llx to EOF into workfd", + (unsigned long long)mrec->fmr_owner, + (unsigned long long)fcr.src_offset); + + ret = clonerange(req->work_fd, &fcr); } - - req->trace_indent++; - ret = csp_freeze_check_attempt(req, mrec, &frozen_len); - req->trace_indent--; - if (ret < 0) - goto out_fd; - if (ret == 1) { + if (ret == EINVAL) { + /* + * If we still can't get the block, it's possible that src_fd + * was punched or truncated out from under us, so we just move + * on to the next fsmap. + */ + trace_freeze(req, "cannot freeze space, moving on", 0); ret = 0; - goto advance; + goto out_fd; + } + if (ret) { + fprintf(stderr, _("freezing space to work file: %s\n"), + strerror(ret)); + goto out_fd; + } + + req->trace_indent++; + outcome = csp_freeze_check_outcome(req, mrec, &frozen_len); + req->trace_indent--; + switch (outcome) { + case FREEZE_FAILED: + ret = -1; + goto out_fd; + case FREEZE_SKIP: + *cursor += frozen_len; + goto out_fd; + case FREEZE_DONE: + break; + } + + /* + * If we tried reflinking to EOF to capture a partially written EOF + * block in the work file, we need to unshare the end of the source + * file before we try to reflink the frozen space into the space + * capture file. + */ + if (fcr.src_length == 0) { + ret = csp_freeze_unaligned_eofblock(req, src_fd, mrec, + &frozen_len); + if (ret) + goto out_fd; } /* @@ -2164,11 +2349,11 @@ csp_freeze_req_fsmap( * the contents of the work file. */ fcr.src_fd = req->work_fd; - fcr.src_offset = mrec->fmr_physical; + fcr.src_offset = 0; fcr.dest_offset = mrec->fmr_physical; fcr.src_length = frozen_len; - trace_freeze(req, "link phys 0x%llx len 0x%llx to spacefd", + trace_freeze(req, "reflink phys 0x%llx len 0x%llx to spacefd", (unsigned long long)mrec->fmr_physical, (unsigned long long)mrec->fmr_length); @@ -2187,7 +2372,6 @@ csp_freeze_req_fsmap( goto out_fd; } -advance: *cursor += frozen_len; out_fd: ret2 = close(src_fd); @@ -2278,6 +2462,79 @@ csp_collect_garbage( return 0; } +static int +csp_prepare( + struct clearspace_req *req) +{ + blkcnt_t old_blocks = 0; + int ret; + + /* + * Empty out CoW forks and speculative post-EOF preallocations before + * starting the clearing process. This may be somewhat overkill. + */ + ret = syncfs(req->xfd->fd); + if (ret) { + perror(_("syncing filesystem")); + return ret; + } + + ret = csp_collect_garbage(req); + if (ret) + return ret; + + /* + * Set up the space capture file as a large sparse file mirroring the + * physical space that we want to defragment. + */ + ret = ftruncate(req->space_fd, req->start + req->length); + if (ret) { + perror(_("setting up space capture file")); + return ret; + } + + /* + * If we don't have reflink, just grab the free space and move on to + * copying and exchanging file contents. + */ + if (!req->use_reflink) + return csp_grab_free_space(req); + + /* + * Try to freeze as much of the requested range as we can, grab the + * free space in that range, and run freeze again to pick up anything + * that may have been allocated while all that was going on. + */ + do { + struct stat statbuf; + + ret = csp_freeze_req_range(req); + if (ret) + return ret; + + ret = csp_grab_free_space(req); + if (ret) + return ret; + + ret = fstat(req->space_fd, &statbuf); + if (ret) + return ret; + + if (old_blocks == statbuf.st_blocks) + break; + old_blocks = statbuf.st_blocks; + } while (1); + + /* + * If reflink is enabled, our strategy is to dedupe to free blocks in + * the area that we're clearing without making any user-visible changes + * to the file contents. For all the written file data blocks in area + * we're clearing, make an identical copy in the work file that is + * backed by blocks that are not in the clearing area. + */ + return csp_prepare_for_dedupe(req); +} + /* Set up the target to clear all metadata from the given range. */ static inline void csp_target_metadata( @@ -2330,50 +2587,10 @@ clearspace_run( return ret; } - /* - * Empty out CoW forks and speculative post-EOF preallocations before - * starting the clearing process. This may be somewhat overkill. - */ - ret = syncfs(req->xfd->fd); - if (ret) { - perror(_("syncing filesystem")); - goto out_bitmap; - } - - ret = csp_collect_garbage(req); - if (ret) - goto out_bitmap; - - /* - * Try to freeze as much of the requested range as we can, grab the - * free space in that range, and run freeze again to pick up anything - * that may have been allocated while all that was going on. - */ - ret = csp_freeze_req_range(req); - if (ret) - goto out_bitmap; - - ret = csp_grab_free_space(req); - if (ret) - goto out_bitmap; - - ret = csp_freeze_req_range(req); + ret = csp_prepare(req); if (ret) goto out_bitmap; - /* - * If reflink is enabled, our strategy is to dedupe to free blocks in - * the area that we're clearing without making any user-visible changes - * to the file contents. For all the written file data blocks in area - * we're clearing, make an identical copy in the work file that is - * backed by blocks that are not in the clearing area. - */ - if (req->use_reflink) { - ret = csp_prepare_for_dedupe(req); - if (ret) - goto out_bitmap; - } - /* Evacuate as many file blocks as we can. */ do { ret = csp_find_target(req, &target); ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/1] xfs_scrub: vectorize kernel calls 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (6 preceding siblings ...) 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong @ 2022-12-30 22:21 ` Darrick J. Wong 2022-12-30 22:21 ` [PATCH 1/1] xfs/122: update for vectored scrub Darrick J. Wong 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: functional test for refcount reporting Darrick J. Wong 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: defragment free space Darrick J. Wong 9 siblings, 1 reply; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:21 UTC (permalink / raw) To: djwong, zlang; +Cc: linux-xfs, fstests, guan Hi all, Create a vectorized version of the metadata scrub and repair ioctl, and adapt xfs_scrub to use that. This is an experiment to measure overhead and to try refactoring xfs_scrub. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=vectorized-scrub xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=vectorized-scrub fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=vectorized-scrub --- tests/xfs/122.out | 2 ++ 1 file changed, 2 insertions(+) ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 1/1] xfs/122: update for vectored scrub 2022-12-30 22:21 ` [PATCHSET 0/1] xfs_scrub: vectorize kernel calls Darrick J. Wong @ 2022-12-30 22:21 ` Darrick J. Wong 0 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:21 UTC (permalink / raw) To: djwong, zlang; +Cc: linux-xfs, fstests, guan From: Darrick J. Wong <djwong@kernel.org> Add the two new vectored scrub structures. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- tests/xfs/122.out | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index 3239a655f9..43461e875c 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -126,6 +126,8 @@ sizeof(struct xfs_rtsb) = 104 sizeof(struct xfs_rud_log_format) = 16 sizeof(struct xfs_rui_log_format) = 16 sizeof(struct xfs_scrub_metadata) = 64 +sizeof(struct xfs_scrub_vec) = 16 +sizeof(struct xfs_scrub_vec_head) = 24 sizeof(struct xfs_swap_extent) = 64 sizeof(struct xfs_sxd_log_format) = 16 sizeof(struct xfs_sxi_log_format) = 80 ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/1] fstests: functional test for refcount reporting 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (7 preceding siblings ...) 2022-12-30 22:21 ` [PATCHSET 0/1] xfs_scrub: vectorize kernel calls Darrick J. Wong @ 2022-12-30 22:21 ` Darrick J. Wong 2022-12-30 22:21 ` [PATCH 1/1] xfs: test output of new FSREFCOUNTS ioctl Darrick J. Wong 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: defragment free space Darrick J. Wong 9 siblings, 1 reply; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:21 UTC (permalink / raw) To: djwong, zlang; +Cc: linux-xfs, fstests, guan Hi all, Add a short functional test for the new GETFSREFCOUNTS ioctl that allows userspace to query reference count information for a given range of physical blocks. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=report-refcounts xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=report-refcounts fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=report-refcounts --- common/rc | 4 + doc/group-names.txt | 1 tests/xfs/921 | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/921.out | 4 + 4 files changed, 175 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/921 create mode 100644 tests/xfs/921.out ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 1/1] xfs: test output of new FSREFCOUNTS ioctl 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: functional test for refcount reporting Darrick J. Wong @ 2022-12-30 22:21 ` Darrick J. Wong 0 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:21 UTC (permalink / raw) To: djwong, zlang; +Cc: linux-xfs, fstests, guan From: Darrick J. Wong <djwong@kernel.org> Make sure the cursors work properly and that refcounts are correct. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- common/rc | 4 + doc/group-names.txt | 1 tests/xfs/921 | 168 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/921.out | 4 + 4 files changed, 175 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/921 create mode 100644 tests/xfs/921.out diff --git a/common/rc b/common/rc index 3c30a444fe..20fe51f502 100644 --- a/common/rc +++ b/common/rc @@ -2543,8 +2543,8 @@ _require_xfs_io_command() echo $testio | grep -q "Operation not supported" && \ _notrun "O_TMPFILE is not supported" ;; - "fsmap") - testio=`$XFS_IO_PROG -f -c "fsmap" $testfile 2>&1` + "fsmap"|"fsrefcounts") + testio=`$XFS_IO_PROG -f -c "$command" $testfile 2>&1` echo $testio | grep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;; diff --git a/doc/group-names.txt b/doc/group-names.txt index e88dcc0fdd..8bcf21919b 100644 --- a/doc/group-names.txt +++ b/doc/group-names.txt @@ -56,6 +56,7 @@ freeze filesystem freeze tests fsck general fsck tests fsmap FS_IOC_GETFSMAP ioctl fsr XFS free space reorganizer +fsrefcounts FS_IOC_GETFSREFCOUNTS ioctl fuzzers filesystem fuzz tests growfs increasing the size of a filesystem hardlink hardlinks diff --git a/tests/xfs/921 b/tests/xfs/921 new file mode 100755 index 0000000000..bc9894b1d7 --- /dev/null +++ b/tests/xfs/921 @@ -0,0 +1,168 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Oracle and/or its affiliates. All Rights Reserved. +# +# FS QA Test No. 921 +# +# Populate filesystem, check that fsrefcounts -n10000 matches fsrefcounts -n1, +# then verify that the refcount information is consistent with the fsmap info. +# +. ./common/preamble +_begin_fstest auto clone fsrefcounts fsmap + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -rf $tmp.* $TEST_DIR/a $TEST_DIR/b +} + +# Import common functions. +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_xfs_io_command "fsmap" +_require_xfs_io_command "fsrefcounts" + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +cpus=$(( $(src/feature -o) * 4)) + +# Use fsstress to create a directory tree with some variability +FSSTRESS_ARGS=$(_scale_fsstress_args -p 4 -d $SCRATCH_MNT -n 4000 $FSSTRESS_AVOID) +$FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full + +echo "Compare fsrefcounts" | tee -a $seqres.full +$XFS_IO_PROG -c 'fsrefcounts -m -n 65536' $SCRATCH_MNT | grep -v 'EXT:' > $TEST_DIR/a +$XFS_IO_PROG -c 'fsrefcounts -m -n 1' $SCRATCH_MNT | grep -v 'EXT:' > $TEST_DIR/b +cat $TEST_DIR/a $TEST_DIR/b >> $seqres.full + +diff -uw $TEST_DIR/a $TEST_DIR/b + +echo "Compare fsrefcounts to fsmap" | tee -a $seqres.full +$XFS_IO_PROG -c 'fsmap -m -n 65536' $SCRATCH_MNT | grep -v 'EXT:' > $TEST_DIR/b +cat $TEST_DIR/b >> $seqres.full + +while IFS=',' read ext major minor pstart pend owners length crap; do + test "$ext" = "EXT" && continue + + awk_args=(-'F' ',' '-v' "major=$major" '-v' "minor=$minor" \ + '-v' "pstart=$pstart" '-v' "pend=$pend" '-v' "owners=$owners") + + if [ "$owners" -eq 1 ]; then + $AWK_PROG "${awk_args[@]}" \ +' +BEGIN { + printf("Q:%s:%s:%s:%s:%s:\n", major, minor, pstart, pend, owners) > "/dev/stderr"; + next_map = -1; +} +{ + if ($2 != major || $3 != minor) { + next; + } + if ($5 <= pstart) { + next; + } + + printf(" A:%s:%s:%s:%s\n", $2, $3, $4, $5) > "/dev/stderr"; + if (next_map < 0) { + if ($4 > pstart) { + exit 1 + } + next_map = $5 + 1; + } else { + if ($4 != next_map) { + exit 1 + } + next_map = $5 + 1; + } + if (next_map >= pend) { + nextfile; + } +} +END { + exit 0; +} +' $TEST_DIR/b 2> $tmp.debug + res=$? + else + $AWK_PROG "${awk_args[@]}" \ +' +function max(a, b) { + return a > b ? a : b; +} +function min(a, b) { + return a < b ? a : b; +} +BEGIN { + printf("Q:%s:%s:%s:%s:%s:\n", major, minor, pstart, pend, owners) > "/dev/stderr"; + refcount_whole = 0; + aborted = 0; +} +{ + if ($2 != major || $3 != minor) { + next; + } + if ($4 >= pend) { + nextfile; + } + if ($5 <= pstart) { + next; + } + if ($6 == "special_0:2") { + /* unknown owner means we cannot distinguish separate owners */ + aborted = 1; + exit 0; + } + + printf(" A:%s:%s:%s:%s -> %d\n", $2, $3, $4, $5, refcount_whole) > "/dev/stderr"; + if ($4 <= pstart && $5 >= pend) { + /* Account for extents that span the whole range */ + refcount_whole++; + } else { + /* Otherwise track refcounts per-block as we find them */ + for (block = max($4, pstart); block <= min($5, pend); block++) { + refcounts[block]++; + } + } +} +END { + if (aborted) { + exit 0; + } + deficit = owners - refcount_whole; + printf(" W:%d:%d\n", owners, refcount_whole, deficit) > "/dev/stderr"; + if (deficit == 0) { + exit 0; + } + + refcount_slivers = 0; + for (block in refcounts) { + printf(" X:%s:%d\n", block, refcounts[block]) > "/dev/stderr"; + if (refcounts[block] == deficit) { + refcount_slivers = deficit; + } else { + exit 1; + } + } + + refcount_whole += refcount_slivers; + exit owners == refcount_whole ? 0 : 1; +} +' $TEST_DIR/b 2> $tmp.debug + res=$? + fi + if [ $res -ne 0 ]; then + echo "$major,$minor,$pstart,$pend,$owners not found in fsmap" + cat $tmp.debug >> $seqres.full + break + fi +done < $TEST_DIR/a + +# success, all done +status=0 +exit diff --git a/tests/xfs/921.out b/tests/xfs/921.out new file mode 100644 index 0000000000..a181d357cd --- /dev/null +++ b/tests/xfs/921.out @@ -0,0 +1,4 @@ +QA output created by 921 +Format and mount +Compare fsrefcounts +Compare fsrefcounts to fsmap ^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCHSET 0/1] fstests: defragment free space 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong ` (8 preceding siblings ...) 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: functional test for refcount reporting Darrick J. Wong @ 2022-12-30 22:21 ` Darrick J. Wong 2022-12-30 22:21 ` [PATCH 1/1] xfs: test clearing of " Darrick J. Wong 9 siblings, 1 reply; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:21 UTC (permalink / raw) To: djwong, zlang; +Cc: linux-xfs, fstests, guan Hi all, These patches contain experimental code to enable userspace to defragment the free space in a filesystem. Two purposes are imagined for this functionality: clearing space at the end of a filesystem before shrinking it, and clearing free space in anticipation of making a large allocation. The first patch adds a new fallocate mode that allows userspace to allocate free space from the filesystem into a file. The goal here is to allow the filesystem shrink process to prevent allocation from a certain part of the filesystem while a free space defragmenter evacuates all the files from the doomed part of the filesystem. The second patch amends the online repair system to allow the sysadmin to forcibly rebuild metadata structures, even if they're not corrupt. Without adding an ioctl to move metadata btree blocks, this is the only way to dislodge metadata. If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=defrag-freespace --- common/rc | 2 + tests/xfs/1400 | 57 +++++++++++++++++++++++++++++++++++++ tests/xfs/1400.out | 2 + tests/xfs/1401 | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1401.out | 2 + 5 files changed, 142 insertions(+), 1 deletion(-) create mode 100755 tests/xfs/1400 create mode 100644 tests/xfs/1400.out create mode 100755 tests/xfs/1401 create mode 100644 tests/xfs/1401.out ^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH 1/1] xfs: test clearing of free space 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: defragment free space Darrick J. Wong @ 2022-12-30 22:21 ` Darrick J. Wong 0 siblings, 0 replies; 40+ messages in thread From: Darrick J. Wong @ 2022-12-30 22:21 UTC (permalink / raw) To: djwong, zlang; +Cc: linux-xfs, fstests, guan From: Darrick J. Wong <djwong@kernel.org> Simple regression test for the spaceman clearspace command, which tries to free all the used space in some part of the filesystem. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- common/rc | 2 + tests/xfs/1400 | 57 +++++++++++++++++++++++++++++++++++++ tests/xfs/1400.out | 2 + tests/xfs/1401 | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1401.out | 2 + 5 files changed, 142 insertions(+), 1 deletion(-) create mode 100755 tests/xfs/1400 create mode 100644 tests/xfs/1400.out create mode 100755 tests/xfs/1401 create mode 100644 tests/xfs/1401.out diff --git a/common/rc b/common/rc index 20fe51f502..bf1d0ded39 100644 --- a/common/rc +++ b/common/rc @@ -2512,7 +2512,7 @@ _require_xfs_io_command() testio=`$XFS_IO_PROG -F -f -c "$command $param 0 1m" $testfile 2>&1` param_checked="$param" ;; - "fpunch" | "fcollapse" | "zero" | "fzero" | "finsert" | "funshare") + "fpunch" | "fcollapse" | "zero" | "fzero" | "finsert" | "funshare" | "fmapfree") local blocksize=$(_get_file_block_size $TEST_DIR) testio=`$XFS_IO_PROG -F -f -c "pwrite 0 $((5 * $blocksize))" \ -c "fsync" -c "$command $blocksize $((2 * $blocksize))" \ diff --git a/tests/xfs/1400 b/tests/xfs/1400 new file mode 100755 index 0000000000..c054bf6ed7 --- /dev/null +++ b/tests/xfs/1400 @@ -0,0 +1,57 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Oracle. All Rights Reserved. +# +# FS QA Test 1400 +# +# Basic functionality testing for FALLOC_FL_MAP_FREE +# +. ./common/preamble +_begin_fstest auto prealloc + +# Import common functions. +. ./common/filter + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_require_scratch +_require_xfs_io_command "fmapfree" + +_scratch_mkfs | _filter_mkfs 2> $tmp.mkfs > /dev/null +_scratch_mount >> $seqres.full +. $tmp.mkfs + +testfile="$SCRATCH_MNT/$seq.txt" +touch $testfile +if $XFS_IO_PROG -c 'stat -v' $testfile | grep -q 'realtime'; then + # realtime + increment=$((dbsize * rtblocks / 10)) + length=$((dbsize * rtblocks)) +else + # data + increment=$((dbsize * dblocks / 10)) + length=$((dbsize * dblocks)) +fi + +free_bytes=$(stat -f -c '%f * %S' $testfile | bc) + +echo "free space: $free_bytes; increment: $increment; length: $length" >> $seqres.full + +# Map all the free space on that device, 10% at a time +for ((start = 0; start < length; start += increment)); do + $XFS_IO_PROG -f -c "fmapfree $start $increment" $testfile +done + +space_used=$(stat -c '%b * %B' $testfile | bc) + +echo "space captured: $space_used" >> $seqres.full +$FILEFRAG_PROG -v $testfile >> $seqres.full + +# Did we get within 10% of the free space? +_within_tolerance "mapfree space used" $space_used $free_bytes 10% -v + +# success, all done +status=0 +exit diff --git a/tests/xfs/1400.out b/tests/xfs/1400.out new file mode 100644 index 0000000000..601404d7a4 --- /dev/null +++ b/tests/xfs/1400.out @@ -0,0 +1,2 @@ +QA output created by 1400 +mapfree space used is in range diff --git a/tests/xfs/1401 b/tests/xfs/1401 new file mode 100755 index 0000000000..8c0a545858 --- /dev/null +++ b/tests/xfs/1401 @@ -0,0 +1,80 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Oracle. All Rights Reserved. +# +# FS QA Test No. 1401 +# +# Basic functionality testing for the free space defragmenter. +# +. ./common/preamble +_begin_fstest auto defrag shrinkfs + +# Override the default cleanup function. +# _cleanup() +# { +# cd / +# rm -r -f $tmp.* +# } + +# Import common functions. +. ./common/filter + +# real QA test starts here + +_notrun "XXX test is not ready yet; you need to deal with tail blocks" + +# Modify as appropriate. +_supported_fs generic +_require_scratch +_require_xfs_spaceman_command "clearfree" + +_scratch_mkfs | _filter_mkfs 2> $tmp.mkfs > /dev/null +cat $tmp.mkfs >> $seqres.full +. $tmp.mkfs +_scratch_mount >> $seqres.full + +cpus=$(( $(src/feature -o) * 4)) + +# Use fsstress to create a directory tree with some variability +FSSTRESS_ARGS=$(_scale_fsstress_args -p 4 -d $SCRATCH_MNT -n 4000 $FSSTRESS_AVOID) +$FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full + +$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT >> $seqres.full + +if $XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT | grep -q 'rt-inherit'; then + # realtime + increment=$((dbsize * rtblocks / agcount)) + length=$((dbsize * rtblocks)) + fsmap_devarg="-r" +else + # data + increment=$((dbsize * agsize)) + length=$((dbsize * dblocks)) + fsmap_devarg="-d" +fi + +echo "start: $start; increment: $increment; length: $length" >> $seqres.full +$DF_PROG $SCRATCH_MNT >> $seqres.full + +TRACE_PROG="strace -s99 -e fallocate,ioctl,openat -o $tmp.strace" + +for ((start = 0; start < length; start += increment)); do + echo "---------------------------" >> $seqres.full + echo "start: $start end: $((start + increment))" >> $seqres.full + echo "---------------------------" >> $seqres.full + + fsmap_args="-vvvv $fsmap_devarg $((start / 512)) $((increment / 512))" + clearfree_args="-vall $start $increment" + + $XFS_IO_PROG -c "fsmap $fsmap_args" $SCRATCH_MNT > $tmp.before + $TRACE_PROG $XFS_SPACEMAN_PROG -c "clearfree $clearfree_args" $SCRATCH_MNT &>> $seqres.full || break + cat $tmp.strace >> $seqres.full + $XFS_IO_PROG -c "fsmap $fsmap_args" $SCRATCH_MNT > $tmp.after + cat $tmp.before >> $seqres.full + cat $tmp.after >> $seqres.full +done + +# success, all done +echo Silence is golden +status=0 +exit diff --git a/tests/xfs/1401.out b/tests/xfs/1401.out new file mode 100644 index 0000000000..504999381e --- /dev/null +++ b/tests/xfs/1401.out @@ -0,0 +1,2 @@ +QA output created by 1401 +Silence is golden ^ permalink raw reply related [flat|nested] 40+ messages in thread
end of thread, other threads:[~2022-12-31 3:30 UTC | newest] Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-12-30 21:14 [NYE DELUGE 4/4] xfs: freespace defrag for online shrink Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: improve post-close eofblocks gc behavior Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: don't free EOF blocks on read close Darrick J. Wong 2022-12-30 22:19 ` [PATCH 3/3] xfs: Don't free EOF blocks on close when extent size hints are set Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/3] xfs: only free posteof blocks on first close Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/3] xfs: vectorize scrub kernel calls Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/3] xfs: whine to dmesg when we encounter errors Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/3] xfs: track deferred ops statistics Darrick J. Wong 2022-12-30 22:19 ` [PATCH 3/3] xfs: introduce vectored scrub mode Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/1] xfs: report refcount information to userspace Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/1] xfs: export reference count " Darrick J. Wong 2022-12-30 22:19 ` [PATCHSET 0/2] xfs: defragment free space Darrick J. Wong 2022-12-30 22:19 ` [PATCH 2/2] xfs: fallocate free space into a file Darrick J. Wong 2022-12-30 22:19 ` [PATCH 1/2] xfs: capture the offset and length in fallocate tracepoints Darrick J. Wong 2022-12-30 22:20 ` [PATCHSET 00/11] xfs_scrub: vectorize kernel calls Darrick J. Wong 2022-12-30 22:20 ` [PATCH 03/11] libfrog: support vectored scrub Darrick J. Wong 2022-12-30 22:20 ` [PATCH 04/11] xfs_io: " Darrick J. Wong 2022-12-30 22:20 ` [PATCH 01/11] xfs: track deferred ops statistics Darrick J. Wong 2022-12-30 22:20 ` [PATCH 02/11] xfs: introduce vectored scrub mode Darrick J. Wong 2022-12-30 22:20 ` [PATCH 08/11] xfs_scrub: vectorize scrub calls Darrick J. Wong 2022-12-30 22:20 ` [PATCH 09/11] xfs_scrub: vectorize repair calls Darrick J. Wong 2022-12-30 22:20 ` [PATCH 07/11] xfs_scrub: convert scrub and repair epilogues to use xfs_scrub_vec Darrick J. Wong 2022-12-30 22:20 ` [PATCH 11/11] xfs_scrub: try spot repairs of metadata items to make scrub progress Darrick J. Wong 2022-12-30 22:20 ` [PATCH 06/11] xfs_scrub: split the repair epilogue code into a separate function Darrick J. Wong 2022-12-30 22:20 ` [PATCH 05/11] xfs_scrub: split the scrub " Darrick J. Wong 2022-12-30 22:20 ` [PATCH 10/11] xfs_scrub: use scrub barriers to reduce kernel calls Darrick J. Wong 2022-12-30 22:20 ` [PATCHSET 0/1] libxfs: report refcount information to userspace Darrick J. Wong 2022-12-30 22:20 ` [PATCH 1/1] xfs_io: dump reference count information Darrick J. Wong 2022-12-30 22:20 ` [PATCHSET 0/5] xfsprogs: defragment free space Darrick J. Wong 2022-12-30 22:20 ` [PATCH 1/5] xfs: fallocate free space into a file Darrick J. Wong 2022-12-30 22:20 ` [PATCH 2/5] xfs_io: support using fallocate to map free space Darrick J. Wong 2022-12-30 22:20 ` [PATCH 4/5] xfs_spaceman: implement clearing " Darrick J. Wong 2022-12-30 22:20 ` [PATCH 3/5] xfs_db: get and put blocks on the AGFL Darrick J. Wong 2022-12-30 22:20 ` [PATCH 5/5] xfs_spaceman: defragment free space with normal files Darrick J. Wong 2022-12-30 22:21 ` [PATCHSET 0/1] xfs_scrub: vectorize kernel calls Darrick J. Wong 2022-12-30 22:21 ` [PATCH 1/1] xfs/122: update for vectored scrub Darrick J. Wong 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: functional test for refcount reporting Darrick J. Wong 2022-12-30 22:21 ` [PATCH 1/1] xfs: test output of new FSREFCOUNTS ioctl Darrick J. Wong 2022-12-30 22:21 ` [PATCHSET 0/1] fstests: defragment free space Darrick J. Wong 2022-12-30 22:21 ` [PATCH 1/1] xfs: test clearing of " Darrick J. Wong
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.