[NYE DELUGE 3/4] xfs: modernize the realtime volume

[NYE DELUGE 3/4] xfs: modernize the realtime volume

[Darrick J. Wong]

Hi all,

This third patchset deluge is for the realtime modernization project.
There are five main parts to this effort -- adding a metadata directory
tree; sharding the realtime volume into allocation groups to reduce
metadata lock contention; adding reverse mapping; adding reflink; and
adding the one piece needed to make quotas work on realtime.  This
brings the robustness of the realtime volume up to par with the data
volume.

Originally, the modernization effort was a side project that was
intended to match XFS up to the proliferation of persistent memory.  The
data device would store metadata on cheap(er) flash storage, and the
realtime volume would be used to map persistent memory to files and take
advantage of the ability to do PMD-aligned allocations.  It's now less
clear how much of that will actually happen (CXL?), but the code's
finished, reasonably well tested, and ready for review.

NOTE: I hacked up metadump to support saving the metadata contents of
external logs and realtime devices so that I could run fuzz testing in
the least thoughtful way possible.  Chandan is working on improving the
deployment image story for our customers, and will likely produce
something better than my rush job.

As a warning, the patches will likely take several days to trickle in.

--D

[PATCHSET 0/1] fstests: test upgrading older features

[Darrick J. Wong]

Hi all,

Here is a general regression test to make sure that we can invoke the
xfs_repair feature to add new features to V5 filesystems without errors.
There are already targeted functionality tests for inobtcount and
bigtime; this new one exists as a general upgrade exerciser.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=upgrade-older-features

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=upgrade-older-features
---
 tests/xfs/769     |  248 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/769.out |    2 
 2 files changed, 250 insertions(+)
 create mode 100755 tests/xfs/769
 create mode 100644 tests/xfs/769.out

[PATCH 1/1] xfs: test upgrading old features

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Test the ability to add older v5 features.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/769     |  248 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/769.out |    2 
 2 files changed, 250 insertions(+)
 create mode 100755 tests/xfs/769
 create mode 100644 tests/xfs/769.out


diff --git a/tests/xfs/769 b/tests/xfs/769
new file mode 100755
index 0000000000..7613048f52
--- /dev/null
+++ b/tests/xfs/769
@@ -0,0 +1,248 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 769
+#
+# Test upgrading filesystems with new features.
+#
+. ./common/preamble
+_begin_fstest auto mkfs repair
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+
+# real QA test starts here
+_supported_fs xfs
+
+test -w /dev/ttyprintk || _notrun "test requires writable /dev/ttyprintk"
+_require_check_dmesg
+_require_scratch_nocheck
+_require_scratch_xfs_crc
+
+# Does repair know how to add a particular feature to a filesystem?
+check_repair_upgrade()
+{
+	$XFS_REPAIR_PROG -c "$1=narf" 2>&1 | \
+		grep -q 'unknown option' && return 1
+	return 0
+}
+
+# Are we configured for realtime?
+rt_configured()
+{
+	test "$USE_EXTERNAL" = "yes" && test -n "$SCRATCH_RTDEV"
+}
+
+# Compute the MKFS_OPTIONS string for a particular feature upgrade test
+compute_mkfs_options()
+{
+	local m_opts=""
+	local caller_options="$MKFS_OPTIONS"
+
+	for feat in "${FEATURES[@]}"; do
+		local feat_state="${FEATURE_STATE["${feat}"]}"
+
+		if echo "$caller_options" | grep -E -w -q "${feat}=[0-9]*"; then
+			# Change the caller's options
+			caller_options="$(echo "$caller_options" | \
+				sed -e "s/\([^[:alnum:]]\)${feat}=[0-9]*/\1${feat}=${feat_state}/g")"
+		else
+			# Add it to our list of new mkfs flags
+			m_opts="${feat}=${feat_state},${m_opts}"
+		fi
+	done
+
+	test -n "$m_opts" && m_opts=" -m $m_opts"
+
+	echo "$caller_options$m_opts"
+}
+
+# Log the start of an upgrade.
+function upgrade_start_message()
+{
+	local feat="$1"
+
+	echo "Add $feat to filesystem"
+}
+
+# Find dmesg log messages since we started a particular upgrade test
+function dmesg_since_feature_upgrade_start()
+{
+	local feat_logmsg="$(upgrade_start_message "$1")"
+
+	# search the dmesg log of last run of $seqnum for possible failures
+	# use sed \cregexpc address type, since $seqnum contains "/"
+	dmesg | \
+		tac | \
+		sed -ne "0,\#run fstests $seqnum at $date_time#p" | \
+		sed -ne "0,\#${feat_logmsg}#p" | \
+		tac
+}
+
+# Did the mount fail because this feature is not supported?
+function feature_unsupported()
+{
+	local feat="$1"
+
+	dmesg_since_feature_upgrade_start "$feat" | \
+		grep -q 'has unknown.*features'
+}
+
+# Exercise the scratch fs
+function scratch_fsstress()
+{
+	echo moo > $SCRATCH_MNT/sample.txt
+	$FSSTRESS_PROG -n $((TIME_FACTOR * 1000)) -p $((LOAD_FACTOR * 4)) \
+		-d $SCRATCH_MNT/data >> $seqres.full
+}
+
+# Exercise the filesystem a little bit and emit a manifest.
+function pre_exercise()
+{
+	local feat="$1"
+
+	_try_scratch_mount &> $tmp.mount
+	res=$?
+	# If the kernel doesn't support the filesystem even after a
+	# fresh format, skip the rest of the upgrade test quietly.
+	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
+		echo "mount failed due to unsupported feature $feat" >> $seqres.full
+		return 1
+	fi
+	if [ $res -ne 0 ]; then
+		cat $tmp.mount
+		echo "mount failed with $res before upgrading to $feat" | \
+			tee -a $seqres.full
+		return 1
+	fi
+
+	scratch_fsstress
+	find $SCRATCH_MNT -type f -print0 | xargs -r -0 md5sum > $tmp.manifest
+	_scratch_unmount
+	return 0
+}
+
+# Check the manifest and exercise the filesystem more
+function post_exercise()
+{
+	local feat="$1"
+
+	_try_scratch_mount &> $tmp.mount
+	res=$?
+	# If the kernel doesn't support the filesystem even after a
+	# fresh format, skip the rest of the upgrade test quietly.
+	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
+		echo "mount failed due to unsupported feature $feat" >> $seqres.full
+		return 1
+	fi
+	if [ $res -ne 0 ]; then
+		cat $tmp.mount
+		echo "mount failed with $res after upgrading to $feat" | \
+			tee -a $seqres.full
+		return 1
+	fi
+
+	md5sum --quiet -c $tmp.manifest || \
+		echo "fs contents ^^^ changed after adding $feat"
+
+	iam="check" _check_scratch_fs || \
+		echo "scratch fs check failed after adding $feat"
+
+	# Try to mount the fs in case the check unmounted it
+	_try_scratch_mount &>> $seqres.full
+
+	scratch_fsstress
+
+	iam="check" _check_scratch_fs || \
+		echo "scratch fs check failed after exercising $feat"
+
+	# Try to unmount the fs in case the check didn't
+	_scratch_unmount &>> $seqres.full
+	return 0
+}
+
+# Create a list of fs features in the order that support for them was added
+# to the kernel driver.  For each feature upgrade test, we enable all the
+# features that came before it and none of the ones after, which means we're
+# testing incremental migrations.  We start each run with a clean fs so that
+# errors and unsatisfied requirements (log size, root ino position, etc) in one
+# upgrade don't spread failure to the rest of the tests.
+FEATURES=()
+if rt_configured; then
+	check_repair_upgrade finobt && FEATURES+=("finobt")
+	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
+	check_repair_upgrade bigtime && FEATURES+=("bigtime")
+else
+	check_repair_upgrade finobt && FEATURES+=("finobt")
+	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
+	check_repair_upgrade reflink && FEATURES+=("reflink")
+	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
+	check_repair_upgrade bigtime && FEATURES+=("bigtime")
+fi
+
+test "${#FEATURES[@]}" -eq 0 && \
+	_notrun "xfs_repair does not know how to add V5 features"
+
+declare -A FEATURE_STATE
+for f in "${FEATURES[@]}"; do
+	FEATURE_STATE["$f"]=0
+done
+
+for feat in "${FEATURES[@]}"; do
+	echo "-----------------------" >> $seqres.full
+
+	upgrade_start_message "$feat" | tee -a $seqres.full /dev/ttyprintk > /dev/null
+
+	opts="$(compute_mkfs_options)"
+	echo "mkfs.xfs $opts" >> $seqres.full
+
+	# Format filesystem
+	MKFS_OPTIONS="$opts" _scratch_mkfs &>> $seqres.full
+	res=$?
+	outcome="mkfs returns $res for $feat upgrade test"
+	echo "$outcome" >> $seqres.full
+	if [ $res -ne 0 ]; then
+		echo "$outcome"
+		continue
+	fi
+
+	# Create some files to make things interesting.
+	pre_exercise "$feat" || break
+
+	# Upgrade the fs
+	_scratch_xfs_repair -c "${feat}=1" &> $tmp.upgrade
+	res=$?
+	cat $tmp.upgrade >> $seqres.full
+	grep -q "^Adding" $tmp.upgrade || \
+		echo "xfs_repair ignored command to add $feat"
+
+	outcome="xfs_repair returns $res while adding $feat"
+	echo "$outcome" >> $seqres.full
+	if [ $res -ne 0 ]; then
+		# Couldn't upgrade filesystem, move on to the next feature.
+		FEATURE_STATE["$feat"]=1
+		continue
+	fi
+
+	# Make sure repair runs cleanly afterwards
+	_scratch_xfs_repair -n &>> $seqres.full
+	res=$?
+	outcome="xfs_repair -n returns $res after adding $feat"
+	echo "$outcome" >> $seqres.full
+	if [ $res -ne 0 ]; then
+		echo "$outcome"
+	fi
+
+	# Make sure we can still exercise the filesystem.
+	post_exercise "$feat" || break
+
+	# Update feature state for next run
+	FEATURE_STATE["$feat"]=1
+done
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/xfs/769.out b/tests/xfs/769.out
new file mode 100644
index 0000000000..332432db97
--- /dev/null
+++ b/tests/xfs/769.out
@@ -0,0 +1,2 @@
+QA output created by 769
+Silence is golden.

Re: [PATCH 1/1] xfs: test upgrading old features

[Zorro Lang]

On Fri, Dec 30, 2022 at 02:20:29PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Test the ability to add older v5 features.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/769     |  248 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/769.out |    2 
>  2 files changed, 250 insertions(+)
>  create mode 100755 tests/xfs/769
>  create mode 100644 tests/xfs/769.out
> 
> 
> diff --git a/tests/xfs/769 b/tests/xfs/769
> new file mode 100755
> index 0000000000..7613048f52
> --- /dev/null
> +++ b/tests/xfs/769
> @@ -0,0 +1,248 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 769
> +#
> +# Test upgrading filesystems with new features.
> +#
> +. ./common/preamble
> +_begin_fstest auto mkfs repair
> +
> +# Import common functions.
> +. ./common/filter
> +. ./common/populate
> +
> +# real QA test starts here
> +_supported_fs xfs
> +
> +test -w /dev/ttyprintk || _notrun "test requires writable /dev/ttyprintk"

Hi Darrick,

I'm not sure why /dev/ttyprintk is necessary. I think sometimes we might not
have this driver, but has /dev/kmsg. Can /dev/kmsg be a replacement of
ttyprintk ?

Thanks,
Zorro


> +_require_check_dmesg
> +_require_scratch_nocheck
> +_require_scratch_xfs_crc
> +
> +# Does repair know how to add a particular feature to a filesystem?
> +check_repair_upgrade()
> +{
> +	$XFS_REPAIR_PROG -c "$1=narf" 2>&1 | \
> +		grep -q 'unknown option' && return 1
> +	return 0
> +}
> +
> +# Are we configured for realtime?
> +rt_configured()
> +{
> +	test "$USE_EXTERNAL" = "yes" && test -n "$SCRATCH_RTDEV"
> +}
> +
> +# Compute the MKFS_OPTIONS string for a particular feature upgrade test
> +compute_mkfs_options()
> +{
> +	local m_opts=""
> +	local caller_options="$MKFS_OPTIONS"
> +
> +	for feat in "${FEATURES[@]}"; do
> +		local feat_state="${FEATURE_STATE["${feat}"]}"
> +
> +		if echo "$caller_options" | grep -E -w -q "${feat}=[0-9]*"; then
> +			# Change the caller's options
> +			caller_options="$(echo "$caller_options" | \
> +				sed -e "s/\([^[:alnum:]]\)${feat}=[0-9]*/\1${feat}=${feat_state}/g")"
> +		else
> +			# Add it to our list of new mkfs flags
> +			m_opts="${feat}=${feat_state},${m_opts}"
> +		fi
> +	done
> +
> +	test -n "$m_opts" && m_opts=" -m $m_opts"
> +
> +	echo "$caller_options$m_opts"
> +}
> +
> +# Log the start of an upgrade.
> +function upgrade_start_message()
> +{
> +	local feat="$1"
> +
> +	echo "Add $feat to filesystem"
> +}
> +
> +# Find dmesg log messages since we started a particular upgrade test
> +function dmesg_since_feature_upgrade_start()
> +{
> +	local feat_logmsg="$(upgrade_start_message "$1")"
> +
> +	# search the dmesg log of last run of $seqnum for possible failures
> +	# use sed \cregexpc address type, since $seqnum contains "/"
> +	dmesg | \
> +		tac | \
> +		sed -ne "0,\#run fstests $seqnum at $date_time#p" | \
> +		sed -ne "0,\#${feat_logmsg}#p" | \
> +		tac
> +}
> +
> +# Did the mount fail because this feature is not supported?
> +function feature_unsupported()
> +{
> +	local feat="$1"
> +
> +	dmesg_since_feature_upgrade_start "$feat" | \
> +		grep -q 'has unknown.*features'
> +}
> +
> +# Exercise the scratch fs
> +function scratch_fsstress()
> +{
> +	echo moo > $SCRATCH_MNT/sample.txt
> +	$FSSTRESS_PROG -n $((TIME_FACTOR * 1000)) -p $((LOAD_FACTOR * 4)) \
> +		-d $SCRATCH_MNT/data >> $seqres.full
> +}
> +
> +# Exercise the filesystem a little bit and emit a manifest.
> +function pre_exercise()
> +{
> +	local feat="$1"
> +
> +	_try_scratch_mount &> $tmp.mount
> +	res=$?
> +	# If the kernel doesn't support the filesystem even after a
> +	# fresh format, skip the rest of the upgrade test quietly.
> +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> +		return 1
> +	fi
> +	if [ $res -ne 0 ]; then
> +		cat $tmp.mount
> +		echo "mount failed with $res before upgrading to $feat" | \
> +			tee -a $seqres.full
> +		return 1
> +	fi
> +
> +	scratch_fsstress
> +	find $SCRATCH_MNT -type f -print0 | xargs -r -0 md5sum > $tmp.manifest
> +	_scratch_unmount
> +	return 0
> +}
> +
> +# Check the manifest and exercise the filesystem more
> +function post_exercise()
> +{
> +	local feat="$1"
> +
> +	_try_scratch_mount &> $tmp.mount
> +	res=$?
> +	# If the kernel doesn't support the filesystem even after a
> +	# fresh format, skip the rest of the upgrade test quietly.
> +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> +		return 1
> +	fi
> +	if [ $res -ne 0 ]; then
> +		cat $tmp.mount
> +		echo "mount failed with $res after upgrading to $feat" | \
> +			tee -a $seqres.full
> +		return 1
> +	fi
> +
> +	md5sum --quiet -c $tmp.manifest || \
> +		echo "fs contents ^^^ changed after adding $feat"
> +
> +	iam="check" _check_scratch_fs || \
> +		echo "scratch fs check failed after adding $feat"
> +
> +	# Try to mount the fs in case the check unmounted it
> +	_try_scratch_mount &>> $seqres.full
> +
> +	scratch_fsstress
> +
> +	iam="check" _check_scratch_fs || \
> +		echo "scratch fs check failed after exercising $feat"
> +
> +	# Try to unmount the fs in case the check didn't
> +	_scratch_unmount &>> $seqres.full
> +	return 0
> +}
> +
> +# Create a list of fs features in the order that support for them was added
> +# to the kernel driver.  For each feature upgrade test, we enable all the
> +# features that came before it and none of the ones after, which means we're
> +# testing incremental migrations.  We start each run with a clean fs so that
> +# errors and unsatisfied requirements (log size, root ino position, etc) in one
> +# upgrade don't spread failure to the rest of the tests.
> +FEATURES=()
> +if rt_configured; then
> +	check_repair_upgrade finobt && FEATURES+=("finobt")
> +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> +else
> +	check_repair_upgrade finobt && FEATURES+=("finobt")
> +	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
> +	check_repair_upgrade reflink && FEATURES+=("reflink")
> +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> +fi
> +
> +test "${#FEATURES[@]}" -eq 0 && \
> +	_notrun "xfs_repair does not know how to add V5 features"
> +
> +declare -A FEATURE_STATE
> +for f in "${FEATURES[@]}"; do
> +	FEATURE_STATE["$f"]=0
> +done
> +
> +for feat in "${FEATURES[@]}"; do
> +	echo "-----------------------" >> $seqres.full
> +
> +	upgrade_start_message "$feat" | tee -a $seqres.full /dev/ttyprintk > /dev/null
> +
> +	opts="$(compute_mkfs_options)"
> +	echo "mkfs.xfs $opts" >> $seqres.full
> +
> +	# Format filesystem
> +	MKFS_OPTIONS="$opts" _scratch_mkfs &>> $seqres.full
> +	res=$?
> +	outcome="mkfs returns $res for $feat upgrade test"
> +	echo "$outcome" >> $seqres.full
> +	if [ $res -ne 0 ]; then
> +		echo "$outcome"
> +		continue
> +	fi
> +
> +	# Create some files to make things interesting.
> +	pre_exercise "$feat" || break
> +
> +	# Upgrade the fs
> +	_scratch_xfs_repair -c "${feat}=1" &> $tmp.upgrade
> +	res=$?
> +	cat $tmp.upgrade >> $seqres.full
> +	grep -q "^Adding" $tmp.upgrade || \
> +		echo "xfs_repair ignored command to add $feat"
> +
> +	outcome="xfs_repair returns $res while adding $feat"
> +	echo "$outcome" >> $seqres.full
> +	if [ $res -ne 0 ]; then
> +		# Couldn't upgrade filesystem, move on to the next feature.
> +		FEATURE_STATE["$feat"]=1
> +		continue
> +	fi
> +
> +	# Make sure repair runs cleanly afterwards
> +	_scratch_xfs_repair -n &>> $seqres.full
> +	res=$?
> +	outcome="xfs_repair -n returns $res after adding $feat"
> +	echo "$outcome" >> $seqres.full
> +	if [ $res -ne 0 ]; then
> +		echo "$outcome"
> +	fi
> +
> +	# Make sure we can still exercise the filesystem.
> +	post_exercise "$feat" || break
> +
> +	# Update feature state for next run
> +	FEATURE_STATE["$feat"]=1
> +done
> +
> +# success, all done
> +echo Silence is golden.
> +status=0
> +exit
> diff --git a/tests/xfs/769.out b/tests/xfs/769.out
> new file mode 100644
> index 0000000000..332432db97
> --- /dev/null
> +++ b/tests/xfs/769.out
> @@ -0,0 +1,2 @@
> +QA output created by 769
> +Silence is golden.
> 

Re: [PATCH 1/1] xfs: test upgrading old features

[Darrick J. Wong]

On Mon, Mar 06, 2023 at 11:56:11PM +0800, Zorro Lang wrote:
> On Fri, Dec 30, 2022 at 02:20:29PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Test the ability to add older v5 features.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/769     |  248 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/769.out |    2 
> >  2 files changed, 250 insertions(+)
> >  create mode 100755 tests/xfs/769
> >  create mode 100644 tests/xfs/769.out
> > 
> > 
> > diff --git a/tests/xfs/769 b/tests/xfs/769
> > new file mode 100755
> > index 0000000000..7613048f52
> > --- /dev/null
> > +++ b/tests/xfs/769
> > @@ -0,0 +1,248 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 769
> > +#
> > +# Test upgrading filesystems with new features.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto mkfs repair
> > +
> > +# Import common functions.
> > +. ./common/filter
> > +. ./common/populate
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +
> > +test -w /dev/ttyprintk || _notrun "test requires writable /dev/ttyprintk"
> 
> Hi Darrick,
> 
> I'm not sure why /dev/ttyprintk is necessary. I think sometimes we might not
> have this driver, but has /dev/kmsg. Can /dev/kmsg be a replacement of
> ttyprintk ?

The kernel logging here is for debugging purposes -- if the upgrade
corrupts the fs and the kernel splats, whoever triages the failure can
have at least some clue as to where things went wrong.

I think the _notrun line should go away though.

--D

> Thanks,
> Zorro
> 
> 
> > +_require_check_dmesg
> > +_require_scratch_nocheck
> > +_require_scratch_xfs_crc
> > +
> > +# Does repair know how to add a particular feature to a filesystem?
> > +check_repair_upgrade()
> > +{
> > +	$XFS_REPAIR_PROG -c "$1=narf" 2>&1 | \
> > +		grep -q 'unknown option' && return 1
> > +	return 0
> > +}
> > +
> > +# Are we configured for realtime?
> > +rt_configured()
> > +{
> > +	test "$USE_EXTERNAL" = "yes" && test -n "$SCRATCH_RTDEV"
> > +}
> > +
> > +# Compute the MKFS_OPTIONS string for a particular feature upgrade test
> > +compute_mkfs_options()
> > +{
> > +	local m_opts=""
> > +	local caller_options="$MKFS_OPTIONS"
> > +
> > +	for feat in "${FEATURES[@]}"; do
> > +		local feat_state="${FEATURE_STATE["${feat}"]}"
> > +
> > +		if echo "$caller_options" | grep -E -w -q "${feat}=[0-9]*"; then
> > +			# Change the caller's options
> > +			caller_options="$(echo "$caller_options" | \
> > +				sed -e "s/\([^[:alnum:]]\)${feat}=[0-9]*/\1${feat}=${feat_state}/g")"
> > +		else
> > +			# Add it to our list of new mkfs flags
> > +			m_opts="${feat}=${feat_state},${m_opts}"
> > +		fi
> > +	done
> > +
> > +	test -n "$m_opts" && m_opts=" -m $m_opts"
> > +
> > +	echo "$caller_options$m_opts"
> > +}
> > +
> > +# Log the start of an upgrade.
> > +function upgrade_start_message()
> > +{
> > +	local feat="$1"
> > +
> > +	echo "Add $feat to filesystem"
> > +}
> > +
> > +# Find dmesg log messages since we started a particular upgrade test
> > +function dmesg_since_feature_upgrade_start()
> > +{
> > +	local feat_logmsg="$(upgrade_start_message "$1")"
> > +
> > +	# search the dmesg log of last run of $seqnum for possible failures
> > +	# use sed \cregexpc address type, since $seqnum contains "/"
> > +	dmesg | \
> > +		tac | \
> > +		sed -ne "0,\#run fstests $seqnum at $date_time#p" | \
> > +		sed -ne "0,\#${feat_logmsg}#p" | \
> > +		tac
> > +}
> > +
> > +# Did the mount fail because this feature is not supported?
> > +function feature_unsupported()
> > +{
> > +	local feat="$1"
> > +
> > +	dmesg_since_feature_upgrade_start "$feat" | \
> > +		grep -q 'has unknown.*features'
> > +}
> > +
> > +# Exercise the scratch fs
> > +function scratch_fsstress()
> > +{
> > +	echo moo > $SCRATCH_MNT/sample.txt
> > +	$FSSTRESS_PROG -n $((TIME_FACTOR * 1000)) -p $((LOAD_FACTOR * 4)) \
> > +		-d $SCRATCH_MNT/data >> $seqres.full
> > +}
> > +
> > +# Exercise the filesystem a little bit and emit a manifest.
> > +function pre_exercise()
> > +{
> > +	local feat="$1"
> > +
> > +	_try_scratch_mount &> $tmp.mount
> > +	res=$?
> > +	# If the kernel doesn't support the filesystem even after a
> > +	# fresh format, skip the rest of the upgrade test quietly.
> > +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> > +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> > +		return 1
> > +	fi
> > +	if [ $res -ne 0 ]; then
> > +		cat $tmp.mount
> > +		echo "mount failed with $res before upgrading to $feat" | \
> > +			tee -a $seqres.full
> > +		return 1
> > +	fi
> > +
> > +	scratch_fsstress
> > +	find $SCRATCH_MNT -type f -print0 | xargs -r -0 md5sum > $tmp.manifest
> > +	_scratch_unmount
> > +	return 0
> > +}
> > +
> > +# Check the manifest and exercise the filesystem more
> > +function post_exercise()
> > +{
> > +	local feat="$1"
> > +
> > +	_try_scratch_mount &> $tmp.mount
> > +	res=$?
> > +	# If the kernel doesn't support the filesystem even after a
> > +	# fresh format, skip the rest of the upgrade test quietly.
> > +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> > +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> > +		return 1
> > +	fi
> > +	if [ $res -ne 0 ]; then
> > +		cat $tmp.mount
> > +		echo "mount failed with $res after upgrading to $feat" | \
> > +			tee -a $seqres.full
> > +		return 1
> > +	fi
> > +
> > +	md5sum --quiet -c $tmp.manifest || \
> > +		echo "fs contents ^^^ changed after adding $feat"
> > +
> > +	iam="check" _check_scratch_fs || \
> > +		echo "scratch fs check failed after adding $feat"
> > +
> > +	# Try to mount the fs in case the check unmounted it
> > +	_try_scratch_mount &>> $seqres.full
> > +
> > +	scratch_fsstress
> > +
> > +	iam="check" _check_scratch_fs || \
> > +		echo "scratch fs check failed after exercising $feat"
> > +
> > +	# Try to unmount the fs in case the check didn't
> > +	_scratch_unmount &>> $seqres.full
> > +	return 0
> > +}
> > +
> > +# Create a list of fs features in the order that support for them was added
> > +# to the kernel driver.  For each feature upgrade test, we enable all the
> > +# features that came before it and none of the ones after, which means we're
> > +# testing incremental migrations.  We start each run with a clean fs so that
> > +# errors and unsatisfied requirements (log size, root ino position, etc) in one
> > +# upgrade don't spread failure to the rest of the tests.
> > +FEATURES=()
> > +if rt_configured; then
> > +	check_repair_upgrade finobt && FEATURES+=("finobt")
> > +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> > +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> > +else
> > +	check_repair_upgrade finobt && FEATURES+=("finobt")
> > +	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
> > +	check_repair_upgrade reflink && FEATURES+=("reflink")
> > +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> > +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> > +fi
> > +
> > +test "${#FEATURES[@]}" -eq 0 && \
> > +	_notrun "xfs_repair does not know how to add V5 features"
> > +
> > +declare -A FEATURE_STATE
> > +for f in "${FEATURES[@]}"; do
> > +	FEATURE_STATE["$f"]=0
> > +done
> > +
> > +for feat in "${FEATURES[@]}"; do
> > +	echo "-----------------------" >> $seqres.full
> > +
> > +	upgrade_start_message "$feat" | tee -a $seqres.full /dev/ttyprintk > /dev/null
> > +
> > +	opts="$(compute_mkfs_options)"
> > +	echo "mkfs.xfs $opts" >> $seqres.full
> > +
> > +	# Format filesystem
> > +	MKFS_OPTIONS="$opts" _scratch_mkfs &>> $seqres.full
> > +	res=$?
> > +	outcome="mkfs returns $res for $feat upgrade test"
> > +	echo "$outcome" >> $seqres.full
> > +	if [ $res -ne 0 ]; then
> > +		echo "$outcome"
> > +		continue
> > +	fi
> > +
> > +	# Create some files to make things interesting.
> > +	pre_exercise "$feat" || break
> > +
> > +	# Upgrade the fs
> > +	_scratch_xfs_repair -c "${feat}=1" &> $tmp.upgrade
> > +	res=$?
> > +	cat $tmp.upgrade >> $seqres.full
> > +	grep -q "^Adding" $tmp.upgrade || \
> > +		echo "xfs_repair ignored command to add $feat"
> > +
> > +	outcome="xfs_repair returns $res while adding $feat"
> > +	echo "$outcome" >> $seqres.full
> > +	if [ $res -ne 0 ]; then
> > +		# Couldn't upgrade filesystem, move on to the next feature.
> > +		FEATURE_STATE["$feat"]=1
> > +		continue
> > +	fi
> > +
> > +	# Make sure repair runs cleanly afterwards
> > +	_scratch_xfs_repair -n &>> $seqres.full
> > +	res=$?
> > +	outcome="xfs_repair -n returns $res after adding $feat"
> > +	echo "$outcome" >> $seqres.full
> > +	if [ $res -ne 0 ]; then
> > +		echo "$outcome"
> > +	fi
> > +
> > +	# Make sure we can still exercise the filesystem.
> > +	post_exercise "$feat" || break
> > +
> > +	# Update feature state for next run
> > +	FEATURE_STATE["$feat"]=1
> > +done
> > +
> > +# success, all done
> > +echo Silence is golden.
> > +status=0
> > +exit
> > diff --git a/tests/xfs/769.out b/tests/xfs/769.out
> > new file mode 100644
> > index 0000000000..332432db97
> > --- /dev/null
> > +++ b/tests/xfs/769.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 769
> > +Silence is golden.
> > 
> 

Re: [PATCH 1/1] xfs: test upgrading old features

[Zorro Lang]

On Mon, Mar 06, 2023 at 08:41:55AM -0800, Darrick J. Wong wrote:
> On Mon, Mar 06, 2023 at 11:56:11PM +0800, Zorro Lang wrote:
> > On Fri, Dec 30, 2022 at 02:20:29PM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > Test the ability to add older v5 features.
> > > 
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  tests/xfs/769     |  248 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> > >  tests/xfs/769.out |    2 
> > >  2 files changed, 250 insertions(+)
> > >  create mode 100755 tests/xfs/769
> > >  create mode 100644 tests/xfs/769.out
> > > 
> > > 
> > > diff --git a/tests/xfs/769 b/tests/xfs/769
> > > new file mode 100755
> > > index 0000000000..7613048f52
> > > --- /dev/null
> > > +++ b/tests/xfs/769
> > > @@ -0,0 +1,248 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0-or-later
> > > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > > +#
> > > +# FS QA Test No. 769
> > > +#
> > > +# Test upgrading filesystems with new features.
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest auto mkfs repair
> > > +
> > > +# Import common functions.
> > > +. ./common/filter
> > > +. ./common/populate
> > > +
> > > +# real QA test starts here
> > > +_supported_fs xfs
> > > +
> > > +test -w /dev/ttyprintk || _notrun "test requires writable /dev/ttyprintk"
> > 
> > Hi Darrick,
> > 
> > I'm not sure why /dev/ttyprintk is necessary. I think sometimes we might not
> > have this driver, but has /dev/kmsg. Can /dev/kmsg be a replacement of
> > ttyprintk ?
> 
> The kernel logging here is for debugging purposes -- if the upgrade
> corrupts the fs and the kernel splats, whoever triages the failure can
> have at least some clue as to where things went wrong.
> 
> I think the _notrun line should go away though.

If only for debugging, better to let this case keep running without ttyprintk.
Or it'll _notrun on many systems without ttyprintk driver.

There's a log_dmesg() helper in common/rc, the /dev/kmsg might be more often.
As you've called *_require_check_dmesg*, it helps to check kmsg file. So
how about use it? Or making a helper to write messages to ttyprintk, then
fallback to kmsg if ttyprintk is missing ?

Thanks,
Zorro

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > 
> > > +_require_check_dmesg
> > > +_require_scratch_nocheck
> > > +_require_scratch_xfs_crc
> > > +
> > > +# Does repair know how to add a particular feature to a filesystem?
> > > +check_repair_upgrade()
> > > +{
> > > +	$XFS_REPAIR_PROG -c "$1=narf" 2>&1 | \
> > > +		grep -q 'unknown option' && return 1
> > > +	return 0
> > > +}
> > > +
> > > +# Are we configured for realtime?
> > > +rt_configured()
> > > +{
> > > +	test "$USE_EXTERNAL" = "yes" && test -n "$SCRATCH_RTDEV"
> > > +}
> > > +
> > > +# Compute the MKFS_OPTIONS string for a particular feature upgrade test
> > > +compute_mkfs_options()
> > > +{
> > > +	local m_opts=""
> > > +	local caller_options="$MKFS_OPTIONS"
> > > +
> > > +	for feat in "${FEATURES[@]}"; do
> > > +		local feat_state="${FEATURE_STATE["${feat}"]}"
> > > +
> > > +		if echo "$caller_options" | grep -E -w -q "${feat}=[0-9]*"; then
> > > +			# Change the caller's options
> > > +			caller_options="$(echo "$caller_options" | \
> > > +				sed -e "s/\([^[:alnum:]]\)${feat}=[0-9]*/\1${feat}=${feat_state}/g")"
> > > +		else
> > > +			# Add it to our list of new mkfs flags
> > > +			m_opts="${feat}=${feat_state},${m_opts}"
> > > +		fi
> > > +	done
> > > +
> > > +	test -n "$m_opts" && m_opts=" -m $m_opts"
> > > +
> > > +	echo "$caller_options$m_opts"
> > > +}
> > > +
> > > +# Log the start of an upgrade.
> > > +function upgrade_start_message()
> > > +{
> > > +	local feat="$1"
> > > +
> > > +	echo "Add $feat to filesystem"
> > > +}
> > > +
> > > +# Find dmesg log messages since we started a particular upgrade test
> > > +function dmesg_since_feature_upgrade_start()
> > > +{
> > > +	local feat_logmsg="$(upgrade_start_message "$1")"
> > > +
> > > +	# search the dmesg log of last run of $seqnum for possible failures
> > > +	# use sed \cregexpc address type, since $seqnum contains "/"
> > > +	dmesg | \
> > > +		tac | \
> > > +		sed -ne "0,\#run fstests $seqnum at $date_time#p" | \
> > > +		sed -ne "0,\#${feat_logmsg}#p" | \
> > > +		tac
> > > +}
> > > +
> > > +# Did the mount fail because this feature is not supported?
> > > +function feature_unsupported()
> > > +{
> > > +	local feat="$1"
> > > +
> > > +	dmesg_since_feature_upgrade_start "$feat" | \
> > > +		grep -q 'has unknown.*features'
> > > +}
> > > +
> > > +# Exercise the scratch fs
> > > +function scratch_fsstress()
> > > +{
> > > +	echo moo > $SCRATCH_MNT/sample.txt
> > > +	$FSSTRESS_PROG -n $((TIME_FACTOR * 1000)) -p $((LOAD_FACTOR * 4)) \
> > > +		-d $SCRATCH_MNT/data >> $seqres.full
> > > +}
> > > +
> > > +# Exercise the filesystem a little bit and emit a manifest.
> > > +function pre_exercise()
> > > +{
> > > +	local feat="$1"
> > > +
> > > +	_try_scratch_mount &> $tmp.mount
> > > +	res=$?
> > > +	# If the kernel doesn't support the filesystem even after a
> > > +	# fresh format, skip the rest of the upgrade test quietly.
> > > +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> > > +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> > > +		return 1
> > > +	fi
> > > +	if [ $res -ne 0 ]; then
> > > +		cat $tmp.mount
> > > +		echo "mount failed with $res before upgrading to $feat" | \
> > > +			tee -a $seqres.full
> > > +		return 1
> > > +	fi
> > > +
> > > +	scratch_fsstress
> > > +	find $SCRATCH_MNT -type f -print0 | xargs -r -0 md5sum > $tmp.manifest
> > > +	_scratch_unmount
> > > +	return 0
> > > +}
> > > +
> > > +# Check the manifest and exercise the filesystem more
> > > +function post_exercise()
> > > +{
> > > +	local feat="$1"
> > > +
> > > +	_try_scratch_mount &> $tmp.mount
> > > +	res=$?
> > > +	# If the kernel doesn't support the filesystem even after a
> > > +	# fresh format, skip the rest of the upgrade test quietly.
> > > +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> > > +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> > > +		return 1
> > > +	fi
> > > +	if [ $res -ne 0 ]; then
> > > +		cat $tmp.mount
> > > +		echo "mount failed with $res after upgrading to $feat" | \
> > > +			tee -a $seqres.full
> > > +		return 1
> > > +	fi
> > > +
> > > +	md5sum --quiet -c $tmp.manifest || \
> > > +		echo "fs contents ^^^ changed after adding $feat"
> > > +
> > > +	iam="check" _check_scratch_fs || \
> > > +		echo "scratch fs check failed after adding $feat"
> > > +
> > > +	# Try to mount the fs in case the check unmounted it
> > > +	_try_scratch_mount &>> $seqres.full
> > > +
> > > +	scratch_fsstress
> > > +
> > > +	iam="check" _check_scratch_fs || \
> > > +		echo "scratch fs check failed after exercising $feat"
> > > +
> > > +	# Try to unmount the fs in case the check didn't
> > > +	_scratch_unmount &>> $seqres.full
> > > +	return 0
> > > +}
> > > +
> > > +# Create a list of fs features in the order that support for them was added
> > > +# to the kernel driver.  For each feature upgrade test, we enable all the
> > > +# features that came before it and none of the ones after, which means we're
> > > +# testing incremental migrations.  We start each run with a clean fs so that
> > > +# errors and unsatisfied requirements (log size, root ino position, etc) in one
> > > +# upgrade don't spread failure to the rest of the tests.
> > > +FEATURES=()
> > > +if rt_configured; then
> > > +	check_repair_upgrade finobt && FEATURES+=("finobt")
> > > +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> > > +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> > > +else
> > > +	check_repair_upgrade finobt && FEATURES+=("finobt")
> > > +	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
> > > +	check_repair_upgrade reflink && FEATURES+=("reflink")
> > > +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> > > +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> > > +fi
> > > +
> > > +test "${#FEATURES[@]}" -eq 0 && \
> > > +	_notrun "xfs_repair does not know how to add V5 features"
> > > +
> > > +declare -A FEATURE_STATE
> > > +for f in "${FEATURES[@]}"; do
> > > +	FEATURE_STATE["$f"]=0
> > > +done
> > > +
> > > +for feat in "${FEATURES[@]}"; do
> > > +	echo "-----------------------" >> $seqres.full
> > > +
> > > +	upgrade_start_message "$feat" | tee -a $seqres.full /dev/ttyprintk > /dev/null
> > > +
> > > +	opts="$(compute_mkfs_options)"
> > > +	echo "mkfs.xfs $opts" >> $seqres.full
> > > +
> > > +	# Format filesystem
> > > +	MKFS_OPTIONS="$opts" _scratch_mkfs &>> $seqres.full
> > > +	res=$?
> > > +	outcome="mkfs returns $res for $feat upgrade test"
> > > +	echo "$outcome" >> $seqres.full
> > > +	if [ $res -ne 0 ]; then
> > > +		echo "$outcome"
> > > +		continue
> > > +	fi
> > > +
> > > +	# Create some files to make things interesting.
> > > +	pre_exercise "$feat" || break
> > > +
> > > +	# Upgrade the fs
> > > +	_scratch_xfs_repair -c "${feat}=1" &> $tmp.upgrade
> > > +	res=$?
> > > +	cat $tmp.upgrade >> $seqres.full
> > > +	grep -q "^Adding" $tmp.upgrade || \
> > > +		echo "xfs_repair ignored command to add $feat"
> > > +
> > > +	outcome="xfs_repair returns $res while adding $feat"
> > > +	echo "$outcome" >> $seqres.full
> > > +	if [ $res -ne 0 ]; then
> > > +		# Couldn't upgrade filesystem, move on to the next feature.
> > > +		FEATURE_STATE["$feat"]=1
> > > +		continue
> > > +	fi
> > > +
> > > +	# Make sure repair runs cleanly afterwards
> > > +	_scratch_xfs_repair -n &>> $seqres.full
> > > +	res=$?
> > > +	outcome="xfs_repair -n returns $res after adding $feat"
> > > +	echo "$outcome" >> $seqres.full
> > > +	if [ $res -ne 0 ]; then
> > > +		echo "$outcome"
> > > +	fi
> > > +
> > > +	# Make sure we can still exercise the filesystem.
> > > +	post_exercise "$feat" || break
> > > +
> > > +	# Update feature state for next run
> > > +	FEATURE_STATE["$feat"]=1
> > > +done
> > > +
> > > +# success, all done
> > > +echo Silence is golden.
> > > +status=0
> > > +exit
> > > diff --git a/tests/xfs/769.out b/tests/xfs/769.out
> > > new file mode 100644
> > > index 0000000000..332432db97
> > > --- /dev/null
> > > +++ b/tests/xfs/769.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 769
> > > +Silence is golden.
> > > 
> > 
> 

Re: [PATCH 1/1] xfs: test upgrading old features

[Darrick J. Wong]

On Tue, Mar 07, 2023 at 12:54:02AM +0800, Zorro Lang wrote:
> On Mon, Mar 06, 2023 at 08:41:55AM -0800, Darrick J. Wong wrote:
> > On Mon, Mar 06, 2023 at 11:56:11PM +0800, Zorro Lang wrote:
> > > On Fri, Dec 30, 2022 at 02:20:29PM -0800, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > Test the ability to add older v5 features.
> > > > 
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > ---
> > > >  tests/xfs/769     |  248 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  tests/xfs/769.out |    2 
> > > >  2 files changed, 250 insertions(+)
> > > >  create mode 100755 tests/xfs/769
> > > >  create mode 100644 tests/xfs/769.out
> > > > 
> > > > 
> > > > diff --git a/tests/xfs/769 b/tests/xfs/769
> > > > new file mode 100755
> > > > index 0000000000..7613048f52
> > > > --- /dev/null
> > > > +++ b/tests/xfs/769
> > > > @@ -0,0 +1,248 @@
> > > > +#! /bin/bash
> > > > +# SPDX-License-Identifier: GPL-2.0-or-later
> > > > +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> > > > +#
> > > > +# FS QA Test No. 769
> > > > +#
> > > > +# Test upgrading filesystems with new features.
> > > > +#
> > > > +. ./common/preamble
> > > > +_begin_fstest auto mkfs repair
> > > > +
> > > > +# Import common functions.
> > > > +. ./common/filter
> > > > +. ./common/populate
> > > > +
> > > > +# real QA test starts here
> > > > +_supported_fs xfs
> > > > +
> > > > +test -w /dev/ttyprintk || _notrun "test requires writable /dev/ttyprintk"
> > > 
> > > Hi Darrick,
> > > 
> > > I'm not sure why /dev/ttyprintk is necessary. I think sometimes we might not
> > > have this driver, but has /dev/kmsg. Can /dev/kmsg be a replacement of
> > > ttyprintk ?
> > 
> > The kernel logging here is for debugging purposes -- if the upgrade
> > corrupts the fs and the kernel splats, whoever triages the failure can
> > have at least some clue as to where things went wrong.
> > 
> > I think the _notrun line should go away though.
> 
> If only for debugging, better to let this case keep running without ttyprintk.
> Or it'll _notrun on many systems without ttyprintk driver.
> 
> There's a log_dmesg() helper in common/rc, the /dev/kmsg might be more often.
> As you've called *_require_check_dmesg*, it helps to check kmsg file. So
> how about use it? Or making a helper to write messages to ttyprintk, then
> fallback to kmsg if ttyprintk is missing ?

Yeah, I can hoist the ttyprintk usage into a helper that emits to
ttyprintk or kmsg depending on what's available.

--D

> Thanks,
> Zorro
> 
> > 
> > --D
> > 
> > > Thanks,
> > > Zorro
> > > 
> > > 
> > > > +_require_check_dmesg
> > > > +_require_scratch_nocheck
> > > > +_require_scratch_xfs_crc
> > > > +
> > > > +# Does repair know how to add a particular feature to a filesystem?
> > > > +check_repair_upgrade()
> > > > +{
> > > > +	$XFS_REPAIR_PROG -c "$1=narf" 2>&1 | \
> > > > +		grep -q 'unknown option' && return 1
> > > > +	return 0
> > > > +}
> > > > +
> > > > +# Are we configured for realtime?
> > > > +rt_configured()
> > > > +{
> > > > +	test "$USE_EXTERNAL" = "yes" && test -n "$SCRATCH_RTDEV"
> > > > +}
> > > > +
> > > > +# Compute the MKFS_OPTIONS string for a particular feature upgrade test
> > > > +compute_mkfs_options()
> > > > +{
> > > > +	local m_opts=""
> > > > +	local caller_options="$MKFS_OPTIONS"
> > > > +
> > > > +	for feat in "${FEATURES[@]}"; do
> > > > +		local feat_state="${FEATURE_STATE["${feat}"]}"
> > > > +
> > > > +		if echo "$caller_options" | grep -E -w -q "${feat}=[0-9]*"; then
> > > > +			# Change the caller's options
> > > > +			caller_options="$(echo "$caller_options" | \
> > > > +				sed -e "s/\([^[:alnum:]]\)${feat}=[0-9]*/\1${feat}=${feat_state}/g")"
> > > > +		else
> > > > +			# Add it to our list of new mkfs flags
> > > > +			m_opts="${feat}=${feat_state},${m_opts}"
> > > > +		fi
> > > > +	done
> > > > +
> > > > +	test -n "$m_opts" && m_opts=" -m $m_opts"
> > > > +
> > > > +	echo "$caller_options$m_opts"
> > > > +}
> > > > +
> > > > +# Log the start of an upgrade.
> > > > +function upgrade_start_message()
> > > > +{
> > > > +	local feat="$1"
> > > > +
> > > > +	echo "Add $feat to filesystem"
> > > > +}
> > > > +
> > > > +# Find dmesg log messages since we started a particular upgrade test
> > > > +function dmesg_since_feature_upgrade_start()
> > > > +{
> > > > +	local feat_logmsg="$(upgrade_start_message "$1")"
> > > > +
> > > > +	# search the dmesg log of last run of $seqnum for possible failures
> > > > +	# use sed \cregexpc address type, since $seqnum contains "/"
> > > > +	dmesg | \
> > > > +		tac | \
> > > > +		sed -ne "0,\#run fstests $seqnum at $date_time#p" | \
> > > > +		sed -ne "0,\#${feat_logmsg}#p" | \
> > > > +		tac
> > > > +}
> > > > +
> > > > +# Did the mount fail because this feature is not supported?
> > > > +function feature_unsupported()
> > > > +{
> > > > +	local feat="$1"
> > > > +
> > > > +	dmesg_since_feature_upgrade_start "$feat" | \
> > > > +		grep -q 'has unknown.*features'
> > > > +}
> > > > +
> > > > +# Exercise the scratch fs
> > > > +function scratch_fsstress()
> > > > +{
> > > > +	echo moo > $SCRATCH_MNT/sample.txt
> > > > +	$FSSTRESS_PROG -n $((TIME_FACTOR * 1000)) -p $((LOAD_FACTOR * 4)) \
> > > > +		-d $SCRATCH_MNT/data >> $seqres.full
> > > > +}
> > > > +
> > > > +# Exercise the filesystem a little bit and emit a manifest.
> > > > +function pre_exercise()
> > > > +{
> > > > +	local feat="$1"
> > > > +
> > > > +	_try_scratch_mount &> $tmp.mount
> > > > +	res=$?
> > > > +	# If the kernel doesn't support the filesystem even after a
> > > > +	# fresh format, skip the rest of the upgrade test quietly.
> > > > +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> > > > +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> > > > +		return 1
> > > > +	fi
> > > > +	if [ $res -ne 0 ]; then
> > > > +		cat $tmp.mount
> > > > +		echo "mount failed with $res before upgrading to $feat" | \
> > > > +			tee -a $seqres.full
> > > > +		return 1
> > > > +	fi
> > > > +
> > > > +	scratch_fsstress
> > > > +	find $SCRATCH_MNT -type f -print0 | xargs -r -0 md5sum > $tmp.manifest
> > > > +	_scratch_unmount
> > > > +	return 0
> > > > +}
> > > > +
> > > > +# Check the manifest and exercise the filesystem more
> > > > +function post_exercise()
> > > > +{
> > > > +	local feat="$1"
> > > > +
> > > > +	_try_scratch_mount &> $tmp.mount
> > > > +	res=$?
> > > > +	# If the kernel doesn't support the filesystem even after a
> > > > +	# fresh format, skip the rest of the upgrade test quietly.
> > > > +	if [ $res -eq 32 ] && feature_unsupported "$feat"; then
> > > > +		echo "mount failed due to unsupported feature $feat" >> $seqres.full
> > > > +		return 1
> > > > +	fi
> > > > +	if [ $res -ne 0 ]; then
> > > > +		cat $tmp.mount
> > > > +		echo "mount failed with $res after upgrading to $feat" | \
> > > > +			tee -a $seqres.full
> > > > +		return 1
> > > > +	fi
> > > > +
> > > > +	md5sum --quiet -c $tmp.manifest || \
> > > > +		echo "fs contents ^^^ changed after adding $feat"
> > > > +
> > > > +	iam="check" _check_scratch_fs || \
> > > > +		echo "scratch fs check failed after adding $feat"
> > > > +
> > > > +	# Try to mount the fs in case the check unmounted it
> > > > +	_try_scratch_mount &>> $seqres.full
> > > > +
> > > > +	scratch_fsstress
> > > > +
> > > > +	iam="check" _check_scratch_fs || \
> > > > +		echo "scratch fs check failed after exercising $feat"
> > > > +
> > > > +	# Try to unmount the fs in case the check didn't
> > > > +	_scratch_unmount &>> $seqres.full
> > > > +	return 0
> > > > +}
> > > > +
> > > > +# Create a list of fs features in the order that support for them was added
> > > > +# to the kernel driver.  For each feature upgrade test, we enable all the
> > > > +# features that came before it and none of the ones after, which means we're
> > > > +# testing incremental migrations.  We start each run with a clean fs so that
> > > > +# errors and unsatisfied requirements (log size, root ino position, etc) in one
> > > > +# upgrade don't spread failure to the rest of the tests.
> > > > +FEATURES=()
> > > > +if rt_configured; then
> > > > +	check_repair_upgrade finobt && FEATURES+=("finobt")
> > > > +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> > > > +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> > > > +else
> > > > +	check_repair_upgrade finobt && FEATURES+=("finobt")
> > > > +	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
> > > > +	check_repair_upgrade reflink && FEATURES+=("reflink")
> > > > +	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
> > > > +	check_repair_upgrade bigtime && FEATURES+=("bigtime")
> > > > +fi
> > > > +
> > > > +test "${#FEATURES[@]}" -eq 0 && \
> > > > +	_notrun "xfs_repair does not know how to add V5 features"
> > > > +
> > > > +declare -A FEATURE_STATE
> > > > +for f in "${FEATURES[@]}"; do
> > > > +	FEATURE_STATE["$f"]=0
> > > > +done
> > > > +
> > > > +for feat in "${FEATURES[@]}"; do
> > > > +	echo "-----------------------" >> $seqres.full
> > > > +
> > > > +	upgrade_start_message "$feat" | tee -a $seqres.full /dev/ttyprintk > /dev/null
> > > > +
> > > > +	opts="$(compute_mkfs_options)"
> > > > +	echo "mkfs.xfs $opts" >> $seqres.full
> > > > +
> > > > +	# Format filesystem
> > > > +	MKFS_OPTIONS="$opts" _scratch_mkfs &>> $seqres.full
> > > > +	res=$?
> > > > +	outcome="mkfs returns $res for $feat upgrade test"
> > > > +	echo "$outcome" >> $seqres.full
> > > > +	if [ $res -ne 0 ]; then
> > > > +		echo "$outcome"
> > > > +		continue
> > > > +	fi
> > > > +
> > > > +	# Create some files to make things interesting.
> > > > +	pre_exercise "$feat" || break
> > > > +
> > > > +	# Upgrade the fs
> > > > +	_scratch_xfs_repair -c "${feat}=1" &> $tmp.upgrade
> > > > +	res=$?
> > > > +	cat $tmp.upgrade >> $seqres.full
> > > > +	grep -q "^Adding" $tmp.upgrade || \
> > > > +		echo "xfs_repair ignored command to add $feat"
> > > > +
> > > > +	outcome="xfs_repair returns $res while adding $feat"
> > > > +	echo "$outcome" >> $seqres.full
> > > > +	if [ $res -ne 0 ]; then
> > > > +		# Couldn't upgrade filesystem, move on to the next feature.
> > > > +		FEATURE_STATE["$feat"]=1
> > > > +		continue
> > > > +	fi
> > > > +
> > > > +	# Make sure repair runs cleanly afterwards
> > > > +	_scratch_xfs_repair -n &>> $seqres.full
> > > > +	res=$?
> > > > +	outcome="xfs_repair -n returns $res after adding $feat"
> > > > +	echo "$outcome" >> $seqres.full
> > > > +	if [ $res -ne 0 ]; then
> > > > +		echo "$outcome"
> > > > +	fi
> > > > +
> > > > +	# Make sure we can still exercise the filesystem.
> > > > +	post_exercise "$feat" || break
> > > > +
> > > > +	# Update feature state for next run
> > > > +	FEATURE_STATE["$feat"]=1
> > > > +done
> > > > +
> > > > +# success, all done
> > > > +echo Silence is golden.
> > > > +status=0
> > > > +exit
> > > > diff --git a/tests/xfs/769.out b/tests/xfs/769.out
> > > > new file mode 100644
> > > > index 0000000000..332432db97
> > > > --- /dev/null
> > > > +++ b/tests/xfs/769.out
> > > > @@ -0,0 +1,2 @@
> > > > +QA output created by 769
> > > > +Silence is golden.
> > > > 
> > > 
> > 
> 

[PATCHSET v1.0 0/9] fstests: test XFS metadata directories

[Darrick J. Wong]

Hi all,

Adjust fstests as needed to support the XFS metadata directory feature,
and add some new tests for online fsck and fuzz testing of the ondisk
metadata.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=metadir

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=metadir

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=metadir

xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=metadir
---
 common/filter      |    7 +++-
 common/repair      |    4 ++
 common/xfs         |   90 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 tests/xfs/007      |   16 +++++----
 tests/xfs/030      |    1 +
 tests/xfs/033      |    1 +
 tests/xfs/050      |    1 +
 tests/xfs/122.out  |    1 +
 tests/xfs/153      |    1 +
 tests/xfs/1546     |   37 +++++++++++++++++++++
 tests/xfs/1546.out |    4 ++
 tests/xfs/1547     |   37 +++++++++++++++++++++
 tests/xfs/1547.out |    4 ++
 tests/xfs/1548     |   37 +++++++++++++++++++++
 tests/xfs/1548.out |    4 ++
 tests/xfs/1549     |   38 ++++++++++++++++++++++
 tests/xfs/1549.out |    4 ++
 tests/xfs/1550     |   37 +++++++++++++++++++++
 tests/xfs/1550.out |    4 ++
 tests/xfs/1551     |   37 +++++++++++++++++++++
 tests/xfs/1551.out |    4 ++
 tests/xfs/1552     |   37 +++++++++++++++++++++
 tests/xfs/1552.out |    4 ++
 tests/xfs/1553     |   38 ++++++++++++++++++++++
 tests/xfs/1553.out |    4 ++
 tests/xfs/1562     |    9 +----
 tests/xfs/1563     |    9 +----
 tests/xfs/1564     |    9 +----
 tests/xfs/1565     |    9 +----
 tests/xfs/1566     |    9 +----
 tests/xfs/1567     |    9 +----
 tests/xfs/1568     |    9 +----
 tests/xfs/1569     |    9 +----
 tests/xfs/178      |    1 +
 tests/xfs/206      |    3 +-
 tests/xfs/299      |    1 +
 tests/xfs/330      |    6 +++
 tests/xfs/509      |   21 +++++++++++-
 tests/xfs/529      |    5 +--
 tests/xfs/530      |    6 +--
 tests/xfs/769      |    2 +
 41 files changed, 491 insertions(+), 78 deletions(-)
 create mode 100755 tests/xfs/1546
 create mode 100644 tests/xfs/1546.out
 create mode 100755 tests/xfs/1547
 create mode 100644 tests/xfs/1547.out
 create mode 100755 tests/xfs/1548
 create mode 100644 tests/xfs/1548.out
 create mode 100755 tests/xfs/1549
 create mode 100644 tests/xfs/1549.out
 create mode 100755 tests/xfs/1550
 create mode 100644 tests/xfs/1550.out
 create mode 100755 tests/xfs/1551
 create mode 100644 tests/xfs/1551.out
 create mode 100755 tests/xfs/1552
 create mode 100644 tests/xfs/1552.out
 create mode 100755 tests/xfs/1553
 create mode 100644 tests/xfs/1553.out

[PATCH 3/9] xfs/{030,033,178}: forcibly disable metadata directory trees

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The golden output for thests tests encode the xfs_repair output when we
fuzz various parts of the filesystem.  With metadata directory trees
enabled, however, the golden output changes dramatically to reflect
reconstruction of the metadata directory tree.

To avoid regressions, add a helper to force metadata directories off via
MKFS_OPTIONS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs    |   13 +++++++++++++
 tests/xfs/030 |    1 +
 tests/xfs/033 |    1 +
 tests/xfs/178 |    1 +
 4 files changed, 16 insertions(+)


diff --git a/common/xfs b/common/xfs
index dafbd1b874..0f69d3eb18 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1770,3 +1770,16 @@ _scratch_xfs_find_metafile()
 	echo "${selector}"
 	return 0
 }
+
+# Force metadata directories off.
+_scratch_xfs_force_no_metadir()
+{
+	if echo "$MKFS_OPTIONS" | grep -q 'metadir='; then
+		MKFS_OPTIONS="$(echo "$MKFS_OPTIONS" | sed -e 's/metadir=\([01]\)/metadir=0/g')"
+		return
+	fi
+
+	if grep -q 'metadir=' $MKFS_XFS_PROG; then
+		MKFS_OPTIONS="-m metadir=0 $MKFS_OPTIONS"
+	fi
+}
diff --git a/tests/xfs/030 b/tests/xfs/030
index 201a901579..a62ea4fab3 100755
--- a/tests/xfs/030
+++ b/tests/xfs/030
@@ -50,6 +50,7 @@ _supported_fs xfs
 
 _require_scratch
 _require_no_large_scratch_dev
+_scratch_xfs_force_no_metadir
 
 DSIZE="-dsize=100m,agcount=6"
 
diff --git a/tests/xfs/033 b/tests/xfs/033
index ef5dc4fa36..e886c15082 100755
--- a/tests/xfs/033
+++ b/tests/xfs/033
@@ -53,6 +53,7 @@ _supported_fs xfs
 
 _require_scratch
 _require_no_large_scratch_dev
+_scratch_xfs_force_no_metadir
 
 # devzero blows away 512byte blocks, so make 512byte inodes (at least)
 _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs >/dev/null
diff --git a/tests/xfs/178 b/tests/xfs/178
index a65197cde3..72b4d347fd 100755
--- a/tests/xfs/178
+++ b/tests/xfs/178
@@ -45,6 +45,7 @@ _supported_fs xfs
 #             fix filesystem, new mkfs.xfs will be fine.
 
 _require_scratch
+_scratch_xfs_force_no_metadir
 _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
 test "${PIPESTATUS[0]}" -eq 0 || _fail "mkfs failed!"
 

[PATCH 1/9] xfs/122: fix metadirino

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix xfs/122 to work properly with metadirino.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122.out |    1 +
 1 file changed, 1 insertion(+)


diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 21549db7fd..eee6c1ee6d 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -35,6 +35,7 @@ offsetof(xfs_sb_t, sb_logsunit) = 196
 offsetof(xfs_sb_t, sb_lsn) = 240
 offsetof(xfs_sb_t, sb_magicnum) = 0
 offsetof(xfs_sb_t, sb_meta_uuid) = 248
+offsetof(xfs_sb_t, sb_metadirino) = 264
 offsetof(xfs_sb_t, sb_pquotino) = 232
 offsetof(xfs_sb_t, sb_qflags) = 176
 offsetof(xfs_sb_t, sb_rblocks) = 16

[PATCH 2/9] various: fix finding metadata inode numbers when metadir is enabled

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

There are a number of tests that use xfs_db to examine the contents of
metadata inodes to check correct functioning.  The logic is scattered
everywhere and won't work with metadata directory trees, so make a
shared helper to find these inodes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs     |   32 ++++++++++++++++++++++++++++++--
 tests/xfs/007  |   16 +++++++++-------
 tests/xfs/1562 |    9 ++-------
 tests/xfs/1563 |    9 ++-------
 tests/xfs/1564 |    9 ++-------
 tests/xfs/1565 |    9 ++-------
 tests/xfs/1566 |    9 ++-------
 tests/xfs/1567 |    9 ++-------
 tests/xfs/1568 |    9 ++-------
 tests/xfs/1569 |    9 ++-------
 tests/xfs/529  |    5 ++---
 tests/xfs/530  |    6 ++----
 12 files changed, 59 insertions(+), 72 deletions(-)


diff --git a/common/xfs b/common/xfs
index 8b365ad18b..dafbd1b874 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1396,7 +1396,7 @@ _scratch_get_bmx_prefix() {
 
 _scratch_get_iext_count()
 {
-	local ino=$1
+	local selector=$1
 	local whichfork=$2
 	local field=""
 
@@ -1411,7 +1411,7 @@ _scratch_get_iext_count()
 			return 1
 	esac
 
-	_scratch_xfs_get_metadata_field $field "inode $ino"
+	_scratch_xfs_get_metadata_field $field "$selector"
 }
 
 #
@@ -1742,3 +1742,31 @@ _require_xfs_scratch_atomicswap()
 		_notrun "atomicswap dependencies not supported by scratch filesystem type: $FSTYP"
 	_scratch_unmount
 }
+
+# Find a metadata file within an xfs filesystem.  The sole argument is the
+# name of the field within the superblock.
+_scratch_xfs_find_metafile()
+{
+	local metafile="$1"
+	local selector=
+
+	if ! _check_scratch_xfs_features METADIR > /dev/null; then
+		sb_field="$(_scratch_xfs_get_sb_field "$metafile")"
+		if echo "$sb_field" | grep -q -w 'not found'; then
+			return 1
+		fi
+		selector="inode $sb_field"
+	else
+		case "${metafile}" in
+		"rootino")	selector="path /";;
+		"uquotino")	selector="path -m /quota/user";;
+		"gquotino")	selector="path -m /quota/group";;
+		"pquotino")	selector="path -m /quota/project";;
+		"rbmino")	selector="path -m /realtime/bitmap";;
+		"rsumino")	selector="path -m /realtime/summary";;
+		esac
+	fi
+
+	echo "${selector}"
+	return 0
+}
diff --git a/tests/xfs/007 b/tests/xfs/007
index 4f864100fd..6d6d828b13 100755
--- a/tests/xfs/007
+++ b/tests/xfs/007
@@ -22,6 +22,11 @@ _require_xfs_quota
 _scratch_mkfs_xfs | _filter_mkfs > /dev/null 2> $tmp.mkfs
 . $tmp.mkfs
 
+get_qfile_nblocks() {
+	local selector="$(_scratch_xfs_find_metafile "$1")"
+	_scratch_xfs_db -c "$selector" -c "p core.nblocks"
+}
+
 do_test()
 {
 	qino_1=$1
@@ -31,12 +36,9 @@ do_test()
 	echo "*** umount"
 	_scratch_unmount
 
-	QINO_1=`_scratch_xfs_get_sb_field $qino_1`
-	QINO_2=`_scratch_xfs_get_sb_field $qino_2`
-
 	echo "*** Usage before quotarm ***"
-	_scratch_xfs_db -c "inode $QINO_1" -c "p core.nblocks"
-	_scratch_xfs_db -c "inode $QINO_2" -c "p core.nblocks"
+	get_qfile_nblocks $qino_1
+	get_qfile_nblocks $qino_2
 
 	_qmount
 	echo "*** turn off $off_opts quotas"
@@ -66,8 +68,8 @@ do_test()
 	_scratch_unmount
 
 	echo "*** Usage after quotarm ***"
-	_scratch_xfs_db -c "inode $QINO_1" -c "p core.nblocks"
-	_scratch_xfs_db -c "inode $QINO_2" -c "p core.nblocks"
+	get_qfile_nblocks $qino_1
+	get_qfile_nblocks $qino_2
 }
 
 # Test user and group first
diff --git a/tests/xfs/1562 b/tests/xfs/1562
index 015209eeb2..1e5b6881ee 100755
--- a/tests/xfs/1562
+++ b/tests/xfs/1562
@@ -27,13 +27,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'online' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/1563 b/tests/xfs/1563
index 2be0870a3d..a9da78106d 100755
--- a/tests/xfs/1563
+++ b/tests/xfs/1563
@@ -27,13 +27,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'online' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/1564 b/tests/xfs/1564
index c0d10ff0e9..4482861d50 100755
--- a/tests/xfs/1564
+++ b/tests/xfs/1564
@@ -27,13 +27,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'offline' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/1565 b/tests/xfs/1565
index 6b4186fb3c..c43ccd848e 100755
--- a/tests/xfs/1565
+++ b/tests/xfs/1565
@@ -27,13 +27,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'offline' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/1566 b/tests/xfs/1566
index 8d0f61ae10..aad4fafb15 100755
--- a/tests/xfs/1566
+++ b/tests/xfs/1566
@@ -28,13 +28,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'both' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/1567 b/tests/xfs/1567
index 7dc2012b67..ff782fc239 100755
--- a/tests/xfs/1567
+++ b/tests/xfs/1567
@@ -28,13 +28,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'both' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/1568 b/tests/xfs/1568
index c80640ef97..e2a28df58a 100755
--- a/tests/xfs/1568
+++ b/tests/xfs/1568
@@ -27,13 +27,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'none' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/1569 b/tests/xfs/1569
index e303f08ff5..dcb07440e8 100755
--- a/tests/xfs/1569
+++ b/tests/xfs/1569
@@ -27,13 +27,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'none' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/529 b/tests/xfs/529
index 83d24da0ac..e10af6753b 100755
--- a/tests/xfs/529
+++ b/tests/xfs/529
@@ -159,9 +159,8 @@ done
 _scratch_unmount >> $seqres.full
 
 echo "Verify uquota inode's extent count"
-uquotino=$(_scratch_xfs_get_metadata_field 'uquotino' 'sb 0')
-
-nextents=$(_scratch_get_iext_count $uquotino data || \
+selector="$(_scratch_xfs_find_metafile uquotino)"
+nextents=$(_scratch_get_iext_count "$selector" data || \
 		   _fail "Unable to obtain inode fork's extent count")
 if (( $nextents > 10 )); then
 	echo "Extent count overflow check failed: nextents = $nextents"
diff --git a/tests/xfs/530 b/tests/xfs/530
index 56f5e7ebdb..cb8c2e3978 100755
--- a/tests/xfs/530
+++ b/tests/xfs/530
@@ -104,10 +104,8 @@ _scratch_unmount >> $seqres.full
 
 echo "Verify rbmino's and rsumino's extent count"
 for rtino in rbmino rsumino; do
-	ino=$(_scratch_xfs_get_metadata_field $rtino "sb 0")
-	echo "$rtino = $ino" >> $seqres.full
-
-	nextents=$(_scratch_get_iext_count $ino data || \
+	selector="$(_scratch_xfs_find_metafile "$rtino")"
+	nextents=$(_scratch_get_iext_count "$selector" data || \
 			_fail "Unable to obtain inode fork's extent count")
 	if (( $nextents > 10 )); then
 		echo "Extent count overflow check failed: nextents = $nextents"

Re: [PATCH 2/9] various: fix finding metadata inode numbers when metadir is enabled

[Zorro Lang]

On Fri, Dec 30, 2022 at 02:20:32PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> There are a number of tests that use xfs_db to examine the contents of
> metadata inodes to check correct functioning.  The logic is scattered
> everywhere and won't work with metadata directory trees, so make a
> shared helper to find these inodes.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/xfs     |   32 ++++++++++++++++++++++++++++++--
>  tests/xfs/007  |   16 +++++++++-------
>  tests/xfs/1562 |    9 ++-------
>  tests/xfs/1563 |    9 ++-------
>  tests/xfs/1564 |    9 ++-------
>  tests/xfs/1565 |    9 ++-------
>  tests/xfs/1566 |    9 ++-------
>  tests/xfs/1567 |    9 ++-------
>  tests/xfs/1568 |    9 ++-------
>  tests/xfs/1569 |    9 ++-------

These case names are temporary names, I've renamed them when I merged them,
so this patch need to rebase. Sorry for this trouble :)

Thanks,
Zorro

>  tests/xfs/529  |    5 ++---
>  tests/xfs/530  |    6 ++----
>  12 files changed, 59 insertions(+), 72 deletions(-)
> 
> 
> diff --git a/common/xfs b/common/xfs
> index 8b365ad18b..dafbd1b874 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -1396,7 +1396,7 @@ _scratch_get_bmx_prefix() {
>  
>  _scratch_get_iext_count()
>  {
> -	local ino=$1
> +	local selector=$1
>  	local whichfork=$2
>  	local field=""
>  
> @@ -1411,7 +1411,7 @@ _scratch_get_iext_count()
>  			return 1
>  	esac
>  
> -	_scratch_xfs_get_metadata_field $field "inode $ino"
> +	_scratch_xfs_get_metadata_field $field "$selector"
>  }
>  
>  #
> @@ -1742,3 +1742,31 @@ _require_xfs_scratch_atomicswap()
>  		_notrun "atomicswap dependencies not supported by scratch filesystem type: $FSTYP"
>  	_scratch_unmount
>  }
> +
> +# Find a metadata file within an xfs filesystem.  The sole argument is the
> +# name of the field within the superblock.
> +_scratch_xfs_find_metafile()
> +{
> +	local metafile="$1"
> +	local selector=
> +
> +	if ! _check_scratch_xfs_features METADIR > /dev/null; then
> +		sb_field="$(_scratch_xfs_get_sb_field "$metafile")"
> +		if echo "$sb_field" | grep -q -w 'not found'; then
> +			return 1
> +		fi
> +		selector="inode $sb_field"
> +	else
> +		case "${metafile}" in
> +		"rootino")	selector="path /";;
> +		"uquotino")	selector="path -m /quota/user";;
> +		"gquotino")	selector="path -m /quota/group";;
> +		"pquotino")	selector="path -m /quota/project";;
> +		"rbmino")	selector="path -m /realtime/bitmap";;
> +		"rsumino")	selector="path -m /realtime/summary";;
> +		esac
> +	fi
> +
> +	echo "${selector}"
> +	return 0
> +}
> diff --git a/tests/xfs/007 b/tests/xfs/007
> index 4f864100fd..6d6d828b13 100755
> --- a/tests/xfs/007
> +++ b/tests/xfs/007
> @@ -22,6 +22,11 @@ _require_xfs_quota
>  _scratch_mkfs_xfs | _filter_mkfs > /dev/null 2> $tmp.mkfs
>  . $tmp.mkfs
>  
> +get_qfile_nblocks() {
> +	local selector="$(_scratch_xfs_find_metafile "$1")"
> +	_scratch_xfs_db -c "$selector" -c "p core.nblocks"
> +}
> +
>  do_test()
>  {
>  	qino_1=$1
> @@ -31,12 +36,9 @@ do_test()
>  	echo "*** umount"
>  	_scratch_unmount
>  
> -	QINO_1=`_scratch_xfs_get_sb_field $qino_1`
> -	QINO_2=`_scratch_xfs_get_sb_field $qino_2`
> -
>  	echo "*** Usage before quotarm ***"
> -	_scratch_xfs_db -c "inode $QINO_1" -c "p core.nblocks"
> -	_scratch_xfs_db -c "inode $QINO_2" -c "p core.nblocks"
> +	get_qfile_nblocks $qino_1
> +	get_qfile_nblocks $qino_2
>  
>  	_qmount
>  	echo "*** turn off $off_opts quotas"
> @@ -66,8 +68,8 @@ do_test()
>  	_scratch_unmount
>  
>  	echo "*** Usage after quotarm ***"
> -	_scratch_xfs_db -c "inode $QINO_1" -c "p core.nblocks"
> -	_scratch_xfs_db -c "inode $QINO_2" -c "p core.nblocks"
> +	get_qfile_nblocks $qino_1
> +	get_qfile_nblocks $qino_2
>  }
>  
>  # Test user and group first
> diff --git a/tests/xfs/1562 b/tests/xfs/1562
> index 015209eeb2..1e5b6881ee 100755
> --- a/tests/xfs/1562
> +++ b/tests/xfs/1562
> @@ -27,13 +27,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtbitmap"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.bitmap')
> -else
> -	path=('sb' 'addr rbmino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'online' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rbmino)"
> +_scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtbitmap"
>  
>  # success, all done
> diff --git a/tests/xfs/1563 b/tests/xfs/1563
> index 2be0870a3d..a9da78106d 100755
> --- a/tests/xfs/1563
> +++ b/tests/xfs/1563
> @@ -27,13 +27,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtsummary"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.summary')
> -else
> -	path=('sb' 'addr rsumino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'online' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rsumino)"
> +_scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtsummary"
>  
>  # success, all done
> diff --git a/tests/xfs/1564 b/tests/xfs/1564
> index c0d10ff0e9..4482861d50 100755
> --- a/tests/xfs/1564
> +++ b/tests/xfs/1564
> @@ -27,13 +27,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtbitmap"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.bitmap')
> -else
> -	path=('sb' 'addr rbmino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'offline' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rbmino)"
> +_scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtbitmap"
>  
>  # success, all done
> diff --git a/tests/xfs/1565 b/tests/xfs/1565
> index 6b4186fb3c..c43ccd848e 100755
> --- a/tests/xfs/1565
> +++ b/tests/xfs/1565
> @@ -27,13 +27,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtsummary"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.summary')
> -else
> -	path=('sb' 'addr rsumino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'offline' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rsumino)"
> +_scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtsummary"
>  
>  # success, all done
> diff --git a/tests/xfs/1566 b/tests/xfs/1566
> index 8d0f61ae10..aad4fafb15 100755
> --- a/tests/xfs/1566
> +++ b/tests/xfs/1566
> @@ -28,13 +28,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtbitmap"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.bitmap')
> -else
> -	path=('sb' 'addr rbmino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'both' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rbmino)"
> +_scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtbitmap"
>  
>  # success, all done
> diff --git a/tests/xfs/1567 b/tests/xfs/1567
> index 7dc2012b67..ff782fc239 100755
> --- a/tests/xfs/1567
> +++ b/tests/xfs/1567
> @@ -28,13 +28,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtsummary"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.summary')
> -else
> -	path=('sb' 'addr rsumino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'both' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rsumino)"
> +_scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtsummary"
>  
>  # success, all done
> diff --git a/tests/xfs/1568 b/tests/xfs/1568
> index c80640ef97..e2a28df58a 100755
> --- a/tests/xfs/1568
> +++ b/tests/xfs/1568
> @@ -27,13 +27,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtbitmap"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.bitmap')
> -else
> -	path=('sb' 'addr rbmino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'none' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rbmino)"
> +_scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtbitmap"
>  
>  # success, all done
> diff --git a/tests/xfs/1569 b/tests/xfs/1569
> index e303f08ff5..dcb07440e8 100755
> --- a/tests/xfs/1569
> +++ b/tests/xfs/1569
> @@ -27,13 +27,8 @@ echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
>  
>  echo "Fuzz rtsummary"
> -is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
> -if [ -n "$is_metadir" ]; then
> -	path=('path -m /realtime/0.summary')
> -else
> -	path=('sb' 'addr rsumino')
> -fi
> -_scratch_xfs_fuzz_metadata '' 'none' "${path[@]}" 'dblock 0' >> $seqres.full
> +path="$(_scratch_xfs_find_metafile rsumino)"
> +_scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
>  echo "Done fuzzing rtsummary"
>  
>  # success, all done
> diff --git a/tests/xfs/529 b/tests/xfs/529
> index 83d24da0ac..e10af6753b 100755
> --- a/tests/xfs/529
> +++ b/tests/xfs/529
> @@ -159,9 +159,8 @@ done
>  _scratch_unmount >> $seqres.full
>  
>  echo "Verify uquota inode's extent count"
> -uquotino=$(_scratch_xfs_get_metadata_field 'uquotino' 'sb 0')
> -
> -nextents=$(_scratch_get_iext_count $uquotino data || \
> +selector="$(_scratch_xfs_find_metafile uquotino)"
> +nextents=$(_scratch_get_iext_count "$selector" data || \
>  		   _fail "Unable to obtain inode fork's extent count")
>  if (( $nextents > 10 )); then
>  	echo "Extent count overflow check failed: nextents = $nextents"
> diff --git a/tests/xfs/530 b/tests/xfs/530
> index 56f5e7ebdb..cb8c2e3978 100755
> --- a/tests/xfs/530
> +++ b/tests/xfs/530
> @@ -104,10 +104,8 @@ _scratch_unmount >> $seqres.full
>  
>  echo "Verify rbmino's and rsumino's extent count"
>  for rtino in rbmino rsumino; do
> -	ino=$(_scratch_xfs_get_metadata_field $rtino "sb 0")
> -	echo "$rtino = $ino" >> $seqres.full
> -
> -	nextents=$(_scratch_get_iext_count $ino data || \
> +	selector="$(_scratch_xfs_find_metafile "$rtino")"
> +	nextents=$(_scratch_get_iext_count "$selector" data || \
>  			_fail "Unable to obtain inode fork's extent count")
>  	if (( $nextents > 10 )); then
>  		echo "Extent count overflow check failed: nextents = $nextents"
> 

[PATCH 4/9] common/repair: patch up repair sb inode value complaints

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we've refactored xfs_repair to be more consistent in how it
reports unexpected superblock inode pointer values, we have to fix up
the fstests repair filters to emulate the old golden output.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/repair |    4 ++++
 1 file changed, 4 insertions(+)


diff --git a/common/repair b/common/repair
index 8945d0028c..c3afcfb3e6 100644
--- a/common/repair
+++ b/common/repair
@@ -28,6 +28,10 @@ _filter_repair()
 	perl -ne '
 # for sb
 /- agno = / && next;	# remove each AG line (variable number)
+s/realtime bitmap inode pointer/realtime bitmap ino pointer/;
+s/sb realtime bitmap inode value/sb realtime bitmap inode/;
+s/realtime summary inode pointer/realtime summary ino pointer/;
+s/sb realtime summary inode value/sb realtime summary inode/;
 s/(pointer to) (\d+)/\1 INO/;
 # Changed inode output in 5.5.0
 s/sb root inode value /sb root inode /;

[PATCH 6/9] xfs/{050,144,153,299,330}: update quota reports to leave out metadir files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Remove the metadata directory tree directories from the quota reporting
in these tests so that we don't regress the golden output.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/filter |    7 +++++--
 common/xfs    |   23 +++++++++++++++++++++++
 tests/xfs/050 |    1 +
 tests/xfs/153 |    1 +
 tests/xfs/299 |    1 +
 tests/xfs/330 |    6 +++++-
 6 files changed, 36 insertions(+), 3 deletions(-)


diff --git a/common/filter b/common/filter
index 3e3fea7ea0..49c6859992 100644
--- a/common/filter
+++ b/common/filter
@@ -618,11 +618,14 @@ _filter_getcap()
 
 # Filter user/group/project id numbers out of quota reports, and standardize
 # the block counts to use filesystem block size.  Callers must set the id and
-# bsize variables before calling this function.
+# bsize variables before calling this function.  The qhidden_rootfiles variable
+# (by default zero) is the number of root files to filter out of the inode
+# count part of the quota report.
 _filter_quota_report()
 {
 	test -n "$id" || echo "id must be set"
 	test -n "$bsize" || echo "block size must be set"
+	test -n "$qhidden_rootfiles" || qhidden_rootfiles=0
 
 	tr -s '[:space:]' | \
 	perl -npe '
@@ -630,7 +633,7 @@ _filter_quota_report()
 		s/^\#0 \d+ /[ROOT] 0 /g;
 		s/6 days/7 days/g' |
 	perl -npe '
-		$val = 0;
+		$val = '"$qhidden_rootfiles"';
 		if ($ENV{'LARGE_SCRATCH_DEV'}) {
 			$val = $ENV{'NUM_SPACE_FILES'};
 		}
diff --git a/common/xfs b/common/xfs
index 0f69d3eb18..99e377631b 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1783,3 +1783,26 @@ _scratch_xfs_force_no_metadir()
 		MKFS_OPTIONS="-m metadir=0 $MKFS_OPTIONS"
 	fi
 }
+
+# Decide if a mount filesystem has metadata directory trees.
+_xfs_mount_has_metadir() {
+	local mount="$1"
+
+	# spaceman (and its info command) predate metadir
+	test ! -e "$XFS_SPACEMAN_PROG" && return 1
+	$XFS_SPACEMAN_PROG -c "info" "$mount" | grep -q 'metadir=1'
+}
+
+# Compute the number of files in the metadata directory tree.
+_xfs_calc_metadir_files() {
+	local mount="$1"
+
+	if ! _xfs_mount_has_metadir "$mount"; then
+		echo 0
+		return
+	fi
+
+	local regfiles="$($XFS_IO_PROG -c 'bulkstat' "$mount" | grep '^bs_ino' | wc -l)"
+	local metafiles="$($XFS_IO_PROG -c 'bulkstat -m' "$mount" 2>&1 | grep '^bs_ino' | wc -l)"
+	echo $((metafiles - regfiles))
+}
diff --git a/tests/xfs/050 b/tests/xfs/050
index 2220e47016..64fbaf687d 100755
--- a/tests/xfs/050
+++ b/tests/xfs/050
@@ -34,6 +34,7 @@ _require_xfs_quota
 _scratch_mkfs >/dev/null 2>&1
 _scratch_mount
 bsize=$(_get_file_block_size $SCRATCH_MNT)
+qhidden_rootfiles=$(_xfs_calc_metadir_files $SCRATCH_MNT)
 _scratch_unmount
 
 bsoft=$(( 200 * $bsize ))
diff --git a/tests/xfs/153 b/tests/xfs/153
index dbe26b6803..fc64bf734a 100755
--- a/tests/xfs/153
+++ b/tests/xfs/153
@@ -39,6 +39,7 @@ _require_test_program "vfs/mount-idmapped"
 _scratch_mkfs >/dev/null 2>&1
 _scratch_mount
 bsize=$(_get_file_block_size $SCRATCH_MNT)
+qhidden_rootfiles=$(_xfs_calc_metadir_files $SCRATCH_MNT)
 _scratch_unmount
 
 bsoft=$(( 200 * $bsize ))
diff --git a/tests/xfs/299 b/tests/xfs/299
index 4b9df3c6aa..2167c492c4 100755
--- a/tests/xfs/299
+++ b/tests/xfs/299
@@ -159,6 +159,7 @@ _qmount_option "uquota,gquota,pquota"
 _qmount
 
 bsize=$(_get_file_block_size $SCRATCH_MNT)
+qhidden_rootfiles=$(_xfs_calc_metadir_files $SCRATCH_MNT)
 
 bsoft=$(( 100 * $bsize ))
 bhard=$(( 500 * $bsize ))
diff --git a/tests/xfs/330 b/tests/xfs/330
index c6e74e67e8..e919ccc1ca 100755
--- a/tests/xfs/330
+++ b/tests/xfs/330
@@ -26,7 +26,10 @@ _require_nobody
 
 do_repquota()
 {
-	repquota $SCRATCH_MNT | grep -E '^(fsgqa|root|nobody)' | sort -r
+	repquota $SCRATCH_MNT | grep -E '^(fsgqa|root|nobody)' | sort -r | \
+	perl -npe '
+		$val = '"$qhidden_rootfiles"';
+		s/(^root\s+--\s+\S+\s+\S+\s+\S+\s+)(\S+)/$1@{[$2 - $val]}/g'
 }
 
 rm -f "$seqres.full"
@@ -35,6 +38,7 @@ echo "Format and mount"
 _scratch_mkfs > "$seqres.full" 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> "$seqres.full" 2>&1
+qhidden_rootfiles=$(_xfs_calc_metadir_files $SCRATCH_MNT)
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 

[PATCH 9/9] xfs: create fuzz tests for metadata directories

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Create fuzz tests to make sure that all the validation works for
metadata directories and subdirectories.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs         |   22 ++++++++++++++++++++++
 tests/xfs/1546     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1546.out |    4 ++++
 tests/xfs/1547     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1547.out |    4 ++++
 tests/xfs/1548     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1548.out |    4 ++++
 tests/xfs/1549     |   38 ++++++++++++++++++++++++++++++++++++++
 tests/xfs/1549.out |    4 ++++
 tests/xfs/1550     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1550.out |    4 ++++
 tests/xfs/1551     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1551.out |    4 ++++
 tests/xfs/1552     |   37 +++++++++++++++++++++++++++++++++++++
 tests/xfs/1552.out |    4 ++++
 tests/xfs/1553     |   38 ++++++++++++++++++++++++++++++++++++++
 tests/xfs/1553.out |    4 ++++
 17 files changed, 352 insertions(+)
 create mode 100755 tests/xfs/1546
 create mode 100644 tests/xfs/1546.out
 create mode 100755 tests/xfs/1547
 create mode 100644 tests/xfs/1547.out
 create mode 100755 tests/xfs/1548
 create mode 100644 tests/xfs/1548.out
 create mode 100755 tests/xfs/1549
 create mode 100644 tests/xfs/1549.out
 create mode 100755 tests/xfs/1550
 create mode 100644 tests/xfs/1550.out
 create mode 100755 tests/xfs/1551
 create mode 100644 tests/xfs/1551.out
 create mode 100755 tests/xfs/1552
 create mode 100644 tests/xfs/1552.out
 create mode 100755 tests/xfs/1553
 create mode 100644 tests/xfs/1553.out


diff --git a/common/xfs b/common/xfs
index 99e377631b..77af8a6d60 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1806,3 +1806,25 @@ _xfs_calc_metadir_files() {
 	local metafiles="$($XFS_IO_PROG -c 'bulkstat -m' "$mount" 2>&1 | grep '^bs_ino' | wc -l)"
 	echo $((metafiles - regfiles))
 }
+
+_require_xfs_mkfs_metadir()
+{
+	_scratch_mkfs_xfs_supported -m metadir=1 >/dev/null 2>&1 || \
+		_notrun "mkfs.xfs doesn't have metadir features"
+}
+
+_require_xfs_scratch_metadir()
+{
+	_require_xfs_mkfs_metadir
+	_require_scratch
+
+	_scratch_mkfs -m metadir=1 &> /dev/null
+	_require_scratch_xfs_features METADIR
+	_try_scratch_mount
+	res=$?
+	if [ $res -ne 0 ]; then
+		_notrun "mounting with metadir not supported by filesystem type: $FSTYP"
+	else
+		_scratch_unmount
+	fi
+}
diff --git a/tests/xfs/1546 b/tests/xfs/1546
new file mode 100755
index 0000000000..5b48463abe
--- /dev/null
+++ b/tests/xfs/1546
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1546
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Use xfs_scrub to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_online_repair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'online' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1546.out b/tests/xfs/1546.out
new file mode 100644
index 0000000000..b72891a758
--- /dev/null
+++ b/tests/xfs/1546.out
@@ -0,0 +1,4 @@
+QA output created by 1546
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1547 b/tests/xfs/1547
new file mode 100755
index 0000000000..ff86bc657e
--- /dev/null
+++ b/tests/xfs/1547
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1547
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Use xfs_repair to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_repair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'offline' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1547.out b/tests/xfs/1547.out
new file mode 100644
index 0000000000..983cc01343
--- /dev/null
+++ b/tests/xfs/1547.out
@@ -0,0 +1,4 @@
+QA output created by 1547
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1548 b/tests/xfs/1548
new file mode 100755
index 0000000000..1f29dfda3b
--- /dev/null
+++ b/tests/xfs/1548
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1548
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Do not fix the filesystem, to test metadata verifiers.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_norepair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'none' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1548.out b/tests/xfs/1548.out
new file mode 100644
index 0000000000..9e395bb059
--- /dev/null
+++ b/tests/xfs/1548.out
@@ -0,0 +1,4 @@
+QA output created by 1548
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1549 b/tests/xfs/1549
new file mode 100755
index 0000000000..865023f218
--- /dev/null
+++ b/tests/xfs/1549
@@ -0,0 +1,38 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1549
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_bothrepair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'both' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1549.out b/tests/xfs/1549.out
new file mode 100644
index 0000000000..22b3d215e3
--- /dev/null
+++ b/tests/xfs/1549.out
@@ -0,0 +1,4 @@
+QA output created by 1549
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1550 b/tests/xfs/1550
new file mode 100755
index 0000000000..62219e65fc
--- /dev/null
+++ b/tests/xfs/1550
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1550
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Use xfs_scrub to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_online_repair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'online' 'path -m /realtime' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1550.out b/tests/xfs/1550.out
new file mode 100644
index 0000000000..7694cd670b
--- /dev/null
+++ b/tests/xfs/1550.out
@@ -0,0 +1,4 @@
+QA output created by 1550
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir
diff --git a/tests/xfs/1551 b/tests/xfs/1551
new file mode 100755
index 0000000000..f101529364
--- /dev/null
+++ b/tests/xfs/1551
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1551
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Use xfs_repair to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_repair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'offline' 'path -m /realtime' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1551.out b/tests/xfs/1551.out
new file mode 100644
index 0000000000..4c3360d08b
--- /dev/null
+++ b/tests/xfs/1551.out
@@ -0,0 +1,4 @@
+QA output created by 1551
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir
diff --git a/tests/xfs/1552 b/tests/xfs/1552
new file mode 100755
index 0000000000..ab3b89ec40
--- /dev/null
+++ b/tests/xfs/1552
@@ -0,0 +1,37 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1552
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Do not fix the filesystem, to test metadata verifiers.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_norepair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'none' 'path -m /realtime' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1552.out b/tests/xfs/1552.out
new file mode 100644
index 0000000000..6636b1b656
--- /dev/null
+++ b/tests/xfs/1552.out
@@ -0,0 +1,4 @@
+QA output created by 1552
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir
diff --git a/tests/xfs/1553 b/tests/xfs/1553
new file mode 100755
index 0000000000..6acbacbe16
--- /dev/null
+++ b/tests/xfs/1553
@@ -0,0 +1,38 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1553
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_bothrepair
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'both' 'path -m /realtime' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1553.out b/tests/xfs/1553.out
new file mode 100644
index 0000000000..0298fcfddb
--- /dev/null
+++ b/tests/xfs/1553.out
@@ -0,0 +1,4 @@
+QA output created by 1553
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir

[PATCH 7/9] xfs/769: add metadir upgrade to test matrix

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add metadata directory trees to the features that this test will try to
upgrade.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/769 |    2 ++
 1 file changed, 2 insertions(+)


diff --git a/tests/xfs/769 b/tests/xfs/769
index 7613048f52..624dd2a338 100755
--- a/tests/xfs/769
+++ b/tests/xfs/769
@@ -174,12 +174,14 @@ if rt_configured; then
 	check_repair_upgrade finobt && FEATURES+=("finobt")
 	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
 	check_repair_upgrade bigtime && FEATURES+=("bigtime")
+	check_repair_upgrade metadir && FEATURES+=("metadir")
 else
 	check_repair_upgrade finobt && FEATURES+=("finobt")
 	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
 	check_repair_upgrade reflink && FEATURES+=("reflink")
 	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
 	check_repair_upgrade bigtime && FEATURES+=("bigtime")
+	check_repair_upgrade metadir && FEATURES+=("metadir")
 fi
 
 test "${#FEATURES[@]}" -eq 0 && \

[PATCH 5/9] xfs/206: update for metadata directory support

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Filter 'metadir=' out of the golden output so that metadata directories
don't cause this test to regress.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/206 |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/206 b/tests/xfs/206
index cb346b6dc9..c181d7dd3e 100755
--- a/tests/xfs/206
+++ b/tests/xfs/206
@@ -64,7 +64,8 @@ mkfs_filter()
 	    -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \
 	    -e "s/, lazy-count=[0-9]//" \
 	    -e "/.*crc=/d" \
-	    -e "/^Default configuration/d"
+	    -e "/^Default configuration/d" \
+	    -e "/metadir=.*/d"
 }
 
 # mkfs slightly smaller than that, small log for speed.

[PATCH 8/9] xfs/509: adjust inumbers accounting for metadata directories

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The INUMBERS ioctl exports data from the inode btree directly -- the
number of inodes it reports is taken from ir_freemask and includes all
the files in the metadata directory tree.  BULKSTAT, on the other hand,
only reports non-metadata files.  When metadir is enabled, this will
(eventually) cause a discrepancy in the inode counts that is large
enough to exceed the tolerances, thereby causing a test failure.

Correct this by counting the files in the metadata directory and
subtracting that from the INUMBERS totals.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/509 |   21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/509 b/tests/xfs/509
index d04dfbbfba..b87abef964 100755
--- a/tests/xfs/509
+++ b/tests/xfs/509
@@ -91,13 +91,13 @@ inumbers_count()
 	bstat_versions | while read v_tag v_flag; do
 		echo -n "inumbers all($v_tag): "
 		nr=$(inumbers_fs $SCRATCH_MNT $v_flag)
-		_within_tolerance "inumbers" $nr $expect $tolerance -v
+		_within_tolerance "inumbers" $((nr - METADATA_FILES)) $expect $tolerance -v
 
 		local agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
 		for batchsize in 71 2 1; do
 			echo -n "inumbers $batchsize($v_tag): "
 			nr=$(inumbers_ag $agcount $batchsize $SCRATCH_MNT $v_flag)
-			_within_tolerance "inumbers" $nr $expect $tolerance -v
+			_within_tolerance "inumbers" $((nr - METADATA_FILES)) $expect $tolerance -v
 		done
 	done
 }
@@ -143,9 +143,26 @@ _supported_fs xfs
 DIRCOUNT=8
 INOCOUNT=$((2048 / DIRCOUNT))
 
+# Count everything in the metadata directory tree.
+count_metadir_files() {
+	local metadirs=('/realtime' '/quota')
+	local db_args=('-f')
+
+	for m in "${metadirs[@]}"; do
+		db_args+=('-c' "ls -m $m")
+	done
+
+	local ret=$(_scratch_xfs_db "${db_args[@]}" 2>/dev/null | grep regular | wc -l)
+	test -z "$ret" && ret=0
+	echo $ret
+}
+
 _scratch_mkfs "-d agcount=$DIRCOUNT" >> $seqres.full 2>&1 || _fail "mkfs failed"
 _scratch_mount
 
+METADATA_FILES=$(count_metadir_files)
+echo "found $METADATA_FILES metadata files" >> $seqres.full
+
 # Figure out if we have v5 bulkstat/inumbers ioctls.
 has_v5=
 bs_root_out="$($XFS_IO_PROG -c 'bulkstat_single root' $SCRATCH_MNT 2>>$seqres.full)"

[PATCHSET v1.0 0/4] fstests: support metadump to external devices

[Darrick J. Wong]

Hi all,

This series modifies fstests to take advantage of the fact that
xfs_metadump and xfs_mdrestore now support capturing the contents of an
external log in a metadump, and restoring it on the other end.  The
first part of this series refactors and cleans up the common code a bit,
and the rest add the actual support.  Once this is merged, we'll be able
to cache metadumps of populated filesystems with external log devices,
which will enable faster fuzz testing.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=metadump-external-devices

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=metadump-external-devices
---
 common/ext4     |   17 ++++++++++++-
 common/fuzzy    |    7 +++++
 common/populate |   72 ++++++++++++++++++++++++++++---------------------------
 common/xfs      |   39 ++++++++++++++++++++++++++++--
 4 files changed, 95 insertions(+), 40 deletions(-)

[PATCH 3/4] common/ext4: reformat external logs during mdrestore operations

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The e2image file format doesn't support the capture of external log
devices, which means that mdrestore ought to reformat the external log
to get the restored filesystem to work again.  The common/populate code
could already do this, so push it to the common ext4 helper.

While we're at it, fix the uncareful usage of SCRATCH_LOGDEV in the
populate code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/ext4     |   17 ++++++++++++++++-
 common/populate |   16 ++--------------
 2 files changed, 18 insertions(+), 15 deletions(-)


diff --git a/common/ext4 b/common/ext4
index 3dcbfe17c9..5171b8df68 100644
--- a/common/ext4
+++ b/common/ext4
@@ -134,7 +134,8 @@ _ext4_mdrestore()
 {
 	local metadump="$1"
 	local device="$2"
-	shift; shift
+	local logdev="$3"
+	shift; shift; shift
 	local options="$@"
 
 	# If we're configured for compressed dumps and there isn't already an
@@ -148,6 +149,20 @@ _ext4_mdrestore()
 	test -r "$metadump" || return 1
 
 	$E2IMAGE_PROG $options -r "${metadump}" "${SCRATCH_DEV}"
+	res=$?
+	test $res -ne 0 && return $res
+
+	# ext4 cannot e2image external logs, so we have to reformat the log
+	# device to match the restored fs
+	if [ "${logdev}" != "none" ]; then
+		local fsuuid="$($DUMPE2FS_PROG -h "${SCRATCH_DEV}" 2>/dev/null | \
+				grep 'Journal UUID:' | \
+				sed -e 's/Journal UUID:[[:space:]]*//g')"
+		$MKFS_EXT4_PROG -O journal_dev "${logdev}" \
+				-F -U "${fsuuid}"
+		res=$?
+	fi
+	return $res
 }
 
 # this test requires the ext4 kernel support crc feature on scratch device
diff --git a/common/populate b/common/populate
index 08c4bdc151..095e771d67 100644
--- a/common/populate
+++ b/common/populate
@@ -912,20 +912,8 @@ _scratch_populate_restore_cached() {
 		return $?
 		;;
 	"ext2"|"ext3"|"ext4")
-		_ext4_mdrestore "${metadump}" "${SCRATCH_DEV}"
-		ret=$?
-		test $ret -ne 0 && return $ret
-
-		# ext4 cannot e2image external logs, so we have to reformat
-		# the scratch device to match the restored fs
-		if [ -n "${SCRATCH_LOGDEV}" ]; then
-			local fsuuid="$($DUMPE2FS_PROG -h "${SCRATCH_DEV}" 2>/dev/null | \
-					grep 'Journal UUID:' | \
-					sed -e 's/Journal UUID:[[:space:]]*//g')"
-			$MKFS_EXT4_PROG -O journal_dev "${SCRATCH_LOGDEV}" \
-					-F -U "${fsuuid}"
-		fi
-		return 0
+		_ext4_mdrestore "${metadump}" "${SCRATCH_DEV}" "${logdev}"
+		return $?
 		;;
 	esac
 	return 1

[PATCH 2/4] common/xfs: wipe external logs during mdrestore operations

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The XFS metadump file format doesn't support the capture of external log
devices, which means that mdrestore ought to wipe the external log and
run xfs_repair to rewrite the log device as needed to get the restored
filesystem to work again.  The common/populate code could already do
this, so push it to the common xfs helper.

While we're at it, fix the uncareful usage of SCRATCH_LOGDEV in the
populate code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/fuzzy    |    7 ++++++-
 common/populate |   19 ++++++-------------
 common/xfs      |   21 +++++++++++++++++++--
 3 files changed, 31 insertions(+), 16 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index ef54f2fe2c..7034ff8c42 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -297,7 +297,12 @@ __scratch_xfs_fuzz_unmount()
 __scratch_xfs_fuzz_mdrestore()
 {
 	__scratch_xfs_fuzz_unmount
-	_xfs_mdrestore "${POPULATE_METADUMP}" "${SCRATCH_DEV}" || \
+
+	local logdev=none
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+		logdev=$SCRATCH_LOGDEV
+
+	_xfs_mdrestore "${POPULATE_METADUMP}" "${SCRATCH_DEV}" "${logdev}" || \
 		_fail "${POPULATE_METADUMP}: Could not find metadump to restore?"
 }
 
diff --git a/common/populate b/common/populate
index 8db7acefb6..08c4bdc151 100644
--- a/common/populate
+++ b/common/populate
@@ -902,21 +902,14 @@ _scratch_populate_cache_tag() {
 _scratch_populate_restore_cached() {
 	local metadump="$1"
 
+	local logdev=none
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+		logdev=$SCRATCH_LOGDEV
+
 	case "${FSTYP}" in
 	"xfs")
-		_xfs_mdrestore "${metadump}" "${SCRATCH_DEV}"
-		res=$?
-		test $res -ne 0 && return $res
-
-		# Cached images should have been unmounted cleanly, so if
-		# there's an external log we need to wipe it and run repair to
-		# format it to match this filesystem.
-		if [ -n "${SCRATCH_LOGDEV}" ]; then
-			$WIPEFS_PROG -a "${SCRATCH_LOGDEV}"
-			_scratch_xfs_repair
-			res=$?
-		fi
-		return $res
+		_xfs_mdrestore "${metadump}" "${SCRATCH_DEV}" "${logdev}"
+		return $?
 		;;
 	"ext2"|"ext3"|"ext4")
 		_ext4_mdrestore "${metadump}" "${SCRATCH_DEV}"
diff --git a/common/xfs b/common/xfs
index 77af8a6d60..29130fabbc 100644
--- a/common/xfs
+++ b/common/xfs
@@ -682,7 +682,8 @@ _xfs_metadump() {
 _xfs_mdrestore() {
 	local metadump="$1"
 	local device="$2"
-	shift; shift
+	local logdev="$3"
+	shift; shift; shift
 	local options="$@"
 
 	# If we're configured for compressed dumps and there isn't already an
@@ -696,6 +697,18 @@ _xfs_mdrestore() {
 	test -r "$metadump" || return 1
 
 	$XFS_MDRESTORE_PROG $options "${metadump}" "${device}"
+	res=$?
+	test $res -ne 0 && return $res
+
+	# Cached images should have been unmounted cleanly, so if there's an
+	# external log we need to wipe it and run repair to format it to match
+	# this filesystem.
+	if [ "${logdev}" != "none" ]; then
+		$WIPEFS_PROG -a "${logdev}"
+		_scratch_xfs_repair
+		res=$?
+	fi
+	return $res
 }
 
 # Snapshot the metadata on the scratch device
@@ -717,7 +730,11 @@ _scratch_xfs_mdrestore()
 	local metadump=$1
 	shift
 
-	_xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$@"
+	local logdev=none
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+		logdev=$SCRATCH_LOGDEV
+
+	_xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$logdev" "$@"
 }
 
 # run xfs_check and friends on a FS.

[PATCH 1/4] common/populate: refactor caching of metadumps to a helper

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Hoist out of _scratch_populate_cached all the code that we use to save a
metadump of the populated filesystem.  We're going to make this more
involved for XFS in the next few patches so that we can take advantage
of the new support for external devices in metadump/mdrestore.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/populate |   37 ++++++++++++++++++++++++++++---------
 1 file changed, 28 insertions(+), 9 deletions(-)


diff --git a/common/populate b/common/populate
index 29ea637ecb..8db7acefb6 100644
--- a/common/populate
+++ b/common/populate
@@ -938,6 +938,31 @@ _scratch_populate_restore_cached() {
 	return 1
 }
 
+# Take a metadump of the scratch filesystem and cache it for later.
+_scratch_populate_save_metadump()
+{
+	local metadump_file="$1"
+
+	case "${FSTYP}" in
+	"xfs")
+		local logdev=none
+		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+			logdev=$SCRATCH_LOGDEV
+
+		_xfs_metadump "$metadump_file" "$SCRATCH_DEV" "$logdev" \
+				compress
+		res=$?
+		;;
+	"ext2"|"ext3"|"ext4")
+		_ext4_metadump "${SCRATCH_DEV}" "${metadump_file}" compress
+		res=$?
+		;;
+	*)
+		_fail "Don't know how to save a ${FSTYP} filesystem."
+	esac
+	return $res
+}
+
 # Populate a scratch FS from scratch or from a cached image.
 _scratch_populate_cached() {
 	local meta_descr="$(_scratch_populate_cache_tag "$@")"
@@ -961,26 +986,20 @@ _scratch_populate_cached() {
 
 	# Oh well, just create one from scratch
 	_scratch_mkfs
-	echo "${meta_descr}" > "${populate_metadump_descr}"
 	case "${FSTYP}" in
 	"xfs")
 		_scratch_xfs_populate $@
 		_scratch_xfs_populate_check
-
-		local logdev=none
-		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
-			logdev=$SCRATCH_LOGDEV
-
-		_xfs_metadump "$POPULATE_METADUMP" "$SCRATCH_DEV" "$logdev" \
-			compress
 		;;
 	"ext2"|"ext3"|"ext4")
 		_scratch_ext4_populate $@
 		_scratch_ext4_populate_check
-		_ext4_metadump "${SCRATCH_DEV}" "${POPULATE_METADUMP}" compress
 		;;
 	*)
 		_fail "Don't know how to populate a ${FSTYP} filesystem."
 		;;
 	esac
+
+	_scratch_populate_save_metadump "${POPULATE_METADUMP}" && \
+			echo "${meta_descr}" > "${populate_metadump_descr}"
 }

[PATCH 4/4] common/xfs: capture external logs during metadump/mdrestore

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If xfs_metadump supports the -x switch to capture the contents of
external log devices and there is an external log device, add the option
to the command line to enable preservation.

Similarly, if xfs_mdrestore supports the -l switch and there's an
external scratch log, pass the option so that we can restore log
contents.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)


diff --git a/common/xfs b/common/xfs
index 29130fabbc..36e02413db 100644
--- a/common/xfs
+++ b/common/xfs
@@ -667,9 +667,20 @@ _xfs_metadump() {
 	shift; shift; shift; shift
 	local options="$@"
 	test -z "$options" && options="-a -o"
+	local metadump_has_dash_x
+
+	# Does metadump support capturing from external devices?
+	$XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-[a-zA-Z]*[wW]x' && \
+			metadump_has_dash_x=1
 
 	if [ "$logdev" != "none" ]; then
 		options="$options -l $logdev"
+
+		# Tell metadump to capture the log device
+		if [ -n "$metadump_has_dash_x" ]; then
+			options="$options -x"
+			unset metadump_has_dash_x
+		fi
 	fi
 
 	$XFS_METADUMP_PROG $options "$device" "$metadump"
@@ -696,6 +707,13 @@ _xfs_mdrestore() {
 	fi
 	test -r "$metadump" || return 1
 
+	# Does mdrestore support restoring to external log devices?  If so,
+	# restore to it, and do not wipe it afterwards.
+	if [ "$logdev" != "none" ] && $XFS_MDRESTORE_PROG --help 2>&1 | grep -q -- '-l logdev'; then
+		options="$options -l $logdev"
+		logdev="none"
+	fi
+
 	$XFS_MDRESTORE_PROG $options "${metadump}" "${device}"
 	res=$?
 	test $res -ne 0 && return $res

[PATCHSET v1.0 00/12] xfsprogs: shard the realtime section

[Darrick J. Wong]

Hi all,

Right now, the realtime section uses a single pair of metadata inodes to
store the free space information.  This presents a scalability problem
since every thread trying to allocate or free rt extents have to lock
these files.  It would be very useful if we could begin to tackle these
problems by sharding the realtime section, so create the notion of
realtime groups, which are similar to allocation groups on the data
section.

While we're at it, define a superblock to be stamped into the start of
each rt section.  This enables utilities such as blkid to identify block
devices containing realtime sections, and helpfully avoids the situation
where a file extent can cross an rtgroup boundary.

The best advantage for rtgroups will become evident later when we get to
adding rmap and reflink to the realtime volume, since the geometry
constraints are the same for rt groups and AGs.  Hence we can reuse all
that code directly.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-groups

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-groups

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-groups
---
 common/fuzzy            |   33 +++++++++++++++----
 common/populate         |   12 ++++++-
 common/xfs              |   83 ++++++++++++++++++++++++++++++++++++++++-------
 src/punch-alternating.c |   28 +++++++++++++++-
 tests/xfs/114           |    4 ++
 tests/xfs/122           |    2 +
 tests/xfs/122.out       |    8 +++++
 tests/xfs/146           |    2 +
 tests/xfs/185           |    2 +
 tests/xfs/187           |    3 +-
 tests/xfs/206           |    3 +-
 tests/xfs/271           |    3 +-
 tests/xfs/341           |    4 +-
 tests/xfs/449           |    6 +++
 tests/xfs/556           |   16 ++++++---
 tests/xfs/800           |    2 +
 tests/xfs/840           |    2 +
 tests/xfs/841           |    2 +
 18 files changed, 176 insertions(+), 39 deletions(-)

[PATCH 03/12] xfs/206: update mkfs filtering for rt groups feature

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Filter out the new mkfs lines that show the rtgroup information, since
this test is heavily dependent on old mkfs output.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/206 |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/206 b/tests/xfs/206
index c181d7dd3e..904d53deb0 100755
--- a/tests/xfs/206
+++ b/tests/xfs/206
@@ -65,7 +65,8 @@ mkfs_filter()
 	    -e "s/, lazy-count=[0-9]//" \
 	    -e "/.*crc=/d" \
 	    -e "/^Default configuration/d" \
-	    -e "/metadir=.*/d"
+	    -e "/metadir=.*/d" \
+	    -e '/rgcount=/d'
 }
 
 # mkfs slightly smaller than that, small log for speed.

[PATCH 01/12] xfs/122: update for rtgroups

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add our new metadata for realtime allocation groups to the ondisk checking.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122.out |    5 +++++
 1 file changed, 5 insertions(+)


diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index eee6c1ee6d..01376180cc 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -44,6 +44,9 @@ offsetof(xfs_sb_t, sb_rbmino) = 64
 offsetof(xfs_sb_t, sb_rextents) = 24
 offsetof(xfs_sb_t, sb_rextsize) = 80
 offsetof(xfs_sb_t, sb_rextslog) = 125
+offsetof(xfs_sb_t, sb_rgblklog) = 280
+offsetof(xfs_sb_t, sb_rgblocks) = 272
+offsetof(xfs_sb_t, sb_rgcount) = 276
 offsetof(xfs_sb_t, sb_rootino) = 56
 offsetof(xfs_sb_t, sb_rrmapino) = 264
 offsetof(xfs_sb_t, sb_rsumino) = 72
@@ -112,9 +115,11 @@ sizeof(struct xfs_refcount_key) = 4
 sizeof(struct xfs_refcount_rec) = 12
 sizeof(struct xfs_rmap_key) = 20
 sizeof(struct xfs_rmap_rec) = 24
+sizeof(struct xfs_rtgroup_geometry) = 128
 sizeof(struct xfs_rtrmap_key) = 24
 sizeof(struct xfs_rtrmap_rec) = 32
 sizeof(struct xfs_rtrmap_root) = 4
+sizeof(struct xfs_rtsb) = 104
 sizeof(struct xfs_rud_log_format) = 16
 sizeof(struct xfs_rui_log_format) = 16
 sizeof(struct xfs_scrub_metadata) = 64

[PATCH 02/12] punch-alternating: detect xfs realtime files with large allocation units

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

For files on the XFS realtime volume, it's possible that the file
allocation unit (aka the minimum size we have to punch to deallocate
file blocks) could be greater than a single fs block.  This utility
assumed that it's always possible to punch a single fs block, but for
these types of files, all that does is zeroes the page cache.  While
that's what most *user applications* want, fstests uses punching to
fragment file mapping metadata and/or fragment free space, so adapt this
test for that purpose by detecting realtime files.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 src/punch-alternating.c |   28 +++++++++++++++++++++++++++-
 tests/xfs/114           |    4 ++++
 tests/xfs/146           |    2 +-
 tests/xfs/187           |    3 ++-
 tests/xfs/341           |    4 ++--
 5 files changed, 36 insertions(+), 5 deletions(-)


diff --git a/src/punch-alternating.c b/src/punch-alternating.c
index 18dd215197..d2bb4b6a22 100644
--- a/src/punch-alternating.c
+++ b/src/punch-alternating.c
@@ -20,6 +20,28 @@ void usage(char *cmd)
 	exit(1);
 }
 
+/* Compute the file allocation unit size for an XFS file. */
+static int detect_xfs_alloc_unit(int fd)
+{
+	struct fsxattr fsx;
+	struct xfs_fsop_geom fsgeom;
+	int ret;
+
+	ret = ioctl(fd, XFS_IOC_FSGEOMETRY, &fsgeom);
+	if (ret)
+		return -1;
+
+	ret = ioctl(fd, XFS_IOC_FSGETXATTR, &fsx);
+	if (ret)
+		return -1;
+
+	ret = fsgeom.blocksize;
+	if (fsx.fsx_xflags & XFS_XFLAG_REALTIME)
+		ret *= fsgeom.rtextsize;
+
+	return ret;
+}
+
 int main(int argc, char *argv[])
 {
 	struct stat	s;
@@ -82,7 +104,11 @@ int main(int argc, char *argv[])
 		goto err;
 
 	sz = s.st_size;
-	blksz = sf.f_bsize;
+	c = detect_xfs_alloc_unit(fd);
+	if (c > 0)
+		blksz = c;
+	else
+		blksz = sf.f_bsize;
 
 	mode = FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE;
 	for (offset = start_offset * blksz;
diff --git a/tests/xfs/114 b/tests/xfs/114
index 0e8a0529ab..7ecb4d217c 100755
--- a/tests/xfs/114
+++ b/tests/xfs/114
@@ -49,6 +49,10 @@ $XFS_IO_PROG -f \
 	-c "pwrite -S 0x68 -b 1048576 0 $len2" \
 	$SCRATCH_MNT/f2 >> $seqres.full
 
+# The arguments to punch-alternating must be specified in units of file
+# allocation units, so we divide the argument by $file_blksz.  We already
+# verified that $blksz is congruent with $file_blksz, so the fpunch parameters
+# will always align with the file allocation unit.
 $here/src/punch-alternating -o $((16 * blksz / file_blksz)) \
 	-s $((blksz / file_blksz)) \
 	-i $((blksz * 2 / file_blksz)) \
diff --git a/tests/xfs/146 b/tests/xfs/146
index 123bdff59f..c1ef5e7e1b 100755
--- a/tests/xfs/146
+++ b/tests/xfs/146
@@ -68,7 +68,7 @@ _xfs_force_bdev realtime $SCRATCH_MNT
 # Allocate some stuff at the start, to force the first falloc of the ouch file
 # to happen somewhere in the middle of the rt volume
 $XFS_IO_PROG -f -c 'falloc 0 64m' "$SCRATCH_MNT/b"
-$here/src/punch-alternating -i $((rextblks * 2)) -s $((rextblks)) "$SCRATCH_MNT/b"
+$here/src/punch-alternating "$SCRATCH_MNT/b"
 
 avail="$(df -P "$SCRATCH_MNT" | awk 'END {print $4}')"1
 toobig="$((avail * 2))"
diff --git a/tests/xfs/187 b/tests/xfs/187
index 7c34d8e630..14c3b37670 100755
--- a/tests/xfs/187
+++ b/tests/xfs/187
@@ -132,7 +132,8 @@ $XFS_IO_PROG -f -c "truncate $required_sz" -c "falloc 0 $remap_sz" $SCRATCH_MNT/
 # Punch out every other extent of the last two sections, to fragment free space.
 frag_sz=$((remap_sz * 3))
 punch_off=$((bigfile_sz - frag_sz))
-$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / fsbsize)) -i $((rtextsize_blks * 2)) -s $rtextsize_blks
+rtextsize_bytes=$((fsbsize * rtextsize_blks))
+$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / rtextsize_bytes))
 
 # Make sure we have some free rtextents.
 free_rtx=$($XFS_IO_PROG -c 'statfs' $SCRATCH_MNT | grep statfs.f_bavail | awk '{print $3}')
diff --git a/tests/xfs/341 b/tests/xfs/341
index 1f734c9015..7d2842b579 100755
--- a/tests/xfs/341
+++ b/tests/xfs/341
@@ -43,8 +43,8 @@ len=$((blocks * rtextsz))
 echo "Create some files"
 $XFS_IO_PROG -f -R -c "falloc 0 $len" -c "pwrite -S 0x68 -b 1048576 0 $len" $SCRATCH_MNT/f1 >> $seqres.full
 $XFS_IO_PROG -f -R -c "falloc 0 $len" -c "pwrite -S 0x68 -b 1048576 0 $len" $SCRATCH_MNT/f2 >> $seqres.full
-$here/src/punch-alternating -i $((2 * rtextsz_blks)) -s $rtextsz_blks $SCRATCH_MNT/f1 >> "$seqres.full"
-$here/src/punch-alternating -i $((2 * rtextsz_blks)) -s $rtextsz_blks $SCRATCH_MNT/f2 >> "$seqres.full"
+$here/src/punch-alternating $SCRATCH_MNT/f1 >> "$seqres.full"
+$here/src/punch-alternating $SCRATCH_MNT/f2 >> "$seqres.full"
 echo garbage > $SCRATCH_MNT/f3
 ino=$(stat -c '%i' $SCRATCH_MNT/f3)
 _scratch_unmount

[PATCH 10/12] xfs/27[46],xfs/556: fix tests to deal with rtgroups output in bmap/fsmap commands

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix these tests to deal with the xfs_io bmap and fsmap commands printing
out realtime group numbers if the feature is enabled.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs    |    4 ++++
 tests/xfs/271 |    3 ++-
 tests/xfs/556 |   16 ++++++++++------
 3 files changed, 16 insertions(+), 7 deletions(-)


diff --git a/common/xfs b/common/xfs
index ccdcf45d0d..6089a05d0e 100644
--- a/common/xfs
+++ b/common/xfs
@@ -486,6 +486,10 @@ _xfs_has_feature()
 		feat="rtextents"
 		feat_regex="[1-9][0-9]*"
 		;;
+	"rtgroups")
+		feat="rgcount"
+		feat_regex="[1-9][0-9]*"
+		;;
 	esac
 
 	local answer="$($XFS_INFO_PROG "$fs" 2>&1 | grep -E -w -c "$feat=$feat_regex")"
diff --git a/tests/xfs/271 b/tests/xfs/271
index d67ac4d6c1..74e2c822c1 100755
--- a/tests/xfs/271
+++ b/tests/xfs/271
@@ -31,6 +31,7 @@ _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 
 agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
+rgcount=$(_xfs_mount_rgcount $SCRATCH_MNT)
 
 # mkfs lays out btree root blocks in the order bnobt, cntbt, inobt, finobt,
 # rmapbt, refcountbt, and then allocates AGFL blocks.  Since GETFSMAP has the
@@ -48,7 +49,7 @@ cat $TEST_DIR/fsmap >> $seqres.full
 
 echo "Check AG header" | tee -a $seqres.full
 grep 'static fs metadata[[:space:]]*[0-9]*[[:space:]]*(0\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout
-_within_tolerance "AG header count" $(wc -l < $TEST_DIR/testout) $agcount 0 -v
+_within_tolerance "AG header count" $(wc -l < $TEST_DIR/testout) $((agcount + rgcount)) 0 -v
 
 echo "Check freesp/rmap btrees" | tee -a $seqres.full
 grep 'per-AG metadata[[:space:]]*[0-9]*[[:space:]]*([0-9]*\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout
diff --git a/tests/xfs/556 b/tests/xfs/556
index 66908a5410..72343e8625 100755
--- a/tests/xfs/556
+++ b/tests/xfs/556
@@ -47,16 +47,20 @@ victim=$SCRATCH_MNT/a
 file_blksz=$(_get_file_block_size $SCRATCH_MNT)
 $XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full
 unset errordev
-_xfs_is_realtime_file $victim && errordev="RT"
+
+awk_len_prog='{print $6}'
+if _xfs_is_realtime_file $victim; then
+	if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then
+		awk_len_prog='{print $4}'
+	fi
+	errordev="RT"
+fi
 bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")"
 echo "$errordev:$bmap_str" >> $seqres.full
 
 phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')"
-if [ "$errordev" = "RT" ]; then
-	len="$(echo "$bmap_str" | $AWK_PROG '{print $4}')"
-else
-	len="$(echo "$bmap_str" | $AWK_PROG '{print $6}')"
-fi
+len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")"
+
 fs_blksz=$(_get_block_size $SCRATCH_MNT)
 echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full
 kernel_sectors_per_fs_block=$((fs_blksz / 512))

[PATCH 07/12] xfs/449: update test to know about xfs_db -R

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The realtime groups feature added a -R flag to xfs_db so that users can
pass in the realtime device.  Since we've now modified the
_scratch_xfs_db to use this facility, we can update the test to do exact
comparisons of the xfs_db info command against the mkfs output.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/449 |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/449 b/tests/xfs/449
index 5374bf2f85..66c443e994 100755
--- a/tests/xfs/449
+++ b/tests/xfs/449
@@ -32,7 +32,11 @@ echo DB >> $seqres.full
 cat $tmp.dbinfo >> $seqres.full
 # xfs_db doesn't take a rtdev argument, so it reports "realtime=external".
 # mkfs does, so make a quick substitution
-diff -u <(cat $tmp.mkfs | sed -e 's/realtime =\/.*extsz=/realtime =external               extsz=/g') $tmp.dbinfo
+if $XFS_DB_PROG --help 2>&1 | grep -q -- '-R rtdev'; then
+	diff -u $tmp.mkfs $tmp.dbinfo
+else
+	diff -u <(cat $tmp.mkfs | sed -e 's/realtime =\/.*extsz=/realtime =external               extsz=/g') $tmp.dbinfo
+fi
 
 _scratch_mount
 

[PATCH 11/12] common/xfs: capture realtime devices during metadump/mdrestore

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If xfs_metadump supports the -x and -R switches to capture the contents
of realtime devices and there is a realtime device, add the option to
the command line to enable preservation.

Similarly, if xfs_mdrestore supports the -R switch and there's an
external scratch rtdev, pass the option so that we can restore rtdev
contents.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/fuzzy    |    6 +++++-
 common/populate |   12 ++++++++++--
 common/xfs      |   48 +++++++++++++++++++++++++++++++++++++++---------
 3 files changed, 54 insertions(+), 12 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index 7034ff8c42..7f96384402 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -302,7 +302,11 @@ __scratch_xfs_fuzz_mdrestore()
 	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 		logdev=$SCRATCH_LOGDEV
 
-	_xfs_mdrestore "${POPULATE_METADUMP}" "${SCRATCH_DEV}" "${logdev}" || \
+	local rtdev=none
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+		rtdev=$SCRATCH_RTDEV
+
+	_xfs_mdrestore "${POPULATE_METADUMP}" "${SCRATCH_DEV}" "${logdev}" "${rtdev}" || \
 		_fail "${POPULATE_METADUMP}: Could not find metadump to restore?"
 }
 
diff --git a/common/populate b/common/populate
index 095e771d67..c0bbbc3f3b 100644
--- a/common/populate
+++ b/common/populate
@@ -908,7 +908,11 @@ _scratch_populate_restore_cached() {
 
 	case "${FSTYP}" in
 	"xfs")
-		_xfs_mdrestore "${metadump}" "${SCRATCH_DEV}" "${logdev}"
+		local rtdev=none
+		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+			rtdev=$SCRATCH_RTDEV
+
+		_xfs_mdrestore "${metadump}" "${SCRATCH_DEV}" "${logdev}" "${rtdev}"
 		return $?
 		;;
 	"ext2"|"ext3"|"ext4")
@@ -930,8 +934,12 @@ _scratch_populate_save_metadump()
 		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 			logdev=$SCRATCH_LOGDEV
 
+		local rtdev=none
+		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+			rtdev=$SCRATCH_RTDEV
+
 		_xfs_metadump "$metadump_file" "$SCRATCH_DEV" "$logdev" \
-				compress
+				"$rtdev" compress
 		res=$?
 		;;
 	"ext2"|"ext3"|"ext4")
diff --git a/common/xfs b/common/xfs
index 6089a05d0e..a37284068f 100644
--- a/common/xfs
+++ b/common/xfs
@@ -679,15 +679,20 @@ _xfs_metadump() {
 	local metadump="$1"
 	local device="$2"
 	local logdev="$3"
-	local compressopt="$4"
-	shift; shift; shift; shift
+	local rtdev="$4"
+	local compressopt="$5"
+	shift; shift; shift; shift; shift
 	local options="$@"
 	test -z "$options" && options="-a -o"
 	local metadump_has_dash_x
+	local metadump_has_dash_R
 
 	# Does metadump support capturing from external devices?
 	$XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-[a-zA-Z]*[wW]x' && \
 			metadump_has_dash_x=1
+	# Does metadump support capturing realtime devices?
+	$XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-R rtdev' && \
+			metadump_has_dash_R=1
 
 	if [ "$logdev" != "none" ]; then
 		options="$options -l $logdev"
@@ -699,6 +704,17 @@ _xfs_metadump() {
 		fi
 	fi
 
+	# Capture the realtime device, if possible
+	if [ "$rtdev" != "none" ] && [ -n "$metadump_has_dash_R" ]; then
+		options="$options -R $rtdev"
+
+		# Tell metadump to capture the rt device
+		if [ -n "$metadump_has_dash_x" ]; then
+			options="$options -x"
+			unset metadump_has_dash_x
+		fi
+	fi
+
 	$XFS_METADUMP_PROG $options "$device" "$metadump"
 	res=$?
 	[ "$compressopt" = "compress" ] && [ -n "$DUMP_COMPRESSOR" ] &&
@@ -710,7 +726,8 @@ _xfs_mdrestore() {
 	local metadump="$1"
 	local device="$2"
 	local logdev="$3"
-	shift; shift; shift
+	local rtdev="$4"
+	shift; shift; shift; shift
 	local options="$@"
 
 	# If we're configured for compressed dumps and there isn't already an
@@ -730,6 +747,11 @@ _xfs_mdrestore() {
 		logdev="none"
 	fi
 
+	# Does mdrestore support restoring to realtime devices?
+	if [ "$rtdev" != "none" ] && $XFS_MDRESTORE_PROG --help 2>&1 | grep -q -- '-R rtdev'; then
+		options="$options -R $rtdev"
+	fi
+
 	$XFS_MDRESTORE_PROG $options "${metadump}" "${device}"
 	res=$?
 	test $res -ne 0 && return $res
@@ -750,12 +772,16 @@ _scratch_xfs_metadump()
 {
 	local metadump=$1
 	shift
+
 	local logdev=none
-
 	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 		logdev=$SCRATCH_LOGDEV
 
-	_xfs_metadump "$metadump" "$SCRATCH_DEV" "$logdev" nocompress "$@"
+	local rtdev=none
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+		rtdev=$SCRATCH_RTDEV
+
+	_xfs_metadump "$metadump" "$SCRATCH_DEV" "$logdev" "$rtdev" nocompress "$@"
 }
 
 # Restore snapshotted metadata on the scratch device
@@ -768,7 +794,11 @@ _scratch_xfs_mdrestore()
 	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 		logdev=$SCRATCH_LOGDEV
 
-	_xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$logdev" "$@"
+	local rtdev=none
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+		rtdev=$SCRATCH_RTDEV
+
+	_xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$logdev" "$rtdev" "$@"
 }
 
 # run xfs_check and friends on a FS.
@@ -895,7 +925,7 @@ _check_xfs_filesystem()
 	if [ "$ok" -ne 1 ] && [ "$DUMP_CORRUPT_FS" = "1" ]; then
 		local flatdev="$(basename "$device")"
 		_xfs_metadump "$seqres.$flatdev.check.md" "$device" "$logdev" \
-			compress >> $seqres.full
+			"$rtdev" compress >> $seqres.full
 	fi
 
 	# Optionally test the index rebuilding behavior.
@@ -928,7 +958,7 @@ _check_xfs_filesystem()
 		if [ "$rebuild_ok" -ne 1 ] && [ "$DUMP_CORRUPT_FS" = "1" ]; then
 			local flatdev="$(basename "$device")"
 			_xfs_metadump "$seqres.$flatdev.rebuild.md" "$device" \
-				"$logdev" compress >> $seqres.full
+				"$logdev" "$rtdev" compress >> $seqres.full
 		fi
 	fi
 
@@ -1009,7 +1039,7 @@ _check_xfs_filesystem()
 		if [ "$orebuild_ok" -ne 1 ] && [ "$DUMP_CORRUPT_FS" = "1" ]; then
 			local flatdev="$(basename "$device")"
 			_xfs_metadump "$seqres.$flatdev.orebuild.md" "$device" \
-				"$logdev" compress >> $seqres.full
+				"$logdev" "$rtdev" compress >> $seqres.full
 		fi
 	fi
 

[PATCH 05/12] common: filter rtgroups when we're disabling metadir

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If we're forcing a filesystem to be created without the metadir feature,
we should forcibly disable rtgroups as well.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs |    4 ++++
 1 file changed, 4 insertions(+)


diff --git a/common/xfs b/common/xfs
index 0d1e0ec4bc..ccdcf45d0d 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1821,6 +1821,10 @@ _scratch_xfs_find_metafile()
 # Force metadata directories off.
 _scratch_xfs_force_no_metadir()
 {
+	if echo "$MKFS_OPTIONS" | grep -q 'rtgroups='; then
+		MKFS_OPTIONS="$(echo "$MKFS_OPTIONS" | sed -e 's/rtgroups=\([01]\)/rtgroups=0/g')"
+	fi
+
 	if echo "$MKFS_OPTIONS" | grep -q 'metadir='; then
 		MKFS_OPTIONS="$(echo "$MKFS_OPTIONS" | sed -e 's/metadir=\([01]\)/metadir=0/g')"
 		return

[PATCH 08/12] xfs/122: update for rtbitmap headers

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122.out |    1 +
 1 file changed, 1 insertion(+)


diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 01376180cc..336618cf7a 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -115,6 +115,7 @@ sizeof(struct xfs_refcount_key) = 4
 sizeof(struct xfs_refcount_rec) = 12
 sizeof(struct xfs_rmap_key) = 20
 sizeof(struct xfs_rmap_rec) = 24
+sizeof(struct xfs_rtbuf_blkinfo) = 48
 sizeof(struct xfs_rtgroup_geometry) = 128
 sizeof(struct xfs_rtrmap_key) = 24
 sizeof(struct xfs_rtrmap_rec) = 32

[PATCH 06/12] xfs/185: update for rtgroups

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Send the fallocate results to seqres.full, since it doesn't matter if
the call fails as long as we get the layout that we wanted.  This test
already has code to check the layout, so there's no point in failing on
random ENOSPC errors.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/185 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/xfs/185 b/tests/xfs/185
index abeb052580..04770fd6c9 100755
--- a/tests/xfs/185
+++ b/tests/xfs/185
@@ -100,7 +100,7 @@ test "$ddbytes" -lt "$((rtbytes + (10 * rtextsize) ))" || \
 # easy because fallocate for the first rt file always starts allocating at
 # physical offset zero.
 alloc_rtx="$((rtbytes / rtextsize))"
-$XFS_IO_PROG -c "falloc 0 $((alloc_rtx * rtextsize))" $rtfile
+$XFS_IO_PROG -c "falloc 0 $((alloc_rtx * rtextsize))" $rtfile &>> $seqres.full
 
 expected_end="$(( (alloc_rtx * rtextsize - 1) / 512 ))"
 

[PATCH 04/12] common: pass the realtime device to xfs_db when possible

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Teach xfstests to pass the realtime device to xfs_db when it supports
that option.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs |   18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)


diff --git a/common/xfs b/common/xfs
index 36e02413db..0d1e0ec4bc 100644
--- a/common/xfs
+++ b/common/xfs
@@ -281,10 +281,10 @@ _xfs_check()
 {
 	OPTS=" "
 	DBOPTS=" "
-	USAGE="Usage: xfs_check [-fsvV] [-l logdev] [-i ino]... [-b bno]... special"
+	USAGE="Usage: xfs_check [-fsvV] [-l logdev] [-R rtdev] [-i ino]... [-b bno]... special"
 
 	OPTIND=1
-	while getopts "b:fi:l:stvV" c; do
+	while getopts "b:fi:l:stvVR:" c; do
 		case $c in
 			s) OPTS=$OPTS"-s ";;
 			t) OPTS=$OPTS"-t ";;
@@ -296,12 +296,14 @@ _xfs_check()
 			V) $XFS_DB_PROG -p xfs_check -V
 			   return $?
 			   ;;
+			R) DBOPTS="$DBOPTS -R $OPTARG";;
 		esac
 	done
 	set -- extra $@
 	shift $OPTIND
 	case $# in
-		1) ${XFS_DB_PROG}${DBOPTS} -F -i -p xfs_check -c "check$OPTS" $1
+		1) echo "${XFS_DB_PROG}${DBOPTS} -F -i -p xfs_check -c check$OPTS $1" >> /dev/ttyprintk
+		   ${XFS_DB_PROG}${DBOPTS} -F -i -p xfs_check -c "check$OPTS" $1
 		   status=$?
 		   ;;
 		2) echo $USAGE 1>&1
@@ -339,6 +341,11 @@ _scratch_xfs_db_options()
 	SCRATCH_OPTIONS=""
 	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 		SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
+	if [ "$USE_EXTERNAL" = yes ] && [ ! -z "$SCRATCH_RTDEV" ]; then
+		$XFS_DB_PROG --help 2>&1 | grep -q -- '-R rtdev' || \
+			_notrun 'xfs_db does not support rt devices'
+		SCRATCH_OPTIONS="$SCRATCH_OPTIONS -R$SCRATCH_RTDEV"
+	fi
 	echo $SCRATCH_OPTIONS $* $SCRATCH_DEV
 }
 
@@ -403,6 +410,11 @@ _scratch_xfs_check()
 		SCRATCH_OPTIONS="-l $SCRATCH_LOGDEV"
 	[ "$LARGE_SCRATCH_DEV" = yes ] && \
 		SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
+	if [ "$USE_EXTERNAL" = yes ] && [ ! -z "$SCRATCH_RTDEV" ]; then
+		$XFS_DB_PROG --help 2>&1 | grep -q -- '-R rtdev' || \
+			_notrun 'xfs_db does not support rt devices'
+		SCRATCH_OPTIONS="$SCRATCH_OPTIONS -R$SCRATCH_RTDEV"
+	fi
 	_xfs_check $SCRATCH_OPTIONS $* $SCRATCH_DEV
 }
 

[PATCH 09/12] xfs/122: udpate test to pick up rtword/suminfo ondisk unions

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Update this test to check that the ondisk unions for rt bitmap word and
rt summary counts are always the correct size.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122     |    2 +-
 tests/xfs/122.out |    2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/122 b/tests/xfs/122
index e616f1987d..fe6c1e36e9 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -187,7 +187,7 @@ echo 'int main(int argc, char *argv[]) {' >>$cprog
 #
 cat /usr/include/xfs/xfs*.h | indent |\
 _attribute_filter |\
-grep -E '(} *xfs_.*_t|^struct xfs_[a-z0-9_]*$)' |\
+grep -E '(} *xfs_.*_t|^(union|struct) xfs_[a-z0-9_]*$)' |\
 grep -E -v -f $tmp.ignore |\
 sed -e 's/^.*}[[:space:]]*//g' -e 's/;.*$//g' -e 's/_t, /_t\n/g' |\
 sort | uniq |\
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 336618cf7a..1379c7b3b5 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -128,6 +128,8 @@ sizeof(struct xfs_swap_extent) = 64
 sizeof(struct xfs_sxd_log_format) = 16
 sizeof(struct xfs_sxi_log_format) = 80
 sizeof(struct xfs_unmount_log_format) = 8
+sizeof(union xfs_rtword_ondisk) = 4
+sizeof(union xfs_suminfo_ondisk) = 4
 sizeof(xfs_agf_t) = 224
 sizeof(xfs_agfl_t) = 36
 sizeof(xfs_agi_t) = 344

[PATCH 12/12] common/fuzzy: adapt the scrub stress tests to support rtgroups

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Adapt the scrub stress testing framework to support checking realtime
group metadata.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/fuzzy  |   27 ++++++++++++++++++++++-----
 common/xfs    |    9 +++++++++
 tests/xfs/800 |    2 +-
 tests/xfs/840 |    2 +-
 tests/xfs/841 |    2 +-
 5 files changed, 34 insertions(+), 8 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index 7f96384402..baba05de0a 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -817,8 +817,10 @@ __stress_one_scrub_loop() {
 	local scrub_tgt="$3"
 	local scrub_startat="$4"
 	local start_agno="$5"
-	shift; shift; shift; shift; shift
+	local start_rgno="$6"
+	shift; shift; shift; shift; shift; shift
 	local agcount="$(_xfs_mount_agcount $SCRATCH_MNT)"
+	local rgcount="$(_xfs_mount_rgcount $SCRATCH_MNT)"
 
 	local xfs_io_args=()
 	for arg in "$@"; do
@@ -831,6 +833,12 @@ __stress_one_scrub_loop() {
 				local ag_arg="$(echo "$arg" | sed -e "s|%agno%|$agno|g")"
 				xfs_io_args+=('-c' "$ag_arg")
 			done
+		elif echo "$arg" | grep -q -w '%rgno%'; then
+			# Substitute the rtgroup number
+			for ((rgno = start_rgno; rgno < rgcount; rgno++)); do
+				local rg_arg="$(echo "$arg" | sed -e "s|%rgno%|$rgno|g")"
+				xfs_io_args+=('-c' "$rg_arg")
+			done
 		else
 			xfs_io_args+=('-c' "$arg")
 		fi
@@ -1201,7 +1209,9 @@ _scratch_xfs_stress_scrub_cleanup() {
 __stress_scrub_check_commands() {
 	local scrub_tgt="$1"
 	local start_agno="$2"
-	shift; shift
+	local start_rgno="$3"
+	shift; shift; shift
+	local rgcount="$(_xfs_mount_rgcount $SCRATCH_MNT)"
 
 	local cooked_tgt="$scrub_tgt"
 	case "$scrub_tgt" in
@@ -1231,6 +1241,10 @@ __stress_scrub_check_commands() {
 			cooked_arg="$(echo "$cooked_arg" | sed -e 's/^repair/repair -R/g')"
 		fi
 		cooked_arg="$(echo "$cooked_arg" | sed -e "s/%agno%/$start_agno/g")"
+		if echo "$cooked_arg" | grep -q -w '%rgno%'; then
+			test "$rgcount" -eq 0 && continue
+			cooked_arg="$(echo "$cooked_arg" | sed -e "s/%rgno%/$start_rgno/g")"
+		fi
 		testio=`$XFS_IO_PROG -x -c "$cooked_arg" "$cooked_tgt" 2>&1`
 		echo $testio | grep -q "Unknown type" && \
 			_notrun "xfs_io scrub subcommand support is missing"
@@ -1256,6 +1270,7 @@ __stress_scrub_check_commands() {
 #	in a separate loop.  If zero -i options are specified, do not run.
 #	Callers must check each of these commands (via _require_xfs_io_command)
 #	before calling here.
+# -R	For %rgno% substitution, start with this rtgroup instead of rtgroup 0.
 # -r	Run fsstress for this amount of time, then remount the fs ro or rw.
 #	The default is to run fsstress continuously with no remount, unless
 #	XFS_SCRUB_STRESS_REMOUNT_PERIOD is set.
@@ -1301,6 +1316,7 @@ _scratch_xfs_stress_scrub() {
 	local remount_period="${XFS_SCRUB_STRESS_REMOUNT_PERIOD}"
 	local stress_tgt="${XFS_SCRUB_STRESS_TARGET:-default}"
 	local start_agno=0
+	local start_rgno=0
 
 	__SCRUB_STRESS_FREEZE_PID=""
 	__SCRUB_STRESS_REMOUNT_LOOP=""
@@ -1308,12 +1324,13 @@ _scratch_xfs_stress_scrub() {
 	touch "$runningfile"
 
 	OPTIND=1
-	while getopts "a:fi:r:s:S:t:w:x:X:" c; do
+	while getopts "a:fi:r:R:s:S:t:w:x:X:" c; do
 		case "$c" in
 			a) start_agno="$OPTARG";;
 			f) freeze=yes;;
 			i) io_args+=("$OPTARG");;
 			r) remount_period="$OPTARG";;
+			R) start_rgno="$OPTARG";;
 			s) one_scrub_args+=("$OPTARG");;
 			S) xfs_scrub_args+=("$OPTARG");;
 			t) scrub_tgt="$OPTARG";;
@@ -1324,7 +1341,7 @@ _scratch_xfs_stress_scrub() {
 		esac
 	done
 
-	__stress_scrub_check_commands "$scrub_tgt" "$start_agno" \
+	__stress_scrub_check_commands "$scrub_tgt" "$start_agno" "$start_rgno" \
 			"${one_scrub_args[@]}"
 
 	if ! command -v "__stress_scrub_${exerciser}_loop" &>/dev/null; then
@@ -1372,7 +1389,7 @@ _scratch_xfs_stress_scrub() {
 
 	if [ "${#one_scrub_args[@]}" -gt 0 ]; then
 		__stress_one_scrub_loop "$end" "$runningfile" "$scrub_tgt" \
-				"$scrub_startat" "$start_agno" \
+				"$scrub_startat" "$start_agno" "$start_rgno" \
 				"${one_scrub_args[@]}" &
 	fi
 
diff --git a/common/xfs b/common/xfs
index a37284068f..f451dfb8ae 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1524,6 +1524,15 @@ _xfs_mount_agcount()
 	$XFS_INFO_PROG "$1" | sed -n "s/^.*agcount=\([[:digit:]]*\).*/\1/p"
 }
 
+# Find rtgroup count of mounted filesystem
+_xfs_mount_rgcount()
+{
+	local rtgroups="$($XFS_INFO_PROG "$1" | grep rgcount= | sed -e 's/^.*rgcount=\([0-9]*\).*$/\1/g')"
+
+	test -z "$rtgroups" && rtgroups=0
+	echo "$rtgroups"
+}
+
 # Wipe the superblock of each XFS AGs
 _try_wipe_scratch_xfs()
 {
diff --git a/tests/xfs/800 b/tests/xfs/800
index cbcfb5f5a6..a2542be5ec 100755
--- a/tests/xfs/800
+++ b/tests/xfs/800
@@ -32,7 +32,7 @@ _require_xfs_stress_scrub
 _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 _require_xfs_has_feature "$SCRATCH_MNT" realtime
-_scratch_xfs_stress_scrub -s "scrub rtbitmap"
+_scratch_xfs_stress_scrub -s "scrub rtbitmap"  -s "scrub rgbitmap %rgno%"
 
 # success, all done
 echo Silence is golden
diff --git a/tests/xfs/840 b/tests/xfs/840
index fff41c5b8a..b9ed0a55b3 100755
--- a/tests/xfs/840
+++ b/tests/xfs/840
@@ -39,7 +39,7 @@ alloc_unit=$(_get_file_block_size $SCRATCH_MNT)
 scratchfile=$SCRATCH_MNT/file
 touch $scratchfile
 $XFS_IO_PROG -x -c 'inject force_repair' $SCRATCH_MNT
-__stress_scrub_check_commands "$scratchfile" "" 'repair bmapbtd'
+__stress_scrub_check_commands "$scratchfile" "" "" 'repair bmapbtd'
 
 # Compute the number of extent records needed to guarantee btree format,
 # assuming 16 bytes for each ondisk extent record
diff --git a/tests/xfs/841 b/tests/xfs/841
index f743454971..5831961f5f 100755
--- a/tests/xfs/841
+++ b/tests/xfs/841
@@ -39,7 +39,7 @@ scratchfile=$SCRATCH_MNT/file
 mkdir $scratchdir
 touch $scratchfile
 $XFS_IO_PROG -x -c 'inject force_repair' $SCRATCH_MNT
-__stress_scrub_check_commands "$scratchdir" "" 'repair directory'
+__stress_scrub_check_commands "$scratchdir" "" "" 'repair directory'
 
 # Create a 2-dirblock directory
 total_size=$((alloc_unit * 2))

[PATCHSET v1.0 00/13] fstests: fixes for realtime rmap

[Darrick J. Wong]

Hi all,

Fix a few regressions in fstests when rmap is enabled on the realtime
volume.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-rmap

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-rmap

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-rmap

xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=realtime-rmap
---
 common/populate    |   36 +++++++++++++++++++++++++++++++++++-
 common/xfs         |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/104      |    1 +
 tests/xfs/122.out  |    5 ++---
 tests/xfs/1528     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1528.out |    4 ++++
 tests/xfs/1529     |   40 ++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1529.out |    4 ++++
 tests/xfs/272      |    2 +-
 tests/xfs/276      |    2 +-
 tests/xfs/277      |    2 +-
 tests/xfs/291      |    3 ++-
 tests/xfs/332      |    6 +-----
 tests/xfs/332.out  |    2 --
 tests/xfs/333      |   45 ---------------------------------------------
 tests/xfs/333.out  |    6 ------
 tests/xfs/337      |    2 +-
 tests/xfs/338      |   21 ++++++++++++++++-----
 tests/xfs/339      |    5 +++--
 tests/xfs/340      |   15 ++++++++++-----
 tests/xfs/341      |   12 ++++--------
 tests/xfs/341.out  |    1 -
 tests/xfs/342      |    4 ++--
 tests/xfs/343      |    2 ++
 tests/xfs/406      |    6 ++++--
 tests/xfs/407      |    6 ++++--
 tests/xfs/408      |    7 +++++--
 tests/xfs/409      |    7 +++++--
 tests/xfs/443      |    9 +++++----
 tests/xfs/481      |    6 ++++--
 tests/xfs/482      |    7 +++++--
 tests/xfs/769      |   29 ++++++++++++++++++++++++++++-
 tests/xfs/781      |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/781.out  |    2 ++
 tests/xfs/817      |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/817.out  |    2 ++
 tests/xfs/821      |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/821.out  |    2 ++
 38 files changed, 414 insertions(+), 107 deletions(-)
 create mode 100755 tests/xfs/1528
 create mode 100644 tests/xfs/1528.out
 create mode 100755 tests/xfs/1529
 create mode 100644 tests/xfs/1529.out
 delete mode 100755 tests/xfs/333
 delete mode 100644 tests/xfs/333.out
 create mode 100755 tests/xfs/781
 create mode 100644 tests/xfs/781.out
 create mode 100755 tests/xfs/817
 create mode 100644 tests/xfs/817.out
 create mode 100755 tests/xfs/821
 create mode 100644 tests/xfs/821.out

[PATCH 03/13] xfs: race fsstress with realtime rmap btree scrub and repair

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Race checking and rebuilding realtime rmap btrees with fsstress.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/781     |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/781.out |    2 ++
 tests/xfs/817     |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/817.out |    2 ++
 tests/xfs/821     |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/821.out |    2 ++
 6 files changed, 132 insertions(+)
 create mode 100755 tests/xfs/781
 create mode 100644 tests/xfs/781.out
 create mode 100755 tests/xfs/817
 create mode 100644 tests/xfs/817.out
 create mode 100755 tests/xfs/821
 create mode 100644 tests/xfs/821.out


diff --git a/tests/xfs/781 b/tests/xfs/781
new file mode 100755
index 0000000000..938777952f
--- /dev/null
+++ b/tests/xfs/781
@@ -0,0 +1,42 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle. Inc.  All Rights Reserved.
+#
+# FS QA Test No. 781
+#
+# Race fsstress and rtrmapbt repair for a while to see if we crash or livelock.
+#
+. ./common/preamble
+_begin_fstest online_repair dangerous_fsstress_repair
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch
+_require_xfs_stress_online_repair
+
+_scratch_mkfs > "$seqres.full" 2>&1
+_scratch_mount
+_require_xfs_has_feature "$SCRATCH_MNT" realtime
+_require_xfs_has_feature "$SCRATCH_MNT" rmapbt
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+_scratch_xfs_stress_online_repair -s "repair rtrmapbt %rgno%"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/781.out b/tests/xfs/781.out
new file mode 100644
index 0000000000..e7f74cf644
--- /dev/null
+++ b/tests/xfs/781.out
@@ -0,0 +1,2 @@
+QA output created by 781
+Silence is golden
diff --git a/tests/xfs/817 b/tests/xfs/817
new file mode 100755
index 0000000000..88d0a18e8d
--- /dev/null
+++ b/tests/xfs/817
@@ -0,0 +1,42 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle. Inc.  All Rights Reserved.
+#
+# FS QA Test No. 817
+#
+# Race fsstress and rtrmapbt scrub for a while to see if we crash or livelock.
+#
+. ./common/preamble
+_begin_fstest scrub dangerous_fsstress_scrub
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch
+_require_xfs_stress_scrub
+
+_scratch_mkfs > "$seqres.full" 2>&1
+_scratch_mount
+_require_xfs_has_feature "$SCRATCH_MNT" realtime
+_require_xfs_has_feature "$SCRATCH_MNT" rmapbt
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+_scratch_xfs_stress_scrub -s "scrub rtrmapbt %rgno%"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/817.out b/tests/xfs/817.out
new file mode 100644
index 0000000000..86920a4fc6
--- /dev/null
+++ b/tests/xfs/817.out
@@ -0,0 +1,2 @@
+QA output created by 817
+Silence is golden
diff --git a/tests/xfs/821 b/tests/xfs/821
new file mode 100755
index 0000000000..45b999e3b5
--- /dev/null
+++ b/tests/xfs/821
@@ -0,0 +1,42 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle. Inc.  All Rights Reserved.
+#
+# FS QA Test No. 821
+#
+# Race fsstress and realtime bitmap repair for a while to see if we crash or
+# livelock.
+#
+. ./common/preamble
+_begin_fstest online_repair dangerous_fsstress_repair
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch
+_require_xfs_stress_online_repair
+
+_scratch_mkfs > "$seqres.full" 2>&1
+_scratch_mount
+_require_xfs_has_feature "$SCRATCH_MNT" realtime
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+_scratch_xfs_stress_online_repair -s "repair rtbitmap" -s "repair rgbitmap %rgno%"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/821.out b/tests/xfs/821.out
new file mode 100644
index 0000000000..17994b8627
--- /dev/null
+++ b/tests/xfs/821.out
@@ -0,0 +1,2 @@
+QA output created by 821
+Silence is golden

[PATCH 04/13] xfs/769: add rtrmapbt upgrade to test matrix

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add realtime reverse mapping btrees to the features that this test will
try to upgrade.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/769 |   29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/769 b/tests/xfs/769
index 624dd2a338..ccc3ea10bc 100755
--- a/tests/xfs/769
+++ b/tests/xfs/769
@@ -35,11 +35,36 @@ rt_configured()
 	test "$USE_EXTERNAL" = "yes" && test -n "$SCRATCH_RTDEV"
 }
 
+need_rtgroups()
+{
+	local feat="$1"
+
+	# if realtime isn't configured, we don't need rt groups
+	rt_configured || return 1
+
+	# rt rmap btrees require rt groups but rt groups cannot be added to
+	# an existing filesystem, so we must force it on at mkfs time
+	test "${FEATURE_STATE["rmapbt"]}" -eq 1 && return 0
+	test "$feat" = "rmapbt" && return 0
+
+	return 1
+}
+
 # Compute the MKFS_OPTIONS string for a particular feature upgrade test
 compute_mkfs_options()
 {
+	local feat="$1"
 	local m_opts=""
 	local caller_options="$MKFS_OPTIONS"
+	local rtgroups
+
+	need_rtgroups "$feat" && rtgroups=1
+	if echo "$caller_options" | grep -q 'rtgroups='; then
+		test -z "$rtgroups" && rtgroups=0
+		caller_options="$(echo "$caller_options" | sed -e 's/rtgroups=*[0-9]*/rtgroups='$rtgroups'/g')"
+	elif [ -n "$rtgroups" ]; then
+		caller_options="$caller_options -r rtgroups=$rtgroups"
+	fi
 
 	for feat in "${FEATURES[@]}"; do
 		local feat_state="${FEATURE_STATE["${feat}"]}"
@@ -171,10 +196,12 @@ function post_exercise()
 # upgrade don't spread failure to the rest of the tests.
 FEATURES=()
 if rt_configured; then
+	# rmap wasn't added to rt devices until after metadir
 	check_repair_upgrade finobt && FEATURES+=("finobt")
 	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
 	check_repair_upgrade bigtime && FEATURES+=("bigtime")
 	check_repair_upgrade metadir && FEATURES+=("metadir")
+	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
 else
 	check_repair_upgrade finobt && FEATURES+=("finobt")
 	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
@@ -197,7 +224,7 @@ for feat in "${FEATURES[@]}"; do
 
 	upgrade_start_message "$feat" | tee -a $seqres.full /dev/ttyprintk > /dev/null
 
-	opts="$(compute_mkfs_options)"
+	opts="$(compute_mkfs_options "$feat")"
 	echo "mkfs.xfs $opts" >> $seqres.full
 
 	# Format filesystem

[PATCH 05/13] xfs/122: update for rtgroups-based realtime rmap btrees

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we've redesigned realtime rmap to require that the rt section
be sharded into allocation groups of no more than 2^31 blocks, we've
reduced the size of the ondisk structures and therefore need to update
this test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122.out |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 53eff0027e..e0801f9660 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -116,8 +116,8 @@ sizeof(struct xfs_rmap_key) = 20
 sizeof(struct xfs_rmap_rec) = 24
 sizeof(struct xfs_rtbuf_blkinfo) = 48
 sizeof(struct xfs_rtgroup_geometry) = 128
-sizeof(struct xfs_rtrmap_key) = 24
-sizeof(struct xfs_rtrmap_rec) = 32
+sizeof(struct xfs_rtrmap_key) = 20
+sizeof(struct xfs_rtrmap_rec) = 24
 sizeof(struct xfs_rtrmap_root) = 4
 sizeof(struct xfs_rtsb) = 104
 sizeof(struct xfs_rud_log_format) = 16

[PATCH 07/13] xfs/341: update test for rtgroup-based rmap

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we're sharding the realtime volume into multiple allocation
groups, update this test to reflect the new reality.  The realtime rmap
btree record and key sizes have shrunk, and we can't guarantee that a
quick file write actually hits the same rt group as the one we fuzzed,
so eliminate the file write test since we're really only curious if
xfs_repair will fix the problem.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/341     |   10 +++-------
 tests/xfs/341.out |    1 -
 2 files changed, 3 insertions(+), 8 deletions(-)


diff --git a/tests/xfs/341 b/tests/xfs/341
index 8861e751a9..561054f0bd 100755
--- a/tests/xfs/341
+++ b/tests/xfs/341
@@ -32,10 +32,10 @@ blksz="$(_get_block_size $SCRATCH_MNT)"
 rtextsz_blks=$((rtextsz / blksz))
 
 # inode core size is at least 176 bytes; btree header is 56 bytes;
-# rtrmap record is 32 bytes; and rtrmap key/pointer are 56 bytes.
+# rtrmap record is 24 bytes; and rtrmap key/pointer are 48 bytes.
 i_core_size="$(_xfs_get_inode_core_bytes $SCRATCH_MNT)"
-i_ptrs=$(( (isize - i_core_size) / 56 ))
-bt_recs=$(( (blksz - 56) / 32 ))
+i_ptrs=$(( (isize - i_core_size) / 48 ))
+bt_recs=$(( (blksz - 56) / 24 ))
 
 blocks=$((i_ptrs * bt_recs + 1))
 len=$((blocks * rtextsz))
@@ -57,10 +57,6 @@ _scratch_xfs_db -x -c 'path -m /realtime/0.rmap' \
 	-c "write u3.rtrmapbt.ptrs[1] $fsbno" -c 'p' >> $seqres.full
 _scratch_mount
 
-echo "Try to create more files"
-$XFS_IO_PROG -f -R -c "pwrite -S 0x68 0 9999" $SCRATCH_MNT/f5 >> $seqres.full 2>&1
-test -e $SCRATCH_MNT/f5 && echo "should not have been able to write f5"
-
 echo "Repair fs"
 _scratch_unmount 2>&1 | _filter_scratch
 _repair_scratch_fs >> $seqres.full 2>&1
diff --git a/tests/xfs/341.out b/tests/xfs/341.out
index 75a5bc6c61..580d788954 100644
--- a/tests/xfs/341.out
+++ b/tests/xfs/341.out
@@ -2,6 +2,5 @@ QA output created by 341
 Format and mount
 Create some files
 Corrupt fs
-Try to create more files
 Repair fs
 Try to create more files (again)

[PATCH 06/13] xfs: fix various problems with fsmap detecting the data device

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Various tests of realtime rmap functionality assumed that the data
device could be picked out from the GETFSMAP output by looking for
static fs metadata.  This is no longer true, since rtgroups have a
static superblock header at the start, so update these tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/272 |    2 +-
 tests/xfs/276 |    2 +-
 tests/xfs/277 |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/tests/xfs/272 b/tests/xfs/272
index 7ed8b95122..42b4a2edb5 100755
--- a/tests/xfs/272
+++ b/tests/xfs/272
@@ -57,7 +57,7 @@ cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total c
 done
 
 echo "Check device field of FS metadata and regular file"
-data_dev=$(grep 'static fs metadata' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
+data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
 rt_dev=$(grep "${ino}[[:space:]]*[0-9]*\.\.[0-9]*" $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
 test "${data_dev}" = "${rt_dev}" || echo "data ${data_dev} realtime ${rt_dev}?"
 
diff --git a/tests/xfs/276 b/tests/xfs/276
index 8cc486752a..a05ca1961d 100755
--- a/tests/xfs/276
+++ b/tests/xfs/276
@@ -61,7 +61,7 @@ cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total c
 done
 
 echo "Check device field of FS metadata and realtime file"
-data_dev=$(grep 'static fs metadata' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
+data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
 rt_dev=$(grep "${ino}[[:space:]]*[0-9]*\.\.[0-9]*[[:space:]]*[0-9]*$" $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
 test "${data_dev}" != "${rt_dev}" || echo "data ${data_dev} realtime ${rt_dev}?"
 
diff --git a/tests/xfs/277 b/tests/xfs/277
index 03208ef233..eff54a2a50 100755
--- a/tests/xfs/277
+++ b/tests/xfs/277
@@ -38,7 +38,7 @@ $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full
 $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT | tr '[]()' '    ' > $TEST_DIR/fsmap
 
 echo "Check device field of FS metadata and journalling log"
-data_dev=$(grep 'static fs metadata' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
+data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
 journal_dev=$(grep 'journalling log' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
 test "${data_dev}" = "${journal_dev}" || echo "data ${data_dev} journal ${journal_dev}?"
 

[PATCH 01/13] xfs: fix tests that try to access the realtime rmap inode

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The realtime rmap tests were added to fstests a long time ago.  Since
they were added, we decided to create a metadata file directory
structure instead of adding more fields to the superblock.  Therefore,
fix all the tests that try to access these paths.

While we're at it, fix xfs/409 to run the *online* scrub program like
it's supposed to.  xfs/408 is the fuzzer for xfs_repair testing.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs        |   18 ++++++++++++++++++
 tests/xfs/122.out |    1 -
 tests/xfs/333     |   45 ---------------------------------------------
 tests/xfs/333.out |    6 ------
 tests/xfs/337     |    2 +-
 tests/xfs/338     |   21 ++++++++++++++++-----
 tests/xfs/339     |    5 +++--
 tests/xfs/340     |   15 ++++++++++-----
 tests/xfs/341     |    2 +-
 tests/xfs/342     |    4 ++--
 10 files changed, 51 insertions(+), 68 deletions(-)
 delete mode 100755 tests/xfs/333
 delete mode 100644 tests/xfs/333.out


diff --git a/common/xfs b/common/xfs
index f451dfb8ae..aea2b678c8 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1922,3 +1922,21 @@ _require_xfs_scratch_metadir()
 		_scratch_unmount
 	fi
 }
+
+# Resolve a metadata directory tree path and return the inode number.
+_scratch_metadir_lookup() {
+	local res="$(_scratch_xfs_db -c "ls -i -m $1")"
+	test "${PIPESTATUS[0]}" -eq 0 && echo "$res"
+}
+
+# Figure out which directory entry we have to change to update the rtrmap
+# inode pointer.  Assumes the /realtime directory is a short format dir.
+_scratch_find_rt_metadir_entry() {
+	local sfkey="$(_scratch_xfs_db -c 'path -m /realtime' -c print | \
+		grep "\"$1\"" | \
+		sed -e 's/.name.*$//g' -e 's/\[/\\[/g' -e 's/\]/\\]/g' )"
+	test -n "$sfkey" || return 1
+	_scratch_xfs_db -c 'path -m /realtime' -c print | \
+		grep "${sfkey}.inumber" | awk '{print $1}'
+	return 0
+}
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 1379c7b3b5..53eff0027e 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -48,7 +48,6 @@ offsetof(xfs_sb_t, sb_rgblklog) = 280
 offsetof(xfs_sb_t, sb_rgblocks) = 272
 offsetof(xfs_sb_t, sb_rgcount) = 276
 offsetof(xfs_sb_t, sb_rootino) = 56
-offsetof(xfs_sb_t, sb_rrmapino) = 264
 offsetof(xfs_sb_t, sb_rsumino) = 72
 offsetof(xfs_sb_t, sb_sectlog) = 121
 offsetof(xfs_sb_t, sb_sectsize) = 102
diff --git a/tests/xfs/333 b/tests/xfs/333
deleted file mode 100755
index 728c518402..0000000000
--- a/tests/xfs/333
+++ /dev/null
@@ -1,45 +0,0 @@
-#! /bin/bash
-# SPDX-License-Identifier: GPL-2.0
-# Copyright (c) 2016, Oracle and/or its affiliates.  All Rights Reserved.
-#
-# FS QA Test No. 333
-#
-# Set rrmapino to another inode on an non-rt rmap fs and see if repair fixes it.
-#
-. ./common/preamble
-_begin_fstest auto quick rmap realtime
-
-# Import common functions.
-. ./common/filter
-
-# real QA test starts here
-_supported_fs xfs
-_require_xfs_scratch_rmapbt
-_disable_dmesg_check
-
-rm -f "$seqres.full"
-
-unset SCRATCH_RTDEV
-
-echo "Format and mount"
-_scratch_mkfs > "$seqres.full" 2>&1
-rrmapino="$(_scratch_xfs_db -c 'sb 0' -c 'p rrmapino' 2>&1)"
-test "${rrmapino}" = "field rrmapino not found" && _notrun "realtime rmapbt not supported"
-_scratch_mount
-
-echo "Create some files"
-$XFS_IO_PROG -f -c "pwrite -S 0x68 0 9999" $SCRATCH_MNT/f1 >> $seqres.full
-$XFS_IO_PROG -f -c "pwrite -S 0x68 0 9999" $SCRATCH_MNT/f2 >> $seqres.full
-echo garbage > $SCRATCH_MNT/f3
-ino=$(stat -c '%i' $SCRATCH_MNT/f3)
-_scratch_unmount
-
-echo "Corrupt fs"
-_scratch_xfs_db -x -c 'sb 0' -c "write rrmapino $ino" >> $seqres.full
-_try_scratch_mount 2>&1 | _filter_error_mount
-
-echo "Test done, mount should have failed"
-
-# success, all done
-status=0
-exit
diff --git a/tests/xfs/333.out b/tests/xfs/333.out
deleted file mode 100644
index b3c698750f..0000000000
--- a/tests/xfs/333.out
+++ /dev/null
@@ -1,6 +0,0 @@
-QA output created by 333
-Format and mount
-Create some files
-Corrupt fs
-mount: Structure needs cleaning
-Test done, mount should have failed
diff --git a/tests/xfs/337 b/tests/xfs/337
index f74baae9b0..9ea8587b27 100755
--- a/tests/xfs/337
+++ b/tests/xfs/337
@@ -53,7 +53,7 @@ echo "+ check fs"
 _scratch_xfs_repair -n >> $seqres.full 2>&1 || echo "xfs_repair should not fail"
 
 echo "+ corrupt image"
-_scratch_xfs_db -x -c "sb" -c "addr rrmapino" -c "addr u3.rtrmapbt.ptrs[1]" \
+_scratch_xfs_db -x -c "path -m /realtime/0.rmap" -c "addr u3.rtrmapbt.ptrs[1]" \
 	-c "stack" -c "blocktrash -x 4096 -y 4096 -n 8 -3 -z" \
 	>> $seqres.full 2>&1
 
diff --git a/tests/xfs/338 b/tests/xfs/338
index 9f36150c7e..9d41a83ec2 100755
--- a/tests/xfs/338
+++ b/tests/xfs/338
@@ -29,13 +29,24 @@ $XFS_IO_PROG -f -R -c "pwrite -S 0x68 0 9999" $SCRATCH_MNT/f2 >> $seqres.full
 _scratch_unmount
 
 echo "Corrupt fs"
-_scratch_xfs_db -x -c 'sb 0' -c 'addr rrmapino' \
-	-c 'write core.nlinkv2 0' -c 'write core.mode 0' -c 'sb 0' \
-	-c 'write rrmapino 0' >> $seqres.full
-_try_scratch_mount >> $seqres.full 2>&1 && echo "mount should have failed"
+rtrmap_sfentry="$(_scratch_find_rt_metadir_entry 0.rmap)"
+test -n "$rtrmap_sfentry" || _fail "Could not find rtrmap metadir entry?"
+_scratch_xfs_db -x -c 'path -m /realtime/0.rmap' \
+	-c 'write core.nlinkv2 0' -c 'write core.mode 0' \
+	-c 'path -m /realtime' \
+	-c "write $rtrmap_sfentry 0" >> $seqres.full
+if _try_scratch_mount >> $seqres.full 2>&1; then
+       echo "mount should have failed"
+       _scratch_unmount
+else
+	# If the verifiers are working properly, the mount will fail because
+	# we fuzzed the metadata root directory.  This causes loud complaints
+	# to dmesg, so we want to ignore those.
+	_disable_dmesg_check
+fi
 
 echo "Repair fs"
-_scratch_unmount 2>&1 | _filter_scratch
+_scratch_unmount 2>&1 | _filter_scratch | _filter_ending_dot
 _repair_scratch_fs >> $seqres.full 2>&1
 
 echo "Try to create more files (again)"
diff --git a/tests/xfs/339 b/tests/xfs/339
index 3e0b4d97ab..24a90d0ba3 100755
--- a/tests/xfs/339
+++ b/tests/xfs/339
@@ -31,7 +31,8 @@ ln $SCRATCH_MNT/f3 $SCRATCH_MNT/f4
 _scratch_unmount
 
 echo "Corrupt fs"
-rrmapino=`_scratch_xfs_get_sb_field rrmapino`
+rrmapino=$(_scratch_metadir_lookup /realtime/0.rmap)
+test -n "$rrmapino" || _fail "Could not find rtrmap inode?"
 _scratch_xfs_set_metadata_field "u3.sfdir3.list[3].inumber.i4" $rrmapino \
 	'sb 0' 'addr rootino' >> $seqres.full
 _scratch_mount
@@ -43,7 +44,7 @@ echo "Try to create more files"
 $XFS_IO_PROG -f -R -c "pwrite -S 0x68 0 9999" $SCRATCH_MNT/f5 >> $seqres.full 2>&1
 
 echo "Repair fs"
-_scratch_unmount 2>&1 | _filter_scratch
+_scratch_unmount 2>&1 | _filter_scratch | _filter_ending_dot
 _repair_scratch_fs >> $seqres.full 2>&1
 
 echo "Try to create more files (again)"
diff --git a/tests/xfs/340 b/tests/xfs/340
index 2c0014513e..1236f6520f 100755
--- a/tests/xfs/340
+++ b/tests/xfs/340
@@ -31,16 +31,21 @@ ino=$(stat -c '%i' $SCRATCH_MNT/f3)
 _scratch_unmount
 
 echo "Corrupt fs"
-rrmapino=$(_scratch_xfs_get_sb_field rrmapino)
-_scratch_xfs_db -x -c "inode $rrmapino" \
+rtrmap_sfentry="$(_scratch_find_rt_metadir_entry 0.rmap)"
+test -n "$rtrmap_sfentry" || _fail "Could not find rtrmap metadir entry?"
+rrmapino=$(_scratch_metadir_lookup /realtime/0.rmap)
+test -n "$rrmapino" || _fail "Could not find rtrmap inode?"
+_scratch_xfs_db -x -c "path -m /realtime/0.rmap" \
 	-c 'write core.format 2' -c 'write core.size 0' \
-	-c 'write core.nblocks 0' -c 'sb 0' -c 'addr rootino' \
+	-c 'write core.nblocks 0' \
+	-c 'sb 0' -c 'addr rootino' \
 	-c "write u3.sfdir3.list[2].inumber.i4 $rrmapino" \
-	-c 'sb 0' -c "write rrmapino $ino" >> $seqres.full
+	-c 'path -m /realtime' \
+	-c "write $rtrmap_sfentry $ino" >> $seqres.full
 _try_scratch_mount >> $seqres.full 2>&1 && echo "mount should have failed"
 
 echo "Repair fs"
-_scratch_unmount 2>&1 | _filter_scratch
+_scratch_unmount 2>&1 | _filter_scratch | _filter_ending_dot
 _repair_scratch_fs >> $seqres.full 2>&1
 
 echo "Try to create more files (again)"
diff --git a/tests/xfs/341 b/tests/xfs/341
index 7d2842b579..8861e751a9 100755
--- a/tests/xfs/341
+++ b/tests/xfs/341
@@ -53,7 +53,7 @@ echo "Corrupt fs"
 fsbno=$(_scratch_xfs_db -c "inode $ino" -c 'bmap' | grep 'flag 0' | head -n 1 | \
 	sed -e 's/^.*startblock \([0-9]*\) .*$/\1/g')
 
-_scratch_xfs_db -x -c 'sb 0' -c 'addr rrmapino' \
+_scratch_xfs_db -x -c 'path -m /realtime/0.rmap' \
 	-c "write u3.rtrmapbt.ptrs[1] $fsbno" -c 'p' >> $seqres.full
 _scratch_mount
 
diff --git a/tests/xfs/342 b/tests/xfs/342
index 538c8987ef..f29bd874e9 100755
--- a/tests/xfs/342
+++ b/tests/xfs/342
@@ -47,9 +47,9 @@ ino=$(stat -c '%i' $SCRATCH_MNT/f3)
 _scratch_unmount
 
 echo "Corrupt fs"
-_scratch_xfs_db -c 'sb 0' -c 'addr rrmapino' -c 'p u3.rtrmapbt.ptrs[1]' >> $seqres.full
+_scratch_xfs_db -c 'path -m /realtime/0.rmap' -c 'p u3.rtrmapbt.ptrs[1]' >> $seqres.full
 
-fsbno=$(_scratch_xfs_db -c 'sb 0' -c 'addr rrmapino' \
+fsbno=$(_scratch_xfs_db -c 'path -m /realtime/0.rmap' \
 	-c 'p u3.rtrmapbt.ptrs[1]' | sed -e 's/^.*://g')
 _scratch_xfs_db -x -c "inode $ino" -c "write u3.bmx[0].startblock $fsbno" >> $seqres.full
 _scratch_mount

[PATCH 02/13] fuzz: for fuzzing the rtrmapbt, find the path to the rt rmap btree file

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The fs population code creates a realtime rmap btree in /some/ realtime
group with at least two levels.  This rmapbt file isn't necessarily the
one for group 0, so we need to find it programmatically.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs    |   33 +++++++++++++++++++++++++++++++++
 tests/xfs/406 |    6 ++++--
 tests/xfs/407 |    6 ++++--
 tests/xfs/408 |    7 +++++--
 tests/xfs/409 |    7 +++++--
 tests/xfs/481 |    6 ++++--
 tests/xfs/482 |    7 +++++--
 7 files changed, 60 insertions(+), 12 deletions(-)


diff --git a/common/xfs b/common/xfs
index aea2b678c8..63eff39d47 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1814,6 +1814,39 @@ _scratch_xfs_find_agbtree_height() {
 	return 1
 }
 
+# Find us the path to the inode containing a realtime btree with a specific
+# height.
+_scratch_xfs_find_rgbtree_height() {
+	local bt_type="$1"
+	local bt_height="$2"
+	local rgcount=$(_xfs_mount_rgcount $SCRATCH_DEV)
+	local path
+	local path_format
+	local bt_prefix
+
+	case "${bt_type}" in
+	"rmap")
+		path_format="/realtime/%u.rmap"
+		bt_prefix="u3.rtrmapbt"
+		;;
+	*)
+		_fail "Don't know about rt btree ${bt_type}"
+		;;
+	esac
+
+	for ((rgno = 0; rgno < rgcount; rgno++)); do
+		path="$(printf "${path_format}" "${rgno}")"
+		bt_level=$(_scratch_xfs_db -c "path -m ${path}" -c "p ${bt_prefix}.level" | awk '{print $3}')
+		# "level" is the actual level within the btree
+		if [ "${bt_level}" -eq "$((bt_height - 1))" ]; then
+			echo "${path}"
+			return 0
+		fi
+	done
+
+	return 1
+}
+
 _require_xfs_mkfs_atomicswap()
 {
 	# atomicswap can be activated on rmap or reflink filesystems.
diff --git a/tests/xfs/406 b/tests/xfs/406
index 78db18077c..8c5570886b 100755
--- a/tests/xfs/406
+++ b/tests/xfs/406
@@ -26,10 +26,12 @@ _require_scratch_xfs_fuzz_fields
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
-inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'sb 0' 'addr rrmapino')
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
 
 echo "Fuzz rtrmapbt recs"
-_scratch_xfs_fuzz_metadata '' 'offline' 'sb 0' 'addr rrmapino' "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
+_scratch_xfs_fuzz_metadata '' 'offline' "path -m $path" "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
 echo "Done fuzzing rtrmapbt recs"
 
 # success, all done
diff --git a/tests/xfs/407 b/tests/xfs/407
index 5a43775b55..2460ea336c 100755
--- a/tests/xfs/407
+++ b/tests/xfs/407
@@ -26,10 +26,12 @@ _require_scratch_xfs_fuzz_fields
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
-inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'sb 0' 'addr rrmapino')
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 1)" || \
+	_fail "could not find two-level rtrmapbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
 
 echo "Fuzz rtrmapbt recs"
-_scratch_xfs_fuzz_metadata '' 'online' 'sb 0' 'addr rrmapino' "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
+_scratch_xfs_fuzz_metadata '' 'online' "path -m $path" "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
 echo "Done fuzzing rtrmapbt recs"
 
 # success, all done
diff --git a/tests/xfs/408 b/tests/xfs/408
index 8049d6bead..3bed3824e8 100755
--- a/tests/xfs/408
+++ b/tests/xfs/408
@@ -4,7 +4,7 @@
 #
 # FS QA Test No. 408
 #
-# Populate a XFS filesystem and fuzz every rtrmapbt keyptr field.
+# Populate a XFS filesystem and fuzz every rtrmapbt key/pointer field.
 # Use xfs_repair to fix the corruption.
 #
 . ./common/preamble
@@ -26,8 +26,11 @@ _require_scratch_xfs_fuzz_fields
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+
 echo "Fuzz rtrmapbt keyptrs"
-_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'offline' 'sb 0' 'addr rrmapino' >> $seqres.full
+_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'offline' "path -m $path" >> $seqres.full
 echo "Done fuzzing rtrmapbt keyptrs"
 
 # success, all done
diff --git a/tests/xfs/409 b/tests/xfs/409
index adac95fea8..ce66175c6e 100755
--- a/tests/xfs/409
+++ b/tests/xfs/409
@@ -4,7 +4,7 @@
 #
 # FS QA Test No. 409
 #
-# Populate a XFS filesystem and fuzz every rtrmapbt keyptr field.
+# Populate a XFS filesystem and fuzz every rtrmapbt key/pointer field.
 # Use xfs_scrub to fix the corruption.
 #
 . ./common/preamble
@@ -26,8 +26,11 @@ _require_scratch_xfs_fuzz_fields
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+
 echo "Fuzz rtrmapbt keyptrs"
-_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'offline' 'sb 0' 'addr rrmapino' >> $seqres.full
+_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'online' "path -m $path" >> $seqres.full
 echo "Done fuzzing rtrmapbt keyptrs"
 
 # success, all done
diff --git a/tests/xfs/481 b/tests/xfs/481
index 48c7580ccb..d303f2c27d 100755
--- a/tests/xfs/481
+++ b/tests/xfs/481
@@ -27,10 +27,12 @@ _disable_dmesg_check
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
-inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'sb 0' 'addr rrmapino')
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
 
 echo "Fuzz rtrmapbt recs"
-_scratch_xfs_fuzz_metadata '' 'none' 'sb 0' 'addr rrmapino' "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
+_scratch_xfs_fuzz_metadata '' 'none' "path -m $path" "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
 echo "Done fuzzing rtrmapbt recs"
 
 # success, all done
diff --git a/tests/xfs/482 b/tests/xfs/482
index 0192b94cc0..32a3012154 100755
--- a/tests/xfs/482
+++ b/tests/xfs/482
@@ -4,7 +4,7 @@
 #
 # FS QA Test No. 482
 #
-# Populate a XFS filesystem and fuzz every rtrmapbt keyptr field.
+# Populate a XFS filesystem and fuzz every rtrmapbt key/pointer field.
 # Do not fix the filesystem, to test metadata verifiers.
 
 . ./common/preamble
@@ -27,8 +27,11 @@ _disable_dmesg_check
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+
 echo "Fuzz rtrmapbt keyptrs"
-_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'offline' 'sb 0' 'addr rrmapino' >> $seqres.full
+_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'offline' "path -m $path" >> $seqres.full
 echo "Done fuzzing rtrmapbt keyptrs"
 
 # success, all done

[PATCH 09/13] xfs: skip tests if formatting small filesystem fails

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

There are a few tests that try to exercise XFS functionality with an
unusually small (< 500MB) filesystem.  Formatting can fail if the test
configuration also specifies a very large realtime device because mkfs
hits ENOSPC when allocating the realtime metadata.  The test proceeds
anyway (which causes an immediate mount failure) so we might as well
skip these.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/104 |    1 +
 tests/xfs/291 |    3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/104 b/tests/xfs/104
index d16f46d8e4..c3d1d18a58 100755
--- a/tests/xfs/104
+++ b/tests/xfs/104
@@ -16,6 +16,7 @@ _create_scratch()
 {
 	echo "*** mkfs"
 	_scratch_mkfs_xfs $@ | tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs
+	test "${PIPESTATUS[0]}" -eq 0 || _notrun "formatting small scratch fs failed"
 	. $tmp.mkfs
 
 	echo "*** mount"
diff --git a/tests/xfs/291 b/tests/xfs/291
index 600dcb2eba..70e5f51cee 100755
--- a/tests/xfs/291
+++ b/tests/xfs/291
@@ -18,7 +18,8 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
 # real QA test starts here
 _require_scratch
 logblks=$(_scratch_find_xfs_min_logblocks -n size=16k -d size=133m)
-_scratch_mkfs_xfs -n size=16k -l size=${logblks}b -d size=133m >> $seqres.full 2>&1
+_scratch_mkfs_xfs -n size=16k -l size=${logblks}b -d size=133m >> $seqres.full 2>&1 || \
+	_notrun "formatting small scratch fs failed"
 _scratch_mount
 
 # First we cause very badly fragmented freespace, then

[PATCH 10/13] xfs/443: use file allocation unit, not dbsize

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

We can only punch in units of file allocation boundaries, so update this
test to use that instead of the fs blocksize.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/443 |    9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)


diff --git a/tests/xfs/443 b/tests/xfs/443
index 56828decae..ab3cda59f3 100755
--- a/tests/xfs/443
+++ b/tests/xfs/443
@@ -40,14 +40,15 @@ _scratch_mount
 
 file1=$SCRATCH_MNT/file1
 file2=$SCRATCH_MNT/file2
+file_blksz=$(_get_file_block_size $SCRATCH_MNT)
 
 # The goal is run an extent swap where one of the associated files has the
 # minimum number of extents to remain in btree format. First, create a couple
 # files with large enough extent counts (200 or so should be plenty) to ensure
 # btree format on the largest possible inode size filesystems.
-$XFS_IO_PROG -fc "falloc 0 $((400 * dbsize))" $file1
+$XFS_IO_PROG -fc "falloc 0 $((400 * file_blksz))" $file1
 $here/src/punch-alternating $file1
-$XFS_IO_PROG -fc "falloc 0 $((400 * dbsize))" $file2
+$XFS_IO_PROG -fc "falloc 0 $((400 * file_blksz))" $file2
 $here/src/punch-alternating $file2
 
 # Now run an extent swap at every possible extent count down to 0. Depending on
@@ -55,12 +56,12 @@ $here/src/punch-alternating $file2
 # btree format.
 for i in $(seq 1 2 399); do
 	# punch one extent from the tmpfile and swap
-	$XFS_IO_PROG -c "fpunch $((i * dbsize)) $dbsize" $file2
+	$XFS_IO_PROG -c "fpunch $((i * file_blksz)) $file_blksz" $file2
 	$XFS_IO_PROG -c "swapext $file2" $file1
 
 	# punch the same extent from the old fork (now in file2) to resync the
 	# extent counts and repeat
-	$XFS_IO_PROG -c "fpunch $((i * dbsize)) $dbsize" $file2
+	$XFS_IO_PROG -c "fpunch $((i * file_blksz)) $file_blksz" $file2
 done
 
 # sanity check that no extents are left over

[PATCH 12/13] populate: check that we created a realtime rmap btree of the given height

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Make sure that we actually create an rt rmap btree of the desired height
somewhere in the filesystem.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/populate |   34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)


diff --git a/common/populate b/common/populate
index 7d57cd1287..6a05177e6d 100644
--- a/common/populate
+++ b/common/populate
@@ -631,6 +631,37 @@ __populate_check_xfs_agbtree_height() {
 	return 1
 }
 
+# Check that there's at least one rt btree with multiple levels
+__populate_check_xfs_rgbtree_height() {
+	local bt_type="$1"
+	local rgcount=$(_scratch_xfs_db -c 'sb 0' -c 'p rgcount' | awk '{print $3}')
+	local path
+	local path_format
+	local bt_prefix
+
+	case "${bt_type}" in
+	"rmap")
+		path_format="/realtime/%u.rmap"
+		bt_prefix="u3.rtrmapbt"
+		;;
+	*)
+		_fail "Don't know about rt btree ${bt_type}"
+		;;
+	esac
+
+	for ((rgno = 0; rgno < rgcount; rgno++)); do
+		path="$(printf "${path_format}" "${rgno}")"
+		bt_level=$(_scratch_xfs_db -c "path -m ${path}" -c "p ${bt_prefix}.level" | awk '{print $3}')
+		# "level" is the actual level within the btree
+		if [ "${bt_level}" -gt 0 ]; then
+			return 0
+		fi
+	done
+
+	__populate_fail "Failed to create rt ${bt_type} of sufficient height!"
+	return 1
+}
+
 # Check that populate created all the types of files we wanted
 _scratch_xfs_populate_check() {
 	_scratch_mount
@@ -654,6 +685,7 @@ _scratch_xfs_populate_check() {
 	is_finobt=$(_xfs_has_feature "$SCRATCH_MNT" finobt -v)
 	is_rmapbt=$(_xfs_has_feature "$SCRATCH_MNT" rmapbt -v)
 	is_reflink=$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)
+	is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")"
 
 	blksz="$(stat -f -c '%s' "${SCRATCH_MNT}")"
 	dblksz="$(_xfs_get_dir_blocksize "$SCRATCH_MNT")"
@@ -684,6 +716,8 @@ _scratch_xfs_populate_check() {
 	test $is_finobt -ne 0 && __populate_check_xfs_agbtree_height "fino"
 	test $is_rmapbt -ne 0 && __populate_check_xfs_agbtree_height "rmap"
 	test $is_reflink -ne 0 && __populate_check_xfs_agbtree_height "refcnt"
+	test $is_rmapbt -ne 0 && test $is_rt -gt 0 && \
+		__populate_check_xfs_rgbtree_height "rmap"
 }
 
 # Check data fork format of ext4 file

[PATCH 13/13] fuzzy: create missing fuzz tests for rt rmap btrees

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Back when I first created the fuzz tests for the realtime rmap btree, I
forgot a couple of things.  Add tests to fuzz rtrmap btree leaf records,
and node keys.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/1528     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1528.out |    4 ++++
 tests/xfs/1529     |   40 ++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1529.out |    4 ++++
 tests/xfs/407      |    2 +-
 5 files changed, 90 insertions(+), 1 deletion(-)
 create mode 100755 tests/xfs/1528
 create mode 100644 tests/xfs/1528.out
 create mode 100755 tests/xfs/1529
 create mode 100644 tests/xfs/1529.out


diff --git a/tests/xfs/1528 b/tests/xfs/1528
new file mode 100755
index 0000000000..b2e1193ebd
--- /dev/null
+++ b/tests/xfs/1528
@@ -0,0 +1,41 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1528
+#
+# Populate a XFS filesystem and fuzz every rtrmapbt record field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_bothrepair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_xfs_scratch_rmapbt
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
+
+echo "Fuzz rtrmapbt recs"
+_scratch_xfs_fuzz_metadata '' 'both' "path -m $path" "addr u${inode_ver}.rtrmapbt.ptrs[1]" >> $seqres.full
+echo "Done fuzzing rtrmapbt recs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1528.out b/tests/xfs/1528.out
new file mode 100644
index 0000000000..b51b640c40
--- /dev/null
+++ b/tests/xfs/1528.out
@@ -0,0 +1,4 @@
+QA output created by 1528
+Format and populate
+Fuzz rtrmapbt recs
+Done fuzzing rtrmapbt recs
diff --git a/tests/xfs/1529 b/tests/xfs/1529
new file mode 100755
index 0000000000..91a673c049
--- /dev/null
+++ b/tests/xfs/1529
@@ -0,0 +1,40 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1529
+#
+# Populate a XFS filesystem and fuzz every rtrmapbt keyptr field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_bothrepair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_xfs_scratch_rmapbt
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
+	_fail "could not find two-level rtrmapbt"
+
+echo "Fuzz rtrmapbt keyptrs"
+_scratch_xfs_fuzz_metadata '(rtrmapbt)' 'offline' "path -m $path" >> $seqres.full
+echo "Done fuzzing rtrmapbt keyptrs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1529.out b/tests/xfs/1529.out
new file mode 100644
index 0000000000..808fcc957f
--- /dev/null
+++ b/tests/xfs/1529.out
@@ -0,0 +1,4 @@
+QA output created by 1529
+Format and populate
+Fuzz rtrmapbt keyptrs
+Done fuzzing rtrmapbt keyptrs
diff --git a/tests/xfs/407 b/tests/xfs/407
index 2460ea336c..bd439105e2 100755
--- a/tests/xfs/407
+++ b/tests/xfs/407
@@ -26,7 +26,7 @@ _require_scratch_xfs_fuzz_fields
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
-path="$(_scratch_xfs_find_rgbtree_height 'rmap' 1)" || \
+path="$(_scratch_xfs_find_rgbtree_height 'rmap' 2)" || \
 	_fail "could not find two-level rtrmapbt"
 inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
 

[PATCH 11/13] populate: adjust rtrmap calculations for rtgroups

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we've sharded the realtime volume and created per-group rmap
btrees, we need to adjust downward the size of rtrmapbt records since
the block counts are now 32-bit instead of 64-bit.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/populate |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/common/populate b/common/populate
index c0bbbc3f3b..7d57cd1287 100644
--- a/common/populate
+++ b/common/populate
@@ -379,7 +379,7 @@ _scratch_xfs_populate() {
 	is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")"
 	if [ $is_rmapbt -gt 0 ] && [ $is_rt -gt 0 ]; then
 		echo "+ rtrmapbt btree"
-		nr="$((blksz * 2 / 32))"
+		nr="$((blksz * 2 / 24))"
 		$XFS_IO_PROG -R -f -c 'truncate 0' "${SCRATCH_MNT}/RTRMAPBT"
 		__populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RTRMAPBT"
 	fi

[PATCH 08/13] xfs/3{43,32}: adapt tests for rt extent size greater than 1

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Both of these tests for the realtime volume can fail when the rt extent
size is larger than a single block.

332 is a read-write functionality test that encodes md5sum in the
output, so we need to skip it if $blksz isn't congruent with the extent
size, because the fcollapse call will fail.

343 is a test of the rmap btree, so the fix here is simpler -- make
$blksz the file allocation unit, and get rid of the md5sum in the
golden output.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/332     |    6 +-----
 tests/xfs/332.out |    2 --
 tests/xfs/343     |    2 ++
 3 files changed, 3 insertions(+), 7 deletions(-)


diff --git a/tests/xfs/332 b/tests/xfs/332
index a2d37ee905..c1ac87adcb 100755
--- a/tests/xfs/332
+++ b/tests/xfs/332
@@ -28,7 +28,7 @@ rm -f "$seqres.full"
 echo "Format and mount"
 _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
-blksz=65536
+blksz=$(_get_file_block_size $SCRATCH_MNT) # 65536
 blocks=16
 len=$((blocks * blksz))
 
@@ -45,10 +45,6 @@ $XFS_IO_PROG -c "fpunch $blksz $blksz" \
 	-c "fcollapse $((9 * blksz)) $blksz" \
 	-c "finsert $((10 * blksz)) $blksz" $SCRATCH_MNT/f1 >> $seqres.full
 
-echo "Check file"
-md5sum $SCRATCH_MNT/f1 | _filter_scratch
-od -tx1 -Ad -c $SCRATCH_MNT/f1 >> $seqres.full
-
 echo "Unmount"
 _scratch_unmount
 
diff --git a/tests/xfs/332.out b/tests/xfs/332.out
index 9beff7cc37..3a7ca95b40 100644
--- a/tests/xfs/332.out
+++ b/tests/xfs/332.out
@@ -2,8 +2,6 @@ QA output created by 332
 Format and mount
 Create some files
 Manipulate file
-Check file
-e45c5707fcf6817e914ffb6ce37a0ac7  SCRATCH_MNT/f1
 Unmount
 Try a regular fsmap
 Try a bad fsmap
diff --git a/tests/xfs/343 b/tests/xfs/343
index bffcc7d9ac..fe461847ed 100755
--- a/tests/xfs/343
+++ b/tests/xfs/343
@@ -31,6 +31,8 @@ blksz=65536
 blocks=16
 len=$((blocks * blksz))
 
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
+
 echo "Create some files"
 $XFS_IO_PROG -f -R -c "falloc 0 $len" -c "pwrite -S 0x68 -b 1048576 0 $len" $SCRATCH_MNT/f1 >> $seqres.full
 

[PATCHSET v1.0 00/10] fstests: reflink on the realtime device

[Darrick J. Wong]

Hi all,

This patchset enables use of the file data block sharing feature (i.e.
reflink) on the realtime device.  It follows the same basic sequence as
the realtime rmap series -- first a few cleanups; then widening of the
API parameters; and introduction of the new btree format and inode fork
format.  Next comes enabling CoW and remapping for the rt device; new
scrub, repair, and health reporting code; and at the end we implement
some code to lengthen write requests so that rt extents are always CoWed
fully.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-reflink

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-reflink

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-reflink

xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=realtime-reflink
---
 common/populate       |   26 ++++++++++++++++++---
 common/xfs            |   15 ++++++++++++
 tests/generic/331     |   12 ++++++++-
 tests/generic/331.out |    2 +-
 tests/xfs/122.out     |    3 ++
 tests/xfs/131         |   48 --------------------------------------
 tests/xfs/131.out     |    5 ----
 tests/xfs/1538        |   41 ++++++++++++++++++++++++++++++++
 tests/xfs/1538.out    |    4 +++
 tests/xfs/1539        |   41 ++++++++++++++++++++++++++++++++
 tests/xfs/1539.out    |    4 +++
 tests/xfs/1540        |   41 ++++++++++++++++++++++++++++++++
 tests/xfs/1540.out    |    4 +++
 tests/xfs/1541        |   42 +++++++++++++++++++++++++++++++++
 tests/xfs/1541.out    |    4 +++
 tests/xfs/1542        |   41 ++++++++++++++++++++++++++++++++
 tests/xfs/1542.out    |    4 +++
 tests/xfs/1543        |   40 ++++++++++++++++++++++++++++++++
 tests/xfs/1543.out    |    4 +++
 tests/xfs/1544        |   40 ++++++++++++++++++++++++++++++++
 tests/xfs/1544.out    |    4 +++
 tests/xfs/1545        |   41 ++++++++++++++++++++++++++++++++
 tests/xfs/1545.out    |    4 +++
 tests/xfs/240         |   13 +++++++++-
 tests/xfs/240.out     |    2 +-
 tests/xfs/243         |    5 ++++
 tests/xfs/272         |   40 +++++++++++++++++++++-----------
 tests/xfs/274         |   62 ++++++++++++++++++++++++++++++++++---------------
 tests/xfs/769         |    3 ++
 tests/xfs/818         |   43 ++++++++++++++++++++++++++++++++++
 tests/xfs/818.out     |    2 ++
 tests/xfs/819         |   43 ++++++++++++++++++++++++++++++++++
 tests/xfs/819.out     |    2 ++
 33 files changed, 590 insertions(+), 95 deletions(-)
 delete mode 100755 tests/xfs/131
 delete mode 100644 tests/xfs/131.out
 create mode 100755 tests/xfs/1538
 create mode 100644 tests/xfs/1538.out
 create mode 100755 tests/xfs/1539
 create mode 100644 tests/xfs/1539.out
 create mode 100755 tests/xfs/1540
 create mode 100644 tests/xfs/1540.out
 create mode 100755 tests/xfs/1541
 create mode 100644 tests/xfs/1541.out
 create mode 100755 tests/xfs/1542
 create mode 100644 tests/xfs/1542.out
 create mode 100755 tests/xfs/1543
 create mode 100644 tests/xfs/1543.out
 create mode 100755 tests/xfs/1544
 create mode 100644 tests/xfs/1544.out
 create mode 100755 tests/xfs/1545
 create mode 100644 tests/xfs/1545.out
 create mode 100755 tests/xfs/818
 create mode 100644 tests/xfs/818.out
 create mode 100755 tests/xfs/819
 create mode 100644 tests/xfs/819.out

[PATCH 01/10] xfs/122: update fields for realtime reflink

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add a few more ondisk structures for realtime reflink.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122.out |    3 +++
 1 file changed, 3 insertions(+)


diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index e0801f9660..3239a655f9 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -116,6 +116,9 @@ sizeof(struct xfs_rmap_key) = 20
 sizeof(struct xfs_rmap_rec) = 24
 sizeof(struct xfs_rtbuf_blkinfo) = 48
 sizeof(struct xfs_rtgroup_geometry) = 128
+sizeof(struct xfs_rtrefcount_key) = 4
+sizeof(struct xfs_rtrefcount_rec) = 12
+sizeof(struct xfs_rtrefcount_root) = 4
 sizeof(struct xfs_rtrmap_key) = 20
 sizeof(struct xfs_rtrmap_rec) = 24
 sizeof(struct xfs_rtrmap_root) = 4

[PATCH 02/10] common/populate: create realtime refcount btree

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Populate a realtime refcount btree when we're creating a sample fs.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/populate |   26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)


diff --git a/common/populate b/common/populate
index 6a05177e6d..734e31345f 100644
--- a/common/populate
+++ b/common/populate
@@ -367,16 +367,30 @@ _scratch_xfs_populate() {
 		rm -f "${dir}/${f}"
 	done
 
-	# Reverse-mapping btree
+	is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")"
 	is_rmapbt="$(_xfs_has_feature "$SCRATCH_MNT" rmapbt -v)"
+	is_reflink="$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)"
+
+	# Reverse-mapping btree
 	if [ $is_rmapbt -gt 0 ]; then
 		echo "+ rmapbt btree"
 		nr="$((blksz * 2 / 24))"
 		__populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RMAPBT"
 	fi
 
+	# Realtime Reference-count btree comes before the rtrmapbt so that
+	# the refcount entries are created in rtgroup 0.
+	if [ $is_reflink -gt 0 ] && [ $is_rt -gt 0 ]; then
+		echo "+ rtreflink btree"
+		rt_blksz=$(_xfs_get_rtextsize "$SCRATCH_MNT")
+		nr="$((rt_blksz * 2 / 12))"
+		$XFS_IO_PROG -R -f -c 'truncate 0' "${SCRATCH_MNT}/RTREFCOUNTBT"
+		__populate_create_file $((blksz * nr)) "${SCRATCH_MNT}/RTREFCOUNTBT"
+		$XFS_IO_PROG -R -f -c 'truncate 0' "${SCRATCH_MNT}/RTREFCOUNTBT2"
+		cp --reflink=always "${SCRATCH_MNT}/RTREFCOUNTBT" "${SCRATCH_MNT}/RTREFCOUNTBT2"
+	fi
+
 	# Realtime Reverse-mapping btree
-	is_rt="$(_xfs_get_rtextents "$SCRATCH_MNT")"
 	if [ $is_rmapbt -gt 0 ] && [ $is_rt -gt 0 ]; then
 		echo "+ rtrmapbt btree"
 		nr="$((blksz * 2 / 24))"
@@ -385,7 +399,6 @@ _scratch_xfs_populate() {
 	fi
 
 	# Reference-count btree
-	is_reflink="$(_xfs_has_feature "$SCRATCH_MNT" reflink -v)"
 	if [ $is_reflink -gt 0 ]; then
 		echo "+ reflink btree"
 		nr="$((blksz * 2 / 12))"
@@ -403,6 +416,7 @@ _scratch_xfs_populate() {
 	__populate_fragment_file "${SCRATCH_MNT}/RMAPBT"
 	__populate_fragment_file "${SCRATCH_MNT}/RTRMAPBT"
 	__populate_fragment_file "${SCRATCH_MNT}/REFCOUNTBT"
+	__populate_fragment_file "${SCRATCH_MNT}/RTREFCOUNTBT"
 
 	umount "${SCRATCH_MNT}"
 }
@@ -644,6 +658,10 @@ __populate_check_xfs_rgbtree_height() {
 		path_format="/realtime/%u.rmap"
 		bt_prefix="u3.rtrmapbt"
 		;;
+	"refcnt")
+		path_format="/realtime/%u.refcount"
+		bt_prefix="u3.rtrefcbt"
+		;;
 	*)
 		_fail "Don't know about rt btree ${bt_type}"
 		;;
@@ -718,6 +736,8 @@ _scratch_xfs_populate_check() {
 	test $is_reflink -ne 0 && __populate_check_xfs_agbtree_height "refcnt"
 	test $is_rmapbt -ne 0 && test $is_rt -gt 0 && \
 		__populate_check_xfs_rgbtree_height "rmap"
+	test $is_reflink -ne 0 && test $is_rt -gt 0 && \
+		__populate_check_xfs_rgbtree_height "refcnt"
 }
 
 # Check data fork format of ext4 file

[PATCH 03/10] xfs: create fuzz tests for the realtime refcount btree

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Create fuzz tests for the realtime refcount btree record and key/ptr
blocks.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs         |    4 ++++
 tests/xfs/1538     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1538.out |    4 ++++
 tests/xfs/1539     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1539.out |    4 ++++
 tests/xfs/1540     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1540.out |    4 ++++
 tests/xfs/1541     |   42 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1541.out |    4 ++++
 tests/xfs/1542     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1542.out |    4 ++++
 tests/xfs/1543     |   40 ++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1543.out |    4 ++++
 tests/xfs/1544     |   40 ++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1544.out |    4 ++++
 tests/xfs/1545     |   41 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1545.out |    4 ++++
 17 files changed, 363 insertions(+)
 create mode 100755 tests/xfs/1538
 create mode 100644 tests/xfs/1538.out
 create mode 100755 tests/xfs/1539
 create mode 100644 tests/xfs/1539.out
 create mode 100755 tests/xfs/1540
 create mode 100644 tests/xfs/1540.out
 create mode 100755 tests/xfs/1541
 create mode 100644 tests/xfs/1541.out
 create mode 100755 tests/xfs/1542
 create mode 100644 tests/xfs/1542.out
 create mode 100755 tests/xfs/1543
 create mode 100644 tests/xfs/1543.out
 create mode 100755 tests/xfs/1544
 create mode 100644 tests/xfs/1544.out
 create mode 100755 tests/xfs/1545
 create mode 100644 tests/xfs/1545.out


diff --git a/common/xfs b/common/xfs
index 63eff39d47..7b7b3a35b5 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1829,6 +1829,10 @@ _scratch_xfs_find_rgbtree_height() {
 		path_format="/realtime/%u.rmap"
 		bt_prefix="u3.rtrmapbt"
 		;;
+	"refcnt")
+		path_format="/realtime/%u.refcount"
+		bt_prefix="u3.rtrefcbt"
+		;;
 	*)
 		_fail "Don't know about rt btree ${bt_type}"
 		;;
diff --git a/tests/xfs/1538 b/tests/xfs/1538
new file mode 100755
index 0000000000..e62bf49b29
--- /dev/null
+++ b/tests/xfs/1538
@@ -0,0 +1,41 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1538
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt record field.
+# Use xfs_scrub to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_online_repair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
+
+echo "Fuzz rtrefcountbt recs"
+_scratch_xfs_fuzz_metadata '' 'online' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full
+echo "Done fuzzing rtrefcountbt recs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1538.out b/tests/xfs/1538.out
new file mode 100644
index 0000000000..968cfd6ef9
--- /dev/null
+++ b/tests/xfs/1538.out
@@ -0,0 +1,4 @@
+QA output created by 1538
+Format and populate
+Fuzz rtrefcountbt recs
+Done fuzzing rtrefcountbt recs
diff --git a/tests/xfs/1539 b/tests/xfs/1539
new file mode 100755
index 0000000000..36cef96a91
--- /dev/null
+++ b/tests/xfs/1539
@@ -0,0 +1,41 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1539
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt record field.
+# Use xfs_repair to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_repair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
+
+echo "Fuzz rtrefcountbt recs"
+_scratch_xfs_fuzz_metadata '' 'offline' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full
+echo "Done fuzzing rtrefcountbt recs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1539.out b/tests/xfs/1539.out
new file mode 100644
index 0000000000..aa3a963dc2
--- /dev/null
+++ b/tests/xfs/1539.out
@@ -0,0 +1,4 @@
+QA output created by 1539
+Format and populate
+Fuzz rtrefcountbt recs
+Done fuzzing rtrefcountbt recs
diff --git a/tests/xfs/1540 b/tests/xfs/1540
new file mode 100755
index 0000000000..fa08d3fb54
--- /dev/null
+++ b/tests/xfs/1540
@@ -0,0 +1,41 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1540
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt record field.
+# Do not fix the filesystem, to test metadata verifiers.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_norepair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
+
+echo "Fuzz rtrefcountbt recs"
+_scratch_xfs_fuzz_metadata '' 'none' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full
+echo "Done fuzzing rtrefcountbt recs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1540.out b/tests/xfs/1540.out
new file mode 100644
index 0000000000..37f3311837
--- /dev/null
+++ b/tests/xfs/1540.out
@@ -0,0 +1,4 @@
+QA output created by 1540
+Format and populate
+Fuzz rtrefcountbt recs
+Done fuzzing rtrefcountbt recs
diff --git a/tests/xfs/1541 b/tests/xfs/1541
new file mode 100755
index 0000000000..ecf6fdc56c
--- /dev/null
+++ b/tests/xfs/1541
@@ -0,0 +1,42 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1541
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt record field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_bothrepair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
+
+echo "Fuzz rtrefcountbt recs"
+_scratch_xfs_fuzz_metadata '' 'both' "path -m $path" "addr u${inode_ver}.rtrefcbt.ptrs[1]" >> $seqres.full
+echo "Done fuzzing rtrefcountbt recs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1541.out b/tests/xfs/1541.out
new file mode 100644
index 0000000000..35a9b73471
--- /dev/null
+++ b/tests/xfs/1541.out
@@ -0,0 +1,4 @@
+QA output created by 1541
+Format and populate
+Fuzz rtrefcountbt recs
+Done fuzzing rtrefcountbt recs
diff --git a/tests/xfs/1542 b/tests/xfs/1542
new file mode 100755
index 0000000000..37ef8a2b4c
--- /dev/null
+++ b/tests/xfs/1542
@@ -0,0 +1,41 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1542
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field.
+# Use xfs_scrub to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_online_repair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" "path -m $path")
+
+echo "Fuzz rtrefcountbt keyptrs"
+_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'online' "path -m $path" >> $seqres.full
+echo "Done fuzzing rtrefcountbt keyptrs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1542.out b/tests/xfs/1542.out
new file mode 100644
index 0000000000..55d820b4b1
--- /dev/null
+++ b/tests/xfs/1542.out
@@ -0,0 +1,4 @@
+QA output created by 1542
+Format and populate
+Fuzz rtrefcountbt keyptrs
+Done fuzzing rtrefcountbt keyptrs
diff --git a/tests/xfs/1543 b/tests/xfs/1543
new file mode 100755
index 0000000000..0acd3203e3
--- /dev/null
+++ b/tests/xfs/1543
@@ -0,0 +1,40 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1543
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field.
+# Use xfs_repair to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_scrub dangerous_repair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+
+echo "Fuzz rtrefcountbt keyptrs"
+_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'offline' "path -m $path" >> $seqres.full
+echo "Done fuzzing rtrefcountbt keyptrs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1543.out b/tests/xfs/1543.out
new file mode 100644
index 0000000000..e7afa10744
--- /dev/null
+++ b/tests/xfs/1543.out
@@ -0,0 +1,4 @@
+QA output created by 1543
+Format and populate
+Fuzz rtrefcountbt keyptrs
+Done fuzzing rtrefcountbt keyptrs
diff --git a/tests/xfs/1544 b/tests/xfs/1544
new file mode 100755
index 0000000000..165f96f6a4
--- /dev/null
+++ b/tests/xfs/1544
@@ -0,0 +1,40 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1544
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field.
+# Do not fix the filesystem, to test metadata verifiers.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_norepair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+
+echo "Fuzz rtrefcountbt keyptrs"
+_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'none' "path -m $path" >> $seqres.full
+echo "Done fuzzing rtrefcountbt keyptrs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1544.out b/tests/xfs/1544.out
new file mode 100644
index 0000000000..b39532c160
--- /dev/null
+++ b/tests/xfs/1544.out
@@ -0,0 +1,4 @@
+QA output created by 1544
+Format and populate
+Fuzz rtrefcountbt keyptrs
+Done fuzzing rtrefcountbt keyptrs
diff --git a/tests/xfs/1545 b/tests/xfs/1545
new file mode 100755
index 0000000000..a467662f2f
--- /dev/null
+++ b/tests/xfs/1545
@@ -0,0 +1,41 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1545
+#
+# Populate a XFS filesystem and fuzz every rtrefcountbt key/ptr field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers dangerous_bothrepair realtime
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch_reflink
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+path="$(_scratch_xfs_find_rgbtree_height 'refcnt' 2)" || \
+	_fail "could not find two-level rtrefcountbt"
+
+echo "Fuzz rtrefcountbt keyptrs"
+_scratch_xfs_fuzz_metadata '(rtrefcbt)' 'both' "path -m $path" >> $seqres.full
+echo "Done fuzzing rtrefcountbt keyptrs"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1545.out b/tests/xfs/1545.out
new file mode 100644
index 0000000000..982a0d64df
--- /dev/null
+++ b/tests/xfs/1545.out
@@ -0,0 +1,4 @@
+QA output created by 1545
+Format and populate
+Fuzz rtrefcountbt keyptrs
+Done fuzzing rtrefcountbt keyptrs

[PATCH 06/10] xfs: race fsstress with realtime refcount btree scrub and repair

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Race checking and rebuilding realtime refcount btrees with fsstress.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/818     |   43 +++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/818.out |    2 ++
 tests/xfs/819     |   43 +++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/819.out |    2 ++
 4 files changed, 90 insertions(+)
 create mode 100755 tests/xfs/818
 create mode 100644 tests/xfs/818.out
 create mode 100755 tests/xfs/819
 create mode 100644 tests/xfs/819.out


diff --git a/tests/xfs/818 b/tests/xfs/818
new file mode 100755
index 0000000000..aabe636750
--- /dev/null
+++ b/tests/xfs/818
@@ -0,0 +1,43 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle. Inc.  All Rights Reserved.
+#
+# FS QA Test No. 818
+#
+# Race fsstress and rt refcount btree scrub for a while to see if we crash or
+# livelock.
+#
+. ./common/preamble
+_begin_fstest scrub dangerous_fsstress_scrub
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch
+_require_xfs_stress_scrub
+
+_scratch_mkfs > "$seqres.full" 2>&1
+_scratch_mount
+_require_xfs_has_feature "$SCRATCH_MNT" realtime
+_require_xfs_has_feature "$SCRATCH_MNT" reflink
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+_scratch_xfs_stress_scrub -s "scrub rtrefcountbt %rgno%"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/818.out b/tests/xfs/818.out
new file mode 100644
index 0000000000..cb0997862e
--- /dev/null
+++ b/tests/xfs/818.out
@@ -0,0 +1,2 @@
+QA output created by 818
+Silence is golden
diff --git a/tests/xfs/819 b/tests/xfs/819
new file mode 100755
index 0000000000..e302ed1fdc
--- /dev/null
+++ b/tests/xfs/819
@@ -0,0 +1,43 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle. Inc.  All Rights Reserved.
+#
+# FS QA Test No. 819
+#
+# Race fsstress and rt refcount btree scrub for a while to see if we crash or
+# livelock.
+#
+. ./common/preamble
+_begin_fstest online_repair dangerous_fsstress_repair
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+# real QA test starts here
+_supported_fs xfs
+_require_realtime
+_require_scratch
+_require_xfs_stress_online_repair
+
+_scratch_mkfs > "$seqres.full" 2>&1
+_scratch_mount
+_require_xfs_has_feature "$SCRATCH_MNT" realtime
+_require_xfs_has_feature "$SCRATCH_MNT" reflink
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+_scratch_xfs_stress_online_repair -s "repair rtrefcountbt %rgno%"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/819.out b/tests/xfs/819.out
new file mode 100644
index 0000000000..f5df7622a7
--- /dev/null
+++ b/tests/xfs/819.out
@@ -0,0 +1,2 @@
+QA output created by 819
+Silence is golden

[PATCH 09/10] generic/331,xfs/240: support files that skip delayed allocation

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The goal of this test is to ensure that log recovery finishes a copy on
write operation in the event of temporary media errors.  It's important
that the test observe some sort of IO error once we switch the scratch
device to fail all IOs, but regrettably the test encoded the specific
behavior of XFS and btrfs when the test was written -- the aio write
to the page cache doesn't have to touch the disk and succeeds, and the
fdatasync flushes things to disk and hits the IO error.

However, this is not how things work on the XFS realtime device.  There
is no delalloc on realtime, so the aio write allocates an unwritten
extent to stage the write.  The allocation fails due to EIO, so it's the
write call that fails.  Therefore, all we need to do is to detect an IO
error at any point between the write and the fdatasync call to be
satisfied that the test does what we want to do.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/331     |   12 ++++++++++--
 tests/generic/331.out |    2 +-
 tests/xfs/240         |   13 +++++++++++--
 tests/xfs/240.out     |    2 +-
 4 files changed, 23 insertions(+), 6 deletions(-)


diff --git a/tests/generic/331 b/tests/generic/331
index 492abedf76..8c665ce4fc 100755
--- a/tests/generic/331
+++ b/tests/generic/331
@@ -59,9 +59,17 @@ echo "CoW and unmount"
 $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full
 $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full
 sync
+
+# If the filesystem supports delalloc, then the fdatasync will report an IO
+# error.  If the write goes directly to disk, then aiocp will return nonzero.
+unset write_failed
 _dmerror_load_error_table
-$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full
-$XFS_IO_PROG -c "fdatasync" $testdir/file2
+$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 &>> $seqres.full || \
+	write_failed=1
+$XFS_IO_PROG -c "fdatasync" $testdir/file2 2>&1 | grep -q 'Input.output error' && \
+	write_failed=1
+test -n $write_failed && echo "write failed"
+
 _dmerror_load_working_table
 _dmerror_unmount
 _dmerror_mount
diff --git a/tests/generic/331.out b/tests/generic/331.out
index adbf841d00..d8ccea704b 100644
--- a/tests/generic/331.out
+++ b/tests/generic/331.out
@@ -5,7 +5,7 @@ Compare files
 1886e67cf8783e89ce6ddc5bb09a3944  SCRATCH_MNT/test-331/file1
 1886e67cf8783e89ce6ddc5bb09a3944  SCRATCH_MNT/test-331/file2
 CoW and unmount
-fdatasync: Input/output error
+write failed
 Compare files
 1886e67cf8783e89ce6ddc5bb09a3944  SCRATCH_MNT/test-331/file1
 d94b0ab13385aba594411c174b1cc13c  SCRATCH_MNT/test-331/file2
diff --git a/tests/xfs/240 b/tests/xfs/240
index a65c270d23..cabe309201 100755
--- a/tests/xfs/240
+++ b/tests/xfs/240
@@ -66,8 +66,17 @@ $XFS_IO_PROG -f -c "pwrite -S 0x63 $bufsize 1" $testdir/file2 >> $seqres.full
 $XFS_IO_PROG -f -c "pwrite -S 0x63 -b $bufsize 0 $filesize" $TEST_DIR/moo >> $seqres.full
 sync
 _dmerror_load_error_table
-$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 >> $seqres.full
-$XFS_IO_PROG -c "fdatasync" $testdir/file2
+
+# If the filesystem supports delalloc, then the fdatasync will report an IO
+# error.  If the write goes directly to disk, then aiocp will return nonzero.
+unset write_failed
+_dmerror_load_error_table
+$AIO_TEST -b $bufsize $TEST_DIR/moo $testdir/file2 &>> $seqres.full || \
+	write_failed=1
+$XFS_IO_PROG -c "fdatasync" $testdir/file2 2>&1 | grep -q 'Input.output error' && \
+	write_failed=1
+test -n $write_failed && echo "write failed"
+
 _dmerror_load_working_table
 _dmerror_unmount
 _dmerror_mount
diff --git a/tests/xfs/240.out b/tests/xfs/240.out
index 1a22e8a389..00bb116e5c 100644
--- a/tests/xfs/240.out
+++ b/tests/xfs/240.out
@@ -5,7 +5,7 @@ Compare files
 1886e67cf8783e89ce6ddc5bb09a3944  SCRATCH_MNT/test-240/file1
 1886e67cf8783e89ce6ddc5bb09a3944  SCRATCH_MNT/test-240/file2
 CoW and unmount
-fdatasync: Input/output error
+write failed
 Compare files
 1886e67cf8783e89ce6ddc5bb09a3944  SCRATCH_MNT/test-240/file1
 d94b0ab13385aba594411c174b1cc13c  SCRATCH_MNT/test-240/file2

[PATCH 05/10] xfs/243: don't run when realtime storage is the default

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Realtime volumes don't support delayed allocation, so don't run this
test when the mkfs configuration specifies realtime creation by default.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/243 |    5 +++++
 1 file changed, 5 insertions(+)


diff --git a/tests/xfs/243 b/tests/xfs/243
index 514fa35667..dda4a0c223 100755
--- a/tests/xfs/243
+++ b/tests/xfs/243
@@ -38,6 +38,11 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
 
+# XFS does not support delayed allocation on realtime volumes (even for COW)
+# so skip this test on those platforms.
+$XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT | grep -q "xflags.*rt-inherit" &&
+	_notrun "delalloc not used for CoW on realtime device"
+
 testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
 

[PATCH 04/10] xfs/27[24]: adapt for checking files on the realtime volume

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Adapt both tests to behave properly if the two files being tested are on
the realtime volume.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/272 |   40 +++++++++++++++++++++++++------------
 tests/xfs/274 |   62 ++++++++++++++++++++++++++++++++++++++++-----------------
 2 files changed, 70 insertions(+), 32 deletions(-)


diff --git a/tests/xfs/272 b/tests/xfs/272
index 42b4a2edb5..2d7fc57d55 100755
--- a/tests/xfs/272
+++ b/tests/xfs/272
@@ -40,26 +40,40 @@ $here/src/punch-alternating $SCRATCH_MNT/urk >> $seqres.full
 ino=$(stat -c '%i' $SCRATCH_MNT/urk)
 
 echo "Get fsmap" | tee -a $seqres.full
-$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full
 $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT | tr '[]()' '    ' > $TEST_DIR/fsmap
+cat $TEST_DIR/fsmap >> $seqres.full
 
 echo "Get bmap" | tee -a $seqres.full
-$XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/urk >> $seqres.full
 $XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/urk | grep '^[[:space:]]*[0-9]*:' | grep -v 'hole' | tr '[]()' '    ' > $TEST_DIR/bmap
+cat $TEST_DIR/bmap >> $seqres.full
 
 echo "Check bmap and fsmap" | tee -a $seqres.full
-cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do
-	qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total}$"
-	echo "${qstr}" >> $seqres.full
-	grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
-	found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
-	test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
-done
+if $XFS_IO_PROG -c 'stat -v' $SCRATCH_MNT/urk | grep -q realtime; then
+	# file on rt volume
+	cat $TEST_DIR/bmap | while read ext offrange colon rtblockrange total crap; do
+		qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${rtblockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${total}$"
+		echo "${qstr}" >> $seqres.full
+		grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
+		found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
+		test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
+	done
 
-echo "Check device field of FS metadata and regular file"
-data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
-rt_dev=$(grep "${ino}[[:space:]]*[0-9]*\.\.[0-9]*" $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
-test "${data_dev}" = "${rt_dev}" || echo "data ${data_dev} realtime ${rt_dev}?"
+	echo "Check device field of FS metadata and regular file"
+else
+	# file on data volume
+	cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do
+		qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total}$"
+		echo "${qstr}" >> $seqres.full
+		grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
+		found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
+		test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
+	done
+
+	echo "Check device field of FS metadata and regular file"
+	data_dev=$(grep 'inode btree' $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
+	rt_dev=$(grep "${ino}[[:space:]]*[0-9]*\.\.[0-9]*" $TEST_DIR/fsmap | head -n 1 | awk '{print $2}')
+	test "${data_dev}" = "${rt_dev}" || echo "data ${data_dev} realtime ${rt_dev}?"
+fi
 
 # success, all done
 status=0
diff --git a/tests/xfs/274 b/tests/xfs/274
index dcaea68804..25dd0c3f74 100755
--- a/tests/xfs/274
+++ b/tests/xfs/274
@@ -40,34 +40,58 @@ _cp_reflink $SCRATCH_MNT/f1 $SCRATCH_MNT/f2
 ino=$(stat -c '%i' $SCRATCH_MNT/f1)
 
 echo "Get fsmap" | tee -a $seqres.full
-$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full
 $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT | tr '[]()' '    ' > $TEST_DIR/fsmap
+cat $TEST_DIR/fsmap >> $seqres.full
 
 echo "Get f1 bmap" | tee -a $seqres.full
-$XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f1 >> $seqres.full
 $XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f1 | grep '^[[:space:]]*[0-9]*:' | grep -v 'hole' | tr '[]()' '    ' > $TEST_DIR/bmap
+cat $TEST_DIR/bmap >> $seqres.full
 
-echo "Check f1 bmap and fsmap" | tee -a $seqres.full
-cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do
-	qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 0100000$"
-	echo "${qstr}" >> $seqres.full
-	grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
-	found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
-	test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
-done
+if _xfs_is_realtime_file $SCRATCH_MNT/f1 && ! _xfs_has_feature $SCRATCH_MNT rtgroups; then
+	# file on rt volume
+	echo "Check f1 bmap and fsmap" | tee -a $seqres.full
+	cat $TEST_DIR/bmap | while read ext offrange colon rtblockrange total crap; do
+		qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${rtblockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${total} 0100000$"
+		echo "${qstr}" >> $seqres.full
+		grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
+		found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
+		test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
+	done
+else
+	# file on data volume
+	echo "Check f1 bmap and fsmap" | tee -a $seqres.full
+	cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do
+		qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 0100000$"
+		echo "${qstr}" >> $seqres.full
+		grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
+		found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
+		test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
+	done
+fi
 
 echo "Get f2 bmap" | tee -a $seqres.full
-$XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f2 >> $seqres.full
 $XFS_IO_PROG -c 'bmap -v' $SCRATCH_MNT/f2 | grep '^[[:space:]]*[0-9]*:' | grep -v 'hole' | tr '[]()' '    ' > $TEST_DIR/bmap
+cat $TEST_DIR/bmap >> $seqres.full
 
-echo "Check f2 bmap and fsmap" | tee -a $seqres.full
-cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do
-	qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 0100000$"
-	echo "${qstr}" >> $seqres.full
-	grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
-	found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
-	test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
-done
+if _xfs_is_realtime_file $SCRATCH_MNT/f2 && ! _xfs_has_feature $SCRATCH_MNT rtgroups; then
+	echo "Check f2 bmap and fsmap" | tee -a $seqres.full
+	cat $TEST_DIR/bmap | while read ext offrange colon rtblockrange total crap; do
+		qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${rtblockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${total} 0100000$"
+		echo "${qstr}" >> $seqres.full
+		grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
+		found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
+		test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
+	done
+else
+	echo "Check f2 bmap and fsmap" | tee -a $seqres.full
+	cat $TEST_DIR/bmap | while read ext offrange colon blockrange ag agrange total crap; do
+		qstr="^[[:space:]]*[0-9]*:[[:space:]]*[0-9]*:[0-9]*[[:space:]]*${blockrange} :[[:space:]]*${ino}[[:space:]]*${offrange}[[:space:]]*${ag}[[:space:]]*${agrange}[[:space:]]*${total} 0100000$"
+		echo "${qstr}" >> $seqres.full
+		grep "${qstr}" $TEST_DIR/fsmap >> $seqres.full
+		found=$(grep -c "${qstr}" $TEST_DIR/fsmap)
+		test $found -eq 1 || echo "Unexpected output for offset ${offrange}."
+	done
+fi
 
 # success, all done
 status=0

[PATCH 08/10] xfs/769: add rtreflink upgrade to test matrix

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add realtime reflink to the features that this test will try to
upgrade.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/769 |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/769 b/tests/xfs/769
index ccc3ea10bc..72863a6e83 100755
--- a/tests/xfs/769
+++ b/tests/xfs/769
@@ -196,12 +196,13 @@ function post_exercise()
 # upgrade don't spread failure to the rest of the tests.
 FEATURES=()
 if rt_configured; then
-	# rmap wasn't added to rt devices until after metadir
+	# reflink & rmap weren't added to rt devices until after metadir
 	check_repair_upgrade finobt && FEATURES+=("finobt")
 	check_repair_upgrade inobtcount && FEATURES+=("inobtcount")
 	check_repair_upgrade bigtime && FEATURES+=("bigtime")
 	check_repair_upgrade metadir && FEATURES+=("metadir")
 	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")
+	check_repair_upgrade reflink && FEATURES+=("reflink")
 else
 	check_repair_upgrade finobt && FEATURES+=("finobt")
 	check_repair_upgrade rmapbt && FEATURES+=("rmapbt")

[PATCH 10/10] common/xfs: fix _xfs_get_file_block_size when rtinherit is set and no rt section

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

It's possible for the sysadmin to set rtinherit on the directory tree
even if there isn't a realtime section attached to the filesystem.  When
this is the case, the realtime flag is /not/ passed to new files, and
file data is written to the data device.  The file allocation unit for
the file is the fs blocksize, and it is not correct to use the rt
extent.

fstests can be fooled into doing the incorrect thing if test runner puts
'-d rtinherit=1 -r extsize=28k' into MKFS_OPTIONS without configuring a
realtime device.  This causes many tests to do the wrong thing because
they think they must operate on units of 28k (and not 4k).  Fix this.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs |   11 +++++++++++
 1 file changed, 11 insertions(+)


diff --git a/common/xfs b/common/xfs
index 7b7b3a35b5..546853247c 100644
--- a/common/xfs
+++ b/common/xfs
@@ -207,6 +207,8 @@ _xfs_get_file_block_size()
 {
 	local path="$1"
 
+	# If rtinherit or realtime are not set on the path, then all files
+	# will be created on the data device.
 	if ! ($XFS_IO_PROG -c "stat -v" "$path" 2>&1 | grep -E -q '(rt-inherit|realtime)'); then
 		_get_block_size "$path"
 		return
@@ -217,6 +219,15 @@ _xfs_get_file_block_size()
 	while ! $XFS_INFO_PROG "$path" &>/dev/null && [ "$path" != "/" ]; do
 		path="$(dirname "$path")"
 	done
+
+	# If there's no realtime section, the rtinherit and rextsize settings
+	# are irrelevant -- all files are created on the data device.
+	if $XFS_INFO_PROG "$path" | grep -q 'realtime =none'; then
+		_get_block_size "$path"
+		return
+	fi
+
+	# Otherwise, report the rt extent size.
 	_xfs_get_rtextsize "$path"
 }
 

[PATCH 07/10] xfs: remove xfs/131 now that we allow reflink on realtime volumes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Remove this test, since we now support reflink on the rt volume.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/131     |   48 ------------------------------------------------
 tests/xfs/131.out |    5 -----
 2 files changed, 53 deletions(-)
 delete mode 100755 tests/xfs/131
 delete mode 100644 tests/xfs/131.out


diff --git a/tests/xfs/131 b/tests/xfs/131
deleted file mode 100755
index 879e2dc6e8..0000000000
--- a/tests/xfs/131
+++ /dev/null
@@ -1,48 +0,0 @@
-#! /bin/bash
-# SPDX-License-Identifier: GPL-2.0
-# Copyright (c) 2015, Oracle and/or its affiliates.  All Rights Reserved.
-#
-# FS QA Test No. 131
-#
-# Ensure that we can't reflink realtime files.
-#
-. ./common/preamble
-_begin_fstest auto quick clone realtime
-
-# Override the default cleanup function.
-_cleanup()
-{
-    cd /
-    umount $SCRATCH_MNT > /dev/null 2>&1
-    rm -rf $tmp.* $testdir $metadump_file
-}
-
-# Import common functions.
-. ./common/filter
-. ./common/reflink
-
-# real QA test starts here
-_supported_fs xfs
-_require_realtime
-_require_scratch_reflink
-_require_cp_reflink
-
-echo "Format and mount scratch device"
-_scratch_mkfs >> $seqres.full
-_scratch_mount
-
-testdir=$SCRATCH_MNT/test-$seq
-mkdir $testdir
-
-echo "Create the original file blocks"
-blksz=65536
-$XFS_IO_PROG -R -f -c "truncate $blksz" $testdir/file1
-
-echo "Reflink every block"
-_cp_reflink $testdir/file1 $testdir/file2 2>&1 | _filter_scratch
-
-test -s $testdir/file2 && _fail "Should not be able to reflink a realtime file."
-
-# success, all done
-status=0
-exit
diff --git a/tests/xfs/131.out b/tests/xfs/131.out
deleted file mode 100644
index 3c0186f0c7..0000000000
--- a/tests/xfs/131.out
+++ /dev/null
@@ -1,5 +0,0 @@
-QA output created by 131
-Format and mount scratch device
-Create the original file blocks
-Reflink every block
-cp: failed to clone 'SCRATCH_MNT/test-131/file2' from 'SCRATCH_MNT/test-131/file1': Invalid argument

[PATCHSET v1.0 0/4] fstests: reflink with large realtime extents

[Darrick J. Wong]

Hi all,

Now that we've landed support for reflink on the realtime device for
cases where the rt extent size is the same as the fs block size, enhance
the reflink code further to support cases where the rt extent size is a
power-of-two multiple of the fs block size.  This enables us to do data
block sharing (for example) for much larger allocation units by dirtying
pagecache around shared extents and expanding writeback to write back
shared extents fully.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-reflink-extsize

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-reflink-extsize

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-reflink-extsize
---
 common/rc         |   23 +++++++
 common/reflink    |   27 +++++++++
 tests/generic/145 |    1 
 tests/generic/147 |    1 
 tests/generic/261 |    1 
 tests/generic/262 |    1 
 tests/generic/303 |    8 ++-
 tests/generic/331 |    1 
 tests/generic/353 |    3 +
 tests/generic/517 |    1 
 tests/generic/657 |    1 
 tests/generic/658 |    1 
 tests/generic/659 |    1 
 tests/generic/660 |    1 
 tests/generic/663 |    1 
 tests/generic/664 |    1 
 tests/generic/665 |    1 
 tests/generic/670 |    1 
 tests/generic/672 |    1 
 tests/xfs/1212    |    1 
 tests/xfs/180     |    1 
 tests/xfs/182     |    1 
 tests/xfs/184     |    1 
 tests/xfs/192     |    1 
 tests/xfs/200     |    1 
 tests/xfs/204     |    1 
 tests/xfs/208     |    1 
 tests/xfs/315     |    1 
 tests/xfs/326     |    6 ++
 tests/xfs/420     |    3 +
 tests/xfs/421     |    3 +
 tests/xfs/919     |  163 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/919.out |   84 +++++++++++++++++++++++++++
 33 files changed, 342 insertions(+), 2 deletions(-)
 create mode 100755 tests/xfs/919
 create mode 100644 tests/xfs/919.out

[PATCH 1/4] xfs: make sure that CoW will write around when rextsize > 1

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Make sure that CoW triggers the intended copy-around behavior when we
write a tiny amount to the middle of a large rt extent.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/919     |  163 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/919.out |   84 +++++++++++++++++++++++++++
 2 files changed, 247 insertions(+)
 create mode 100755 tests/xfs/919
 create mode 100644 tests/xfs/919.out


diff --git a/tests/xfs/919 b/tests/xfs/919
new file mode 100755
index 0000000000..45bb42f91f
--- /dev/null
+++ b/tests/xfs/919
@@ -0,0 +1,163 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 919
+#
+# Make sure that copy on write actually does the intended write-around when we
+# stage a tiny modification to a large shared realtime extent.  We should never
+# end up with multiple rt extents mapped to the same region.
+#
+. ./common/preamble
+_begin_fstest auto quick clone realtime
+
+# Import common functions.
+. ./common/filter
+. ./common/reflink
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_io_command "fpunch"
+_require_xfs_io_command "fzero"
+_require_xfs_io_command "fcollapse"
+_require_xfs_io_command "finsert"
+_require_xfs_io_command "funshare"
+_require_realtime
+_require_scratch_reflink
+
+rtextsz=262144
+filesz=$((rtextsz * 3))
+
+echo "Format filesystem and populate"
+_scratch_mkfs -m reflink=1 -r extsize=$rtextsz > $seqres.full
+_scratch_mount >> $seqres.full
+
+# Force all our files to be on the realtime device
+_xfs_force_bdev realtime $SCRATCH_MNT
+
+check_file() {
+	$XFS_IO_PROG -c fsync -c 'bmap -elpv' $1 >> $seqres.full
+	md5sum $SCRATCH_MNT/a | _filter_scratch
+	md5sum $1 | _filter_scratch
+}
+
+rtextsz_got=$(_xfs_get_rtextsize "$SCRATCH_MNT")
+test $rtextsz_got -eq $rtextsz || \
+	_notrun "got rtextsize $rtextsz_got, wanted $rtextsz"
+
+_pwrite_byte 0x59 0 $filesz $SCRATCH_MNT/a >> $seqres.full
+sync
+md5sum $SCRATCH_MNT/a | _filter_scratch
+$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/a >> $seqres.full
+
+echo "pwrite 1 byte in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/b
+_pwrite_byte 0x00 345678 1 $SCRATCH_MNT/b >> $seqres.full
+check_file $SCRATCH_MNT/b
+
+echo "mwrite 1 byte in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/c
+$XFS_IO_PROG -c "mmap -rw 0 $filesz" -c "mwrite -S 0x00 345678 1" -c msync $SCRATCH_MNT/c
+check_file $SCRATCH_MNT/c
+
+echo "fzero 1 byte in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/d
+$XFS_IO_PROG -c "fzero 345678 1" $SCRATCH_MNT/d
+check_file $SCRATCH_MNT/d
+
+echo "fpunch 1 byte in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/e
+$XFS_IO_PROG -c "fpunch 345678 1" $SCRATCH_MNT/e
+check_file $SCRATCH_MNT/e
+
+echo "funshare 1 byte in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/f
+$XFS_IO_PROG -c "funshare 345678 1" $SCRATCH_MNT/f
+check_file $SCRATCH_MNT/f
+
+echo "pwrite 1 block in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/g
+_pwrite_byte 0x00 327680 65536 $SCRATCH_MNT/g >> $seqres.full
+check_file $SCRATCH_MNT/g
+
+echo "mwrite 1 block in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/h
+$XFS_IO_PROG -c "mmap -rw 0 $filesz" -c "mwrite -S 0x00 327680 65536" -c msync $SCRATCH_MNT/h
+check_file $SCRATCH_MNT/h
+
+echo "fzero 1 block in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/i
+$XFS_IO_PROG -c "fzero 327680 65536" $SCRATCH_MNT/i
+check_file $SCRATCH_MNT/i
+
+echo "fpunch 1 block in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/j
+$XFS_IO_PROG -c "fpunch 327680 65536" $SCRATCH_MNT/j
+check_file $SCRATCH_MNT/j
+
+echo "funshare 1 block in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/k
+$XFS_IO_PROG -c "funshare 327680 65536" $SCRATCH_MNT/k
+check_file $SCRATCH_MNT/k
+
+echo "pwrite 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/l
+_pwrite_byte 0x00 262144 262144 $SCRATCH_MNT/l >> $seqres.full
+check_file $SCRATCH_MNT/l
+
+echo "mwrite 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/m
+$XFS_IO_PROG -c "mmap -rw 0 $filesz" -c "mwrite -S 0x00 262144 262144" -c msync $SCRATCH_MNT/m
+check_file $SCRATCH_MNT/m
+
+echo "fzero 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/n
+$XFS_IO_PROG -c "fzero 262144 262144" $SCRATCH_MNT/n
+check_file $SCRATCH_MNT/n
+
+echo "fpunch 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/o
+$XFS_IO_PROG -c "fpunch 262144 262144" $SCRATCH_MNT/o
+check_file $SCRATCH_MNT/o
+
+echo "funshare 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/p
+$XFS_IO_PROG -c "funshare 262144 262144" $SCRATCH_MNT/p
+check_file $SCRATCH_MNT/p
+
+echo "fcollapse 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/q
+$XFS_IO_PROG -c "fcollapse 262144 262144" $SCRATCH_MNT/q
+check_file $SCRATCH_MNT/q
+
+echo "finsert 1 extent in the middle" | tee -a $seqres.full
+_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/r
+$XFS_IO_PROG -c "finsert 262144 262144" $SCRATCH_MNT/r
+check_file $SCRATCH_MNT/r
+
+echo "copy unwritten blocks in large rtext" | tee -a $seqres.full
+$XFS_IO_PROG -f -c "falloc 0 $filesz" -c 'pwrite -S 0x59 345678 1' $SCRATCH_MNT/s >> $seqres.full
+$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/s >> $seqres.full
+_cp_reflink $SCRATCH_MNT/s $SCRATCH_MNT/t
+$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/s >> $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x59 1048576 1' $SCRATCH_MNT/s >> $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x59 1048576 1' $SCRATCH_MNT/t >> $seqres.full
+check_file $SCRATCH_MNT/s
+check_file $SCRATCH_MNT/t
+
+echo "test writing to shared unwritten extent" | tee -a $seqres.full
+$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/s >> $seqres.full
+_cp_reflink $SCRATCH_MNT/s $SCRATCH_MNT/u
+$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/s >> $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x59 345678 1' $SCRATCH_MNT/u >> $seqres.full
+check_file $SCRATCH_MNT/u
+
+echo "Remount and recheck" | tee -a $seqres.full
+md5sum $SCRATCH_MNT/a | _filter_scratch
+for i in b c d e f g h i j k l m n o p q r s t u; do
+	check_file $SCRATCH_MNT/$i | grep -v SCRATCH_MNT.a
+done
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/919.out b/tests/xfs/919.out
new file mode 100644
index 0000000000..6ab3f70d17
--- /dev/null
+++ b/tests/xfs/919.out
@@ -0,0 +1,84 @@
+QA output created by 919
+Format filesystem and populate
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+pwrite 1 byte in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/b
+mwrite 1 byte in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/c
+fzero 1 byte in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/d
+fpunch 1 byte in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/e
+funshare 1 byte in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/f
+pwrite 1 block in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/g
+mwrite 1 block in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/h
+fzero 1 block in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/i
+fpunch 1 block in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/j
+funshare 1 block in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/k
+pwrite 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/l
+mwrite 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/m
+fzero 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/n
+fpunch 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/o
+funshare 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/p
+fcollapse 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+b0581637c15320958874ef3f082111da  SCRATCH_MNT/q
+finsert 1 extent in the middle
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+a7359d0c100367c2cd430be334dffbd3  SCRATCH_MNT/r
+copy unwritten blocks in large rtext
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+bff4e0a70430429c92d6139065e6949b  SCRATCH_MNT/s
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+bff4e0a70430429c92d6139065e6949b  SCRATCH_MNT/t
+test writing to shared unwritten extent
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+bff4e0a70430429c92d6139065e6949b  SCRATCH_MNT/u
+Remount and recheck
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/a
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/b
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/c
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/d
+ea0b05f13c8cce703accaffe56d59bd3  SCRATCH_MNT/e
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/f
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/g
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/h
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/i
+2a508e23efc80e468efa7004fd8a1839  SCRATCH_MNT/j
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/k
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/l
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/m
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/n
+352abe71b0d40f194b9d701750b0d7f3  SCRATCH_MNT/o
+924a97fdaa2ab30e2768081469e728a7  SCRATCH_MNT/p
+b0581637c15320958874ef3f082111da  SCRATCH_MNT/q
+a7359d0c100367c2cd430be334dffbd3  SCRATCH_MNT/r
+bff4e0a70430429c92d6139065e6949b  SCRATCH_MNT/s
+bff4e0a70430429c92d6139065e6949b  SCRATCH_MNT/t
+bff4e0a70430429c92d6139065e6949b  SCRATCH_MNT/u

[PATCH 2/4] xfs: skip cowextsize hint fragmentation tests on realtime volumes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The XFS CoW extent size hint is ignored on realtime filesystems when the
rt extent size set to a unit larger than a single filesystem block
because it is assumed that the larger allocation unit is the
administrator's sole and mandatory anti-fragmentation strategy.  As
such, we can skip these tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/reflink |   27 +++++++++++++++++++++++++++
 tests/xfs/180  |    1 +
 tests/xfs/182  |    1 +
 tests/xfs/184  |    1 +
 tests/xfs/192  |    1 +
 tests/xfs/200  |    1 +
 tests/xfs/204  |    1 +
 tests/xfs/208  |    1 +
 tests/xfs/315  |    1 +
 tests/xfs/326  |    6 ++++++
 10 files changed, 41 insertions(+)


diff --git a/common/reflink b/common/reflink
index 22adc4449b..1082642e4e 100644
--- a/common/reflink
+++ b/common/reflink
@@ -521,3 +521,30 @@ _sweave_reflink_rainbow_delalloc() {
 		_pwrite_byte 0x62 $((blksz * i)) $blksz $dfile.chk
 	done
 }
+
+# Require that the COW extent size hint can actually be used to combat
+# fragmentation on the scratch filesystem.  This is (so far) true for any
+# filesystem except for the ones where the realtime extent size is larger
+# than one fs block, for it is assumed that setting a rt extent size is the
+# preferred fragmentation avoidance strategy.
+_require_scratch_cowextsize_useful() {
+	local testfile=$SCRATCH_MNT/hascowextsize
+	local param="${1:-1m}"
+
+	rm -f $testfile
+	touch $testfile
+	local before="$($XFS_IO_PROG -c 'cowextsize' $testfile)"
+
+	$XFS_IO_PROG -c "cowextsize $param" $testfile
+	local after="$($XFS_IO_PROG -c 'cowextsize' $testfile)"
+	rm -f $testfile
+
+	test "$before" != "$after" || \
+		_notrun "setting cowextsize to $param had no effect"
+
+	local fileblocksize=$(_get_file_block_size $SCRATCH_MNT)
+	local fsblocksize=$(_get_block_size $SCRATCH_MNT)
+
+	test $fsblocksize -eq $fileblocksize || \
+		_notrun "XFS does not support cowextsize when rt extsize ($fileblocksize) > 1FSB ($fsblocksize)"
+}
diff --git a/tests/xfs/180 b/tests/xfs/180
index cfea2020ce..06b4b69d52 100755
--- a/tests/xfs/180
+++ b/tests/xfs/180
@@ -37,6 +37,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_block_size $testdir)
diff --git a/tests/xfs/182 b/tests/xfs/182
index 511aca6f2d..7c0713b248 100755
--- a/tests/xfs/182
+++ b/tests/xfs/182
@@ -39,6 +39,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_block_size $testdir)
diff --git a/tests/xfs/184 b/tests/xfs/184
index 3bdd86addf..a0dc2741f5 100755
--- a/tests/xfs/184
+++ b/tests/xfs/184
@@ -39,6 +39,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_block_size $testdir)
diff --git a/tests/xfs/192 b/tests/xfs/192
index eb577f15fc..daa4fcb1e0 100755
--- a/tests/xfs/192
+++ b/tests/xfs/192
@@ -41,6 +41,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_block_size $testdir)
diff --git a/tests/xfs/200 b/tests/xfs/200
index b51b9a54f5..8eb54b5755 100755
--- a/tests/xfs/200
+++ b/tests/xfs/200
@@ -41,6 +41,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_block_size $testdir)
diff --git a/tests/xfs/204 b/tests/xfs/204
index 7d6b79a86d..85b7ed7cd9 100755
--- a/tests/xfs/204
+++ b/tests/xfs/204
@@ -43,6 +43,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_block_size $testdir)
diff --git a/tests/xfs/208 b/tests/xfs/208
index 9a71b74f6f..3a4a3e4df1 100755
--- a/tests/xfs/208
+++ b/tests/xfs/208
@@ -40,6 +40,7 @@ nr=128
 filesize=$((blksz * nr))
 bufnr=16
 bufsize=$((blksz * bufnr))
+_require_scratch_cowextsize_useful $bufsize
 
 _require_fs_space $SCRATCH_MNT $((filesize / 1024 * 3 * 5 / 4))
 real_blksz=$(_get_file_block_size $testdir)
diff --git a/tests/xfs/315 b/tests/xfs/315
index 9f6b39c8cc..3a618a3680 100755
--- a/tests/xfs/315
+++ b/tests/xfs/315
@@ -38,6 +38,7 @@ echo "Format filesystem"
 _scratch_mkfs >/dev/null 2>&1
 _scratch_mount >> $seqres.full
 _require_congruent_file_oplen $SCRATCH_MNT $blksz
+_require_scratch_cowextsize_useful $sz
 
 $XFS_IO_PROG -c "cowextsize $sz" $SCRATCH_MNT
 
diff --git a/tests/xfs/326 b/tests/xfs/326
index ac620fc433..a3fed8b6ac 100755
--- a/tests/xfs/326
+++ b/tests/xfs/326
@@ -55,6 +55,12 @@ $XFS_IO_PROG -c "cowextsize $sz" $SCRATCH_MNT
 # staging extent for an unshared extent and trips over the injected error.
 _require_no_xfs_always_cow
 
+# This test uses a very large cowextszhint to manipulate the COW fork to
+# contain a large unwritten extent before injecting the error.  XFS ignores
+# cowextsize when the realtime extent size is greater than 1FSB, so this test
+# cannot set up the preconditions for the test.
+_require_scratch_cowextsize_useful $sz
+
 echo "Create files"
 _pwrite_byte 0x66 0 $sz $SCRATCH_MNT/file1 >> $seqres.full
 _cp_reflink $SCRATCH_MNT/file1 $SCRATCH_MNT/file2

[PATCH 4/4] generic/303: avoid test failures on weird rt extent sizes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix this test to skip the high offset reflink test if (on XFS) the rt
extent size isn't congruent with the chosen target offset.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc         |   23 +++++++++++++++++++++++
 tests/generic/303 |    8 +++++++-
 2 files changed, 30 insertions(+), 1 deletion(-)


diff --git a/common/rc b/common/rc
index cfe765de2e..3c30a444fe 100644
--- a/common/rc
+++ b/common/rc
@@ -4488,6 +4488,29 @@ _get_file_block_size()
 	esac
 }
 
+_test_congruent_file_oplen()
+{
+	local file="$1"
+	local alloc_unit=$(_get_file_block_size "$file")
+	local oplen="$2"
+
+	case $FSTYP in
+	nfs*|cifs|9p|virtiofs|ceph|glusterfs|overlay|pvfs2)
+		# Network filesystems don't know about (or tell the client
+		# about) the underlying file allocation unit and they generally
+		# pass the file IO request to the underlying filesystem, so we
+		# don't have anything to check here.
+		return
+		;;
+	esac
+
+	if [ $alloc_unit -gt $oplen ]; then
+		return 1
+	fi
+	test $((oplen % alloc_unit)) -eq 0 || return 1
+	return 0
+}
+
 # Given a file path and a byte length of a file operation under test, ensure
 # that the length is an integer multiple of the file's allocation unit size.
 # In other words, skip the test unless (oplen ≡ alloc_unit mod 0).  This is
diff --git a/tests/generic/303 b/tests/generic/303
index 95679569e4..ef88d2357b 100755
--- a/tests/generic/303
+++ b/tests/generic/303
@@ -48,7 +48,13 @@ echo "Reflink past maximum file size in dest file (should fail)"
 _reflink_range $testdir/file1 0 $testdir/file5 4611686018427322368 $len >> $seqres.full
 
 echo "Reflink high offset to low offset"
-_reflink_range $testdir/file1 $bigoff_64k $testdir/file6 1048576 65535 >> $seqres.full
+oplen=1048576
+if _test_congruent_file_oplen $testdir $oplen; then
+	_reflink_range $testdir/file1 $bigoff_64k $testdir/file6 $oplen 65535 >> $seqres.full
+else
+	# If we can't do the ficlonerange test, fake it in the output file
+	$XFS_IO_PROG -f -c 'pwrite -S 0x61 1114110 1' $testdir/file6 >> $seqres.full
+fi
 
 echo "Reflink past source file EOF (should fail)"
 _reflink_range $testdir/file2 524288 $testdir/file7 0 1048576 >> $seqres.full

[PATCH 3/4] misc: add more congruent oplen testing

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Do more checking for file allocation operation op length congruency.
This prevents tests from failing with EINVAL when the realtime extent
size is something weird like 28k or 1GB.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/145 |    1 +
 tests/generic/147 |    1 +
 tests/generic/261 |    1 +
 tests/generic/262 |    1 +
 tests/generic/331 |    1 +
 tests/generic/353 |    3 ++-
 tests/generic/517 |    1 +
 tests/generic/657 |    1 +
 tests/generic/658 |    1 +
 tests/generic/659 |    1 +
 tests/generic/660 |    1 +
 tests/generic/663 |    1 +
 tests/generic/664 |    1 +
 tests/generic/665 |    1 +
 tests/generic/670 |    1 +
 tests/generic/672 |    1 +
 tests/xfs/1212    |    1 +
 tests/xfs/420     |    3 +++
 tests/xfs/421     |    3 +++
 19 files changed, 24 insertions(+), 1 deletion(-)


diff --git a/tests/generic/145 b/tests/generic/145
index f213f53be8..81fc5f6c2f 100755
--- a/tests/generic/145
+++ b/tests/generic/145
@@ -36,6 +36,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $TEST_DIR $blksz
 _pwrite_byte 0x61 0 $blksz $testdir/file1 >> $seqres.full
 _pwrite_byte 0x62 $blksz $blksz $testdir/file1 >> $seqres.full
 _pwrite_byte 0x63 $((blksz * 2)) $blksz $testdir/file1 >> $seqres.full
diff --git a/tests/generic/147 b/tests/generic/147
index 113800944b..bb17bb1c0b 100755
--- a/tests/generic/147
+++ b/tests/generic/147
@@ -35,6 +35,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $TEST_DIR $blksz
 _pwrite_byte 0x61 0 $blksz $testdir/file1 >> $seqres.full
 _pwrite_byte 0x62 $blksz $blksz $testdir/file1 >> $seqres.full
 _pwrite_byte 0x63 $((blksz * 2)) $blksz $testdir/file1 >> $seqres.full
diff --git a/tests/generic/261 b/tests/generic/261
index 93c1c349b1..deb360288e 100755
--- a/tests/generic/261
+++ b/tests/generic/261
@@ -29,6 +29,7 @@ testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
 
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=5
 filesize=$((blksz * nr))
 
diff --git a/tests/generic/262 b/tests/generic/262
index 46e88f8731..f296e37e02 100755
--- a/tests/generic/262
+++ b/tests/generic/262
@@ -29,6 +29,7 @@ testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
 
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=4
 filesize=$((blksz * nr))
 
diff --git a/tests/generic/331 b/tests/generic/331
index 8c665ce4fc..9b6801e16f 100755
--- a/tests/generic/331
+++ b/tests/generic/331
@@ -38,6 +38,7 @@ testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
 
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=640
 bufnr=128
 filesize=$((blksz * nr))
diff --git a/tests/generic/353 b/tests/generic/353
index 9a1471bd81..94c9ac2273 100755
--- a/tests/generic/353
+++ b/tests/generic/353
@@ -29,7 +29,8 @@ _require_xfs_io_command "fiemap"
 _scratch_mkfs > /dev/null 2>&1
 _scratch_mount
 
-blocksize=64k
+blocksize=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blocksize
 file1="$SCRATCH_MNT/file1"
 file2="$SCRATCH_MNT/file2"
 
diff --git a/tests/generic/517 b/tests/generic/517
index cf3031ed2d..229358d06b 100755
--- a/tests/generic/517
+++ b/tests/generic/517
@@ -21,6 +21,7 @@ _require_scratch_dedupe
 
 _scratch_mkfs >>$seqres.full 2>&1
 _scratch_mount
+_require_congruent_file_oplen $SCRATCH_MNT 65536
 
 # The first byte with a value of 0xae starts at an offset (512Kb + 100) which is
 # not a multiple of the block size.
diff --git a/tests/generic/657 b/tests/generic/657
index e0fecd544c..9f4673dda3 100755
--- a/tests/generic/657
+++ b/tests/generic/657
@@ -30,6 +30,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _pwrite_byte 0x61 0 $filesize $testdir/file1 >> $seqres.full
diff --git a/tests/generic/658 b/tests/generic/658
index a5cbadaaa5..e9519c25e2 100755
--- a/tests/generic/658
+++ b/tests/generic/658
@@ -31,6 +31,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _weave_reflink_regular $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
diff --git a/tests/generic/659 b/tests/generic/659
index ccc2d7950d..05436edfab 100755
--- a/tests/generic/659
+++ b/tests/generic/659
@@ -31,6 +31,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _weave_reflink_unwritten $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
diff --git a/tests/generic/660 b/tests/generic/660
index bc17dc5e59..52b0d1ea9e 100755
--- a/tests/generic/660
+++ b/tests/generic/660
@@ -31,6 +31,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _weave_reflink_holes $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
diff --git a/tests/generic/663 b/tests/generic/663
index 658a5b7004..692c77b745 100755
--- a/tests/generic/663
+++ b/tests/generic/663
@@ -32,6 +32,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _sweave_reflink_regular $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
diff --git a/tests/generic/664 b/tests/generic/664
index 3009101fdc..40fb8c6d92 100755
--- a/tests/generic/664
+++ b/tests/generic/664
@@ -34,6 +34,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _sweave_reflink_unwritten $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
diff --git a/tests/generic/665 b/tests/generic/665
index 86ba578720..ee511755e6 100755
--- a/tests/generic/665
+++ b/tests/generic/665
@@ -34,6 +34,7 @@ mkdir $testdir
 
 echo "Create the original files"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=64
 filesize=$((blksz * nr))
 _sweave_reflink_holes $blksz $nr $testdir/file1 $testdir/file3 >> $seqres.full
diff --git a/tests/generic/670 b/tests/generic/670
index 67de167405..80f9fe6d4f 100755
--- a/tests/generic/670
+++ b/tests/generic/670
@@ -31,6 +31,7 @@ mkdir $testdir
 loops=512
 nr_loops=$((loops - 1))
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 
 echo "Initialize files"
 echo >> $seqres.full
diff --git a/tests/generic/672 b/tests/generic/672
index 9e3a97ec5e..0710a04294 100755
--- a/tests/generic/672
+++ b/tests/generic/672
@@ -30,6 +30,7 @@ mkdir $testdir
 loops=1024
 nr_loops=$((loops - 1))
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 
 echo "Initialize files"
 echo >> $seqres.full
diff --git a/tests/xfs/1212 b/tests/xfs/1212
index d2292d65a2..655cd14021 100755
--- a/tests/xfs/1212
+++ b/tests/xfs/1212
@@ -32,6 +32,7 @@ _require_xfs_io_error_injection "bmap_finish_one"
 
 _scratch_mkfs >> $seqres.full
 _scratch_mount
+_require_congruent_file_oplen $SCRATCH_MNT 65536
 
 # Create original file
 _pwrite_byte 0x58 0 1m $SCRATCH_MNT/a >> $seqres.full
diff --git a/tests/xfs/420 b/tests/xfs/420
index d38772c9d9..51f87bc304 100755
--- a/tests/xfs/420
+++ b/tests/xfs/420
@@ -69,6 +69,9 @@ exercise_lseek() {
 }
 
 blksz=65536
+# Golden output encodes SEEK_HOLE/DATA output, which depends on COW only
+# happening on $blksz granularity
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=8
 filesize=$((blksz * nr))
 
diff --git a/tests/xfs/421 b/tests/xfs/421
index 027ae47c21..429333e349 100755
--- a/tests/xfs/421
+++ b/tests/xfs/421
@@ -51,6 +51,9 @@ testdir=$SCRATCH_MNT/test-$seq
 mkdir $testdir
 
 blksz=65536
+# Golden output encodes SEEK_HOLE/DATA output, which depends on COW only
+# happening on $blksz granularity
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 nr=8
 filesize=$((blksz * nr))
 

[PATCHSET v1.0 0/1] fstests: functional tests for rt quota

[Darrick J. Wong]

Hi all,

The sole patch in this series sets up functional testing for quota on
the xfs realtime device.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-quotas

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-quotas

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-quotas
---
 common/quota      |   30 ++++++++++
 tests/xfs/767     |  167 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/767.out |   41 +++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100755 tests/xfs/767
 create mode 100644 tests/xfs/767.out

[PATCH 1/1] xfs: regression testing of quota on the realtime device

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Make sure that quota accounting and enforcement work correctly for
realtime volumes on XFS.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/quota      |   30 ++++++++++
 tests/xfs/767     |  167 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/767.out |   41 +++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100755 tests/xfs/767
 create mode 100644 tests/xfs/767.out


diff --git a/common/quota b/common/quota
index 96b8d04424..f4c528c836 100644
--- a/common/quota
+++ b/common/quota
@@ -117,6 +117,36 @@ _require_xfs_quota_acct_enabled()
 	_notrun "$qtype: accounting not enabled on $fsname filesystem."
 }
 
+# Decide if the mounted filesystem supports realtime quotas.
+_require_rtquota()
+{
+	local dev="$1"
+	test -z "$dev" && dev="$TEST_DEV"
+	local rtdev="$2"
+	test -z "$rtdev" && rtdev="$TEST_RTDEV"
+
+	test "$FSTYP" = "xfs" || \
+		_notrun "Realtime quota only supported on xfs"
+
+	[ -n "$XFS_QUOTA_PROG" ] || \
+		_notrun "xfs_quota user tool not installed"
+
+	$here/src/feature -q $dev || \
+		_notrun "Installed kernel does not support XFS quotas"
+
+	test -b "$rtdev" || \
+		_notrun "No realtime device supplied?"
+
+	test "$USE_EXTERNAL" = "yes" || \
+		_notrun "Realtime requires USE_EXTERNAL='yes'"
+
+	$here/src/feature -U $dev || \
+	$here/src/feature -G $dev || \
+	$here/src/feature -P $dev || \
+		_notrun "Mounted rt filesystem does not have quotas enabled"
+
+}
+
 #
 # checks that xfs_quota can operate on foreign (non-xfs) filesystems
 # Skips check on xfs filesystems, old xfs_quota is fine there.
diff --git a/tests/xfs/767 b/tests/xfs/767
new file mode 100755
index 0000000000..f30321d7fc
--- /dev/null
+++ b/tests/xfs/767
@@ -0,0 +1,167 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 767
+#
+# Functional testing for realtime quotas.
+
+. ./common/preamble
+_begin_fstest auto quick quota realtime
+
+# Import common functions.
+. ./common/quota
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_require_test_program "punch-alternating"
+_require_scratch
+_require_user
+
+echo "Format filesystem" | tee -a $seqres.full
+_scratch_mkfs > $seqres.full
+_qmount_option 'usrquota'
+_qmount
+_require_rtquota $SCRATCH_DEV $SCRATCH_RTDEV
+
+# Make sure all our files are on the rt device
+_xfs_force_bdev realtime $SCRATCH_MNT
+chmod a+rwx $SCRATCH_MNT
+
+# Record rt geometry
+bmbt_blksz=$(_get_block_size $SCRATCH_MNT)
+file_blksz=$(_get_file_block_size $SCRATCH_MNT)
+rextsize=$((file_blksz / bmbt_blksz))
+echo "bmbt_blksz $bmbt_blksz" >> $seqres.full
+echo "file_blksz $file_blksz" >> $seqres.full
+echo "rextsize $rextsize" >> $seqres.full
+
+note() {
+	echo -e "\n$@" | tee -a $seqres.full
+}
+
+# Report on the user's block and rt block usage, soft limit, hard limit, and
+# warning count for rt volumes
+report_rtusage() {
+	local user="$1"
+	local timeout_arg="$2"
+	local print_timeout=0
+
+	test -z "$user" && user=$qa_user
+	test -n "$timeout_arg" && print_timeout=1
+
+	$XFS_QUOTA_PROG -c "quota -u -r -n -N $user" $SCRATCH_MNT | \
+		sed -e 's/ days/_days/g' >> $seqres.full
+
+	$XFS_QUOTA_PROG -c "quota -u -r -n -N $user" $SCRATCH_MNT | \
+		sed -e 's/ days/_days/g' | \
+		awk -v user=$user -v print_timeout=$print_timeout -v file_blksz=$file_blksz \
+			'{printf("%s[real] %d %d %d %d %s\n", user, $2 * 1024 / file_blksz, $3 * 1024 / file_blksz, $4 * 1024 / file_blksz, $5, print_timeout ? $6 : "---");}'
+}
+
+note "Write 128rx to root"
+$XFS_IO_PROG -f -c "pwrite 0 $((128 * file_blksz))" $SCRATCH_MNT/file1 > /dev/null
+chmod a+r $SCRATCH_MNT/file1
+sync
+report_rtusage 0
+
+note "Write 64rx to root, 4444, and 5555."
+$XFS_IO_PROG -f -c "pwrite 0 $((64 * file_blksz))" $SCRATCH_MNT/file3.5555 > /dev/null
+chown 5555 $SCRATCH_MNT/file3.5555
+$XFS_IO_PROG -f -c "pwrite 0 $((64 * file_blksz))" $SCRATCH_MNT/file3.4444 > /dev/null
+chown 4444 $SCRATCH_MNT/file3.4444
+$XFS_IO_PROG -f -c "pwrite 0 $((64 * file_blksz))" $SCRATCH_MNT/file3 > /dev/null
+sync
+report_rtusage 0
+report_rtusage 4444
+report_rtusage 5555
+
+note "Move 64rx from root to 5555"
+chown 5555 $SCRATCH_MNT/file3
+report_rtusage 0
+report_rtusage 4444
+report_rtusage 5555
+
+note "Move 64rx from 5555 to 4444"
+chown 4444 $SCRATCH_MNT/file3
+report_rtusage 0
+report_rtusage 4444
+report_rtusage 5555
+
+note "Set hard limit of 1024rx and check enforcement"
+$XFS_QUOTA_PROG -x -c "limit -u rtbhard=$((1024 * file_blksz)) $qa_user" $SCRATCH_MNT
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite 0 $((2048 * file_blksz))' $SCRATCH_MNT/file2"
+report_rtusage
+
+note "Set soft limit of 512rx and check timelimit enforcement"
+rm -f $SCRATCH_MNT/file2 $SCRATCH_MNT/file2.1
+$XFS_QUOTA_PROG -x -c "limit -u rtbsoft=$((512 * file_blksz)) rtbhard=0 $qa_user" $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c "timer -u -r -d 2" $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'state -u' $SCRATCH_MNT >> $seqres.full
+
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite 0 $((512 * file_blksz))' $SCRATCH_MNT/file2" > /dev/null
+report_rtusage
+
+overflow=$(date +%s)
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -b $file_blksz 0 $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage
+sleep 1
+echo "Try again after 1s"
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -b $file_blksz $file_blksz $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage
+sleep 2
+echo "Try again after 3s"
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -b $file_blksz $((2 * file_blksz)) $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage
+
+note "Extend time limits and warnings"
+rm -f $SCRATCH_MNT/file2 $SCRATCH_MNT/file2.1
+$XFS_QUOTA_PROG -x -c "limit -u rtbsoft=$((512 * file_blksz)) rtbhard=0 $qa_user" $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c "timer -u -r -d 49h" $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'state -u' $SCRATCH_MNT >> $seqres.full
+
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite 0 $((512 * file_blksz))' $SCRATCH_MNT/file2" > /dev/null
+report_rtusage $qa_user want_timeout
+
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -b $file_blksz 0 $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage $qa_user want_timeout
+
+$XFS_QUOTA_PROG -x -c "timer -u -r 73h $qa_user" $SCRATCH_MNT
+
+su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -b $file_blksz $file_blksz $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage $qa_user want_timeout
+
+note "Test quota applied to bmbt"
+
+# Testing quota enforcement for bmbt shape changes is tricky.  The block
+# reservation will be for enough blocks to handle the maximal btree split.
+# This is (approximately) 9 blocks no matter the size of the existing extent
+# map structure, so we set the hard limit to one more than this quantity.
+#
+# However, that means that we need to make a file of at least twice that size
+# to ensure that we create enough extent records even in the rextsize==1 case
+# where punching doesn't just create unwritten records.
+#
+# Unfortunately, it's very difficult to predict when exactly the EDQUOT will
+# come down, so we just look for the error message.
+extent_records=$(( (25 * bmbt_blksz) / 16))
+echo "extent_records $extent_records" >> $seqres.full
+
+rm -f $SCRATCH_MNT/file2
+$XFS_QUOTA_PROG -x -c "limit -u rtbsoft=0 rtbhard=0 $qa_user" $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c "limit -u bhard=$((bmbt_blksz * 10)) bsoft=0 $qa_user" $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'state -u' $SCRATCH_MNT >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x58 -b 64m 0 $((extent_records * file_blksz))" $SCRATCH_MNT/file2 > /dev/null
+sync
+chown $qa_user $SCRATCH_MNT/file2
+$here/src/punch-alternating $SCRATCH_MNT/file2 2>&1 | _filter_scratch
+
+$XFS_QUOTA_PROG -c "quota -u -r -n -N $qa_user" -c "quota -u -b -n -N $qa_user" $SCRATCH_MNT >> $seqres.full
+$XFS_IO_PROG -c "bmap -e -l -p -v" $SCRATCH_MNT/file2 >> $seqres.full
+
+# success, all done
+$XFS_QUOTA_PROG -x -c 'report -a -u' -c 'report -a -u -r' $SCRATCH_MNT >> $seqres.full
+ls -latr $SCRATCH_MNT >> $seqres.full
+status=0
+exit
diff --git a/tests/xfs/767.out b/tests/xfs/767.out
new file mode 100644
index 0000000000..bff7c0f44c
--- /dev/null
+++ b/tests/xfs/767.out
@@ -0,0 +1,41 @@
+QA output created by 767
+Format filesystem
+
+Write 128rx to root
+0[real] 128 0 0 0 ---
+
+Write 64rx to root, 4444, and 5555.
+0[real] 192 0 0 0 ---
+4444[real] 64 0 0 0 ---
+5555[real] 64 0 0 0 ---
+
+Move 64rx from root to 5555
+0[real] 128 0 0 0 ---
+4444[real] 64 0 0 0 ---
+5555[real] 128 0 0 0 ---
+
+Move 64rx from 5555 to 4444
+0[real] 128 0 0 0 ---
+4444[real] 128 0 0 0 ---
+5555[real] 64 0 0 0 ---
+
+Set hard limit of 1024rx and check enforcement
+pwrite: Disk quota exceeded
+fsgqa[real] 1024 0 1024 0 ---
+
+Set soft limit of 512rx and check timelimit enforcement
+fsgqa[real] 512 512 0 0 ---
+fsgqa[real] 513 512 0 0 ---
+Try again after 1s
+fsgqa[real] 514 512 0 0 ---
+Try again after 3s
+pwrite: Disk quota exceeded
+fsgqa[real] 514 512 0 0 ---
+
+Extend time limits and warnings
+fsgqa[real] 512 512 0 0 [--------]
+fsgqa[real] 513 512 0 0 [2_days]
+fsgqa[real] 514 512 0 0 [3_days]
+
+Test quota applied to bmbt
+SCRATCH_MNT/file2: Disk quota exceeded