[PATCHSET] fstests: random fixes for v2024.01.14

[PATCHSET] fstests: random fixes for v2024.01.14

[Darrick J. Wong]

Hi all,

Here's the usual odd fixes for fstests.  Most of these are cleanups and
bug fixes for the recently merged xfs metadump v2 testing code.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
Commits in this patchset:
 * generic/256: constrain runtime with TIME_FACTOR
 * common/xfs: simplify maximum metadump format detection
 * common/populate: always metadump full metadata blocks
 * xfs/336: fix omitted -a and -o in metadump call
 * common: refactor metadump v1 and v2 tests
 * xfs/{129,234,253,605}: disable metadump v1 testing with external devices
 * xfs/503: test metadump obfuscation, not progressbars
 * xfs/503: split copy and metadump into two tests
 * common/xfs: only pass -l in _xfs_mdrestore for v2 metadumps
 * xfs/122: fix for xfs_attr_shortform removal in 6.8
---
 common/populate           |    2 -
 common/rc                 |   10 ---
 common/xfs                |   49 +++++++++++++++-
 common/xfs_metadump_tests |  138 +++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/256         |    7 ++
 tests/xfs/122.out         |    2 +
 tests/xfs/129             |   91 ++----------------------------
 tests/xfs/1876            |   54 ++++++++++++++++++
 tests/xfs/1876.out        |    4 +
 tests/xfs/234             |   92 ++----------------------------
 tests/xfs/253             |   90 ++---------------------------
 tests/xfs/284             |    4 +
 tests/xfs/291             |   32 ++++------
 tests/xfs/336             |    2 -
 tests/xfs/432             |   31 ++--------
 tests/xfs/503             |   82 ++++-----------------------
 tests/xfs/503.out         |    6 +-
 tests/xfs/605             |   92 +-----------------------------
 18 files changed, 304 insertions(+), 484 deletions(-)
 create mode 100644 common/xfs_metadump_tests
 create mode 100755 tests/xfs/1876
 create mode 100755 tests/xfs/1876.out

[PATCH 01/10] generic/256: constrain runtime with TIME_FACTOR

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test runs 500 iterations of a "fill the fs and try to punch" test.
Hole punching can be particularly slow if, say, the filesystem is
mounted with -odiscard and the DISCARD operation takes a very long time.
In extreme cases, I can see test runtimes of 4+ hours.

Constrain the runtime of _test_full_fs_punch by establishing a deadline
of (30 seconds * TIME_FACTOR) and breaking out of the for loop if the
test goes beyond the time budget.  This keeps the runtime within the
customary 30 seconds.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/256 |    7 +++++++
 1 file changed, 7 insertions(+)


diff --git a/tests/generic/256 b/tests/generic/256
index 808a730f3a..ea6cc2938a 100755
--- a/tests/generic/256
+++ b/tests/generic/256
@@ -44,6 +44,8 @@ _test_full_fs_punch()
 	local file_len=$(( $(( $hole_len + $hole_interval )) * $iterations ))
 	local path=`dirname $file_name`
 	local hole_offset=0
+	local start_time
+	local stop_time
 
 	if [ $# -ne 5 ]
 	then
@@ -57,6 +59,9 @@ _test_full_fs_punch()
 		-c "fsync" $file_name &> /dev/null
 	chmod 666 $file_name
 
+	start_time="$(date +%s)"
+	stop_time=$(( start_time + (30 * TIME_FACTOR) ))
+
 	# All files are created as a non root user to prevent reserved blocks
 	# from being consumed.
 	_fill_fs $(( 1024 * 1024 * 1024 )) $path/fill $block_size 1 \
@@ -64,6 +69,8 @@ _test_full_fs_punch()
 
 	for (( i=0; i<$iterations; i++ ))
 	do
+		test "$(date +%s)" -ge "$stop_time" && break
+
 		# This part must not be done as root in order to
 		# test that reserved blocks are used when needed
 		_user_do "$XFS_IO_PROG -f -c \"fpunch $hole_offset $hole_len\" $file_name"

[PATCH 02/10] common/xfs: simplify maximum metadump format detection

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

xfs_metadump (aka the wrapper around xfs_db -c metadump) advertises the
-v switch to turn on v2 format in its help screen.  There's no need to
fire up xfs_db on the scratch device which will load the AGs and take
much longer.

While we're at it, reduce the amount of boilerplate in the test files by
changing the function to emit the max version supported.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs    |   10 +++++++---
 tests/xfs/129 |    3 +--
 tests/xfs/234 |    3 +--
 tests/xfs/253 |    3 +--
 tests/xfs/291 |    3 +--
 tests/xfs/432 |    3 +--
 tests/xfs/503 |    3 +--
 tests/xfs/605 |    3 +--
 8 files changed, 14 insertions(+), 17 deletions(-)


diff --git a/common/xfs b/common/xfs
index 11cfd79562..248ccefda3 100644
--- a/common/xfs
+++ b/common/xfs
@@ -713,10 +713,14 @@ _xfs_mdrestore() {
 	$XFS_MDRESTORE_PROG $options "${metadump}" "${device}"
 }
 
-_scratch_metadump_v2_supported()
+# What is the maximum metadump file format supported by xfs_metadump?
+_xfs_metadump_max_version()
 {
-	$XFS_DB_PROG -c "help metadump" $SCRATCH_DEV | \
-		grep -q "Metadump version to be used"
+	if $XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-v version'; then
+		echo 2
+	else
+		echo 1
+	fi
 }
 
 # Snapshot the metadata on the scratch device
diff --git a/tests/xfs/129 b/tests/xfs/129
index 8a817b416c..cdac2349df 100755
--- a/tests/xfs/129
+++ b/tests/xfs/129
@@ -106,8 +106,7 @@ verify_metadump_v2()
 
 _scratch_mkfs >/dev/null 2>&1
 
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 _scratch_mount
 
diff --git a/tests/xfs/234 b/tests/xfs/234
index c9bdb674ab..f4f8af6d3a 100755
--- a/tests/xfs/234
+++ b/tests/xfs/234
@@ -106,8 +106,7 @@ verify_metadump_v2()
 
 _scratch_mkfs >/dev/null 2>&1
 
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 _scratch_mount
 
diff --git a/tests/xfs/253 b/tests/xfs/253
index 8e18ddb83a..3b567999d8 100755
--- a/tests/xfs/253
+++ b/tests/xfs/253
@@ -233,8 +233,7 @@ cd $here
 
 _scratch_unmount
 
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 verify_metadump_v1 $max_md_version
 
diff --git a/tests/xfs/291 b/tests/xfs/291
index 33193eb78e..1433140821 100755
--- a/tests/xfs/291
+++ b/tests/xfs/291
@@ -92,8 +92,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
 
 # Yes they can!  Now...
 # Can xfs_metadump cope with this monster?
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 for md_version in $(seq 1 $max_md_version); do
 	version=""
diff --git a/tests/xfs/432 b/tests/xfs/432
index a215d3ce2e..7e402aa88f 100755
--- a/tests/xfs/432
+++ b/tests/xfs/432
@@ -87,8 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
 test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
 
 echo "Try to metadump, restore and check restored metadump image"
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 for md_version in $(seq 1 $max_md_version); do
 	version=""
diff --git a/tests/xfs/503 b/tests/xfs/503
index a1479eb613..8643c3d483 100755
--- a/tests/xfs/503
+++ b/tests/xfs/503
@@ -54,8 +54,7 @@ check_restored_metadump_image()
 	_destroy_loop_device $loop_dev
 }
 
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 echo "metadump and mdrestore"
 for md_version in $(seq 1 $max_md_version); do
diff --git a/tests/xfs/605 b/tests/xfs/605
index 5cbf5d2550..f2cd7aba98 100755
--- a/tests/xfs/605
+++ b/tests/xfs/605
@@ -44,8 +44,7 @@ testfile=${SCRATCH_MNT}/testfile
 echo "Format filesystem on scratch device"
 _scratch_mkfs >> $seqres.full 2>&1
 
-max_md_version=1
-_scratch_metadump_v2_supported && max_md_version=2
+max_md_version=$(_xfs_metadump_max_version)
 
 external_log=0
 if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then

[PATCH 03/10] common/populate: always metadump full metadata blocks

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Commit e443cadcea pushed the -a and -o options to the
_scratch_xfs_metadump callsites.  Unfortunately, it missed the
_xfs_metadump callsite in common/populate, so fix that now.

Fixes: e443cadcea ("common/xfs: Do not append -a and -o options to metadump")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/populate |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/common/populate b/common/populate
index cfbfd88a7f..33f2db8d4a 100644
--- a/common/populate
+++ b/common/populate
@@ -1029,7 +1029,7 @@ _scratch_populate_cached() {
 			logdev=$SCRATCH_LOGDEV
 
 		_xfs_metadump "$POPULATE_METADUMP" "$SCRATCH_DEV" "$logdev" \
-			compress
+			compress -a -o
 		;;
 	"ext2"|"ext3"|"ext4")
 		_scratch_ext4_populate $@

[PATCH 04/10] xfs/336: fix omitted -a and -o in metadump call

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Commit e443cadcea reworked _xfs_metadump to require that all callers
always pass the arguments they want -- no more defaulting to "-a -o".
Unfortunately, a few got missed.  Fix some of them now; the rest will
get cleaned up in the next patch.

Fixes: e443cadcea ("common/xfs: Do not append -a and -o options to metadump")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/284 |    4 ++--
 tests/xfs/336 |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/tests/xfs/284 b/tests/xfs/284
index 58f330035e..443c375759 100755
--- a/tests/xfs/284
+++ b/tests/xfs/284
@@ -42,12 +42,12 @@ COPY_FILE="${TEST_DIR}/${seq}_copyfile"
 # xfs_metadump should refuse to dump a mounted device
 _scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
-_scratch_xfs_metadump $METADUMP_FILE 2>&1 | filter_mounted
+_scratch_xfs_metadump $METADUMP_FILE -a -o 2>&1 | filter_mounted
 _scratch_unmount
 
 # Test restore to a mounted device
 # xfs_mdrestore should refuse to restore to a mounted device
-_scratch_xfs_metadump $METADUMP_FILE
+_scratch_xfs_metadump $METADUMP_FILE -a -o
 _scratch_mount
 _scratch_xfs_mdrestore $METADUMP_FILE 2>&1 | filter_mounted
 _scratch_unmount
diff --git a/tests/xfs/336 b/tests/xfs/336
index 43b3790cbb..3c30f1a40b 100755
--- a/tests/xfs/336
+++ b/tests/xfs/336
@@ -62,7 +62,7 @@ _scratch_cycle_mount
 
 echo "Create metadump file"
 _scratch_unmount
-_scratch_xfs_metadump $metadump_file -a
+_scratch_xfs_metadump $metadump_file -a -o
 
 # Now restore the obfuscated one back and take a look around
 echo "Restore metadump"

[PATCH 05/10] common: refactor metadump v1 and v2 tests

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
duplicate copies of the same code.

While we're at it, fix the fsck so that it includes xfs_scrub.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc                 |   10 ----
 common/xfs                |   14 +++++
 common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/129             |   90 ++-------------------------------
 tests/xfs/234             |   91 ++-------------------------------
 tests/xfs/253             |   89 ++-------------------------------
 tests/xfs/291             |   31 ++++-------
 tests/xfs/432             |   30 ++---------
 tests/xfs/503             |   60 +++-------------------
 tests/xfs/605             |   84 ++-----------------------------
 10 files changed, 181 insertions(+), 441 deletions(-)
 create mode 100644 common/xfs_metadump_tests


diff --git a/common/rc b/common/rc
index 524ffa02aa..0b69f7f54f 100644
--- a/common/rc
+++ b/common/rc
@@ -3320,15 +3320,7 @@ _check_scratch_fs()
 
     case $FSTYP in
     xfs)
-	local scratch_log="none"
-	local scratch_rt="none"
-	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
-	    scratch_log="$SCRATCH_LOGDEV"
-
-	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
-	    scratch_rt="$SCRATCH_RTDEV"
-
-	_check_xfs_filesystem $device $scratch_log $scratch_rt
+	_check_xfs_scratch_fs $device
 	;;
     udf)
 	_check_udf_filesystem $device $udf_fsize
diff --git a/common/xfs b/common/xfs
index 248ccefda3..6a48960a7f 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
 	return $?
 }
 
+_check_xfs_scratch_fs()
+{
+	local device="${1:-$SCRATCH_DEV}"
+	local scratch_log="none"
+	local scratch_rt="none"
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+	    scratch_log="$SCRATCH_LOGDEV"
+
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+	    scratch_rt="$SCRATCH_RTDEV"
+
+	_check_xfs_filesystem $device $scratch_log $scratch_rt
+}
+
 # modeled after _scratch_xfs_repair
 _test_xfs_repair()
 {
diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
new file mode 100644
index 0000000000..dd3dec1fb4
--- /dev/null
+++ b/common/xfs_metadump_tests
@@ -0,0 +1,123 @@
+#
+# XFS specific metadump testing functions.
+#
+
+# Set up environment variables for a metadump test.  Requires the test and
+# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
+_setup_verify_metadump()
+{
+	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
+	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
+	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
+
+	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
+}
+
+_cleanup_verify_metadump()
+{
+	_scratch_unmount &>> $seqres.full
+
+	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
+		losetup -d "$ldev"
+	done
+	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
+}
+
+# Create a metadump in v1 format, restore it to fs image files, then mount the
+# images and fsck them.
+_verify_metadump_v1()
+{
+	local metadump_args="$1"
+	local extra_test="$2"
+
+	local metadump_file="$XFS_METADUMP_FILE"
+	local version=""
+	local data_img="$XFS_METADUMP_IMG.data"
+	local data_loop
+
+	# Force v1 if we detect v2 support
+	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
+		version="-v 1"
+	fi
+
+	# Capture metadump, which creates metadump_file
+	_scratch_xfs_metadump $metadump_file $metadump_args $version
+
+	# Restore metadump, which creates data_img
+	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
+
+	# Create loopdev for data device so we can mount the fs
+	data_loop=$(_create_loop_device $data_img)
+
+	# Mount fs, run an extra test, fsck, and unmount
+	SCRATCH_DEV=$data_loop _scratch_mount
+	if [ -n "$extra_test" ]; then
+		SCRATCH_DEV=$data_loop $extra_test
+	fi
+	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
+	SCRATCH_DEV=$data_loop _scratch_unmount
+
+	# Tear down what we created
+	_destroy_loop_device $data_loop
+	rm -f $data_img
+}
+
+# Create a metadump in v2 format, restore it to fs image files, then mount the
+# images and fsck them.
+_verify_metadump_v2()
+{
+	local metadump_args="$1"
+	local extra_test="$2"
+
+	local metadump_file="$XFS_METADUMP_FILE"
+	local version="-v 2"
+	local data_img="$XFS_METADUMP_IMG.data"
+	local data_loop
+	local log_img=""
+	local log_loop
+
+	# Capture metadump, which creates metadump_file
+	_scratch_xfs_metadump $metadump_file $metadump_args $version
+
+	#
+	# Metadump v2 files can contain contents dumped from an external log
+	# device. Use a temporary file to hold the log device contents restored
+	# from such a metadump file.
+	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
+
+	# Restore metadump, which creates data_img and log_img
+	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
+		_scratch_xfs_mdrestore $metadump_file
+
+	# Create loopdev for data device so we can mount the fs
+	data_loop=$(_create_loop_device $data_img)
+
+	# Create loopdev for log device if we recovered anything
+	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
+
+	# Mount fs, run an extra test, fsck, and unmount
+	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
+	if [ -n "$extra_test" ]; then
+		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
+	fi
+	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
+	SCRATCH_DEV=$data_loop _scratch_unmount
+
+	# Tear down what we created
+	if [ -b "$log_loop" ]; then
+		_destroy_loop_device $log_loop
+		rm -f $log_img
+	fi
+	_destroy_loop_device $data_loop
+	rm -f $data_img
+}
+
+# Verify both metadump formats if possible
+_verify_metadumps()
+{
+	_verify_metadump_v1 "$@"
+
+	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
+		_verify_metadump_v2 "$@"
+	fi
+}
diff --git a/tests/xfs/129 b/tests/xfs/129
index cdac2349df..c3a9bcefee 100755
--- a/tests/xfs/129
+++ b/tests/xfs/129
@@ -16,98 +16,23 @@ _cleanup()
 {
     cd /
     _scratch_unmount > /dev/null 2>&1
-    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
-	    _destroy_loop_device $logdev
-    [[ -n $datadev ]] && _destroy_loop_device $datadev
-    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
-       $TEST_DIR/log-image
+    _cleanup_verify_metadump
+    rm -rf $tmp.* $testdir
 }
 
 # Import common functions.
 . ./common/filter
 . ./common/reflink
+. ./common/xfs_metadump_tests
 
 # real QA test starts here
 _supported_fs xfs
 _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
 _require_loop
 _require_scratch_reflink
-
-metadump_file=$TEST_DIR/${seq}_metadump
-
-verify_metadump_v1()
-{
-	local max_version=$1
-	local version=""
-
-	if [[ $max_version == 2 ]]; then
-		version="-v 1"
-	fi
-
-	_scratch_xfs_metadump $metadump_file $version
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
-		   _scratch_xfs_mdrestore $metadump_file
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	SCRATCH_DEV=$datadev _scratch_mount
-	SCRATCH_DEV=$datadev _scratch_unmount
-
-	logdev=$SCRATCH_LOGDEV
-	[[ -z $logdev ]] && logdev=none
-	_check_xfs_filesystem $datadev $logdev none
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
-
-verify_metadump_v2()
-{
-	version="-v 2"
-
-	_scratch_xfs_metadump $metadump_file $version
-
-	# Metadump v2 files can contain contents dumped from an external log
-	# device. Use a temporary file to hold the log device contents restored
-	# from such a metadump file.
-	slogdev=""
-	if [[ -n $SCRATCH_LOGDEV ]]; then
-		slogdev=$TEST_DIR/log-image
-	fi
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
-		   _scratch_xfs_mdrestore $metadump_file
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	logdev=${SCRATCH_LOGDEV}
-	if [[ -s $TEST_DIR/log-image ]]; then
-		logdev=$(_create_loop_device $TEST_DIR/log-image)
-	fi
-
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
-
-	[[ -z $logdev ]] && logdev=none
-	_check_xfs_filesystem $datadev $logdev none
-
-	if [[ -s $TEST_DIR/log-image ]]; then
-		_destroy_loop_device $logdev
-		logdev=""
-		rm -f $TEST_DIR/log-image
-	fi
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
+_setup_verify_metadump
 
 _scratch_mkfs >/dev/null 2>&1
-
-max_md_version=$(_xfs_metadump_max_version)
-
 _scratch_mount
 
 testdir=$SCRATCH_MNT/test-$seq
@@ -127,12 +52,7 @@ done
 _scratch_unmount
 
 echo "Create metadump file, restore it and check restored fs"
-
-verify_metadump_v1 $max_md_version
-
-if [[ $max_md_version == 2 ]]; then
-	verify_metadump_v2
-fi
+_verify_metadumps
 
 # success, all done
 status=0
diff --git a/tests/xfs/234 b/tests/xfs/234
index f4f8af6d3a..8f808c7507 100755
--- a/tests/xfs/234
+++ b/tests/xfs/234
@@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
 _cleanup()
 {
     cd /
-    _scratch_unmount > /dev/null 2>&1
-    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
-	    _destroy_loop_device $logdev
-    [[ -n $datadev ]] && _destroy_loop_device $datadev
-    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
-       $TEST_DIR/log-image
+    _cleanup_verify_metadump
+    rm -rf $tmp.* $testdir
 }
 
 # Import common functions.
 . ./common/filter
+. ./common/xfs_metadump_tests
 
 # real QA test starts here
 _supported_fs xfs
@@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
 _require_loop
 _require_xfs_scratch_rmapbt
 _require_xfs_io_command "fpunch"
-
-metadump_file=$TEST_DIR/${seq}_metadump
-
-verify_metadump_v1()
-{
-	local max_version=$1
-	local version=""
-
-	if [[ $max_version == 2 ]]; then
-		version="-v 1"
-	fi
-
-	_scratch_xfs_metadump $metadump_file $version
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
-		   _scratch_xfs_mdrestore $metadump_file
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	SCRATCH_DEV=$datadev _scratch_mount
-	SCRATCH_DEV=$datadev _scratch_unmount
-
-	logdev=$SCRATCH_LOGDEV
-	[[ -z $logdev ]] && logdev=none
-	_check_xfs_filesystem $datadev $logdev none
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
-
-verify_metadump_v2()
-{
-	version="-v 2"
-
-	_scratch_xfs_metadump $metadump_file $version
-
-	# Metadump v2 files can contain contents dumped from an external log
-	# device. Use a temporary file to hold the log device contents restored
-	# from such a metadump file.
-	slogdev=""
-	if [[ -n $SCRATCH_LOGDEV ]]; then
-		slogdev=$TEST_DIR/log-image
-	fi
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
-		   _scratch_xfs_mdrestore $metadump_file
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	logdev=${SCRATCH_LOGDEV}
-	if [[ -s $TEST_DIR/log-image ]]; then
-		logdev=$(_create_loop_device $TEST_DIR/log-image)
-	fi
-
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
-
-	[[ -z $logdev ]] && logdev=none
-	_check_xfs_filesystem $datadev $logdev none
-
-	if [[ -s $TEST_DIR/log-image ]]; then
-		_destroy_loop_device $logdev
-		logdev=""
-		rm -f $TEST_DIR/log-image
-	fi
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
+_setup_verify_metadump
 
 _scratch_mkfs >/dev/null 2>&1
-
-max_md_version=$(_xfs_metadump_max_version)
-
 _scratch_mount
 
 testdir=$SCRATCH_MNT/test-$seq
@@ -127,12 +51,7 @@ done
 _scratch_unmount
 
 echo "Create metadump file, restore it and check restored fs"
-
-verify_metadump_v1 $max_md_version
-
-if [[ $max_md_version == 2 ]]; then
-	verify_metadump_v2
-fi
+_verify_metadumps
 
 # success, all done
 status=0
diff --git a/tests/xfs/253 b/tests/xfs/253
index 3b567999d8..6623c435e5 100755
--- a/tests/xfs/253
+++ b/tests/xfs/253
@@ -26,23 +26,21 @@ _cleanup()
     cd /
     rm -f $tmp.*
     rm -rf "${OUTPUT_DIR}"
-    rm -f "${METADUMP_FILE}"
-    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
-	    _destroy_loop_device $logdev
-    [[ -n $datadev ]] && _destroy_loop_device $datadev
+    _cleanup_verify_metadump
 }
 
 # Import common functions.
 . ./common/filter
+. ./common/xfs_metadump_tests
 
 _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
 _require_test
 _require_scratch
+_setup_verify_metadump
 
 # real QA test starts here
 
 OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
-METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
 ORPHANAGE="lost+found"
 
 _supported_fs xfs
@@ -52,24 +50,7 @@ function create_file() {
 	touch $(printf "$@")
 }
 
-verify_metadump_v1()
-{
-	local max_version=$1
-	local version=""
-
-	if [[ $max_version == 2 ]]; then
-		version="-v 1"
-	fi
-
-	_scratch_xfs_metadump $METADUMP_FILE $version
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
-		   _scratch_xfs_mdrestore $METADUMP_FILE
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	SCRATCH_DEV=$datadev _scratch_mount
-
+extra_test() {
 	cd "${SCRATCH_MNT}"
 
 	# Get a listing of all the files after obfuscation
@@ -78,60 +59,6 @@ verify_metadump_v1()
 	ls -R | od -c >> $seqres.full
 
 	cd /
-
-	SCRATCH_DEV=$datadev _scratch_unmount
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
-
-verify_metadump_v2()
-{
-	version="-v 2"
-
-	_scratch_xfs_metadump $METADUMP_FILE $version
-
-	# Metadump v2 files can contain contents dumped from an external log
-	# device. Use a temporary file to hold the log device contents restored
-	# from such a metadump file.
-	slogdev=""
-	if [[ -n $SCRATCH_LOGDEV ]]; then
-		slogdev=$TEST_DIR/log-image
-	fi
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
-		   _scratch_xfs_mdrestore $METADUMP_FILE
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	logdev=${SCRATCH_LOGDEV}
-	if [[ -s $TEST_DIR/log-image ]]; then
-		logdev=$(_create_loop_device $TEST_DIR/log-image)
-	fi
-
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
-
-	cd "${SCRATCH_MNT}"
-
-	# Get a listing of all the files after obfuscation
-	echo "Metadump v2" >> $seqres.full
-	ls -R >> $seqres.full
-	ls -R | od -c >> $seqres.full
-
-	cd /
-
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
-
-	if [[ -s $TEST_DIR/log-image ]]; then
-		_destroy_loop_device $logdev
-		logdev=""
-		rm -f $TEST_DIR/log-image
-	fi
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
 }
 
 echo "Disciplyne of silence is goed."
@@ -233,13 +160,7 @@ cd $here
 
 _scratch_unmount
 
-max_md_version=$(_xfs_metadump_max_version)
-
-verify_metadump_v1 $max_md_version
-
-if [[ $max_md_version == 2 ]]; then
-	verify_metadump_v2
-fi
+_verify_metadumps '' extra_test
 
 # Finally, re-make the filesystem since to ensure we don't
 # leave a directory with duplicate entries lying around.
diff --git a/tests/xfs/291 b/tests/xfs/291
index 1433140821..c475d89ad9 100755
--- a/tests/xfs/291
+++ b/tests/xfs/291
@@ -9,11 +9,21 @@
 . ./common/preamble
 _begin_fstest auto repair metadump
 
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.*
+	_cleanup_verify_metadump
+}
+
 # Import common functions.
 . ./common/filter
+. ./common/xfs_metadump_tests
 
 _supported_fs xfs
 _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
+_setup_verify_metadump
 
 # real QA test starts here
 _require_scratch
@@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
 
 # Yes they can!  Now...
 # Can xfs_metadump cope with this monster?
-max_md_version=$(_xfs_metadump_max_version)
-
-for md_version in $(seq 1 $max_md_version); do
-	version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v $md_version"
-	fi
-
-	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
-		_fail "xfs_metadump failed"
-
-	slogdev=$SCRATCH_LOGDEV
-	if [[ -z $version || $version == "-v 1" ]]; then
-		slogdev=""
-	fi
-	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
-		   $tmp.metadump || _fail "xfs_mdrestore failed"
-	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
-		_fail "xfs_repair of metadump failed"
-done
+_verify_metadumps '-a -o'
 
 # Yes it can; success, all done
 status=0
diff --git a/tests/xfs/432 b/tests/xfs/432
index 7e402aa88f..579e1b556a 100755
--- a/tests/xfs/432
+++ b/tests/xfs/432
@@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
 _cleanup()
 {
 	cd /
-	rm -f "$tmp".* $metadump_file $metadump_img
+	rm -f "$tmp".*
+	_cleanup_verify_metadump
 }
 
 # Import common functions.
 . ./common/filter
+. ./common/xfs_metadump_tests
 
 # real QA test starts here
 _supported_fs xfs
 _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
 _require_scratch
+_setup_verify_metadump
 
 rm -f "$seqres.full"
 
@@ -54,9 +57,6 @@ echo "Format and mount"
 _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
 _scratch_mount >> "$seqres.full" 2>&1
 
-metadump_file="$TEST_DIR/meta-$seq"
-metadump_img="$TEST_DIR/img-$seq"
-rm -f $metadump_file $metadump_img
 testdir="$SCRATCH_MNT/test-$seq"
 max_fname_len=255
 blksz=$(_get_block_size $SCRATCH_MNT)
@@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
 test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
 
 echo "Try to metadump, restore and check restored metadump image"
-max_md_version=$(_xfs_metadump_max_version)
-
-for md_version in $(seq 1 $max_md_version); do
-	version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v $md_version"
-	fi
-
-	_scratch_xfs_metadump $metadump_file -a -o -w $version
-
-	slogdev=$SCRATCH_LOGDEV
-	if [[ -z $version || $version == "-v 1" ]]; then
-		slogdev=""
-	fi
-
-	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
-		   _scratch_xfs_mdrestore $metadump_file
-
-	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
-		echo "xfs_repair on restored fs returned $?"
-done
+_verify_metadumps '-a -o -w'
 
 # success, all done
 status=0
diff --git a/tests/xfs/503 b/tests/xfs/503
index 8643c3d483..ff6b344a9c 100755
--- a/tests/xfs/503
+++ b/tests/xfs/503
@@ -17,11 +17,13 @@ _cleanup()
 {
 	cd /
 	rm -rf $tmp.* $testdir
+	_cleanup_verify_metadump
 }
 
 # Import common functions.
 . ./common/filter
 . ./common/populate
+. ./common/xfs_metadump_tests
 
 testdir=$TEST_DIR/test-$seq
 
@@ -35,6 +37,7 @@ _require_scratch_nocheck
 _require_populate_commands
 _xfs_skip_online_rebuild
 _xfs_skip_offline_rebuild
+_setup_verify_metadump
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
@@ -43,66 +46,17 @@ mkdir -p $testdir
 metadump_file=$testdir/scratch.md
 copy_file=$testdir/copy.img
 
-check_restored_metadump_image()
-{
-	local image=$1
-
-	loop_dev=$(_create_loop_device $image)
-	SCRATCH_DEV=$loop_dev _scratch_mount
-	SCRATCH_DEV=$loop_dev _check_scratch_fs
-	SCRATCH_DEV=$loop_dev _scratch_unmount
-	_destroy_loop_device $loop_dev
-}
-
-max_md_version=$(_xfs_metadump_max_version)
-
 echo "metadump and mdrestore"
-for md_version in $(seq 1 $max_md_version); do
-	version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v $md_version"
-	fi
-
-	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
-	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
-	check_restored_metadump_image $TEST_DIR/image
-done
+_verify_metadumps '-a -o'
 
 echo "metadump a and mdrestore"
-for md_version in $(seq 1 $max_md_version); do
-	version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v $md_version"
-	fi
-
-	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
-	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
-	check_restored_metadump_image $TEST_DIR/image
-done
+_verify_metadumps '-a'
 
 echo "metadump g and mdrestore"
-for md_version in $(seq 1 $max_md_version); do
-	version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v $md_version"
-	fi
-
-	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
-	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
-	check_restored_metadump_image $TEST_DIR/image
-done
+_verify_metadumps '-g' >> $seqres.full
 
 echo "metadump ag and mdrestore"
-for md_version in $(seq 1 $max_md_version); do
-	version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v $md_version"
-	fi
-
-	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
-	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
-	check_restored_metadump_image $TEST_DIR/image
-done
+_verify_metadumps '-a -g' >> $seqres.full
 
 echo copy
 $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
diff --git a/tests/xfs/605 b/tests/xfs/605
index f2cd7aba98..af917f0f32 100755
--- a/tests/xfs/605
+++ b/tests/xfs/605
@@ -15,17 +15,13 @@ _cleanup()
 {
 	cd /
 	rm -r -f $tmp.*
-	_scratch_unmount > /dev/null 2>&1
-	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
-		_destroy_loop_device $logdev
-	[[ -n $datadev ]] && _destroy_loop_device $datadev
-	rm -r -f $metadump_file $TEST_DIR/data-image \
-	   $TEST_DIR/log-image
+	_cleanup_verify_metadump
 }
 
 # Import common functions.
 . ./common/dmflakey
 . ./common/inject
+. ./common/xfs_metadump_tests
 
 # real QA test starts here
 _supported_fs xfs
@@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
 _require_dm_target flakey
 _require_xfs_io_command "pwrite"
 _require_test_program "punch-alternating"
+_setup_verify_metadump
 
-metadump_file=${TEST_DIR}/${seq}.md
 testfile=${SCRATCH_MNT}/testfile
 
 echo "Format filesystem on scratch device"
 _scratch_mkfs >> $seqres.full 2>&1
 
-max_md_version=$(_xfs_metadump_max_version)
-
 external_log=0
 if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
 	external_log=1
 fi
 
-if [[ $max_md_version == 1 && $external_log == 1 ]]; then
+if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
 	_notrun "metadump v1 does not support external log device"
 fi
 
-verify_metadump_v1()
-{
-	local version=""
-	if [[ $max_md_version == 2 ]]; then
-		version="-v 1"
-	fi
-
-	_scratch_xfs_metadump $metadump_file -a -o $version
-
-	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	SCRATCH_DEV=$datadev _scratch_mount
-	SCRATCH_DEV=$datadev _check_scratch_fs
-	SCRATCH_DEV=$datadev _scratch_unmount
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
-
-verify_metadump_v2()
-{
-	local version="-v 2"
-
-	_scratch_xfs_metadump $metadump_file -a -o $version
-
-	# Metadump v2 files can contain contents dumped from an external log
-	# device. Use a temporary file to hold the log device contents restored
-	# from such a metadump file.
-	slogdev=""
-	if [[ -n $SCRATCH_LOGDEV ]]; then
-		slogdev=$TEST_DIR/log-image
-	fi
-
-	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
-		   _scratch_xfs_mdrestore $metadump_file
-
-	datadev=$(_create_loop_device $TEST_DIR/data-image)
-
-	logdev=""
-	if [[ -s $slogdev ]]; then
-		logdev=$(_create_loop_device $slogdev)
-	fi
-
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
-	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
-
-	if [[ -s $logdev ]]; then
-		_destroy_loop_device $logdev
-		logdev=""
-		rm -f $slogdev
-	fi
-
-	_destroy_loop_device $datadev
-	datadev=""
-	rm -f $TEST_DIR/data-image
-}
-
 echo "Initialize and mount filesystem on flakey device"
 _init_flakey
 _load_flakey_table $FLAKEY_ALLOW_WRITES
@@ -160,14 +93,7 @@ echo -n "Filesystem has a "
 _print_logstate
 
 echo "Create metadump file, restore it and check restored fs"
-
-if [[ $external_log == 0 ]]; then
-	verify_metadump_v1 $max_md_version
-fi
-
-if [[ $max_md_version == 2 ]]; then
-	verify_metadump_v2
-fi
+_verify_metadumps '-a -o'
 
 # Mount the fs to replay the contents from the dirty log.
 _scratch_mount

[PATCH 06/10] xfs/{129,234,253,605}: disable metadump v1 testing with external devices

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The metadump v1 format does not support capturing content from log
devices or realtime devices.  Hence it does not make sense to test these
scenarios.  Create predicates to decide if we want to test a particular
metadump format, then convert existing tests to check formats
explicitly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs_metadump_tests |   25 ++++++++++++++++++++-----
 tests/xfs/605             |    9 ---------
 2 files changed, 20 insertions(+), 14 deletions(-)


diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
index dd3dec1fb4..fdc3a1fb10 100644
--- a/common/xfs_metadump_tests
+++ b/common/xfs_metadump_tests
@@ -23,6 +23,24 @@ _cleanup_verify_metadump()
 	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
 }
 
+# Can xfs_metadump snapshot the fs metadata to a v1 metadump file?
+_scratch_xfs_can_metadump_v1()
+{
+	# metadump v1 does not support log devices
+	[ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_LOGDEV" ] && return 1
+
+	# metadump v1 does not support realtime devices
+	[ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_RTDEV" ] && return 1
+
+	return 0
+}
+
+# Can xfs_metadump snapshot the fs metadata to a v2 metadump file?
+_scratch_xfs_can_metadump_v2()
+{
+	test "$MAX_XFS_METADUMP_VERSION" -ge 2
+}
+
 # Create a metadump in v1 format, restore it to fs image files, then mount the
 # images and fsck them.
 _verify_metadump_v1()
@@ -115,9 +133,6 @@ _verify_metadump_v2()
 # Verify both metadump formats if possible
 _verify_metadumps()
 {
-	_verify_metadump_v1 "$@"
-
-	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
-		_verify_metadump_v2 "$@"
-	fi
+	_scratch_xfs_can_metadump_v1 && _verify_metadump_v1 "$@"
+	_scratch_xfs_can_metadump_v2 && _verify_metadump_v2 "$@"
 }
diff --git a/tests/xfs/605 b/tests/xfs/605
index af917f0f32..4b6ffcb2b4 100755
--- a/tests/xfs/605
+++ b/tests/xfs/605
@@ -40,15 +40,6 @@ testfile=${SCRATCH_MNT}/testfile
 echo "Format filesystem on scratch device"
 _scratch_mkfs >> $seqres.full 2>&1
 
-external_log=0
-if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
-	external_log=1
-fi
-
-if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
-	_notrun "metadump v1 does not support external log device"
-fi
-
 echo "Initialize and mount filesystem on flakey device"
 _init_flakey
 _load_flakey_table $FLAKEY_ALLOW_WRITES

[PATCH 07/10] xfs/503: test metadump obfuscation, not progressbars

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The -g switch to xfs_metadump turns on progress reporting, but nothing
in this test actually checks that it works.

The -o switch turns off obfuscation, which is much more critical to
support teams.

Change this test to check -o and -ao instead of -g or -ag.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/503     |   10 +++++-----
 tests/xfs/503.out |    4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)


diff --git a/tests/xfs/503 b/tests/xfs/503
index ff6b344a9c..79bbbdd26d 100755
--- a/tests/xfs/503
+++ b/tests/xfs/503
@@ -47,16 +47,16 @@ metadump_file=$testdir/scratch.md
 copy_file=$testdir/copy.img
 
 echo "metadump and mdrestore"
-_verify_metadumps '-a -o'
+_verify_metadumps
 
 echo "metadump a and mdrestore"
 _verify_metadumps '-a'
 
-echo "metadump g and mdrestore"
-_verify_metadumps '-g' >> $seqres.full
+echo "metadump o and mdrestore"
+_verify_metadumps '-o'
 
-echo "metadump ag and mdrestore"
-_verify_metadumps '-a -g' >> $seqres.full
+echo "metadump ao and mdrestore"
+_verify_metadumps '-a -o'
 
 echo copy
 $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
diff --git a/tests/xfs/503.out b/tests/xfs/503.out
index 496f2516e4..5e7488456d 100644
--- a/tests/xfs/503.out
+++ b/tests/xfs/503.out
@@ -2,7 +2,7 @@ QA output created by 503
 Format and populate
 metadump and mdrestore
 metadump a and mdrestore
-metadump g and mdrestore
-metadump ag and mdrestore
+metadump o and mdrestore
+metadump ao and mdrestore
 copy
 recopy

[PATCH 08/10] xfs/503: split copy and metadump into two tests

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test examines the behavior of xfs_copy and xfs_metadump.  Metadump
now supports capturing external log contents, but copy does not.  Split
the test into two to improve coverage on multidevice filesystems.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/1876     |   54 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1876.out |    4 ++++
 tests/xfs/503      |   17 +++-------------
 tests/xfs/503.out  |    2 --
 4 files changed, 61 insertions(+), 16 deletions(-)
 create mode 100755 tests/xfs/1876
 create mode 100755 tests/xfs/1876.out


diff --git a/tests/xfs/1876 b/tests/xfs/1876
new file mode 100755
index 0000000000..feeb82fca0
--- /dev/null
+++ b/tests/xfs/1876
@@ -0,0 +1,54 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+# Copyright (c) 2019 Oracle, Inc.  All Rights Reserved.
+#
+# FS QA Test No. 1876
+#
+# Populate a XFS filesystem and ensure that xfs_copy works properly.
+#
+. ./common/preamble
+_begin_fstest auto copy
+
+_register_cleanup "_cleanup" BUS
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -rf $tmp.* $testdir
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+
+testdir=$TEST_DIR/test-$seq
+
+# real QA test starts here
+_supported_fs xfs
+
+_require_xfs_copy
+_require_scratch_nocheck
+_require_populate_commands
+_xfs_skip_online_rebuild
+_xfs_skip_offline_rebuild
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+mkdir -p $testdir
+copy_file=$testdir/copy.img
+
+echo copy
+$XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
+_check_scratch_fs $copy_file
+
+echo recopy
+$XFS_COPY_PROG $copy_file $SCRATCH_DEV >> $seqres.full
+_scratch_mount
+_check_scratch_fs
+_scratch_unmount
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1876.out b/tests/xfs/1876.out
new file mode 100755
index 0000000000..0bf8e156c0
--- /dev/null
+++ b/tests/xfs/1876.out
@@ -0,0 +1,4 @@
+QA output created by 1876
+Format and populate
+copy
+recopy
diff --git a/tests/xfs/503 b/tests/xfs/503
index 79bbbdd26d..606fcbdcc4 100755
--- a/tests/xfs/503
+++ b/tests/xfs/503
@@ -4,11 +4,11 @@
 #
 # FS QA Test No. 503
 #
-# Populate a XFS filesystem and ensure that metadump, mdrestore, and copy
-# all work properly.
+# Populate a XFS filesystem and ensure that metadump and mdrestore all work
+# properly.
 #
 . ./common/preamble
-_begin_fstest auto copy metadump
+_begin_fstest auto metadump
 
 _register_cleanup "_cleanup" BUS
 
@@ -32,7 +32,6 @@ _supported_fs xfs
 
 _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
 _require_loop
-_require_xfs_copy
 _require_scratch_nocheck
 _require_populate_commands
 _xfs_skip_online_rebuild
@@ -58,16 +57,6 @@ _verify_metadumps '-o'
 echo "metadump ao and mdrestore"
 _verify_metadumps '-a -o'
 
-echo copy
-$XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
-_check_scratch_fs $copy_file
-
-echo recopy
-$XFS_COPY_PROG $copy_file $SCRATCH_DEV >> $seqres.full
-_scratch_mount
-_check_scratch_fs
-_scratch_unmount
-
 # success, all done
 status=0
 exit
diff --git a/tests/xfs/503.out b/tests/xfs/503.out
index 5e7488456d..7f3d3a5f24 100644
--- a/tests/xfs/503.out
+++ b/tests/xfs/503.out
@@ -4,5 +4,3 @@ metadump and mdrestore
 metadump a and mdrestore
 metadump o and mdrestore
 metadump ao and mdrestore
-copy
-recopy

[PATCH 09/10] common/xfs: only pass -l in _xfs_mdrestore for v2 metadumps

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

fstests has a weird history with external log devices -- prior to the
introduction of metadump v2, a dump/restore cycle would leave an
external log unaltered, and most tests worked just fine.  Were those
tests ignorant?  Or did they pass intentionally?

Either way, we don't want to pass -l to xfs_mdrestore just because we
have an external log, because that switch is new and causes regressions
when testing with xfsprogs from before 6.5.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs |   25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)


diff --git a/common/xfs b/common/xfs
index 6a48960a7f..65b509691b 100644
--- a/common/xfs
+++ b/common/xfs
@@ -689,12 +689,25 @@ _xfs_metadump() {
 	return $res
 }
 
+# What is the version of this metadump file?
+_xfs_metadumpfile_version() {
+	local file="$1"
+	local magic
+
+	magic="$($XFS_IO_PROG -c 'pread -q -v 0 4' "$file")"
+	case "$magic" in
+	"00000000:  58 4d 44 32  XMD2") echo 2;;
+	"00000000:  58 46 53 4d  XFSM") echo 1;;
+	esac
+}
+
 _xfs_mdrestore() {
 	local metadump="$1"
 	local device="$2"
 	local logdev="$3"
 	shift; shift; shift
 	local options="$@"
+	local dumpfile_ver
 
 	# If we're configured for compressed dumps and there isn't already an
 	# uncompressed dump, see if we can use DUMP_COMPRESSOR to decompress
@@ -705,8 +718,18 @@ _xfs_mdrestore() {
 		done
 	fi
 	test -r "$metadump" || return 1
+	dumpfile_ver="$(_xfs_metadumpfile_version "$metadump")"
 
-	if [ "$logdev" != "none" ]; then
+	if [ "$logdev" != "none" ] && [[ $dumpfile_ver > 1 ]]; then
+		# metadump and mdrestore began capturing and restoring the
+		# contents of external log devices with the addition of the
+		# metadump v2 format.  Hence it only makes sense to specify -l
+		# here if the dump file itself is in v2 format.
+		#
+		# With a v1 metadump, the log device is not changed by the dump
+		# and restore process.  Historically, fstests either didn't
+		# notice or _notrun themselves when external logs were in use.
+		# Don't break that for people testing with xfsprogs < 6.5.
 		options="$options -l $logdev"
 	fi
 

[PATCH 10/10] xfs/122: fix for xfs_attr_shortform removal in 6.8

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The xfs_attr_shortform struct (with multiple flexarrays) was removed in
6.8.  Check the two surviving structures (the attr sf header and entry)
instead.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/122.out |    2 ++
 1 file changed, 2 insertions(+)


diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 89f7b735b0..067a0ec76b 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -62,6 +62,8 @@ sizeof(struct xfs_agfl) = 36
 sizeof(struct xfs_attr3_leaf_hdr) = 80
 sizeof(struct xfs_attr3_leafblock) = 88
 sizeof(struct xfs_attr3_rmt_hdr) = 56
+sizeof(struct xfs_attr_sf_entry) = 3
+sizeof(struct xfs_attr_sf_hdr) = 4
 sizeof(struct xfs_attr_shortform) = 8
 sizeof(struct xfs_attrd_log_format) = 16
 sizeof(struct xfs_attri_log_format) = 40

Re: [PATCH 01/10] generic/256: constrain runtime with TIME_FACTOR

[Andrey Albershteyn]

On 2024-01-25 11:04:14, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This test runs 500 iterations of a "fill the fs and try to punch" test.
> Hole punching can be particularly slow if, say, the filesystem is
> mounted with -odiscard and the DISCARD operation takes a very long time.
> In extreme cases, I can see test runtimes of 4+ hours.
> 
> Constrain the runtime of _test_full_fs_punch by establishing a deadline
> of (30 seconds * TIME_FACTOR) and breaking out of the for loop if the
> test goes beyond the time budget.  This keeps the runtime within the
> customary 30 seconds.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/generic/256 |    7 +++++++
>  1 file changed, 7 insertions(+)
> 
> 
> diff --git a/tests/generic/256 b/tests/generic/256
> index 808a730f3a..ea6cc2938a 100755
> --- a/tests/generic/256
> +++ b/tests/generic/256
> @@ -44,6 +44,8 @@ _test_full_fs_punch()
>  	local file_len=$(( $(( $hole_len + $hole_interval )) * $iterations ))
>  	local path=`dirname $file_name`
>  	local hole_offset=0
> +	local start_time
> +	local stop_time
>  
>  	if [ $# -ne 5 ]
>  	then
> @@ -57,6 +59,9 @@ _test_full_fs_punch()
>  		-c "fsync" $file_name &> /dev/null
>  	chmod 666 $file_name
>  
> +	start_time="$(date +%s)"
> +	stop_time=$(( start_time + (30 * TIME_FACTOR) ))
> +
>  	# All files are created as a non root user to prevent reserved blocks
>  	# from being consumed.
>  	_fill_fs $(( 1024 * 1024 * 1024 )) $path/fill $block_size 1 \
> @@ -64,6 +69,8 @@ _test_full_fs_punch()
>  
>  	for (( i=0; i<$iterations; i++ ))
>  	do
> +		test "$(date +%s)" -ge "$stop_time" && break
> +
>  		# This part must not be done as root in order to
>  		# test that reserved blocks are used when needed
>  		_user_do "$XFS_IO_PROG -f -c \"fpunch $hole_offset $hole_len\" $file_name"
> 
> 

LGTM
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com>

-- 
- Andrey

Re: [PATCH 10/10] xfs/122: fix for xfs_attr_shortform removal in 6.8

[Andrey Albershteyn]

On 2024-01-25 11:06:35, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> The xfs_attr_shortform struct (with multiple flexarrays) was removed in
> 6.8.  Check the two surviving structures (the attr sf header and entry)
> instead.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/122.out |    2 ++
>  1 file changed, 2 insertions(+)
> 
> 
> diff --git a/tests/xfs/122.out b/tests/xfs/122.out
> index 89f7b735b0..067a0ec76b 100644
> --- a/tests/xfs/122.out
> +++ b/tests/xfs/122.out
> @@ -62,6 +62,8 @@ sizeof(struct xfs_agfl) = 36
>  sizeof(struct xfs_attr3_leaf_hdr) = 80
>  sizeof(struct xfs_attr3_leafblock) = 88
>  sizeof(struct xfs_attr3_rmt_hdr) = 56
> +sizeof(struct xfs_attr_sf_entry) = 3
> +sizeof(struct xfs_attr_sf_hdr) = 4
>  sizeof(struct xfs_attr_shortform) = 8
>  sizeof(struct xfs_attrd_log_format) = 16
>  sizeof(struct xfs_attri_log_format) = 40
> 
> 

LGTM
Reviewed-by: Andrey Albershteyn <aalbersh@redhat.com>

-- 
- Andrey

Re: [PATCH 01/10] generic/256: constrain runtime with TIME_FACTOR

[Christoph Hellwig]

On Thu, Jan 25, 2024 at 11:04:14AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This test runs 500 iterations of a "fill the fs and try to punch" test.
> Hole punching can be particularly slow if, say, the filesystem is
> mounted with -odiscard and the DISCARD operation takes a very long time.
> In extreme cases, I can see test runtimes of 4+ hours.
> 
> Constrain the runtime of _test_full_fs_punch by establishing a deadline
> of (30 seconds * TIME_FACTOR) and breaking out of the for loop if the
> test goes beyond the time budget.  This keeps the runtime within the
> customary 30 seconds.

Sounds good, this is can one of the slower 'quick' tests for sure:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 02/10] common/xfs: simplify maximum metadump format detection

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 03/10] common/populate: always metadump full metadata blocks

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 04/10] xfs/336: fix omitted -a and -o in metadump call

[Christoph Hellwig]

On Thu, Jan 25, 2024 at 11:05:01AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Commit e443cadcea reworked _xfs_metadump to require that all callers
> always pass the arguments they want -- no more defaulting to "-a -o".
> Unfortunately, a few got missed.  Fix some of them now; the rest will
> get cleaned up in the next patch.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 06/10] xfs/{129,234,253,605}: disable metadump v1 testing with external devices

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 07/10] xfs/503: test metadump obfuscation, not progressbars

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 08/10] xfs/503: split copy and metadump into two tests

[Christoph Hellwig]

> +++ b/tests/xfs/1876

What's it with these weird high test numbers?


Otherwise looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 09/10] common/xfs: only pass -l in _xfs_mdrestore for v2 metadumps

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 10/10] xfs/122: fix for xfs_attr_shortform removal in 6.8

[Christoph Hellwig]

On Thu, Jan 25, 2024 at 11:06:35AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> The xfs_attr_shortform struct (with multiple flexarrays) was removed in
> 6.8.  Check the two surviving structures (the attr sf header and entry)
> instead.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Although we really need to just kill this test.  Let me resubmit my
series to do the checking in libxfs using the newly shared kernel code.

Re: [PATCH 08/10] xfs/503: split copy and metadump into two tests

[Darrick J. Wong]

On Fri, Jan 26, 2024 at 05:36:17AM -0800, Christoph Hellwig wrote:
> > +++ b/tests/xfs/1876
> 
> What's it with these weird high test numbers?

Two reasons: I don't have to renumber all my new tests (and update the
spreadsheet I use to track upstreaming status) every time I rebase with
upstream; and the maintainer can git-am the patch as-is, without needing
to hand-edit the patches.

(Hmmm maybe fstests needs a git hook to detect a new test and
automatically renumber it.)

--D

> 
> Otherwise looks good:
> 
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> duplicate copies of the same code.
> 
> While we're at it, fix the fsck so that it includes xfs_scrub.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/rc                 |   10 ----
>  common/xfs                |   14 +++++
>  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/129             |   90 ++-------------------------------
>  tests/xfs/234             |   91 ++-------------------------------
>  tests/xfs/253             |   89 ++-------------------------------
>  tests/xfs/291             |   31 ++++-------
>  tests/xfs/432             |   30 ++---------
>  tests/xfs/503             |   60 +++-------------------
>  tests/xfs/605             |   84 ++-----------------------------
>  10 files changed, 181 insertions(+), 441 deletions(-)
>  create mode 100644 common/xfs_metadump_tests
> 
> 
> diff --git a/common/rc b/common/rc
> index 524ffa02aa..0b69f7f54f 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -3320,15 +3320,7 @@ _check_scratch_fs()
>  
>      case $FSTYP in
>      xfs)
> -	local scratch_log="none"
> -	local scratch_rt="none"
> -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> -	    scratch_log="$SCRATCH_LOGDEV"
> -
> -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> -	    scratch_rt="$SCRATCH_RTDEV"
> -
> -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> +	_check_xfs_scratch_fs $device
>  	;;
>      udf)
>  	_check_udf_filesystem $device $udf_fsize
> diff --git a/common/xfs b/common/xfs
> index 248ccefda3..6a48960a7f 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
>  	return $?
>  }
>  
> +_check_xfs_scratch_fs()
> +{
> +	local device="${1:-$SCRATCH_DEV}"
> +	local scratch_log="none"
> +	local scratch_rt="none"
> +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> +	    scratch_log="$SCRATCH_LOGDEV"
> +
> +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> +	    scratch_rt="$SCRATCH_RTDEV"
> +
> +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> +}
> +
>  # modeled after _scratch_xfs_repair
>  _test_xfs_repair()
>  {
> diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests

Hi Darrick,

Thanks for this improvement.

I'm wondering do we need a separated common file only for xfs metadump
helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
_xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
in common/xfs.

Thanks,
Zorro

> new file mode 100644
> index 0000000000..dd3dec1fb4
> --- /dev/null
> +++ b/common/xfs_metadump_tests
> @@ -0,0 +1,123 @@
> +#
> +# XFS specific metadump testing functions.
> +#
> +
> +# Set up environment variables for a metadump test.  Requires the test and
> +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> +_setup_verify_metadump()
> +{
> +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> +
> +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> +}
> +
> +_cleanup_verify_metadump()
> +{
> +	_scratch_unmount &>> $seqres.full
> +
> +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> +		losetup -d "$ldev"
> +	done
> +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> +}
> +
> +# Create a metadump in v1 format, restore it to fs image files, then mount the
> +# images and fsck them.
> +_verify_metadump_v1()
> +{
> +	local metadump_args="$1"
> +	local extra_test="$2"
> +
> +	local metadump_file="$XFS_METADUMP_FILE"
> +	local version=""
> +	local data_img="$XFS_METADUMP_IMG.data"
> +	local data_loop
> +
> +	# Force v1 if we detect v2 support
> +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> +		version="-v 1"
> +	fi
> +
> +	# Capture metadump, which creates metadump_file
> +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> +
> +	# Restore metadump, which creates data_img
> +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> +
> +	# Create loopdev for data device so we can mount the fs
> +	data_loop=$(_create_loop_device $data_img)
> +
> +	# Mount fs, run an extra test, fsck, and unmount
> +	SCRATCH_DEV=$data_loop _scratch_mount
> +	if [ -n "$extra_test" ]; then
> +		SCRATCH_DEV=$data_loop $extra_test
> +	fi
> +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> +	SCRATCH_DEV=$data_loop _scratch_unmount
> +
> +	# Tear down what we created
> +	_destroy_loop_device $data_loop
> +	rm -f $data_img
> +}
> +
> +# Create a metadump in v2 format, restore it to fs image files, then mount the
> +# images and fsck them.
> +_verify_metadump_v2()
> +{
> +	local metadump_args="$1"
> +	local extra_test="$2"
> +
> +	local metadump_file="$XFS_METADUMP_FILE"
> +	local version="-v 2"
> +	local data_img="$XFS_METADUMP_IMG.data"
> +	local data_loop
> +	local log_img=""
> +	local log_loop
> +
> +	# Capture metadump, which creates metadump_file
> +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> +
> +	#
> +	# Metadump v2 files can contain contents dumped from an external log
> +	# device. Use a temporary file to hold the log device contents restored
> +	# from such a metadump file.
> +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> +
> +	# Restore metadump, which creates data_img and log_img
> +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> +		_scratch_xfs_mdrestore $metadump_file
> +
> +	# Create loopdev for data device so we can mount the fs
> +	data_loop=$(_create_loop_device $data_img)
> +
> +	# Create loopdev for log device if we recovered anything
> +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> +
> +	# Mount fs, run an extra test, fsck, and unmount
> +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> +	if [ -n "$extra_test" ]; then
> +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> +	fi
> +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> +	SCRATCH_DEV=$data_loop _scratch_unmount
> +
> +	# Tear down what we created
> +	if [ -b "$log_loop" ]; then
> +		_destroy_loop_device $log_loop
> +		rm -f $log_img
> +	fi
> +	_destroy_loop_device $data_loop
> +	rm -f $data_img
> +}
> +
> +# Verify both metadump formats if possible
> +_verify_metadumps()
> +{
> +	_verify_metadump_v1 "$@"
> +
> +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> +		_verify_metadump_v2 "$@"
> +	fi
> +}
> diff --git a/tests/xfs/129 b/tests/xfs/129
> index cdac2349df..c3a9bcefee 100755
> --- a/tests/xfs/129
> +++ b/tests/xfs/129
> @@ -16,98 +16,23 @@ _cleanup()
>  {
>      cd /
>      _scratch_unmount > /dev/null 2>&1
> -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> -	    _destroy_loop_device $logdev
> -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> -       $TEST_DIR/log-image
> +    _cleanup_verify_metadump
> +    rm -rf $tmp.* $testdir
>  }
>  
>  # Import common functions.
>  . ./common/filter
>  . ./common/reflink
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_loop
>  _require_scratch_reflink
> -
> -metadump_file=$TEST_DIR/${seq}_metadump
> -
> -verify_metadump_v1()
> -{
> -	local max_version=$1
> -	local version=""
> -
> -	if [[ $max_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	logdev=$SCRATCH_LOGDEV
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	version="-v 2"
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=${SCRATCH_LOGDEV}
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $TEST_DIR/log-image
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> +_setup_verify_metadump
>  
>  _scratch_mkfs >/dev/null 2>&1
> -
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  _scratch_mount
>  
>  testdir=$SCRATCH_MNT/test-$seq
> @@ -127,12 +52,7 @@ done
>  _scratch_unmount
>  
>  echo "Create metadump file, restore it and check restored fs"
> -
> -verify_metadump_v1 $max_md_version
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/234 b/tests/xfs/234
> index f4f8af6d3a..8f808c7507 100755
> --- a/tests/xfs/234
> +++ b/tests/xfs/234
> @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
>  _cleanup()
>  {
>      cd /
> -    _scratch_unmount > /dev/null 2>&1
> -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> -	    _destroy_loop_device $logdev
> -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> -       $TEST_DIR/log-image
> +    _cleanup_verify_metadump
> +    rm -rf $tmp.* $testdir
>  }
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
> @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_loop
>  _require_xfs_scratch_rmapbt
>  _require_xfs_io_command "fpunch"
> -
> -metadump_file=$TEST_DIR/${seq}_metadump
> -
> -verify_metadump_v1()
> -{
> -	local max_version=$1
> -	local version=""
> -
> -	if [[ $max_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	logdev=$SCRATCH_LOGDEV
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	version="-v 2"
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=${SCRATCH_LOGDEV}
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $TEST_DIR/log-image
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> +_setup_verify_metadump
>  
>  _scratch_mkfs >/dev/null 2>&1
> -
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  _scratch_mount
>  
>  testdir=$SCRATCH_MNT/test-$seq
> @@ -127,12 +51,7 @@ done
>  _scratch_unmount
>  
>  echo "Create metadump file, restore it and check restored fs"
> -
> -verify_metadump_v1 $max_md_version
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/253 b/tests/xfs/253
> index 3b567999d8..6623c435e5 100755
> --- a/tests/xfs/253
> +++ b/tests/xfs/253
> @@ -26,23 +26,21 @@ _cleanup()
>      cd /
>      rm -f $tmp.*
>      rm -rf "${OUTPUT_DIR}"
> -    rm -f "${METADUMP_FILE}"
> -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> -	    _destroy_loop_device $logdev
> -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> +    _cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_test
>  _require_scratch
> +_setup_verify_metadump
>  
>  # real QA test starts here
>  
>  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
>  ORPHANAGE="lost+found"
>  
>  _supported_fs xfs
> @@ -52,24 +50,7 @@ function create_file() {
>  	touch $(printf "$@")
>  }
>  
> -verify_metadump_v1()
> -{
> -	local max_version=$1
> -	local version=""
> -
> -	if [[ $max_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $METADUMP_FILE $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> -		   _scratch_xfs_mdrestore $METADUMP_FILE
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -
> +extra_test() {
>  	cd "${SCRATCH_MNT}"
>  
>  	# Get a listing of all the files after obfuscation
> @@ -78,60 +59,6 @@ verify_metadump_v1()
>  	ls -R | od -c >> $seqres.full
>  
>  	cd /
> -
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	version="-v 2"
> -
> -	_scratch_xfs_metadump $METADUMP_FILE $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $METADUMP_FILE
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=${SCRATCH_LOGDEV}
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -
> -	cd "${SCRATCH_MNT}"
> -
> -	# Get a listing of all the files after obfuscation
> -	echo "Metadump v2" >> $seqres.full
> -	ls -R >> $seqres.full
> -	ls -R | od -c >> $seqres.full
> -
> -	cd /
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $TEST_DIR/log-image
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
>  }
>  
>  echo "Disciplyne of silence is goed."
> @@ -233,13 +160,7 @@ cd $here
>  
>  _scratch_unmount
>  
> -max_md_version=$(_xfs_metadump_max_version)
> -
> -verify_metadump_v1 $max_md_version
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps '' extra_test
>  
>  # Finally, re-make the filesystem since to ensure we don't
>  # leave a directory with duplicate entries lying around.
> diff --git a/tests/xfs/291 b/tests/xfs/291
> index 1433140821..c475d89ad9 100755
> --- a/tests/xfs/291
> +++ b/tests/xfs/291
> @@ -9,11 +9,21 @@
>  . ./common/preamble
>  _begin_fstest auto repair metadump
>  
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +	cd /
> +	rm -r -f $tmp.*
> +	_cleanup_verify_metadump
> +}
> +
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  _supported_fs xfs
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> +_setup_verify_metadump
>  
>  # real QA test starts here
>  _require_scratch
> @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
>  
>  # Yes they can!  Now...
>  # Can xfs_metadump cope with this monster?
> -max_md_version=$(_xfs_metadump_max_version)
> -
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> -		_fail "xfs_metadump failed"
> -
> -	slogdev=$SCRATCH_LOGDEV
> -	if [[ -z $version || $version == "-v 1" ]]; then
> -		slogdev=""
> -	fi
> -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> -		_fail "xfs_repair of metadump failed"
> -done
> +_verify_metadumps '-a -o'
>  
>  # Yes it can; success, all done
>  status=0
> diff --git a/tests/xfs/432 b/tests/xfs/432
> index 7e402aa88f..579e1b556a 100755
> --- a/tests/xfs/432
> +++ b/tests/xfs/432
> @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
>  _cleanup()
>  {
>  	cd /
> -	rm -f "$tmp".* $metadump_file $metadump_img
> +	rm -f "$tmp".*
> +	_cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_scratch
> +_setup_verify_metadump
>  
>  rm -f "$seqres.full"
>  
> @@ -54,9 +57,6 @@ echo "Format and mount"
>  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
>  _scratch_mount >> "$seqres.full" 2>&1
>  
> -metadump_file="$TEST_DIR/meta-$seq"
> -metadump_img="$TEST_DIR/img-$seq"
> -rm -f $metadump_file $metadump_img
>  testdir="$SCRATCH_MNT/test-$seq"
>  max_fname_len=255
>  blksz=$(_get_block_size $SCRATCH_MNT)
> @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
>  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
>  
>  echo "Try to metadump, restore and check restored metadump image"
> -max_md_version=$(_xfs_metadump_max_version)
> -
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> -
> -	slogdev=$SCRATCH_LOGDEV
> -	if [[ -z $version || $version == "-v 1" ]]; then
> -		slogdev=""
> -	fi
> -
> -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> -		echo "xfs_repair on restored fs returned $?"
> -done
> +_verify_metadumps '-a -o -w'
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/503 b/tests/xfs/503
> index 8643c3d483..ff6b344a9c 100755
> --- a/tests/xfs/503
> +++ b/tests/xfs/503
> @@ -17,11 +17,13 @@ _cleanup()
>  {
>  	cd /
>  	rm -rf $tmp.* $testdir
> +	_cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/filter
>  . ./common/populate
> +. ./common/xfs_metadump_tests
>  
>  testdir=$TEST_DIR/test-$seq
>  
> @@ -35,6 +37,7 @@ _require_scratch_nocheck
>  _require_populate_commands
>  _xfs_skip_online_rebuild
>  _xfs_skip_offline_rebuild
> +_setup_verify_metadump
>  
>  echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
> @@ -43,66 +46,17 @@ mkdir -p $testdir
>  metadump_file=$testdir/scratch.md
>  copy_file=$testdir/copy.img
>  
> -check_restored_metadump_image()
> -{
> -	local image=$1
> -
> -	loop_dev=$(_create_loop_device $image)
> -	SCRATCH_DEV=$loop_dev _scratch_mount
> -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> -	SCRATCH_DEV=$loop_dev _scratch_unmount
> -	_destroy_loop_device $loop_dev
> -}
> -
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  echo "metadump and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-a -o'
>  
>  echo "metadump a and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-a'
>  
>  echo "metadump g and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-g' >> $seqres.full
>  
>  echo "metadump ag and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-a -g' >> $seqres.full
>  
>  echo copy
>  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> diff --git a/tests/xfs/605 b/tests/xfs/605
> index f2cd7aba98..af917f0f32 100755
> --- a/tests/xfs/605
> +++ b/tests/xfs/605
> @@ -15,17 +15,13 @@ _cleanup()
>  {
>  	cd /
>  	rm -r -f $tmp.*
> -	_scratch_unmount > /dev/null 2>&1
> -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> -		_destroy_loop_device $logdev
> -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> -	rm -r -f $metadump_file $TEST_DIR/data-image \
> -	   $TEST_DIR/log-image
> +	_cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/dmflakey
>  . ./common/inject
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
> @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
>  _require_dm_target flakey
>  _require_xfs_io_command "pwrite"
>  _require_test_program "punch-alternating"
> +_setup_verify_metadump
>  
> -metadump_file=${TEST_DIR}/${seq}.md
>  testfile=${SCRATCH_MNT}/testfile
>  
>  echo "Format filesystem on scratch device"
>  _scratch_mkfs >> $seqres.full 2>&1
>  
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  external_log=0
>  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
>  	external_log=1
>  fi
>  
> -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
>  	_notrun "metadump v1 does not support external log device"
>  fi
>  
> -verify_metadump_v1()
> -{
> -	local version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -o $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -	SCRATCH_DEV=$datadev _check_scratch_fs
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	local version="-v 2"
> -
> -	_scratch_xfs_metadump $metadump_file -a -o $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=""
> -	if [[ -s $slogdev ]]; then
> -		logdev=$(_create_loop_device $slogdev)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	if [[ -s $logdev ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $slogdev
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
>  echo "Initialize and mount filesystem on flakey device"
>  _init_flakey
>  _load_flakey_table $FLAKEY_ALLOW_WRITES
> @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
>  _print_logstate
>  
>  echo "Create metadump file, restore it and check restored fs"
> -
> -if [[ $external_log == 0 ]]; then
> -	verify_metadump_v1 $max_md_version
> -fi
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps '-a -o'
>  
>  # Mount the fs to replay the contents from the dirty log.
>  _scratch_mount
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Darrick J. Wong]

On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > duplicate copies of the same code.
> > 
> > While we're at it, fix the fsck so that it includes xfs_scrub.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  common/rc                 |   10 ----
> >  common/xfs                |   14 +++++
> >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/129             |   90 ++-------------------------------
> >  tests/xfs/234             |   91 ++-------------------------------
> >  tests/xfs/253             |   89 ++-------------------------------
> >  tests/xfs/291             |   31 ++++-------
> >  tests/xfs/432             |   30 ++---------
> >  tests/xfs/503             |   60 +++-------------------
> >  tests/xfs/605             |   84 ++-----------------------------
> >  10 files changed, 181 insertions(+), 441 deletions(-)
> >  create mode 100644 common/xfs_metadump_tests
> > 
> > 
> > diff --git a/common/rc b/common/rc
> > index 524ffa02aa..0b69f7f54f 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> >  
> >      case $FSTYP in
> >      xfs)
> > -	local scratch_log="none"
> > -	local scratch_rt="none"
> > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > -	    scratch_log="$SCRATCH_LOGDEV"
> > -
> > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > -	    scratch_rt="$SCRATCH_RTDEV"
> > -
> > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > +	_check_xfs_scratch_fs $device
> >  	;;
> >      udf)
> >  	_check_udf_filesystem $device $udf_fsize
> > diff --git a/common/xfs b/common/xfs
> > index 248ccefda3..6a48960a7f 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> >  	return $?
> >  }
> >  
> > +_check_xfs_scratch_fs()
> > +{
> > +	local device="${1:-$SCRATCH_DEV}"
> > +	local scratch_log="none"
> > +	local scratch_rt="none"
> > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > +	    scratch_log="$SCRATCH_LOGDEV"
> > +
> > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > +	    scratch_rt="$SCRATCH_RTDEV"
> > +
> > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > +}
> > +
> >  # modeled after _scratch_xfs_repair
> >  _test_xfs_repair()
> >  {
> > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> 
> Hi Darrick,
> 
> Thanks for this improvement.
> 
> I'm wondering do we need a separated common file only for xfs metadump
> helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> in common/xfs.

Yes, that's certainly possible, but keep in mind that common/$FSTYP
files are getting big:

   509 fstests/common/overlay
   523 fstests/common/quota
   550 fstests/common/reflink
   638 fstests/common/log
   640 fstests/common/punch
   663 fstests/common/filter
   794 fstests/common/btrfs
   936 fstests/common/config
  1030 fstests/common/encrypt
  1162 fstests/common/populate
  1519 fstests/common/fuzzy
  1531 fstests/common/dump
  2218 fstests/common/xfs
  5437 fstests/common/rc

with common/xfs being particularly larger than most everything else.

Since the common/xfs_metadump_tests functions are shared helper
functions for testing metadump/mdrestore that are not use by most tests,
I decided that a split was appropriate both to maintain the (ha!)
cohesion of common/xfs and not add more bash parsing costs to every
single testcase.

--D

> Thanks,
> Zorro
> 
> > new file mode 100644
> > index 0000000000..dd3dec1fb4
> > --- /dev/null
> > +++ b/common/xfs_metadump_tests
> > @@ -0,0 +1,123 @@
> > +#
> > +# XFS specific metadump testing functions.
> > +#
> > +
> > +# Set up environment variables for a metadump test.  Requires the test and
> > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > +_setup_verify_metadump()
> > +{
> > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > +
> > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > +}
> > +
> > +_cleanup_verify_metadump()
> > +{
> > +	_scratch_unmount &>> $seqres.full
> > +
> > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > +		losetup -d "$ldev"
> > +	done
> > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > +}
> > +
> > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > +# images and fsck them.
> > +_verify_metadump_v1()
> > +{
> > +	local metadump_args="$1"
> > +	local extra_test="$2"
> > +
> > +	local metadump_file="$XFS_METADUMP_FILE"
> > +	local version=""
> > +	local data_img="$XFS_METADUMP_IMG.data"
> > +	local data_loop
> > +
> > +	# Force v1 if we detect v2 support
> > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > +		version="-v 1"
> > +	fi
> > +
> > +	# Capture metadump, which creates metadump_file
> > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > +
> > +	# Restore metadump, which creates data_img
> > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > +
> > +	# Create loopdev for data device so we can mount the fs
> > +	data_loop=$(_create_loop_device $data_img)
> > +
> > +	# Mount fs, run an extra test, fsck, and unmount
> > +	SCRATCH_DEV=$data_loop _scratch_mount
> > +	if [ -n "$extra_test" ]; then
> > +		SCRATCH_DEV=$data_loop $extra_test
> > +	fi
> > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > +
> > +	# Tear down what we created
> > +	_destroy_loop_device $data_loop
> > +	rm -f $data_img
> > +}
> > +
> > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > +# images and fsck them.
> > +_verify_metadump_v2()
> > +{
> > +	local metadump_args="$1"
> > +	local extra_test="$2"
> > +
> > +	local metadump_file="$XFS_METADUMP_FILE"
> > +	local version="-v 2"
> > +	local data_img="$XFS_METADUMP_IMG.data"
> > +	local data_loop
> > +	local log_img=""
> > +	local log_loop
> > +
> > +	# Capture metadump, which creates metadump_file
> > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > +
> > +	#
> > +	# Metadump v2 files can contain contents dumped from an external log
> > +	# device. Use a temporary file to hold the log device contents restored
> > +	# from such a metadump file.
> > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > +
> > +	# Restore metadump, which creates data_img and log_img
> > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > +		_scratch_xfs_mdrestore $metadump_file
> > +
> > +	# Create loopdev for data device so we can mount the fs
> > +	data_loop=$(_create_loop_device $data_img)
> > +
> > +	# Create loopdev for log device if we recovered anything
> > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > +
> > +	# Mount fs, run an extra test, fsck, and unmount
> > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > +	if [ -n "$extra_test" ]; then
> > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > +	fi
> > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > +
> > +	# Tear down what we created
> > +	if [ -b "$log_loop" ]; then
> > +		_destroy_loop_device $log_loop
> > +		rm -f $log_img
> > +	fi
> > +	_destroy_loop_device $data_loop
> > +	rm -f $data_img
> > +}
> > +
> > +# Verify both metadump formats if possible
> > +_verify_metadumps()
> > +{
> > +	_verify_metadump_v1 "$@"
> > +
> > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > +		_verify_metadump_v2 "$@"
> > +	fi
> > +}
> > diff --git a/tests/xfs/129 b/tests/xfs/129
> > index cdac2349df..c3a9bcefee 100755
> > --- a/tests/xfs/129
> > +++ b/tests/xfs/129
> > @@ -16,98 +16,23 @@ _cleanup()
> >  {
> >      cd /
> >      _scratch_unmount > /dev/null 2>&1
> > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > -	    _destroy_loop_device $logdev
> > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > -       $TEST_DIR/log-image
> > +    _cleanup_verify_metadump
> > +    rm -rf $tmp.* $testdir
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> >  . ./common/reflink
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_loop
> >  _require_scratch_reflink
> > -
> > -metadump_file=$TEST_DIR/${seq}_metadump
> > -
> > -verify_metadump_v1()
> > -{
> > -	local max_version=$1
> > -	local version=""
> > -
> > -	if [[ $max_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	logdev=$SCRATCH_LOGDEV
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	version="-v 2"
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=${SCRATCH_LOGDEV}
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $TEST_DIR/log-image
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > +_setup_verify_metadump
> >  
> >  _scratch_mkfs >/dev/null 2>&1
> > -
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  _scratch_mount
> >  
> >  testdir=$SCRATCH_MNT/test-$seq
> > @@ -127,12 +52,7 @@ done
> >  _scratch_unmount
> >  
> >  echo "Create metadump file, restore it and check restored fs"
> > -
> > -verify_metadump_v1 $max_md_version
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps
> >  
> >  # success, all done
> >  status=0
> > diff --git a/tests/xfs/234 b/tests/xfs/234
> > index f4f8af6d3a..8f808c7507 100755
> > --- a/tests/xfs/234
> > +++ b/tests/xfs/234
> > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> >  _cleanup()
> >  {
> >      cd /
> > -    _scratch_unmount > /dev/null 2>&1
> > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > -	    _destroy_loop_device $logdev
> > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > -       $TEST_DIR/log-image
> > +    _cleanup_verify_metadump
> > +    rm -rf $tmp.* $testdir
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_loop
> >  _require_xfs_scratch_rmapbt
> >  _require_xfs_io_command "fpunch"
> > -
> > -metadump_file=$TEST_DIR/${seq}_metadump
> > -
> > -verify_metadump_v1()
> > -{
> > -	local max_version=$1
> > -	local version=""
> > -
> > -	if [[ $max_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	logdev=$SCRATCH_LOGDEV
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	version="-v 2"
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=${SCRATCH_LOGDEV}
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $TEST_DIR/log-image
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > +_setup_verify_metadump
> >  
> >  _scratch_mkfs >/dev/null 2>&1
> > -
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  _scratch_mount
> >  
> >  testdir=$SCRATCH_MNT/test-$seq
> > @@ -127,12 +51,7 @@ done
> >  _scratch_unmount
> >  
> >  echo "Create metadump file, restore it and check restored fs"
> > -
> > -verify_metadump_v1 $max_md_version
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps
> >  
> >  # success, all done
> >  status=0
> > diff --git a/tests/xfs/253 b/tests/xfs/253
> > index 3b567999d8..6623c435e5 100755
> > --- a/tests/xfs/253
> > +++ b/tests/xfs/253
> > @@ -26,23 +26,21 @@ _cleanup()
> >      cd /
> >      rm -f $tmp.*
> >      rm -rf "${OUTPUT_DIR}"
> > -    rm -f "${METADUMP_FILE}"
> > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > -	    _destroy_loop_device $logdev
> > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > +    _cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_test
> >  _require_scratch
> > +_setup_verify_metadump
> >  
> >  # real QA test starts here
> >  
> >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> >  ORPHANAGE="lost+found"
> >  
> >  _supported_fs xfs
> > @@ -52,24 +50,7 @@ function create_file() {
> >  	touch $(printf "$@")
> >  }
> >  
> > -verify_metadump_v1()
> > -{
> > -	local max_version=$1
> > -	local version=""
> > -
> > -	if [[ $max_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -
> > +extra_test() {
> >  	cd "${SCRATCH_MNT}"
> >  
> >  	# Get a listing of all the files after obfuscation
> > @@ -78,60 +59,6 @@ verify_metadump_v1()
> >  	ls -R | od -c >> $seqres.full
> >  
> >  	cd /
> > -
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	version="-v 2"
> > -
> > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=${SCRATCH_LOGDEV}
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -
> > -	cd "${SCRATCH_MNT}"
> > -
> > -	# Get a listing of all the files after obfuscation
> > -	echo "Metadump v2" >> $seqres.full
> > -	ls -R >> $seqres.full
> > -	ls -R | od -c >> $seqres.full
> > -
> > -	cd /
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $TEST_DIR/log-image
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> >  }
> >  
> >  echo "Disciplyne of silence is goed."
> > @@ -233,13 +160,7 @@ cd $here
> >  
> >  _scratch_unmount
> >  
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> > -verify_metadump_v1 $max_md_version
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps '' extra_test
> >  
> >  # Finally, re-make the filesystem since to ensure we don't
> >  # leave a directory with duplicate entries lying around.
> > diff --git a/tests/xfs/291 b/tests/xfs/291
> > index 1433140821..c475d89ad9 100755
> > --- a/tests/xfs/291
> > +++ b/tests/xfs/291
> > @@ -9,11 +9,21 @@
> >  . ./common/preamble
> >  _begin_fstest auto repair metadump
> >  
> > +# Override the default cleanup function.
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -r -f $tmp.*
> > +	_cleanup_verify_metadump
> > +}
> > +
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  _supported_fs xfs
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > +_setup_verify_metadump
> >  
> >  # real QA test starts here
> >  _require_scratch
> > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> >  
> >  # Yes they can!  Now...
> >  # Can xfs_metadump cope with this monster?
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > -		_fail "xfs_metadump failed"
> > -
> > -	slogdev=$SCRATCH_LOGDEV
> > -	if [[ -z $version || $version == "-v 1" ]]; then
> > -		slogdev=""
> > -	fi
> > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > -		_fail "xfs_repair of metadump failed"
> > -done
> > +_verify_metadumps '-a -o'
> >  
> >  # Yes it can; success, all done
> >  status=0
> > diff --git a/tests/xfs/432 b/tests/xfs/432
> > index 7e402aa88f..579e1b556a 100755
> > --- a/tests/xfs/432
> > +++ b/tests/xfs/432
> > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> >  _cleanup()
> >  {
> >  	cd /
> > -	rm -f "$tmp".* $metadump_file $metadump_img
> > +	rm -f "$tmp".*
> > +	_cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_scratch
> > +_setup_verify_metadump
> >  
> >  rm -f "$seqres.full"
> >  
> > @@ -54,9 +57,6 @@ echo "Format and mount"
> >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> >  _scratch_mount >> "$seqres.full" 2>&1
> >  
> > -metadump_file="$TEST_DIR/meta-$seq"
> > -metadump_img="$TEST_DIR/img-$seq"
> > -rm -f $metadump_file $metadump_img
> >  testdir="$SCRATCH_MNT/test-$seq"
> >  max_fname_len=255
> >  blksz=$(_get_block_size $SCRATCH_MNT)
> > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> >  
> >  echo "Try to metadump, restore and check restored metadump image"
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > -
> > -	slogdev=$SCRATCH_LOGDEV
> > -	if [[ -z $version || $version == "-v 1" ]]; then
> > -		slogdev=""
> > -	fi
> > -
> > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > -		echo "xfs_repair on restored fs returned $?"
> > -done
> > +_verify_metadumps '-a -o -w'
> >  
> >  # success, all done
> >  status=0
> > diff --git a/tests/xfs/503 b/tests/xfs/503
> > index 8643c3d483..ff6b344a9c 100755
> > --- a/tests/xfs/503
> > +++ b/tests/xfs/503
> > @@ -17,11 +17,13 @@ _cleanup()
> >  {
> >  	cd /
> >  	rm -rf $tmp.* $testdir
> > +	_cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> >  . ./common/populate
> > +. ./common/xfs_metadump_tests
> >  
> >  testdir=$TEST_DIR/test-$seq
> >  
> > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> >  _require_populate_commands
> >  _xfs_skip_online_rebuild
> >  _xfs_skip_offline_rebuild
> > +_setup_verify_metadump
> >  
> >  echo "Format and populate"
> >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > @@ -43,66 +46,17 @@ mkdir -p $testdir
> >  metadump_file=$testdir/scratch.md
> >  copy_file=$testdir/copy.img
> >  
> > -check_restored_metadump_image()
> > -{
> > -	local image=$1
> > -
> > -	loop_dev=$(_create_loop_device $image)
> > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > -	_destroy_loop_device $loop_dev
> > -}
> > -
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  echo "metadump and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-a -o'
> >  
> >  echo "metadump a and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-a'
> >  
> >  echo "metadump g and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-g' >> $seqres.full
> >  
> >  echo "metadump ag and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-a -g' >> $seqres.full
> >  
> >  echo copy
> >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > diff --git a/tests/xfs/605 b/tests/xfs/605
> > index f2cd7aba98..af917f0f32 100755
> > --- a/tests/xfs/605
> > +++ b/tests/xfs/605
> > @@ -15,17 +15,13 @@ _cleanup()
> >  {
> >  	cd /
> >  	rm -r -f $tmp.*
> > -	_scratch_unmount > /dev/null 2>&1
> > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > -		_destroy_loop_device $logdev
> > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > -	   $TEST_DIR/log-image
> > +	_cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/dmflakey
> >  . ./common/inject
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> >  _require_dm_target flakey
> >  _require_xfs_io_command "pwrite"
> >  _require_test_program "punch-alternating"
> > +_setup_verify_metadump
> >  
> > -metadump_file=${TEST_DIR}/${seq}.md
> >  testfile=${SCRATCH_MNT}/testfile
> >  
> >  echo "Format filesystem on scratch device"
> >  _scratch_mkfs >> $seqres.full 2>&1
> >  
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  external_log=0
> >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> >  	external_log=1
> >  fi
> >  
> > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> >  	_notrun "metadump v1 does not support external log device"
> >  fi
> >  
> > -verify_metadump_v1()
> > -{
> > -	local version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	local version="-v 2"
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=""
> > -	if [[ -s $slogdev ]]; then
> > -		logdev=$(_create_loop_device $slogdev)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	if [[ -s $logdev ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $slogdev
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> >  echo "Initialize and mount filesystem on flakey device"
> >  _init_flakey
> >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> >  _print_logstate
> >  
> >  echo "Create metadump file, restore it and check restored fs"
> > -
> > -if [[ $external_log == 0 ]]; then
> > -	verify_metadump_v1 $max_md_version
> > -fi
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps '-a -o'
> >  
> >  # Mount the fs to replay the contents from the dirty log.
> >  _scratch_mount
> > 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > duplicate copies of the same code.
> > > 
> > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > 
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  common/rc                 |   10 ----
> > >  common/xfs                |   14 +++++
> > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > >  tests/xfs/129             |   90 ++-------------------------------
> > >  tests/xfs/234             |   91 ++-------------------------------
> > >  tests/xfs/253             |   89 ++-------------------------------
> > >  tests/xfs/291             |   31 ++++-------
> > >  tests/xfs/432             |   30 ++---------
> > >  tests/xfs/503             |   60 +++-------------------
> > >  tests/xfs/605             |   84 ++-----------------------------
> > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > >  create mode 100644 common/xfs_metadump_tests
> > > 
> > > 
> > > diff --git a/common/rc b/common/rc
> > > index 524ffa02aa..0b69f7f54f 100644
> > > --- a/common/rc
> > > +++ b/common/rc
> > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > >  
> > >      case $FSTYP in
> > >      xfs)
> > > -	local scratch_log="none"
> > > -	local scratch_rt="none"
> > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > -
> > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > -
> > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > +	_check_xfs_scratch_fs $device
> > >  	;;
> > >      udf)
> > >  	_check_udf_filesystem $device $udf_fsize
> > > diff --git a/common/xfs b/common/xfs
> > > index 248ccefda3..6a48960a7f 100644
> > > --- a/common/xfs
> > > +++ b/common/xfs
> > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > >  	return $?
> > >  }
> > >  
> > > +_check_xfs_scratch_fs()
> > > +{
> > > +	local device="${1:-$SCRATCH_DEV}"
> > > +	local scratch_log="none"
> > > +	local scratch_rt="none"
> > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > +
> > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > +
> > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > +}
> > > +
> > >  # modeled after _scratch_xfs_repair
> > >  _test_xfs_repair()
> > >  {
> > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > 
> > Hi Darrick,
> > 
> > Thanks for this improvement.
> > 
> > I'm wondering do we need a separated common file only for xfs metadump
> > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > in common/xfs.
> 
> Yes, that's certainly possible, but keep in mind that common/$FSTYP
> files are getting big:
> 
>    509 fstests/common/overlay
>    523 fstests/common/quota
>    550 fstests/common/reflink
>    638 fstests/common/log
>    640 fstests/common/punch
>    663 fstests/common/filter
>    794 fstests/common/btrfs
>    936 fstests/common/config
>   1030 fstests/common/encrypt
>   1162 fstests/common/populate
>   1519 fstests/common/fuzzy
>   1531 fstests/common/dump
>   2218 fstests/common/xfs
>   5437 fstests/common/rc
> 
> with common/xfs being particularly larger than most everything else.

Haha, maybe we'll have a common/xfs/ directory in one day :)

> 
> Since the common/xfs_metadump_tests functions are shared helper
> functions for testing metadump/mdrestore that are not use by most tests,
> I decided that a split was appropriate both to maintain the (ha!)
> cohesion of common/xfs and not add more bash parsing costs to every
> single testcase.

OK, a split makes sense, but I have 3 questions:
1) Will you move all metadump helpers from common/xfs to this new file?
2) Can we call it common/metadump? (Not sure if any other fs has metadump
   things:)
3) Or move to common/dump directly? (looks not proper ;-)

Thanks,
Zorro

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > > new file mode 100644
> > > index 0000000000..dd3dec1fb4
> > > --- /dev/null
> > > +++ b/common/xfs_metadump_tests
> > > @@ -0,0 +1,123 @@
> > > +#
> > > +# XFS specific metadump testing functions.
> > > +#
> > > +
> > > +# Set up environment variables for a metadump test.  Requires the test and
> > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > +_setup_verify_metadump()
> > > +{
> > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > +
> > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > +}
> > > +
> > > +_cleanup_verify_metadump()
> > > +{
> > > +	_scratch_unmount &>> $seqres.full
> > > +
> > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > +		losetup -d "$ldev"
> > > +	done
> > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > +}
> > > +
> > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > +# images and fsck them.
> > > +_verify_metadump_v1()
> > > +{
> > > +	local metadump_args="$1"
> > > +	local extra_test="$2"
> > > +
> > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > +	local version=""
> > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > +	local data_loop
> > > +
> > > +	# Force v1 if we detect v2 support
> > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > +		version="-v 1"
> > > +	fi
> > > +
> > > +	# Capture metadump, which creates metadump_file
> > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > +
> > > +	# Restore metadump, which creates data_img
> > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > +
> > > +	# Create loopdev for data device so we can mount the fs
> > > +	data_loop=$(_create_loop_device $data_img)
> > > +
> > > +	# Mount fs, run an extra test, fsck, and unmount
> > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > +	if [ -n "$extra_test" ]; then
> > > +		SCRATCH_DEV=$data_loop $extra_test
> > > +	fi
> > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > +
> > > +	# Tear down what we created
> > > +	_destroy_loop_device $data_loop
> > > +	rm -f $data_img
> > > +}
> > > +
> > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > +# images and fsck them.
> > > +_verify_metadump_v2()
> > > +{
> > > +	local metadump_args="$1"
> > > +	local extra_test="$2"
> > > +
> > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > +	local version="-v 2"
> > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > +	local data_loop
> > > +	local log_img=""
> > > +	local log_loop
> > > +
> > > +	# Capture metadump, which creates metadump_file
> > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > +
> > > +	#
> > > +	# Metadump v2 files can contain contents dumped from an external log
> > > +	# device. Use a temporary file to hold the log device contents restored
> > > +	# from such a metadump file.
> > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > +
> > > +	# Restore metadump, which creates data_img and log_img
> > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > +		_scratch_xfs_mdrestore $metadump_file
> > > +
> > > +	# Create loopdev for data device so we can mount the fs
> > > +	data_loop=$(_create_loop_device $data_img)
> > > +
> > > +	# Create loopdev for log device if we recovered anything
> > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > +
> > > +	# Mount fs, run an extra test, fsck, and unmount
> > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > +	if [ -n "$extra_test" ]; then
> > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > +	fi
> > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > +
> > > +	# Tear down what we created
> > > +	if [ -b "$log_loop" ]; then
> > > +		_destroy_loop_device $log_loop
> > > +		rm -f $log_img
> > > +	fi
> > > +	_destroy_loop_device $data_loop
> > > +	rm -f $data_img
> > > +}
> > > +
> > > +# Verify both metadump formats if possible
> > > +_verify_metadumps()
> > > +{
> > > +	_verify_metadump_v1 "$@"
> > > +
> > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > +		_verify_metadump_v2 "$@"
> > > +	fi
> > > +}
> > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > index cdac2349df..c3a9bcefee 100755
> > > --- a/tests/xfs/129
> > > +++ b/tests/xfs/129
> > > @@ -16,98 +16,23 @@ _cleanup()
> > >  {
> > >      cd /
> > >      _scratch_unmount > /dev/null 2>&1
> > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > -	    _destroy_loop_device $logdev
> > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > -       $TEST_DIR/log-image
> > > +    _cleanup_verify_metadump
> > > +    rm -rf $tmp.* $testdir
> > >  }
> > >  
> > >  # Import common functions.
> > >  . ./common/filter
> > >  . ./common/reflink
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  # real QA test starts here
> > >  _supported_fs xfs
> > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > >  _require_loop
> > >  _require_scratch_reflink
> > > -
> > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > -
> > > -verify_metadump_v1()
> > > -{
> > > -	local max_version=$1
> > > -	local version=""
> > > -
> > > -	if [[ $max_version == 2 ]]; then
> > > -		version="-v 1"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file $version
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > -		   _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > -
> > > -	logdev=$SCRATCH_LOGDEV
> > > -	[[ -z $logdev ]] && logdev=none
> > > -	_check_xfs_filesystem $datadev $logdev none
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > -
> > > -verify_metadump_v2()
> > > -{
> > > -	version="-v 2"
> > > -
> > > -	_scratch_xfs_metadump $metadump_file $version
> > > -
> > > -	# Metadump v2 files can contain contents dumped from an external log
> > > -	# device. Use a temporary file to hold the log device contents restored
> > > -	# from such a metadump file.
> > > -	slogdev=""
> > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > -		slogdev=$TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > -		   _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	logdev=${SCRATCH_LOGDEV}
> > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > -
> > > -	[[ -z $logdev ]] && logdev=none
> > > -	_check_xfs_filesystem $datadev $logdev none
> > > -
> > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > -		_destroy_loop_device $logdev
> > > -		logdev=""
> > > -		rm -f $TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > +_setup_verify_metadump
> > >  
> > >  _scratch_mkfs >/dev/null 2>&1
> > > -
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > >  _scratch_mount
> > >  
> > >  testdir=$SCRATCH_MNT/test-$seq
> > > @@ -127,12 +52,7 @@ done
> > >  _scratch_unmount
> > >  
> > >  echo "Create metadump file, restore it and check restored fs"
> > > -
> > > -verify_metadump_v1 $max_md_version
> > > -
> > > -if [[ $max_md_version == 2 ]]; then
> > > -	verify_metadump_v2
> > > -fi
> > > +_verify_metadumps
> > >  
> > >  # success, all done
> > >  status=0
> > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > index f4f8af6d3a..8f808c7507 100755
> > > --- a/tests/xfs/234
> > > +++ b/tests/xfs/234
> > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > >  _cleanup()
> > >  {
> > >      cd /
> > > -    _scratch_unmount > /dev/null 2>&1
> > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > -	    _destroy_loop_device $logdev
> > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > -       $TEST_DIR/log-image
> > > +    _cleanup_verify_metadump
> > > +    rm -rf $tmp.* $testdir
> > >  }
> > >  
> > >  # Import common functions.
> > >  . ./common/filter
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  # real QA test starts here
> > >  _supported_fs xfs
> > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > >  _require_loop
> > >  _require_xfs_scratch_rmapbt
> > >  _require_xfs_io_command "fpunch"
> > > -
> > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > -
> > > -verify_metadump_v1()
> > > -{
> > > -	local max_version=$1
> > > -	local version=""
> > > -
> > > -	if [[ $max_version == 2 ]]; then
> > > -		version="-v 1"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file $version
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > -		   _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > -
> > > -	logdev=$SCRATCH_LOGDEV
> > > -	[[ -z $logdev ]] && logdev=none
> > > -	_check_xfs_filesystem $datadev $logdev none
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > -
> > > -verify_metadump_v2()
> > > -{
> > > -	version="-v 2"
> > > -
> > > -	_scratch_xfs_metadump $metadump_file $version
> > > -
> > > -	# Metadump v2 files can contain contents dumped from an external log
> > > -	# device. Use a temporary file to hold the log device contents restored
> > > -	# from such a metadump file.
> > > -	slogdev=""
> > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > -		slogdev=$TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > -		   _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	logdev=${SCRATCH_LOGDEV}
> > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > -
> > > -	[[ -z $logdev ]] && logdev=none
> > > -	_check_xfs_filesystem $datadev $logdev none
> > > -
> > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > -		_destroy_loop_device $logdev
> > > -		logdev=""
> > > -		rm -f $TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > +_setup_verify_metadump
> > >  
> > >  _scratch_mkfs >/dev/null 2>&1
> > > -
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > >  _scratch_mount
> > >  
> > >  testdir=$SCRATCH_MNT/test-$seq
> > > @@ -127,12 +51,7 @@ done
> > >  _scratch_unmount
> > >  
> > >  echo "Create metadump file, restore it and check restored fs"
> > > -
> > > -verify_metadump_v1 $max_md_version
> > > -
> > > -if [[ $max_md_version == 2 ]]; then
> > > -	verify_metadump_v2
> > > -fi
> > > +_verify_metadumps
> > >  
> > >  # success, all done
> > >  status=0
> > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > index 3b567999d8..6623c435e5 100755
> > > --- a/tests/xfs/253
> > > +++ b/tests/xfs/253
> > > @@ -26,23 +26,21 @@ _cleanup()
> > >      cd /
> > >      rm -f $tmp.*
> > >      rm -rf "${OUTPUT_DIR}"
> > > -    rm -f "${METADUMP_FILE}"
> > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > -	    _destroy_loop_device $logdev
> > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > +    _cleanup_verify_metadump
> > >  }
> > >  
> > >  # Import common functions.
> > >  . ./common/filter
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > >  _require_test
> > >  _require_scratch
> > > +_setup_verify_metadump
> > >  
> > >  # real QA test starts here
> > >  
> > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > >  ORPHANAGE="lost+found"
> > >  
> > >  _supported_fs xfs
> > > @@ -52,24 +50,7 @@ function create_file() {
> > >  	touch $(printf "$@")
> > >  }
> > >  
> > > -verify_metadump_v1()
> > > -{
> > > -	local max_version=$1
> > > -	local version=""
> > > -
> > > -	if [[ $max_version == 2 ]]; then
> > > -		version="-v 1"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > -
> > > +extra_test() {
> > >  	cd "${SCRATCH_MNT}"
> > >  
> > >  	# Get a listing of all the files after obfuscation
> > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > >  	ls -R | od -c >> $seqres.full
> > >  
> > >  	cd /
> > > -
> > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > -
> > > -verify_metadump_v2()
> > > -{
> > > -	version="-v 2"
> > > -
> > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > -
> > > -	# Metadump v2 files can contain contents dumped from an external log
> > > -	# device. Use a temporary file to hold the log device contents restored
> > > -	# from such a metadump file.
> > > -	slogdev=""
> > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > -		slogdev=$TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	logdev=${SCRATCH_LOGDEV}
> > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > -
> > > -	cd "${SCRATCH_MNT}"
> > > -
> > > -	# Get a listing of all the files after obfuscation
> > > -	echo "Metadump v2" >> $seqres.full
> > > -	ls -R >> $seqres.full
> > > -	ls -R | od -c >> $seqres.full
> > > -
> > > -	cd /
> > > -
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > -
> > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > -		_destroy_loop_device $logdev
> > > -		logdev=""
> > > -		rm -f $TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > >  }
> > >  
> > >  echo "Disciplyne of silence is goed."
> > > @@ -233,13 +160,7 @@ cd $here
> > >  
> > >  _scratch_unmount
> > >  
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > > -verify_metadump_v1 $max_md_version
> > > -
> > > -if [[ $max_md_version == 2 ]]; then
> > > -	verify_metadump_v2
> > > -fi
> > > +_verify_metadumps '' extra_test
> > >  
> > >  # Finally, re-make the filesystem since to ensure we don't
> > >  # leave a directory with duplicate entries lying around.
> > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > index 1433140821..c475d89ad9 100755
> > > --- a/tests/xfs/291
> > > +++ b/tests/xfs/291
> > > @@ -9,11 +9,21 @@
> > >  . ./common/preamble
> > >  _begin_fstest auto repair metadump
> > >  
> > > +# Override the default cleanup function.
> > > +_cleanup()
> > > +{
> > > +	cd /
> > > +	rm -r -f $tmp.*
> > > +	_cleanup_verify_metadump
> > > +}
> > > +
> > >  # Import common functions.
> > >  . ./common/filter
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  _supported_fs xfs
> > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > +_setup_verify_metadump
> > >  
> > >  # real QA test starts here
> > >  _require_scratch
> > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > >  
> > >  # Yes they can!  Now...
> > >  # Can xfs_metadump cope with this monster?
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > > -for md_version in $(seq 1 $max_md_version); do
> > > -	version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v $md_version"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > -		_fail "xfs_metadump failed"
> > > -
> > > -	slogdev=$SCRATCH_LOGDEV
> > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > -		slogdev=""
> > > -	fi
> > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > -		_fail "xfs_repair of metadump failed"
> > > -done
> > > +_verify_metadumps '-a -o'
> > >  
> > >  # Yes it can; success, all done
> > >  status=0
> > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > index 7e402aa88f..579e1b556a 100755
> > > --- a/tests/xfs/432
> > > +++ b/tests/xfs/432
> > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > >  _cleanup()
> > >  {
> > >  	cd /
> > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > +	rm -f "$tmp".*
> > > +	_cleanup_verify_metadump
> > >  }
> > >  
> > >  # Import common functions.
> > >  . ./common/filter
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  # real QA test starts here
> > >  _supported_fs xfs
> > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > >  _require_scratch
> > > +_setup_verify_metadump
> > >  
> > >  rm -f "$seqres.full"
> > >  
> > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > >  _scratch_mount >> "$seqres.full" 2>&1
> > >  
> > > -metadump_file="$TEST_DIR/meta-$seq"
> > > -metadump_img="$TEST_DIR/img-$seq"
> > > -rm -f $metadump_file $metadump_img
> > >  testdir="$SCRATCH_MNT/test-$seq"
> > >  max_fname_len=255
> > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > >  
> > >  echo "Try to metadump, restore and check restored metadump image"
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > > -for md_version in $(seq 1 $max_md_version); do
> > > -	version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v $md_version"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > -
> > > -	slogdev=$SCRATCH_LOGDEV
> > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > -		slogdev=""
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > -		   _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > -		echo "xfs_repair on restored fs returned $?"
> > > -done
> > > +_verify_metadumps '-a -o -w'
> > >  
> > >  # success, all done
> > >  status=0
> > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > index 8643c3d483..ff6b344a9c 100755
> > > --- a/tests/xfs/503
> > > +++ b/tests/xfs/503
> > > @@ -17,11 +17,13 @@ _cleanup()
> > >  {
> > >  	cd /
> > >  	rm -rf $tmp.* $testdir
> > > +	_cleanup_verify_metadump
> > >  }
> > >  
> > >  # Import common functions.
> > >  . ./common/filter
> > >  . ./common/populate
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  testdir=$TEST_DIR/test-$seq
> > >  
> > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > >  _require_populate_commands
> > >  _xfs_skip_online_rebuild
> > >  _xfs_skip_offline_rebuild
> > > +_setup_verify_metadump
> > >  
> > >  echo "Format and populate"
> > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > >  metadump_file=$testdir/scratch.md
> > >  copy_file=$testdir/copy.img
> > >  
> > > -check_restored_metadump_image()
> > > -{
> > > -	local image=$1
> > > -
> > > -	loop_dev=$(_create_loop_device $image)
> > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > -	_destroy_loop_device $loop_dev
> > > -}
> > > -
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > >  echo "metadump and mdrestore"
> > > -for md_version in $(seq 1 $max_md_version); do
> > > -	version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v $md_version"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > -	check_restored_metadump_image $TEST_DIR/image
> > > -done
> > > +_verify_metadumps '-a -o'
> > >  
> > >  echo "metadump a and mdrestore"
> > > -for md_version in $(seq 1 $max_md_version); do
> > > -	version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v $md_version"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > -	check_restored_metadump_image $TEST_DIR/image
> > > -done
> > > +_verify_metadumps '-a'
> > >  
> > >  echo "metadump g and mdrestore"
> > > -for md_version in $(seq 1 $max_md_version); do
> > > -	version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v $md_version"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > -	check_restored_metadump_image $TEST_DIR/image
> > > -done
> > > +_verify_metadumps '-g' >> $seqres.full
> > >  
> > >  echo "metadump ag and mdrestore"
> > > -for md_version in $(seq 1 $max_md_version); do
> > > -	version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v $md_version"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > -	check_restored_metadump_image $TEST_DIR/image
> > > -done
> > > +_verify_metadumps '-a -g' >> $seqres.full
> > >  
> > >  echo copy
> > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > index f2cd7aba98..af917f0f32 100755
> > > --- a/tests/xfs/605
> > > +++ b/tests/xfs/605
> > > @@ -15,17 +15,13 @@ _cleanup()
> > >  {
> > >  	cd /
> > >  	rm -r -f $tmp.*
> > > -	_scratch_unmount > /dev/null 2>&1
> > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > -		_destroy_loop_device $logdev
> > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > -	   $TEST_DIR/log-image
> > > +	_cleanup_verify_metadump
> > >  }
> > >  
> > >  # Import common functions.
> > >  . ./common/dmflakey
> > >  . ./common/inject
> > > +. ./common/xfs_metadump_tests
> > >  
> > >  # real QA test starts here
> > >  _supported_fs xfs
> > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > >  _require_dm_target flakey
> > >  _require_xfs_io_command "pwrite"
> > >  _require_test_program "punch-alternating"
> > > +_setup_verify_metadump
> > >  
> > > -metadump_file=${TEST_DIR}/${seq}.md
> > >  testfile=${SCRATCH_MNT}/testfile
> > >  
> > >  echo "Format filesystem on scratch device"
> > >  _scratch_mkfs >> $seqres.full 2>&1
> > >  
> > > -max_md_version=$(_xfs_metadump_max_version)
> > > -
> > >  external_log=0
> > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > >  	external_log=1
> > >  fi
> > >  
> > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > >  	_notrun "metadump v1 does not support external log device"
> > >  fi
> > >  
> > > -verify_metadump_v1()
> > > -{
> > > -	local version=""
> > > -	if [[ $max_md_version == 2 ]]; then
> > > -		version="-v 1"
> > > -	fi
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > -
> > > -verify_metadump_v2()
> > > -{
> > > -	local version="-v 2"
> > > -
> > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > -
> > > -	# Metadump v2 files can contain contents dumped from an external log
> > > -	# device. Use a temporary file to hold the log device contents restored
> > > -	# from such a metadump file.
> > > -	slogdev=""
> > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > -		slogdev=$TEST_DIR/log-image
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > -		   _scratch_xfs_mdrestore $metadump_file
> > > -
> > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > -
> > > -	logdev=""
> > > -	if [[ -s $slogdev ]]; then
> > > -		logdev=$(_create_loop_device $slogdev)
> > > -	fi
> > > -
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > -
> > > -	if [[ -s $logdev ]]; then
> > > -		_destroy_loop_device $logdev
> > > -		logdev=""
> > > -		rm -f $slogdev
> > > -	fi
> > > -
> > > -	_destroy_loop_device $datadev
> > > -	datadev=""
> > > -	rm -f $TEST_DIR/data-image
> > > -}
> > > -
> > >  echo "Initialize and mount filesystem on flakey device"
> > >  _init_flakey
> > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > >  _print_logstate
> > >  
> > >  echo "Create metadump file, restore it and check restored fs"
> > > -
> > > -if [[ $external_log == 0 ]]; then
> > > -	verify_metadump_v1 $max_md_version
> > > -fi
> > > -
> > > -if [[ $max_md_version == 2 ]]; then
> > > -	verify_metadump_v2
> > > -fi
> > > +_verify_metadumps '-a -o'
> > >  
> > >  # Mount the fs to replay the contents from the dirty log.
> > >  _scratch_mount
> > > 
> > 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Darrick J. Wong]

On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > duplicate copies of the same code.
> > > > 
> > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > 
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > ---
> > > >  common/rc                 |   10 ----
> > > >  common/xfs                |   14 +++++
> > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > >  tests/xfs/129             |   90 ++-------------------------------
> > > >  tests/xfs/234             |   91 ++-------------------------------
> > > >  tests/xfs/253             |   89 ++-------------------------------
> > > >  tests/xfs/291             |   31 ++++-------
> > > >  tests/xfs/432             |   30 ++---------
> > > >  tests/xfs/503             |   60 +++-------------------
> > > >  tests/xfs/605             |   84 ++-----------------------------
> > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > >  create mode 100644 common/xfs_metadump_tests
> > > > 
> > > > 
> > > > diff --git a/common/rc b/common/rc
> > > > index 524ffa02aa..0b69f7f54f 100644
> > > > --- a/common/rc
> > > > +++ b/common/rc
> > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > >  
> > > >      case $FSTYP in
> > > >      xfs)
> > > > -	local scratch_log="none"
> > > > -	local scratch_rt="none"
> > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > -
> > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > -
> > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > +	_check_xfs_scratch_fs $device
> > > >  	;;
> > > >      udf)
> > > >  	_check_udf_filesystem $device $udf_fsize
> > > > diff --git a/common/xfs b/common/xfs
> > > > index 248ccefda3..6a48960a7f 100644
> > > > --- a/common/xfs
> > > > +++ b/common/xfs
> > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > >  	return $?
> > > >  }
> > > >  
> > > > +_check_xfs_scratch_fs()
> > > > +{
> > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > +	local scratch_log="none"
> > > > +	local scratch_rt="none"
> > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > +
> > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > +
> > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > +}
> > > > +
> > > >  # modeled after _scratch_xfs_repair
> > > >  _test_xfs_repair()
> > > >  {
> > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > 
> > > Hi Darrick,
> > > 
> > > Thanks for this improvement.
> > > 
> > > I'm wondering do we need a separated common file only for xfs metadump
> > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > in common/xfs.
> > 
> > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > files are getting big:
> > 
> >    509 fstests/common/overlay
> >    523 fstests/common/quota
> >    550 fstests/common/reflink
> >    638 fstests/common/log
> >    640 fstests/common/punch
> >    663 fstests/common/filter
> >    794 fstests/common/btrfs
> >    936 fstests/common/config
> >   1030 fstests/common/encrypt
> >   1162 fstests/common/populate
> >   1519 fstests/common/fuzzy
> >   1531 fstests/common/dump
> >   2218 fstests/common/xfs
> >   5437 fstests/common/rc
> > 
> > with common/xfs being particularly larger than most everything else.
> 
> Haha, maybe we'll have a common/xfs/ directory in one day :)
> 
> > 
> > Since the common/xfs_metadump_tests functions are shared helper
> > functions for testing metadump/mdrestore that are not use by most tests,
> > I decided that a split was appropriate both to maintain the (ha!)
> > cohesion of common/xfs and not add more bash parsing costs to every
> > single testcase.
> 
> OK, a split makes sense, but I have 3 questions:
> 1) Will you move all metadump helpers from common/xfs to this new file?

I don't really want to, because that's another patch and would require
careful auditing of all the tests to find the ones that want to use
metadump but aren't themselves functional tests of metadump.

IOWs, this new file really is for shared metadump functional testing and
not much else.

> 2) Can we call it common/metadump? (Not sure if any other fs has metadump
>    things:)

Yes, e2image does this for ext*.

> 3) Or move to common/dump directly? (looks not proper ;-)

dump != metadump; one is for all the files in the fs and none of the
non-file metadata; the other is for metadata and none of the files.

--D

> Thanks,
> Zorro
> 
> > 
> > --D
> > 
> > > Thanks,
> > > Zorro
> > > 
> > > > new file mode 100644
> > > > index 0000000000..dd3dec1fb4
> > > > --- /dev/null
> > > > +++ b/common/xfs_metadump_tests
> > > > @@ -0,0 +1,123 @@
> > > > +#
> > > > +# XFS specific metadump testing functions.
> > > > +#
> > > > +
> > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > +_setup_verify_metadump()
> > > > +{
> > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > +
> > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > +}
> > > > +
> > > > +_cleanup_verify_metadump()
> > > > +{
> > > > +	_scratch_unmount &>> $seqres.full
> > > > +
> > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > +		losetup -d "$ldev"
> > > > +	done
> > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > +}
> > > > +
> > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > +# images and fsck them.
> > > > +_verify_metadump_v1()
> > > > +{
> > > > +	local metadump_args="$1"
> > > > +	local extra_test="$2"
> > > > +
> > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > +	local version=""
> > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > +	local data_loop
> > > > +
> > > > +	# Force v1 if we detect v2 support
> > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > +		version="-v 1"
> > > > +	fi
> > > > +
> > > > +	# Capture metadump, which creates metadump_file
> > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > +
> > > > +	# Restore metadump, which creates data_img
> > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > +
> > > > +	# Create loopdev for data device so we can mount the fs
> > > > +	data_loop=$(_create_loop_device $data_img)
> > > > +
> > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > +	if [ -n "$extra_test" ]; then
> > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > +	fi
> > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > +
> > > > +	# Tear down what we created
> > > > +	_destroy_loop_device $data_loop
> > > > +	rm -f $data_img
> > > > +}
> > > > +
> > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > +# images and fsck them.
> > > > +_verify_metadump_v2()
> > > > +{
> > > > +	local metadump_args="$1"
> > > > +	local extra_test="$2"
> > > > +
> > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > +	local version="-v 2"
> > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > +	local data_loop
> > > > +	local log_img=""
> > > > +	local log_loop
> > > > +
> > > > +	# Capture metadump, which creates metadump_file
> > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > +
> > > > +	#
> > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > +	# from such a metadump file.
> > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > +
> > > > +	# Restore metadump, which creates data_img and log_img
> > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > +
> > > > +	# Create loopdev for data device so we can mount the fs
> > > > +	data_loop=$(_create_loop_device $data_img)
> > > > +
> > > > +	# Create loopdev for log device if we recovered anything
> > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > +
> > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > +	if [ -n "$extra_test" ]; then
> > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > +	fi
> > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > +
> > > > +	# Tear down what we created
> > > > +	if [ -b "$log_loop" ]; then
> > > > +		_destroy_loop_device $log_loop
> > > > +		rm -f $log_img
> > > > +	fi
> > > > +	_destroy_loop_device $data_loop
> > > > +	rm -f $data_img
> > > > +}
> > > > +
> > > > +# Verify both metadump formats if possible
> > > > +_verify_metadumps()
> > > > +{
> > > > +	_verify_metadump_v1 "$@"
> > > > +
> > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > +		_verify_metadump_v2 "$@"
> > > > +	fi
> > > > +}
> > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > index cdac2349df..c3a9bcefee 100755
> > > > --- a/tests/xfs/129
> > > > +++ b/tests/xfs/129
> > > > @@ -16,98 +16,23 @@ _cleanup()
> > > >  {
> > > >      cd /
> > > >      _scratch_unmount > /dev/null 2>&1
> > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > -	    _destroy_loop_device $logdev
> > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > -       $TEST_DIR/log-image
> > > > +    _cleanup_verify_metadump
> > > > +    rm -rf $tmp.* $testdir
> > > >  }
> > > >  
> > > >  # Import common functions.
> > > >  . ./common/filter
> > > >  . ./common/reflink
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  # real QA test starts here
> > > >  _supported_fs xfs
> > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > >  _require_loop
> > > >  _require_scratch_reflink
> > > > -
> > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > -
> > > > -verify_metadump_v1()
> > > > -{
> > > > -	local max_version=$1
> > > > -	local version=""
> > > > -
> > > > -	if [[ $max_version == 2 ]]; then
> > > > -		version="-v 1"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > -
> > > > -	logdev=$SCRATCH_LOGDEV
> > > > -	[[ -z $logdev ]] && logdev=none
> > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > -
> > > > -verify_metadump_v2()
> > > > -{
> > > > -	version="-v 2"
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > -
> > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > -	# from such a metadump file.
> > > > -	slogdev=""
> > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > -		slogdev=$TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	logdev=${SCRATCH_LOGDEV}
> > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > -
> > > > -	[[ -z $logdev ]] && logdev=none
> > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > -
> > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > -		_destroy_loop_device $logdev
> > > > -		logdev=""
> > > > -		rm -f $TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > +_setup_verify_metadump
> > > >  
> > > >  _scratch_mkfs >/dev/null 2>&1
> > > > -
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > >  _scratch_mount
> > > >  
> > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > @@ -127,12 +52,7 @@ done
> > > >  _scratch_unmount
> > > >  
> > > >  echo "Create metadump file, restore it and check restored fs"
> > > > -
> > > > -verify_metadump_v1 $max_md_version
> > > > -
> > > > -if [[ $max_md_version == 2 ]]; then
> > > > -	verify_metadump_v2
> > > > -fi
> > > > +_verify_metadumps
> > > >  
> > > >  # success, all done
> > > >  status=0
> > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > index f4f8af6d3a..8f808c7507 100755
> > > > --- a/tests/xfs/234
> > > > +++ b/tests/xfs/234
> > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > >  _cleanup()
> > > >  {
> > > >      cd /
> > > > -    _scratch_unmount > /dev/null 2>&1
> > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > -	    _destroy_loop_device $logdev
> > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > -       $TEST_DIR/log-image
> > > > +    _cleanup_verify_metadump
> > > > +    rm -rf $tmp.* $testdir
> > > >  }
> > > >  
> > > >  # Import common functions.
> > > >  . ./common/filter
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  # real QA test starts here
> > > >  _supported_fs xfs
> > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > >  _require_loop
> > > >  _require_xfs_scratch_rmapbt
> > > >  _require_xfs_io_command "fpunch"
> > > > -
> > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > -
> > > > -verify_metadump_v1()
> > > > -{
> > > > -	local max_version=$1
> > > > -	local version=""
> > > > -
> > > > -	if [[ $max_version == 2 ]]; then
> > > > -		version="-v 1"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > -
> > > > -	logdev=$SCRATCH_LOGDEV
> > > > -	[[ -z $logdev ]] && logdev=none
> > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > -
> > > > -verify_metadump_v2()
> > > > -{
> > > > -	version="-v 2"
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > -
> > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > -	# from such a metadump file.
> > > > -	slogdev=""
> > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > -		slogdev=$TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	logdev=${SCRATCH_LOGDEV}
> > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > -
> > > > -	[[ -z $logdev ]] && logdev=none
> > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > -
> > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > -		_destroy_loop_device $logdev
> > > > -		logdev=""
> > > > -		rm -f $TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > +_setup_verify_metadump
> > > >  
> > > >  _scratch_mkfs >/dev/null 2>&1
> > > > -
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > >  _scratch_mount
> > > >  
> > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > @@ -127,12 +51,7 @@ done
> > > >  _scratch_unmount
> > > >  
> > > >  echo "Create metadump file, restore it and check restored fs"
> > > > -
> > > > -verify_metadump_v1 $max_md_version
> > > > -
> > > > -if [[ $max_md_version == 2 ]]; then
> > > > -	verify_metadump_v2
> > > > -fi
> > > > +_verify_metadumps
> > > >  
> > > >  # success, all done
> > > >  status=0
> > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > index 3b567999d8..6623c435e5 100755
> > > > --- a/tests/xfs/253
> > > > +++ b/tests/xfs/253
> > > > @@ -26,23 +26,21 @@ _cleanup()
> > > >      cd /
> > > >      rm -f $tmp.*
> > > >      rm -rf "${OUTPUT_DIR}"
> > > > -    rm -f "${METADUMP_FILE}"
> > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > -	    _destroy_loop_device $logdev
> > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > +    _cleanup_verify_metadump
> > > >  }
> > > >  
> > > >  # Import common functions.
> > > >  . ./common/filter
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > >  _require_test
> > > >  _require_scratch
> > > > +_setup_verify_metadump
> > > >  
> > > >  # real QA test starts here
> > > >  
> > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > >  ORPHANAGE="lost+found"
> > > >  
> > > >  _supported_fs xfs
> > > > @@ -52,24 +50,7 @@ function create_file() {
> > > >  	touch $(printf "$@")
> > > >  }
> > > >  
> > > > -verify_metadump_v1()
> > > > -{
> > > > -	local max_version=$1
> > > > -	local version=""
> > > > -
> > > > -	if [[ $max_version == 2 ]]; then
> > > > -		version="-v 1"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > -
> > > > +extra_test() {
> > > >  	cd "${SCRATCH_MNT}"
> > > >  
> > > >  	# Get a listing of all the files after obfuscation
> > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > >  	ls -R | od -c >> $seqres.full
> > > >  
> > > >  	cd /
> > > > -
> > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > -
> > > > -verify_metadump_v2()
> > > > -{
> > > > -	version="-v 2"
> > > > -
> > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > -
> > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > -	# from such a metadump file.
> > > > -	slogdev=""
> > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > -		slogdev=$TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	logdev=${SCRATCH_LOGDEV}
> > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > -
> > > > -	cd "${SCRATCH_MNT}"
> > > > -
> > > > -	# Get a listing of all the files after obfuscation
> > > > -	echo "Metadump v2" >> $seqres.full
> > > > -	ls -R >> $seqres.full
> > > > -	ls -R | od -c >> $seqres.full
> > > > -
> > > > -	cd /
> > > > -
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > -
> > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > -		_destroy_loop_device $logdev
> > > > -		logdev=""
> > > > -		rm -f $TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > >  }
> > > >  
> > > >  echo "Disciplyne of silence is goed."
> > > > @@ -233,13 +160,7 @@ cd $here
> > > >  
> > > >  _scratch_unmount
> > > >  
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > > -verify_metadump_v1 $max_md_version
> > > > -
> > > > -if [[ $max_md_version == 2 ]]; then
> > > > -	verify_metadump_v2
> > > > -fi
> > > > +_verify_metadumps '' extra_test
> > > >  
> > > >  # Finally, re-make the filesystem since to ensure we don't
> > > >  # leave a directory with duplicate entries lying around.
> > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > index 1433140821..c475d89ad9 100755
> > > > --- a/tests/xfs/291
> > > > +++ b/tests/xfs/291
> > > > @@ -9,11 +9,21 @@
> > > >  . ./common/preamble
> > > >  _begin_fstest auto repair metadump
> > > >  
> > > > +# Override the default cleanup function.
> > > > +_cleanup()
> > > > +{
> > > > +	cd /
> > > > +	rm -r -f $tmp.*
> > > > +	_cleanup_verify_metadump
> > > > +}
> > > > +
> > > >  # Import common functions.
> > > >  . ./common/filter
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  _supported_fs xfs
> > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > +_setup_verify_metadump
> > > >  
> > > >  # real QA test starts here
> > > >  _require_scratch
> > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > >  
> > > >  # Yes they can!  Now...
> > > >  # Can xfs_metadump cope with this monster?
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > > -for md_version in $(seq 1 $max_md_version); do
> > > > -	version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v $md_version"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > -		_fail "xfs_metadump failed"
> > > > -
> > > > -	slogdev=$SCRATCH_LOGDEV
> > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > -		slogdev=""
> > > > -	fi
> > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > -		_fail "xfs_repair of metadump failed"
> > > > -done
> > > > +_verify_metadumps '-a -o'
> > > >  
> > > >  # Yes it can; success, all done
> > > >  status=0
> > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > index 7e402aa88f..579e1b556a 100755
> > > > --- a/tests/xfs/432
> > > > +++ b/tests/xfs/432
> > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > >  _cleanup()
> > > >  {
> > > >  	cd /
> > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > +	rm -f "$tmp".*
> > > > +	_cleanup_verify_metadump
> > > >  }
> > > >  
> > > >  # Import common functions.
> > > >  . ./common/filter
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  # real QA test starts here
> > > >  _supported_fs xfs
> > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > >  _require_scratch
> > > > +_setup_verify_metadump
> > > >  
> > > >  rm -f "$seqres.full"
> > > >  
> > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > >  
> > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > -rm -f $metadump_file $metadump_img
> > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > >  max_fname_len=255
> > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > >  
> > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > > -for md_version in $(seq 1 $max_md_version); do
> > > > -	version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v $md_version"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > -
> > > > -	slogdev=$SCRATCH_LOGDEV
> > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > -		slogdev=""
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > -		echo "xfs_repair on restored fs returned $?"
> > > > -done
> > > > +_verify_metadumps '-a -o -w'
> > > >  
> > > >  # success, all done
> > > >  status=0
> > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > index 8643c3d483..ff6b344a9c 100755
> > > > --- a/tests/xfs/503
> > > > +++ b/tests/xfs/503
> > > > @@ -17,11 +17,13 @@ _cleanup()
> > > >  {
> > > >  	cd /
> > > >  	rm -rf $tmp.* $testdir
> > > > +	_cleanup_verify_metadump
> > > >  }
> > > >  
> > > >  # Import common functions.
> > > >  . ./common/filter
> > > >  . ./common/populate
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  testdir=$TEST_DIR/test-$seq
> > > >  
> > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > >  _require_populate_commands
> > > >  _xfs_skip_online_rebuild
> > > >  _xfs_skip_offline_rebuild
> > > > +_setup_verify_metadump
> > > >  
> > > >  echo "Format and populate"
> > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > >  metadump_file=$testdir/scratch.md
> > > >  copy_file=$testdir/copy.img
> > > >  
> > > > -check_restored_metadump_image()
> > > > -{
> > > > -	local image=$1
> > > > -
> > > > -	loop_dev=$(_create_loop_device $image)
> > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > -	_destroy_loop_device $loop_dev
> > > > -}
> > > > -
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > >  echo "metadump and mdrestore"
> > > > -for md_version in $(seq 1 $max_md_version); do
> > > > -	version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v $md_version"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > -done
> > > > +_verify_metadumps '-a -o'
> > > >  
> > > >  echo "metadump a and mdrestore"
> > > > -for md_version in $(seq 1 $max_md_version); do
> > > > -	version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v $md_version"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > -done
> > > > +_verify_metadumps '-a'
> > > >  
> > > >  echo "metadump g and mdrestore"
> > > > -for md_version in $(seq 1 $max_md_version); do
> > > > -	version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v $md_version"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > -done
> > > > +_verify_metadumps '-g' >> $seqres.full
> > > >  
> > > >  echo "metadump ag and mdrestore"
> > > > -for md_version in $(seq 1 $max_md_version); do
> > > > -	version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v $md_version"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > -done
> > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > >  
> > > >  echo copy
> > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > index f2cd7aba98..af917f0f32 100755
> > > > --- a/tests/xfs/605
> > > > +++ b/tests/xfs/605
> > > > @@ -15,17 +15,13 @@ _cleanup()
> > > >  {
> > > >  	cd /
> > > >  	rm -r -f $tmp.*
> > > > -	_scratch_unmount > /dev/null 2>&1
> > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > -		_destroy_loop_device $logdev
> > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > -	   $TEST_DIR/log-image
> > > > +	_cleanup_verify_metadump
> > > >  }
> > > >  
> > > >  # Import common functions.
> > > >  . ./common/dmflakey
> > > >  . ./common/inject
> > > > +. ./common/xfs_metadump_tests
> > > >  
> > > >  # real QA test starts here
> > > >  _supported_fs xfs
> > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > >  _require_dm_target flakey
> > > >  _require_xfs_io_command "pwrite"
> > > >  _require_test_program "punch-alternating"
> > > > +_setup_verify_metadump
> > > >  
> > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > >  testfile=${SCRATCH_MNT}/testfile
> > > >  
> > > >  echo "Format filesystem on scratch device"
> > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > >  
> > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > -
> > > >  external_log=0
> > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > >  	external_log=1
> > > >  fi
> > > >  
> > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > >  	_notrun "metadump v1 does not support external log device"
> > > >  fi
> > > >  
> > > > -verify_metadump_v1()
> > > > -{
> > > > -	local version=""
> > > > -	if [[ $max_md_version == 2 ]]; then
> > > > -		version="-v 1"
> > > > -	fi
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > -
> > > > -verify_metadump_v2()
> > > > -{
> > > > -	local version="-v 2"
> > > > -
> > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > -
> > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > -	# from such a metadump file.
> > > > -	slogdev=""
> > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > -		slogdev=$TEST_DIR/log-image
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > -
> > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > -
> > > > -	logdev=""
> > > > -	if [[ -s $slogdev ]]; then
> > > > -		logdev=$(_create_loop_device $slogdev)
> > > > -	fi
> > > > -
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > -
> > > > -	if [[ -s $logdev ]]; then
> > > > -		_destroy_loop_device $logdev
> > > > -		logdev=""
> > > > -		rm -f $slogdev
> > > > -	fi
> > > > -
> > > > -	_destroy_loop_device $datadev
> > > > -	datadev=""
> > > > -	rm -f $TEST_DIR/data-image
> > > > -}
> > > > -
> > > >  echo "Initialize and mount filesystem on flakey device"
> > > >  _init_flakey
> > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > >  _print_logstate
> > > >  
> > > >  echo "Create metadump file, restore it and check restored fs"
> > > > -
> > > > -if [[ $external_log == 0 ]]; then
> > > > -	verify_metadump_v1 $max_md_version
> > > > -fi
> > > > -
> > > > -if [[ $max_md_version == 2 ]]; then
> > > > -	verify_metadump_v2
> > > > -fi
> > > > +_verify_metadumps '-a -o'
> > > >  
> > > >  # Mount the fs to replay the contents from the dirty log.
> > > >  _scratch_mount
> > > > 
> > > 
> > 
> 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Mon, Jan 29, 2024 at 05:32:07PM -0800, Darrick J. Wong wrote:
> On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> > On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > 
> > > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > > duplicate copies of the same code.
> > > > > 
> > > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > > 
> > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > ---
> > > > >  common/rc                 |   10 ----
> > > > >  common/xfs                |   14 +++++
> > > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > > >  tests/xfs/129             |   90 ++-------------------------------
> > > > >  tests/xfs/234             |   91 ++-------------------------------
> > > > >  tests/xfs/253             |   89 ++-------------------------------
> > > > >  tests/xfs/291             |   31 ++++-------
> > > > >  tests/xfs/432             |   30 ++---------
> > > > >  tests/xfs/503             |   60 +++-------------------
> > > > >  tests/xfs/605             |   84 ++-----------------------------
> > > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > > >  create mode 100644 common/xfs_metadump_tests
> > > > > 
> > > > > 
> > > > > diff --git a/common/rc b/common/rc
> > > > > index 524ffa02aa..0b69f7f54f 100644
> > > > > --- a/common/rc
> > > > > +++ b/common/rc
> > > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > > >  
> > > > >      case $FSTYP in
> > > > >      xfs)
> > > > > -	local scratch_log="none"
> > > > > -	local scratch_rt="none"
> > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > > -
> > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > > -
> > > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > +	_check_xfs_scratch_fs $device
> > > > >  	;;
> > > > >      udf)
> > > > >  	_check_udf_filesystem $device $udf_fsize
> > > > > diff --git a/common/xfs b/common/xfs
> > > > > index 248ccefda3..6a48960a7f 100644
> > > > > --- a/common/xfs
> > > > > +++ b/common/xfs
> > > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > > >  	return $?
> > > > >  }
> > > > >  
> > > > > +_check_xfs_scratch_fs()
> > > > > +{
> > > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > > +	local scratch_log="none"
> > > > > +	local scratch_rt="none"
> > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > > +
> > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > > +
> > > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > +}
> > > > > +
> > > > >  # modeled after _scratch_xfs_repair
> > > > >  _test_xfs_repair()
> > > > >  {
> > > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > > 
> > > > Hi Darrick,
> > > > 
> > > > Thanks for this improvement.
> > > > 
> > > > I'm wondering do we need a separated common file only for xfs metadump
> > > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > > in common/xfs.
> > > 
> > > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > > files are getting big:
> > > 
> > >    509 fstests/common/overlay
> > >    523 fstests/common/quota
> > >    550 fstests/common/reflink
> > >    638 fstests/common/log
> > >    640 fstests/common/punch
> > >    663 fstests/common/filter
> > >    794 fstests/common/btrfs
> > >    936 fstests/common/config
> > >   1030 fstests/common/encrypt
> > >   1162 fstests/common/populate
> > >   1519 fstests/common/fuzzy
> > >   1531 fstests/common/dump
> > >   2218 fstests/common/xfs
> > >   5437 fstests/common/rc
> > > 
> > > with common/xfs being particularly larger than most everything else.
> > 
> > Haha, maybe we'll have a common/xfs/ directory in one day :)
> > 
> > > 
> > > Since the common/xfs_metadump_tests functions are shared helper
> > > functions for testing metadump/mdrestore that are not use by most tests,
> > > I decided that a split was appropriate both to maintain the (ha!)
> > > cohesion of common/xfs and not add more bash parsing costs to every
> > > single testcase.
> > 
> > OK, a split makes sense, but I have 3 questions:
> > 1) Will you move all metadump helpers from common/xfs to this new file?
> 
> I don't really want to, because that's another patch and would require
> careful auditing of all the tests to find the ones that want to use
> metadump but aren't themselves functional tests of metadump.
> 
> IOWs, this new file really is for shared metadump functional testing and
> not much else.

OK, I think we can care about this part step by step in the future.

> 
> > 2) Can we call it common/metadump? (Not sure if any other fs has metadump
> >    things:)
> 
> Yes, e2image does this for ext*.

OK, I'll rename this file to common/metadump, and change other patches to
souce the new name when I merge this patchset. Is that good to you?

Thanks,
Zorro

> 
> > 3) Or move to common/dump directly? (looks not proper ;-)
> 
> dump != metadump; one is for all the files in the fs and none of the
> non-file metadata; the other is for metadata and none of the files.
> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > > 
> > > --D
> > > 
> > > > Thanks,
> > > > Zorro
> > > > 
> > > > > new file mode 100644
> > > > > index 0000000000..dd3dec1fb4
> > > > > --- /dev/null
> > > > > +++ b/common/xfs_metadump_tests
> > > > > @@ -0,0 +1,123 @@
> > > > > +#
> > > > > +# XFS specific metadump testing functions.
> > > > > +#
> > > > > +
> > > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > > +_setup_verify_metadump()
> > > > > +{
> > > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > > +
> > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > +}
> > > > > +
> > > > > +_cleanup_verify_metadump()
> > > > > +{
> > > > > +	_scratch_unmount &>> $seqres.full
> > > > > +
> > > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > > +		losetup -d "$ldev"
> > > > > +	done
> > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > +}
> > > > > +
> > > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > > +# images and fsck them.
> > > > > +_verify_metadump_v1()
> > > > > +{
> > > > > +	local metadump_args="$1"
> > > > > +	local extra_test="$2"
> > > > > +
> > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > +	local version=""
> > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > +	local data_loop
> > > > > +
> > > > > +	# Force v1 if we detect v2 support
> > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > > +		version="-v 1"
> > > > > +	fi
> > > > > +
> > > > > +	# Capture metadump, which creates metadump_file
> > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > +
> > > > > +	# Restore metadump, which creates data_img
> > > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > > +
> > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > +
> > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > > +	if [ -n "$extra_test" ]; then
> > > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > > +	fi
> > > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > +
> > > > > +	# Tear down what we created
> > > > > +	_destroy_loop_device $data_loop
> > > > > +	rm -f $data_img
> > > > > +}
> > > > > +
> > > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > > +# images and fsck them.
> > > > > +_verify_metadump_v2()
> > > > > +{
> > > > > +	local metadump_args="$1"
> > > > > +	local extra_test="$2"
> > > > > +
> > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > +	local version="-v 2"
> > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > +	local data_loop
> > > > > +	local log_img=""
> > > > > +	local log_loop
> > > > > +
> > > > > +	# Capture metadump, which creates metadump_file
> > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > +
> > > > > +	#
> > > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > > +	# from such a metadump file.
> > > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > > +
> > > > > +	# Restore metadump, which creates data_img and log_img
> > > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > > +
> > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > +
> > > > > +	# Create loopdev for log device if we recovered anything
> > > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > > +
> > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > > +	if [ -n "$extra_test" ]; then
> > > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > > +	fi
> > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > +
> > > > > +	# Tear down what we created
> > > > > +	if [ -b "$log_loop" ]; then
> > > > > +		_destroy_loop_device $log_loop
> > > > > +		rm -f $log_img
> > > > > +	fi
> > > > > +	_destroy_loop_device $data_loop
> > > > > +	rm -f $data_img
> > > > > +}
> > > > > +
> > > > > +# Verify both metadump formats if possible
> > > > > +_verify_metadumps()
> > > > > +{
> > > > > +	_verify_metadump_v1 "$@"
> > > > > +
> > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > > +		_verify_metadump_v2 "$@"
> > > > > +	fi
> > > > > +}
> > > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > > index cdac2349df..c3a9bcefee 100755
> > > > > --- a/tests/xfs/129
> > > > > +++ b/tests/xfs/129
> > > > > @@ -16,98 +16,23 @@ _cleanup()
> > > > >  {
> > > > >      cd /
> > > > >      _scratch_unmount > /dev/null 2>&1
> > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > -	    _destroy_loop_device $logdev
> > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > > -       $TEST_DIR/log-image
> > > > > +    _cleanup_verify_metadump
> > > > > +    rm -rf $tmp.* $testdir
> > > > >  }
> > > > >  
> > > > >  # Import common functions.
> > > > >  . ./common/filter
> > > > >  . ./common/reflink
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  # real QA test starts here
> > > > >  _supported_fs xfs
> > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > >  _require_loop
> > > > >  _require_scratch_reflink
> > > > > -
> > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > -
> > > > > -verify_metadump_v1()
> > > > > -{
> > > > > -	local max_version=$1
> > > > > -	local version=""
> > > > > -
> > > > > -	if [[ $max_version == 2 ]]; then
> > > > > -		version="-v 1"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > -
> > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > -
> > > > > -verify_metadump_v2()
> > > > > -{
> > > > > -	version="-v 2"
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > -
> > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > -	# from such a metadump file.
> > > > > -	slogdev=""
> > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > -		slogdev=$TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > -
> > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > -
> > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > -		_destroy_loop_device $logdev
> > > > > -		logdev=""
> > > > > -		rm -f $TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > +_setup_verify_metadump
> > > > >  
> > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > -
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > >  _scratch_mount
> > > > >  
> > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > @@ -127,12 +52,7 @@ done
> > > > >  _scratch_unmount
> > > > >  
> > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > -
> > > > > -verify_metadump_v1 $max_md_version
> > > > > -
> > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > -	verify_metadump_v2
> > > > > -fi
> > > > > +_verify_metadumps
> > > > >  
> > > > >  # success, all done
> > > > >  status=0
> > > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > > index f4f8af6d3a..8f808c7507 100755
> > > > > --- a/tests/xfs/234
> > > > > +++ b/tests/xfs/234
> > > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > > >  _cleanup()
> > > > >  {
> > > > >      cd /
> > > > > -    _scratch_unmount > /dev/null 2>&1
> > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > -	    _destroy_loop_device $logdev
> > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > > -       $TEST_DIR/log-image
> > > > > +    _cleanup_verify_metadump
> > > > > +    rm -rf $tmp.* $testdir
> > > > >  }
> > > > >  
> > > > >  # Import common functions.
> > > > >  . ./common/filter
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  # real QA test starts here
> > > > >  _supported_fs xfs
> > > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > >  _require_loop
> > > > >  _require_xfs_scratch_rmapbt
> > > > >  _require_xfs_io_command "fpunch"
> > > > > -
> > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > -
> > > > > -verify_metadump_v1()
> > > > > -{
> > > > > -	local max_version=$1
> > > > > -	local version=""
> > > > > -
> > > > > -	if [[ $max_version == 2 ]]; then
> > > > > -		version="-v 1"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > -
> > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > -
> > > > > -verify_metadump_v2()
> > > > > -{
> > > > > -	version="-v 2"
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > -
> > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > -	# from such a metadump file.
> > > > > -	slogdev=""
> > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > -		slogdev=$TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > -
> > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > -
> > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > -		_destroy_loop_device $logdev
> > > > > -		logdev=""
> > > > > -		rm -f $TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > +_setup_verify_metadump
> > > > >  
> > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > -
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > >  _scratch_mount
> > > > >  
> > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > @@ -127,12 +51,7 @@ done
> > > > >  _scratch_unmount
> > > > >  
> > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > -
> > > > > -verify_metadump_v1 $max_md_version
> > > > > -
> > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > -	verify_metadump_v2
> > > > > -fi
> > > > > +_verify_metadumps
> > > > >  
> > > > >  # success, all done
> > > > >  status=0
> > > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > > index 3b567999d8..6623c435e5 100755
> > > > > --- a/tests/xfs/253
> > > > > +++ b/tests/xfs/253
> > > > > @@ -26,23 +26,21 @@ _cleanup()
> > > > >      cd /
> > > > >      rm -f $tmp.*
> > > > >      rm -rf "${OUTPUT_DIR}"
> > > > > -    rm -f "${METADUMP_FILE}"
> > > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > -	    _destroy_loop_device $logdev
> > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > +    _cleanup_verify_metadump
> > > > >  }
> > > > >  
> > > > >  # Import common functions.
> > > > >  . ./common/filter
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > >  _require_test
> > > > >  _require_scratch
> > > > > +_setup_verify_metadump
> > > > >  
> > > > >  # real QA test starts here
> > > > >  
> > > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > > >  ORPHANAGE="lost+found"
> > > > >  
> > > > >  _supported_fs xfs
> > > > > @@ -52,24 +50,7 @@ function create_file() {
> > > > >  	touch $(printf "$@")
> > > > >  }
> > > > >  
> > > > > -verify_metadump_v1()
> > > > > -{
> > > > > -	local max_version=$1
> > > > > -	local version=""
> > > > > -
> > > > > -	if [[ $max_version == 2 ]]; then
> > > > > -		version="-v 1"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > -
> > > > > +extra_test() {
> > > > >  	cd "${SCRATCH_MNT}"
> > > > >  
> > > > >  	# Get a listing of all the files after obfuscation
> > > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > > >  	ls -R | od -c >> $seqres.full
> > > > >  
> > > > >  	cd /
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > -
> > > > > -verify_metadump_v2()
> > > > > -{
> > > > > -	version="-v 2"
> > > > > -
> > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > -
> > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > -	# from such a metadump file.
> > > > > -	slogdev=""
> > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > -		slogdev=$TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > -
> > > > > -	cd "${SCRATCH_MNT}"
> > > > > -
> > > > > -	# Get a listing of all the files after obfuscation
> > > > > -	echo "Metadump v2" >> $seqres.full
> > > > > -	ls -R >> $seqres.full
> > > > > -	ls -R | od -c >> $seqres.full
> > > > > -
> > > > > -	cd /
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > -
> > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > -		_destroy_loop_device $logdev
> > > > > -		logdev=""
> > > > > -		rm -f $TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > >  }
> > > > >  
> > > > >  echo "Disciplyne of silence is goed."
> > > > > @@ -233,13 +160,7 @@ cd $here
> > > > >  
> > > > >  _scratch_unmount
> > > > >  
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > > -verify_metadump_v1 $max_md_version
> > > > > -
> > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > -	verify_metadump_v2
> > > > > -fi
> > > > > +_verify_metadumps '' extra_test
> > > > >  
> > > > >  # Finally, re-make the filesystem since to ensure we don't
> > > > >  # leave a directory with duplicate entries lying around.
> > > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > > index 1433140821..c475d89ad9 100755
> > > > > --- a/tests/xfs/291
> > > > > +++ b/tests/xfs/291
> > > > > @@ -9,11 +9,21 @@
> > > > >  . ./common/preamble
> > > > >  _begin_fstest auto repair metadump
> > > > >  
> > > > > +# Override the default cleanup function.
> > > > > +_cleanup()
> > > > > +{
> > > > > +	cd /
> > > > > +	rm -r -f $tmp.*
> > > > > +	_cleanup_verify_metadump
> > > > > +}
> > > > > +
> > > > >  # Import common functions.
> > > > >  . ./common/filter
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  _supported_fs xfs
> > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > +_setup_verify_metadump
> > > > >  
> > > > >  # real QA test starts here
> > > > >  _require_scratch
> > > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > > >  
> > > > >  # Yes they can!  Now...
> > > > >  # Can xfs_metadump cope with this monster?
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > -	version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v $md_version"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > > -		_fail "xfs_metadump failed"
> > > > > -
> > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > -		slogdev=""
> > > > > -	fi
> > > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > > -		_fail "xfs_repair of metadump failed"
> > > > > -done
> > > > > +_verify_metadumps '-a -o'
> > > > >  
> > > > >  # Yes it can; success, all done
> > > > >  status=0
> > > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > > index 7e402aa88f..579e1b556a 100755
> > > > > --- a/tests/xfs/432
> > > > > +++ b/tests/xfs/432
> > > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > > >  _cleanup()
> > > > >  {
> > > > >  	cd /
> > > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > > +	rm -f "$tmp".*
> > > > > +	_cleanup_verify_metadump
> > > > >  }
> > > > >  
> > > > >  # Import common functions.
> > > > >  . ./common/filter
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  # real QA test starts here
> > > > >  _supported_fs xfs
> > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > >  _require_scratch
> > > > > +_setup_verify_metadump
> > > > >  
> > > > >  rm -f "$seqres.full"
> > > > >  
> > > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > > >  
> > > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > > -rm -f $metadump_file $metadump_img
> > > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > > >  max_fname_len=255
> > > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > > >  
> > > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > -	version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v $md_version"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > > -
> > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > -		slogdev=""
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > > -		echo "xfs_repair on restored fs returned $?"
> > > > > -done
> > > > > +_verify_metadumps '-a -o -w'
> > > > >  
> > > > >  # success, all done
> > > > >  status=0
> > > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > > index 8643c3d483..ff6b344a9c 100755
> > > > > --- a/tests/xfs/503
> > > > > +++ b/tests/xfs/503
> > > > > @@ -17,11 +17,13 @@ _cleanup()
> > > > >  {
> > > > >  	cd /
> > > > >  	rm -rf $tmp.* $testdir
> > > > > +	_cleanup_verify_metadump
> > > > >  }
> > > > >  
> > > > >  # Import common functions.
> > > > >  . ./common/filter
> > > > >  . ./common/populate
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  testdir=$TEST_DIR/test-$seq
> > > > >  
> > > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > > >  _require_populate_commands
> > > > >  _xfs_skip_online_rebuild
> > > > >  _xfs_skip_offline_rebuild
> > > > > +_setup_verify_metadump
> > > > >  
> > > > >  echo "Format and populate"
> > > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > > >  metadump_file=$testdir/scratch.md
> > > > >  copy_file=$testdir/copy.img
> > > > >  
> > > > > -check_restored_metadump_image()
> > > > > -{
> > > > > -	local image=$1
> > > > > -
> > > > > -	loop_dev=$(_create_loop_device $image)
> > > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > > -	_destroy_loop_device $loop_dev
> > > > > -}
> > > > > -
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > >  echo "metadump and mdrestore"
> > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > -	version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v $md_version"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > -done
> > > > > +_verify_metadumps '-a -o'
> > > > >  
> > > > >  echo "metadump a and mdrestore"
> > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > -	version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v $md_version"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > -done
> > > > > +_verify_metadumps '-a'
> > > > >  
> > > > >  echo "metadump g and mdrestore"
> > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > -	version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v $md_version"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > -done
> > > > > +_verify_metadumps '-g' >> $seqres.full
> > > > >  
> > > > >  echo "metadump ag and mdrestore"
> > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > -	version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v $md_version"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > -done
> > > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > > >  
> > > > >  echo copy
> > > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > > index f2cd7aba98..af917f0f32 100755
> > > > > --- a/tests/xfs/605
> > > > > +++ b/tests/xfs/605
> > > > > @@ -15,17 +15,13 @@ _cleanup()
> > > > >  {
> > > > >  	cd /
> > > > >  	rm -r -f $tmp.*
> > > > > -	_scratch_unmount > /dev/null 2>&1
> > > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > -		_destroy_loop_device $logdev
> > > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > > -	   $TEST_DIR/log-image
> > > > > +	_cleanup_verify_metadump
> > > > >  }
> > > > >  
> > > > >  # Import common functions.
> > > > >  . ./common/dmflakey
> > > > >  . ./common/inject
> > > > > +. ./common/xfs_metadump_tests
> > > > >  
> > > > >  # real QA test starts here
> > > > >  _supported_fs xfs
> > > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > > >  _require_dm_target flakey
> > > > >  _require_xfs_io_command "pwrite"
> > > > >  _require_test_program "punch-alternating"
> > > > > +_setup_verify_metadump
> > > > >  
> > > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > > >  testfile=${SCRATCH_MNT}/testfile
> > > > >  
> > > > >  echo "Format filesystem on scratch device"
> > > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > > >  
> > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > -
> > > > >  external_log=0
> > > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > > >  	external_log=1
> > > > >  fi
> > > > >  
> > > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > > >  	_notrun "metadump v1 does not support external log device"
> > > > >  fi
> > > > >  
> > > > > -verify_metadump_v1()
> > > > > -{
> > > > > -	local version=""
> > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > -		version="-v 1"
> > > > > -	fi
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > -
> > > > > -verify_metadump_v2()
> > > > > -{
> > > > > -	local version="-v 2"
> > > > > -
> > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > -
> > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > -	# from such a metadump file.
> > > > > -	slogdev=""
> > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > -		slogdev=$TEST_DIR/log-image
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > -
> > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > -
> > > > > -	logdev=""
> > > > > -	if [[ -s $slogdev ]]; then
> > > > > -		logdev=$(_create_loop_device $slogdev)
> > > > > -	fi
> > > > > -
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > -
> > > > > -	if [[ -s $logdev ]]; then
> > > > > -		_destroy_loop_device $logdev
> > > > > -		logdev=""
> > > > > -		rm -f $slogdev
> > > > > -	fi
> > > > > -
> > > > > -	_destroy_loop_device $datadev
> > > > > -	datadev=""
> > > > > -	rm -f $TEST_DIR/data-image
> > > > > -}
> > > > > -
> > > > >  echo "Initialize and mount filesystem on flakey device"
> > > > >  _init_flakey
> > > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > > >  _print_logstate
> > > > >  
> > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > -
> > > > > -if [[ $external_log == 0 ]]; then
> > > > > -	verify_metadump_v1 $max_md_version
> > > > > -fi
> > > > > -
> > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > -	verify_metadump_v2
> > > > > -fi
> > > > > +_verify_metadumps '-a -o'
> > > > >  
> > > > >  # Mount the fs to replay the contents from the dirty log.
> > > > >  _scratch_mount
> > > > > 
> > > > 
> > > 
> > 
> > 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Darrick J. Wong]

On Wed, Jan 31, 2024 at 10:09:19PM +0800, Zorro Lang wrote:
> On Mon, Jan 29, 2024 at 05:32:07PM -0800, Darrick J. Wong wrote:
> > On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> > > On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > > > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > > 
> > > > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > > > duplicate copies of the same code.
> > > > > > 
> > > > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > > > 
> > > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > > ---
> > > > > >  common/rc                 |   10 ----
> > > > > >  common/xfs                |   14 +++++
> > > > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > > > >  tests/xfs/129             |   90 ++-------------------------------
> > > > > >  tests/xfs/234             |   91 ++-------------------------------
> > > > > >  tests/xfs/253             |   89 ++-------------------------------
> > > > > >  tests/xfs/291             |   31 ++++-------
> > > > > >  tests/xfs/432             |   30 ++---------
> > > > > >  tests/xfs/503             |   60 +++-------------------
> > > > > >  tests/xfs/605             |   84 ++-----------------------------
> > > > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > > > >  create mode 100644 common/xfs_metadump_tests
> > > > > > 
> > > > > > 
> > > > > > diff --git a/common/rc b/common/rc
> > > > > > index 524ffa02aa..0b69f7f54f 100644
> > > > > > --- a/common/rc
> > > > > > +++ b/common/rc
> > > > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > > > >  
> > > > > >      case $FSTYP in
> > > > > >      xfs)
> > > > > > -	local scratch_log="none"
> > > > > > -	local scratch_rt="none"
> > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > -
> > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > -
> > > > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > +	_check_xfs_scratch_fs $device
> > > > > >  	;;
> > > > > >      udf)
> > > > > >  	_check_udf_filesystem $device $udf_fsize
> > > > > > diff --git a/common/xfs b/common/xfs
> > > > > > index 248ccefda3..6a48960a7f 100644
> > > > > > --- a/common/xfs
> > > > > > +++ b/common/xfs
> > > > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > > > >  	return $?
> > > > > >  }
> > > > > >  
> > > > > > +_check_xfs_scratch_fs()
> > > > > > +{
> > > > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > > > +	local scratch_log="none"
> > > > > > +	local scratch_rt="none"
> > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > +
> > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > +
> > > > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > +}
> > > > > > +
> > > > > >  # modeled after _scratch_xfs_repair
> > > > > >  _test_xfs_repair()
> > > > > >  {
> > > > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > > > 
> > > > > Hi Darrick,
> > > > > 
> > > > > Thanks for this improvement.
> > > > > 
> > > > > I'm wondering do we need a separated common file only for xfs metadump
> > > > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > > > in common/xfs.
> > > > 
> > > > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > > > files are getting big:
> > > > 
> > > >    509 fstests/common/overlay
> > > >    523 fstests/common/quota
> > > >    550 fstests/common/reflink
> > > >    638 fstests/common/log
> > > >    640 fstests/common/punch
> > > >    663 fstests/common/filter
> > > >    794 fstests/common/btrfs
> > > >    936 fstests/common/config
> > > >   1030 fstests/common/encrypt
> > > >   1162 fstests/common/populate
> > > >   1519 fstests/common/fuzzy
> > > >   1531 fstests/common/dump
> > > >   2218 fstests/common/xfs
> > > >   5437 fstests/common/rc
> > > > 
> > > > with common/xfs being particularly larger than most everything else.
> > > 
> > > Haha, maybe we'll have a common/xfs/ directory in one day :)
> > > 
> > > > 
> > > > Since the common/xfs_metadump_tests functions are shared helper
> > > > functions for testing metadump/mdrestore that are not use by most tests,
> > > > I decided that a split was appropriate both to maintain the (ha!)
> > > > cohesion of common/xfs and not add more bash parsing costs to every
> > > > single testcase.
> > > 
> > > OK, a split makes sense, but I have 3 questions:
> > > 1) Will you move all metadump helpers from common/xfs to this new file?
> > 
> > I don't really want to, because that's another patch and would require
> > careful auditing of all the tests to find the ones that want to use
> > metadump but aren't themselves functional tests of metadump.
> > 
> > IOWs, this new file really is for shared metadump functional testing and
> > not much else.
> 
> OK, I think we can care about this part step by step in the future.

<nod>

> > 
> > > 2) Can we call it common/metadump? (Not sure if any other fs has metadump
> > >    things:)
> > 
> > Yes, e2image does this for ext*.
> 
> OK, I'll rename this file to common/metadump, and change other patches to
> souce the new name when I merge this patchset. Is that good to you?

Sorry, I realized that my statement was ambiguous -- the "yes" applies
to "Not sure if any other fs has metadump things", not "Can we call it
common/metadump?".  The code in common/xfs_metadump_tests is specific to
xfs and is not used for e2image testing, so let's leave the name as-is.

--D

> Thanks,
> Zorro
> 
> > 
> > > 3) Or move to common/dump directly? (looks not proper ;-)
> > 
> > dump != metadump; one is for all the files in the fs and none of the
> > non-file metadata; the other is for metadata and none of the files.
> > 
> > --D
> > 
> > > Thanks,
> > > Zorro
> > > 
> > > > 
> > > > --D
> > > > 
> > > > > Thanks,
> > > > > Zorro
> > > > > 
> > > > > > new file mode 100644
> > > > > > index 0000000000..dd3dec1fb4
> > > > > > --- /dev/null
> > > > > > +++ b/common/xfs_metadump_tests
> > > > > > @@ -0,0 +1,123 @@
> > > > > > +#
> > > > > > +# XFS specific metadump testing functions.
> > > > > > +#
> > > > > > +
> > > > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > > > +_setup_verify_metadump()
> > > > > > +{
> > > > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > > > +
> > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > +}
> > > > > > +
> > > > > > +_cleanup_verify_metadump()
> > > > > > +{
> > > > > > +	_scratch_unmount &>> $seqres.full
> > > > > > +
> > > > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > > > +		losetup -d "$ldev"
> > > > > > +	done
> > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > +}
> > > > > > +
> > > > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > > > +# images and fsck them.
> > > > > > +_verify_metadump_v1()
> > > > > > +{
> > > > > > +	local metadump_args="$1"
> > > > > > +	local extra_test="$2"
> > > > > > +
> > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > +	local version=""
> > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > +	local data_loop
> > > > > > +
> > > > > > +	# Force v1 if we detect v2 support
> > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > > > +		version="-v 1"
> > > > > > +	fi
> > > > > > +
> > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > +
> > > > > > +	# Restore metadump, which creates data_img
> > > > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > > > +
> > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > +
> > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > > > +	fi
> > > > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > +
> > > > > > +	# Tear down what we created
> > > > > > +	_destroy_loop_device $data_loop
> > > > > > +	rm -f $data_img
> > > > > > +}
> > > > > > +
> > > > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > > > +# images and fsck them.
> > > > > > +_verify_metadump_v2()
> > > > > > +{
> > > > > > +	local metadump_args="$1"
> > > > > > +	local extra_test="$2"
> > > > > > +
> > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > +	local version="-v 2"
> > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > +	local data_loop
> > > > > > +	local log_img=""
> > > > > > +	local log_loop
> > > > > > +
> > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > +
> > > > > > +	#
> > > > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > > > +	# from such a metadump file.
> > > > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > > > +
> > > > > > +	# Restore metadump, which creates data_img and log_img
> > > > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > > > +
> > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > +
> > > > > > +	# Create loopdev for log device if we recovered anything
> > > > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > > > +
> > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > > > +	fi
> > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > +
> > > > > > +	# Tear down what we created
> > > > > > +	if [ -b "$log_loop" ]; then
> > > > > > +		_destroy_loop_device $log_loop
> > > > > > +		rm -f $log_img
> > > > > > +	fi
> > > > > > +	_destroy_loop_device $data_loop
> > > > > > +	rm -f $data_img
> > > > > > +}
> > > > > > +
> > > > > > +# Verify both metadump formats if possible
> > > > > > +_verify_metadumps()
> > > > > > +{
> > > > > > +	_verify_metadump_v1 "$@"
> > > > > > +
> > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > > > +		_verify_metadump_v2 "$@"
> > > > > > +	fi
> > > > > > +}
> > > > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > > > index cdac2349df..c3a9bcefee 100755
> > > > > > --- a/tests/xfs/129
> > > > > > +++ b/tests/xfs/129
> > > > > > @@ -16,98 +16,23 @@ _cleanup()
> > > > > >  {
> > > > > >      cd /
> > > > > >      _scratch_unmount > /dev/null 2>&1
> > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > -	    _destroy_loop_device $logdev
> > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > > > -       $TEST_DIR/log-image
> > > > > > +    _cleanup_verify_metadump
> > > > > > +    rm -rf $tmp.* $testdir
> > > > > >  }
> > > > > >  
> > > > > >  # Import common functions.
> > > > > >  . ./common/filter
> > > > > >  . ./common/reflink
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  # real QA test starts here
> > > > > >  _supported_fs xfs
> > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > >  _require_loop
> > > > > >  _require_scratch_reflink
> > > > > > -
> > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > -
> > > > > > -verify_metadump_v1()
> > > > > > -{
> > > > > > -	local max_version=$1
> > > > > > -	local version=""
> > > > > > -
> > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > -		version="-v 1"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > -
> > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > -
> > > > > > -verify_metadump_v2()
> > > > > > -{
> > > > > > -	version="-v 2"
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > -
> > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > -	# from such a metadump file.
> > > > > > -	slogdev=""
> > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > -
> > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > -
> > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > -		_destroy_loop_device $logdev
> > > > > > -		logdev=""
> > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > -
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > >  _scratch_mount
> > > > > >  
> > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > @@ -127,12 +52,7 @@ done
> > > > > >  _scratch_unmount
> > > > > >  
> > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > -
> > > > > > -verify_metadump_v1 $max_md_version
> > > > > > -
> > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > -	verify_metadump_v2
> > > > > > -fi
> > > > > > +_verify_metadumps
> > > > > >  
> > > > > >  # success, all done
> > > > > >  status=0
> > > > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > > > index f4f8af6d3a..8f808c7507 100755
> > > > > > --- a/tests/xfs/234
> > > > > > +++ b/tests/xfs/234
> > > > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > > > >  _cleanup()
> > > > > >  {
> > > > > >      cd /
> > > > > > -    _scratch_unmount > /dev/null 2>&1
> > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > -	    _destroy_loop_device $logdev
> > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > > > -       $TEST_DIR/log-image
> > > > > > +    _cleanup_verify_metadump
> > > > > > +    rm -rf $tmp.* $testdir
> > > > > >  }
> > > > > >  
> > > > > >  # Import common functions.
> > > > > >  . ./common/filter
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  # real QA test starts here
> > > > > >  _supported_fs xfs
> > > > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > >  _require_loop
> > > > > >  _require_xfs_scratch_rmapbt
> > > > > >  _require_xfs_io_command "fpunch"
> > > > > > -
> > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > -
> > > > > > -verify_metadump_v1()
> > > > > > -{
> > > > > > -	local max_version=$1
> > > > > > -	local version=""
> > > > > > -
> > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > -		version="-v 1"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > -
> > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > -
> > > > > > -verify_metadump_v2()
> > > > > > -{
> > > > > > -	version="-v 2"
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > -
> > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > -	# from such a metadump file.
> > > > > > -	slogdev=""
> > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > -
> > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > -
> > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > -		_destroy_loop_device $logdev
> > > > > > -		logdev=""
> > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > -
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > >  _scratch_mount
> > > > > >  
> > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > @@ -127,12 +51,7 @@ done
> > > > > >  _scratch_unmount
> > > > > >  
> > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > -
> > > > > > -verify_metadump_v1 $max_md_version
> > > > > > -
> > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > -	verify_metadump_v2
> > > > > > -fi
> > > > > > +_verify_metadumps
> > > > > >  
> > > > > >  # success, all done
> > > > > >  status=0
> > > > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > > > index 3b567999d8..6623c435e5 100755
> > > > > > --- a/tests/xfs/253
> > > > > > +++ b/tests/xfs/253
> > > > > > @@ -26,23 +26,21 @@ _cleanup()
> > > > > >      cd /
> > > > > >      rm -f $tmp.*
> > > > > >      rm -rf "${OUTPUT_DIR}"
> > > > > > -    rm -f "${METADUMP_FILE}"
> > > > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > -	    _destroy_loop_device $logdev
> > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > +    _cleanup_verify_metadump
> > > > > >  }
> > > > > >  
> > > > > >  # Import common functions.
> > > > > >  . ./common/filter
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > >  _require_test
> > > > > >  _require_scratch
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > >  # real QA test starts here
> > > > > >  
> > > > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > > > >  ORPHANAGE="lost+found"
> > > > > >  
> > > > > >  _supported_fs xfs
> > > > > > @@ -52,24 +50,7 @@ function create_file() {
> > > > > >  	touch $(printf "$@")
> > > > > >  }
> > > > > >  
> > > > > > -verify_metadump_v1()
> > > > > > -{
> > > > > > -	local max_version=$1
> > > > > > -	local version=""
> > > > > > -
> > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > -		version="-v 1"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > -
> > > > > > +extra_test() {
> > > > > >  	cd "${SCRATCH_MNT}"
> > > > > >  
> > > > > >  	# Get a listing of all the files after obfuscation
> > > > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > > > >  	ls -R | od -c >> $seqres.full
> > > > > >  
> > > > > >  	cd /
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > -
> > > > > > -verify_metadump_v2()
> > > > > > -{
> > > > > > -	version="-v 2"
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > -
> > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > -	# from such a metadump file.
> > > > > > -	slogdev=""
> > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > -
> > > > > > -	cd "${SCRATCH_MNT}"
> > > > > > -
> > > > > > -	# Get a listing of all the files after obfuscation
> > > > > > -	echo "Metadump v2" >> $seqres.full
> > > > > > -	ls -R >> $seqres.full
> > > > > > -	ls -R | od -c >> $seqres.full
> > > > > > -
> > > > > > -	cd /
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > -
> > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > -		_destroy_loop_device $logdev
> > > > > > -		logdev=""
> > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > >  }
> > > > > >  
> > > > > >  echo "Disciplyne of silence is goed."
> > > > > > @@ -233,13 +160,7 @@ cd $here
> > > > > >  
> > > > > >  _scratch_unmount
> > > > > >  
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > > -verify_metadump_v1 $max_md_version
> > > > > > -
> > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > -	verify_metadump_v2
> > > > > > -fi
> > > > > > +_verify_metadumps '' extra_test
> > > > > >  
> > > > > >  # Finally, re-make the filesystem since to ensure we don't
> > > > > >  # leave a directory with duplicate entries lying around.
> > > > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > > > index 1433140821..c475d89ad9 100755
> > > > > > --- a/tests/xfs/291
> > > > > > +++ b/tests/xfs/291
> > > > > > @@ -9,11 +9,21 @@
> > > > > >  . ./common/preamble
> > > > > >  _begin_fstest auto repair metadump
> > > > > >  
> > > > > > +# Override the default cleanup function.
> > > > > > +_cleanup()
> > > > > > +{
> > > > > > +	cd /
> > > > > > +	rm -r -f $tmp.*
> > > > > > +	_cleanup_verify_metadump
> > > > > > +}
> > > > > > +
> > > > > >  # Import common functions.
> > > > > >  . ./common/filter
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  _supported_fs xfs
> > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > >  # real QA test starts here
> > > > > >  _require_scratch
> > > > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > > > >  
> > > > > >  # Yes they can!  Now...
> > > > > >  # Can xfs_metadump cope with this monster?
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > -	version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v $md_version"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > > > -		_fail "xfs_metadump failed"
> > > > > > -
> > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > -		slogdev=""
> > > > > > -	fi
> > > > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > > > -		_fail "xfs_repair of metadump failed"
> > > > > > -done
> > > > > > +_verify_metadumps '-a -o'
> > > > > >  
> > > > > >  # Yes it can; success, all done
> > > > > >  status=0
> > > > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > > > index 7e402aa88f..579e1b556a 100755
> > > > > > --- a/tests/xfs/432
> > > > > > +++ b/tests/xfs/432
> > > > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > > > >  _cleanup()
> > > > > >  {
> > > > > >  	cd /
> > > > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > > > +	rm -f "$tmp".*
> > > > > > +	_cleanup_verify_metadump
> > > > > >  }
> > > > > >  
> > > > > >  # Import common functions.
> > > > > >  . ./common/filter
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  # real QA test starts here
> > > > > >  _supported_fs xfs
> > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > >  _require_scratch
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > >  rm -f "$seqres.full"
> > > > > >  
> > > > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > > > >  
> > > > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > > > -rm -f $metadump_file $metadump_img
> > > > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > > > >  max_fname_len=255
> > > > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > > > >  
> > > > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > -	version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v $md_version"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > > > -
> > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > -		slogdev=""
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > > > -		echo "xfs_repair on restored fs returned $?"
> > > > > > -done
> > > > > > +_verify_metadumps '-a -o -w'
> > > > > >  
> > > > > >  # success, all done
> > > > > >  status=0
> > > > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > > > index 8643c3d483..ff6b344a9c 100755
> > > > > > --- a/tests/xfs/503
> > > > > > +++ b/tests/xfs/503
> > > > > > @@ -17,11 +17,13 @@ _cleanup()
> > > > > >  {
> > > > > >  	cd /
> > > > > >  	rm -rf $tmp.* $testdir
> > > > > > +	_cleanup_verify_metadump
> > > > > >  }
> > > > > >  
> > > > > >  # Import common functions.
> > > > > >  . ./common/filter
> > > > > >  . ./common/populate
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  testdir=$TEST_DIR/test-$seq
> > > > > >  
> > > > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > > > >  _require_populate_commands
> > > > > >  _xfs_skip_online_rebuild
> > > > > >  _xfs_skip_offline_rebuild
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > >  echo "Format and populate"
> > > > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > > > >  metadump_file=$testdir/scratch.md
> > > > > >  copy_file=$testdir/copy.img
> > > > > >  
> > > > > > -check_restored_metadump_image()
> > > > > > -{
> > > > > > -	local image=$1
> > > > > > -
> > > > > > -	loop_dev=$(_create_loop_device $image)
> > > > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > > > -	_destroy_loop_device $loop_dev
> > > > > > -}
> > > > > > -
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > >  echo "metadump and mdrestore"
> > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > -	version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v $md_version"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > -done
> > > > > > +_verify_metadumps '-a -o'
> > > > > >  
> > > > > >  echo "metadump a and mdrestore"
> > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > -	version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v $md_version"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > -done
> > > > > > +_verify_metadumps '-a'
> > > > > >  
> > > > > >  echo "metadump g and mdrestore"
> > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > -	version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v $md_version"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > -done
> > > > > > +_verify_metadumps '-g' >> $seqres.full
> > > > > >  
> > > > > >  echo "metadump ag and mdrestore"
> > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > -	version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v $md_version"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > -done
> > > > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > > > >  
> > > > > >  echo copy
> > > > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > > > index f2cd7aba98..af917f0f32 100755
> > > > > > --- a/tests/xfs/605
> > > > > > +++ b/tests/xfs/605
> > > > > > @@ -15,17 +15,13 @@ _cleanup()
> > > > > >  {
> > > > > >  	cd /
> > > > > >  	rm -r -f $tmp.*
> > > > > > -	_scratch_unmount > /dev/null 2>&1
> > > > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > -		_destroy_loop_device $logdev
> > > > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > > > -	   $TEST_DIR/log-image
> > > > > > +	_cleanup_verify_metadump
> > > > > >  }
> > > > > >  
> > > > > >  # Import common functions.
> > > > > >  . ./common/dmflakey
> > > > > >  . ./common/inject
> > > > > > +. ./common/xfs_metadump_tests
> > > > > >  
> > > > > >  # real QA test starts here
> > > > > >  _supported_fs xfs
> > > > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > > > >  _require_dm_target flakey
> > > > > >  _require_xfs_io_command "pwrite"
> > > > > >  _require_test_program "punch-alternating"
> > > > > > +_setup_verify_metadump
> > > > > >  
> > > > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > > > >  testfile=${SCRATCH_MNT}/testfile
> > > > > >  
> > > > > >  echo "Format filesystem on scratch device"
> > > > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > > > >  
> > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > -
> > > > > >  external_log=0
> > > > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > > > >  	external_log=1
> > > > > >  fi
> > > > > >  
> > > > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > > > >  	_notrun "metadump v1 does not support external log device"
> > > > > >  fi
> > > > > >  
> > > > > > -verify_metadump_v1()
> > > > > > -{
> > > > > > -	local version=""
> > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > -		version="-v 1"
> > > > > > -	fi
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > -
> > > > > > -verify_metadump_v2()
> > > > > > -{
> > > > > > -	local version="-v 2"
> > > > > > -
> > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > -
> > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > -	# from such a metadump file.
> > > > > > -	slogdev=""
> > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > -
> > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > -
> > > > > > -	logdev=""
> > > > > > -	if [[ -s $slogdev ]]; then
> > > > > > -		logdev=$(_create_loop_device $slogdev)
> > > > > > -	fi
> > > > > > -
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > -
> > > > > > -	if [[ -s $logdev ]]; then
> > > > > > -		_destroy_loop_device $logdev
> > > > > > -		logdev=""
> > > > > > -		rm -f $slogdev
> > > > > > -	fi
> > > > > > -
> > > > > > -	_destroy_loop_device $datadev
> > > > > > -	datadev=""
> > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > -}
> > > > > > -
> > > > > >  echo "Initialize and mount filesystem on flakey device"
> > > > > >  _init_flakey
> > > > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > > > >  _print_logstate
> > > > > >  
> > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > -
> > > > > > -if [[ $external_log == 0 ]]; then
> > > > > > -	verify_metadump_v1 $max_md_version
> > > > > > -fi
> > > > > > -
> > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > -	verify_metadump_v2
> > > > > > -fi
> > > > > > +_verify_metadumps '-a -o'
> > > > > >  
> > > > > >  # Mount the fs to replay the contents from the dirty log.
> > > > > >  _scratch_mount
> > > > > > 
> > > > > 
> > > > 
> > > 
> > > 
> > 
> 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Wed, Jan 31, 2024 at 11:24:33AM -0800, Darrick J. Wong wrote:
> On Wed, Jan 31, 2024 at 10:09:19PM +0800, Zorro Lang wrote:
> > On Mon, Jan 29, 2024 at 05:32:07PM -0800, Darrick J. Wong wrote:
> > > On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> > > > On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > > > > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > > > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > > > 
> > > > > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > > > > duplicate copies of the same code.
> > > > > > > 
> > > > > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > > > > 
> > > > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > > > ---
> > > > > > >  common/rc                 |   10 ----
> > > > > > >  common/xfs                |   14 +++++
> > > > > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > > > > >  tests/xfs/129             |   90 ++-------------------------------
> > > > > > >  tests/xfs/234             |   91 ++-------------------------------
> > > > > > >  tests/xfs/253             |   89 ++-------------------------------
> > > > > > >  tests/xfs/291             |   31 ++++-------
> > > > > > >  tests/xfs/432             |   30 ++---------
> > > > > > >  tests/xfs/503             |   60 +++-------------------
> > > > > > >  tests/xfs/605             |   84 ++-----------------------------
> > > > > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > > > > >  create mode 100644 common/xfs_metadump_tests
> > > > > > > 
> > > > > > > 
> > > > > > > diff --git a/common/rc b/common/rc
> > > > > > > index 524ffa02aa..0b69f7f54f 100644
> > > > > > > --- a/common/rc
> > > > > > > +++ b/common/rc
> > > > > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > > > > >  
> > > > > > >      case $FSTYP in
> > > > > > >      xfs)
> > > > > > > -	local scratch_log="none"
> > > > > > > -	local scratch_rt="none"
> > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > -
> > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > -
> > > > > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > +	_check_xfs_scratch_fs $device
> > > > > > >  	;;
> > > > > > >      udf)
> > > > > > >  	_check_udf_filesystem $device $udf_fsize
> > > > > > > diff --git a/common/xfs b/common/xfs
> > > > > > > index 248ccefda3..6a48960a7f 100644
> > > > > > > --- a/common/xfs
> > > > > > > +++ b/common/xfs
> > > > > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > > > > >  	return $?
> > > > > > >  }
> > > > > > >  
> > > > > > > +_check_xfs_scratch_fs()
> > > > > > > +{
> > > > > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > > > > +	local scratch_log="none"
> > > > > > > +	local scratch_rt="none"
> > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > +
> > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > +
> > > > > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > +}
> > > > > > > +
> > > > > > >  # modeled after _scratch_xfs_repair
> > > > > > >  _test_xfs_repair()
> > > > > > >  {
> > > > > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > > > > 
> > > > > > Hi Darrick,
> > > > > > 
> > > > > > Thanks for this improvement.
> > > > > > 
> > > > > > I'm wondering do we need a separated common file only for xfs metadump
> > > > > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > > > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > > > > in common/xfs.
> > > > > 
> > > > > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > > > > files are getting big:
> > > > > 
> > > > >    509 fstests/common/overlay
> > > > >    523 fstests/common/quota
> > > > >    550 fstests/common/reflink
> > > > >    638 fstests/common/log
> > > > >    640 fstests/common/punch
> > > > >    663 fstests/common/filter
> > > > >    794 fstests/common/btrfs
> > > > >    936 fstests/common/config
> > > > >   1030 fstests/common/encrypt
> > > > >   1162 fstests/common/populate
> > > > >   1519 fstests/common/fuzzy
> > > > >   1531 fstests/common/dump
> > > > >   2218 fstests/common/xfs
> > > > >   5437 fstests/common/rc
> > > > > 
> > > > > with common/xfs being particularly larger than most everything else.
> > > > 
> > > > Haha, maybe we'll have a common/xfs/ directory in one day :)
> > > > 
> > > > > 
> > > > > Since the common/xfs_metadump_tests functions are shared helper
> > > > > functions for testing metadump/mdrestore that are not use by most tests,
> > > > > I decided that a split was appropriate both to maintain the (ha!)
> > > > > cohesion of common/xfs and not add more bash parsing costs to every
> > > > > single testcase.
> > > > 
> > > > OK, a split makes sense, but I have 3 questions:
> > > > 1) Will you move all metadump helpers from common/xfs to this new file?
> > > 
> > > I don't really want to, because that's another patch and would require
> > > careful auditing of all the tests to find the ones that want to use
> > > metadump but aren't themselves functional tests of metadump.
> > > 
> > > IOWs, this new file really is for shared metadump functional testing and
> > > not much else.
> > 
> > OK, I think we can care about this part step by step in the future.
> 
> <nod>
> 
> > > 
> > > > 2) Can we call it common/metadump? (Not sure if any other fs has metadump
> > > >    things:)
> > > 
> > > Yes, e2image does this for ext*.
> > 
> > OK, I'll rename this file to common/metadump, and change other patches to
> > souce the new name when I merge this patchset. Is that good to you?
> 
> Sorry, I realized that my statement was ambiguous -- the "yes" applies
> to "Not sure if any other fs has metadump things", not "Can we call it
> common/metadump?".  The code in common/xfs_metadump_tests is specific to
> xfs and is not used for e2image testing, so let's leave the name as-is.

Oh, maybe it can be used for e2image or other metadump helpers later, if
we call it common/metadump?

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > > 
> > > > 3) Or move to common/dump directly? (looks not proper ;-)
> > > 
> > > dump != metadump; one is for all the files in the fs and none of the
> > > non-file metadata; the other is for metadata and none of the files.
> > > 
> > > --D
> > > 
> > > > Thanks,
> > > > Zorro
> > > > 
> > > > > 
> > > > > --D
> > > > > 
> > > > > > Thanks,
> > > > > > Zorro
> > > > > > 
> > > > > > > new file mode 100644
> > > > > > > index 0000000000..dd3dec1fb4
> > > > > > > --- /dev/null
> > > > > > > +++ b/common/xfs_metadump_tests
> > > > > > > @@ -0,0 +1,123 @@
> > > > > > > +#
> > > > > > > +# XFS specific metadump testing functions.
> > > > > > > +#
> > > > > > > +
> > > > > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > > > > +_setup_verify_metadump()
> > > > > > > +{
> > > > > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > > > > +
> > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > +}
> > > > > > > +
> > > > > > > +_cleanup_verify_metadump()
> > > > > > > +{
> > > > > > > +	_scratch_unmount &>> $seqres.full
> > > > > > > +
> > > > > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > > > > +		losetup -d "$ldev"
> > > > > > > +	done
> > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > +}
> > > > > > > +
> > > > > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > > > > +# images and fsck them.
> > > > > > > +_verify_metadump_v1()
> > > > > > > +{
> > > > > > > +	local metadump_args="$1"
> > > > > > > +	local extra_test="$2"
> > > > > > > +
> > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > +	local version=""
> > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > +	local data_loop
> > > > > > > +
> > > > > > > +	# Force v1 if we detect v2 support
> > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > > > > +		version="-v 1"
> > > > > > > +	fi
> > > > > > > +
> > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > +
> > > > > > > +	# Restore metadump, which creates data_img
> > > > > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > > > > +
> > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > +
> > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > > > > +	fi
> > > > > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > +
> > > > > > > +	# Tear down what we created
> > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > +	rm -f $data_img
> > > > > > > +}
> > > > > > > +
> > > > > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > > > > +# images and fsck them.
> > > > > > > +_verify_metadump_v2()
> > > > > > > +{
> > > > > > > +	local metadump_args="$1"
> > > > > > > +	local extra_test="$2"
> > > > > > > +
> > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > +	local version="-v 2"
> > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > +	local data_loop
> > > > > > > +	local log_img=""
> > > > > > > +	local log_loop
> > > > > > > +
> > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > +
> > > > > > > +	#
> > > > > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > > > > +	# from such a metadump file.
> > > > > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > > > > +
> > > > > > > +	# Restore metadump, which creates data_img and log_img
> > > > > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > > > > +
> > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > +
> > > > > > > +	# Create loopdev for log device if we recovered anything
> > > > > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > > > > +
> > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > > > > +	fi
> > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > +
> > > > > > > +	# Tear down what we created
> > > > > > > +	if [ -b "$log_loop" ]; then
> > > > > > > +		_destroy_loop_device $log_loop
> > > > > > > +		rm -f $log_img
> > > > > > > +	fi
> > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > +	rm -f $data_img
> > > > > > > +}
> > > > > > > +
> > > > > > > +# Verify both metadump formats if possible
> > > > > > > +_verify_metadumps()
> > > > > > > +{
> > > > > > > +	_verify_metadump_v1 "$@"
> > > > > > > +
> > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > > > > +		_verify_metadump_v2 "$@"
> > > > > > > +	fi
> > > > > > > +}
> > > > > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > > > > index cdac2349df..c3a9bcefee 100755
> > > > > > > --- a/tests/xfs/129
> > > > > > > +++ b/tests/xfs/129
> > > > > > > @@ -16,98 +16,23 @@ _cleanup()
> > > > > > >  {
> > > > > > >      cd /
> > > > > > >      _scratch_unmount > /dev/null 2>&1
> > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > > > > -       $TEST_DIR/log-image
> > > > > > > +    _cleanup_verify_metadump
> > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > >  }
> > > > > > >  
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/filter
> > > > > > >  . ./common/reflink
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  # real QA test starts here
> > > > > > >  _supported_fs xfs
> > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > >  _require_loop
> > > > > > >  _require_scratch_reflink
> > > > > > > -
> > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > -
> > > > > > > -verify_metadump_v1()
> > > > > > > -{
> > > > > > > -	local max_version=$1
> > > > > > > -	local version=""
> > > > > > > -
> > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > -		version="-v 1"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > -
> > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > -
> > > > > > > -verify_metadump_v2()
> > > > > > > -{
> > > > > > > -	version="-v 2"
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > -
> > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > -	# from such a metadump file.
> > > > > > > -	slogdev=""
> > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > -
> > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > -
> > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > -		_destroy_loop_device $logdev
> > > > > > > -		logdev=""
> > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > -
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > >  _scratch_mount
> > > > > > >  
> > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > @@ -127,12 +52,7 @@ done
> > > > > > >  _scratch_unmount
> > > > > > >  
> > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > -
> > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > -
> > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > -	verify_metadump_v2
> > > > > > > -fi
> > > > > > > +_verify_metadumps
> > > > > > >  
> > > > > > >  # success, all done
> > > > > > >  status=0
> > > > > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > > > > index f4f8af6d3a..8f808c7507 100755
> > > > > > > --- a/tests/xfs/234
> > > > > > > +++ b/tests/xfs/234
> > > > > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > > > > >  _cleanup()
> > > > > > >  {
> > > > > > >      cd /
> > > > > > > -    _scratch_unmount > /dev/null 2>&1
> > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > > > > -       $TEST_DIR/log-image
> > > > > > > +    _cleanup_verify_metadump
> > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > >  }
> > > > > > >  
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/filter
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  # real QA test starts here
> > > > > > >  _supported_fs xfs
> > > > > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > >  _require_loop
> > > > > > >  _require_xfs_scratch_rmapbt
> > > > > > >  _require_xfs_io_command "fpunch"
> > > > > > > -
> > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > -
> > > > > > > -verify_metadump_v1()
> > > > > > > -{
> > > > > > > -	local max_version=$1
> > > > > > > -	local version=""
> > > > > > > -
> > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > -		version="-v 1"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > -
> > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > -
> > > > > > > -verify_metadump_v2()
> > > > > > > -{
> > > > > > > -	version="-v 2"
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > -
> > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > -	# from such a metadump file.
> > > > > > > -	slogdev=""
> > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > -
> > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > -
> > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > -		_destroy_loop_device $logdev
> > > > > > > -		logdev=""
> > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > -
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > >  _scratch_mount
> > > > > > >  
> > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > @@ -127,12 +51,7 @@ done
> > > > > > >  _scratch_unmount
> > > > > > >  
> > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > -
> > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > -
> > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > -	verify_metadump_v2
> > > > > > > -fi
> > > > > > > +_verify_metadumps
> > > > > > >  
> > > > > > >  # success, all done
> > > > > > >  status=0
> > > > > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > > > > index 3b567999d8..6623c435e5 100755
> > > > > > > --- a/tests/xfs/253
> > > > > > > +++ b/tests/xfs/253
> > > > > > > @@ -26,23 +26,21 @@ _cleanup()
> > > > > > >      cd /
> > > > > > >      rm -f $tmp.*
> > > > > > >      rm -rf "${OUTPUT_DIR}"
> > > > > > > -    rm -f "${METADUMP_FILE}"
> > > > > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > +    _cleanup_verify_metadump
> > > > > > >  }
> > > > > > >  
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/filter
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > >  _require_test
> > > > > > >  _require_scratch
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > >  # real QA test starts here
> > > > > > >  
> > > > > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > > > > >  ORPHANAGE="lost+found"
> > > > > > >  
> > > > > > >  _supported_fs xfs
> > > > > > > @@ -52,24 +50,7 @@ function create_file() {
> > > > > > >  	touch $(printf "$@")
> > > > > > >  }
> > > > > > >  
> > > > > > > -verify_metadump_v1()
> > > > > > > -{
> > > > > > > -	local max_version=$1
> > > > > > > -	local version=""
> > > > > > > -
> > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > -		version="-v 1"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > -
> > > > > > > +extra_test() {
> > > > > > >  	cd "${SCRATCH_MNT}"
> > > > > > >  
> > > > > > >  	# Get a listing of all the files after obfuscation
> > > > > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > > > > >  	ls -R | od -c >> $seqres.full
> > > > > > >  
> > > > > > >  	cd /
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > -
> > > > > > > -verify_metadump_v2()
> > > > > > > -{
> > > > > > > -	version="-v 2"
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > -
> > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > -	# from such a metadump file.
> > > > > > > -	slogdev=""
> > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > -
> > > > > > > -	cd "${SCRATCH_MNT}"
> > > > > > > -
> > > > > > > -	# Get a listing of all the files after obfuscation
> > > > > > > -	echo "Metadump v2" >> $seqres.full
> > > > > > > -	ls -R >> $seqres.full
> > > > > > > -	ls -R | od -c >> $seqres.full
> > > > > > > -
> > > > > > > -	cd /
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > -
> > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > -		_destroy_loop_device $logdev
> > > > > > > -		logdev=""
> > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > >  }
> > > > > > >  
> > > > > > >  echo "Disciplyne of silence is goed."
> > > > > > > @@ -233,13 +160,7 @@ cd $here
> > > > > > >  
> > > > > > >  _scratch_unmount
> > > > > > >  
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > -
> > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > -	verify_metadump_v2
> > > > > > > -fi
> > > > > > > +_verify_metadumps '' extra_test
> > > > > > >  
> > > > > > >  # Finally, re-make the filesystem since to ensure we don't
> > > > > > >  # leave a directory with duplicate entries lying around.
> > > > > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > > > > index 1433140821..c475d89ad9 100755
> > > > > > > --- a/tests/xfs/291
> > > > > > > +++ b/tests/xfs/291
> > > > > > > @@ -9,11 +9,21 @@
> > > > > > >  . ./common/preamble
> > > > > > >  _begin_fstest auto repair metadump
> > > > > > >  
> > > > > > > +# Override the default cleanup function.
> > > > > > > +_cleanup()
> > > > > > > +{
> > > > > > > +	cd /
> > > > > > > +	rm -r -f $tmp.*
> > > > > > > +	_cleanup_verify_metadump
> > > > > > > +}
> > > > > > > +
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/filter
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  _supported_fs xfs
> > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > >  # real QA test starts here
> > > > > > >  _require_scratch
> > > > > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > > > > >  
> > > > > > >  # Yes they can!  Now...
> > > > > > >  # Can xfs_metadump cope with this monster?
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > -	version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v $md_version"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > > > > -		_fail "xfs_metadump failed"
> > > > > > > -
> > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > -		slogdev=""
> > > > > > > -	fi
> > > > > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > > > > -		_fail "xfs_repair of metadump failed"
> > > > > > > -done
> > > > > > > +_verify_metadumps '-a -o'
> > > > > > >  
> > > > > > >  # Yes it can; success, all done
> > > > > > >  status=0
> > > > > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > > > > index 7e402aa88f..579e1b556a 100755
> > > > > > > --- a/tests/xfs/432
> > > > > > > +++ b/tests/xfs/432
> > > > > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > > > > >  _cleanup()
> > > > > > >  {
> > > > > > >  	cd /
> > > > > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > > > > +	rm -f "$tmp".*
> > > > > > > +	_cleanup_verify_metadump
> > > > > > >  }
> > > > > > >  
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/filter
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  # real QA test starts here
> > > > > > >  _supported_fs xfs
> > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > >  _require_scratch
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > >  rm -f "$seqres.full"
> > > > > > >  
> > > > > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > > > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > > > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > > > > >  
> > > > > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > > > > -rm -f $metadump_file $metadump_img
> > > > > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > > > > >  max_fname_len=255
> > > > > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > > > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > > > > >  
> > > > > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > -	version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v $md_version"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > > > > -
> > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > -		slogdev=""
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > > > > -		echo "xfs_repair on restored fs returned $?"
> > > > > > > -done
> > > > > > > +_verify_metadumps '-a -o -w'
> > > > > > >  
> > > > > > >  # success, all done
> > > > > > >  status=0
> > > > > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > > > > index 8643c3d483..ff6b344a9c 100755
> > > > > > > --- a/tests/xfs/503
> > > > > > > +++ b/tests/xfs/503
> > > > > > > @@ -17,11 +17,13 @@ _cleanup()
> > > > > > >  {
> > > > > > >  	cd /
> > > > > > >  	rm -rf $tmp.* $testdir
> > > > > > > +	_cleanup_verify_metadump
> > > > > > >  }
> > > > > > >  
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/filter
> > > > > > >  . ./common/populate
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  testdir=$TEST_DIR/test-$seq
> > > > > > >  
> > > > > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > > > > >  _require_populate_commands
> > > > > > >  _xfs_skip_online_rebuild
> > > > > > >  _xfs_skip_offline_rebuild
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > >  echo "Format and populate"
> > > > > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > > > > >  metadump_file=$testdir/scratch.md
> > > > > > >  copy_file=$testdir/copy.img
> > > > > > >  
> > > > > > > -check_restored_metadump_image()
> > > > > > > -{
> > > > > > > -	local image=$1
> > > > > > > -
> > > > > > > -	loop_dev=$(_create_loop_device $image)
> > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > > > > -	_destroy_loop_device $loop_dev
> > > > > > > -}
> > > > > > > -
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > >  echo "metadump and mdrestore"
> > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > -	version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v $md_version"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > -done
> > > > > > > +_verify_metadumps '-a -o'
> > > > > > >  
> > > > > > >  echo "metadump a and mdrestore"
> > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > -	version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v $md_version"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > -done
> > > > > > > +_verify_metadumps '-a'
> > > > > > >  
> > > > > > >  echo "metadump g and mdrestore"
> > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > -	version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v $md_version"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > -done
> > > > > > > +_verify_metadumps '-g' >> $seqres.full
> > > > > > >  
> > > > > > >  echo "metadump ag and mdrestore"
> > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > -	version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v $md_version"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > -done
> > > > > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > > > > >  
> > > > > > >  echo copy
> > > > > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > > > > index f2cd7aba98..af917f0f32 100755
> > > > > > > --- a/tests/xfs/605
> > > > > > > +++ b/tests/xfs/605
> > > > > > > @@ -15,17 +15,13 @@ _cleanup()
> > > > > > >  {
> > > > > > >  	cd /
> > > > > > >  	rm -r -f $tmp.*
> > > > > > > -	_scratch_unmount > /dev/null 2>&1
> > > > > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > -		_destroy_loop_device $logdev
> > > > > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > > > > -	   $TEST_DIR/log-image
> > > > > > > +	_cleanup_verify_metadump
> > > > > > >  }
> > > > > > >  
> > > > > > >  # Import common functions.
> > > > > > >  . ./common/dmflakey
> > > > > > >  . ./common/inject
> > > > > > > +. ./common/xfs_metadump_tests
> > > > > > >  
> > > > > > >  # real QA test starts here
> > > > > > >  _supported_fs xfs
> > > > > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > > > > >  _require_dm_target flakey
> > > > > > >  _require_xfs_io_command "pwrite"
> > > > > > >  _require_test_program "punch-alternating"
> > > > > > > +_setup_verify_metadump
> > > > > > >  
> > > > > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > > > > >  testfile=${SCRATCH_MNT}/testfile
> > > > > > >  
> > > > > > >  echo "Format filesystem on scratch device"
> > > > > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > > > > >  
> > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > -
> > > > > > >  external_log=0
> > > > > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > > > > >  	external_log=1
> > > > > > >  fi
> > > > > > >  
> > > > > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > > > > >  	_notrun "metadump v1 does not support external log device"
> > > > > > >  fi
> > > > > > >  
> > > > > > > -verify_metadump_v1()
> > > > > > > -{
> > > > > > > -	local version=""
> > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > -		version="-v 1"
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > -
> > > > > > > -verify_metadump_v2()
> > > > > > > -{
> > > > > > > -	local version="-v 2"
> > > > > > > -
> > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > -
> > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > -	# from such a metadump file.
> > > > > > > -	slogdev=""
> > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > -
> > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > -
> > > > > > > -	logdev=""
> > > > > > > -	if [[ -s $slogdev ]]; then
> > > > > > > -		logdev=$(_create_loop_device $slogdev)
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > -
> > > > > > > -	if [[ -s $logdev ]]; then
> > > > > > > -		_destroy_loop_device $logdev
> > > > > > > -		logdev=""
> > > > > > > -		rm -f $slogdev
> > > > > > > -	fi
> > > > > > > -
> > > > > > > -	_destroy_loop_device $datadev
> > > > > > > -	datadev=""
> > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > -}
> > > > > > > -
> > > > > > >  echo "Initialize and mount filesystem on flakey device"
> > > > > > >  _init_flakey
> > > > > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > > > > >  _print_logstate
> > > > > > >  
> > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > -
> > > > > > > -if [[ $external_log == 0 ]]; then
> > > > > > > -	verify_metadump_v1 $max_md_version
> > > > > > > -fi
> > > > > > > -
> > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > -	verify_metadump_v2
> > > > > > > -fi
> > > > > > > +_verify_metadumps '-a -o'
> > > > > > >  
> > > > > > >  # Mount the fs to replay the contents from the dirty log.
> > > > > > >  _scratch_mount
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > > 
> > > 
> > 
> > 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Darrick J. Wong]

On Thu, Feb 01, 2024 at 02:29:22PM +0800, Zorro Lang wrote:
> On Wed, Jan 31, 2024 at 11:24:33AM -0800, Darrick J. Wong wrote:
> > On Wed, Jan 31, 2024 at 10:09:19PM +0800, Zorro Lang wrote:
> > > On Mon, Jan 29, 2024 at 05:32:07PM -0800, Darrick J. Wong wrote:
> > > > On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> > > > > On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > > > > > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > > > > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > > > > 
> > > > > > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > > > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > > > > > duplicate copies of the same code.
> > > > > > > > 
> > > > > > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > > > > > 
> > > > > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > > > > ---
> > > > > > > >  common/rc                 |   10 ----
> > > > > > > >  common/xfs                |   14 +++++
> > > > > > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > > > > > >  tests/xfs/129             |   90 ++-------------------------------
> > > > > > > >  tests/xfs/234             |   91 ++-------------------------------
> > > > > > > >  tests/xfs/253             |   89 ++-------------------------------
> > > > > > > >  tests/xfs/291             |   31 ++++-------
> > > > > > > >  tests/xfs/432             |   30 ++---------
> > > > > > > >  tests/xfs/503             |   60 +++-------------------
> > > > > > > >  tests/xfs/605             |   84 ++-----------------------------
> > > > > > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > > > > > >  create mode 100644 common/xfs_metadump_tests
> > > > > > > > 
> > > > > > > > 
> > > > > > > > diff --git a/common/rc b/common/rc
> > > > > > > > index 524ffa02aa..0b69f7f54f 100644
> > > > > > > > --- a/common/rc
> > > > > > > > +++ b/common/rc
> > > > > > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > > > > > >  
> > > > > > > >      case $FSTYP in
> > > > > > > >      xfs)
> > > > > > > > -	local scratch_log="none"
> > > > > > > > -	local scratch_rt="none"
> > > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > > -
> > > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > > -
> > > > > > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > > +	_check_xfs_scratch_fs $device
> > > > > > > >  	;;
> > > > > > > >      udf)
> > > > > > > >  	_check_udf_filesystem $device $udf_fsize
> > > > > > > > diff --git a/common/xfs b/common/xfs
> > > > > > > > index 248ccefda3..6a48960a7f 100644
> > > > > > > > --- a/common/xfs
> > > > > > > > +++ b/common/xfs
> > > > > > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > > > > > >  	return $?
> > > > > > > >  }
> > > > > > > >  
> > > > > > > > +_check_xfs_scratch_fs()
> > > > > > > > +{
> > > > > > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > > > > > +	local scratch_log="none"
> > > > > > > > +	local scratch_rt="none"
> > > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > > +
> > > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > > +
> > > > > > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > > +}
> > > > > > > > +
> > > > > > > >  # modeled after _scratch_xfs_repair
> > > > > > > >  _test_xfs_repair()
> > > > > > > >  {
> > > > > > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > > > > > 
> > > > > > > Hi Darrick,
> > > > > > > 
> > > > > > > Thanks for this improvement.
> > > > > > > 
> > > > > > > I'm wondering do we need a separated common file only for xfs metadump
> > > > > > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > > > > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > > > > > in common/xfs.
> > > > > > 
> > > > > > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > > > > > files are getting big:
> > > > > > 
> > > > > >    509 fstests/common/overlay
> > > > > >    523 fstests/common/quota
> > > > > >    550 fstests/common/reflink
> > > > > >    638 fstests/common/log
> > > > > >    640 fstests/common/punch
> > > > > >    663 fstests/common/filter
> > > > > >    794 fstests/common/btrfs
> > > > > >    936 fstests/common/config
> > > > > >   1030 fstests/common/encrypt
> > > > > >   1162 fstests/common/populate
> > > > > >   1519 fstests/common/fuzzy
> > > > > >   1531 fstests/common/dump
> > > > > >   2218 fstests/common/xfs
> > > > > >   5437 fstests/common/rc
> > > > > > 
> > > > > > with common/xfs being particularly larger than most everything else.
> > > > > 
> > > > > Haha, maybe we'll have a common/xfs/ directory in one day :)
> > > > > 
> > > > > > 
> > > > > > Since the common/xfs_metadump_tests functions are shared helper
> > > > > > functions for testing metadump/mdrestore that are not use by most tests,
> > > > > > I decided that a split was appropriate both to maintain the (ha!)
> > > > > > cohesion of common/xfs and not add more bash parsing costs to every
> > > > > > single testcase.
> > > > > 
> > > > > OK, a split makes sense, but I have 3 questions:
> > > > > 1) Will you move all metadump helpers from common/xfs to this new file?
> > > > 
> > > > I don't really want to, because that's another patch and would require
> > > > careful auditing of all the tests to find the ones that want to use
> > > > metadump but aren't themselves functional tests of metadump.
> > > > 
> > > > IOWs, this new file really is for shared metadump functional testing and
> > > > not much else.
> > > 
> > > OK, I think we can care about this part step by step in the future.
> > 
> > <nod>
> > 
> > > > 
> > > > > 2) Can we call it common/metadump? (Not sure if any other fs has metadump
> > > > >    things:)
> > > > 
> > > > Yes, e2image does this for ext*.
> > > 
> > > OK, I'll rename this file to common/metadump, and change other patches to
> > > souce the new name when I merge this patchset. Is that good to you?
> > 
> > Sorry, I realized that my statement was ambiguous -- the "yes" applies
> > to "Not sure if any other fs has metadump things", not "Can we call it
> > common/metadump?".  The code in common/xfs_metadump_tests is specific to
> > xfs and is not used for e2image testing, so let's leave the name as-is.
> 
> Oh, maybe it can be used for e2image or other metadump helpers later, if
> we call it common/metadump?

Sounds ok to me.  Want me to respin?

--D

> > 
> > --D
> > 
> > > Thanks,
> > > Zorro
> > > 
> > > > 
> > > > > 3) Or move to common/dump directly? (looks not proper ;-)
> > > > 
> > > > dump != metadump; one is for all the files in the fs and none of the
> > > > non-file metadata; the other is for metadata and none of the files.
> > > > 
> > > > --D
> > > > 
> > > > > Thanks,
> > > > > Zorro
> > > > > 
> > > > > > 
> > > > > > --D
> > > > > > 
> > > > > > > Thanks,
> > > > > > > Zorro
> > > > > > > 
> > > > > > > > new file mode 100644
> > > > > > > > index 0000000000..dd3dec1fb4
> > > > > > > > --- /dev/null
> > > > > > > > +++ b/common/xfs_metadump_tests
> > > > > > > > @@ -0,0 +1,123 @@
> > > > > > > > +#
> > > > > > > > +# XFS specific metadump testing functions.
> > > > > > > > +#
> > > > > > > > +
> > > > > > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > > > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > > > > > +_setup_verify_metadump()
> > > > > > > > +{
> > > > > > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > > > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > > > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > > > > > +
> > > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +_cleanup_verify_metadump()
> > > > > > > > +{
> > > > > > > > +	_scratch_unmount &>> $seqres.full
> > > > > > > > +
> > > > > > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > > > > > +		losetup -d "$ldev"
> > > > > > > > +	done
> > > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > > > > > +# images and fsck them.
> > > > > > > > +_verify_metadump_v1()
> > > > > > > > +{
> > > > > > > > +	local metadump_args="$1"
> > > > > > > > +	local extra_test="$2"
> > > > > > > > +
> > > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > > +	local version=""
> > > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > > +	local data_loop
> > > > > > > > +
> > > > > > > > +	# Force v1 if we detect v2 support
> > > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > > > > > +		version="-v 1"
> > > > > > > > +	fi
> > > > > > > > +
> > > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > > +
> > > > > > > > +	# Restore metadump, which creates data_img
> > > > > > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > > > > > +
> > > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > > +
> > > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > > > > > +	fi
> > > > > > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > > +
> > > > > > > > +	# Tear down what we created
> > > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > > +	rm -f $data_img
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > > > > > +# images and fsck them.
> > > > > > > > +_verify_metadump_v2()
> > > > > > > > +{
> > > > > > > > +	local metadump_args="$1"
> > > > > > > > +	local extra_test="$2"
> > > > > > > > +
> > > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > > +	local version="-v 2"
> > > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > > +	local data_loop
> > > > > > > > +	local log_img=""
> > > > > > > > +	local log_loop
> > > > > > > > +
> > > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > > +
> > > > > > > > +	#
> > > > > > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > +	# from such a metadump file.
> > > > > > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > > > > > +
> > > > > > > > +	# Restore metadump, which creates data_img and log_img
> > > > > > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > > > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > > > > > +
> > > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > > +
> > > > > > > > +	# Create loopdev for log device if we recovered anything
> > > > > > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > > > > > +
> > > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > > > > > +	fi
> > > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > > +
> > > > > > > > +	# Tear down what we created
> > > > > > > > +	if [ -b "$log_loop" ]; then
> > > > > > > > +		_destroy_loop_device $log_loop
> > > > > > > > +		rm -f $log_img
> > > > > > > > +	fi
> > > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > > +	rm -f $data_img
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +# Verify both metadump formats if possible
> > > > > > > > +_verify_metadumps()
> > > > > > > > +{
> > > > > > > > +	_verify_metadump_v1 "$@"
> > > > > > > > +
> > > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > > > > > +		_verify_metadump_v2 "$@"
> > > > > > > > +	fi
> > > > > > > > +}
> > > > > > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > > > > > index cdac2349df..c3a9bcefee 100755
> > > > > > > > --- a/tests/xfs/129
> > > > > > > > +++ b/tests/xfs/129
> > > > > > > > @@ -16,98 +16,23 @@ _cleanup()
> > > > > > > >  {
> > > > > > > >      cd /
> > > > > > > >      _scratch_unmount > /dev/null 2>&1
> > > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > > > > > -       $TEST_DIR/log-image
> > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/filter
> > > > > > > >  . ./common/reflink
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  # real QA test starts here
> > > > > > > >  _supported_fs xfs
> > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > >  _require_loop
> > > > > > > >  _require_scratch_reflink
> > > > > > > > -
> > > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > > -
> > > > > > > > -verify_metadump_v1()
> > > > > > > > -{
> > > > > > > > -	local max_version=$1
> > > > > > > > -	local version=""
> > > > > > > > -
> > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > -		version="-v 1"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -verify_metadump_v2()
> > > > > > > > -{
> > > > > > > > -	version="-v 2"
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > -
> > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > -	# from such a metadump file.
> > > > > > > > -	slogdev=""
> > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > -
> > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > -		logdev=""
> > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > > -
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > >  _scratch_mount
> > > > > > > >  
> > > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > > @@ -127,12 +52,7 @@ done
> > > > > > > >  _scratch_unmount
> > > > > > > >  
> > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > -
> > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > -
> > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > -	verify_metadump_v2
> > > > > > > > -fi
> > > > > > > > +_verify_metadumps
> > > > > > > >  
> > > > > > > >  # success, all done
> > > > > > > >  status=0
> > > > > > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > > > > > index f4f8af6d3a..8f808c7507 100755
> > > > > > > > --- a/tests/xfs/234
> > > > > > > > +++ b/tests/xfs/234
> > > > > > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > > > > > >  _cleanup()
> > > > > > > >  {
> > > > > > > >      cd /
> > > > > > > > -    _scratch_unmount > /dev/null 2>&1
> > > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > > > > > -       $TEST_DIR/log-image
> > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/filter
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  # real QA test starts here
> > > > > > > >  _supported_fs xfs
> > > > > > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > >  _require_loop
> > > > > > > >  _require_xfs_scratch_rmapbt
> > > > > > > >  _require_xfs_io_command "fpunch"
> > > > > > > > -
> > > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > > -
> > > > > > > > -verify_metadump_v1()
> > > > > > > > -{
> > > > > > > > -	local max_version=$1
> > > > > > > > -	local version=""
> > > > > > > > -
> > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > -		version="-v 1"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -verify_metadump_v2()
> > > > > > > > -{
> > > > > > > > -	version="-v 2"
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > -
> > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > -	# from such a metadump file.
> > > > > > > > -	slogdev=""
> > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > -
> > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > -		logdev=""
> > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > > -
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > >  _scratch_mount
> > > > > > > >  
> > > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > > @@ -127,12 +51,7 @@ done
> > > > > > > >  _scratch_unmount
> > > > > > > >  
> > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > -
> > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > -
> > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > -	verify_metadump_v2
> > > > > > > > -fi
> > > > > > > > +_verify_metadumps
> > > > > > > >  
> > > > > > > >  # success, all done
> > > > > > > >  status=0
> > > > > > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > > > > > index 3b567999d8..6623c435e5 100755
> > > > > > > > --- a/tests/xfs/253
> > > > > > > > +++ b/tests/xfs/253
> > > > > > > > @@ -26,23 +26,21 @@ _cleanup()
> > > > > > > >      cd /
> > > > > > > >      rm -f $tmp.*
> > > > > > > >      rm -rf "${OUTPUT_DIR}"
> > > > > > > > -    rm -f "${METADUMP_FILE}"
> > > > > > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > +    _cleanup_verify_metadump
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/filter
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > >  _require_test
> > > > > > > >  _require_scratch
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > >  # real QA test starts here
> > > > > > > >  
> > > > > > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > > > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > > > > > >  ORPHANAGE="lost+found"
> > > > > > > >  
> > > > > > > >  _supported_fs xfs
> > > > > > > > @@ -52,24 +50,7 @@ function create_file() {
> > > > > > > >  	touch $(printf "$@")
> > > > > > > >  }
> > > > > > > >  
> > > > > > > > -verify_metadump_v1()
> > > > > > > > -{
> > > > > > > > -	local max_version=$1
> > > > > > > > -	local version=""
> > > > > > > > -
> > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > -		version="-v 1"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > -
> > > > > > > > +extra_test() {
> > > > > > > >  	cd "${SCRATCH_MNT}"
> > > > > > > >  
> > > > > > > >  	# Get a listing of all the files after obfuscation
> > > > > > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > > > > > >  	ls -R | od -c >> $seqres.full
> > > > > > > >  
> > > > > > > >  	cd /
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -verify_metadump_v2()
> > > > > > > > -{
> > > > > > > > -	version="-v 2"
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > > -
> > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > -	# from such a metadump file.
> > > > > > > > -	slogdev=""
> > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > -
> > > > > > > > -	cd "${SCRATCH_MNT}"
> > > > > > > > -
> > > > > > > > -	# Get a listing of all the files after obfuscation
> > > > > > > > -	echo "Metadump v2" >> $seqres.full
> > > > > > > > -	ls -R >> $seqres.full
> > > > > > > > -	ls -R | od -c >> $seqres.full
> > > > > > > > -
> > > > > > > > -	cd /
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > -		logdev=""
> > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  echo "Disciplyne of silence is goed."
> > > > > > > > @@ -233,13 +160,7 @@ cd $here
> > > > > > > >  
> > > > > > > >  _scratch_unmount
> > > > > > > >  
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > -
> > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > -	verify_metadump_v2
> > > > > > > > -fi
> > > > > > > > +_verify_metadumps '' extra_test
> > > > > > > >  
> > > > > > > >  # Finally, re-make the filesystem since to ensure we don't
> > > > > > > >  # leave a directory with duplicate entries lying around.
> > > > > > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > > > > > index 1433140821..c475d89ad9 100755
> > > > > > > > --- a/tests/xfs/291
> > > > > > > > +++ b/tests/xfs/291
> > > > > > > > @@ -9,11 +9,21 @@
> > > > > > > >  . ./common/preamble
> > > > > > > >  _begin_fstest auto repair metadump
> > > > > > > >  
> > > > > > > > +# Override the default cleanup function.
> > > > > > > > +_cleanup()
> > > > > > > > +{
> > > > > > > > +	cd /
> > > > > > > > +	rm -r -f $tmp.*
> > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > +}
> > > > > > > > +
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/filter
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  _supported_fs xfs
> > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > >  # real QA test starts here
> > > > > > > >  _require_scratch
> > > > > > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > > > > > >  
> > > > > > > >  # Yes they can!  Now...
> > > > > > > >  # Can xfs_metadump cope with this monster?
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > -	version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v $md_version"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > > > > > -		_fail "xfs_metadump failed"
> > > > > > > > -
> > > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > > -		slogdev=""
> > > > > > > > -	fi
> > > > > > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > > > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > > > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > > > > > -		_fail "xfs_repair of metadump failed"
> > > > > > > > -done
> > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > >  
> > > > > > > >  # Yes it can; success, all done
> > > > > > > >  status=0
> > > > > > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > > > > > index 7e402aa88f..579e1b556a 100755
> > > > > > > > --- a/tests/xfs/432
> > > > > > > > +++ b/tests/xfs/432
> > > > > > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > > > > > >  _cleanup()
> > > > > > > >  {
> > > > > > > >  	cd /
> > > > > > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > > > > > +	rm -f "$tmp".*
> > > > > > > > +	_cleanup_verify_metadump
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/filter
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  # real QA test starts here
> > > > > > > >  _supported_fs xfs
> > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > >  _require_scratch
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > >  rm -f "$seqres.full"
> > > > > > > >  
> > > > > > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > > > > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > > > > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > > > > > >  
> > > > > > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > > > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > > > > > -rm -f $metadump_file $metadump_img
> > > > > > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > > > > > >  max_fname_len=255
> > > > > > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > > > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > > > > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > > > > > >  
> > > > > > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > -	version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v $md_version"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > > > > > -
> > > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > > -		slogdev=""
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > > > > > -		echo "xfs_repair on restored fs returned $?"
> > > > > > > > -done
> > > > > > > > +_verify_metadumps '-a -o -w'
> > > > > > > >  
> > > > > > > >  # success, all done
> > > > > > > >  status=0
> > > > > > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > > > > > index 8643c3d483..ff6b344a9c 100755
> > > > > > > > --- a/tests/xfs/503
> > > > > > > > +++ b/tests/xfs/503
> > > > > > > > @@ -17,11 +17,13 @@ _cleanup()
> > > > > > > >  {
> > > > > > > >  	cd /
> > > > > > > >  	rm -rf $tmp.* $testdir
> > > > > > > > +	_cleanup_verify_metadump
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/filter
> > > > > > > >  . ./common/populate
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  testdir=$TEST_DIR/test-$seq
> > > > > > > >  
> > > > > > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > > > > > >  _require_populate_commands
> > > > > > > >  _xfs_skip_online_rebuild
> > > > > > > >  _xfs_skip_offline_rebuild
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > >  echo "Format and populate"
> > > > > > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > > > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > > > > > >  metadump_file=$testdir/scratch.md
> > > > > > > >  copy_file=$testdir/copy.img
> > > > > > > >  
> > > > > > > > -check_restored_metadump_image()
> > > > > > > > -{
> > > > > > > > -	local image=$1
> > > > > > > > -
> > > > > > > > -	loop_dev=$(_create_loop_device $image)
> > > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > > > > > -	_destroy_loop_device $loop_dev
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > >  echo "metadump and mdrestore"
> > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > -	version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v $md_version"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > -done
> > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > >  
> > > > > > > >  echo "metadump a and mdrestore"
> > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > -	version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v $md_version"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > -done
> > > > > > > > +_verify_metadumps '-a'
> > > > > > > >  
> > > > > > > >  echo "metadump g and mdrestore"
> > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > -	version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v $md_version"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > -done
> > > > > > > > +_verify_metadumps '-g' >> $seqres.full
> > > > > > > >  
> > > > > > > >  echo "metadump ag and mdrestore"
> > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > -	version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v $md_version"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > -done
> > > > > > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > > > > > >  
> > > > > > > >  echo copy
> > > > > > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > > > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > > > > > index f2cd7aba98..af917f0f32 100755
> > > > > > > > --- a/tests/xfs/605
> > > > > > > > +++ b/tests/xfs/605
> > > > > > > > @@ -15,17 +15,13 @@ _cleanup()
> > > > > > > >  {
> > > > > > > >  	cd /
> > > > > > > >  	rm -r -f $tmp.*
> > > > > > > > -	_scratch_unmount > /dev/null 2>&1
> > > > > > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > > > > > -	   $TEST_DIR/log-image
> > > > > > > > +	_cleanup_verify_metadump
> > > > > > > >  }
> > > > > > > >  
> > > > > > > >  # Import common functions.
> > > > > > > >  . ./common/dmflakey
> > > > > > > >  . ./common/inject
> > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > >  
> > > > > > > >  # real QA test starts here
> > > > > > > >  _supported_fs xfs
> > > > > > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > > > > > >  _require_dm_target flakey
> > > > > > > >  _require_xfs_io_command "pwrite"
> > > > > > > >  _require_test_program "punch-alternating"
> > > > > > > > +_setup_verify_metadump
> > > > > > > >  
> > > > > > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > > > > > >  testfile=${SCRATCH_MNT}/testfile
> > > > > > > >  
> > > > > > > >  echo "Format filesystem on scratch device"
> > > > > > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > > > > > >  
> > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > -
> > > > > > > >  external_log=0
> > > > > > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > > > > > >  	external_log=1
> > > > > > > >  fi
> > > > > > > >  
> > > > > > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > > > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > > > > > >  	_notrun "metadump v1 does not support external log device"
> > > > > > > >  fi
> > > > > > > >  
> > > > > > > > -verify_metadump_v1()
> > > > > > > > -{
> > > > > > > > -	local version=""
> > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > -		version="-v 1"
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -verify_metadump_v2()
> > > > > > > > -{
> > > > > > > > -	local version="-v 2"
> > > > > > > > -
> > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > > -
> > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > -	# from such a metadump file.
> > > > > > > > -	slogdev=""
> > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > -
> > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > -
> > > > > > > > -	logdev=""
> > > > > > > > -	if [[ -s $slogdev ]]; then
> > > > > > > > -		logdev=$(_create_loop_device $slogdev)
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > -
> > > > > > > > -	if [[ -s $logdev ]]; then
> > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > -		logdev=""
> > > > > > > > -		rm -f $slogdev
> > > > > > > > -	fi
> > > > > > > > -
> > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > -	datadev=""
> > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > -}
> > > > > > > > -
> > > > > > > >  echo "Initialize and mount filesystem on flakey device"
> > > > > > > >  _init_flakey
> > > > > > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > > > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > > > > > >  _print_logstate
> > > > > > > >  
> > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > -
> > > > > > > > -if [[ $external_log == 0 ]]; then
> > > > > > > > -	verify_metadump_v1 $max_md_version
> > > > > > > > -fi
> > > > > > > > -
> > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > -	verify_metadump_v2
> > > > > > > > -fi
> > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > >  
> > > > > > > >  # Mount the fs to replay the contents from the dirty log.
> > > > > > > >  _scratch_mount
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > 
> > > 
> > 
> 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Fri, Feb 02, 2024 at 08:47:08AM -0800, Darrick J. Wong wrote:
> On Thu, Feb 01, 2024 at 02:29:22PM +0800, Zorro Lang wrote:
> > On Wed, Jan 31, 2024 at 11:24:33AM -0800, Darrick J. Wong wrote:
> > > On Wed, Jan 31, 2024 at 10:09:19PM +0800, Zorro Lang wrote:
> > > > On Mon, Jan 29, 2024 at 05:32:07PM -0800, Darrick J. Wong wrote:
> > > > > On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> > > > > > On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > > > > > > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > > > > > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > > > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > > > > > 
> > > > > > > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > > > > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > > > > > > duplicate copies of the same code.
> > > > > > > > > 
> > > > > > > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > > > > > > 
> > > > > > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > > > > > ---
> > > > > > > > >  common/rc                 |   10 ----
> > > > > > > > >  common/xfs                |   14 +++++
> > > > > > > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > > > > > > >  tests/xfs/129             |   90 ++-------------------------------
> > > > > > > > >  tests/xfs/234             |   91 ++-------------------------------
> > > > > > > > >  tests/xfs/253             |   89 ++-------------------------------
> > > > > > > > >  tests/xfs/291             |   31 ++++-------
> > > > > > > > >  tests/xfs/432             |   30 ++---------
> > > > > > > > >  tests/xfs/503             |   60 +++-------------------
> > > > > > > > >  tests/xfs/605             |   84 ++-----------------------------
> > > > > > > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > > > > > > >  create mode 100644 common/xfs_metadump_tests
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > diff --git a/common/rc b/common/rc
> > > > > > > > > index 524ffa02aa..0b69f7f54f 100644
> > > > > > > > > --- a/common/rc
> > > > > > > > > +++ b/common/rc
> > > > > > > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > > > > > > >  
> > > > > > > > >      case $FSTYP in
> > > > > > > > >      xfs)
> > > > > > > > > -	local scratch_log="none"
> > > > > > > > > -	local scratch_rt="none"
> > > > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > > > -
> > > > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > > > -
> > > > > > > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > > > +	_check_xfs_scratch_fs $device
> > > > > > > > >  	;;
> > > > > > > > >      udf)
> > > > > > > > >  	_check_udf_filesystem $device $udf_fsize
> > > > > > > > > diff --git a/common/xfs b/common/xfs
> > > > > > > > > index 248ccefda3..6a48960a7f 100644
> > > > > > > > > --- a/common/xfs
> > > > > > > > > +++ b/common/xfs
> > > > > > > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > > > > > > >  	return $?
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > > +_check_xfs_scratch_fs()
> > > > > > > > > +{
> > > > > > > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > > > > > > +	local scratch_log="none"
> > > > > > > > > +	local scratch_rt="none"
> > > > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > > > +
> > > > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > > > +
> > > > > > > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > >  # modeled after _scratch_xfs_repair
> > > > > > > > >  _test_xfs_repair()
> > > > > > > > >  {
> > > > > > > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > > > > > > 
> > > > > > > > Hi Darrick,
> > > > > > > > 
> > > > > > > > Thanks for this improvement.
> > > > > > > > 
> > > > > > > > I'm wondering do we need a separated common file only for xfs metadump
> > > > > > > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > > > > > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > > > > > > in common/xfs.
> > > > > > > 
> > > > > > > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > > > > > > files are getting big:
> > > > > > > 
> > > > > > >    509 fstests/common/overlay
> > > > > > >    523 fstests/common/quota
> > > > > > >    550 fstests/common/reflink
> > > > > > >    638 fstests/common/log
> > > > > > >    640 fstests/common/punch
> > > > > > >    663 fstests/common/filter
> > > > > > >    794 fstests/common/btrfs
> > > > > > >    936 fstests/common/config
> > > > > > >   1030 fstests/common/encrypt
> > > > > > >   1162 fstests/common/populate
> > > > > > >   1519 fstests/common/fuzzy
> > > > > > >   1531 fstests/common/dump
> > > > > > >   2218 fstests/common/xfs
> > > > > > >   5437 fstests/common/rc
> > > > > > > 
> > > > > > > with common/xfs being particularly larger than most everything else.
> > > > > > 
> > > > > > Haha, maybe we'll have a common/xfs/ directory in one day :)
> > > > > > 
> > > > > > > 
> > > > > > > Since the common/xfs_metadump_tests functions are shared helper
> > > > > > > functions for testing metadump/mdrestore that are not use by most tests,
> > > > > > > I decided that a split was appropriate both to maintain the (ha!)
> > > > > > > cohesion of common/xfs and not add more bash parsing costs to every
> > > > > > > single testcase.
> > > > > > 
> > > > > > OK, a split makes sense, but I have 3 questions:
> > > > > > 1) Will you move all metadump helpers from common/xfs to this new file?
> > > > > 
> > > > > I don't really want to, because that's another patch and would require
> > > > > careful auditing of all the tests to find the ones that want to use
> > > > > metadump but aren't themselves functional tests of metadump.
> > > > > 
> > > > > IOWs, this new file really is for shared metadump functional testing and
> > > > > not much else.
> > > > 
> > > > OK, I think we can care about this part step by step in the future.
> > > 
> > > <nod>
> > > 
> > > > > 
> > > > > > 2) Can we call it common/metadump? (Not sure if any other fs has metadump
> > > > > >    things:)
> > > > > 
> > > > > Yes, e2image does this for ext*.
> > > > 
> > > > OK, I'll rename this file to common/metadump, and change other patches to
> > > > souce the new name when I merge this patchset. Is that good to you?
> > > 
> > > Sorry, I realized that my statement was ambiguous -- the "yes" applies
> > > to "Not sure if any other fs has metadump things", not "Can we call it
> > > common/metadump?".  The code in common/xfs_metadump_tests is specific to
> > > xfs and is not used for e2image testing, so let's leave the name as-is.
> > 
> > Oh, maybe it can be used for e2image or other metadump helpers later, if
> > we call it common/metadump?
> 
> Sounds ok to me.  Want me to respin?

No, if it's good to you, I can change that on my side. But your might need to
rebase on it later :)

> 
> --D
> 
> > > 
> > > --D
> > > 
> > > > Thanks,
> > > > Zorro
> > > > 
> > > > > 
> > > > > > 3) Or move to common/dump directly? (looks not proper ;-)
> > > > > 
> > > > > dump != metadump; one is for all the files in the fs and none of the
> > > > > non-file metadata; the other is for metadata and none of the files.
> > > > > 
> > > > > --D
> > > > > 
> > > > > > Thanks,
> > > > > > Zorro
> > > > > > 
> > > > > > > 
> > > > > > > --D
> > > > > > > 
> > > > > > > > Thanks,
> > > > > > > > Zorro
> > > > > > > > 
> > > > > > > > > new file mode 100644
> > > > > > > > > index 0000000000..dd3dec1fb4
> > > > > > > > > --- /dev/null
> > > > > > > > > +++ b/common/xfs_metadump_tests
> > > > > > > > > @@ -0,0 +1,123 @@
> > > > > > > > > +#
> > > > > > > > > +# XFS specific metadump testing functions.
> > > > > > > > > +#
> > > > > > > > > +
> > > > > > > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > > > > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > > > > > > +_setup_verify_metadump()
> > > > > > > > > +{
> > > > > > > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > > > > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > > > > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > > > > > > +
> > > > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +_cleanup_verify_metadump()
> > > > > > > > > +{
> > > > > > > > > +	_scratch_unmount &>> $seqres.full
> > > > > > > > > +
> > > > > > > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > > > > > > +		losetup -d "$ldev"
> > > > > > > > > +	done
> > > > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > > > > > > +# images and fsck them.
> > > > > > > > > +_verify_metadump_v1()
> > > > > > > > > +{
> > > > > > > > > +	local metadump_args="$1"
> > > > > > > > > +	local extra_test="$2"
> > > > > > > > > +
> > > > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > > > +	local version=""
> > > > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > > > +	local data_loop
> > > > > > > > > +
> > > > > > > > > +	# Force v1 if we detect v2 support
> > > > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > > > > > > +		version="-v 1"
> > > > > > > > > +	fi
> > > > > > > > > +
> > > > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > > > +
> > > > > > > > > +	# Restore metadump, which creates data_img
> > > > > > > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > +
> > > > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > > > +
> > > > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > > > > > > +	fi
> > > > > > > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > > > +
> > > > > > > > > +	# Tear down what we created
> > > > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > > > +	rm -f $data_img
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > > > > > > +# images and fsck them.
> > > > > > > > > +_verify_metadump_v2()
> > > > > > > > > +{
> > > > > > > > > +	local metadump_args="$1"
> > > > > > > > > +	local extra_test="$2"
> > > > > > > > > +
> > > > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > > > +	local version="-v 2"
> > > > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > > > +	local data_loop
> > > > > > > > > +	local log_img=""
> > > > > > > > > +	local log_loop
> > > > > > > > > +
> > > > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > > > +
> > > > > > > > > +	#
> > > > > > > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > +	# from such a metadump file.
> > > > > > > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > > > > > > +
> > > > > > > > > +	# Restore metadump, which creates data_img and log_img
> > > > > > > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > > > > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > > > > > > +
> > > > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > > > +
> > > > > > > > > +	# Create loopdev for log device if we recovered anything
> > > > > > > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > > > > > > +
> > > > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > > > > > > +	fi
> > > > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > > > +
> > > > > > > > > +	# Tear down what we created
> > > > > > > > > +	if [ -b "$log_loop" ]; then
> > > > > > > > > +		_destroy_loop_device $log_loop
> > > > > > > > > +		rm -f $log_img
> > > > > > > > > +	fi
> > > > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > > > +	rm -f $data_img
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +# Verify both metadump formats if possible
> > > > > > > > > +_verify_metadumps()
> > > > > > > > > +{
> > > > > > > > > +	_verify_metadump_v1 "$@"
> > > > > > > > > +
> > > > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > > > > > > +		_verify_metadump_v2 "$@"
> > > > > > > > > +	fi
> > > > > > > > > +}
> > > > > > > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > > > > > > index cdac2349df..c3a9bcefee 100755
> > > > > > > > > --- a/tests/xfs/129
> > > > > > > > > +++ b/tests/xfs/129
> > > > > > > > > @@ -16,98 +16,23 @@ _cleanup()
> > > > > > > > >  {
> > > > > > > > >      cd /
> > > > > > > > >      _scratch_unmount > /dev/null 2>&1
> > > > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > > > > > > -       $TEST_DIR/log-image
> > > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/filter
> > > > > > > > >  . ./common/reflink
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  # real QA test starts here
> > > > > > > > >  _supported_fs xfs
> > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > >  _require_loop
> > > > > > > > >  _require_scratch_reflink
> > > > > > > > > -
> > > > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v1()
> > > > > > > > > -{
> > > > > > > > > -	local max_version=$1
> > > > > > > > > -	local version=""
> > > > > > > > > -
> > > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > > -		version="-v 1"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v2()
> > > > > > > > > -{
> > > > > > > > > -	version="-v 2"
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > -
> > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > -	# from such a metadump file.
> > > > > > > > > -	slogdev=""
> > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > -
> > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > -		logdev=""
> > > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > > > -
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > >  _scratch_mount
> > > > > > > > >  
> > > > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > > > @@ -127,12 +52,7 @@ done
> > > > > > > > >  _scratch_unmount
> > > > > > > > >  
> > > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > > -
> > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -	verify_metadump_v2
> > > > > > > > > -fi
> > > > > > > > > +_verify_metadumps
> > > > > > > > >  
> > > > > > > > >  # success, all done
> > > > > > > > >  status=0
> > > > > > > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > > > > > > index f4f8af6d3a..8f808c7507 100755
> > > > > > > > > --- a/tests/xfs/234
> > > > > > > > > +++ b/tests/xfs/234
> > > > > > > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > > > > > > >  _cleanup()
> > > > > > > > >  {
> > > > > > > > >      cd /
> > > > > > > > > -    _scratch_unmount > /dev/null 2>&1
> > > > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > > > > > > -       $TEST_DIR/log-image
> > > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/filter
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  # real QA test starts here
> > > > > > > > >  _supported_fs xfs
> > > > > > > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > >  _require_loop
> > > > > > > > >  _require_xfs_scratch_rmapbt
> > > > > > > > >  _require_xfs_io_command "fpunch"
> > > > > > > > > -
> > > > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v1()
> > > > > > > > > -{
> > > > > > > > > -	local max_version=$1
> > > > > > > > > -	local version=""
> > > > > > > > > -
> > > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > > -		version="-v 1"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v2()
> > > > > > > > > -{
> > > > > > > > > -	version="-v 2"
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > -
> > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > -	# from such a metadump file.
> > > > > > > > > -	slogdev=""
> > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > -
> > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > -		logdev=""
> > > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > > > -
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > >  _scratch_mount
> > > > > > > > >  
> > > > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > > > @@ -127,12 +51,7 @@ done
> > > > > > > > >  _scratch_unmount
> > > > > > > > >  
> > > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > > -
> > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -	verify_metadump_v2
> > > > > > > > > -fi
> > > > > > > > > +_verify_metadumps
> > > > > > > > >  
> > > > > > > > >  # success, all done
> > > > > > > > >  status=0
> > > > > > > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > > > > > > index 3b567999d8..6623c435e5 100755
> > > > > > > > > --- a/tests/xfs/253
> > > > > > > > > +++ b/tests/xfs/253
> > > > > > > > > @@ -26,23 +26,21 @@ _cleanup()
> > > > > > > > >      cd /
> > > > > > > > >      rm -f $tmp.*
> > > > > > > > >      rm -rf "${OUTPUT_DIR}"
> > > > > > > > > -    rm -f "${METADUMP_FILE}"
> > > > > > > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/filter
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > >  _require_test
> > > > > > > > >  _require_scratch
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > >  # real QA test starts here
> > > > > > > > >  
> > > > > > > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > > > > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > > > > > > >  ORPHANAGE="lost+found"
> > > > > > > > >  
> > > > > > > > >  _supported_fs xfs
> > > > > > > > > @@ -52,24 +50,7 @@ function create_file() {
> > > > > > > > >  	touch $(printf "$@")
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > > -verify_metadump_v1()
> > > > > > > > > -{
> > > > > > > > > -	local max_version=$1
> > > > > > > > > -	local version=""
> > > > > > > > > -
> > > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > > -		version="-v 1"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > -
> > > > > > > > > +extra_test() {
> > > > > > > > >  	cd "${SCRATCH_MNT}"
> > > > > > > > >  
> > > > > > > > >  	# Get a listing of all the files after obfuscation
> > > > > > > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > > > > > > >  	ls -R | od -c >> $seqres.full
> > > > > > > > >  
> > > > > > > > >  	cd /
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v2()
> > > > > > > > > -{
> > > > > > > > > -	version="-v 2"
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > > > -
> > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > -	# from such a metadump file.
> > > > > > > > > -	slogdev=""
> > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > -
> > > > > > > > > -	cd "${SCRATCH_MNT}"
> > > > > > > > > -
> > > > > > > > > -	# Get a listing of all the files after obfuscation
> > > > > > > > > -	echo "Metadump v2" >> $seqres.full
> > > > > > > > > -	ls -R >> $seqres.full
> > > > > > > > > -	ls -R | od -c >> $seqres.full
> > > > > > > > > -
> > > > > > > > > -	cd /
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > -		logdev=""
> > > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  echo "Disciplyne of silence is goed."
> > > > > > > > > @@ -233,13 +160,7 @@ cd $here
> > > > > > > > >  
> > > > > > > > >  _scratch_unmount
> > > > > > > > >  
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > > -
> > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -	verify_metadump_v2
> > > > > > > > > -fi
> > > > > > > > > +_verify_metadumps '' extra_test
> > > > > > > > >  
> > > > > > > > >  # Finally, re-make the filesystem since to ensure we don't
> > > > > > > > >  # leave a directory with duplicate entries lying around.
> > > > > > > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > > > > > > index 1433140821..c475d89ad9 100755
> > > > > > > > > --- a/tests/xfs/291
> > > > > > > > > +++ b/tests/xfs/291
> > > > > > > > > @@ -9,11 +9,21 @@
> > > > > > > > >  . ./common/preamble
> > > > > > > > >  _begin_fstest auto repair metadump
> > > > > > > > >  
> > > > > > > > > +# Override the default cleanup function.
> > > > > > > > > +_cleanup()
> > > > > > > > > +{
> > > > > > > > > +	cd /
> > > > > > > > > +	rm -r -f $tmp.*
> > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/filter
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  _supported_fs xfs
> > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > >  # real QA test starts here
> > > > > > > > >  _require_scratch
> > > > > > > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > > > > > > >  
> > > > > > > > >  # Yes they can!  Now...
> > > > > > > > >  # Can xfs_metadump cope with this monster?
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > -	version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v $md_version"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > > > > > > -		_fail "xfs_metadump failed"
> > > > > > > > > -
> > > > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > > > -		slogdev=""
> > > > > > > > > -	fi
> > > > > > > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > > > > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > > > > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > > > > > > -		_fail "xfs_repair of metadump failed"
> > > > > > > > > -done
> > > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > > >  
> > > > > > > > >  # Yes it can; success, all done
> > > > > > > > >  status=0
> > > > > > > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > > > > > > index 7e402aa88f..579e1b556a 100755
> > > > > > > > > --- a/tests/xfs/432
> > > > > > > > > +++ b/tests/xfs/432
> > > > > > > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > > > > > > >  _cleanup()
> > > > > > > > >  {
> > > > > > > > >  	cd /
> > > > > > > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > > > > > > +	rm -f "$tmp".*
> > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/filter
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  # real QA test starts here
> > > > > > > > >  _supported_fs xfs
> > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > >  _require_scratch
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > >  rm -f "$seqres.full"
> > > > > > > > >  
> > > > > > > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > > > > > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > > > > > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > > > > > > >  
> > > > > > > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > > > > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > > > > > > -rm -f $metadump_file $metadump_img
> > > > > > > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > > > > > > >  max_fname_len=255
> > > > > > > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > > > > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > > > > > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > > > > > > >  
> > > > > > > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > -	version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v $md_version"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > > > > > > -
> > > > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > > > -		slogdev=""
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > > > > > > -		echo "xfs_repair on restored fs returned $?"
> > > > > > > > > -done
> > > > > > > > > +_verify_metadumps '-a -o -w'
> > > > > > > > >  
> > > > > > > > >  # success, all done
> > > > > > > > >  status=0
> > > > > > > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > > > > > > index 8643c3d483..ff6b344a9c 100755
> > > > > > > > > --- a/tests/xfs/503
> > > > > > > > > +++ b/tests/xfs/503
> > > > > > > > > @@ -17,11 +17,13 @@ _cleanup()
> > > > > > > > >  {
> > > > > > > > >  	cd /
> > > > > > > > >  	rm -rf $tmp.* $testdir
> > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/filter
> > > > > > > > >  . ./common/populate
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  testdir=$TEST_DIR/test-$seq
> > > > > > > > >  
> > > > > > > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > > > > > > >  _require_populate_commands
> > > > > > > > >  _xfs_skip_online_rebuild
> > > > > > > > >  _xfs_skip_offline_rebuild
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > >  echo "Format and populate"
> > > > > > > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > > > > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > > > > > > >  metadump_file=$testdir/scratch.md
> > > > > > > > >  copy_file=$testdir/copy.img
> > > > > > > > >  
> > > > > > > > > -check_restored_metadump_image()
> > > > > > > > > -{
> > > > > > > > > -	local image=$1
> > > > > > > > > -
> > > > > > > > > -	loop_dev=$(_create_loop_device $image)
> > > > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > > > > > > -	_destroy_loop_device $loop_dev
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > >  echo "metadump and mdrestore"
> > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > -	version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v $md_version"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > -done
> > > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > > >  
> > > > > > > > >  echo "metadump a and mdrestore"
> > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > -	version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v $md_version"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > -done
> > > > > > > > > +_verify_metadumps '-a'
> > > > > > > > >  
> > > > > > > > >  echo "metadump g and mdrestore"
> > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > -	version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v $md_version"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > -done
> > > > > > > > > +_verify_metadumps '-g' >> $seqres.full
> > > > > > > > >  
> > > > > > > > >  echo "metadump ag and mdrestore"
> > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > -	version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v $md_version"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > -done
> > > > > > > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > > > > > > >  
> > > > > > > > >  echo copy
> > > > > > > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > > > > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > > > > > > index f2cd7aba98..af917f0f32 100755
> > > > > > > > > --- a/tests/xfs/605
> > > > > > > > > +++ b/tests/xfs/605
> > > > > > > > > @@ -15,17 +15,13 @@ _cleanup()
> > > > > > > > >  {
> > > > > > > > >  	cd /
> > > > > > > > >  	rm -r -f $tmp.*
> > > > > > > > > -	_scratch_unmount > /dev/null 2>&1
> > > > > > > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > > > > > > -	   $TEST_DIR/log-image
> > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > >  # Import common functions.
> > > > > > > > >  . ./common/dmflakey
> > > > > > > > >  . ./common/inject
> > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > >  
> > > > > > > > >  # real QA test starts here
> > > > > > > > >  _supported_fs xfs
> > > > > > > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > > > > > > >  _require_dm_target flakey
> > > > > > > > >  _require_xfs_io_command "pwrite"
> > > > > > > > >  _require_test_program "punch-alternating"
> > > > > > > > > +_setup_verify_metadump
> > > > > > > > >  
> > > > > > > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > > > > > > >  testfile=${SCRATCH_MNT}/testfile
> > > > > > > > >  
> > > > > > > > >  echo "Format filesystem on scratch device"
> > > > > > > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > > > > > > >  
> > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > -
> > > > > > > > >  external_log=0
> > > > > > > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > > > > > > >  	external_log=1
> > > > > > > > >  fi
> > > > > > > > >  
> > > > > > > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > > > > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > > > > > > >  	_notrun "metadump v1 does not support external log device"
> > > > > > > > >  fi
> > > > > > > > >  
> > > > > > > > > -verify_metadump_v1()
> > > > > > > > > -{
> > > > > > > > > -	local version=""
> > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -		version="-v 1"
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -verify_metadump_v2()
> > > > > > > > > -{
> > > > > > > > > -	local version="-v 2"
> > > > > > > > > -
> > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > > > -
> > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > -	# from such a metadump file.
> > > > > > > > > -	slogdev=""
> > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > -
> > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > -
> > > > > > > > > -	logdev=""
> > > > > > > > > -	if [[ -s $slogdev ]]; then
> > > > > > > > > -		logdev=$(_create_loop_device $slogdev)
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > -
> > > > > > > > > -	if [[ -s $logdev ]]; then
> > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > -		logdev=""
> > > > > > > > > -		rm -f $slogdev
> > > > > > > > > -	fi
> > > > > > > > > -
> > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > -	datadev=""
> > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > >  echo "Initialize and mount filesystem on flakey device"
> > > > > > > > >  _init_flakey
> > > > > > > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > > > > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > > > > > > >  _print_logstate
> > > > > > > > >  
> > > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > > -
> > > > > > > > > -if [[ $external_log == 0 ]]; then
> > > > > > > > > -	verify_metadump_v1 $max_md_version
> > > > > > > > > -fi
> > > > > > > > > -
> > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > -	verify_metadump_v2
> > > > > > > > > -fi
> > > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > > >  
> > > > > > > > >  # Mount the fs to replay the contents from the dirty log.
> > > > > > > > >  _scratch_mount
> > > > > > > > > 
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > > 
> > > 
> > 
> > 
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Sun, Feb 04, 2024 at 02:59:42PM +0800, Zorro Lang wrote:
> On Fri, Feb 02, 2024 at 08:47:08AM -0800, Darrick J. Wong wrote:
> > On Thu, Feb 01, 2024 at 02:29:22PM +0800, Zorro Lang wrote:
> > > On Wed, Jan 31, 2024 at 11:24:33AM -0800, Darrick J. Wong wrote:
> > > > On Wed, Jan 31, 2024 at 10:09:19PM +0800, Zorro Lang wrote:
> > > > > On Mon, Jan 29, 2024 at 05:32:07PM -0800, Darrick J. Wong wrote:
> > > > > > On Sun, Jan 28, 2024 at 09:23:04PM +0800, Zorro Lang wrote:
> > > > > > > On Sat, Jan 27, 2024 at 09:22:22AM -0800, Darrick J. Wong wrote:
> > > > > > > > On Sat, Jan 27, 2024 at 04:47:14PM +0800, Zorro Lang wrote:
> > > > > > > > > On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > > > > > > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > > > > > > 
> > > > > > > > > > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > > > > > > > > > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > > > > > > > > > duplicate copies of the same code.
> > > > > > > > > > 
> > > > > > > > > > While we're at it, fix the fsck so that it includes xfs_scrub.
> > > > > > > > > > 
> > > > > > > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > > > > > > ---
> > > > > > > > > >  common/rc                 |   10 ----
> > > > > > > > > >  common/xfs                |   14 +++++
> > > > > > > > > >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> > > > > > > > > >  tests/xfs/129             |   90 ++-------------------------------
> > > > > > > > > >  tests/xfs/234             |   91 ++-------------------------------
> > > > > > > > > >  tests/xfs/253             |   89 ++-------------------------------
> > > > > > > > > >  tests/xfs/291             |   31 ++++-------
> > > > > > > > > >  tests/xfs/432             |   30 ++---------
> > > > > > > > > >  tests/xfs/503             |   60 +++-------------------
> > > > > > > > > >  tests/xfs/605             |   84 ++-----------------------------
> > > > > > > > > >  10 files changed, 181 insertions(+), 441 deletions(-)
> > > > > > > > > >  create mode 100644 common/xfs_metadump_tests
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > diff --git a/common/rc b/common/rc
> > > > > > > > > > index 524ffa02aa..0b69f7f54f 100644
> > > > > > > > > > --- a/common/rc
> > > > > > > > > > +++ b/common/rc
> > > > > > > > > > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> > > > > > > > > >  
> > > > > > > > > >      case $FSTYP in
> > > > > > > > > >      xfs)
> > > > > > > > > > -	local scratch_log="none"
> > > > > > > > > > -	local scratch_rt="none"
> > > > > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > > > > -	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > > > > -
> > > > > > > > > > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > > > > -	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > > > > -
> > > > > > > > > > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > > > > +	_check_xfs_scratch_fs $device
> > > > > > > > > >  	;;
> > > > > > > > > >      udf)
> > > > > > > > > >  	_check_udf_filesystem $device $udf_fsize
> > > > > > > > > > diff --git a/common/xfs b/common/xfs
> > > > > > > > > > index 248ccefda3..6a48960a7f 100644
> > > > > > > > > > --- a/common/xfs
> > > > > > > > > > +++ b/common/xfs
> > > > > > > > > > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> > > > > > > > > >  	return $?
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > > +_check_xfs_scratch_fs()
> > > > > > > > > > +{
> > > > > > > > > > +	local device="${1:-$SCRATCH_DEV}"
> > > > > > > > > > +	local scratch_log="none"
> > > > > > > > > > +	local scratch_rt="none"
> > > > > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > > > > > > > > > +	    scratch_log="$SCRATCH_LOGDEV"
> > > > > > > > > > +
> > > > > > > > > > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > > > > > > > > > +	    scratch_rt="$SCRATCH_RTDEV"
> > > > > > > > > > +
> > > > > > > > > > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > >  # modeled after _scratch_xfs_repair
> > > > > > > > > >  _test_xfs_repair()
> > > > > > > > > >  {
> > > > > > > > > > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > > > > > > > > 
> > > > > > > > > Hi Darrick,
> > > > > > > > > 
> > > > > > > > > Thanks for this improvement.
> > > > > > > > > 
> > > > > > > > > I'm wondering do we need a separated common file only for xfs metadump
> > > > > > > > > helpers ? Can't they be in common/xfs ? There're already _xfs_metadump(),
> > > > > > > > > _xfs_mdrestore(), _scratch_xfs_metadump(), _scratch_xfs_mdrestore() etc
> > > > > > > > > in common/xfs.
> > > > > > > > 
> > > > > > > > Yes, that's certainly possible, but keep in mind that common/$FSTYP
> > > > > > > > files are getting big:
> > > > > > > > 
> > > > > > > >    509 fstests/common/overlay
> > > > > > > >    523 fstests/common/quota
> > > > > > > >    550 fstests/common/reflink
> > > > > > > >    638 fstests/common/log
> > > > > > > >    640 fstests/common/punch
> > > > > > > >    663 fstests/common/filter
> > > > > > > >    794 fstests/common/btrfs
> > > > > > > >    936 fstests/common/config
> > > > > > > >   1030 fstests/common/encrypt
> > > > > > > >   1162 fstests/common/populate
> > > > > > > >   1519 fstests/common/fuzzy
> > > > > > > >   1531 fstests/common/dump
> > > > > > > >   2218 fstests/common/xfs
> > > > > > > >   5437 fstests/common/rc
> > > > > > > > 
> > > > > > > > with common/xfs being particularly larger than most everything else.
> > > > > > > 
> > > > > > > Haha, maybe we'll have a common/xfs/ directory in one day :)
> > > > > > > 
> > > > > > > > 
> > > > > > > > Since the common/xfs_metadump_tests functions are shared helper
> > > > > > > > functions for testing metadump/mdrestore that are not use by most tests,
> > > > > > > > I decided that a split was appropriate both to maintain the (ha!)
> > > > > > > > cohesion of common/xfs and not add more bash parsing costs to every
> > > > > > > > single testcase.
> > > > > > > 
> > > > > > > OK, a split makes sense, but I have 3 questions:
> > > > > > > 1) Will you move all metadump helpers from common/xfs to this new file?
> > > > > > 
> > > > > > I don't really want to, because that's another patch and would require
> > > > > > careful auditing of all the tests to find the ones that want to use
> > > > > > metadump but aren't themselves functional tests of metadump.
> > > > > > 
> > > > > > IOWs, this new file really is for shared metadump functional testing and
> > > > > > not much else.
> > > > > 
> > > > > OK, I think we can care about this part step by step in the future.
> > > > 
> > > > <nod>
> > > > 
> > > > > > 
> > > > > > > 2) Can we call it common/metadump? (Not sure if any other fs has metadump
> > > > > > >    things:)
> > > > > > 
> > > > > > Yes, e2image does this for ext*.
> > > > > 
> > > > > OK, I'll rename this file to common/metadump, and change other patches to
> > > > > souce the new name when I merge this patchset. Is that good to you?
> > > > 
> > > > Sorry, I realized that my statement was ambiguous -- the "yes" applies
> > > > to "Not sure if any other fs has metadump things", not "Can we call it
> > > > common/metadump?".  The code in common/xfs_metadump_tests is specific to
> > > > xfs and is not used for e2image testing, so let's leave the name as-is.
> > > 
> > > Oh, maybe it can be used for e2image or other metadump helpers later, if
> > > we call it common/metadump?
> > 
> > Sounds ok to me.  Want me to respin?
> 
> No, if it's good to you, I can change that on my side. But your might need to
> rebase on it later :)

Oh... it's not simple to rename the file name only. All helpers are xfs specific,
if we change the file name to common/metadump, those xfs specific function names
should all have "xfs" prefix. I'd like not to change too much locally without
reviewing. I think we can have this patchset at first, then change it later when
we have more metadump helpers for other filesystems, or when you'd like to change
that in later "random fixes from Darrick".

Thanks,
Zorro

> 
> > 
> > --D
> > 
> > > > 
> > > > --D
> > > > 
> > > > > Thanks,
> > > > > Zorro
> > > > > 
> > > > > > 
> > > > > > > 3) Or move to common/dump directly? (looks not proper ;-)
> > > > > > 
> > > > > > dump != metadump; one is for all the files in the fs and none of the
> > > > > > non-file metadata; the other is for metadata and none of the files.
> > > > > > 
> > > > > > --D
> > > > > > 
> > > > > > > Thanks,
> > > > > > > Zorro
> > > > > > > 
> > > > > > > > 
> > > > > > > > --D
> > > > > > > > 
> > > > > > > > > Thanks,
> > > > > > > > > Zorro
> > > > > > > > > 
> > > > > > > > > > new file mode 100644
> > > > > > > > > > index 0000000000..dd3dec1fb4
> > > > > > > > > > --- /dev/null
> > > > > > > > > > +++ b/common/xfs_metadump_tests
> > > > > > > > > > @@ -0,0 +1,123 @@
> > > > > > > > > > +#
> > > > > > > > > > +# XFS specific metadump testing functions.
> > > > > > > > > > +#
> > > > > > > > > > +
> > > > > > > > > > +# Set up environment variables for a metadump test.  Requires the test and
> > > > > > > > > > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > > > > > > > > > +_setup_verify_metadump()
> > > > > > > > > > +{
> > > > > > > > > > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > > > > > > > > > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > > > > > > > > > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > > > > > > > > > +
> > > > > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +_cleanup_verify_metadump()
> > > > > > > > > > +{
> > > > > > > > > > +	_scratch_unmount &>> $seqres.full
> > > > > > > > > > +
> > > > > > > > > > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > > > > > > > > > +		losetup -d "$ldev"
> > > > > > > > > > +	done
> > > > > > > > > > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > > > > > > > > > +# images and fsck them.
> > > > > > > > > > +_verify_metadump_v1()
> > > > > > > > > > +{
> > > > > > > > > > +	local metadump_args="$1"
> > > > > > > > > > +	local extra_test="$2"
> > > > > > > > > > +
> > > > > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > > > > +	local version=""
> > > > > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > > > > +	local data_loop
> > > > > > > > > > +
> > > > > > > > > > +	# Force v1 if we detect v2 support
> > > > > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > > > > > > > > > +		version="-v 1"
> > > > > > > > > > +	fi
> > > > > > > > > > +
> > > > > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > > > > +
> > > > > > > > > > +	# Restore metadump, which creates data_img
> > > > > > > > > > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > +
> > > > > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > > > > +
> > > > > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_mount
> > > > > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > > > > +		SCRATCH_DEV=$data_loop $extra_test
> > > > > > > > > > +	fi
> > > > > > > > > > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > > > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > > > > +
> > > > > > > > > > +	# Tear down what we created
> > > > > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > > > > +	rm -f $data_img
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > > > > > > > > > +# images and fsck them.
> > > > > > > > > > +_verify_metadump_v2()
> > > > > > > > > > +{
> > > > > > > > > > +	local metadump_args="$1"
> > > > > > > > > > +	local extra_test="$2"
> > > > > > > > > > +
> > > > > > > > > > +	local metadump_file="$XFS_METADUMP_FILE"
> > > > > > > > > > +	local version="-v 2"
> > > > > > > > > > +	local data_img="$XFS_METADUMP_IMG.data"
> > > > > > > > > > +	local data_loop
> > > > > > > > > > +	local log_img=""
> > > > > > > > > > +	local log_loop
> > > > > > > > > > +
> > > > > > > > > > +	# Capture metadump, which creates metadump_file
> > > > > > > > > > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > > > > > > > > > +
> > > > > > > > > > +	#
> > > > > > > > > > +	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > > +	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > > +	# from such a metadump file.
> > > > > > > > > > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > > > > > > > > > +
> > > > > > > > > > +	# Restore metadump, which creates data_img and log_img
> > > > > > > > > > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > > > > > > > > > +		_scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > +
> > > > > > > > > > +	# Create loopdev for data device so we can mount the fs
> > > > > > > > > > +	data_loop=$(_create_loop_device $data_img)
> > > > > > > > > > +
> > > > > > > > > > +	# Create loopdev for log device if we recovered anything
> > > > > > > > > > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > > > > > > > > > +
> > > > > > > > > > +	# Mount fs, run an extra test, fsck, and unmount
> > > > > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > > > > > > > > > +	if [ -n "$extra_test" ]; then
> > > > > > > > > > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > > > > > > > > > +	fi
> > > > > > > > > > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > > > > > > > > > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > > > > > > > > > +
> > > > > > > > > > +	# Tear down what we created
> > > > > > > > > > +	if [ -b "$log_loop" ]; then
> > > > > > > > > > +		_destroy_loop_device $log_loop
> > > > > > > > > > +		rm -f $log_img
> > > > > > > > > > +	fi
> > > > > > > > > > +	_destroy_loop_device $data_loop
> > > > > > > > > > +	rm -f $data_img
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +# Verify both metadump formats if possible
> > > > > > > > > > +_verify_metadumps()
> > > > > > > > > > +{
> > > > > > > > > > +	_verify_metadump_v1 "$@"
> > > > > > > > > > +
> > > > > > > > > > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > > > > > > > > > +		_verify_metadump_v2 "$@"
> > > > > > > > > > +	fi
> > > > > > > > > > +}
> > > > > > > > > > diff --git a/tests/xfs/129 b/tests/xfs/129
> > > > > > > > > > index cdac2349df..c3a9bcefee 100755
> > > > > > > > > > --- a/tests/xfs/129
> > > > > > > > > > +++ b/tests/xfs/129
> > > > > > > > > > @@ -16,98 +16,23 @@ _cleanup()
> > > > > > > > > >  {
> > > > > > > > > >      cd /
> > > > > > > > > >      _scratch_unmount > /dev/null 2>&1
> > > > > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > > > > > > > > > -       $TEST_DIR/log-image
> > > > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/filter
> > > > > > > > > >  . ./common/reflink
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  # real QA test starts here
> > > > > > > > > >  _supported_fs xfs
> > > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > > >  _require_loop
> > > > > > > > > >  _require_scratch_reflink
> > > > > > > > > > -
> > > > > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v1()
> > > > > > > > > > -{
> > > > > > > > > > -	local max_version=$1
> > > > > > > > > > -	local version=""
> > > > > > > > > > -
> > > > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > > > -		version="-v 1"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v2()
> > > > > > > > > > -{
> > > > > > > > > > -	version="-v 2"
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > > -
> > > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > > -	# from such a metadump file.
> > > > > > > > > > -	slogdev=""
> > > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > > -
> > > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > > -		logdev=""
> > > > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > > > > -
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > >  _scratch_mount
> > > > > > > > > >  
> > > > > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > > > > @@ -127,12 +52,7 @@ done
> > > > > > > > > >  _scratch_unmount
> > > > > > > > > >  
> > > > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > > > -
> > > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -	verify_metadump_v2
> > > > > > > > > > -fi
> > > > > > > > > > +_verify_metadumps
> > > > > > > > > >  
> > > > > > > > > >  # success, all done
> > > > > > > > > >  status=0
> > > > > > > > > > diff --git a/tests/xfs/234 b/tests/xfs/234
> > > > > > > > > > index f4f8af6d3a..8f808c7507 100755
> > > > > > > > > > --- a/tests/xfs/234
> > > > > > > > > > +++ b/tests/xfs/234
> > > > > > > > > > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> > > > > > > > > >  _cleanup()
> > > > > > > > > >  {
> > > > > > > > > >      cd /
> > > > > > > > > > -    _scratch_unmount > /dev/null 2>&1
> > > > > > > > > > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > > > > > > > > > -       $TEST_DIR/log-image
> > > > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > > > +    rm -rf $tmp.* $testdir
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/filter
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  # real QA test starts here
> > > > > > > > > >  _supported_fs xfs
> > > > > > > > > > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > > >  _require_loop
> > > > > > > > > >  _require_xfs_scratch_rmapbt
> > > > > > > > > >  _require_xfs_io_command "fpunch"
> > > > > > > > > > -
> > > > > > > > > > -metadump_file=$TEST_DIR/${seq}_metadump
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v1()
> > > > > > > > > > -{
> > > > > > > > > > -	local max_version=$1
> > > > > > > > > > -	local version=""
> > > > > > > > > > -
> > > > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > > > -		version="-v 1"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	logdev=$SCRATCH_LOGDEV
> > > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v2()
> > > > > > > > > > -{
> > > > > > > > > > -	version="-v 2"
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file $version
> > > > > > > > > > -
> > > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > > -	# from such a metadump file.
> > > > > > > > > > -	slogdev=""
> > > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	[[ -z $logdev ]] && logdev=none
> > > > > > > > > > -	_check_xfs_filesystem $datadev $logdev none
> > > > > > > > > > -
> > > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > > -		logdev=""
> > > > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > >  _scratch_mkfs >/dev/null 2>&1
> > > > > > > > > > -
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > >  _scratch_mount
> > > > > > > > > >  
> > > > > > > > > >  testdir=$SCRATCH_MNT/test-$seq
> > > > > > > > > > @@ -127,12 +51,7 @@ done
> > > > > > > > > >  _scratch_unmount
> > > > > > > > > >  
> > > > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > > > -
> > > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -	verify_metadump_v2
> > > > > > > > > > -fi
> > > > > > > > > > +_verify_metadumps
> > > > > > > > > >  
> > > > > > > > > >  # success, all done
> > > > > > > > > >  status=0
> > > > > > > > > > diff --git a/tests/xfs/253 b/tests/xfs/253
> > > > > > > > > > index 3b567999d8..6623c435e5 100755
> > > > > > > > > > --- a/tests/xfs/253
> > > > > > > > > > +++ b/tests/xfs/253
> > > > > > > > > > @@ -26,23 +26,21 @@ _cleanup()
> > > > > > > > > >      cd /
> > > > > > > > > >      rm -f $tmp.*
> > > > > > > > > >      rm -rf "${OUTPUT_DIR}"
> > > > > > > > > > -    rm -f "${METADUMP_FILE}"
> > > > > > > > > > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > > -	    _destroy_loop_device $logdev
> > > > > > > > > > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > > +    _cleanup_verify_metadump
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/filter
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > > >  _require_test
> > > > > > > > > >  _require_scratch
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > >  # real QA test starts here
> > > > > > > > > >  
> > > > > > > > > >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > > > > > > > > > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> > > > > > > > > >  ORPHANAGE="lost+found"
> > > > > > > > > >  
> > > > > > > > > >  _supported_fs xfs
> > > > > > > > > > @@ -52,24 +50,7 @@ function create_file() {
> > > > > > > > > >  	touch $(printf "$@")
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > > -verify_metadump_v1()
> > > > > > > > > > -{
> > > > > > > > > > -	local max_version=$1
> > > > > > > > > > -	local version=""
> > > > > > > > > > -
> > > > > > > > > > -	if [[ $max_version == 2 ]]; then
> > > > > > > > > > -		version="-v 1"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > > -
> > > > > > > > > > +extra_test() {
> > > > > > > > > >  	cd "${SCRATCH_MNT}"
> > > > > > > > > >  
> > > > > > > > > >  	# Get a listing of all the files after obfuscation
> > > > > > > > > > @@ -78,60 +59,6 @@ verify_metadump_v1()
> > > > > > > > > >  	ls -R | od -c >> $seqres.full
> > > > > > > > > >  
> > > > > > > > > >  	cd /
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v2()
> > > > > > > > > > -{
> > > > > > > > > > -	version="-v 2"
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > > > > > > > > > -
> > > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > > -	# from such a metadump file.
> > > > > > > > > > -	slogdev=""
> > > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	logdev=${SCRATCH_LOGDEV}
> > > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > > -
> > > > > > > > > > -	cd "${SCRATCH_MNT}"
> > > > > > > > > > -
> > > > > > > > > > -	# Get a listing of all the files after obfuscation
> > > > > > > > > > -	echo "Metadump v2" >> $seqres.full
> > > > > > > > > > -	ls -R >> $seqres.full
> > > > > > > > > > -	ls -R | od -c >> $seqres.full
> > > > > > > > > > -
> > > > > > > > > > -	cd /
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	if [[ -s $TEST_DIR/log-image ]]; then
> > > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > > -		logdev=""
> > > > > > > > > > -		rm -f $TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  echo "Disciplyne of silence is goed."
> > > > > > > > > > @@ -233,13 +160,7 @@ cd $here
> > > > > > > > > >  
> > > > > > > > > >  _scratch_unmount
> > > > > > > > > >  
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v1 $max_md_version
> > > > > > > > > > -
> > > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -	verify_metadump_v2
> > > > > > > > > > -fi
> > > > > > > > > > +_verify_metadumps '' extra_test
> > > > > > > > > >  
> > > > > > > > > >  # Finally, re-make the filesystem since to ensure we don't
> > > > > > > > > >  # leave a directory with duplicate entries lying around.
> > > > > > > > > > diff --git a/tests/xfs/291 b/tests/xfs/291
> > > > > > > > > > index 1433140821..c475d89ad9 100755
> > > > > > > > > > --- a/tests/xfs/291
> > > > > > > > > > +++ b/tests/xfs/291
> > > > > > > > > > @@ -9,11 +9,21 @@
> > > > > > > > > >  . ./common/preamble
> > > > > > > > > >  _begin_fstest auto repair metadump
> > > > > > > > > >  
> > > > > > > > > > +# Override the default cleanup function.
> > > > > > > > > > +_cleanup()
> > > > > > > > > > +{
> > > > > > > > > > +	cd /
> > > > > > > > > > +	rm -r -f $tmp.*
> > > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/filter
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  _supported_fs xfs
> > > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > >  # real QA test starts here
> > > > > > > > > >  _require_scratch
> > > > > > > > > > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> > > > > > > > > >  
> > > > > > > > > >  # Yes they can!  Now...
> > > > > > > > > >  # Can xfs_metadump cope with this monster?
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > > -	version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v $md_version"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > > > > > > > > > -		_fail "xfs_metadump failed"
> > > > > > > > > > -
> > > > > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > > > > -		slogdev=""
> > > > > > > > > > -	fi
> > > > > > > > > > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > > > > > > > > > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > > > > > > > > > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > > > > > > > > > -		_fail "xfs_repair of metadump failed"
> > > > > > > > > > -done
> > > > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > > > >  
> > > > > > > > > >  # Yes it can; success, all done
> > > > > > > > > >  status=0
> > > > > > > > > > diff --git a/tests/xfs/432 b/tests/xfs/432
> > > > > > > > > > index 7e402aa88f..579e1b556a 100755
> > > > > > > > > > --- a/tests/xfs/432
> > > > > > > > > > +++ b/tests/xfs/432
> > > > > > > > > > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> > > > > > > > > >  _cleanup()
> > > > > > > > > >  {
> > > > > > > > > >  	cd /
> > > > > > > > > > -	rm -f "$tmp".* $metadump_file $metadump_img
> > > > > > > > > > +	rm -f "$tmp".*
> > > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/filter
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  # real QA test starts here
> > > > > > > > > >  _supported_fs xfs
> > > > > > > > > >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > > > > > > > > >  _require_scratch
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > >  rm -f "$seqres.full"
> > > > > > > > > >  
> > > > > > > > > > @@ -54,9 +57,6 @@ echo "Format and mount"
> > > > > > > > > >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> > > > > > > > > >  _scratch_mount >> "$seqres.full" 2>&1
> > > > > > > > > >  
> > > > > > > > > > -metadump_file="$TEST_DIR/meta-$seq"
> > > > > > > > > > -metadump_img="$TEST_DIR/img-$seq"
> > > > > > > > > > -rm -f $metadump_file $metadump_img
> > > > > > > > > >  testdir="$SCRATCH_MNT/test-$seq"
> > > > > > > > > >  max_fname_len=255
> > > > > > > > > >  blksz=$(_get_block_size $SCRATCH_MNT)
> > > > > > > > > > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> > > > > > > > > >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> > > > > > > > > >  
> > > > > > > > > >  echo "Try to metadump, restore and check restored metadump image"
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > > -	version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v $md_version"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > > > > > > > > > -
> > > > > > > > > > -	slogdev=$SCRATCH_LOGDEV
> > > > > > > > > > -	if [[ -z $version || $version == "-v 1" ]]; then
> > > > > > > > > > -		slogdev=""
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > > > > > > > > > -		echo "xfs_repair on restored fs returned $?"
> > > > > > > > > > -done
> > > > > > > > > > +_verify_metadumps '-a -o -w'
> > > > > > > > > >  
> > > > > > > > > >  # success, all done
> > > > > > > > > >  status=0
> > > > > > > > > > diff --git a/tests/xfs/503 b/tests/xfs/503
> > > > > > > > > > index 8643c3d483..ff6b344a9c 100755
> > > > > > > > > > --- a/tests/xfs/503
> > > > > > > > > > +++ b/tests/xfs/503
> > > > > > > > > > @@ -17,11 +17,13 @@ _cleanup()
> > > > > > > > > >  {
> > > > > > > > > >  	cd /
> > > > > > > > > >  	rm -rf $tmp.* $testdir
> > > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/filter
> > > > > > > > > >  . ./common/populate
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  testdir=$TEST_DIR/test-$seq
> > > > > > > > > >  
> > > > > > > > > > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> > > > > > > > > >  _require_populate_commands
> > > > > > > > > >  _xfs_skip_online_rebuild
> > > > > > > > > >  _xfs_skip_offline_rebuild
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > >  echo "Format and populate"
> > > > > > > > > >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > > > > > > > > > @@ -43,66 +46,17 @@ mkdir -p $testdir
> > > > > > > > > >  metadump_file=$testdir/scratch.md
> > > > > > > > > >  copy_file=$testdir/copy.img
> > > > > > > > > >  
> > > > > > > > > > -check_restored_metadump_image()
> > > > > > > > > > -{
> > > > > > > > > > -	local image=$1
> > > > > > > > > > -
> > > > > > > > > > -	loop_dev=$(_create_loop_device $image)
> > > > > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > > > > > > > > > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > > > > > > > > > -	_destroy_loop_device $loop_dev
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > >  echo "metadump and mdrestore"
> > > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > > -	version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v $md_version"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > > -done
> > > > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > > > >  
> > > > > > > > > >  echo "metadump a and mdrestore"
> > > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > > -	version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v $md_version"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > > -done
> > > > > > > > > > +_verify_metadumps '-a'
> > > > > > > > > >  
> > > > > > > > > >  echo "metadump g and mdrestore"
> > > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > > -	version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v $md_version"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > > -done
> > > > > > > > > > +_verify_metadumps '-g' >> $seqres.full
> > > > > > > > > >  
> > > > > > > > > >  echo "metadump ag and mdrestore"
> > > > > > > > > > -for md_version in $(seq 1 $max_md_version); do
> > > > > > > > > > -	version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v $md_version"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -	check_restored_metadump_image $TEST_DIR/image
> > > > > > > > > > -done
> > > > > > > > > > +_verify_metadumps '-a -g' >> $seqres.full
> > > > > > > > > >  
> > > > > > > > > >  echo copy
> > > > > > > > > >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > > > > > > > > > diff --git a/tests/xfs/605 b/tests/xfs/605
> > > > > > > > > > index f2cd7aba98..af917f0f32 100755
> > > > > > > > > > --- a/tests/xfs/605
> > > > > > > > > > +++ b/tests/xfs/605
> > > > > > > > > > @@ -15,17 +15,13 @@ _cleanup()
> > > > > > > > > >  {
> > > > > > > > > >  	cd /
> > > > > > > > > >  	rm -r -f $tmp.*
> > > > > > > > > > -	_scratch_unmount > /dev/null 2>&1
> > > > > > > > > > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > > > > > > > > > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > > > > > > > > > -	   $TEST_DIR/log-image
> > > > > > > > > > +	_cleanup_verify_metadump
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > >  # Import common functions.
> > > > > > > > > >  . ./common/dmflakey
> > > > > > > > > >  . ./common/inject
> > > > > > > > > > +. ./common/xfs_metadump_tests
> > > > > > > > > >  
> > > > > > > > > >  # real QA test starts here
> > > > > > > > > >  _supported_fs xfs
> > > > > > > > > > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> > > > > > > > > >  _require_dm_target flakey
> > > > > > > > > >  _require_xfs_io_command "pwrite"
> > > > > > > > > >  _require_test_program "punch-alternating"
> > > > > > > > > > +_setup_verify_metadump
> > > > > > > > > >  
> > > > > > > > > > -metadump_file=${TEST_DIR}/${seq}.md
> > > > > > > > > >  testfile=${SCRATCH_MNT}/testfile
> > > > > > > > > >  
> > > > > > > > > >  echo "Format filesystem on scratch device"
> > > > > > > > > >  _scratch_mkfs >> $seqres.full 2>&1
> > > > > > > > > >  
> > > > > > > > > > -max_md_version=$(_xfs_metadump_max_version)
> > > > > > > > > > -
> > > > > > > > > >  external_log=0
> > > > > > > > > >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> > > > > > > > > >  	external_log=1
> > > > > > > > > >  fi
> > > > > > > > > >  
> > > > > > > > > > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > > > > > > > > > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> > > > > > > > > >  	_notrun "metadump v1 does not support external log device"
> > > > > > > > > >  fi
> > > > > > > > > >  
> > > > > > > > > > -verify_metadump_v1()
> > > > > > > > > > -{
> > > > > > > > > > -	local version=""
> > > > > > > > > > -	if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -		version="-v 1"
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > > > > > > > > > -	SCRATCH_DEV=$datadev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -verify_metadump_v2()
> > > > > > > > > > -{
> > > > > > > > > > -	local version="-v 2"
> > > > > > > > > > -
> > > > > > > > > > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > > > > > > > > > -
> > > > > > > > > > -	# Metadump v2 files can contain contents dumped from an external log
> > > > > > > > > > -	# device. Use a temporary file to hold the log device contents restored
> > > > > > > > > > -	# from such a metadump file.
> > > > > > > > > > -	slogdev=""
> > > > > > > > > > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > > > > > > > > > -		slogdev=$TEST_DIR/log-image
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > > > > > > > > > -		   _scratch_xfs_mdrestore $metadump_file
> > > > > > > > > > -
> > > > > > > > > > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > > > > > > > > > -
> > > > > > > > > > -	logdev=""
> > > > > > > > > > -	if [[ -s $slogdev ]]; then
> > > > > > > > > > -		logdev=$(_create_loop_device $slogdev)
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > > > > > > > > > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > > > > > > > > > -
> > > > > > > > > > -	if [[ -s $logdev ]]; then
> > > > > > > > > > -		_destroy_loop_device $logdev
> > > > > > > > > > -		logdev=""
> > > > > > > > > > -		rm -f $slogdev
> > > > > > > > > > -	fi
> > > > > > > > > > -
> > > > > > > > > > -	_destroy_loop_device $datadev
> > > > > > > > > > -	datadev=""
> > > > > > > > > > -	rm -f $TEST_DIR/data-image
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > >  echo "Initialize and mount filesystem on flakey device"
> > > > > > > > > >  _init_flakey
> > > > > > > > > >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > > > > > > > > > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> > > > > > > > > >  _print_logstate
> > > > > > > > > >  
> > > > > > > > > >  echo "Create metadump file, restore it and check restored fs"
> > > > > > > > > > -
> > > > > > > > > > -if [[ $external_log == 0 ]]; then
> > > > > > > > > > -	verify_metadump_v1 $max_md_version
> > > > > > > > > > -fi
> > > > > > > > > > -
> > > > > > > > > > -if [[ $max_md_version == 2 ]]; then
> > > > > > > > > > -	verify_metadump_v2
> > > > > > > > > > -fi
> > > > > > > > > > +_verify_metadumps '-a -o'
> > > > > > > > > >  
> > > > > > > > > >  # Mount the fs to replay the contents from the dirty log.
> > > > > > > > > >  _scratch_mount
> > > > > > > > > > 
> > > > > > > > > 
> > > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > 
> > > 
> > 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Zorro Lang]

On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> duplicate copies of the same code.
> 
> While we're at it, fix the fsck so that it includes xfs_scrub.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/rc                 |   10 ----
>  common/xfs                |   14 +++++
>  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/129             |   90 ++-------------------------------
>  tests/xfs/234             |   91 ++-------------------------------
>  tests/xfs/253             |   89 ++-------------------------------
>  tests/xfs/291             |   31 ++++-------
>  tests/xfs/432             |   30 ++---------
>  tests/xfs/503             |   60 +++-------------------
>  tests/xfs/605             |   84 ++-----------------------------
>  10 files changed, 181 insertions(+), 441 deletions(-)
>  create mode 100644 common/xfs_metadump_tests
> 
> 
> diff --git a/common/rc b/common/rc
> index 524ffa02aa..0b69f7f54f 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -3320,15 +3320,7 @@ _check_scratch_fs()
>  
>      case $FSTYP in
>      xfs)
> -	local scratch_log="none"
> -	local scratch_rt="none"
> -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> -	    scratch_log="$SCRATCH_LOGDEV"
> -
> -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> -	    scratch_rt="$SCRATCH_RTDEV"
> -
> -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> +	_check_xfs_scratch_fs $device
>  	;;
>      udf)
>  	_check_udf_filesystem $device $udf_fsize
> diff --git a/common/xfs b/common/xfs
> index 248ccefda3..6a48960a7f 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
>  	return $?
>  }
>  
> +_check_xfs_scratch_fs()
> +{
> +	local device="${1:-$SCRATCH_DEV}"
> +	local scratch_log="none"
> +	local scratch_rt="none"
> +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> +	    scratch_log="$SCRATCH_LOGDEV"
> +
> +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> +	    scratch_rt="$SCRATCH_RTDEV"
> +
> +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> +}
> +
>  # modeled after _scratch_xfs_repair
>  _test_xfs_repair()
>  {
> diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> new file mode 100644
> index 0000000000..dd3dec1fb4
> --- /dev/null
> +++ b/common/xfs_metadump_tests
> @@ -0,0 +1,123 @@
> +#
> +# XFS specific metadump testing functions.
> +#
> +
> +# Set up environment variables for a metadump test.  Requires the test and
> +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> +_setup_verify_metadump()
> +{
> +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> +
> +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> +}
> +
> +_cleanup_verify_metadump()
> +{
> +	_scratch_unmount &>> $seqres.full
> +
> +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> +		losetup -d "$ldev"
> +	done

Hi Darrick,

If $XFS_METADUMP_IMG is null, this line will delete all loop device. If
someone uses loop devices to be TEST_DEV and SCRATCH_DEV, then it might
break the whole testing.

> +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*

This's dangerous too, I've explained that in :
  https://lore.kernel.org/fstests/20240205065642.3hqhipmnxkg442kg@dell-per750-06-vm-08.rhts.eng.pek2.redhat.com/T/#m496e69db25755292ed9ac4bee08a22f227ebf7d2

Sorry I didn't notice that when I tested with new upstream kernel. But after I
tested on old kernel, some metadump cases start to _notrun, then trigger this
bug. This bug is too dangerous, one of my regular system has gone, I have to
reinstall. So I decide to reset my last fstests release, to avoid destroying
other folks' machine.

I think we'd better to check XFS_METADUMP_FILE and XFS_METADUMP_IMG aren't
empty at first, e.g.

if [ -n "$XFS_METADUMP_FILE" -a -n "$XFS_METADUMP_IMG" ];then
	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
		losetup -d "$ldev"
	done
	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
fi

Or if you'd like to change more :)

Thanks,
Zorro

> +}
> +
> +# Create a metadump in v1 format, restore it to fs image files, then mount the
> +# images and fsck them.
> +_verify_metadump_v1()
> +{
> +	local metadump_args="$1"
> +	local extra_test="$2"
> +
> +	local metadump_file="$XFS_METADUMP_FILE"
> +	local version=""
> +	local data_img="$XFS_METADUMP_IMG.data"
> +	local data_loop
> +
> +	# Force v1 if we detect v2 support
> +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> +		version="-v 1"
> +	fi
> +
> +	# Capture metadump, which creates metadump_file
> +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> +
> +	# Restore metadump, which creates data_img
> +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> +
> +	# Create loopdev for data device so we can mount the fs
> +	data_loop=$(_create_loop_device $data_img)
> +
> +	# Mount fs, run an extra test, fsck, and unmount
> +	SCRATCH_DEV=$data_loop _scratch_mount
> +	if [ -n "$extra_test" ]; then
> +		SCRATCH_DEV=$data_loop $extra_test
> +	fi
> +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> +	SCRATCH_DEV=$data_loop _scratch_unmount
> +
> +	# Tear down what we created
> +	_destroy_loop_device $data_loop
> +	rm -f $data_img
> +}
> +
> +# Create a metadump in v2 format, restore it to fs image files, then mount the
> +# images and fsck them.
> +_verify_metadump_v2()
> +{
> +	local metadump_args="$1"
> +	local extra_test="$2"
> +
> +	local metadump_file="$XFS_METADUMP_FILE"
> +	local version="-v 2"
> +	local data_img="$XFS_METADUMP_IMG.data"
> +	local data_loop
> +	local log_img=""
> +	local log_loop
> +
> +	# Capture metadump, which creates metadump_file
> +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> +
> +	#
> +	# Metadump v2 files can contain contents dumped from an external log
> +	# device. Use a temporary file to hold the log device contents restored
> +	# from such a metadump file.
> +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> +
> +	# Restore metadump, which creates data_img and log_img
> +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> +		_scratch_xfs_mdrestore $metadump_file
> +
> +	# Create loopdev for data device so we can mount the fs
> +	data_loop=$(_create_loop_device $data_img)
> +
> +	# Create loopdev for log device if we recovered anything
> +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> +
> +	# Mount fs, run an extra test, fsck, and unmount
> +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> +	if [ -n "$extra_test" ]; then
> +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> +	fi
> +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> +	SCRATCH_DEV=$data_loop _scratch_unmount
> +
> +	# Tear down what we created
> +	if [ -b "$log_loop" ]; then
> +		_destroy_loop_device $log_loop
> +		rm -f $log_img
> +	fi
> +	_destroy_loop_device $data_loop
> +	rm -f $data_img
> +}
> +
> +# Verify both metadump formats if possible
> +_verify_metadumps()
> +{
> +	_verify_metadump_v1 "$@"
> +
> +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> +		_verify_metadump_v2 "$@"
> +	fi
> +}
> diff --git a/tests/xfs/129 b/tests/xfs/129
> index cdac2349df..c3a9bcefee 100755
> --- a/tests/xfs/129
> +++ b/tests/xfs/129
> @@ -16,98 +16,23 @@ _cleanup()
>  {
>      cd /
>      _scratch_unmount > /dev/null 2>&1
> -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> -	    _destroy_loop_device $logdev
> -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> -       $TEST_DIR/log-image
> +    _cleanup_verify_metadump
> +    rm -rf $tmp.* $testdir
>  }
>  
>  # Import common functions.
>  . ./common/filter
>  . ./common/reflink
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_loop
>  _require_scratch_reflink
> -
> -metadump_file=$TEST_DIR/${seq}_metadump
> -
> -verify_metadump_v1()
> -{
> -	local max_version=$1
> -	local version=""
> -
> -	if [[ $max_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	logdev=$SCRATCH_LOGDEV
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	version="-v 2"
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=${SCRATCH_LOGDEV}
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $TEST_DIR/log-image
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> +_setup_verify_metadump
>  
>  _scratch_mkfs >/dev/null 2>&1
> -
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  _scratch_mount
>  
>  testdir=$SCRATCH_MNT/test-$seq
> @@ -127,12 +52,7 @@ done
>  _scratch_unmount
>  
>  echo "Create metadump file, restore it and check restored fs"
> -
> -verify_metadump_v1 $max_md_version
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/234 b/tests/xfs/234
> index f4f8af6d3a..8f808c7507 100755
> --- a/tests/xfs/234
> +++ b/tests/xfs/234
> @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
>  _cleanup()
>  {
>      cd /
> -    _scratch_unmount > /dev/null 2>&1
> -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> -	    _destroy_loop_device $logdev
> -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> -       $TEST_DIR/log-image
> +    _cleanup_verify_metadump
> +    rm -rf $tmp.* $testdir
>  }
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
> @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_loop
>  _require_xfs_scratch_rmapbt
>  _require_xfs_io_command "fpunch"
> -
> -metadump_file=$TEST_DIR/${seq}_metadump
> -
> -verify_metadump_v1()
> -{
> -	local max_version=$1
> -	local version=""
> -
> -	if [[ $max_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	logdev=$SCRATCH_LOGDEV
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	version="-v 2"
> -
> -	_scratch_xfs_metadump $metadump_file $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=${SCRATCH_LOGDEV}
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	[[ -z $logdev ]] && logdev=none
> -	_check_xfs_filesystem $datadev $logdev none
> -
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $TEST_DIR/log-image
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> +_setup_verify_metadump
>  
>  _scratch_mkfs >/dev/null 2>&1
> -
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  _scratch_mount
>  
>  testdir=$SCRATCH_MNT/test-$seq
> @@ -127,12 +51,7 @@ done
>  _scratch_unmount
>  
>  echo "Create metadump file, restore it and check restored fs"
> -
> -verify_metadump_v1 $max_md_version
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/253 b/tests/xfs/253
> index 3b567999d8..6623c435e5 100755
> --- a/tests/xfs/253
> +++ b/tests/xfs/253
> @@ -26,23 +26,21 @@ _cleanup()
>      cd /
>      rm -f $tmp.*
>      rm -rf "${OUTPUT_DIR}"
> -    rm -f "${METADUMP_FILE}"
> -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> -	    _destroy_loop_device $logdev
> -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> +    _cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_test
>  _require_scratch
> +_setup_verify_metadump
>  
>  # real QA test starts here
>  
>  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
>  ORPHANAGE="lost+found"
>  
>  _supported_fs xfs
> @@ -52,24 +50,7 @@ function create_file() {
>  	touch $(printf "$@")
>  }
>  
> -verify_metadump_v1()
> -{
> -	local max_version=$1
> -	local version=""
> -
> -	if [[ $max_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $METADUMP_FILE $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> -		   _scratch_xfs_mdrestore $METADUMP_FILE
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -
> +extra_test() {
>  	cd "${SCRATCH_MNT}"
>  
>  	# Get a listing of all the files after obfuscation
> @@ -78,60 +59,6 @@ verify_metadump_v1()
>  	ls -R | od -c >> $seqres.full
>  
>  	cd /
> -
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	version="-v 2"
> -
> -	_scratch_xfs_metadump $METADUMP_FILE $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $METADUMP_FILE
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=${SCRATCH_LOGDEV}
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -
> -	cd "${SCRATCH_MNT}"
> -
> -	# Get a listing of all the files after obfuscation
> -	echo "Metadump v2" >> $seqres.full
> -	ls -R >> $seqres.full
> -	ls -R | od -c >> $seqres.full
> -
> -	cd /
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	if [[ -s $TEST_DIR/log-image ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $TEST_DIR/log-image
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
>  }
>  
>  echo "Disciplyne of silence is goed."
> @@ -233,13 +160,7 @@ cd $here
>  
>  _scratch_unmount
>  
> -max_md_version=$(_xfs_metadump_max_version)
> -
> -verify_metadump_v1 $max_md_version
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps '' extra_test
>  
>  # Finally, re-make the filesystem since to ensure we don't
>  # leave a directory with duplicate entries lying around.
> diff --git a/tests/xfs/291 b/tests/xfs/291
> index 1433140821..c475d89ad9 100755
> --- a/tests/xfs/291
> +++ b/tests/xfs/291
> @@ -9,11 +9,21 @@
>  . ./common/preamble
>  _begin_fstest auto repair metadump
>  
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +	cd /
> +	rm -r -f $tmp.*
> +	_cleanup_verify_metadump
> +}
> +
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  _supported_fs xfs
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> +_setup_verify_metadump
>  
>  # real QA test starts here
>  _require_scratch
> @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
>  
>  # Yes they can!  Now...
>  # Can xfs_metadump cope with this monster?
> -max_md_version=$(_xfs_metadump_max_version)
> -
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> -		_fail "xfs_metadump failed"
> -
> -	slogdev=$SCRATCH_LOGDEV
> -	if [[ -z $version || $version == "-v 1" ]]; then
> -		slogdev=""
> -	fi
> -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> -		_fail "xfs_repair of metadump failed"
> -done
> +_verify_metadumps '-a -o'
>  
>  # Yes it can; success, all done
>  status=0
> diff --git a/tests/xfs/432 b/tests/xfs/432
> index 7e402aa88f..579e1b556a 100755
> --- a/tests/xfs/432
> +++ b/tests/xfs/432
> @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
>  _cleanup()
>  {
>  	cd /
> -	rm -f "$tmp".* $metadump_file $metadump_img
> +	rm -f "$tmp".*
> +	_cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
>  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
>  _require_scratch
> +_setup_verify_metadump
>  
>  rm -f "$seqres.full"
>  
> @@ -54,9 +57,6 @@ echo "Format and mount"
>  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
>  _scratch_mount >> "$seqres.full" 2>&1
>  
> -metadump_file="$TEST_DIR/meta-$seq"
> -metadump_img="$TEST_DIR/img-$seq"
> -rm -f $metadump_file $metadump_img
>  testdir="$SCRATCH_MNT/test-$seq"
>  max_fname_len=255
>  blksz=$(_get_block_size $SCRATCH_MNT)
> @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
>  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
>  
>  echo "Try to metadump, restore and check restored metadump image"
> -max_md_version=$(_xfs_metadump_max_version)
> -
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> -
> -	slogdev=$SCRATCH_LOGDEV
> -	if [[ -z $version || $version == "-v 1" ]]; then
> -		slogdev=""
> -	fi
> -
> -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> -		echo "xfs_repair on restored fs returned $?"
> -done
> +_verify_metadumps '-a -o -w'
>  
>  # success, all done
>  status=0
> diff --git a/tests/xfs/503 b/tests/xfs/503
> index 8643c3d483..ff6b344a9c 100755
> --- a/tests/xfs/503
> +++ b/tests/xfs/503
> @@ -17,11 +17,13 @@ _cleanup()
>  {
>  	cd /
>  	rm -rf $tmp.* $testdir
> +	_cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/filter
>  . ./common/populate
> +. ./common/xfs_metadump_tests
>  
>  testdir=$TEST_DIR/test-$seq
>  
> @@ -35,6 +37,7 @@ _require_scratch_nocheck
>  _require_populate_commands
>  _xfs_skip_online_rebuild
>  _xfs_skip_offline_rebuild
> +_setup_verify_metadump
>  
>  echo "Format and populate"
>  _scratch_populate_cached nofill > $seqres.full 2>&1
> @@ -43,66 +46,17 @@ mkdir -p $testdir
>  metadump_file=$testdir/scratch.md
>  copy_file=$testdir/copy.img
>  
> -check_restored_metadump_image()
> -{
> -	local image=$1
> -
> -	loop_dev=$(_create_loop_device $image)
> -	SCRATCH_DEV=$loop_dev _scratch_mount
> -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> -	SCRATCH_DEV=$loop_dev _scratch_unmount
> -	_destroy_loop_device $loop_dev
> -}
> -
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  echo "metadump and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-a -o'
>  
>  echo "metadump a and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-a'
>  
>  echo "metadump g and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-g' >> $seqres.full
>  
>  echo "metadump ag and mdrestore"
> -for md_version in $(seq 1 $max_md_version); do
> -	version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v $md_version"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> -	check_restored_metadump_image $TEST_DIR/image
> -done
> +_verify_metadumps '-a -g' >> $seqres.full
>  
>  echo copy
>  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> diff --git a/tests/xfs/605 b/tests/xfs/605
> index f2cd7aba98..af917f0f32 100755
> --- a/tests/xfs/605
> +++ b/tests/xfs/605
> @@ -15,17 +15,13 @@ _cleanup()
>  {
>  	cd /
>  	rm -r -f $tmp.*
> -	_scratch_unmount > /dev/null 2>&1
> -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> -		_destroy_loop_device $logdev
> -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> -	rm -r -f $metadump_file $TEST_DIR/data-image \
> -	   $TEST_DIR/log-image
> +	_cleanup_verify_metadump
>  }
>  
>  # Import common functions.
>  . ./common/dmflakey
>  . ./common/inject
> +. ./common/xfs_metadump_tests
>  
>  # real QA test starts here
>  _supported_fs xfs
> @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
>  _require_dm_target flakey
>  _require_xfs_io_command "pwrite"
>  _require_test_program "punch-alternating"
> +_setup_verify_metadump
>  
> -metadump_file=${TEST_DIR}/${seq}.md
>  testfile=${SCRATCH_MNT}/testfile
>  
>  echo "Format filesystem on scratch device"
>  _scratch_mkfs >> $seqres.full 2>&1
>  
> -max_md_version=$(_xfs_metadump_max_version)
> -
>  external_log=0
>  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
>  	external_log=1
>  fi
>  
> -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
>  	_notrun "metadump v1 does not support external log device"
>  fi
>  
> -verify_metadump_v1()
> -{
> -	local version=""
> -	if [[ $max_md_version == 2 ]]; then
> -		version="-v 1"
> -	fi
> -
> -	_scratch_xfs_metadump $metadump_file -a -o $version
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	SCRATCH_DEV=$datadev _scratch_mount
> -	SCRATCH_DEV=$datadev _check_scratch_fs
> -	SCRATCH_DEV=$datadev _scratch_unmount
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
> -verify_metadump_v2()
> -{
> -	local version="-v 2"
> -
> -	_scratch_xfs_metadump $metadump_file -a -o $version
> -
> -	# Metadump v2 files can contain contents dumped from an external log
> -	# device. Use a temporary file to hold the log device contents restored
> -	# from such a metadump file.
> -	slogdev=""
> -	if [[ -n $SCRATCH_LOGDEV ]]; then
> -		slogdev=$TEST_DIR/log-image
> -	fi
> -
> -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> -		   _scratch_xfs_mdrestore $metadump_file
> -
> -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> -
> -	logdev=""
> -	if [[ -s $slogdev ]]; then
> -		logdev=$(_create_loop_device $slogdev)
> -	fi
> -
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> -
> -	if [[ -s $logdev ]]; then
> -		_destroy_loop_device $logdev
> -		logdev=""
> -		rm -f $slogdev
> -	fi
> -
> -	_destroy_loop_device $datadev
> -	datadev=""
> -	rm -f $TEST_DIR/data-image
> -}
> -
>  echo "Initialize and mount filesystem on flakey device"
>  _init_flakey
>  _load_flakey_table $FLAKEY_ALLOW_WRITES
> @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
>  _print_logstate
>  
>  echo "Create metadump file, restore it and check restored fs"
> -
> -if [[ $external_log == 0 ]]; then
> -	verify_metadump_v1 $max_md_version
> -fi
> -
> -if [[ $max_md_version == 2 ]]; then
> -	verify_metadump_v2
> -fi
> +_verify_metadumps '-a -o'
>  
>  # Mount the fs to replay the contents from the dirty log.
>  _scratch_mount
> 

Re: [PATCH 05/10] common: refactor metadump v1 and v2 tests

[Darrick J. Wong]

On Mon, Feb 05, 2024 at 05:37:44PM +0800, Zorro Lang wrote:
> On Thu, Jan 25, 2024 at 11:05:16AM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Refactor the copy-pasta'd code in xfs/129, xfs/234, xfs/253, xfs/291,
> > xfs/432, xfs/503, and xfs/605 so that we don't have to maintain nearly
> > duplicate copies of the same code.
> > 
> > While we're at it, fix the fsck so that it includes xfs_scrub.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  common/rc                 |   10 ----
> >  common/xfs                |   14 +++++
> >  common/xfs_metadump_tests |  123 +++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/129             |   90 ++-------------------------------
> >  tests/xfs/234             |   91 ++-------------------------------
> >  tests/xfs/253             |   89 ++-------------------------------
> >  tests/xfs/291             |   31 ++++-------
> >  tests/xfs/432             |   30 ++---------
> >  tests/xfs/503             |   60 +++-------------------
> >  tests/xfs/605             |   84 ++-----------------------------
> >  10 files changed, 181 insertions(+), 441 deletions(-)
> >  create mode 100644 common/xfs_metadump_tests
> > 
> > 
> > diff --git a/common/rc b/common/rc
> > index 524ffa02aa..0b69f7f54f 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -3320,15 +3320,7 @@ _check_scratch_fs()
> >  
> >      case $FSTYP in
> >      xfs)
> > -	local scratch_log="none"
> > -	local scratch_rt="none"
> > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > -	    scratch_log="$SCRATCH_LOGDEV"
> > -
> > -	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > -	    scratch_rt="$SCRATCH_RTDEV"
> > -
> > -	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > +	_check_xfs_scratch_fs $device
> >  	;;
> >      udf)
> >  	_check_udf_filesystem $device $udf_fsize
> > diff --git a/common/xfs b/common/xfs
> > index 248ccefda3..6a48960a7f 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -1035,6 +1035,20 @@ _check_xfs_test_fs()
> >  	return $?
> >  }
> >  
> > +_check_xfs_scratch_fs()
> > +{
> > +	local device="${1:-$SCRATCH_DEV}"
> > +	local scratch_log="none"
> > +	local scratch_rt="none"
> > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> > +	    scratch_log="$SCRATCH_LOGDEV"
> > +
> > +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
> > +	    scratch_rt="$SCRATCH_RTDEV"
> > +
> > +	_check_xfs_filesystem $device $scratch_log $scratch_rt
> > +}
> > +
> >  # modeled after _scratch_xfs_repair
> >  _test_xfs_repair()
> >  {
> > diff --git a/common/xfs_metadump_tests b/common/xfs_metadump_tests
> > new file mode 100644
> > index 0000000000..dd3dec1fb4
> > --- /dev/null
> > +++ b/common/xfs_metadump_tests
> > @@ -0,0 +1,123 @@
> > +#
> > +# XFS specific metadump testing functions.
> > +#
> > +
> > +# Set up environment variables for a metadump test.  Requires the test and
> > +# scratch devices.  Sets XFS_METADUMP_{FILE,IMG} and MAX_XFS_METADUMP_VERSION.
> > +_setup_verify_metadump()
> > +{
> > +	XFS_METADUMP_FILE="$TEST_DIR/${seq}_metadump"
> > +	XFS_METADUMP_IMG="$TEST_DIR/${seq}_image"
> > +	MAX_XFS_METADUMP_VERSION="$(_xfs_metadump_max_version)"
> > +
> > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> > +}
> > +
> > +_cleanup_verify_metadump()
> > +{
> > +	_scratch_unmount &>> $seqres.full
> > +
> > +	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> > +		losetup -d "$ldev"
> > +	done
> 
> Hi Darrick,
> 
> If $XFS_METADUMP_IMG is null, this line will delete all loop device. If
> someone uses loop devices to be TEST_DEV and SCRATCH_DEV, then it might
> break the whole testing.
> 
> > +	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> 
> This's dangerous too, I've explained that in :
>   https://lore.kernel.org/fstests/20240205065642.3hqhipmnxkg442kg@dell-per750-06-vm-08.rhts.eng.pek2.redhat.com/T/#m496e69db25755292ed9ac4bee08a22f227ebf7d2

Aaaaggghgh f*cking bash!

> Sorry I didn't notice that when I tested with new upstream kernel. But after I
> tested on old kernel, some metadump cases start to _notrun, then trigger this
> bug. This bug is too dangerous, one of my regular system has gone, I have to
> reinstall. So I decide to reset my last fstests release, to avoid destroying
> other folks' machine.
> 
> I think we'd better to check XFS_METADUMP_FILE and XFS_METADUMP_IMG aren't
> empty at first, e.g.
> 
> if [ -n "$XFS_METADUMP_FILE" -a -n "$XFS_METADUMP_IMG" ];then
> 	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
> 		losetup -d "$ldev"
> 	done
> 	rm -f "$XFS_METADUMP_FILE" "$XFS_METADUMP_IMG"*
> fi
> 
> Or if you'd like to change more :)

That's mostly it, though I'd split the cleanup of the two variables:

test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE"

if [ -n "$XFS_METADUMP_IMG" ];then
	losetup -n -a -O BACK-FILE,NAME | grep "^$XFS_METADUMP_IMG" | while read backing ldev; do
		losetup -d "$ldev"
	done
	for fubar in "$XFS_METADUMP_IMG"*; do
		rm -f "$fubar"
	done
fi

Anyway I'll try this out and send a patch once I've got this fixed.
Sorry I blew up your machine. :( :(

ALSO: should we call shellcheck (or some other linter) as part of make
all?  That might be a good idea, though I actually /did/ run shellcheck
and it didn't say anything about the rm glob trap.

--D

> 
> Thanks,
> Zorro
> 
> > +}
> > +
> > +# Create a metadump in v1 format, restore it to fs image files, then mount the
> > +# images and fsck them.
> > +_verify_metadump_v1()
> > +{
> > +	local metadump_args="$1"
> > +	local extra_test="$2"
> > +
> > +	local metadump_file="$XFS_METADUMP_FILE"
> > +	local version=""
> > +	local data_img="$XFS_METADUMP_IMG.data"
> > +	local data_loop
> > +
> > +	# Force v1 if we detect v2 support
> > +	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
> > +		version="-v 1"
> > +	fi
> > +
> > +	# Capture metadump, which creates metadump_file
> > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > +
> > +	# Restore metadump, which creates data_img
> > +	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
> > +
> > +	# Create loopdev for data device so we can mount the fs
> > +	data_loop=$(_create_loop_device $data_img)
> > +
> > +	# Mount fs, run an extra test, fsck, and unmount
> > +	SCRATCH_DEV=$data_loop _scratch_mount
> > +	if [ -n "$extra_test" ]; then
> > +		SCRATCH_DEV=$data_loop $extra_test
> > +	fi
> > +	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
> > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > +
> > +	# Tear down what we created
> > +	_destroy_loop_device $data_loop
> > +	rm -f $data_img
> > +}
> > +
> > +# Create a metadump in v2 format, restore it to fs image files, then mount the
> > +# images and fsck them.
> > +_verify_metadump_v2()
> > +{
> > +	local metadump_args="$1"
> > +	local extra_test="$2"
> > +
> > +	local metadump_file="$XFS_METADUMP_FILE"
> > +	local version="-v 2"
> > +	local data_img="$XFS_METADUMP_IMG.data"
> > +	local data_loop
> > +	local log_img=""
> > +	local log_loop
> > +
> > +	# Capture metadump, which creates metadump_file
> > +	_scratch_xfs_metadump $metadump_file $metadump_args $version
> > +
> > +	#
> > +	# Metadump v2 files can contain contents dumped from an external log
> > +	# device. Use a temporary file to hold the log device contents restored
> > +	# from such a metadump file.
> > +	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
> > +
> > +	# Restore metadump, which creates data_img and log_img
> > +	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
> > +		_scratch_xfs_mdrestore $metadump_file
> > +
> > +	# Create loopdev for data device so we can mount the fs
> > +	data_loop=$(_create_loop_device $data_img)
> > +
> > +	# Create loopdev for log device if we recovered anything
> > +	test -s "$log_img" && log_loop=$(_create_loop_device $log_img)
> > +
> > +	# Mount fs, run an extra test, fsck, and unmount
> > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _scratch_mount
> > +	if [ -n "$extra_test" ]; then
> > +		SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop $extra_test
> > +	fi
> > +	SCRATCH_DEV=$data_loop SCRATCH_LOGDEV=$log_loop _check_xfs_scratch_fs
> > +	SCRATCH_DEV=$data_loop _scratch_unmount
> > +
> > +	# Tear down what we created
> > +	if [ -b "$log_loop" ]; then
> > +		_destroy_loop_device $log_loop
> > +		rm -f $log_img
> > +	fi
> > +	_destroy_loop_device $data_loop
> > +	rm -f $data_img
> > +}
> > +
> > +# Verify both metadump formats if possible
> > +_verify_metadumps()
> > +{
> > +	_verify_metadump_v1 "$@"
> > +
> > +	if [[ $MAX_XFS_METADUMP_FORMAT == 2 ]]; then
> > +		_verify_metadump_v2 "$@"
> > +	fi
> > +}
> > diff --git a/tests/xfs/129 b/tests/xfs/129
> > index cdac2349df..c3a9bcefee 100755
> > --- a/tests/xfs/129
> > +++ b/tests/xfs/129
> > @@ -16,98 +16,23 @@ _cleanup()
> >  {
> >      cd /
> >      _scratch_unmount > /dev/null 2>&1
> > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > -	    _destroy_loop_device $logdev
> > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/data-image \
> > -       $TEST_DIR/log-image
> > +    _cleanup_verify_metadump
> > +    rm -rf $tmp.* $testdir
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> >  . ./common/reflink
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_loop
> >  _require_scratch_reflink
> > -
> > -metadump_file=$TEST_DIR/${seq}_metadump
> > -
> > -verify_metadump_v1()
> > -{
> > -	local max_version=$1
> > -	local version=""
> > -
> > -	if [[ $max_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	logdev=$SCRATCH_LOGDEV
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	version="-v 2"
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=${SCRATCH_LOGDEV}
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $TEST_DIR/log-image
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > +_setup_verify_metadump
> >  
> >  _scratch_mkfs >/dev/null 2>&1
> > -
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  _scratch_mount
> >  
> >  testdir=$SCRATCH_MNT/test-$seq
> > @@ -127,12 +52,7 @@ done
> >  _scratch_unmount
> >  
> >  echo "Create metadump file, restore it and check restored fs"
> > -
> > -verify_metadump_v1 $max_md_version
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps
> >  
> >  # success, all done
> >  status=0
> > diff --git a/tests/xfs/234 b/tests/xfs/234
> > index f4f8af6d3a..8f808c7507 100755
> > --- a/tests/xfs/234
> > +++ b/tests/xfs/234
> > @@ -15,16 +15,13 @@ _begin_fstest auto quick rmap punch metadump
> >  _cleanup()
> >  {
> >      cd /
> > -    _scratch_unmount > /dev/null 2>&1
> > -    [[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > -	    _destroy_loop_device $logdev
> > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > -    rm -rf $tmp.* $testdir $metadump_file $TEST_DIR/image \
> > -       $TEST_DIR/log-image
> > +    _cleanup_verify_metadump
> > +    rm -rf $tmp.* $testdir
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> > @@ -32,82 +29,9 @@ _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_loop
> >  _require_xfs_scratch_rmapbt
> >  _require_xfs_io_command "fpunch"
> > -
> > -metadump_file=$TEST_DIR/${seq}_metadump
> > -
> > -verify_metadump_v1()
> > -{
> > -	local max_version=$1
> > -	local version=""
> > -
> > -	if [[ $max_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	logdev=$SCRATCH_LOGDEV
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	version="-v 2"
> > -
> > -	_scratch_xfs_metadump $metadump_file $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=${SCRATCH_LOGDEV}
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	[[ -z $logdev ]] && logdev=none
> > -	_check_xfs_filesystem $datadev $logdev none
> > -
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $TEST_DIR/log-image
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > +_setup_verify_metadump
> >  
> >  _scratch_mkfs >/dev/null 2>&1
> > -
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  _scratch_mount
> >  
> >  testdir=$SCRATCH_MNT/test-$seq
> > @@ -127,12 +51,7 @@ done
> >  _scratch_unmount
> >  
> >  echo "Create metadump file, restore it and check restored fs"
> > -
> > -verify_metadump_v1 $max_md_version
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps
> >  
> >  # success, all done
> >  status=0
> > diff --git a/tests/xfs/253 b/tests/xfs/253
> > index 3b567999d8..6623c435e5 100755
> > --- a/tests/xfs/253
> > +++ b/tests/xfs/253
> > @@ -26,23 +26,21 @@ _cleanup()
> >      cd /
> >      rm -f $tmp.*
> >      rm -rf "${OUTPUT_DIR}"
> > -    rm -f "${METADUMP_FILE}"
> > -    [[ -n $logdev && $logdev != $SCRATCH_LOGDEV ]] && \
> > -	    _destroy_loop_device $logdev
> > -    [[ -n $datadev ]] && _destroy_loop_device $datadev
> > +    _cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_test
> >  _require_scratch
> > +_setup_verify_metadump
> >  
> >  # real QA test starts here
> >  
> >  OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> > -METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> >  ORPHANAGE="lost+found"
> >  
> >  _supported_fs xfs
> > @@ -52,24 +50,7 @@ function create_file() {
> >  	touch $(printf "$@")
> >  }
> >  
> > -verify_metadump_v1()
> > -{
> > -	local max_version=$1
> > -	local version=""
> > -
> > -	if [[ $max_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV="" \
> > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -
> > +extra_test() {
> >  	cd "${SCRATCH_MNT}"
> >  
> >  	# Get a listing of all the files after obfuscation
> > @@ -78,60 +59,6 @@ verify_metadump_v1()
> >  	ls -R | od -c >> $seqres.full
> >  
> >  	cd /
> > -
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	version="-v 2"
> > -
> > -	_scratch_xfs_metadump $METADUMP_FILE $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $METADUMP_FILE
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=${SCRATCH_LOGDEV}
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		logdev=$(_create_loop_device $TEST_DIR/log-image)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -
> > -	cd "${SCRATCH_MNT}"
> > -
> > -	# Get a listing of all the files after obfuscation
> > -	echo "Metadump v2" >> $seqres.full
> > -	ls -R >> $seqres.full
> > -	ls -R | od -c >> $seqres.full
> > -
> > -	cd /
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	if [[ -s $TEST_DIR/log-image ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $TEST_DIR/log-image
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> >  }
> >  
> >  echo "Disciplyne of silence is goed."
> > @@ -233,13 +160,7 @@ cd $here
> >  
> >  _scratch_unmount
> >  
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> > -verify_metadump_v1 $max_md_version
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps '' extra_test
> >  
> >  # Finally, re-make the filesystem since to ensure we don't
> >  # leave a directory with duplicate entries lying around.
> > diff --git a/tests/xfs/291 b/tests/xfs/291
> > index 1433140821..c475d89ad9 100755
> > --- a/tests/xfs/291
> > +++ b/tests/xfs/291
> > @@ -9,11 +9,21 @@
> >  . ./common/preamble
> >  _begin_fstest auto repair metadump
> >  
> > +# Override the default cleanup function.
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -r -f $tmp.*
> > +	_cleanup_verify_metadump
> > +}
> > +
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  _supported_fs xfs
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> > +_setup_verify_metadump
> >  
> >  # real QA test starts here
> >  _require_scratch
> > @@ -92,26 +102,7 @@ _scratch_xfs_check >> $seqres.full 2>&1 || _fail "xfs_check failed"
> >  
> >  # Yes they can!  Now...
> >  # Can xfs_metadump cope with this monster?
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $tmp.metadump -a -o $version || \
> > -		_fail "xfs_metadump failed"
> > -
> > -	slogdev=$SCRATCH_LOGDEV
> > -	if [[ -z $version || $version == "-v 1" ]]; then
> > -		slogdev=""
> > -	fi
> > -	SCRATCH_DEV=$tmp.img SCRATCH_LOGDEV=$slogdev _scratch_xfs_mdrestore \
> > -		   $tmp.metadump || _fail "xfs_mdrestore failed"
> > -	SCRATCH_DEV=$tmp.img _scratch_xfs_repair -f &>> $seqres.full || \
> > -		_fail "xfs_repair of metadump failed"
> > -done
> > +_verify_metadumps '-a -o'
> >  
> >  # Yes it can; success, all done
> >  status=0
> > diff --git a/tests/xfs/432 b/tests/xfs/432
> > index 7e402aa88f..579e1b556a 100755
> > --- a/tests/xfs/432
> > +++ b/tests/xfs/432
> > @@ -20,16 +20,19 @@ _begin_fstest auto quick dir metadata metadump
> >  _cleanup()
> >  {
> >  	cd /
> > -	rm -f "$tmp".* $metadump_file $metadump_img
> > +	rm -f "$tmp".*
> > +	_cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> >  _require_command "$XFS_MDRESTORE_PROG" "xfs_mdrestore"
> >  _require_scratch
> > +_setup_verify_metadump
> >  
> >  rm -f "$seqres.full"
> >  
> > @@ -54,9 +57,6 @@ echo "Format and mount"
> >  _scratch_mkfs -b size=1k -n size=64k > "$seqres.full" 2>&1
> >  _scratch_mount >> "$seqres.full" 2>&1
> >  
> > -metadump_file="$TEST_DIR/meta-$seq"
> > -metadump_img="$TEST_DIR/img-$seq"
> > -rm -f $metadump_file $metadump_img
> >  testdir="$SCRATCH_MNT/test-$seq"
> >  max_fname_len=255
> >  blksz=$(_get_block_size $SCRATCH_MNT)
> > @@ -87,27 +87,7 @@ echo "qualifying extent: $extlen blocks" >> $seqres.full
> >  test -n "$extlen" || _notrun "could not create dir extent > 1000 blocks"
> >  
> >  echo "Try to metadump, restore and check restored metadump image"
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o -w $version
> > -
> > -	slogdev=$SCRATCH_LOGDEV
> > -	if [[ -z $version || $version == "-v 1" ]]; then
> > -		slogdev=""
> > -	fi
> > -
> > -	SCRATCH_DEV=$metadump_img SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	SCRATCH_DEV=$metadump_img _scratch_xfs_repair -n &>> $seqres.full || \
> > -		echo "xfs_repair on restored fs returned $?"
> > -done
> > +_verify_metadumps '-a -o -w'
> >  
> >  # success, all done
> >  status=0
> > diff --git a/tests/xfs/503 b/tests/xfs/503
> > index 8643c3d483..ff6b344a9c 100755
> > --- a/tests/xfs/503
> > +++ b/tests/xfs/503
> > @@ -17,11 +17,13 @@ _cleanup()
> >  {
> >  	cd /
> >  	rm -rf $tmp.* $testdir
> > +	_cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/filter
> >  . ./common/populate
> > +. ./common/xfs_metadump_tests
> >  
> >  testdir=$TEST_DIR/test-$seq
> >  
> > @@ -35,6 +37,7 @@ _require_scratch_nocheck
> >  _require_populate_commands
> >  _xfs_skip_online_rebuild
> >  _xfs_skip_offline_rebuild
> > +_setup_verify_metadump
> >  
> >  echo "Format and populate"
> >  _scratch_populate_cached nofill > $seqres.full 2>&1
> > @@ -43,66 +46,17 @@ mkdir -p $testdir
> >  metadump_file=$testdir/scratch.md
> >  copy_file=$testdir/copy.img
> >  
> > -check_restored_metadump_image()
> > -{
> > -	local image=$1
> > -
> > -	loop_dev=$(_create_loop_device $image)
> > -	SCRATCH_DEV=$loop_dev _scratch_mount
> > -	SCRATCH_DEV=$loop_dev _check_scratch_fs
> > -	SCRATCH_DEV=$loop_dev _scratch_unmount
> > -	_destroy_loop_device $loop_dev
> > -}
> > -
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  echo "metadump and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-a -o'
> >  
> >  echo "metadump a and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-a'
> >  
> >  echo "metadump g and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -g $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-g' >> $seqres.full
> >  
> >  echo "metadump ag and mdrestore"
> > -for md_version in $(seq 1 $max_md_version); do
> > -	version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v $md_version"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -g $version >> $seqres.full
> > -	SCRATCH_DEV=$TEST_DIR/image _scratch_xfs_mdrestore $metadump_file
> > -	check_restored_metadump_image $TEST_DIR/image
> > -done
> > +_verify_metadumps '-a -g' >> $seqres.full
> >  
> >  echo copy
> >  $XFS_COPY_PROG $SCRATCH_DEV $copy_file >> $seqres.full
> > diff --git a/tests/xfs/605 b/tests/xfs/605
> > index f2cd7aba98..af917f0f32 100755
> > --- a/tests/xfs/605
> > +++ b/tests/xfs/605
> > @@ -15,17 +15,13 @@ _cleanup()
> >  {
> >  	cd /
> >  	rm -r -f $tmp.*
> > -	_scratch_unmount > /dev/null 2>&1
> > -	[[ -n $logdev && $logdev != "none" && $logdev != $SCRATCH_LOGDEV ]] && \
> > -		_destroy_loop_device $logdev
> > -	[[ -n $datadev ]] && _destroy_loop_device $datadev
> > -	rm -r -f $metadump_file $TEST_DIR/data-image \
> > -	   $TEST_DIR/log-image
> > +	_cleanup_verify_metadump
> >  }
> >  
> >  # Import common functions.
> >  . ./common/dmflakey
> >  . ./common/inject
> > +. ./common/xfs_metadump_tests
> >  
> >  # real QA test starts here
> >  _supported_fs xfs
> > @@ -37,85 +33,22 @@ _require_xfs_io_error_injection log_item_pin
> >  _require_dm_target flakey
> >  _require_xfs_io_command "pwrite"
> >  _require_test_program "punch-alternating"
> > +_setup_verify_metadump
> >  
> > -metadump_file=${TEST_DIR}/${seq}.md
> >  testfile=${SCRATCH_MNT}/testfile
> >  
> >  echo "Format filesystem on scratch device"
> >  _scratch_mkfs >> $seqres.full 2>&1
> >  
> > -max_md_version=$(_xfs_metadump_max_version)
> > -
> >  external_log=0
> >  if [[ $USE_EXTERNAL = yes && -n "$SCRATCH_LOGDEV" ]]; then
> >  	external_log=1
> >  fi
> >  
> > -if [[ $max_md_version == 1 && $external_log == 1 ]]; then
> > +if [[ $MAX_XFS_METADUMP_FORMAT == 1 && $external_log == 1 ]]; then
> >  	_notrun "metadump v1 does not support external log device"
> >  fi
> >  
> > -verify_metadump_v1()
> > -{
> > -	local version=""
> > -	if [[ $max_md_version == 2 ]]; then
> > -		version="-v 1"
> > -	fi
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	SCRATCH_DEV=$datadev _scratch_mount
> > -	SCRATCH_DEV=$datadev _check_scratch_fs
> > -	SCRATCH_DEV=$datadev _scratch_unmount
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> > -verify_metadump_v2()
> > -{
> > -	local version="-v 2"
> > -
> > -	_scratch_xfs_metadump $metadump_file -a -o $version
> > -
> > -	# Metadump v2 files can contain contents dumped from an external log
> > -	# device. Use a temporary file to hold the log device contents restored
> > -	# from such a metadump file.
> > -	slogdev=""
> > -	if [[ -n $SCRATCH_LOGDEV ]]; then
> > -		slogdev=$TEST_DIR/log-image
> > -	fi
> > -
> > -	SCRATCH_DEV=$TEST_DIR/data-image SCRATCH_LOGDEV=$slogdev \
> > -		   _scratch_xfs_mdrestore $metadump_file
> > -
> > -	datadev=$(_create_loop_device $TEST_DIR/data-image)
> > -
> > -	logdev=""
> > -	if [[ -s $slogdev ]]; then
> > -		logdev=$(_create_loop_device $slogdev)
> > -	fi
> > -
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_mount
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _check_scratch_fs
> > -	SCRATCH_DEV=$datadev SCRATCH_LOGDEV=$logdev _scratch_unmount
> > -
> > -	if [[ -s $logdev ]]; then
> > -		_destroy_loop_device $logdev
> > -		logdev=""
> > -		rm -f $slogdev
> > -	fi
> > -
> > -	_destroy_loop_device $datadev
> > -	datadev=""
> > -	rm -f $TEST_DIR/data-image
> > -}
> > -
> >  echo "Initialize and mount filesystem on flakey device"
> >  _init_flakey
> >  _load_flakey_table $FLAKEY_ALLOW_WRITES
> > @@ -160,14 +93,7 @@ echo -n "Filesystem has a "
> >  _print_logstate
> >  
> >  echo "Create metadump file, restore it and check restored fs"
> > -
> > -if [[ $external_log == 0 ]]; then
> > -	verify_metadump_v1 $max_md_version
> > -fi
> > -
> > -if [[ $max_md_version == 2 ]]; then
> > -	verify_metadump_v2
> > -fi
> > +_verify_metadumps '-a -o'
> >  
> >  # Mount the fs to replay the contents from the dirty log.
> >  _scratch_mount
> > 
> 
> 

[PATCHSET] fstests: random fixes for v2024.01.14

[Darrick J. Wong]

Hi all,

Here's the usual odd fixes for fstests.  Most of these are cleanups and
bug fixes for the recently merged xfs metadump v2 testing code.

This second attempt fixes a severe bug in the cleanup code from the
refactored metadump testing code and adds a few adjustments that the
maintainer asked for.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes-2024.01.14
---
Commits in this patchset:
 * generic/256: constrain runtime with TIME_FACTOR
 * common/xfs: simplify maximum metadump format detection
 * common/populate: always metadump full metadata blocks
 * xfs/336: fix omitted -a and -o in metadump call
 * common: refactor metadump v1 and v2 tests, version 2
 * xfs/{129,234,253,605}: disable metadump v1 testing with external devices
 * xfs/503: test metadump obfuscation, not progressbars
 * xfs/503: split copy and metadump into two tests
 * common/xfs: only pass -l in _xfs_mdrestore for v2 metadumps
 * xfs/122: fix for xfs_attr_shortform removal in 6.8
---
 common/metadump   |  152 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 common/populate   |    2 -
 common/rc         |   10 ---
 common/xfs        |   49 ++++++++++++++++-
 tests/generic/256 |    7 ++
 tests/xfs/122.out |    2 +
 tests/xfs/129     |   91 ++------------------------------
 tests/xfs/234     |   92 ++------------------------------
 tests/xfs/253     |   90 ++-----------------------------
 tests/xfs/284     |    4 +
 tests/xfs/291     |   32 ++++-------
 tests/xfs/336     |    2 -
 tests/xfs/432     |   31 ++---------
 tests/xfs/503     |   82 ++++-------------------------
 tests/xfs/503.out |    6 +-
 tests/xfs/601     |   54 +++++++++++++++++++
 tests/xfs/601.out |    4 +
 tests/xfs/605     |   92 +-------------------------------
 18 files changed, 318 insertions(+), 484 deletions(-)
 create mode 100644 common/metadump
 create mode 100755 tests/xfs/601
 create mode 100755 tests/xfs/601.out