[PATCHSET 0/9] fstests: random fixes

[PATCHSET 0/9] fstests: random fixes

[Darrick J. Wong]

Hi all,

This branch contains fixes to various tests to fix miscellaneous test
bugs and unnecessary regressions when XFS is configured with somewhat
unusual configurations (e.g. always-cow mode, external logs, and/or
realtime devices).

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
 common/dmthin     |    9 ++++++++-
 common/dump       |    1 +
 common/rc         |   13 ++++++++++++-
 common/repair     |    2 +-
 tests/generic/094 |    5 +++++
 tests/generic/223 |    3 +++
 tests/generic/225 |    5 +++++
 tests/generic/347 |    2 +-
 tests/generic/500 |    2 +-
 tests/generic/563 |    7 ++++++-
 tests/generic/620 |    2 +-
 tests/xfs/030     |    2 +-
 tests/xfs/083     |    2 +-
 tests/xfs/305     |    2 +-
 tests/xfs/506     |    1 +
 tests/xfs/521     |    2 +-
 tests/xfs/530     |    2 +-
 17 files changed, 50 insertions(+), 12 deletions(-)

[PATCH 1/9] xfs/506: fix regression on freshly quotachecked filesystem

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The first thing this test checks is that the health command reports that
nothing has been checked.  This isn't true if we regenerated the quota
counts when we mounted the filesystem (and hence they're marked healthy
and checked), so cycle the mount to get rid of that state.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/506 |    1 +
 1 file changed, 1 insertion(+)


diff --git a/tests/xfs/506 b/tests/xfs/506
index ddda8614..7e553849 100755
--- a/tests/xfs/506
+++ b/tests/xfs/506
@@ -36,6 +36,7 @@ rm -f $seqres.full
 
 _scratch_mkfs > $seqres.full 2>&1
 _scratch_mount
+_scratch_cycle_mount	# make sure we haven't run quotacheck on this mount
 
 # Haven't checked anything, it should tell us to run scrub
 $XFS_SPACEMAN_PROG -c "health" $SCRATCH_MNT

[PATCH 2/9] generic/563: selectively remove the io cgroup controller

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If a system configuration tool such as systemd sets up the io cgroup
controller for its own purposes, it's possible that the last line of
this test will not be able to remove the io controller from the system
configuration.  This causes the test to fail even though the inability
to tear down systemd should not be considered (in this case) a failure.

Change this test to set the "io" component of subtree control back to
whatever it was when the test started.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/563 |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)


diff --git a/tests/generic/563 b/tests/generic/563
index b113eacf..fe7394c0 100755
--- a/tests/generic/563
+++ b/tests/generic/563
@@ -103,6 +103,9 @@ sminor=$((0x`stat -L -c %T $LOOP_DEV`))
 _mkfs_dev $LOOP_DEV >> $seqres.full 2>&1
 _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed"
 
+drop_io_cgroup=
+grep -q -w io $cgdir/cgroup.subtree_control || drop_io_cgroup=1
+
 echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
 
 # Read and write from a single group.
@@ -143,7 +146,9 @@ $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
 check_cg $cgdir/$seq-cg $iosize $iosize
 check_cg $cgdir/$seq-cg-2 0 0
 
-echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
+if [ "$drop_io_cgroup" = 1 ]; then
+	echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
+fi
 
 # success, all done
 status=0

[PATCH 3/9] xfs/521,530: refactor scratch fs check

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Use the existing _check_scratch_fs helper to check the (modified)
scratch filesystem in these tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/521 |    2 +-
 tests/xfs/530 |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/521 b/tests/xfs/521
index e6c417b8..b8026d45 100755
--- a/tests/xfs/521
+++ b/tests/xfs/521
@@ -75,7 +75,7 @@ echo "Create more copies to make sure the bitmap really works"
 cp -p $testdir/original $testdir/copy3
 
 echo "Check filesystem"
-_check_xfs_filesystem $SCRATCH_DEV none $rtdev
+_check_scratch_fs
 
 # success, all done
 status=0
diff --git a/tests/xfs/530 b/tests/xfs/530
index 65c17af2..0e4dd6b5 100755
--- a/tests/xfs/530
+++ b/tests/xfs/530
@@ -114,7 +114,7 @@ for rtino in rbmino rsumino; do
 done
 
 echo "Check filesystem"
-_check_xfs_filesystem $SCRATCH_DEV none $rtdev
+_check_scratch_fs
 
 losetup -d $rtdev
 rm -f $TEST_DIR/$seq.rtvol

[PATCH 4/9] common/dump: filter out xfs_restore messages about fallocate failures

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

fallocate is not supported on XFS filesystems that are running in
"always COW" mode.  This leads to false test regressions because
xfs_restore complains about EOPNOTSUPP in the golden output.  The
preallocation isn't required for correct xfs_restore operation, so
filter out the EOPNOTSUPP messages.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/dump |    1 +
 1 file changed, 1 insertion(+)


diff --git a/common/dump b/common/dump
index 2b8e0893..3c4029ff 100644
--- a/common/dump
+++ b/common/dump
@@ -866,6 +866,7 @@ _dump_filter_main()
       -e 's/id:[[:space:]]*[0-9a-f]\{8\}-[0-9a-f]\{4\}-[0-9a-f]\{4\}-[0-9a-f]\{4\}-[0-9a-f]\{12\}/ID: ID/'						\
       -e 's/\[y\/n\][- ]----------------------*/\[y\/n\]/'		\
       -e '/skip attribute set/d'				\
+      -e '/xfsrestore: NOTE: attempt to reserve [0-9]* bytes for.*Operation not supported/d' \
   | perl -ne '
 	# filter out all the output between the lines "Dump Summary:"
 	# and "Dump Status:"

[PATCH 5/9] common/dmthin: make this work with external log devices

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Provide a mkfs helper to format the dm thin device when external devices
are in use, and fix the dmthin mount helper to support them.  This fixes
regressions in generic/347 and generic/500 when external logs are in
use.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/dmthin     |    9 ++++++++-
 tests/generic/223 |    3 +++
 tests/generic/347 |    2 +-
 tests/generic/500 |    2 +-
 4 files changed, 13 insertions(+), 3 deletions(-)


diff --git a/common/dmthin b/common/dmthin
index c58c3948..3b1c7d45 100644
--- a/common/dmthin
+++ b/common/dmthin
@@ -218,10 +218,17 @@ _dmthin_set_fail()
 
 _dmthin_mount_options()
 {
-	echo `_common_dev_mount_options $*` $DMTHIN_VOL_DEV $SCRATCH_MNT
+	_scratch_options mount
+	echo `_common_dev_mount_options $*` $SCRATCH_OPTIONS $DMTHIN_VOL_DEV $SCRATCH_MNT
 }
 
 _dmthin_mount()
 {
 	_mount -t $FSTYP `_dmthin_mount_options $*`
 }
+
+_dmthin_mkfs()
+{
+	_scratch_options mkfs
+	_mkfs_dev $SCRATCH_OPTIONS $@ $DMTHIN_VOL_DEV
+}
diff --git a/tests/generic/223 b/tests/generic/223
index 1f85efe5..a5ace82f 100755
--- a/tests/generic/223
+++ b/tests/generic/223
@@ -43,6 +43,9 @@ for SUNIT_K in 8 16 32 64 128; do
 	_scratch_mkfs_geom $SUNIT_BYTES 4 $BLOCKSIZE >> $seqres.full 2>&1
 	_scratch_mount
 
+	# Make sure everything is on the data device
+	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
+
 	for SIZE_MULT in 1 2 8 64 256; do
 		let SIZE=$SIZE_MULT*$SUNIT_BYTES
 
diff --git a/tests/generic/347 b/tests/generic/347
index cbc5150a..e970ac10 100755
--- a/tests/generic/347
+++ b/tests/generic/347
@@ -31,7 +31,7 @@ _setup_thin()
 {
 	_dmthin_init $BACKING_SIZE $VIRTUAL_SIZE
 	_dmthin_set_queue
-	_mkfs_dev $DMTHIN_VOL_DEV
+	_dmthin_mkfs
 	_dmthin_mount
 }
 
diff --git a/tests/generic/500 b/tests/generic/500
index 085ddbf3..5ab2f78c 100755
--- a/tests/generic/500
+++ b/tests/generic/500
@@ -68,7 +68,7 @@ CLUSTER_SIZE=$((64 * 1024 / 512))		# 64K
 
 _dmthin_init $BACKING_SIZE $VIRTUAL_SIZE $CLUSTER_SIZE 0
 _dmthin_set_fail
-_mkfs_dev $DMTHIN_VOL_DEV
+_dmthin_mkfs
 _dmthin_mount
 
 # There're two bugs at here, one is dm-thin bug, the other is filesystem

[PATCH 6/9] misc: replace more open-coded _scratch_xfs_db calls

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Replace the last remaining open-coded calls to xfs_db for the scratch
device with calls to _scratch_xfs_db.  This fixes these tests when
external logs are enabled.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/repair |    2 +-
 tests/xfs/030 |    2 +-
 tests/xfs/083 |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/common/repair b/common/repair
index 8adc2178..5a957f4e 100644
--- a/common/repair
+++ b/common/repair
@@ -10,7 +10,7 @@ _zero_position()
 	struct="$2"
 
 	# set values for off/len variables provided by db
-	eval `xfs_db -r -c "$struct" -c stack $SCRATCH_DEV | perl -ne '
+	eval `_scratch_xfs_db -r -c "$struct" -c stack | perl -ne '
 		if (/byte offset (\d+), length (\d+)/) {
 			print "offset=$1\nlength=$2\n"; exit
 		}'`
diff --git a/tests/xfs/030 b/tests/xfs/030
index 906d9019..81198155 100755
--- a/tests/xfs/030
+++ b/tests/xfs/030
@@ -81,7 +81,7 @@ else
 	_scratch_unmount
 fi
 clear=""
-eval `xfs_db -r -c "sb 1" -c stack $SCRATCH_DEV | perl -ne '
+eval `_scratch_xfs_db -r -c "sb 1" -c stack | perl -ne '
 	if (/byte offset (\d+), length (\d+)/) {
 		print "clear=", $1 / 512, "\n"; exit
 	}'`
diff --git a/tests/xfs/083 b/tests/xfs/083
index a548be2a..a3f32cb7 100755
--- a/tests/xfs/083
+++ b/tests/xfs/083
@@ -108,7 +108,7 @@ echo "+ check fs" >> $seqres.full
 _scratch_xfs_repair >> $seqres.full 2>&1 || _fail "should pass initial fsck"
 
 echo "++ corrupt image" >> $seqres.full
-xfs_db -x -c blockget -c "blocktrash ${FUZZ_ARGS}" "${SCRATCH_DEV}" >> $seqres.full 2>&1
+_scratch_xfs_db -x -c blockget -c "blocktrash ${FUZZ_ARGS}" >> $seqres.full 2>&1
 
 echo "++ mount image" >> $seqres.full
 _try_scratch_mount >> $seqres.full 2>&1

[PATCH 7/9] generic/620: fix order of require_scratch calls

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

_require_scratch_16T_support does not itself check that the scratch
device exists, which means that it depends on someone else to call
_require_scratch.  Document this dependency and fix this test so that we
can run:

./check --exact-order generic/374 generic/620

on an ext4 filesystem without g/620 tripping over the mess left by g/374
when it calls _notrun.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc         |    3 ++-
 tests/generic/620 |    2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)


diff --git a/common/rc b/common/rc
index 23f86ce6..bb54df56 100644
--- a/common/rc
+++ b/common/rc
@@ -1608,7 +1608,8 @@ _require_scratch_size_nocheck()
 	[ $devsize -lt $1 ] && _notrun "scratch dev too small"
 }
 
-# require scratch fs which supports >16T of filesystem size.
+# Require scratch fs which supports >16T of filesystem size.
+# _require_scratch must be called before this function is called.
 _require_scratch_16T_support()
 {
 	case $FSTYP in
diff --git a/tests/generic/620 b/tests/generic/620
index d0e58ca4..60559441 100755
--- a/tests/generic/620
+++ b/tests/generic/620
@@ -41,8 +41,8 @@ rm -f $seqres.full
 
 # Modify as appropriate.
 _supported_fs generic
-_require_scratch_16T_support
 _require_scratch_size_nocheck $((4 * 1024 * 1024)) #kB
+_require_scratch_16T_support
 _require_dmhugedisk
 
 # 17TB dm huge-test-zer0 device

[PATCH 8/9] generic/{094,225}: skip test when the xfs rt extent size is larger than 1 fsb

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

These two tests call various fallocate modes on a file and compare the
FIEMAP output to some golden output.  Unfortunately, the golden output
doesn't take into account the possibility that (on XFS) the files could be
created on a realtime volume with a large rt extent size set.

Under such a configuration, fpunch operations that are aligned to the fs
block size but not the rt extent size simply result in those blocks
being set to unwritten status.  Unfortunately, the test expects holes
and fails.  Therefore, detect the situation and skip the tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc         |   10 ++++++++++
 tests/generic/094 |    5 +++++
 tests/generic/225 |    5 +++++
 3 files changed, 20 insertions(+)


diff --git a/common/rc b/common/rc
index bb54df56..b5e930de 100644
--- a/common/rc
+++ b/common/rc
@@ -4062,6 +4062,16 @@ _get_block_size()
 	stat -f -c %S $1
 }
 
+# Require that the fundamental allocation unit of a file is the same as the
+# filesystem block size.
+_require_file_block_size_equals_fs_block_size()
+{
+	local file_alloc_unit="$(_get_file_block_size $1)"
+	local fs_block_size="$(_get_block_size $1)"
+	test "$file_alloc_unit" != "$fs_block_size" && \
+		_notrun "File allocation unit is larger than a filesystem block"
+}
+
 get_page_size()
 {
 	echo $(getconf PAGE_SIZE)
diff --git a/tests/generic/094 b/tests/generic/094
index d371e951..8c292473 100755
--- a/tests/generic/094
+++ b/tests/generic/094
@@ -40,6 +40,11 @@ fiemapfile=$SCRATCH_MNT/$seq.fiemap
 
 _require_test_program "fiemap-tester"
 
+# FIEMAP test doesn't like finding unwritten blocks after it punches out
+# a partial rt extent.
+test "$FSTYP" = "xfs" && \
+	_require_file_block_size_equals_fs_block_size $fiemapfile
+
 seed=`date +%s`
 
 echo "using seed $seed" >> $seqres.full
diff --git a/tests/generic/225 b/tests/generic/225
index 1228a256..fac688df 100755
--- a/tests/generic/225
+++ b/tests/generic/225
@@ -40,6 +40,11 @@ fiemaplog=$SCRATCH_MNT/$seq.log
 
 _require_test_program "fiemap-tester"
 
+# FIEMAP test doesn't like finding unwritten blocks after it punches out
+# a partial rt extent.
+test "$FSTYP" = "xfs" && \
+	_require_file_block_size_equals_fs_block_size $fiemapfile
+
 seed=`date +%s`
 
 echo "using seed $seed" >> $fiemaplog

[PATCH 9/9] xfs/305: make sure that fsstress is still running when we quotaoff

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Greatly increase the number of fs ops that fsstress is supposed to run
in in this test so that we can ensure that it's still running when the
quotaoff gets run.  1000 might have been sufficient in 2013, but it
isn't now.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/305 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/xfs/305 b/tests/xfs/305
index fdb48a1d..9a3f0e03 100755
--- a/tests/xfs/305
+++ b/tests/xfs/305
@@ -50,7 +50,7 @@ _exercise()
 	_qmount
 	mkdir -p $QUOTA_DIR
 
-	$FSSTRESS_PROG -d $QUOTA_DIR -n 1000 -p 100 $FSSTRESS_AVOID >/dev/null 2>&1 &
+	$FSSTRESS_PROG -d $QUOTA_DIR -n 1000000 -p 100 $FSSTRESS_AVOID >/dev/null 2>&1 &
 	sleep 10
 	xfs_quota -x -c "off -$type" $SCRATCH_DEV
 	sleep 5

Re: [PATCH 1/9] xfs/506: fix regression on freshly quotachecked filesystem

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 2/9] generic/563: selectively remove the io cgroup controller

[Christoph Hellwig]

On Tue, Apr 13, 2021 at 06:04:42PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> If a system configuration tool such as systemd sets up the io cgroup
> controller for its own purposes, it's possible that the last line of
> this test will not be able to remove the io controller from the system
> configuration.  This causes the test to fail even though the inability
> to tear down systemd should not be considered (in this case) a failure.
> 
> Change this test to set the "io" component of subtree control back to
> whatever it was when the test started.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 3/9] xfs/521,530: refactor scratch fs check

[Christoph Hellwig]

On Tue, Apr 13, 2021 at 06:04:48PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Use the existing _check_scratch_fs helper to check the (modified)
> scratch filesystem in these tests.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 4/9] common/dump: filter out xfs_restore messages about fallocate failures

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 5/9] common/dmthin: make this work with external log devices

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 6/9] misc: replace more open-coded _scratch_xfs_db calls

[Christoph Hellwig]

On Tue, Apr 13, 2021 at 06:05:07PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Replace the last remaining open-coded calls to xfs_db for the scratch
> device with calls to _scratch_xfs_db.  This fixes these tests when
> external logs are enabled.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 7/9] generic/620: fix order of require_scratch calls

[Christoph Hellwig]

On Tue, Apr 13, 2021 at 06:05:14PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> _require_scratch_16T_support does not itself check that the scratch
> device exists, which means that it depends on someone else to call
> _require_scratch.  Document this dependency and fix this test so that we
> can run:
> 
> ./check --exact-order generic/374 generic/620
> 
> on an ext4 filesystem without g/620 tripping over the mess left by g/374
> when it calls _notrun.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 8/9] generic/{094,225}: skip test when the xfs rt extent size is larger than 1 fsb

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 9/9] xfs/305: make sure that fsstress is still running when we quotaoff

[Christoph Hellwig]

On Tue, Apr 13, 2021 at 06:05:26PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Greatly increase the number of fs ops that fsstress is supposed to run
> in in this test so that we can ensure that it's still running when the
> quotaoff gets run.  1000 might have been sufficient in 2013, but it
> isn't now.

How long does this now run?

Re: [PATCH 9/9] xfs/305: make sure that fsstress is still running when we quotaoff

[Darrick J. Wong]

On Wed, Apr 14, 2021 at 07:21:35AM +0100, Christoph Hellwig wrote:
> On Tue, Apr 13, 2021 at 06:05:26PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Greatly increase the number of fs ops that fsstress is supposed to run
> > in in this test so that we can ensure that it's still running when the
> > quotaoff gets run.  1000 might have been sufficient in 2013, but it
> > isn't now.
> 
> How long does this now run?

The same amount of time (~15s per _exercise) as before -- we start
fsstress in the background, wait 10 seconds, run quotaoff, wait 5 more
seconds, and then kill the fsstress process.  Bumping nrops to 1 million
gives fsstress enough work to do that it probably won't exit on its own
before the quotaoff completes.

FWIW the fastest storage I have can run about ~100000 fsstress ops per
minute, so I figured that 1 million ought to do for now.

--D

Re: [PATCH 5/9] common/dmthin: make this work with external log devices

[Eryu Guan]

On Tue, Apr 13, 2021 at 06:05:01PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Provide a mkfs helper to format the dm thin device when external devices
> are in use, and fix the dmthin mount helper to support them.  This fixes
> regressions in generic/347 and generic/500 when external logs are in
> use.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/dmthin     |    9 ++++++++-
>  tests/generic/223 |    3 +++
>  tests/generic/347 |    2 +-
>  tests/generic/500 |    2 +-
>  4 files changed, 13 insertions(+), 3 deletions(-)
> 
> 
> diff --git a/common/dmthin b/common/dmthin
> index c58c3948..3b1c7d45 100644
> --- a/common/dmthin
> +++ b/common/dmthin
> @@ -218,10 +218,17 @@ _dmthin_set_fail()
>  
>  _dmthin_mount_options()
>  {
> -	echo `_common_dev_mount_options $*` $DMTHIN_VOL_DEV $SCRATCH_MNT
> +	_scratch_options mount
> +	echo `_common_dev_mount_options $*` $SCRATCH_OPTIONS $DMTHIN_VOL_DEV $SCRATCH_MNT
>  }
>  
>  _dmthin_mount()
>  {
>  	_mount -t $FSTYP `_dmthin_mount_options $*`
>  }
> +
> +_dmthin_mkfs()
> +{
> +	_scratch_options mkfs
> +	_mkfs_dev $SCRATCH_OPTIONS $@ $DMTHIN_VOL_DEV
> +}
> diff --git a/tests/generic/223 b/tests/generic/223
> index 1f85efe5..a5ace82f 100755
> --- a/tests/generic/223
> +++ b/tests/generic/223
> @@ -43,6 +43,9 @@ for SUNIT_K in 8 16 32 64 128; do
>  	_scratch_mkfs_geom $SUNIT_BYTES 4 $BLOCKSIZE >> $seqres.full 2>&1
>  	_scratch_mount
>  
> +	# Make sure everything is on the data device
> +	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT

What does this do for non-xfs filesystems? Do we need a FSTYP check and
do chattr only on XFS?

Thanks,
Eryu

> +
>  	for SIZE_MULT in 1 2 8 64 256; do
>  		let SIZE=$SIZE_MULT*$SUNIT_BYTES
>  
> diff --git a/tests/generic/347 b/tests/generic/347
> index cbc5150a..e970ac10 100755
> --- a/tests/generic/347
> +++ b/tests/generic/347
> @@ -31,7 +31,7 @@ _setup_thin()
>  {
>  	_dmthin_init $BACKING_SIZE $VIRTUAL_SIZE
>  	_dmthin_set_queue
> -	_mkfs_dev $DMTHIN_VOL_DEV
> +	_dmthin_mkfs
>  	_dmthin_mount
>  }
>  
> diff --git a/tests/generic/500 b/tests/generic/500
> index 085ddbf3..5ab2f78c 100755
> --- a/tests/generic/500
> +++ b/tests/generic/500
> @@ -68,7 +68,7 @@ CLUSTER_SIZE=$((64 * 1024 / 512))		# 64K
>  
>  _dmthin_init $BACKING_SIZE $VIRTUAL_SIZE $CLUSTER_SIZE 0
>  _dmthin_set_fail
> -_mkfs_dev $DMTHIN_VOL_DEV
> +_dmthin_mkfs
>  _dmthin_mount
>  
>  # There're two bugs at here, one is dm-thin bug, the other is filesystem

Re: [PATCHSET 0/9] fstests: random fixes

[Eryu Guan]

On Tue, Apr 13, 2021 at 06:04:30PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> This branch contains fixes to various tests to fix miscellaneous test
> bugs and unnecessary regressions when XFS is configured with somewhat
> unusual configurations (e.g. always-cow mode, external logs, and/or
> realtime devices).

Thanks for all the fixes! I've applied all patches except the 5th one.

Thanks,
Eryu

Re: [PATCH 5/9] common/dmthin: make this work with external log devices

[Theodore Ts'o]

On Sun, Apr 18, 2021 at 08:25:48PM +0800, Eryu Guan wrote:
> > diff --git a/tests/generic/223 b/tests/generic/223
> > index 1f85efe5..a5ace82f 100755
> > --- a/tests/generic/223
> > +++ b/tests/generic/223
> > @@ -43,6 +43,9 @@ for SUNIT_K in 8 16 32 64 128; do
> >  	_scratch_mkfs_geom $SUNIT_BYTES 4 $BLOCKSIZE >> $seqres.full 2>&1
> >  	_scratch_mount
> >  
> > +	# Make sure everything is on the data device
> > +	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
> 
> What does this do for non-xfs filesystems? Do we need a FSTYP check and
> do chattr only on XFS?

This clears the FS_NOTAIL_FL flag, which prevents tail merging, on the
root directory of the mounted scratch file system.  That should be
harmless on non-xfs file systems; in fact, the only file system that
even uses NOTAIL_FL flag is reiserfs, and the NOTAIL_FL flag is not
set by default on the root directory of a newly created reiserfs file
system.

However, by default reiserfs does not support the
FS_IOC_{GET,SET}FLAGS ioctl unless the mount option "attrs" is given.
Why, I have no idea:

root@kvm-xfstests:~# mount -t reiserfs /vtmp/foo.img /mnt
root@kvm-xfstests:~# xfs_io -c 'chattr -t' /mnt
xfs_io: cannot get flags on /mnt: Inappropriate ioctl for device

So it might be a good idea to redirect stderr for the xfs_io
invocation to /dev/null, for those file systems which do not support
the FS_IOC_{GET,SET}FLAGS ioctls.

I also have no idea why this helps for xfs --- I would think it's a
no-op, but I'm guessing there's some magical side-effect which is
taking place when FS_IOC_SETFLAGS ioctl is processed?  Maybe it would
be worth a comment explaining what is going on --- and whether this is
going to make any difference if the patch series which unifies
FS_IOC_{GETSET}FLAGS handling is merged?

						- Ted

Re: [PATCH 5/9] common/dmthin: make this work with external log devices

[Dave Chinner]

On Sun, Apr 18, 2021 at 10:44:08AM -0400, Theodore Ts'o wrote:
> On Sun, Apr 18, 2021 at 08:25:48PM +0800, Eryu Guan wrote:
> > > diff --git a/tests/generic/223 b/tests/generic/223
> > > index 1f85efe5..a5ace82f 100755
> > > --- a/tests/generic/223
> > > +++ b/tests/generic/223
> > > @@ -43,6 +43,9 @@ for SUNIT_K in 8 16 32 64 128; do
> > >  	_scratch_mkfs_geom $SUNIT_BYTES 4 $BLOCKSIZE >> $seqres.full 2>&1
> > >  	_scratch_mount
> > >  
> > > +	# Make sure everything is on the data device
> > > +	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
> > 
> > What does this do for non-xfs filesystems? Do we need a FSTYP check and
> > do chattr only on XFS?
> 
> This clears the FS_NOTAIL_FL flag, which prevents tail merging, on the

No, this is not the 'chattr' CLI program. This is the xfs_io
'chattr' command, and they have different attribute namespaces. See
xfs_io for the definitions, but in this case:

	 t    inherit realtime flag (XFS_XFLAG_RTINHERIT)

And so clearing that flag ensures that all newly created files
are on the data device, as per the comment...

> I also have no idea why this helps for xfs --- I would think it's a
> no-op,

Because it's not what you think it is. :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 5/9] common/dmthin: make this work with external log devices

[Darrick J. Wong]

On Mon, Apr 19, 2021 at 12:04:42PM +1000, Dave Chinner wrote:
> On Sun, Apr 18, 2021 at 10:44:08AM -0400, Theodore Ts'o wrote:
> > On Sun, Apr 18, 2021 at 08:25:48PM +0800, Eryu Guan wrote:
> > > > diff --git a/tests/generic/223 b/tests/generic/223
> > > > index 1f85efe5..a5ace82f 100755
> > > > --- a/tests/generic/223
> > > > +++ b/tests/generic/223
> > > > @@ -43,6 +43,9 @@ for SUNIT_K in 8 16 32 64 128; do
> > > >  	_scratch_mkfs_geom $SUNIT_BYTES 4 $BLOCKSIZE >> $seqres.full 2>&1
> > > >  	_scratch_mount
> > > >  
> > > > +	# Make sure everything is on the data device
> > > > +	$XFS_IO_PROG -c 'chattr -t' $SCRATCH_MNT
> > > 
> > > What does this do for non-xfs filesystems? Do we need a FSTYP check and
> > > do chattr only on XFS?

Yes, I think so.  Will fix.

> > This clears the FS_NOTAIL_FL flag, which prevents tail merging, on the
> 
> No, this is not the 'chattr' CLI program. This is the xfs_io
> 'chattr' command, and they have different attribute namespaces. See
> xfs_io for the definitions, but in this case:
> 
> 	 t    inherit realtime flag (XFS_XFLAG_RTINHERIT)
> 
> And so clearing that flag ensures that all newly created files
> are on the data device, as per the comment...
> 
> > I also have no idea why this helps for xfs --- I would think it's a
> > no-op,
> 
> Because it's not what you think it is. :)

(Indeed.)

--D

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH 9/9] xfs/305: make sure that fsstress is still running when we quotaoff

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCHSET 0/9] fstests: random fixes

[Darrick J. Wong]

On Tue, Jul 05, 2022 at 10:28:40PM +0800, Zorro Lang wrote:
> On Tue, Jun 28, 2022 at 01:21:17PM -0700, Darrick J. Wong wrote:
> > Hi all,
> > 
> > Here's the usual batch of odd fixes for fstests.
> > 
> > If you're going to start using this mess, you probably ought to just
> > pull from my git trees, which are linked below.
> > 
> > This is an extraordinary way to destroy everything.  Enjoy!
> > Comments and questions are, as always, welcome.
> > 
> > --D
> 
> Hi Darrick,
> 
> JFYI, to push the proceeding of your patch merging, I've merged/pushed this
> patchset, except patch 7/9 and 9/9 due to they haven't gotten any review.
> I can review them, but patch 7/9 changes the case which Dave might care about,
> so I'd like to wait more response.
> 
> Feel free to send these 2 patches with more your new patches, if you'd like to
> do that :)

Done.  Thanks for taking last week's bundle, btw.

--D

> Thanks,
> Zorro
> 
> > 
> > kernel git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes
> > 
> > xfsprogs git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes
> > 
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
> > ---
> >  check                  |    3 +++
> >  common/repair          |    1 +
> >  src/seek_sanity_test.c |   12 ++++++++++-
> >  tests/xfs/018          |   52 +++++++++++++++++++++++++++++++++++++++++++-----
> >  tests/xfs/018.out      |   16 ++++-----------
> >  tests/xfs/109          |    2 +-
> >  tests/xfs/166          |   19 ++++++++++++++----
> >  tests/xfs/547          |   14 +++++++++----
> >  tests/xfs/843          |   51 +++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/843.out      |    2 ++
> >  tests/xfs/844          |   33 ++++++++++++++++++++++++++++++
> >  tests/xfs/844.out      |    3 +++
> >  12 files changed, 181 insertions(+), 27 deletions(-)
> >  create mode 100755 tests/xfs/843
> >  create mode 100644 tests/xfs/843.out
> >  create mode 100755 tests/xfs/844
> >  create mode 100644 tests/xfs/844.out
> > 
> 

Re: [PATCHSET 0/9] fstests: random fixes

[Zorro Lang]

On Tue, Jun 28, 2022 at 01:21:17PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> Here's the usual batch of odd fixes for fstests.
> 
> If you're going to start using this mess, you probably ought to just
> pull from my git trees, which are linked below.
> 
> This is an extraordinary way to destroy everything.  Enjoy!
> Comments and questions are, as always, welcome.
> 
> --D

Hi Darrick,

JFYI, to push the proceeding of your patch merging, I've merged/pushed this
patchset, except patch 7/9 and 9/9 due to they haven't gotten any review.
I can review them, but patch 7/9 changes the case which Dave might care about,
so I'd like to wait more response.

Feel free to send these 2 patches with more your new patches, if you'd like to
do that :)

Thanks,
Zorro

> 
> kernel git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes
> 
> xfsprogs git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes
> 
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
> ---
>  check                  |    3 +++
>  common/repair          |    1 +
>  src/seek_sanity_test.c |   12 ++++++++++-
>  tests/xfs/018          |   52 +++++++++++++++++++++++++++++++++++++++++++-----
>  tests/xfs/018.out      |   16 ++++-----------
>  tests/xfs/109          |    2 +-
>  tests/xfs/166          |   19 ++++++++++++++----
>  tests/xfs/547          |   14 +++++++++----
>  tests/xfs/843          |   51 +++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/843.out      |    2 ++
>  tests/xfs/844          |   33 ++++++++++++++++++++++++++++++
>  tests/xfs/844.out      |    3 +++
>  12 files changed, 181 insertions(+), 27 deletions(-)
>  create mode 100755 tests/xfs/843
>  create mode 100644 tests/xfs/843.out
>  create mode 100755 tests/xfs/844
>  create mode 100644 tests/xfs/844.out
> 

[PATCHSET 0/9] fstests: random fixes

[Darrick J. Wong]

Hi all,

Here's the usual batch of odd fixes for fstests.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
 check                  |    3 +++
 common/repair          |    1 +
 src/seek_sanity_test.c |   12 ++++++++++-
 tests/xfs/018          |   52 +++++++++++++++++++++++++++++++++++++++++++-----
 tests/xfs/018.out      |   16 ++++-----------
 tests/xfs/109          |    2 +-
 tests/xfs/166          |   19 ++++++++++++++----
 tests/xfs/547          |   14 +++++++++----
 tests/xfs/843          |   51 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/843.out      |    2 ++
 tests/xfs/844          |   33 ++++++++++++++++++++++++++++++
 tests/xfs/844.out      |    3 +++
 12 files changed, 181 insertions(+), 27 deletions(-)
 create mode 100755 tests/xfs/843
 create mode 100644 tests/xfs/843.out
 create mode 100755 tests/xfs/844
 create mode 100644 tests/xfs/844.out