[PATCHSET 0/4] fstests: various fixes

[PATCHSET 0/4] fstests: various fixes

[Darrick J. Wong]

Hi all,

Here's a few fixes for some false negatives on reflink filesystems, and
fixes to make sure the dax tests work properly even when the filesystem
is formatted with the dax iflag set everywhere.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
 common/rc         |    3 ++-
 tests/generic/607 |    4 ++++
 tests/generic/608 |    3 +++
 tests/generic/623 |    1 +
 tests/xfs/271     |   12 +++++++++++-
 5 files changed, 21 insertions(+), 2 deletions(-)

[PATCH 1/4] generic/623: don't fail on core dumps

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test is designed to fail an mmap write and see what happens.
Typically this is a segmentation fault.  If the user's computer is
configured to capture core dumps, this will cause the test to fail, even
though we got the reaction we wanted.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/623 |    1 +
 1 file changed, 1 insertion(+)


diff --git a/tests/generic/623 b/tests/generic/623
index 7be38955..04411405 100755
--- a/tests/generic/623
+++ b/tests/generic/623
@@ -37,6 +37,7 @@ _scratch_mount
 # status on the page.
 file=$SCRATCH_MNT/file
 $XFS_IO_PROG -fc "pwrite 0 4k" -c fsync $file | _filter_xfs_io
+ulimit -c 0
 $XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \
 	-c "mwrite 0 4k" $file | _filter_xfs_io
 

Re: [PATCH 1/4] generic/623: don't fail on core dumps

[Chandan Babu R]

On 03 Mar 2021 at 04:52, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> This test is designed to fail an mmap write and see what happens.
> Typically this is a segmentation fault.  If the user's computer is
> configured to capture core dumps, this will cause the test to fail, even
> though we got the reaction we wanted.
>

Looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/generic/623 |    1 +
>  1 file changed, 1 insertion(+)
>
>
> diff --git a/tests/generic/623 b/tests/generic/623
> index 7be38955..04411405 100755
> --- a/tests/generic/623
> +++ b/tests/generic/623
> @@ -37,6 +37,7 @@ _scratch_mount
>  # status on the page.
>  file=$SCRATCH_MNT/file
>  $XFS_IO_PROG -fc "pwrite 0 4k" -c fsync $file | _filter_xfs_io
> +ulimit -c 0
>  $XFS_IO_PROG -x -c "mmap 0 4k" -c "mwrite 0 4k" -c shutdown -c fsync \
>  	-c "mwrite 0 4k" $file | _filter_xfs_io
>  


-- 
chandan

Re: [PATCH 1/4] generic/623: don't fail on core dumps

[Christoph Hellwig]

On Tue, Mar 02, 2021 at 03:22:39PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This test is designed to fail an mmap write and see what happens.
> Typically this is a segmentation fault.  If the user's computer is
> configured to capture core dumps, this will cause the test to fail, even
> though we got the reaction we wanted.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 2/4] xfs/271: fix test failure on non-reflink filesystems

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test creates an empty filesystem with rmap btrees enabled, and then
checks that GETFSMAP corresponds (roughly) with what we expect mkfs to
have written to the filesystem.

Unfortunately, the test's calculation for the number of "per-AG
metadata" extents is not quite correct.  For a filesystem with a
refcount btree, the rmapbt and agfl blocks will be reported separately,
but for non-reflink filesystems, GETFSMAP merges the records.

Since this test counts the number of records, fix the calculation.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/271 |   12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/271 b/tests/xfs/271
index 48a3eb8f..35c23b84 100755
--- a/tests/xfs/271
+++ b/tests/xfs/271
@@ -38,6 +38,16 @@ _scratch_mount
 
 agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
 
+# mkfs lays out btree root blocks in the order bnobt, cntbt, inobt, finobt,
+# rmapbt, refcountbt, and then allocates AGFL blocks.  Since GETFSMAP has the
+# same owner (per-AG metadata) for rmap btree blocks and blocks on the AGFL and
+# the reverse mapping index merges records, the number of per-AG extents
+# reported will vary depending on whether the refcount btree is enabled.
+$XFS_INFO_PROG $SCRATCH_MNT | grep -q reflink=1
+has_reflink=$(( 1 - $? ))
+perag_metadata_exts=2
+test $has_reflink -gt 0 && perag_metadata_exts=$((perag_metadata_exts + 1))
+
 echo "Get fsmap" | tee -a $seqres.full
 $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT > $TEST_DIR/fsmap
 cat $TEST_DIR/fsmap >> $seqres.full
@@ -48,7 +58,7 @@ _within_tolerance "AG header count" $(wc -l < $TEST_DIR/testout) $agcount 0 -v
 
 echo "Check freesp/rmap btrees" | tee -a $seqres.full
 grep 'per-AG metadata[[:space:]]*[0-9]*[[:space:]]*([0-9]*\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout
-_within_tolerance "freesp extent count" $(wc -l < $TEST_DIR/testout) $((agcount * 3)) 0 999999 -v
+_within_tolerance "freesp extent count" $(wc -l < $TEST_DIR/testout) $((agcount * perag_metadata_exts)) 0 999999 -v
 
 echo "Check inode btrees" | tee -a $seqres.full
 grep 'inode btree[[:space:]]*[0-9]*[[:space:]]*([0-9]*\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout

Re: [PATCH 2/4] xfs/271: fix test failure on non-reflink filesystems

[Chandan Babu R]

On 03 Mar 2021 at 04:52, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> This test creates an empty filesystem with rmap btrees enabled, and then
> checks that GETFSMAP corresponds (roughly) with what we expect mkfs to
> have written to the filesystem.
>
> Unfortunately, the test's calculation for the number of "per-AG
> metadata" extents is not quite correct.  For a filesystem with a
> refcount btree, the rmapbt and agfl blocks will be reported separately,
> but for non-reflink filesystems, GETFSMAP merges the records.
>
> Since this test counts the number of records, fix the calculation.
>

Looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>


> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/271 |   12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
>
>
> diff --git a/tests/xfs/271 b/tests/xfs/271
> index 48a3eb8f..35c23b84 100755
> --- a/tests/xfs/271
> +++ b/tests/xfs/271
> @@ -38,6 +38,16 @@ _scratch_mount
>  
>  agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
>  
> +# mkfs lays out btree root blocks in the order bnobt, cntbt, inobt, finobt,
> +# rmapbt, refcountbt, and then allocates AGFL blocks.  Since GETFSMAP has the
> +# same owner (per-AG metadata) for rmap btree blocks and blocks on the AGFL and
> +# the reverse mapping index merges records, the number of per-AG extents
> +# reported will vary depending on whether the refcount btree is enabled.
> +$XFS_INFO_PROG $SCRATCH_MNT | grep -q reflink=1
> +has_reflink=$(( 1 - $? ))
> +perag_metadata_exts=2
> +test $has_reflink -gt 0 && perag_metadata_exts=$((perag_metadata_exts + 1))
> +
>  echo "Get fsmap" | tee -a $seqres.full
>  $XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT > $TEST_DIR/fsmap
>  cat $TEST_DIR/fsmap >> $seqres.full
> @@ -48,7 +58,7 @@ _within_tolerance "AG header count" $(wc -l < $TEST_DIR/testout) $agcount 0 -v
>  
>  echo "Check freesp/rmap btrees" | tee -a $seqres.full
>  grep 'per-AG metadata[[:space:]]*[0-9]*[[:space:]]*([0-9]*\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout
> -_within_tolerance "freesp extent count" $(wc -l < $TEST_DIR/testout) $((agcount * 3)) 0 999999 -v
> +_within_tolerance "freesp extent count" $(wc -l < $TEST_DIR/testout) $((agcount * perag_metadata_exts)) 0 999999 -v
>  
>  echo "Check inode btrees" | tee -a $seqres.full
>  grep 'inode btree[[:space:]]*[0-9]*[[:space:]]*([0-9]*\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout


-- 
chandan

Re: [PATCH 2/4] xfs/271: fix test failure on non-reflink filesystems

[Christoph Hellwig]

On Tue, Mar 02, 2021 at 03:22:45PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This test creates an empty filesystem with rmap btrees enabled, and then
> checks that GETFSMAP corresponds (roughly) with what we expect mkfs to
> have written to the filesystem.
> 
> Unfortunately, the test's calculation for the number of "per-AG
> metadata" extents is not quite correct.  For a filesystem with a
> refcount btree, the rmapbt and agfl blocks will be reported separately,
> but for non-reflink filesystems, GETFSMAP merges the records.
> 
> Since this test counts the number of records, fix the calculation.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 3/4] common/rc: fix detection of device-mapper/persistent memory incompatibility

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In commit fc7b3903, we tried to make _require_dm_target smart enough to
_notrun tests that require a device mapper target that isn't compatible
with "DAX".  However, as of this writing, the incompatibility stems from
device mapper's unwillingness to switch access modes when running atop
DAX (persistent memory) devices, and has nothing to do with the
filesystem mount options.

Since filesystems supporting DAX don't universally require "dax" in the
mount options to enable that functionality, switch the test to query
sysfs to see if the scratch device supports DAX.

Fixes: fc7b3903 ("dax/dm: disable testing on devices that don't support dax")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


diff --git a/common/rc b/common/rc
index 7254130f..10e5f40b 100644
--- a/common/rc
+++ b/common/rc
@@ -1889,7 +1889,8 @@ _require_dm_target()
 	_require_sane_bdev_flush $SCRATCH_DEV
 	_require_command "$DMSETUP_PROG" dmsetup
 
-	_normalize_mount_options | egrep -q "dax(=always| |$)"
+	_normalize_mount_options | egrep -q "dax(=always| |$)" || \
+			test -e "/sys/block/$(_short_dev $SCRATCH_DEV)/dax"
 	if [ $? -eq 0 ]; then
 		case $target in
 		stripe|linear|log-writes)

Re: [PATCH 3/4] common/rc: fix detection of device-mapper/persistent memory incompatibility

[Chandan Babu R]

On 03 Mar 2021 at 04:52, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> In commit fc7b3903, we tried to make _require_dm_target smart enough to
> _notrun tests that require a device mapper target that isn't compatible
> with "DAX".  However, as of this writing, the incompatibility stems from
> device mapper's unwillingness to switch access modes when running atop
> DAX (persistent memory) devices, and has nothing to do with the
> filesystem mount options.
>
> Since filesystems supporting DAX don't universally require "dax" in the
> mount options to enable that functionality, switch the test to query
> sysfs to see if the scratch device supports DAX.

Looks good to me.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

>
> Fixes: fc7b3903 ("dax/dm: disable testing on devices that don't support dax")
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/rc |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
>
> diff --git a/common/rc b/common/rc
> index 7254130f..10e5f40b 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1889,7 +1889,8 @@ _require_dm_target()
>  	_require_sane_bdev_flush $SCRATCH_DEV
>  	_require_command "$DMSETUP_PROG" dmsetup
>  
> -	_normalize_mount_options | egrep -q "dax(=always| |$)"
> +	_normalize_mount_options | egrep -q "dax(=always| |$)" || \
> +			test -e "/sys/block/$(_short_dev $SCRATCH_DEV)/dax"
>  	if [ $? -eq 0 ]; then
>  		case $target in
>  		stripe|linear|log-writes)


-- 
chandan

Re: [PATCH 3/4] common/rc: fix detection of device-mapper/persistent memory incompatibility

[Christoph Hellwig]

On Tue, Mar 02, 2021 at 03:22:50PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> In commit fc7b3903, we tried to make _require_dm_target smart enough to
> _notrun tests that require a device mapper target that isn't compatible
> with "DAX".  However, as of this writing, the incompatibility stems from
> device mapper's unwillingness to switch access modes when running atop
> DAX (persistent memory) devices, and has nothing to do with the
> filesystem mount options.
> 
> Since filesystems supporting DAX don't universally require "dax" in the
> mount options to enable that functionality, switch the test to query
> sysfs to see if the scratch device supports DAX.
> 
> Fixes: fc7b3903 ("dax/dm: disable testing on devices that don't support dax")
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 4/4] generic/60[78]: ensure the initial DAX file flag state before test

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Since this test checks the behaviors of both the in-core S_DAX flag and
the ondisk FS_XFLAG_DAX inode flags, it must be careful about the
initial state of the filesystem w.r.t. the inode flag.

Make sure that the root directory does /not/ have the inode flag set
before we begin testing, so that the initial state doesn't throw off
inheritance testing.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/generic/607 |    4 ++++
 tests/generic/608 |    3 +++
 2 files changed, 7 insertions(+)


diff --git a/tests/generic/607 b/tests/generic/607
index dd6dbd65..ba7da11b 100755
--- a/tests/generic/607
+++ b/tests/generic/607
@@ -156,6 +156,10 @@ do_xflag_tests()
 	local option=$1
 
 	_scratch_mount "$option"
+
+	# Make sure the root dir doesn't have FS_XFLAG_DAX set before we start.
+	chattr -x $SCRATCH_MNT &>> $seqres.full
+
 	cd $SCRATCH_MNT
 
 	for i in $(seq 1 5); do
diff --git a/tests/generic/608 b/tests/generic/608
index dd89d91c..13a751d7 100755
--- a/tests/generic/608
+++ b/tests/generic/608
@@ -98,6 +98,9 @@ do_tests()
 
 	_scratch_mount "$mount_option"
 
+	# Make sure the root dir doesn't have FS_XFLAG_DAX set before we start.
+	chattr -x $SCRATCH_MNT &>> $seqres.full
+
 	test_drop_caches
 
 	test_cycle_mount "$cycle_mount_option"

Re: [PATCH 4/4] generic/60[78]: ensure the initial DAX file flag state before test

[Chandan Babu R]

On 03 Mar 2021 at 04:52, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Since this test checks the behaviors of both the in-core S_DAX flag and
> the ondisk FS_XFLAG_DAX inode flags, it must be careful about the
> initial state of the filesystem w.r.t. the inode flag.
>
> Make sure that the root directory does /not/ have the inode flag set
> before we begin testing, so that the initial state doesn't throw off
> inheritance testing.

If $MKFS_OPTIONS has daxinherit=1 option then this will cause all the inodes
created in the new filesystem to have the corresponding flag set. Hence the
patch is correct.

Reviewed-by: Chandan Babu R <chandanrlinux@gmail.com>

>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/generic/607 |    4 ++++
>  tests/generic/608 |    3 +++
>  2 files changed, 7 insertions(+)
>
>
> diff --git a/tests/generic/607 b/tests/generic/607
> index dd6dbd65..ba7da11b 100755
> --- a/tests/generic/607
> +++ b/tests/generic/607
> @@ -156,6 +156,10 @@ do_xflag_tests()
>  	local option=$1
>
>  	_scratch_mount "$option"
> +
> +	# Make sure the root dir doesn't have FS_XFLAG_DAX set before we start.
> +	chattr -x $SCRATCH_MNT &>> $seqres.full
> +
>  	cd $SCRATCH_MNT
>
>  	for i in $(seq 1 5); do
> diff --git a/tests/generic/608 b/tests/generic/608
> index dd89d91c..13a751d7 100755
> --- a/tests/generic/608
> +++ b/tests/generic/608
> @@ -98,6 +98,9 @@ do_tests()
>
>  	_scratch_mount "$mount_option"
>
> +	# Make sure the root dir doesn't have FS_XFLAG_DAX set before we start.
> +	chattr -x $SCRATCH_MNT &>> $seqres.full
> +
>  	test_drop_caches
>
>  	test_cycle_mount "$cycle_mount_option"


--
chandan