[PATCHSET 0/3] fstests: fix dax+reflink tests

[PATCHSET 0/3] fstests: fix dax+reflink tests

[Darrick J. Wong]

Hi all,

This patchset adjusts a few tests to handle fsdax and reflink being
enabled at the same time.  This is new functionality for 6.2.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fix-dax-reflink
---
 tests/xfs/128     |    2 +-
 tests/xfs/182     |    4 ++--
 tests/xfs/182.out |    1 -
 tests/xfs/184     |    1 +
 tests/xfs/192     |    1 +
 tests/xfs/200     |    1 +
 tests/xfs/204     |    1 +
 tests/xfs/232     |    1 +
 tests/xfs/440     |    1 +
 9 files changed, 9 insertions(+), 4 deletions(-)

[PATCH 1/3] xfs: fix dax inode flag test failures

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Filter out the DAX inode flag because it's causing problems with this
test.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/128 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/xfs/128 b/tests/xfs/128
index 5591342d41..8c1663c6c5 100755
--- a/tests/xfs/128
+++ b/tests/xfs/128
@@ -81,7 +81,7 @@ c13=$(_md5_checksum $testdir/file3)
 c14=$(_md5_checksum $testdir/file4)
 
 echo "Defragment"
-lsattr -l $testdir/ | _filter_scratch | _filter_spaces
+lsattr -l $testdir/ | _filter_scratch | _filter_spaces | sed -e 's/DAX/---/g'
 $XFS_FSR_PROG -v -d $testdir/file1 >> $seqres.full
 $XFS_FSR_PROG -v -d $testdir/file2 >> $seqres.full # fsr probably breaks the link
 $XFS_FSR_PROG -v -d $testdir/file3 >> $seqres.full # fsr probably breaks the link

Re: [PATCH 1/3] xfs: fix dax inode flag test failures

[Xiao Yang]

Hi Darrick,

It looks good to me.
Reviewed-by: Xiao Yang <yangx.jy@fujitsu.com>

Best Regards,
Xiao Yang

On 2023/1/18 8:42, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Filter out the DAX inode flag because it's causing problems with this
> test.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>   tests/xfs/128 |    2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> 
> diff --git a/tests/xfs/128 b/tests/xfs/128
> index 5591342d41..8c1663c6c5 100755
> --- a/tests/xfs/128
> +++ b/tests/xfs/128
> @@ -81,7 +81,7 @@ c13=$(_md5_checksum $testdir/file3)
>   c14=$(_md5_checksum $testdir/file4)
>   
>   echo "Defragment"
> -lsattr -l $testdir/ | _filter_scratch | _filter_spaces
> +lsattr -l $testdir/ | _filter_scratch | _filter_spaces | sed -e 's/DAX/---/g'
>   $XFS_FSR_PROG -v -d $testdir/file1 >> $seqres.full
>   $XFS_FSR_PROG -v -d $testdir/file2 >> $seqres.full # fsr probably breaks the link
>   $XFS_FSR_PROG -v -d $testdir/file3 >> $seqres.full # fsr probably breaks the link
> 

[PATCH 2/3] xfs: fix reflink test failures when dax is enabled

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Turn off reflink tests that require delayed allocation to work, because
we don't use delayed allocation when fsdax mode is turned on.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/184 |    1 +
 tests/xfs/192 |    1 +
 tests/xfs/200 |    1 +
 tests/xfs/204 |    1 +
 tests/xfs/232 |    1 +
 tests/xfs/440 |    1 +
 6 files changed, 6 insertions(+)


diff --git a/tests/xfs/184 b/tests/xfs/184
index c251040e8a..3bdd86addf 100755
--- a/tests/xfs/184
+++ b/tests/xfs/184
@@ -19,6 +19,7 @@ _begin_fstest auto quick clone fiemap unshare
 
 # real QA test starts here
 _supported_fs xfs
+_require_scratch_delalloc
 _require_scratch_reflink
 _require_cp_reflink
 _require_xfs_io_command "fiemap"
diff --git a/tests/xfs/192 b/tests/xfs/192
index 85ed7a48fc..ced18fa3c1 100755
--- a/tests/xfs/192
+++ b/tests/xfs/192
@@ -19,6 +19,7 @@ _begin_fstest auto quick clone fiemap unshare
 
 # real QA test starts here
 _supported_fs xfs
+_require_scratch_delalloc
 _require_scratch_reflink
 _require_cp_reflink
 _require_xfs_io_command "fiemap"
diff --git a/tests/xfs/200 b/tests/xfs/200
index f91bfbf478..b51b9a54f5 100755
--- a/tests/xfs/200
+++ b/tests/xfs/200
@@ -21,6 +21,7 @@ _begin_fstest auto quick clone fiemap unshare
 
 # real QA test starts here
 _supported_fs xfs
+_require_scratch_delalloc
 _require_scratch_reflink
 _require_cp_reflink
 _require_xfs_io_command "fiemap"
diff --git a/tests/xfs/204 b/tests/xfs/204
index d034446bbc..ca21dfe722 100755
--- a/tests/xfs/204
+++ b/tests/xfs/204
@@ -21,6 +21,7 @@ _begin_fstest auto quick clone fiemap unshare
 
 # real QA test starts here
 _supported_fs xfs
+_require_scratch_delalloc
 _require_scratch_reflink
 _require_cp_reflink
 _require_xfs_io_command "fiemap"
diff --git a/tests/xfs/232 b/tests/xfs/232
index f402ad6cf3..59bbc43686 100755
--- a/tests/xfs/232
+++ b/tests/xfs/232
@@ -30,6 +30,7 @@ _cleanup()
 
 # real QA test starts here
 _supported_fs xfs
+_require_scratch_delalloc
 _require_xfs_io_command "cowextsize"
 _require_scratch_reflink
 _require_cp_reflink
diff --git a/tests/xfs/440 b/tests/xfs/440
index 496ee04edf..368ee8a05d 100755
--- a/tests/xfs/440
+++ b/tests/xfs/440
@@ -20,6 +20,7 @@ _begin_fstest auto quick clone quota
 _supported_fs xfs
 
 _require_quota
+_require_scratch_delalloc
 _require_scratch_reflink
 _require_cp_reflink
 _require_user

Re: [PATCH 2/3] xfs: fix reflink test failures when dax is enabled

[Xiao Yang]

Hi Darrick,

Thanks for your quick fix. It's better than my patch.
Reviewed-by: Xiao Yang <yangx.jy@fujitsu.com>

Best Regards,
Xiao Yang

On 2023/1/18 8:42, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Turn off reflink tests that require delayed allocation to work, because
> we don't use delayed allocation when fsdax mode is turned on.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>   tests/xfs/184 |    1 +
>   tests/xfs/192 |    1 +
>   tests/xfs/200 |    1 +
>   tests/xfs/204 |    1 +
>   tests/xfs/232 |    1 +
>   tests/xfs/440 |    1 +
>   6 files changed, 6 insertions(+)
> 
> 
> diff --git a/tests/xfs/184 b/tests/xfs/184
> index c251040e8a..3bdd86addf 100755
> --- a/tests/xfs/184
> +++ b/tests/xfs/184
> @@ -19,6 +19,7 @@ _begin_fstest auto quick clone fiemap unshare
>   
>   # real QA test starts here
>   _supported_fs xfs
> +_require_scratch_delalloc
>   _require_scratch_reflink
>   _require_cp_reflink
>   _require_xfs_io_command "fiemap"
> diff --git a/tests/xfs/192 b/tests/xfs/192
> index 85ed7a48fc..ced18fa3c1 100755
> --- a/tests/xfs/192
> +++ b/tests/xfs/192
> @@ -19,6 +19,7 @@ _begin_fstest auto quick clone fiemap unshare
>   
>   # real QA test starts here
>   _supported_fs xfs
> +_require_scratch_delalloc
>   _require_scratch_reflink
>   _require_cp_reflink
>   _require_xfs_io_command "fiemap"
> diff --git a/tests/xfs/200 b/tests/xfs/200
> index f91bfbf478..b51b9a54f5 100755
> --- a/tests/xfs/200
> +++ b/tests/xfs/200
> @@ -21,6 +21,7 @@ _begin_fstest auto quick clone fiemap unshare
>   
>   # real QA test starts here
>   _supported_fs xfs
> +_require_scratch_delalloc
>   _require_scratch_reflink
>   _require_cp_reflink
>   _require_xfs_io_command "fiemap"
> diff --git a/tests/xfs/204 b/tests/xfs/204
> index d034446bbc..ca21dfe722 100755
> --- a/tests/xfs/204
> +++ b/tests/xfs/204
> @@ -21,6 +21,7 @@ _begin_fstest auto quick clone fiemap unshare
>   
>   # real QA test starts here
>   _supported_fs xfs
> +_require_scratch_delalloc
>   _require_scratch_reflink
>   _require_cp_reflink
>   _require_xfs_io_command "fiemap"
> diff --git a/tests/xfs/232 b/tests/xfs/232
> index f402ad6cf3..59bbc43686 100755
> --- a/tests/xfs/232
> +++ b/tests/xfs/232
> @@ -30,6 +30,7 @@ _cleanup()
>   
>   # real QA test starts here
>   _supported_fs xfs
> +_require_scratch_delalloc
>   _require_xfs_io_command "cowextsize"
>   _require_scratch_reflink
>   _require_cp_reflink
> diff --git a/tests/xfs/440 b/tests/xfs/440
> index 496ee04edf..368ee8a05d 100755
> --- a/tests/xfs/440
> +++ b/tests/xfs/440
> @@ -20,6 +20,7 @@ _begin_fstest auto quick clone quota
>   _supported_fs xfs
>   
>   _require_quota
> +_require_scratch_delalloc
>   _require_scratch_reflink
>   _require_cp_reflink
>   _require_user
> 

[PATCH 3/3] xfs/182: fix spurious direct write failure

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This test has some weird behavior that causes regressions when fsdax and
reflink are enabled.  The goal of this test is to set a cow extent size
hint, perform some random directio writes, perform a directio rewrite of
the entire file, and make sure that the file content (and extent count)
are sane afterwards.

Most of the time, the random directio writes will never touch the
8388609th byte, though if they do randomly select that EOF block, they'd
end up extending the file by $real_blksz bytes and causing spurious test
failures.

Then, the rewrite does this:

pwrite -S 0x63 -b $real_blksz 0 $((filesize + 1))

Note that we previously set filesize=8388608, which means that we're
asking for a series of direct writes that fill the first 8388608 bytes
with 'c'.  The last write in the series becomes a single byte direct
write.  For regular file access mode, this last write will fail with
EINVAL, since block devices do not support byte granularity writes and
XFS does not fall back to the pagecache for unaligned direct wites.
Hence we never wrote the 8388609th byte of the file.

However, fsdax *does* allow byte-granularity direct writes, which means
that the single-byte write succeeds.  There is no EINVAL return code,
and the 8388609th byte of the file is now 'c' instead of 'a'.  As a
result, the md5 of file2 is different.

Since fsdax+reflink is the newcomer, amend the direct writes in this
test so that they always end at the 8388608th byte, since we were never
really testing that last byte anyway.  This makes the test behavior
consistent across both access modes.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/182     |    4 ++--
 tests/xfs/182.out |    1 -
 2 files changed, 2 insertions(+), 3 deletions(-)


diff --git a/tests/xfs/182 b/tests/xfs/182
index ec3f7dc026..696b933e60 100755
--- a/tests/xfs/182
+++ b/tests/xfs/182
@@ -55,9 +55,9 @@ md5sum $testdir/file2 | _filter_scratch
 
 echo "CoW and unmount"
 $XFS_IO_PROG -f -c "cowextsize" $testdir/file2 >> $seqres.full
-$XFS_IO_PROG -d -f -c "pwrite -R -S 0x63 -b $real_blksz 0 $((filesize + 1))" \
+$XFS_IO_PROG -d -f -c "pwrite -R -S 0x63 -b $real_blksz 0 $filesize" \
 	$testdir/file2 2>&1 >> $seqres.full | _filter_xfs_io_error
-$XFS_IO_PROG -d -f -c "pwrite -S 0x63 -b $real_blksz 0 $((filesize + 1))" \
+$XFS_IO_PROG -d -f -c "pwrite -S 0x63 -b $real_blksz 0 $filesize" \
 	$testdir/file2 2>&1 >> $seqres.full | _filter_xfs_io_error
 _scratch_cycle_mount
 
diff --git a/tests/xfs/182.out b/tests/xfs/182.out
index 41384437ad..8821bcd5bd 100644
--- a/tests/xfs/182.out
+++ b/tests/xfs/182.out
@@ -5,7 +5,6 @@ Compare files
 2909feb63a37b0e95fe5cfb7f274f7b1  SCRATCH_MNT/test-182/file1
 2909feb63a37b0e95fe5cfb7f274f7b1  SCRATCH_MNT/test-182/file2
 CoW and unmount
-pwrite: Invalid argument
 Compare files
 2909feb63a37b0e95fe5cfb7f274f7b1  SCRATCH_MNT/test-182/file1
 c6ba35da9f73ced20d7781a448cc11d4  SCRATCH_MNT/test-182/file2

Re: [PATCH 3/3] xfs/182: fix spurious direct write failure

[Xiao Yang]

Hi Darrick,

It looks good to me. It actually fixed the failure of xfs/182 with 
enabled fsdax and reflink.
Reviewed-by: Xiao Yang <yangx.jy@fujitsu.com>

Best Regards,
Xiao Yang

On 2023/1/18 8:42, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> This test has some weird behavior that causes regressions when fsdax and
> reflink are enabled.  The goal of this test is to set a cow extent size
> hint, perform some random directio writes, perform a directio rewrite of
> the entire file, and make sure that the file content (and extent count)
> are sane afterwards.
> 
> Most of the time, the random directio writes will never touch the
> 8388609th byte, though if they do randomly select that EOF block, they'd
> end up extending the file by $real_blksz bytes and causing spurious test
> failures.
> 
> Then, the rewrite does this:
> 
> pwrite -S 0x63 -b $real_blksz 0 $((filesize + 1))
> 
> Note that we previously set filesize=8388608, which means that we're
> asking for a series of direct writes that fill the first 8388608 bytes
> with 'c'.  The last write in the series becomes a single byte direct
> write.  For regular file access mode, this last write will fail with
> EINVAL, since block devices do not support byte granularity writes and
> XFS does not fall back to the pagecache for unaligned direct wites.
> Hence we never wrote the 8388609th byte of the file.
> 
> However, fsdax *does* allow byte-granularity direct writes, which means
> that the single-byte write succeeds.  There is no EINVAL return code,
> and the 8388609th byte of the file is now 'c' instead of 'a'.  As a
> result, the md5 of file2 is different.
> 
> Since fsdax+reflink is the newcomer, amend the direct writes in this
> test so that they always end at the 8388608th byte, since we were never
> really testing that last byte anyway.  This makes the test behavior
> consistent across both access modes.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>   tests/xfs/182     |    4 ++--
>   tests/xfs/182.out |    1 -
>   2 files changed, 2 insertions(+), 3 deletions(-)
> 
> 
> diff --git a/tests/xfs/182 b/tests/xfs/182
> index ec3f7dc026..696b933e60 100755
> --- a/tests/xfs/182
> +++ b/tests/xfs/182
> @@ -55,9 +55,9 @@ md5sum $testdir/file2 | _filter_scratch
>   
>   echo "CoW and unmount"
>   $XFS_IO_PROG -f -c "cowextsize" $testdir/file2 >> $seqres.full
> -$XFS_IO_PROG -d -f -c "pwrite -R -S 0x63 -b $real_blksz 0 $((filesize + 1))" \
> +$XFS_IO_PROG -d -f -c "pwrite -R -S 0x63 -b $real_blksz 0 $filesize" \
>   	$testdir/file2 2>&1 >> $seqres.full | _filter_xfs_io_error
> -$XFS_IO_PROG -d -f -c "pwrite -S 0x63 -b $real_blksz 0 $((filesize + 1))" \
> +$XFS_IO_PROG -d -f -c "pwrite -S 0x63 -b $real_blksz 0 $filesize" \
>   	$testdir/file2 2>&1 >> $seqres.full | _filter_xfs_io_error
>   _scratch_cycle_mount
>   
> diff --git a/tests/xfs/182.out b/tests/xfs/182.out
> index 41384437ad..8821bcd5bd 100644
> --- a/tests/xfs/182.out
> +++ b/tests/xfs/182.out
> @@ -5,7 +5,6 @@ Compare files
>   2909feb63a37b0e95fe5cfb7f274f7b1  SCRATCH_MNT/test-182/file1
>   2909feb63a37b0e95fe5cfb7f274f7b1  SCRATCH_MNT/test-182/file2
>   CoW and unmount
> -pwrite: Invalid argument
>   Compare files
>   2909feb63a37b0e95fe5cfb7f274f7b1  SCRATCH_MNT/test-182/file1
>   c6ba35da9f73ced20d7781a448cc11d4  SCRATCH_MNT/test-182/file2
>