[PATCH] generic: add test for zero range over a file range with many small extents

[PATCH] generic: add test for zero range over a file range with many small extents

[fdmanana]

From: Filipe Manana <fdmanana@suse.com>

Test a fallocate() zero range operation against a large file range for which
there are many small extents allocated. Verify the operation does not fail
and the respective range return zeroes on subsequent reads.

This test is motivated by a bug found on btrfs. The patch that fixes the
bug on btrfs has the following subject:

 "btrfs: fix metadata reservation for fallocate that leads to transaction aborts"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/generic/609     | 61 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/609.out |  5 ++++
 tests/generic/group   |  1 +
 3 files changed, 67 insertions(+)
 create mode 100755 tests/generic/609
 create mode 100644 tests/generic/609.out

diff --git a/tests/generic/609 b/tests/generic/609
new file mode 100755
index 00000000..cda2b3dc
--- /dev/null
+++ b/tests/generic/609
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test No. 609
+#
+# Test a fallocate() zero range operation against a large file range for which
+# there are many small extents allocated. Verify the operation does not fail
+# and the respective range return zeroes on subsequent reads.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_xfs_io_command "fzero"
+_require_xfs_io_command "fpunch"
+_require_test_program "punch-alternating"
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# Create a file with many small extents. To speed up file creation, do
+# buffered writes and then punch a hole on every other block.
+$XFS_IO_PROG -f -c "pwrite -S 0xab -b 10M 0 100M" \
+	$SCRATCH_MNT/foobar >>$seqres.full
+$here/src/punch-alternating $SCRATCH_MNT/foobar >>$seqres.full
+
+# For btrfs, trigger a transaction commit to force metadata COW for the
+# following fallocate zero range operation.
+sync
+
+$XFS_IO_PROG -c "fzero 0 100M" $SCRATCH_MNT/foobar
+
+# Check the file content after umounting and mounting again the fs, to verify
+# everything was persisted.
+_scratch_cycle_mount
+
+echo "File content after zero range operation:"
+od -A d -t x1 $SCRATCH_MNT/foobar
+
+status=0
+exit
diff --git a/tests/generic/609.out b/tests/generic/609.out
new file mode 100644
index 00000000..feb8c211
--- /dev/null
+++ b/tests/generic/609.out
@@ -0,0 +1,5 @@
+QA output created by 609
+File content after zero range operation:
+0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+*
+104857600
diff --git a/tests/generic/group b/tests/generic/group
index aa969bcb..f8eabc0a 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -611,3 +611,4 @@
 606 auto attr quick dax
 607 auto attr quick dax
 608 auto attr quick dax
+609 auto quick prealloc zero
-- 
2.26.2

Re: [PATCH] generic: add test for zero range over a file range with many small extents

[Darrick J. Wong]

On Tue, Sep 08, 2020 at 11:32:02AM +0100, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Test a fallocate() zero range operation against a large file range for which
> there are many small extents allocated. Verify the operation does not fail
> and the respective range return zeroes on subsequent reads.

LOL, we didn't already have a stress test for fzero? :/

> This test is motivated by a bug found on btrfs. The patch that fixes the
> bug on btrfs has the following subject:
> 
>  "btrfs: fix metadata reservation for fallocate that leads to transaction aborts"
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>

Looks good to me!

Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>

--D

> ---
>  tests/generic/609     | 61 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/609.out |  5 ++++
>  tests/generic/group   |  1 +
>  3 files changed, 67 insertions(+)
>  create mode 100755 tests/generic/609
>  create mode 100644 tests/generic/609.out
> 
> diff --git a/tests/generic/609 b/tests/generic/609
> new file mode 100755
> index 00000000..cda2b3dc
> --- /dev/null
> +++ b/tests/generic/609
> @@ -0,0 +1,61 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test No. 609
> +#
> +# Test a fallocate() zero range operation against a large file range for which
> +# there are many small extents allocated. Verify the operation does not fail
> +# and the respective range return zeroes on subsequent reads.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_xfs_io_command "fzero"
> +_require_xfs_io_command "fpunch"
> +_require_test_program "punch-alternating"
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_scratch_mount
> +
> +# Create a file with many small extents. To speed up file creation, do
> +# buffered writes and then punch a hole on every other block.
> +$XFS_IO_PROG -f -c "pwrite -S 0xab -b 10M 0 100M" \
> +	$SCRATCH_MNT/foobar >>$seqres.full
> +$here/src/punch-alternating $SCRATCH_MNT/foobar >>$seqres.full
> +
> +# For btrfs, trigger a transaction commit to force metadata COW for the
> +# following fallocate zero range operation.
> +sync
> +
> +$XFS_IO_PROG -c "fzero 0 100M" $SCRATCH_MNT/foobar
> +
> +# Check the file content after umounting and mounting again the fs, to verify
> +# everything was persisted.
> +_scratch_cycle_mount
> +
> +echo "File content after zero range operation:"
> +od -A d -t x1 $SCRATCH_MNT/foobar
> +
> +status=0
> +exit
> diff --git a/tests/generic/609.out b/tests/generic/609.out
> new file mode 100644
> index 00000000..feb8c211
> --- /dev/null
> +++ b/tests/generic/609.out
> @@ -0,0 +1,5 @@
> +QA output created by 609
> +File content after zero range operation:
> +0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> +*
> +104857600
> diff --git a/tests/generic/group b/tests/generic/group
> index aa969bcb..f8eabc0a 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -611,3 +611,4 @@
>  606 auto attr quick dax
>  607 auto attr quick dax
>  608 auto attr quick dax
> +609 auto quick prealloc zero
> -- 
> 2.26.2
> 

Re: [PATCH] generic: add test for zero range over a file range with many small extents

[Zorro Lang]

On Wed, Sep 09, 2020 at 12:02:13PM -0700, Darrick J. Wong wrote:
> On Tue, Sep 08, 2020 at 11:32:02AM +0100, fdmanana@kernel.org wrote:
> > From: Filipe Manana <fdmanana@suse.com>
> > 
> > Test a fallocate() zero range operation against a large file range for which
> > there are many small extents allocated. Verify the operation does not fail
> > and the respective range return zeroes on subsequent reads.
> 
> LOL, we didn't already have a stress test for fzero? :/

We have lots of test for fzero, but they always test on a file with about 2~2000
blocks, none of them test with so many small extents.

We might need to think about this missing test coverage more, not only fzero
operation :)

Thanks,
Zorro

> 
> > This test is motivated by a bug found on btrfs. The patch that fixes the
> > bug on btrfs has the following subject:
> > 
> >  "btrfs: fix metadata reservation for fallocate that leads to transaction aborts"
> > 
> > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> 
> Looks good to me!
> 
> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> --D
> 
> > ---
> >  tests/generic/609     | 61 +++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/609.out |  5 ++++
> >  tests/generic/group   |  1 +
> >  3 files changed, 67 insertions(+)
> >  create mode 100755 tests/generic/609
> >  create mode 100644 tests/generic/609.out
> > 
> > diff --git a/tests/generic/609 b/tests/generic/609
> > new file mode 100755
> > index 00000000..cda2b3dc
> > --- /dev/null
> > +++ b/tests/generic/609
> > @@ -0,0 +1,61 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> > +#
> > +# FS QA Test No. 609
> > +#
> > +# Test a fallocate() zero range operation against a large file range for which
> > +# there are many small extents allocated. Verify the operation does not fail
> > +# and the respective range return zeroes on subsequent reads.
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +tmp=/tmp/$$
> > +status=1	# failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_supported_os Linux
> > +_require_scratch
> > +_require_xfs_io_command "fzero"
> > +_require_xfs_io_command "fpunch"
> > +_require_test_program "punch-alternating"
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs >>$seqres.full 2>&1
> > +_scratch_mount
> > +
> > +# Create a file with many small extents. To speed up file creation, do
> > +# buffered writes and then punch a hole on every other block.
> > +$XFS_IO_PROG -f -c "pwrite -S 0xab -b 10M 0 100M" \
> > +	$SCRATCH_MNT/foobar >>$seqres.full
> > +$here/src/punch-alternating $SCRATCH_MNT/foobar >>$seqres.full
> > +
> > +# For btrfs, trigger a transaction commit to force metadata COW for the
> > +# following fallocate zero range operation.
> > +sync
> > +
> > +$XFS_IO_PROG -c "fzero 0 100M" $SCRATCH_MNT/foobar
> > +
> > +# Check the file content after umounting and mounting again the fs, to verify
> > +# everything was persisted.
> > +_scratch_cycle_mount
> > +
> > +echo "File content after zero range operation:"
> > +od -A d -t x1 $SCRATCH_MNT/foobar
> > +
> > +status=0
> > +exit
> > diff --git a/tests/generic/609.out b/tests/generic/609.out
> > new file mode 100644
> > index 00000000..feb8c211
> > --- /dev/null
> > +++ b/tests/generic/609.out
> > @@ -0,0 +1,5 @@
> > +QA output created by 609
> > +File content after zero range operation:
> > +0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > +*
> > +104857600
> > diff --git a/tests/generic/group b/tests/generic/group
> > index aa969bcb..f8eabc0a 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -611,3 +611,4 @@
> >  606 auto attr quick dax
> >  607 auto attr quick dax
> >  608 auto attr quick dax
> > +609 auto quick prealloc zero
> > -- 
> > 2.26.2
> > 
>