* [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior
@ 2020-01-31 6:05 Qu Wenruo
2020-01-31 9:23 ` Filipe Manana
2020-01-31 9:26 ` Filipe Manana
0 siblings, 2 replies; 3+ messages in thread
From: Qu Wenruo @ 2020-01-31 6:05 UTC (permalink / raw)
To: fstests, linux-btrfs; +Cc: Martin Doucha
There is a new LTP test case (*) doing hole punching with the following
conditions:
- Hole is unaligned on exiting data
Which involves data writes to zero exiting data.
- The fs is full
- The involved file has NOCOW bit set
Even for fs like btrfs, such write should no allocate new space.
For other fses which don't support NOCOW bit, they either default to
NOCOW or don't support COW at all.
Thus the behavior should still be the same.
Btrfs currently fails such test, the fix is titled
"btrfs: Allow btrfs_truncate_block() to fallback to nocow for data space
reservation".
XFS and EXT4 all pass.
*: https://patchwork.ozlabs.org/patch/1224176/
Reported-by: Martin Doucha <mdoucha@suse.cz>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
Please note that, for EXT4 there seems to be a bug in mkfs.ext4, as it
always output the version string ("mke2fs 1.45.5 (07-Jan-2020)") to
stderr, polluting the golden output.
But the unaligned hole punching behavior is still correct for EXT4.
---
tests/generic/593 | 75 +++++++++++++++++++++++++++++++++++++++++++
tests/generic/593.out | 2 ++
tests/generic/group | 1 +
3 files changed, 78 insertions(+)
create mode 100755 tests/generic/593
create mode 100644 tests/generic/593.out
diff --git a/tests/generic/593 b/tests/generic/593
new file mode 100755
index 00000000..884a142a
--- /dev/null
+++ b/tests/generic/593
@@ -0,0 +1,75 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 593
+#
+# Test if a fs can still punch unaligned hole for NOCOW files when the fs
+# is full.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_command "$CHATTR_PROG" chattr
+_require_command "$LSATTR_PROG" lsattr
+
+# Create a small fs so filling it should be pretty fast
+fssize=$(( 1024 * 1024 * 1024 )) # In bytes
+
+_scratch_mkfs_sized $fssize > $seqres.full
+_scratch_mount
+
+blocksize=$(_get_block_size $SCRATCH_MNT)
+echo "blocksize = $blocksize" >> $seqres.full
+nr_blocks=5
+
+touch $SCRATCH_MNT/target
+# - Completely ignore the error
+# Either the fs supports COW, this will success and mark the file NOCOW
+# Or the fs doesn't support COW, we can still go ahead.
+$CHATTR_PROG +C $SCRATCH_MNT/target >> $seqres.full 2>&1
+
+$LSATTR_PROG $SCRATCH_MNT/target >> $seqres.full
+
+$XFS_IO_PROG -c "pwrite -b $blocksize 0 $(( $nr_blocks * $blocksize))" \
+ $SCRATCH_MNT/target >> $seqres.full
+
+# ENOSPC expected
+$XFS_IO_PROG -f -c "pwrite -b $blocksize 0 $fssize" \
+ $SCRATCH_MNT/padding >> $seqres.full 2>&1
+
+# All these fpunch calls should success
+for ((i = 0; i < $nr_blocks; i++)); do
+ $XFS_IO_PROG -c "fpunch $(( $i * $blocksize)) $(( $blocksize / 2))" \
+ $SCRATCH_MNT/target >> $seqres.full
+done
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/generic/593.out b/tests/generic/593.out
new file mode 100644
index 00000000..bac4d7d9
--- /dev/null
+++ b/tests/generic/593.out
@@ -0,0 +1,2 @@
+QA output created by 593
+Silence is golden
diff --git a/tests/generic/group b/tests/generic/group
index 6fe62505..ca4df435 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -595,3 +595,4 @@
590 auto prealloc preallocrw
591 auto quick rw pipe splice
592 auto quick encrypt
+593 auto quick enospc
--
2.23.0
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior
2020-01-31 6:05 [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior Qu Wenruo
@ 2020-01-31 9:23 ` Filipe Manana
2020-01-31 9:26 ` Filipe Manana
1 sibling, 0 replies; 3+ messages in thread
From: Filipe Manana @ 2020-01-31 9:23 UTC (permalink / raw)
To: Qu Wenruo; +Cc: fstests, linux-btrfs, Martin Doucha
On Fri, Jan 31, 2020 at 6:06 AM Qu Wenruo <wqu@suse.com> wrote:
>
> There is a new LTP test case (*) doing hole punching with the following
> conditions:
> - Hole is unaligned on exiting data
> Which involves data writes to zero exiting data.
>
> - The fs is full
>
> - The involved file has NOCOW bit set
> Even for fs like btrfs, such write should no allocate new space.
> For other fses which don't support NOCOW bit, they either default to
> NOCOW or don't support COW at all.
> Thus the behavior should still be the same.
>
> Btrfs currently fails such test, the fix is titled
> "btrfs: Allow btrfs_truncate_block() to fallback to nocow for data space
> reservation".
>
> XFS and EXT4 all pass.
>
> *: https://patchwork.ozlabs.org/patch/1224176/
>
> Reported-by: Martin Doucha <mdoucha@suse.cz>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> Please note that, for EXT4 there seems to be a bug in mkfs.ext4, as it
> always output the version string ("mke2fs 1.45.5 (07-Jan-2020)") to
> stderr, polluting the golden output.
>
> But the unaligned hole punching behavior is still correct for EXT4.
> ---
> tests/generic/593 | 75 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/593.out | 2 ++
> tests/generic/group | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100755 tests/generic/593
> create mode 100644 tests/generic/593.out
>
> diff --git a/tests/generic/593 b/tests/generic/593
> new file mode 100755
> index 00000000..884a142a
> --- /dev/null
> +++ b/tests/generic/593
> @@ -0,0 +1,75 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 593
> +#
> +# Test if a fs can still punch unaligned hole for NOCOW files when the fs
> +# is full.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_command "$CHATTR_PROG" chattr
> +_require_command "$LSATTR_PROG" lsattr
> +
> +# Create a small fs so filling it should be pretty fast
> +fssize=$(( 1024 * 1024 * 1024 )) # In bytes
Someone not so familiar with btrfs, looking at a generic test, might
ask, why 1Gb?
Why not 128Mb for example, that would make it even faster to fill the
fs... right?
You might add a comment mentioning that 1Gb is a safe value to
guarantee btrfs' mkfs will not use mixed block groups.
Because if it does, the hole punching might fail due to lack of
metadata space, that is, by not using mixed block groups we can be
sure we will have enough metadata free space while having exhausted
all data space.
Other than that, it looks good to me.
Thanks.
> +
> +_scratch_mkfs_sized $fssize > $seqres.full
> +_scratch_mount
> +
> +blocksize=$(_get_block_size $SCRATCH_MNT)
> +echo "blocksize = $blocksize" >> $seqres.full
> +nr_blocks=5
> +
> +touch $SCRATCH_MNT/target
> +# - Completely ignore the error
> +# Either the fs supports COW, this will success and mark the file NOCOW
> +# Or the fs doesn't support COW, we can still go ahead.
> +$CHATTR_PROG +C $SCRATCH_MNT/target >> $seqres.full 2>&1
> +
> +$LSATTR_PROG $SCRATCH_MNT/target >> $seqres.full
> +
> +$XFS_IO_PROG -c "pwrite -b $blocksize 0 $(( $nr_blocks * $blocksize))" \
> + $SCRATCH_MNT/target >> $seqres.full
> +
> +# ENOSPC expected
> +$XFS_IO_PROG -f -c "pwrite -b $blocksize 0 $fssize" \
> + $SCRATCH_MNT/padding >> $seqres.full 2>&1
> +
> +# All these fpunch calls should success
> +for ((i = 0; i < $nr_blocks; i++)); do
> + $XFS_IO_PROG -c "fpunch $(( $i * $blocksize)) $(( $blocksize / 2))" \
> + $SCRATCH_MNT/target >> $seqres.full
> +done
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/593.out b/tests/generic/593.out
> new file mode 100644
> index 00000000..bac4d7d9
> --- /dev/null
> +++ b/tests/generic/593.out
> @@ -0,0 +1,2 @@
> +QA output created by 593
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index 6fe62505..ca4df435 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -595,3 +595,4 @@
> 590 auto prealloc preallocrw
> 591 auto quick rw pipe splice
> 592 auto quick encrypt
> +593 auto quick enospc
> --
> 2.23.0
>
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior
2020-01-31 6:05 [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior Qu Wenruo
2020-01-31 9:23 ` Filipe Manana
@ 2020-01-31 9:26 ` Filipe Manana
1 sibling, 0 replies; 3+ messages in thread
From: Filipe Manana @ 2020-01-31 9:26 UTC (permalink / raw)
To: Qu Wenruo; +Cc: fstests, linux-btrfs, Martin Doucha
On Fri, Jan 31, 2020 at 6:06 AM Qu Wenruo <wqu@suse.com> wrote:
>
> There is a new LTP test case (*) doing hole punching with the following
> conditions:
> - Hole is unaligned on exiting data
> Which involves data writes to zero exiting data.
>
> - The fs is full
>
> - The involved file has NOCOW bit set
> Even for fs like btrfs, such write should no allocate new space.
> For other fses which don't support NOCOW bit, they either default to
> NOCOW or don't support COW at all.
> Thus the behavior should still be the same.
>
> Btrfs currently fails such test, the fix is titled
> "btrfs: Allow btrfs_truncate_block() to fallback to nocow for data space
> reservation".
>
> XFS and EXT4 all pass.
>
> *: https://patchwork.ozlabs.org/patch/1224176/
>
> Reported-by: Martin Doucha <mdoucha@suse.cz>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> Please note that, for EXT4 there seems to be a bug in mkfs.ext4, as it
> always output the version string ("mke2fs 1.45.5 (07-Jan-2020)") to
> stderr, polluting the golden output.
>
> But the unaligned hole punching behavior is still correct for EXT4.
> ---
> tests/generic/593 | 75 +++++++++++++++++++++++++++++++++++++++++++
> tests/generic/593.out | 2 ++
> tests/generic/group | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100755 tests/generic/593
> create mode 100644 tests/generic/593.out
>
> diff --git a/tests/generic/593 b/tests/generic/593
> new file mode 100755
> index 00000000..884a142a
> --- /dev/null
> +++ b/tests/generic/593
> @@ -0,0 +1,75 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 593
> +#
> +# Test if a fs can still punch unaligned hole for NOCOW files when the fs
> +# is full.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_command "$CHATTR_PROG" chattr
> +_require_command "$LSATTR_PROG" lsattr
_require_xfs_io_command "fpunch"
> +
> +# Create a small fs so filling it should be pretty fast
> +fssize=$(( 1024 * 1024 * 1024 )) # In bytes
(Repeating the fs size thing from the other mail since I forgot a few
more things)
Someone not so familiar with btrfs, looking at a generic test, might
ask, why 1Gb?
Why not 128Mb for example, that would make it even faster to fill the
fs... right?
You might add a comment mentioning that 1Gb is a safe value to
guarantee btrfs' mkfs will not use mixed block groups.
Because if it does, the hole punching might fail due to lack of
metadata space, that is, by not using mixed block groups we can be
sure we will have enough metadata free space while having exhausted
all data space.
> +
> +_scratch_mkfs_sized $fssize > $seqres.full
> +_scratch_mount
> +
> +blocksize=$(_get_block_size $SCRATCH_MNT)
> +echo "blocksize = $blocksize" >> $seqres.full
> +nr_blocks=5
> +
> +touch $SCRATCH_MNT/target
> +# - Completely ignore the error
> +# Either the fs supports COW, this will success and mark the file NOCOW
> +# Or the fs doesn't support COW, we can still go ahead.
> +$CHATTR_PROG +C $SCRATCH_MNT/target >> $seqres.full 2>&1
> +
> +$LSATTR_PROG $SCRATCH_MNT/target >> $seqres.full
> +
> +$XFS_IO_PROG -c "pwrite -b $blocksize 0 $(( $nr_blocks * $blocksize))" \
> + $SCRATCH_MNT/target >> $seqres.full
> +
> +# ENOSPC expected
> +$XFS_IO_PROG -f -c "pwrite -b $blocksize 0 $fssize" \
> + $SCRATCH_MNT/padding >> $seqres.full 2>&1
> +
> +# All these fpunch calls should success
> +for ((i = 0; i < $nr_blocks; i++)); do
> + $XFS_IO_PROG -c "fpunch $(( $i * $blocksize)) $(( $blocksize / 2))" \
> + $SCRATCH_MNT/target >> $seqres.full
> +done
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/593.out b/tests/generic/593.out
> new file mode 100644
> index 00000000..bac4d7d9
> --- /dev/null
> +++ b/tests/generic/593.out
> @@ -0,0 +1,2 @@
> +QA output created by 593
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index 6fe62505..ca4df435 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -595,3 +595,4 @@
> 590 auto prealloc preallocrw
> 591 auto quick rw pipe splice
> 592 auto quick encrypt
> +593 auto quick enospc
Also missing the group 'punch'.
Thanks.
> --
> 2.23.0
>
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-01-31 9:26 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-31 6:05 [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior Qu Wenruo
2020-01-31 9:23 ` Filipe Manana
2020-01-31 9:26 ` Filipe Manana
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).