FSTests Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior
@ 2020-01-31  6:05 Qu Wenruo
  2020-01-31  9:23 ` Filipe Manana
  2020-01-31  9:26 ` Filipe Manana
  0 siblings, 2 replies; 3+ messages in thread
From: Qu Wenruo @ 2020-01-31  6:05 UTC (permalink / raw)
  To: fstests, linux-btrfs; +Cc: Martin Doucha

There is a new LTP test case (*) doing hole punching with the following
conditions:
- Hole is unaligned on exiting data
  Which involves data writes to zero exiting data.

- The fs is full

- The involved file has NOCOW bit set
  Even for fs like btrfs, such write should no allocate new space.
  For other fses which don't support NOCOW bit, they either default to
  NOCOW or don't support COW at all.
  Thus the behavior should still be the same.

Btrfs currently fails such test, the fix is titled
"btrfs: Allow btrfs_truncate_block() to fallback to nocow for data space
 reservation".

XFS and EXT4 all pass.

*: https://patchwork.ozlabs.org/patch/1224176/

Reported-by: Martin Doucha <mdoucha@suse.cz>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
Please note that, for EXT4 there seems to be a bug in mkfs.ext4, as it
always output the version string ("mke2fs 1.45.5 (07-Jan-2020)") to
stderr, polluting the golden output.

But the unaligned hole punching behavior is still correct for EXT4.
---
 tests/generic/593     | 75 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/593.out |  2 ++
 tests/generic/group   |  1 +
 3 files changed, 78 insertions(+)
 create mode 100755 tests/generic/593
 create mode 100644 tests/generic/593.out

diff --git a/tests/generic/593 b/tests/generic/593
new file mode 100755
index 00000000..884a142a
--- /dev/null
+++ b/tests/generic/593
@@ -0,0 +1,75 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 593
+#
+# Test if a fs can still punch unaligned hole for NOCOW files when the fs
+# is full.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_command "$CHATTR_PROG" chattr
+_require_command "$LSATTR_PROG" lsattr 
+
+# Create a small fs so filling it should be pretty fast
+fssize=$(( 1024 * 1024 * 1024 )) # In bytes
+
+_scratch_mkfs_sized $fssize > $seqres.full
+_scratch_mount
+
+blocksize=$(_get_block_size $SCRATCH_MNT)
+echo "blocksize = $blocksize" >> $seqres.full
+nr_blocks=5
+
+touch $SCRATCH_MNT/target
+# - Completely ignore the error
+#   Either the fs supports COW, this will success and mark the file NOCOW
+#   Or the fs doesn't support COW, we can still go ahead.
+$CHATTR_PROG +C $SCRATCH_MNT/target >> $seqres.full 2>&1
+
+$LSATTR_PROG $SCRATCH_MNT/target >> $seqres.full
+
+$XFS_IO_PROG -c "pwrite -b $blocksize 0 $(( $nr_blocks * $blocksize))" \
+	$SCRATCH_MNT/target >> $seqres.full
+
+# ENOSPC expected
+$XFS_IO_PROG -f -c "pwrite -b $blocksize 0 $fssize" \
+	$SCRATCH_MNT/padding >> $seqres.full 2>&1
+
+# All these fpunch calls should success
+for ((i = 0; i < $nr_blocks; i++)); do
+	$XFS_IO_PROG -c "fpunch $(( $i * $blocksize)) $(( $blocksize / 2))" \
+		$SCRATCH_MNT/target >> $seqres.full
+done
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
diff --git a/tests/generic/593.out b/tests/generic/593.out
new file mode 100644
index 00000000..bac4d7d9
--- /dev/null
+++ b/tests/generic/593.out
@@ -0,0 +1,2 @@
+QA output created by 593
+Silence is golden
diff --git a/tests/generic/group b/tests/generic/group
index 6fe62505..ca4df435 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -595,3 +595,4 @@
 590 auto prealloc preallocrw
 591 auto quick rw pipe splice
 592 auto quick encrypt
+593 auto quick enospc
-- 
2.23.0


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior
  2020-01-31  6:05 [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior Qu Wenruo
@ 2020-01-31  9:23 ` Filipe Manana
  2020-01-31  9:26 ` Filipe Manana
  1 sibling, 0 replies; 3+ messages in thread
From: Filipe Manana @ 2020-01-31  9:23 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: fstests, linux-btrfs, Martin Doucha

On Fri, Jan 31, 2020 at 6:06 AM Qu Wenruo <wqu@suse.com> wrote:
>
> There is a new LTP test case (*) doing hole punching with the following
> conditions:
> - Hole is unaligned on exiting data
>   Which involves data writes to zero exiting data.
>
> - The fs is full
>
> - The involved file has NOCOW bit set
>   Even for fs like btrfs, such write should no allocate new space.
>   For other fses which don't support NOCOW bit, they either default to
>   NOCOW or don't support COW at all.
>   Thus the behavior should still be the same.
>
> Btrfs currently fails such test, the fix is titled
> "btrfs: Allow btrfs_truncate_block() to fallback to nocow for data space
>  reservation".
>
> XFS and EXT4 all pass.
>
> *: https://patchwork.ozlabs.org/patch/1224176/
>
> Reported-by: Martin Doucha <mdoucha@suse.cz>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> Please note that, for EXT4 there seems to be a bug in mkfs.ext4, as it
> always output the version string ("mke2fs 1.45.5 (07-Jan-2020)") to
> stderr, polluting the golden output.
>
> But the unaligned hole punching behavior is still correct for EXT4.
> ---
>  tests/generic/593     | 75 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/593.out |  2 ++
>  tests/generic/group   |  1 +
>  3 files changed, 78 insertions(+)
>  create mode 100755 tests/generic/593
>  create mode 100644 tests/generic/593.out
>
> diff --git a/tests/generic/593 b/tests/generic/593
> new file mode 100755
> index 00000000..884a142a
> --- /dev/null
> +++ b/tests/generic/593
> @@ -0,0 +1,75 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 593
> +#
> +# Test if a fs can still punch unaligned hole for NOCOW files when the fs
> +# is full.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1       # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_command "$CHATTR_PROG" chattr
> +_require_command "$LSATTR_PROG" lsattr
> +
> +# Create a small fs so filling it should be pretty fast
> +fssize=$(( 1024 * 1024 * 1024 )) # In bytes

Someone not so familiar with btrfs, looking at a generic test, might
ask, why 1Gb?
Why not 128Mb for example, that would make it even faster to fill the
fs... right?

You might add a comment mentioning that 1Gb is a safe value to
guarantee btrfs' mkfs will not use mixed block groups.
Because if it does, the hole punching might fail due to lack of
metadata space, that is, by not using mixed block groups we can be
sure we will have enough metadata free space while having exhausted
all data space.

Other than that, it looks good to me.

Thanks.

> +
> +_scratch_mkfs_sized $fssize > $seqres.full
> +_scratch_mount
> +
> +blocksize=$(_get_block_size $SCRATCH_MNT)
> +echo "blocksize = $blocksize" >> $seqres.full
> +nr_blocks=5
> +
> +touch $SCRATCH_MNT/target
> +# - Completely ignore the error
> +#   Either the fs supports COW, this will success and mark the file NOCOW
> +#   Or the fs doesn't support COW, we can still go ahead.
> +$CHATTR_PROG +C $SCRATCH_MNT/target >> $seqres.full 2>&1
> +
> +$LSATTR_PROG $SCRATCH_MNT/target >> $seqres.full
> +
> +$XFS_IO_PROG -c "pwrite -b $blocksize 0 $(( $nr_blocks * $blocksize))" \
> +       $SCRATCH_MNT/target >> $seqres.full
> +
> +# ENOSPC expected
> +$XFS_IO_PROG -f -c "pwrite -b $blocksize 0 $fssize" \
> +       $SCRATCH_MNT/padding >> $seqres.full 2>&1
> +
> +# All these fpunch calls should success
> +for ((i = 0; i < $nr_blocks; i++)); do
> +       $XFS_IO_PROG -c "fpunch $(( $i * $blocksize)) $(( $blocksize / 2))" \
> +               $SCRATCH_MNT/target >> $seqres.full
> +done
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/593.out b/tests/generic/593.out
> new file mode 100644
> index 00000000..bac4d7d9
> --- /dev/null
> +++ b/tests/generic/593.out
> @@ -0,0 +1,2 @@
> +QA output created by 593
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index 6fe62505..ca4df435 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -595,3 +595,4 @@
>  590 auto prealloc preallocrw
>  591 auto quick rw pipe splice
>  592 auto quick encrypt
> +593 auto quick enospc
> --
> 2.23.0
>


-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior
  2020-01-31  6:05 [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior Qu Wenruo
  2020-01-31  9:23 ` Filipe Manana
@ 2020-01-31  9:26 ` Filipe Manana
  1 sibling, 0 replies; 3+ messages in thread
From: Filipe Manana @ 2020-01-31  9:26 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: fstests, linux-btrfs, Martin Doucha

On Fri, Jan 31, 2020 at 6:06 AM Qu Wenruo <wqu@suse.com> wrote:
>
> There is a new LTP test case (*) doing hole punching with the following
> conditions:
> - Hole is unaligned on exiting data
>   Which involves data writes to zero exiting data.
>
> - The fs is full
>
> - The involved file has NOCOW bit set
>   Even for fs like btrfs, such write should no allocate new space.
>   For other fses which don't support NOCOW bit, they either default to
>   NOCOW or don't support COW at all.
>   Thus the behavior should still be the same.
>
> Btrfs currently fails such test, the fix is titled
> "btrfs: Allow btrfs_truncate_block() to fallback to nocow for data space
>  reservation".
>
> XFS and EXT4 all pass.
>
> *: https://patchwork.ozlabs.org/patch/1224176/
>
> Reported-by: Martin Doucha <mdoucha@suse.cz>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> Please note that, for EXT4 there seems to be a bug in mkfs.ext4, as it
> always output the version string ("mke2fs 1.45.5 (07-Jan-2020)") to
> stderr, polluting the golden output.
>
> But the unaligned hole punching behavior is still correct for EXT4.
> ---
>  tests/generic/593     | 75 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/593.out |  2 ++
>  tests/generic/group   |  1 +
>  3 files changed, 78 insertions(+)
>  create mode 100755 tests/generic/593
>  create mode 100644 tests/generic/593.out
>
> diff --git a/tests/generic/593 b/tests/generic/593
> new file mode 100755
> index 00000000..884a142a
> --- /dev/null
> +++ b/tests/generic/593
> @@ -0,0 +1,75 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2020 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 593
> +#
> +# Test if a fs can still punch unaligned hole for NOCOW files when the fs
> +# is full.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1       # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_require_command "$CHATTR_PROG" chattr
> +_require_command "$LSATTR_PROG" lsattr

_require_xfs_io_command "fpunch"

> +
> +# Create a small fs so filling it should be pretty fast
> +fssize=$(( 1024 * 1024 * 1024 )) # In bytes

(Repeating the fs size thing from the other mail since I forgot a few
more things)

Someone not so familiar with btrfs, looking at a generic test, might
ask, why 1Gb?
Why not 128Mb for example, that would make it even faster to fill the
fs... right?

You might add a comment mentioning that 1Gb is a safe value to
guarantee btrfs' mkfs will not use mixed block groups.
Because if it does, the hole punching might fail due to lack of
metadata space, that is, by not using mixed block groups we can be
sure we will have enough metadata free space while having exhausted
all data space.

> +
> +_scratch_mkfs_sized $fssize > $seqres.full
> +_scratch_mount
> +
> +blocksize=$(_get_block_size $SCRATCH_MNT)
> +echo "blocksize = $blocksize" >> $seqres.full
> +nr_blocks=5
> +
> +touch $SCRATCH_MNT/target
> +# - Completely ignore the error
> +#   Either the fs supports COW, this will success and mark the file NOCOW
> +#   Or the fs doesn't support COW, we can still go ahead.
> +$CHATTR_PROG +C $SCRATCH_MNT/target >> $seqres.full 2>&1
> +
> +$LSATTR_PROG $SCRATCH_MNT/target >> $seqres.full
> +
> +$XFS_IO_PROG -c "pwrite -b $blocksize 0 $(( $nr_blocks * $blocksize))" \
> +       $SCRATCH_MNT/target >> $seqres.full
> +
> +# ENOSPC expected
> +$XFS_IO_PROG -f -c "pwrite -b $blocksize 0 $fssize" \
> +       $SCRATCH_MNT/padding >> $seqres.full 2>&1
> +
> +# All these fpunch calls should success
> +for ((i = 0; i < $nr_blocks; i++)); do
> +       $XFS_IO_PROG -c "fpunch $(( $i * $blocksize)) $(( $blocksize / 2))" \
> +               $SCRATCH_MNT/target >> $seqres.full
> +done
> +
> +echo "Silence is golden"
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/593.out b/tests/generic/593.out
> new file mode 100644
> index 00000000..bac4d7d9
> --- /dev/null
> +++ b/tests/generic/593.out
> @@ -0,0 +1,2 @@
> +QA output created by 593
> +Silence is golden
> diff --git a/tests/generic/group b/tests/generic/group
> index 6fe62505..ca4df435 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -595,3 +595,4 @@
>  590 auto prealloc preallocrw
>  591 auto quick rw pipe splice
>  592 auto quick encrypt
> +593 auto quick enospc

Also missing the group 'punch'.

Thanks.
> --
> 2.23.0
>


-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-31  6:05 [PATCH] fstests: generic: Introduce new test case to verify the NOCOW unaligned hole punch behavior Qu Wenruo
2020-01-31  9:23 ` Filipe Manana
2020-01-31  9:26 ` Filipe Manana

FSTests Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/fstests/0 fstests/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 fstests fstests/ https://lore.kernel.org/fstests \
		fstests@vger.kernel.org
	public-inbox-index fstests

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.fstests


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git