* [PATCH] btrfs: test unaligned punch hole at ENOSPC
@ 2020-01-31 4:39 Anand Jain
2020-01-31 4:49 ` Qu Wenruo
0 siblings, 1 reply; 4+ messages in thread
From: Anand Jain @ 2020-01-31 4:39 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Try to punch hole with unaligned size and offset when the FS is
full. Mainly holes are punched at locations which are unaligned
with the file extent boundaries when the FS is full by data.
As the punching holes at unaligned location will involve
truncating blocks instead of just dropping the extents, it shall
involve reserving data and metadata space for delalloc and so data
alloc fails as the FS is full.
btrfs_punch_hole()
btrfs_truncate_block()
btrfs_check_data_free_space() <-- ENOSPC
We don't fail punch hole if the holes are aligned with the file
extent boundaries as it shall involve just dropping the related
extents, without truncating data extent blocks.
Link: https://patchwork.kernel.org/patch/11357415/
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
(cherry picked from commit 4c2c678cd56a81a210cb16f9f9347073e91e2fb0)
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Conflicts:
tests/btrfs/group
---
Its decided to bring back this test case, now the problem is better understood
and the fix is available in the ML as in [Link].
tests/btrfs/172 | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/172.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 76 insertions(+)
create mode 100755 tests/btrfs/172
create mode 100644 tests/btrfs/172.out
diff --git a/tests/btrfs/172 b/tests/btrfs/172
new file mode 100755
index 000000000000..0dffb2dff40b
--- /dev/null
+++ b/tests/btrfs/172
@@ -0,0 +1,73 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2018 Oracle. All Rights Reserved.
+#
+# FS QA Test 172
+#
+# Test if the unaligned (by size and offset) punch hole is successful when FS
+# is at ENOSPC.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_xfs_io_command "fpunch"
+
+_scratch_mkfs_sized $((256 * 1024 *1024)) >> $seqres.full
+
+# max_inline ensures data is not inlined within metadata extents
+_scratch_mount "-o max_inline=0,nodatacow"
+
+cat /proc/self/mounts | grep $SCRATCH_DEV >> $seqres.full
+$BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full
+
+extent_size=$(_scratch_btrfs_sectorsize)
+unalign_by=512
+echo extent_size=$extent_size unalign_by=$unalign_by >> $seqres.full
+
+$XFS_IO_PROG -f -c "pwrite -S 0xab 0 $((extent_size * 10))" \
+ $SCRATCH_MNT/testfile >> $seqres.full
+
+echo "Fill all space available for data and all unallocated space." >> $seqres.full
+dd status=none if=/dev/zero of=$SCRATCH_MNT/filler bs=512 >> $seqres.full 2>&1
+
+hole_offset=0
+hole_len=$unalign_by
+$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
+
+hole_offset=$(($extent_size + $unalign_by))
+hole_len=$(($extent_size - $unalign_by))
+$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
+
+hole_offset=$(($extent_size * 2 + $unalign_by))
+hole_len=$(($extent_size * 5))
+$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
+
+# success, all done
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/btrfs/172.out b/tests/btrfs/172.out
new file mode 100644
index 000000000000..ce2de3f0d107
--- /dev/null
+++ b/tests/btrfs/172.out
@@ -0,0 +1,2 @@
+QA output created by 172
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 4b64bf8b6d2f..697b6a38ea00 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -174,6 +174,7 @@
169 auto quick send
170 auto quick snapshot
171 auto quick qgroup
+172 auto quick punch
173 auto quick swap
174 auto quick swap
175 auto quick swap volume
--
1.8.3.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH] btrfs: test unaligned punch hole at ENOSPC
2020-01-31 4:39 [PATCH] btrfs: test unaligned punch hole at ENOSPC Anand Jain
@ 2020-01-31 4:49 ` Qu Wenruo
2020-01-31 5:09 ` [PATCH v2] " Anand Jain
0 siblings, 1 reply; 4+ messages in thread
From: Qu Wenruo @ 2020-01-31 4:49 UTC (permalink / raw)
To: Anand Jain, fstests; +Cc: linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 4655 bytes --]
On 2020/1/31 下午12:39, Anand Jain wrote:
> Try to punch hole with unaligned size and offset when the FS is
> full. Mainly holes are punched at locations which are unaligned
> with the file extent boundaries when the FS is full by data.
> As the punching holes at unaligned location will involve
> truncating blocks instead of just dropping the extents, it shall
> involve reserving data and metadata space for delalloc and so data
> alloc fails as the FS is full.
It's better to mention the nocow part.
For COW part, the ENOSPC is still expected.
As that's the main reason I failed to understand the use case.
Thanks,
Qu
>
> btrfs_punch_hole()
> btrfs_truncate_block()
> btrfs_check_data_free_space() <-- ENOSPC
>
> We don't fail punch hole if the holes are aligned with the file
> extent boundaries as it shall involve just dropping the related
> extents, without truncating data extent blocks.
>
> Link: https://patchwork.kernel.org/patch/11357415/
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
> (cherry picked from commit 4c2c678cd56a81a210cb16f9f9347073e91e2fb0)
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>
> Conflicts:
> tests/btrfs/group
> ---
> Its decided to bring back this test case, now the problem is better understood
> and the fix is available in the ML as in [Link].
>
> tests/btrfs/172 | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/172.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 76 insertions(+)
> create mode 100755 tests/btrfs/172
> create mode 100644 tests/btrfs/172.out
>
> diff --git a/tests/btrfs/172 b/tests/btrfs/172
> new file mode 100755
> index 000000000000..0dffb2dff40b
> --- /dev/null
> +++ b/tests/btrfs/172
> @@ -0,0 +1,73 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 172
> +#
> +# Test if the unaligned (by size and offset) punch hole is successful when FS
> +# is at ENOSPC.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_xfs_io_command "fpunch"
> +
> +_scratch_mkfs_sized $((256 * 1024 *1024)) >> $seqres.full
> +
> +# max_inline ensures data is not inlined within metadata extents
> +_scratch_mount "-o max_inline=0,nodatacow"
> +
> +cat /proc/self/mounts | grep $SCRATCH_DEV >> $seqres.full
> +$BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full
> +
> +extent_size=$(_scratch_btrfs_sectorsize)
> +unalign_by=512
> +echo extent_size=$extent_size unalign_by=$unalign_by >> $seqres.full
> +
> +$XFS_IO_PROG -f -c "pwrite -S 0xab 0 $((extent_size * 10))" \
> + $SCRATCH_MNT/testfile >> $seqres.full
> +
> +echo "Fill all space available for data and all unallocated space." >> $seqres.full
> +dd status=none if=/dev/zero of=$SCRATCH_MNT/filler bs=512 >> $seqres.full 2>&1
> +
> +hole_offset=0
> +hole_len=$unalign_by
> +$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
> +
> +hole_offset=$(($extent_size + $unalign_by))
> +hole_len=$(($extent_size - $unalign_by))
> +$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
> +
> +hole_offset=$(($extent_size * 2 + $unalign_by))
> +hole_len=$(($extent_size * 5))
> +$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
> +
> +# success, all done
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/172.out b/tests/btrfs/172.out
> new file mode 100644
> index 000000000000..ce2de3f0d107
> --- /dev/null
> +++ b/tests/btrfs/172.out
> @@ -0,0 +1,2 @@
> +QA output created by 172
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 4b64bf8b6d2f..697b6a38ea00 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -174,6 +174,7 @@
> 169 auto quick send
> 170 auto quick snapshot
> 171 auto quick qgroup
> +172 auto quick punch
> 173 auto quick swap
> 174 auto quick swap
> 175 auto quick swap volume
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v2] btrfs: test unaligned punch hole at ENOSPC
2020-01-31 4:49 ` Qu Wenruo
@ 2020-01-31 5:09 ` Anand Jain
2020-02-17 12:43 ` Eryu Guan
0 siblings, 1 reply; 4+ messages in thread
From: Anand Jain @ 2020-01-31 5:09 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Try to punch hole with unaligned size and offset when the FS is
full and mounted with nodatacow option.
Mainly holes are punched at locations which are unaligned
with the file extent boundaries when the FS is full by data.
As the punching holes at unaligned location will involve
truncating blocks instead of just dropping the extents, it shall
involve reserving data and metadata space for delalloc and so data
alloc fails as the FS is full.
btrfs_punch_hole()
btrfs_truncate_block()
btrfs_check_data_free_space() <-- ENOSPC
We don't fail punch hole if the holes are aligned with the file
extent boundaries as it shall involve just dropping the related
extents, without truncating data extent blocks.
Link: https://patchwork.kernel.org/patch/11357415/
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
(cherry picked from commit 4c2c678cd56a81a210cb16f9f9347073e91e2fb0)
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Conflicts:
tests/btrfs/group
---
Its decided to bring back this test case, now the problem is better understood
and the fix is available in the ML as in [Link].
v2: mention nodatacow option used in the testcase in the commit log.
tests/btrfs/172 | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/172.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 76 insertions(+)
create mode 100755 tests/btrfs/172
create mode 100644 tests/btrfs/172.out
diff --git a/tests/btrfs/172 b/tests/btrfs/172
new file mode 100755
index 000000000000..0dffb2dff40b
--- /dev/null
+++ b/tests/btrfs/172
@@ -0,0 +1,73 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2018 Oracle. All Rights Reserved.
+#
+# FS QA Test 172
+#
+# Test if the unaligned (by size and offset) punch hole is successful when FS
+# is at ENOSPC.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_xfs_io_command "fpunch"
+
+_scratch_mkfs_sized $((256 * 1024 *1024)) >> $seqres.full
+
+# max_inline ensures data is not inlined within metadata extents
+_scratch_mount "-o max_inline=0,nodatacow"
+
+cat /proc/self/mounts | grep $SCRATCH_DEV >> $seqres.full
+$BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full
+
+extent_size=$(_scratch_btrfs_sectorsize)
+unalign_by=512
+echo extent_size=$extent_size unalign_by=$unalign_by >> $seqres.full
+
+$XFS_IO_PROG -f -c "pwrite -S 0xab 0 $((extent_size * 10))" \
+ $SCRATCH_MNT/testfile >> $seqres.full
+
+echo "Fill all space available for data and all unallocated space." >> $seqres.full
+dd status=none if=/dev/zero of=$SCRATCH_MNT/filler bs=512 >> $seqres.full 2>&1
+
+hole_offset=0
+hole_len=$unalign_by
+$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
+
+hole_offset=$(($extent_size + $unalign_by))
+hole_len=$(($extent_size - $unalign_by))
+$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
+
+hole_offset=$(($extent_size * 2 + $unalign_by))
+hole_len=$(($extent_size * 5))
+$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
+
+# success, all done
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/btrfs/172.out b/tests/btrfs/172.out
new file mode 100644
index 000000000000..ce2de3f0d107
--- /dev/null
+++ b/tests/btrfs/172.out
@@ -0,0 +1,2 @@
+QA output created by 172
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 4b64bf8b6d2f..697b6a38ea00 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -174,6 +174,7 @@
169 auto quick send
170 auto quick snapshot
171 auto quick qgroup
+172 auto quick punch
173 auto quick swap
174 auto quick swap
175 auto quick swap volume
--
1.8.3.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH v2] btrfs: test unaligned punch hole at ENOSPC
2020-01-31 5:09 ` [PATCH v2] " Anand Jain
@ 2020-02-17 12:43 ` Eryu Guan
0 siblings, 0 replies; 4+ messages in thread
From: Eryu Guan @ 2020-02-17 12:43 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Fri, Jan 31, 2020 at 01:09:57PM +0800, Anand Jain wrote:
> Try to punch hole with unaligned size and offset when the FS is
> full and mounted with nodatacow option.
> Mainly holes are punched at locations which are unaligned
> with the file extent boundaries when the FS is full by data.
> As the punching holes at unaligned location will involve
> truncating blocks instead of just dropping the extents, it shall
> involve reserving data and metadata space for delalloc and so data
> alloc fails as the FS is full.
>
> btrfs_punch_hole()
> btrfs_truncate_block()
> btrfs_check_data_free_space() <-- ENOSPC
>
> We don't fail punch hole if the holes are aligned with the file
> extent boundaries as it shall involve just dropping the related
> extents, without truncating data extent blocks.
>
> Link: https://patchwork.kernel.org/patch/11357415/
I've went through above link, and all btrfs developers involved there
agreed to restore the original btrfs/172 case. Then I'm fine with it
too. But we really should avoid such remove/reconstruct again,
especially this time btrfs/172 is already taken by other test..
Thanks,
Eryu
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> Signed-off-by: Eryu Guan <guaneryu@gmail.com>
> (cherry picked from commit 4c2c678cd56a81a210cb16f9f9347073e91e2fb0)
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>
> Conflicts:
> tests/btrfs/group
> ---
> Its decided to bring back this test case, now the problem is better understood
> and the fix is available in the ML as in [Link].
> v2: mention nodatacow option used in the testcase in the commit log.
>
> tests/btrfs/204 | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/204.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 76 insertions(+)
> create mode 100755 tests/btrfs/204
> create mode 100644 tests/btrfs/204.out
>
> diff --git a/tests/btrfs/204 b/tests/btrfs/204
> new file mode 100755
> index 000000000000..0dffb2dff40b
> --- /dev/null
> +++ b/tests/btrfs/204
> @@ -0,0 +1,73 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 Oracle. All Rights Reserved.
> +#
> +# FS QA Test 204
> +#
> +# Test if the unaligned (by size and offset) punch hole is successful when FS
> +# is at ENOSPC.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +_require_xfs_io_command "fpunch"
> +
> +_scratch_mkfs_sized $((256 * 1024 *1024)) >> $seqres.full
> +
> +# max_inline ensures data is not inlined within metadata extents
> +_scratch_mount "-o max_inline=0,nodatacow"
> +
> +cat /proc/self/mounts | grep $SCRATCH_DEV >> $seqres.full
> +$BTRFS_UTIL_PROG filesystem df $SCRATCH_MNT >> $seqres.full
> +
> +extent_size=$(_scratch_btrfs_sectorsize)
> +unalign_by=512
> +echo extent_size=$extent_size unalign_by=$unalign_by >> $seqres.full
> +
> +$XFS_IO_PROG -f -c "pwrite -S 0xab 0 $((extent_size * 10))" \
> + $SCRATCH_MNT/testfile >> $seqres.full
> +
> +echo "Fill all space available for data and all unallocated space." >> $seqres.full
> +dd status=none if=/dev/zero of=$SCRATCH_MNT/filler bs=512 >> $seqres.full 2>&1
> +
> +hole_offset=0
> +hole_len=$unalign_by
> +$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
> +
> +hole_offset=$(($extent_size + $unalign_by))
> +hole_len=$(($extent_size - $unalign_by))
> +$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
> +
> +hole_offset=$(($extent_size * 2 + $unalign_by))
> +hole_len=$(($extent_size * 5))
> +$XFS_IO_PROG -c "fpunch $hole_offset $hole_len" $SCRATCH_MNT/testfile
> +
> +# success, all done
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/204.out b/tests/btrfs/204.out
> new file mode 100644
> index 000000000000..ce2de3f0d107
> --- /dev/null
> +++ b/tests/btrfs/204.out
> @@ -0,0 +1,2 @@
> +QA output created by 204
> +Silence is golden
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-02-17 12:43 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-31 4:39 [PATCH] btrfs: test unaligned punch hole at ENOSPC Anand Jain
2020-01-31 4:49 ` Qu Wenruo
2020-01-31 5:09 ` [PATCH v2] " Anand Jain
2020-02-17 12:43 ` Eryu Guan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).