* [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
@ 2021-02-11 17:01 fdmanana
2021-03-07 14:35 ` Eryu Guan
0 siblings, 1 reply; 6+ messages in thread
From: fdmanana @ 2021-02-11 17:01 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Filipe Manana
From: Filipe Manana <fdmanana@suse.com>
Test cases where a direct IO write, with O_DSYNC, can not be done and has
to fallback to a buffered write.
This is motivated by a regression that was introduced in kernel 5.10 by
commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
DSYNC workaround").
Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
tests/btrfs/231 | 61 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/231.out | 21 ++++++++++++++++
tests/btrfs/group | 1 +
3 files changed, 83 insertions(+)
create mode 100755 tests/btrfs/231
create mode 100644 tests/btrfs/231.out
diff --git a/tests/btrfs/231 b/tests/btrfs/231
new file mode 100755
index 00000000..9a404f57
--- /dev/null
+++ b/tests/btrfs/231
@@ -0,0 +1,61 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test No. btrfs/231
+#
+# Test cases where a direct IO write, with O_DSYNC, can not be done and has to
+# fallback to a buffered write.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs btrfs
+_require_scratch
+_require_odirect
+_require_chattr c
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# First lets test with an attempt to write into a file range with compressed
+# extents.
+touch $SCRATCH_MNT/foo
+$CHATTR_PROG +c $SCRATCH_MNT/foo
+
+$XFS_IO_PROG -s -c "pwrite -S 0xab -b 1M 0 1M" $SCRATCH_MNT/foo | _filter_xfs_io
+# Now do the direct IO write...
+$XFS_IO_PROG -d -s -c "pwrite -S 0xcd 512K 512K" $SCRATCH_MNT/foo | _filter_xfs_io
+
+# Now try doing a write into an unaligned offset...
+$XFS_IO_PROG -f -d -s -c "pwrite -S 0xef 1111 512K" $SCRATCH_MNT/bar | _filter_xfs_io
+
+# Unmount, mount again, and verify we have the expected data.
+_scratch_cycle_mount
+
+echo "File foo data:"
+od -A d -t x1 $SCRATCH_MNT/foo
+echo "File bar data:"
+od -A d -t x1 $SCRATCH_MNT/bar
+
+status=0
+exit
diff --git a/tests/btrfs/231.out b/tests/btrfs/231.out
new file mode 100644
index 00000000..def05769
--- /dev/null
+++ b/tests/btrfs/231.out
@@ -0,0 +1,21 @@
+QA output created by 231
+wrote 1048576/1048576 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 524288/524288 bytes at offset 524288
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 524288/524288 bytes at offset 1111
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+File foo data:
+0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
+*
+0524288 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
+*
+1048576
+File bar data:
+0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+*
+0001104 00 00 00 00 00 00 00 ef ef ef ef ef ef ef ef ef
+0001120 ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef
+*
+0525392 ef ef ef ef ef ef ef
+0525399
diff --git a/tests/btrfs/group b/tests/btrfs/group
index a7c65983..9f63db69 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -233,3 +233,4 @@
228 auto quick volume
229 auto quick send clone
230 auto quick qgroup limit
+231 auto quick compress rw
--
2.28.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
2021-02-11 17:01 [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write fdmanana
@ 2021-03-07 14:35 ` Eryu Guan
2021-03-07 15:07 ` Filipe Manana
0 siblings, 1 reply; 6+ messages in thread
From: Eryu Guan @ 2021-03-07 14:35 UTC (permalink / raw)
To: fdmanana; +Cc: fstests, linux-btrfs, Filipe Manana
On Thu, Feb 11, 2021 at 05:01:18PM +0000, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
>
> Test cases where a direct IO write, with O_DSYNC, can not be done and has
> to fallback to a buffered write.
>
> This is motivated by a regression that was introduced in kernel 5.10 by
> commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> DSYNC workaround").
>
> Signed-off-by: Filipe Manana <fdmanana@suse.com>
Sorry for the late review..
So this is supposed to fail with v5.10 kernel, right? But I got it
passed
[root@fedoravm xfstests]# ./check -s btrfs btrfs/231
SECTION -- btrfs
RECREATING -- btrfs on /dev/mapper/testvg-lv1
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 fedoravm 5.10.0 #6 SMP Sun Mar 7 22:25:35 CST 2021
MKFS_OPTIONS -- /dev/mapper/testvg-lv2
MOUNT_OPTIONS -- /dev/mapper/testvg-lv2 /mnt/scratch
btrfs/231 13s ... 8s
Ran: btrfs/231
Passed all 1 tests
SECTION -- btrfs
=========================
Ran: btrfs/231
Passed all 1 tests
> ---
> tests/btrfs/231 | 61 +++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/231.out | 21 ++++++++++++++++
> tests/btrfs/group | 1 +
> 3 files changed, 83 insertions(+)
> create mode 100755 tests/btrfs/231
> create mode 100644 tests/btrfs/231.out
>
> diff --git a/tests/btrfs/231 b/tests/btrfs/231
> new file mode 100755
> index 00000000..9a404f57
> --- /dev/null
> +++ b/tests/btrfs/231
> @@ -0,0 +1,61 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test No. btrfs/231
> +#
> +# Test cases where a direct IO write, with O_DSYNC, can not be done and has to
> +# fallback to a buffered write.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/attr
> +
> +# real QA test starts here
> +_supported_fs btrfs
> +_require_scratch
> +_require_odirect
> +_require_chattr c
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_scratch_mount
> +
> +# First lets test with an attempt to write into a file range with compressed
> +# extents.
> +touch $SCRATCH_MNT/foo
> +$CHATTR_PROG +c $SCRATCH_MNT/foo
It's not so clear to me why writing into a compressed file is required,
would you please add more comments?
Thanks,
Eryu
> +
> +$XFS_IO_PROG -s -c "pwrite -S 0xab -b 1M 0 1M" $SCRATCH_MNT/foo | _filter_xfs_io
> +# Now do the direct IO write...
> +$XFS_IO_PROG -d -s -c "pwrite -S 0xcd 512K 512K" $SCRATCH_MNT/foo | _filter_xfs_io
> +
> +# Now try doing a write into an unaligned offset...
> +$XFS_IO_PROG -f -d -s -c "pwrite -S 0xef 1111 512K" $SCRATCH_MNT/bar | _filter_xfs_io
> +
> +# Unmount, mount again, and verify we have the expected data.
> +_scratch_cycle_mount
> +
> +echo "File foo data:"
> +od -A d -t x1 $SCRATCH_MNT/foo
> +echo "File bar data:"
> +od -A d -t x1 $SCRATCH_MNT/bar
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/231.out b/tests/btrfs/231.out
> new file mode 100644
> index 00000000..def05769
> --- /dev/null
> +++ b/tests/btrfs/231.out
> @@ -0,0 +1,21 @@
> +QA output created by 231
> +wrote 1048576/1048576 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 524288/524288 bytes at offset 524288
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 524288/524288 bytes at offset 1111
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +File foo data:
> +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
> +*
> +0524288 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
> +*
> +1048576
> +File bar data:
> +0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> +*
> +0001104 00 00 00 00 00 00 00 ef ef ef ef ef ef ef ef ef
> +0001120 ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef
> +*
> +0525392 ef ef ef ef ef ef ef
> +0525399
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index a7c65983..9f63db69 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -233,3 +233,4 @@
> 228 auto quick volume
> 229 auto quick send clone
> 230 auto quick qgroup limit
> +231 auto quick compress rw
> --
> 2.28.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
2021-03-07 14:35 ` Eryu Guan
@ 2021-03-07 15:07 ` Filipe Manana
2021-03-07 15:19 ` Eryu Guan
0 siblings, 1 reply; 6+ messages in thread
From: Filipe Manana @ 2021-03-07 15:07 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs, Filipe Manana
On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan <guan@eryu.me> wrote:
>
> On Thu, Feb 11, 2021 at 05:01:18PM +0000, fdmanana@kernel.org wrote:
> > From: Filipe Manana <fdmanana@suse.com>
> >
> > Test cases where a direct IO write, with O_DSYNC, can not be done and has
> > to fallback to a buffered write.
> >
> > This is motivated by a regression that was introduced in kernel 5.10 by
> > commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> > fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> > DSYNC workaround").
> >
> > Signed-off-by: Filipe Manana <fdmanana@suse.com>
>
> Sorry for the late review..
>
> So this is supposed to fail with v5.10 kernel, right? But I got it
> passed
Because either you are testing with a patched 5.10.x kernel, or you
don't have CONFIG_BTRFS_ASSERT=y in your config.
The fix landed in 5.10.18:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.10.18&id=a6703c71153438d3ebdf58a75d53dd5e57b33095
>
> [root@fedoravm xfstests]# ./check -s btrfs btrfs/231
> SECTION -- btrfs
> RECREATING -- btrfs on /dev/mapper/testvg-lv1
> FSTYP -- btrfs
> PLATFORM -- Linux/x86_64 fedoravm 5.10.0 #6 SMP Sun Mar 7 22:25:35 CST 2021
> MKFS_OPTIONS -- /dev/mapper/testvg-lv2
> MOUNT_OPTIONS -- /dev/mapper/testvg-lv2 /mnt/scratch
>
> btrfs/231 13s ... 8s
> Ran: btrfs/231
> Passed all 1 tests
>
> SECTION -- btrfs
> =========================
> Ran: btrfs/231
> Passed all 1 tests
>
> > ---
> > tests/btrfs/231 | 61 +++++++++++++++++++++++++++++++++++++++++++++
> > tests/btrfs/231.out | 21 ++++++++++++++++
> > tests/btrfs/group | 1 +
> > 3 files changed, 83 insertions(+)
> > create mode 100755 tests/btrfs/231
> > create mode 100644 tests/btrfs/231.out
> >
> > diff --git a/tests/btrfs/231 b/tests/btrfs/231
> > new file mode 100755
> > index 00000000..9a404f57
> > --- /dev/null
> > +++ b/tests/btrfs/231
> > @@ -0,0 +1,61 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
> > +#
> > +# FS QA Test No. btrfs/231
> > +#
> > +# Test cases where a direct IO write, with O_DSYNC, can not be done and has to
> > +# fallback to a buffered write.
> > +#
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +tmp=/tmp/$$
> > +status=1 # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +. ./common/attr
> > +
> > +# real QA test starts here
> > +_supported_fs btrfs
> > +_require_scratch
> > +_require_odirect
> > +_require_chattr c
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs >>$seqres.full 2>&1
> > +_scratch_mount
> > +
> > +# First lets test with an attempt to write into a file range with compressed
> > +# extents.
> > +touch $SCRATCH_MNT/foo
> > +$CHATTR_PROG +c $SCRATCH_MNT/foo
>
> It's not so clear to me why writing into a compressed file is required,
> would you please add more comments?
The test is meant to test cases where we can deterministically make a
direct IO write fallback to buffered IO.
There are 2 such cases:
1) Attempting to write to an unaligned offset - this was the bug in
5.10 that resulted in a crash when CONFIG_BTRFS_ASSERT=y (default in
many distros, such as openSUSE).
2) Writing to a range that has compressed extents. This has nothing to
do with the 5.10 regression, I just added it since there's no existing
test that explicitly and deterministically triggers this.
So yes, I decided to add a test case for all possible cases of
direct IO falling back to buffered instead of adding one just to test
a regression (and help detecting any possible future regressions).
Thanks.
>
> Thanks,
> Eryu
>
> > +
> > +$XFS_IO_PROG -s -c "pwrite -S 0xab -b 1M 0 1M" $SCRATCH_MNT/foo | _filter_xfs_io
> > +# Now do the direct IO write...
> > +$XFS_IO_PROG -d -s -c "pwrite -S 0xcd 512K 512K" $SCRATCH_MNT/foo | _filter_xfs_io
> > +
> > +# Now try doing a write into an unaligned offset...
> > +$XFS_IO_PROG -f -d -s -c "pwrite -S 0xef 1111 512K" $SCRATCH_MNT/bar | _filter_xfs_io
> > +
> > +# Unmount, mount again, and verify we have the expected data.
> > +_scratch_cycle_mount
> > +
> > +echo "File foo data:"
> > +od -A d -t x1 $SCRATCH_MNT/foo
> > +echo "File bar data:"
> > +od -A d -t x1 $SCRATCH_MNT/bar
> > +
> > +status=0
> > +exit
> > diff --git a/tests/btrfs/231.out b/tests/btrfs/231.out
> > new file mode 100644
> > index 00000000..def05769
> > --- /dev/null
> > +++ b/tests/btrfs/231.out
> > @@ -0,0 +1,21 @@
> > +QA output created by 231
> > +wrote 1048576/1048576 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 524288/524288 bytes at offset 524288
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 524288/524288 bytes at offset 1111
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +File foo data:
> > +0000000 ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab
> > +*
> > +0524288 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
> > +*
> > +1048576
> > +File bar data:
> > +0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > +*
> > +0001104 00 00 00 00 00 00 00 ef ef ef ef ef ef ef ef ef
> > +0001120 ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef ef
> > +*
> > +0525392 ef ef ef ef ef ef ef
> > +0525399
> > diff --git a/tests/btrfs/group b/tests/btrfs/group
> > index a7c65983..9f63db69 100644
> > --- a/tests/btrfs/group
> > +++ b/tests/btrfs/group
> > @@ -233,3 +233,4 @@
> > 228 auto quick volume
> > 229 auto quick send clone
> > 230 auto quick qgroup limit
> > +231 auto quick compress rw
> > --
> > 2.28.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
2021-03-07 15:07 ` Filipe Manana
@ 2021-03-07 15:19 ` Eryu Guan
2021-03-10 10:48 ` Filipe Manana
0 siblings, 1 reply; 6+ messages in thread
From: Eryu Guan @ 2021-03-07 15:19 UTC (permalink / raw)
To: Filipe Manana; +Cc: fstests, linux-btrfs, Filipe Manana
On Sun, Mar 07, 2021 at 03:07:43PM +0000, Filipe Manana wrote:
> On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan <guan@eryu.me> wrote:
> >
> > On Thu, Feb 11, 2021 at 05:01:18PM +0000, fdmanana@kernel.org wrote:
> > > From: Filipe Manana <fdmanana@suse.com>
> > >
> > > Test cases where a direct IO write, with O_DSYNC, can not be done and has
> > > to fallback to a buffered write.
> > >
> > > This is motivated by a regression that was introduced in kernel 5.10 by
> > > commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> > > fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> > > DSYNC workaround").
> > >
> > > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> >
> > Sorry for the late review..
> >
> > So this is supposed to fail with v5.10 kernel, right? But I got it
> > passed
>
> Because either you are testing with a patched 5.10.x kernel, or you
> don't have CONFIG_BTRFS_ASSERT=y in your config.
> The fix landed in 5.10.18:
You're right, I don't have CONFIG_BTRFS_ASSERT=y. As the test dumps the
od output of the file content, so I thought the failure would be a data
corruption, and expected a od output diff failure.
>
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.10.18&id=a6703c71153438d3ebdf58a75d53dd5e57b33095
>
> >
> > [root@fedoravm xfstests]# ./check -s btrfs btrfs/231
> > SECTION -- btrfs
> > RECREATING -- btrfs on /dev/mapper/testvg-lv1
> > FSTYP -- btrfs
> > PLATFORM -- Linux/x86_64 fedoravm 5.10.0 #6 SMP Sun Mar 7 22:25:35 CST 2021
> > MKFS_OPTIONS -- /dev/mapper/testvg-lv2
> > MOUNT_OPTIONS -- /dev/mapper/testvg-lv2 /mnt/scratch
> >
> > btrfs/231 13s ... 8s
> > Ran: btrfs/231
> > Passed all 1 tests
> >
> > SECTION -- btrfs
> > =========================
> > Ran: btrfs/231
> > Passed all 1 tests
> >
> > > ---
> > > tests/btrfs/231 | 61 +++++++++++++++++++++++++++++++++++++++++++++
> > > tests/btrfs/231.out | 21 ++++++++++++++++
> > > tests/btrfs/group | 1 +
> > > 3 files changed, 83 insertions(+)
> > > create mode 100755 tests/btrfs/231
> > > create mode 100644 tests/btrfs/231.out
> > >
> > > diff --git a/tests/btrfs/231 b/tests/btrfs/231
> > > new file mode 100755
> > > index 00000000..9a404f57
> > > --- /dev/null
> > > +++ b/tests/btrfs/231
> > > @@ -0,0 +1,61 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
> > > +#
> > > +# FS QA Test No. btrfs/231
> > > +#
> > > +# Test cases where a direct IO write, with O_DSYNC, can not be done and has to
> > > +# fallback to a buffered write.
> > > +#
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +. ./common/attr
> > > +
> > > +# real QA test starts here
> > > +_supported_fs btrfs
> > > +_require_scratch
> > > +_require_odirect
> > > +_require_chattr c
> > > +
> > > +rm -f $seqres.full
> > > +
> > > +_scratch_mkfs >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +
> > > +# First lets test with an attempt to write into a file range with compressed
> > > +# extents.
> > > +touch $SCRATCH_MNT/foo
> > > +$CHATTR_PROG +c $SCRATCH_MNT/foo
> >
> > It's not so clear to me why writing into a compressed file is required,
> > would you please add more comments?
>
> The test is meant to test cases where we can deterministically make a
> direct IO write fallback to buffered IO.
> There are 2 such cases:
>
> 1) Attempting to write to an unaligned offset - this was the bug in
> 5.10 that resulted in a crash when CONFIG_BTRFS_ASSERT=y (default in
> many distros, such as openSUSE).
>
> 2) Writing to a range that has compressed extents. This has nothing to
> do with the 5.10 regression, I just added it since there's no existing
> test that explicitly and deterministically triggers this.
> So yes, I decided to add a test case for all possible cases of
> direct IO falling back to buffered instead of adding one just to test
> a regression (and help detecting any possible future regressions).
Sounds good, thanks!
Eryu
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
2021-03-07 15:19 ` Eryu Guan
@ 2021-03-10 10:48 ` Filipe Manana
2021-03-12 6:43 ` Eryu Guan
0 siblings, 1 reply; 6+ messages in thread
From: Filipe Manana @ 2021-03-10 10:48 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs, Filipe Manana
On Sun, Mar 7, 2021 at 3:24 PM Eryu Guan <guan@eryu.me> wrote:
>
> On Sun, Mar 07, 2021 at 03:07:43PM +0000, Filipe Manana wrote:
> > On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan <guan@eryu.me> wrote:
> > >
> > > On Thu, Feb 11, 2021 at 05:01:18PM +0000, fdmanana@kernel.org wrote:
> > > > From: Filipe Manana <fdmanana@suse.com>
> > > >
> > > > Test cases where a direct IO write, with O_DSYNC, can not be done and has
> > > > to fallback to a buffered write.
> > > >
> > > > This is motivated by a regression that was introduced in kernel 5.10 by
> > > > commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> > > > fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> > > > DSYNC workaround").
> > > >
> > > > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> > >
> > > Sorry for the late review..
> > >
> > > So this is supposed to fail with v5.10 kernel, right? But I got it
> > > passed
> >
> > Because either you are testing with a patched 5.10.x kernel, or you
> > don't have CONFIG_BTRFS_ASSERT=y in your config.
> > The fix landed in 5.10.18:
>
> You're right, I don't have CONFIG_BTRFS_ASSERT=y. As the test dumps the
> od output of the file content, so I thought the failure would be a data
> corruption, and expected a od output diff failure.
I see the test was not merged yet, do you expect me to update anything
in the patch?
Thanks.
>
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.10.18&id=a6703c71153438d3ebdf58a75d53dd5e57b33095
> >
> > >
> > > [root@fedoravm xfstests]# ./check -s btrfs btrfs/231
> > > SECTION -- btrfs
> > > RECREATING -- btrfs on /dev/mapper/testvg-lv1
> > > FSTYP -- btrfs
> > > PLATFORM -- Linux/x86_64 fedoravm 5.10.0 #6 SMP Sun Mar 7 22:25:35 CST 2021
> > > MKFS_OPTIONS -- /dev/mapper/testvg-lv2
> > > MOUNT_OPTIONS -- /dev/mapper/testvg-lv2 /mnt/scratch
> > >
> > > btrfs/231 13s ... 8s
> > > Ran: btrfs/231
> > > Passed all 1 tests
> > >
> > > SECTION -- btrfs
> > > =========================
> > > Ran: btrfs/231
> > > Passed all 1 tests
> > >
> > > > ---
> > > > tests/btrfs/231 | 61 +++++++++++++++++++++++++++++++++++++++++++++
> > > > tests/btrfs/231.out | 21 ++++++++++++++++
> > > > tests/btrfs/group | 1 +
> > > > 3 files changed, 83 insertions(+)
> > > > create mode 100755 tests/btrfs/231
> > > > create mode 100644 tests/btrfs/231.out
> > > >
> > > > diff --git a/tests/btrfs/231 b/tests/btrfs/231
> > > > new file mode 100755
> > > > index 00000000..9a404f57
> > > > --- /dev/null
> > > > +++ b/tests/btrfs/231
> > > > @@ -0,0 +1,61 @@
> > > > +#! /bin/bash
> > > > +# SPDX-License-Identifier: GPL-2.0
> > > > +# Copyright (C) 2021 SUSE Linux Products GmbH. All Rights Reserved.
> > > > +#
> > > > +# FS QA Test No. btrfs/231
> > > > +#
> > > > +# Test cases where a direct IO write, with O_DSYNC, can not be done and has to
> > > > +# fallback to a buffered write.
> > > > +#
> > > > +seq=`basename $0`
> > > > +seqres=$RESULT_DIR/$seq
> > > > +echo "QA output created by $seq"
> > > > +
> > > > +tmp=/tmp/$$
> > > > +status=1 # failure is the default!
> > > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > > +
> > > > +_cleanup()
> > > > +{
> > > > + cd /
> > > > + rm -f $tmp.*
> > > > +}
> > > > +
> > > > +# get standard environment, filters and checks
> > > > +. ./common/rc
> > > > +. ./common/filter
> > > > +. ./common/attr
> > > > +
> > > > +# real QA test starts here
> > > > +_supported_fs btrfs
> > > > +_require_scratch
> > > > +_require_odirect
> > > > +_require_chattr c
> > > > +
> > > > +rm -f $seqres.full
> > > > +
> > > > +_scratch_mkfs >>$seqres.full 2>&1
> > > > +_scratch_mount
> > > > +
> > > > +# First lets test with an attempt to write into a file range with compressed
> > > > +# extents.
> > > > +touch $SCRATCH_MNT/foo
> > > > +$CHATTR_PROG +c $SCRATCH_MNT/foo
> > >
> > > It's not so clear to me why writing into a compressed file is required,
> > > would you please add more comments?
> >
> > The test is meant to test cases where we can deterministically make a
> > direct IO write fallback to buffered IO.
> > There are 2 such cases:
> >
> > 1) Attempting to write to an unaligned offset - this was the bug in
> > 5.10 that resulted in a crash when CONFIG_BTRFS_ASSERT=y (default in
> > many distros, such as openSUSE).
> >
> > 2) Writing to a range that has compressed extents. This has nothing to
> > do with the 5.10 regression, I just added it since there's no existing
> > test that explicitly and deterministically triggers this.
> > So yes, I decided to add a test case for all possible cases of
> > direct IO falling back to buffered instead of adding one just to test
> > a regression (and help detecting any possible future regressions).
>
> Sounds good, thanks!
>
> Eryu
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write
2021-03-10 10:48 ` Filipe Manana
@ 2021-03-12 6:43 ` Eryu Guan
0 siblings, 0 replies; 6+ messages in thread
From: Eryu Guan @ 2021-03-12 6:43 UTC (permalink / raw)
To: Filipe Manana; +Cc: fstests, linux-btrfs, Filipe Manana
On Wed, Mar 10, 2021 at 10:48:35AM +0000, Filipe Manana wrote:
> On Sun, Mar 7, 2021 at 3:24 PM Eryu Guan <guan@eryu.me> wrote:
> >
> > On Sun, Mar 07, 2021 at 03:07:43PM +0000, Filipe Manana wrote:
> > > On Sun, Mar 7, 2021 at 2:41 PM Eryu Guan <guan@eryu.me> wrote:
> > > >
> > > > On Thu, Feb 11, 2021 at 05:01:18PM +0000, fdmanana@kernel.org wrote:
> > > > > From: Filipe Manana <fdmanana@suse.com>
> > > > >
> > > > > Test cases where a direct IO write, with O_DSYNC, can not be done and has
> > > > > to fallback to a buffered write.
> > > > >
> > > > > This is motivated by a regression that was introduced in kernel 5.10 by
> > > > > commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround")) and was
> > > > > fixed in kernel 5.11 by commit ecfdc08b8cc65d ("btrfs: remove dio iomap
> > > > > DSYNC workaround").
> > > > >
> > > > > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> > > >
> > > > Sorry for the late review..
> > > >
> > > > So this is supposed to fail with v5.10 kernel, right? But I got it
> > > > passed
> > >
> > > Because either you are testing with a patched 5.10.x kernel, or you
> > > don't have CONFIG_BTRFS_ASSERT=y in your config.
> > > The fix landed in 5.10.18:
> >
> > You're right, I don't have CONFIG_BTRFS_ASSERT=y. As the test dumps the
> > od output of the file content, so I thought the failure would be a data
> > corruption, and expected a od output diff failure.
>
> I see the test was not merged yet, do you expect me to update anything
> in the patch?
Sorry, it was lost in my to-review queue.. But I'd be great if you could
add more descriptions about the CONFIG_BTRFS_ASSERT and the compress
case either in commit log or in test description.
Thanks,
Eryu
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-03-12 6:44 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-11 17:01 [PATCH] btrfs: add test for cases when a dio write has to fallback to a buffered write fdmanana
2021-03-07 14:35 ` Eryu Guan
2021-03-07 15:07 ` Filipe Manana
2021-03-07 15:19 ` Eryu Guan
2021-03-10 10:48 ` Filipe Manana
2021-03-12 6:43 ` Eryu Guan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.