* btrfs read repair: more tests
@ 2022-06-22 4:58 Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Hi all,
this series adds a few more btrfs read repair tests. Still all for user
data as I haven't managed to successfully inject corruption into metadata
yet.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 4:58 ` [PATCH 2/4] btrfs; add a test for impossible repair cases Christoph Hellwig
` (3 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
The -b flag without an argument is not supported, so remove it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
common/btrfs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/common/btrfs b/common/btrfs
index c7058918..14ad890e 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -537,7 +537,7 @@ _btrfs_get_physical()
_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
- $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
$BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
$AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
}
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 2/4] btrfs; add a test for impossible repair cases
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 4:58 ` [PATCH 3/4] btrfs: test checker pattern corruption on raid10 Christoph Hellwig
` (2 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Verify that a repair attempt that can't succeed because all copies are
bad returns a proper I/O error and doesn't cause any deadlocks.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/268 | 62 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/268.out | 7 +++++
2 files changed, 69 insertions(+)
create mode 100755 tests/btrfs/268
create mode 100644 tests/btrfs/268.out
diff --git a/tests/btrfs/268 b/tests/btrfs/268
new file mode 100755
index 00000000..5043bb02
--- /dev/null
+++ b/tests/btrfs/268
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 268
+#
+# Test that btrfs read repair on a raid1 profile won't loop forever if data
+# is corrupted on both mirrors and can't be recovered.
+#
+
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_odirect
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 2
+_scratch_dev_pool_get 2
+
+echo "step 1......mkfs.btrfs"
+
+_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical2=$(_btrfs_get_physical ${logical} 2)
+devpath2=$(_btrfs_get_device_path ${logical} 2)
+
+physical3=$(_btrfs_get_physical ${logical} 3)
+devpath3=$(_btrfs_get_device_path ${logical} 3)
+
+_scratch_unmount
+
+echo "step 2......corrupt file extent"
+$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical1 4K" \
+ $devpath1 > /dev/null
+$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical2 4K" \
+ $devpath2 > /dev/null
+
+_scratch_mount
+
+echo "step 3......try to repair"
+$XFS_IO_PROG -d -c "pread -b 4K 0 4K" $SCRATCH_MNT/foobar
+
+_scratch_unmount
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/268.out b/tests/btrfs/268.out
new file mode 100644
index 00000000..d28b37b3
--- /dev/null
+++ b/tests/btrfs/268.out
@@ -0,0 +1,7 @@
+QA output created by 268
+step 1......mkfs.btrfs
+wrote 262144/262144 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......try to repair
+pread: Input/output error
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/4] btrfs: test checker pattern corruption on raid10
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
2022-06-22 4:58 ` [PATCH 2/4] btrfs; add a test for impossible repair cases Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
2022-06-29 14:00 ` btrfs read repair: more tests Zorro Lang
4 siblings, 0 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Check read repair for the case where the corruption is spread over
the different legs of a raid10 set.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/269 | 73 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/269.out | 41 +++++++++++++++++++++++++
2 files changed, 114 insertions(+)
create mode 100755 tests/btrfs/269
create mode 100644 tests/btrfs/269.out
diff --git a/tests/btrfs/269 b/tests/btrfs/269
new file mode 100755
index 00000000..3c11da3f
--- /dev/null
+++ b/tests/btrfs/269
@@ -0,0 +1,73 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 269
+#
+# Test btrfs read repair over tricky stripe boundaries on the raid10 profile:
+#
+# | stripe 0 | stripe 2
+# --------------------------------
+# mirror 1 | I/O FAIL | GOOD
+# mirror 2 | GOOD | CSUM FAIL
+#
+
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_odirect
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 4
+_scratch_dev_pool_get 4
+
+echo "step 1......mkfs.btrfs"
+
+_scratch_pool_mkfs "-d raid10 -b 1G" >>$seqres.full 2>&1
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical4=$(_btrfs_get_physical ${logical} 3)
+devpath4=$(_btrfs_get_device_path ${logical} 3)
+
+_scratch_unmount
+
+echo "step 2......corrupt file extent"
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical1 64K" \
+ $devpath1 > /dev/null
+$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical4 + 65536)) 64K" \
+ $devpath4 > /dev/null
+
+_scratch_mount
+
+echo "step 3......repair the bad copy"
+
+_btrfs_direct_read_on_mirror 0 2 "$SCRATCH_MNT/foobar" 0 128K
+_btrfs_direct_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $((physical4 + 65536)) 512" $devpath4 |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/269.out b/tests/btrfs/269.out
new file mode 100644
index 00000000..d3ad7f07
--- /dev/null
+++ b/tests/btrfs/269.out
@@ -0,0 +1,41 @@
+QA output created by 269
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
` (2 preceding siblings ...)
2022-06-22 4:58 ` [PATCH 3/4] btrfs: test checker pattern corruption on raid10 Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 9:21 ` Christoph Hellwig
2022-06-29 14:00 ` btrfs read repair: more tests Zorro Lang
4 siblings, 1 reply; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Exercise read repair on a corrupted compressed sector.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/270 | 79 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/270.out | 41 +++++++++++++++++++++++
2 files changed, 120 insertions(+)
create mode 100755 tests/btrfs/270
create mode 100644 tests/btrfs/270.out
diff --git a/tests/btrfs/270 b/tests/btrfs/270
new file mode 100755
index 00000000..4229a02c
--- /dev/null
+++ b/tests/btrfs/270
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+#
+# FS QA Test 270
+#
+# Regression test for btrfs buffered read repair of compressed data.
+#
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 2
+_scratch_dev_pool_get 2
+
+get_physical()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
+}
+
+get_devid()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
+}
+
+get_device_path()
+{
+ local devid=$1
+ echo "$SCRATCH_DEV_POOL" | $AWK_PROG "{print \$$devid}"
+}
+
+
+echo "step 1......mkfs.btrfs"
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
+_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -S 0xaa -W -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | _filter_xfs_io_offset
+
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+physical=$(get_physical ${logical_in_btrfs} 1)
+devid=$(get_devid ${logical_in_btrfs} 1)
+devpath=$(get_device_path ${devid})
+
+_scratch_unmount
+echo "step 2......corrupt file extent"
+echo " corrupt stripe #1, devid $devid devpath $devpath physical $physical" \
+ >> $seqres.full
+$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $physical 64K" $devpath > /dev/null
+
+_scratch_mount
+
+echo "step 3......repair the bad copy"
+_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -c "pread -v -b 512 $physical 512" $devpath |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/270.out b/tests/btrfs/270.out
new file mode 100644
index 00000000..53a80692
--- /dev/null
+++ b/tests/btrfs/270.out
@@ -0,0 +1,41 @@
+QA output created by 270
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
@ 2022-06-22 9:21 ` Christoph Hellwig
2022-06-22 12:41 ` Zorro Lang
0 siblings, 1 reply; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 9:21 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
So while this test properly documents the current behavior, it failed
to grasp how broken that behavior ist: the current read repair code
writes back the uncompressed data to disk even for a compressed extent,
and this test verified the behavior.
Below is a correct test that fails on current mainline. I'll send fixes
but right now they depend on a lot of prep work.
---
From 6b6c505f75c6c7cc15359f14053b1db43e3d3091 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch@lst.de>
Date: Wed, 22 Jun 2022 06:55:36 +0200
Subject: btrfs: test read repair on a corrupted compressed extent
Exercise read repair on a corrupted compressed sector.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/270 | 82 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/270.out | 7 ++++
2 files changed, 89 insertions(+)
create mode 100755 tests/btrfs/270
create mode 100644 tests/btrfs/270.out
diff --git a/tests/btrfs/270 b/tests/btrfs/270
new file mode 100755
index 00000000..5b73fb15
--- /dev/null
+++ b/tests/btrfs/270
@@ -0,0 +1,82 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+#
+# FS QA Test 270
+#
+# Regression test for btrfs buffered read repair of compressed data.
+#
+. ./common/preamble
+_begin_fstest auto quick read_repair compress
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 2
+_scratch_dev_pool_get 2
+
+get_physical()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
+}
+
+get_devid()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
+}
+
+get_device_path()
+{
+ local devid=$1
+ echo "$SCRATCH_DEV_POOL" | $AWK_PROG "{print \$$devid}"
+}
+
+
+echo "step 1......mkfs.btrfs"
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
+_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -S 0xaa -W -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | _filter_xfs_io_offset
+
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+physical=$(get_physical ${logical_in_btrfs} 1)
+devid=$(get_devid ${logical_in_btrfs} 1)
+devpath=$(get_device_path ${devid})
+
+_scratch_unmount
+echo "step 2......corrupt file extent"
+echo " corrupt stripe #1, devid $devid devpath $devpath physical $physical" \
+ >> $seqres.full
+dd if=$devpath of=$TEST_DIR/$seq.dump.good skip=$physical bs=1 count=4096 \
+ 2>/dev/null
+$XFS_IO_PROG -c "pwrite -S 0xbb -b 4K $physical 4K" $devpath > /dev/null
+
+_scratch_mount
+
+echo "step 3......repair the bad copy"
+_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+dd if=$devpath of=$TEST_DIR/$seq.dump skip=$physical bs=1 count=4096 \
+ 2>/dev/null
+cmp -bl $TEST_DIR/$seq.dump.good $TEST_DIR/$seq.dump
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/270.out b/tests/btrfs/270.out
new file mode 100644
index 00000000..6d744c02
--- /dev/null
+++ b/tests/btrfs/270.out
@@ -0,0 +1,7 @@
+QA output created by 270
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 9:21 ` Christoph Hellwig
@ 2022-06-22 12:41 ` Zorro Lang
2022-06-22 13:07 ` Christoph Hellwig
0 siblings, 1 reply; 10+ messages in thread
From: Zorro Lang @ 2022-06-22 12:41 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs
On Wed, Jun 22, 2022 at 11:21:40AM +0200, Christoph Hellwig wrote:
> So while this test properly documents the current behavior, it failed
> to grasp how broken that behavior ist: the current read repair code
> writes back the uncompressed data to disk even for a compressed extent,
> and this test verified the behavior.
>
> Below is a correct test that fails on current mainline. I'll send fixes
> but right now they depend on a lot of prep work.
>
> ---
> From 6b6c505f75c6c7cc15359f14053b1db43e3d3091 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch@lst.de>
> Date: Wed, 22 Jun 2022 06:55:36 +0200
> Subject: btrfs: test read repair on a corrupted compressed extent
>
> Exercise read repair on a corrupted compressed sector.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> tests/btrfs/270 | 82 +++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/270.out | 7 ++++
> 2 files changed, 89 insertions(+)
> create mode 100755 tests/btrfs/270
> create mode 100644 tests/btrfs/270.out
>
> diff --git a/tests/btrfs/270 b/tests/btrfs/270
> new file mode 100755
> index 00000000..5b73fb15
> --- /dev/null
> +++ b/tests/btrfs/270
> @@ -0,0 +1,82 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
^^^^^^^^^^^
Is it a wrong copy&paste ?
> +#
> +# FS QA Test 270
> +#
> +# Regression test for btrfs buffered read repair of compressed data.
If this's a regression test, I'd like to see the fix be reviewed/acked
at first :)
Thanks,
Zorro
> +#
> +. ./common/preamble
> +_begin_fstest auto quick read_repair compress
> +
> +. ./common/filter
> +
> +_supported_fs btrfs
> +_require_btrfs_command inspect-internal dump-tree
> +_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
> +_require_scratch_dev_pool 2
> +_scratch_dev_pool_get 2
> +
> +get_physical()
> +{
> + local logical=$1
> + local stripe=$2
> + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
> + grep $logical -A 6 | \
> + $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
> +}
> +
> +get_devid()
> +{
> + local logical=$1
> + local stripe=$2
> + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
> + grep $logical -A 6 | \
> + $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
> +}
> +
> +get_device_path()
> +{
> + local devid=$1
> + echo "$SCRATCH_DEV_POOL" | $AWK_PROG "{print \$$devid}"
> +}
> +
> +
> +echo "step 1......mkfs.btrfs"
> +_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
> +_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
> +_scratch_mount -ocompress
> +
> +# Create a file with all data being compressed
> +$XFS_IO_PROG -f -c "pwrite -S 0xaa -W -b 128K 0 128K" \
> + "$SCRATCH_MNT/foobar" | _filter_xfs_io_offset
> +
> +logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> +physical=$(get_physical ${logical_in_btrfs} 1)
> +devid=$(get_devid ${logical_in_btrfs} 1)
> +devpath=$(get_device_path ${devid})
> +
> +_scratch_unmount
> +echo "step 2......corrupt file extent"
> +echo " corrupt stripe #1, devid $devid devpath $devpath physical $physical" \
> + >> $seqres.full
> +dd if=$devpath of=$TEST_DIR/$seq.dump.good skip=$physical bs=1 count=4096 \
> + 2>/dev/null
> +$XFS_IO_PROG -c "pwrite -S 0xbb -b 4K $physical 4K" $devpath > /dev/null
> +
> +_scratch_mount
> +
> +echo "step 3......repair the bad copy"
> +_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
> +
> +_scratch_unmount
> +
> +echo "step 4......check if the repair worked"
> +dd if=$devpath of=$TEST_DIR/$seq.dump skip=$physical bs=1 count=4096 \
> + 2>/dev/null
> +cmp -bl $TEST_DIR/$seq.dump.good $TEST_DIR/$seq.dump
> +
> +_scratch_dev_pool_put
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/270.out b/tests/btrfs/270.out
> new file mode 100644
> index 00000000..6d744c02
> --- /dev/null
> +++ b/tests/btrfs/270.out
> @@ -0,0 +1,7 @@
> +QA output created by 270
> +step 1......mkfs.btrfs
> +wrote 131072/131072 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +step 2......corrupt file extent
> +step 3......repair the bad copy
> +step 4......check if the repair worked
> --
> 2.30.2
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 12:41 ` Zorro Lang
@ 2022-06-22 13:07 ` Christoph Hellwig
2022-06-24 2:25 ` Zorro Lang
0 siblings, 1 reply; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 13:07 UTC (permalink / raw)
To: Zorro Lang; +Cc: Christoph Hellwig, fstests, linux-btrfs
On Wed, Jun 22, 2022 at 08:41:18PM +0800, Zorro Lang wrote:
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> ^^^^^^^^^^^
> Is it a wrong copy&paste ?
A lot of the test is copied from btrfs/141, so I wanted to keep the
copyright intact.
> > +#
> > +# FS QA Test 270
> > +#
> > +# Regression test for btrfs buffered read repair of compressed data.
>
> If this's a regression test, I'd like to see the fix be reviewed/acked
> at first :)
Heh. Actually I'm not sure it is a regression, this was copy and
pasted as well. I think it actually has been broken since day 1.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 13:07 ` Christoph Hellwig
@ 2022-06-24 2:25 ` Zorro Lang
0 siblings, 0 replies; 10+ messages in thread
From: Zorro Lang @ 2022-06-24 2:25 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs
On Wed, Jun 22, 2022 at 03:07:45PM +0200, Christoph Hellwig wrote:
> On Wed, Jun 22, 2022 at 08:41:18PM +0800, Zorro Lang wrote:
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> > ^^^^^^^^^^^
> > Is it a wrong copy&paste ?
>
> A lot of the test is copied from btrfs/141, so I wanted to keep the
> copyright intact.
Oh, there're some cases copy from old cases in fstests, but they are changed
a little to have different coverage. Generally they metioned they copy from
which one case (number). I don't mind if you'd like to keep original
Copyright name, but the copyright time is old for a new test case, I don't
know if we'd better to update it. Hmm... sorry, not good at these copyright
things...
>
> > > +#
> > > +# FS QA Test 270
> > > +#
> > > +# Regression test for btrfs buffered read repair of compressed data.
> >
> > If this's a regression test, I'd like to see the fix be reviewed/acked
> > at first :)
>
> Heh. Actually I'm not sure it is a regression, this was copy and
> pasted as well. I think it actually has been broken since day 1.
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: btrfs read repair: more tests
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
` (3 preceding siblings ...)
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
@ 2022-06-29 14:00 ` Zorro Lang
4 siblings, 0 replies; 10+ messages in thread
From: Zorro Lang @ 2022-06-29 14:00 UTC (permalink / raw)
To: linux-btrfs; +Cc: fstests, Christoph Hellwig
On Wed, Jun 22, 2022 at 06:58:40AM +0200, Christoph Hellwig wrote:
> Hi all,
>
> this series adds a few more btrfs read repair tests. Still all for user
> data as I haven't managed to successfully inject corruption into metadata
> yet.
This patchset is good to me (from fstests side). If there's not more review
points or objections from btrfs list, I'll merge it. Welcome more review.
Thanks,
Zorro
>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-06-29 14:00 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
2022-06-22 4:58 ` [PATCH 2/4] btrfs; add a test for impossible repair cases Christoph Hellwig
2022-06-22 4:58 ` [PATCH 3/4] btrfs: test checker pattern corruption on raid10 Christoph Hellwig
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
2022-06-22 9:21 ` Christoph Hellwig
2022-06-22 12:41 ` Zorro Lang
2022-06-22 13:07 ` Christoph Hellwig
2022-06-24 2:25 ` Zorro Lang
2022-06-29 14:00 ` btrfs read repair: more tests Zorro Lang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).