* btrfs read repair: more tests
@ 2022-06-22 4:58 Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Hi all,
this series adds a few more btrfs read repair tests. Still all for user
data as I haven't managed to successfully inject corruption into metadata
yet.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 4:58 ` [PATCH 2/4] btrfs; add a test for impossible repair cases Christoph Hellwig
` (3 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
The -b flag without an argument is not supported, so remove it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
common/btrfs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/common/btrfs b/common/btrfs
index c7058918..14ad890e 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -537,7 +537,7 @@ _btrfs_get_physical()
_require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
- $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
$BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
$AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
}
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 2/4] btrfs; add a test for impossible repair cases
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 4:58 ` [PATCH 3/4] btrfs: test checker pattern corruption on raid10 Christoph Hellwig
` (2 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Verify that a repair attempt that can't succeed because all copies are
bad returns a proper I/O error and doesn't cause any deadlocks.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/268 | 62 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/268.out | 7 +++++
2 files changed, 69 insertions(+)
create mode 100755 tests/btrfs/268
create mode 100644 tests/btrfs/268.out
diff --git a/tests/btrfs/268 b/tests/btrfs/268
new file mode 100755
index 00000000..5043bb02
--- /dev/null
+++ b/tests/btrfs/268
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 268
+#
+# Test that btrfs read repair on a raid1 profile won't loop forever if data
+# is corrupted on both mirrors and can't be recovered.
+#
+
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_odirect
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 2
+_scratch_dev_pool_get 2
+
+echo "step 1......mkfs.btrfs"
+
+_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical2=$(_btrfs_get_physical ${logical} 2)
+devpath2=$(_btrfs_get_device_path ${logical} 2)
+
+physical3=$(_btrfs_get_physical ${logical} 3)
+devpath3=$(_btrfs_get_device_path ${logical} 3)
+
+_scratch_unmount
+
+echo "step 2......corrupt file extent"
+$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical1 4K" \
+ $devpath1 > /dev/null
+$XFS_IO_PROG -d -c "pwrite -S 0xba -b 4K $physical2 4K" \
+ $devpath2 > /dev/null
+
+_scratch_mount
+
+echo "step 3......try to repair"
+$XFS_IO_PROG -d -c "pread -b 4K 0 4K" $SCRATCH_MNT/foobar
+
+_scratch_unmount
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/268.out b/tests/btrfs/268.out
new file mode 100644
index 00000000..d28b37b3
--- /dev/null
+++ b/tests/btrfs/268.out
@@ -0,0 +1,7 @@
+QA output created by 268
+step 1......mkfs.btrfs
+wrote 262144/262144 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......try to repair
+pread: Input/output error
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/4] btrfs: test checker pattern corruption on raid10
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
2022-06-22 4:58 ` [PATCH 2/4] btrfs; add a test for impossible repair cases Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
2022-06-29 14:00 ` btrfs read repair: more tests Zorro Lang
4 siblings, 0 replies; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Check read repair for the case where the corruption is spread over
the different legs of a raid10 set.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/269 | 73 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/269.out | 41 +++++++++++++++++++++++++
2 files changed, 114 insertions(+)
create mode 100755 tests/btrfs/269
create mode 100644 tests/btrfs/269.out
diff --git a/tests/btrfs/269 b/tests/btrfs/269
new file mode 100755
index 00000000..3c11da3f
--- /dev/null
+++ b/tests/btrfs/269
@@ -0,0 +1,73 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 269
+#
+# Test btrfs read repair over tricky stripe boundaries on the raid10 profile:
+#
+# | stripe 0 | stripe 2
+# --------------------------------
+# mirror 1 | I/O FAIL | GOOD
+# mirror 2 | GOOD | CSUM FAIL
+#
+
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_odirect
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 4
+_scratch_dev_pool_get 4
+
+echo "step 1......mkfs.btrfs"
+
+_scratch_pool_mkfs "-d raid10 -b 1G" >>$seqres.full 2>&1
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical4=$(_btrfs_get_physical ${logical} 3)
+devpath4=$(_btrfs_get_device_path ${logical} 3)
+
+_scratch_unmount
+
+echo "step 2......corrupt file extent"
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical1 64K" \
+ $devpath1 > /dev/null
+$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical4 + 65536)) 64K" \
+ $devpath4 > /dev/null
+
+_scratch_mount
+
+echo "step 3......repair the bad copy"
+
+_btrfs_direct_read_on_mirror 0 2 "$SCRATCH_MNT/foobar" 0 128K
+_btrfs_direct_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $((physical4 + 65536)) 512" $devpath4 |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/269.out b/tests/btrfs/269.out
new file mode 100644
index 00000000..d3ad7f07
--- /dev/null
+++ b/tests/btrfs/269.out
@@ -0,0 +1,41 @@
+QA output created by 269
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
` (2 preceding siblings ...)
2022-06-22 4:58 ` [PATCH 3/4] btrfs: test checker pattern corruption on raid10 Christoph Hellwig
@ 2022-06-22 4:58 ` Christoph Hellwig
2022-06-22 9:21 ` Christoph Hellwig
2022-06-29 14:00 ` btrfs read repair: more tests Zorro Lang
4 siblings, 1 reply; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 4:58 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Exercise read repair on a corrupted compressed sector.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/270 | 79 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/270.out | 41 +++++++++++++++++++++++
2 files changed, 120 insertions(+)
create mode 100755 tests/btrfs/270
create mode 100644 tests/btrfs/270.out
diff --git a/tests/btrfs/270 b/tests/btrfs/270
new file mode 100755
index 00000000..4229a02c
--- /dev/null
+++ b/tests/btrfs/270
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+#
+# FS QA Test 270
+#
+# Regression test for btrfs buffered read repair of compressed data.
+#
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 2
+_scratch_dev_pool_get 2
+
+get_physical()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
+}
+
+get_devid()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
+}
+
+get_device_path()
+{
+ local devid=$1
+ echo "$SCRATCH_DEV_POOL" | $AWK_PROG "{print \$$devid}"
+}
+
+
+echo "step 1......mkfs.btrfs"
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
+_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -S 0xaa -W -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | _filter_xfs_io_offset
+
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+physical=$(get_physical ${logical_in_btrfs} 1)
+devid=$(get_devid ${logical_in_btrfs} 1)
+devpath=$(get_device_path ${devid})
+
+_scratch_unmount
+echo "step 2......corrupt file extent"
+echo " corrupt stripe #1, devid $devid devpath $devpath physical $physical" \
+ >> $seqres.full
+$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $physical 64K" $devpath > /dev/null
+
+_scratch_mount
+
+echo "step 3......repair the bad copy"
+_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -c "pread -v -b 512 $physical 512" $devpath |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/270.out b/tests/btrfs/270.out
new file mode 100644
index 00000000..53a80692
--- /dev/null
+++ b/tests/btrfs/270.out
@@ -0,0 +1,41 @@
+QA output created by 270
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
@ 2022-06-22 9:21 ` Christoph Hellwig
2022-06-22 12:41 ` Zorro Lang
0 siblings, 1 reply; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 9:21 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
So while this test properly documents the current behavior, it failed
to grasp how broken that behavior ist: the current read repair code
writes back the uncompressed data to disk even for a compressed extent,
and this test verified the behavior.
Below is a correct test that fails on current mainline. I'll send fixes
but right now they depend on a lot of prep work.
---
From 6b6c505f75c6c7cc15359f14053b1db43e3d3091 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch@lst.de>
Date: Wed, 22 Jun 2022 06:55:36 +0200
Subject: btrfs: test read repair on a corrupted compressed extent
Exercise read repair on a corrupted compressed sector.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/270 | 82 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/270.out | 7 ++++
2 files changed, 89 insertions(+)
create mode 100755 tests/btrfs/270
create mode 100644 tests/btrfs/270.out
diff --git a/tests/btrfs/270 b/tests/btrfs/270
new file mode 100755
index 00000000..5b73fb15
--- /dev/null
+++ b/tests/btrfs/270
@@ -0,0 +1,82 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+#
+# FS QA Test 270
+#
+# Regression test for btrfs buffered read repair of compressed data.
+#
+. ./common/preamble
+_begin_fstest auto quick read_repair compress
+
+. ./common/filter
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
+_require_scratch_dev_pool 2
+_scratch_dev_pool_get 2
+
+get_physical()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
+}
+
+get_devid()
+{
+ local logical=$1
+ local stripe=$2
+ $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+ grep $logical -A 6 | \
+ $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
+}
+
+get_device_path()
+{
+ local devid=$1
+ echo "$SCRATCH_DEV_POOL" | $AWK_PROG "{print \$$devid}"
+}
+
+
+echo "step 1......mkfs.btrfs"
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
+_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
+_scratch_mount -ocompress
+
+# Create a file with all data being compressed
+$XFS_IO_PROG -f -c "pwrite -S 0xaa -W -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | _filter_xfs_io_offset
+
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+physical=$(get_physical ${logical_in_btrfs} 1)
+devid=$(get_devid ${logical_in_btrfs} 1)
+devpath=$(get_device_path ${devid})
+
+_scratch_unmount
+echo "step 2......corrupt file extent"
+echo " corrupt stripe #1, devid $devid devpath $devpath physical $physical" \
+ >> $seqres.full
+dd if=$devpath of=$TEST_DIR/$seq.dump.good skip=$physical bs=1 count=4096 \
+ 2>/dev/null
+$XFS_IO_PROG -c "pwrite -S 0xbb -b 4K $physical 4K" $devpath > /dev/null
+
+_scratch_mount
+
+echo "step 3......repair the bad copy"
+_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+dd if=$devpath of=$TEST_DIR/$seq.dump skip=$physical bs=1 count=4096 \
+ 2>/dev/null
+cmp -bl $TEST_DIR/$seq.dump.good $TEST_DIR/$seq.dump
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/270.out b/tests/btrfs/270.out
new file mode 100644
index 00000000..6d744c02
--- /dev/null
+++ b/tests/btrfs/270.out
@@ -0,0 +1,7 @@
+QA output created by 270
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 9:21 ` Christoph Hellwig
@ 2022-06-22 12:41 ` Zorro Lang
2022-06-22 13:07 ` Christoph Hellwig
0 siblings, 1 reply; 10+ messages in thread
From: Zorro Lang @ 2022-06-22 12:41 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs
On Wed, Jun 22, 2022 at 11:21:40AM +0200, Christoph Hellwig wrote:
> So while this test properly documents the current behavior, it failed
> to grasp how broken that behavior ist: the current read repair code
> writes back the uncompressed data to disk even for a compressed extent,
> and this test verified the behavior.
>
> Below is a correct test that fails on current mainline. I'll send fixes
> but right now they depend on a lot of prep work.
>
> ---
> From 6b6c505f75c6c7cc15359f14053b1db43e3d3091 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch@lst.de>
> Date: Wed, 22 Jun 2022 06:55:36 +0200
> Subject: btrfs: test read repair on a corrupted compressed extent
>
> Exercise read repair on a corrupted compressed sector.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> tests/btrfs/270 | 82 +++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/270.out | 7 ++++
> 2 files changed, 89 insertions(+)
> create mode 100755 tests/btrfs/270
> create mode 100644 tests/btrfs/270.out
>
> diff --git a/tests/btrfs/270 b/tests/btrfs/270
> new file mode 100755
> index 00000000..5b73fb15
> --- /dev/null
> +++ b/tests/btrfs/270
> @@ -0,0 +1,82 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
^^^^^^^^^^^
Is it a wrong copy&paste ?
> +#
> +# FS QA Test 270
> +#
> +# Regression test for btrfs buffered read repair of compressed data.
If this's a regression test, I'd like to see the fix be reviewed/acked
at first :)
Thanks,
Zorro
> +#
> +. ./common/preamble
> +_begin_fstest auto quick read_repair compress
> +
> +. ./common/filter
> +
> +_supported_fs btrfs
> +_require_btrfs_command inspect-internal dump-tree
> +_require_non_zoned_device "${SCRATCH_DEV}" # no overwrites on zoned devices
> +_require_scratch_dev_pool 2
> +_scratch_dev_pool_get 2
> +
> +get_physical()
> +{
> + local logical=$1
> + local stripe=$2
> + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
> + grep $logical -A 6 | \
> + $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
> +}
> +
> +get_devid()
> +{
> + local logical=$1
> + local stripe=$2
> + $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
> + grep $logical -A 6 | \
> + $AWK_PROG "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
> +}
> +
> +get_device_path()
> +{
> + local devid=$1
> + echo "$SCRATCH_DEV_POOL" | $AWK_PROG "{print \$$devid}"
> +}
> +
> +
> +echo "step 1......mkfs.btrfs"
> +_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
> +_scratch_pool_mkfs "-d raid1 -b 1G" >>$seqres.full 2>&1
> +_scratch_mount -ocompress
> +
> +# Create a file with all data being compressed
> +$XFS_IO_PROG -f -c "pwrite -S 0xaa -W -b 128K 0 128K" \
> + "$SCRATCH_MNT/foobar" | _filter_xfs_io_offset
> +
> +logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> +physical=$(get_physical ${logical_in_btrfs} 1)
> +devid=$(get_devid ${logical_in_btrfs} 1)
> +devpath=$(get_device_path ${devid})
> +
> +_scratch_unmount
> +echo "step 2......corrupt file extent"
> +echo " corrupt stripe #1, devid $devid devpath $devpath physical $physical" \
> + >> $seqres.full
> +dd if=$devpath of=$TEST_DIR/$seq.dump.good skip=$physical bs=1 count=4096 \
> + 2>/dev/null
> +$XFS_IO_PROG -c "pwrite -S 0xbb -b 4K $physical 4K" $devpath > /dev/null
> +
> +_scratch_mount
> +
> +echo "step 3......repair the bad copy"
> +_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
> +
> +_scratch_unmount
> +
> +echo "step 4......check if the repair worked"
> +dd if=$devpath of=$TEST_DIR/$seq.dump skip=$physical bs=1 count=4096 \
> + 2>/dev/null
> +cmp -bl $TEST_DIR/$seq.dump.good $TEST_DIR/$seq.dump
> +
> +_scratch_dev_pool_put
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/270.out b/tests/btrfs/270.out
> new file mode 100644
> index 00000000..6d744c02
> --- /dev/null
> +++ b/tests/btrfs/270.out
> @@ -0,0 +1,7 @@
> +QA output created by 270
> +step 1......mkfs.btrfs
> +wrote 131072/131072 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +step 2......corrupt file extent
> +step 3......repair the bad copy
> +step 4......check if the repair worked
> --
> 2.30.2
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 12:41 ` Zorro Lang
@ 2022-06-22 13:07 ` Christoph Hellwig
2022-06-24 2:25 ` Zorro Lang
0 siblings, 1 reply; 10+ messages in thread
From: Christoph Hellwig @ 2022-06-22 13:07 UTC (permalink / raw)
To: Zorro Lang; +Cc: Christoph Hellwig, fstests, linux-btrfs
On Wed, Jun 22, 2022 at 08:41:18PM +0800, Zorro Lang wrote:
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> ^^^^^^^^^^^
> Is it a wrong copy&paste ?
A lot of the test is copied from btrfs/141, so I wanted to keep the
copyright intact.
> > +#
> > +# FS QA Test 270
> > +#
> > +# Regression test for btrfs buffered read repair of compressed data.
>
> If this's a regression test, I'd like to see the fix be reviewed/acked
> at first :)
Heh. Actually I'm not sure it is a regression, this was copy and
pasted as well. I think it actually has been broken since day 1.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent
2022-06-22 13:07 ` Christoph Hellwig
@ 2022-06-24 2:25 ` Zorro Lang
0 siblings, 0 replies; 10+ messages in thread
From: Zorro Lang @ 2022-06-24 2:25 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs
On Wed, Jun 22, 2022 at 03:07:45PM +0200, Christoph Hellwig wrote:
> On Wed, Jun 22, 2022 at 08:41:18PM +0800, Zorro Lang wrote:
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> > ^^^^^^^^^^^
> > Is it a wrong copy&paste ?
>
> A lot of the test is copied from btrfs/141, so I wanted to keep the
> copyright intact.
Oh, there're some cases copy from old cases in fstests, but they are changed
a little to have different coverage. Generally they metioned they copy from
which one case (number). I don't mind if you'd like to keep original
Copyright name, but the copyright time is old for a new test case, I don't
know if we'd better to update it. Hmm... sorry, not good at these copyright
things...
>
> > > +#
> > > +# FS QA Test 270
> > > +#
> > > +# Regression test for btrfs buffered read repair of compressed data.
> >
> > If this's a regression test, I'd like to see the fix be reviewed/acked
> > at first :)
>
> Heh. Actually I'm not sure it is a regression, this was copy and
> pasted as well. I think it actually has been broken since day 1.
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: btrfs read repair: more tests
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
` (3 preceding siblings ...)
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
@ 2022-06-29 14:00 ` Zorro Lang
4 siblings, 0 replies; 10+ messages in thread
From: Zorro Lang @ 2022-06-29 14:00 UTC (permalink / raw)
To: linux-btrfs; +Cc: fstests, Christoph Hellwig
On Wed, Jun 22, 2022 at 06:58:40AM +0200, Christoph Hellwig wrote:
> Hi all,
>
> this series adds a few more btrfs read repair tests. Still all for user
> data as I haven't managed to successfully inject corruption into metadata
> yet.
This patchset is good to me (from fstests side). If there's not more review
points or objections from btrfs list, I'll merge it. Welcome more review.
Thanks,
Zorro
>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-06-29 14:00 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-22 4:58 btrfs read repair: more tests Christoph Hellwig
2022-06-22 4:58 ` [PATCH 1/4] btrfs: fix the_btrfs_get_physical invocation in btrfs-map-logical Christoph Hellwig
2022-06-22 4:58 ` [PATCH 2/4] btrfs; add a test for impossible repair cases Christoph Hellwig
2022-06-22 4:58 ` [PATCH 3/4] btrfs: test checker pattern corruption on raid10 Christoph Hellwig
2022-06-22 4:58 ` [PATCH 4/4] btrfs: test read repair on a corrupted compressed extent Christoph Hellwig
2022-06-22 9:21 ` Christoph Hellwig
2022-06-22 12:41 ` Zorro Lang
2022-06-22 13:07 ` Christoph Hellwig
2022-06-24 2:25 ` Zorro Lang
2022-06-29 14:00 ` btrfs read repair: more tests Zorro Lang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.