* btrfs read repair: new tests and cleanups
@ 2022-05-27 8:19 Christoph Hellwig
2022-05-27 8:19 ` [PATCH 01/10] btrfs: add a helpers for read repair testing Christoph Hellwig
` (10 more replies)
0 siblings, 11 replies; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Hi all,
this series adds more tests for the btrfs read-repair code that exercise
the raid1c3 profile and adds helpers to avoid duplicating too much code
for read repair testing
Changes since v2:
- use the -b option to btrfs-map-logical-blocks for the $seqres.full
output
- don't use _btrfs_no_v1_cache_opt for the new tests
- add a new test to test interleaved sector corruption repair for
direct I/O.
Changes since v1:
- add common helpers for read repair
- increase the offsets so thay they should be fine with 64k block size
(although I don't have a system to actually verify that)
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-27 14:54 ` Zorro Lang
2022-05-30 0:23 ` Anand Jain
2022-05-27 8:19 ` [PATCH 02/10] btrfs/140: use common read repair helpers Christoph Hellwig
` (9 subsequent siblings)
10 siblings, 2 replies; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Add a few helpers to consolidate code for btrfs read repair testing:
- _btrfs_get_first_logical() gets the btrfs logical address for the
first extent in a file
- _btrfs_get_device_path and _btrfs_get_physical use the
btrfs-map-logical tool to find the device path and physical address
for btrfs logical address for a specific mirror
- _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
read the data from a specific mirror
These will be used to consolidate the read repair tests and avoid
duplication for new tests.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
common/btrfs | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
common/config | 1 +
2 files changed, 76 insertions(+)
diff --git a/common/btrfs b/common/btrfs
index ac597ca4..b69feeee 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -505,3 +505,78 @@ _btrfs_metadump()
$BTRFS_IMAGE_PROG "$device" "$dumpfile"
[ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &> /dev/null
}
+
+# Return the btrfs logical address for the first block in a file
+_btrfs_get_first_logical()
+{
+ local file=$1
+ _require_command "$FILEFRAG_PROG" filefrag
+
+ ${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
+ ${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
+}
+
+# Find the device path for a btrfs logical offset
+_btrfs_get_device_path()
+{
+ local logical=$1
+ local stripe=$2
+
+ _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
+
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
+ $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
+}
+
+
+# Find the device physical sector for a btrfs logical offset
+_btrfs_get_physical()
+{
+ local logical=$1
+ local stripe=$2
+
+ _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
+
+ $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
+ $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
+}
+
+# Read from a specific stripe to test read recovery that corrupted a specific
+# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
+# xfs_io process that performed the read was executed with a PID that ends up
+# on the intended mirror.
+_btrfs_direct_read_on_mirror()
+{
+ local mirror=$1
+ local nr_mirrors=$2
+ local file=$3
+ local offset=$4
+ local size=$5
+
+ while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
+ exec $XFS_IO_PROG -d \
+ -c "pread -b $size $offset $size" $file) ]]; do
+ :
+ done
+}
+
+# Read from a specific stripe to test read recovery that corrupted a specific
+# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
+# xfs_io process that performed the read was executed with a PID that ends up
+# on the intended mirror.
+_btrfs_buffered_read_on_mirror()
+{
+ local mirror=$1
+ local nr_mirrors=$2
+ local file=$3
+ local offset=$4
+ local size=$5
+
+ echo 3 > /proc/sys/vm/drop_caches
+ while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
+ exec $XFS_IO_PROG \
+ -c "pread -b $size $offset $size" $file) ]]; do
+ :
+ done
+}
diff --git a/common/config b/common/config
index c6428f90..df20afc1 100644
--- a/common/config
+++ b/common/config
@@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
export BLKZONE_PROG="$(type -P blkzone)"
export GZIP_PROG="$(type -P gzip)"
export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
+export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
# use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
# newer systems have udevadm command but older systems like RHEL5 don't.
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 02/10] btrfs/140: use common read repair helpers
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
2022-05-27 8:19 ` [PATCH 01/10] btrfs: add a helpers for read repair testing Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:35 ` Anand Jain
2022-05-27 8:19 ` [PATCH 03/10] btrfs/141: " Christoph Hellwig
` (8 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Use the common helpers to find the btrfs logical address and to read from
a specific mirror.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/140 | 13 ++-----------
1 file changed, 2 insertions(+), 11 deletions(-)
diff --git a/tests/btrfs/140 b/tests/btrfs/140
index c680fe0a..fdff6eb2 100755
--- a/tests/btrfs/140
+++ b/tests/btrfs/140
@@ -24,7 +24,6 @@ _supported_fs btrfs
_require_scratch_dev_pool 2
_require_btrfs_command inspect-internal dump-tree
-_require_command "$FILEFRAG_PROG" filefrag
_require_odirect
# Overwriting data is forbidden on a zoned block device
_require_non_zoned_device "${SCRATCH_DEV}"
@@ -71,7 +70,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
echo "step 2......corrupt file extent" >>$seqres.full
${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
-logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
physical=$(get_physical ${logical_in_btrfs} 1)
devid=$(get_devid ${logical_in_btrfs} 1)
devpath=$(get_device_path ${devid})
@@ -87,15 +86,7 @@ _scratch_mount
# step 3, 128k dio read (this read can repair bad copy)
echo "step 3......repair the bad copy" >>$seqres.full
-# since raid1 consists of two copies, and the bad copy was put on stripe #1
-# while the good copy lies on stripe #0, the bad copy only gets access when the
-# reader's pid % 2 == 1 is true
-while true; do
- $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null &
- pid=$!
- wait
- [ $((pid % 2)) == 1 ] && break
-done
+_btrfs_direct_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
_scratch_unmount
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 03/10] btrfs/141: use common read repair helpers
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
2022-05-27 8:19 ` [PATCH 01/10] btrfs: add a helpers for read repair testing Christoph Hellwig
2022-05-27 8:19 ` [PATCH 02/10] btrfs/140: use common read repair helpers Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:36 ` Anand Jain
2022-05-27 8:19 ` [PATCH 04/10] btrfs/142: " Christoph Hellwig
` (7 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Use the common helpers to find the btrfs logical address and to read from
a specific mirror.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/141 | 15 ++-------------
1 file changed, 2 insertions(+), 13 deletions(-)
diff --git a/tests/btrfs/141 b/tests/btrfs/141
index 9fdcb2ab..90a90d00 100755
--- a/tests/btrfs/141
+++ b/tests/btrfs/141
@@ -25,7 +25,6 @@ _supported_fs btrfs
_require_scratch_dev_pool 2
_require_btrfs_command inspect-internal dump-tree
-_require_command "$FILEFRAG_PROG" filefrag
get_physical()
{
@@ -69,8 +68,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
# one in $SCRATCH_DEV_POOL
echo "step 2......corrupt file extent" >>$seqres.full
-${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
-logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
physical=$(get_physical ${logical_in_btrfs} 1)
devid=$(get_devid ${logical_in_btrfs} 1)
devpath=$(get_device_path ${devid})
@@ -85,16 +83,7 @@ _scratch_mount
# step 3, 128k buffered read (this read can repair bad copy)
echo "step 3......repair the bad copy" >>$seqres.full
-# since raid1 consists of two copies, and the bad copy was put on stripe #1
-# while the good copy lies on stripe #0, the bad copy only gets access when the
-# reader's pid % 2 == 1 is true
-while true; do
- echo 3 > /proc/sys/vm/drop_caches
- $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null &
- pid=$!
- wait
- [ $((pid % 2)) == 1 ] && break
-done
+_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
_scratch_unmount
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 04/10] btrfs/142: use common read repair helpers
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (2 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 03/10] btrfs/141: " Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:41 ` Anand Jain
2022-05-27 8:19 ` [PATCH 05/10] btrfs/143: " Christoph Hellwig
` (6 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Use the common helpers to find the btrfs logical address and to read from
a specific mirror.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/142 | 10 +++-------
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/tests/btrfs/142 b/tests/btrfs/142
index 58d01add..7a63fb83 100755
--- a/tests/btrfs/142
+++ b/tests/btrfs/142
@@ -27,7 +27,6 @@ _require_scratch_dev_pool 2
_require_dm_target dust
_require_btrfs_command inspect-internal dump-tree
-_require_command "$FILEFRAG_PROG" filefrag
_scratch_dev_pool_get 2
# step 1, create a raid1 btrfs which contains one 128k file.
@@ -46,8 +45,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
# step 2, corrupt the first 64k of stripe #1
echo "step 2......corrupt file extent" >>$seqres.full
-${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
-logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
echo "Logical offset is $logical_in_btrfs" >>$seqres.full
_scratch_unmount
@@ -67,10 +65,8 @@ echo "step 3......repair the bad copy" >>$seqres.full
$DMSETUP_PROG message dust-test 0 addbadblock $((physical / 512))
$DMSETUP_PROG message dust-test 0 enable
-while [[ -z $( (( BASHPID % 2 == stripe )) &&
- exec $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar") ]]; do
- :
-done
+
+_btrfs_direct_read_on_mirror $stripe 2 "$SCRATCH_MNT/foobar" 0 128K
_cleanup_dust
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 05/10] btrfs/143: use common read repair helpers
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (3 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 04/10] btrfs/142: " Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:43 ` Anand Jain
2022-05-27 8:19 ` [PATCH 06/10] btrfs/157: use _btrfs_get_first_logical Christoph Hellwig
` (5 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Use the common helpers to find the btrfs logical address and to read from
a specific mirror.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/143 | 10 +++-------
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/tests/btrfs/143 b/tests/btrfs/143
index 71db861d..b6ff2a42 100755
--- a/tests/btrfs/143
+++ b/tests/btrfs/143
@@ -34,7 +34,6 @@ _require_scratch_dev_pool 2
_require_dm_target dust
_require_btrfs_command inspect-internal dump-tree
-_require_command "$FILEFRAG_PROG" filefrag
_scratch_dev_pool_get 2
# step 1, create a raid1 btrfs which contains one 128k file.
@@ -53,8 +52,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
# step 2, corrupt the first 64k of stripe #1
echo "step 2......corrupt file extent" >>$seqres.full
-${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
-logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
+logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
echo "Logical offset is $logical_in_btrfs" >>$seqres.full
_scratch_unmount
@@ -74,10 +72,8 @@ echo "step 3......repair the bad copy" >>$seqres.full
$DMSETUP_PROG message dust-test 0 addbadblock $((physical / 512))
$DMSETUP_PROG message dust-test 0 enable
-while [[ -z $( (( BASHPID % 2 == stripe )) &&
- exec $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar") ]]; do
- :
-done
+
+_btrfs_buffered_read_on_mirror $stripe 2 "$SCRATCH_MNT/foobar" 0 128K
_cleanup_dust
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 06/10] btrfs/157: use _btrfs_get_first_logical
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (4 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 05/10] btrfs/143: " Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:44 ` Anand Jain
2022-05-27 8:19 ` [PATCH 07/10] btrfs/215: " Christoph Hellwig
` (4 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Use the _btrfs_get_first_logical helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/157 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/btrfs/157 b/tests/btrfs/157
index 343178b7..022db511 100755
--- a/tests/btrfs/157
+++ b/tests/btrfs/157
@@ -71,7 +71,7 @@ _scratch_mount $(_btrfs_no_v1_cache_opt)
$XFS_IO_PROG -f -d -c "pwrite -S 0xaa 0 128K" -c "fsync" \
"$SCRATCH_MNT/foobar" | _filter_xfs_io
-logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
_scratch_unmount
phy0=$(get_physical 0)
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 07/10] btrfs/215: use _btrfs_get_first_logical
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (5 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 06/10] btrfs/157: use _btrfs_get_first_logical Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:44 ` Anand Jain
2022-05-27 8:19 ` [PATCH 08/10] btrfs: test repair with sectors corrupted in multiple mirrors Christoph Hellwig
` (3 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Use the _btrfs_get_first_logical helper instead of open coding it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/215 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/btrfs/215 b/tests/btrfs/215
index 0dcbce2a..928f365c 100755
--- a/tests/btrfs/215
+++ b/tests/btrfs/215
@@ -56,7 +56,7 @@ fi
#create an 8 block file
$XFS_IO_PROG -d -f -c "pwrite -S 0xbb -b $filesize 0 $filesize" "$SCRATCH_MNT/foobar" > /dev/null
-logical_extent=$($FILEFRAG_PROG -v "$SCRATCH_MNT/foobar" | _filter_filefrag | cut -d '#' -f 1)
+logical_extent=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
physical_extent=$(get_physical $logical_extent)
echo "logical = $logical_extent physical=$physical_extent" >> $seqres.full
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 08/10] btrfs: test repair with sectors corrupted in multiple mirrors
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (6 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 07/10] btrfs/215: " Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 0:50 ` Anand Jain
2022-05-27 8:19 ` [PATCH 09/10] btrfs: test buffered I/O read repair with interleaved corrupted sectors Christoph Hellwig
` (2 subsequent siblings)
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Test that repair handles the case where it needs to read from more than
a single mirror on the raid1c3 profile.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/265 | 85 +++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/265.out | 75 +++++++++++++++++++++++++++++++++++++++
2 files changed, 160 insertions(+)
create mode 100755 tests/btrfs/265
create mode 100644 tests/btrfs/265.out
diff --git a/tests/btrfs/265 b/tests/btrfs/265
new file mode 100755
index 00000000..b75d9c84
--- /dev/null
+++ b/tests/btrfs/265
@@ -0,0 +1,85 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 265
+#
+# Test that btrfs raid repair on a raid1c3 profile can repair corruption on two
+# mirrors for the same logical offset.
+#
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+
+_require_odirect
+# Overwriting data is forbidden on a zoned block device
+_require_non_zoned_device "${SCRATCH_DEV}"
+
+_scratch_dev_pool_get 3
+# step 1, create a raid1 btrfs which contains one 128k file.
+echo "step 1......mkfs.btrfs"
+
+mkfs_opts="-d raid1c3 -b 1G"
+_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
+
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# step 2, corrupt the first 64k of one copy (on SCRATCH_DEV which is the first
+# one in $SCRATCH_DEV_POOL
+echo "step 2......corrupt file extent"
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical2=$(_btrfs_get_physical ${logical} 2)
+devpath2=$(_btrfs_get_device_path ${logical} 2)
+
+_scratch_unmount
+
+echo " corrupt stripe #1, devpath $devpath1 physical $physical1" \
+ >> $seqres.full
+$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical1 64K" $devpath1 \
+ > /dev/null
+
+echo " corrupt stripe #2, devpath $devpath2 physical $physical2" \
+ >> $seqres.full
+$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical2 64K" $devpath2 \
+ > /dev/null
+
+_scratch_mount
+
+# step 3, 128k dio read (this read can repair bad copy)
+echo "step 3......repair the bad copy"
+
+_btrfs_direct_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 128K
+_btrfs_direct_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 128K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/265.out b/tests/btrfs/265.out
new file mode 100644
index 00000000..c62c7a39
--- /dev/null
+++ b/tests/btrfs/265.out
@@ -0,0 +1,75 @@
+QA output created by 265
+step 1......mkfs.btrfs
+wrote 131072/131072 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 09/10] btrfs: test buffered I/O read repair with interleaved corrupted sectors
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (7 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 08/10] btrfs: test repair with sectors corrupted in multiple mirrors Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-30 2:02 ` Anand Jain
2022-05-27 8:19 ` [PATCH 10/10] btrfs: test direct " Christoph Hellwig
2022-05-30 2:07 ` btrfs read repair: new tests and cleanups Anand Jain
10 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs, Qu Wenruo
Test that repair handles the case where it needs to read from more than
a single mirror on the raid1c3 profile and needs to take turns over the
mirrors to recover data for the whole read.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
tests/btrfs/266 | 93 +++++++++++++++++++++++++++++++++++++
tests/btrfs/266.out | 109 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 202 insertions(+)
create mode 100755 tests/btrfs/266
create mode 100644 tests/btrfs/266.out
diff --git a/tests/btrfs/266 b/tests/btrfs/266
new file mode 100755
index 00000000..c70e4b2e
--- /dev/null
+++ b/tests/btrfs/266
@@ -0,0 +1,93 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 266
+#
+# Test that btrfs raid repair on a raid1c3 profile can repair interleaving
+# errors on all mirrors.
+#
+
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+
+_require_odirect
+# Overwriting data is forbidden on a zoned block device
+_require_non_zoned_device "${SCRATCH_DEV}"
+
+_scratch_dev_pool_get 3
+# step 1, create a raid1 btrfs which contains one 128k file.
+echo "step 1......mkfs.btrfs"
+
+mkfs_opts="-d raid1c3 -b 1G"
+_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
+
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# step 2, corrupt 4k in each copy
+echo "step 2......corrupt file extent"
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical2=$(_btrfs_get_physical ${logical} 2)
+devpath2=$(_btrfs_get_device_path ${logical} 2)
+
+physical3=$(_btrfs_get_physical ${logical} 3)
+devpath3=$(_btrfs_get_device_path ${logical} 3)
+
+_scratch_unmount
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical3 64K" \
+ $devpath3 > /dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S 0xba -b 64K $physical1 128K" \
+ $devpath1 > /dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical2 + 65536)) 128K" \
+ $devpath2 > /dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 64K $((physical3 + (2 * 65536))) 128K" \
+ $devpath3 > /dev/null
+
+_scratch_mount
+
+# step 3, 128k dio read (this read can repair bad copy)
+echo "step 3......repair the bad copy"
+
+_btrfs_buffered_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 256K
+_btrfs_buffered_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 256K
+_btrfs_buffered_read_on_mirror 2 3 "$SCRATCH_MNT/foobar" 0 256K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/266.out b/tests/btrfs/266.out
new file mode 100644
index 00000000..fcf2f5b8
--- /dev/null
+++ b/tests/btrfs/266.out
@@ -0,0 +1,109 @@
+QA output created by 266
+step 1......mkfs.btrfs
+wrote 262144/262144 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH 10/10] btrfs: test direct I/O read repair with interleaved corrupted sectors
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (8 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 09/10] btrfs: test buffered I/O read repair with interleaved corrupted sectors Christoph Hellwig
@ 2022-05-27 8:19 ` Christoph Hellwig
2022-05-27 10:23 ` Qu Wenruo
2022-05-30 2:03 ` Anand Jain
2022-05-30 2:07 ` btrfs read repair: new tests and cleanups Anand Jain
10 siblings, 2 replies; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 8:19 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Test that repair handles the case where it needs to read from more than
a single mirror on the raid1c3 profile and needs to take turns over the
mirrors to recover data for the whole read.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
tests/btrfs/267 | 93 +++++++++++++++++++++++++++++++++++++
tests/btrfs/267.out | 109 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 202 insertions(+)
create mode 100755 tests/btrfs/267
create mode 100644 tests/btrfs/267.out
diff --git a/tests/btrfs/267 b/tests/btrfs/267
new file mode 100755
index 00000000..cf19fdc8
--- /dev/null
+++ b/tests/btrfs/267
@@ -0,0 +1,93 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2017 Liu Bo. All Rights Reserved.
+# Copyright (c) 2022 Christoph Hellwig.
+#
+# FS QA Test 267
+#
+# Test that btrfs buffered read repair on a raid1c3 profile can repair
+# interleaving errors on all mirrors.
+#
+
+. ./common/preamble
+_begin_fstest auto quick read_repair
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+
+_require_odirect
+# Overwriting data is forbidden on a zoned block device
+_require_non_zoned_device "${SCRATCH_DEV}"
+
+_scratch_dev_pool_get 3
+# step 1, create a raid1 btrfs which contains one 128k file.
+echo "step 1......mkfs.btrfs"
+
+mkfs_opts="-d raid1c3 -b 1G"
+_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
+
+_scratch_mount
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
+ "$SCRATCH_MNT/foobar" | \
+ _filter_xfs_io_offset
+
+# step 2, corrupt 4k in each copy
+echo "step 2......corrupt file extent"
+
+# ensure btrfs-map-logical sees the tree updates
+sync
+
+logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
+
+physical1=$(_btrfs_get_physical ${logical} 1)
+devpath1=$(_btrfs_get_device_path ${logical} 1)
+
+physical2=$(_btrfs_get_physical ${logical} 2)
+devpath2=$(_btrfs_get_device_path ${logical} 2)
+
+physical3=$(_btrfs_get_physical ${logical} 3)
+devpath3=$(_btrfs_get_device_path ${logical} 3)
+
+_scratch_unmount
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical3 64K" \
+ $devpath3 > /dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S 0xba -b 64K $physical1 128K" \
+ $devpath1 > /dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical2 + 65536)) 128K" \
+ $devpath2 > /dev/null
+
+$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 64K $((physical3 + (2 * 65536))) 128K" \
+ $devpath3 > /dev/null
+
+_scratch_mount
+
+# step 3, 128k dio read (this read can repair bad copy)
+echo "step 3......repair the bad copy"
+
+_btrfs_direct_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 256K
+_btrfs_direct_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 256K
+_btrfs_direct_read_on_mirror 2 3 "$SCRATCH_MNT/foobar" 0 256K
+
+_scratch_unmount
+
+echo "step 4......check if the repair worked"
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
+ _filter_xfs_io_offset
+$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\
+ _filter_xfs_io_offset
+
+_scratch_dev_pool_put
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/267.out b/tests/btrfs/267.out
new file mode 100644
index 00000000..2bdd32ea
--- /dev/null
+++ b/tests/btrfs/267.out
@@ -0,0 +1,109 @@
+QA output created by 267
+step 1......mkfs.btrfs
+wrote 262144/262144 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+step 2......corrupt file extent
+step 3......repair the bad copy
+step 4......check if the repair worked
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
+read 512/512 bytes
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH 10/10] btrfs: test direct I/O read repair with interleaved corrupted sectors
2022-05-27 8:19 ` [PATCH 10/10] btrfs: test direct " Christoph Hellwig
@ 2022-05-27 10:23 ` Qu Wenruo
2022-05-27 13:37 ` Christoph Hellwig
2022-05-30 2:03 ` Anand Jain
1 sibling, 1 reply; 31+ messages in thread
From: Qu Wenruo @ 2022-05-27 10:23 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs
On 2022/5/27 16:19, Christoph Hellwig wrote:
> Test that repair handles the case where it needs to read from more than
> a single mirror on the raid1c3 profile and needs to take turns over the
> mirrors to recover data for the whole read.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> tests/btrfs/267 | 93 +++++++++++++++++++++++++++++++++++++
> tests/btrfs/267.out | 109 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 202 insertions(+)
> create mode 100755 tests/btrfs/267
> create mode 100644 tests/btrfs/267.out
>
> diff --git a/tests/btrfs/267 b/tests/btrfs/267
> new file mode 100755
> index 00000000..cf19fdc8
> --- /dev/null
> +++ b/tests/btrfs/267
> @@ -0,0 +1,93 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> +# Copyright (c) 2022 Christoph Hellwig.
> +#
> +# FS QA Test 267
> +#
> +# Test that btrfs buffered read repair on a raid1c3 profile can repair
> +# interleaving errors on all mirrors.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto quick read_repair
> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_require_scratch_dev_pool 3
> +
> +_require_odirect
> +# Overwriting data is forbidden on a zoned block device
> +_require_non_zoned_device "${SCRATCH_DEV}"
> +
> +_scratch_dev_pool_get 3
> +# step 1, create a raid1 btrfs which contains one 128k file.
> +echo "step 1......mkfs.btrfs"
> +
> +mkfs_opts="-d raid1c3 -b 1G"
> +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
> +
> +_scratch_mount
> +
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
> + "$SCRATCH_MNT/foobar" | \
> + _filter_xfs_io_offset
> +
> +# step 2, corrupt 4k in each copy
I guess you mean 64K?
Otherwise looks good to me.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Thanks,
Qu
> +echo "step 2......corrupt file extent"
> +
> +# ensure btrfs-map-logical sees the tree updates
> +sync
> +
> +logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> +
> +physical1=$(_btrfs_get_physical ${logical} 1)
> +devpath1=$(_btrfs_get_device_path ${logical} 1)
> +
> +physical2=$(_btrfs_get_physical ${logical} 2)
> +devpath2=$(_btrfs_get_device_path ${logical} 2)
> +
> +physical3=$(_btrfs_get_physical ${logical} 3)
> +devpath3=$(_btrfs_get_device_path ${logical} 3)
> +
> +_scratch_unmount
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical3 64K" \
> + $devpath3 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 64K $physical1 128K" \
> + $devpath1 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical2 + 65536)) 128K" \
> + $devpath2 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 64K $((physical3 + (2 * 65536))) 128K" \
> + $devpath3 > /dev/null
> +
> +_scratch_mount
> +
> +# step 3, 128k dio read (this read can repair bad copy)
> +echo "step 3......repair the bad copy"
> +
> +_btrfs_direct_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 256K
> +_btrfs_direct_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 256K
> +_btrfs_direct_read_on_mirror 2 3 "$SCRATCH_MNT/foobar" 0 256K
> +
> +_scratch_unmount
> +
> +echo "step 4......check if the repair worked"
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\
> + _filter_xfs_io_offset
> +
> +_scratch_dev_pool_put
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/267.out b/tests/btrfs/267.out
> new file mode 100644
> index 00000000..2bdd32ea
> --- /dev/null
> +++ b/tests/btrfs/267.out
> @@ -0,0 +1,109 @@
> +QA output created by 267
> +step 1......mkfs.btrfs
> +wrote 262144/262144 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +step 2......corrupt file extent
> +step 3......repair the bad copy
> +step 4......check if the repair worked
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 10/10] btrfs: test direct I/O read repair with interleaved corrupted sectors
2022-05-27 10:23 ` Qu Wenruo
@ 2022-05-27 13:37 ` Christoph Hellwig
0 siblings, 0 replies; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 13:37 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Christoph Hellwig, fstests, linux-btrfs
On Fri, May 27, 2022 at 06:23:13PM +0800, Qu Wenruo wrote:
>> +# step 2, corrupt 4k in each copy
>
> I guess you mean 64K?
Yes. Same for btrfs/266.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-27 8:19 ` [PATCH 01/10] btrfs: add a helpers for read repair testing Christoph Hellwig
@ 2022-05-27 14:54 ` Zorro Lang
2022-05-27 15:03 ` Christoph Hellwig
2022-05-30 0:23 ` Anand Jain
1 sibling, 1 reply; 31+ messages in thread
From: Zorro Lang @ 2022-05-27 14:54 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs, Qu Wenruo
On Fri, May 27, 2022 at 10:19:06AM +0200, Christoph Hellwig wrote:
> Add a few helpers to consolidate code for btrfs read repair testing:
>
> - _btrfs_get_first_logical() gets the btrfs logical address for the
> first extent in a file
> - _btrfs_get_device_path and _btrfs_get_physical use the
> btrfs-map-logical tool to find the device path and physical address
> for btrfs logical address for a specific mirror
> - _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
> read the data from a specific mirror
>
> These will be used to consolidate the read repair tests and avoid
> duplication for new tests.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> common/btrfs | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
> common/config | 1 +
> 2 files changed, 76 insertions(+)
>
> diff --git a/common/btrfs b/common/btrfs
> index ac597ca4..b69feeee 100644
> --- a/common/btrfs
> +++ b/common/btrfs
> @@ -505,3 +505,78 @@ _btrfs_metadump()
> $BTRFS_IMAGE_PROG "$device" "$dumpfile"
> [ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &> /dev/null
> }
> +
> +# Return the btrfs logical address for the first block in a file
> +_btrfs_get_first_logical()
> +{
> + local file=$1
> + _require_command "$FILEFRAG_PROG" filefrag
> +
> + ${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
^^
$file
You can send a single fixed patch for this one
Thanks,
Zorro
> + ${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
> +}
> +
> +# Find the device path for a btrfs logical offset
> +_btrfs_get_device_path()
> +{
> + local logical=$1
> + local stripe=$2
> +
> + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> +
> + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
> +}
> +
> +
> +# Find the device physical sector for a btrfs logical offset
> +_btrfs_get_physical()
> +{
> + local logical=$1
> + local stripe=$2
> +
> + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> +
> + $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
> + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
> +}
> +
> +# Read from a specific stripe to test read recovery that corrupted a specific
> +# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
> +# xfs_io process that performed the read was executed with a PID that ends up
> +# on the intended mirror.
> +_btrfs_direct_read_on_mirror()
> +{
> + local mirror=$1
> + local nr_mirrors=$2
> + local file=$3
> + local offset=$4
> + local size=$5
> +
> + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> + exec $XFS_IO_PROG -d \
> + -c "pread -b $size $offset $size" $file) ]]; do
> + :
> + done
> +}
> +
> +# Read from a specific stripe to test read recovery that corrupted a specific
> +# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
> +# xfs_io process that performed the read was executed with a PID that ends up
> +# on the intended mirror.
> +_btrfs_buffered_read_on_mirror()
> +{
> + local mirror=$1
> + local nr_mirrors=$2
> + local file=$3
> + local offset=$4
> + local size=$5
> +
> + echo 3 > /proc/sys/vm/drop_caches
> + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> + exec $XFS_IO_PROG \
> + -c "pread -b $size $offset $size" $file) ]]; do
> + :
> + done
> +}
> diff --git a/common/config b/common/config
> index c6428f90..df20afc1 100644
> --- a/common/config
> +++ b/common/config
> @@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
> export BLKZONE_PROG="$(type -P blkzone)"
> export GZIP_PROG="$(type -P gzip)"
> export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
> +export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
>
> # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
> # newer systems have udevadm command but older systems like RHEL5 don't.
> --
> 2.30.2
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-27 14:54 ` Zorro Lang
@ 2022-05-27 15:03 ` Christoph Hellwig
2022-05-28 3:34 ` Zorro Lang
0 siblings, 1 reply; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-27 15:03 UTC (permalink / raw)
To: Zorro Lang; +Cc: Christoph Hellwig, fstests, linux-btrfs, Qu Wenruo
On Fri, May 27, 2022 at 10:54:45PM +0800, Zorro Lang wrote:
> You can send a single fixed patch for this one
I could do that, but I also need to fix up the comments in 266 and 267
to say 64k instead of 4k.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-27 15:03 ` Christoph Hellwig
@ 2022-05-28 3:34 ` Zorro Lang
2022-05-28 4:56 ` Christoph Hellwig
0 siblings, 1 reply; 31+ messages in thread
From: Zorro Lang @ 2022-05-28 3:34 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs
On Fri, May 27, 2022 at 05:03:06PM +0200, Christoph Hellwig wrote:
> On Fri, May 27, 2022 at 10:54:45PM +0800, Zorro Lang wrote:
> > You can send a single fixed patch for this one
>
> I could do that, but I also need to fix up the comments in 266 and 267
> to say 64k instead of 4k.
Hi Christoph,
I can help to do this change when I merge this patchset, if you don't have
more changes need to do. Actually I'm trying to wait this patchset for this
weekend fstests release, due to it's almost done. I need to do a regression
test today before pushing these patches on Sunday (if no exception:)
Thanks,
Zorro
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-28 3:34 ` Zorro Lang
@ 2022-05-28 4:56 ` Christoph Hellwig
0 siblings, 0 replies; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-28 4:56 UTC (permalink / raw)
To: Zorro Lang; +Cc: Christoph Hellwig, fstests, linux-btrfs
On Sat, May 28, 2022 at 11:34:14AM +0800, Zorro Lang wrote:
> I can help to do this change when I merge this patchset, if you don't have
> more changes need to do. Actually I'm trying to wait this patchset for this
> weekend fstests release, due to it's almost done. I need to do a regression
> test today before pushing these patches on Sunday (if no exception:)
Ok. Here is the updated version of this first patch:
---
From 3de26208af33e9aa83deac77a9d34dec3ccb67c9 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch@lst.de>
Date: Tue, 24 May 2022 09:06:38 +0200
Subject: btrfs: add a helpers for read repair testing
Add a few helpers to consolidate code for btrfs read repair testing:
- _btrfs_get_first_logical() gets the btrfs logical address for the
first extent in a file
- _btrfs_get_device_path and _btrfs_get_physical use the
btrfs-map-logical tool to find the device path and physical address
for btrfs logical address for a specific mirror
- _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
read the data from a specific mirror
These will be used to consolidate the read repair tests and avoid
duplication for new tests.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Qu Wenruo <wqu@suse.com>
---
common/btrfs | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
common/config | 1 +
2 files changed, 76 insertions(+)
diff --git a/common/btrfs b/common/btrfs
index ac597ca4..c7058918 100644
--- a/common/btrfs
+++ b/common/btrfs
@@ -505,3 +505,78 @@ _btrfs_metadump()
$BTRFS_IMAGE_PROG "$device" "$dumpfile"
[ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &> /dev/null
}
+
+# Return the btrfs logical address for the first block in a file
+_btrfs_get_first_logical()
+{
+ local file=$1
+ _require_command "$FILEFRAG_PROG" filefrag
+
+ ${FILEFRAG_PROG} -v $file >> $seqres.full
+ ${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
+}
+
+# Find the device path for a btrfs logical offset
+_btrfs_get_device_path()
+{
+ local logical=$1
+ local stripe=$2
+
+ _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
+
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
+ $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
+}
+
+
+# Find the device physical sector for a btrfs logical offset
+_btrfs_get_physical()
+{
+ local logical=$1
+ local stripe=$2
+
+ _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
+
+ $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
+ $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
+ $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
+}
+
+# Read from a specific stripe to test read recovery that corrupted a specific
+# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
+# xfs_io process that performed the read was executed with a PID that ends up
+# on the intended mirror.
+_btrfs_direct_read_on_mirror()
+{
+ local mirror=$1
+ local nr_mirrors=$2
+ local file=$3
+ local offset=$4
+ local size=$5
+
+ while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
+ exec $XFS_IO_PROG -d \
+ -c "pread -b $size $offset $size" $file) ]]; do
+ :
+ done
+}
+
+# Read from a specific stripe to test read recovery that corrupted a specific
+# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
+# xfs_io process that performed the read was executed with a PID that ends up
+# on the intended mirror.
+_btrfs_buffered_read_on_mirror()
+{
+ local mirror=$1
+ local nr_mirrors=$2
+ local file=$3
+ local offset=$4
+ local size=$5
+
+ echo 3 > /proc/sys/vm/drop_caches
+ while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
+ exec $XFS_IO_PROG \
+ -c "pread -b $size $offset $size" $file) ]]; do
+ :
+ done
+}
diff --git a/common/config b/common/config
index c6428f90..df20afc1 100644
--- a/common/config
+++ b/common/config
@@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
export BLKZONE_PROG="$(type -P blkzone)"
export GZIP_PROG="$(type -P gzip)"
export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
+export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
# use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
# newer systems have udevadm command but older systems like RHEL5 don't.
--
2.30.2
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-27 8:19 ` [PATCH 01/10] btrfs: add a helpers for read repair testing Christoph Hellwig
2022-05-27 14:54 ` Zorro Lang
@ 2022-05-30 0:23 ` Anand Jain
2022-05-30 1:20 ` Qu Wenruo
1 sibling, 1 reply; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:23 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
On 5/27/22 13:49, Christoph Hellwig wrote:
> Add a few helpers to consolidate code for btrfs read repair testing:
>
> - _btrfs_get_first_logical() gets the btrfs logical address for the
> first extent in a file
> - _btrfs_get_device_path and _btrfs_get_physical use the
> btrfs-map-logical tool to find the device path and physical address
> for btrfs logical address for a specific mirror
> - _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
> read the data from a specific mirror
>
> These will be used to consolidate the read repair tests and avoid
> duplication for new tests.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> common/btrfs | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
> common/config | 1 +
> 2 files changed, 76 insertions(+)
>
> diff --git a/common/btrfs b/common/btrfs
> index ac597ca4..b69feeee 100644
> --- a/common/btrfs
> +++ b/common/btrfs
> @@ -505,3 +505,78 @@ _btrfs_metadump()
> $BTRFS_IMAGE_PROG "$device" "$dumpfile"
> [ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &> /dev/null
> }
> +
> +# Return the btrfs logical address for the first block in a file
> +_btrfs_get_first_logical()
> +{
> + local file=$1
> + _require_command "$FILEFRAG_PROG" filefrag
> +
> + ${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> + ${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
> +}
> +
> +# Find the device path for a btrfs logical offset
> +_btrfs_get_device_path()
> +{
> + local logical=$1
> + local stripe=$2
> +
> + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> +
> + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
> +}
> +
> +
> +# Find the device physical sector for a btrfs logical offset
> +_btrfs_get_physical()
> +{
> + local logical=$1
> + local stripe=$2
> +
> + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> +
> + $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >> $seqres.full 2>&1
> + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
> +}
> +
> +# Read from a specific stripe to test read recovery that corrupted a specific
> +# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
> +# xfs_io process that performed the read was executed with a PID that ends up
> +# on the intended mirror.
> +_btrfs_direct_read_on_mirror()
> +{
> + local mirror=$1
> + local nr_mirrors=$2
> + local file=$3
> + local offset=$4
> + local size=$5
> +
> + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> + exec $XFS_IO_PROG -d \
> + -c "pread -b $size $offset $size" $file) ]]; do
> + :
> + done
> +}
> +
> +# Read from a specific stripe to test read recovery that corrupted a specific
> +# stripe. Btrfs uses the PID to select the mirror, so keep reading until the
> +# xfs_io process that performed the read was executed with a PID that ends up
> +# on the intended mirror.
> +_btrfs_buffered_read_on_mirror()
> +{
> + local mirror=$1
> + local nr_mirrors=$2
> + local file=$3
> + local offset=$4
> + local size=$5
> +
> + echo 3 > /proc/sys/vm/drop_caches
> + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> + exec $XFS_IO_PROG \
> + -c "pread -b $size $offset $size" $file) ]]; do
I am confused if it should be BASHPID or PID?
Next, it is ok if the xfs_io_prog fails and returns != 0.
(Part of the test).
But then we will continue in the while loop. No?
Sorry, I am sceptical about this. Could you please clarify
how this works?
Thanks, Anand
> + :
> + done
> +}
> diff --git a/common/config b/common/config
> index c6428f90..df20afc1 100644
> --- a/common/config
> +++ b/common/config
> @@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
> export BLKZONE_PROG="$(type -P blkzone)"
> export GZIP_PROG="$(type -P gzip)"
> export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
> +export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
>
> # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
> # newer systems have udevadm command but older systems like RHEL5 don't.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 02/10] btrfs/140: use common read repair helpers
2022-05-27 8:19 ` [PATCH 02/10] btrfs/140: use common read repair helpers Christoph Hellwig
@ 2022-05-30 0:35 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:35 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
On 5/27/22 13:49, Christoph Hellwig wrote:
> Use the common helpers to find the btrfs logical address and to read from
> a specific mirror.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/140 | 13 ++-----------
> 1 file changed, 2 insertions(+), 11 deletions(-)
>
> diff --git a/tests/btrfs/140 b/tests/btrfs/140
> index c680fe0a..fdff6eb2 100755
> --- a/tests/btrfs/140
> +++ b/tests/btrfs/140
> @@ -24,7 +24,6 @@ _supported_fs btrfs
> _require_scratch_dev_pool 2
>
> _require_btrfs_command inspect-internal dump-tree
> -_require_command "$FILEFRAG_PROG" filefrag
> _require_odirect
> # Overwriting data is forbidden on a zoned block device
> _require_non_zoned_device "${SCRATCH_DEV}"
> @@ -71,7 +70,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
> echo "step 2......corrupt file extent" >>$seqres.full
>
> ${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> -logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
> +logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> physical=$(get_physical ${logical_in_btrfs} 1)
> devid=$(get_devid ${logical_in_btrfs} 1)
> devpath=$(get_device_path ${devid})
> @@ -87,15 +86,7 @@ _scratch_mount
> # step 3, 128k dio read (this read can repair bad copy)
> echo "step 3......repair the bad copy" >>$seqres.full
>
> -# since raid1 consists of two copies, and the bad copy was put on stripe #1
> -# while the good copy lies on stripe #0, the bad copy only gets access when the
> -# reader's pid % 2 == 1 is true
> -while true; do
> - $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null &
> - pid=$!
> - wait
> - [ $((pid % 2)) == 1 ] && break
> -done
> +_btrfs_direct_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
>
(_btrfs_direct_read_on_mirror() is being reviewed in 1/10)
Otherwise. Looks good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
> _scratch_unmount
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 03/10] btrfs/141: use common read repair helpers
2022-05-27 8:19 ` [PATCH 03/10] btrfs/141: " Christoph Hellwig
@ 2022-05-30 0:36 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:36 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
On 5/27/22 13:49, Christoph Hellwig wrote:
> Use the common helpers to find the btrfs logical address and to read from
> a specific mirror.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/141 | 15 ++-------------
> 1 file changed, 2 insertions(+), 13 deletions(-)
>
> diff --git a/tests/btrfs/141 b/tests/btrfs/141
> index 9fdcb2ab..90a90d00 100755
> --- a/tests/btrfs/141
> +++ b/tests/btrfs/141
> @@ -25,7 +25,6 @@ _supported_fs btrfs
> _require_scratch_dev_pool 2
>
> _require_btrfs_command inspect-internal dump-tree
> -_require_command "$FILEFRAG_PROG" filefrag
>
> get_physical()
> {
> @@ -69,8 +68,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
> # one in $SCRATCH_DEV_POOL
> echo "step 2......corrupt file extent" >>$seqres.full
>
> -${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> -logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
> +logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> physical=$(get_physical ${logical_in_btrfs} 1)
> devid=$(get_devid ${logical_in_btrfs} 1)
> devpath=$(get_device_path ${devid})
> @@ -85,16 +83,7 @@ _scratch_mount
> # step 3, 128k buffered read (this read can repair bad copy)
> echo "step 3......repair the bad copy" >>$seqres.full
>
> -# since raid1 consists of two copies, and the bad copy was put on stripe #1
> -# while the good copy lies on stripe #0, the bad copy only gets access when the
> -# reader's pid % 2 == 1 is true
> -while true; do
> - echo 3 > /proc/sys/vm/drop_caches
> - $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar" > /dev/null &
> - pid=$!
> - wait
> - [ $((pid % 2)) == 1 ] && break
> -done
> +_btrfs_buffered_read_on_mirror 1 2 "$SCRATCH_MNT/foobar" 0 128K
>
Same here too.
Otherwise. Looks good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
> _scratch_unmount
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 04/10] btrfs/142: use common read repair helpers
2022-05-27 8:19 ` [PATCH 04/10] btrfs/142: " Christoph Hellwig
@ 2022-05-30 0:41 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:41 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
On 5/27/22 13:49, Christoph Hellwig wrote:
> Use the common helpers to find the btrfs logical address and to read from
> a specific mirror.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/142 | 10 +++-------
> 1 file changed, 3 insertions(+), 7 deletions(-)
>
> diff --git a/tests/btrfs/142 b/tests/btrfs/142
> index 58d01add..7a63fb83 100755
> --- a/tests/btrfs/142
> +++ b/tests/btrfs/142
> @@ -27,7 +27,6 @@ _require_scratch_dev_pool 2
> _require_dm_target dust
>
> _require_btrfs_command inspect-internal dump-tree
> -_require_command "$FILEFRAG_PROG" filefrag
>
> _scratch_dev_pool_get 2
> # step 1, create a raid1 btrfs which contains one 128k file.
> @@ -46,8 +45,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
> # step 2, corrupt the first 64k of stripe #1
> echo "step 2......corrupt file extent" >>$seqres.full
>
> -${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> -logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
> +logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> echo "Logical offset is $logical_in_btrfs" >>$seqres.full
> _scratch_unmount
>
> @@ -67,10 +65,8 @@ echo "step 3......repair the bad copy" >>$seqres.full
>
> $DMSETUP_PROG message dust-test 0 addbadblock $((physical / 512))
> $DMSETUP_PROG message dust-test 0 enable
> -while [[ -z $( (( BASHPID % 2 == stripe )) &&
> - exec $XFS_IO_PROG -d -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar") ]]; do
> - :
> -done
Ah. It is an existing code, we are discussing this in 1/10.
> +
> +_btrfs_direct_read_on_mirror $stripe 2 "$SCRATCH_MNT/foobar" 0 128K
>
Otherwise look good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
> _cleanup_dust
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 05/10] btrfs/143: use common read repair helpers
2022-05-27 8:19 ` [PATCH 05/10] btrfs/143: " Christoph Hellwig
@ 2022-05-30 0:43 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:43 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
looks good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
On 5/27/22 13:49, Christoph Hellwig wrote:
> Use the common helpers to find the btrfs logical address and to read from
> a specific mirror.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/143 | 10 +++-------
> 1 file changed, 3 insertions(+), 7 deletions(-)
>
> diff --git a/tests/btrfs/143 b/tests/btrfs/143
> index 71db861d..b6ff2a42 100755
> --- a/tests/btrfs/143
> +++ b/tests/btrfs/143
> @@ -34,7 +34,6 @@ _require_scratch_dev_pool 2
> _require_dm_target dust
>
> _require_btrfs_command inspect-internal dump-tree
> -_require_command "$FILEFRAG_PROG" filefrag
>
> _scratch_dev_pool_get 2
> # step 1, create a raid1 btrfs which contains one 128k file.
> @@ -53,8 +52,7 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" "$SCRATCH_MNT/foobar" |\
> # step 2, corrupt the first 64k of stripe #1
> echo "step 2......corrupt file extent" >>$seqres.full
>
> -${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> -logical_in_btrfs=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
> +logical_in_btrfs=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> echo "Logical offset is $logical_in_btrfs" >>$seqres.full
> _scratch_unmount
>
> @@ -74,10 +72,8 @@ echo "step 3......repair the bad copy" >>$seqres.full
>
> $DMSETUP_PROG message dust-test 0 addbadblock $((physical / 512))
> $DMSETUP_PROG message dust-test 0 enable
> -while [[ -z $( (( BASHPID % 2 == stripe )) &&
> - exec $XFS_IO_PROG -c "pread -b 128K 0 128K" "$SCRATCH_MNT/foobar") ]]; do
> - :
> -done
> +
> +_btrfs_buffered_read_on_mirror $stripe 2 "$SCRATCH_MNT/foobar" 0 128K
>
> _cleanup_dust
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 06/10] btrfs/157: use _btrfs_get_first_logical
2022-05-27 8:19 ` [PATCH 06/10] btrfs/157: use _btrfs_get_first_logical Christoph Hellwig
@ 2022-05-30 0:44 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:44 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
looks good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
On 5/27/22 13:49, Christoph Hellwig wrote:
> Use the _btrfs_get_first_logical helper instead of open coding it.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/157 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/btrfs/157 b/tests/btrfs/157
> index 343178b7..022db511 100755
> --- a/tests/btrfs/157
> +++ b/tests/btrfs/157
> @@ -71,7 +71,7 @@ _scratch_mount $(_btrfs_no_v1_cache_opt)
> $XFS_IO_PROG -f -d -c "pwrite -S 0xaa 0 128K" -c "fsync" \
> "$SCRATCH_MNT/foobar" | _filter_xfs_io
>
> -logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
> +logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> _scratch_unmount
>
> phy0=$(get_physical 0)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 07/10] btrfs/215: use _btrfs_get_first_logical
2022-05-27 8:19 ` [PATCH 07/10] btrfs/215: " Christoph Hellwig
@ 2022-05-30 0:44 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:44 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
looks good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
On 5/27/22 13:49, Christoph Hellwig wrote:
> Use the _btrfs_get_first_logical helper instead of open coding it.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/215 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/btrfs/215 b/tests/btrfs/215
> index 0dcbce2a..928f365c 100755
> --- a/tests/btrfs/215
> +++ b/tests/btrfs/215
> @@ -56,7 +56,7 @@ fi
> #create an 8 block file
> $XFS_IO_PROG -d -f -c "pwrite -S 0xbb -b $filesize 0 $filesize" "$SCRATCH_MNT/foobar" > /dev/null
>
> -logical_extent=$($FILEFRAG_PROG -v "$SCRATCH_MNT/foobar" | _filter_filefrag | cut -d '#' -f 1)
> +logical_extent=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> physical_extent=$(get_physical $logical_extent)
>
> echo "logical = $logical_extent physical=$physical_extent" >> $seqres.full
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 08/10] btrfs: test repair with sectors corrupted in multiple mirrors
2022-05-27 8:19 ` [PATCH 08/10] btrfs: test repair with sectors corrupted in multiple mirrors Christoph Hellwig
@ 2022-05-30 0:50 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 0:50 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
looks good.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Thanks, Anand
On 5/27/22 13:49, Christoph Hellwig wrote:
> Test that repair handles the case where it needs to read from more than
> a single mirror on the raid1c3 profile.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/265 | 85 +++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/265.out | 75 +++++++++++++++++++++++++++++++++++++++
> 2 files changed, 160 insertions(+)
> create mode 100755 tests/btrfs/265
> create mode 100644 tests/btrfs/265.out
>
> diff --git a/tests/btrfs/265 b/tests/btrfs/265
> new file mode 100755
> index 00000000..b75d9c84
> --- /dev/null
> +++ b/tests/btrfs/265
> @@ -0,0 +1,85 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> +# Copyright (c) 2022 Christoph Hellwig.
> +#
> +# FS QA Test 265
> +#
> +# Test that btrfs raid repair on a raid1c3 profile can repair corruption on two
> +# mirrors for the same logical offset.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick read_repair
> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_require_scratch_dev_pool 3
> +
> +_require_odirect
> +# Overwriting data is forbidden on a zoned block device
> +_require_non_zoned_device "${SCRATCH_DEV}"
> +
> +_scratch_dev_pool_get 3
> +# step 1, create a raid1 btrfs which contains one 128k file.
> +echo "step 1......mkfs.btrfs"
> +
> +mkfs_opts="-d raid1c3 -b 1G"
> +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
> +
> +_scratch_mount
> +
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 128K 0 128K" \
> + "$SCRATCH_MNT/foobar" | \
> + _filter_xfs_io_offset
> +
> +# step 2, corrupt the first 64k of one copy (on SCRATCH_DEV which is the first
> +# one in $SCRATCH_DEV_POOL
> +echo "step 2......corrupt file extent"
> +
> +# ensure btrfs-map-logical sees the tree updates
> +sync
> +
> +logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> +
> +physical1=$(_btrfs_get_physical ${logical} 1)
> +devpath1=$(_btrfs_get_device_path ${logical} 1)
> +
> +physical2=$(_btrfs_get_physical ${logical} 2)
> +devpath2=$(_btrfs_get_device_path ${logical} 2)
> +
> +_scratch_unmount
> +
> +echo " corrupt stripe #1, devpath $devpath1 physical $physical1" \
> + >> $seqres.full
> +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical1 64K" $devpath1 \
> + > /dev/null
> +
> +echo " corrupt stripe #2, devpath $devpath2 physical $physical2" \
> + >> $seqres.full
> +$XFS_IO_PROG -d -c "pwrite -S 0xbf -b 64K $physical2 64K" $devpath2 \
> + > /dev/null
> +
> +_scratch_mount
> +
> +# step 3, 128k dio read (this read can repair bad copy)
> +echo "step 3......repair the bad copy"
> +
> +_btrfs_direct_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 128K
> +_btrfs_direct_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 128K
> +
> +_scratch_unmount
> +
> +echo "step 4......check if the repair worked"
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
> + _filter_xfs_io_offset
> +
> +_scratch_dev_pool_put
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/265.out b/tests/btrfs/265.out
> new file mode 100644
> index 00000000..c62c7a39
> --- /dev/null
> +++ b/tests/btrfs/265.out
> @@ -0,0 +1,75 @@
> +QA output created by 265
> +step 1......mkfs.btrfs
> +wrote 131072/131072 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +step 2......corrupt file extent
> +step 3......repair the bad copy
> +step 4......check if the repair worked
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-30 0:23 ` Anand Jain
@ 2022-05-30 1:20 ` Qu Wenruo
2022-05-30 4:36 ` Zorro Lang
0 siblings, 1 reply; 31+ messages in thread
From: Qu Wenruo @ 2022-05-30 1:20 UTC (permalink / raw)
To: Anand Jain, Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
On 2022/5/30 08:23, Anand Jain wrote:
> On 5/27/22 13:49, Christoph Hellwig wrote:
>> Add a few helpers to consolidate code for btrfs read repair testing:
>>
>> - _btrfs_get_first_logical() gets the btrfs logical address for the
>> first extent in a file
>> - _btrfs_get_device_path and _btrfs_get_physical use the
>> btrfs-map-logical tool to find the device path and physical address
>> for btrfs logical address for a specific mirror
>> - _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
>> read the data from a specific mirror
>>
>> These will be used to consolidate the read repair tests and avoid
>> duplication for new tests.
>>
>> Signed-off-by: Christoph Hellwig <hch@lst.de>
>> Reviewed-by: Qu Wenruo <wqu@suse.com>
>> ---
>> common/btrfs | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
>> common/config | 1 +
>> 2 files changed, 76 insertions(+)
>>
>> diff --git a/common/btrfs b/common/btrfs
>> index ac597ca4..b69feeee 100644
>> --- a/common/btrfs
>> +++ b/common/btrfs
>> @@ -505,3 +505,78 @@ _btrfs_metadump()
>> $BTRFS_IMAGE_PROG "$device" "$dumpfile"
>> [ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &>
>> /dev/null
>> }
>> +
>> +# Return the btrfs logical address for the first block in a file
>> +_btrfs_get_first_logical()
>> +{
>> + local file=$1
>> + _require_command "$FILEFRAG_PROG" filefrag
>> +
>> + ${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
>> + ${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
>> +}
>> +
>> +# Find the device path for a btrfs logical offset
>> +_btrfs_get_device_path()
>> +{
>> + local logical=$1
>> + local stripe=$2
>> +
>> + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
>> +
>> + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
>> + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
>> +}
>> +
>> +
>> +# Find the device physical sector for a btrfs logical offset
>> +_btrfs_get_physical()
>> +{
>> + local logical=$1
>> + local stripe=$2
>> +
>> + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
>> +
>> + $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >>
>> $seqres.full 2>&1
>> + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
>> + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
>> +}
>> +
>> +# Read from a specific stripe to test read recovery that corrupted a
>> specific
>> +# stripe. Btrfs uses the PID to select the mirror, so keep reading
>> until the
>> +# xfs_io process that performed the read was executed with a PID that
>> ends up
>> +# on the intended mirror.
>> +_btrfs_direct_read_on_mirror()
>> +{
>> + local mirror=$1
>> + local nr_mirrors=$2
>> + local file=$3
>> + local offset=$4
>> + local size=$5
>> +
>> + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
>> + exec $XFS_IO_PROG -d \
>> + -c "pread -b $size $offset $size" $file) ]]; do
>> + :
>> + done
>> +}
>> +
>> +# Read from a specific stripe to test read recovery that corrupted a
>> specific
>> +# stripe. Btrfs uses the PID to select the mirror, so keep reading
>> until the
>> +# xfs_io process that performed the read was executed with a PID that
>> ends up
>> +# on the intended mirror.
>> +_btrfs_buffered_read_on_mirror()
>> +{
>> + local mirror=$1
>> + local nr_mirrors=$2
>> + local file=$3
>> + local offset=$4
>> + local size=$5
>> +
>> + echo 3 > /proc/sys/vm/drop_caches
>
>
>> + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
>> + exec $XFS_IO_PROG \
>> + -c "pread -b $size $offset $size" $file) ]]; do
>
> I am confused if it should be BASHPID or PID?
>
> Next, it is ok if the xfs_io_prog fails and returns != 0.
> (Part of the test).
> But then we will continue in the while loop. No?
The loop goes like this:
If "BASHPID % nr_mirrors" isn't mirror, then the whole condition inside
the $() part is already 0, no need to exec the xfs_io command.
Then -z 0 return true, we continue the while loop (aka, doing nothing).
Only when (BASHPID % nr_mirrors == mirror) is true, we will try to exec
c XFS_IO_PROGS in the same PID, and normally we will exit.
Some of your concern are valid.
Like if some code bug makes the pread failed, then we will continue the
loop, possibly lead to an infinite loop.
The pid-check-then-exec idea is pretty good, it allows us to only
initiate the read, without doing some unnecessary read then check the pid.
But putting the whole exec into the pid check is indeed hacky and can
lead to problems.
Can we make it less hacky?
Thanks,
Qu
Thanks,
Qu
>
> Sorry, I am sceptical about this. Could you please clarify
> how this works?
>
>
> Thanks, Anand
>
>
>
>> + :
>> + done
>
>
>
>
>> +}
>> diff --git a/common/config b/common/config
>> index c6428f90..df20afc1 100644
>> --- a/common/config
>> +++ b/common/config
>> @@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
>> export BLKZONE_PROG="$(type -P blkzone)"
>> export GZIP_PROG="$(type -P gzip)"
>> export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
>> +export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
>> # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
>> # newer systems have udevadm command but older systems like RHEL5
>> don't.
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 09/10] btrfs: test buffered I/O read repair with interleaved corrupted sectors
2022-05-27 8:19 ` [PATCH 09/10] btrfs: test buffered I/O read repair with interleaved corrupted sectors Christoph Hellwig
@ 2022-05-30 2:02 ` Anand Jain
0 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 2:02 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs, Qu Wenruo
LGTM
Reviewed-by: Anand Jain <anand.jain@oracle.com>
On 5/27/22 13:49, Christoph Hellwig wrote:
> Test that repair handles the case where it needs to read from more than
> a single mirror on the raid1c3 profile and needs to take turns over the
> mirrors to recover data for the whole read.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
> ---
> tests/btrfs/266 | 93 +++++++++++++++++++++++++++++++++++++
> tests/btrfs/266.out | 109 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 202 insertions(+)
> create mode 100755 tests/btrfs/266
> create mode 100644 tests/btrfs/266.out
>
> diff --git a/tests/btrfs/266 b/tests/btrfs/266
> new file mode 100755
> index 00000000..c70e4b2e
> --- /dev/null
> +++ b/tests/btrfs/266
> @@ -0,0 +1,93 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> +# Copyright (c) 2022 Christoph Hellwig.
> +#
> +# FS QA Test 266
> +#
> +# Test that btrfs raid repair on a raid1c3 profile can repair interleaving
> +# errors on all mirrors.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto quick read_repair
> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_require_scratch_dev_pool 3
> +
> +_require_odirect
> +# Overwriting data is forbidden on a zoned block device
> +_require_non_zoned_device "${SCRATCH_DEV}"
> +
> +_scratch_dev_pool_get 3
> +# step 1, create a raid1 btrfs which contains one 128k file.
> +echo "step 1......mkfs.btrfs"
> +
> +mkfs_opts="-d raid1c3 -b 1G"
> +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
> +
> +_scratch_mount
> +
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
> + "$SCRATCH_MNT/foobar" | \
> + _filter_xfs_io_offset
> +
> +# step 2, corrupt 4k in each copy
> +echo "step 2......corrupt file extent"
> +
> +# ensure btrfs-map-logical sees the tree updates
> +sync
> +
> +logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> +
> +physical1=$(_btrfs_get_physical ${logical} 1)
> +devpath1=$(_btrfs_get_device_path ${logical} 1)
> +
> +physical2=$(_btrfs_get_physical ${logical} 2)
> +devpath2=$(_btrfs_get_device_path ${logical} 2)
> +
> +physical3=$(_btrfs_get_physical ${logical} 3)
> +devpath3=$(_btrfs_get_device_path ${logical} 3)
> +
> +_scratch_unmount
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical3 64K" \
> + $devpath3 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 64K $physical1 128K" \
> + $devpath1 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical2 + 65536)) 128K" \
> + $devpath2 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 64K $((physical3 + (2 * 65536))) 128K" \
> + $devpath3 > /dev/null
> +
> +_scratch_mount
> +
> +# step 3, 128k dio read (this read can repair bad copy)
> +echo "step 3......repair the bad copy"
> +
> +_btrfs_buffered_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 256K
> +_btrfs_buffered_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 256K
> +_btrfs_buffered_read_on_mirror 2 3 "$SCRATCH_MNT/foobar" 0 256K
> +
> +_scratch_unmount
> +
> +echo "step 4......check if the repair worked"
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\
> + _filter_xfs_io_offset
> +
> +_scratch_dev_pool_put
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/266.out b/tests/btrfs/266.out
> new file mode 100644
> index 00000000..fcf2f5b8
> --- /dev/null
> +++ b/tests/btrfs/266.out
> @@ -0,0 +1,109 @@
> +QA output created by 266
> +step 1......mkfs.btrfs
> +wrote 262144/262144 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +step 2......corrupt file extent
> +step 3......repair the bad copy
> +step 4......check if the repair worked
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 10/10] btrfs: test direct I/O read repair with interleaved corrupted sectors
2022-05-27 8:19 ` [PATCH 10/10] btrfs: test direct " Christoph Hellwig
2022-05-27 10:23 ` Qu Wenruo
@ 2022-05-30 2:03 ` Anand Jain
1 sibling, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 2:03 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs
LGTM
Reviewed-by: Anand Jain <anand.jain@oracle.com>
On 5/27/22 13:49, Christoph Hellwig wrote:
> Test that repair handles the case where it needs to read from more than
> a single mirror on the raid1c3 profile and needs to take turns over the
> mirrors to recover data for the whole read.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> tests/btrfs/267 | 93 +++++++++++++++++++++++++++++++++++++
> tests/btrfs/267.out | 109 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 202 insertions(+)
> create mode 100755 tests/btrfs/267
> create mode 100644 tests/btrfs/267.out
>
> diff --git a/tests/btrfs/267 b/tests/btrfs/267
> new file mode 100755
> index 00000000..cf19fdc8
> --- /dev/null
> +++ b/tests/btrfs/267
> @@ -0,0 +1,93 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2017 Liu Bo. All Rights Reserved.
> +# Copyright (c) 2022 Christoph Hellwig.
> +#
> +# FS QA Test 267
> +#
> +# Test that btrfs buffered read repair on a raid1c3 profile can repair
> +# interleaving errors on all mirrors.
> +#
> +
> +. ./common/preamble
> +_begin_fstest auto quick read_repair
> +
> +# Import common functions.
> +. ./common/filter
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_require_scratch_dev_pool 3
> +
> +_require_odirect
> +# Overwriting data is forbidden on a zoned block device
> +_require_non_zoned_device "${SCRATCH_DEV}"
> +
> +_scratch_dev_pool_get 3
> +# step 1, create a raid1 btrfs which contains one 128k file.
> +echo "step 1......mkfs.btrfs"
> +
> +mkfs_opts="-d raid1c3 -b 1G"
> +_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
> +
> +_scratch_mount
> +
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xaa -b 256K 0 256K" \
> + "$SCRATCH_MNT/foobar" | \
> + _filter_xfs_io_offset
> +
> +# step 2, corrupt 4k in each copy
> +echo "step 2......corrupt file extent"
> +
> +# ensure btrfs-map-logical sees the tree updates
> +sync
> +
> +logical=$(_btrfs_get_first_logical $SCRATCH_MNT/foobar)
> +
> +physical1=$(_btrfs_get_physical ${logical} 1)
> +devpath1=$(_btrfs_get_device_path ${logical} 1)
> +
> +physical2=$(_btrfs_get_physical ${logical} 2)
> +devpath2=$(_btrfs_get_device_path ${logical} 2)
> +
> +physical3=$(_btrfs_get_physical ${logical} 3)
> +devpath3=$(_btrfs_get_device_path ${logical} 3)
> +
> +_scratch_unmount
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbd -b 64K $physical3 64K" \
> + $devpath3 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xba -b 64K $physical1 128K" \
> + $devpath1 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbb -b 64K $((physical2 + 65536)) 128K" \
> + $devpath2 > /dev/null
> +
> +$XFS_IO_PROG -d -c "pwrite -S 0xbc -b 64K $((physical3 + (2 * 65536))) 128K" \
> + $devpath3 > /dev/null
> +
> +_scratch_mount
> +
> +# step 3, 128k dio read (this read can repair bad copy)
> +echo "step 3......repair the bad copy"
> +
> +_btrfs_direct_read_on_mirror 0 3 "$SCRATCH_MNT/foobar" 0 256K
> +_btrfs_direct_read_on_mirror 1 3 "$SCRATCH_MNT/foobar" 0 256K
> +_btrfs_direct_read_on_mirror 2 3 "$SCRATCH_MNT/foobar" 0 256K
> +
> +_scratch_unmount
> +
> +echo "step 4......check if the repair worked"
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical1 512" $devpath1 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical2 512" $devpath2 |\
> + _filter_xfs_io_offset
> +$XFS_IO_PROG -d -c "pread -v -b 512 $physical3 512" $devpath3 |\
> + _filter_xfs_io_offset
> +
> +_scratch_dev_pool_put
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/267.out b/tests/btrfs/267.out
> new file mode 100644
> index 00000000..2bdd32ea
> --- /dev/null
> +++ b/tests/btrfs/267.out
> @@ -0,0 +1,109 @@
> +QA output created by 267
> +step 1......mkfs.btrfs
> +wrote 262144/262144 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +step 2......corrupt file extent
> +step 3......repair the bad copy
> +step 4......check if the repair worked
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +XXXXXXXX: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa ................
> +read 512/512 bytes
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: btrfs read repair: new tests and cleanups
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
` (9 preceding siblings ...)
2022-05-27 8:19 ` [PATCH 10/10] btrfs: test direct " Christoph Hellwig
@ 2022-05-30 2:07 ` Anand Jain
10 siblings, 0 replies; 31+ messages in thread
From: Anand Jain @ 2022-05-30 2:07 UTC (permalink / raw)
To: Christoph Hellwig, fstests; +Cc: linux-btrfs
On 5/27/22 13:49, Christoph Hellwig wrote:
> Hi all,
>
> this series adds more tests for the btrfs read-repair code that exercise
> the raid1c3 profile and adds helpers to avoid duplicating too much code
> for read repair testing
>
> Changes since v2:
> - use the -b option to btrfs-map-logical-blocks for the $seqres.full
> output
> - don't use _btrfs_no_v1_cache_opt for the new tests
> - add a new test to test interleaved sector corruption repair for
> direct I/O.
>
> Changes since v1:
> - add common helpers for read repair
> - increase the offsets so thay they should be fine with 64k block size
> (although I don't have a system to actually verify that)
Verified on pagesize=sectorsize=64K.
For the whole series.
Test-by: Anand Jain <anand.jain@oracle.com>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-30 1:20 ` Qu Wenruo
@ 2022-05-30 4:36 ` Zorro Lang
2022-05-30 5:34 ` Christoph Hellwig
0 siblings, 1 reply; 31+ messages in thread
From: Zorro Lang @ 2022-05-30 4:36 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Anand Jain, Christoph Hellwig, fstests, linux-btrfs, Qu Wenruo
On Mon, May 30, 2022 at 09:20:27AM +0800, Qu Wenruo wrote:
>
>
> On 2022/5/30 08:23, Anand Jain wrote:
> > On 5/27/22 13:49, Christoph Hellwig wrote:
> > > Add a few helpers to consolidate code for btrfs read repair testing:
> > >
> > > - _btrfs_get_first_logical() gets the btrfs logical address for the
> > > first extent in a file
> > > - _btrfs_get_device_path and _btrfs_get_physical use the
> > > btrfs-map-logical tool to find the device path and physical address
> > > for btrfs logical address for a specific mirror
> > > - _btrfs_direct_read_on_mirror and _btrfs_buffered_read_on_mirror
> > > read the data from a specific mirror
> > >
> > > These will be used to consolidate the read repair tests and avoid
> > > duplication for new tests.
> > >
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > > Reviewed-by: Qu Wenruo <wqu@suse.com>
> > > ---
> > > common/btrfs | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
> > > common/config | 1 +
> > > 2 files changed, 76 insertions(+)
> > >
> > > diff --git a/common/btrfs b/common/btrfs
> > > index ac597ca4..b69feeee 100644
> > > --- a/common/btrfs
> > > +++ b/common/btrfs
> > > @@ -505,3 +505,78 @@ _btrfs_metadump()
> > > $BTRFS_IMAGE_PROG "$device" "$dumpfile"
> > > [ -n "$DUMP_COMPRESSOR" ] && $DUMP_COMPRESSOR -f "$dumpfile" &>
> > > /dev/null
> > > }
> > > +
> > > +# Return the btrfs logical address for the first block in a file
> > > +_btrfs_get_first_logical()
> > > +{
> > > + local file=$1
> > > + _require_command "$FILEFRAG_PROG" filefrag
> > > +
> > > + ${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar >> $seqres.full
> > > + ${FILEFRAG_PROG} -v $file | _filter_filefrag | cut -d '#' -f 1
> > > +}
> > > +
> > > +# Find the device path for a btrfs logical offset
> > > +_btrfs_get_device_path()
> > > +{
> > > + local logical=$1
> > > + local stripe=$2
> > > +
> > > + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> > > +
> > > + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> > > + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$8 }"
> > > +}
> > > +
> > > +
> > > +# Find the device physical sector for a btrfs logical offset
> > > +_btrfs_get_physical()
> > > +{
> > > + local logical=$1
> > > + local stripe=$2
> > > +
> > > + _require_command "$BTRFS_MAP_LOGICAL_PROG" btrfs-map-logical
> > > +
> > > + $BTRFS_MAP_LOGICAL_PROG -b -l $logical $SCRATCH_DEV >>
> > > $seqres.full 2>&1
> > > + $BTRFS_MAP_LOGICAL_PROG -l $logical $SCRATCH_DEV | \
> > > + $AWK_PROG "(\$1 ~ /mirror/ && \$2 ~ /$stripe/) { print \$6 }"
> > > +}
> > > +
> > > +# Read from a specific stripe to test read recovery that corrupted a
> > > specific
> > > +# stripe. Btrfs uses the PID to select the mirror, so keep reading
> > > until the
> > > +# xfs_io process that performed the read was executed with a PID that
> > > ends up
> > > +# on the intended mirror.
> > > +_btrfs_direct_read_on_mirror()
> > > +{
> > > + local mirror=$1
> > > + local nr_mirrors=$2
> > > + local file=$3
> > > + local offset=$4
> > > + local size=$5
> > > +
> > > + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> > > + exec $XFS_IO_PROG -d \
> > > + -c "pread -b $size $offset $size" $file) ]]; do
> > > + :
> > > + done
> > > +}
> > > +
> > > +# Read from a specific stripe to test read recovery that corrupted a
> > > specific
> > > +# stripe. Btrfs uses the PID to select the mirror, so keep reading
> > > until the
> > > +# xfs_io process that performed the read was executed with a PID that
> > > ends up
> > > +# on the intended mirror.
> > > +_btrfs_buffered_read_on_mirror()
> > > +{
> > > + local mirror=$1
> > > + local nr_mirrors=$2
> > > + local file=$3
> > > + local offset=$4
> > > + local size=$5
> > > +
> > > + echo 3 > /proc/sys/vm/drop_caches
> >
> >
> > > + while [[ -z $( (( BASHPID % nr_mirrors == mirror )) &&
> > > + exec $XFS_IO_PROG \
> > > + -c "pread -b $size $offset $size" $file) ]]; do
> >
> > I am confused if it should be BASHPID or PID?
> >
> > Next, it is ok if the xfs_io_prog fails and returns != 0.
> > (Part of the test).
> > But then we will continue in the while loop. No?
>
> The loop goes like this:
>
> If "BASHPID % nr_mirrors" isn't mirror, then the whole condition inside
> the $() part is already 0, no need to exec the xfs_io command.
>
> Then -z 0 return true, we continue the while loop (aka, doing nothing).
>
> Only when (BASHPID % nr_mirrors == mirror) is true, we will try to exec
> c XFS_IO_PROGS in the same PID, and normally we will exit.
>
>
> Some of your concern are valid.
> Like if some code bug makes the pread failed, then we will continue the
> loop, possibly lead to an infinite loop.
>
> The pid-check-then-exec idea is pretty good, it allows us to only
> initiate the read, without doing some unnecessary read then check the pid.
>
> But putting the whole exec into the pid check is indeed hacky and can
> lead to problems.
>
> Can we make it less hacky?
Hi Anand and Wenruo,
This patchset has been merged into fstests release v2022.05.29 last weekend.
If you feel something isn't good enough, feel free to send new patches to
fix/improve it.
Thanks,
Zorro
>
> Thanks,
> Qu
>
> Thanks,
> Qu
>
> >
> > Sorry, I am sceptical about this. Could you please clarify
> > how this works?
> >
> >
> > Thanks, Anand
> >
> >
> >
> > > + :
> > > + done
> >
> >
> >
> >
> > > +}
> > > diff --git a/common/config b/common/config
> > > index c6428f90..df20afc1 100644
> > > --- a/common/config
> > > +++ b/common/config
> > > @@ -228,6 +228,7 @@ export E2IMAGE_PROG="$(type -P e2image)"
> > > export BLKZONE_PROG="$(type -P blkzone)"
> > > export GZIP_PROG="$(type -P gzip)"
> > > export BTRFS_IMAGE_PROG="$(type -P btrfs-image)"
> > > +export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical)
> > > # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled.
> > > # newer systems have udevadm command but older systems like RHEL5
> > > don't.
> >
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH 01/10] btrfs: add a helpers for read repair testing
2022-05-30 4:36 ` Zorro Lang
@ 2022-05-30 5:34 ` Christoph Hellwig
0 siblings, 0 replies; 31+ messages in thread
From: Christoph Hellwig @ 2022-05-30 5:34 UTC (permalink / raw)
To: Zorro Lang
Cc: Qu Wenruo, Anand Jain, Christoph Hellwig, fstests, linux-btrfs,
Qu Wenruo
On Mon, May 30, 2022 at 12:36:32PM +0800, Zorro Lang wrote:
> This patchset has been merged into fstests release v2022.05.29 last weekend.
> If you feel something isn't good enough, feel free to send new patches to
> fix/improve it.
Also this exact same code existed in two tests before and was just
factored into a common helper.
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2022-05-30 5:34 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-27 8:19 btrfs read repair: new tests and cleanups Christoph Hellwig
2022-05-27 8:19 ` [PATCH 01/10] btrfs: add a helpers for read repair testing Christoph Hellwig
2022-05-27 14:54 ` Zorro Lang
2022-05-27 15:03 ` Christoph Hellwig
2022-05-28 3:34 ` Zorro Lang
2022-05-28 4:56 ` Christoph Hellwig
2022-05-30 0:23 ` Anand Jain
2022-05-30 1:20 ` Qu Wenruo
2022-05-30 4:36 ` Zorro Lang
2022-05-30 5:34 ` Christoph Hellwig
2022-05-27 8:19 ` [PATCH 02/10] btrfs/140: use common read repair helpers Christoph Hellwig
2022-05-30 0:35 ` Anand Jain
2022-05-27 8:19 ` [PATCH 03/10] btrfs/141: " Christoph Hellwig
2022-05-30 0:36 ` Anand Jain
2022-05-27 8:19 ` [PATCH 04/10] btrfs/142: " Christoph Hellwig
2022-05-30 0:41 ` Anand Jain
2022-05-27 8:19 ` [PATCH 05/10] btrfs/143: " Christoph Hellwig
2022-05-30 0:43 ` Anand Jain
2022-05-27 8:19 ` [PATCH 06/10] btrfs/157: use _btrfs_get_first_logical Christoph Hellwig
2022-05-30 0:44 ` Anand Jain
2022-05-27 8:19 ` [PATCH 07/10] btrfs/215: " Christoph Hellwig
2022-05-30 0:44 ` Anand Jain
2022-05-27 8:19 ` [PATCH 08/10] btrfs: test repair with sectors corrupted in multiple mirrors Christoph Hellwig
2022-05-30 0:50 ` Anand Jain
2022-05-27 8:19 ` [PATCH 09/10] btrfs: test buffered I/O read repair with interleaved corrupted sectors Christoph Hellwig
2022-05-30 2:02 ` Anand Jain
2022-05-27 8:19 ` [PATCH 10/10] btrfs: test direct " Christoph Hellwig
2022-05-27 10:23 ` Qu Wenruo
2022-05-27 13:37 ` Christoph Hellwig
2022-05-30 2:03 ` Anand Jain
2022-05-30 2:07 ` btrfs read repair: new tests and cleanups Anand Jain
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.