All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qu Wenruo <wqu@suse.com>
To: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org
Cc: Filipe Manana <fdmanana@suse.com>
Subject: [PATCH 3/3] fstests: btrfs/15[78]: Use proper helper to get both devid and physical offset for corruption
Date: Wed, 11 Dec 2019 18:40:29 +0800	[thread overview]
Message-ID: <20191211104029.25541-4-wqu@suse.com> (raw)
In-Reply-To: <20191211104029.25541-1-wqu@suse.com>

[BUG]
When using btrfs-progs v5.4, btrfs/157 and btrfs/158 will fail:

btrfs/157 1s ... - output mismatch (see xfstests/results//btrfs/157.out.bad)
    --- tests/btrfs/157.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/157.out.bad
2019-12-10 15:35:43.112390076 +0000
    @@ -1,9 +1,9 @@
     QA output created by 157
     wrote 131072/131072 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    +wrote 65536/65536 bytes at offset 22020096
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    ...
    (Run 'diff -u xfstests/tests/btrfs/157.out xfstests/results//btrfs/157.out.bad'  to see the entire diff)
btrfs/158 2s ... - output mismatch (see xfstests/results//btrfs/158.out.bad)
    --- tests/btrfs/158.out 2018-09-16 21:30:48.505104287 +0100
    +++ xfstests/results//btrfs/158.out.bad
2019-12-10 15:35:44.844388521 +0000
    @@ -1,9 +1,9 @@
     QA output created by 158
     wrote 131072/131072 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    +wrote 65536/65536 bytes at offset 22020096
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -wrote 65536/65536 bytes at offset 9437184
    ...
    (Run 'diff -u xfstests/tests/btrfs/158.out xfstests/results//btrfs/158.out.bad'  to see the entire diff)

[CAUSE]
This two tests use physical offset as golden output, while mkfs.btrfs
can do whatever it likes to arrange its chunk layout, thus physical
offset is never reliable.

And btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") just changed the layout.

So the output mismatch and failed.

[FIX]
In fact, that btrfs-progs commit not only changed offset, but also the
device sequence.

So we can't just simply remove the physical offset, but also need to use
proper helper to get both devid (as its device path) and physical offset
for corruption.

As long as mkfs.btrfs still uses sequential devid, these tests should
handle future chunk layout change without problem.

Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/btrfs/157     | 53 +++++++++++++++++++++++++++++----------------
 tests/btrfs/157.out |  4 ----
 tests/btrfs/158     | 48 +++++++++++++++++++++++++---------------
 tests/btrfs/158.out |  4 ----
 4 files changed, 65 insertions(+), 44 deletions(-)

diff --git a/tests/btrfs/157 b/tests/btrfs/157
index 7f75c407..80b01b8d 100755
--- a/tests/btrfs/157
+++ b/tests/btrfs/157
@@ -51,22 +51,30 @@ _require_scratch_dev_pool 4
 _require_btrfs_command inspect-internal dump-tree
 _require_btrfs_fs_feature raid56
 
-get_physical_stripe0()
+get_physical()
 {
-	$BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
-	grep " DATA\|RAID6" -A 10 | \
-	$AWK_PROG '($1 ~ /stripe/ && $3 ~ /devid/ && $2 ~ /0/) { print $6 }'
+	local stripe=$1
+        $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+		grep " DATA\|RAID6" -A 10 | \
+		awk "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
 }
 
-get_physical_stripe1()
+get_devid()
 {
-	$BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
-	grep " DATA\|RAID6" -A 10 | \
-	$AWK_PROG '($1 ~ /stripe/ && $3 ~ /devid/ && $2 ~ /1/) { print $6 }'
+	local stripe=$1
+        $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+		grep " DATA\|RAID6" -A 10 | \
+		awk "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
+}
+
+get_device_path()
+{
+	local devid=$1
+	echo "$SCRATCH_DEV_POOL" | awk "{print \$$devid}"
 }
 
 _scratch_dev_pool_get 4
-# step 1: create a raid6 btrfs and create a 4K file
+# step 1: create a raid6 btrfs and create a 128K file
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
 mkfs_opts="-d raid6 -b 1G"
@@ -80,18 +88,25 @@ _scratch_mount -o nospace_cache
 $XFS_IO_PROG -f -d -c "pwrite -S 0xaa 0 128K" -c "fsync" \
 	"$SCRATCH_MNT/foobar" | _filter_xfs_io
 
+logical=`${FILEFRAG_PROG} -v $SCRATCH_MNT/foobar | _filter_filefrag | cut -d '#' -f 1`
 _scratch_unmount
 
-stripe_0=`get_physical_stripe0`
-stripe_1=`get_physical_stripe1`
-dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
-dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
-
-# step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
-echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
-
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
+phy0=$(get_physical 0)
+devid0=$(get_devid 0)
+devpath0=$(get_device_path $devid0)
+phy1=$(get_physical 1)
+devid1=$(get_devid 1)
+devpath1=$(get_device_path $devid1)
+
+# step 2: corrupt stripe #0 and #1
+echo "step 2......simulate bitrot at:" >>$seqres.full
+echo "      ......stripe #0: devid $devid0 devpath $devpath0 phy $phy0" \
+	>>$seqres.full
+echo "      ......stripe #1: devid $devid1 devpath $devpath1 phy $phy1" \
+	>>$seqres.full
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $phy0 64K" $devpath0 > /dev/null
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $phy1 64K" $devpath1 > /dev/null
 
 # step 3: read foobar to repair the bitrot
 echo "step 3......repair the bitrot" >> $seqres.full
diff --git a/tests/btrfs/157.out b/tests/btrfs/157.out
index 08d592c4..d69c0f1d 100644
--- a/tests/btrfs/157.out
+++ b/tests/btrfs/157.out
@@ -1,10 +1,6 @@
 QA output created by 157
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 0200000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
 *
 0400000
diff --git a/tests/btrfs/158 b/tests/btrfs/158
index 603e8bea..c8d5af82 100755
--- a/tests/btrfs/158
+++ b/tests/btrfs/158
@@ -43,22 +43,30 @@ _require_scratch_dev_pool 4
 _require_btrfs_command inspect-internal dump-tree
 _require_btrfs_fs_feature raid56
 
-get_physical_stripe0()
+get_physical()
 {
-	$BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
-	grep " DATA\|RAID6" -A 10 | \
-	$AWK_PROG '($1 ~ /stripe/ && $3 ~ /devid/ && $2 ~ /0/) { print $6 }'
+	local stripe=$1
+        $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+		grep " DATA\|RAID6" -A 10 | \
+		awk "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$6 }"
 }
 
-get_physical_stripe1()
+get_devid()
 {
-	$BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
-	grep " DATA\|RAID6" -A 10 | \
-	$AWK_PROG '($1 ~ /stripe/ && $3 ~ /devid/ && $2 ~ /1/) { print $6 }'
+	local stripe=$1
+        $BTRFS_UTIL_PROG inspect-internal dump-tree -t 3 $SCRATCH_DEV | \
+		grep " DATA\|RAID6" -A 10 | \
+		awk "(\$1 ~ /stripe/ && \$3 ~ /devid/ && \$2 ~ /$stripe/) { print \$4 }"
+}
+
+get_device_path()
+{
+	local devid=$1
+	echo "$SCRATCH_DEV_POOL" | awk "{print \$$devid}"
 }
 
 _scratch_dev_pool_get 4
-# step 1: create a raid6 btrfs and create a 4K file
+# step 1: create a raid6 btrfs and create a 128K file
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
 mkfs_opts="-d raid6 -b 1G"
@@ -74,16 +82,22 @@ $XFS_IO_PROG -f -d -c "pwrite -S 0xaa 0 128K" -c "fsync" \
 
 _scratch_unmount
 
-stripe_0=`get_physical_stripe0`
-stripe_1=`get_physical_stripe1`
-dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
-dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
+phy0=$(get_physical 0)
+devid0=$(get_devid 0)
+devpath0=$(get_device_path $devid0)
+phy1=$(get_physical 1)
+devid1=$(get_devid 1)
+devpath1=$(get_device_path $devid1)
 
 # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
-echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
-
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
+echo "step 2......simulate bitrot at:" >>$seqres.full
+echo "      ......stripe #0: devid $devid0 devpath $devpath0 phy $phy0" \
+	>>$seqres.full
+echo "      ......stripe #1: devid $devid1 devpath $devpath1 phy $phy1" \
+	>>$seqres.full
+
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $phy0 64K" $devpath0 > /dev/null
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $phy1 64K" $devpath1 > /dev/null
 
 # step 3: scrub filesystem to repair the bitrot
 echo "step 3......repair the bitrot" >> $seqres.full
diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
index 1f5ad3f7..95562f49 100644
--- a/tests/btrfs/158.out
+++ b/tests/btrfs/158.out
@@ -1,10 +1,6 @@
 QA output created by 158
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
 *
 0400000
-- 
2.23.0


  parent reply	other threads:[~2019-12-11 10:40 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-11 10:40 [PATCH 0/3] fstests: btrfs/15[78] btrfs/14[23]: Use more accurate devid/phsyical for corruption Qu Wenruo
2019-12-11 10:40 ` [PATCH 1/3] fstests: common: Use more accurate kernel config for _require_fail_make_request Qu Wenruo
2019-12-17 17:24   ` Filipe Manana
2019-12-11 10:40 ` [PATCH 2/3] fstests: btrfs/14[23]: Use proper help to get both devid and physical offset for corruption Qu Wenruo
2019-12-11 11:24   ` Nikolay Borisov
2019-12-17 17:30   ` Filipe Manana
2019-12-11 10:40 ` Qu Wenruo [this message]
2019-12-11 11:24   ` [PATCH 3/3] fstests: btrfs/15[78]: Use proper helper " Nikolay Borisov
2019-12-17 17:33   ` Filipe Manana

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191211104029.25541-4-wqu@suse.com \
    --to=wqu@suse.com \
    --cc=fdmanana@suse.com \
    --cc=fstests@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.