[PATCH] fstests: btrfs/157 btrfs/158: Prevent stripe offset to pollute golden output

[PATCH] fstests: btrfs/157 btrfs/158: Prevent stripe offset to pollute golden output

[Qu Wenruo]

Test btrfs/157 and btrfs/158 are verifying the repair functionality of
supported RAID profile, thus it needs to corrupt the fs data manually using
physical offset of data.

However that physical offset of data is dependent on chunk layout, which
is further dependent on mkfs, so such physical offset is never reliable.

And btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
order to the stripe index") changed the mkfs stripe layout, the golden
output no longer matches the output.

This patch will remove the physical offset from golden output,
especially since we already have those offsets output in seqres.full.

Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/btrfs/157     | 5 +++--
 tests/btrfs/157.out | 4 ----
 tests/btrfs/158     | 4 ++--
 tests/btrfs/158.out | 4 ----
 4 files changed, 5 insertions(+), 12 deletions(-)

diff --git a/tests/btrfs/157 b/tests/btrfs/157
index 7f75c407..9895f1fd 100755
--- a/tests/btrfs/157
+++ b/tests/btrfs/157
@@ -90,8 +90,9 @@ dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
 # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
 echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
 
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
+# These stripe offset is mkfs dependent, don't pollute golden output
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 > /dev/null
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 > /dev/null
 
 # step 3: read foobar to repair the bitrot
 echo "step 3......repair the bitrot" >> $seqres.full
diff --git a/tests/btrfs/157.out b/tests/btrfs/157.out
index 08d592c4..d69c0f1d 100644
--- a/tests/btrfs/157.out
+++ b/tests/btrfs/157.out
@@ -1,10 +1,6 @@
 QA output created by 157
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 0200000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
 *
 0400000
diff --git a/tests/btrfs/158 b/tests/btrfs/158
index 603e8bea..99ee7fb7 100755
--- a/tests/btrfs/158
+++ b/tests/btrfs/158
@@ -82,8 +82,8 @@ dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
 # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
 echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
 
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 > /dev/null
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 > /dev/null
 
 # step 3: scrub filesystem to repair the bitrot
 echo "step 3......repair the bitrot" >> $seqres.full
diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
index 1f5ad3f7..95562f49 100644
--- a/tests/btrfs/158.out
+++ b/tests/btrfs/158.out
@@ -1,10 +1,6 @@
 QA output created by 158
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
 *
 0400000
-- 
2.23.0

[PATCH] fstests: common: Allow user to keep $seqres.dmesg for all tests

[Qu Wenruo]

Currently fstests will remove $seqres.dmesg if nothing wrong happened.
It saves some space, but sometimes it may not provide good enough
history for developers to check.
E.g. some unexpected dmesg from fs, but not serious enough to be caught
by current filter.

So instead of deleting the ordinary $seqres.dmesg, provide a new config:
KEEP_DMESG, to allow user to choose whether to keep the dmesg.

The default value for it is 0, which keeps the existing behavior by
deleting ordinary dmesg.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 common/config | 4 ++++
 common/rc     | 4 +++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/common/config b/common/config
index 1b75777f..b409f32c 100644
--- a/common/config
+++ b/common/config
@@ -22,6 +22,9 @@
 # RMT_IRIXTAPE_DEV- the IRIX remote tape device for the xfsdump tests
 # RMT_TAPE_USER -   remote user for tape device
 # SELINUX_MOUNT_OPTIONS - Options to use when SELinux is enabled.
+# KEEP_DMESG -      whether to keep all dmesg for each test case.
+#                   1: keep all dmesg
+#                   0: only keep dmesg with error/warning (default)
 #
 # - These can be added to $HOST_CONFIG_DIR (witch default to ./config)
 #   below or a separate local configuration file can be used (using
@@ -757,6 +760,7 @@ if [ -z "$CONFIG_INCLUDED" ]; then
 	[ -z "$TEST_FS_MOUNT_OPTS" ] && _test_mount_opts
 	[ -z "$MKFS_OPTIONS" ] && _mkfs_opts
 	[ -z "$FSCK_OPTIONS" ] && _fsck_opts
+	[ -z "$KEEP_DMESG" ] && export KEEP_DMESG=0
 else
 	# We get here for the non multi section case, on every test that sources
 	# common/rc after re-sourcing the HOST_OPTIONS config file.
diff --git a/common/rc b/common/rc
index e5535279..a1386f61 100644
--- a/common/rc
+++ b/common/rc
@@ -3634,7 +3634,9 @@ _check_dmesg()
 		_dump_err "_check_dmesg: something found in dmesg (see $seqres.dmesg)"
 		return 1
 	else
-		rm -f $seqres.dmesg
+		if [ "$KEEP_DMESG" != 1 ]; then
+			rm -f $seqres.dmesg
+		fi
 		return 0
 	fi
 }
-- 
2.23.0

Re: [PATCH] fstests: btrfs/157 btrfs/158: Prevent stripe offset to pollute golden output

[Qu WenRuo]

[-- Attachment #1.1: Type: text/plain, Size: 4160 bytes --]



On 2019/12/11 上午10:22, Qu Wenruo wrote:
> Test btrfs/157 and btrfs/158 are verifying the repair functionality of
> supported RAID profile, thus it needs to corrupt the fs data manually using
> physical offset of data.
> 
> However that physical offset of data is dependent on chunk layout, which
> is further dependent on mkfs, so such physical offset is never reliable.
> 
> And btrfs-progs commit c501c9e3b816 ("btrfs-progs: mkfs: match devid
> order to the stripe index") changed the mkfs stripe layout, the golden
> output no longer matches the output.
> 
> This patch will remove the physical offset from golden output,
> especially since we already have those offsets output in seqres.full.
> 
> Reported-by: Filipe Manana <fdmanana@suse.com>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  tests/btrfs/157     | 5 +++--
>  tests/btrfs/157.out | 4 ----
>  tests/btrfs/158     | 4 ++--
>  tests/btrfs/158.out | 4 ----
>  4 files changed, 5 insertions(+), 12 deletions(-)
> 
> diff --git a/tests/btrfs/157 b/tests/btrfs/157
> index 7f75c407..9895f1fd 100755
> --- a/tests/btrfs/157
> +++ b/tests/btrfs/157
> @@ -90,8 +90,9 @@ dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
>  # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
>  echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
>  
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
> +# These stripe offset is mkfs dependent, don't pollute golden output
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 > /dev/null
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 > /dev/null

Please discard this version, it doesn't take devid into consideration.

We need a more flex way to not only get physical offset from a stripe,
but also its devid.

I'll update the patch to be more flex.

Thanks,
Qu

>  
>  # step 3: read foobar to repair the bitrot
>  echo "step 3......repair the bitrot" >> $seqres.full
> diff --git a/tests/btrfs/157.out b/tests/btrfs/157.out
> index 08d592c4..d69c0f1d 100644
> --- a/tests/btrfs/157.out
> +++ b/tests/btrfs/157.out
> @@ -1,10 +1,6 @@
>  QA output created by 157
>  wrote 131072/131072 bytes at offset 0
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  0200000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>  *
>  0400000
> diff --git a/tests/btrfs/158 b/tests/btrfs/158
> index 603e8bea..99ee7fb7 100755
> --- a/tests/btrfs/158
> +++ b/tests/btrfs/158
> @@ -82,8 +82,8 @@ dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
>  # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
>  echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
>  
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 > /dev/null
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 > /dev/null
>  
>  # step 3: scrub filesystem to repair the bitrot
>  echo "step 3......repair the bitrot" >> $seqres.full
> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
> index 1f5ad3f7..95562f49 100644
> --- a/tests/btrfs/158.out
> +++ b/tests/btrfs/158.out
> @@ -1,10 +1,6 @@
>  QA output created by 158
>  wrote 131072/131072 bytes at offset 0
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>  *
>  0400000
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCHi RFC] fstest: btrfs/158 fix miss-aligned stripe and device

[Anand Jain]

We changed the order of the allocation on the devices, and
so the test cases which are hard coded to find specific stripe
on the specific device gets failed. So fix it with the new layout.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
Qu, Right we need to fix the dev in the test case as well.
    I saw your patches bit late. Here is what I had.. you may
    use it. So I am marking this patch as RFC.
Thanks.

 tests/btrfs/158     | 10 +++++-----
 tests/btrfs/158.out |  4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/btrfs/158 b/tests/btrfs/158
index 603e8bea9b7e..7f2066384f55 100755
--- a/tests/btrfs/158
+++ b/tests/btrfs/158
@@ -76,14 +76,14 @@ _scratch_unmount
 
 stripe_0=`get_physical_stripe0`
 stripe_1=`get_physical_stripe1`
-dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
-dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
+dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
 
 # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
-echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
+echo "step 2......simulate bitrot at offset $stripe_0 of device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
 
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
-$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 | _filter_xfs_io
+$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 | _filter_xfs_io
 
 # step 3: scrub filesystem to repair the bitrot
 echo "step 3......repair the bitrot" >> $seqres.full
diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
index 1f5ad3f76917..5cdaeb238c62 100644
--- a/tests/btrfs/158.out
+++ b/tests/btrfs/158.out
@@ -1,9 +1,9 @@
 QA output created by 158
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
+wrote 65536/65536 bytes at offset 22020096
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 9437184
+wrote 65536/65536 bytes at offset 1048576
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
 *
-- 
1.8.3.1

Re: [PATCHi RFC] fstest: btrfs/158 fix miss-aligned stripe and device

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3083 bytes --]



On 2019/12/11 下午2:08, Anand Jain wrote:
> We changed the order of the allocation on the devices, and
> so the test cases which are hard coded to find specific stripe
> on the specific device gets failed. So fix it with the new layout.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
> Qu, Right we need to fix the dev in the test case as well.
>     I saw your patches bit late. Here is what I had.. you may
>     use it. So I am marking this patch as RFC.

I am crafting a better solution, to handle both behavior (and even
future behavior), by getting both devid and physical offset.

And I tend to remove the fail_make_request requirement from some tests,
and direct read with multiple try should be enough to trigger repair for
test btrfs/142 and btrfs/143.


In fact, I don't believe your current fix is good enough to handle both
old and new mkfs.btrfs.

So we need to investigate more for raid repair test cases to make them
future proof.

Thanks,
Qu
> Thanks.
> 
>  tests/btrfs/158     | 10 +++++-----
>  tests/btrfs/158.out |  4 ++--
>  2 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/btrfs/158 b/tests/btrfs/158
> index 603e8bea9b7e..7f2066384f55 100755
> --- a/tests/btrfs/158
> +++ b/tests/btrfs/158
> @@ -76,14 +76,14 @@ _scratch_unmount
>  
>  stripe_0=`get_physical_stripe0`
>  stripe_1=`get_physical_stripe1`
> -dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
> -dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
> +dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
> +dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>  
>  # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
> -echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
> +echo "step 2......simulate bitrot at offset $stripe_0 of device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
>  
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 | _filter_xfs_io
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 | _filter_xfs_io
>  
>  # step 3: scrub filesystem to repair the bitrot
>  echo "step 3......repair the bitrot" >> $seqres.full
> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
> index 1f5ad3f76917..5cdaeb238c62 100644
> --- a/tests/btrfs/158.out
> +++ b/tests/btrfs/158.out
> @@ -1,9 +1,9 @@
>  QA output created by 158
>  wrote 131072/131072 bytes at offset 0
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> +wrote 65536/65536 bytes at offset 22020096
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> -wrote 65536/65536 bytes at offset 9437184
> +wrote 65536/65536 bytes at offset 1048576
>  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>  *
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCHi RFC] fstest: btrfs/158 fix miss-aligned stripe and device

[Anand Jain]

On 12/11/19 3:08 PM, Qu Wenruo wrote:
> 
> 
> On 2019/12/11 下午2:08, Anand Jain wrote:
>> We changed the order of the allocation on the devices, and
>> so the test cases which are hard coded to find specific stripe
>> on the specific device gets failed. So fix it with the new layout.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>> ---
>> Qu, Right we need to fix the dev in the test case as well.
>>      I saw your patches bit late. Here is what I had.. you may
>>      use it. So I am marking this patch as RFC.
> 
> I am crafting a better solution, to handle both behavior (and even
> future behavior), by getting both devid and physical offset.

  Yep helper function using either,
    devid and btrfs fi show
    or
    uuid  and blkid

  would dynamically find the right device.

  I am ok with either.

> And I tend to remove the fail_make_request requirement from some tests,
> and direct read with multiple try should be enough to trigger repair for
> test btrfs/142 and btrfs/143.
> 

> In fact, I don't believe your current fix is good enough to handle both
> old and new mkfs.btrfs.

  It was designed to handle only forward compatible.

Thanks, Anand

> So we need to investigate more for raid repair test cases to make them
> future proof.
> 
> Thanks,
> Qu
>> Thanks.
>>
>>   tests/btrfs/158     | 10 +++++-----
>>   tests/btrfs/158.out |  4 ++--
>>   2 files changed, 7 insertions(+), 7 deletions(-)
>>
>> diff --git a/tests/btrfs/158 b/tests/btrfs/158
>> index 603e8bea9b7e..7f2066384f55 100755
>> --- a/tests/btrfs/158
>> +++ b/tests/btrfs/158
>> @@ -76,14 +76,14 @@ _scratch_unmount
>>   
>>   stripe_0=`get_physical_stripe0`
>>   stripe_1=`get_physical_stripe1`
>> -dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
>> -dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
>> +dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>> +dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>>   
>>   # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
>> -echo "step 2......simulate bitrot at offset $stripe_0 of device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
>> +echo "step 2......simulate bitrot at offset $stripe_0 of device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
>>   
>> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 | _filter_xfs_io
>> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 | _filter_xfs_io
>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 | _filter_xfs_io
>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 | _filter_xfs_io
>>   
>>   # step 3: scrub filesystem to repair the bitrot
>>   echo "step 3......repair the bitrot" >> $seqres.full
>> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
>> index 1f5ad3f76917..5cdaeb238c62 100644
>> --- a/tests/btrfs/158.out
>> +++ b/tests/btrfs/158.out
>> @@ -1,9 +1,9 @@
>>   QA output created by 158
>>   wrote 131072/131072 bytes at offset 0
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>> -wrote 65536/65536 bytes at offset 9437184
>> +wrote 65536/65536 bytes at offset 22020096
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>> -wrote 65536/65536 bytes at offset 9437184
>> +wrote 65536/65536 bytes at offset 1048576
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>   0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>>   *
>>
> 

Re: [PATCHi RFC] fstest: btrfs/158 fix miss-aligned stripe and device

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 4065 bytes --]



On 2019/12/11 下午3:23, Anand Jain wrote:
> 
> 
> On 12/11/19 3:08 PM, Qu Wenruo wrote:
>>
>>
>> On 2019/12/11 下午2:08, Anand Jain wrote:
>>> We changed the order of the allocation on the devices, and
>>> so the test cases which are hard coded to find specific stripe
>>> on the specific device gets failed. So fix it with the new layout.
>>>
>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>> ---
>>> Qu, Right we need to fix the dev in the test case as well.
>>>      I saw your patches bit late. Here is what I had.. you may
>>>      use it. So I am marking this patch as RFC.
>>
>> I am crafting a better solution, to handle both behavior (and even
>> future behavior), by getting both devid and physical offset.
> 
>  Yep helper function using either,
>    devid and btrfs fi show
>    or
>    uuid  and blkid
> 
>  would dynamically find the right device.

My current helper is going to rely on the fact that all mkfs.btrfs
assigned devid sequentially.
Which means "mkfs.btrfs -f $dev1 $dev2" will always assigned devid 1 to
$dev1, and devid 2 to $dev2.

As long as we don't touch that part, we should be OK.
(And I really hope we won't touch that part).

> 
>  I am ok with either.
> 
>> And I tend to remove the fail_make_request requirement from some tests,
>> and direct read with multiple try should be enough to trigger repair for
>> test btrfs/142 and btrfs/143.
>>
> 
>> In fact, I don't believe your current fix is good enough to handle both
>> old and new mkfs.btrfs.
> 
>  It was designed to handle only forward compatible.

Then both Oracle and SUSE QA people will hate such tests...

Thanks,
Qu

> 
> Thanks, Anand
> 
>> So we need to investigate more for raid repair test cases to make them
>> future proof.
>>
>> Thanks,
>> Qu
>>> Thanks.
>>>
>>>   tests/btrfs/158     | 10 +++++-----
>>>   tests/btrfs/158.out |  4 ++--
>>>   2 files changed, 7 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/tests/btrfs/158 b/tests/btrfs/158
>>> index 603e8bea9b7e..7f2066384f55 100755
>>> --- a/tests/btrfs/158
>>> +++ b/tests/btrfs/158
>>> @@ -76,14 +76,14 @@ _scratch_unmount
>>>     stripe_0=`get_physical_stripe0`
>>>   stripe_1=`get_physical_stripe1`
>>> -dev4=`echo $SCRATCH_DEV_POOL | awk '{print $4}'`
>>> -dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
>>> +dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>>> +dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>>>     # step 2: corrupt the 1st and 2nd stripe (stripe 0 and 1)
>>> -echo "step 2......simulate bitrot at offset $stripe_0 of
>>> device_4($dev4) and offset $stripe_1 of device_3($dev3)" >>$seqres.full
>>> +echo "step 2......simulate bitrot at offset $stripe_0 of
>>> device_1($dev1) and offset $stripe_1 of device_2($dev2)" >>$seqres.full
>>>   -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev4 |
>>> _filter_xfs_io
>>> -$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev3 |
>>> _filter_xfs_io
>>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_0 64K" $dev1 |
>>> _filter_xfs_io
>>> +$XFS_IO_PROG -f -d -c "pwrite -S 0xbb $stripe_1 64K" $dev2 |
>>> _filter_xfs_io
>>>     # step 3: scrub filesystem to repair the bitrot
>>>   echo "step 3......repair the bitrot" >> $seqres.full
>>> diff --git a/tests/btrfs/158.out b/tests/btrfs/158.out
>>> index 1f5ad3f76917..5cdaeb238c62 100644
>>> --- a/tests/btrfs/158.out
>>> +++ b/tests/btrfs/158.out
>>> @@ -1,9 +1,9 @@
>>>   QA output created by 158
>>>   wrote 131072/131072 bytes at offset 0
>>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>> -wrote 65536/65536 bytes at offset 9437184
>>> +wrote 65536/65536 bytes at offset 22020096
>>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>> -wrote 65536/65536 bytes at offset 9437184
>>> +wrote 65536/65536 bytes at offset 1048576
>>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>   0000000 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
>>>   *
>>>
>>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCHi RFC] fstest: btrfs/158 fix miss-aligned stripe and device

[David Sterba]

On Wed, Dec 11, 2019 at 03:23:06PM +0800, Anand Jain wrote:
> > And I tend to remove the fail_make_request requirement from some tests,
> > and direct read with multiple try should be enough to trigger repair for
> > test btrfs/142 and btrfs/143.
> > 
> 
> > In fact, I don't believe your current fix is good enough to handle both
> > old and new mkfs.btrfs.
> 
>   It was designed to handle only forward compatible.

If possible the tests should be robust and check assumptions and also
not hardcode eg. fixed block offset values that could change for various
reasons. Forward compatibility makes it hard to test backports, not only
on enterprise kernels but also on older stable kernels. We're in a state
where developers have list of tests to ignore because they known to be
bogus or unreliable.