* [PATCH] btrfs/154: test for device dynamic rescan
@ 2017-11-13 2:25 Anand Jain
2017-11-14 12:12 ` Eryu Guan
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Anand Jain @ 2017-11-13 2:25 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Make sure missing device is included in the alloc list when it is
scanned on a mounted FS.
This test case needs btrfs kernel patch which is in the ML
[PATCH] btrfs: handle dynamically reappearing missing device
Without the kernel patch, the test will run, but reports as
failed, as the device scanned won't appear in the alloc_list.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
tests/btrfs/154 | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/154.out | 10 +++
tests/btrfs/group | 1 +
3 files changed, 199 insertions(+)
create mode 100755 tests/btrfs/154
create mode 100644 tests/btrfs/154.out
diff --git a/tests/btrfs/154 b/tests/btrfs/154
new file mode 100755
index 000000000000..8b06fc4d9347
--- /dev/null
+++ b/tests/btrfs/154
@@ -0,0 +1,188 @@
+#! /bin/bash
+# FS QA Test 154
+#
+# Test for reappearing missing device functionality.
+# This test will fail without the btrfs kernel patch
+# [PATCH] btrfs: handle dynamically reappearing missing device
+#
+#---------------------------------------------------------------------
+# Copyright (c) 2017 Oracle. All Rights Reserved.
+# Author: Anand Jain <anand.jain@oracle.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#---------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/module
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch_dev_pool 2
+_test_unmount
+_require_loadable_fs_module "btrfs"
+
+_scratch_dev_pool_get 2
+
+DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+
+echo DEV1=$DEV1 >> $seqres.full
+echo DEV2=$DEV2 >> $seqres.full
+
+# Balance won't be successful if filled too much
+DEV1_SZ=`blockdev --getsize64 $DEV1`
+DEV2_SZ=`blockdev --getsize64 $DEV2`
+
+# get min
+MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
+# Need disks with more than 2G
+if [ $MAX_FS_SZ -lt 2000000000 ]; then
+ _scratch_dev_pool_put
+ _test_mount
+ _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
+fi
+
+MAX_FS_SZ=100000000
+bs="1M"
+COUNT=$(($MAX_FS_SZ / 1000000))
+CHECKPOINT1=0
+CHECKPOINT2=0
+
+setup()
+{
+ echo >> $seqres.full
+ echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
+ echo "setup"
+ echo "-----setup-----" >> $seqres.full
+ _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
+ _scratch_mount >> $seqres.full 2>&1
+ dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
+ >>$seqres.full 2>&1
+ _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
+ _run_btrfs_util_prog filesystem df $SCRATCH_MNT
+ COUNT=$(( $COUNT - 1 ))
+ echo "unmount" >> $seqres.full
+ _scratch_unmount
+}
+
+degrade_mount_write()
+{
+ echo >> $seqres.full
+ echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
+ echo
+ echo "degraded mount"
+
+ echo "clean btrfs ko" >> $seqres.full
+ # un-scan the btrfs devices
+ _reload_fs_module "btrfs"
+ _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
+ cnt=$(( $COUNT/10 ))
+ dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
+ >>$seqres.full 2>&1
+ COUNT=$(( $COUNT - $cnt ))
+ _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+ _run_btrfs_util_prog filesystem df $SCRATCH_MNT
+ CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
+ echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full 2>&1
+}
+
+scan_missing_dev_and_write()
+{
+ echo >> $seqres.full
+ echo "--scan missing $DEV2--" >> $seqres.full
+ echo
+ echo "scan missing dev and write"
+
+ _run_btrfs_util_prog device scan $DEV2
+
+ echo >> $seqres.full
+
+ _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
+ _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
+
+ dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
+ >>$seqres.full 2>&1
+ CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
+ echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full 2>&1
+}
+
+balance_convert()
+{
+ echo >> $seqres.full
+ echo "----- run balance -----" >> $seqres.full
+ echo
+ echo "run balance"
+
+ _run_btrfs_util_prog bal start --full-balance -dconvert=raid1 \
+ -mconvert=raid1 ${SCRATCH_MNT}
+}
+
+verify()
+{
+ echo >> $seqres.full
+ echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
+ echo
+ echo "mount reconstructed dev only and check md5sum"
+ echo "unmount" >> $seqres.full
+
+ _scratch_unmount
+ _reload_fs_module "btrfs"
+ _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
+ verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
+ verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
+
+ if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
+ echo "checkpoint1 does not match with after scrub"
+ fi
+
+ if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
+ echo "checkpoint2 does not match with after scrub"
+ fi
+}
+
+setup
+degrade_mount_write
+scan_missing_dev_and_write
+balance_convert
+verify
+
+$UMOUNT_PROG $DEV2
+_scratch_dev_pool_put
+_test_mount
+
+status=0
+exit
diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
new file mode 100644
index 000000000000..4a4939d448aa
--- /dev/null
+++ b/tests/btrfs/154.out
@@ -0,0 +1,10 @@
+QA output created by 154
+setup
+
+degraded mount
+
+scan missing dev and write
+
+run balance
+
+mount reconstructed dev only and check md5sum
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 15c2ecbf553c..c98cf823224e 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -156,3 +156,4 @@
151 auto quick
152 auto quick metadata qgroup send
153 auto quick qgroup
+154 auto quick
--
2.13.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] btrfs/154: test for device dynamic rescan
2017-11-13 2:25 [PATCH] btrfs/154: test for device dynamic rescan Anand Jain
@ 2017-11-14 12:12 ` Eryu Guan
2017-11-15 3:04 ` Anand Jain
2017-11-15 3:05 ` [PATCH v2] " Anand Jain
2018-10-21 9:16 ` [PATCH] " Filipe Manana
2 siblings, 1 reply; 10+ messages in thread
From: Eryu Guan @ 2017-11-14 12:12 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Mon, Nov 13, 2017 at 10:25:41AM +0800, Anand Jain wrote:
> Make sure missing device is included in the alloc list when it is
> scanned on a mounted FS.
>
> This test case needs btrfs kernel patch which is in the ML
> [PATCH] btrfs: handle dynamically reappearing missing device
> Without the kernel patch, the test will run, but reports as
> failed, as the device scanned won't appear in the alloc_list.
>
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
Tested without the fix and test failed as expected, test passed after
applying the fix.
Some minor nits below.
> ---
> tests/btrfs/154 | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/154.out | 10 +++
> tests/btrfs/group | 1 +
> 3 files changed, 199 insertions(+)
> create mode 100755 tests/btrfs/154
> create mode 100644 tests/btrfs/154.out
>
> diff --git a/tests/btrfs/154 b/tests/btrfs/154
> new file mode 100755
> index 000000000000..8b06fc4d9347
> --- /dev/null
> +++ b/tests/btrfs/154
> @@ -0,0 +1,188 @@
> +#! /bin/bash
> +# FS QA Test 154
> +#
> +# Test for reappearing missing device functionality.
> +# This test will fail without the btrfs kernel patch
> +# [PATCH] btrfs: handle dynamically reappearing missing device
> +#
> +#---------------------------------------------------------------------
> +# Copyright (c) 2017 Oracle. All Rights Reserved.
> +# Author: Anand Jain <anand.jain@oracle.com>
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/module
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_test_unmount
This is not needed now, _require_loadable_fs_module will umount & mount
test dev as necessary.
> +_require_loadable_fs_module "btrfs"
> +
> +_scratch_dev_pool_get 2
> +
> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
> +
> +echo DEV1=$DEV1 >> $seqres.full
> +echo DEV2=$DEV2 >> $seqres.full
> +
> +# Balance won't be successful if filled too much
> +DEV1_SZ=`blockdev --getsize64 $DEV1`
> +DEV2_SZ=`blockdev --getsize64 $DEV2`
> +
> +# get min
> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
> +# Need disks with more than 2G
> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
> + _scratch_dev_pool_put
> + _test_mount
Then no need to _test_mount.
> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
> +fi
> +
> +MAX_FS_SZ=100000000
> +bs="1M"
> +COUNT=$(($MAX_FS_SZ / 1000000))
> +CHECKPOINT1=0
> +CHECKPOINT2=0
> +
> +setup()
> +{
> + echo >> $seqres.full
> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
> + echo "setup"
> + echo "-----setup-----" >> $seqres.full
> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
> + _scratch_mount >> $seqres.full 2>&1
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
> + >>$seqres.full 2>&1
> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> + COUNT=$(( $COUNT - 1 ))
> + echo "unmount" >> $seqres.full
> + _scratch_unmount
> +}
> +
> +degrade_mount_write()
> +{
> + echo >> $seqres.full
> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
> + echo
> + echo "degraded mount"
> +
> + echo "clean btrfs ko" >> $seqres.full
> + # un-scan the btrfs devices
> + _reload_fs_module "btrfs"
> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
> + cnt=$(( $COUNT/10 ))
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
> + >>$seqres.full 2>&1
> + COUNT=$(( $COUNT - $cnt ))
> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full 2>&1
"2>&1" not needed.
> +}
> +
> +scan_missing_dev_and_write()
> +{
> + echo >> $seqres.full
> + echo "--scan missing $DEV2--" >> $seqres.full
> + echo
> + echo "scan missing dev and write"
> +
> + _run_btrfs_util_prog device scan $DEV2
> +
> + echo >> $seqres.full
> +
> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
> +
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
> + >>$seqres.full 2>&1
> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full 2>&1
Same here.
> +}
> +
> +balance_convert()
> +{
> + echo >> $seqres.full
> + echo "----- run balance -----" >> $seqres.full
> + echo
> + echo "run balance"
> +
> + _run_btrfs_util_prog bal start --full-balance -dconvert=raid1 \
All btrfs tests use full btrfs sub-command now, use "balance" too?
> + -mconvert=raid1 ${SCRATCH_MNT}
> +}
> +
> +verify()
> +{
> + echo >> $seqres.full
> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
> + echo
> + echo "mount reconstructed dev only and check md5sum"
> + echo "unmount" >> $seqres.full
> +
> + _scratch_unmount
> + _reload_fs_module "btrfs"
> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
> +
> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
> + echo "checkpoint1 does not match with after scrub"
I don't see scrub involved, you meant "scan"?
Thanks,
Eryu
> + fi
> +
> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
> + echo "checkpoint2 does not match with after scrub"
> + fi
> +}
> +
> +setup
> +degrade_mount_write
> +scan_missing_dev_and_write
> +balance_convert
> +verify
> +
> +$UMOUNT_PROG $DEV2
> +_scratch_dev_pool_put
> +_test_mount
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
> new file mode 100644
> index 000000000000..4a4939d448aa
> --- /dev/null
> +++ b/tests/btrfs/154.out
> @@ -0,0 +1,10 @@
> +QA output created by 154
> +setup
> +
> +degraded mount
> +
> +scan missing dev and write
> +
> +run balance
> +
> +mount reconstructed dev only and check md5sum
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 15c2ecbf553c..c98cf823224e 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -156,3 +156,4 @@
> 151 auto quick
> 152 auto quick metadata qgroup send
> 153 auto quick qgroup
> +154 auto quick
> --
> 2.13.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] btrfs/154: test for device dynamic rescan
2017-11-14 12:12 ` Eryu Guan
@ 2017-11-15 3:04 ` Anand Jain
0 siblings, 0 replies; 10+ messages in thread
From: Anand Jain @ 2017-11-15 3:04 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs
On 11/14/2017 08:12 PM, Eryu Guan wrote:
> On Mon, Nov 13, 2017 at 10:25:41AM +0800, Anand Jain wrote:
>> Make sure missing device is included in the alloc list when it is
>> scanned on a mounted FS.
>>
>> This test case needs btrfs kernel patch which is in the ML
>> [PATCH] btrfs: handle dynamically reappearing missing device
>> Without the kernel patch, the test will run, but reports as
>> failed, as the device scanned won't appear in the alloc_list.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>
> Tested without the fix and test failed as expected, test passed after
> applying the fix.
> Some minor nits below.
>
>> ---
>> tests/btrfs/154 | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/154.out | 10 +++
>> tests/btrfs/group | 1 +
>> 3 files changed, 199 insertions(+)
>> create mode 100755 tests/btrfs/154
>> create mode 100644 tests/btrfs/154.out
>>
>> diff --git a/tests/btrfs/154 b/tests/btrfs/154
>> new file mode 100755
>> index 000000000000..8b06fc4d9347
>> --- /dev/null
>> +++ b/tests/btrfs/154
>> @@ -0,0 +1,188 @@
>> +#! /bin/bash
>> +# FS QA Test 154
>> +#
>> +# Test for reappearing missing device functionality.
>> +# This test will fail without the btrfs kernel patch
>> +# [PATCH] btrfs: handle dynamically reappearing missing device
>> +#
>> +#---------------------------------------------------------------------
>> +# Copyright (c) 2017 Oracle. All Rights Reserved.
>> +# Author: Anand Jain <anand.jain@oracle.com>
>> +#
>> +# This program is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU General Public License as
>> +# published by the Free Software Foundation.
>> +#
>> +# This program is distributed in the hope that it would be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program; if not, write the Free Software Foundation,
>> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>> +#---------------------------------------------------------------------
>> +#
>> +
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1 # failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> + cd /
>> + rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/module
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs btrfs
>> +_supported_os Linux
>> +_require_scratch_dev_pool 2
>> +_test_unmount
>
> This is not needed now, _require_loadable_fs_module will umount & mount
> test dev as necessary.
Right will fix it.
>> +_require_loadable_fs_module "btrfs"
>> +
>> +_scratch_dev_pool_get 2
>> +
>> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>> +
>> +echo DEV1=$DEV1 >> $seqres.full
>> +echo DEV2=$DEV2 >> $seqres.full
>> +
>> +# Balance won't be successful if filled too much
>> +DEV1_SZ=`blockdev --getsize64 $DEV1`
>> +DEV2_SZ=`blockdev --getsize64 $DEV2`
>> +
>> +# get min
>> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
>> +# Need disks with more than 2G
>> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
>> + _scratch_dev_pool_put
>> + _test_mount
>
> Then no need to _test_mount.
Fixed this in v2.
>> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
>> +fi
>> +
>> +MAX_FS_SZ=100000000
>> +bs="1M"
>> +COUNT=$(($MAX_FS_SZ / 1000000))
>> +CHECKPOINT1=0
>> +CHECKPOINT2=0
>> +
>> +setup()
>> +{
>> + echo >> $seqres.full
>> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
>> + echo "setup"
>> + echo "-----setup-----" >> $seqres.full
>> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
>> + _scratch_mount >> $seqres.full 2>&1
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
>> + >>$seqres.full 2>&1
>> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
>> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
>> + COUNT=$(( $COUNT - 1 ))
>> + echo "unmount" >> $seqres.full
>> + _scratch_unmount
>> +}
>> +
>> +degrade_mount_write()
>> +{
>> + echo >> $seqres.full
>> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
>> + echo
>> + echo "degraded mount"
>> +
>> + echo "clean btrfs ko" >> $seqres.full
>> + # un-scan the btrfs devices
>> + _reload_fs_module "btrfs"
>> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
>> + cnt=$(( $COUNT/10 ))
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
>> + >>$seqres.full 2>&1
>> + COUNT=$(( $COUNT - $cnt ))
>> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
>> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
>> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
>> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full 2>&1
>
> "2>&1" not needed.
Fixed in v2.
>> +}
>> +
>> +scan_missing_dev_and_write()
>> +{
>> + echo >> $seqres.full
>> + echo "--scan missing $DEV2--" >> $seqres.full
>> + echo
>> + echo "scan missing dev and write"
>> +
>> + _run_btrfs_util_prog device scan $DEV2
>> +
>> + echo >> $seqres.full
>> +
>> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
>> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
>> +
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
>> + >>$seqres.full 2>&1
>> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
>> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full 2>&1
>
> Same here.
Fixed in v2.
>> +}
>> +
>> +balance_convert()
>> +{
>> + echo >> $seqres.full
>> + echo "----- run balance -----" >> $seqres.full
>> + echo
>> + echo "run balance"
>> +
>> + _run_btrfs_util_prog bal start --full-balance -dconvert=raid1 \
>
> All btrfs tests use full btrfs sub-command now, use "balance" too?
Will replace with balance sub-command in v2.
>> + -mconvert=raid1 ${SCRATCH_MNT}
>> +}
>> +
>> +verify()
>> +{
>> + echo >> $seqres.full
>> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
>> + echo
>> + echo "mount reconstructed dev only and check md5sum"
>> + echo "unmount" >> $seqres.full
>> +
>> + _scratch_unmount
>> + _reload_fs_module "btrfs"
>> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
>> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
>> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
>> +
>> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
>> + echo "checkpoint1 does not match with after scrub"
>
> I don't see scrub involved, you meant "scan"?
Right. I have updated this as well.
Sent out V2 with these fixes.
Thanks, Anand
> Thanks,
> Eryu
>
>> + fi
>> +
>> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
>> + echo "checkpoint2 does not match with after scrub"
>> + fi
>> +}
>> +
>> +setup
>> +degrade_mount_write
>> +scan_missing_dev_and_write
>> +balance_convert
>> +verify
>> +
>> +$UMOUNT_PROG $DEV2
>> +_scratch_dev_pool_put
>> +_test_mount
>> +
>> +status=0
>> +exit
>> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
>> new file mode 100644
>> index 000000000000..4a4939d448aa
>> --- /dev/null
>> +++ b/tests/btrfs/154.out
>> @@ -0,0 +1,10 @@
>> +QA output created by 154
>> +setup
>> +
>> +degraded mount
>> +
>> +scan missing dev and write
>> +
>> +run balance
>> +
>> +mount reconstructed dev only and check md5sum
>> diff --git a/tests/btrfs/group b/tests/btrfs/group
>> index 15c2ecbf553c..c98cf823224e 100644
>> --- a/tests/btrfs/group
>> +++ b/tests/btrfs/group
>> @@ -156,3 +156,4 @@
>> 151 auto quick
>> 152 auto quick metadata qgroup send
>> 153 auto quick qgroup
>> +154 auto quick
>> --
>> 2.13.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe fstests" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2] btrfs/154: test for device dynamic rescan
2017-11-13 2:25 [PATCH] btrfs/154: test for device dynamic rescan Anand Jain
2017-11-14 12:12 ` Eryu Guan
@ 2017-11-15 3:05 ` Anand Jain
2017-11-15 6:47 ` Eryu Guan
2018-10-21 9:16 ` [PATCH] " Filipe Manana
2 siblings, 1 reply; 10+ messages in thread
From: Anand Jain @ 2017-11-15 3:05 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Make sure missing device is included in the alloc list when it is
scanned on a mounted FS.
This test case needs btrfs kernel patch which is in the ML
[PATCH] btrfs: handle dynamically reappearing missing device
Without the kernel patch, the test will run, but reports as
failed, as the device scanned won't appear in the alloc_list.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
v2: Fixed review comments.
tests/btrfs/154 | 186 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/154.out | 10 +++
tests/btrfs/group | 1 +
3 files changed, 197 insertions(+)
create mode 100755 tests/btrfs/154
create mode 100644 tests/btrfs/154.out
diff --git a/tests/btrfs/154 b/tests/btrfs/154
new file mode 100755
index 000000000000..73a185157389
--- /dev/null
+++ b/tests/btrfs/154
@@ -0,0 +1,186 @@
+#! /bin/bash
+# FS QA Test 154
+#
+# Test for reappearing missing device functionality.
+# This test will fail without the btrfs kernel patch
+# [PATCH] btrfs: handle dynamically reappearing missing device
+#
+#---------------------------------------------------------------------
+# Copyright (c) 2017 Oracle. All Rights Reserved.
+# Author: Anand Jain <anand.jain@oracle.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#---------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/module
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch_dev_pool 2
+_require_loadable_fs_module "btrfs"
+
+_scratch_dev_pool_get 2
+
+DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+
+echo DEV1=$DEV1 >> $seqres.full
+echo DEV2=$DEV2 >> $seqres.full
+
+# Balance won't be successful if filled too much
+DEV1_SZ=`blockdev --getsize64 $DEV1`
+DEV2_SZ=`blockdev --getsize64 $DEV2`
+
+# get min
+MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
+# Need disks with more than 2G
+if [ $MAX_FS_SZ -lt 2000000000 ]; then
+ _scratch_dev_pool_put
+ _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
+fi
+
+MAX_FS_SZ=100000000
+bs="1M"
+COUNT=$(($MAX_FS_SZ / 1000000))
+CHECKPOINT1=0
+CHECKPOINT2=0
+
+setup()
+{
+ echo >> $seqres.full
+ echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
+ echo "setup"
+ echo "-----setup-----" >> $seqres.full
+ _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
+ _scratch_mount >> $seqres.full 2>&1
+ dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
+ >>$seqres.full 2>&1
+ _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
+ _run_btrfs_util_prog filesystem df $SCRATCH_MNT
+ COUNT=$(( $COUNT - 1 ))
+ echo "unmount" >> $seqres.full
+ _scratch_unmount
+}
+
+degrade_mount_write()
+{
+ echo >> $seqres.full
+ echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
+ echo
+ echo "degraded mount"
+
+ echo "clean btrfs ko" >> $seqres.full
+ # un-scan the btrfs devices
+ _reload_fs_module "btrfs"
+ _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
+ cnt=$(( $COUNT/10 ))
+ dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
+ >>$seqres.full 2>&1
+ COUNT=$(( $COUNT - $cnt ))
+ _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+ _run_btrfs_util_prog filesystem df $SCRATCH_MNT
+ CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
+ echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full
+}
+
+scan_missing_dev_and_write()
+{
+ echo >> $seqres.full
+ echo "--scan missing $DEV2--" >> $seqres.full
+ echo
+ echo "scan missing dev and write"
+
+ _run_btrfs_util_prog device scan $DEV2
+
+ echo >> $seqres.full
+
+ _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
+ _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
+
+ dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
+ >>$seqres.full 2>&1
+ CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
+ echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full
+}
+
+balance_convert()
+{
+ echo >> $seqres.full
+ echo "----- run balance -----" >> $seqres.full
+ echo
+ echo "run balance"
+
+ _run_btrfs_util_prog balance start --full-balance -dconvert=raid1 \
+ -mconvert=raid1 ${SCRATCH_MNT}
+}
+
+verify()
+{
+ echo >> $seqres.full
+ echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
+ echo
+ echo "mount reconstructed dev only and check md5sum"
+ echo "unmount" >> $seqres.full
+
+ _scratch_unmount
+ _reload_fs_module "btrfs"
+ _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
+ verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
+ verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
+
+ if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
+ echo "checkpoint1 on reappeared disk does not match after balance"
+ fi
+
+ if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
+ echo "checkpoint2 on reappeared disk does not match after balance"
+ fi
+}
+
+setup
+degrade_mount_write
+scan_missing_dev_and_write
+balance_convert
+verify
+
+$UMOUNT_PROG $DEV2
+_scratch_dev_pool_put
+_test_mount
+
+status=0
+exit
diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
new file mode 100644
index 000000000000..4a4939d448aa
--- /dev/null
+++ b/tests/btrfs/154.out
@@ -0,0 +1,10 @@
+QA output created by 154
+setup
+
+degraded mount
+
+scan missing dev and write
+
+run balance
+
+mount reconstructed dev only and check md5sum
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 15c2ecbf553c..c98cf823224e 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -156,3 +156,4 @@
151 auto quick
152 auto quick metadata qgroup send
153 auto quick qgroup
+154 auto quick
--
2.13.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2] btrfs/154: test for device dynamic rescan
2017-11-15 3:05 ` [PATCH v2] " Anand Jain
@ 2017-11-15 6:47 ` Eryu Guan
2017-11-15 10:13 ` Anand Jain
0 siblings, 1 reply; 10+ messages in thread
From: Eryu Guan @ 2017-11-15 6:47 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Wed, Nov 15, 2017 at 11:05:15AM +0800, Anand Jain wrote:
> Make sure missing device is included in the alloc list when it is
> scanned on a mounted FS.
>
> This test case needs btrfs kernel patch which is in the ML
> [PATCH] btrfs: handle dynamically reappearing missing device
> Without the kernel patch, the test will run, but reports as
> failed, as the device scanned won't appear in the alloc_list.
>
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
> v2: Fixed review comments.
> tests/btrfs/154 | 186 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/154.out | 10 +++
> tests/btrfs/group | 1 +
> 3 files changed, 197 insertions(+)
> create mode 100755 tests/btrfs/154
> create mode 100644 tests/btrfs/154.out
>
> diff --git a/tests/btrfs/154 b/tests/btrfs/154
> new file mode 100755
> index 000000000000..73a185157389
> --- /dev/null
> +++ b/tests/btrfs/154
> @@ -0,0 +1,186 @@
> +#! /bin/bash
> +# FS QA Test 154
> +#
> +# Test for reappearing missing device functionality.
> +# This test will fail without the btrfs kernel patch
> +# [PATCH] btrfs: handle dynamically reappearing missing device
> +#
> +#---------------------------------------------------------------------
> +# Copyright (c) 2017 Oracle. All Rights Reserved.
> +# Author: Anand Jain <anand.jain@oracle.com>
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/module
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_require_loadable_fs_module "btrfs"
> +
> +_scratch_dev_pool_get 2
> +
> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
> +
> +echo DEV1=$DEV1 >> $seqres.full
> +echo DEV2=$DEV2 >> $seqres.full
> +
> +# Balance won't be successful if filled too much
> +DEV1_SZ=`blockdev --getsize64 $DEV1`
> +DEV2_SZ=`blockdev --getsize64 $DEV2`
> +
> +# get min
> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
> +# Need disks with more than 2G
> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
> + _scratch_dev_pool_put
> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
> +fi
> +
> +MAX_FS_SZ=100000000
> +bs="1M"
> +COUNT=$(($MAX_FS_SZ / 1000000))
> +CHECKPOINT1=0
> +CHECKPOINT2=0
> +
> +setup()
> +{
> + echo >> $seqres.full
> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
> + echo "setup"
> + echo "-----setup-----" >> $seqres.full
> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
> + _scratch_mount >> $seqres.full 2>&1
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
> + >>$seqres.full 2>&1
> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> + COUNT=$(( $COUNT - 1 ))
> + echo "unmount" >> $seqres.full
> + _scratch_unmount
> +}
> +
> +degrade_mount_write()
> +{
> + echo >> $seqres.full
> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
> + echo
> + echo "degraded mount"
> +
> + echo "clean btrfs ko" >> $seqres.full
> + # un-scan the btrfs devices
> + _reload_fs_module "btrfs"
> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
> + cnt=$(( $COUNT/10 ))
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
> + >>$seqres.full 2>&1
> + COUNT=$(( $COUNT - $cnt ))
> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full
> +}
> +
> +scan_missing_dev_and_write()
> +{
> + echo >> $seqres.full
> + echo "--scan missing $DEV2--" >> $seqres.full
> + echo
> + echo "scan missing dev and write"
> +
> + _run_btrfs_util_prog device scan $DEV2
> +
> + echo >> $seqres.full
> +
> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
> +
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
> + >>$seqres.full 2>&1
> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full
> +}
> +
> +balance_convert()
> +{
> + echo >> $seqres.full
> + echo "----- run balance -----" >> $seqres.full
> + echo
> + echo "run balance"
> +
> + _run_btrfs_util_prog balance start --full-balance -dconvert=raid1 \
> + -mconvert=raid1 ${SCRATCH_MNT}
> +}
> +
> +verify()
> +{
> + echo >> $seqres.full
> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
> + echo
> + echo "mount reconstructed dev only and check md5sum"
> + echo "unmount" >> $seqres.full
> +
> + _scratch_unmount
> + _reload_fs_module "btrfs"
> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
> +
> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
> + echo "checkpoint1 on reappeared disk does not match after balance"
> + fi
> +
> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
> + echo "checkpoint2 on reappeared disk does not match after balance"
> + fi
> +}
> +
> +setup
> +degrade_mount_write
> +scan_missing_dev_and_write
> +balance_convert
> +verify
> +
> +$UMOUNT_PROG $DEV2
> +_scratch_dev_pool_put
> +_test_mount
This _test_mount is not needed either, and it breaks test now as
+mount: /dev/sdc1 is already mounted or /sdc1 busy
+ /dev/sdc1 is already mounted on /sdc1
I'll fix it on commit.
Thanks,
Eryu
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
> new file mode 100644
> index 000000000000..4a4939d448aa
> --- /dev/null
> +++ b/tests/btrfs/154.out
> @@ -0,0 +1,10 @@
> +QA output created by 154
> +setup
> +
> +degraded mount
> +
> +scan missing dev and write
> +
> +run balance
> +
> +mount reconstructed dev only and check md5sum
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 15c2ecbf553c..c98cf823224e 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -156,3 +156,4 @@
> 151 auto quick
> 152 auto quick metadata qgroup send
> 153 auto quick qgroup
> +154 auto quick
> --
> 2.13.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] btrfs/154: test for device dynamic rescan
2017-11-15 6:47 ` Eryu Guan
@ 2017-11-15 10:13 ` Anand Jain
0 siblings, 0 replies; 10+ messages in thread
From: Anand Jain @ 2017-11-15 10:13 UTC (permalink / raw)
To: Eryu Guan; +Cc: fstests, linux-btrfs
On 11/15/2017 02:47 PM, Eryu Guan wrote:
> On Wed, Nov 15, 2017 at 11:05:15AM +0800, Anand Jain wrote:
>> Make sure missing device is included in the alloc list when it is
>> scanned on a mounted FS.
>>
>> This test case needs btrfs kernel patch which is in the ML
>> [PATCH] btrfs: handle dynamically reappearing missing device
>> Without the kernel patch, the test will run, but reports as
>> failed, as the device scanned won't appear in the alloc_list.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>> ---
>> v2: Fixed review comments.
>> tests/btrfs/154 | 186 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/154.out | 10 +++
>> tests/btrfs/group | 1 +
>> 3 files changed, 197 insertions(+)
>> create mode 100755 tests/btrfs/154
>> create mode 100644 tests/btrfs/154.out
>>
>> diff --git a/tests/btrfs/154 b/tests/btrfs/154
>> new file mode 100755
>> index 000000000000..73a185157389
>> --- /dev/null
>> +++ b/tests/btrfs/154
>> @@ -0,0 +1,186 @@
>> +#! /bin/bash
>> +# FS QA Test 154
>> +#
>> +# Test for reappearing missing device functionality.
>> +# This test will fail without the btrfs kernel patch
>> +# [PATCH] btrfs: handle dynamically reappearing missing device
>> +#
>> +#---------------------------------------------------------------------
>> +# Copyright (c) 2017 Oracle. All Rights Reserved.
>> +# Author: Anand Jain <anand.jain@oracle.com>
>> +#
>> +# This program is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU General Public License as
>> +# published by the Free Software Foundation.
>> +#
>> +# This program is distributed in the hope that it would be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program; if not, write the Free Software Foundation,
>> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>> +#---------------------------------------------------------------------
>> +#
>> +
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1 # failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> + cd /
>> + rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/module
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs btrfs
>> +_supported_os Linux
>> +_require_scratch_dev_pool 2
>> +_require_loadable_fs_module "btrfs"
>> +
>> +_scratch_dev_pool_get 2
>> +
>> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>> +
>> +echo DEV1=$DEV1 >> $seqres.full
>> +echo DEV2=$DEV2 >> $seqres.full
>> +
>> +# Balance won't be successful if filled too much
>> +DEV1_SZ=`blockdev --getsize64 $DEV1`
>> +DEV2_SZ=`blockdev --getsize64 $DEV2`
>> +
>> +# get min
>> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
>> +# Need disks with more than 2G
>> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
>> + _scratch_dev_pool_put
>> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
>> +fi
>> +
>> +MAX_FS_SZ=100000000
>> +bs="1M"
>> +COUNT=$(($MAX_FS_SZ / 1000000))
>> +CHECKPOINT1=0
>> +CHECKPOINT2=0
>> +
>> +setup()
>> +{
>> + echo >> $seqres.full
>> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
>> + echo "setup"
>> + echo "-----setup-----" >> $seqres.full
>> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
>> + _scratch_mount >> $seqres.full 2>&1
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
>> + >>$seqres.full 2>&1
>> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
>> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
>> + COUNT=$(( $COUNT - 1 ))
>> + echo "unmount" >> $seqres.full
>> + _scratch_unmount
>> +}
>> +
>> +degrade_mount_write()
>> +{
>> + echo >> $seqres.full
>> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
>> + echo
>> + echo "degraded mount"
>> +
>> + echo "clean btrfs ko" >> $seqres.full
>> + # un-scan the btrfs devices
>> + _reload_fs_module "btrfs"
>> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
>> + cnt=$(( $COUNT/10 ))
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
>> + >>$seqres.full 2>&1
>> + COUNT=$(( $COUNT - $cnt ))
>> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
>> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
>> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
>> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full
>> +}
>> +
>> +scan_missing_dev_and_write()
>> +{
>> + echo >> $seqres.full
>> + echo "--scan missing $DEV2--" >> $seqres.full
>> + echo
>> + echo "scan missing dev and write"
>> +
>> + _run_btrfs_util_prog device scan $DEV2
>> +
>> + echo >> $seqres.full
>> +
>> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
>> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
>> +
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
>> + >>$seqres.full 2>&1
>> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
>> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full
>> +}
>> +
>> +balance_convert()
>> +{
>> + echo >> $seqres.full
>> + echo "----- run balance -----" >> $seqres.full
>> + echo
>> + echo "run balance"
>> +
>> + _run_btrfs_util_prog balance start --full-balance -dconvert=raid1 \
>> + -mconvert=raid1 ${SCRATCH_MNT}
>> +}
>> +
>> +verify()
>> +{
>> + echo >> $seqres.full
>> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
>> + echo
>> + echo "mount reconstructed dev only and check md5sum"
>> + echo "unmount" >> $seqres.full
>> +
>> + _scratch_unmount
>> + _reload_fs_module "btrfs"
>> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
>> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
>> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
>> +
>> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
>> + echo "checkpoint1 on reappeared disk does not match after balance"
>> + fi
>> +
>> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
>> + echo "checkpoint2 on reappeared disk does not match after balance"
>> + fi
>> +}
>> +
>> +setup
>> +degrade_mount_write
>> +scan_missing_dev_and_write
>> +balance_convert
>> +verify
>> +
>> +$UMOUNT_PROG $DEV2
>> +_scratch_dev_pool_put
>> +_test_mount
>
> This _test_mount is not needed either, and it breaks test now as
>
> +mount: /dev/sdc1 is already mounted or /sdc1 busy
> + /dev/sdc1 is already mounted on /sdc1
>
> I'll fix it on commit.
oops. I just verified _reload_fs_module() and missed to remove
_test_mount here. Thanks for fixing Eryu.
-Anand
> Thanks,
> Eryu
>
>> +
>> +status=0
>> +exit
>> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
>> new file mode 100644
>> index 000000000000..4a4939d448aa
>> --- /dev/null
>> +++ b/tests/btrfs/154.out
>> @@ -0,0 +1,10 @@
>> +QA output created by 154
>> +setup
>> +
>> +degraded mount
>> +
>> +scan missing dev and write
>> +
>> +run balance
>> +
>> +mount reconstructed dev only and check md5sum
>> diff --git a/tests/btrfs/group b/tests/btrfs/group
>> index 15c2ecbf553c..c98cf823224e 100644
>> --- a/tests/btrfs/group
>> +++ b/tests/btrfs/group
>> @@ -156,3 +156,4 @@
>> 151 auto quick
>> 152 auto quick metadata qgroup send
>> 153 auto quick qgroup
>> +154 auto quick
>> --
>> 2.13.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe fstests" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] btrfs/154: test for device dynamic rescan
2017-11-13 2:25 [PATCH] btrfs/154: test for device dynamic rescan Anand Jain
2017-11-14 12:12 ` Eryu Guan
2017-11-15 3:05 ` [PATCH v2] " Anand Jain
@ 2018-10-21 9:16 ` Filipe Manana
2018-10-21 9:20 ` Nikolay Borisov
2 siblings, 1 reply; 10+ messages in thread
From: Filipe Manana @ 2018-10-21 9:16 UTC (permalink / raw)
To: Anand Jain; +Cc: fstests, linux-btrfs
On Mon, Nov 13, 2017 at 2:26 AM Anand Jain <anand.jain@oracle.com> wrote:
>
> Make sure missing device is included in the alloc list when it is
> scanned on a mounted FS.
>
> This test case needs btrfs kernel patch which is in the ML
> [PATCH] btrfs: handle dynamically reappearing missing device
> Without the kernel patch, the test will run, but reports as
> failed, as the device scanned won't appear in the alloc_list.
So that patch was never merged, at least not with that subject.
What happened?
>
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
> tests/btrfs/154 | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/154.out | 10 +++
> tests/btrfs/group | 1 +
> 3 files changed, 199 insertions(+)
> create mode 100755 tests/btrfs/154
> create mode 100644 tests/btrfs/154.out
>
> diff --git a/tests/btrfs/154 b/tests/btrfs/154
> new file mode 100755
> index 000000000000..8b06fc4d9347
> --- /dev/null
> +++ b/tests/btrfs/154
> @@ -0,0 +1,188 @@
> +#! /bin/bash
> +# FS QA Test 154
> +#
> +# Test for reappearing missing device functionality.
> +# This test will fail without the btrfs kernel patch
> +# [PATCH] btrfs: handle dynamically reappearing missing device
> +#
> +#---------------------------------------------------------------------
> +# Copyright (c) 2017 Oracle. All Rights Reserved.
> +# Author: Anand Jain <anand.jain@oracle.com>
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#---------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/module
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch_dev_pool 2
> +_test_unmount
> +_require_loadable_fs_module "btrfs"
> +
> +_scratch_dev_pool_get 2
> +
> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
> +
> +echo DEV1=$DEV1 >> $seqres.full
> +echo DEV2=$DEV2 >> $seqres.full
> +
> +# Balance won't be successful if filled too much
> +DEV1_SZ=`blockdev --getsize64 $DEV1`
> +DEV2_SZ=`blockdev --getsize64 $DEV2`
> +
> +# get min
> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
> +# Need disks with more than 2G
> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
> + _scratch_dev_pool_put
> + _test_mount
> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
> +fi
> +
> +MAX_FS_SZ=100000000
> +bs="1M"
> +COUNT=$(($MAX_FS_SZ / 1000000))
> +CHECKPOINT1=0
> +CHECKPOINT2=0
> +
> +setup()
> +{
> + echo >> $seqres.full
> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
> + echo "setup"
> + echo "-----setup-----" >> $seqres.full
> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
> + _scratch_mount >> $seqres.full 2>&1
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
> + >>$seqres.full 2>&1
> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> + COUNT=$(( $COUNT - 1 ))
> + echo "unmount" >> $seqres.full
> + _scratch_unmount
> +}
> +
> +degrade_mount_write()
> +{
> + echo >> $seqres.full
> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
> + echo
> + echo "degraded mount"
> +
> + echo "clean btrfs ko" >> $seqres.full
> + # un-scan the btrfs devices
> + _reload_fs_module "btrfs"
> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
> + cnt=$(( $COUNT/10 ))
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
> + >>$seqres.full 2>&1
> + COUNT=$(( $COUNT - $cnt ))
> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full 2>&1
> +}
> +
> +scan_missing_dev_and_write()
> +{
> + echo >> $seqres.full
> + echo "--scan missing $DEV2--" >> $seqres.full
> + echo
> + echo "scan missing dev and write"
> +
> + _run_btrfs_util_prog device scan $DEV2
> +
> + echo >> $seqres.full
> +
> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
> +
> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
> + >>$seqres.full 2>&1
> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full 2>&1
> +}
> +
> +balance_convert()
> +{
> + echo >> $seqres.full
> + echo "----- run balance -----" >> $seqres.full
> + echo
> + echo "run balance"
> +
> + _run_btrfs_util_prog bal start --full-balance -dconvert=raid1 \
> + -mconvert=raid1 ${SCRATCH_MNT}
> +}
> +
> +verify()
> +{
> + echo >> $seqres.full
> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
> + echo
> + echo "mount reconstructed dev only and check md5sum"
> + echo "unmount" >> $seqres.full
> +
> + _scratch_unmount
> + _reload_fs_module "btrfs"
> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
> +
> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
> + echo "checkpoint1 does not match with after scrub"
> + fi
> +
> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
> + echo "checkpoint2 does not match with after scrub"
> + fi
> +}
> +
> +setup
> +degrade_mount_write
> +scan_missing_dev_and_write
> +balance_convert
> +verify
> +
> +$UMOUNT_PROG $DEV2
> +_scratch_dev_pool_put
> +_test_mount
> +
> +status=0
> +exit
> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
> new file mode 100644
> index 000000000000..4a4939d448aa
> --- /dev/null
> +++ b/tests/btrfs/154.out
> @@ -0,0 +1,10 @@
> +QA output created by 154
> +setup
> +
> +degraded mount
> +
> +scan missing dev and write
> +
> +run balance
> +
> +mount reconstructed dev only and check md5sum
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index 15c2ecbf553c..c98cf823224e 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -156,3 +156,4 @@
> 151 auto quick
> 152 auto quick metadata qgroup send
> 153 auto quick qgroup
> +154 auto quick
> --
> 2.13.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] btrfs/154: test for device dynamic rescan
2018-10-21 9:16 ` [PATCH] " Filipe Manana
@ 2018-10-21 9:20 ` Nikolay Borisov
2018-10-21 9:31 ` Filipe Manana
0 siblings, 1 reply; 10+ messages in thread
From: Nikolay Borisov @ 2018-10-21 9:20 UTC (permalink / raw)
To: fdmanana, Anand Jain; +Cc: fstests, linux-btrfs
On 21.10.2018 10:16, Filipe Manana wrote:
> On Mon, Nov 13, 2017 at 2:26 AM Anand Jain <anand.jain@oracle.com> wrote:
>>
>> Make sure missing device is included in the alloc list when it is
>> scanned on a mounted FS.
>>
>> This test case needs btrfs kernel patch which is in the ML
>> [PATCH] btrfs: handle dynamically reappearing missing device
>> Without the kernel patch, the test will run, but reports as
>> failed, as the device scanned won't appear in the alloc_list.
>
> So that patch was never merged, at least not with that subject.
> What happened?
https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/btrfs/154
In my testing of misc-next this test has been failing for me.
>
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>> ---
>> tests/btrfs/154 | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/154.out | 10 +++
>> tests/btrfs/group | 1 +
>> 3 files changed, 199 insertions(+)
>> create mode 100755 tests/btrfs/154
>> create mode 100644 tests/btrfs/154.out
>>
>> diff --git a/tests/btrfs/154 b/tests/btrfs/154
>> new file mode 100755
>> index 000000000000..8b06fc4d9347
>> --- /dev/null
>> +++ b/tests/btrfs/154
>> @@ -0,0 +1,188 @@
>> +#! /bin/bash
>> +# FS QA Test 154
>> +#
>> +# Test for reappearing missing device functionality.
>> +# This test will fail without the btrfs kernel patch
>> +# [PATCH] btrfs: handle dynamically reappearing missing device
>> +#
>> +#---------------------------------------------------------------------
>> +# Copyright (c) 2017 Oracle. All Rights Reserved.
>> +# Author: Anand Jain <anand.jain@oracle.com>
>> +#
>> +# This program is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU General Public License as
>> +# published by the Free Software Foundation.
>> +#
>> +# This program is distributed in the hope that it would be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program; if not, write the Free Software Foundation,
>> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>> +#---------------------------------------------------------------------
>> +#
>> +
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1 # failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> + cd /
>> + rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +. ./common/module
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +_supported_fs btrfs
>> +_supported_os Linux
>> +_require_scratch_dev_pool 2
>> +_test_unmount
>> +_require_loadable_fs_module "btrfs"
>> +
>> +_scratch_dev_pool_get 2
>> +
>> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
>> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
>> +
>> +echo DEV1=$DEV1 >> $seqres.full
>> +echo DEV2=$DEV2 >> $seqres.full
>> +
>> +# Balance won't be successful if filled too much
>> +DEV1_SZ=`blockdev --getsize64 $DEV1`
>> +DEV2_SZ=`blockdev --getsize64 $DEV2`
>> +
>> +# get min
>> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
>> +# Need disks with more than 2G
>> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
>> + _scratch_dev_pool_put
>> + _test_mount
>> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
>> +fi
>> +
>> +MAX_FS_SZ=100000000
>> +bs="1M"
>> +COUNT=$(($MAX_FS_SZ / 1000000))
>> +CHECKPOINT1=0
>> +CHECKPOINT2=0
>> +
>> +setup()
>> +{
>> + echo >> $seqres.full
>> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
>> + echo "setup"
>> + echo "-----setup-----" >> $seqres.full
>> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
>> + _scratch_mount >> $seqres.full 2>&1
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
>> + >>$seqres.full 2>&1
>> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
>> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
>> + COUNT=$(( $COUNT - 1 ))
>> + echo "unmount" >> $seqres.full
>> + _scratch_unmount
>> +}
>> +
>> +degrade_mount_write()
>> +{
>> + echo >> $seqres.full
>> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
>> + echo
>> + echo "degraded mount"
>> +
>> + echo "clean btrfs ko" >> $seqres.full
>> + # un-scan the btrfs devices
>> + _reload_fs_module "btrfs"
>> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
>> + cnt=$(( $COUNT/10 ))
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
>> + >>$seqres.full 2>&1
>> + COUNT=$(( $COUNT - $cnt ))
>> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
>> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
>> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
>> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full 2>&1
>> +}
>> +
>> +scan_missing_dev_and_write()
>> +{
>> + echo >> $seqres.full
>> + echo "--scan missing $DEV2--" >> $seqres.full
>> + echo
>> + echo "scan missing dev and write"
>> +
>> + _run_btrfs_util_prog device scan $DEV2
>> +
>> + echo >> $seqres.full
>> +
>> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
>> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
>> +
>> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
>> + >>$seqres.full 2>&1
>> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
>> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full 2>&1
>> +}
>> +
>> +balance_convert()
>> +{
>> + echo >> $seqres.full
>> + echo "----- run balance -----" >> $seqres.full
>> + echo
>> + echo "run balance"
>> +
>> + _run_btrfs_util_prog bal start --full-balance -dconvert=raid1 \
>> + -mconvert=raid1 ${SCRATCH_MNT}
>> +}
>> +
>> +verify()
>> +{
>> + echo >> $seqres.full
>> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
>> + echo
>> + echo "mount reconstructed dev only and check md5sum"
>> + echo "unmount" >> $seqres.full
>> +
>> + _scratch_unmount
>> + _reload_fs_module "btrfs"
>> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
>> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
>> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
>> +
>> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
>> + echo "checkpoint1 does not match with after scrub"
>> + fi
>> +
>> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
>> + echo "checkpoint2 does not match with after scrub"
>> + fi
>> +}
>> +
>> +setup
>> +degrade_mount_write
>> +scan_missing_dev_and_write
>> +balance_convert
>> +verify
>> +
>> +$UMOUNT_PROG $DEV2
>> +_scratch_dev_pool_put
>> +_test_mount
>> +
>> +status=0
>> +exit
>> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
>> new file mode 100644
>> index 000000000000..4a4939d448aa
>> --- /dev/null
>> +++ b/tests/btrfs/154.out
>> @@ -0,0 +1,10 @@
>> +QA output created by 154
>> +setup
>> +
>> +degraded mount
>> +
>> +scan missing dev and write
>> +
>> +run balance
>> +
>> +mount reconstructed dev only and check md5sum
>> diff --git a/tests/btrfs/group b/tests/btrfs/group
>> index 15c2ecbf553c..c98cf823224e 100644
>> --- a/tests/btrfs/group
>> +++ b/tests/btrfs/group
>> @@ -156,3 +156,4 @@
>> 151 auto quick
>> 152 auto quick metadata qgroup send
>> 153 auto quick qgroup
>> +154 auto quick
>> --
>> 2.13.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] btrfs/154: test for device dynamic rescan
2018-10-21 9:20 ` Nikolay Borisov
@ 2018-10-21 9:31 ` Filipe Manana
2018-10-21 14:13 ` Anand Jain
0 siblings, 1 reply; 10+ messages in thread
From: Filipe Manana @ 2018-10-21 9:31 UTC (permalink / raw)
To: Nikolay Borisov; +Cc: Anand Jain, fstests, linux-btrfs
On Sun, Oct 21, 2018 at 10:20 AM Nikolay Borisov <nborisov@suse.com> wrote:
>
>
>
> On 21.10.2018 10:16, Filipe Manana wrote:
> > On Mon, Nov 13, 2017 at 2:26 AM Anand Jain <anand.jain@oracle.com> wrote:
> >>
> >> Make sure missing device is included in the alloc list when it is
> >> scanned on a mounted FS.
> >>
> >> This test case needs btrfs kernel patch which is in the ML
> >> [PATCH] btrfs: handle dynamically reappearing missing device
> >> Without the kernel patch, the test will run, but reports as
> >> failed, as the device scanned won't appear in the alloc_list.
> >
> > So that patch was never merged, at least not with that subject.
> > What happened?
>
> https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/btrfs/154
>
> In my testing of misc-next this test has been failing for me.
Yes, that's why I'm asking.
It's not uncommon for Anand to submit tests that never get the
corresponding kernel patch merged or, like in his recent hole punch
test, that doesn't have a corresponding kernel fix.
Anand, what's the deal here? Are you planning on pushing the fixes to
the kernel? And for the hole punch test, are you working on a fix or
just hoping someone else will fix it?
> >
> >>
> >> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> >> ---
> >> tests/btrfs/154 | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> tests/btrfs/154.out | 10 +++
> >> tests/btrfs/group | 1 +
> >> 3 files changed, 199 insertions(+)
> >> create mode 100755 tests/btrfs/154
> >> create mode 100644 tests/btrfs/154.out
> >>
> >> diff --git a/tests/btrfs/154 b/tests/btrfs/154
> >> new file mode 100755
> >> index 000000000000..8b06fc4d9347
> >> --- /dev/null
> >> +++ b/tests/btrfs/154
> >> @@ -0,0 +1,188 @@
> >> +#! /bin/bash
> >> +# FS QA Test 154
> >> +#
> >> +# Test for reappearing missing device functionality.
> >> +# This test will fail without the btrfs kernel patch
> >> +# [PATCH] btrfs: handle dynamically reappearing missing device
> >> +#
> >> +#---------------------------------------------------------------------
> >> +# Copyright (c) 2017 Oracle. All Rights Reserved.
> >> +# Author: Anand Jain <anand.jain@oracle.com>
> >> +#
> >> +# This program is free software; you can redistribute it and/or
> >> +# modify it under the terms of the GNU General Public License as
> >> +# published by the Free Software Foundation.
> >> +#
> >> +# This program is distributed in the hope that it would be useful,
> >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> +# GNU General Public License for more details.
> >> +#
> >> +# You should have received a copy of the GNU General Public License
> >> +# along with this program; if not, write the Free Software Foundation,
> >> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> >> +#---------------------------------------------------------------------
> >> +#
> >> +
> >> +seq=`basename $0`
> >> +seqres=$RESULT_DIR/$seq
> >> +echo "QA output created by $seq"
> >> +
> >> +here=`pwd`
> >> +tmp=/tmp/$$
> >> +status=1 # failure is the default!
> >> +trap "_cleanup; exit \$status" 0 1 2 3 15
> >> +
> >> +_cleanup()
> >> +{
> >> + cd /
> >> + rm -f $tmp.*
> >> +}
> >> +
> >> +# get standard environment, filters and checks
> >> +. ./common/rc
> >> +. ./common/filter
> >> +. ./common/module
> >> +
> >> +# remove previous $seqres.full before test
> >> +rm -f $seqres.full
> >> +
> >> +# real QA test starts here
> >> +
> >> +_supported_fs btrfs
> >> +_supported_os Linux
> >> +_require_scratch_dev_pool 2
> >> +_test_unmount
> >> +_require_loadable_fs_module "btrfs"
> >> +
> >> +_scratch_dev_pool_get 2
> >> +
> >> +DEV1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
> >> +DEV2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
> >> +
> >> +echo DEV1=$DEV1 >> $seqres.full
> >> +echo DEV2=$DEV2 >> $seqres.full
> >> +
> >> +# Balance won't be successful if filled too much
> >> +DEV1_SZ=`blockdev --getsize64 $DEV1`
> >> +DEV2_SZ=`blockdev --getsize64 $DEV2`
> >> +
> >> +# get min
> >> +MAX_FS_SZ=`echo -e "$DEV1_SZ\n$DEV2_SZ" | sort | head -1`
> >> +# Need disks with more than 2G
> >> +if [ $MAX_FS_SZ -lt 2000000000 ]; then
> >> + _scratch_dev_pool_put
> >> + _test_mount
> >> + _notrun "Smallest dev size $MAX_FS_SZ, Need at least 2G"
> >> +fi
> >> +
> >> +MAX_FS_SZ=100000000
> >> +bs="1M"
> >> +COUNT=$(($MAX_FS_SZ / 1000000))
> >> +CHECKPOINT1=0
> >> +CHECKPOINT2=0
> >> +
> >> +setup()
> >> +{
> >> + echo >> $seqres.full
> >> + echo "MAX_FS_SZ=$MAX_FS_SZ COUNT=$COUNT" >> $seqres.full
> >> + echo "setup"
> >> + echo "-----setup-----" >> $seqres.full
> >> + _scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
> >> + _scratch_mount >> $seqres.full 2>&1
> >> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf bs=$bs count=1 \
> >> + >>$seqres.full 2>&1
> >> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> >> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> >> + COUNT=$(( $COUNT - 1 ))
> >> + echo "unmount" >> $seqres.full
> >> + _scratch_unmount
> >> +}
> >> +
> >> +degrade_mount_write()
> >> +{
> >> + echo >> $seqres.full
> >> + echo "--degraded mount: max_fs_sz $max_fs_sz bytes--" >> $seqres.full
> >> + echo
> >> + echo "degraded mount"
> >> +
> >> + echo "clean btrfs ko" >> $seqres.full
> >> + # un-scan the btrfs devices
> >> + _reload_fs_module "btrfs"
> >> + _mount -o degraded $DEV1 $SCRATCH_MNT >>$seqres.full 2>&1
> >> + cnt=$(( $COUNT/10 ))
> >> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf1 bs=$bs count=$cnt \
> >> + >>$seqres.full 2>&1
> >> + COUNT=$(( $COUNT - $cnt ))
> >> + _run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> >> + _run_btrfs_util_prog filesystem df $SCRATCH_MNT
> >> + CHECKPOINT1=`md5sum $SCRATCH_MNT/tf1`
> >> + echo $SCRATCH_MNT/tf1:$CHECKPOINT1 >> $seqres.full 2>&1
> >> +}
> >> +
> >> +scan_missing_dev_and_write()
> >> +{
> >> + echo >> $seqres.full
> >> + echo "--scan missing $DEV2--" >> $seqres.full
> >> + echo
> >> + echo "scan missing dev and write"
> >> +
> >> + _run_btrfs_util_prog device scan $DEV2
> >> +
> >> + echo >> $seqres.full
> >> +
> >> + _run_btrfs_util_prog filesystem show -m ${SCRATCH_MNT}
> >> + _run_btrfs_util_prog filesystem df ${SCRATCH_MNT}
> >> +
> >> + dd if=/dev/urandom of="$SCRATCH_MNT"/tf2 bs=$bs count=$COUNT \
> >> + >>$seqres.full 2>&1
> >> + CHECKPOINT2=`md5sum $SCRATCH_MNT/tf2`
> >> + echo $SCRATCH_MNT/tf2:$CHECKPOINT2 >> $seqres.full 2>&1
> >> +}
> >> +
> >> +balance_convert()
> >> +{
> >> + echo >> $seqres.full
> >> + echo "----- run balance -----" >> $seqres.full
> >> + echo
> >> + echo "run balance"
> >> +
> >> + _run_btrfs_util_prog bal start --full-balance -dconvert=raid1 \
> >> + -mconvert=raid1 ${SCRATCH_MNT}
> >> +}
> >> +
> >> +verify()
> >> +{
> >> + echo >> $seqres.full
> >> + echo "--mount reconstructed dev only and check md5sum--" >> $seqres.full
> >> + echo
> >> + echo "mount reconstructed dev only and check md5sum"
> >> + echo "unmount" >> $seqres.full
> >> +
> >> + _scratch_unmount
> >> + _reload_fs_module "btrfs"
> >> + _mount -o degraded $DEV2 $SCRATCH_MNT >>$seqres.full 2>&1
> >> + verify_checkpoint1=`md5sum $SCRATCH_MNT/tf1`
> >> + verify_checkpoint2=`md5sum $SCRATCH_MNT/tf2`
> >> +
> >> + if [ "$verify_checkpoint1" != "$CHECKPOINT1" ]; then
> >> + echo "checkpoint1 does not match with after scrub"
> >> + fi
> >> +
> >> + if [ "$verify_checkpoint2" != "$CHECKPOINT2" ]; then
> >> + echo "checkpoint2 does not match with after scrub"
> >> + fi
> >> +}
> >> +
> >> +setup
> >> +degrade_mount_write
> >> +scan_missing_dev_and_write
> >> +balance_convert
> >> +verify
> >> +
> >> +$UMOUNT_PROG $DEV2
> >> +_scratch_dev_pool_put
> >> +_test_mount
> >> +
> >> +status=0
> >> +exit
> >> diff --git a/tests/btrfs/154.out b/tests/btrfs/154.out
> >> new file mode 100644
> >> index 000000000000..4a4939d448aa
> >> --- /dev/null
> >> +++ b/tests/btrfs/154.out
> >> @@ -0,0 +1,10 @@
> >> +QA output created by 154
> >> +setup
> >> +
> >> +degraded mount
> >> +
> >> +scan missing dev and write
> >> +
> >> +run balance
> >> +
> >> +mount reconstructed dev only and check md5sum
> >> diff --git a/tests/btrfs/group b/tests/btrfs/group
> >> index 15c2ecbf553c..c98cf823224e 100644
> >> --- a/tests/btrfs/group
> >> +++ b/tests/btrfs/group
> >> @@ -156,3 +156,4 @@
> >> 151 auto quick
> >> 152 auto quick metadata qgroup send
> >> 153 auto quick qgroup
> >> +154 auto quick
> >> --
> >> 2.13.1
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> >
> >
--
Filipe David Manana,
“Whether you think you can, or you think you can't — you're right.”
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] btrfs/154: test for device dynamic rescan
2018-10-21 9:31 ` Filipe Manana
@ 2018-10-21 14:13 ` Anand Jain
0 siblings, 0 replies; 10+ messages in thread
From: Anand Jain @ 2018-10-21 14:13 UTC (permalink / raw)
To: fdmanana, Nikolay Borisov; +Cc: fstests, linux-btrfs
On 10/21/2018 05:31 PM, Filipe Manana wrote:
> On Sun, Oct 21, 2018 at 10:20 AM Nikolay Borisov <nborisov@suse.com> wrote:
>>
>>
>>
>> On 21.10.2018 10:16, Filipe Manana wrote:
>>> On Mon, Nov 13, 2017 at 2:26 AM Anand Jain <anand.jain@oracle.com> wrote:
>>>>
>>>> Make sure missing device is included in the alloc list when it is
>>>> scanned on a mounted FS.
>>>>
>>>> This test case needs btrfs kernel patch which is in the ML
>>>> [PATCH] btrfs: handle dynamically reappearing missing device
>>>> Without the kernel patch, the test will run, but reports as
>>>> failed, as the device scanned won't appear in the alloc_list.
>>>
>>> So that patch was never merged, at least not with that subject.
>>> What happened?
The contending patch at V5 (with fixed comments as received) was
working fine at the then latest kernel version, now its out dated,
and I am not sure where to invest my time, it will be better to
have David's comment, if he has any concern on the idea/approach
in general. I am ok to revive the patch.
>> https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/btrfs/154
>>
>> In my testing of misc-next this test has been failing for me.
Don't bother about that patch I am sure its not good on the current
kernel.
> recent hole punch
> test, that doesn't have a corresponding kernel fix.
Hole punching kernel patch, fixes a corner case, I tried but failed.
Until my current priority list recedes I won't be able to look into
it again. If you would like to pick this up pls go ahead. I think it
will easy for you.
Lastly, I think its a good idea to record the current known limitation
as a test cases, even if it fails, in that way it gets documented.
Thanks, Anand
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2018-10-21 14:13 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-13 2:25 [PATCH] btrfs/154: test for device dynamic rescan Anand Jain
2017-11-14 12:12 ` Eryu Guan
2017-11-15 3:04 ` Anand Jain
2017-11-15 3:05 ` [PATCH v2] " Anand Jain
2017-11-15 6:47 ` Eryu Guan
2017-11-15 10:13 ` Anand Jain
2018-10-21 9:16 ` [PATCH] " Filipe Manana
2018-10-21 9:20 ` Nikolay Borisov
2018-10-21 9:31 ` Filipe Manana
2018-10-21 14:13 ` Anand Jain
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.