[PATCH v5 0/3] dm error based test cases

[PATCH v5 0/3] dm error based test cases

[Anand Jain]

This is v5 of this patch set. Mainly accepts Filipe latest review comments.

Anand Jain (3):
  xfstests: btrfs: add functions to create dm-error device
  xfstests: btrfs: test device replace, with EIO on the src dev
  xfstests: btrfs: test device delete with EIO on src dev

 common/dmerror      | 69 ++++++++++++++++++++++++++++++++++++++++++++
 common/rc           | 16 +++++++++++
 tests/btrfs/098     | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/098.out | 11 +++++++
 tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/099.out | 11 +++++++
 tests/btrfs/group   |  2 ++
 7 files changed, 272 insertions(+)
 create mode 100644 common/dmerror
 create mode 100755 tests/btrfs/098
 create mode 100644 tests/btrfs/098.out
 create mode 100755 tests/btrfs/099
 create mode 100644 tests/btrfs/099.out

-- 
2.4.1

[PATCH v5 1/3] xfstests: btrfs: add functions to create dm-error device

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

Controlled EIO from the device is achieved using the dm device.
Helper functions are at common/dmerror.

Broadly steps will include calling _init_dmerror().
_init_dmerror() will use SCRATCH_DEV to create dm linear device and assign
DMERROR_DEV to /dev/mapper/error-test.

When test script is ready to get EIO, the test cases can call
_load_dmerror_table() which then it will load the dm error.
so that reading DMERROR_DEV will cause EIO. After the test case is
complete, cleanup must be done by calling _cleanup_dmerror().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
v4->v5: No Change. keep up with the patch set
v3->v4: rebase on latest xfstests code
v2.1->v3: accepts Filipe Manana's review comments, thanks
v2->v2.1: fixed missed typo error fixup in the commit.
v1->v2: accepts Dave Chinner's review comments, thanks
 common/dmerror | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 common/rc      |  9 ++++++++
 2 files changed, 78 insertions(+)
 create mode 100644 common/dmerror

diff --git a/common/dmerror b/common/dmerror
new file mode 100644
index 0000000..f895d90
--- /dev/null
+++ b/common/dmerror
@@ -0,0 +1,69 @@
+##/bin/bash
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#
+# common functions for setting up and tearing down a dmerror device
+
+_init_dmerror()
+{
+	$DMSETUP_PROG remove error-test > /dev/null 2>&1
+
+	local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
+
+	DMERROR_DEV='/dev/mapper/error-test'
+
+	DMLINEAR_TABLE="0 $BLK_DEV_SIZE linear $SCRATCH_DEV 0"
+
+	$DMSETUP_PROG create error-test --table "$DMLINEAR_TABLE" || \
+		_fatal "failed to create dm linear device"
+
+	DMERROR_TABLE="0 $BLK_DEV_SIZE error $SCRATCH_DEV 0"
+}
+
+_scratch_mkfs_dmerror()
+{
+	$MKFS_BTRFS_PROG $* $DMERROR_DEV >> $seqres.full 2>&1 || \
+			_fatal "failed to create mkfs.btrfs $* $DMERROR_DEV"
+}
+
+_mount_dmerror()
+{
+	mount -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT
+}
+
+_unmount_dmerror()
+{
+	$UMOUNT_PROGS $SCRATCH_MNT
+}
+
+_cleanup_dmerror()
+{
+	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
+	$DMSETUP_PROG remove error-test > /dev/null 2>&1
+}
+
+_load_dmerror_table()
+{
+	$DMSETUP_PROG suspend error-test
+	[ $? -ne 0 ] && _fatal  "failed to suspend error-test"
+
+	$DMSETUP_PROG load error-test --table "$DMERROR_TABLE"
+	[ $? -ne 0 ] && _fatal "failed to load error table error-test"
+
+	$DMSETUP_PROG resume error-test
+	[ $? -ne 0 ] && _fatal  "failed to resume error-test"
+}
diff --git a/common/rc b/common/rc
index 70d2fa8..8d4da0e 100644
--- a/common/rc
+++ b/common/rc
@@ -1337,6 +1337,15 @@ _require_sane_bdev_flush()
 	fi
 }
 
+# this test requires the device mapper error target
+#
+_require_dmerror()
+{
+	_require_command "$DMSETUP_PROG" dmsetup
+	$DMSETUP_PROG targets | grep error >/dev/null 2>&1
+	[ $? -ne 0 ] && _notrun "This test requires dm error support"
+}
+
 # this test requires the device mapper flakey target
 #
 _require_dm_flakey()
-- 
2.4.1

[PATCH v5 2/3] xfstests: btrfs: test device replace, with EIO on the src dev

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

This test case will test to confirm the replace works with
the failed (EIO) replacing source device. EIO condition is
achieved using the DM device.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
v4->v5: rebase on latest xfstests code and accepts Filipe comment
v3->v4: rebase on latest xfstests code
v2->v3: accepts Filipe Manana's review comments, thanks
v1->v2: accepts Dave Chinner's review comments, thanks
 tests/btrfs/098     | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/098.out | 11 ++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 93 insertions(+)
 create mode 100755 tests/btrfs/098
 create mode 100644 tests/btrfs/098.out

diff --git a/tests/btrfs/098 b/tests/btrfs/098
new file mode 100755
index 0000000..afb41d1
--- /dev/null
+++ b/tests/btrfs/098
@@ -0,0 +1,81 @@
+#! /bin/bash
+# FS QA Test No. btrfs/098
+#
+#test device replace works when the source device has EIO
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+
+_cleanup()
+{
+	_cleanup_dmerror
+	rm -f $tmp
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/filter.btrfs
+. ./common/dmerror
+
+_supported_fs btrfs
+_supported_os Linux
+_need_to_be_root
+_require_scratch_dev_pool 3
+_require_dmerror
+
+rm -f $seqres.full
+
+dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
+dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
+
+_init_dmerror
+_scratch_mkfs_dmerror "-f -d raid1 -m raid1 $dev1"
+_mount_dmerror
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
+			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
+
+snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
+snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
+						"$snapshot_cmd" -X 50 >&/dev/null
+
+# now load the error into the DMERROR_DEV
+_load_dmerror_table
+
+_run_btrfs_util_prog replace start -B $error_devid $dev2 $SCRATCH_MNT
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+echo "=== device replace completed"
+
+status=0; exit
diff --git a/tests/btrfs/098.out b/tests/btrfs/098.out
new file mode 100644
index 0000000..eb2f87f
--- /dev/null
+++ b/tests/btrfs/098.out
@@ -0,0 +1,11 @@
+QA output created by 098
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
+
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+
+=== device replace completed
diff --git a/tests/btrfs/group b/tests/btrfs/group
index e13865a..c8a53b5 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -100,3 +100,4 @@
 095 auto quick metadata
 096 auto quick clone
 097 auto quick send clone
+098 auto quick replace
-- 
2.4.1

[PATCH v5 3/3] xfstests: btrfs: test device delete with EIO on src dev

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

This test case tests if the device delete works with
the failed (EIO) source device. EIO errors are achieved
usign the DM device.

This test would need following btrfs-progs and btrfs
kernel patch
   btrfs-progs: device delete to accept devid
   Btrfs: device delete by devid

However when btrfs-progs patch is not found this test will
not run, and when kernel patch is not found btrfs-progs
will fail gracefully and thus the test script.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
v4->v5: rebase on latest xfstests code, and accepts Filipe comment
v3->v4: rebase on latest xfstests code
v2->v3: accepts Filipe Manana's review comments, thanks
v1->v2: accepts Dave Chinner's review comments, thanks
 common/rc           |  7 +++++
 tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/099.out | 11 +++++++
 tests/btrfs/group   |  1 +
 4 files changed, 101 insertions(+)
 create mode 100755 tests/btrfs/099
 create mode 100644 tests/btrfs/099.out

diff --git a/common/rc b/common/rc
index 8d4da0e..31a0328 100644
--- a/common/rc
+++ b/common/rc
@@ -2737,6 +2737,13 @@ _require_meta_uuid()
 	umount $SCRATCH_MNT
 }
 
+_require_btrfs_dev_del_by_devid()
+{
+	$BTRFS_UTIL_PROG device delete --help | egrep devid > /dev/null 2>&1
+	[ $? -eq 0 ] || _notrun "$BTRFS_UTIL_PROG too old "\
+			"(must support 'btrfs device delete <devid> /<mnt>')"
+}
+
 _get_total_inode()
 {
 	if [ -z "$1" ]; then
diff --git a/tests/btrfs/099 b/tests/btrfs/099
new file mode 100755
index 0000000..4464e24
--- /dev/null
+++ b/tests/btrfs/099
@@ -0,0 +1,82 @@
+#! /bin/bash
+# FS QA Test No. btrfs/099
+#
+# test device delete when the source device has EIO
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+
+_cleanup()
+{
+	_cleanup_dmerror
+	rm -f $tmp
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/filter.btrfs
+. ./common/dmerror
+
+_supported_fs btrfs
+_supported_os Linux
+_need_to_be_root
+_require_scratch_dev_pool 3
+_require_btrfs_dev_del_by_devid
+_require_dmerror
+
+rm -f $seqres.full
+
+dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
+dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
+
+_init_dmerror
+_scratch_mkfs_dmerror "-f -d raid1 -m raid1 $dev1 $dev2"
+_mount_dmerror
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
+			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
+
+snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
+snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
+						"$snapshot_cmd" -X 50 >&/dev/null
+
+# now load the error into the DMERROR_DEV
+_load_dmerror_table
+
+_run_btrfs_util_prog device delete $error_devid $SCRATCH_MNT
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+echo "=== device delete completed"
+
+status=0; exit
diff --git a/tests/btrfs/099.out b/tests/btrfs/099.out
new file mode 100644
index 0000000..ec74e45
--- /dev/null
+++ b/tests/btrfs/099.out
@@ -0,0 +1,11 @@
+QA output created by 099
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
+
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+
+=== device delete completed
diff --git a/tests/btrfs/group b/tests/btrfs/group
index c8a53b5..968ee63 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -101,3 +101,4 @@
 096 auto quick clone
 097 auto quick send clone
 098 auto quick replace
+099 auto quick replace
-- 
2.4.1

Re: [PATCH v5 1/3] xfstests: btrfs: add functions to create dm-error device

[Eryu Guan]

On Fri, Aug 14, 2015 at 06:47:02PM +0800, Anand Jain wrote:
> From: Anand Jain <Anand.Jain@oracle.com>
> 
> Controlled EIO from the device is achieved using the dm device.
> Helper functions are at common/dmerror.
> 
> Broadly steps will include calling _init_dmerror().
> _init_dmerror() will use SCRATCH_DEV to create dm linear device and assign
> DMERROR_DEV to /dev/mapper/error-test.
> 
> When test script is ready to get EIO, the test cases can call
> _load_dmerror_table() which then it will load the dm error.
> so that reading DMERROR_DEV will cause EIO. After the test case is
> complete, cleanup must be done by calling _cleanup_dmerror().
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> ---
> v4->v5: No Change. keep up with the patch set
> v3->v4: rebase on latest xfstests code
> v2.1->v3: accepts Filipe Manana's review comments, thanks
> v2->v2.1: fixed missed typo error fixup in the commit.
> v1->v2: accepts Dave Chinner's review comments, thanks
>  common/dmerror | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  common/rc      |  9 ++++++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 common/dmerror
> 
> diff --git a/common/dmerror b/common/dmerror
> new file mode 100644
> index 0000000..f895d90
> --- /dev/null
> +++ b/common/dmerror
> @@ -0,0 +1,69 @@
> +##/bin/bash
> +#
> +# Copyright (c) 2015 Oracle.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +#
> +# common functions for setting up and tearing down a dmerror device
> +
> +_init_dmerror()
> +{
> +	$DMSETUP_PROG remove error-test > /dev/null 2>&1
> +
> +	local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
> +
> +	DMERROR_DEV='/dev/mapper/error-test'
> +
> +	DMLINEAR_TABLE="0 $BLK_DEV_SIZE linear $SCRATCH_DEV 0"
> +
> +	$DMSETUP_PROG create error-test --table "$DMLINEAR_TABLE" || \
> +		_fatal "failed to create dm linear device"
> +
> +	DMERROR_TABLE="0 $BLK_DEV_SIZE error $SCRATCH_DEV 0"
> +}
> +
> +_scratch_mkfs_dmerror()
> +{
> +	$MKFS_BTRFS_PROG $* $DMERROR_DEV >> $seqres.full 2>&1 || \
> +			_fatal "failed to create mkfs.btrfs $* $DMERROR_DEV"

I didn't follow previous reviews, please correct me if I miss anything.

Is dmerror only for btrfs testing? I saw $MKFS_BTRFS_PROG here. And do
we need $MKFS_OPTIONS too?

> +}
> +
> +_mount_dmerror()
> +{
> +	mount -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT

$MOUNT_PROG ?

> +}
> +
> +_unmount_dmerror()
> +{
> +	$UMOUNT_PROGS $SCRATCH_MNT

$UMOUNT_PROG, no S at the end.

Thanks,
Eryu

> +}
> +
> +_cleanup_dmerror()
> +{
> +	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
> +	$DMSETUP_PROG remove error-test > /dev/null 2>&1
> +}
> +
> +_load_dmerror_table()
> +{
> +	$DMSETUP_PROG suspend error-test
> +	[ $? -ne 0 ] && _fatal  "failed to suspend error-test"
> +
> +	$DMSETUP_PROG load error-test --table "$DMERROR_TABLE"
> +	[ $? -ne 0 ] && _fatal "failed to load error table error-test"
> +
> +	$DMSETUP_PROG resume error-test
> +	[ $? -ne 0 ] && _fatal  "failed to resume error-test"
> +}
> diff --git a/common/rc b/common/rc
> index 70d2fa8..8d4da0e 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1337,6 +1337,15 @@ _require_sane_bdev_flush()
>  	fi
>  }
>  
> +# this test requires the device mapper error target
> +#
> +_require_dmerror()
> +{
> +	_require_command "$DMSETUP_PROG" dmsetup
> +	$DMSETUP_PROG targets | grep error >/dev/null 2>&1
> +	[ $? -ne 0 ] && _notrun "This test requires dm error support"
> +}
> +
>  # this test requires the device mapper flakey target
>  #
>  _require_dm_flakey()
> -- 
> 2.4.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 2/3] xfstests: btrfs: test device replace, with EIO on the src dev

[Eryu Guan]

On Fri, Aug 14, 2015 at 06:47:03PM +0800, Anand Jain wrote:
> From: Anand Jain <Anand.Jain@oracle.com>
> 
> This test case will test to confirm the replace works with
> the failed (EIO) replacing source device. EIO condition is
> achieved using the DM device.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> ---
> v4->v5: rebase on latest xfstests code and accepts Filipe comment
> v3->v4: rebase on latest xfstests code
> v2->v3: accepts Filipe Manana's review comments, thanks
> v1->v2: accepts Dave Chinner's review comments, thanks
>  tests/btrfs/098     | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/098.out | 11 ++++++++
>  tests/btrfs/group   |  1 +
>  3 files changed, 93 insertions(+)
>  create mode 100755 tests/btrfs/098
>  create mode 100644 tests/btrfs/098.out
> 
> diff --git a/tests/btrfs/098 b/tests/btrfs/098
> new file mode 100755
> index 0000000..afb41d1
> --- /dev/null
> +++ b/tests/btrfs/098
> @@ -0,0 +1,81 @@
> +#! /bin/bash
> +# FS QA Test No. btrfs/098
> +#
> +#test device replace works when the source device has EIO

Nitpick here, need a space after # :)

> +#
> +# Copyright (c) 2015 Oracle.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +
> +_cleanup()
> +{
> +	_cleanup_dmerror
> +	rm -f $tmp

should be rm -f $tmp.* as many functions in common/rc and check create
tmp files like $tmp.xxx

Thanks,
Eryu

> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/filter.btrfs
> +. ./common/dmerror
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_need_to_be_root
> +_require_scratch_dev_pool 3
> +_require_dmerror
> +
> +rm -f $seqres.full
> +
> +dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
> +dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
> +
> +_init_dmerror
> +_scratch_mkfs_dmerror "-f -d raid1 -m raid1 $dev1"
> +_mount_dmerror
> +
> +_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> +$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
> +
> +error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
> +			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
> +
> +snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
> +snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
> +run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
> +						"$snapshot_cmd" -X 50 >&/dev/null
> +
> +# now load the error into the DMERROR_DEV
> +_load_dmerror_table
> +
> +_run_btrfs_util_prog replace start -B $error_devid $dev2 $SCRATCH_MNT
> +
> +_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> +$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
> +
> +echo "=== device replace completed"
> +
> +status=0; exit
> diff --git a/tests/btrfs/098.out b/tests/btrfs/098.out
> new file mode 100644
> index 0000000..eb2f87f
> --- /dev/null
> +++ b/tests/btrfs/098.out
> @@ -0,0 +1,11 @@
> +QA output created by 098
> +Label: none  uuid:  <UUID>
> +	Total devices <NUM> FS bytes used <SIZE>
> +	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
> +	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
> +
> +Label: none  uuid:  <UUID>
> +	Total devices <NUM> FS bytes used <SIZE>
> +	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
> +
> +=== device replace completed
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index e13865a..c8a53b5 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -100,3 +100,4 @@
>  095 auto quick metadata
>  096 auto quick clone
>  097 auto quick send clone
> +098 auto quick replace
> -- 
> 2.4.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 3/3] xfstests: btrfs: test device delete with EIO on src dev

[Eryu Guan]

On Fri, Aug 14, 2015 at 06:47:04PM +0800, Anand Jain wrote:
> From: Anand Jain <Anand.Jain@oracle.com>
> 
> This test case tests if the device delete works with
> the failed (EIO) source device. EIO errors are achieved
> usign the DM device.
> 
> This test would need following btrfs-progs and btrfs
> kernel patch
>    btrfs-progs: device delete to accept devid
>    Btrfs: device delete by devid
> 
> However when btrfs-progs patch is not found this test will
> not run, and when kernel patch is not found btrfs-progs
> will fail gracefully and thus the test script.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
> v4->v5: rebase on latest xfstests code, and accepts Filipe comment
> v3->v4: rebase on latest xfstests code
> v2->v3: accepts Filipe Manana's review comments, thanks
> v1->v2: accepts Dave Chinner's review comments, thanks
>  common/rc           |  7 +++++
>  tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/099.out | 11 +++++++
>  tests/btrfs/group   |  1 +
>  4 files changed, 101 insertions(+)
>  create mode 100755 tests/btrfs/099
>  create mode 100644 tests/btrfs/099.out
> 
> diff --git a/common/rc b/common/rc
> index 8d4da0e..31a0328 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -2737,6 +2737,13 @@ _require_meta_uuid()
>  	umount $SCRATCH_MNT
>  }
>  
> +_require_btrfs_dev_del_by_devid()
> +{
> +	$BTRFS_UTIL_PROG device delete --help | egrep devid > /dev/null 2>&1
> +	[ $? -eq 0 ] || _notrun "$BTRFS_UTIL_PROG too old "\
> +			"(must support 'btrfs device delete <devid> /<mnt>')"
> +}
> +
>  _get_total_inode()
>  {
>  	if [ -z "$1" ]; then
> diff --git a/tests/btrfs/099 b/tests/btrfs/099
> new file mode 100755
> index 0000000..4464e24
> --- /dev/null
> +++ b/tests/btrfs/099
> @@ -0,0 +1,82 @@
> +#! /bin/bash
> +# FS QA Test No. btrfs/099
> +#
> +# test device delete when the source device has EIO
> +#
> +# Copyright (c) 2015 Oracle.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +
> +_cleanup()
> +{
> +	_cleanup_dmerror
> +	rm -f $tmp

And here too, rm -f $tmp.*

Thanks,
Eryu

Re: [PATCH v5 2/3] xfstests: btrfs: test device replace, with EIO on the src dev

[Filipe David Manana]

On Fri, Aug 14, 2015 at 11:47 AM, Anand Jain <anand.jain@oracle.com> wrote:
> From: Anand Jain <Anand.Jain@oracle.com>
>
> This test case will test to confirm the replace works with
> the failed (EIO) replacing source device. EIO condition is
> achieved using the DM device.
>
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> ---
> v4->v5: rebase on latest xfstests code and accepts Filipe comment
> v3->v4: rebase on latest xfstests code
> v2->v3: accepts Filipe Manana's review comments, thanks
> v1->v2: accepts Dave Chinner's review comments, thanks
>  tests/btrfs/098     | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/098.out | 11 ++++++++
>  tests/btrfs/group   |  1 +
>  3 files changed, 93 insertions(+)
>  create mode 100755 tests/btrfs/098
>  create mode 100644 tests/btrfs/098.out
>
> diff --git a/tests/btrfs/098 b/tests/btrfs/098
> new file mode 100755
> index 0000000..afb41d1
> --- /dev/null
> +++ b/tests/btrfs/098
> @@ -0,0 +1,81 @@
> +#! /bin/bash
> +# FS QA Test No. btrfs/098
> +#
> +#test device replace works when the source device has EIO
> +#
> +# Copyright (c) 2015 Oracle.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +
> +status=1       # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +
> +_cleanup()
> +{
> +       _cleanup_dmerror
> +       rm -f $tmp
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/filter.btrfs
> +. ./common/dmerror
> +
> +_supported_fs btrfs
> +_supported_os Linux
> +_need_to_be_root
> +_require_scratch_dev_pool 3
> +_require_dmerror
> +
> +rm -f $seqres.full
> +
> +dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
> +dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
> +
> +_init_dmerror
> +_scratch_mkfs_dmerror "-f -d raid1 -m raid1 $dev1"
> +_mount_dmerror
> +
> +_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> +$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
> +
> +error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
> +                       egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
> +
> +snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
> +snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
> +run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
> +                                               "$snapshot_cmd" -X 50 >&/dev/null

Sorry missed this before, but you don't need to redirect stdout/stderr
to /dev/null.
run_check redirects them to $seqres.full where it's actually useful -
when we have the test failing, we can check $seqres.full to see what
seed fsstress used (fsstress prints it to stdout/stderr). That's for
the case where it's failing only for some seeds of course.

Same observation applies to the other test/patch.

Thanks.

> +
> +# now load the error into the DMERROR_DEV
> +_load_dmerror_table
> +
> +_run_btrfs_util_prog replace start -B $error_devid $dev2 $SCRATCH_MNT
> +
> +_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
> +$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
> +
> +echo "=== device replace completed"
> +
> +status=0; exit
> diff --git a/tests/btrfs/098.out b/tests/btrfs/098.out
> new file mode 100644
> index 0000000..eb2f87f
> --- /dev/null
> +++ b/tests/btrfs/098.out
> @@ -0,0 +1,11 @@
> +QA output created by 098
> +Label: none  uuid:  <UUID>
> +       Total devices <NUM> FS bytes used <SIZE>
> +       devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
> +       devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
> +
> +Label: none  uuid:  <UUID>
> +       Total devices <NUM> FS bytes used <SIZE>
> +       devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
> +
> +=== device replace completed
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index e13865a..c8a53b5 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -100,3 +100,4 @@
>  095 auto quick metadata
>  096 auto quick clone
>  097 auto quick send clone
> +098 auto quick replace
> --
> 2.4.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

[PATCH v6 0/3] dm error based test cases

[Anand Jain]

This is v6 of this patch set. Mainly accepts Filipe latest review comments and Eryu's review comments with thanks.


Anand Jain (3):
  xfstests: btrfs: add functions to create dm-error device
  xfstests: btrfs: test device replace, with EIO on the src dev
  xfstests: btrfs: test device delete with EIO on src dev

 common/dmerror      | 69 ++++++++++++++++++++++++++++++++++++++++++++
 common/rc           | 16 +++++++++++
 tests/btrfs/098     | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/098.out | 11 +++++++
 tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/099.out | 11 +++++++
 tests/btrfs/group   |  2 ++
 7 files changed, 272 insertions(+)
 create mode 100644 common/dmerror
 create mode 100755 tests/btrfs/098
 create mode 100644 tests/btrfs/098.out
 create mode 100755 tests/btrfs/099
 create mode 100644 tests/btrfs/099.out

-- 
2.4.1

[PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

Controlled EIO from the device is achieved using the dm device.
Helper functions are at common/dmerror.

Broadly steps will include calling _init_dmerror().
_init_dmerror() will use SCRATCH_DEV to create dm linear device and assign
DMERROR_DEV to /dev/mapper/error-test.

When test script is ready to get EIO, the test cases can call
_load_dmerror_table() which then it will load the dm error.
so that reading DMERROR_DEV will cause EIO. After the test case is
complete, cleanup must be done by calling _cleanup_dmerror().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
v5->v6: accepts Eryu's comments
v4->v5: No Change. keep up with the patch set
v3->v4: rebase on latest xfstests code
v2.1->v3: accepts Filipe Manana's review comments, thanks
v2->v2.1: fixed missed typo error fixup in the commit.
v1->v2: accepts Dave Chinner's review comments, thanks
 common/dmerror | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 common/rc      |  9 ++++++++
 2 files changed, 78 insertions(+)
 create mode 100644 common/dmerror

diff --git a/common/dmerror b/common/dmerror
new file mode 100644
index 0000000..928e998
--- /dev/null
+++ b/common/dmerror
@@ -0,0 +1,69 @@
+##/bin/bash
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#
+# common functions for setting up and tearing down a dmerror device
+
+_init_dmerror()
+{
+	$DMSETUP_PROG remove error-test > /dev/null 2>&1
+
+	local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
+
+	DMERROR_DEV='/dev/mapper/error-test'
+
+	DMLINEAR_TABLE="0 $BLK_DEV_SIZE linear $SCRATCH_DEV 0"
+
+	$DMSETUP_PROG create error-test --table "$DMLINEAR_TABLE" || \
+		_fatal "failed to create dm linear device"
+
+	DMERROR_TABLE="0 $BLK_DEV_SIZE error $SCRATCH_DEV 0"
+}
+
+_scratch_mkfs_dmerror()
+{
+	$MKFS_BTRFS_PROG $MKFS_OPTIONS $* $DMERROR_DEV >> $seqres.full 2>&1 || \
+			_fatal "failed to create mkfs.btrfs $* $DMERROR_DEV"
+}
+
+_mount_dmerror()
+{
+	$MOUNT_PROG -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT
+}
+
+_unmount_dmerror()
+{
+	$UMOUNT_PROG $SCRATCH_MNT
+}
+
+_cleanup_dmerror()
+{
+	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
+	$DMSETUP_PROG remove error-test > /dev/null 2>&1
+}
+
+_load_dmerror_table()
+{
+	$DMSETUP_PROG suspend error-test
+	[ $? -ne 0 ] && _fatal  "failed to suspend error-test"
+
+	$DMSETUP_PROG load error-test --table "$DMERROR_TABLE"
+	[ $? -ne 0 ] && _fatal "failed to load error table error-test"
+
+	$DMSETUP_PROG resume error-test
+	[ $? -ne 0 ] && _fatal  "failed to resume error-test"
+}
diff --git a/common/rc b/common/rc
index 70d2fa8..8d4da0e 100644
--- a/common/rc
+++ b/common/rc
@@ -1337,6 +1337,15 @@ _require_sane_bdev_flush()
 	fi
 }
 
+# this test requires the device mapper error target
+#
+_require_dmerror()
+{
+	_require_command "$DMSETUP_PROG" dmsetup
+	$DMSETUP_PROG targets | grep error >/dev/null 2>&1
+	[ $? -ne 0 ] && _notrun "This test requires dm error support"
+}
+
 # this test requires the device mapper flakey target
 #
 _require_dm_flakey()
-- 
2.4.1

[PATCH v6 2/3] xfstests: btrfs: test device replace, with EIO on the src dev

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

This test case will test to confirm the replace works with
the failed (EIO) replacing source device. EIO condition is
achieved using the DM device.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
v5->v6: accepts Eryu and Filipe's comments
v4->v5: rebase on latest xfstests code and accepts Filipe comment
v3->v4: rebase on latest xfstests code
v2->v3: accepts Filipe Manana's review comments, thanks
v1->v2: accepts Dave Chinner's review comments, thanks
 tests/btrfs/098     | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/098.out | 11 ++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 93 insertions(+)
 create mode 100755 tests/btrfs/098
 create mode 100644 tests/btrfs/098.out

diff --git a/tests/btrfs/098 b/tests/btrfs/098
new file mode 100755
index 0000000..a41ea86
--- /dev/null
+++ b/tests/btrfs/098
@@ -0,0 +1,81 @@
+#! /bin/bash
+# FS QA Test No. btrfs/098
+#
+# Test device replace works when the source device has EIO
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+
+_cleanup()
+{
+	_cleanup_dmerror
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/filter.btrfs
+. ./common/dmerror
+
+_supported_fs btrfs
+_supported_os Linux
+_need_to_be_root
+_require_scratch_dev_pool 3
+_require_dmerror
+
+rm -f $seqres.full
+
+dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
+dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
+
+_init_dmerror
+_scratch_mkfs_dmerror "-f -d raid1 -m raid1 $dev1"
+_mount_dmerror
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
+			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
+
+snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
+snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
+						"$snapshot_cmd" -X 50
+
+# now load the error into the DMERROR_DEV
+_load_dmerror_table
+
+_run_btrfs_util_prog replace start -B $error_devid $dev2 $SCRATCH_MNT
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+echo "=== device replace completed"
+
+status=0; exit
diff --git a/tests/btrfs/098.out b/tests/btrfs/098.out
new file mode 100644
index 0000000..eb2f87f
--- /dev/null
+++ b/tests/btrfs/098.out
@@ -0,0 +1,11 @@
+QA output created by 098
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
+
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+
+=== device replace completed
diff --git a/tests/btrfs/group b/tests/btrfs/group
index e13865a..c8a53b5 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -100,3 +100,4 @@
 095 auto quick metadata
 096 auto quick clone
 097 auto quick send clone
+098 auto quick replace
-- 
2.4.1

[PATCH v6 3/3] xfstests: btrfs: test device delete with EIO on src dev

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

This test case tests if the device delete works with
the failed (EIO) source device. EIO errors are achieved
usign the DM device.

This test would need following btrfs-progs and btrfs
kernel patch
   btrfs-progs: device delete to accept devid
   Btrfs: device delete by devid

However when btrfs-progs patch is not found this test will
not run, and when kernel patch is not found btrfs-progs
will fail gracefully and thus the test script.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
v5->v6: accepts Eryu and Filipe's comments, thanks
v4->v5: rebase on latest xfstests code, and accepts Filipe comment
v3->v4: rebase on latest xfstests code
v2->v3: accepts Filipe Manana's review comments, thanks
v1->v2: accepts Dave Chinner's review comments, thanks
 common/rc           |  7 +++++
 tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/099.out | 11 +++++++
 tests/btrfs/group   |  1 +
 4 files changed, 101 insertions(+)
 create mode 100755 tests/btrfs/099
 create mode 100644 tests/btrfs/099.out

diff --git a/common/rc b/common/rc
index 8d4da0e..31a0328 100644
--- a/common/rc
+++ b/common/rc
@@ -2737,6 +2737,13 @@ _require_meta_uuid()
 	umount $SCRATCH_MNT
 }
 
+_require_btrfs_dev_del_by_devid()
+{
+	$BTRFS_UTIL_PROG device delete --help | egrep devid > /dev/null 2>&1
+	[ $? -eq 0 ] || _notrun "$BTRFS_UTIL_PROG too old "\
+			"(must support 'btrfs device delete <devid> /<mnt>')"
+}
+
 _get_total_inode()
 {
 	if [ -z "$1" ]; then
diff --git a/tests/btrfs/099 b/tests/btrfs/099
new file mode 100755
index 0000000..a0761c7
--- /dev/null
+++ b/tests/btrfs/099
@@ -0,0 +1,82 @@
+#! /bin/bash
+# FS QA Test No. btrfs/099
+#
+# test device delete when the source device has EIO
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+
+_cleanup()
+{
+	_cleanup_dmerror
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/filter.btrfs
+. ./common/dmerror
+
+_supported_fs btrfs
+_supported_os Linux
+_need_to_be_root
+_require_scratch_dev_pool 3
+_require_btrfs_dev_del_by_devid
+_require_dmerror
+
+rm -f $seqres.full
+
+dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
+dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
+
+_init_dmerror
+_scratch_mkfs_dmerror "-f -d raid1 -m raid1 $dev1 $dev2"
+_mount_dmerror
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
+			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
+
+snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
+snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
+						"$snapshot_cmd" -X 50
+
+# now load the error into the DMERROR_DEV
+_load_dmerror_table
+
+_run_btrfs_util_prog device delete $error_devid $SCRATCH_MNT
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+echo "=== device delete completed"
+
+status=0; exit
diff --git a/tests/btrfs/099.out b/tests/btrfs/099.out
new file mode 100644
index 0000000..ec74e45
--- /dev/null
+++ b/tests/btrfs/099.out
@@ -0,0 +1,11 @@
+QA output created by 099
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
+
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+
+=== device delete completed
diff --git a/tests/btrfs/group b/tests/btrfs/group
index c8a53b5..968ee63 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -101,3 +101,4 @@
 096 auto quick clone
 097 auto quick send clone
 098 auto quick replace
+099 auto quick replace
-- 
2.4.1

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Dave Chinner]

On Fri, Aug 14, 2015 at 11:31:43PM +0800, Anand Jain wrote:
> From: Anand Jain <Anand.Jain@oracle.com>
> 
> Controlled EIO from the device is achieved using the dm device.
> Helper functions are at common/dmerror.
> 
> Broadly steps will include calling _init_dmerror().
> _init_dmerror() will use SCRATCH_DEV to create dm linear device and assign
> DMERROR_DEV to /dev/mapper/error-test.
> 
> When test script is ready to get EIO, the test cases can call
> _load_dmerror_table() which then it will load the dm error.
> so that reading DMERROR_DEV will cause EIO. After the test case is
> complete, cleanup must be done by calling _cleanup_dmerror().
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> Reviewed-by: Filipe Manana <fdmanana@suse.com>
> ---
> v5->v6: accepts Eryu's comments
> v4->v5: No Change. keep up with the patch set
> v3->v4: rebase on latest xfstests code
> v2.1->v3: accepts Filipe Manana's review comments, thanks
> v2->v2.1: fixed missed typo error fixup in the commit.
> v1->v2: accepts Dave Chinner's review comments, thanks

This is not a change log. A change log describes the changes that
were made, not who asked for changes to be made. i.e. I have no idea
what changes you actually made from the previous version in this
patch set....

....
> +# common functions for setting up and tearing down a dmerror device
> +
> +_init_dmerror()

All these function names shoul dbe prefixed _dmerror_<blah> so that
it is clear the namespace we are working on. (similar to
_scratch-<blah>, _test_<blah>, etc).

> +{
> +	$DMSETUP_PROG remove error-test > /dev/null 2>&1
> +
> +	local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`

I think I've said this before - local variables are lower case,
global/exported variables are upper case.

> +
> +	DMERROR_DEV='/dev/mapper/error-test'
> +
> +	DMLINEAR_TABLE="0 $BLK_DEV_SIZE linear $SCRATCH_DEV 0"
> +
> +	$DMSETUP_PROG create error-test --table "$DMLINEAR_TABLE" || \
> +		_fatal "failed to create dm linear device"

That should be _fail...

_fatal() is local to common/config (as it can be included in scripts
without including common/rc). Tests include common/rc (and not
common/config directly), so they should all use _fail...


> +	DMERROR_TABLE="0 $BLK_DEV_SIZE error $SCRATCH_DEV 0"
> +}
> +
> +_scratch_mkfs_dmerror()
> +{
> +	$MKFS_BTRFS_PROG $MKFS_OPTIONS $* $DMERROR_DEV >> $seqres.full 2>&1 || \
> +			_fatal "failed to create mkfs.btrfs $* $DMERROR_DEV"
> +}

This has nothing to do with the scratch device. The test should
simply call '_mkfs_dev $DMERROR_DEV' as there's no need for a
wrapper here.



> +
> +_mount_dmerror()
> +{
> +	$MOUNT_PROG -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT
> +}

Should mirror _scratch_mount.

_mount -t $FSTYP `_scratch_mount_options` $DMERROR_DEV $SCRATCH_MNT

> +
> +_unmount_dmerror()
> +{
> +	$UMOUNT_PROG $SCRATCH_MNT
> +}

Doesn't need a wrapper.

> +
> +_cleanup_dmerror()
> +{
> +	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
> +	$DMSETUP_PROG remove error-test > /dev/null 2>&1
> +}
> +
> +_load_dmerror_table()
> +{
> +	$DMSETUP_PROG suspend error-test
> +	[ $? -ne 0 ] && _fatal  "failed to suspend error-test"

Error message makes no sense when read as the reason for a test
failing. '_fail "dmsetup suspend failed"' makes more sense, as that is the
command that failed.

> +
> +	$DMSETUP_PROG load error-test --table "$DMERROR_TABLE"
> +	[ $? -ne 0 ] && _fatal "failed to load error table error-test"

_fail "dmsetup failed to load error table"

> +
> +	$DMSETUP_PROG resume error-test
> +	[ $? -ne 0 ] && _fatal  "failed to resume error-test"

_fail "dmsetup resume failed"

> +}
> diff --git a/common/rc b/common/rc
> index 70d2fa8..8d4da0e 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1337,6 +1337,15 @@ _require_sane_bdev_flush()
>  	fi
>  }
>  
> +# this test requires the device mapper error target
> +#
> +_require_dmerror()
> +{
> +	_require_command "$DMSETUP_PROG" dmsetup
> +	$DMSETUP_PROG targets | grep error >/dev/null 2>&1
> +	[ $? -ne 0 ] && _notrun "This test requires dm error support"
> +}

Why isn't this in common/dmerror?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[anand jain]

Hi Dave,

  All comments accepted thanks. except for this.


>> +_mount_dmerror()
>> +{
>> +	$MOUNT_PROG -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT
>> +}
>
> Should mirror _scratch_mount.
>
> _mount -t $FSTYP `_scratch_mount_options` $DMERROR_DEV $SCRATCH_MNT


`_scratch_mount_options` also returns $SCRATCH_DEV.
in case of tests involving dmerror module, dmerror_init would use 
$SCRATCH_DEV as backing device and provide $DMERROR_DEV to be used 
instead of $SCRATCH_DEV. So I am proposing..

+	_mount -t $FSTYP $SCRATCH_OPTIONS $MOUNT_OPTIONS 
$SELINUX_MOUNT_OPTIONS $* $DMERROR_DEV $SCRATCH_MNT



Thanks, Anand

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Dave Chinner]

On Tue, Aug 18, 2015 at 12:26:54PM +0800, anand jain wrote:
> 
> Hi Dave,
> 
>  All comments accepted thanks. except for this.
> 
> 
> >>+_mount_dmerror()
> >>+{
> >>+	$MOUNT_PROG -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT
> >>+}
> >
> >Should mirror _scratch_mount.
> >
> >_mount -t $FSTYP `_scratch_mount_options` $DMERROR_DEV $SCRATCH_MNT
> 
> 
> `_scratch_mount_options` also returns $SCRATCH_DEV.
> in case of tests involving dmerror module, dmerror_init would use
> $SCRATCH_DEV as backing device and provide $DMERROR_DEV to be used
> instead of $SCRATCH_DEV. So I am proposing..
> 
> +	_mount -t $FSTYP $SCRATCH_OPTIONS $MOUNT_OPTIONS
> $SELINUX_MOUNT_OPTIONS $* $DMERROR_DEV $SCRATCH_MNT

Ok, SCRATCH_OPTIONS might not be the best idea here, so feel free to
drop it.

However, you've still missed the primary reason I suggested
_scratch_mount_options in the first place: what do we do instead of
copy'n'paste of random code fragments?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[anand jain]

> Ok, SCRATCH_OPTIONS might not be the best idea here, so feel free to
> drop it.

  I have dropped $SCRATCH_OPTIONS. (waiting to submit v8). Thanks.

> However, you've still missed the primary reason I suggested
> _scratch_mount_options in the first place:

  I am shocked. What was missed ?

Cheers, Anand

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Dave Chinner]

On Wed, Aug 19, 2015 at 11:16:10AM +0800, anand jain wrote:
> 
> 
> >Ok, SCRATCH_OPTIONS might not be the best idea here, so feel free to
> >drop it.
> 
>  I have dropped $SCRATCH_OPTIONS. (waiting to submit v8). Thanks.
> 
> >However, you've still missed the primary reason I suggested
> >_scratch_mount_options in the first place:
> 
>  I am shocked. What was missed ?

Answering the question I asked:

Q: "what do we do instead of copy'n'paste of random code fragments?"

Think on it, because the answer to that question (and it's not a
hard one) should tell you exactly what you need to do.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Anand Jain]

  (thanks for the off-ML emails from the people who helped me
  to understand).

Dave,

  looks like you are suggesting something like..

-------
+_dmerror_mount_options()
+{
+       _scratch_options mount
+       echo $SCRATCH_OPTIONS $MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS \
+                                       $* $DMERROR_DEV $SCRATCH_MNT
+}
+
+_dmerror_mount()
+{
+       _mount -t $FSTYP `_dmerror_mount_options $*`
+}
--------

  Sorry that the word 'should mirror _scratch_mount()' confused me.

  Pls clarify.

Thanks, Anand


 >>> +_mount_dmerror()
 >>> +{
 >>> +	$MOUNT_PROG -t $FSTYP $MOUNT_OPTIONS $DMERROR_DEV $SCRATCH_MNT
 >>> +}


 >> Should mirror _scratch_mount.
 >>
 >> _mount -t $FSTYP `_scratch_mount_options` $DMERROR_DEV $SCRATCH_MNT


 > So I am proposing..

 > +	_mount -t $FSTYP $SCRATCH_OPTIONS $MOUNT_OPTIONS
 > $SELINUX_MOUNT_OPTIONS $* $DMERROR_DEV $SCRATCH_MNT


However, you've still missed the primary reason I suggested
_scratch_mount_options in the first place: what do we do instead of
copy'n'paste of random code fragments?

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Dave Chinner]

On Thu, Aug 20, 2015 at 03:09:05PM +0800, Anand Jain wrote:
> 
>  (thanks for the off-ML emails from the people who helped me
>  to understand).
> 
> Dave,
> 
>  looks like you are suggesting something like..
> 
> -------
> +_dmerror_mount_options()
> +{
> +       _scratch_options mount
> +       echo $SCRATCH_OPTIONS $MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS \
> +                                       $* $DMERROR_DEV $SCRATCH_MNT
> +}
> +
> +_dmerror_mount()
> +{
> +       _mount -t $FSTYP `_dmerror_mount_options $*`
> +}
> --------

Very close. :)

You've got the structure right, but missed one more thing: the
options are still cut'n'paste from _scratch_mount_options, and so
should be /factored/ into a common function...

_scratch_options is only useful to XFS to set external log/realtime
devices, and we don't want to do that for the dmerror device.
Everthing else is almost the same as _scratch_mount_options(),
however.  So:

# Used for mounting non-scratch devices (e.g. loop, dm constructs)
# with the safe set of scratch mount options (e.g. loop image may be
# hosted on $SCRATCH_DEV, so can't use external scratch devices).
_common_dev_mount_options()
{
	echo $MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS $*
}

_scratch_mount_options()
{
	_scratch_options mount

	echo `_common_dev_mount_options $*` $SCRATCH_OPTIONS \
					    $SCRATCH_DEV $SCRATCH_MNT

}

_dmerror_mount_options()
{
       echo `_common_dev_mount_options $*` $DMERROR_DEV $SCRATCH_MNT
}

And to demonstrate why factoring like this is useful, this can also
be applied to _mount_flakey(), and we could also replace most of the
open-coded loop device mounts with a generic "mount device" function
like:

_mount_dev()
{
	_mount `_common_dev_mount_options $*`
}

and call it like:

_mount_dev $LOOP_DEV $MNT_POINT

or even

_mount_dev -o loop $MNT_POINT

We can then take it further: if we have a _mount_dev() wrapper we
can factor _common_dev_mount_options() from all the
_*_mount_options() functions. i.e:

_scratch_mount_options()
{
	_scratch_options mount

	echo $SCRATCH_OPTIONS $SCRATCH_DEV $SCRATCH_MNT
}

_scratch_mount()
{
	_mount_dev $* `_scratch_mount_options`
}

The result is we end up with a much cleaner set of mount functions
that are more generic, more extensible and easier to use, as well as
being more maintainable....

>  Sorry that the word 'should mirror _scratch_mount()' confused me.

Ok, I'll try to be more clear in future.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v6 1/3] xfstests: btrfs: add functions to create dm-error device

[Anand Jain]

Dave,

On 08/21/2015 05:45 AM, Dave Chinner wrote:
> On Thu, Aug 20, 2015 at 03:09:05PM +0800, Anand Jain wrote:
>>
>>   (thanks for the off-ML emails from the people who helped me
>>   to understand).
>>
>> Dave,
>>
>>   looks like you are suggesting something like..
>>
>> -------
>> +_dmerror_mount_options()
>> +{
>> +       _scratch_options mount
>> +       echo $SCRATCH_OPTIONS $MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS \
>> +                                       $* $DMERROR_DEV $SCRATCH_MNT
>> +}
>> +
>> +_dmerror_mount()
>> +{
>> +       _mount -t $FSTYP `_dmerror_mount_options $*`
>> +}
>> --------
>
> Very close. :)
>
> You've got the structure right, but missed one more thing: the
> options are still cut'n'paste from _scratch_mount_options, and so
> should be /factored/ into a common function...
>
> _scratch_options is only useful to XFS to set external log/realtime
> devices, and we don't want to do that for the dmerror device.
> Everthing else is almost the same as _scratch_mount_options(),
> however.  So:
>
> # Used for mounting non-scratch devices (e.g. loop, dm constructs)
> # with the safe set of scratch mount options (e.g. loop image may be
> # hosted on $SCRATCH_DEV, so can't use external scratch devices).
> _common_dev_mount_options()
> {
> 	echo $MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS $*
> }
>
> _scratch_mount_options()
> {
> 	_scratch_options mount
>
> 	echo `_common_dev_mount_options $*` $SCRATCH_OPTIONS \
> 					    $SCRATCH_DEV $SCRATCH_MNT
>
> }


> _dmerror_mount_options()
> {
>         echo `_common_dev_mount_options $*` $DMERROR_DEV $SCRATCH_MNT
> }


   this would make test cases nice and sleek. Thanks. I have this in v8.


> And to demonstrate why factoring like this is useful, this can also
> be applied to _mount_flakey(), and we could also replace most of the
> open-coded loop device mounts with a generic "mount device" function
> like:

   yes indeed. I suggest if these could be handled in a separate patch
   as this would deviate from the intentions of this patch set ?

Thanks, Anand


> _mount_dev()
> {
> 	_mount `_common_dev_mount_options $*`
> }
>
> and call it like:
>
> _mount_dev $LOOP_DEV $MNT_POINT
>
> or even
>
> _mount_dev -o loop $MNT_POINT
>
> We can then take it further: if we have a _mount_dev() wrapper we
> can factor _common_dev_mount_options() from all the
> _*_mount_options() functions. i.e:
>
> _scratch_mount_options()
> {
> 	_scratch_options mount
>
> 	echo $SCRATCH_OPTIONS $SCRATCH_DEV $SCRATCH_MNT
> }
>
> _scratch_mount()
> {
> 	_mount_dev $* `_scratch_mount_options`
> }
>
> The result is we end up with a much cleaner set of mount functions
> that are more generic, more extensible and easier to use, as well as
> being more maintainable....
>
>>   Sorry that the word 'should mirror _scratch_mount()' confused me.
>
> Ok, I'll try to be more clear in future.
>
> Cheers,
>
> Dave.
>