[PATCH v2 0/3] Refactor and revamp the 'test' script

[PATCH v2 0/3] Refactor and revamp the 'test' script

[Zhilong Liu]

Hi, Jes;
  Mainly added the suggestions from v1. I'm awared that these patches
maybe still to improve, draft them just from my understanding. I would
improve for any defect, and thanks for your patience to point out.

Still need ideas about testdev():
- Now the 'test' has supported new arguments like --disks= to test physical
  devices, but testdev() in all cases was designed by loop mode, we should
  decide whether or not the 'disk' mode also supports the testdev() testing.
  Currently, I have ignored the testdev() testing and make it 'return 0' if
  has set --dev=disk mode in this patch version.

For --raidtype:
- I have devided the raid levels to
  raid0|linear|raid1|raid456|raid10|ddf|imsm
For --disks:
- It supports user to specify physical devices as follow.
  Example: ./test --dev=disk --disks=/dev/sda{5..16} or
           ./test --dev=disk --disks=/dev/sd{a..m}

Changes from v1:
- Refactor and improve all functions of test 'script'.
- Add --raidtype argument to support testing different raid
  level cases.
- Add --disks argument to support testing physical devices

v1 mail-tree:
[PATCH 0/3] mdadm/test: Refactor the codes of 'test' script
RFC patch mail-tree:
[PATCH RFC] test: revise 'test' and make it easier to understand


Thanks very much,
-Zhilong

Zhilong Liu (3):
  mdadm/test: Refactor and revamp 'test' script
  mdadm/test: Add '--raidtype=' to run different raidlevel cases
  mdadm/test: Add '--disks=' to support testing phsical devices

 test | 464 +++++++++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 285 insertions(+), 179 deletions(-)

-- 
2.6.6

[PATCH v2 1/3] mdadm/test: Refactor and revamp 'test' script

[Zhilong Liu]

Adding functions:
die()
  uniform the abnormal situations that have to abort.
check_env()
  do various basic checking before running test suite.
save_log()
  collect array infos, include of dmesg, superblock,
  bitmap and /proc/mdstat.
main()
  the core function of this script.

Improve functions:
cleanup()
  clear dmesg and remove the /vat/tmp/mdtest* files.
mdadm()
  clear superblock once creating or building arrays
  every time, because it's always creating arrays
  many times in a test case.
check()
  just tidy up with die(), didn't change code meanings.
testdev()
  add checking $1 must be a block device, add 'return 0'
  in final because this function exists in last line of
  test case, such as tests/05r6tor0.
do_test()
  add checking abnormal dmesg and changing log management.
do_help()
  just recommend a better way to print Usage.
parse_args()
  revamp and improve.

Delete function:
fast_sync()
  It's no longer used, so get rid of it.

Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
 test | 408 +++++++++++++++++++++++++++++++++++++------------------------------
 1 file changed, 228 insertions(+), 180 deletions(-)

diff --git a/test b/test
index 7942d6e..7ee523b 100755
--- a/test
+++ b/test
@@ -1,37 +1,20 @@
 #!/bin/bash
 #
 # run test suite for mdadm
-user=`id -un`
-if [ " $user" != " root" ]
-then
-	echo >&2 "test: testing can only be done as 'root'."
-	exit 1
-fi
-
-prefix='[0-9][0-9]'
-
-dir=`pwd`
+dir=$(pwd)
 mdadm=$dir/mdadm
-if [ \! -x $mdadm ]
-then
-	echo >&2 "test: $mdadm isn't usable."
-fi
-
 testdir="tests"
-logdir="$testdir/logs"
-logsave=0
-exitonerror=1
+targetdir="/var/tmp"
+logdir="$targetdir"
+config=/tmp/mdadm.conf
 
-echo "Testing on linux-$(uname -r) kernel"
+savelogs=0
+exitonerror=1
+prefix='[0-9][0-9]'
 
-# Check whether to run multipath tests
-modprobe multipath 2> /dev/null
-if grep -s 'Personalities : .*multipath' > /dev/null /proc/mdstat
-then
-	MULTIPATH="yes"
-fi
-INTEGRITY=yes
+# use loop devices by default if doesn't specify --dev
 DEVTYPE=loop
+INTEGRITY=yes
 LVM_VOLGROUP=mdtest
 
 # make sure to test local mdmon, not system one
@@ -46,7 +29,6 @@ mdp1=/dev/md_d1
 
 # We test mdadm on loop-back block devices.
 # dir for storing files should be settable by command line maybe
-targetdir=/var/tmp
 size=20000
 # super0, round down to multiple of 64 and substract 64
 mdsize0=19904
@@ -68,20 +50,65 @@ mdsize12=19988
 # ddf needs bigger devices as 32Meg is reserved!
 ddfsize=65536
 
-config=/tmp/mdadm.conf
+# $1 is optional parameter, it shows why to save log
+save_log() {
+	status=$1
+	logfile="$status""$_basename".log
+
+	cat $targetdir/stderr >> $targetdir/log
+	cp $targetdir/log $logdir/$_basename.log
+	echo "## $HOSTNAME: saving dmesg." >> $logdir/$logfile
+	dmesg -c >> $logdir/$logfile
+	$mdadm -As 2> /dev/null
+	echo "## $HOSTNAME: saving proc mdstat." >> $logdir/$logfile
+	cat /proc/mdstat >> $logdir/$logfile
+	array=($(mdadm -Ds | cut -d' ' -f2))
+	echo "## $HOSTNAME: mdadm -D ${array[@]}" >> $logdir/$logfile
+	$mdadm -D ${array[@]} >> $logdir/$logfile
+	[ "$1" == "fail" ] &&
+		echo "FAILED - see $logdir/$_basename.log and $logdir/$logfile for details"
+	# ignore saving external(external file, imsm...) bitmap
+	cat /proc/mdstat | grep -q "linear\|external" && return 0
+	if [ $DEVTYPE == 'lvm' ]
+	then
+		# not supported lvm type yet
+		echo
+	elif [ $DEVTYPE == 'loop' ]
+	then
+		if [ ! -z ${array[@]} -a ${#array[@]} -ge 1 ]
+		then
+			md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/$DEVTYPE" | cut -d'=' -f2))
+			cat /proc/mdstat | grep -q "bitmap"
+			if [ $? -eq 0 ]
+			then
+				echo "## $HOSTNAME: mdadm -X ${md_disks[@]}" >> $logdir/$logfile
+				$mdadm -X ${md_disks[@]} >> $logdir/$logfile
+			fi
+		else
+			echo "## $HOSTNAME: no array assembled!" >> $logdir/$logfile
+		fi
+	fi
+}
+
+die() {
+	echo -e "\n\tERROR: $* \n"
+	save_log fail
+	exit 2
+}
 
 cleanup() {
 	udevadm settle
 	$mdadm -Ssq 2> /dev/null
 	case $DEVTYPE in
-	loop)
+	loop )
 		for d in 0 1 2 3 4 5 6 7  8 9 10 11 12 13
 		do
-			losetup -d /dev/loop$d # rm -f $targetdir/mdtest$d
+			losetup -d /dev/loop$d
 			rm -f /dev/disk/by-path/loop*
+			rm -f /var/tmp/mdtest$d
 		done
 		;;
-	lvm)
+	lvm )
 		for d in 0 1 2 3 4 5 6 7  8 9 10 11 12 13
 		do
 			eval "lvremove --quiet -f \$dev$d"
@@ -98,23 +125,26 @@ do_setup() {
 	trap cleanup 0 1 3 15
 	trap ctrl_c 2
 
-	# make sure there are no loop devices remaining.
-	# udev started things can sometimes prevent them being stopped
-	# immediately
-	while grep loop /proc/partitions > /dev/null 2>&1
-	do
-		mdadm -Ss
-		losetup -d /dev/loop[0-9]* 2> /dev/null
-		sleep 1
-	done
+	[ -d $logdir ] || mkdir -p $logdir
+	dmesg -c > /dev/null
+
+	if [ "$DEVTYPE" == "loop" ]
+	then
+		# make sure there are no loop devices remaining.
+		# udev started things can sometimes prevent them being stopped
+		# immediately
+		while grep loop /proc/partitions > /dev/null 2>&1
+		do
+			$mdadm -Ssq
+			losetup -d /dev/loop[0-9]* 2> /dev/null
+			sleep 0.2
+		done
+	fi
 	devlist=
 	for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
 	do
 		sz=$size
-		if [ $d -gt 7 ]
-		then
-			sz=$ddfsize
-		fi
+		[ $d -gt 7 ] && sz=$ddfsize
 		case $DEVTYPE in
 		loop)
 			[ -f $targetdir/mdtest$d ] ||
@@ -169,7 +199,17 @@ mdadm() {
 		;;
 	esac
 	case $* in
-	*-C* )
+	*-C* | *--create* | *-B* | *--build* )
+		# clear superblock every time once creating or
+		# building arrays, because it's always creating
+		# and building array many times in a test case.
+		for args in $*
+		do
+			[[ $args =~ "/dev/" ]] && {
+				[[ $args =~ "md" ]] ||
+					$mdadm --zero $args > /dev/null
+			}
+		done
 		$mdadm 2> $targetdir/stderr --quiet "$@" --auto=yes
 		;;
 	* )
@@ -191,39 +231,28 @@ mdadm() {
 check() {
 	case $1 in
 	spares )
-		spares=`tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0`
-		if [ $spares -ne $2 ]
-		then
-			echo >&2 "ERROR expected $2 spares, found $spares"
-			exit 1
-		fi
+		spares=$(tr '] ' '\012\012' < /proc/mdstat | grep -c '(S)' || exit 0)
+		[ $spares -ne $2 ] &&
+			die "expected $2 spares, found $spares"
 		;;
 	raid* | linear )
-		grep -s "active $1 " /proc/mdstat > /dev/null || {
-			echo >&2 "ERROR active $1 not found"
-			cat /proc/mdstat
-			exit 1
-		}
+		grep -sq "active $1 " /proc/mdstat ||
+			die "active $1 not found"
 		;;
 	algorithm )
-		grep -s " algorithm $2 " /proc/mdstat > /dev/null || {
-			echo >&2 "ERROR algorithm $2 not found"
-			cat /proc/mdstat
-			exit 1
-		}
+		grep -sq " algorithm $2 " /proc/mdstat ||
+			die "algorithm $2 not found"
 		;;
 	resync | recovery | reshape )
 		cnt=5
-		while ! grep -s $1 /proc/mdstat > /dev/null
+		while ! grep -sq $1 /proc/mdstat
 		do
 			if [ $cnt -gt 0 ] && grep -v idle /sys/block/md*/md/sync_action > /dev/null
 			then # Something isn't idle - wait a bit
 				sleep 0.5
 				cnt=$[cnt-1]
 			else
-				echo >&2 ERROR no $1 happening
-				cat /proc/mdstat
-				exit 1
+				die "no $1 happening"
 			fi
 		done
 		;;
@@ -234,22 +263,18 @@ check() {
 		# to do can still take a little longer than expected.
 		# add an extra check: is sync_completed shows the end is reached, assume
 		# there is no recovery.
-		if grep -s -E '(resync|recovery|reshape) *=' > /dev/null /proc/mdstat
+		if grep -sq -E '(resync|recovery|reshape) *=' /proc/mdstat
 		then
 			incomplete=`grep / /sys/block/md*/md/sync_completed 2> /dev/null | sed '/^ *\([0-9]*\) \/ \1/d'`
-			if [ -n "$incomplete" ]
-			then
-				echo >&2 "ERROR resync or recovery is happening!"
-				cat /proc/mdstat
-				exit 1
-			fi
+			[ -n "$incomplete" ] &&
+				die "resync or recovery is happening!"
 		fi
 		;;
 	wait )
 		p=`cat /proc/sys/dev/raid/speed_limit_max`
 		echo 2000000 > /proc/sys/dev/raid/speed_limit_max
 		sleep 0.1
-		while grep -E '(resync|recovery|reshape|check|repair) *=' > /dev/null /proc/mdstat ||
+		while grep -Eq '(resync|recovery|reshape|check|repair) *=' /proc/mdstat ||
 			grep -v idle > /dev/null /sys/block/md*/md/sync_action
 		do
 			sleep 0.5
@@ -257,45 +282,28 @@ check() {
 		echo $p > /proc/sys/dev/raid/speed_limit_max
 		;;
 	state )
-		grep -s "blocks.*\[$2\]\$" /proc/mdstat > /dev/null || {
-			echo >&2 "ERROR state $2 not found!"
-			cat /proc/mdstat
-			exit 1
-		}
+		grep -sq "blocks.*\[$2\]\$" /proc/mdstat ||
+			die "state $2 not found!"
 		sleep 0.5
 		;;
 	bitmap )
-		grep -s bitmap > /dev/null /proc/mdstat || {
-			echo >&2 ERROR no bitmap
-			cat /proc/mdstat
-			exit 1
-		}
+		grep -sq bitmap /proc/mdstat ||
+			die "no bitmap"
 		;;
 	nobitmap )
-		if grep -s "bitmap" > /dev/null /proc/mdstat
-		then
-			echo >&2 ERROR bitmap present
-			cat /proc/mdstat
-			exit 1
-		fi
+		grep -sq "bitmap" /proc/mdstat &&
+			die "bitmap present"
 		;;
 	readonly )
-		grep -s "read-only" > /dev/null /proc/mdstat || {
-			echo >&2 "ERROR array is not read-only!"
-			cat /proc/mdstat
-			exit 1
-		}
+		grep -sq "read-only" /proc/mdstat ||
+			die "array is not read-only!"
 		;;
 	inactive )
-		grep -s "inactive" > /dev/null /proc/mdstat || {
-			echo >&2 "ERROR array is not inactive!"
-			cat /proc/mdstat
-			exit 1
-		}
+		grep -sq "inactive" /proc/mdstat ||
+			die "array is not inactive!"
 		;;
 	* )
-		echo >&2 ERROR unknown check $1
-		exit 1
+		die "unknown check $1"
 		;;
 	esac
 }
@@ -311,6 +319,7 @@ no_errors() {
 
 # basic device test
 testdev() {
+	[ -b $1 ] || die "$1 isn't a block device."
 	udevadm settle
 	dev=$1
 	cnt=$2
@@ -329,20 +338,11 @@ testdev() {
 		rasize=$[rasize/DEV_ROUND_K/2]
 		rasize=$[rasize*DEV_ROUND_K*2]
 	fi
-	if [ `/sbin/blockdev --getsize $dev` -eq 0 ]
-	then
-		sleep 2
-	fi
+	[ `/sbin/blockdev --getsize $dev` -eq 0 ] && sleep 2
 	_sz=`/sbin/blockdev --getsize $dev`
-	if [ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ]
-	then
-		echo "ERROR: size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
-		exit 1
-	fi
-}
-
-fast_sync() {
-	echo 200000 > /proc/sys/dev/raid/speed_limit_max
+	[ $rasize -lt $_sz -o $[rasize*4/5] -gt $_sz ] &&
+		die "size is wrong for $dev: $cnt * $dvsize (chunk=$chunk) = $rasize, not $_sz"
+	return 0
 }
 
 rotest() {
@@ -359,7 +359,6 @@ do_test() {
 		# stop all arrays, just incase some script left an array active.
 		$mdadm -Ssq 2> /dev/null
 		mdadm --zero $devlist 2> /dev/null
-		mdadm --zero $devlist 2> /dev/null
 		# this might have been reset: restore the default.
 		echo 2000 > /proc/sys/dev/raid/speed_limit_max
 		# source script in a subshell, so it has access to our
@@ -367,52 +366,44 @@ do_test() {
 		echo -ne "$_script... "
 		if ( set -ex ; . $_script ) &> $targetdir/log
 		then
+			dmesg | grep -iq "error\|call trace\|segfault" &&
+				die "dmesg prints errors when testing $_basename!"
 			echo "succeeded"
 			_fail=0
 		else
-			log=log
-			cat $targetdir/stderr >> $targetdir/log
-			echo "=======================dmesg=================" >> $targetdir/log
-			dmesg | tail -n 200 >> $targetdir/log
-			if [ $exitonerror == 0 ]; then
-				log=log-`basename $_script`
-				mv $targetdir/log $logdir/$log
-			fi
-			echo "FAILED - see $logdir/$log for details"
+			save_log fail
 			_fail=1
 		fi
-		if [ "$savelogs" == "1" ]
-		then
-			cp $targetdir/log $logdir/$_basename.log
-		fi
-		if [ "$_fail" == "1" -a "$exitonerror" == "1" ]
-		then
-			exit 1
-		fi
+		[ "$savelogs" == "1" ] &&
+			mv -f $targetdir/log $logdir/$_basename.log
+		[ "$_fail" == "1" -a "$exitonerror" == "1" ] && exit 1
 	fi
 }
 
 do_help() {
-	echo "Usage: $0 [options]"
-	echo " Options:"
-	echo "    --tests=<test1,test2,..>    Comma separated list of tests to run"
-	echo "    --disable-multipath         Disable any tests involving multipath"
-	echo "    --disable-integrity         Disable slow tests of RAID[56] consistency"
-	echo "    --logdir=<directory>        Directory to save logfiles in"
-	echo "    --save-logs                 Save all logs in <logdir>"
-	echo "    --keep-going                Don't stop on error, ie. run all tests"
-	echo "    --dev=[loop|lvm|ram]        Use loop devices (default), LVM, or RAM disk"
-	echo "    --volgroup=<name>           LVM volume group for LVM test"
-	echo "    setup                       Setup test environment and exit"
-	echo "    cleanup                     Cleanup test environment"
-	echo "    <prefix>                    Run tests with <prefix>"
+	cat <<-EOF
+	Usage: $0 [options]
+	Options:
+		--tests=test1,test2,...     Comma separated list of tests to run
+		--disable-multipath         Disable any tests involving multipath
+		--disable-integrity         Disable slow tests of RAID[56] consistency
+		--logdir=directory          Directory to save all logfiles in
+		--save-logs                 Usually use with --logdir together
+		--keep-going | --no-error   Don't stop on error, ie. run all tests
+		--dev=loop|lvm|ram          Use loop devices (default), LVM, or RAM disk
+		--volgroup=name             LVM volume group for LVM test
+		setup                       Setup test environment and exit
+		cleanup                     Cleanup test environment
+		prefix                      Run tests with <prefix>
+		--help | -h                 Print this usage
+	EOF
 }
 
 parse_args() {
 	for i in $*
 	do
 		case $i in
-		[0-9]* )
+		[0-9][0-9] )
 			prefix=$i
 			;;
 		setup )
@@ -426,10 +417,10 @@ parse_args() {
 			exit 0
 			;;
 		--tests=* )
-			TESTLIST=`expr "x$i" : 'x[^=]*=\(.*\)' | sed -e 's/,/ /g'`
+			TESTLIST=($(echo ${i##*=} | sed -e 's/,/ /g'))
 			;;
 		--logdir=* )
-			logdir=`expr "x$i" : 'x[^=]*=\(.*\)'`
+			logdir="${i##*=}"
 			;;
 		--save-logs )
 			savelogs=1
@@ -443,52 +434,109 @@ parse_args() {
 		--disable-integrity )
 			unset INTEGRITY
 			;;
-		--dev=loop )
-			DEVTYPE=loop
-			;;
-		--dev=lvm )
-			DEVTYPE=lvm
-			;;
-		--dev=ram )
-			DEVTYPE=ram
+		--dev=* )
+			case ${i##*=} in
+			loop )
+				DEVTYPE=loop
+				;;
+			lvm )
+				DEVTYPE=lvm
+				;;
+			ram )
+				DEVTYPE=ram
+				;;
+			* )
+				echo "Unknown argument: $i"
+				do_help
+				exit 1
+				;;
+			esac
 			;;
 		--volgroup=* )
 			LVM_VOLGROUP=`expr "x$i" : 'x[^=]*=\(.*\)'`
 			;;
-		--help )
+		--help | -h )
 			do_help
 			exit 0
 			;;
-		-* )
+		* )
 			echo " $0: Unknown argument: $i"
 			do_help
-			exit 0
+			exit 1
 			;;
 		esac
 	done
 }
 
-logdir=$targetdir
-parse_args $@
+check_env() {
+	user=$(id -un)
+	[ "X$user" != "Xroot" ] && {
+		echo "test: testing can only be done as 'root'."
+		exit 1
+	}
+	[ -x "raid6check" -a -x $mdadm ] || {
+		echo "test: please run 'make everything' before perform testing."
+		exit 1
+	}
+	cmds=(mdadm lsblk df udevadm losetup mkfs.ext3 fsck)
+	for cmd in ${cmds[@]}
+	do
+		which $cmd > /dev/null || {
+			echo "$cmd command not found!"
+			exit 1
+		}
+	done
+	mdadm_src_ver="$($mdadm -V 2>&1)"
+	mdadm_sbin_ver="$($(which mdadm) -V 2>&1)"
+	if [ "$mdadm_src_ver" != "$mdadm_sbin_ver" ]
+	then
+		# it's nessesary to 'make install' mdadm to /SBIN/DIR,
+		# such as systemd/mdadm-grow-continue@.service, would
+		# run as an instance by systemd when reshape happens,
+		# thus ensure that the correct mdadm is in testing.
+		echo "test: please run 'make install' before testing."
+		exit 1
+	fi
+	if ! $(df -T . | grep -iq ext)
+	then
+		# 'external file' bitmap only supports with ext[2-4] file system
+		echo "test: please run test suite with ext[2-4] file system."
+		exit 1
+	fi
+	if $(lsblk -a | grep -iq raid)
+	then
+		# donot run mdadm -Ss directly if there are RAIDs working.
+		echo "test: please run test suite without running RAIDs environment."
+		exit 1
+	fi
+	# Check whether to run multipath tests
+	modprobe multipath 2> /dev/null
+	grep -sq 'Personalities : .*multipath' /proc/mdstat &&
+		MULTIPATH="yes"
+}
 
-do_setup
-mkdir -p $logdir
+main() {
+	check_env
+	do_setup
 
-if [ "$savelogs" == "1" ]
-then
-	echo "Saving logs to $logdir"
-fi
+	echo "Testing on linux-$(uname -r) kernel"
+	[ "$savelogs" == "1" ] &&
+		echo "Saving logs to $logdir"
+	if [ "x$TESTLIST" != "x" ]
+	then
+		for script in ${TESTLIST[@]}
+		do
+			do_test $testdir/$script
+		done
+	else
+		for script in $testdir/$prefix $testdir/$prefix*[^~]
+		do
+			do_test $script
+		done
+	fi
 
-if [ "x$TESTLIST" != "x" ]
-then
-	for script in $TESTLIST
-	do
-		do_test $testdir/$script
-	done
-else
-	for script in $testdir/$prefix $testdir/$prefix*[^~]
-	do
-		do_test $script
-	done
-fi
-exit 0
+	exit 0
+}
+
+parse_args $@
+main
-- 
2.6.6

Re: [PATCH v2 1/3] mdadm/test: Refactor and revamp 'test' script

[Anthony Youngman]

On 14/06/17 14:02, Zhilong Liu wrote:
> Improve functions:
> cleanup()
>    clear dmesg and remove the /vat/tmp/mdtest* files.

c/vat/var/ ???

Cheers,
Wol

Re: [PATCH v2 1/3] mdadm/test: Refactor and revamp 'test' script

[Zhilong Liu]

On 06/29/2017 06:20 AM, Anthony Youngman wrote:
>
>
> On 14/06/17 14:02, Zhilong Liu wrote:
>> Improve functions:
>> cleanup()
>>    clear dmesg and remove the /vat/tmp/mdtest* files.
>
> c/vat/var/ ???
>

Thanks for pointing it out, will do.

Regards,
Zhilong

> Cheers,
> Wol
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [PATCH v2 1/3] mdadm/test: Refactor and revamp 'test' script

[Jes Sorensen]

On 06/14/2017 09:02 AM, Zhilong Liu wrote:
> Adding functions:
> die()
>    uniform the abnormal situations that have to abort.
> check_env()
>    do various basic checking before running test suite.
> save_log()
>    collect array infos, include of dmesg, superblock,
>    bitmap and /proc/mdstat.
> main()
>    the core function of this script.
> 
> Improve functions:
> cleanup()
>    clear dmesg and remove the /vat/tmp/mdtest* files.
> mdadm()
>    clear superblock once creating or building arrays
>    every time, because it's always creating arrays
>    many times in a test case.
> check()
>    just tidy up with die(), didn't change code meanings.
> testdev()
>    add checking $1 must be a block device, add 'return 0'
>    in final because this function exists in last line of
>    test case, such as tests/05r6tor0.
> do_test()
>    add checking abnormal dmesg and changing log management.
> do_help()
>    just recommend a better way to print Usage.
> parse_args()
>    revamp and improve.
> 
> Delete function:
> fast_sync()
>    It's no longer used, so get rid of it.
> 
> Signed-off-by: Zhilong Liu <zlliu@suse.com>

Applied!

I fixed the s/vat/var/ issue.

Thanks,
Jes

[PATCH v2 2/3] mdadm/test: Add '--raidtype=' to run different raidlevel cases

[Zhilong Liu]

It supports to specify the argument of "--raidtype"
to run the different raid level cases. Details refer
to the do_help() usage.
For example: ./test --raidtype=raid1
could execute all the raid1 test cases under tests/.

Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
 test | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/test b/test
index 7ee523b..df2eeab 100755
--- a/test
+++ b/test
@@ -385,6 +385,7 @@ do_help() {
 	Usage: $0 [options]
 	Options:
 		--tests=test1,test2,...     Comma separated list of tests to run
+		--raidtype=                 raid0|linear|raid1|raid456|raid10|ddf|imsm
 		--disable-multipath         Disable any tests involving multipath
 		--disable-integrity         Disable slow tests of RAID[56] consistency
 		--logdir=directory          Directory to save all logfiles in
@@ -419,6 +420,36 @@ parse_args() {
 		--tests=* )
 			TESTLIST=($(echo ${i##*=} | sed -e 's/,/ /g'))
 			;;
+		--raidtype=* )
+			case ${i##*=} in
+			raid0 )
+				TESTLIST=($(ls $testdir | grep "[0-9][0-9]r0\|raid0"))
+				;;
+			linear )
+				TESTLIST=($(ls $testdir | grep "linear"))
+				;;
+			raid1 )
+				TESTLIST=($(ls $testdir | grep "[0-9][0-9]r1\|raid1" | grep -vi raid10))
+				;;
+			raid456 )
+				TESTLIST=($(ls $testdir | grep "[0-9][0-9]r[4-6]\|raid[4-6]"))
+				;;
+			raid10 )
+				TESTLIST=($(ls $testdir | grep "[0-9][0-9]r10\|raid10"))
+				;;
+			ddf )
+				TESTLIST=($(ls $testdir | grep "[0-9][0-9]ddf"))
+				;;
+			imsm )
+				TESTLIST=($(ls $testdir | grep "[0-9][0-9]imsm"))
+				;;
+			* )
+				echo "Unknown argument: $i"
+				do_help
+				exit 1
+				;;
+			esac
+			;;
 		--logdir=* )
 			logdir="${i##*=}"
 			;;
-- 
2.6.6

Re: [PATCH v2 2/3] mdadm/test: Add '--raidtype=' to run different raidlevel cases

[Jes Sorensen]

On 06/14/2017 09:02 AM, Zhilong Liu wrote:
> It supports to specify the argument of "--raidtype"
> to run the different raid level cases. Details refer
> to the do_help() usage.
> For example: ./test --raidtype=raid1
> could execute all the raid1 test cases under tests/.
> 
> Signed-off-by: Zhilong Liu <zlliu@suse.com>

Applied!

Thanks,
Jes

[PATCH v2 3/3] mdadm/test: Add '--disks=' to support testing phsical devices

[Zhilong Liu]

If test mode has set as '--dev=disk', then users can specify
the argument of "--disks" to test a bunch of physical devices.
For example: ./test --dev=disk --disks=/dev/sda{2..15}
could execute all test cases on physical devices.

Currently, the --dev=disk mode would confilct with testdev()
in current test cases, thus ignore testing testdev() if has
set --dev=disk mode.

Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
 test | 37 ++++++++++++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 5 deletions(-)

diff --git a/test b/test
index df2eeab..87e2df2 100755
--- a/test
+++ b/test
@@ -73,11 +73,11 @@ save_log() {
 	then
 		# not supported lvm type yet
 		echo
-	elif [ $DEVTYPE == 'loop' ]
+	elif [ "$DEVTYPE" == 'loop' -o "$DEVTYPE" == 'disk' ]
 	then
 		if [ ! -z ${array[@]} -a ${#array[@]} -ge 1 ]
 		then
-			md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/$DEVTYPE" | cut -d'=' -f2))
+			md_disks=($($mdadm -D -Y ${array[@]} | grep "/dev/" | cut -d'=' -f2))
 			cat /proc/mdstat | grep -q "bitmap"
 			if [ $? -eq 0 ]
 			then
@@ -114,6 +114,9 @@ cleanup() {
 			eval "lvremove --quiet -f \$dev$d"
 		done
 		;;
+	disk )
+		$mdadm --zero ${disks[@]} &> /dev/null
+		;;
 	esac
 }
 
@@ -128,6 +131,7 @@ do_setup() {
 	[ -d $logdir ] || mkdir -p $logdir
 	dmesg -c > /dev/null
 
+	devlist=
 	if [ "$DEVTYPE" == "loop" ]
 	then
 		# make sure there are no loop devices remaining.
@@ -139,8 +143,22 @@ do_setup() {
 			losetup -d /dev/loop[0-9]* 2> /dev/null
 			sleep 0.2
 		done
+	elif [ "$DEVTYPE" == "disk" ]
+	then
+		if [ ! -z "$disks" ]
+		then
+			for d in $(seq 0 ${#disks[@]})
+			do
+				eval "dev$d=${disks[$d]}"
+				eval devlist=\"\$devlist \$dev$d\"
+				eval devlist$d=\"\$devlist\"
+			done
+			$mdadm --zero ${disks[@]} &> /dev/null
+		else
+			echo "Forget to provide physical devices for disk mode."
+			exit 1
+		fi
 	fi
-	devlist=
 	for d in 0 1 2 3 4 5 6 7 8 9 10 11 12 13
 	do
 		sz=$size
@@ -320,6 +338,7 @@ no_errors() {
 # basic device test
 testdev() {
 	[ -b $1 ] || die "$1 isn't a block device."
+	[ "$DEVTYPE" == "disk" ] && return 0
 	udevadm settle
 	dev=$1
 	cnt=$2
@@ -383,6 +402,7 @@ do_test() {
 do_help() {
 	cat <<-EOF
 	Usage: $0 [options]
+	Example for disk mode: ./test --dev=disk --disks=/dev/sda{2..15}
 	Options:
 		--tests=test1,test2,...     Comma separated list of tests to run
 		--raidtype=                 raid0|linear|raid1|raid456|raid10|ddf|imsm
@@ -391,7 +411,8 @@ do_help() {
 		--logdir=directory          Directory to save all logfiles in
 		--save-logs                 Usually use with --logdir together
 		--keep-going | --no-error   Don't stop on error, ie. run all tests
-		--dev=loop|lvm|ram          Use loop devices (default), LVM, or RAM disk
+		--dev=loop|lvm|ram|disk     Use loop devices (default), LVM, RAM or disk
+		--disks=                    Provide a bunch of physical devices for test
 		--volgroup=name             LVM volume group for LVM test
 		setup                       Setup test environment and exit
 		cleanup                     Cleanup test environment
@@ -476,6 +497,9 @@ parse_args() {
 			ram )
 				DEVTYPE=ram
 				;;
+			disk )
+				DEVTYPE=disk
+				;;
 			* )
 				echo "Unknown argument: $i"
 				do_help
@@ -483,6 +507,9 @@ parse_args() {
 				;;
 			esac
 			;;
+		--disks=* )
+			disks=(${disks[*]} ${i##*=})
+			;;
 		--volgroup=* )
 			LVM_VOLGROUP=`expr "x$i" : 'x[^=]*=\(.*\)'`
 			;;
@@ -509,7 +536,7 @@ check_env() {
 		echo "test: please run 'make everything' before perform testing."
 		exit 1
 	}
-	cmds=(mdadm lsblk df udevadm losetup mkfs.ext3 fsck)
+	cmds=(mdadm lsblk df udevadm losetup mkfs.ext3 fsck seq)
 	for cmd in ${cmds[@]}
 	do
 		which $cmd > /dev/null || {
-- 
2.6.6