All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: djwong@kernel.org, guaneryu@gmail.com
Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me
Subject: [PATCH 5/6] check: run tests in exactly the order specified
Date: Tue, 09 Feb 2021 18:56:47 -0800	[thread overview]
Message-ID: <161292580772.3504537.14460569826738892955.stgit@magnolia> (raw)
In-Reply-To: <161292577956.3504537.3260962158197387248.stgit@magnolia>

From: Darrick J. Wong <djwong@kernel.org>

Introduce a new --exact-order switch to disable all sorting, filtering
of repeated lines, and shuffling of test order.  The goal of this is to
be able to run tests in a specific order, namely to try to reproduce
test failures that could be the result of a -r(andomize) run getting
lucky.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 check |   36 ++++++++++++++++++++++++++++--------
 1 file changed, 28 insertions(+), 8 deletions(-)


diff --git a/check b/check
index 6f8db858..106ec8e1 100755
--- a/check
+++ b/check
@@ -20,6 +20,7 @@ diff="diff -u"
 showme=false
 have_test_arg=false
 randomize=false
+exact_order=false
 export here=`pwd`
 xfile=""
 subdir_xfile=""
@@ -67,6 +68,7 @@ check options
     -n			show me, do not run tests
     -T			output timestamps
     -r			randomize test order
+    --exact-order	run tests in the exact order specified
     -i <n>		iterate the test list <n> times
     -d			dump test output to stdout
     -b			brief test summary
@@ -249,17 +251,22 @@ _prepare_test_list()
 		trim_test_list $list
 	done
 
-	# sort the list of tests into numeric order
-	if $randomize; then
-		if type shuf >& /dev/null; then
-			sorter="shuf"
+	# sort the list of tests into numeric order unless we're running tests
+	# in the exact order specified
+	if ! $exact_order; then
+		if $randomize; then
+			if type shuf >& /dev/null; then
+				sorter="shuf"
+			else
+				sorter="awk -v seed=$RANDOM -f randomize.awk"
+			fi
 		else
-			sorter="awk -v seed=$RANDOM -f randomize.awk"
+			sorter="cat"
 		fi
+		list=`sort -n $tmp.list | uniq | $sorter`
 	else
-		sorter="cat"
+		list=`cat $tmp.list`
 	fi
-	list=`sort -n $tmp.list | uniq | $sorter`
 	rm -f $tmp.list
 }
 
@@ -304,7 +311,20 @@ while [ $# -gt 0 ]; do
 	-udiff)	diff="$diff -u" ;;
 
 	-n)	showme=true ;;
-        -r)	randomize=true ;;
+	-r)
+		if $exact_order; then
+			echo "Cannot specify -r and --exact-order."
+			exit 1
+		fi
+		randomize=true
+		;;
+	--exact-order)
+		if $randomize; then
+			echo "Cannnot specify --exact-order and -r."
+			exit 1
+		fi
+		exact_order=true
+		;;
 	-i)	iterations=$2; shift ;;
 	-T)	timestamp=true ;;
 	-d)	DUMP_OUTPUT=true ;;


  parent reply	other threads:[~2021-02-10  2:57 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-10  2:56 [PATCHSET 0/6] fstests: various improvements to the test framework Darrick J. Wong
2021-02-10  2:56 ` [PATCH 1/6] config: wrap xfs_metadump as $XFS_METADUMP_PROG like the other tools Darrick J. Wong
2021-02-11 13:58   ` Brian Foster
2021-02-10  2:56 ` [PATCH 2/6] common: capture metadump output if xfs filesystem check fails Darrick J. Wong
2021-02-11 13:59   ` Brian Foster
2021-02-11 18:12     ` Darrick J. Wong
2021-02-11 18:35       ` Brian Foster
2021-02-11 19:05         ` Darrick J. Wong
2021-02-10  2:56 ` [PATCH 3/6] check: allow '-e testid' to exclude a single test Darrick J. Wong
2021-02-11 14:00   ` Brian Foster
2021-02-10  2:56 ` [PATCH 4/6] check: don't abort on non-existent excluded groups Darrick J. Wong
2021-02-11 14:00   ` Brian Foster
2021-02-11 17:27     ` Darrick J. Wong
2021-02-11 18:01       ` Brian Foster
2021-02-10  2:56 ` Darrick J. Wong [this message]
2021-02-11 14:00   ` [PATCH 5/6] check: run tests in exactly the order specified Brian Foster
2021-02-11 17:28     ` Darrick J. Wong
2021-02-10  2:56 ` [PATCH 6/6] fuzzy: capture core dumps from repair utilities Darrick J. Wong
2021-02-11 14:00   ` Brian Foster

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=161292580772.3504537.14460569826738892955.stgit@magnolia \
    --to=djwong@kernel.org \
    --cc=fstests@vger.kernel.org \
    --cc=guan@eryu.me \
    --cc=guaneryu@gmail.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.