All of
 help / color / mirror / Atom feed
From: Luis Chamberlain <>
Cc:,,, Luis Chamberlain <>
Subject: [PATCH v3 fstests] check: add support for --start-after
Date: Fri, 16 Feb 2024 10:09:46 -0800	[thread overview]
Message-ID: <> (raw)

Often times one is running a new test baseline we want to continue to
start testing where we left off if the last test was a crash. This is
in particular useful if you are doing automation and want to kick off
where the last test crashed.

To do this the first thing that occurred to me was to use the check.time
file as an expunge file but that doesn't work so well if you crashed
as the file turns out empty.

So instead add super simple argument --start-after which let's you
skip all tests inclusive of the test you specified, letting you pick up
where you last left off testing from a crash. This is intended to work
if you are not using a random order.

If the target test is not found in your test list we complain and
bail. This is not as obvious when you specify groups, so likewise
we do a special check when you use groups to ensure the test is at
least part of one group.


root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink -n -g soak --start-after generic/025
Start after test generic/025 not found in any group specified.
Be sure you specify a test present in one of your test run groups if using --start-after.

Your set of groups have these tests:

generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/650

root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink -n -g soak --start-after generic/522
SECTION       -- xfs_reflink
RECREATING    -- xfs on /dev/loop16
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 demo-xfs-reflink 6.5.0-5-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.13-1 (2023-11-29)
MKFS_OPTIONS  -- -f -f -m reflink=1,rmapbt=1, -i sparse=1, /dev/loop5
MOUNT_OPTIONS -- /dev/loop5 /media/scratch


Reviewed-by: Andrey Albershteyn <>
Signed-off-by: Luis Chamberlain <>

Changes since v2:

- Simplify to just one variable as requested by Zorro
- Remove the unused grep_start_after variable as pointed out by Andrey Albersht
- Make the conflict with randomize explicit, although we could support
  it, it is just odd to use it with randomize...
- Replace the not-needed echo -e, with just echo

Changes since v1:                                                                                                                                                                             
This all addresses Anand Jain's feedback.                                                                                                                                                     
 - Skip tests completely which are not going to be run                                                                                                                                        
 - Sanity test to ensure the test is part of a group, if you listed                                                                                                                           
   groups, and if not provide a useful output giving the list of all                                                                                                                          
   tests in your group so you can know better which one is a valid test                                                                                                                       
   to skip                                                                                                                                                                                    
 - Sanity test to ensure the test you specified is valid                                                                                                                                      
 - Moves the trim during file processing now using a routine                                                                                                                                  

 check | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/check b/check
index 71b9fbd07522..f081bf8ce685 100755
--- a/check
+++ b/check
@@ -18,6 +18,7 @@ showme=false
 export here=`pwd`
@@ -80,6 +81,7 @@ check options
     -b			brief test summary
     -R fmt[,fmt]	generate report in formats specified. Supported formats: xunit, xunit-quiet
     --large-fs		optimise scratch device for large filesystems
+    --start-after	only start testing after the test specified
     -s section		run only specified section from config file
     -S section		exclude the specified section from the config file
     -L <n>		loop tests <n> times following a failure, measuring aggregate pass/fail metrics
@@ -120,6 +122,8 @@ examples:
  check -x stress xfs/*
  check -X .exclude -g auto
  check -E ~/.xfstests.exclude
+ check --start-after btrfs/010
+ check -n -g soak --start-after generic/522
 	    exit 1
@@ -204,6 +208,23 @@ trim_test_list()
 	rm -f $tmp.grep
+# takes the list of tests to run in $tmp.list and skips all tests until
+# the specified test is found. This will ensure the tests start after the
+# test specified, it skips the test specified.
+	local skip_test="$1"
+	local starts_regexp=$(echo $skip_test | sed -e 's|\/|\\/|')
+	if grep -q $skip_test $tmp.list ; then
+		rm -f $tmp.grep
+		awk 'f;/.*'$starts_regexp'/{f=1}' $tmp.list > $tmp.tmp
+		mv $tmp.tmp $tmp.list
+	else
+		echo "Test $skip_test not found in test list, be sure to use a valid test if using --start-after"
+		exit 1
+	fi
@@ -233,6 +254,9 @@ _prepare_test_list()
 		# no test numbers, do everything
+		local group_all
+		local start_after_found=0
+		list=""
 		for group in $GROUP_LIST; do
 			list=$(get_group_list $group)
 			if [ -z "$list" ]; then
@@ -240,11 +264,28 @@ _prepare_test_list()
 				exit 1
+			if [[ "$start_after_test" != "" && $start_after_found -ne 1 ]]; then
+				echo $list | grep -q $start_after_test
+				if [[ $? -eq 0 ]]; then
+					start_after_found=1
+				fi
+			fi
 			for t in $list; do
 				grep -s "^$t\$" $tmp.list >/dev/null || \
 							echo "$t" >>$tmp.list
+			group_all="$group_all $list"
+		if [[ "$start_after_test" != "" && $start_after_found -ne 1 ]]; then
+			group_all=$(echo $group_all | sed -e 's|tests/||g')
+			echo "Start after test $start_after_test not found in any group specified."
+			echo "Be sure you specify a test present in one of your test run groups if using --start-after."
+			echo
+			echo "Your set of groups have these tests:"
+			echo
+			echo $group_all
+			exit 1
+		fi
 	# Specified groups to exclude
@@ -258,6 +299,10 @@ _prepare_test_list()
 		trim_test_list $list
+	if [[ "$start_after_test" != "" ]]; then
+		trim_start_after $start_after_test
+	fi
 	# sort the list of tests into numeric order unless we're running tests
 	# in the exact order specified
 	if ! $exact_order; then
@@ -313,6 +358,14 @@ while [ $# -gt 0 ]; do
 				<(sed "s/#.*$//" $xfile)
+	--start-after)
+		if $randomize; then
+			echo "Cannot specify -r and --start-after."
+			exit 1
+		fi
+		start_after_test="$2"
+		shift
+		;;
 	-s)	RUN_SECTION="$RUN_SECTION $2"; shift ;;
 	-l)	diff="diff" ;;

             reply	other threads:[~2024-02-16 18:09 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-16 18:09 Luis Chamberlain [this message]
2024-02-19 13:20 ` [PATCH v3 fstests] check: add support for --start-after David Disseldorp

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.