linux-nvme.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH blktests v3 00/15] support test case repeat by different conditions
@ 2024-04-24  7:59 Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 01/15] check: factor out _run_test() Shin'ichiro Kawasaki
                   ` (14 more replies)
  0 siblings, 15 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

In the recent discussion for nvme test group [1], two pain points were mentioned
regarding the test case runs.

1) Several test cases in nvme test group do exactly the same test except the
   NVME transport backend set up condition difference (device vs. file). This
   results in duplicate test script codes. It is desired to unify the test cases
   and run them repeatedly with the different conditions.

2) NVME transport types can be specified with nvme_trtype parameter so that the
   same tests can be run for various transport types. However, some test cases
   do not depend on the transport types. They are repeated in multiple runs for
   the various transport types under the exact same conditions. It is desired to
   repeat the test cases only when such repetition is required.

[1] https://lore.kernel.org/linux-block/w2eaegjopbah5qbjsvpnrwln2t5dr7mv3v4n2e63m5tjqiochm@uonrjm2i2g72/

One idea to address these pain points is to add the test repeat feature to the
nvme test group. However, Daniel questioned if the feature could be implemented
in the blktests framework. Actually, a similar feature has already been
implemented to repeat some test cases for non-zoned block devices and zoned
block devices. However, this feature is implemented only for the zoned and non-
zoned device conditions. It can not fulfill the desires for nvme test group.

This series proposes to generalize the feature in the blktests framework to
repeat the test cases with different conditions. Introduce a new function
set_conditions() that each test case can define and instruct the framework to
repeat the test case. This series applies this feature to nvme test group so
that the test cases can be repeated for NVME transport types and backend types
in the ideal way. For this purpose, this series introduces new config parameters
NVMET_TRTYPES and NVMET_BLKDEV_TYPES. Taking this chance, it renames other
lowercase config parameters nvme_img_size, nvme_num_iter and use_rxe to
uppercase to follow the guide for environment variables.

The first four patches introduce the feature and apply it to the repetition for
non-zoned and zoned block devices. The fifth patch introduces a helper function
to prepare config parameter rename to uppercase. The following seven patches
apply the test case repeat feature to nvme group. Three of the seven patches are
reused from the work by Daniel. The last three patches rename the config
parameters. The all patches are listed in the order that does not lose the test
coverage with the default set up.

As an example of the repeated test run, let's say NVMET_TRTYPES and
NVMET_BLKDEV_TYPES are defined in the config file as follows:

  NVMET_TRTYPES="loop rdma tcp"
  NVMET_BLKDEV_TYPES="device file"

In this case, the test cases which depend on these parameters are repeated
3 x 2 = 6 times. For example, nvme/006 is repeated as follows.

nvme/006 (nvmet bd=device tr=loop) (create an NVMeOF target) [passed]
    runtime  0.148s  ...  0.165s
nvme/006 (nvmet bd=device tr=rdma) (create an NVMeOF target) [passed]
    runtime  0.273s  ...  0.235s
nvme/006 (nvmet bd=device tr=tcp) (create an NVMeOF target)  [passed]
    runtime  0.162s  ...  0.164s
nvme/006 (nvmet bd=file tr=loop) (create an NVMeOF target)   [passed]
    runtime  0.138s  ...  0.134s
nvme/006 (nvmet bd=file tr=rdma) (create an NVMeOF target)   [passed]
    runtime  0.216s  ...  0.201s
nvme/006 (nvmet bd=file tr=tcp) (create an NVMeOF target)    [passed]
    runtime  0.154s  ...  0.146s


Changes from v2:
* 5th patch: added to prepare for the parameter rename
* 6th patch: modified nvme_trtype description and added NVMET_TRTYPES check
* 13-15th patches: added for the parameter rename
* Applied Reviewed-by tags
* Rebased to the latest master branch tip

Changes from v1:
* Renamed NVMET_TR_TYPES to NVMET_TRTYPES
* 1st patch: reflected comments on the list and added Reviewed-by tag
* 5th patch: changed NVMET_TRTYPES from array to variable
* 7th patch: changed NVMET_BLKDEV_TYPES from array to variable
* Reflected other comments on the list


Daniel Wagner (3):
  nvme/rc: add blkdev type environment variable
  nvme/{007,009,011,013,015,020,024}: drop duplicate nvmet blkdev type
    tests
  nvme/{021,022,025,026,027,028}: do not hard code target blkdev type

Shin'ichiro Kawasaki (12):
  check: factor out _run_test()
  check: support test case repeat by different conditions
  check: use set_conditions() for the CAN_BE_ZONED test cases
  meta/{016,017}: add test cases to check repeated test case runs
  common/rc: introduce _check_conflict_and_set_default()
  nvme/rc: introduce NVMET_TRTYPES
  nvme/rc: introduce NVMET_BLKDEV_TYPES
  nvme/{002-031,033-038,040-045,047,048}: support NMVET_TRTYPES
  nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE
  nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER
  nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE

 Documentation/running-tests.md |  33 ++++++---
 Makefile                       |   3 +-
 check                          | 129 ++++++++++++++++++++++-----------
 common/multipath-over-rdma     |   5 +-
 common/rc                      |  27 +++++++
 common/shellcheck              |   2 +-
 common/zoned                   |  22 ++++++
 new                            |  21 ++++++
 tests/meta/016                 |  29 ++++++++
 tests/meta/016.out             |   2 +
 tests/meta/017                 |  29 ++++++++
 tests/meta/017.out             |   2 +
 tests/nvme/002                 |   4 +
 tests/nvme/003                 |   4 +
 tests/nvme/004                 |   4 +
 tests/nvme/005                 |   4 +
 tests/nvme/006                 |   8 +-
 tests/nvme/007                 |  27 -------
 tests/nvme/007.out             |   2 -
 tests/nvme/008                 |   8 +-
 tests/nvme/009                 |  36 ---------
 tests/nvme/009.out             |   3 -
 tests/nvme/010                 |  10 ++-
 tests/nvme/011                 |  39 ----------
 tests/nvme/011.out             |   3 -
 tests/nvme/012                 |   8 +-
 tests/nvme/013                 |  43 -----------
 tests/nvme/013.out             |   3 -
 tests/nvme/014                 |   8 +-
 tests/nvme/015                 |  48 ------------
 tests/nvme/015.out             |   4 -
 tests/nvme/016                 |   6 +-
 tests/nvme/017                 |   8 +-
 tests/nvme/018                 |   4 +
 tests/nvme/019                 |   8 +-
 tests/nvme/020                 |  40 ----------
 tests/nvme/020.out             |   4 -
 tests/nvme/021                 |  10 ++-
 tests/nvme/022                 |  10 ++-
 tests/nvme/023                 |   8 +-
 tests/nvme/024                 |  40 ----------
 tests/nvme/024.out             |   2 -
 tests/nvme/025                 |  10 ++-
 tests/nvme/026                 |  10 ++-
 tests/nvme/027                 |  10 ++-
 tests/nvme/028                 |  10 ++-
 tests/nvme/029                 |   4 +
 tests/nvme/030                 |   4 +
 tests/nvme/031                 |   6 +-
 tests/nvme/033                 |   4 +
 tests/nvme/034                 |   6 +-
 tests/nvme/035                 |   8 +-
 tests/nvme/036                 |   4 +
 tests/nvme/037                 |   4 +
 tests/nvme/038                 |   4 +
 tests/nvme/040                 |   4 +
 tests/nvme/041                 |   3 +
 tests/nvme/042                 |   3 +
 tests/nvme/043                 |   3 +
 tests/nvme/044                 |   3 +
 tests/nvme/045                 |   3 +
 tests/nvme/047                 |   4 +
 tests/nvme/048                 |   4 +
 tests/nvme/rc                  |  74 ++++++++++++++++---
 tests/srp/rc                   |   2 +-
 65 files changed, 493 insertions(+), 404 deletions(-)
 create mode 100644 common/zoned
 create mode 100755 tests/meta/016
 create mode 100644 tests/meta/016.out
 create mode 100755 tests/meta/017
 create mode 100644 tests/meta/017.out
 delete mode 100755 tests/nvme/007
 delete mode 100644 tests/nvme/007.out
 delete mode 100755 tests/nvme/009
 delete mode 100644 tests/nvme/009.out
 delete mode 100755 tests/nvme/011
 delete mode 100644 tests/nvme/011.out
 delete mode 100755 tests/nvme/013
 delete mode 100644 tests/nvme/013.out
 delete mode 100755 tests/nvme/015
 delete mode 100644 tests/nvme/015.out
 delete mode 100755 tests/nvme/020
 delete mode 100644 tests/nvme/020.out
 delete mode 100755 tests/nvme/024
 delete mode 100644 tests/nvme/024.out

-- 
2.44.0



^ permalink raw reply	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 01/15] check: factor out _run_test()
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 02/15] check: support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

The function _run_test() is rather complex and has deep nests. Before
modifying it for repeated test case runs, simplify it. Factor out some
part of the function to the new functions _check_and_call_test() and
_check_and_call_test_device().

Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com>
Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 check | 93 +++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 56 insertions(+), 37 deletions(-)

diff --git a/check b/check
index 55871b0..7d09ec0 100755
--- a/check
+++ b/check
@@ -463,6 +463,59 @@ _unload_modules() {
 	unset MODULES_TO_UNLOAD
 }
 
+_check_and_call_test() {
+	local ret
+
+	if declare -fF requires >/dev/null; then
+		requires
+	fi
+
+	RESULTS_DIR="$OUTPUT/nodev"
+	_call_test test
+	ret=$?
+	if (( RUN_ZONED_TESTS && CAN_BE_ZONED )); then
+		RESULTS_DIR="$OUTPUT/nodev_zoned"
+		RUN_FOR_ZONED=1
+		_call_test test
+		ret=$(( ret || $? ))
+	fi
+
+	return $ret
+}
+
+_check_and_call_test_device() {
+	local unset_skip_reason
+	local ret
+
+	if declare -fF requires >/dev/null; then
+		requires
+	fi
+
+	for TEST_DEV in "${TEST_DEVS[@]}"; do
+		TEST_DEV_SYSFS="${TEST_DEV_SYSFS_DIRS["$TEST_DEV"]}"
+		TEST_DEV_PART_SYSFS="${TEST_DEV_PART_SYSFS_DIRS["$TEST_DEV"]}"
+
+		unset_skip_reason=0
+		if [[ ! -v SKIP_REASONS ]]; then
+			unset_skip_reason=1
+			if (( !CAN_BE_ZONED )) && _test_dev_is_zoned; then
+				SKIP_REASONS+=("${TEST_DEV} is a zoned block device")
+			elif declare -fF device_requires >/dev/null; then
+				device_requires
+			fi
+		fi
+		RESULTS_DIR="$OUTPUT/$(basename "$TEST_DEV")"
+		if ! _call_test test_device; then
+			ret=1
+		fi
+		if (( unset_skip_reason )); then
+			unset SKIP_REASONS
+		fi
+	done
+
+	return $ret
+}
+
 _run_test() {
 	TEST_NAME="$1"
 	CAN_BE_ZONED=0
@@ -482,19 +535,8 @@ _run_test() {
 	. "tests/${TEST_NAME}"
 
 	if declare -fF test >/dev/null; then
-		if declare -fF requires >/dev/null; then
-			requires
-		fi
-
-		RESULTS_DIR="$OUTPUT/nodev"
-		_call_test test
+		_check_and_call_test
 		ret=$?
-		if (( RUN_ZONED_TESTS && CAN_BE_ZONED )); then
-			RESULTS_DIR="$OUTPUT/nodev_zoned"
-			RUN_FOR_ZONED=1
-			_call_test test
-			ret=$(( ret || $? ))
-		fi
 	else
 		if [[ ${#TEST_DEVS[@]} -eq 0 ]] && \
 			declare -fF fallback_device >/dev/null; then
@@ -516,31 +558,8 @@ _run_test() {
 			return 0
 		fi
 
-		if declare -fF requires >/dev/null; then
-			requires
-		fi
-
-		for TEST_DEV in "${TEST_DEVS[@]}"; do
-			TEST_DEV_SYSFS="${TEST_DEV_SYSFS_DIRS["$TEST_DEV"]}"
-			TEST_DEV_PART_SYSFS="${TEST_DEV_PART_SYSFS_DIRS["$TEST_DEV"]}"
-
-			local unset_skip_reason=0
-			if [[ ! -v SKIP_REASONS ]]; then
-				unset_skip_reason=1
-				if (( !CAN_BE_ZONED )) && _test_dev_is_zoned; then
-					SKIP_REASONS+=("${TEST_DEV} is a zoned block device")
-				elif declare -fF device_requires >/dev/null; then
-					device_requires
-				fi
-			fi
-			RESULTS_DIR="$OUTPUT/$(basename "$TEST_DEV")"
-			if ! _call_test test_device; then
-				ret=1
-			fi
-			if (( unset_skip_reason )); then
-				unset SKIP_REASONS
-			fi
-		done
+		_check_and_call_test_device
+		ret=$?
 
 		if (( FALLBACK_DEVICE )); then
 			cleanup_fallback_device
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 02/15] check: support test case repeat by different conditions
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 01/15] check: factor out _run_test() Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 03/15] check: use set_conditions() for the CAN_BE_ZONED test cases Shin'ichiro Kawasaki
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

It is often required to run the same test with slightly different test
conditions. If we create each test case for each test condition, those
test cases are almost same and have code duplication. Such duplication
is seen in many of the nvme test cases that set up nvme transport.

To avoid the code duplication, introduce a new feature to support test
case repetition with different conditions. When a test case implements
the function set_conditions(), blktests repeat the test case. When
set_conditions() is called without an argument, it returns how many
times the test case is to be repeated. Before each test case run,
blktests calls set_conditions() with an argument number from 0 to the
number of repetitions minus 1. set_conditions() sets up the condition
for each test run referring to the argument as the index of the
condition to set up. set_conditions() also sets up a short string in
the COND_DESC variable. This string is printed to stdout to identify the
condition of each run. It is also used as the directory path name to
hold result files.

Document the usage of set_conditions() in the new script. Separate out
shellcheck command line for the new script to avoid a false-positive
warning unique to the file.

Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Makefile          |  3 ++-
 check             | 57 ++++++++++++++++++++++++++++++++++-------------
 common/shellcheck |  2 +-
 new               | 21 +++++++++++++++++
 4 files changed, 65 insertions(+), 18 deletions(-)

diff --git a/Makefile b/Makefile
index 43f2ab0..1c685fe 100644
--- a/Makefile
+++ b/Makefile
@@ -18,8 +18,9 @@ install:
 SHELLCHECK_EXCLUDE := SC2119
 
 check:
-	shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc check new common/* \
+	shellcheck -x -e $(SHELLCHECK_EXCLUDE) -f gcc check common/* \
 		tests/*/rc tests/*/[0-9]*[0-9] src/*.sh
+	shellcheck --exclude=$(SHELLCHECK_EXCLUDE),SC2154 --format=gcc new
 	! grep TODO tests/*/rc tests/*/[0-9]*[0-9]
 	! find -name '*.out' -perm /u=x+g=x+o=x -printf '%p is executable\n' | grep .
 
diff --git a/check b/check
index 7d09ec0..edc421d 100755
--- a/check
+++ b/check
@@ -17,7 +17,9 @@ _found_test() {
 	local test_name="$1"
 	local explicit="$2"
 
-	unset CAN_BE_ZONED DESCRIPTION QUICK TIMED requires device_requires test test_device fallback_device cleanup_fallback_device
+	unset CAN_BE_ZONED DESCRIPTION QUICK TIMED requires device_requires \
+	      test test_device fallback_device cleanup_fallback_device \
+	      set_conditions
 
 	# shellcheck disable=SC1090
 	if ! . "tests/${test_name}"; then
@@ -190,15 +192,12 @@ _write_test_run() {
 _output_status() {
 	local test="$1"
 	local status="$2"
-	local zoned=" "
+	local str="${test} "
 
-	if (( RUN_FOR_ZONED )); then zoned=" (zoned) "; fi
-
-	if [[ "${DESCRIPTION:-}" ]]; then
-		printf '%-60s' "${test}${zoned}($DESCRIPTION)"
-	else
-		printf '%-60s' "${test}${zoned}"
-	fi
+	(( RUN_FOR_ZONED )) && str="$str(zoned) "
+	[[ ${COND_DESC:-} ]] && str="$str(${COND_DESC}) "
+	[[ ${DESCRIPTION:-} ]] && str="$str(${DESCRIPTION})"
+	printf '%-60s' "${str}"
 	if [[ -z $status ]]; then
 		echo
 		return
@@ -464,17 +463,19 @@ _unload_modules() {
 }
 
 _check_and_call_test() {
+	local postfix
 	local ret
 
 	if declare -fF requires >/dev/null; then
 		requires
 	fi
 
-	RESULTS_DIR="$OUTPUT/nodev"
+	[[ -n $COND_DESC ]] && postfix=_${COND_DESC//[ =]/_}
+	RESULTS_DIR="$OUTPUT/nodev${postfix}"
 	_call_test test
 	ret=$?
 	if (( RUN_ZONED_TESTS && CAN_BE_ZONED )); then
-		RESULTS_DIR="$OUTPUT/nodev_zoned"
+		RESULTS_DIR="$OUTPUT/nodev_zoned${postfix}"
 		RUN_FOR_ZONED=1
 		_call_test test
 		ret=$(( ret || $? ))
@@ -484,6 +485,7 @@ _check_and_call_test() {
 }
 
 _check_and_call_test_device() {
+	local postfix
 	local unset_skip_reason
 	local ret
 
@@ -491,6 +493,7 @@ _check_and_call_test_device() {
 		requires
 	fi
 
+	[[ -n $COND_DESC ]] && postfix=_${COND_DESC//[ =]/_}
 	for TEST_DEV in "${TEST_DEVS[@]}"; do
 		TEST_DEV_SYSFS="${TEST_DEV_SYSFS_DIRS["$TEST_DEV"]}"
 		TEST_DEV_PART_SYSFS="${TEST_DEV_PART_SYSFS_DIRS["$TEST_DEV"]}"
@@ -504,7 +507,7 @@ _check_and_call_test_device() {
 				device_requires
 			fi
 		fi
-		RESULTS_DIR="$OUTPUT/$(basename "$TEST_DEV")"
+		RESULTS_DIR="$OUTPUT/$(basename "$TEST_DEV")""$postfix"
 		if ! _call_test test_device; then
 			ret=1
 		fi
@@ -522,9 +525,11 @@ _run_test() {
 	CHECK_DMESG=1
 	DMESG_FILTER="cat"
 	RUN_FOR_ZONED=0
+	COND_DESC=""
 	FALLBACK_DEVICE=0
 	MODULES_TO_UNLOAD=()
 
+	local nr_conds cond_i
 	local ret=0
 
 	# Ensure job control monitor mode is off in the sub-shell for test case
@@ -535,8 +540,18 @@ _run_test() {
 	. "tests/${TEST_NAME}"
 
 	if declare -fF test >/dev/null; then
-		_check_and_call_test
-		ret=$?
+		if declare -fF set_conditions >/dev/null; then
+			nr_conds=$(set_conditions)
+			for ((cond_i = 0; cond_i < nr_conds; cond_i++)); do
+				set_conditions $cond_i
+				_check_and_call_test
+				ret=$(( ret || $? ))
+				unset SKIP_REASONS
+			done
+		else
+			_check_and_call_test
+			ret=$?
+		fi
 	else
 		if [[ ${#TEST_DEVS[@]} -eq 0 ]] && \
 			declare -fF fallback_device >/dev/null; then
@@ -558,8 +573,18 @@ _run_test() {
 			return 0
 		fi
 
-		_check_and_call_test_device
-		ret=$?
+		if declare -fF set_conditions >/dev/null; then
+			nr_conds=$(set_conditions)
+			for ((cond_i = 0; cond_i < nr_conds; cond_i++)); do
+				set_conditions $cond_i
+				_check_and_call_test_device
+				ret=$(( ret || $? ))
+				unset SKIP_REASONS
+			done
+		else
+			_check_and_call_test_device
+			ret=$?
+		fi
 
 		if (( FALLBACK_DEVICE )); then
 			cleanup_fallback_device
diff --git a/common/shellcheck b/common/shellcheck
index 8c324bd..ac0a51e 100644
--- a/common/shellcheck
+++ b/common/shellcheck
@@ -6,5 +6,5 @@
 
 # Suppress unused global variable warnings.
 _silence_sc2034() {
-	echo "$CAN_BE_ZONED $CGROUP2_DIR $CHECK_DMESG $DESCRIPTION $DMESG_FILTER $FIO_PERF_FIELDS $FIO_PERF_PREFIX $QUICK $SKIP_REASONS ${TEST_RUN[*]} $TIMED" > /dev/null
+	echo "$CAN_BE_ZONED $CGROUP2_DIR $CHECK_DMESG $COND_DESC $DESCRIPTION $DMESG_FILTER $FIO_PERF_FIELDS $FIO_PERF_PREFIX $QUICK $SKIP_REASONS ${TEST_RUN[*]} $TIMED" > /dev/null
 }
diff --git a/new b/new
index 574d8b4..cb5fab2 100755
--- a/new
+++ b/new
@@ -180,6 +180,27 @@ DESCRIPTION=""
 # 	_require_test_dev_is_foo && _require_test_dev_supports_bar
 # }
 
+# TODO: if the test case can run the same test for different conditions, define
+# the helper function "set_condition". When no argument is specified, return the
+# number of condition variations. Blktests repeats the test case as many times
+# as the returned number. When its argument is specified, refer to it as the
+# condition variation index and set up the conditions for it. Also set the
+# global variable COND_DESC which is printed at the test case run and used for
+# the result directory name. Blktests calls set_condition() before each run of
+# the test case incrementing the argument index from 0.
+# set_conditions() {
+# 	local index=\$1
+# 
+# 	if [[ -z \$index ]]; then
+# 		echo 2  # return number of condition variations
+# 		return
+# 	fi
+# 
+# 	# Set test conditions based on the $index
+# 	...
+# 	COND_DESC="Describe the conditions shortly"
+# }
+
 # TODO: define the test. The output of this function (stdout and stderr) will
 # be compared to tests/\${TEST_NAME}.out. If it does not match, the test is
 # considered a failure. If the test runs a command which has unnecessary
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 03/15] check: use set_conditions() for the CAN_BE_ZONED test cases
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 01/15] check: factor out _run_test() Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 02/15] check: support test case repeat by different conditions Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 04/15] meta/{016,017}: add test cases to check repeated test case runs Shin'ichiro Kawasaki
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

When the test case with test() function is marked as CAN_BE_ZONED,
blktests runs the test case twice: once for non-zoned device, and the
second for zoned device. This is now implemented as a special logic in
the check script.

To simplify the implementation, use the feature to repeat test cases
with different conditions. Use set_conditions() and move out the special
logic from the check script to the common/zoned script file.

Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 check        | 19 ++++++++-----------
 common/zoned | 22 ++++++++++++++++++++++
 2 files changed, 30 insertions(+), 11 deletions(-)
 create mode 100644 common/zoned

diff --git a/check b/check
index edc421d..3ed4510 100755
--- a/check
+++ b/check
@@ -56,6 +56,11 @@ _found_test() {
 		return 1
 	fi
 
+	if [[ -n $CAN_BE_ZONED ]] && declare -fF test >/dev/null && declare -fF set_conditions >/dev/null; then
+		_warning "${test_name} defines both CAN_BE_ZONED and set_conditions()"
+		return 1
+	fi
+
 	if (( QUICK && TIMED )); then
 		_warning "${test_name} cannot be both QUICK and TIMED"
 		return 1
@@ -194,7 +199,6 @@ _output_status() {
 	local status="$2"
 	local str="${test} "
 
-	(( RUN_FOR_ZONED )) && str="$str(zoned) "
 	[[ ${COND_DESC:-} ]] && str="$str(${COND_DESC}) "
 	[[ ${DESCRIPTION:-} ]] && str="$str(${DESCRIPTION})"
 	printf '%-60s' "${str}"
@@ -464,7 +468,6 @@ _unload_modules() {
 
 _check_and_call_test() {
 	local postfix
-	local ret
 
 	if declare -fF requires >/dev/null; then
 		requires
@@ -473,15 +476,6 @@ _check_and_call_test() {
 	[[ -n $COND_DESC ]] && postfix=_${COND_DESC//[ =]/_}
 	RESULTS_DIR="$OUTPUT/nodev${postfix}"
 	_call_test test
-	ret=$?
-	if (( RUN_ZONED_TESTS && CAN_BE_ZONED )); then
-		RESULTS_DIR="$OUTPUT/nodev_zoned${postfix}"
-		RUN_FOR_ZONED=1
-		_call_test test
-		ret=$(( ret || $? ))
-	fi
-
-	return $ret
 }
 
 _check_and_call_test_device() {
@@ -540,6 +534,9 @@ _run_test() {
 	. "tests/${TEST_NAME}"
 
 	if declare -fF test >/dev/null; then
+		if ((RUN_ZONED_TESTS && CAN_BE_ZONED)); then
+			. "common/zoned"
+		fi
 		if declare -fF set_conditions >/dev/null; then
 			nr_conds=$(set_conditions)
 			for ((cond_i = 0; cond_i < nr_conds; cond_i++)); do
diff --git a/common/zoned b/common/zoned
new file mode 100644
index 0000000..6a8f1e5
--- /dev/null
+++ b/common/zoned
@@ -0,0 +1,22 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2024 Western Digital Corporation or its affiliates.
+
+# The helper function for test cases with CAN_BE_ZONED flag and test()
+# function. Run the test case twice for non-zoned and zoned conditions.
+set_conditions() {
+	local index=$1
+
+	if [[ -z $index ]]; then
+		echo 2
+		return
+	fi
+
+	if ((index == 0)); then
+		export RUN_FOR_ZONED=0
+		export COND_DESC=
+	elif ((index == 1)); then
+		export RUN_FOR_ZONED=1
+		export COND_DESC="zoned"
+	fi
+}
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 04/15] meta/{016,017}: add test cases to check repeated test case runs
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (2 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 03/15] check: use set_conditions() for the CAN_BE_ZONED test cases Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 05/15] common/rc: introduce _check_conflict_and_set_default() Shin'ichiro Kawasaki
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

Add test cases to confirm the feature to repeat test case runs with
different conditions is working.

Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/meta/016     | 29 +++++++++++++++++++++++++++++
 tests/meta/016.out |  2 ++
 tests/meta/017     | 29 +++++++++++++++++++++++++++++
 tests/meta/017.out |  2 ++
 4 files changed, 62 insertions(+)
 create mode 100755 tests/meta/016
 create mode 100644 tests/meta/016.out
 create mode 100755 tests/meta/017
 create mode 100644 tests/meta/017.out

diff --git a/tests/meta/016 b/tests/meta/016
new file mode 100755
index 0000000..caf876d
--- /dev/null
+++ b/tests/meta/016
@@ -0,0 +1,29 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2024 Western Digital Corporation or its affiliates.
+#
+# Test repeated test() run with set_conditions()
+
+. tests/meta/rc
+
+DESCRIPTION="repeat test()"
+
+declare cond_set_index
+
+set_conditions() {
+	local index=$1
+
+	if [[ -z $index ]]; then
+		echo 2
+		return
+	fi
+
+	cond_set_index=$index
+	COND_DESC="condition set $index"
+}
+
+test() {
+	echo "Running ${TEST_NAME}"
+	echo "condition set $cond_set_index" >> "$FULL"
+	echo "Test complete"
+}
diff --git a/tests/meta/016.out b/tests/meta/016.out
new file mode 100644
index 0000000..cccfec4
--- /dev/null
+++ b/tests/meta/016.out
@@ -0,0 +1,2 @@
+Running meta/016
+Test complete
diff --git a/tests/meta/017 b/tests/meta/017
new file mode 100755
index 0000000..03f92d6
--- /dev/null
+++ b/tests/meta/017
@@ -0,0 +1,29 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2024 Western Digital Corporation or its affiliates.
+#
+# Test repeated test_device() run with set_conditions()
+
+. tests/meta/rc
+
+DESCRIPTION="repeat test_device()"
+
+declare cond_set_index
+
+set_conditions() {
+	local index=$1
+
+	if [[ -z $index ]]; then
+		echo 2
+		return
+	fi
+
+	cond_set_index=$index
+	COND_DESC="condition set $index"
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+	echo "condition set $cond_set_index" >> "$FULL"
+	echo "Test complete"
+}
diff --git a/tests/meta/017.out b/tests/meta/017.out
new file mode 100644
index 0000000..7fc55ff
--- /dev/null
+++ b/tests/meta/017.out
@@ -0,0 +1,2 @@
+Running meta/017
+Test complete
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 05/15] common/rc: introduce _check_conflict_and_set_default()
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (3 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 04/15] meta/{016,017}: add test cases to check repeated test case runs Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24 12:27   ` Daniel Wagner
  2024-04-24  7:59 ` [PATCH blktests v3 06/15] nvme/rc: introduce NVMET_TRTYPES Shin'ichiro Kawasaki
                   ` (9 subsequent siblings)
  14 siblings, 1 reply; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

Following commits are going to rename some config option parameters from
lowercase letters to uppercase. The old lowercase options will be
deprecated but still be kept usable to not cause confusions. When these
changes are made, it will be required to check that both new and old
parameters are not set at once and ensure they do not have two different
values.

To simplify the code to check the two parameters, introduce the helper
_check_conflict_and_set_default(). If the both two parameters are
set, it errors out. If the old option is set, it propagates the old
option value to the new option. Also, when neither of them is set, it
sets the default value to the new option.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 common/rc | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/common/rc b/common/rc
index c3680f5..9ff31cf 100644
--- a/common/rc
+++ b/common/rc
@@ -470,3 +470,30 @@ convert_to_mb()
 		echo "$((res * 1024))"
 	fi
 }
+
+# Check both old and new parameters are not configured. If the old parameter is
+# set, propagate to the new parameter. If neither is set, set the default value
+# to the new parameter.
+_check_conflict_and_set_default()
+{
+	local new_name="$1"
+	local old_name="$2"
+	local default_val="$3"
+	local new_name_checked="$new_name"_checked
+
+	if [[ -n ${!new_name_checked} ]]; then
+		return
+	fi
+
+	if [[ -n ${!old_name} ]]; then
+		if [[ -n ${!new_name} ]]; then
+			echo "Both ${old_name} and ${new_name} are specified"
+			exit 1
+		fi
+		eval "${new_name}=\"${!old_name}\""
+	elif [[ -z ${!new_name} ]]; then
+		eval "${new_name}=\"${default_val}\""
+	fi
+
+	eval "${new_name_checked}=true"
+}
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 06/15] nvme/rc: introduce NVMET_TRTYPES
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (4 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 05/15] common/rc: introduce _check_conflict_and_set_default() Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24 12:28   ` Daniel Wagner
  2024-04-24  7:59 ` [PATCH blktests v3 07/15] nvme/rc: add blkdev type environment variable Shin'ichiro Kawasaki
                   ` (8 subsequent siblings)
  14 siblings, 1 reply; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

Some of the test cases in nvme test group can be run under various nvme
target transport types. The configuration parameter nvme_trtype
specifies the transport to use. But this configuration method has two
drawbacks. Firstly, the blktests check script needs to be invoked
multiple times to cover multiple transport types. Secondly, the test
cases irrelevant to the transport types are executed exactly same
conditions in the multiple blktests runs.

To avoid the drawbacks, allow setting multiple transport types. Taking
this chance, rename the parameter from nvme_trtype to NVMET_TRTYPES to
follow the uppercase letter naming guide for environment variables.
NVMET_TRTYPES can take multiple transport types like:

    NVMET_TRTYPES="loop tcp"

Introduce _nvmet_set_nvme_trtype() which can be called from the
set_conditions() hook of the transport type dependent test cases.
Blktests will repeat the test case as many as the number of elements in
NVMET_TRTYPES, and set nvme_trtype for each test case run.

Also introduce _NVMET_TRTYPES_is_valid() to check NVMET_TRTYPES value
before test run.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Documentation/running-tests.md | 12 +++++++----
 tests/nvme/rc                  | 37 ++++++++++++++++++++++++++++++----
 2 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/Documentation/running-tests.md b/Documentation/running-tests.md
index ae80860..571ee04 100644
--- a/Documentation/running-tests.md
+++ b/Documentation/running-tests.md
@@ -102,8 +102,12 @@ RUN_ZONED_TESTS=1
 
 The NVMe tests can be additionally parameterized via environment variables.
 
-- nvme_trtype: 'loop' (default), 'tcp', 'rdma' and 'fc'
-  Run the tests with the given transport.
+- NVMET_TRTYPES: 'loop' (default), 'tcp', 'rdma' and 'fc'
+  Set up NVME target backends with the specified transport. Multiple transports
+  can be listed with separating spaces, e.g., "loop tcp rdma". In this case, the
+  tests are repeated to cover all of the transports specified.
+  This parameter had an old name 'nvme_trtype'. The old name is still usable,
+  but not recommended.
 - nvme_img_size: '1G' (default)
   Run the tests with given image size in bytes. 'm', 'M', 'g'
 	and 'G' postfix are supported.
@@ -117,11 +121,11 @@ These tests will use the siw (soft-iWARP) driver by default. The rdma_rxe
 
 ```sh
 To use the siw driver:
-nvme_trtype=rdma ./check nvme/
+NVMET_TRTYPES=rdma ./check nvme/
 ./check srp/
 
 To use the rdma_rxe driver:
-use_rxe=1 nvme_trtype=rdma ./check nvme/
+use_rxe=1 NVMET_TRTYPES=rdma ./check nvme/
 use_rxe=1 ./check srp/
 ```
 
diff --git a/tests/nvme/rc b/tests/nvme/rc
index 1f5ff44..f35ed09 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -18,10 +18,41 @@ def_hostid="0f01fb42-9f7f-4856-b0b3-51e60b8de349"
 def_hostnqn="nqn.2014-08.org.nvmexpress:uuid:${def_hostid}"
 export def_subsysnqn="blktests-subsystem-1"
 export def_subsys_uuid="91fdba0d-f87b-4c25-b80f-db7be1418b9e"
-nvme_trtype=${nvme_trtype:-"loop"}
+_check_conflict_and_set_default NVMET_TRTYPES nvme_trtype "loop"
 nvme_img_size=${nvme_img_size:-"1G"}
 nvme_num_iter=${nvme_num_iter:-"1000"}
 
+_NVMET_TRTYPES_is_valid() {
+	local type
+
+	for type in $NVMET_TRTYPES; do
+		case $type in
+		loop | rdma | tcp | fc)
+			;;
+		*)
+			SKIP_REASONS+=("Invalid NVMET_TRTYPE value: $type")
+			return 1
+			;;
+		esac
+	done
+	return 0
+}
+
+_set_nvme_trtype() {
+	local index=$1
+	local -a types
+
+	read -r -a types <<< "$NVMET_TRTYPES"
+
+	if [[ -z $index ]]; then
+		echo ${#types[@]}
+		return
+	fi
+
+	nvme_trtype=${types[index]}
+	COND_DESC="nvmet tr=${nvme_trtype}"
+}
+
 # TMPDIR can not be referred out of test() or test_device() context. Instead of
 # global variable def_flie_path, use this getter function.
 _nvme_def_file_path() {
@@ -61,9 +92,6 @@ _nvme_requires() {
 		_have_configfs
 		def_adrfam="fc"
 		;;
-	*)
-		SKIP_REASONS+=("unsupported nvme_trtype=${nvme_trtype}")
-		return 1
 	esac
 
 	if [[ -n ${nvme_adrfam} ]]; then
@@ -92,6 +120,7 @@ _nvme_requires() {
 
 group_requires() {
 	_have_root
+	_NVMET_TRTYPES_is_valid
 }
 
 group_device_requires() {
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 07/15] nvme/rc: add blkdev type environment variable
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (5 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 06/15] nvme/rc: introduce NVMET_TRTYPES Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 08/15] nvme/rc: introduce NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

From: Daniel Wagner <dwagner@suse.de>

Introduce nvmet_blkdev_type environment variable which allows to control
the target setup. This allows us to drop duplicate tests which just
differ how the target is setup.

Signed-off-by: Daniel Wagner <dwagner@suse.de>
[Shin'ichiro: dropped description in Documentation/running-tests.md]
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/nvme/rc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/nvme/rc b/tests/nvme/rc
index f35ed09..ec54609 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -21,6 +21,7 @@ export def_subsys_uuid="91fdba0d-f87b-4c25-b80f-db7be1418b9e"
 _check_conflict_and_set_default NVMET_TRTYPES nvme_trtype "loop"
 nvme_img_size=${nvme_img_size:-"1G"}
 nvme_num_iter=${nvme_num_iter:-"1000"}
+nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
 
 _NVMET_TRTYPES_is_valid() {
 	local type
@@ -855,7 +856,7 @@ _find_nvme_passthru_loop_dev() {
 }
 
 _nvmet_target_setup() {
-	local blkdev_type="device"
+	local blkdev_type="${nvmet_blkdev_type}"
 	local blkdev
 	local ctrlkey=""
 	local hostkey=""
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 08/15] nvme/rc: introduce NVMET_BLKDEV_TYPES
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (6 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 07/15] nvme/rc: add blkdev type environment variable Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 09/15] nvme/{002-031,033-038,040-045,047,048}: support NMVET_TRTYPES Shin'ichiro Kawasaki
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

Some of the test cases in nvme test group do the exact same test for two
blkdev types: device type and file type. Except for this difference, the
test cases are pure duplication. It is desired to avoid the duplication.
When the duplication is avoided, it is required to control which
condition to run the test.
    
To avoid the duplication and also to allow the blkdev type control,
introduce a new configuration parameter NVMET_BLKDEV_TYPES. This
parameter specifies which blkdev type to setup for the tests. It can
take one of the blkdev types. Or it can take both types, which is the
default. When both types are specified, the test cases are repeated
twice to cover the types.
    
Also add the helper function _set_nvme_trtype_and_nvmet_blkdev_type().
It sets up nvmet_blkdev_type variable for each test case run from
NVMET_BLKDEV_TYPES. It also sets nvme_trtype from NVMET_TRTYPES.

When NVMET_BLKDEV_TYPES and NVMET_TRTYPES are set as follows, the test
case with _set_nvme_trtype_and_nvmet_blkdev_type in set_condition() hook
is called 2 x 3 = 6 times.

  NVMET_BLKDEV_TYPES="device file"
  NVMET_TRTYPES="loop rdma tcp"

Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Documentation/running-tests.md |  5 +++++
 tests/nvme/rc                  | 22 ++++++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/Documentation/running-tests.md b/Documentation/running-tests.md
index 571ee04..64aff7c 100644
--- a/Documentation/running-tests.md
+++ b/Documentation/running-tests.md
@@ -108,6 +108,11 @@ The NVMe tests can be additionally parameterized via environment variables.
   tests are repeated to cover all of the transports specified.
   This parameter had an old name 'nvme_trtype'. The old name is still usable,
   but not recommended.
+- NVMET_BLKDEV_TYPES: 'device', 'file'
+  Set up NVME target backends with the specified block device type. Multiple
+  block device types can be listed with separating spaces. In this case, the
+  tests are repeated to cover all of the block device types specified. Default
+  value is "device file".
 - nvme_img_size: '1G' (default)
   Run the tests with given image size in bytes. 'm', 'M', 'g'
 	and 'G' postfix are supported.
diff --git a/tests/nvme/rc b/tests/nvme/rc
index ec54609..a31690d 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -22,6 +22,7 @@ _check_conflict_and_set_default NVMET_TRTYPES nvme_trtype "loop"
 nvme_img_size=${nvme_img_size:-"1G"}
 nvme_num_iter=${nvme_num_iter:-"1000"}
 nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
+NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
 
 _NVMET_TRTYPES_is_valid() {
 	local type
@@ -54,6 +55,27 @@ _set_nvme_trtype() {
 	COND_DESC="nvmet tr=${nvme_trtype}"
 }
 
+_set_nvme_trtype_and_nvmet_blkdev_type() {
+	local index=$1
+	local bd_index tr_index
+	local -a blkdev_types
+	local -a trtypes
+
+	read -r -a blkdev_types <<< "$NVMET_BLKDEV_TYPES"
+	read -r -a trtypes <<< "$NVMET_TRTYPES"
+
+	if [[ -z $index ]]; then
+		echo $(( ${#trtypes[@]} * ${#blkdev_types[@]} ))
+		return
+	fi
+
+	bd_index=$((index / ${#trtypes[@]}))
+	tr_index=$((index % ${#trtypes[@]}))
+	nvmet_blkdev_type=${blkdev_types[bd_index]}
+	nvme_trtype=${trtypes[tr_index]}
+	COND_DESC="nvmet bd=${nvmet_blkdev_type} tr=${nvme_trtype}"
+}
+
 # TMPDIR can not be referred out of test() or test_device() context. Instead of
 # global variable def_flie_path, use this getter function.
 _nvme_def_file_path() {
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 09/15] nvme/{002-031,033-038,040-045,047,048}: support NMVET_TRTYPES
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (7 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 08/15] nvme/rc: introduce NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

Add set_conditions() hook and call _set_nvme_trtype() so that the test
cases are repeated for NMVET_TRTYPES.

Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/nvme/002 | 4 ++++
 tests/nvme/003 | 4 ++++
 tests/nvme/004 | 4 ++++
 tests/nvme/005 | 4 ++++
 tests/nvme/006 | 4 ++++
 tests/nvme/007 | 4 ++++
 tests/nvme/008 | 4 ++++
 tests/nvme/009 | 4 ++++
 tests/nvme/010 | 4 ++++
 tests/nvme/011 | 4 ++++
 tests/nvme/012 | 4 ++++
 tests/nvme/013 | 4 ++++
 tests/nvme/014 | 4 ++++
 tests/nvme/015 | 4 ++++
 tests/nvme/016 | 4 ++++
 tests/nvme/017 | 4 ++++
 tests/nvme/018 | 4 ++++
 tests/nvme/019 | 4 ++++
 tests/nvme/020 | 4 ++++
 tests/nvme/021 | 4 ++++
 tests/nvme/022 | 4 ++++
 tests/nvme/023 | 4 ++++
 tests/nvme/024 | 4 ++++
 tests/nvme/025 | 4 ++++
 tests/nvme/026 | 4 ++++
 tests/nvme/027 | 4 ++++
 tests/nvme/028 | 4 ++++
 tests/nvme/029 | 4 ++++
 tests/nvme/030 | 4 ++++
 tests/nvme/031 | 4 ++++
 tests/nvme/033 | 4 ++++
 tests/nvme/034 | 4 ++++
 tests/nvme/035 | 4 ++++
 tests/nvme/036 | 4 ++++
 tests/nvme/037 | 4 ++++
 tests/nvme/038 | 4 ++++
 tests/nvme/040 | 4 ++++
 tests/nvme/041 | 3 +++
 tests/nvme/042 | 3 +++
 tests/nvme/043 | 3 +++
 tests/nvme/044 | 3 +++
 tests/nvme/045 | 3 +++
 tests/nvme/047 | 4 ++++
 tests/nvme/048 | 4 ++++
 44 files changed, 171 insertions(+)

diff --git a/tests/nvme/002 b/tests/nvme/002
index 6b84848..f613c78 100755
--- a/tests/nvme/002
+++ b/tests/nvme/002
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_loop
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/003 b/tests/nvme/003
index bb96d1d..36c0753 100755
--- a/tests/nvme/003
+++ b/tests/nvme/003
@@ -17,6 +17,10 @@ requires() {
 	_have_writeable_kmsg
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/004 b/tests/nvme/004
index b751746..a6b4949 100755
--- a/tests/nvme/004
+++ b/tests/nvme/004
@@ -17,6 +17,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/005 b/tests/nvme/005
index f17174d..66c12fd 100755
--- a/tests/nvme/005
+++ b/tests/nvme/005
@@ -16,6 +16,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/006 b/tests/nvme/006
index ccf752c..ff0a9eb 100755
--- a/tests/nvme/006
+++ b/tests/nvme/006
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/007 b/tests/nvme/007
index b92f5d9..cb2637e 100755
--- a/tests/nvme/007
+++ b/tests/nvme/007
@@ -14,6 +14,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/008 b/tests/nvme/008
index a5d0681..1877d8a 100755
--- a/tests/nvme/008
+++ b/tests/nvme/008
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/009 b/tests/nvme/009
index a1655d4..d7b1307 100755
--- a/tests/nvme/009
+++ b/tests/nvme/009
@@ -14,6 +14,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/010 b/tests/nvme/010
index 6feb391..34914a7 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/011 b/tests/nvme/011
index 4810459..bd29129 100755
--- a/tests/nvme/011
+++ b/tests/nvme/011
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/012 b/tests/nvme/012
index 64cb6ec..e06bf8d 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -19,6 +19,10 @@ requires() {
 	_require_nvme_test_img_size 350m
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/013 b/tests/nvme/013
index 24fc910..91da498 100755
--- a/tests/nvme/013
+++ b/tests/nvme/013
@@ -18,6 +18,10 @@ requires() {
 	_require_nvme_test_img_size 350m
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/014 b/tests/nvme/014
index 839b91f..ff0ebfb 100755
--- a/tests/nvme/014
+++ b/tests/nvme/014
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/015 b/tests/nvme/015
index f0621da..b5ec10c 100755
--- a/tests/nvme/015
+++ b/tests/nvme/015
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/016 b/tests/nvme/016
index 908abbd..a65cffd 100755
--- a/tests/nvme/016
+++ b/tests/nvme/016
@@ -13,6 +13,10 @@ requires() {
 	_require_nvme_trtype_is_loop
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/017 b/tests/nvme/017
index c8d9b32..9410cdc 100755
--- a/tests/nvme/017
+++ b/tests/nvme/017
@@ -13,6 +13,10 @@ requires() {
 	_require_nvme_trtype_is_loop
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/018 b/tests/nvme/018
index b8c1635..5b0f57b 100755
--- a/tests/nvme/018
+++ b/tests/nvme/018
@@ -16,6 +16,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/019 b/tests/nvme/019
index 1cd5378..31020d9 100755
--- a/tests/nvme/019
+++ b/tests/nvme/019
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/020 b/tests/nvme/020
index 0364c4e..4993e36 100755
--- a/tests/nvme/020
+++ b/tests/nvme/020
@@ -14,6 +14,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/021 b/tests/nvme/021
index 7ee1f07..270d90e 100755
--- a/tests/nvme/021
+++ b/tests/nvme/021
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/022 b/tests/nvme/022
index 7ce33dd..adaa765 100755
--- a/tests/nvme/022
+++ b/tests/nvme/022
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/023 b/tests/nvme/023
index d8f17ae..da99406 100755
--- a/tests/nvme/023
+++ b/tests/nvme/023
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/024 b/tests/nvme/024
index a512194..cab1818 100755
--- a/tests/nvme/024
+++ b/tests/nvme/024
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/025 b/tests/nvme/025
index 3f9a615..224492b 100755
--- a/tests/nvme/025
+++ b/tests/nvme/025
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/026 b/tests/nvme/026
index 28fd151..6ee6a51 100755
--- a/tests/nvme/026
+++ b/tests/nvme/026
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/027 b/tests/nvme/027
index 053fd58..a63e42b 100755
--- a/tests/nvme/027
+++ b/tests/nvme/027
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/028 b/tests/nvme/028
index 9f4a905..65c52a9 100755
--- a/tests/nvme/028
+++ b/tests/nvme/028
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/029 b/tests/nvme/029
index 559c0b4..10acc58 100755
--- a/tests/nvme/029
+++ b/tests/nvme/029
@@ -16,6 +16,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test_user_io()
 {
 	local disk="$1"
diff --git a/tests/nvme/030 b/tests/nvme/030
index 9251e17..b1ed8bc 100755
--- a/tests/nvme/030
+++ b/tests/nvme/030
@@ -15,6 +15,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/031 b/tests/nvme/031
index 0bf823d..b98630a 100755
--- a/tests/nvme/031
+++ b/tests/nvme/031
@@ -23,6 +23,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/033 b/tests/nvme/033
index 70a73b8..7a69b94 100755
--- a/tests/nvme/033
+++ b/tests/nvme/033
@@ -13,6 +13,10 @@ requires() {
 	_have_kernel_option NVME_TARGET_PASSTHRU
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 nvme_info() {
 	local ns=$1
 
diff --git a/tests/nvme/034 b/tests/nvme/034
index 409324a..522ffe3 100755
--- a/tests/nvme/034
+++ b/tests/nvme/034
@@ -14,6 +14,10 @@ requires() {
 	_have_fio
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test_device() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/035 b/tests/nvme/035
index ecf6b72..cfca5fd 100755
--- a/tests/nvme/035
+++ b/tests/nvme/035
@@ -20,6 +20,10 @@ device_requires() {
 	_require_test_dev_size "${nvme_img_size}"
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test_device() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/036 b/tests/nvme/036
index 36ea792..ef6c29d 100755
--- a/tests/nvme/036
+++ b/tests/nvme/036
@@ -13,6 +13,10 @@ requires() {
 	_have_kernel_option NVME_TARGET_PASSTHRU
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test_device() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/037 b/tests/nvme/037
index 66d6656..ef7ac59 100755
--- a/tests/nvme/037
+++ b/tests/nvme/037
@@ -12,6 +12,10 @@ requires() {
 	_have_kernel_option NVME_TARGET_PASSTHRU
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test_device() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/038 b/tests/nvme/038
index 007b5f4..8435415 100755
--- a/tests/nvme/038
+++ b/tests/nvme/038
@@ -18,6 +18,10 @@ requires() {
 	_nvme_requires
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/040 b/tests/nvme/040
index bb9ed5e..9536f35 100755
--- a/tests/nvme/040
+++ b/tests/nvme/040
@@ -16,6 +16,10 @@ requires() {
 	_require_nvme_trtype_is_fabrics
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/041 b/tests/nvme/041
index f1fa00c..aa44f04 100755
--- a/tests/nvme/041
+++ b/tests/nvme/041
@@ -19,6 +19,9 @@ requires() {
 	_require_nvme_cli_auth
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
 
 test() {
 	echo "Running ${TEST_NAME}"
diff --git a/tests/nvme/042 b/tests/nvme/042
index a9e79c6..70c9056 100755
--- a/tests/nvme/042
+++ b/tests/nvme/042
@@ -19,6 +19,9 @@ requires() {
 	_require_nvme_cli_auth
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
 
 test() {
 	echo "Running ${TEST_NAME}"
diff --git a/tests/nvme/043 b/tests/nvme/043
index 4589423..cf99865 100755
--- a/tests/nvme/043
+++ b/tests/nvme/043
@@ -20,6 +20,9 @@ requires() {
 	_have_driver dh_generic
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
 
 test() {
 	echo "Running ${TEST_NAME}"
diff --git a/tests/nvme/044 b/tests/nvme/044
index 8b88590..9ed46c9 100755
--- a/tests/nvme/044
+++ b/tests/nvme/044
@@ -20,6 +20,9 @@ requires() {
 	_have_driver dh_generic
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
 
 test() {
 	echo "Running ${TEST_NAME}"
diff --git a/tests/nvme/045 b/tests/nvme/045
index f387ead..be81316 100755
--- a/tests/nvme/045
+++ b/tests/nvme/045
@@ -21,6 +21,9 @@ requires() {
 	_have_driver dh_generic
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
 
 test() {
 	echo "Running ${TEST_NAME}"
diff --git a/tests/nvme/047 b/tests/nvme/047
index 9bbe84d..1ab68f8 100755
--- a/tests/nvme/047
+++ b/tests/nvme/047
@@ -17,6 +17,10 @@ requires() {
 	_have_kver 4 21
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 test() {
 	echo "Running ${TEST_NAME}"
 
diff --git a/tests/nvme/048 b/tests/nvme/048
index 0b299a9..bd41fae 100755
--- a/tests/nvme/048
+++ b/tests/nvme/048
@@ -15,6 +15,10 @@ requires() {
 	_require_min_cpus 2
 }
 
+set_conditions() {
+	_set_nvme_trtype "$@"
+}
+
 nvmf_wait_for_state() {
 	local def_state_timeout=5
 	local subsys_name="$1"
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (8 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 09/15] nvme/{002-031,033-038,040-045,047,048}: support NMVET_TRTYPES Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-28  8:58   ` Sagi Grimberg
  2024-04-24  7:59 ` [PATCH blktests v3 11/15] nvme/{007,009,011,013,015,020,024}: drop duplicate nvmet blkdev type tests Shin'ichiro Kawasaki
                   ` (4 subsequent siblings)
  14 siblings, 1 reply; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

Enable repeated test runs for the listed test cases for
NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
_set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
so that the test cases are repeated for listed conditions in
NVMET_BLKDEV_TYPES and NVMET_TRTYPES.

The default values of NVMET_BLKDEV_TYPES is (device file). With this
default set up, each of the listed test cases are run twice. The second
runs of the test cases for 'file' blkdev type do exact same test as
other test cases nvme/007, 009, 011, 013, 015, 020 and 024.

Reviewed-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/nvme/006 | 2 +-
 tests/nvme/008 | 2 +-
 tests/nvme/010 | 2 +-
 tests/nvme/012 | 2 +-
 tests/nvme/014 | 2 +-
 tests/nvme/019 | 2 +-
 tests/nvme/023 | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/nvme/006 b/tests/nvme/006
index ff0a9eb..c543b40 100755
--- a/tests/nvme/006
+++ b/tests/nvme/006
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
diff --git a/tests/nvme/008 b/tests/nvme/008
index 1877d8a..b53ecdb 100755
--- a/tests/nvme/008
+++ b/tests/nvme/008
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
diff --git a/tests/nvme/010 b/tests/nvme/010
index 34914a7..0417daf 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
diff --git a/tests/nvme/012 b/tests/nvme/012
index e06bf8d..37b9056 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -20,7 +20,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
diff --git a/tests/nvme/014 b/tests/nvme/014
index ff0ebfb..bcfbc87 100755
--- a/tests/nvme/014
+++ b/tests/nvme/014
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
diff --git a/tests/nvme/019 b/tests/nvme/019
index 31020d9..fb11d41 100755
--- a/tests/nvme/019
+++ b/tests/nvme/019
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
diff --git a/tests/nvme/023 b/tests/nvme/023
index da99406..a723b73 100755
--- a/tests/nvme/023
+++ b/tests/nvme/023
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 11/15] nvme/{007,009,011,013,015,020,024}: drop duplicate nvmet blkdev type tests
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (9 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 12/15] nvme/{021,022,025,026,027,028}: do not hard code target blkdev type Shin'ichiro Kawasaki
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

From: Daniel Wagner <dwagner@suse.de>

There are various tests which only differ on the blkdev type of the
target. With the newly added feature which allows to control the target
blkdev type via the environment, these duplicate tests are not necessary
anymore and reduces the maintenance overhead.

The removed tests are covered by the other test cases nvme/006 ,008,
010, 012, 014, 019 and 023 using 'file' blkdev type.

Signed-off-by: Daniel Wagner <dwagner@suse.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/nvme/006     |  4 ++--
 tests/nvme/007     | 31 ---------------------------
 tests/nvme/007.out |  2 --
 tests/nvme/008     |  4 ++--
 tests/nvme/009     | 40 -----------------------------------
 tests/nvme/009.out |  3 ---
 tests/nvme/010     |  4 ++--
 tests/nvme/011     | 43 --------------------------------------
 tests/nvme/011.out |  3 ---
 tests/nvme/012     |  4 ++--
 tests/nvme/013     | 47 -----------------------------------------
 tests/nvme/013.out |  3 ---
 tests/nvme/014     |  4 ++--
 tests/nvme/015     | 52 ----------------------------------------------
 tests/nvme/015.out |  4 ----
 tests/nvme/019     |  4 ++--
 tests/nvme/020     | 44 ---------------------------------------
 tests/nvme/020.out |  4 ----
 tests/nvme/023     |  4 ++--
 tests/nvme/024     | 44 ---------------------------------------
 tests/nvme/024.out |  2 --
 21 files changed, 14 insertions(+), 336 deletions(-)
 delete mode 100755 tests/nvme/007
 delete mode 100644 tests/nvme/007.out
 delete mode 100755 tests/nvme/009
 delete mode 100644 tests/nvme/009.out
 delete mode 100755 tests/nvme/011
 delete mode 100644 tests/nvme/011.out
 delete mode 100755 tests/nvme/013
 delete mode 100644 tests/nvme/013.out
 delete mode 100755 tests/nvme/015
 delete mode 100644 tests/nvme/015.out
 delete mode 100755 tests/nvme/020
 delete mode 100644 tests/nvme/020.out
 delete mode 100755 tests/nvme/024
 delete mode 100644 tests/nvme/024.out

diff --git a/tests/nvme/006 b/tests/nvme/006
index c543b40..0ea679b 100755
--- a/tests/nvme/006
+++ b/tests/nvme/006
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMeOF target creation with a block device backed ns.
+# Test NVMeOF target creation.
 
 . tests/nvme/rc
 
-DESCRIPTION="create an NVMeOF target with a block device-backed ns"
+DESCRIPTION="create an NVMeOF target"
 QUICK=1
 
 requires() {
diff --git a/tests/nvme/007 b/tests/nvme/007
deleted file mode 100755
index cb2637e..0000000
--- a/tests/nvme/007
+++ /dev/null
@@ -1,31 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# Test NVMeOF target creation with a file backed ns.
-
-. tests/nvme/rc
-
-DESCRIPTION="create an NVMeOF target with a file-backed ns"
-QUICK=1
-
-requires() {
-	_nvme_requires
-	_require_nvme_trtype_is_fabrics
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	_nvmet_target_setup --blkdev file
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/007.out b/tests/nvme/007.out
deleted file mode 100644
index fdb3472..0000000
--- a/tests/nvme/007.out
+++ /dev/null
@@ -1,2 +0,0 @@
-Running nvme/007
-Test complete
diff --git a/tests/nvme/008 b/tests/nvme/008
index b53ecdb..838eb07 100755
--- a/tests/nvme/008
+++ b/tests/nvme/008
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMeOF host creation with a block device backed ns.
+# Test NVMeOF host creation.
 
 . tests/nvme/rc
 
-DESCRIPTION="create an NVMeOF host with a block device-backed ns"
+DESCRIPTION="create an NVMeOF host"
 QUICK=1
 
 requires() {
diff --git a/tests/nvme/009 b/tests/nvme/009
deleted file mode 100755
index d7b1307..0000000
--- a/tests/nvme/009
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# Test NVMeOF host creation with a file backed ns.
-
-. tests/nvme/rc
-
-DESCRIPTION="create an NVMeOF host with a file-backed ns"
-QUICK=1
-
-requires() {
-	_nvme_requires
-	_require_nvme_trtype_is_fabrics
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	local nvmedev
-
-	_nvmet_target_setup --blkdev file
-
-	_nvme_connect_subsys
-
-	nvmedev=$(_find_nvme_dev "${def_subsysnqn}")
-	_check_uuid "${nvmedev}"
-
-	_nvme_disconnect_subsys
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/009.out b/tests/nvme/009.out
deleted file mode 100644
index 4d53a8e..0000000
--- a/tests/nvme/009.out
+++ /dev/null
@@ -1,3 +0,0 @@
-Running nvme/009
-disconnected 1 controller(s)
-Test complete
diff --git a/tests/nvme/010 b/tests/nvme/010
index 0417daf..9ea2561 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# This is a data verification test for block device backed ns.
+# This is a data verification test.
 
 . tests/nvme/rc
 
-DESCRIPTION="run data verification fio job on NVMeOF block device-backed ns"
+DESCRIPTION="run data verification fio job"
 TIMED=1
 
 requires() {
diff --git a/tests/nvme/011 b/tests/nvme/011
deleted file mode 100755
index bd29129..0000000
--- a/tests/nvme/011
+++ /dev/null
@@ -1,43 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# This is a data verification test for file backed ns.
-
-. tests/nvme/rc
-
-DESCRIPTION="run data verification fio job on NVMeOF file-backed ns"
-TIMED=1
-
-requires() {
-	_nvme_requires
-	_have_fio
-	_require_nvme_trtype_is_fabrics
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	local ns
-
-	_nvmet_target_setup --blkdev file
-
-	_nvme_connect_subsys
-
-	ns=$(_find_nvme_ns "${def_subsys_uuid}")
-
-	_run_fio_verify_io --size="${nvme_img_size}" \
-		--filename="/dev/${ns}"
-
-	_nvme_disconnect_subsys
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/011.out b/tests/nvme/011.out
deleted file mode 100644
index ebbb4f7..0000000
--- a/tests/nvme/011.out
+++ /dev/null
@@ -1,3 +0,0 @@
-Running nvme/011
-disconnected 1 controller(s)
-Test complete
diff --git a/tests/nvme/012 b/tests/nvme/012
index 37b9056..d0eb487 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -2,12 +2,12 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test mkfs with data verification for block device backed ns.
+# Test mkfs with data verification.
 
 . tests/nvme/rc
 . common/xfs
 
-DESCRIPTION="run mkfs and data verification fio job on NVMeOF block device-backed ns"
+DESCRIPTION="run mkfs and data verification fio"
 TIMED=1
 
 requires() {
diff --git a/tests/nvme/013 b/tests/nvme/013
deleted file mode 100755
index 91da498..0000000
--- a/tests/nvme/013
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# Test mkfs with data verification for file backed ns.
-
-. tests/nvme/rc
-. common/xfs
-
-DESCRIPTION="run mkfs and data verification fio job on NVMeOF file-backed ns"
-TIMED=1
-
-requires() {
-	_nvme_requires
-	_have_xfs
-	_have_fio
-	_require_nvme_trtype_is_fabrics
-	_require_nvme_test_img_size 350m
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	local ns
-
-	_nvmet_target_setup --blkdev file
-
-	_nvme_connect_subsys
-
-	ns=$(_find_nvme_ns "${def_subsys_uuid}")
-
-	if ! _xfs_run_fio_verify_io "/dev/${ns}"; then
-		echo "FAIL: fio verify failed"
-	fi
-
-	_nvme_disconnect_subsys
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/013.out b/tests/nvme/013.out
deleted file mode 100644
index a727170..0000000
--- a/tests/nvme/013.out
+++ /dev/null
@@ -1,3 +0,0 @@
-Running nvme/013
-disconnected 1 controller(s)
-Test complete
diff --git a/tests/nvme/014 b/tests/nvme/014
index bcfbc87..1429180 100755
--- a/tests/nvme/014
+++ b/tests/nvme/014
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMeOF flush command from host with a block device backed ns.
+# Test NVMeOF flush command from host.
 
 . tests/nvme/rc
 
-DESCRIPTION="flush a NVMeOF block device-backed ns"
+DESCRIPTION="flush a command from host"
 QUICK=1
 
 requires() {
diff --git a/tests/nvme/015 b/tests/nvme/015
deleted file mode 100755
index b5ec10c..0000000
--- a/tests/nvme/015
+++ /dev/null
@@ -1,52 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# Test NVMeOF flush command from host with a file backed ns.
-
-. tests/nvme/rc
-
-DESCRIPTION="unit test for NVMe flush for file backed ns"
-QUICK=1
-
-requires() {
-	_nvme_requires
-	_have_loop
-	_require_nvme_trtype_is_fabrics
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	local ns
-	local size
-	local bs
-	local count
-
-	_nvmet_target_setup --blkdev file
-
-	_nvme_connect_subsys
-
-	ns=$(_find_nvme_ns "${def_subsys_uuid}")
-
-	size="$(blockdev --getsize64 "/dev/${ns}")"
-	bs="$(blockdev --getbsz "/dev/${ns}")"
-	count=$((size / bs))
-
-	dd if=/dev/urandom of="/dev/${ns}" \
-		count="${count}" bs="${bs}" status=none
-
-	nvme flush "/dev/${ns}"
-
-	_nvme_disconnect_subsys
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/015.out b/tests/nvme/015.out
deleted file mode 100644
index f854f0b..0000000
--- a/tests/nvme/015.out
+++ /dev/null
@@ -1,4 +0,0 @@
-Running nvme/015
-NVMe Flush: success
-disconnected 1 controller(s)
-Test complete
diff --git a/tests/nvme/019 b/tests/nvme/019
index fb11d41..d4cb926 100755
--- a/tests/nvme/019
+++ b/tests/nvme/019
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe DSM Discard command on NVMeOF with a block-device ns.
+# Test NVMe DSM Discard command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe DSM Discard command on NVMeOF block-device ns"
+DESCRIPTION="test NVMe DSM Discard command"
 QUICK=1
 
 requires() {
diff --git a/tests/nvme/020 b/tests/nvme/020
deleted file mode 100755
index 4993e36..0000000
--- a/tests/nvme/020
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# Test NVMe DSM Discard command on NVMeOF with a file-backed ns.
-
-. tests/nvme/rc
-
-DESCRIPTION="test NVMe DSM Discard command on NVMeOF file-backed ns"
-QUICK=1
-
-requires() {
-	_nvme_requires
-	_require_nvme_trtype_is_fabrics
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	local ns
-	local nblk_range="10,10,10,10,10,10,10,10,10,10"
-	local sblk_range="100,200,300,400,500,600,700,800,900,1000"
-
-	_nvmet_target_setup --blkdev file
-
-	_nvme_connect_subsys
-
-	ns=$(_find_nvme_ns "${def_subsys_uuid}")
-
-	nvme dsm "/dev/${ns}" --ad \
-		--slbs "${sblk_range}" --blocks "${nblk_range}"
-
-	_nvme_disconnect_subsys
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/020.out b/tests/nvme/020.out
deleted file mode 100644
index 61be280..0000000
--- a/tests/nvme/020.out
+++ /dev/null
@@ -1,4 +0,0 @@
-Running nvme/020
-NVMe DSM: success
-disconnected 1 controller(s)
-Test complete
diff --git a/tests/nvme/023 b/tests/nvme/023
index a723b73..78dfb9e 100755
--- a/tests/nvme/023
+++ b/tests/nvme/023
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe smart-log command on NVMeOF with a block-device ns.
+# Test NVMe smart-log command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe smart-log command on NVMeOF block-device ns"
+DESCRIPTION="test NVMe smart-log command"
 QUICK=1
 
 requires() {
diff --git a/tests/nvme/024 b/tests/nvme/024
deleted file mode 100755
index cab1818..0000000
--- a/tests/nvme/024
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/bin/bash
-# SPDX-License-Identifier: GPL-2.0+
-# Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
-#
-# Test NVMe smart-log command on NVMeOF with a file-backed ns.
-
-. tests/nvme/rc
-
-DESCRIPTION="test NVMe smart-log command on NVMeOF file-backed ns"
-QUICK=1
-
-requires() {
-	_nvme_requires
-	_have_loop
-	_require_nvme_trtype_is_fabrics
-}
-
-set_conditions() {
-	_set_nvme_trtype "$@"
-}
-
-test() {
-	echo "Running ${TEST_NAME}"
-
-	_setup_nvmet
-
-	local ns
-
-	_nvmet_target_setup --blkdev file
-
-	_nvme_connect_subsys
-
-	ns=$(_find_nvme_ns ${def_subsys_uuid})
-
-	if ! nvme smart-log "/dev/${ns}" >> "$FULL" 2>&1; then
-		echo "ERROR: smart-log file-ns failed"
-	fi
-
-	_nvme_disconnect_subsys >> "$FULL" 2>&1
-
-	_nvmet_target_cleanup
-
-	echo "Test complete"
-}
diff --git a/tests/nvme/024.out b/tests/nvme/024.out
deleted file mode 100644
index 76c3e29..0000000
--- a/tests/nvme/024.out
+++ /dev/null
@@ -1,2 +0,0 @@
-Running nvme/024
-Test complete
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 12/15] nvme/{021,022,025,026,027,028}: do not hard code target blkdev type
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (10 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 11/15] nvme/{007,009,011,013,015,020,024}: drop duplicate nvmet blkdev type tests Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 13/15] nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE Shin'ichiro Kawasaki
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

From: Daniel Wagner <dwagner@suse.de>

There is no need to hardcode the target blkdev type. This allows
the user to select different blkdev types via the nvmet_blkdev_type
environment variable. Also modify set_conditions() hooks to call
_set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvme_trtype(),
so that the test cases are run for all blkdev types set in
NVMET_BLKDEV_TYPES.

Signed-off-by: Daniel Wagner <dwagner@suse.de>
[Shin'ichiro: modify set_conditions()]
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/nvme/021 | 8 ++++----
 tests/nvme/022 | 8 ++++----
 tests/nvme/025 | 8 ++++----
 tests/nvme/026 | 8 ++++----
 tests/nvme/027 | 8 ++++----
 tests/nvme/028 | 8 ++++----
 6 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/tests/nvme/021 b/tests/nvme/021
index 270d90e..d7add6f 100755
--- a/tests/nvme/021
+++ b/tests/nvme/021
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe list command on NVMeOF with a file-backed ns.
+# Test NVMe list command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe list command on NVMeOF file-backed ns"
+DESCRIPTION="test NVMe list command"
 QUICK=1
 
 requires() {
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
@@ -26,7 +26,7 @@ test() {
 
 	local ns
 
-	_nvmet_target_setup --blkdev file
+	_nvmet_target_setup
 
 	_nvme_connect_subsys
 
diff --git a/tests/nvme/022 b/tests/nvme/022
index adaa765..0f2868c 100755
--- a/tests/nvme/022
+++ b/tests/nvme/022
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe reset command on NVMeOF with a file-backed ns.
+# Test NVMe reset command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe reset command on NVMeOF file-backed ns"
+DESCRIPTION="test NVMe reset command"
 QUICK=1
 
 requires() {
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
@@ -26,7 +26,7 @@ test() {
 
 	local nvmedev
 
-	_nvmet_target_setup --blkdev file
+	_nvmet_target_setup
 
 	_nvme_connect_subsys
 
diff --git a/tests/nvme/025 b/tests/nvme/025
index 224492b..a171099 100755
--- a/tests/nvme/025
+++ b/tests/nvme/025
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe effects-log command on NVMeOF with a file-backed ns.
+# Test NVMe effects-log command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe effects-log command on NVMeOF file-backed ns"
+DESCRIPTION="test NVMe effects-log"
 QUICK=1
 
 requires() {
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
@@ -26,7 +26,7 @@ test() {
 
 	local ns
 
-	_nvmet_target_setup --blkdev file
+	_nvmet_target_setup
 
 	_nvme_connect_subsys
 
diff --git a/tests/nvme/026 b/tests/nvme/026
index 6ee6a51..7196c60 100755
--- a/tests/nvme/026
+++ b/tests/nvme/026
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe ns-descs command on NVMeOF with a file-backed ns.
+# Test NVMe ns-descs command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe ns-descs command on NVMeOF file-backed ns"
+DESCRIPTION="test NVMe ns-descs"
 QUICK=1
 
 requires() {
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
@@ -26,7 +26,7 @@ test() {
 
 	local ns
 
-	_nvmet_target_setup --blkdev file
+	_nvmet_target_setup
 
 	_nvme_connect_subsys
 
diff --git a/tests/nvme/027 b/tests/nvme/027
index a63e42b..b117cc1 100755
--- a/tests/nvme/027
+++ b/tests/nvme/027
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe ns-rescan command on NVMeOF with a file-backed ns.
+# Test NVMe ns-rescan command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe ns-rescan command on NVMeOF file-backed ns"
+DESCRIPTION="test NVMe ns-rescan command"
 QUICK=1
 
 requires() {
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
@@ -26,7 +26,7 @@ test() {
 
 	local nvmedev
 
-	_nvmet_target_setup --blkdev file
+	_nvmet_target_setup
 
 	_nvme_connect_subsys
 
diff --git a/tests/nvme/028 b/tests/nvme/028
index 65c52a9..d6ffdd6 100755
--- a/tests/nvme/028
+++ b/tests/nvme/028
@@ -2,11 +2,11 @@
 # SPDX-License-Identifier: GPL-2.0+
 # Copyright (c) 2017-2018 Western Digital Corporation or its affiliates.
 #
-# Test NVMe list-subsys command on NVMeOF with a file-backed ns.
+# Test NVMe list-subsys command.
 
 . tests/nvme/rc
 
-DESCRIPTION="test NVMe list-subsys command on NVMeOF file-backed ns"
+DESCRIPTION="test NVMe list-subsys"
 QUICK=1
 
 requires() {
@@ -16,7 +16,7 @@ requires() {
 }
 
 set_conditions() {
-	_set_nvme_trtype "$@"
+	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
 }
 
 test() {
@@ -24,7 +24,7 @@ test() {
 
 	_setup_nvmet
 
-	_nvmet_target_setup --blkdev file
+	_nvmet_target_setup
 
 	_nvme_connect_subsys
 
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 13/15] nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (11 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 12/15] nvme/{021,022,025,026,027,028}: do not hard code target blkdev type Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24 12:29   ` Daniel Wagner
  2024-04-24  7:59 ` [PATCH blktests v3 14/15] nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER Shin'ichiro Kawasaki
  2024-04-24  7:59 ` [PATCH blktests v3 15/15] nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE Shin'ichiro Kawasaki
  14 siblings, 1 reply; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

To follow uppercase letter guide of environment variables, rename
nvme_img_size to NVME_IMG_SIZE.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Documentation/running-tests.md | 7 ++++---
 tests/nvme/010                 | 2 +-
 tests/nvme/017                 | 2 +-
 tests/nvme/031                 | 2 +-
 tests/nvme/034                 | 2 +-
 tests/nvme/035                 | 4 ++--
 tests/nvme/rc                  | 8 ++++----
 7 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/Documentation/running-tests.md b/Documentation/running-tests.md
index 64aff7c..736ab48 100644
--- a/Documentation/running-tests.md
+++ b/Documentation/running-tests.md
@@ -113,9 +113,10 @@ The NVMe tests can be additionally parameterized via environment variables.
   block device types can be listed with separating spaces. In this case, the
   tests are repeated to cover all of the block device types specified. Default
   value is "device file".
-- nvme_img_size: '1G' (default)
-  Run the tests with given image size in bytes. 'm', 'M', 'g'
-	and 'G' postfix are supported.
+- NVME_IMG_SIZE: '1G' (default)
+  Run the tests with given image size in bytes. 'm', 'M', 'g' and 'G' postfix
+  are supported. This parameter had an old name 'nvme_img_size'. The old name
+  is still usable, but not recommended.
 - nvme_num_iter: 1000 (default)
   The number of iterations a test should do.
 
diff --git a/tests/nvme/010 b/tests/nvme/010
index 9ea2561..d99067a 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -32,7 +32,7 @@ test() {
 
 	ns=$(_find_nvme_ns "${def_subsys_uuid}")
 
-	_run_fio_verify_io --size="${nvme_img_size}" \
+	_run_fio_verify_io --size="${NVME_IMG_SIZE}" \
 		--filename="/dev/${ns}"
 
 	_nvme_disconnect_subsys
diff --git a/tests/nvme/017 b/tests/nvme/017
index 9410cdc..4f14471 100755
--- a/tests/nvme/017
+++ b/tests/nvme/017
@@ -25,7 +25,7 @@ test() {
 	local port
 	local iterations="${nvme_num_iter}"
 
-	truncate -s "${nvme_img_size}" "$(_nvme_def_file_path)"
+	truncate -s "${NVME_IMG_SIZE}" "$(_nvme_def_file_path)"
 
 	local genctr=1
 
diff --git a/tests/nvme/031 b/tests/nvme/031
index b98630a..00d3d18 100755
--- a/tests/nvme/031
+++ b/tests/nvme/031
@@ -37,7 +37,7 @@ test() {
 	local loop_dev
 	local port
 
-	truncate -s "${nvme_img_size}" "$(_nvme_def_file_path)"
+	truncate -s "${NVME_IMG_SIZE}" "$(_nvme_def_file_path)"
 
 	loop_dev="$(losetup -f --show "$(_nvme_def_file_path)")"
 
diff --git a/tests/nvme/034 b/tests/nvme/034
index 522ffe3..239757c 100755
--- a/tests/nvme/034
+++ b/tests/nvme/034
@@ -29,7 +29,7 @@ test_device() {
 	_nvmet_passthru_target_setup
 	nsdev=$(_nvmet_passthru_target_connect)
 
-	_run_fio_verify_io --size="${nvme_img_size}" --filename="${nsdev}"
+	_run_fio_verify_io --size="${NVME_IMG_SIZE}" --filename="${nsdev}"
 
 	_nvme_disconnect_subsys
 	_nvmet_passthru_target_cleanup
diff --git a/tests/nvme/035 b/tests/nvme/035
index cfca5fd..8286178 100755
--- a/tests/nvme/035
+++ b/tests/nvme/035
@@ -17,7 +17,7 @@ requires() {
 }
 
 device_requires() {
-	_require_test_dev_size "${nvme_img_size}"
+	_require_test_dev_size "${NVME_IMG_SIZE}"
 }
 
 set_conditions() {
@@ -35,7 +35,7 @@ test_device() {
 	_nvmet_passthru_target_setup
 	nsdev=$(_nvmet_passthru_target_connect)
 
-	if ! _xfs_run_fio_verify_io "${nsdev}" "${nvme_img_size}"; then
+	if ! _xfs_run_fio_verify_io "${nsdev}" "${NVME_IMG_SIZE}"; then
 		echo "FAIL: fio verify failed"
 	fi
 
diff --git a/tests/nvme/rc b/tests/nvme/rc
index a31690d..c018f7f 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -19,7 +19,7 @@ def_hostnqn="nqn.2014-08.org.nvmexpress:uuid:${def_hostid}"
 export def_subsysnqn="blktests-subsystem-1"
 export def_subsys_uuid="91fdba0d-f87b-4c25-b80f-db7be1418b9e"
 _check_conflict_and_set_default NVMET_TRTYPES nvme_trtype "loop"
-nvme_img_size=${nvme_img_size:-"1G"}
+_check_conflict_and_set_default NVME_IMG_SIZE nvme_img_size 1G
 nvme_num_iter=${nvme_num_iter:-"1000"}
 nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
 NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
@@ -165,10 +165,10 @@ _require_nvme_test_img_size() {
 	local nvme_img_size_mb
 
 	require_sz_mb="$(convert_to_mb "$1")"
-	nvme_img_size_mb="$(convert_to_mb "${nvme_img_size}")"
+	nvme_img_size_mb="$(convert_to_mb "${NVME_IMG_SIZE}")"
 
 	if ((nvme_img_size_mb < require_sz_mb)); then
-		SKIP_REASONS+=("nvme_img_size must be at least ${require_sz_mb}m")
+		SKIP_REASONS+=("NVME_IMG_SIZE must be at least ${require_sz_mb}m")
 		return 1
 	fi
 	return 0
@@ -915,7 +915,7 @@ _nvmet_target_setup() {
 		esac
 	done
 
-	truncate -s "${nvme_img_size}" "$(_nvme_def_file_path)"
+	truncate -s "${NVME_IMG_SIZE}" "$(_nvme_def_file_path)"
 	if [[ "${blkdev_type}" == "device" ]]; then
 		blkdev="$(losetup -f --show "$(_nvme_def_file_path)")"
 	else
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 14/15] nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (12 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 13/15] nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24 12:30   ` Daniel Wagner
  2024-04-24  7:59 ` [PATCH blktests v3 15/15] nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE Shin'ichiro Kawasaki
  14 siblings, 1 reply; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

To follow uppercase letter guide of environment variables, rename
nvme_num_iter to NVME_NUM_ITER.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Documentation/running-tests.md | 5 +++--
 tests/nvme/016                 | 2 +-
 tests/nvme/017                 | 2 +-
 tests/nvme/rc                  | 2 +-
 4 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/Documentation/running-tests.md b/Documentation/running-tests.md
index 736ab48..7bd0885 100644
--- a/Documentation/running-tests.md
+++ b/Documentation/running-tests.md
@@ -117,8 +117,9 @@ The NVMe tests can be additionally parameterized via environment variables.
   Run the tests with given image size in bytes. 'm', 'M', 'g' and 'G' postfix
   are supported. This parameter had an old name 'nvme_img_size'. The old name
   is still usable, but not recommended.
-- nvme_num_iter: 1000 (default)
-  The number of iterations a test should do.
+- NVME_NUM_ITER: 1000 (default)
+  The number of iterations a test should do. This parameter had an old name
+  'nvme_num_iter'. The old name is still usable, but not recommended.
 
 ### Running nvme-rdma and SRP tests
 
diff --git a/tests/nvme/016 b/tests/nvme/016
index a65cffd..d1fdb35 100755
--- a/tests/nvme/016
+++ b/tests/nvme/016
@@ -23,7 +23,7 @@ test() {
 	_setup_nvmet
 
 	local port
-	local iterations="${nvme_num_iter}"
+	local iterations="${NVME_NUM_ITER}"
 	local loop_dev
 
 	loop_dev="$(losetup -f)"
diff --git a/tests/nvme/017 b/tests/nvme/017
index 4f14471..114be60 100755
--- a/tests/nvme/017
+++ b/tests/nvme/017
@@ -23,7 +23,7 @@ test() {
 	_setup_nvmet
 
 	local port
-	local iterations="${nvme_num_iter}"
+	local iterations="${NVME_NUM_ITER}"
 
 	truncate -s "${NVME_IMG_SIZE}" "$(_nvme_def_file_path)"
 
diff --git a/tests/nvme/rc b/tests/nvme/rc
index c018f7f..8762a56 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -20,7 +20,7 @@ export def_subsysnqn="blktests-subsystem-1"
 export def_subsys_uuid="91fdba0d-f87b-4c25-b80f-db7be1418b9e"
 _check_conflict_and_set_default NVMET_TRTYPES nvme_trtype "loop"
 _check_conflict_and_set_default NVME_IMG_SIZE nvme_img_size 1G
-nvme_num_iter=${nvme_num_iter:-"1000"}
+_check_conflict_and_set_default NVME_NUM_ITER nvme_num_iter 1000
 nvmet_blkdev_type=${nvmet_blkdev_type:-"device"}
 NVMET_BLKDEV_TYPES=${NVMET_BLKDEV_TYPES:-"device file"}
 
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH blktests v3 15/15] nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE
  2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
                   ` (13 preceding siblings ...)
  2024-04-24  7:59 ` [PATCH blktests v3 14/15] nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER Shin'ichiro Kawasaki
@ 2024-04-24  7:59 ` Shin'ichiro Kawasaki
  2024-04-24 12:33   ` Daniel Wagner
  14 siblings, 1 reply; 28+ messages in thread
From: Shin'ichiro Kawasaki @ 2024-04-24  7:59 UTC (permalink / raw)
  To: linux-block; +Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni, Sagi Grimberg

To follow uppercase letter guide of environment variables, rename
use_rxe to USE_RXE.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Documentation/running-tests.md | 6 ++++--
 common/multipath-over-rdma     | 5 +++--
 tests/nvme/rc                  | 2 +-
 tests/srp/rc                   | 2 +-
 4 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/Documentation/running-tests.md b/Documentation/running-tests.md
index 7bd0885..968702e 100644
--- a/Documentation/running-tests.md
+++ b/Documentation/running-tests.md
@@ -132,9 +132,11 @@ NVMET_TRTYPES=rdma ./check nvme/
 ./check srp/
 
 To use the rdma_rxe driver:
-use_rxe=1 NVMET_TRTYPES=rdma ./check nvme/
-use_rxe=1 ./check srp/
+USE_RXE=1 NVMET_TRTYPES=rdma ./check nvme/
+USE_RXE=1 ./check srp/
 ```
+'USE_RXE' had the old name 'use_rxe'. The old name is still usable but not
+recommended.
 
 ### Normal user
 
diff --git a/common/multipath-over-rdma b/common/multipath-over-rdma
index ee05100..d0f4d26 100644
--- a/common/multipath-over-rdma
+++ b/common/multipath-over-rdma
@@ -12,12 +12,13 @@ filesystem_type=ext4
 fio_aux_path=/tmp/fio-state-files
 memtotal=$(sed -n 's/^MemTotal:[[:blank:]]*\([0-9]*\)[[:blank:]]*kB$/\1/p' /proc/meminfo)
 max_ramdisk_size=$((1<<25))
-use_rxe=${use_rxe:-""}
 ramdisk_size=$((memtotal*(1024/16)))  # in bytes
 if [ $ramdisk_size -gt $max_ramdisk_size ]; then
 	ramdisk_size=$max_ramdisk_size
 fi
 
+_check_conflict_and_set_default USE_RXE use_rxe ""
+
 _have_legacy_dm() {
 	_have_kernel_config_file || return
 	if ! _check_kernel_option DM_MQ_DEFAULT; then
@@ -396,7 +397,7 @@ start_soft_rdma() {
 	local type
 
 	{
-	if [ -z "$use_rxe" ]; then
+	if [ -z "$USE_RXE" ]; then
 		modprobe siw || return $?
 		type=siw
 	else
diff --git a/tests/nvme/rc b/tests/nvme/rc
index 8762a56..24eafd2 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -103,7 +103,7 @@ _nvme_requires() {
 		_have_driver nvmet-rdma
 		_have_configfs
 		_have_program rdma
-		if [ -n "$use_rxe" ]; then
+		if [ -n "$USE_RXE" ]; then
 			_have_driver rdma_rxe
 		else
 			_have_driver siw
diff --git a/tests/srp/rc b/tests/srp/rc
index c77ef6c..85bd1dd 100755
--- a/tests/srp/rc
+++ b/tests/srp/rc
@@ -51,7 +51,7 @@ group_requires() {
 	_have_module ib_uverbs
 	_have_module null_blk
 	_have_module rdma_cm
-	if [ -n "$use_rxe" ]; then
+	if [ -n "$USE_RXE" ]; then
 		_have_module rdma_rxe
 	else
 		_have_module siw
-- 
2.44.0



^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 05/15] common/rc: introduce _check_conflict_and_set_default()
  2024-04-24  7:59 ` [PATCH blktests v3 05/15] common/rc: introduce _check_conflict_and_set_default() Shin'ichiro Kawasaki
@ 2024-04-24 12:27   ` Daniel Wagner
  0 siblings, 0 replies; 28+ messages in thread
From: Daniel Wagner @ 2024-04-24 12:27 UTC (permalink / raw)
  To: Shin'ichiro Kawasaki
  Cc: linux-block, linux-nvme, Chaitanya Kulkarni, Sagi Grimberg

On Wed, Apr 24, 2024 at 04:59:45PM +0900, Shin'ichiro Kawasaki wrote:
> Following commits are going to rename some config option parameters from
> lowercase letters to uppercase. The old lowercase options will be
> deprecated but still be kept usable to not cause confusions. When these
> changes are made, it will be required to check that both new and old
> parameters are not set at once and ensure they do not have two different
> values.
> 
> To simplify the code to check the two parameters, introduce the helper
> _check_conflict_and_set_default(). If the both two parameters are
> set, it errors out. If the old option is set, it propagates the old
> option value to the new option. Also, when neither of them is set, it
> sets the default value to the new option.
> 
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Reviewed-by: Daniel Wagner <dwagner@suse.de>


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 06/15] nvme/rc: introduce NVMET_TRTYPES
  2024-04-24  7:59 ` [PATCH blktests v3 06/15] nvme/rc: introduce NVMET_TRTYPES Shin'ichiro Kawasaki
@ 2024-04-24 12:28   ` Daniel Wagner
  0 siblings, 0 replies; 28+ messages in thread
From: Daniel Wagner @ 2024-04-24 12:28 UTC (permalink / raw)
  To: Shin'ichiro Kawasaki
  Cc: linux-block, linux-nvme, Chaitanya Kulkarni, Sagi Grimberg

On Wed, Apr 24, 2024 at 04:59:46PM +0900, Shin'ichiro Kawasaki wrote:
> Some of the test cases in nvme test group can be run under various nvme
> target transport types. The configuration parameter nvme_trtype
> specifies the transport to use. But this configuration method has two
> drawbacks. Firstly, the blktests check script needs to be invoked
> multiple times to cover multiple transport types. Secondly, the test
> cases irrelevant to the transport types are executed exactly same
> conditions in the multiple blktests runs.
> 
> To avoid the drawbacks, allow setting multiple transport types. Taking
> this chance, rename the parameter from nvme_trtype to NVMET_TRTYPES to
> follow the uppercase letter naming guide for environment variables.
> NVMET_TRTYPES can take multiple transport types like:
> 
>     NVMET_TRTYPES="loop tcp"
> 
> Introduce _nvmet_set_nvme_trtype() which can be called from the
> set_conditions() hook of the transport type dependent test cases.
> Blktests will repeat the test case as many as the number of elements in
> NVMET_TRTYPES, and set nvme_trtype for each test case run.
> 
> Also introduce _NVMET_TRTYPES_is_valid() to check NVMET_TRTYPES value
> before test run.
> 
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Reviewed-by: Daniel Wagner <dwagner@suse.de>


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 13/15] nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE
  2024-04-24  7:59 ` [PATCH blktests v3 13/15] nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE Shin'ichiro Kawasaki
@ 2024-04-24 12:29   ` Daniel Wagner
  0 siblings, 0 replies; 28+ messages in thread
From: Daniel Wagner @ 2024-04-24 12:29 UTC (permalink / raw)
  To: Shin'ichiro Kawasaki
  Cc: linux-block, linux-nvme, Chaitanya Kulkarni, Sagi Grimberg

On Wed, Apr 24, 2024 at 04:59:53PM +0900, Shin'ichiro Kawasaki wrote:
> To follow uppercase letter guide of environment variables, rename
> nvme_img_size to NVME_IMG_SIZE.
> 
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Reviewed-by: Daniel Wagner <dwagner@suse.de>


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 14/15] nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER
  2024-04-24  7:59 ` [PATCH blktests v3 14/15] nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER Shin'ichiro Kawasaki
@ 2024-04-24 12:30   ` Daniel Wagner
  0 siblings, 0 replies; 28+ messages in thread
From: Daniel Wagner @ 2024-04-24 12:30 UTC (permalink / raw)
  To: Shin'ichiro Kawasaki
  Cc: linux-block, linux-nvme, Chaitanya Kulkarni, Sagi Grimberg

On Wed, Apr 24, 2024 at 04:59:54PM +0900, Shin'ichiro Kawasaki wrote:
> To follow uppercase letter guide of environment variables, rename
> nvme_num_iter to NVME_NUM_ITER.
> 
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Reviewed-by: Daniel Wagner <dwagner@suse.de>


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 15/15] nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE
  2024-04-24  7:59 ` [PATCH blktests v3 15/15] nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE Shin'ichiro Kawasaki
@ 2024-04-24 12:33   ` Daniel Wagner
  0 siblings, 0 replies; 28+ messages in thread
From: Daniel Wagner @ 2024-04-24 12:33 UTC (permalink / raw)
  To: Shin'ichiro Kawasaki
  Cc: linux-block, linux-nvme, Chaitanya Kulkarni, Sagi Grimberg

On Wed, Apr 24, 2024 at 04:59:55PM +0900, Shin'ichiro Kawasaki wrote:
> To follow uppercase letter guide of environment variables, rename
> use_rxe to USE_RXE.
> 
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Reviewed-by: Daniel Wagner <dwagner@suse.de>

>  	{
> -	if [ -z "$use_rxe" ]; then
> +	if [ -z "$USE_RXE" ]; then

Maybe use the opportunity to use switch to "${USE_RXE}" form of the
variable access when you are touching these lines?


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-24  7:59 ` [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
@ 2024-04-28  8:58   ` Sagi Grimberg
  2024-04-28 10:32     ` Shinichiro Kawasaki
  0 siblings, 1 reply; 28+ messages in thread
From: Sagi Grimberg @ 2024-04-28  8:58 UTC (permalink / raw)
  To: Shin'ichiro Kawasaki, linux-block
  Cc: linux-nvme, Daniel Wagner, Chaitanya Kulkarni



On 24/04/2024 10:59, Shin'ichiro Kawasaki wrote:
> Enable repeated test runs for the listed test cases for
> NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
> _set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
> so that the test cases are repeated for listed conditions in
> NVMET_BLKDEV_TYPES and NVMET_TRTYPES.
>
> The default values of NVMET_BLKDEV_TYPES is (device file). With this
> default set up, each of the listed test cases are run twice. The second
> runs of the test cases for 'file' blkdev type do exact same test as
> other test cases nvme/007, 009, 011, 013, 015, 020 and 024.
>
> Reviewed-by: Daniel Wagner <dwagner@suse.de>
> Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
> Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
> ---
>   tests/nvme/006 | 2 +-
>   tests/nvme/008 | 2 +-
>   tests/nvme/010 | 2 +-
>   tests/nvme/012 | 2 +-
>   tests/nvme/014 | 2 +-
>   tests/nvme/019 | 2 +-
>   tests/nvme/023 | 2 +-
>   7 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/tests/nvme/006 b/tests/nvme/006
> index ff0a9eb..c543b40 100755
> --- a/tests/nvme/006
> +++ b/tests/nvme/006
> @@ -16,7 +16,7 @@ requires() {
>   }
>   
>   set_conditions() {
> -	_set_nvme_trtype "$@"
> +	_set_nvme_trtype_and_nvmet_blkdev_type "$@"

Why not calling separate functions? having func do_a_and_b interface is 
not great.


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-28  8:58   ` Sagi Grimberg
@ 2024-04-28 10:32     ` Shinichiro Kawasaki
  2024-04-28 13:12       ` Sagi Grimberg
  0 siblings, 1 reply; 28+ messages in thread
From: Shinichiro Kawasaki @ 2024-04-28 10:32 UTC (permalink / raw)
  To: Sagi Grimberg; +Cc: linux-block, linux-nvme, Daniel Wagner, Chaitanya Kulkarni

On Apr 28, 2024 / 11:58, Sagi Grimberg wrote:
> 
> 
> On 24/04/2024 10:59, Shin'ichiro Kawasaki wrote:
> > Enable repeated test runs for the listed test cases for
> > NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
> > _set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
> > so that the test cases are repeated for listed conditions in
> > NVMET_BLKDEV_TYPES and NVMET_TRTYPES.
> > 
> > The default values of NVMET_BLKDEV_TYPES is (device file). With this
> > default set up, each of the listed test cases are run twice. The second
> > runs of the test cases for 'file' blkdev type do exact same test as
> > other test cases nvme/007, 009, 011, 013, 015, 020 and 024.
> > 
> > Reviewed-by: Daniel Wagner <dwagner@suse.de>
> > Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
> > Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
> > Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
> > ---
> >   tests/nvme/006 | 2 +-
> >   tests/nvme/008 | 2 +-
> >   tests/nvme/010 | 2 +-
> >   tests/nvme/012 | 2 +-
> >   tests/nvme/014 | 2 +-
> >   tests/nvme/019 | 2 +-
> >   tests/nvme/023 | 2 +-
> >   7 files changed, 7 insertions(+), 7 deletions(-)
> > 
> > diff --git a/tests/nvme/006 b/tests/nvme/006
> > index ff0a9eb..c543b40 100755
> > --- a/tests/nvme/006
> > +++ b/tests/nvme/006
> > @@ -16,7 +16,7 @@ requires() {
> >   }
> >   set_conditions() {
> > -	_set_nvme_trtype "$@"
> > +	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
> 
> Why not calling separate functions? having func do_a_and_b interface is not
> great.

In this case, we want to repeat the test cases to cover combination of two
conditions: M trtypes and N blkdev_types. The test case should be repeated to
cover all of M x N matrix elements, then the hook set_conditions() should
iterate the elements. I can not think of the way to handle this iteration with
separated two functions.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-28 10:32     ` Shinichiro Kawasaki
@ 2024-04-28 13:12       ` Sagi Grimberg
  2024-04-28 23:58         ` Shinichiro Kawasaki
  0 siblings, 1 reply; 28+ messages in thread
From: Sagi Grimberg @ 2024-04-28 13:12 UTC (permalink / raw)
  To: Shinichiro Kawasaki
  Cc: linux-block, linux-nvme, Daniel Wagner, Chaitanya Kulkarni



On 28/04/2024 13:32, Shinichiro Kawasaki wrote:
> On Apr 28, 2024 / 11:58, Sagi Grimberg wrote:
>>
>> On 24/04/2024 10:59, Shin'ichiro Kawasaki wrote:
>>> Enable repeated test runs for the listed test cases for
>>> NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
>>> _set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
>>> so that the test cases are repeated for listed conditions in
>>> NVMET_BLKDEV_TYPES and NVMET_TRTYPES.
>>>
>>> The default values of NVMET_BLKDEV_TYPES is (device file). With this
>>> default set up, each of the listed test cases are run twice. The second
>>> runs of the test cases for 'file' blkdev type do exact same test as
>>> other test cases nvme/007, 009, 011, 013, 015, 020 and 024.
>>>
>>> Reviewed-by: Daniel Wagner <dwagner@suse.de>
>>> Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
>>> Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
>>> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
>>> ---
>>>    tests/nvme/006 | 2 +-
>>>    tests/nvme/008 | 2 +-
>>>    tests/nvme/010 | 2 +-
>>>    tests/nvme/012 | 2 +-
>>>    tests/nvme/014 | 2 +-
>>>    tests/nvme/019 | 2 +-
>>>    tests/nvme/023 | 2 +-
>>>    7 files changed, 7 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/tests/nvme/006 b/tests/nvme/006
>>> index ff0a9eb..c543b40 100755
>>> --- a/tests/nvme/006
>>> +++ b/tests/nvme/006
>>> @@ -16,7 +16,7 @@ requires() {
>>>    }
>>>    set_conditions() {
>>> -	_set_nvme_trtype "$@"
>>> +	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
>> Why not calling separate functions? having func do_a_and_b interface is not
>> great.
> In this case, we want to repeat the test cases to cover combination of two
> conditions: M trtypes and N blkdev_types. The test case should be repeated to
> cover all of M x N matrix elements, then the hook set_conditions() should
> iterate the elements. I can not think of the way to handle this iteration with
> separated two functions.

What happens when you add another condition to iterate against, you 
introduce set_a_and_b_and_c interface?


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-28 13:12       ` Sagi Grimberg
@ 2024-04-28 23:58         ` Shinichiro Kawasaki
  2024-04-30 12:00           ` Sagi Grimberg
  0 siblings, 1 reply; 28+ messages in thread
From: Shinichiro Kawasaki @ 2024-04-28 23:58 UTC (permalink / raw)
  To: Sagi Grimberg; +Cc: linux-block, linux-nvme, Daniel Wagner, Chaitanya Kulkarni

On Apr 28, 2024 / 16:12, Sagi Grimberg wrote:
> 
> 
> On 28/04/2024 13:32, Shinichiro Kawasaki wrote:
> > On Apr 28, 2024 / 11:58, Sagi Grimberg wrote:
> > > 
> > > On 24/04/2024 10:59, Shin'ichiro Kawasaki wrote:
> > > > Enable repeated test runs for the listed test cases for
> > > > NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
> > > > _set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
> > > > so that the test cases are repeated for listed conditions in
> > > > NVMET_BLKDEV_TYPES and NVMET_TRTYPES.
> > > > 
> > > > The default values of NVMET_BLKDEV_TYPES is (device file). With this
> > > > default set up, each of the listed test cases are run twice. The second
> > > > runs of the test cases for 'file' blkdev type do exact same test as
> > > > other test cases nvme/007, 009, 011, 013, 015, 020 and 024.
> > > > 
> > > > Reviewed-by: Daniel Wagner <dwagner@suse.de>
> > > > Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
> > > > Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
> > > > Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
> > > > ---
> > > >    tests/nvme/006 | 2 +-
> > > >    tests/nvme/008 | 2 +-
> > > >    tests/nvme/010 | 2 +-
> > > >    tests/nvme/012 | 2 +-
> > > >    tests/nvme/014 | 2 +-
> > > >    tests/nvme/019 | 2 +-
> > > >    tests/nvme/023 | 2 +-
> > > >    7 files changed, 7 insertions(+), 7 deletions(-)
> > > > 
> > > > diff --git a/tests/nvme/006 b/tests/nvme/006
> > > > index ff0a9eb..c543b40 100755
> > > > --- a/tests/nvme/006
> > > > +++ b/tests/nvme/006
> > > > @@ -16,7 +16,7 @@ requires() {
> > > >    }
> > > >    set_conditions() {
> > > > -	_set_nvme_trtype "$@"
> > > > +	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
> > > Why not calling separate functions? having func do_a_and_b interface is not
> > > great.
> > In this case, we want to repeat the test cases to cover combination of two
> > conditions: M trtypes and N blkdev_types. The test case should be repeated to
> > cover all of M x N matrix elements, then the hook set_conditions() should
> > iterate the elements. I can not think of the way to handle this iteration with
> > separated two functions.
> 
> What happens when you add another condition to iterate against, you
> introduce set_a_and_b_and_c interface?

That is my current intent.

Another question is how it is likely to have more conditions to add on. I guess
such many, multiplied conditions will result in combination explosion and long
test runtime, so I'm not sure how much it will be useful.

Do we have potential candidates of the third or fourth conditions?

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-28 23:58         ` Shinichiro Kawasaki
@ 2024-04-30 12:00           ` Sagi Grimberg
  2024-05-03  6:23             ` Shinichiro Kawasaki
  0 siblings, 1 reply; 28+ messages in thread
From: Sagi Grimberg @ 2024-04-30 12:00 UTC (permalink / raw)
  To: Shinichiro Kawasaki
  Cc: linux-block, linux-nvme, Daniel Wagner, Chaitanya Kulkarni



On 29/04/2024 2:58, Shinichiro Kawasaki wrote:
> On Apr 28, 2024 / 16:12, Sagi Grimberg wrote:
>>
>> On 28/04/2024 13:32, Shinichiro Kawasaki wrote:
>>> On Apr 28, 2024 / 11:58, Sagi Grimberg wrote:
>>>> On 24/04/2024 10:59, Shin'ichiro Kawasaki wrote:
>>>>> Enable repeated test runs for the listed test cases for
>>>>> NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
>>>>> _set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
>>>>> so that the test cases are repeated for listed conditions in
>>>>> NVMET_BLKDEV_TYPES and NVMET_TRTYPES.
>>>>>
>>>>> The default values of NVMET_BLKDEV_TYPES is (device file). With this
>>>>> default set up, each of the listed test cases are run twice. The second
>>>>> runs of the test cases for 'file' blkdev type do exact same test as
>>>>> other test cases nvme/007, 009, 011, 013, 015, 020 and 024.
>>>>>
>>>>> Reviewed-by: Daniel Wagner <dwagner@suse.de>
>>>>> Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
>>>>> Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
>>>>> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
>>>>> ---
>>>>>     tests/nvme/006 | 2 +-
>>>>>     tests/nvme/008 | 2 +-
>>>>>     tests/nvme/010 | 2 +-
>>>>>     tests/nvme/012 | 2 +-
>>>>>     tests/nvme/014 | 2 +-
>>>>>     tests/nvme/019 | 2 +-
>>>>>     tests/nvme/023 | 2 +-
>>>>>     7 files changed, 7 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/tests/nvme/006 b/tests/nvme/006
>>>>> index ff0a9eb..c543b40 100755
>>>>> --- a/tests/nvme/006
>>>>> +++ b/tests/nvme/006
>>>>> @@ -16,7 +16,7 @@ requires() {
>>>>>     }
>>>>>     set_conditions() {
>>>>> -	_set_nvme_trtype "$@"
>>>>> +	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
>>>> Why not calling separate functions? having func do_a_and_b interface is not
>>>> great.
>>> In this case, we want to repeat the test cases to cover combination of two
>>> conditions: M trtypes and N blkdev_types. The test case should be repeated to
>>> cover all of M x N matrix elements, then the hook set_conditions() should
>>> iterate the elements. I can not think of the way to handle this iteration with
>>> separated two functions.
>> What happens when you add another condition to iterate against, you
>> introduce set_a_and_b_and_c interface?
> That is my current intent.

I don't think its very maintainable.

>
> Another question is how it is likely to have more conditions to add on.

I expect that people will want to add more flavors moving forward. For 
example
ADDRESS_FAMILIES="ipv4 ipv6" RDMA_TRANSPORT="siw rxe" and possibly other
features that can grow in the future.


>   I guess
> such many, multiplied conditions will result in combination explosion and long
> test runtime, so I'm not sure how much it will be useful.

I think that running multiple flavors of a test suite is a capability 
that is bound to be
reused as more test flavors emerge. But that may be just my opinion.

>
> Do we have potential candidates of the third or fourth conditions?

Yes, see above.


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-04-30 12:00           ` Sagi Grimberg
@ 2024-05-03  6:23             ` Shinichiro Kawasaki
  2024-05-03  7:33               ` Sagi Grimberg
  0 siblings, 1 reply; 28+ messages in thread
From: Shinichiro Kawasaki @ 2024-05-03  6:23 UTC (permalink / raw)
  To: Sagi Grimberg; +Cc: linux-block, linux-nvme, Daniel Wagner, Chaitanya Kulkarni

On Apr 30, 2024 / 15:00, Sagi Grimberg wrote:
> 
> 
> On 29/04/2024 2:58, Shinichiro Kawasaki wrote:
> > On Apr 28, 2024 / 16:12, Sagi Grimberg wrote:
> > > 
> > > On 28/04/2024 13:32, Shinichiro Kawasaki wrote:
> > > > On Apr 28, 2024 / 11:58, Sagi Grimberg wrote:
> > > > > On 24/04/2024 10:59, Shin'ichiro Kawasaki wrote:
> > > > > > Enable repeated test runs for the listed test cases for
> > > > > > NVMET_BLKDEV_TYPES. Modify the set_conditions() hooks to call
> > > > > > _set_nvme_trtype_and_nvmet_blkdev_type() instead of _set_nvmet_trtype()
> > > > > > so that the test cases are repeated for listed conditions in
> > > > > > NVMET_BLKDEV_TYPES and NVMET_TRTYPES.
> > > > > > 
> > > > > > The default values of NVMET_BLKDEV_TYPES is (device file). With this
> > > > > > default set up, each of the listed test cases are run twice. The second
> > > > > > runs of the test cases for 'file' blkdev type do exact same test as
> > > > > > other test cases nvme/007, 009, 011, 013, 015, 020 and 024.
> > > > > > 
> > > > > > Reviewed-by: Daniel Wagner <dwagner@suse.de>
> > > > > > Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
> > > > > > Acked-by: Nitesh Shetty <nj.shetty@samsung.com>
> > > > > > Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
> > > > > > ---
> > > > > >     tests/nvme/006 | 2 +-
> > > > > >     tests/nvme/008 | 2 +-
> > > > > >     tests/nvme/010 | 2 +-
> > > > > >     tests/nvme/012 | 2 +-
> > > > > >     tests/nvme/014 | 2 +-
> > > > > >     tests/nvme/019 | 2 +-
> > > > > >     tests/nvme/023 | 2 +-
> > > > > >     7 files changed, 7 insertions(+), 7 deletions(-)
> > > > > > 
> > > > > > diff --git a/tests/nvme/006 b/tests/nvme/006
> > > > > > index ff0a9eb..c543b40 100755
> > > > > > --- a/tests/nvme/006
> > > > > > +++ b/tests/nvme/006
> > > > > > @@ -16,7 +16,7 @@ requires() {
> > > > > >     }
> > > > > >     set_conditions() {
> > > > > > -	_set_nvme_trtype "$@"
> > > > > > +	_set_nvme_trtype_and_nvmet_blkdev_type "$@"
> > > > > Why not calling separate functions? having func do_a_and_b interface is not
> > > > > great.
> > > > In this case, we want to repeat the test cases to cover combination of two
> > > > conditions: M trtypes and N blkdev_types. The test case should be repeated to
> > > > cover all of M x N matrix elements, then the hook set_conditions() should
> > > > iterate the elements. I can not think of the way to handle this iteration with
> > > > separated two functions.
> > > What happens when you add another condition to iterate against, you
> > > introduce set_a_and_b_and_c interface?
> > That is my current intent.
> 
> I don't think its very maintainable.
> 
> > 
> > Another question is how it is likely to have more conditions to add on.
> 
> I expect that people will want to add more flavors moving forward. For
> example
> ADDRESS_FAMILIES="ipv4 ipv6" RDMA_TRANSPORT="siw rxe" and possibly other
> features that can grow in the future.
> 
> 
> >   I guess
> > such many, multiplied conditions will result in combination explosion and long
> > test runtime, so I'm not sure how much it will be useful.
> 
> I think that running multiple flavors of a test suite is a capability that
> is bound to be
> reused as more test flavors emerge. But that may be just my opinion.
> 
> > 
> > Do we have potential candidates of the third or fourth conditions?
> 
> Yes, see above.

Okay, the candidates look useful in the future. Fortunately, I came up with an
idea to make up the condition matrix from multiple set_conditions() hooks. Will
try to implement it and respin the series.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES
  2024-05-03  6:23             ` Shinichiro Kawasaki
@ 2024-05-03  7:33               ` Sagi Grimberg
  0 siblings, 0 replies; 28+ messages in thread
From: Sagi Grimberg @ 2024-05-03  7:33 UTC (permalink / raw)
  To: Shinichiro Kawasaki
  Cc: linux-block, linux-nvme, Daniel Wagner, Chaitanya Kulkarni

>>> Another question is how it is likely to have more conditions to add on.
>> I expect that people will want to add more flavors moving forward. For
>> example
>> ADDRESS_FAMILIES="ipv4 ipv6" RDMA_TRANSPORT="siw rxe" and possibly other
>> features that can grow in the future.
>>
>>
>>>    I guess
>>> such many, multiplied conditions will result in combination explosion and long
>>> test runtime, so I'm not sure how much it will be useful.
>> I think that running multiple flavors of a test suite is a capability that
>> is bound to be
>> reused as more test flavors emerge. But that may be just my opinion.
>>
>>> Do we have potential candidates of the third or fourth conditions?
>> Yes, see above.
> Okay, the candidates look useful in the future. Fortunately, I came up with an
> idea to make up the condition matrix from multiple set_conditions() hooks. Will
> try to implement it and respin the series.

Cool.


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2024-05-03  7:33 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-24  7:59 [PATCH blktests v3 00/15] support test case repeat by different conditions Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 01/15] check: factor out _run_test() Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 02/15] check: support test case repeat by different conditions Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 03/15] check: use set_conditions() for the CAN_BE_ZONED test cases Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 04/15] meta/{016,017}: add test cases to check repeated test case runs Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 05/15] common/rc: introduce _check_conflict_and_set_default() Shin'ichiro Kawasaki
2024-04-24 12:27   ` Daniel Wagner
2024-04-24  7:59 ` [PATCH blktests v3 06/15] nvme/rc: introduce NVMET_TRTYPES Shin'ichiro Kawasaki
2024-04-24 12:28   ` Daniel Wagner
2024-04-24  7:59 ` [PATCH blktests v3 07/15] nvme/rc: add blkdev type environment variable Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 08/15] nvme/rc: introduce NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 09/15] nvme/{002-031,033-038,040-045,047,048}: support NMVET_TRTYPES Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 10/15] nvme/{006,008,010,012,014,019,023}: support NVMET_BLKDEV_TYPES Shin'ichiro Kawasaki
2024-04-28  8:58   ` Sagi Grimberg
2024-04-28 10:32     ` Shinichiro Kawasaki
2024-04-28 13:12       ` Sagi Grimberg
2024-04-28 23:58         ` Shinichiro Kawasaki
2024-04-30 12:00           ` Sagi Grimberg
2024-05-03  6:23             ` Shinichiro Kawasaki
2024-05-03  7:33               ` Sagi Grimberg
2024-04-24  7:59 ` [PATCH blktests v3 11/15] nvme/{007,009,011,013,015,020,024}: drop duplicate nvmet blkdev type tests Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 12/15] nvme/{021,022,025,026,027,028}: do not hard code target blkdev type Shin'ichiro Kawasaki
2024-04-24  7:59 ` [PATCH blktests v3 13/15] nvme/{rc,010,017,031,034,035}: rename nvme_img_size to NVME_IMG_SIZE Shin'ichiro Kawasaki
2024-04-24 12:29   ` Daniel Wagner
2024-04-24  7:59 ` [PATCH blktests v3 14/15] nvme/{rc,016,017}: rename nvme_num_iter to NVME_NUM_ITER Shin'ichiro Kawasaki
2024-04-24 12:30   ` Daniel Wagner
2024-04-24  7:59 ` [PATCH blktests v3 15/15] nvme/rc,srp/rc,common/multipath-over-rdma: rename use_rxe to USE_RXE Shin'ichiro Kawasaki
2024-04-24 12:33   ` Daniel Wagner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).