From: Dmitry Fomichev <dmitry.fomichev@wdc.com>
To: Jens Axboe <axboe@kernel.dk>,
fio@vger.kernel.org, Aravind Ramesh <aravind.ramesh@wdc.com>,
Bart Van Assche <bvanassche@acm.org>,
Naohiro Aota <naohiro.aota@wdc.com>,
Niklas Cassel <niklas.cassel@wdc.com>
Cc: Damien Le Moal <damien.lemoal@wdc.com>,
Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>,
Dmitry Fomichev <dmitry.fomichev@wdc.com>
Subject: [PATCH v3 26/38] t/zbd: skip tests when test prerequisites are not met
Date: Thu, 7 Jan 2021 06:57:27 +0900 [thread overview]
Message-ID: <20210106215739.264524-27-dmitry.fomichev@wdc.com> (raw)
In-Reply-To: <20210106215739.264524-1-dmitry.fomichev@wdc.com>
From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Some of the test cases in t/zbd/test-zbd-support require test target
devices to have certain features. When these prerequisites are not met,
they skip the actual test and report the test result to be "PASS".
This does not help users to understand the true test outcome.
As the tests expand to cover a wider variety of zoned devices and
layouts, reporting skipped tests becomes more and more beneficial.
Modify test-zbd-support script to report skipped test cases.
Introduce helper functions require_*() to check test target
prerequisites. If they are not met, set the variable SKIP_REASON and
return the constant SKIP_TESTCASE from the test function. In the main
loo, print "SKIP" status and SKIP_REASON if the test case is skipped.
Also, output the total number of skipped cases at the end of the test
script run.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
---
t/zbd/test-zbd-support | 93 ++++++++++++++++++++++++++++++------------
1 file changed, 67 insertions(+), 26 deletions(-)
diff --git a/t/zbd/test-zbd-support b/t/zbd/test-zbd-support
index 652dddfc..d94b5125 100755
--- a/t/zbd/test-zbd-support
+++ b/t/zbd/test-zbd-support
@@ -190,6 +190,42 @@ prep_write() {
reset_zone "${dev}" -1
}
+SKIP_TESTCASE=255
+
+require_scsi_dev() {
+ if ! is_scsi_device "$dev"; then
+ SKIP_REASON="$dev is not a SCSI device"
+ return 1
+ fi
+ return 0
+}
+
+require_conv_zone_bytes() {
+ local req_bytes=${1}
+
+ if ((req_bytes > first_sequential_zone_sector * 512)); then
+ SKIP_REASON="$dev does not have enough conventional zones"
+ return 1
+ fi
+ return 0
+}
+
+require_zbd() {
+ if [[ -z ${is_zbd} ]]; then
+ SKIP_REASON="$dev is not a zoned block device"
+ return 1
+ fi
+ return 0
+}
+
+require_regular_block_dev() {
+ if [[ -n ${is_zbd} ]]; then
+ SKIP_REASON="$dev is not a regular block device"
+ return 1
+ fi
+ return 0
+}
+
# Check whether buffered writes are refused.
test1() {
run_fio --name=job1 --filename="$dev" --rw=write --direct=0 --bs=4K \
@@ -327,10 +363,7 @@ test8() {
test9() {
local size
- if ! is_scsi_device "$dev"; then
- echo "$dev is not a SCSI device" >>"${logfile}.${test_number}"
- return 0
- fi
+ require_scsi_dev || return $SKIP_TESTCASE
prep_write
size=$((4 * zone_size))
@@ -346,10 +379,7 @@ test9() {
test10() {
local size
- if ! is_scsi_device "$dev"; then
- echo "$dev is not a SCSI device" >>"${logfile}.${test_number}"
- return 0
- fi
+ require_scsi_dev || return $SKIP_TESTCASE
prep_write
size=$((4 * zone_size))
@@ -413,11 +443,8 @@ test14() {
prep_write
size=$((16 * 2**20)) # 20 MB
- if [ $size -gt $((first_sequential_zone_sector * 512)) ]; then
- echo "$dev does not have enough sequential zones" \
- >>"${logfile}.${test_number}"
- return 0
- fi
+ require_conv_zone_bytes "${size}" || return $SKIP_TESTCASE
+
run_one_fio_job "$(ioengine "libaio")" --iodepth=64 --rw=randwrite --bs=16K \
--zonemode=zbd --zonesize="${zone_size}" --do_verify=1 \
--verify=md5 --size=$size \
@@ -696,6 +723,8 @@ test31() {
test32() {
local off opts=() size
+ require_zbd || return $SKIP_TESTCASE
+
prep_write
off=$((first_sequential_zone_sector * 512))
size=$((disk_size - off))
@@ -814,7 +843,7 @@ read_one_block() {
# Check whether fio accepts --zonemode=none for zoned block devices.
test39() {
- [ -n "$is_zbd" ] || return 0
+ require_zbd || return $SKIP_TESTCASE
read_one_block --zonemode=none >/dev/null || return $?
check_read $((logical_block_size)) || return $?
}
@@ -824,7 +853,7 @@ test40() {
local bs
bs=$((logical_block_size))
- [ -n "$is_zbd" ] || return 0
+ require_zbd || return $SKIP_TESTCASE
read_one_block --zonemode=strided |
grep -q 'fio: --zonesize must be specified when using --zonemode=strided' ||
return $?
@@ -834,21 +863,21 @@ test40() {
# Check whether fio checks the zone size for zoned block devices.
test41() {
- [ -n "$is_zbd" ] || return 0
+ require_zbd || return $SKIP_TESTCASE
read_one_block --zonemode=zbd --zonesize=$((2 * zone_size)) |
grep -q 'job parameter zonesize.*does not match disk zone size'
}
# Check whether fio handles --zonesize=0 correctly for regular block devices.
test42() {
- [ -n "$is_zbd" ] && return 0
+ require_regular_block_dev || return $SKIP_TESTCASE
read_one_block --zonemode=zbd --zonesize=0 |
grep -q 'Specifying the zone size is mandatory for regular block devices with --zonemode=zbd'
}
# Check whether fio handles --zonesize=1 correctly for regular block devices.
test43() {
- [ -n "$is_zbd" ] && return 0
+ require_regular_block_dev || return $SKIP_TESTCASE
read_one_block --zonemode=zbd --zonesize=1 |
grep -q 'zone size must be at least 512 bytes for --zonemode=zbd'
}
@@ -862,7 +891,7 @@ test44() {
test45() {
local bs i
- [ -z "$is_zbd" ] && return 0
+ require_zbd || return $SKIP_TESTCASE
prep_write
bs=$((logical_block_size))
run_one_fio_job "$(ioengine "psync")" --iodepth=1 --rw=randwrite --bs=$bs\
@@ -901,6 +930,8 @@ test47() {
test48() {
local i jobs=16 off opts=()
+ require_zbd || return $SKIP_TESTCASE
+
off=$((first_sequential_zone_sector * 512 + 64 * zone_size))
size=$((16*zone_size))
prep_write
@@ -930,11 +961,7 @@ test48() {
# Check if fio handles --zonecapacity on a normal block device correctly
test49() {
- if [ -n "$is_zbd" ]; then
- echo "$dev is not a regular block device" \
- >>"${logfile}.${test_number}"
- return 0
- fi
+ require_regular_block_dev || return $SKIP_TESTCASE
size=$((2 * zone_size))
capacity=$((zone_size * 3 / 4))
@@ -1087,10 +1114,12 @@ fi
logfile=$0.log
passed=0
+skipped=0
failed=0
if [ -t 1 ]; then
red="\e[1;31m"
green="\e[1;32m"
+ cyan="\e[1;36m"
end="\e[m"
else
red=""
@@ -1101,14 +1130,23 @@ rc=0
intr=0
trap 'intr=1' SIGINT
+ret=0
for test_number in "${tests[@]}"; do
rm -f "${logfile}.${test_number}"
+ unset SKIP_REASON
echo -n "Running test $(printf "%02d" $test_number) ... "
- if eval "test$test_number" && check_log $test_number; then
+ eval "test$test_number"
+ ret=$?
+ if ((!ret)) && check_log $test_number; then
status="PASS"
cc_status="${green}${status}${end}"
((passed++))
+ elif ((ret==SKIP_TESTCASE)); then
+ status="SKIP"
+ echo "${SKIP_REASON}" >> "${logfile}.${test_number}"
+ cc_status="${cyan}${status}${end} ${SKIP_REASON}"
+ ((skipped++))
else
status="FAIL"
cc_status="${red}${status}${end}"
@@ -1121,7 +1159,10 @@ for test_number in "${tests[@]}"; do
done
echo "$passed tests passed"
+if [ $skipped -gt 0 ]; then
+ echo " $skipped tests skipped"
+fi
if [ $failed -gt 0 ]; then
- echo " and $failed tests failed"
+ echo " $failed tests failed"
fi
exit $rc
--
2.28.0
next prev parent reply other threads:[~2021-01-06 21:57 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-06 21:57 [PATCH v3 00/38] ZBD fixes and improvements Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 01/38] zbd: return ENOMEM if zone buffer allocation fails Dmitry Fomichev
2021-01-22 2:07 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 02/38] zbd: use zbd_zone_nr() more actively in the code Dmitry Fomichev
2021-01-22 2:14 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 03/38] zbd: add get_zone() helper function Dmitry Fomichev
2021-01-22 2:19 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 04/38] zbd: introduce zone_unlock() Dmitry Fomichev
2021-01-22 2:23 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 05/38] zbd: engines/libzbc: don't fail on assert for offline zones Dmitry Fomichev
2021-01-22 2:27 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 06/38] zbd: remove dependency on zone type during i/o Dmitry Fomichev
2021-01-22 3:56 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 07/38] zbd: skip offline zones in zbd_convert_to_open_zone() Dmitry Fomichev
2021-01-22 3:59 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 08/38] zbd: avoid zone buffer overrun Dmitry Fomichev
2021-01-22 4:02 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 09/38] zbd: don't unlock zone mutex after verify replay Dmitry Fomichev
2021-01-22 4:13 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 10/38] zbd: do not lock conventional zones on I/O adjustment Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 11/38] zbd: do not set zbd handlers for conventional zones Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 12/38] zbd: count sectors with data for write pointer zones Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 13/38] zbd: initialize min_zone and max_zone for all zone types Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 14/38] zbd: initialize sectors with data at start time Dmitry Fomichev
2021-01-22 4:19 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 15/38] zbd: use zone_lock() in zbd_process_swd() Dmitry Fomichev
2021-01-22 4:28 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 16/38] zbd: disable crossing from conventional to sequential zones Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 17/38] zbd: don't log "zone nnnn is not open" message Dmitry Fomichev
2021-01-22 4:31 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 18/38] zbd: handle conventional start zone in zbd_convert_to_open_zone() Dmitry Fomichev
2021-01-22 4:36 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 19/38] zbd: improve replay range validation Dmitry Fomichev
2021-01-22 4:47 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 20/38] engines/libzbc: enable block backend Dmitry Fomichev
2021-01-22 4:49 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 21/38] zbd: avoid failing assertion in zbd_convert_to_open_zone() Dmitry Fomichev
2021-01-22 5:05 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 22/38] zbd: set thread errors in zbd_adjust_block() Dmitry Fomichev
2021-01-22 5:12 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 23/38] t/zbd: check for error in test #2 Dmitry Fomichev
2021-01-22 5:13 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 24/38] t/zbd: add run-tests-against-nullb script Dmitry Fomichev
2021-01-22 8:47 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 25/38] t/zbd: add -t option to run-tests-against-nullb Dmitry Fomichev
2021-01-06 21:57 ` Dmitry Fomichev [this message]
2021-01-06 21:57 ` [PATCH v3 27/38] t/zbd: skip tests that need too many sequential zones Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 28/38] t/zbd: test that conventional zones are not locked during random i/o Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 29/38] t/zbd: test that zone_reset_threshold calculation is correct Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 30/38] t/zbd: test random I/O direction in all-conventional case Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 31/38] t/zbd: fix wrong units in test case #37 Dmitry Fomichev
2021-01-06 21:57 ` [PATCH v3 32/38] t/zbd: add an option to bail on a failed test Dmitry Fomichev
2021-01-22 8:53 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 33/38] t/zbd: prevent test #31 from looping Dmitry Fomichev
2021-01-22 8:56 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 34/38] t/zbd: add checks for offline zone condition Dmitry Fomichev
2021-01-22 9:06 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 35/38] t/zbd: add test #54 to exercise ZBD verification Dmitry Fomichev
2021-01-22 9:10 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 36/38] t/zbd: show elapsed time in test-zbd-support Dmitry Fomichev
2021-01-22 9:11 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 37/38] t/zbd: increase timeout in test #48 Dmitry Fomichev
2021-01-22 9:12 ` Shinichiro Kawasaki
2021-01-06 21:57 ` [PATCH v3 38/38] t/zbd: avoid looping on invalid command line options Dmitry Fomichev
2021-01-22 9:14 ` Shinichiro Kawasaki
2021-01-22 9:24 ` [PATCH v3 00/38] ZBD fixes and improvements Shinichiro Kawasaki
2021-01-22 20:31 ` Dmitry Fomichev
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210106215739.264524-27-dmitry.fomichev@wdc.com \
--to=dmitry.fomichev@wdc.com \
--cc=aravind.ramesh@wdc.com \
--cc=axboe@kernel.dk \
--cc=bvanassche@acm.org \
--cc=damien.lemoal@wdc.com \
--cc=fio@vger.kernel.org \
--cc=naohiro.aota@wdc.com \
--cc=niklas.cassel@wdc.com \
--cc=shinichiro.kawasaki@wdc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.