From: Paolo Bonzini <pbonzini@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: shuah@kernel.org, linux-kselftest@vger.kernel.org
Subject: [PATCH v2 1/6] kselftest: fix TAP output for skipped tests
Date: Mon, 22 Jun 2020 20:15:42 -0400 [thread overview]
Message-ID: <20200623001547.22255-2-pbonzini@redhat.com> (raw)
In-Reply-To: <20200623001547.22255-1-pbonzini@redhat.com>
According to the TAP specification, a skipped test must be marked as "ok"
and annotated with the SKIP directive, for example
ok 23 # skip Insufficient flogiston pressure.
(https://testanything.org/tap-specification.html)
Fix the kselftest infrastructure to match this.
For ksft_exit_skip, it is preferrable to emit a dummy plan line that
indicates the whole test was skipped, but this is not always possible
because of ksft_exit_skip being used as a "shortcut" by the tests.
In that case, print the test counts and a normal "ok" line. The format
is now the same independent of whether msg is NULL or not (but it is
never NULL in any caller right now).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
tools/testing/selftests/kselftest.h | 28 +++++++++++++++------
tools/testing/selftests/kselftest/runner.sh | 2 +-
2 files changed, 21 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
index 862eee734553..1b0075359734 100644
--- a/tools/testing/selftests/kselftest.h
+++ b/tools/testing/selftests/kselftest.h
@@ -128,7 +128,7 @@ static inline void ksft_test_result_skip(const char *msg, ...)
ksft_cnt.ksft_xskip++;
va_start(args, msg);
- printf("not ok %d # SKIP ", ksft_test_num());
+ printf("ok %d # SKIP ", ksft_test_num());
errno = saved_errno;
vprintf(msg, args);
va_end(args);
@@ -190,18 +190,30 @@ static inline int ksft_exit_xpass(void)
static inline int ksft_exit_skip(const char *msg, ...)
{
- if (msg) {
- int saved_errno = errno;
- va_list args;
+ int saved_errno = errno;
+ va_list args;
- va_start(args, msg);
- printf("not ok %d # SKIP ", 1 + ksft_test_num());
+ va_start(args, msg);
+
+ /*
+ * FIXME: several tests misuse ksft_exit_skip so produce
+ * something sensible if some tests have already been run
+ * or a plan has been printed. Those tests should use
+ * ksft_test_result_skip or ksft_exit_fail_msg instead.
+ */
+ if (ksft_plan || ksft_test_num()) {
+ ksft_cnt.ksft_xskip++;
+ printf("ok %d # SKIP ", 1 + ksft_test_num());
+ } else {
+ printf("1..0 # SKIP ");
+ }
+ if (msg) {
errno = saved_errno;
vprintf(msg, args);
va_end(args);
- } else {
- ksft_print_cnts();
}
+ if (ksft_test_num())
+ ksft_print_cnts();
exit(KSFT_SKIP);
}
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 676b3a8b114d..f4815cbcd60f 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -77,7 +77,7 @@ run_one()
echo "ok $test_num $TEST_HDR_MSG") ||
(rc=$?; \
if [ $rc -eq $skip_rc ]; then \
- echo "not ok $test_num $TEST_HDR_MSG # SKIP"
+ echo "ok $test_num $TEST_HDR_MSG # SKIP"
elif [ $rc -eq $timeout_rc ]; then \
echo "#"
echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT"
--
2.26.2
next prev parent reply other threads:[~2020-06-23 0:15 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-23 0:15 [PATCH v2 0/6] kselftest: fix TAP output for skipped test and ksft_set_plan misuse Paolo Bonzini
2020-06-23 0:15 ` Paolo Bonzini [this message]
2020-06-23 0:15 ` [PATCH v2 2/6] selftests: breakpoints: fix computation of test plan Paolo Bonzini
2020-06-23 0:15 ` [PATCH v2 3/6] selftests: breakpoints: do not use ksft_exit_skip after ksft_set_plan Paolo Bonzini
2020-06-23 0:15 ` [PATCH v2 4/6] selftests: pidfd: " Paolo Bonzini
2020-06-23 20:44 ` Christian Brauner
2020-06-24 6:21 ` Paolo Bonzini
2020-07-06 20:55 ` Shuah Khan
2020-07-07 9:49 ` Paolo Bonzini
2020-06-23 0:15 ` [PATCH v2 5/6] selftests: sigaltstack: " Paolo Bonzini
2020-06-23 0:15 ` [PATCH v2 6/6] selftests: sync_test: " Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200623001547.22255-2-pbonzini@redhat.com \
--to=pbonzini@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).