From: Rae Moar <rmoar@google.com>
To: brendanhiggins@google.com, davidgow@google.com,
dlatypov@google.com, keescook@chromium.org, shuah@kernel.org
Cc: kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org,
linux-kernel@vger.kernel.org, Rae Moar <rmoar@google.com>
Subject: [PATCH 0/2] selftests: tool: update format of kselftest TAP results to improve compatibility with KTAP
Date: Fri, 27 Aug 2021 22:58:10 +0000 [thread overview]
Message-ID: <20210827225812.3247919-1-rmoar@google.com> (raw)
This series of patches updates the format of kselftest TAP results to improve
compatibility with the proposed KTAP specification
(https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqaJk+r-K1YJzPggFDQ@mail.gmail.com/).
Three changes:
- Change from "# " to " " for indentation of nested tests
- Add subtest header line at start of tests with subtests. Line format
is "# Subtest: [name of test]".
- Remove TAP header in nested tests
Standardizing TAP results would not only allow for clearer documentation and ease of reading but by standardizing the format across different testing frameworks, we could also share the use of tools.
As an example:
This is a truncated version of TAP results from the kselftest ptrace with the new format changes:
TAP version 13
1..1
# selftests: ptrace: get_syscall_info
# Subtest: selftests: ptrace: get_syscall_info
1..1
# Starting 1 tests from 1 test cases.
# RUN global.get_syscall_info ...
# OK global.get_syscall_info
ok 1 global.get_syscall_info
# PASSED: 1 / 1 tests passed.
# Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 1 selftests: ptrace: get_syscall_info
With the new patch to update the KUnit parser to improve compatibility with the proposed KTAP specification, (https://lore.kernel.org/linux-kselftest/20210826195505.3066755-1-rmoar@google.com/) the above TAP results would be parsed as the following:
[20:46:09] ============================================================
[20:46:09] ===== selftests: ptrace: get_syscall_info (1 subtest) ======
[20:46:09] [PASSED] global.get_syscall_info
[20:46:09] ======= [PASSED] selftests: ptrace: get_syscall_info =======
[20:46:09] ============================================================
[20:46:09] Testing complete. Passed: 1, Failed: 0, Crashed: 0, Skipped: 0, Errors: 0
Thus, the kunit parser could become a useful tool for kselftest users.
Rae Moar (2):
selftests: tool: Add subtest header line and change indentation format
in TAP results
Revert "selftests: Remove KSFT_TAP_LEVEL"
tools/testing/selftests/Makefile | 6 ++++++
tools/testing/selftests/kselftest/prefix.pl | 2 +-
tools/testing/selftests/kselftest/runner.sh | 7 ++++---
3 files changed, 11 insertions(+), 4 deletions(-)
--
2.33.0.259.gc128427fd7-goog
next reply other threads:[~2021-08-27 22:58 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-27 22:58 Rae Moar [this message]
2021-08-27 22:58 ` [PATCH 1/2] selftests: tool: Add subtest header line and change indentation format in TAP results Rae Moar
2021-08-28 6:26 ` Kees Cook
2021-08-27 22:58 ` [PATCH 2/2] Revert "selftests: Remove KSFT_TAP_LEVEL" Rae Moar
2021-08-28 6:29 ` Kees Cook
2021-08-28 6:10 ` [PATCH 0/2] selftests: tool: update format of kselftest TAP results to improve compatibility with KTAP Kees Cook
2021-08-31 2:26 ` Rae Moar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210827225812.3247919-1-rmoar@google.com \
--to=rmoar@google.com \
--cc=brendanhiggins@google.com \
--cc=davidgow@google.com \
--cc=dlatypov@google.com \
--cc=keescook@chromium.org \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).