From: David Gow <davidgow@google.com>
To: Daniel Latypov <dlatypov@google.com>
Cc: Brendan Higgins <brendanhiggins@google.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
KUnit Development <kunit-dev@googlegroups.com>,
"open list:KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@vger.kernel.org>,
Shuah Khan <skhan@linuxfoundation.org>
Subject: Re: [PATCH] kunit: tool: fix --json output for skipped tests
Date: Sat, 9 Oct 2021 08:29:33 +0800 [thread overview]
Message-ID: <CABVgOS=GanpOft7jtKNtyjucE1aaLH8-n3PPmQNCzvdnLAZ1bA@mail.gmail.com> (raw)
In-Reply-To: <20211008232421.1264857-1-dlatypov@google.com>
On Sat, Oct 9, 2021 at 7:24 AM Daniel Latypov <dlatypov@google.com> wrote:
>
> Currently, KUnit will report SKIPPED tests as having failed if one uses
> --json.
>
> Add the missing if statement to set the appropriate status ("SKIP").
> See https://api.kernelci.org/schema-test-case.html:
> "status": {
> "type": "string",
> "description": "The status of the execution of this test case",
> "enum": ["PASS", "FAIL", "SKIP", "ERROR"],
> "default": "PASS"
> },
> with this, we now can properly produce all four of the statuses.
>
> Fixes: 5acaf6031f53 ("kunit: tool: Support skipped tests in kunit_tool")
> Signed-off-by: Daniel Latypov <dlatypov@google.com>
> ---
Ah: thanks! I totally missed this!
Reviewed-by: David Gow <davidgow@google.com>
-- David
> tools/testing/kunit/kunit_json.py | 2 ++
> tools/testing/kunit/kunit_tool_test.py | 6 ++++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py
> index 746bec72b9ac..b6e66c5d64d1 100644
> --- a/tools/testing/kunit/kunit_json.py
> +++ b/tools/testing/kunit/kunit_json.py
> @@ -30,6 +30,8 @@ def _get_group_json(test: Test, def_config: str,
> test_case = {"name": subtest.name, "status": "FAIL"}
> if subtest.status == TestStatus.SUCCESS:
> test_case["status"] = "PASS"
> + elif subtest.status == TestStatus.SKIPPED:
> + test_case["status"] = "SKIP"
> elif subtest.status == TestStatus.TEST_CRASHED:
> test_case["status"] = "ERROR"
> test_cases.append(test_case)
> diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
> index 77e61b0a40e8..b3cc0227843c 100755
> --- a/tools/testing/kunit/kunit_tool_test.py
> +++ b/tools/testing/kunit/kunit_tool_test.py
> @@ -426,6 +426,12 @@ class KUnitJsonTest(unittest.TestCase):
> {'name': 'example_simple_test', 'status': 'ERROR'},
> result["sub_groups"][1]["test_cases"][0])
>
> + def test_skipped_test_json(self):
> + result = self._json_for('test_skip_tests.log')
> + self.assertEqual(
> + {'name': 'example_skip_test', 'status': 'SKIP'},
> + result["sub_groups"][1]["test_cases"][1])
> +
> def test_no_tests_json(self):
> result = self._json_for('test_is_test_passed-no_tests_run_with_header.log')
> self.assertEqual(0, len(result['sub_groups']))
>
> base-commit: 4a86e2973c31902a2a72f4f25f99b2367188ec5d
> --
> 2.33.0.882.g93a45727a2-goog
>
next prev parent reply other threads:[~2021-10-09 0:29 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-08 23:24 [PATCH] kunit: tool: fix --json output for skipped tests Daniel Latypov
2021-10-09 0:29 ` David Gow [this message]
2021-12-07 20:03 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABVgOS=GanpOft7jtKNtyjucE1aaLH8-n3PPmQNCzvdnLAZ1bA@mail.gmail.com' \
--to=davidgow@google.com \
--cc=brendanhiggins@google.com \
--cc=dlatypov@google.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=skhan@linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).