From: Daniel Latypov <dlatypov@google.com>
To: David Gow <davidgow@google.com>
Cc: Brendan Higgins <brendanhiggins@google.com>,
Rae Moar <rmr167@gmail.com>,
Shuah Khan <skhan@linuxfoundation.org>,
kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 1/4] kunit: tool: Do not error on tests without test plans
Date: Wed, 27 Oct 2021 15:12:52 -0700 [thread overview]
Message-ID: <CAGS_qxo2=NmXEnEnkg4E_1k-DXwniGK8xFZi08nKZZkb8otXzQ@mail.gmail.com> (raw)
In-Reply-To: <20211027013702.2039566-1-davidgow@google.com>
On Tue, Oct 26, 2021 at 6:37 PM David Gow <davidgow@google.com> wrote:
>
> The (K)TAP spec encourages test output to begin with a 'test plan': a
> count of the number of tests being run of the form:
> 1..n
>
> However, some test suites might not know the number of subtests in
> advance (for example, KUnit's parameterised tests use a generator
> function). In this case, it's not possible to print the test plan in
> advance.
>
> kunit_tool already parses test output which doesn't contain a plan, but
> reports an error. Since we want to use nested subtests with KUnit
> paramterised tests, remove this error.
>
> Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
This looks to be unchanged from v1.
Looks good to me given kunit itself will report SKIPPED for parameterised tests.
> ---
> tools/testing/kunit/kunit_parser.py | 5 ++---
> tools/testing/kunit/kunit_tool_test.py | 5 ++++-
> 2 files changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
> index 3355196d0515..50ded55c168c 100644
> --- a/tools/testing/kunit/kunit_parser.py
> +++ b/tools/testing/kunit/kunit_parser.py
> @@ -340,8 +340,8 @@ def parse_test_plan(lines: LineStream, test: Test) -> bool:
> """
> Parses test plan line and stores the expected number of subtests in
> test object. Reports an error if expected count is 0.
> - Returns False and reports missing test plan error if fails to parse
> - test plan.
> + Returns False and sets expected_count to None if there is no valid test
> + plan.
>
> Accepted format:
> - '1..[number of subtests]'
> @@ -356,7 +356,6 @@ def parse_test_plan(lines: LineStream, test: Test) -> bool:
> match = TEST_PLAN.match(lines.peek())
> if not match:
> test.expected_count = None
> - test.add_error('missing plan line!')
> return False
> test.log.append(lines.pop())
> expected_count = int(match.group(1))
> diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
> index 9c4126731457..bc8793145713 100755
> --- a/tools/testing/kunit/kunit_tool_test.py
> +++ b/tools/testing/kunit/kunit_tool_test.py
> @@ -191,7 +191,10 @@ class KUnitParserTest(unittest.TestCase):
> result = kunit_parser.parse_run_tests(
> kunit_parser.extract_tap_lines(
> file.readlines()))
> - self.assertEqual(2, result.test.counts.errors)
> + # A missing test plan is not an error.
> + self.assertEqual(0, result.test.counts.errors)
> + # All tests should be accounted for.
> + self.assertEqual(10, result.test.counts.total())
> self.assertEqual(
> kunit_parser.TestStatus.SUCCESS,
> result.status)
> --
> 2.33.0.1079.g6e70778dc9-goog
>
next prev parent reply other threads:[~2021-10-27 22:13 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-27 1:36 [PATCH v2 1/4] kunit: tool: Do not error on tests without test plans David Gow
2021-10-27 1:37 ` [PATCH v2 2/4] kunit: tool: Report an error if any test has no subtests David Gow
2021-10-27 1:37 ` [PATCH v2 3/4] kunit: Don't crash if no parameters are generated David Gow
2021-10-27 18:59 ` Daniel Latypov
2021-10-27 1:37 ` [PATCH v2 4/4] kunit: Report test parameter results as (K)TAP subtests David Gow
2021-10-27 22:12 ` Daniel Latypov [this message]
2021-10-27 23:02 ` [PATCH v2 1/4] kunit: tool: Do not error on tests without test plans David Gow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGS_qxo2=NmXEnEnkg4E_1k-DXwniGK8xFZi08nKZZkb8otXzQ@mail.gmail.com' \
--to=dlatypov@google.com \
--cc=brendanhiggins@google.com \
--cc=davidgow@google.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=rmr167@gmail.com \
--cc=skhan@linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).