* [PATCH] kunit: Report test parameter results as (K)TAP subtests
@ 2021-10-06 7:11 David Gow
2021-10-06 20:32 ` Daniel Latypov
0 siblings, 1 reply; 3+ messages in thread
From: David Gow @ 2021-10-06 7:11 UTC (permalink / raw)
To: Brendan Higgins, Rae Moar, Daniel Latypov, Shuah Khan
Cc: kunit-dev, linux-kselftest, linux-kernel, David Gow
Currently, the results for individial parameters in a parameterised test
are simply output as (K)TAP diagnostic lines. However, the plan was
always[1] to make these (K)TAP subtests when kunit_tool supported them.
With [2], these are now supported. (v5 will print out an error about the
missing plan line, but this can safely be ignored, and will hopefully be
changed). As a result, individual test parameter results are parsed,
displayed in the formatted results, and counted for test statistics.
[1]: https://lore.kernel.org/linux-kselftest/CABVgOSnJAgWvTTABaF082LuYjAoAWzrBsyu9sT7x4GGMVsOD6Q@mail.gmail.com/
[2]: https://lore.kernel.org/linux-kselftest/20211006001447.20919-1-dlatypov@google.com/
Signed-off-by: David Gow <davidgow@google.com>
---
lib/kunit/test.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index f246b847024e..02a9fdadcae2 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -508,6 +508,8 @@ int kunit_run_tests(struct kunit_suite *suite)
/* Get initial param. */
param_desc[0] = '\0';
test.param_value = test_case->generate_params(NULL, param_desc);
+ kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+ "# Subtest: %s", test_case->name);
}
do {
@@ -520,9 +522,8 @@ int kunit_run_tests(struct kunit_suite *suite)
}
kunit_log(KERN_INFO, &test,
- KUNIT_SUBTEST_INDENT
- "# %s: %s %d - %s",
- test_case->name,
+ KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+ "%s %d - %s",
kunit_status_to_ok_not_ok(test.status),
test.param_index + 1, param_desc);
--
2.33.0.800.g4c38ced690-goog
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] kunit: Report test parameter results as (K)TAP subtests
2021-10-06 7:11 [PATCH] kunit: Report test parameter results as (K)TAP subtests David Gow
@ 2021-10-06 20:32 ` Daniel Latypov
2021-10-06 20:57 ` Brendan Higgins
0 siblings, 1 reply; 3+ messages in thread
From: Daniel Latypov @ 2021-10-06 20:32 UTC (permalink / raw)
To: David Gow
Cc: Brendan Higgins, Rae Moar, Shuah Khan, kunit-dev,
linux-kselftest, linux-kernel
On Wed, Oct 6, 2021 at 12:11 AM David Gow <davidgow@google.com> wrote:
>
> Currently, the results for individial parameters in a parameterised test
> are simply output as (K)TAP diagnostic lines. However, the plan was
> always[1] to make these (K)TAP subtests when kunit_tool supported them.
>
> With [2], these are now supported. (v5 will print out an error about the
> missing plan line, but this can safely be ignored, and will hopefully be
> changed). As a result, individual test parameter results are parsed,
Hmm, I'd rather not condition users to ignore warnings.
It's possible we can get this all fixed up in time for 5.16, but we
have quite a bit we're trying to get in already, so I'm not sure.
> displayed in the formatted results, and counted for test statistics.
>
> [1]: https://lore.kernel.org/linux-kselftest/CABVgOSnJAgWvTTABaF082LuYjAoAWzrBsyu9sT7x4GGMVsOD6Q@mail.gmail.com/
> [2]: https://lore.kernel.org/linux-kselftest/20211006001447.20919-1-dlatypov@google.com/
>
> Signed-off-by: David Gow <davidgow@google.com>
> ---
> lib/kunit/test.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index f246b847024e..02a9fdadcae2 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -508,6 +508,8 @@ int kunit_run_tests(struct kunit_suite *suite)
> /* Get initial param. */
> param_desc[0] = '\0';
> test.param_value = test_case->generate_params(NULL, param_desc);
> + kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
> + "# Subtest: %s", test_case->name);
> }
>
> do {
> @@ -520,9 +522,8 @@ int kunit_run_tests(struct kunit_suite *suite)
> }
>
> kunit_log(KERN_INFO, &test,
> - KUNIT_SUBTEST_INDENT
> - "# %s: %s %d - %s",
> - test_case->name,
> + KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
> + "%s %d - %s",
> kunit_status_to_ok_not_ok(test.status),
> test.param_index + 1, param_desc);
>
> --
> 2.33.0.800.g4c38ced690-goog
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] kunit: Report test parameter results as (K)TAP subtests
2021-10-06 20:32 ` Daniel Latypov
@ 2021-10-06 20:57 ` Brendan Higgins
0 siblings, 0 replies; 3+ messages in thread
From: Brendan Higgins @ 2021-10-06 20:57 UTC (permalink / raw)
To: Daniel Latypov
Cc: David Gow, Rae Moar, Shuah Khan, kunit-dev, linux-kselftest,
linux-kernel
On Wed, Oct 6, 2021 at 1:32 PM Daniel Latypov <dlatypov@google.com> wrote:
>
> On Wed, Oct 6, 2021 at 12:11 AM David Gow <davidgow@google.com> wrote:
> >
> > Currently, the results for individial parameters in a parameterised test
> > are simply output as (K)TAP diagnostic lines. However, the plan was
> > always[1] to make these (K)TAP subtests when kunit_tool supported them.
> >
> > With [2], these are now supported. (v5 will print out an error about the
> > missing plan line, but this can safely be ignored, and will hopefully be
> > changed). As a result, individual test parameter results are parsed,
>
> Hmm, I'd rather not condition users to ignore warnings.
> It's possible we can get this all fixed up in time for 5.16, but we
> have quite a bit we're trying to get in already, so I'm not sure.
I agree with Daniel. I think we should just get that fixed first. I
will poke Shuah to start applying patches for 5.16 to give us a place
to work.
> > displayed in the formatted results, and counted for test statistics.
> >
> > [1]: https://lore.kernel.org/linux-kselftest/CABVgOSnJAgWvTTABaF082LuYjAoAWzrBsyu9sT7x4GGMVsOD6Q@mail.gmail.com/
> > [2]: https://lore.kernel.org/linux-kselftest/20211006001447.20919-1-dlatypov@google.com/
> >
> > Signed-off-by: David Gow <davidgow@google.com>
> > ---
> > lib/kunit/test.c | 7 ++++---
> > 1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> > index f246b847024e..02a9fdadcae2 100644
> > --- a/lib/kunit/test.c
> > +++ b/lib/kunit/test.c
> > @@ -508,6 +508,8 @@ int kunit_run_tests(struct kunit_suite *suite)
> > /* Get initial param. */
> > param_desc[0] = '\0';
> > test.param_value = test_case->generate_params(NULL, param_desc);
> > + kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
> > + "# Subtest: %s", test_case->name);
> > }
> >
> > do {
> > @@ -520,9 +522,8 @@ int kunit_run_tests(struct kunit_suite *suite)
> > }
> >
> > kunit_log(KERN_INFO, &test,
> > - KUNIT_SUBTEST_INDENT
> > - "# %s: %s %d - %s",
> > - test_case->name,
> > + KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
> > + "%s %d - %s",
> > kunit_status_to_ok_not_ok(test.status),
> > test.param_index + 1, param_desc);
> >
> > --
> > 2.33.0.800.g4c38ced690-goog
> >
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-10-06 20:57 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-06 7:11 [PATCH] kunit: Report test parameter results as (K)TAP subtests David Gow
2021-10-06 20:32 ` Daniel Latypov
2021-10-06 20:57 ` Brendan Higgins
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).