From: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
To: "André Almeida" <andrealmeid@collabora.com>
Cc: Christoph Hellwig <hch@lst.de>,
linux-kernel@vger.kernel.org,
Brendan Higgins <brendanhiggins@google.com>,
linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
Shuah Khan <shuah@kernel.org>,
~lkcamp/patches@lists.sr.ht, nfraprado@collabora.com,
leandro.ribeiro@collabora.com,
Vitor Massaru Iha <vitor@massaru.org>,
lucmaga@gmail.com, David Gow <davidgow@google.com>,
Daniel Latypov <dlatypov@google.com>,
tales.aparecida@gmail.com
Subject: Re: [PATCH v2 0/1] lib: Convert UUID runtime test to KUnit
Date: Thu, 10 Jun 2021 12:13:58 +0300 [thread overview]
Message-ID: <YMHX1mKuqoRCiHAF@smile.fi.intel.com> (raw)
In-Reply-To: <20210609233730.164082-1-andrealmeid@collabora.com>
On Wed, Jun 09, 2021 at 08:37:29PM -0300, André Almeida wrote:
> Hi,
>
> This patch converts existing UUID runtime test to use KUnit framework.
>
> Below, there's a comparison between the old output format and the new
> one. Keep in mind that even if KUnit seems very verbose, this is the
> corner case where _every_ test has failed.
>
> * This is how the current output looks like in success:
>
> test_uuid: all 18 tests passed
>
> * And when it fails:
>
> test_uuid: conversion test #1 failed on LE data: 'c33f4995-3701-450e-9fbf-206a2e98e576'
> test_uuid: cmp test #2 failed on LE data: 'c33f4995-3701-450e-9fbf-206a2e98e576'
> test_uuid: cmp test #2 actual data: 'c33f4995-3701-450e-9fbf-206a2e98e576'
> test_uuid: conversion test #3 failed on BE data: 'c33f4995-3701-450e-9fbf-206a2e98e576'
> test_uuid: cmp test #4 failed on BE data: 'c33f4995-3701-450e-9fbf-206a2e98e576'
> test_uuid: cmp test #4 actual data: 'c33f4995-3701-450e-9fbf-206a2e98e576'
> test_uuid: conversion test #5 failed on LE data: '64b4371c-77c1-48f9-8221-29f054fc023b'
> test_uuid: cmp test #6 failed on LE data: '64b4371c-77c1-48f9-8221-29f054fc023b'
> test_uuid: cmp test #6 actual data: '64b4371c-77c1-48f9-8221-29f054fc023b'
> test_uuid: conversion test #7 failed on BE data: '64b4371c-77c1-48f9-8221-29f054fc023b'
> test_uuid: cmp test #8 failed on BE data: '64b4371c-77c1-48f9-8221-29f054fc023b'
> test_uuid: cmp test #8 actual data: '64b4371c-77c1-48f9-8221-29f054fc023b'
> test_uuid: conversion test #9 failed on LE data: '0cb4ddff-a545-4401-9d06-688af53e7f84'
> test_uuid: cmp test #10 failed on LE data: '0cb4ddff-a545-4401-9d06-688af53e7f84'
> test_uuid: cmp test #10 actual data: '0cb4ddff-a545-4401-9d06-688af53e7f84'
> test_uuid: conversion test #11 failed on BE data: '0cb4ddff-a545-4401-9d06-688af53e7f84'
> test_uuid: cmp test #12 failed on BE data: '0cb4ddff-a545-4401-9d06-688af53e7f84'
> test_uuid: cmp test #12 actual data: '0cb4ddff-a545-4401-9d06-688af53e7f84'
> test_uuid: negative test #13 passed on wrong LE data: 'c33f4995-3701-450e-9fbf206a2e98e576 '
> test_uuid: negative test #14 passed on wrong BE data: 'c33f4995-3701-450e-9fbf206a2e98e576 '
> test_uuid: negative test #15 passed on wrong LE data: '64b4371c-77c1-48f9-8221-29f054XX023b'
> test_uuid: negative test #16 passed on wrong BE data: '64b4371c-77c1-48f9-8221-29f054XX023b'
> test_uuid: negative test #17 passed on wrong LE data: '0cb4ddff-a545-4401-9d06-688af53e'
> test_uuid: negative test #18 passed on wrong BE data: '0cb4ddff-a545-4401-9d06-688af53e'
> test_uuid: failed 18 out of 18 tests
>
>
> * Now, here's how it looks like with KUnit:
>
> ======== [PASSED] uuid ========
> [PASSED] uuid_correct_be
> [PASSED] uuid_correct_le
> [PASSED] uuid_wrong_be
> [PASSED] uuid_wrong_le
>
> * And if every test fail with KUnit:
>
> ======== [FAILED] uuid ========
> [FAILED] uuid_correct_be
> # uuid_correct_be: ASSERTION FAILED at lib/test_uuid.c:57
> Expected uuid_parse(data->uuid, &be) == 1, but
> uuid_parse(data->uuid, &be) == 0
>
> failed to parse 'c33f4995-3701-450e-9fbf-206a2e98e576'
> # uuid_correct_be: not ok 1 - c33f4995-3701-450e-9fbf-206a2e98e576
> # uuid_correct_be: ASSERTION FAILED at lib/test_uuid.c:57
> Expected uuid_parse(data->uuid, &be) == 1, but
> uuid_parse(data->uuid, &be) == 0
>
> failed to parse '64b4371c-77c1-48f9-8221-29f054fc023b'
> # uuid_correct_be: not ok 2 - 64b4371c-77c1-48f9-8221-29f054fc023b
> # uuid_correct_be: ASSERTION FAILED at lib/test_uuid.c:57
> Expected uuid_parse(data->uuid, &be) == 1, but
> uuid_parse(data->uuid, &be) == 0
>
> failed to parse '0cb4ddff-a545-4401-9d06-688af53e7f84'
> # uuid_correct_be: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e7f84
> not ok 1 - uuid_correct_be
>
> [FAILED] uuid_correct_le
> # uuid_correct_le: ASSERTION FAILED at lib/test_uuid.c:46
> Expected guid_parse(data->uuid, &le) == 1, but
> guid_parse(data->uuid, &le) == 0
>
> failed to parse 'c33f4995-3701-450e-9fbf-206a2e98e576'
> # uuid_correct_le: not ok 1 - c33f4995-3701-450e-9fbf-206a2e98e576
> # uuid_correct_le: ASSERTION FAILED at lib/test_uuid.c:46
> Expected guid_parse(data->uuid, &le) == 1, but
> guid_parse(data->uuid, &le) == 0
>
> failed to parse '64b4371c-77c1-48f9-8221-29f054fc023b'
> # uuid_correct_le: not ok 2 - 64b4371c-77c1-48f9-8221-29f054fc023b
> # uuid_correct_le: ASSERTION FAILED at lib/test_uuid.c:46
> Expected guid_parse(data->uuid, &le) == 1, but
> guid_parse(data->uuid, &le) == 0
>
> failed to parse '0cb4ddff-a545-4401-9d06-688af53e7f84'
> # uuid_correct_le: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e7f84
> not ok 2 - uuid_correct_le
>
> [FAILED] uuid_wrong_be
> # uuid_wrong_be: ASSERTION FAILED at lib/test_uuid.c:77
> Expected uuid_parse(*data, &be) == 0, but
> uuid_parse(*data, &be) == -22
>
> parsing of 'c33f4995-3701-450e-9fbf206a2e98e576 ' should've failed
> # uuid_wrong_be: not ok 1 - c33f4995-3701-450e-9fbf206a2e98e576
> # uuid_wrong_be: ASSERTION FAILED at lib/test_uuid.c:77
> Expected uuid_parse(*data, &be) == 0, but
> uuid_parse(*data, &be) == -22
>
> parsing of '64b4371c-77c1-48f9-8221-29f054XX023b' should've failed
> # uuid_wrong_be: not ok 2 - 64b4371c-77c1-48f9-8221-29f054XX023b
> # uuid_wrong_be: ASSERTION FAILED at lib/test_uuid.c:77
> Expected uuid_parse(*data, &be) == 0, but
> uuid_parse(*data, &be) == -22
>
> parsing of '0cb4ddff-a545-4401-9d06-688af53e' should've failed
> # uuid_wrong_be: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e
> not ok 3 - uuid_wrong_be
>
> [FAILED] uuid_wrong_le
> # uuid_wrong_le: ASSERTION FAILED at lib/test_uuid.c:68
> Expected guid_parse(*data, &le) == 0, but
> guid_parse(*data, &le) == -22
>
> parsing of 'c33f4995-3701-450e-9fbf206a2e98e576 ' should've failed
> # uuid_wrong_le: not ok 1 - c33f4995-3701-450e-9fbf206a2e98e576
> # uuid_wrong_le: ASSERTION FAILED at lib/test_uuid.c:68
> Expected guid_parse(*data, &le) == 0, but
> guid_parse(*data, &le) == -22
>
> parsing of '64b4371c-77c1-48f9-8221-29f054XX023b' should've failed
> # uuid_wrong_le: not ok 2 - 64b4371c-77c1-48f9-8221-29f054XX023b
> # uuid_wrong_le: ASSERTION FAILED at lib/test_uuid.c:68
> Expected guid_parse(*data, &le) == 0, but
> guid_parse(*data, &le) == -22
>
> parsing of '0cb4ddff-a545-4401-9d06-688af53e' should've failed
> # uuid_wrong_le: not ok 3 - 0cb4ddff-a545-4401-9d06-688af53e
> not ok 4 - uuid_wrong_le
Thanks!
It's not your fault but I think we need to defer this until KUnit gains support
of the run statistics. My guts telling me if we allow more and more conversions
like this the point will vanish and nobody will care.
I like the code, but I can give my tag after KUnit prints some kind of this:
* This is how the current output looks like in success:
test_uuid: all 18 tests passed
* And when it fails:
test_uuid: failed 18 out of 18 tests
> Changes from v1:
> - Test suite name: uuid_test -> uuid
> - Config name: TEST_UUID -> UUID_KUNIT_TEST
> - Config entry in the Kconfig file left where it is
> - Converted tests to use _MSG variant
>
> André Almeida (1):
> lib: Convert UUID runtime test to KUnit
>
> lib/Kconfig.debug | 11 +++-
> lib/Makefile | 2 +-
> lib/test_uuid.c | 137 +++++++++++++++++++---------------------------
> 3 files changed, 67 insertions(+), 83 deletions(-)
>
> --
> 2.31.1
>
--
With Best Regards,
Andy Shevchenko
next prev parent reply other threads:[~2021-06-10 9:14 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-09 23:37 [PATCH v2 0/1] lib: Convert UUID runtime test to KUnit André Almeida
2021-06-09 23:37 ` [PATCH v2 1/1] " André Almeida
2021-06-10 9:13 ` Andy Shevchenko [this message]
2021-06-10 11:53 ` [PATCH v2 0/1] " David Gow
2021-06-10 12:39 ` Andy Shevchenko
2021-06-10 13:08 ` André Almeida
2021-06-10 13:52 ` Andy Shevchenko
2021-06-10 13:56 ` André Almeida
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YMHX1mKuqoRCiHAF@smile.fi.intel.com \
--to=andriy.shevchenko@linux.intel.com \
--cc=andrealmeid@collabora.com \
--cc=brendanhiggins@google.com \
--cc=davidgow@google.com \
--cc=dlatypov@google.com \
--cc=hch@lst.de \
--cc=kunit-dev@googlegroups.com \
--cc=leandro.ribeiro@collabora.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=lucmaga@gmail.com \
--cc=nfraprado@collabora.com \
--cc=shuah@kernel.org \
--cc=tales.aparecida@gmail.com \
--cc=vitor@massaru.org \
--cc=~lkcamp/patches@lists.sr.ht \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).