From: Frank Rowand <frowand.list@gmail.com>
To: Brendan Higgins <brendanhiggins@google.com>
Cc: Kees Cook <keescook@google.com>,
Luis Chamberlain <mcgrof@kernel.org>,
shuah@kernel.org, Rob Herring <robh@kernel.org>,
Kieran Bingham <kieran.bingham@ideasonboard.com>,
Greg KH <gregkh@linuxfoundation.org>,
Joel Stanley <joel@jms.id.au>,
Michael Ellerman <mpe@ellerman.id.au>,
Joe Perches <joe@perches.com>,
brakmo@fb.com, Steven Rostedt <rostedt@goodmis.org>,
"Bird, Timothy" <Tim.Bird@sony.com>,
Kevin Hilman <khilman@baylibre.com>,
Julia Lawall <julia.lawall@lip6.fr>,
linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Jeff Dike <jdike@addtoit.com>,
Richard Weinberger <richard@nod.at>,
linux-um@lists.infradead.org, Daniel Vetter <daniel@ffwll.ch>,
dri-devel <dri-devel@lists.freedesktop.org>,
Dan Williams <dan.j.williams@intel.com>,
linux-nvdimm <linux-nvdimm@lists.01.org>,
Knut Omang <knut.omang@oracle.com>,
devicetree <devicetree@vger.kernel.org>,
Petr Mladek <pmladek@suse.com>,
Sasha Levin <Alexander.Levin@microsoft.com>,
Amir Goldstein <amir73il@gmail.com>,
dan.carpenter@oracle.com, wfg@linux.intel.com
Subject: Re: [RFC v4 08/17] kunit: test: add support for test abort
Date: Tue, 19 Feb 2019 22:44:23 -0800 [thread overview]
Message-ID: <0124ed28-466c-e954-ddde-495419630a9f@gmail.com> (raw)
In-Reply-To: <CAFd5g467OV8wTB7zEshgZw8g8=zoq0dLVfwx3wDH-8UA+9GWFA@mail.gmail.com>
On 2/19/19 7:39 PM, Brendan Higgins wrote:
> On Mon, Feb 18, 2019 at 11:52 AM Frank Rowand <frowand.list@gmail.com> wrote:
>>
>> On 2/14/19 1:37 PM, Brendan Higgins wrote:
>>> Add support for aborting/bailing out of test cases. Needed for
>>> implementing assertions.
>>>
>>> Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
>>> ---
>>> Changes Since Last Version
>>> - This patch is new introducing a new cross-architecture way to abort
>>> out of a test case (needed for KUNIT_ASSERT_*, see next patch for
>>> details).
>>> - On a side note, this is not a complete replacement for the UML abort
>>> mechanism, but covers the majority of necessary functionality. UML
>>> architecture specific featurs have been dropped from the initial
>>> patchset.
>>> ---
>>> include/kunit/test.h | 24 +++++
>>> kunit/Makefile | 3 +-
>>> kunit/test-test.c | 127 ++++++++++++++++++++++++++
>>> kunit/test.c | 208 +++++++++++++++++++++++++++++++++++++++++--
>>> 4 files changed, 353 insertions(+), 9 deletions(-)
>>> create mode 100644 kunit/test-test.c
>>
>> < snip >
>>
>>> diff --git a/kunit/test.c b/kunit/test.c
>>> index d18c50d5ed671..6e5244642ab07 100644
>>> --- a/kunit/test.c
>>> +++ b/kunit/test.c
>>> @@ -6,9 +6,9 @@
>>> * Author: Brendan Higgins <brendanhiggins@google.com>
>>> */
>>>
>>> -#include <linux/sched.h>
>>> #include <linux/sched/debug.h>
>>> -#include <os.h>
>>> +#include <linux/completion.h>
>>> +#include <linux/kthread.h>
>>> #include <kunit/test.h>
>>>
>>> static bool kunit_get_success(struct kunit *test)
>>> @@ -32,6 +32,27 @@ static void kunit_set_success(struct kunit *test, bool success)
>>> spin_unlock_irqrestore(&test->lock, flags);
>>> }
>>>
>>> +static bool kunit_get_death_test(struct kunit *test)
>>> +{
>>> + unsigned long flags;
>>> + bool death_test;
>>> +
>>> + spin_lock_irqsave(&test->lock, flags);
>>> + death_test = test->death_test;
>>> + spin_unlock_irqrestore(&test->lock, flags);
>>> +
>>> + return death_test;
>>> +}
>>> +
>>> +static void kunit_set_death_test(struct kunit *test, bool death_test)
>>> +{
>>> + unsigned long flags;
>>> +
>>> + spin_lock_irqsave(&test->lock, flags);
>>> + test->death_test = death_test;
>>> + spin_unlock_irqrestore(&test->lock, flags);
>>> +}
>>> +
>>> static int kunit_vprintk_emit(const struct kunit *test,
>>> int level,
>>> const char *fmt,
>>> @@ -70,13 +91,29 @@ static void kunit_fail(struct kunit *test, struct kunit_stream *stream)
>>> stream->commit(stream);
>>> }
>>>
>>> +static void __noreturn kunit_abort(struct kunit *test)
>>> +{
>>> + kunit_set_death_test(test, true);
>>> +
>>> + test->try_catch.throw(&test->try_catch);
>>> +
>>> + /*
>>> + * Throw could not abort from test.
>>> + */
>>> + kunit_err(test, "Throw could not abort from test!");
>>> + show_stack(NULL, NULL);
>>> + BUG();
>>
>> kunit_abort() is what will be call as the result of an assert failure.
>
> Yep. Does that need clarified somewhere.
>>
>> BUG(), which is a panic, which is crashing the system is not acceptable
>> in the Linux kernel. You will just annoy Linus if you submit this.
>
> Sorry, I thought this was an acceptable use case since, a) this should
> never be compiled in a production kernel, b) we are in a pretty bad,
> unpredictable state if we get here and keep going. I think you might
> have said elsewhere that you think "a" is not valid? In any case, I
> can replace this with a WARN, would that be acceptable?
A WARN may or may not make sense, depending on the context. It may
be sufficient to simply report a test failure (as in the old version
of case (2) below.
Answers to "a)" and "b)":
a) it might be in a production kernel
a') it is not acceptable in my development kernel either
b) No. You don't crash a developer's kernel either unless it is
required to avoid data corruption.
b') And you can not do replacements like:
(1) in of_unittest_check_tree_linkage()
----- old -----
if (!of_root)
return;
----- new -----
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, of_root);
(2) in of_unittest_property_string()
----- old -----
/* of_property_read_string_index() tests */
rc = of_property_read_string_index(np, "string-property", 0, strings);
unittest(rc == 0 && !strcmp(strings[0], "foobar"), "of_property_read_string_index() failure; rc=%i\n", rc);
----- new -----
/* of_property_read_string_index() tests */
rc = of_property_read_string_index(np, "string-property", 0, strings);
KUNIT_ASSERT_EQ(test, rc, 0);
KUNIT_EXPECT_STREQ(test, strings[0], "foobar");
If a test fails, that is no reason to abort testing. The remainder of the unit
tests can still run. There may be cascading failures, but that is ok.
next prev parent reply other threads:[~2019-02-20 6:44 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-14 21:37 [RFC v4 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2019-02-14 21:37 ` [RFC v4 01/17] kunit: test: add KUnit test runner core Brendan Higgins
2019-02-14 21:37 ` [RFC v4 02/17] kunit: test: add test resource management API Brendan Higgins
2019-02-15 21:01 ` Stephen Boyd
2019-02-19 23:24 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 03/17] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2019-02-14 21:37 ` [RFC v4 04/17] kunit: test: add test_stream a std::stream like logger Brendan Higgins
2019-02-14 21:37 ` [RFC v4 05/17] kunit: test: add the concept of expectations Brendan Higgins
2019-02-14 21:37 ` [RFC v4 06/17] kbuild: enable building KUnit Brendan Higgins
2019-02-14 21:37 ` [RFC v4 07/17] kunit: test: add initial tests Brendan Higgins
2019-02-14 21:37 ` [RFC v4 08/17] kunit: test: add support for test abort Brendan Higgins
2019-02-18 19:52 ` Frank Rowand
2019-02-20 3:39 ` Brendan Higgins
2019-02-20 6:44 ` Frank Rowand [this message]
2019-02-28 7:42 ` Brendan Higgins
2019-03-22 1:09 ` Frank Rowand
2019-03-22 1:41 ` Brendan Higgins
2019-03-22 7:10 ` Knut Omang
2019-03-25 22:32 ` Brendan Higgins
2019-03-26 7:44 ` Knut Omang
2019-02-26 20:35 ` Stephen Boyd
2019-02-28 9:03 ` Brendan Higgins
2019-02-28 13:54 ` Dan Carpenter
2019-03-04 22:28 ` Brendan Higgins
[not found] ` <155137694423.260864.2846034318906225490@swboyd.mtv.corp.google.com>
2019-03-04 22:39 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 09/17] kunit: test: add the concept of assertions Brendan Higgins
2019-02-14 21:37 ` [RFC v4 10/17] kunit: test: add test managed resource tests Brendan Higgins
2019-02-15 20:54 ` Stephen Boyd
2019-02-19 23:20 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 11/17] kunit: tool: add Python wrappers for running KUnit tests Brendan Higgins
2019-02-14 21:37 ` [RFC v4 12/17] kunit: defconfig: add defconfigs for building " Brendan Higgins
2019-02-14 21:37 ` [RFC v4 13/17] Documentation: kunit: add documentation for KUnit Brendan Higgins
2019-02-14 21:37 ` [RFC v4 14/17] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2019-02-14 21:37 ` [RFC v4 15/17] of: unittest: migrate tests to run on KUnit Brendan Higgins
2019-02-16 0:24 ` Frank Rowand
2019-02-20 2:24 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 16/17] of: unittest: split out a couple of test cases from unittest Brendan Higgins
2019-03-22 1:14 ` Frank Rowand
2019-03-22 1:45 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 17/17] of: unittest: split up some super large test cases Brendan Higgins
2019-03-22 1:16 ` Frank Rowand
2019-03-22 1:45 ` Brendan Higgins
2019-02-18 20:02 ` [RFC v4 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework Frank Rowand
2019-02-20 6:34 ` Brendan Higgins
2019-02-20 6:46 ` Frank Rowand
2019-02-22 20:52 ` Thiago Jung Bauermann
2019-02-28 4:18 ` Brendan Higgins
2019-02-28 4:15 ` Brendan Higgins
2019-03-04 23:01 ` Brendan Higgins
2019-03-22 1:23 ` Frank Rowand
2019-03-25 22:11 ` Brendan Higgins
2019-03-21 1:07 ` Logan Gunthorpe
2019-03-21 5:23 ` Knut Omang
2019-03-21 15:56 ` Logan Gunthorpe
2019-03-21 16:55 ` Brendan Higgins
2019-03-21 19:13 ` Knut Omang
2019-03-21 19:29 ` Logan Gunthorpe
2019-03-21 20:14 ` Knut Omang
2019-03-21 22:07 ` Brendan Higgins
2019-03-21 22:26 ` Logan Gunthorpe
2019-03-21 23:33 ` Brendan Higgins
2019-03-22 1:12 ` Frank Rowand
2019-03-25 22:12 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0124ed28-466c-e954-ddde-495419630a9f@gmail.com \
--to=frowand.list@gmail.com \
--cc=Alexander.Levin@microsoft.com \
--cc=Tim.Bird@sony.com \
--cc=amir73il@gmail.com \
--cc=brakmo@fb.com \
--cc=brendanhiggins@google.com \
--cc=dan.carpenter@oracle.com \
--cc=dan.j.williams@intel.com \
--cc=daniel@ffwll.ch \
--cc=devicetree@vger.kernel.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=gregkh@linuxfoundation.org \
--cc=jdike@addtoit.com \
--cc=joe@perches.com \
--cc=joel@jms.id.au \
--cc=julia.lawall@lip6.fr \
--cc=keescook@google.com \
--cc=khilman@baylibre.com \
--cc=kieran.bingham@ideasonboard.com \
--cc=knut.omang@oracle.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-um@lists.infradead.org \
--cc=mcgrof@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=pmladek@suse.com \
--cc=richard@nod.at \
--cc=robh@kernel.org \
--cc=rostedt@goodmis.org \
--cc=shuah@kernel.org \
--cc=wfg@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).