From: Brendan Higgins <brendanhiggins@google.com>
To: Knut Omang <knut.omang@oracle.com>
Cc: Frank Rowand <frowand.list@gmail.com>,
Kees Cook <keescook@google.com>,
Luis Chamberlain <mcgrof@kernel.org>,
shuah@kernel.org, Rob Herring <robh@kernel.org>,
Kieran Bingham <kieran.bingham@ideasonboard.com>,
Greg KH <gregkh@linuxfoundation.org>,
Joel Stanley <joel@jms.id.au>,
Michael Ellerman <mpe@ellerman.id.au>,
Joe Perches <joe@perches.com>,
brakmo@fb.com, Steven Rostedt <rostedt@goodmis.org>,
"Bird, Timothy" <Tim.Bird@sony.com>,
Kevin Hilman <khilman@baylibre.com>,
Julia Lawall <julia.lawall@lip6.fr>,
linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Jeff Dike <jdike@addtoit.com>,
Richard Weinberger <richard@nod.at>,
linux-um@lists.infradead.org, Daniel Vetter <daniel@ffwll.ch>,
dri-devel <dri-devel@lists.freedesktop.org>,
Dan Williams <dan.j.williams@intel.com>,
linux-nvdimm <linux-nvdimm@lists.01.org>,
devicetree <devicetree@vger.kernel.org>,
Petr Mladek <pmladek@suse.com>,
Sasha Levin <Alexander.Levin@microsoft.com>,
Amir Goldstein <amir73il@gmail.com>,
Dan Carpenter <dan.carpenter@oracle.com>,
wfg@linux.intel.com
Subject: Re: [RFC v4 08/17] kunit: test: add support for test abort
Date: Mon, 25 Mar 2019 15:32:07 -0700 [thread overview]
Message-ID: <CAFd5g44eqjN-nVCJuoeYFCxwVa5AorWiAnXe-tFCAc11zDgJFA@mail.gmail.com> (raw)
In-Reply-To: <e29d487f5ca0a5caf4723bc5d96b8a523a53a318.camel@oracle.com>
On Fri, Mar 22, 2019 at 12:11 AM Knut Omang <knut.omang@oracle.com> wrote:
>
> On Thu, 2019-03-21 at 18:41 -0700, Brendan Higgins wrote:
> > On Thu, Mar 21, 2019 at 6:10 PM Frank Rowand <frowand.list@gmail.com> wrote:
> > > On 2/27/19 11:42 PM, Brendan Higgins wrote:
> > > > On Tue, Feb 19, 2019 at 10:44 PM Frank Rowand <frowand.list@gmail.com>
> > > > wrote:
> > > > > On 2/19/19 7:39 PM, Brendan Higgins wrote:
> > > > > > On Mon, Feb 18, 2019 at 11:52 AM Frank Rowand <frowand.list@gmail.com>
> > > > > > wrote:
> > > > > > > On 2/14/19 1:37 PM, Brendan Higgins wrote:
< snip >
> > > > > > > kunit_abort() is what will be call as the result of an assert
> > > > > > > failure.
> > > > > >
> > > > > > Yep. Does that need clarified somewhere.
> > > > > > > BUG(), which is a panic, which is crashing the system is not
> > > > > > > acceptable
> > > > > > > in the Linux kernel. You will just annoy Linus if you submit this.
> > > > > >
> > > > > > Sorry, I thought this was an acceptable use case since, a) this should
> > > > > > never be compiled in a production kernel, b) we are in a pretty bad,
> > > > > > unpredictable state if we get here and keep going. I think you might
> > > > > > have said elsewhere that you think "a" is not valid? In any case, I
> > > > > > can replace this with a WARN, would that be acceptable?
> > > > >
> > > > > A WARN may or may not make sense, depending on the context. It may
> > > > > be sufficient to simply report a test failure (as in the old version
> > > > > of case (2) below.
> > > > >
> > > > > Answers to "a)" and "b)":
> > > > >
> > > > > a) it might be in a production kernel
> > > >
> > > > Sorry for a possibly stupid question, how might it be so? Why would
> > > > someone intentionally build unit tests into a production kernel?
> > >
> > > People do things. Just expect it.
> >
> > Huh, alright. I will take your word for it then.
>
> I have a better explanation: Production kernels have bugs, unfortunately.
> And sometimes those need to be investigated on systems than cannot be
> brought down or affected more than absolutely necessary, maybe via a third party
> doing the execution. A light weight, precise test (well tested ahead :) ) might
> be a way of proving or disproving assumptions that can lead to the development
> and application of a fix.
Sorry, you are not suggesting testing in production are you? To be
clear, I am not concerned about someone using testing, KUnit, or
whatever in a *production-like* environment: that's not what we are
talking about here. My assumption is that no one will deploy tests
into actual production.
>
> IMHO you're confusing "building into" with temporary applying, then removing
> again - like the difference between running a local user space program vs
> installing it under /usr and have it in everyone's PATH.
I don't really see the point of distinguishing between "building into"
and "temporary applying" in this case; that's part of my point. Maybe
it makes sense in whitebox end-to-end testing, but in the case of unit
testing, I don't think so.
>
> > > > > a') it is not acceptable in my development kernel either
>
> I think one of the fundamental properties of a good test framework is that it
> should not require changes to the code under test by itself.
>
Sure, but that has nothing to do with the environment the code/tests
are running in.
< snip >
Cheers
next prev parent reply other threads:[~2019-03-25 22:32 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-14 21:37 [RFC v4 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2019-02-14 21:37 ` [RFC v4 01/17] kunit: test: add KUnit test runner core Brendan Higgins
2019-02-14 21:37 ` [RFC v4 02/17] kunit: test: add test resource management API Brendan Higgins
2019-02-15 21:01 ` Stephen Boyd
2019-02-19 23:24 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 03/17] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2019-02-14 21:37 ` [RFC v4 04/17] kunit: test: add test_stream a std::stream like logger Brendan Higgins
2019-02-14 21:37 ` [RFC v4 05/17] kunit: test: add the concept of expectations Brendan Higgins
2019-02-14 21:37 ` [RFC v4 06/17] kbuild: enable building KUnit Brendan Higgins
2019-02-14 21:37 ` [RFC v4 07/17] kunit: test: add initial tests Brendan Higgins
2019-02-14 21:37 ` [RFC v4 08/17] kunit: test: add support for test abort Brendan Higgins
2019-02-18 19:52 ` Frank Rowand
2019-02-20 3:39 ` Brendan Higgins
2019-02-20 6:44 ` Frank Rowand
2019-02-28 7:42 ` Brendan Higgins
2019-03-22 1:09 ` Frank Rowand
2019-03-22 1:41 ` Brendan Higgins
2019-03-22 7:10 ` Knut Omang
2019-03-25 22:32 ` Brendan Higgins [this message]
2019-03-26 7:44 ` Knut Omang
2019-02-26 20:35 ` Stephen Boyd
2019-02-28 9:03 ` Brendan Higgins
2019-02-28 13:54 ` Dan Carpenter
2019-03-04 22:28 ` Brendan Higgins
[not found] ` <155137694423.260864.2846034318906225490@swboyd.mtv.corp.google.com>
2019-03-04 22:39 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 09/17] kunit: test: add the concept of assertions Brendan Higgins
2019-02-14 21:37 ` [RFC v4 10/17] kunit: test: add test managed resource tests Brendan Higgins
2019-02-15 20:54 ` Stephen Boyd
2019-02-19 23:20 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 11/17] kunit: tool: add Python wrappers for running KUnit tests Brendan Higgins
2019-02-14 21:37 ` [RFC v4 12/17] kunit: defconfig: add defconfigs for building " Brendan Higgins
2019-02-14 21:37 ` [RFC v4 13/17] Documentation: kunit: add documentation for KUnit Brendan Higgins
2019-02-14 21:37 ` [RFC v4 14/17] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2019-02-14 21:37 ` [RFC v4 15/17] of: unittest: migrate tests to run on KUnit Brendan Higgins
2019-02-16 0:24 ` Frank Rowand
2019-02-20 2:24 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 16/17] of: unittest: split out a couple of test cases from unittest Brendan Higgins
2019-03-22 1:14 ` Frank Rowand
2019-03-22 1:45 ` Brendan Higgins
2019-02-14 21:37 ` [RFC v4 17/17] of: unittest: split up some super large test cases Brendan Higgins
2019-03-22 1:16 ` Frank Rowand
2019-03-22 1:45 ` Brendan Higgins
2019-02-18 20:02 ` [RFC v4 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework Frank Rowand
2019-02-20 6:34 ` Brendan Higgins
2019-02-20 6:46 ` Frank Rowand
2019-02-22 20:52 ` Thiago Jung Bauermann
2019-02-28 4:18 ` Brendan Higgins
2019-02-28 4:15 ` Brendan Higgins
2019-03-04 23:01 ` Brendan Higgins
2019-03-22 1:23 ` Frank Rowand
2019-03-25 22:11 ` Brendan Higgins
2019-03-21 1:07 ` Logan Gunthorpe
2019-03-21 5:23 ` Knut Omang
2019-03-21 15:56 ` Logan Gunthorpe
2019-03-21 16:55 ` Brendan Higgins
2019-03-21 19:13 ` Knut Omang
2019-03-21 19:29 ` Logan Gunthorpe
2019-03-21 20:14 ` Knut Omang
2019-03-21 22:07 ` Brendan Higgins
2019-03-21 22:26 ` Logan Gunthorpe
2019-03-21 23:33 ` Brendan Higgins
2019-03-22 1:12 ` Frank Rowand
2019-03-25 22:12 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAFd5g44eqjN-nVCJuoeYFCxwVa5AorWiAnXe-tFCAc11zDgJFA@mail.gmail.com \
--to=brendanhiggins@google.com \
--cc=Alexander.Levin@microsoft.com \
--cc=Tim.Bird@sony.com \
--cc=amir73il@gmail.com \
--cc=brakmo@fb.com \
--cc=dan.carpenter@oracle.com \
--cc=dan.j.williams@intel.com \
--cc=daniel@ffwll.ch \
--cc=devicetree@vger.kernel.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=frowand.list@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=jdike@addtoit.com \
--cc=joe@perches.com \
--cc=joel@jms.id.au \
--cc=julia.lawall@lip6.fr \
--cc=keescook@google.com \
--cc=khilman@baylibre.com \
--cc=kieran.bingham@ideasonboard.com \
--cc=knut.omang@oracle.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-um@lists.infradead.org \
--cc=mcgrof@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=pmladek@suse.com \
--cc=richard@nod.at \
--cc=robh@kernel.org \
--cc=rostedt@goodmis.org \
--cc=shuah@kernel.org \
--cc=wfg@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).