Linux-NVDIMM Archive on lore.kernel.org
 help / color / Atom feed
From: Rob Herring <robh@kernel.org>
To: Brendan Higgins <brendanhiggins@google.com>
Cc: brakmo@fb.com, dri-devel <dri-devel@lists.freedesktop.org>,
	linux-kselftest@vger.kernel.org, shuah@kernel.org,
	Frank Rowand <frowand.list@gmail.com>,
	linux-nvdimm <linux-nvdimm@lists.01.org>,
	Richard Weinberger <richard@nod.at>,
	Knut Omang <knut.omang@oracle.com>,
	Kieran Bingham <kieran.bingham@ideasonboard.com>,
	Joel Stanley <joel@jms.id.au>, Jeff Dike <jdike@addtoit.com>,
	"Bird," Timothy" <Tim.Bird@sony.com>,
	Kees Cook <keescook@google.com>," linux-um@lists.infradead.org,
	Steven Rostedt <rostedt@goodmis.org>,
	Julia Lawall <julia.lawall@lip6.fr>,
	kunit-dev@googlegroups.com,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"Luis R. Rodriguez" <mcgrof@kernel.org>,
	Daniel Vetter <daniel@ffwll.ch>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Joe Perches <joe@perches.com>,
	Kevin Hilman <khilman@baylibre.com>
Subject: Re: [RFC v3 17/19] of: unittest: migrate tests to run on KUnit
Date: Thu, 14 Feb 2019 14:10:17 -0600
Message-ID: <CAL_JsqL=NthWW6q==TXjado83AAWuC1jYZ9CR+T9wPtgQLC4hw@mail.gmail.com> (raw)
In-Reply-To: <CAFd5g46NehKo=TT4reQ8r74bDTG9BkR=ELg+Q3n5YooUxFwBmA@mail.gmail.com>

On Tue, Feb 12, 2019 at 7:44 PM Brendan Higgins
<brendanhiggins@google.com> wrote:
>
> On Wed, Nov 28, 2018 at 12:56 PM Rob Herring <robh@kernel.org> wrote:
> >
> > On Wed, Nov 28, 2018 at 1:38 PM Brendan Higgins
> > <brendanhiggins@google.com> wrote:
> > >
> > > Migrate tests without any cleanup, or modifying test logic in anyway to
> > > run under KUnit using the KUnit expectation and assertion API.
> >
> > Nice! You beat me to it. This is probably going to conflict with what
> > is in the DT tree for 4.21. Also, please Cc the DT list for
> > drivers/of/ changes.
> >
> > Looks good to me, but a few mostly formatting comments below.
>
> I just realized that we never talked about your other comments, and I
> still have some questions. (Sorry, it was the last thing I looked at
> while getting v4 ready.) No worries if you don't get to it before I
> send v4 out, I just didn't want you to think I was ignoring you.
>
> >
> > >
> > > Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
> > > ---
> > >  drivers/of/Kconfig    |    1 +
> > >  drivers/of/unittest.c | 1405 ++++++++++++++++++++++-------------------
> > >  2 files changed, 752 insertions(+), 654 deletions(-)
> > >
> <snip>
> > > diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
> > > index 41b49716ac75f..a5ef44730ffdb 100644
> > > --- a/drivers/of/unittest.c
> > > +++ b/drivers/of/unittest.c
> <snip>
> > > -
> > > -static void __init of_unittest_find_node_by_name(void)
> > > +static void of_unittest_find_node_by_name(struct kunit *test)
> >
> > Why do we have to drop __init everywhere? The tests run later?
>
> From the standpoint of a unit test __init doesn't really make any
> sense, right? I know that right now we are running as part of a
> kernel, but the goal should be that a unit test is not part of a
> kernel and we just include what we need.

Well, the test only runs during boot and better to free the space when
done with it. There was some desire to make it a kernel module and
then we'd also need to get rid of __init too.

> Even so, that's the future. For now, I did not put the KUnit
> infrastructure in the .init section because I didn't think it belonged
> there. In practice, KUnit only knows how to run during the init phase
> of the kernel, but I don't think it should be restricted there. You
> should be able to run tests whenever you want because you should be
> able to test anything right? I figured any restriction on that is
> misleading and will potentially get in the way at worst, and
> unnecessary at best especially since people shouldn't build a
> production kernel with all kinds of unit tests inside.

More folks will run things if they can be enabled on production
kernels. If size is the only issue, modules mitigate that. However,
there's probably APIs to test which we don't want to export to
modules.

I think in general, we change things in the kernel when needed, not
for something in the future. Changing __init is simple enough to do
later.

OTOH, things get copied and maybe this we don't want copied, so we can
remove it if you want to.

> <snip>
> > >
> > > -static void __init of_unittest_property_string(void)
> > > +static void of_unittest_property_string(struct kunit *test)
> > >  {
> > >         const char *strings[4];
> > >         struct device_node *np;
> > >         int rc;
> > >
> > >         np = of_find_node_by_path("/testcase-data/phandle-tests/consumer-a");
> > > -       if (!np) {
> > > -               pr_err("No testcase data in device tree\n");
> > > -               return;
> > > -       }
> > > -
> > > -       rc = of_property_match_string(np, "phandle-list-names", "first");
> > > -       unittest(rc == 0, "first expected:0 got:%i\n", rc);
> > > -       rc = of_property_match_string(np, "phandle-list-names", "second");
> > > -       unittest(rc == 1, "second expected:1 got:%i\n", rc);
> > > -       rc = of_property_match_string(np, "phandle-list-names", "third");
> > > -       unittest(rc == 2, "third expected:2 got:%i\n", rc);
> > > -       rc = of_property_match_string(np, "phandle-list-names", "fourth");
> > > -       unittest(rc == -ENODATA, "unmatched string; rc=%i\n", rc);
> > > -       rc = of_property_match_string(np, "missing-property", "blah");
> > > -       unittest(rc == -EINVAL, "missing property; rc=%i\n", rc);
> > > -       rc = of_property_match_string(np, "empty-property", "blah");
> > > -       unittest(rc == -ENODATA, "empty property; rc=%i\n", rc);
> > > -       rc = of_property_match_string(np, "unterminated-string", "blah");
> > > -       unittest(rc == -EILSEQ, "unterminated string; rc=%i\n", rc);
> > > +       KUNIT_ASSERT_NOT_ERR_OR_NULL(test, np);
> > > +
> > > +       KUNIT_EXPECT_EQ(test,
> > > +                       of_property_match_string(np,
> > > +                                                "phandle-list-names",
> > > +                                                "first"),
> > > +                       0);
> > > +       KUNIT_EXPECT_EQ(test,
> > > +                       of_property_match_string(np,
> > > +                                                "phandle-list-names",
> > > +                                                "second"),
> > > +                       1);
> >
> > Fewer lines on these would be better even if we go over 80 chars.
>
> On the of_property_match_string(...), I have no opinion. I will do
> whatever you like best.
>
> Nevertheless, as far as the KUNIT_EXPECT_*(...), I do have an opinion: I am
> trying to establish a good, readable convention. Given an expect statement
> structured as
> ```
> KUNIT_EXPECT_*(
>     test,
>     expect_arg_0, ..., expect_arg_n,
>     fmt_str, fmt_arg_0, ..., fmt_arg_n)
> ```
> where `test` is the `struct kunit` context argument, `expect_arg_{0, ..., n}`
> are the arguments the expectations is being made about (so in the above example,
> `of_property_match_string(...)` and `1`), and `fmt_*` is the optional format
> string that comes at the end of some expectations.
>
> The pattern I had been trying to promote is the following:
>
> 1) If everything fits on 1 line, do that.
> 2) If you must make a line split, prefer to keep `test` on its own line,
> `expect_arg_{0, ..., n}` should be kept together, if possible, and the format
> string should follow the conventions already most commonly used with format
> strings.
> 3) If you must split up `expect_arg_{0, ..., n}` each argument should get its
> own line and should not share a line with either `test` or any `fmt_*`.

You'd better write a checkpatch.pl check or else good luck enforcing that. :)

> The reason I care about this so much is because expectations should be
> extremely easy to read; they are the most important part of a unit
> test because they tell you what the test is verifying. I am not
> married to the formatting I proposed above, but I want something that
> will be extremely easy to identify the arguments that the expectation
> is on. Maybe that means that I need to add some syntactic fluff to
> make it clearer, I don't know, but this is definitely something we
> need to get right, especially in the earliest examples.

Makes sense. I think putting the test (of_property_match_string) on
one line furthers the readability.

Rob
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

  reply index

Thread overview: 118+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-28 19:36 [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2018-11-28 19:36 ` [RFC v3 01/19] kunit: test: add KUnit test runner core Brendan Higgins
2018-11-30  3:14   ` Luis Chamberlain
2018-12-01  1:51     ` Brendan Higgins
2018-12-01  2:57       ` Luis Chamberlain
2018-12-05 13:15     ` Anton Ivanov
2018-12-05 14:45       ` Arnd Bergmann
2018-12-05 14:49         ` Anton Ivanov
2018-11-30  3:28   ` Luis Chamberlain
     [not found]     ` <20181130032802.GG18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-01  2:08       ` Brendan Higgins
2018-12-01  3:10         ` Luis Chamberlain
     [not found]           ` <20181201031049.GL28501-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 22:47             ` Brendan Higgins
2018-12-01  3:02   ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 10/19] kunit: test: add test managed resource tests Brendan Higgins
     [not found] ` <20181128193636.254378-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2018-11-28 19:36   ` [RFC v3 02/19] kunit: test: add test resource management API Brendan Higgins
2018-11-28 19:36   ` [RFC v3 03/19] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2018-11-30  3:29     ` Luis Chamberlain
2018-12-01  2:14       ` Brendan Higgins
2018-12-01  3:12         ` Luis Chamberlain
2018-12-03 10:55       ` Petr Mladek
2018-12-04  0:35         ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 04/19] kunit: test: add test_stream a std::stream like logger Brendan Higgins
2018-11-28 19:36   ` [RFC v3 05/19] kunit: test: add the concept of expectations Brendan Higgins
2018-11-28 19:36   ` [RFC v3 06/19] arch: um: enable running kunit from User Mode Linux Brendan Higgins
2018-11-28 21:26     ` Rob Herring
2018-11-30  3:37       ` Luis Chamberlain
2018-11-30 14:05         ` Rob Herring
2018-11-30 18:22           ` Luis Chamberlain
     [not found]             ` <20181130182203.GS18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:22               ` Brendan Higgins
2018-11-30  3:30     ` Luis Chamberlain
2018-11-28 19:36   ` [RFC v3 07/19] kunit: test: add initial tests Brendan Higgins
2018-11-30  3:40     ` Luis Chamberlain
2018-12-03 23:26       ` Brendan Higgins
2018-12-03 23:43         ` Luis Chamberlain
2018-11-28 19:36   ` [RFC v3 08/19] arch: um: add shim to trap to allow installing a fault catcher for tests Brendan Higgins
2018-11-30  3:34     ` Luis Chamberlain
     [not found]       ` <20181130033429.GK18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:34         ` Brendan Higgins
     [not found]           ` <CAFd5g45+MAVaSW8HN9x57Y8Um=TV1Oa=-K8yExPBS-4KjLyciQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-03 23:46             ` Luis Chamberlain
     [not found]               ` <20181203234628.GR28501-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-04  0:44                 ` Brendan Higgins
2018-11-30  3:41     ` Luis Chamberlain
2018-12-03 23:37       ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 09/19] kunit: test: add the concept of assertions Brendan Higgins
2018-11-28 19:36   ` [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel Brendan Higgins
2018-11-29 13:54     ` Kieran Bingham
     [not found]       ` <841cf4ae-501b-05ae-5863-a51010709b67-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
2018-12-03 23:48         ` Brendan Higgins
2018-12-04 20:47           ` Luis Chamberlain
2018-12-06 12:32             ` Kieran Bingham
2018-12-06 15:37               ` Matthew Wilcox
2018-12-07 11:30                 ` Kieran Bingham
2018-12-11 14:09                 ` Petr Mladek
2018-12-11 14:41                   ` Steven Rostedt
2018-12-11 17:01                     ` Anton Ivanov
2019-02-09  0:40                       ` Brendan Higgins
2018-12-07  1:05               ` Luis Chamberlain
2018-12-07 18:35               ` Kent Overstreet
2018-11-30  3:44     ` Luis Chamberlain
2018-12-03 23:50       ` Brendan Higgins
2018-12-04 20:48         ` Luis Chamberlain
2018-11-28 19:36   ` [RFC v3 12/19] kunit: add KUnit wrapper script and simple output parser Brendan Higgins
2018-11-28 19:36   ` [RFC v3 13/19] kunit: improve output from python wrapper Brendan Higgins
2018-11-28 19:36   ` [RFC v3 14/19] Documentation: kunit: add documentation for KUnit Brendan Higgins
2018-11-29 13:56     ` Kieran Bingham
2018-11-30  3:45       ` Luis Chamberlain
     [not found]         ` <20181130034525.GP18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:53           ` Brendan Higgins
2018-12-06 12:16             ` Kieran Bingham
2019-02-09  0:56               ` Brendan Higgins
2019-02-11 12:16                 ` Kieran Bingham
2019-02-12 22:10                   ` Brendan Higgins
2019-02-13 21:55                     ` Kieran Bingham
2019-02-14  0:17                       ` Brendan Higgins
2019-02-14 17:26                         ` Luis Chamberlain
2019-02-14 22:07                           ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 15/19] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2018-11-28 19:36   ` [RFC v3 17/19] of: unittest: migrate tests to run on KUnit Brendan Higgins
2018-11-28 20:56     ` Rob Herring
2018-11-30  0:39       ` Randy Dunlap
     [not found]         ` <18814973-8f0a-4647-a097-fcc3dc0b3cd3-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2018-12-04  0:13           ` Brendan Higgins
2018-12-04 13:40             ` Rob Herring
     [not found]               ` <CAL_JsqL_PivQbrJFEusdKAy-2EQtKL3OHbmyYSK9bzuTOQegqA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-05 23:42                 ` Brendan Higgins
2018-12-07  0:41                   ` Rob Herring
2018-12-04  0:08       ` Brendan Higgins
2019-02-13  1:44       ` Brendan Higgins
2019-02-14 20:10         ` Rob Herring [this message]
2019-02-14 21:52           ` Brendan Higgins
2019-02-18 22:56         ` Frank Rowand
2019-02-28  0:29           ` Brendan Higgins
2018-12-04 10:56     ` Frank Rowand
2018-11-28 19:36   ` [RFC v3 18/19] of: unittest: split out a couple of test cases from unittest Brendan Higgins
2018-12-04 10:58     ` Frank Rowand
2018-12-05 23:54       ` Brendan Higgins
2019-02-14 23:57         ` Frank Rowand
2019-02-15  0:56           ` Brendan Higgins
2019-02-15  2:05             ` Frank Rowand
2019-02-15 10:56               ` Brendan Higgins
2019-02-18 22:25                 ` Frank Rowand
2019-02-20 20:44                   ` Frank Rowand
2019-02-20 20:47                     ` Frank Rowand
2019-02-28  3:52                     ` Brendan Higgins
2019-03-22  0:22                       ` Frank Rowand
2019-03-22  1:30                         ` Brendan Higgins
2019-03-22  1:47                           ` Frank Rowand
2019-03-25 22:15                             ` Brendan Higgins
2019-09-20 16:57                           ` Rob Herring
2019-09-21 23:57                             ` Frank Rowand
2019-03-22  1:34                         ` Frank Rowand
2019-03-25 22:18                           ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 19/19] of: unittest: split up some super large test cases Brendan Higgins
2018-11-28 19:36 ` [RFC v3 16/19] arch: um: make UML unflatten device tree when testing Brendan Higgins
2018-11-28 21:16   ` Rob Herring
     [not found]     ` <CAL_JsqK5cG=QzMBFSZ31_-3ujnxqxv=jj3XYajbRLT7yWYZbfw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-04  0:00       ` Brendan Higgins
2018-11-30  3:46   ` Luis Chamberlain
2018-12-04  0:02     ` Brendan Higgins
2018-12-04 10:52 ` [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Frank Rowand
2018-12-04 11:40 ` Frank Rowand
2018-12-04 13:49   ` Rob Herring
2018-12-05 23:10     ` Brendan Higgins
2019-03-22  0:27       ` Frank Rowand
2019-03-25 22:04         ` Brendan Higgins

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAL_JsqL=NthWW6q==TXjado83AAWuC1jYZ9CR+T9wPtgQLC4hw@mail.gmail.com' \
    --to=robh@kernel.org \
    --cc=Tim.Bird@sony.com \
    --cc=brakmo@fb.com \
    --cc=brendanhiggins@google.com \
    --cc=daniel@ffwll.ch \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jdike@addtoit.com \
    --cc=joe@perches.com \
    --cc=joel@jms.id.au \
    --cc=julia.lawall@lip6.fr \
    --cc=keescook@google.com \
    --cc=khilman@baylibre.com \
    --cc=kieran.bingham@ideasonboard.com \
    --cc=knut.omang@oracle.com \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-nvdimm@lists.01.org \
    --cc=linux-um@lists.infradead.org \
    --cc=mcgrof@kernel.org \
    --cc=mpe@ellerman.id.au \
    --cc=richard@nod.at \
    --cc=rostedt@goodmis.org \
    --cc=shuah@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NVDIMM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nvdimm/0 linux-nvdimm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nvdimm linux-nvdimm/ https://lore.kernel.org/linux-nvdimm \
		linux-nvdimm@lists.01.org
	public-inbox-index linux-nvdimm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.01.lists.linux-nvdimm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git