From: Brendan Higgins <brendanhiggins@google.com>
To: Rob Herring <robh@kernel.org>
Cc: brakmo@fb.com, dri-devel <dri-devel@lists.freedesktop.org>,
linux-kselftest@vger.kernel.org, shuah@kernel.org,
Frank Rowand <frowand.list@gmail.com>,
linux-nvdimm <linux-nvdimm@lists.01.org>,
Richard Weinberger <richard@nod.at>,
Knut Omang <knut.omang@oracle.com>,
Kieran Bingham <kieran.bingham@ideasonboard.com>,
Joel Stanley <joel@jms.id.au>, Jeff Dike <jdike@addtoit.com>,
"Bird," Timothy" <Tim.Bird@sony.com>,
Kees Cook <keescook@google.com>," linux-um@lists.infradead.org,
Steven Rostedt <rostedt@goodmis.org>,
Julia Lawall <julia.lawall@lip6.fr>,
kunit-dev@googlegroups.com,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"Luis R. Rodriguez" <mcgrof@kernel.org>,
Daniel Vetter <daniel@ffwll.ch>,
Michael Ellerman <mpe@ellerman.id.au>,
Joe Perches <joe@perches.com>,
Kevin Hilman <khilman@baylibre.com>
Subject: Re: [RFC v3 17/19] of: unittest: migrate tests to run on KUnit
Date: Tue, 12 Feb 2019 17:44:23 -0800 [thread overview]
Message-ID: <CAFd5g46NehKo=TT4reQ8r74bDTG9BkR=ELg+Q3n5YooUxFwBmA@mail.gmail.com> (raw)
In-Reply-To: <CAL_Jsq+09Kx7yMBC_Jw45QGmk6U_fp4N6HOZDwYrM4tWw+_dOA@mail.gmail.com>
On Wed, Nov 28, 2018 at 12:56 PM Rob Herring <robh@kernel.org> wrote:
>
> On Wed, Nov 28, 2018 at 1:38 PM Brendan Higgins
> <brendanhiggins@google.com> wrote:
> >
> > Migrate tests without any cleanup, or modifying test logic in anyway to
> > run under KUnit using the KUnit expectation and assertion API.
>
> Nice! You beat me to it. This is probably going to conflict with what
> is in the DT tree for 4.21. Also, please Cc the DT list for
> drivers/of/ changes.
>
> Looks good to me, but a few mostly formatting comments below.
I just realized that we never talked about your other comments, and I
still have some questions. (Sorry, it was the last thing I looked at
while getting v4 ready.) No worries if you don't get to it before I
send v4 out, I just didn't want you to think I was ignoring you.
>
> >
> > Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
> > ---
> > drivers/of/Kconfig | 1 +
> > drivers/of/unittest.c | 1405 ++++++++++++++++++++++-------------------
> > 2 files changed, 752 insertions(+), 654 deletions(-)
> >
<snip>
> > diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
> > index 41b49716ac75f..a5ef44730ffdb 100644
> > --- a/drivers/of/unittest.c
> > +++ b/drivers/of/unittest.c
<snip>
> > -
> > -static void __init of_unittest_find_node_by_name(void)
> > +static void of_unittest_find_node_by_name(struct kunit *test)
>
> Why do we have to drop __init everywhere? The tests run later?
>From the standpoint of a unit test __init doesn't really make any
sense, right? I know that right now we are running as part of a
kernel, but the goal should be that a unit test is not part of a
kernel and we just include what we need.
Even so, that's the future. For now, I did not put the KUnit
infrastructure in the .init section because I didn't think it belonged
there. In practice, KUnit only knows how to run during the init phase
of the kernel, but I don't think it should be restricted there. You
should be able to run tests whenever you want because you should be
able to test anything right? I figured any restriction on that is
misleading and will potentially get in the way at worst, and
unnecessary at best especially since people shouldn't build a
production kernel with all kinds of unit tests inside.
>
> > {
> > struct device_node *np;
> > const char *options, *name;
> >
<snip>
> >
> >
> > - np = of_find_node_by_path("/testcase-data/missing-path");
> > - unittest(!np, "non-existent path returned node %pOF\n", np);
> > + KUNIT_EXPECT_EQ_MSG(test,
> > + of_find_node_by_path("/testcase-data/missing-path"),
> > + NULL,
> > + "non-existent path returned node %pOF\n", np);
>
> 1 tab indent would help with less vertical code (in general, not this
> one so much).
Will do.
>
> > of_node_put(np);
> >
> > - np = of_find_node_by_path("missing-alias");
> > - unittest(!np, "non-existent alias returned node %pOF\n", np);
> > + KUNIT_EXPECT_EQ_MSG(test, of_find_node_by_path("missing-alias"), NULL,
> > + "non-existent alias returned node %pOF\n", np);
> > of_node_put(np);
> >
> > - np = of_find_node_by_path("testcase-alias/missing-path");
> > - unittest(!np, "non-existent alias with relative path returned node %pOF\n", np);
> > + KUNIT_EXPECT_EQ_MSG(test,
> > + of_find_node_by_path("testcase-alias/missing-path"),
> > + NULL,
> > + "non-existent alias with relative path returned node %pOF\n",
> > + np);
> > of_node_put(np);
> >
<snip>
> >
> > -static void __init of_unittest_property_string(void)
> > +static void of_unittest_property_string(struct kunit *test)
> > {
> > const char *strings[4];
> > struct device_node *np;
> > int rc;
> >
> > np = of_find_node_by_path("/testcase-data/phandle-tests/consumer-a");
> > - if (!np) {
> > - pr_err("No testcase data in device tree\n");
> > - return;
> > - }
> > -
> > - rc = of_property_match_string(np, "phandle-list-names", "first");
> > - unittest(rc == 0, "first expected:0 got:%i\n", rc);
> > - rc = of_property_match_string(np, "phandle-list-names", "second");
> > - unittest(rc == 1, "second expected:1 got:%i\n", rc);
> > - rc = of_property_match_string(np, "phandle-list-names", "third");
> > - unittest(rc == 2, "third expected:2 got:%i\n", rc);
> > - rc = of_property_match_string(np, "phandle-list-names", "fourth");
> > - unittest(rc == -ENODATA, "unmatched string; rc=%i\n", rc);
> > - rc = of_property_match_string(np, "missing-property", "blah");
> > - unittest(rc == -EINVAL, "missing property; rc=%i\n", rc);
> > - rc = of_property_match_string(np, "empty-property", "blah");
> > - unittest(rc == -ENODATA, "empty property; rc=%i\n", rc);
> > - rc = of_property_match_string(np, "unterminated-string", "blah");
> > - unittest(rc == -EILSEQ, "unterminated string; rc=%i\n", rc);
> > + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, np);
> > +
> > + KUNIT_EXPECT_EQ(test,
> > + of_property_match_string(np,
> > + "phandle-list-names",
> > + "first"),
> > + 0);
> > + KUNIT_EXPECT_EQ(test,
> > + of_property_match_string(np,
> > + "phandle-list-names",
> > + "second"),
> > + 1);
>
> Fewer lines on these would be better even if we go over 80 chars.
On the of_property_match_string(...), I have no opinion. I will do
whatever you like best.
Nevertheless, as far as the KUNIT_EXPECT_*(...), I do have an opinion: I am
trying to establish a good, readable convention. Given an expect statement
structured as
```
KUNIT_EXPECT_*(
test,
expect_arg_0, ..., expect_arg_n,
fmt_str, fmt_arg_0, ..., fmt_arg_n)
```
where `test` is the `struct kunit` context argument, `expect_arg_{0, ..., n}`
are the arguments the expectations is being made about (so in the above example,
`of_property_match_string(...)` and `1`), and `fmt_*` is the optional format
string that comes at the end of some expectations.
The pattern I had been trying to promote is the following:
1) If everything fits on 1 line, do that.
2) If you must make a line split, prefer to keep `test` on its own line,
`expect_arg_{0, ..., n}` should be kept together, if possible, and the format
string should follow the conventions already most commonly used with format
strings.
3) If you must split up `expect_arg_{0, ..., n}` each argument should get its
own line and should not share a line with either `test` or any `fmt_*`.
The reason I care about this so much is because expectations should be
extremely easy to read; they are the most important part of a unit
test because they tell you what the test is verifying. I am not
married to the formatting I proposed above, but I want something that
will be extremely easy to identify the arguments that the expectation
is on. Maybe that means that I need to add some syntactic fluff to
make it clearer, I don't know, but this is definitely something we
need to get right, especially in the earliest examples.
>
> > + KUNIT_EXPECT_EQ(test,
> > + of_property_match_string(np,
> > + "phandle-list-names",
> > + "third"),
> > + 2);
> > + KUNIT_EXPECT_EQ_MSG(test,
> > + of_property_match_string(np,
> > + "phandle-list-names",
> > + "fourth"),
> > + -ENODATA,
> > + "unmatched string");
> > + KUNIT_EXPECT_EQ_MSG(test,
> > + of_property_match_string(np,
> > + "missing-property",
> > + "blah"),
> > + -EINVAL,
> > + "missing property");
> > + KUNIT_EXPECT_EQ_MSG(test,
> > + of_property_match_string(np,
> > + "empty-property",
> > + "blah"),
> > + -ENODATA,
> > + "empty property");
> > + KUNIT_EXPECT_EQ_MSG(test,
> > + of_property_match_string(np,
> > + "unterminated-string",
> > + "blah"),
> > + -EILSEQ,
> > + "unterminated string");
<snip>
> > /* test insertion of a bus with parent devices */
> > -static void __init of_unittest_overlay_10(void)
> > +static void of_unittest_overlay_10(struct kunit *test)
> > {
> > - int ret;
> > char *child_path;
> >
> > /* device should disable */
> > - ret = of_unittest_apply_overlay_check(10, 10, 0, 1, PDEV_OVERLAY);
> > - if (unittest(ret == 0,
> > - "overlay test %d failed; overlay application\n", 10))
> > - return;
> > + KUNIT_ASSERT_EQ_MSG(test,
> > + of_unittest_apply_overlay_check(test,
> > + 10,
> > + 10,
> > + 0,
> > + 1,
> > + PDEV_OVERLAY),
>
> I prefer putting multiple args on a line and having fewer lines.
Looking at this now, I tend to agree, but I don't think I saw a
consistent way to break them up for these functions. I figured there
should be some type of pattern.
>
> > + 0,
> > + "overlay test %d failed; overlay application\n",
> > + 10);
> >
> > child_path = kasprintf(GFP_KERNEL, "%s/test-unittest101",
> > unittest_path(10, PDEV_OVERLAY));
> > - if (unittest(child_path, "overlay test %d failed; kasprintf\n", 10))
> > - return;
> > + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, child_path);
> >
> > - ret = of_path_device_type_exists(child_path, PDEV_OVERLAY);
> > + KUNIT_EXPECT_TRUE_MSG(test,
> > + of_path_device_type_exists(child_path,
> > + PDEV_OVERLAY),
> > + "overlay test %d failed; no child device\n", 10);
> > kfree(child_path);
> > -
> > - unittest(ret, "overlay test %d failed; no child device\n", 10);
> > }
<snip>
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
next prev parent reply other threads:[~2019-02-13 1:44 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-28 19:36 [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2018-11-28 19:36 ` [RFC v3 01/19] kunit: test: add KUnit test runner core Brendan Higgins
2018-11-30 3:14 ` Luis Chamberlain
2018-12-01 1:51 ` Brendan Higgins
2018-12-01 2:57 ` Luis Chamberlain
2018-12-05 13:15 ` Anton Ivanov
2018-12-05 14:45 ` Arnd Bergmann
2018-12-05 14:49 ` Anton Ivanov
2018-11-30 3:28 ` Luis Chamberlain
[not found] ` <20181130032802.GG18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-01 2:08 ` Brendan Higgins
2018-12-01 3:10 ` Luis Chamberlain
[not found] ` <20181201031049.GL28501-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 22:47 ` Brendan Higgins
2018-12-01 3:02 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 10/19] kunit: test: add test managed resource tests Brendan Higgins
[not found] ` <20181128193636.254378-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2018-11-28 19:36 ` [RFC v3 02/19] kunit: test: add test resource management API Brendan Higgins
2018-11-28 19:36 ` [RFC v3 03/19] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2018-11-30 3:29 ` Luis Chamberlain
2018-12-01 2:14 ` Brendan Higgins
2018-12-01 3:12 ` Luis Chamberlain
2018-12-03 10:55 ` Petr Mladek
2018-12-04 0:35 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 04/19] kunit: test: add test_stream a std::stream like logger Brendan Higgins
2018-11-28 19:36 ` [RFC v3 05/19] kunit: test: add the concept of expectations Brendan Higgins
2018-11-28 19:36 ` [RFC v3 06/19] arch: um: enable running kunit from User Mode Linux Brendan Higgins
2018-11-28 21:26 ` Rob Herring
2018-11-30 3:37 ` Luis Chamberlain
2018-11-30 14:05 ` Rob Herring
2018-11-30 18:22 ` Luis Chamberlain
[not found] ` <20181130182203.GS18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:22 ` Brendan Higgins
2018-11-30 3:30 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 07/19] kunit: test: add initial tests Brendan Higgins
2018-11-30 3:40 ` Luis Chamberlain
2018-12-03 23:26 ` Brendan Higgins
2018-12-03 23:43 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 08/19] arch: um: add shim to trap to allow installing a fault catcher for tests Brendan Higgins
2018-11-30 3:34 ` Luis Chamberlain
[not found] ` <20181130033429.GK18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:34 ` Brendan Higgins
[not found] ` <CAFd5g45+MAVaSW8HN9x57Y8Um=TV1Oa=-K8yExPBS-4KjLyciQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-03 23:46 ` Luis Chamberlain
[not found] ` <20181203234628.GR28501-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-04 0:44 ` Brendan Higgins
2018-11-30 3:41 ` Luis Chamberlain
2018-12-03 23:37 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 09/19] kunit: test: add the concept of assertions Brendan Higgins
2018-11-28 19:36 ` [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel Brendan Higgins
2018-11-29 13:54 ` Kieran Bingham
[not found] ` <841cf4ae-501b-05ae-5863-a51010709b67-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
2018-12-03 23:48 ` Brendan Higgins
2018-12-04 20:47 ` Luis Chamberlain
2018-12-06 12:32 ` Kieran Bingham
2018-12-06 15:37 ` Matthew Wilcox
2018-12-07 11:30 ` Kieran Bingham
2018-12-11 14:09 ` Petr Mladek
2018-12-11 14:41 ` Steven Rostedt
2018-12-11 17:01 ` Anton Ivanov
2019-02-09 0:40 ` Brendan Higgins
2018-12-07 1:05 ` Luis Chamberlain
2018-12-07 18:35 ` Kent Overstreet
2018-11-30 3:44 ` Luis Chamberlain
2018-12-03 23:50 ` Brendan Higgins
2018-12-04 20:48 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 12/19] kunit: add KUnit wrapper script and simple output parser Brendan Higgins
2018-11-28 19:36 ` [RFC v3 13/19] kunit: improve output from python wrapper Brendan Higgins
2018-11-28 19:36 ` [RFC v3 14/19] Documentation: kunit: add documentation for KUnit Brendan Higgins
2018-11-29 13:56 ` Kieran Bingham
2018-11-30 3:45 ` Luis Chamberlain
[not found] ` <20181130034525.GP18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:53 ` Brendan Higgins
2018-12-06 12:16 ` Kieran Bingham
2019-02-09 0:56 ` Brendan Higgins
2019-02-11 12:16 ` Kieran Bingham
2019-02-12 22:10 ` Brendan Higgins
2019-02-13 21:55 ` Kieran Bingham
2019-02-14 0:17 ` Brendan Higgins
2019-02-14 17:26 ` Luis Chamberlain
2019-02-14 22:07 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 15/19] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2018-11-28 19:36 ` [RFC v3 17/19] of: unittest: migrate tests to run on KUnit Brendan Higgins
2018-11-28 20:56 ` Rob Herring
2018-11-30 0:39 ` Randy Dunlap
[not found] ` <18814973-8f0a-4647-a097-fcc3dc0b3cd3-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2018-12-04 0:13 ` Brendan Higgins
2018-12-04 13:40 ` Rob Herring
[not found] ` <CAL_JsqL_PivQbrJFEusdKAy-2EQtKL3OHbmyYSK9bzuTOQegqA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-05 23:42 ` Brendan Higgins
2018-12-07 0:41 ` Rob Herring
2018-12-04 0:08 ` Brendan Higgins
2019-02-13 1:44 ` Brendan Higgins [this message]
2019-02-14 20:10 ` Rob Herring
2019-02-14 21:52 ` Brendan Higgins
2019-02-18 22:56 ` Frank Rowand
2019-02-28 0:29 ` Brendan Higgins
2018-12-04 10:56 ` Frank Rowand
2018-11-28 19:36 ` [RFC v3 18/19] of: unittest: split out a couple of test cases from unittest Brendan Higgins
2018-12-04 10:58 ` Frank Rowand
2018-12-05 23:54 ` Brendan Higgins
2019-02-14 23:57 ` Frank Rowand
2019-02-15 0:56 ` Brendan Higgins
2019-02-15 2:05 ` Frank Rowand
2019-02-15 10:56 ` Brendan Higgins
2019-02-18 22:25 ` Frank Rowand
2019-02-20 20:44 ` Frank Rowand
2019-02-20 20:47 ` Frank Rowand
2019-02-28 3:52 ` Brendan Higgins
2019-03-22 0:22 ` Frank Rowand
2019-03-22 1:30 ` Brendan Higgins
2019-03-22 1:47 ` Frank Rowand
2019-03-25 22:15 ` Brendan Higgins
2019-09-20 16:57 ` Rob Herring
2019-09-21 23:57 ` Frank Rowand
2019-03-22 1:34 ` Frank Rowand
2019-03-25 22:18 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 19/19] of: unittest: split up some super large test cases Brendan Higgins
2018-11-28 19:36 ` [RFC v3 16/19] arch: um: make UML unflatten device tree when testing Brendan Higgins
2018-11-28 21:16 ` Rob Herring
[not found] ` <CAL_JsqK5cG=QzMBFSZ31_-3ujnxqxv=jj3XYajbRLT7yWYZbfw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-04 0:00 ` Brendan Higgins
2018-11-30 3:46 ` Luis Chamberlain
2018-12-04 0:02 ` Brendan Higgins
2018-12-04 10:52 ` [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Frank Rowand
2018-12-04 11:40 ` Frank Rowand
2018-12-04 13:49 ` Rob Herring
2018-12-05 23:10 ` Brendan Higgins
2019-03-22 0:27 ` Frank Rowand
2019-03-25 22:04 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFd5g46NehKo=TT4reQ8r74bDTG9BkR=ELg+Q3n5YooUxFwBmA@mail.gmail.com' \
--to=brendanhiggins@google.com \
--cc=brakmo@fb.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=frowand.list@gmail.com \
--cc=jdike@addtoit.com \
--cc=joel@jms.id.au \
--cc=kieran.bingham@ideasonboard.com \
--cc=knut.omang@oracle.com \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=richard@nod.at \
--cc=robh@kernel.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).