From: Kieran Bingham <kieran.bingham@ideasonboard.com>
To: Brendan Higgins <brendanhiggins@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>,
Greg KH <gregkh@linuxfoundation.org>,
Kees Cook <keescook@google.com>,
shuah@kernel.org, Joel Stanley <joel@jms.id.au>,
mpe@ellerman.id.au, joe@perches.com, brakmo@fb.com,
Steven Rostedt <rostedt@goodmis.org>,
Tim.Bird@sony.com, khilman@baylibre.com,
Julia Lawall <julia.lawall@lip6.fr>,
linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
jdike@addtoit.com, richard@nod.at, linux-um@lists.infradead.org,
Daniel Vetter <daniel@ffwll.ch>,
dri-devel@lists.freedesktop.org, Rob Herring <robh@kernel.org>,
dan.j.williams@intel.com, linux-nvdimm@lists.01.org,
Frank Rowand <frowand.list@gmail.com>,
Knut Omang <knut.omang@oracle.com>,
Felix Guo <felixguoxiuping@gmail.com>
Subject: Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit
Date: Wed, 13 Feb 2019 21:55:39 +0000 [thread overview]
Message-ID: <eade3052-0aa4-90b4-d55b-8c44556f98bc@ideasonboard.com> (raw)
In-Reply-To: <CAFd5g46AEquMKzfrjLDVi+PP5-7aGs6C6pCunGAXDn3VRkJP+g@mail.gmail.com>
Hi Brendan,
On 12/02/2019 22:10, Brendan Higgins wrote:
> On Mon, Feb 11, 2019 at 4:16 AM Kieran Bingham
> <kieran.bingham@ideasonboard.com> wrote:
>>
>> Hi Brendan,
>>
>> On 09/02/2019 00:56, Brendan Higgins wrote:
>>> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
>>> <kieran.bingham@ideasonboard.com> wrote:
>>>>
>>>> Hi Brendan,
>>>>
>>>> On 03/12/2018 23:53, Brendan Higgins wrote:
>>>>> On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
>>>>>>
>>>>>> On Thu, Nov 29, 2018 at 01:56:37PM +0000, Kieran Bingham wrote:
>>>>>>> Hi Brendan,
>>>>>>>
>>>>>>> Please excuse the top posting, but I'm replying here as I'm following
>>>>>>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
>>>>>>>
>>>>>>> Could the three line kunitconfig file live under say
>>>>>>> arch/um/configs/kunit_defconfig?
>>>>
>>>>
>>>> Further consideration to this topic - I mentioned putting it in
>>>> arch/um/configs
>>>>
>>>> - but I think this is wrong.
>>>>
>>>> We now have a location for config-fragments, which is essentially what
>>>> this is, under kernel/configs
>>>>
>>>> So perhaps an addition as :
>>>>
>>>> kernel/configs/kunit.config
>>>>
>>>> Would be more appropriate - and less (UM) architecture specific.
>>>
>>> Sorry for the long radio silence.
>>>
>>> I just got around to doing this and I found that there are some
>>> configs that are desirable to have when running KUnit under x86 in a
>>> VM, but not UML.
>>
>> Should this behaviour you mention be handled by the KCONFIG depends flags?
>>
>> depends on (KUMIT & UML)
>> or
>> depends on (KUNIT & !UML)
>>
>> or such?
>
> Not really. Anything that is strictly necessary to run KUnit on an
> architectures should of course be turned on as a dependency like you
> suggest, but I am talking about stuff that you would probably want to
> get yourself going, but is by no means necessary.
>
>>
>> An example of which configs you are referring to would help to
>> understand the issue perhaps.
>>
>
> For example, you might want to enable a serial console that is known
> to work with a fairly generic qemu setup when building for x86:
> CONFIG_SERIAL_8250=y
> CONFIG_SERIAL_8250_CONSOLE=y
>
> Obviously not a dependency, and not even particularly useful to people
> who know what they are doing, but to someone who is new or just wants
> something to work out of the box would probably want that.
It sounds like that would be a config fragment for qemu ?
Although - perhaps this is already covered by the following fragment:
kernel/configs/kvm_guest.config
>>> So should we have one that goes in with
>>> config-fragments and others that go into architectures? Another idea,
>>> it would be nice to have a KUnit config that runs all known tests
>>
>> This might also be a config option added to the tests directly like
>> COMPILE_TEST perhaps?
>
> That just allows a bunch of drivers to be compiled, it does not
> actually go through and turn the configs on, right? I mean, there is
> no a priori way to know that there is a configuration which spans all
> possible options available under COMPILE_TEST, right? Maybe I
> misunderstand what you are suggesting...
Bah - you're right of course. I was mis-remembering the functionality of
COMPILE_TEST as if it were some sort of 'select' but it's just an enable..
Sorry for the confusion.
>> (Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)
>>
>> I think that might be more maintainable as otherwise each new test would
>> have to modify the {min,def}{config,fragment} ...
>>
>
> Looking at kselftest-merge, they just start out with a set of
> fragments in which the union should contain all tests and then merge
> it with a base .config (probably intended to be $(ARCH)_defconfig).
> However, I don't know if that is the state of the art.
>
>>
>>> (this probably won't work in practice once we start testing mutually
>>> exclusive things or things with lots of ifdeffery, but it probably
>>> something we should try to maintain as best as we can?); this probably
>>> shouldn't go in with the fragments, right?
>>
>> Sounds like we agree there :)
>
> Totally. Long term we will need something a lot more sophisticated
> than anything under discussion here. I was talking about this with
> Luis on another thread:
> https://groups.google.com/forum/#!topic/kunit-dev/EQ1x0SzrUus (feel
> free to chime in!). Nevertheless, that's a really hard problem and I
> figure some variant of defconfigs and config fragments will work well
> enough until we reach that point.
>
>>
>>>
>>> I will be sending another revision out soon, but I figured I might be
>>> able to catch you before I did so.
>>
>> Thanks for thinking of me.
>
> How can I forget? You have been super helpful!
>
>> I hope I managed to reply in time to help and not hinder your progress.
>
> Yep, no trouble at all. You are the one helping me :-)
>
> Thanks!
>
--
Regards
--
Kieran
next prev parent reply other threads:[~2019-02-13 21:55 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-28 19:36 [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2018-11-28 19:36 ` [RFC v3 01/19] kunit: test: add KUnit test runner core Brendan Higgins
2018-11-30 3:14 ` Luis Chamberlain
2018-12-01 1:51 ` Brendan Higgins
2018-12-01 2:57 ` Luis Chamberlain
2018-12-05 13:15 ` Anton Ivanov
2018-12-05 14:45 ` Arnd Bergmann
2018-12-05 14:49 ` Anton Ivanov
2018-11-30 3:28 ` Luis Chamberlain
2018-12-01 2:08 ` Brendan Higgins
2018-12-01 3:10 ` Luis Chamberlain
2018-12-03 22:47 ` Brendan Higgins
2018-12-01 3:02 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 02/19] kunit: test: add test resource management API Brendan Higgins
2018-11-28 19:36 ` [RFC v3 03/19] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2018-11-30 3:29 ` Luis Chamberlain
2018-12-01 2:14 ` Brendan Higgins
2018-12-01 3:12 ` Luis Chamberlain
2018-12-03 10:55 ` Petr Mladek
2018-12-04 0:35 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 04/19] kunit: test: add test_stream a std::stream like logger Brendan Higgins
2018-11-28 19:36 ` [RFC v3 05/19] kunit: test: add the concept of expectations Brendan Higgins
2018-11-28 19:36 ` [RFC v3 06/19] arch: um: enable running kunit from User Mode Linux Brendan Higgins
2018-11-28 21:26 ` Rob Herring
2018-11-30 3:37 ` Luis Chamberlain
2018-11-30 14:05 ` Rob Herring
2018-11-30 18:22 ` Luis Chamberlain
2018-12-03 23:22 ` Brendan Higgins
2018-11-30 3:30 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 07/19] kunit: test: add initial tests Brendan Higgins
2018-11-30 3:40 ` Luis Chamberlain
2018-12-03 23:26 ` Brendan Higgins
2018-12-03 23:43 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 08/19] arch: um: add shim to trap to allow installing a fault catcher for tests Brendan Higgins
2018-11-30 3:34 ` Luis Chamberlain
2018-12-03 23:34 ` Brendan Higgins
2018-12-03 23:46 ` Luis Chamberlain
2018-12-04 0:44 ` Brendan Higgins
2018-11-30 3:41 ` Luis Chamberlain
2018-12-03 23:37 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 09/19] kunit: test: add the concept of assertions Brendan Higgins
2018-11-28 19:36 ` [RFC v3 10/19] kunit: test: add test managed resource tests Brendan Higgins
2018-11-28 19:36 ` [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel Brendan Higgins
2018-11-29 13:54 ` Kieran Bingham
2018-12-03 23:48 ` Brendan Higgins
2018-12-04 20:47 ` Luis Chamberlain
2018-12-06 12:32 ` Kieran Bingham
2018-12-06 15:37 ` Matthew Wilcox
2018-12-07 11:30 ` Kieran Bingham
2018-12-11 14:09 ` Petr Mladek
2018-12-11 14:41 ` Steven Rostedt
2018-12-11 17:01 ` Anton Ivanov
2019-02-09 0:40 ` Brendan Higgins
2018-12-07 1:05 ` Luis Chamberlain
2018-12-07 18:35 ` Kent Overstreet
2018-11-30 3:44 ` Luis Chamberlain
2018-12-03 23:50 ` Brendan Higgins
2018-12-04 20:48 ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 12/19] kunit: add KUnit wrapper script and simple output parser Brendan Higgins
2018-11-28 19:36 ` [RFC v3 13/19] kunit: improve output from python wrapper Brendan Higgins
2018-11-28 19:36 ` [RFC v3 14/19] Documentation: kunit: add documentation for KUnit Brendan Higgins
2018-11-29 13:56 ` Kieran Bingham
2018-11-30 3:45 ` Luis Chamberlain
2018-12-03 23:53 ` Brendan Higgins
2018-12-06 12:16 ` Kieran Bingham
2019-02-09 0:56 ` Brendan Higgins
2019-02-11 12:16 ` Kieran Bingham
2019-02-12 22:10 ` Brendan Higgins
2019-02-13 21:55 ` Kieran Bingham [this message]
2019-02-14 0:17 ` Brendan Higgins
2019-02-14 17:26 ` Luis Chamberlain
2019-02-14 22:07 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 15/19] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2018-11-28 19:36 ` [RFC v3 16/19] arch: um: make UML unflatten device tree when testing Brendan Higgins
2018-11-28 21:16 ` Rob Herring
2018-12-04 0:00 ` Brendan Higgins
2018-11-30 3:46 ` Luis Chamberlain
2018-12-04 0:02 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 17/19] of: unittest: migrate tests to run on KUnit Brendan Higgins
2018-11-28 20:56 ` Rob Herring
2018-11-30 0:39 ` Randy Dunlap
2018-12-04 0:13 ` Brendan Higgins
2018-12-04 13:40 ` Rob Herring
2018-12-05 23:42 ` Brendan Higgins
2018-12-07 0:41 ` Rob Herring
2018-12-04 0:08 ` Brendan Higgins
2019-02-13 1:44 ` Brendan Higgins
2019-02-14 20:10 ` Rob Herring
2019-02-14 21:52 ` Brendan Higgins
2019-02-18 22:56 ` Frank Rowand
2019-02-28 0:29 ` Brendan Higgins
2018-12-04 10:56 ` Frank Rowand
2018-11-28 19:36 ` [RFC v3 18/19] of: unittest: split out a couple of test cases from unittest Brendan Higgins
2018-12-04 10:58 ` Frank Rowand
2018-12-05 23:54 ` Brendan Higgins
2019-02-14 23:57 ` Frank Rowand
2019-02-15 0:56 ` Brendan Higgins
2019-02-15 2:05 ` Frank Rowand
2019-02-15 10:56 ` Brendan Higgins
2019-02-18 22:25 ` Frank Rowand
2019-02-20 20:44 ` Frank Rowand
2019-02-20 20:47 ` Frank Rowand
2019-02-28 3:52 ` Brendan Higgins
2019-03-22 0:22 ` Frank Rowand
2019-03-22 1:30 ` Brendan Higgins
2019-03-22 1:47 ` Frank Rowand
2019-03-25 22:15 ` Brendan Higgins
2019-09-20 16:57 ` Rob Herring
2019-09-21 23:57 ` Frank Rowand
2019-03-22 1:34 ` Frank Rowand
2019-03-25 22:18 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 19/19] of: unittest: split up some super large test cases Brendan Higgins
2018-12-04 10:52 ` [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Frank Rowand
2018-12-04 11:40 ` Frank Rowand
2018-12-04 13:49 ` Rob Herring
2018-12-05 23:10 ` Brendan Higgins
2019-03-22 0:27 ` Frank Rowand
2019-03-25 22:04 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=eade3052-0aa4-90b4-d55b-8c44556f98bc@ideasonboard.com \
--to=kieran.bingham@ideasonboard.com \
--cc=Tim.Bird@sony.com \
--cc=brakmo@fb.com \
--cc=brendanhiggins@google.com \
--cc=dan.j.williams@intel.com \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=felixguoxiuping@gmail.com \
--cc=frowand.list@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=jdike@addtoit.com \
--cc=joe@perches.com \
--cc=joel@jms.id.au \
--cc=julia.lawall@lip6.fr \
--cc=keescook@google.com \
--cc=khilman@baylibre.com \
--cc=knut.omang@oracle.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-um@lists.infradead.org \
--cc=mcgrof@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=richard@nod.at \
--cc=robh@kernel.org \
--cc=rostedt@goodmis.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).