linux-kselftest.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: kieran.bingham at ideasonboard.com (Kieran Bingham)
Subject: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit
Date: Mon, 11 Feb 2019 12:16:08 +0000	[thread overview]
Message-ID: <57c3dc86-236f-e981-249a-8bbfe5c19f0e@ideasonboard.com> (raw)
In-Reply-To: <CAFd5g44fKzriEs_pGyANxe7GuC9Ji2SD7EEz76__h8q9jjg8yA@mail.gmail.com>

Hi Brendan,

On 09/02/2019 00:56, Brendan Higgins wrote:
> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
> <kieran.bingham at ideasonboard.com> wrote:
>>
>> Hi Brendan,
>>
>> On 03/12/2018 23:53, Brendan Higgins wrote:
>>> On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain <mcgrof at kernel.org> wrote:
>>>>
>>>> On Thu, Nov 29, 2018 at 01:56:37PM +0000, Kieran Bingham wrote:
>>>>> Hi Brendan,
>>>>>
>>>>> Please excuse the top posting, but I'm replying here as I'm following
>>>>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
>>>>>
>>>>> Could the three line kunitconfig file live under say
>>>>>        arch/um/configs/kunit_defconfig?
>>
>>
>> Further consideration to this topic - I mentioned putting it in
>>   arch/um/configs
>>
>> - but I think this is wrong.
>>
>> We now have a location for config-fragments, which is essentially what
>> this is, under kernel/configs
>>
>> So perhaps an addition as :
>>
>>  kernel/configs/kunit.config
>>
>> Would be more appropriate - and less (UM) architecture specific.
> 
> Sorry for the long radio silence.
> 
> I just got around to doing this and I found that there are some
> configs that are desirable to have when running KUnit under x86 in a
> VM, but not UML. 

Should this behaviour you mention be handled by the KCONFIG depends flags?

depends on (KUMIT & UML)
or
depends on (KUNIT & !UML)

or such?

An example of which configs you are referring to would help to
understand the issue perhaps.


> So should we have one that goes in with
> config-fragments and others that go into architectures? Another idea,
> it would be nice to have a KUnit config that runs all known tests

This might also be a config option added to the tests directly like
COMPILE_TEST perhaps?

(Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)

I think that might be more maintainable as otherwise each new test would
have to modify the {min,def}{config,fragment} ...


> (this probably won't work in practice once we start testing mutually
> exclusive things or things with lots of ifdeffery, but it probably
> something we should try to maintain as best as we can?); this probably
> shouldn't go in with the fragments, right?

Sounds like we agree there :)

> 
> I will be sending another revision out soon, but I figured I might be
> able to catch you before I did so.

Thanks for thinking of me.
I hope I managed to reply in time to help and not hinder your progress.

-- 
Regards
--
Kieran

WARNING: multiple messages have this Message-ID (diff)
From: kieran.bingham@ideasonboard.com (Kieran Bingham)
Subject: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit
Date: Mon, 11 Feb 2019 12:16:08 +0000	[thread overview]
Message-ID: <57c3dc86-236f-e981-249a-8bbfe5c19f0e@ideasonboard.com> (raw)
Message-ID: <20190211121608.F-5BnQufqRS1gwZjD2eixGkhQD3Mdolj8kEqyt-wDoo@z> (raw)
In-Reply-To: <CAFd5g44fKzriEs_pGyANxe7GuC9Ji2SD7EEz76__h8q9jjg8yA@mail.gmail.com>

Hi Brendan,

On 09/02/2019 00:56, Brendan Higgins wrote:
> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
> <kieran.bingham@ideasonboard.com> wrote:
>>
>> Hi Brendan,
>>
>> On 03/12/2018 23:53, Brendan Higgins wrote:
>>> On Thu, Nov 29, 2018@7:45 PM Luis Chamberlain <mcgrof@kernel.org> wrote:
>>>>
>>>> On Thu, Nov 29, 2018@01:56:37PM +0000, Kieran Bingham wrote:
>>>>> Hi Brendan,
>>>>>
>>>>> Please excuse the top posting, but I'm replying here as I'm following
>>>>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
>>>>>
>>>>> Could the three line kunitconfig file live under say
>>>>>        arch/um/configs/kunit_defconfig?
>>
>>
>> Further consideration to this topic - I mentioned putting it in
>>   arch/um/configs
>>
>> - but I think this is wrong.
>>
>> We now have a location for config-fragments, which is essentially what
>> this is, under kernel/configs
>>
>> So perhaps an addition as :
>>
>>  kernel/configs/kunit.config
>>
>> Would be more appropriate - and less (UM) architecture specific.
> 
> Sorry for the long radio silence.
> 
> I just got around to doing this and I found that there are some
> configs that are desirable to have when running KUnit under x86 in a
> VM, but not UML. 

Should this behaviour you mention be handled by the KCONFIG depends flags?

depends on (KUMIT & UML)
or
depends on (KUNIT & !UML)

or such?

An example of which configs you are referring to would help to
understand the issue perhaps.


> So should we have one that goes in with
> config-fragments and others that go into architectures? Another idea,
> it would be nice to have a KUnit config that runs all known tests

This might also be a config option added to the tests directly like
COMPILE_TEST perhaps?

(Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)

I think that might be more maintainable as otherwise each new test would
have to modify the {min,def}{config,fragment} ...


> (this probably won't work in practice once we start testing mutually
> exclusive things or things with lots of ifdeffery, but it probably
> something we should try to maintain as best as we can?); this probably
> shouldn't go in with the fragments, right?

Sounds like we agree there :)

> 
> I will be sending another revision out soon, but I figured I might be
> able to catch you before I did so.

Thanks for thinking of me.
I hope I managed to reply in time to help and not hinder your progress.

-- 
Regards
--
Kieran

  parent reply	other threads:[~2019-02-11 12:16 UTC|newest]

Thread overview: 232+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-28 19:36 [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework brendanhiggins
2018-11-28 19:36 ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 01/19] kunit: test: add KUnit test runner core brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-30  3:14   ` mcgrof
2018-11-30  3:14     ` Luis Chamberlain
2018-12-01  1:51     ` brendanhiggins
2018-12-01  1:51       ` Brendan Higgins
2018-12-01  2:57       ` mcgrof
2018-12-01  2:57         ` Luis Chamberlain
2018-12-05 13:15     ` anton.ivanov
2018-12-05 13:15       ` Anton Ivanov
2018-12-05 14:45       ` arnd
2018-12-05 14:45         ` Arnd Bergmann
2018-12-05 14:49         ` anton.ivanov
2018-12-05 14:49           ` Anton Ivanov
2018-11-30  3:28   ` mcgrof
2018-11-30  3:28     ` Luis Chamberlain
2018-12-01  2:08     ` brendanhiggins
2018-12-01  2:08       ` Brendan Higgins
2018-12-01  3:10       ` mcgrof
2018-12-01  3:10         ` Luis Chamberlain
2018-12-03 22:47         ` brendanhiggins
2018-12-03 22:47           ` Brendan Higgins
2018-12-01  3:02   ` mcgrof
2018-12-01  3:02     ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 02/19] kunit: test: add test resource management API brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 03/19] kunit: test: add string_stream a std::stream like string builder brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-30  3:29   ` mcgrof
2018-11-30  3:29     ` Luis Chamberlain
2018-12-01  2:14     ` brendanhiggins
2018-12-01  2:14       ` Brendan Higgins
2018-12-01  3:12       ` mcgrof
2018-12-01  3:12         ` Luis Chamberlain
2018-12-03 10:55     ` pmladek
2018-12-03 10:55       ` Petr Mladek
2018-12-04  0:35       ` brendanhiggins
2018-12-04  0:35         ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 04/19] kunit: test: add test_stream a std::stream like logger brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 05/19] kunit: test: add the concept of expectations brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 06/19] arch: um: enable running kunit from User Mode Linux brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 21:26   ` robh
2018-11-28 21:26     ` Rob Herring
2018-11-30  3:37     ` mcgrof
2018-11-30  3:37       ` Luis Chamberlain
2018-11-30 14:05       ` robh
2018-11-30 14:05         ` Rob Herring
2018-11-30 18:22         ` mcgrof
2018-11-30 18:22           ` Luis Chamberlain
2018-12-03 23:22           ` brendanhiggins
2018-12-03 23:22             ` Brendan Higgins
2018-11-30  3:30   ` mcgrof
2018-11-30  3:30     ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 07/19] kunit: test: add initial tests brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-30  3:40   ` mcgrof
2018-11-30  3:40     ` Luis Chamberlain
2018-12-03 23:26     ` brendanhiggins
2018-12-03 23:26       ` Brendan Higgins
2018-12-03 23:43       ` mcgrof
2018-12-03 23:43         ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 08/19] arch: um: add shim to trap to allow installing a fault catcher for tests brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-30  3:34   ` mcgrof
2018-11-30  3:34     ` Luis Chamberlain
2018-12-03 23:34     ` brendanhiggins
2018-12-03 23:34       ` Brendan Higgins
2018-12-03 23:46       ` mcgrof
2018-12-03 23:46         ` Luis Chamberlain
2018-12-04  0:44         ` brendanhiggins
2018-12-04  0:44           ` Brendan Higgins
2018-11-30  3:41   ` mcgrof
2018-11-30  3:41     ` Luis Chamberlain
2018-12-03 23:37     ` brendanhiggins
2018-12-03 23:37       ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 09/19] kunit: test: add the concept of assertions brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 10/19] kunit: test: add test managed resource tests brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-29 13:54   ` kieran.bingham
2018-11-29 13:54     ` Kieran Bingham
2018-12-03 23:48     ` brendanhiggins
2018-12-03 23:48       ` Brendan Higgins
2018-12-04 20:47       ` mcgrof
2018-12-04 20:47         ` Luis Chamberlain
2018-12-06 12:32         ` kieran.bingham
2018-12-06 12:32           ` Kieran Bingham
2018-12-06 15:37           ` willy
2018-12-06 15:37             ` Matthew Wilcox
2018-12-07 11:30             ` kieran.bingham
2018-12-07 11:30               ` Kieran Bingham
2018-12-11 14:09             ` pmladek
2018-12-11 14:09               ` Petr Mladek
2018-12-11 14:41               ` rostedt
2018-12-11 14:41                 ` Steven Rostedt
2018-12-11 17:01                 ` anton.ivanov
2018-12-11 17:01                   ` Anton Ivanov
2019-02-09  0:40                   ` brendanhiggins
2019-02-09  0:40                     ` Brendan Higgins
2018-12-07  1:05           ` mcgrof
2018-12-07  1:05             ` Luis Chamberlain
2018-12-07 18:35           ` kent.overstreet
2018-12-07 18:35             ` Kent Overstreet
2018-11-30  3:44   ` mcgrof
2018-11-30  3:44     ` Luis Chamberlain
2018-12-03 23:50     ` brendanhiggins
2018-12-03 23:50       ` Brendan Higgins
2018-12-04 20:48       ` mcgrof
2018-12-04 20:48         ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 12/19] kunit: add KUnit wrapper script and simple output parser brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 13/19] kunit: improve output from python wrapper brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 14/19] Documentation: kunit: add documentation for KUnit brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-29 13:56   ` kieran.bingham
2018-11-29 13:56     ` Kieran Bingham
2018-11-30  3:45     ` mcgrof
2018-11-30  3:45       ` Luis Chamberlain
2018-12-03 23:53       ` brendanhiggins
2018-12-03 23:53         ` Brendan Higgins
2018-12-06 12:16         ` kieran.bingham
2018-12-06 12:16           ` Kieran Bingham
2019-02-09  0:56           ` brendanhiggins
2019-02-09  0:56             ` Brendan Higgins
2019-02-11 12:16             ` kieran.bingham [this message]
2019-02-11 12:16               ` Kieran Bingham
2019-02-12 22:10               ` brendanhiggins
2019-02-12 22:10                 ` Brendan Higgins
2019-02-13 21:55                 ` kieran.bingham
2019-02-13 21:55                   ` Kieran Bingham
2019-02-14  0:17                   ` brendanhiggins
2019-02-14  0:17                     ` Brendan Higgins
2019-02-14 17:26                     ` mcgrof
2019-02-14 17:26                       ` Luis Chamberlain
2019-02-14 22:07                       ` brendanhiggins
2019-02-14 22:07                         ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 15/19] MAINTAINERS: add entry for KUnit the unit testing framework brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 16/19] arch: um: make UML unflatten device tree when testing brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-11-28 21:16   ` robh
2018-11-28 21:16     ` Rob Herring
2018-12-04  0:00     ` brendanhiggins
2018-12-04  0:00       ` Brendan Higgins
2018-11-30  3:46   ` mcgrof
2018-11-30  3:46     ` Luis Chamberlain
2018-12-04  0:02     ` brendanhiggins
2018-12-04  0:02       ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 17/19] of: unittest: migrate tests to run on KUnit brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
     [not found]   ` <CAL_Jsq+09Kx7yMBC_Jw45QGmk6U_fp4N6HOZDwYrM4tWw+_dOA@mail.gmail.com>
2018-11-30  0:39     ` rdunlap
2018-11-30  0:39       ` Randy Dunlap
2018-12-04  0:13       ` brendanhiggins
2018-12-04  0:13         ` Brendan Higgins
2018-12-04 13:40         ` robh
2018-12-04 13:40           ` Rob Herring
2018-12-05 23:42           ` brendanhiggins
2018-12-05 23:42             ` Brendan Higgins
2018-12-07  0:41             ` robh
2018-12-07  0:41               ` Rob Herring
2018-12-04  0:08     ` brendanhiggins
2018-12-04  0:08       ` Brendan Higgins
2019-02-13  1:44     ` brendanhiggins
2019-02-13  1:44       ` Brendan Higgins
2019-02-14 20:10       ` robh
2019-02-14 20:10         ` Rob Herring
2019-02-14 21:52         ` brendanhiggins
2019-02-14 21:52           ` Brendan Higgins
2019-02-18 22:56       ` frowand.list
2019-02-18 22:56         ` Frank Rowand
2019-02-28  0:29         ` brendanhiggins
2019-02-28  0:29           ` Brendan Higgins
2018-12-04 10:56   ` frowand.list
2018-12-04 10:56     ` Frank Rowand
2018-11-28 19:36 ` [RFC v3 18/19] of: unittest: split out a couple of test cases from unittest brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-12-04 10:58   ` frowand.list
2018-12-04 10:58     ` Frank Rowand
2018-12-05 23:54     ` brendanhiggins
2018-12-05 23:54       ` Brendan Higgins
2019-02-14 23:57       ` frowand.list
2019-02-14 23:57         ` Frank Rowand
2019-02-15  0:56         ` brendanhiggins
2019-02-15  0:56           ` Brendan Higgins
2019-02-15  2:05           ` frowand.list
2019-02-15  2:05             ` Frank Rowand
2019-02-15 10:56             ` brendanhiggins
2019-02-15 10:56               ` Brendan Higgins
2019-02-18 22:25               ` frowand.list
2019-02-18 22:25                 ` Frank Rowand
2019-02-20 20:44                 ` frowand.list
2019-02-20 20:44                   ` Frank Rowand
2019-02-20 20:47                   ` frowand.list
2019-02-20 20:47                     ` Frank Rowand
2019-02-28  3:52                   ` brendanhiggins
2019-02-28  3:52                     ` Brendan Higgins
2019-03-22  0:22                     ` frowand.list
2019-03-22  0:22                       ` Frank Rowand
2019-03-22  1:30                       ` brendanhiggins
2019-03-22  1:30                         ` Brendan Higgins
2019-03-22  1:47                         ` frowand.list
2019-03-22  1:47                           ` Frank Rowand
2019-03-25 22:15                           ` brendanhiggins
2019-03-25 22:15                             ` Brendan Higgins
2019-09-20 16:57                         ` Rob Herring
2019-09-21 23:57                           ` Frank Rowand
2019-03-22  1:34                       ` frowand.list
2019-03-22  1:34                         ` Frank Rowand
2019-03-25 22:18                         ` brendanhiggins
2019-03-25 22:18                           ` Brendan Higgins
2018-11-28 19:36 ` [RFC v3 19/19] of: unittest: split up some super large test cases brendanhiggins
2018-11-28 19:36   ` Brendan Higgins
2018-12-04 10:52 ` [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework frowand.list
2018-12-04 10:52   ` Frank Rowand
2018-12-04 11:40 ` frowand.list
2018-12-04 11:40   ` Frank Rowand
2018-12-04 13:49   ` robh
2018-12-04 13:49     ` Rob Herring
2018-12-05 23:10     ` brendanhiggins
2018-12-05 23:10       ` Brendan Higgins
2019-03-22  0:27       ` frowand.list
2019-03-22  0:27         ` Frank Rowand
2019-03-25 22:04         ` brendanhiggins
2019-03-25 22:04           ` Brendan Higgins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=57c3dc86-236f-e981-249a-8bbfe5c19f0e@ideasonboard.com \
    --to=linux-kselftest@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).