From mboxrd@z Thu Jan 1 00:00:00 1970 From: brendanhiggins@google.com (Brendan Higgins) Date: Wed, 13 Feb 2019 16:17:13 -0800 Subject: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit In-Reply-To: References: <20181128193636.254378-1-brendanhiggins@google.com> <20181128193636.254378-15-brendanhiggins@google.com> <20181130034525.GP18410@garbanzo.do-not-panic.com> <0927c42a-8e65-f410-e6ed-27576572577f@ideasonboard.com> <57c3dc86-236f-e981-249a-8bbfe5c19f0e@ideasonboard.com> Message-ID: Content-Type: text/plain; charset="UTF-8" Message-ID: <20190214001713.f5Q7fEeh9oyJfxaWCdXmyLdqJAf3Bo_ynW0zxYJHRyE@z> On Wed, Feb 13, 2019 at 1:55 PM Kieran Bingham wrote: > > Hi Brendan, > > On 12/02/2019 22:10, Brendan Higgins wrote: > > On Mon, Feb 11, 2019 at 4:16 AM Kieran Bingham > > wrote: > >> > >> Hi Brendan, > >> > >> On 09/02/2019 00:56, Brendan Higgins wrote: > >>> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham > >>> wrote: > >>>> > >>>> Hi Brendan, > >>>> > >>>> On 03/12/2018 23:53, Brendan Higgins wrote: > >>>>> On Thu, Nov 29, 2018@7:45 PM Luis Chamberlain wrote: > >>>>>> > >>>>>> On Thu, Nov 29, 2018@01:56:37PM +0000, Kieran Bingham wrote: > >>>>>>> Hi Brendan, > >>>>>>> > >>>>>>> Please excuse the top posting, but I'm replying here as I'm following > >>>>>>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst. > >>>>>>> > >>>>>>> Could the three line kunitconfig file live under say > >>>>>>> arch/um/configs/kunit_defconfig? > >>>> > >>>> > >>>> Further consideration to this topic - I mentioned putting it in > >>>> arch/um/configs > >>>> > >>>> - but I think this is wrong. > >>>> > >>>> We now have a location for config-fragments, which is essentially what > >>>> this is, under kernel/configs > >>>> > >>>> So perhaps an addition as : > >>>> > >>>> kernel/configs/kunit.config > >>>> > >>>> Would be more appropriate - and less (UM) architecture specific. > >>> > >>> Sorry for the long radio silence. > >>> > >>> I just got around to doing this and I found that there are some > >>> configs that are desirable to have when running KUnit under x86 in a > >>> VM, but not UML. > >> > >> Should this behaviour you mention be handled by the KCONFIG depends flags? > >> > >> depends on (KUMIT & UML) > >> or > >> depends on (KUNIT & !UML) > >> > >> or such? > > > > Not really. Anything that is strictly necessary to run KUnit on an > > architectures should of course be turned on as a dependency like you > > suggest, but I am talking about stuff that you would probably want to > > get yourself going, but is by no means necessary. > > > >> > >> An example of which configs you are referring to would help to > >> understand the issue perhaps. > >> > > > > For example, you might want to enable a serial console that is known > > to work with a fairly generic qemu setup when building for x86: > > CONFIG_SERIAL_8250=y > > CONFIG_SERIAL_8250_CONSOLE=y > > > > Obviously not a dependency, and not even particularly useful to people > > who know what they are doing, but to someone who is new or just wants > > something to work out of the box would probably want that. > > It sounds like that would be a config fragment for qemu ? > > Although - perhaps this is already covered by the following fragment: > kernel/configs/kvm_guest.config > Oh, yep, you are right. Does that mean we should bother at all with a defconfig? Luis, I know you said you wanted one. I am thinking just stick with the UML one? The downside there is we then get stuck having to maintain the fragment and the defconfig. I right now (in the new revision I am working on) have the Python kunit_tool copy the defconfig if no kunitconfig is provided and a flag is set. It would be pretty straightforward to make it merge in the fragment instead. All that being said, I think I am going to drop the arch/x86 defconfig, since I think we all agree that it is not very useful, but keep the UML defconfig and the fragment. That will at least given something concrete to discuss. > > >>> So should we have one that goes in with > >>> config-fragments and others that go into architectures? Another idea, > >>> it would be nice to have a KUnit config that runs all known tests > >> > >> This might also be a config option added to the tests directly like > >> COMPILE_TEST perhaps? > > > > That just allows a bunch of drivers to be compiled, it does not > > actually go through and turn the configs on, right? I mean, there is > > no a priori way to know that there is a configuration which spans all > > possible options available under COMPILE_TEST, right? Maybe I > > misunderstand what you are suggesting... > > Bah - you're right of course. I was mis-remembering the functionality of > COMPILE_TEST as if it were some sort of 'select' but it's just an enable.. > > Sorry for the confusion. > No problem, I thought for a second that was a good example too (and I wish it were. It would make my life so much easier!). I remember getting emails with a COMPILE_TEST config attached that demonstrates an invalid build caused by my changes, presumably that email bot just tries random configs with a new change until it finds one that breaks. > > >> (Not sure what that would be called though ... KUNIT_RUNTIME_TEST?) > >> > >> I think that might be more maintainable as otherwise each new test would > >> have to modify the {min,def}{config,fragment} ... > >> > > > > Looking at kselftest-merge, they just start out with a set of > > fragments in which the union should contain all tests and then merge > > it with a base .config (probably intended to be $(ARCH)_defconfig). > > However, I don't know if that is the state of the art. > > > >> > >>> (this probably won't work in practice once we start testing mutually > >>> exclusive things or things with lots of ifdeffery, but it probably > >>> something we should try to maintain as best as we can?); this probably > >>> shouldn't go in with the fragments, right? > >> > >> Sounds like we agree there :) > > > > Totally. Long term we will need something a lot more sophisticated > > than anything under discussion here. I was talking about this with > > Luis on another thread: > > https://groups.google.com/forum/#!topic/kunit-dev/EQ1x0SzrUus (feel > > free to chime in!). Nevertheless, that's a really hard problem and I > > figure some variant of defconfigs and config fragments will work well > > enough until we reach that point. > > > >> > >>> > >>> I will be sending another revision out soon, but I figured I might be > >>> able to catch you before I did so. > >> > >> Thanks for thinking of me. > > > > How can I forget? You have been super helpful! > > > >> I hope I managed to reply in time to help and not hinder your progress. > > > > Yep, no trouble at all. You are the one helping me :-) > > > > Thanks! > > > > -- > Regards > -- > Kieran