From: Luis Chamberlain <email@example.com> To: shuah <firstname.lastname@example.org> Cc: email@example.com, Brendan Higgins <firstname.lastname@example.org>, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, Knut Omang <firstname.lastname@example.org>, email@example.com, firstname.lastname@example.org, Tim.Bird@sony.com, Joel Stanley <email@example.com>, firstname.lastname@example.org, Kees Cook <email@example.com>, firstname.lastname@example.org, email@example.com, Julia Lawall <firstname.lastname@example.org>, email@example.com, Greg KH <firstname.lastname@example.org>, Linux Kernel Mailing List <email@example.com>, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org Subject: Re: [RFC v2 00/14] kunit: introduce KUnit, the Linux kernel unit testing framework Date: Thu, 29 Nov 2018 16:59:42 -0800 [thread overview] Message-ID: <20181130005942.GL4922@garbanzo.do-not-panic.com> (raw) In-Reply-To: <email@example.com> On Wed, Nov 28, 2018 at 01:50:01PM -0700, shuah wrote: > On 11/28/18 12:54 PM, Knut Omang wrote: > > On Mon, 2018-11-26 at 17:41 -0800, Brendan Higgins wrote: > > Both approaches provide assertion macros for running tests inside the kernel, > > I doubt the kernel community would like to see yet two such sets of macros, > > with differing syntax merged. One of the good reasons to have a *framework* > > is that it is widely used and understood, so that people coming from one part of the > > kernel can easily run, understand and relate to selected tests from another part. > > > > The goal with KTF is to allow this for any kernel, old or new, not just kernels > > built specifically for testing purposes. We felt that providing it as a separate git > > module (still open source, for anyone to contribute to) is a more efficient approach > > until we have more examples and experience with using it in different parts > > of the kernel. We can definitely post the kernel side of KTF as a patch series fairly soon > > if the community wants us to. Except for hybrid tests, the ktf.ko module works fine > > independently of any user side support, just using the debugfs interface to run and > > examine tests. > > > > Having test framework in the kernel sources tree has benefits. It allows > us (kernel developers) to do co-development of kernel features and tests > for these features. Agreed! > It encourages developers to write regression tests. More importantly, > kernel features and tests for these features are included in the same > release in most cases and/or allows us freedom to do so if test framework > and tests are part of the kernel sources. > > We have seen this with our experience with kselftest. It would not have > see the same level of attention and growth if it stayed outside the > kernel sources. > > Most kernel developers would not want to include a externally maintained > module for testing. As a general rule, it has to be easy to run tests. > > > I think there are good uses cases for having the ability to maintain a > > single source for tests that can be run against multiple kernels, > > also distro kernels that the test framework cannot expect to be able to modify, > > except from using the module interfaces. > > Same reasons as above. Having the tests included in the kernel sources > makes it easier for distros to run those tests and include running them > during their qualification. Also... selftests are an example of tests which *are* upstream and yet there are teams out there using them to test these tests on older kernels. So the scripts for instance are supposed to work with older kernels. So if you expand on a feature your selftest script should detect if the new mechanism is present or not, and also be backward compatible with older kernels. > > And there are good arguments for having (at least parts of) > > the test framework easily available within the kernel under test. > > > > When Kernel unit, functional, and regressions tests reside in the kernel > sources, they evolve quicker as kernel developers contribute tests as > part of their kernel work-flow. Maintaining tests and framework > separately will make it harder to maintain them and keep them updated > for us the kernel community. Agreed! Also, I actually see no issue with having *both* kunit / ktest merged upstream. IMHO we should not be forcing people to pick one or the other but rather we should: let the best test framework win. Similar as we did with LSMs. Each test framework has its own gains / advantages. Luis _______________________________________________ Linux-nvdimm mailing list Linuxfirstname.lastname@example.org https://lists.01.org/mailman/listinfo/linux-nvdimm
prev parent reply other threads:[~2018-11-30 0:59 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-10-23 23:57 Brendan Higgins [not found] ` <20181023235750.103146-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> 2018-10-23 23:57 ` [RFC v2 01/14] kunit: test: add KUnit test runner core Brendan Higgins 2018-11-02 18:44 ` Shuah Khan [not found] ` <017b111f-d960-c1ef-46ae-eb0eb639fe5b-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> 2018-11-07 1:28 ` Brendan Higgins 2018-11-07 20:02 ` Shuah Khan 2018-10-23 23:57 ` [RFC v2 02/14] kunit: test: add test resource management API Brendan Higgins 2018-10-23 23:57 ` [RFC v2 03/14] kunit: test: add string_stream a std::stream like string builder Brendan Higgins 2018-10-23 23:57 ` [RFC v2 04/14] kunit: test: add test_stream a std::stream like logger Brendan Higgins 2018-10-23 23:57 ` [RFC v2 05/14] kunit: test: add the concept of expectations Brendan Higgins 2018-10-23 23:57 ` [RFC v2 06/14] arch: um: enable running kunit from User Mode Linux Brendan Higgins 2018-10-23 23:57 ` [RFC v2 07/14] kunit: test: add initial tests Brendan Higgins 2018-10-23 23:57 ` [RFC v2 08/14] arch: um: add shim to trap to allow installing a fault catcher for tests Brendan Higgins 2018-10-23 23:57 ` [RFC v2 09/14] kunit: test: add the concept of assertions Brendan Higgins 2018-10-23 23:57 ` [RFC v2 10/14] kunit: add Python libraries for handing KUnit config and kernel Brendan Higgins 2018-10-23 23:57 ` [RFC v2 11/14] kunit: add KUnit wrapper script and simple output parser Brendan Higgins 2018-10-23 23:57 ` [RFC v2 12/14] kunit.py: improve output from python wrapper Brendan Higgins 2018-10-23 23:57 ` [RFC v2 13/14] Documentation: kunit: add documentation for KUnit Brendan Higgins 2018-10-23 23:57 ` [RFC v2 14/14] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins 2018-10-24 9:14 ` [RFC v2 00/14] kunit: introduce KUnit, the Linux kernel " Daniel Vetter 2018-10-25 21:25 ` Brendan Higgins 2018-10-25 17:40 ` Shuah Khan 2018-11-02 18:23 ` Shuah Khan 2018-11-07 1:17 ` Brendan Higgins 2018-11-07 17:46 ` Frank Rowand [not found] ` <04f677b1-bc44-f004-cf2a-51b47baf0965-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2018-11-13 10:10 ` Brendan Higgins 2018-11-24 5:15 ` Knut Omang [not found] ` <1543036529.4680.655.camel-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2018-11-27 1:41 ` Brendan Higgins 2018-11-28 19:54 ` Knut Omang 2018-11-28 20:50 ` shuah 2018-11-30 0:59 ` Luis Chamberlain [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181130005942.GL4922@garbanzo.do-not-panic.com \ --email@example.com \ --cc=Tim.Bird@sony.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: [RFC v2 00/14] kunit: introduce KUnit, the Linux kernel unit testing framework' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).