From: daniel@ffwll.ch (Daniel Vetter)
Subject: [PATCH v2 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework
Date: Fri, 10 May 2019 10:12:13 +0200 [thread overview]
Message-ID: <CAKMK7uFd1xUx8u3xWLwifVSq4OEnMO4S-m0hESe68UzONXnMFg@mail.gmail.com> (raw)
Message-ID: <20190510081213.fe78-h58KGQLSStgJnv1J-6Nfsn_joBZ2PpCAZNwoaM@z> (raw)
In-Reply-To: <a1b88d5add15d43de0468c32d9a2427629337abb.camel@oracle.com>
On Fri, May 10, 2019@7:49 AM Knut Omang <knut.omang@oracle.com> wrote:
>
> On Thu, 2019-05-09@22:18 -0700, Frank Rowand wrote:
> > On 5/9/19 4:40 PM, Logan Gunthorpe wrote:
> > >
> > >
> > > On 2019-05-09 5:30 p.m., Theodore Ts'o wrote:
> > >> On Thu, May 09, 2019@04:20:05PM -0600, Logan Gunthorpe wrote:
> > >>>
> > >>> The second item, arguably, does have significant overlap with kselftest.
> > >>> Whether you are running short tests in a light weight UML environment or
> > >>> higher level tests in an heavier VM the two could be using the same
> > >>> framework for writing or defining in-kernel tests. It *may* also be valuable
> > >>> for some people to be able to run all the UML tests in the heavy VM
> > >>> environment along side other higher level tests.
> > >>>
> > >>> Looking at the selftests tree in the repo, we already have similar items to
> > >>> what Kunit is adding as I described in point (2) above. kselftest_harness.h
> > >>> contains macros like EXPECT_* and ASSERT_* with very similar intentions to
> > >>> the new KUNIT_EXECPT_* and KUNIT_ASSERT_* macros.
> > >>>
> > >>> However, the number of users of this harness appears to be quite small. Most
> > >>> of the code in the selftests tree seems to be a random mismash of scripts
> > >>> and userspace code so it's not hard to see it as something completely
> > >>> different from the new Kunit:
> > >>>
> > >>> $ git grep --files-with-matches kselftest_harness.h *
> > >>
> > >> To the extent that we can unify how tests are written, I agree that
> > >> this would be a good thing. However, you should note that
> > >> kselftest_harness.h is currently assums that it will be included in
> > >> userspace programs. This is most obviously seen if you look closely
> > >> at the functions defined in the header files which makes calls to
> > >> fork(), abort() and fprintf().
> > >
> > > Ah, yes. I obviously did not dig deep enough. Using kunit for
> > > in-kernel tests and kselftest_harness for userspace tests seems like
> > > a sensible line to draw to me. Trying to unify kernel and userspace
> > > here sounds like it could be difficult so it's probably not worth
> > > forcing the issue unless someone wants to do some really fancy work
> > > to get it done.
> > >
> > > Based on some of the other commenters, I was under the impression
> > > that kselftests had in-kernel tests but I'm not sure where or if they
> > > exist.
> >
> > YES, kselftest has in-kernel tests. (Excuse the shouting...)
> >
> > Here is a likely list of them in the kernel source tree:
> >
> > $ grep module_init lib/test_*.c
> > lib/test_bitfield.c:module_init(test_bitfields)
> > lib/test_bitmap.c:module_init(test_bitmap_init);
> > lib/test_bpf.c:module_init(test_bpf_init);
> > lib/test_debug_virtual.c:module_init(test_debug_virtual_init);
> > lib/test_firmware.c:module_init(test_firmware_init);
> > lib/test_hash.c:module_init(test_hash_init); /* Does everything */
> > lib/test_hexdump.c:module_init(test_hexdump_init);
> > lib/test_ida.c:module_init(ida_checks);
> > lib/test_kasan.c:module_init(kmalloc_tests_init);
> > lib/test_list_sort.c:module_init(list_sort_test);
> > lib/test_memcat_p.c:module_init(test_memcat_p_init);
> > lib/test_module.c:static int __init test_module_init(void)
> > lib/test_module.c:module_init(test_module_init);
> > lib/test_objagg.c:module_init(test_objagg_init);
> > lib/test_overflow.c:static int __init test_module_init(void)
> > lib/test_overflow.c:module_init(test_module_init);
> > lib/test_parman.c:module_init(test_parman_init);
> > lib/test_printf.c:module_init(test_printf_init);
> > lib/test_rhashtable.c:module_init(test_rht_init);
> > lib/test_siphash.c:module_init(siphash_test_init);
> > lib/test_sort.c:module_init(test_sort_init);
> > lib/test_stackinit.c:module_init(test_stackinit_init);
> > lib/test_static_key_base.c:module_init(test_static_key_base_init);
> > lib/test_static_keys.c:module_init(test_static_key_init);
> > lib/test_string.c:module_init(string_selftest_init);
> > lib/test_ubsan.c:module_init(test_ubsan_init);
> > lib/test_user_copy.c:module_init(test_user_copy_init);
> > lib/test_uuid.c:module_init(test_uuid_init);
> > lib/test_vmalloc.c:module_init(vmalloc_test_init)
> > lib/test_xarray.c:module_init(xarray_checks);
> >
> >
> > > If they do exists, it seems like it would make sense to
> > > convert those to kunit and have Kunit tests run-able in a VM or
> > > baremetal instance.
> >
> > They already run in a VM.
> >
> > They already run on bare metal.
> >
> > They already run in UML.
> >
> > This is not to say that KUnit does not make sense. But I'm still trying
> > to get a better description of the KUnit features (and there are
> > some).
>
> FYI, I have a master student who looks at converting some of these to KTF, such as for
> instance the XArray tests, which lended themselves quite good to a semi-automated
> conversion.
>
> The result is also a somewhat more compact code as well as the flexibility
> provided by the Googletest executor and the KTF frameworks, such as running selected
> tests, output formatting, debugging features etc.
So is KTF already in upstream? Or is the plan to unify the KTF and
Kunit in-kernel test harnesses? Because there's tons of these
in-kernel unit tests already, and every merge we get more (Frank's
list didn't even look into drivers or anywhere else, e.g. it's missing
the locking self tests I worked on in the past), and a more structured
approach would really be good.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
next prev parent reply other threads:[~2019-05-10 8:12 UTC|newest]
Thread overview: 262+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-01 23:01 [PATCH v2 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 01/17] kunit: test: add KUnit test runner core brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 02/17] kunit: test: add test resource management API brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 03/17] kunit: test: add string_stream a std::stream like string builder brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-03 1:26 ` shuah
2019-05-03 1:26 ` shuah
2019-05-03 4:37 ` brendanhiggins
2019-05-03 4:37 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 04/17] kunit: test: add kunit_stream a std::stream like logger brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-02 11:00 ` gregkh
2019-05-02 11:00 ` Greg KH
2019-05-02 20:25 ` brendanhiggins
2019-05-02 20:25 ` Brendan Higgins
2019-05-02 21:18 ` frowand.list
2019-05-02 21:18 ` Frank Rowand
2019-05-03 1:50 ` shuah
2019-05-03 1:50 ` shuah
2019-05-03 5:48 ` brendanhiggins
2019-05-03 5:48 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 05/17] kunit: test: add the concept of expectations brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 06/17] kbuild: enable building KUnit brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-10 3:03 ` yamada.masahiro
2019-05-10 3:03 ` Masahiro Yamada
2019-05-10 10:27 ` brendanhiggins
2019-05-10 10:27 ` Brendan Higgins
2019-05-10 10:30 ` yamada.masahiro
2019-05-10 10:30 ` Masahiro Yamada
2019-05-10 10:33 ` brendanhiggins
2019-05-10 10:33 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 07/17] kunit: test: add initial tests brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-02 10:58 ` gregkh
2019-05-02 10:58 ` Greg KH
2019-05-02 20:30 ` brendanhiggins
2019-05-02 20:30 ` Brendan Higgins
2019-05-03 1:27 ` shuah
2019-05-03 1:27 ` shuah
2019-05-03 5:18 ` brendanhiggins
2019-05-03 5:18 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 08/17] kunit: test: add support for test abort brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-03 3:14 ` logang
2019-05-03 3:14 ` Logan Gunthorpe
2019-05-03 6:48 ` brendanhiggins
2019-05-03 6:48 ` Brendan Higgins
2019-05-03 12:33 ` logang
2019-05-03 12:33 ` Logan Gunthorpe
2019-05-06 8:48 ` brendanhiggins
2019-05-06 8:48 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 09/17] kunit: test: add tests for kunit " brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 10/17] kunit: test: add the concept of assertions brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 11/17] kunit: test: add test managed resource tests brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-03 14:34 ` shuah
2019-05-03 14:34 ` shuah
2019-05-06 9:03 ` brendanhiggins
2019-05-06 9:03 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 12/17] kunit: tool: add Python wrappers for running KUnit tests brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-02 11:02 ` gregkh
2019-05-02 11:02 ` Greg KH
2019-05-02 18:07 ` brendanhiggins
2019-05-02 18:07 ` Brendan Higgins
2019-05-02 21:16 ` frowand.list
2019-05-02 21:16 ` Frank Rowand
2019-05-02 23:45 ` brendanhiggins
2019-05-02 23:45 ` Brendan Higgins
2019-05-03 1:45 ` frowand.list
2019-05-03 1:45 ` Frank Rowand
2019-05-03 5:36 ` brendanhiggins
2019-05-03 5:36 ` Brendan Higgins
2019-05-03 18:59 ` frowand.list
2019-05-03 18:59 ` Frank Rowand
2019-05-03 23:14 ` brendanhiggins
2019-05-03 23:14 ` Brendan Higgins
2019-05-04 10:42 ` gregkh
2019-05-04 10:42 ` Greg KH
2019-05-06 0:19 ` frowand.list
2019-05-06 0:19 ` Frank Rowand
2019-05-06 17:43 ` keescook
2019-05-06 17:43 ` Kees Cook
2019-05-06 21:42 ` brendanhiggins
2019-05-06 21:42 ` Brendan Higgins
2019-05-06 21:39 ` brendanhiggins
2019-05-06 21:39 ` Brendan Higgins
2019-05-07 19:13 ` Tim.Bird
2019-05-07 19:13 ` Tim.Bird
2019-05-03 6:41 ` gregkh
2019-05-03 6:41 ` Greg KH
2019-05-01 23:01 ` [PATCH v2 13/17] kunit: defconfig: add defconfigs for building " brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 14/17] Documentation: kunit: add documentation for KUnit brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-09 5:08 ` rdunlap
2019-05-09 5:08 ` Randy Dunlap
2019-05-09 17:38 ` brendanhiggins
2019-05-09 17:38 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 15/17] MAINTAINERS: add entry for KUnit the unit testing framework brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-03 14:38 ` shuah
2019-05-03 14:38 ` shuah
2019-05-06 9:18 ` brendanhiggins
2019-05-06 9:18 ` Brendan Higgins
2019-05-01 23:01 ` [PATCH v2 16/17] kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec() brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-02 11:03 ` gregkh
2019-05-02 11:03 ` Greg KH
2019-05-02 18:14 ` Tim.Bird
2019-05-02 18:14 ` Tim.Bird
2019-05-02 18:45 ` brendanhiggins
2019-05-02 18:45 ` Brendan Higgins
2019-05-03 6:42 ` gregkh
2019-05-03 6:42 ` Greg KH
2019-05-03 23:41 ` brendanhiggins
2019-05-03 23:41 ` Brendan Higgins
2019-05-04 10:40 ` gregkh
2019-05-04 10:40 ` Greg KH
2019-05-01 23:01 ` [PATCH v2 17/17] MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section brendanhiggins
2019-05-01 23:01 ` Brendan Higgins
2019-05-02 10:50 ` [PATCH v2 00/17] kunit: introduce KUnit, the Linux kernel unit testing framework gregkh
2019-05-02 10:50 ` Greg KH
2019-05-02 11:05 ` gregkh
2019-05-02 11:05 ` Greg KH
2019-05-03 0:41 ` brendanhiggins
2019-05-03 0:41 ` Brendan Higgins
2019-05-02 14:04 ` shuah
2019-05-02 14:04 ` shuah
2019-05-03 0:44 ` brendanhiggins
2019-05-03 0:44 ` Brendan Higgins
2019-05-03 3:18 ` logang
2019-05-03 3:18 ` Logan Gunthorpe
2019-05-07 3:14 ` frowand.list
2019-05-07 3:14 ` Frank Rowand
2019-05-07 8:01 ` gregkh
2019-05-07 8:01 ` Greg KH
2019-05-07 15:23 ` shuah
2019-05-07 15:23 ` shuah
2019-05-09 1:01 ` frowand.list
2019-05-09 1:01 ` Frank Rowand
2019-05-07 17:22 ` tytso
2019-05-07 17:22 ` Theodore Ts'o
2019-05-08 19:17 ` brendanhiggins
2019-05-08 19:17 ` Brendan Higgins
2019-05-09 0:58 ` frowand.list
2019-05-09 0:58 ` Frank Rowand
2019-05-09 1:44 ` tytso
2019-05-09 1:44 ` Theodore Ts'o
2019-05-09 2:18 ` frowand.list
2019-05-09 2:18 ` Frank Rowand
2019-05-14 8:22 ` brendanhiggins
2019-05-14 8:22 ` Brendan Higgins
2019-05-09 0:43 ` frowand.list
2019-05-09 0:43 ` Frank Rowand
2019-05-09 1:58 ` tytso
2019-05-09 1:58 ` Theodore Ts'o
2019-05-09 2:13 ` frowand.list
2019-05-09 2:13 ` Frank Rowand
2019-05-09 3:20 ` tytso
2019-05-09 3:20 ` Theodore Ts'o
2019-05-09 11:52 ` knut.omang
2019-05-09 11:52 ` Knut Omang
2019-05-09 13:35 ` tytso
2019-05-09 13:35 ` Theodore Ts'o
2019-05-09 14:48 ` knut.omang
2019-05-09 14:48 ` Knut Omang
2019-05-09 17:00 ` Tim.Bird
2019-05-09 17:00 ` Tim.Bird
2019-05-09 17:42 ` daniel
2019-05-09 17:42 ` Daniel Vetter
2019-05-09 18:12 ` frowand.list
2019-05-09 18:12 ` Frank Rowand
2019-05-09 21:42 ` tytso
2019-05-09 21:42 ` Theodore Ts'o
2019-05-09 22:20 ` logang
2019-05-09 22:20 ` Logan Gunthorpe
2019-05-09 23:30 ` tytso
2019-05-09 23:30 ` Theodore Ts'o
2019-05-09 23:40 ` logang
2019-05-09 23:40 ` Logan Gunthorpe
2019-05-10 4:47 ` tytso
2019-05-10 4:47 ` Theodore Ts'o
2019-05-10 5:18 ` frowand.list
2019-05-10 5:18 ` Frank Rowand
2019-05-10 5:48 ` knut.omang
2019-05-10 5:48 ` Knut Omang
2019-05-10 8:12 ` daniel [this message]
2019-05-10 8:12 ` Daniel Vetter
2019-05-10 10:23 ` brendanhiggins
2019-05-10 10:23 ` Brendan Higgins
2019-05-10 12:12 ` knut.omang
2019-05-10 12:12 ` Knut Omang
2019-05-10 20:54 ` brendanhiggins
2019-05-10 20:54 ` Brendan Higgins
2019-05-10 22:18 ` frowand.list
2019-05-10 22:18 ` Frank Rowand
2019-05-11 6:17 ` knut.omang
2019-05-11 6:17 ` Knut Omang
2019-05-14 6:39 ` brendanhiggins
2019-05-14 6:39 ` Brendan Higgins
2019-05-10 21:59 ` frowand.list
2019-05-10 21:59 ` Frank Rowand
2019-05-11 6:43 ` knut.omang
2019-05-11 6:43 ` Knut Omang
2019-05-14 8:00 ` brendanhiggins
2019-05-14 8:00 ` Brendan Higgins
2019-05-10 11:36 ` knut.omang
2019-05-10 11:36 ` Knut Omang
2019-05-10 16:17 ` logang
2019-05-10 16:17 ` Logan Gunthorpe
2019-05-10 22:13 ` frowand.list
2019-05-10 22:13 ` Frank Rowand
2019-05-14 8:38 ` brendanhiggins
2019-05-14 8:38 ` Brendan Higgins
2019-05-15 0:14 ` frowand.list
2019-05-15 0:14 ` Frank Rowand
2019-05-15 0:26 ` logang
2019-05-15 0:26 ` Logan Gunthorpe
2019-05-10 21:52 ` frowand.list
2019-05-10 21:52 ` Frank Rowand
2019-05-14 20:54 ` brendanhiggins
2019-05-14 20:54 ` Brendan Higgins
2019-05-10 21:12 ` frowand.list
2019-05-10 21:12 ` Frank Rowand
2019-05-11 17:33 ` tytso
2019-05-11 17:33 ` Theodore Ts'o
2019-05-13 14:44 ` daniel
2019-05-13 14:44 ` Daniel Vetter
2019-05-14 6:04 ` brendanhiggins
2019-05-14 6:04 ` Brendan Higgins
2019-05-14 12:05 ` daniel
2019-05-14 12:05 ` Daniel Vetter
2019-05-14 18:36 ` brendanhiggins
2019-05-14 18:36 ` Brendan Higgins
2019-05-15 7:41 ` daniel
2019-05-15 7:41 ` Daniel Vetter
2019-05-22 21:38 ` brendanhiggins
2019-05-22 21:38 ` Brendan Higgins
2019-05-23 8:40 ` daniel
2019-05-23 8:40 ` Daniel Vetter
2019-05-15 0:26 ` frowand.list
2019-05-15 0:26 ` Frank Rowand
2019-05-15 4:28 ` tytso
2019-05-15 4:28 ` Theodore Ts'o
2019-05-10 5:11 ` frowand.list
2019-05-10 5:11 ` Frank Rowand
2019-05-10 10:43 ` tytso
2019-05-10 10:43 ` Theodore Ts'o
2019-05-10 21:05 ` frowand.list
2019-05-10 21:05 ` Frank Rowand
2019-05-09 15:19 ` yamada.masahiro
2019-05-09 15:19 ` Masahiro Yamada
2019-05-10 10:25 ` brendanhiggins
2019-05-10 10:25 ` Brendan Higgins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKMK7uFd1xUx8u3xWLwifVSq4OEnMO4S-m0hESe68UzONXnMFg@mail.gmail.com \
--to=daniel@ffwll.ch \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).