From: Alan Maguire <alan.maguire@oracle.com>
To: David Gow <davidgow@google.com>
Cc: Kees Cook <keescook@chromium.org>,
Rasmus Villemoes <linux@rasmusvillemoes.dk>,
Brendan Higgins <brendanhiggins@google.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"open list:KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@vger.kernel.org>,
linux-kernel-mentees@lists.linuxfoundation.org,
KUnit Development <kunit-dev@googlegroups.com>
Subject: Re: [Linux-kernel-mentees] common KUnit Kconfig and file naming (was: Re: [PATCH] lib: kunit_test_overflow: add KUnit test of check_*_overflow functions)
Date: Tue, 16 Jun 2020 10:40:22 +0100 (BST) [thread overview]
Message-ID: <alpine.LRH.2.21.2006160956170.16027@localhost> (raw)
In-Reply-To: <CABVgOSkPYBL8Qy9Fsr0owNhh2=tXZmdwCsWhnq+PshJLr06YKA@mail.gmail.com>
On Tue, 16 Jun 2020, David Gow wrote:
> CONFIG_PM_QOS_KUNIT_TESTOn Mon, Jun 15, 2020 at 1:48 AM Kees Cook
> <keescook@chromium.org> wrote:
> >
> > On Sat, Jun 13, 2020 at 02:51:17PM +0800, David Gow wrote:
> > > Yeah, _KUNIT_TEST was what we've sort-of implicitly decided on for
> > > config names, but the documentation does need to happen.
> >
> > That works for me. It still feels redundant, but all I really want is a
> > standard name. :)
> >
> > > We haven't put as much thought into standardising the filenames much, though.
> >
> > I actually find this to be much more important because it is more
> > end-user-facing (i.e. in module naming, in build logs, in scripts, on
> > filesystem, etc -- CONFIG is basically only present during kernel build).
> > Trying to do any sorting or greping really needs a way to find all the
> > kunit pieces.
> >
>
> Certainly this is more of an issue now we support building KUnit tests
> as modules, rather than having them always be built-in.
>
> Having some halfway consistent config-name <-> filename <-> test suite
> name could be useful down the line, too. Unfortunately, not
> necessarily a 1:1 mapping, e.g.:
> - CONFIG_KUNIT_TEST compiles both kunit-test.c and string-stream-test.c
> - kunit-test.c has several test suites within it:
> kunit-try-catch-test, kunit-resource-test & kunit-log-test.
> - CONFIG_EXT4_KUNIT_TESTS currently only builds ext4-inode-test.c, but
> as the plural name suggests, might build others later.
> - CONFIG_SECURITY_APPARMOR_KUNIT_TEST doesn't actually have its own
> source file: the test is built into policy_unpack.c
> - &cetera
>
> Indeed, this made me quickly look up the names of suites, and there
> are a few inconsistencies there:
> - most have "-test" as a suffix
> - some have "_test" as a suffix
> - some have no suffix
>
> (I'm inclined to say that these don't need a suffix at all.)
>
A good convention for module names - which I _think_ is along the lines
of what Kees is suggesting - might be something like
<subsystem>[_<optional_test-area>]_kunit.ko
So for example
kunit_test -> test_kunit.ko
string_stream_test.ko -> test_string_stream_kunit.ko
kunit_example_test -> example_kunit.ko
ext4_inode_test.ko -> ext4_inode_kunit.ko
For the kunit selftests, "selftest_" might be a better name
than "test_", as the latter might encourage people to reintroduce
a redundant "test" into their module name.
> Within test suites, we're also largely prefixing all of the tests with
> a suite name (even if it's not actually the specified suite name). For
> example, CONFIG_PM_QOS_KUNIT_TEST builds
> drivers/base/power/qos-test.c which contains a suite called
> "qos-kunit-test", with tests prefixed "freq_qos_test_". Some of this
> clearly comes down to wanting to namespace things a bit more
> ("qos-test" as a name could refer to a few things, I imagine), but
> specifying how to do so consistently could help.
>
Could we add some definitions to help standardize this?
For example, adding a "subsystem" field to "struct kunit_suite"?
So for the ext4 tests the "subsystem" would be "ext4" and the
name "inode" would specify the test area within that subsystem.
For the KUnit selftests, the subsystem would be "test"/"selftest".
Logging could utilize the subsystem definition to allow test
writers to use less redundant test names too. For example
the suite name logged could be constructed from the
subsystem + area values associated with the kunit_suite,
and individual test names could be shown as the
suite area + test_name.
Thanks!
Alan
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
next prev parent reply other threads:[~2020-06-16 9:40 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-11 21:55 [Linux-kernel-mentees] [PATCH] lib: kunit_test_overflow: add KUnit test of check_*_overflow functions Vitor Massaru Iha
2020-06-12 19:06 ` Brendan Higgins via Linux-kernel-mentees
2020-06-12 22:36 ` Kees Cook
2020-06-13 6:51 ` David Gow via Linux-kernel-mentees
2020-06-14 17:48 ` [Linux-kernel-mentees] common KUnit Kconfig and file naming (was: Re: [PATCH] lib: kunit_test_overflow: add KUnit test of check_*_overflow functions) Kees Cook
2020-06-16 7:25 ` David Gow via Linux-kernel-mentees
2020-06-16 9:40 ` Alan Maguire [this message]
2020-06-17 4:20 ` David Gow via Linux-kernel-mentees
2020-06-18 20:27 ` Brendan Higgins via Linux-kernel-mentees
2020-06-19 3:42 ` Kees Cook
2020-06-19 6:39 ` David Gow via Linux-kernel-mentees
2020-06-19 20:12 ` Brendan Higgins via Linux-kernel-mentees
2020-06-15 16:30 ` [Linux-kernel-mentees] [PATCH] lib: kunit_test_overflow: add KUnit test of check_*_overflow functions Vitor Massaru Iha
2020-06-15 18:37 ` Kees Cook
2020-06-13 6:56 ` David Gow via Linux-kernel-mentees
2020-06-15 16:33 ` Vitor Massaru Iha
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.LRH.2.21.2006160956170.16027@localhost \
--to=alan.maguire@oracle.com \
--cc=brendanhiggins@google.com \
--cc=davidgow@google.com \
--cc=keescook@chromium.org \
--cc=kunit-dev@googlegroups.com \
--cc=linux-kernel-mentees@lists.linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux@rasmusvillemoes.dk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).