archive mirror
 help / color / mirror / Atom feed
From: Marco Elver <>
To: David Gow <>
Cc: Brendan Higgins <>,
	Jonathan Corbet <>,
	Kees Cook <>,
	Alan Maguire <>,
	Randy Dunlap <>,
	"Theodore Ts'o" <>, Tim Bird <>,
	KUnit Development <>,
	"open list:DOCUMENTATION" <>,
	Linux Kernel Mailing List <>
Subject: Re: [PATCH] Documentation: kunit: Add naming guidelines
Date: Thu, 27 Aug 2020 20:28:03 +0200	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Thu, 27 Aug 2020 at 18:17, David Gow <> wrote:
> > First of all, thanks for the talk yesterday! I only looked at this
> > because somebody pasted the LKML link. :-)
> No worries! Clearly this document needed linking -- even I was
> starting to suspect the reason no-one was complaining about this was
> that no-one had read it. :-)
> >
> > While I guess this ship has sailed, and *_kunit.c is the naming
> > convention now, I hope this is still just a recommendation and names of
> > the form *-test.c are not banned!
> The ship hasn't technically sailed until this patch is actually
> accepted. Thus far, we hadn't had any real complaints about the
> _kunit.c idea, though that may have been due to this email not
> reaching enough people. If you haven't read the discussion in
> it's worthwhile: the _kunit.c name is discussed, and Kees lays out
> some more detailed rationale for it as well.

Thanks, I can see the rationale. AFAIK the main concern was "it does
not distinguish it from other tests".

> > $> git grep 'KUNIT.*-test.o'
> >         drivers/base/power/Makefile:obj-$(CONFIG_PM_QOS_KUNIT_TEST) += qos-test.o
> >         drivers/base/test/Makefile:obj-$(CONFIG_KUNIT_DRIVER_PE_TEST) += property-entry-test.o
> >         fs/ext4/Makefile:obj-$(CONFIG_EXT4_KUNIT_TESTS)         += ext4-inode-test.o
> >         kernel/Makefile:obj-$(CONFIG_SYSCTL_KUNIT_TEST) += sysctl-test.o
> >         lib/Makefile:obj-$(CONFIG_LIST_KUNIT_TEST) += list-test.o
> >         lib/kunit/Makefile:obj-$(CONFIG_KUNIT_TEST) +=          kunit-test.o
> >         lib/kunit/Makefile:obj-$(CONFIG_KUNIT_TEST) +=          string-stream-test.o
> >         lib/kunit/Makefile:obj-$(CONFIG_KUNIT_EXAMPLE_TEST) +=  kunit-example-test.o
> >
> > $> git grep 'KUNIT.*_kunit.o'
> > # Returns nothing
> >
> This was definitely something we noted. Part of the goal of having
> _kunit.c as a filename suffix (and, perhaps more importantly, the
> _kunit module name suffix) was to have a way of both distinguishing
> KUnit tests from non-KUnit ones (of which there are quite a few
> already with -test names), and to have a way of quickly determining
> what modules contain KUnit tests. That really only works if everyone
> is using it, so the plan was to push this as much as possible. This'd
> probably include renaming most of the existing test files to match,
> which is a bit of a pain (particularly when converting non-KUnit tests
> to KUnit and similar), but is a one-time thing.

Feel free to ignore the below, but here might be one concern:

I think the problem of distinguishing KUnit tests from non-KUnit tests
is a technical problem (in fact, the Kconfig entries have all the info
we need), but a solution that sacrifices readability might cause
unnecessary friction.

The main issue I foresee is that the _kunit.c name is opaque as to
what the file does, but merely indicates one of its dependencies. Most
of us clearly know that KUnit is a test framework, but it's a level of
indirection nevertheless. (But _kunit_test.c is also bad, because it's
unnecessarily long.) For a dozen tests, that's probably still fine.
But now imagine 100s of tests, people will quickly wonder "what's this
_kunit thing?". And if KUnit tests are eventually the dominant tests,
does it still matter?

I worry that because of the difficulty of enforcing the name, choosing
something unintuitive will also achieve the opposite result:
proliferation of even more diverse names. A generic convention like
"*-test.c" will be close enough to what's intuitive for most people,
and we might actually have a chance of getting everyone to stick to

The problem of identifying all KUnit tests can be solved with a script.

> > Just an idea: Maybe the names are also an opportunity to distinguish
> > real _unit_ style tests and then the rarer integration-style tests. I
> > personally prefer using the more generic *-test.c, at least for the
> > integration-style tests I've been working on (KUnit is still incredibly
> > valuable for integration-style tests, because otherwise I'd have to roll
> > my own poor-man's version of KUnit, so thank you!). Using *_kunit.c for
> > such tests is unintuitive, because the word "unit" hints at "unit tests"
> > -- and having descriptive (and not misleading) filenames is still
> > important. So I hope you won't mind if *-test.c are still used where
> > appropriate.
> As Brendan alluded to in the talk, the popularity of KUnit for these
> integration-style tests came as something of a surprise (more due to
> our lack of imagination than anything else, I suspect). It's great
> that it's working, though: I don't think anyone wants the world filled
> with more single-use test "frameworks" than is necessary!
> I guess the interesting thing to note is that we've to date not really
> made a distinction between KUnit the framework and the suite of all
> KUnit tests. Maybe having a separate file/module naming scheme could
> be a way of making that distinction, though it'd really only appear
> when loading tests as modules -- there'd be no indication in e.g.,
> suite names or test results. The more obvious solution to me (at
> least, based on the current proposal) would be to have "integration"
> or similar be part of the suite name (and hence the filename, so
> _integration_kunit.c or similar), though even I admit that that's much
> uglier.

Yeah, that's not great either.  Again, in the end it's probably
entirely up to you, but it'd be good if the filenames are descriptive
and readable (vs. a puzzle).

> Maybe the idea of having the subsystem/suite distinction be
> represented in the code could pave the way to having different suites
> support different suffixes like that.

> Do you know of any cases where something has/would have both
> unit-style tests and integration-style tests built with KUnit where
> the distinction needs to be clear?

None I know of, so probably not a big deal.

> Brendan, Kees: do you have any thoughts?

> Cheers,
> -- David

-- Marco

  reply	other threads:[~2020-08-27 18:28 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-02  7:14 [PATCH] Documentation: kunit: Add naming guidelines David Gow
2020-07-31 22:02 ` Brendan Higgins
2020-08-27 13:14 ` Marco Elver
2020-08-27 16:17   ` David Gow
2020-08-27 18:28     ` Marco Elver [this message]
2020-08-27 19:34       ` Brendan Higgins
2020-08-31 23:47     ` Kees Cook
2020-09-01  5:31       ` David Gow
2020-09-01 12:23         ` Marco Elver
2020-09-04  4:22           ` David Gow
2020-09-07  8:57             ` Marco Elver
  -- strict thread matches above, loose matches on Subject: below --
2020-06-20  5:49 David Gow
2020-06-22  3:55 ` Randy Dunlap
2020-06-22 21:33 ` Brendan Higgins
2020-06-22 21:41 ` Kees Cook

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='' \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).