Linux-NVDIMM Archive on lore.kernel.org
 help / color / Atom feed
From: Randy Dunlap <rdunlap@infradead.org>
To: Brendan Higgins <brendanhiggins@google.com>,
	frowand.list@gmail.com, gregkh@linuxfoundation.org,
	jpoimboe@redhat.com, keescook@google.com,
	kieran.bingham@ideasonboard.com, mcgrof@kernel.org,
	peterz@infradead.org, robh@kernel.org, sboyd@kernel.org,
	shuah@kernel.org, tytso@mit.edu, yamada.masahiro@socionext.com
Cc: pmladek@suse.com, linux-doc@vger.kernel.org, amir73il@gmail.com,
	dri-devel@lists.freedesktop.org, Alexander.Levin@microsoft.com,
	linux-kselftest@vger.kernel.org, linux-nvdimm@lists.01.org,
	khilman@baylibre.com, knut.omang@oracle.com,
	Felix Guo <felixguoxiuping@gmail.com>,
	wfg@linux.intel.com, joel@jms.id.au, rientjes@google.com,
	jdike@addtoit.com, dan.carpenter@oracle.com,
	devicetree@vger.kernel.org, linux-kbuild@vger.kernel.org,
	Tim.Bird@sony.com, linux-um@lists.infradead.org,
	rostedt@goodmis.org, julia.lawall@lip6.fr,
	kunit-dev@googlegroups.com, richard@nod.at,
	torvalds@linux-foundation.org, Jonathan Corbet <corbet@lwn.net>,
	linux-kernel@vger.kernel.org, daniel@ffwll.ch,
	mpe@ellerman.id.au, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit
Date: Mon, 23 Sep 2019 17:51:41 -0700
Message-ID: <9cd80aa2-fc8d-1fed-838b-cf4951692b6d@infradead.org> (raw)
In-Reply-To: <20190923090249.127984-16-brendanhiggins@google.com>

On 9/23/19 2:02 AM, Brendan Higgins wrote:

> diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
> new file mode 100644
> index 000000000000..c6e69634e274
> --- /dev/null
> +++ b/Documentation/dev-tools/kunit/usage.rst
> @@ -0,0 +1,576 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +===========
> +Using KUnit
> +===========
> +
> +The purpose of this document is to describe what KUnit is, how it works, how it
> +is intended to be used, and all the concepts and terminology that are needed to
> +understand it. This guide assumes a working knowledge of the Linux kernel and
> +some basic knowledge of testing.
> +
> +For a high level introduction to KUnit, including setting up KUnit for your
> +project, see :doc:`start`.
> +
> +Organization of this document
> +=============================
> +
> +This document is organized into two main sections: Testing and Isolating
> +Behavior. The first covers what a unit test is and how to use KUnit to write

                              what unit tests are
would agree with the following "them."

> +them. The second covers how to use KUnit to isolate code and make it possible
> +to unit test code that was otherwise un-unit-testable.
> +
> +Testing
> +=======
> +

[snip]


> +
> +Test Suites
> +~~~~~~~~~~~
> +
> +Now obviously one unit test isn't very helpful; the power comes from having
> +many test cases covering all of your behaviors. Consequently it is common to

                   covering all of a unit's behaviors.

> +have many *similar* tests; in order to reduce duplication in these closely
> +related tests most unit testing frameworks provide the concept of a *test
> +suite*, in KUnit we call it a *test suite*; all it is is just a collection of

                                             . This is just a collection of

> +test cases for a unit of code with a set up function that gets invoked before
> +every test cases and then a tear down function that gets invoked after every

   every test case

> +test case completes.
> +
> +Example:
> +
> +.. code-block:: c
> +
> +	static struct kunit_case example_test_cases[] = {
> +		KUNIT_CASE(example_test_foo),
> +		KUNIT_CASE(example_test_bar),
> +		KUNIT_CASE(example_test_baz),
> +		{}
> +	};
> +
> +	static struct kunit_suite example_test_suite = {
> +		.name = "example",
> +		.init = example_test_init,
> +		.exit = example_test_exit,
> +		.test_cases = example_test_cases,
> +	};
> +	kunit_test_suite(example_test_suite);
> +
> +In the above example the test suite, ``example_test_suite``, would run the test
> +cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``,
> +each would have ``example_test_init`` called immediately before it and would
> +have ``example_test_exit`` called immediately after it.
> +``kunit_test_suite(example_test_suite)`` registers the test suite with the
> +KUnit test framework.
> +
> +.. note::
> +   A test case will only be run if it is associated with a test suite.
> +
> +For a more information on these types of things see the :doc:`api/test`.

   For more

> +
> +Isolating Behavior
> +==================
> +

[snip]

> +
> +.. _kunit-on-non-uml:
> +
> +KUnit on non-UML architectures
> +==============================
> +
> +By default KUnit uses UML as a way to provide dependencies for code under test.
> +Under most circumstances KUnit's usage of UML should be treated as an
> +implementation detail of how KUnit works under the hood. Nevertheless, there
> +are instances where being able to run architecture specific code, or test

                           I would drop the comma above.

> +against real hardware is desirable. For these reasons KUnit supports running on
> +other architectures.
> +
> +Running existing KUnit tests on non-UML architectures
> +-----------------------------------------------------
> +

[snip]

> +Writing new tests for other architectures
> +-----------------------------------------
> +
> +The first thing you must do is ask yourself whether it is necessary to write a
> +KUnit test for a specific architecture, and then whether it is necessary to
> +write that test for a particular piece of hardware. In general, writing a test
> +that depends on having access to a particular piece of hardware or software (not
> +included in the Linux source repo) should be avoided at all costs.
> +
> +Even if you only ever plan on running your KUnit test on your hardware
> +configuration, other people may want to run your tests and may not have access
> +to your hardware. If you write your test to run on UML, then anyone can run your
> +tests without knowing anything about your particular setup, and you can still
> +run your tests on your hardware setup just by compiling for your architecture.
> +
> +.. important::
> +   Always prefer tests that run on UML to tests that only run under a particular
> +   architecture, and always prefer tests that run under QEMU or another easy
> +   (and monitarily free) to obtain software environment to a specific piece of

           monetarily

> +   hardware.
> +
> +Nevertheless, there are still valid reasons to write an architecture or hardware
> +specific test: for example, you might want to test some code that really belongs
> +in ``arch/some-arch/*``. Even so, try your best to write the test so that it
> +does not depend on physical hardware: if some of your test cases don't need the
> +hardware, only require the hardware for tests that actually need it.
> +
> +Now that you have narrowed down exactly what bits are hardware specific, the
> +actual procedure for writing and running the tests is pretty much the same as
> +writing normal KUnit tests. One special caveat is that you have to reset
> +hardware state in between test cases; if this is not possible, you may only be
> +able to run one test case per invocation.
> +
> +.. TODO(brendanhiggins@google.com): Add an actual example of an architecture
> +   dependent KUnit test.


-- 
~Randy
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

  parent reply index

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-23  9:02 [PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 01/19] kunit: test: add KUnit test runner core Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 02/19] kunit: test: add test resource management API Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 03/19] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 04/19] kunit: test: add assertion printing library Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 05/19] kunit: test: add the concept of expectations Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 06/19] lib: enable building KUnit in lib/ Brendan Higgins
2019-09-23 15:35   ` Stephen Boyd
2019-09-23  9:02 ` [PATCH v18 07/19] kunit: test: add initial tests Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 08/19] objtool: add kunit_try_catch_throw to the noreturn list Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 09/19] kunit: test: add support for test abort Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 10/19] kunit: test: add tests for kunit " Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 11/19] kunit: test: add the concept of assertions Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 12/19] kunit: test: add tests for KUnit managed resources Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 13/19] kunit: tool: add Python wrappers for running KUnit tests Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 14/19] kunit: defconfig: add defconfigs for building " Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 15/19] Documentation: kunit: add documentation for KUnit Brendan Higgins
2019-09-23 15:47   ` Randy Dunlap
2019-09-23 18:06     ` Brendan Higgins
2019-09-23 19:49       ` Randy Dunlap
2019-09-23 21:18         ` shuah
2019-09-23 21:30           ` Randy Dunlap
2019-09-24  0:51   ` Randy Dunlap [this message]
2019-09-23  9:02 ` [PATCH v18 16/19] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 17/19] kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec() Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 18/19] MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section Brendan Higgins
2019-09-23  9:02 ` [PATCH v18 19/19] kunit: fix failure to build without printk Brendan Higgins
2019-10-04 21:38 ` [PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Theodore Y. Ts'o
2019-10-04 21:42   ` Linus Torvalds
2019-10-04 21:59     ` shuah
2019-10-04 22:27       ` Brendan Higgins
2019-10-04 22:47         ` shuah
2019-10-04 23:10           ` Brendan Higgins
2019-10-04 23:15             ` shuah
2019-10-04 23:29           ` Theodore Y. Ts'o
2019-10-04 23:52             ` Brendan Higgins
2019-10-04 23:57               ` shuah
2019-10-05  0:33                 ` Brendan Higgins
2019-10-05  0:49                   ` shuah
2019-10-05  1:18                     ` Brendan Higgins
2019-10-06 16:54                       ` Theodore Y. Ts'o
2019-10-06 17:18                         ` Linus Torvalds
2019-10-07  8:40                           ` Brendan Higgins
2019-10-07 14:42                             ` shuah
2019-10-07 14:40                           ` Steven Rostedt
2019-10-07 14:57                             ` shuah
2019-10-07  8:20                         ` Brendan Higgins

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9cd80aa2-fc8d-1fed-838b-cf4951692b6d@infradead.org \
    --to=rdunlap@infradead.org \
    --cc=Alexander.Levin@microsoft.com \
    --cc=Tim.Bird@sony.com \
    --cc=amir73il@gmail.com \
    --cc=brendanhiggins@google.com \
    --cc=corbet@lwn.net \
    --cc=dan.carpenter@oracle.com \
    --cc=daniel@ffwll.ch \
    --cc=devicetree@vger.kernel.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=felixguoxiuping@gmail.com \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jdike@addtoit.com \
    --cc=joel@jms.id.au \
    --cc=jpoimboe@redhat.com \
    --cc=julia.lawall@lip6.fr \
    --cc=keescook@google.com \
    --cc=khilman@baylibre.com \
    --cc=kieran.bingham@ideasonboard.com \
    --cc=knut.omang@oracle.com \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-nvdimm@lists.01.org \
    --cc=linux-um@lists.infradead.org \
    --cc=mcgrof@kernel.org \
    --cc=mpe@ellerman.id.au \
    --cc=peterz@infradead.org \
    --cc=pmladek@suse.com \
    --cc=richard@nod.at \
    --cc=rientjes@google.com \
    --cc=robh@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=sboyd@kernel.org \
    --cc=shuah@kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=tytso@mit.edu \
    --cc=wfg@linux.intel.com \
    --cc=yamada.masahiro@socionext.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NVDIMM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nvdimm/0 linux-nvdimm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nvdimm linux-nvdimm/ https://lore.kernel.org/linux-nvdimm \
		linux-nvdimm@lists.01.org
	public-inbox-index linux-nvdimm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.01.lists.linux-nvdimm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git