linux-kselftest.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Tales Aparecida <tales.aparecida@gmail.com>
To: Sadiya Kazi <sadiyakazi@google.com>
Cc: linux-kernel@vger.kernel.org, kunit-dev@googlegroups.com,
	linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org,
	davidgow@google.com, corbet@lwn.net, brendan.higgins@linux.dev,
	Trevor Woerner <twoerner@gmail.com>,
	siqueirajordao@riseup.net, mwen@igalia.com,
	andrealmeid@riseup.net, mairacanal@riseup.net,
	Isabella Basso <isabbasso@riseup.net>,
	magalilemes00@gmail.com, tales.aparecida@gmail.com
Subject: [PATCH v2 4/8] Documentation: KUnit: Reword start guide for selecting tests
Date: Sun, 21 Aug 2022 23:26:42 -0300	[thread overview]
Message-ID: <20220822022646.98581-5-tales.aparecida@gmail.com> (raw)
In-Reply-To: <20220822022646.98581-1-tales.aparecida@gmail.com>

Reword "Creating a ``.kunitconfig``" into "Selecting which tests to run"
covering the current alternatives for editing configs and glob-filtering

Signed-off-by: Tales Aparecida <tales.aparecida@gmail.com>
Reviewed-by: Maíra Canal <mairacanal@riseup.net>

---
Notes:
    Avoid hyphen in "test case" and "test suite"
    Fix nit: "any test case that match" -> "...matches"
---
 Documentation/dev-tools/kunit/start.rst | 90 +++++++++++++++++--------
 1 file changed, 63 insertions(+), 27 deletions(-)

diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 9beec7d6ac4b..adf782507999 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -52,27 +52,20 @@ The tests will pass or fail.
 For detailed information on this wrapper, see:
 Documentation/dev-tools/kunit/run_wrapper.rst.
 
-Creating a ``.kunitconfig``
----------------------------
-
-By default, kunit_tool runs a selection of tests. However, you can specify which
-unit tests to run by creating a ``.kunitconfig`` file with kernel config options
-that enable only a specific set of tests and their dependencies.
-The ``.kunitconfig`` file contains a list of kconfig options which are required
-to run the desired targets. The ``.kunitconfig`` also contains any other test
-specific config options, such as test dependencies. For example: the
-``FAT_FS`` tests - ``FAT_KUNIT_TEST``, depends on
-``FAT_FS``. ``FAT_FS`` can be enabled by selecting either ``MSDOS_FS``
-or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
+Selecting which tests to run
+----------------------------
 
-.. code-block:: none
+By default, kunit_tool runs all tests reachable with minimal configuration,
+that is, using default values for most of the kconfig options.  However,
+you can select which tests to run by:
 
-	CONFIG_KUNIT=y
-	CONFIG_MSDOS_FS=y
-	CONFIG_FAT_KUNIT_TEST=y
+- `Customizing Kconfig`_ used to compile the kernel, or
+- `Filtering tests by name`_ to select specifically which compiled tests to run.
 
-1. A good starting point for the ``.kunitconfig`` is the KUnit default config.
-   You can generate it by running:
+Customizing Kconfig
+~~~~~~~~~~~~~~~~~~~
+A good starting point for the ``.kunitconfig`` is the KUnit default config.
+If you didn't run ``kunit.py run`` yet, you can generate it by running:
 
 .. code-block:: bash
 
@@ -84,27 +77,70 @@ or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
    ``.kunitconfig`` lives in the ``--build_dir`` used by kunit.py, which is
    ``.kunit`` by default.
 
-.. note ::
+Before running the tests, kunit_tool ensures that all config options
+set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
+you if you have not included dependencies for the options used.
+
+There are many ways to customize the configurations:
+
+a. Edit ``.kunit/.kunitconfig``. The file should contain the list of kconfig
+   options required to run the desired tests, including their dependencies.
    You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig`` as
    it will enable a number of additional tests that you may not want.
+   If you need to run on an architecture other than UML see :ref:`kunit-on-qemu`.
 
-2. You can then add any other Kconfig options, for example:
+b. Enable additional kconfig options on top of ``.kunit/.kunitconfig``.
+   For example, to include the kernel's linked-list test you can run::
 
-.. code-block:: none
+	./tools/testing/kunit/kunit.py run \
+		--kconfig_add CONFIG_LIST_KUNIT_TEST=y
 
-	CONFIG_LIST_KUNIT_TEST=y
+c. Provide the path of one or more .kunitconfig files from the tree.
+   For example, to run only ``FAT_FS`` and ``EXT4`` tests you can run::
 
-Before running the tests, kunit_tool ensures that all config options
-set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
-you if you have not included dependencies for the options used.
+	./tools/testing/kunit/kunit.py run \
+		--kunitconfig ./fs/fat/.kunitconfig \
+		--kunitconfig ./fs/ext4/.kunitconfig
 
-.. note ::
-   If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
+d. If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
    ``.config`` file. But you can edit the ``.config`` file directly or with
    tools like ``make menuconfig O=.kunit``. As long as its a superset of
    ``.kunitconfig``, kunit.py won't overwrite your changes.
 
 
+.. note ::
+
+	To save a .kunitconfig after finding a satisfactory configuration::
+
+		make savedefconfig O=.kunit
+		cp .kunit/defconfig .kunit/.kunitconfig
+
+Filtering tests by name
+~~~~~~~~~~~~~~~~~~~~~~~
+If you want to be more specific than Kconfig can provide, it is also possible
+to select which tests to execute at boot-time by passing a glob filter
+(read instructions regarding the pattern in the manpage :manpage:`glob(7)`).
+If there is a ``"."`` (period) in the filter, it will be interpreted as a
+separator between the name of the test suite and the test case,
+otherwise, it will be interpreted as the name of the test suite.
+For example, let's assume we are using the default config:
+
+a. inform the name of a test suite, like ``"kunit_executor_test"``,
+   to run every test case it contains::
+
+	./tools/testing/kunit/kunit.py run "kunit_executor_test"
+
+b. inform the name of a test case prefixed by its test suite,
+   like ``"example.example_simple_test"``, to run specifically that test case::
+
+	./tools/testing/kunit/kunit.py run "example.example_simple_test"
+
+c. use wildcard characters (``*?[``) to run any test case that matches the pattern,
+   like ``"*.*64*"`` to run test cases containing ``"64"`` in the name inside
+   any test suite::
+
+	./tools/testing/kunit/kunit.py run "*.*64*"
+
 Running Tests without the KUnit Wrapper
 =======================================
 If you do not want to use the KUnit Wrapper (for example: you want code
-- 
2.37.2


  parent reply	other threads:[~2022-08-22  2:30 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-22  2:26 [PATCH v2 0/8] Documentation: Kunit: clean kunit-tool.rst and start.rst Tales Aparecida
2022-08-22  2:26 ` [PATCH v2 1/8] Documentation: KUnit: remove duplicated docs for kunit_tool Tales Aparecida
2022-08-26  7:29   ` David Gow
2022-08-22  2:26 ` [PATCH v2 2/8] Documentation: KUnit: avoid repeating "kunit.py run" in start.rst Tales Aparecida
2022-08-22 11:32   ` Maíra Canal
2022-08-22 17:36   ` Sadiya Kazi
2022-08-26  7:29   ` David Gow
2022-08-22  2:26 ` [PATCH v2 3/8] Documentation: KUnit: add note about mrproper " Tales Aparecida
2022-08-26  7:29   ` David Gow
2022-08-22  2:26 ` Tales Aparecida [this message]
2022-08-22  6:07   ` [PATCH v2 4/8] Documentation: KUnit: Reword start guide for selecting tests Sadiya Kazi
2022-08-26  7:30   ` David Gow
2022-08-22  2:26 ` [PATCH v2 5/8] Documentation: KUnit: add intro to the getting-started page Tales Aparecida
2022-08-22  6:23   ` Sadiya Kazi
2022-08-26  7:30   ` David Gow
2022-08-22  2:26 ` [PATCH v2 6/8] Documentation: KUnit: update links in the index page Tales Aparecida
2022-08-26  7:30   ` David Gow
2022-08-22  2:26 ` [PATCH v2 7/8] lib: overflow: update reference to kunit-tool Tales Aparecida
2022-08-26  7:30   ` David Gow
2022-08-22  2:26 ` [PATCH v2 8/8] lib: stackinit: " Tales Aparecida
2022-08-26  7:31   ` David Gow

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220822022646.98581-5-tales.aparecida@gmail.com \
    --to=tales.aparecida@gmail.com \
    --cc=andrealmeid@riseup.net \
    --cc=brendan.higgins@linux.dev \
    --cc=corbet@lwn.net \
    --cc=davidgow@google.com \
    --cc=isabbasso@riseup.net \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=magalilemes00@gmail.com \
    --cc=mairacanal@riseup.net \
    --cc=mwen@igalia.com \
    --cc=sadiyakazi@google.com \
    --cc=siqueirajordao@riseup.net \
    --cc=twoerner@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).