linux-arch.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Brendan Higgins <brendanhiggins@google.com>
To: jdike@addtoit.com, richard@nod.at,
	anton.ivanov@cambridgegreys.com, arnd@arndb.de,
	keescook@chromium.org, skhan@linuxfoundation.org,
	alan.maguire@oracle.com, yzaikin@google.com, davidgow@google.com,
	akpm@linux-foundation.org, rppt@linux.ibm.com,
	frowand.list@gmail.com
Cc: gregkh@linuxfoundation.org, sboyd@kernel.org,
	logang@deltatee.com, mcgrof@kernel.org,
	linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org,
	kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org,
	linux-doc@vger.kernel.org,
	Brendan Higgins <brendanhiggins@google.com>
Subject: [PATCH v6 0/5] kunit: create a centralized executor to dispatch all KUnit tests
Date: Tue,  4 Aug 2020 13:47:40 -0700	[thread overview]
Message-ID: <20200804204745.987648-1-brendanhiggins@google.com> (raw)
Message-ID: <20200804204740.HwxJrf2O3QZ8a8HDm3doI2NFHyUHkGPzY7HP2lsW2DI@z> (raw)

## TL;DR

This patchset adds a centralized executor to dispatch tests rather than
relying on late_initcall to schedule each test suite separately along
with a couple of new features that depend on it.

## What am I trying to do?

Conceptually, I am trying to provide a mechanism by which test suites
can be grouped together so that they can be reasoned about collectively.
The second to last patch in this series add features which depend on
this:

PATCH 04/05 Prints out a test plan[1] right before KUnit tests are run;
            this is valuable because it makes it possible for a test
            harness to detect whether the number of tests run matches
            the number of tests expected to be run, ensuring that no
            tests silently failed. The test plan includes a count of
            tests that will run. With the centralized executor, the
            tests are located in a single data structure and thus can be
            counted.

In addition, by dispatching tests from a single location, we can
guarantee that all KUnit tests run after late_init is complete, which
was a concern during the initial KUnit patchset review (this has not
been a problem in practice, but resolving with certainty is nevertheless
desirable).

Other use cases for this exist, but the above features should provide an
idea of the value that this could provide.

## Changes since last revision:
 - Renamed the KUNIT_TEST_SUITES the KUNIT_TABLE section and moved it
   from INIT_DATA_SECTION to INIT_DATA; this had the additional
   consequence of making the first several architecture specific patches
   unnecessary - suggested by Kees.
 - Dropped the kunit_shutdown patches; I think it makes more sense to
   reintroduce them in a later patchset.

Alan Maguire (1):
  kunit: test: create a single centralized executor for all tests

Brendan Higgins (4):
  vmlinux.lds.h: add linker section for KUnit test suites
  init: main: add KUnit to kernel init
  kunit: test: add test plan to KUnit TAP format
  Documentation: kunit: add a brief blurb about kunit_test_suite

 Documentation/dev-tools/kunit/usage.rst       |   5 ++
 include/asm-generic/vmlinux.lds.h             |  10 ++-
 include/kunit/test.h                          |  76 +++++++++++++-----
 init/main.c                                   |   4 +
 lib/kunit/Makefile                            |   3 +-
 lib/kunit/executor.c                          |  43 ++++++++++
 lib/kunit/test.c                              |  13 +--
 tools/testing/kunit/kunit_parser.py           |  76 ++++++++++++++----
 .../test_is_test_passed-all_passed.log        | Bin 1562 -> 1567 bytes
 .../test_data/test_is_test_passed-crash.log   | Bin 3016 -> 3021 bytes
 .../test_data/test_is_test_passed-failure.log | Bin 1700 -> 1705 bytes
 11 files changed, 180 insertions(+), 50 deletions(-)
 create mode 100644 lib/kunit/executor.c


base-commit: 145ff1ec090dce9beb5a9590b5dc288e7bb2e65d
-- 
2.28.0.163.g6104cc2f0b6-goog

             reply	other threads:[~2020-08-04 20:47 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-04 20:47 Brendan Higgins [this message]
2020-08-04 20:47 ` [PATCH v6 0/5] kunit: create a centralized executor to dispatch all KUnit tests Brendan Higgins
2020-08-04 20:47 ` [PATCH v6 1/5] vmlinux.lds.h: add linker section for KUnit test suites Brendan Higgins
2020-08-04 20:47 ` [PATCH v6 2/5] kunit: test: create a single centralized executor for all tests Brendan Higgins
2020-10-13 11:09   ` Mauro Carvalho Chehab
2020-08-04 20:47 ` [PATCH v6 3/5] init: main: add KUnit to kernel init Brendan Higgins
2020-08-04 20:47 ` [PATCH v6 4/5] kunit: test: add test plan to KUnit TAP format Brendan Higgins
2020-08-04 20:47   ` Brendan Higgins
2020-08-04 20:47 ` [PATCH v6 5/5] Documentation: kunit: add a brief blurb about kunit_test_suite Brendan Higgins
2020-08-05  6:01   ` Stephen Boyd
2020-08-05  6:01     ` Stephen Boyd

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200804204745.987648-1-brendanhiggins@google.com \
    --to=brendanhiggins@google.com \
    --cc=akpm@linux-foundation.org \
    --cc=alan.maguire@oracle.com \
    --cc=anton.ivanov@cambridgegreys.com \
    --cc=arnd@arndb.de \
    --cc=davidgow@google.com \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jdike@addtoit.com \
    --cc=keescook@chromium.org \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=logang@deltatee.com \
    --cc=mcgrof@kernel.org \
    --cc=richard@nod.at \
    --cc=rppt@linux.ibm.com \
    --cc=sboyd@kernel.org \
    --cc=skhan@linuxfoundation.org \
    --cc=yzaikin@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).