From: Simon Glass <sjg@chromium.org>
To: u-boot@lists.denx.de
Subject: [PATCH v2 00/40] test: Refactor tests to have a single test runner
Date: Sat, 30 Jan 2021 20:32:08 -0700 [thread overview]
Message-ID: <20210131033248.1502385-1-sjg@chromium.org> (raw)
At present U-Boot has two broad sets of tests in the C code: driver model
tests which do a lot of pre-/post-init and command tests which do not.
This separation makes it slightly harder to write a test, since there are
two different test-state structures and different rules for running the
two different test types. At present these rules are determined by where
the test is (actually its prefix).
All unit tests can be run from the command line with the 'ut' command.
Since SPL does not have commands, it currently calls the test runner
directly and offers no control of which tests are run.
This seems like a good time to refactor the tests into a unified test
runner, allowing U-Boot proper and SPL to use the same path, perhaps with
some different conditions along the way.
This series sets up a unified runner called ut_run_list(), which runs a
set of tests from a linker_list. Driver model tests are distinguished by
a new UT_TESTF_DM flag so that the necessary init and cleanup can still
be done.
The runner is modified to support running SPL tests that are not solely
for driver model. An example test for FIT loading is added as a
demonstration.
In addition, some documentation is added to explain how to write tests.
This series is available at u-boot-dm/test-working
Changes in v2:
- Use correct rst format for 'Ad-hoc tests' section
- Expand docs on how each type of test is marked
- Put the docs in tests_sandbox since it is more related to sandbox
- Put in a mention of tests_sandbox in the main testing docs
- Add a note that SPL tests can in fact be run individualy
- Document how to run all C tests with 'ut all'
- Fix 'get list' typo
- Fix conditions so non-DM SPL tests are actually run
- Allow for prefix to be NULL, to match function comment
- Add new patches to cover running an SPL test
Simon Glass (40):
doc: Tidy up testing section
doc: Document make tcheck
sandbox: Drop the 'starting...' message unless testing
doc: Explain how to run tests without pytest
doc: Document how sandbox_spl_tests are run
test: Correct setexpr test prefix
test: Mark all driver model tests with a flag
test: Rename test-main.c to test-dm.c
test: Add an overall test runner
test: Create pre/post-run functions
test: Call test_pre/post_run() from driver model tests
test: Move dm_extended_scan() to test_pre_run()
test: Move do_autoprobe() to test_pre_run()
test: Move dm_scan_plat() to test_pre_run()
test: Drop mallinfo() work-around
test: Move console silencing to test_pre_run()
test: Move delay skipping to test_pre_run()
test: Handle driver model reinit in test_pre_run()
test: Drop struct dm_test_state
test: Move dm_test_init() into test-main.c
test: Move dm_test_destroy() into test-main.c
test: Move test running into a separate function
test: Use ut_run_test() to run driver model tests
test: Drop dm_do_test()
test: Add ut_run_test_live_flat() to run tests twice
test: Use a local variable for test state
test: Run driver-model tests using ut_run_list()
test: Use return values in dm_test_run()
test: Move the devicetree check into ut_run_list()
test: Move restoring of driver model state to ut_run_list()
test: log: Rename log main test file to log_ut.c
test: Add a macros for finding tests in linker_lists
test: Rename all linker lists to have a ut_ prefix
test: Allow SPL to run any available test
sandbox: Update os_find_u_boot() to find the .img file
spl: Convert spl_fit to work with sandbox
doc: Move coccinelle into its own section
spl: test: Add a test for spl_load_simple_fit()
test: sandbox: Move sandbox test docs into doc/develop
doc: Explain briefly how to write new tests
arch/sandbox/cpu/os.c | 8 +-
arch/sandbox/cpu/spl.c | 9 +-
arch/sandbox/cpu/start.c | 15 +-
arch/sandbox/include/asm/state.h | 1 +
common/spl/spl.c | 3 +-
common/spl/spl_fit.c | 27 +-
configs/sandbox_spl_defconfig | 2 +-
doc/arch/sandbox.rst | 48 +---
doc/develop/index.rst | 11 +-
doc/develop/py_testing.rst | 3 +-
doc/develop/testing.rst | 46 +++-
doc/develop/tests_sandbox.rst | 203 ++++++++++++++
doc/develop/tests_writing.rst | 335 +++++++++++++++++++++++
include/dm/test.h | 20 +-
include/os.h | 3 +-
include/spl.h | 9 +
include/test/test.h | 38 ++-
include/test/ut.h | 45 ++++
test/Makefile | 3 +
test/bloblist.c | 5 +-
test/bootm.c | 4 +-
test/cmd/mem.c | 4 +-
test/cmd/setexpr.c | 28 +-
test/cmd_ut.c | 38 +--
test/compression.c | 5 +-
test/dm/Makefile | 2 +-
test/dm/core.c | 47 ++--
test/dm/test-dm.c | 50 ++++
test/dm/test-driver.c | 12 +-
test/dm/test-main.c | 229 ----------------
test/dm/test-uclass.c | 10 +-
test/env/cmd_ut_env.c | 4 +-
test/image/Makefile | 5 +
test/image/spl_load.c | 91 +++++++
test/lib/cmd_ut_lib.c | 4 +-
test/log/Makefile | 2 +-
test/log/{test-main.c => log_ut.c} | 4 +-
test/optee/cmd_ut_optee.c | 5 +-
test/overlay/cmd_ut_overlay.c | 5 +-
test/py/tests/test_log.py | 2 +-
test/str_ut.c | 5 +-
test/test-main.c | 416 +++++++++++++++++++++++++++++
test/unicode_ut.c | 4 +-
test/ut.c | 7 +
44 files changed, 1382 insertions(+), 435 deletions(-)
create mode 100644 doc/develop/tests_sandbox.rst
create mode 100644 doc/develop/tests_writing.rst
create mode 100644 test/dm/test-dm.c
delete mode 100644 test/dm/test-main.c
create mode 100644 test/image/Makefile
create mode 100644 test/image/spl_load.c
rename test/log/{test-main.c => log_ut.c} (75%)
create mode 100644 test/test-main.c
--
2.30.0.365.g02bc693789-goog
next reply other threads:[~2021-01-31 3:32 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-31 3:32 Simon Glass [this message]
2021-01-31 3:32 ` [PATCH v2 01/40] doc: Tidy up testing section Simon Glass
2021-01-31 3:32 ` [PATCH v2 02/40] doc: Document make tcheck Simon Glass
2021-01-31 3:32 ` [PATCH v2 03/40] sandbox: Drop the 'starting...' message unless testing Simon Glass
2021-01-31 3:32 ` [PATCH v2 04/40] doc: Explain how to run tests without pytest Simon Glass
2021-01-31 3:32 ` [PATCH v2 05/40] doc: Document how sandbox_spl_tests are run Simon Glass
2021-02-01 15:06 ` Pratyush Yadav
2021-01-31 3:32 ` [PATCH v2 06/40] test: Correct setexpr test prefix Simon Glass
2021-01-31 3:32 ` [PATCH v2 07/40] test: Mark all driver model tests with a flag Simon Glass
2021-01-31 3:32 ` [PATCH v2 08/40] test: Rename test-main.c to test-dm.c Simon Glass
2021-01-31 3:32 ` [PATCH v2 09/40] test: Add an overall test runner Simon Glass
2021-01-31 3:32 ` [PATCH v2 10/40] test: Create pre/post-run functions Simon Glass
2021-01-31 3:32 ` [PATCH v2 11/40] test: Call test_pre/post_run() from driver model tests Simon Glass
2021-01-31 3:32 ` [PATCH v2 12/40] test: Move dm_extended_scan() to test_pre_run() Simon Glass
2021-01-31 3:32 ` [PATCH v2 13/40] test: Move do_autoprobe() " Simon Glass
2021-01-31 3:32 ` [PATCH v2 14/40] test: Move dm_scan_plat() " Simon Glass
2021-01-31 3:32 ` [PATCH v2 15/40] test: Drop mallinfo() work-around Simon Glass
2021-01-31 3:32 ` [PATCH v2 16/40] test: Move console silencing to test_pre_run() Simon Glass
2021-01-31 3:32 ` [PATCH v2 17/40] test: Move delay skipping " Simon Glass
2021-01-31 3:32 ` [PATCH v2 18/40] test: Handle driver model reinit in test_pre_run() Simon Glass
2021-01-31 3:32 ` [PATCH v2 19/40] test: Drop struct dm_test_state Simon Glass
2021-01-31 3:32 ` [PATCH v2 20/40] test: Move dm_test_init() into test-main.c Simon Glass
2021-01-31 3:32 ` [PATCH v2 21/40] test: Move dm_test_destroy() " Simon Glass
2021-01-31 3:32 ` [PATCH v2 22/40] test: Move test running into a separate function Simon Glass
2021-01-31 3:32 ` [PATCH v2 23/40] test: Use ut_run_test() to run driver model tests Simon Glass
2021-01-31 3:32 ` [PATCH v2 24/40] test: Drop dm_do_test() Simon Glass
2021-01-31 3:32 ` [PATCH v2 25/40] test: Add ut_run_test_live_flat() to run tests twice Simon Glass
2021-01-31 3:32 ` [PATCH v2 26/40] test: Use a local variable for test state Simon Glass
2021-02-01 16:47 ` Alex G.
2021-01-31 3:32 ` [PATCH v2 27/40] test: Run driver-model tests using ut_run_list() Simon Glass
2021-01-31 3:32 ` [PATCH v2 28/40] test: Use return values in dm_test_run() Simon Glass
2021-01-31 3:32 ` [PATCH v2 29/40] test: Move the devicetree check into ut_run_list() Simon Glass
2021-01-31 3:32 ` [PATCH v2 30/40] test: Move restoring of driver model state to ut_run_list() Simon Glass
2021-01-31 3:32 ` [PATCH v2 31/40] test: log: Rename log main test file to log_ut.c Simon Glass
2021-01-31 3:32 ` [PATCH v2 32/40] test: Add a macros for finding tests in linker_lists Simon Glass
2021-01-31 3:32 ` [PATCH v2 33/40] test: Rename all linker lists to have a ut_ prefix Simon Glass
2021-01-31 3:32 ` [PATCH v2 34/40] test: Allow SPL to run any available test Simon Glass
2021-01-31 3:32 ` [PATCH v2 35/40] sandbox: Update os_find_u_boot() to find the .img file Simon Glass
2021-01-31 3:32 ` [PATCH v2 36/40] spl: Convert spl_fit to work with sandbox Simon Glass
2021-01-31 3:32 ` [PATCH v2 37/40] doc: Move coccinelle into its own section Simon Glass
2021-01-31 3:32 ` [PATCH v2 38/40] spl: test: Add a test for spl_load_simple_fit() Simon Glass
2021-01-31 3:32 ` [PATCH v2 39/40] test: sandbox: Move sandbox test docs into doc/develop Simon Glass
2021-01-31 3:32 ` [PATCH v2 40/40] doc: Explain briefly how to write new tests Simon Glass
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210131033248.1502385-1-sjg@chromium.org \
--to=sjg@chromium.org \
--cc=u-boot@lists.denx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.