linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 00/11] Add Kconfig unit tests
@ 2018-03-02  4:31 Masahiro Yamada
  2018-03-02  4:31 ` [PATCH v2 01/11] kbuild: define PYTHON2 and PYTHON3 variables instead of PYTHON Masahiro Yamada
                   ` (10 more replies)
  0 siblings, 11 replies; 20+ messages in thread
From: Masahiro Yamada @ 2018-03-02  4:31 UTC (permalink / raw)
  To: linux-kbuild
  Cc: Sam Ravnborg, Michal Marek, Ulf Magnusson, Randy Dunlap,
	Luis R . Rodriguez, Masahiro Yamada, Tony Luck, linux-ia64,
	linux-kernel, Fenghua Yu


I am applying various cleanups to Kconfig these days.

However, I fear regressions.  I have been thinking of unit-tests.

There are various cryptic parts in Kconfig and corner cases where
it is difficult to notice breakage.  If unit-tests cover those,
I will be able to apply changes more confidently.

This series introduces a unit-tests.  The framework is written based
on pytest.  Also, this is written in Python 3.  Python 2 will retire
in 2020.  So, I believe new python tools should be written in Python 3.

This is my Python 3 and pytest versions.

$ python3 --version
Python 3.5.2
$ python3 -m pytest --version
This is pytest version 3.4.1, imported from /home/masahiro/.local/lib/python3.5/site-packages/pytest.py

How to use?
-----------

Please make sure Python3 and pytest are installed on your system.

Then, run "make testconfig"

The result looks like as follows:

$ make testconfig
python3 -B -m pytest ./scripts/kconfig/tests \
-o cache_dir=/home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests/.cache \

================================= test session starts =================================
platform linux -- Python 3.5.2, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
cachedir: scripts/kconfig/tests/.cache
rootdir: /home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests, inifile: pytest.ini
collected 14 items

scripts/kconfig/tests/auto_submenu/__init__.py::test PASSED                     [  7%]
scripts/kconfig/tests/choice/__init__.py::test_oldask0 PASSED                   [ 14%]
scripts/kconfig/tests/choice/__init__.py::test_oldask1 PASSED                   [ 21%]
scripts/kconfig/tests/choice/__init__.py::test_allyes PASSED                    [ 28%]
scripts/kconfig/tests/choice/__init__.py::test_allmod PASSED                    [ 35%]
scripts/kconfig/tests/choice/__init__.py::test_allno PASSED                     [ 42%]
scripts/kconfig/tests/choice/__init__.py::test_alldef PASSED                    [ 50%]
scripts/kconfig/tests/choice_value_with_m_dep/__init__.py::test PASSED          [ 57%]
scripts/kconfig/tests/err_recursive_inc/__init__.py::test PASSED                [ 64%]
scripts/kconfig/tests/inter_choice/__init__.py::test PASSED                     [ 71%]
scripts/kconfig/tests/new_choice_with_dep/__init__.py::test PASSED              [ 78%]
scripts/kconfig/tests/no_write_if_dep_unmet/__init__.py::test PASSED            [ 85%]
scripts/kconfig/tests/rand_nested_choice/__init__.py::test PASSED               [ 92%]
scripts/kconfig/tests/warn_recursive_dep/__init__.py::test PASSED               [100%]


Changes in v2:
  - Add backward compatibility for PYTHON
  - Add 'randconfig' support
  - Add docstring
  - Fix various style errors detected by 'pep8' and 'pep257'
  - coding style clean-up based on PEP8, PEP257
  - shorten directory name  'auto_submenu_creation' -> 'auto_submenu'
  - coding style clean-up based on PEP8, PEP257
  - coding style clean-up based on PEP8, PEP257
  - coding style clean-up based on PEP8, PEP257
  - coding style clean-up based on PEP8, PEP257
  - Newly added
  - Newly added
  - Fix missing end quote
  - coding style clean-up based on PEP8, PEP257
  - coding style clean-up based on PEP8, PEP257

Masahiro Yamada (11):
  kbuild: define PYTHON2 and PYTHON3 variables instead of PYTHON
  kconfig: unittest: add framework for Kconfig unit testing
  kconfig: unittest: add basic 'choice' tests
  kconfig: unittest: test automatic submenu creation
  kconfig: unittest: test if new symbols in choice are asked
  kconfig: unittest: check unneeded "is not set" with unmet dependency
  kconfig: unittest: check visibility of tri-choice values in y choice
  kconfig: unittest: test defconfig when two choices interact
  kconfig: unittest: test randconfig for choice in choice
  kconfig: unittest: test if recursive dependencies are detected
  kconfig: unittest: test if recursive inclusion is detected

 Makefile                                           |   5 +-
 arch/ia64/Makefile                                 |   2 +-
 scripts/kconfig/Makefile                           |   8 +
 scripts/kconfig/tests/auto_submenu/Kconfig         |  50 ++++
 scripts/kconfig/tests/auto_submenu/__init__.py     |  12 +
 scripts/kconfig/tests/auto_submenu/expected_stdout |  10 +
 scripts/kconfig/tests/choice/Kconfig               |  54 ++++
 scripts/kconfig/tests/choice/__init__.py           |  40 +++
 .../kconfig/tests/choice/alldef_expected_config    |   5 +
 .../kconfig/tests/choice/allmod_expected_config    |   9 +
 scripts/kconfig/tests/choice/allno_expected_config |   5 +
 .../kconfig/tests/choice/allyes_expected_config    |   9 +
 .../kconfig/tests/choice/oldask0_expected_stdout   |  10 +
 scripts/kconfig/tests/choice/oldask1_config        |   2 +
 .../kconfig/tests/choice/oldask1_expected_stdout   |  15 ++
 .../kconfig/tests/choice_value_with_m_dep/Kconfig  |  20 ++
 .../tests/choice_value_with_m_dep/__init__.py      |  15 ++
 .../kconfig/tests/choice_value_with_m_dep/config   |   2 +
 .../tests/choice_value_with_m_dep/expected_config  |   3 +
 .../tests/choice_value_with_m_dep/expected_stdout  |   4 +
 scripts/kconfig/tests/conftest.py                  | 291 +++++++++++++++++++++
 scripts/kconfig/tests/err_recursive_inc/Kconfig    |   1 +
 .../kconfig/tests/err_recursive_inc/Kconfig.inc    |   1 +
 .../kconfig/tests/err_recursive_inc/__init__.py    |  10 +
 .../tests/err_recursive_inc/expected_stderr        |   4 +
 scripts/kconfig/tests/inter_choice/Kconfig         |  24 ++
 scripts/kconfig/tests/inter_choice/__init__.py     |  14 +
 scripts/kconfig/tests/inter_choice/defconfig       |   1 +
 scripts/kconfig/tests/inter_choice/expected_config |   4 +
 scripts/kconfig/tests/new_choice_with_dep/Kconfig  |  37 +++
 .../kconfig/tests/new_choice_with_dep/__init__.py  |  14 +
 scripts/kconfig/tests/new_choice_with_dep/config   |   3 +
 .../tests/new_choice_with_dep/expected_stdout      |  10 +
 .../kconfig/tests/no_write_if_dep_unmet/Kconfig    |  14 +
 .../tests/no_write_if_dep_unmet/__init__.py        |  19 ++
 scripts/kconfig/tests/no_write_if_dep_unmet/config |   1 +
 .../tests/no_write_if_dep_unmet/expected_config    |   5 +
 scripts/kconfig/tests/pytest.ini                   |   6 +
 scripts/kconfig/tests/rand_nested_choice/Kconfig   |  33 +++
 .../kconfig/tests/rand_nested_choice/__init__.py   |  16 ++
 .../tests/rand_nested_choice/expected_stdout0      |   2 +
 .../tests/rand_nested_choice/expected_stdout1      |   4 +
 .../tests/rand_nested_choice/expected_stdout2      |   5 +
 scripts/kconfig/tests/warn_recursive_dep/Kconfig   |  62 +++++
 .../kconfig/tests/warn_recursive_dep/__init__.py   |   9 +
 .../tests/warn_recursive_dep/expected_stderr       |  30 +++
 46 files changed, 897 insertions(+), 3 deletions(-)
 create mode 100644 scripts/kconfig/tests/auto_submenu/Kconfig
 create mode 100644 scripts/kconfig/tests/auto_submenu/__init__.py
 create mode 100644 scripts/kconfig/tests/auto_submenu/expected_stdout
 create mode 100644 scripts/kconfig/tests/choice/Kconfig
 create mode 100644 scripts/kconfig/tests/choice/__init__.py
 create mode 100644 scripts/kconfig/tests/choice/alldef_expected_config
 create mode 100644 scripts/kconfig/tests/choice/allmod_expected_config
 create mode 100644 scripts/kconfig/tests/choice/allno_expected_config
 create mode 100644 scripts/kconfig/tests/choice/allyes_expected_config
 create mode 100644 scripts/kconfig/tests/choice/oldask0_expected_stdout
 create mode 100644 scripts/kconfig/tests/choice/oldask1_config
 create mode 100644 scripts/kconfig/tests/choice/oldask1_expected_stdout
 create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/Kconfig
 create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/__init__.py
 create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/config
 create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/expected_config
 create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/expected_stdout
 create mode 100644 scripts/kconfig/tests/conftest.py
 create mode 100644 scripts/kconfig/tests/err_recursive_inc/Kconfig
 create mode 100644 scripts/kconfig/tests/err_recursive_inc/Kconfig.inc
 create mode 100644 scripts/kconfig/tests/err_recursive_inc/__init__.py
 create mode 100644 scripts/kconfig/tests/err_recursive_inc/expected_stderr
 create mode 100644 scripts/kconfig/tests/inter_choice/Kconfig
 create mode 100644 scripts/kconfig/tests/inter_choice/__init__.py
 create mode 100644 scripts/kconfig/tests/inter_choice/defconfig
 create mode 100644 scripts/kconfig/tests/inter_choice/expected_config
 create mode 100644 scripts/kconfig/tests/new_choice_with_dep/Kconfig
 create mode 100644 scripts/kconfig/tests/new_choice_with_dep/__init__.py
 create mode 100644 scripts/kconfig/tests/new_choice_with_dep/config
 create mode 100644 scripts/kconfig/tests/new_choice_with_dep/expected_stdout
 create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/Kconfig
 create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/__init__.py
 create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/config
 create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/expected_config
 create mode 100644 scripts/kconfig/tests/pytest.ini
 create mode 100644 scripts/kconfig/tests/rand_nested_choice/Kconfig
 create mode 100644 scripts/kconfig/tests/rand_nested_choice/__init__.py
 create mode 100644 scripts/kconfig/tests/rand_nested_choice/expected_stdout0
 create mode 100644 scripts/kconfig/tests/rand_nested_choice/expected_stdout1
 create mode 100644 scripts/kconfig/tests/rand_nested_choice/expected_stdout2
 create mode 100644 scripts/kconfig/tests/warn_recursive_dep/Kconfig
 create mode 100644 scripts/kconfig/tests/warn_recursive_dep/__init__.py
 create mode 100644 scripts/kconfig/tests/warn_recursive_dep/expected_stderr

-- 
2.7.4

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2018-03-07  2:23 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-02  4:31 [PATCH v2 00/11] Add Kconfig unit tests Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 01/11] kbuild: define PYTHON2 and PYTHON3 variables instead of PYTHON Masahiro Yamada
2018-03-06 17:16   ` Tony Luck
2018-03-07  2:22     ` Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 02/11] kconfig: unittest: add framework for Kconfig unit testing Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 03/11] kconfig: unittest: add basic 'choice' tests Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 04/11] kconfig: unittest: test automatic submenu creation Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 05/11] kconfig: unittest: test if new symbols in choice are asked Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 06/11] kconfig: unittest: check unneeded "is not set" with unmet dependency Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 07/11] kconfig: unittest: check visibility of tri-choice values in y choice Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 08/11] kconfig: unittest: test defconfig when two choices interact Masahiro Yamada
2018-03-02 10:41   ` Ulf Magnusson
2018-03-06  9:21     ` Masahiro Yamada
2018-03-02  4:31 ` [PATCH v2 09/11] kconfig: unittest: test randconfig for choice in choice Masahiro Yamada
2018-03-02 11:14   ` Ulf Magnusson
     [not found]     ` <CAB=NE6V+5MK-ytS+-QBppSBLz7T4hRQV8eMA2Vv9-crcbj7NYg@mail.gmail.com>
2018-03-03  9:34       ` Ulf Magnusson
2018-03-06  9:36         ` Masahiro Yamada
2018-03-02  4:32 ` [PATCH v2 10/11] kconfig: unittest: test if recursive dependencies are detected Masahiro Yamada
2018-03-02  4:32 ` [PATCH v2 11/11] kconfig: unittest: test if recursive inclusion is detected Masahiro Yamada
2018-03-02  7:08   ` Masahiro Yamada

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).