All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 00/16] Preventing style regressions using check-package
@ 2022-07-24  5:48 Ricardo Martincoski
  2022-07-24  5:48 ` [Buildroot] [PATCH 01/16] DEVELOPERS: update entries for Ricardo Martincoski Ricardo Martincoski
                   ` (15 more replies)
  0 siblings, 16 replies; 32+ messages in thread
From: Ricardo Martincoski @ 2022-07-24  5:48 UTC (permalink / raw)
  To: buildroot; +Cc: Ricardo Martincoski

Hello,

This series:
- adds 'make check-package' to the workflow of sending patches to the
  list;
- ensures reproducible results for check-package (that will call
  shellcheck and flake8);
- merges check-flake8 into check-package;
- makes check-package to check all shell scripts in the tree;
- makes check-package to check new directories in the tree;

The results of 'make check-package' already depends on the version of
shellcheck installed on the host and would also depend on the version of
flake8 installed on the host, making the results even less reproducible.
So in order to ensure reproducible results, this series makes
'make check-package' always run inside the docker image.

After merging check-flake8 into check-package, 'make check-package'
takes much more time to run, 1-2 minutes instead of seconds.
But:
 - it ensures reproducible results, since all tests run inside the
   docker image;
 - it simplifies the dependency to the host tools when checking style:
   instead of installing shellcheck, flake8, and a growing number of
   python modules, the developer only needs docker and the docker image;
 - it standardizes the use and the developer must know only about one
   target: 'make check-package' instead of 'make check-flake8', 'make
   check-shellcheck', 'make check-somethingelse';

Patch 1 is completely unrelated, just to update my entries on DEVELOPERS
file before changing it in a later patch.

Patches 2 to 7 ensure reproducible on check-package results, standardize
the check for all files in the tree (yet limited by the list of
directories the check-package script knows that it understand) and add a
fine-grained ignore list for warnings in the tree, making the creation
of a new check_function in check-package to take effect immediately when
added to the tree.

Patch 8 adds 'make check-package' to the default workflow of sending
patches, since its results are now reproducible and depend only on
docker installed on the host.

Patches 9 to 14 expand the list of files check-package do understand:
all shell scripts, all python scripts, and new directories: board,
support, utils.
At the end of this series, this is the output from check-package:
374966 lines processed
0 warnings generated

Patch 15 is an example of fixing an ignored warning (and removing the
entry from the ignored list).

Patch 16 is an example of adding a new check_function to check-package
without fixing all warnings it would generate first.
Notice that even we are adding warnings to the ignore list, any new
patch sent to the list that triggers that new check_funtion will
generate warnings, preventing regressions.

This series intentionally do NOT make pkg-stats to check shell and
python scripts in the tree. pkg-stats remains counting warnings only to
files that do not depend on external tools (shellcheck, flake8) and
therefore already have reproducible results.
This change of checking all files in pkg-stats to count in the Warning
column is technically feasible, but in order to ensure reproducible
results, pkg-stats would need to run inside the docker image, and new
python modules would need to be added to the image: python3-aiohttp,
requests, ... too much complexity, I think.

Regards,
Ricardo

Ricardo Martincoski (16):
  DEVELOPERS: update entries for Ricardo Martincoski
  utils/check-package: improve shellcheck reproducibility
  utils/check-package: create an ignore list
  support/testing: test check-package ignore list
  utils/check-package: add --failed-only
  Makefile: make check-package assume a git tree
  Makefile: run check-* inside docker image
  docs/manual: check-package before submitting patch
  support/docker: add python3-magic
  utils/check-package: check all shell scripts
  utils/check-package: check files in utils/
  utils/check-package: check files in board/
  utils/check-package: check files in support/
  Makefile: merge check-flake8 into check-package
  utils/docker-run: fix shellcheck warnings
  utils/checkpackagelib: warn about $(HOST_DIR)/usr

 .checkpackageignore                           | 1102 +++++++++++++++++
 .shellcheckrc                                 |    0
 DEVELOPERS                                    |   10 +-
 Makefile                                      |   20 +-
 docs/manual/contribute.txt                    |    6 +
 support/docker/Dockerfile                     |    2 +
 support/misc/gitlab-ci.yml.in                 |    4 -
 support/scripts/generate-gitlab-ci-yml        |    2 +-
 support/scripts/pkg-stats                     |    2 +-
 .../utils/br2-external/.checkpackageignore    |    1 +
 .../br2-external/package/.checkpackageignore  |    1 +
 .../package/.checkpackageignore_outdated      |    1 +
 .../tests/utils/br2-external/utils/x-python   |    2 +
 .../utils/br2-external/utils/x-shellscript    |    2 +
 .../testing/tests/utils/test_check_package.py |   65 +
 utils/check-package                           |  122 +-
 utils/checkpackagelib/lib_mk.py               |   12 +
 utils/checkpackagelib/lib_python.py           |    1 +
 utils/checkpackagelib/lib_shellscript.py      |    5 +
 utils/checkpackagelib/lib_sysv.py             |    3 +
 utils/checkpackagelib/test_lib_mk.py          |   23 +
 utils/checkpackagelib/test_tool.py            |   28 +
 utils/checkpackagelib/tool.py                 |   20 +
 utils/docker-run                              |    2 +-
 24 files changed, 1396 insertions(+), 40 deletions(-)
 create mode 100644 .checkpackageignore
 create mode 100644 .shellcheckrc
 create mode 100644 support/testing/tests/utils/br2-external/.checkpackageignore
 create mode 100644 support/testing/tests/utils/br2-external/package/.checkpackageignore
 create mode 100644 support/testing/tests/utils/br2-external/package/.checkpackageignore_outdated
 create mode 100644 support/testing/tests/utils/br2-external/utils/x-python
 create mode 100755 support/testing/tests/utils/br2-external/utils/x-shellscript
 create mode 100644 utils/checkpackagelib/lib_python.py
 create mode 100644 utils/checkpackagelib/lib_shellscript.py

-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 32+ messages in thread

end of thread, other threads:[~2023-04-09 21:05 UTC | newest]

Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-24  5:48 [Buildroot] [PATCH 00/16] Preventing style regressions using check-package Ricardo Martincoski
2022-07-24  5:48 ` [Buildroot] [PATCH 01/16] DEVELOPERS: update entries for Ricardo Martincoski Ricardo Martincoski
2022-07-25 22:21   ` Arnout Vandecappelle
2022-07-24  5:48 ` [Buildroot] [PATCH 02/16] utils/check-package: improve shellcheck reproducibility Ricardo Martincoski
2022-07-25 22:21   ` Arnout Vandecappelle
2022-07-24  5:48 ` [Buildroot] [PATCH 03/16] utils/check-package: create an ignore list Ricardo Martincoski
2022-07-24  5:49 ` [Buildroot] [PATCH 04/16] support/testing: test check-package " Ricardo Martincoski
2022-07-24  5:49 ` [Buildroot] [PATCH 05/16] utils/check-package: add --failed-only Ricardo Martincoski
2022-07-24  5:49 ` [Buildroot] [PATCH 06/16] Makefile: make check-package assume a git tree Ricardo Martincoski
2022-07-27 12:54   ` Romain Naour
2022-07-31 14:31     ` Ricardo Martincoski
2022-07-31 19:23       ` Thomas Petazzoni via buildroot
2022-07-24  5:49 ` [Buildroot] [PATCH 07/16] Makefile: run check-* inside docker image Ricardo Martincoski
2022-07-27 13:16   ` Romain Naour
2022-07-31 14:34     ` Ricardo Martincoski
2022-07-24  5:49 ` [Buildroot] [PATCH 08/16] docs/manual: check-package before submitting patch Ricardo Martincoski
2022-07-27 13:22   ` Romain Naour
2022-07-31 14:37     ` Ricardo Martincoski
2022-07-24  5:49 ` [Buildroot] [PATCH 09/16] support/docker: add python3-magic Ricardo Martincoski
2022-07-24  5:49 ` [Buildroot] [PATCH 10/16] utils/check-package: check all shell scripts Ricardo Martincoski
2023-04-09 21:01   ` Arnout Vandecappelle
2022-07-24  5:49 ` [Buildroot] [PATCH 11/16] utils/check-package: check files in utils/ Ricardo Martincoski
2023-04-09 21:02   ` Arnout Vandecappelle
2022-07-24  5:49 ` [Buildroot] [PATCH 12/16] utils/check-package: check files in board/ Ricardo Martincoski
2023-04-09 21:02   ` Arnout Vandecappelle
2022-07-24  5:49 ` [Buildroot] [PATCH 13/16] utils/check-package: check files in support/ Ricardo Martincoski
2023-04-09 21:03   ` Arnout Vandecappelle
2022-07-24  5:49 ` [Buildroot] [PATCH 14/16] Makefile: merge check-flake8 into check-package Ricardo Martincoski
2023-04-09 21:04   ` Arnout Vandecappelle
2022-07-24  5:49 ` [Buildroot] [PATCH 15/16] utils/docker-run: fix shellcheck warnings Ricardo Martincoski
2023-04-09 21:05   ` Arnout Vandecappelle
2022-07-24  5:49 ` [Buildroot] [PATCH 16/16] utils/checkpackagelib: warn about $(HOST_DIR)/usr Ricardo Martincoski

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.