Re: [PATCH v3 00/16] python/iotests: Run iotest linters during Python CI

[John Snow <jsnow@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 4908 bytes --]

On Thu, Sep 16, 2021 at 12:10 AM John Snow <jsnow@redhat.com> wrote:

> GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-iotest
> CI: https://gitlab.com/jsnow/qemu/-/pipelines/371611883
> Based-On: <20210915175318.853225-1-hreitz@redhat.com>
>           "[PULL 00/32] Block patches"
>
> Since iotests are such a heavy and prominent user of the Python qemu.qmp
> and qemu.machine packages, it would be convenient if the Python linting
> suite also checked this client for any possible regressions introduced
> by shifting around signatures, types, or interfaces in these packages.
>
> (We'd eventually find those problems when iotest 297 ran, but with
> increasing distance between Python development and Block development,
> the risk of an accidental breakage in this regard increases. I,
> personally, know to run iotests (and especially 297) after changing
> Python code, but not everyone in the future might. Plus, I am lazy, and
> I like only having to push one button.)
>
> Add the ability for the Python CI to run the iotest linters too, which
> means that the iotest linters would be checked against:
>
> - Python 3.6, using a frozen set of linting packages at their oldest
>   supported versions, using 'pipenv'
> - Python 3.6 through Python 3.10 inclusive, using 'tox' and the latest
>   versions of mypy/pylint that happen to be installed during test
>   time. This CI test is allowed to fail with a warning, and can serve
>   as a bellwether for when new incompatible changes may disrupt the
>   linters. Testing against old and new Python interpreters alike can
>   help surface incompatibility issues we may need to be aware of.)
>
> Here are example outputs of those CI jobs with this series applied:
>  - "check-python-pipenv": https://gitlab.com/jsnow/qemu/-/jobs/1377735087
>  - "check-python-tox": https://gitlab.com/jsnow/qemu/-/jobs/1377735088
>
> You can also run these same tests locally from ./python, plus one more:
>
> - "make check-dev" to test against whatever python you have.
> - "make check-pipenv", if you have Python 3.6 and pipenv installed.
> - "make check-tox", to test against multiple python versions you have
> installed,
>                     from 3.6 to 3.10 inclusive. (CI tests against all 5.)
>
> See the old commit message for more sample output, etc.
>
> https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg07056.html
>
> V3:
>  - Added patch 1 which has been submitted separately upstream,
>    but was necessary for testing.
>  - Rebased on top of hreitz/block, which fixed some linting issues.
>  - Added a workaround for a rather nasty mypy bug ... >:(
>
> V2:
>  - Added patches 1-5 which do some more delinting.
>  - Added patch 8, which scans subdirs for tests to lint.
>  - Added patch 17, which improves the speed of mypy analysis.
>  - Patch 14 is different because of the new patch 8.
>
> John Snow (16):
>   python: Update for pylint 2.10
>   iotests/mirror-top-perms: Adjust imports
>   iotests/migrate-bitmaps-postcopy-test: declare instance variables
>   iotests/migrate-bitmaps-test: delint
>   iotests/297: modify is_python_file to work from any CWD
>   iotests/297: Add get_files() function
>   iotests/297: Don't rely on distro-specific linter binaries
>   iotests/297: Create main() function
>   iotests/297: Separate environment setup from test execution
>   iotests/297: Add 'directory' argument to run_linters
>   iotests/297: return error code from run_linters()
>   iotests/297: split linters.py off from 297
>   iotests/linters: Add entry point for Python CI linters
>   iotests/linters: Add workaround for mypy bug #9852
>   python: Add iotest linters to test suite
>   iotests/linters: check mypy files all at once
>
>  python/qemu/machine/machine.py                |   9 +-
>  python/setup.cfg                              |   1 +
>  python/tests/iotests.sh                       |   4 +
>  tests/qemu-iotests/297                        |  81 ++---------
>  tests/qemu-iotests/linters.py                 | 129 ++++++++++++++++++
>  .../tests/migrate-bitmaps-postcopy-test       |   3 +
>  tests/qemu-iotests/tests/migrate-bitmaps-test |  50 ++++---
>  tests/qemu-iotests/tests/mirror-top-perms     |   7 +-
>  8 files changed, 186 insertions(+), 98 deletions(-)
>  create mode 100755 python/tests/iotests.sh
>  create mode 100755 tests/qemu-iotests/linters.py
>
> --
> 2.31.1
>
>
>
FWIW: I sent a new version of a pull request that adds pylint 2.10 *and*
2.11 support; the 2.11 release happened just yesterday, so I am going to
rebase this series.

Additionally, I found a new way to avoid sys.path hacking in all of our
test files entirely, so I will include that in this series, rebase, and
resend extremely soon. If you have difficulties applying this patchset or
testing it, sit tight for a refreshed version -- but most of these patches
can still be reviewed on their own merits in the meantime.

Thanks,
--js

[-- Attachment #2: Type: text/html, Size: 6410 bytes --]