From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4291CC433F5 for ; Fri, 17 Sep 2021 05:58:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C279C61019 for ; Fri, 17 Sep 2021 05:58:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C279C61019 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:39748 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6tM-0004eG-W5 for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:58:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49054) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6iP-0004YY-In for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:47:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:31922) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6iM-0008TN-IL for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:47:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Evr6ermBDoLDDPRcjDpmHYST5gs0KUORaBD78YmWkIE=; b=OOGccpbwMgeib544Xamos47HroSdnEIjhQiYZ3bLv5sBhW5nBn6UTHgShDGNkihtq1aBj9 BlXrCBCTazZVNmzi84LALIX9eiytiHcxj4zMGP0Jd4phhXZwHIuAff7XQ2l5uC9pjs+y98 gdNB10jXOW48FP50CVDekGglB1yMOw8= Received: from mail-oo1-f69.google.com (mail-oo1-f69.google.com [209.85.161.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-147-XdolPulGOUCC2y0fqz26iQ-1; Fri, 17 Sep 2021 01:47:07 -0400 X-MC-Unique: XdolPulGOUCC2y0fqz26iQ-1 Received: by mail-oo1-f69.google.com with SMTP id w6-20020a4adec6000000b0028b7d13a4c8so38057678oou.13 for ; Thu, 16 Sep 2021 22:47:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Evr6ermBDoLDDPRcjDpmHYST5gs0KUORaBD78YmWkIE=; b=dRYBrGdv/YD7tf7qqN069/MCQG9Nt4AZSd4PqT2TT05g2c20EYtlUNp2W/Lzlapp2B HObr/Np3cdU14fVx+P70UC2uIHTc+2CCP1hTIQxLejdXIGnOQluuCG/FZkbOH/0+enTI 2v09ccDdzp89UAM+uEvBO2oeR/8DzV+HO1oSLimhkWyQ0f5NEowVJypBtKfa3CaAmcU2 cYkWSYooWeV25aaeM74tm2wQosHIWkZ0lsbulMdB64m4FxtMjYZ3TrVDJ6lzOSOXGpx4 2S6J4GsPR0aR8NOIqCjEWW4/Lm/VnbTukaZk/mvzHtLLlqJcRRqSnQI5LfU7pFRQFbXj 4N/w== X-Gm-Message-State: AOAM533bf8rdoKOWlQtEsqyVr2mG2Y7+jUAPfKglddAS6kqWh+VpjaGn TYtrjZMEOVX5cQX/sXPvJ4uDJ93MLhHv9YgGM6R54Yp6Q6rMaO7tbMc4is/kyq1wf7jr0UfCCsJ +UE48B8OnBgbIO2UQMgOYDX1rSeaYCzg= X-Received: by 2002:aca:f386:: with SMTP id r128mr11970164oih.168.1631857626405; Thu, 16 Sep 2021 22:47:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwy8UmIy2Aok1llwis3koHVtCVtZmBBxnck4SNTnOn10vtBSfKWxwkoToXISGmtfJvlJ16wwY/NaARcinUB77o= X-Received: by 2002:aca:f386:: with SMTP id r128mr11970147oih.168.1631857626156; Thu, 16 Sep 2021 22:47:06 -0700 (PDT) MIME-Version: 1.0 References: <20210916040955.628560-1-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> From: John Snow Date: Fri, 17 Sep 2021 01:46:55 -0400 Message-ID: Subject: Re: [PATCH v3 00/16] python/iotests: Run iotest linters during Python CI To: qemu-devel Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/alternative; boundary="00000000000021abcd05cc2a76c7" Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --00000000000021abcd05cc2a76c7 Content-Type: text/plain; charset="UTF-8" On Thu, Sep 16, 2021 at 12:10 AM John Snow wrote: > GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-iotest > CI: https://gitlab.com/jsnow/qemu/-/pipelines/371611883 > Based-On: <20210915175318.853225-1-hreitz@redhat.com> > "[PULL 00/32] Block patches" > > Since iotests are such a heavy and prominent user of the Python qemu.qmp > and qemu.machine packages, it would be convenient if the Python linting > suite also checked this client for any possible regressions introduced > by shifting around signatures, types, or interfaces in these packages. > > (We'd eventually find those problems when iotest 297 ran, but with > increasing distance between Python development and Block development, > the risk of an accidental breakage in this regard increases. I, > personally, know to run iotests (and especially 297) after changing > Python code, but not everyone in the future might. Plus, I am lazy, and > I like only having to push one button.) > > Add the ability for the Python CI to run the iotest linters too, which > means that the iotest linters would be checked against: > > - Python 3.6, using a frozen set of linting packages at their oldest > supported versions, using 'pipenv' > - Python 3.6 through Python 3.10 inclusive, using 'tox' and the latest > versions of mypy/pylint that happen to be installed during test > time. This CI test is allowed to fail with a warning, and can serve > as a bellwether for when new incompatible changes may disrupt the > linters. Testing against old and new Python interpreters alike can > help surface incompatibility issues we may need to be aware of.) > > Here are example outputs of those CI jobs with this series applied: > - "check-python-pipenv": https://gitlab.com/jsnow/qemu/-/jobs/1377735087 > - "check-python-tox": https://gitlab.com/jsnow/qemu/-/jobs/1377735088 > > You can also run these same tests locally from ./python, plus one more: > > - "make check-dev" to test against whatever python you have. > - "make check-pipenv", if you have Python 3.6 and pipenv installed. > - "make check-tox", to test against multiple python versions you have > installed, > from 3.6 to 3.10 inclusive. (CI tests against all 5.) > > See the old commit message for more sample output, etc. > > https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg07056.html > > V3: > - Added patch 1 which has been submitted separately upstream, > but was necessary for testing. > - Rebased on top of hreitz/block, which fixed some linting issues. > - Added a workaround for a rather nasty mypy bug ... >:( > > V2: > - Added patches 1-5 which do some more delinting. > - Added patch 8, which scans subdirs for tests to lint. > - Added patch 17, which improves the speed of mypy analysis. > - Patch 14 is different because of the new patch 8. > > John Snow (16): > python: Update for pylint 2.10 > iotests/mirror-top-perms: Adjust imports > iotests/migrate-bitmaps-postcopy-test: declare instance variables > iotests/migrate-bitmaps-test: delint > iotests/297: modify is_python_file to work from any CWD > iotests/297: Add get_files() function > iotests/297: Don't rely on distro-specific linter binaries > iotests/297: Create main() function > iotests/297: Separate environment setup from test execution > iotests/297: Add 'directory' argument to run_linters > iotests/297: return error code from run_linters() > iotests/297: split linters.py off from 297 > iotests/linters: Add entry point for Python CI linters > iotests/linters: Add workaround for mypy bug #9852 > python: Add iotest linters to test suite > iotests/linters: check mypy files all at once > > python/qemu/machine/machine.py | 9 +- > python/setup.cfg | 1 + > python/tests/iotests.sh | 4 + > tests/qemu-iotests/297 | 81 ++--------- > tests/qemu-iotests/linters.py | 129 ++++++++++++++++++ > .../tests/migrate-bitmaps-postcopy-test | 3 + > tests/qemu-iotests/tests/migrate-bitmaps-test | 50 ++++--- > tests/qemu-iotests/tests/mirror-top-perms | 7 +- > 8 files changed, 186 insertions(+), 98 deletions(-) > create mode 100755 python/tests/iotests.sh > create mode 100755 tests/qemu-iotests/linters.py > > -- > 2.31.1 > > > FWIW: I sent a new version of a pull request that adds pylint 2.10 *and* 2.11 support; the 2.11 release happened just yesterday, so I am going to rebase this series. Additionally, I found a new way to avoid sys.path hacking in all of our test files entirely, so I will include that in this series, rebase, and resend extremely soon. If you have difficulties applying this patchset or testing it, sit tight for a refreshed version -- but most of these patches can still be reviewed on their own merits in the meantime. Thanks, --js --00000000000021abcd05cc2a76c7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, Sep 16, 2021 at 12:10 AM John Snow <jsnow@redhat.com> wrote:
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-iotest
CI: https://gitlab.com/jsnow/qemu/-/pipelines/371= 611883
Based-On: <20210915175318.853225-1-hreitz@redhat.com>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "[PULL 00/32] Block patches"
Since iotests are such a heavy and prominent user of the Python qemu.qmp and qemu.machine packages, it would be convenient if the Python linting
suite also checked this client for any possible regressions introduced
by shifting around signatures, types, or interfaces in these packages.

(We'd eventually find those problems when iotest 297 ran, but with
increasing distance between Python development and Block development,
the risk of an accidental breakage in this regard increases. I,
personally, know to run iotests (and especially 297) after changing
Python code, but not everyone in the future might. Plus, I am lazy, and
I like only having to push one button.)

Add the ability for the Python CI to run the iotest linters too, which
means that the iotest linters would be checked against:

- Python 3.6, using a frozen set of linting packages at their oldest
=C2=A0 supported versions, using 'pipenv'
- Python 3.6 through Python 3.10 inclusive, using 'tox' and the lat= est
=C2=A0 versions of mypy/pylint that happen to be installed during test
=C2=A0 time. This CI test is allowed to fail with a warning, and can serve<= br> =C2=A0 as a bellwether for when new incompatible changes may disrupt the =C2=A0 linters. Testing against old and new Python interpreters alike can =C2=A0 help surface incompatibility issues we may need to be aware of.)

Here are example outputs of those CI jobs with this series applied:
=C2=A0- "check-python-pipenv": https://gitl= ab.com/jsnow/qemu/-/jobs/1377735087
=C2=A0- "check-python-tox": https://gitlab.= com/jsnow/qemu/-/jobs/1377735088

You can also run these same tests locally from ./python, plus one more:

- "make check-dev" to test against whatever python you have.
- "make check-pipenv", if you have Python 3.6 and pipenv installe= d.
- "make check-tox", to test against multiple python versions you = have installed,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 from = 3.6 to 3.10 inclusive. (CI tests against all 5.)

See the old commit message for more sample output, etc.

https://lists.gnu.org/archive/htm= l/qemu-devel/2021-06/msg07056.html

V3:
=C2=A0- Added patch 1 which has been submitted separately upstream,
=C2=A0 =C2=A0but was necessary for testing.
=C2=A0- Rebased on top of hreitz/block, which fixed some linting issues. =C2=A0- Added a workaround for a rather nasty mypy bug ... >:(

V2:
=C2=A0- Added patches 1-5 which do some more delinting.
=C2=A0- Added patch 8, which scans subdirs for tests to lint.
=C2=A0- Added patch 17, which improves the speed of mypy analysis.
=C2=A0- Patch 14 is different because of the new patch 8.

John Snow (16):
=C2=A0 python: Update for pylint 2.10
=C2=A0 iotests/mirror-top-perms: Adjust imports
=C2=A0 iotests/migrate-bitmaps-postcopy-test: declare instance variables =C2=A0 iotests/migrate-bitmaps-test: delint
=C2=A0 iotests/297: modify is_python_file to work from any CWD
=C2=A0 iotests/297: Add get_files() function
=C2=A0 iotests/297: Don't rely on distro-specific linter binaries
=C2=A0 iotests/297: Create main() function
=C2=A0 iotests/297: Separate environment setup from test execution
=C2=A0 iotests/297: Add 'directory' argument to run_linters
=C2=A0 iotests/297: return error code from run_linters()
=C2=A0 iotests/297: split linters.py off from 297
=C2=A0 iotests/linters: Add entry point for Python CI linters
=C2=A0 iotests/linters: Add workaround for mypy bug #9852
=C2=A0 python: Add iotest linters to test suite
=C2=A0 iotests/linters: check mypy files all at once

=C2=A0python/qemu/machine/machine.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A09 +-
=C2=A0python/setup.cfg=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A01 +
=C2=A0python/tests/iotests.sh=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A04 +
=C2=A0tests/qemu-iotests/297=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 81 ++---------
=C2=A0tests/qemu-iotests/linters.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0| 129 ++++++++++++++++++
=C2=A0.../tests/migrate-bitmaps-postcopy-test=C2=A0 =C2=A0 =C2=A0 =C2=A0|= =C2=A0 =C2=A03 +
=C2=A0tests/qemu-iotests/tests/migrate-bitmaps-test |=C2=A0 50 ++++---
=C2=A0tests/qemu-iotests/tests/mirror-top-perms=C2=A0 =C2=A0 =C2=A0|=C2=A0 = =C2=A07 +-
=C2=A08 files changed, 186 insertions(+), 98 deletions(-)
=C2=A0create mode 100755 python/tests/iotests.sh
=C2=A0create mode 100755 tests/qemu-iotests/linters.py

--
2.31.1



FWIW: I sent a new ve= rsion of a pull request that adds pylint 2.10 *and* 2.11 support; the 2.11 = release happened just yesterday, so I am going to rebase this series.
=

Additionally, I found a new way to avoid sy= s.path hacking in all of our test files entirely, so I will include that in= this series, rebase, and resend extremely soon. If you have difficulties a= pplying this patchset or testing it, sit tight for a refreshed version -- b= ut most of these patches can still be reviewed on their own merits in the m= eantime.

Thanks,
--js
=C2=A0
--00000000000021abcd05cc2a76c7--