From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6653BC4707F for ; Fri, 28 May 2021 02:56:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CF6AE613DA for ; Fri, 28 May 2021 02:56:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF6AE613DA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lmSfL-0002O3-Vx for qemu-devel@archiver.kernel.org; Thu, 27 May 2021 22:56:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lmSLR-00070R-4C for qemu-devel@nongnu.org; Thu, 27 May 2021 22:35:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28956) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lmSLN-0001An-0g for qemu-devel@nongnu.org; Thu, 27 May 2021 22:35:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622169324; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ud9e7mjQJ0SHNQA2haf8QpAQ47fOzGTX6yYCxnTg30s=; b=L1GU0IojaBG34QU7ZH4KpT+/GN/Jg7zO2G5dDh6yY9TKwSr3SjazC9/vIAOtR1gEaCVzDP W3cY4vw0Z30CCn4g9ijcf4X0+ShSSWYoelmTtOrGVknWEHqyT1bVetuFGBmUd7ERXNmeN/ CleSi/J+UWP22OG+T6VPGdjU01EgxQQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-109-KLnpTloUNq-K0tG2NO3N8Q-1; Thu, 27 May 2021 22:35:22 -0400 X-MC-Unique: KLnpTloUNq-K0tG2NO3N8Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1543F800D55; Fri, 28 May 2021 02:35:21 +0000 (UTC) Received: from scv.redhat.com (ovpn-116-137.rdu2.redhat.com [10.10.116.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 76DF71A875; Fri, 28 May 2021 02:35:19 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 40/44] python: add avocado-framework and tests Date: Thu, 27 May 2021 22:32:16 -0400 Message-Id: <20210528023220.417057-41-jsnow@redhat.com> In-Reply-To: <20210528023220.417057-1-jsnow@redhat.com> References: <20210528023220.417057-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.374, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , Max Reitz , Willian Rampazzo , Cleber Rosa , John Snow , Aurelien Jarno Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add avocado-framework to the setup.cfg development dependencies, and add avocado.cfg to store some preferences for how we'd like the test output to look. Finally, add avocado-framework to the Pipfile environment and lock the new dependencies. We are using avocado >= 87.0 here to take advantage of some features that Cleber has helpfully added to make the test output here *very* friendly and easy to read for developers that might chance upon the output in Gitlab CI. [Note: ALL of the dependencies get updated to the most modern versions that exist at the time of this writing. No way around it that I have seen. Not ideal, but so it goes.] Provided you have the right development dependencies (mypy, flake8, isort, pylint, and now avocado-framework) You should be able to run "avocado --config avocado.cfg run tests/" from the python folder to run all of these linters with the correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-id: 20210527211715.394144-28-jsnow@redhat.com Signed-off-by: John Snow --- python/README.rst | 2 ++ python/Pipfile.lock | 8 ++++++++ python/avocado.cfg | 10 ++++++++++ python/setup.cfg | 1 + python/tests/flake8.sh | 2 ++ python/tests/isort.sh | 2 ++ python/tests/mypy.sh | 2 ++ python/tests/pylint.sh | 2 ++ 8 files changed, 29 insertions(+) create mode 100644 python/avocado.cfg create mode 100755 python/tests/flake8.sh create mode 100755 python/tests/isort.sh create mode 100755 python/tests/mypy.sh create mode 100755 python/tests/pylint.sh diff --git a/python/README.rst b/python/README.rst index 954870973d0..6bd2c6b3547 100644 --- a/python/README.rst +++ b/python/README.rst @@ -37,6 +37,8 @@ Files in this directory ----------------------- - ``qemu/`` Python package source directory. +- ``tests/`` Python package tests directory. +- ``avocado.cfg`` Configuration for the Avocado test-runner. - ``MANIFEST.in`` is read by python setuptools, it specifies additional files that should be included by a source distribution. - ``PACKAGE.rst`` is used as the README file that is visible on PyPI.org. diff --git a/python/Pipfile.lock b/python/Pipfile.lock index a2cdc1c50ea..6e344f5fadf 100644 --- a/python/Pipfile.lock +++ b/python/Pipfile.lock @@ -30,6 +30,14 @@ "markers": "python_version ~= '3.6'", "version": "==2.5.6" }, + "avocado-framework": { + "hashes": [ + "sha256:42aa7962df98d6b78d4efd9afa2177226dc630f3d83a2a7d5baf7a0a7da7fa1b", + "sha256:d96ae343abf890e1ef3b3a6af5ce49e35f6bded0715770c4acb325bca555c515" + ], + "markers": "python_version >= '3.6'", + "version": "==88.1" + }, "flake8": { "hashes": [ "sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b", diff --git a/python/avocado.cfg b/python/avocado.cfg new file mode 100644 index 00000000000..10dc6fb6054 --- /dev/null +++ b/python/avocado.cfg @@ -0,0 +1,10 @@ +[simpletests] +# Don't show stdout/stderr in the test *summary* +status.failure_fields = ['status'] + +[job] +# Don't show the full debug.log output; only select stdout/stderr. +output.testlogs.logfiles = ['stdout', 'stderr'] + +# Show full stdout/stderr only on tests that FAIL +output.testlogs.statuses = ['FAIL'] diff --git a/python/setup.cfg b/python/setup.cfg index 39dc135e601..fd325194901 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -25,6 +25,7 @@ packages = [options.extras_require] # Run `pipenv lock --dev` when changing these requirements. devel = + avocado-framework >= 87.0 flake8 >= 3.6.0 isort >= 5.1.2 mypy >= 0.770 diff --git a/python/tests/flake8.sh b/python/tests/flake8.sh new file mode 100755 index 00000000000..51e0788462b --- /dev/null +++ b/python/tests/flake8.sh @@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m flake8 diff --git a/python/tests/isort.sh b/python/tests/isort.sh new file mode 100755 index 00000000000..4480405bfb0 --- /dev/null +++ b/python/tests/isort.sh @@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m isort -c qemu/ diff --git a/python/tests/mypy.sh b/python/tests/mypy.sh new file mode 100755 index 00000000000..5f980f563bb --- /dev/null +++ b/python/tests/mypy.sh @@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m mypy -p qemu diff --git a/python/tests/pylint.sh b/python/tests/pylint.sh new file mode 100755 index 00000000000..4b10b34db7c --- /dev/null +++ b/python/tests/pylint.sh @@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m pylint qemu/ -- 2.31.1