[PULL 40/44] python: add avocado-framework and tests

From: John Snow <jsnow@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Kevin Wolf" <kwolf@redhat.com>,
	"Peter Maydell" <peter.maydell@linaro.org>,
	"Thomas Huth" <thuth@redhat.com>,
	"Eduardo Habkost" <ehabkost@redhat.com>,
	qemu-block@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Max Reitz" <mreitz@redhat.com>,
	"Willian Rampazzo" <willianr@redhat.com>,
	"Cleber Rosa" <crosa@redhat.com>, "John Snow" <jsnow@redhat.com>,
	"Aurelien Jarno" <aurelien@aurel32.net>
Subject: [PULL 40/44] python: add avocado-framework and tests
Date: Thu, 27 May 2021 22:32:16 -0400	[thread overview]
Message-ID: <20210528023220.417057-41-jsnow@redhat.com> (raw)
In-Reply-To: <20210528023220.417057-1-jsnow@redhat.com>

Try using avocado to manage our various tests; even though right now
they're only invoking shell scripts and not really running any
python-native code.

Create tests/, and add shell scripts which call out to mypy, flake8,
pylint and isort to enforce the standards in this directory.

Add avocado-framework to the setup.cfg development dependencies, and add
avocado.cfg to store some preferences for how we'd like the test output
to look.

Finally, add avocado-framework to the Pipfile environment and lock the
new dependencies. We are using avocado >= 87.0 here to take advantage of
some features that Cleber has helpfully added to make the test output
here *very* friendly and easy to read for developers that might chance
upon the output in Gitlab CI.

[Note: ALL of the dependencies get updated to the most modern versions
that exist at the time of this writing. No way around it that I have
seen. Not ideal, but so it goes.]

Provided you have the right development dependencies (mypy, flake8,
isort, pylint, and now avocado-framework) You should be able to run
"avocado --config avocado.cfg run tests/" from the python folder to run
all of these linters with the correct arguments.

(A forthcoming commit adds the much easier 'make check'.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-28-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/README.rst      |  2 ++
 python/Pipfile.lock    |  8 ++++++++
 python/avocado.cfg     | 10 ++++++++++
 python/setup.cfg       |  1 +
 python/tests/flake8.sh |  2 ++
 python/tests/isort.sh  |  2 ++
 python/tests/mypy.sh   |  2 ++
 python/tests/pylint.sh |  2 ++
 8 files changed, 29 insertions(+)
 create mode 100644 python/avocado.cfg
 create mode 100755 python/tests/flake8.sh
 create mode 100755 python/tests/isort.sh
 create mode 100755 python/tests/mypy.sh
 create mode 100755 python/tests/pylint.sh

diff --git a/python/README.rst b/python/README.rst
index 954870973d0..6bd2c6b3547 100644
--- a/python/README.rst
+++ b/python/README.rst
@@ -37,6 +37,8 @@ Files in this directory
 -----------------------
 
 - ``qemu/`` Python package source directory.
+- ``tests/`` Python package tests directory.
+- ``avocado.cfg`` Configuration for the Avocado test-runner.
 - ``MANIFEST.in`` is read by python setuptools, it specifies additional files
   that should be included by a source distribution.
 - ``PACKAGE.rst`` is used as the README file that is visible on PyPI.org.
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index a2cdc1c50ea..6e344f5fadf 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -30,6 +30,14 @@
             "markers": "python_version ~= '3.6'",
             "version": "==2.5.6"
         },
+        "avocado-framework": {
+            "hashes": [
+                "sha256:42aa7962df98d6b78d4efd9afa2177226dc630f3d83a2a7d5baf7a0a7da7fa1b",
+                "sha256:d96ae343abf890e1ef3b3a6af5ce49e35f6bded0715770c4acb325bca555c515"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==88.1"
+        },
         "flake8": {
             "hashes": [
                 "sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b",
diff --git a/python/avocado.cfg b/python/avocado.cfg
new file mode 100644
index 00000000000..10dc6fb6054
--- /dev/null
+++ b/python/avocado.cfg
@@ -0,0 +1,10 @@
+[simpletests]
+# Don't show stdout/stderr in the test *summary*
+status.failure_fields = ['status']
+
+[job]
+# Don't show the full debug.log output; only select stdout/stderr.
+output.testlogs.logfiles = ['stdout', 'stderr']
+
+# Show full stdout/stderr only on tests that FAIL
+output.testlogs.statuses = ['FAIL']
diff --git a/python/setup.cfg b/python/setup.cfg
index 39dc135e601..fd325194901 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -25,6 +25,7 @@ packages =
 [options.extras_require]
 # Run `pipenv lock --dev` when changing these requirements.
 devel =
+    avocado-framework >= 87.0
     flake8 >= 3.6.0
     isort >= 5.1.2
     mypy >= 0.770
diff --git a/python/tests/flake8.sh b/python/tests/flake8.sh
new file mode 100755
index 00000000000..51e0788462b
--- /dev/null
+++ b/python/tests/flake8.sh
@@ -0,0 +1,2 @@
+#!/bin/sh -e
+python3 -m flake8
diff --git a/python/tests/isort.sh b/python/tests/isort.sh
new file mode 100755
index 00000000000..4480405bfb0
--- /dev/null
+++ b/python/tests/isort.sh
@@ -0,0 +1,2 @@
+#!/bin/sh -e
+python3 -m isort -c qemu/
diff --git a/python/tests/mypy.sh b/python/tests/mypy.sh
new file mode 100755
index 00000000000..5f980f563bb
--- /dev/null
+++ b/python/tests/mypy.sh
@@ -0,0 +1,2 @@
+#!/bin/sh -e
+python3 -m mypy -p qemu
diff --git a/python/tests/pylint.sh b/python/tests/pylint.sh
new file mode 100755
index 00000000000..4b10b34db7c
--- /dev/null
+++ b/python/tests/pylint.sh
@@ -0,0 +1,2 @@
+#!/bin/sh -e
+python3 -m pylint qemu/
-- 
2.31.1


