All of lore.kernel.org
 help / color / mirror / Atom feed
From: Cleber Rosa <crosa@redhat.com>
To: qemu-devel@nongnu.org
Cc: qemu-ppc@nongnu.org, "Thomas Huth" <thuth@redhat.com>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Yoshinori Sato" <ysato@users.sourceforge.jp>,
	"Pavel Dovgalyuk" <pavel.dovgaluk@ispras.ru>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Hervé Poussineau" <hpoussin@reactos.org>,
	"Greg Kurz" <groug@kaod.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Auger Eric" <eric.auger@redhat.com>,
	"Kamil Rytarowski" <kamil@netbsd.org>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Reinoud Zandijk" <reinoud@netbsd.org>,
	"David Gibson" <david@gibson.dropbear.id.au>,
	"Willian Rampazzo" <wrampazz@redhat.com>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Alexandre Iooss" <erdnaxe@crans.org>,
	"Mahmoud Mandour" <ma.mandourr@gmail.com>,
	"Ryo ONODERA" <ryoon@netbsd.org>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Eduardo Habkost" <ehabkost@redhat.com>
Subject: [PATCH 01/16] Acceptance Tests: bump Avocado requirement to 91.0
Date: Fri, 24 Sep 2021 14:54:51 -0400	[thread overview]
Message-ID: <20210924185506.2542588-2-crosa@redhat.com> (raw)
In-Reply-To: <20210924185506.2542588-1-crosa@redhat.com>

Avocado 91.0, brings, among other changes, a switch to a new runner
implementation, known as "nrunner".  While my personal testing shows
that it's suitable for running the QEMU tests, there are some
considerations that should be addressed before the change.

For instance, the "nrunner" implementation will run tests in parallel
by default.  With more tests running, and resources shared, some tests
may fail with timeouts on some environments.

So, for now, let's bump the Avocado version to allow for the other
features to be used.  And the "nrunner" implementation to be used by
those that choose to do so.

More information can be found at the release notes at:
   https://avocado-framework.readthedocs.io/en/latest/releases/91_0.html

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 docs/devel/testing.rst | 12 ++++++++++++
 tests/Makefile.include |  2 ++
 tests/requirements.txt |  2 +-
 3 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4a0abbf23d..2818c58ff8 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -740,6 +740,18 @@ may be invoked by running:
 
   tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/
 
+Running tests in parallel
+-------------------------
+
+The current ``make check-acceptance`` target continues to use the
+Avocado runner implementation simply known as "runner".  But, it's now
+possible to opt-in and choose the "nrunner" implementation, which,
+among other things, allows for parallel execution of tests:
+
+.. code::
+
+  make AVOCADO_RUNNER=nrunner check-acceptance
+
 Manual Installation
 -------------------
 
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6e16c05f10..ac289a2e41 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -92,6 +92,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
 AVOCADO_SHOW=app
+AVOCADO_RUNNER=runner
 AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
 
 $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
@@ -128,6 +129,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
 	$(call quiet-command, \
             $(TESTS_VENV_DIR)/bin/python -m avocado \
             --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
+            --test-runner=$(AVOCADO_RUNNER) \
             --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
             $(AVOCADO_TAGS) \
             $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
diff --git a/tests/requirements.txt b/tests/requirements.txt
index a21b59b443..40af24c664 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,5 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==88.1
+avocado-framework==91.0
 pycdlib==1.11.0
-- 
2.31.1



  reply	other threads:[~2021-09-24 19:08 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-24 18:54 [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements Cleber Rosa
2021-09-24 18:54 ` Cleber Rosa [this message]
2021-10-06 18:39   ` [PATCH 01/16] Acceptance Tests: bump Avocado requirement to 91.0 Willian Rampazzo
2021-09-24 18:54 ` [PATCH 02/16] Acceptance Tests: improve check-acceptance description Cleber Rosa
2021-09-27 14:40   ` Philippe Mathieu-Daudé
2021-09-24 18:54 ` [PATCH 03/16] Acceptance Tests: add mechanism for listing tests Cleber Rosa
2021-10-06 18:47   ` Willian Rampazzo
2021-09-24 18:54 ` [PATCH 04/16] Acceptance Tests: keep track and disable tests with known issues Cleber Rosa
2021-09-24 18:54 ` [PATCH 05/16] Acceptance Tests: add standard clean up at test tearDown() Cleber Rosa
2021-09-24 19:38   ` Willian Rampazzo
2021-09-24 18:54 ` [PATCH 06/16] Acceptance Tests: use extract from package from avocado.utils Cleber Rosa
2021-09-24 18:54 ` [PATCH 07/16] Acceptance Tests: workaround expired mipsdistros.mips.com HTTPS cert Cleber Rosa
2021-09-24 18:54 ` [PATCH 08/16] acceptance/tests/vnc.py: use explicit syntax for enabling passwords Cleber Rosa
2021-09-27 14:41   ` Philippe Mathieu-Daudé
2021-09-24 18:54 ` [PATCH 09/16] tests/acceptance/boot_xen.py: merge base classes Cleber Rosa
2021-10-06 18:57   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 10/16] tests/acceptance/boot_xen.py: unify tags Cleber Rosa
2021-10-06 18:56   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 11/16] tests/acceptance/boot_xen.py: fetch kernel during test setUp() Cleber Rosa
2021-10-06 18:55   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 12/16] tests/acceptance/boot_xen.py: removed unused import Cleber Rosa
2021-09-27 14:43   ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 13/16] tests/acceptance/boot_xen.py: use class attribute Cleber Rosa
2021-10-06 18:58   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 14/16] tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update Cleber Rosa
2021-09-27 15:26   ` Philippe Mathieu-Daudé
2021-09-30 19:19     ` Reinoud Zandijk
2021-09-30 19:44       ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 15/16] tests/acceptance/ppc_prep_40p.py: clean up unused import Cleber Rosa
2021-09-27 14:42   ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 16/16] tests/acceptance/ppc_prep_40p.py: unify tags Cleber Rosa
2021-09-27 14:45   ` Philippe Mathieu-Daudé
2021-10-06 19:17   ` Willian Rampazzo
2021-10-06 19:19     ` Willian Rampazzo
2021-09-27  9:09 ` [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements Pavel Dovgalyuk
2021-09-27 16:46 ` Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210924185506.2542588-2-crosa@redhat.com \
    --to=crosa@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=bleal@redhat.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=ehabkost@redhat.com \
    --cc=erdnaxe@crans.org \
    --cc=eric.auger@redhat.com \
    --cc=groug@kaod.org \
    --cc=hpoussin@reactos.org \
    --cc=kamil@netbsd.org \
    --cc=ma.mandourr@gmail.com \
    --cc=pavel.dovgaluk@ispras.ru \
    --cc=pbonzini@redhat.com \
    --cc=philmd@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=reinoud@netbsd.org \
    --cc=ryoon@netbsd.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=wrampazz@redhat.com \
    --cc=ysato@users.sourceforge.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.