From: Thomas Huth <thuth@redhat.com>
To: Willian Rampazzo <willianr@redhat.com>,
f4bug@amsat.org, philmd@redhat.com, crosa@redhat.com,
alex.bennee@linaro.org, qemu-devel@nongnu.org,
wainersm@redhat.com
Subject: Re: [PATCH v3] tests/acceptance: allow control over tags during check-acceptance
Date: Fri, 27 Aug 2021 09:46:34 +0200 [thread overview]
Message-ID: <0a55ff1b-122d-3cbb-0070-942cac8bd7e0@redhat.com> (raw)
In-Reply-To: <20210713210111.173349-1-willianr@redhat.com>
On 13/07/2021 23.01, Willian Rampazzo wrote:
> Although it is possible to run a specific test using the avocado
> command-line, a user may want to use a specific tag while running the
> ``make check-acceptance`` during the development or debugging.
>
> This allows using the AVOCADO_TAGS environment variable where the user
> takes total control of which tests should run based on the tags defined.
>
> This also makes the check-acceptance command flexible to restrict tests
> based on tags while running on CI.
>
> e.g.:
>
> AVOCADO_TAGS="foo bar baz" make check-acceptance
>
> Signed-off-by: Willian Rampazzo <willianr@redhat.com>
> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
> docs/devel/testing.rst | 14 ++++++++++++++
> tests/Makefile.include | 12 +++++++++---
> 2 files changed, 23 insertions(+), 3 deletions(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 4e42392810..a81b44cf18 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -760,6 +760,20 @@ in the current directory, tagged as "quick", run:
>
> avocado run -t quick .
>
> +It is also possible to run tests based on tags using the
> +``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment
> +variable:
> +
> +.. code::
> +
> + AVOCADO_TAGS=quick make check-acceptance
> +
> +Note that tags separated with commas have an AND behavior, while tags
> +separated by spaces have an OR behavior. For more information on Avocado
> +tags, see:
> +
> + https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
> +
> The ``avocado_qemu.Test`` base test class
> -----------------------------------------
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index e4dcb17329..0ce331f9a9 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
> # Any number of command separated loggers are accepted. For more
> # information please refer to "avocado --help".
> AVOCADO_SHOW=app
> -AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
> +ifndef AVOCADO_TAGS
> + AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
> + $(filter %-softmmu,$(TARGETS)))
> +else
> + AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
> +endif
>
> $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> $(call quiet-command, \
> @@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> $(call quiet-command, \
> $(TESTS_VENV_DIR)/bin/python -m avocado \
> --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> - --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
> - $(AVOCADO_TAGS) \
> + $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
> + --filter-by-tags-include-empty-key) \
> + $(AVOCADO_CMDLINE_TAGS) \
> $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
> "AVOCADO", "tests/acceptance")
>
>
Reviewed-by: Thomas Huth <thuth@redhat.com>
next prev parent reply other threads:[~2021-08-27 8:14 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-13 21:01 [PATCH v3] tests/acceptance: allow control over tags during check-acceptance Willian Rampazzo
2021-08-27 7:46 ` Thomas Huth [this message]
2021-09-22 18:33 ` Willian Rampazzo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0a55ff1b-122d-3cbb-0070-942cac8bd7e0@redhat.com \
--to=thuth@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=crosa@redhat.com \
--cc=f4bug@amsat.org \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=wainersm@redhat.com \
--cc=willianr@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).