* [Qemu-devel] [PATCH v7 1/3] Bootstrap Python venv for tests
2018-10-18 15:31 [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests Cleber Rosa
@ 2018-10-18 15:31 ` Cleber Rosa
2018-10-23 7:56 ` Philippe Mathieu-Daudé
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 2/3] Acceptance tests: add make rule for running them Cleber Rosa
` (2 subsequent siblings)
3 siblings, 1 reply; 8+ messages in thread
From: Cleber Rosa @ 2018-10-18 15:31 UTC (permalink / raw)
To: qemu-devel
Cc: Caio Carrara, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Laszlo Ersek, Fam Zheng,
Eduardo Habkost, Stefan Hajnoczi, Philippe Mathieu-Daudé,
Cleber Rosa
A number of QEMU tests are written in Python, and may benefit
from an untainted Python venv.
By using make rules, tests that depend on specific Python libs
can set that rule as a requirement, along with rules that require
the presence or installation of specific libraries.
The tests/requirements.txt is supposed to contain the Python
requirements that should be added to the venv created by check-venv.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
---
tests/Makefile.include | 26 ++++++++++++++++++++++++++
tests/requirements.txt | 3 +++
2 files changed, 29 insertions(+)
create mode 100644 tests/requirements.txt
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 7fe8578972..004a86f274 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -12,6 +12,7 @@ check-help:
@echo " $(MAKE) check-block Run block tests"
@echo " $(MAKE) check-tcg Run TCG tests"
@echo " $(MAKE) check-report.html Generates an HTML test report"
+ @echo " $(MAKE) check-venv Creates a Python venv for tests"
@echo " $(MAKE) check-clean Clean the tests"
@echo
@echo "Please note that HTML reports do not regenerate if the unit tests"
@@ -899,6 +900,30 @@ check-decodetree:
./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
TEST, decodetree.py)
+# Python venv for running tests
+
+.PHONY: check-venv
+
+TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
+TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
+
+$(shell $(PYTHON) -c 'import sys; assert sys.version_info >= (3,0)' >/dev/null 2>&1)
+ifeq ($(.SHELLSTATUS),0)
+$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
+ $(call quiet-command, \
+ $(PYTHON) -m venv --system-site-packages $@, \
+ VENV, $@)
+ $(call quiet-command, \
+ $(TESTS_VENV_DIR)/bin/python -m pip -q install -r $(TESTS_VENV_REQ), \
+ PIP, $(TESTS_VENV_REQ))
+ $(call quiet-command, touch $@)
+else
+$(TESTS_VENV_DIR):
+ $(error "venv directory for tests requires Python 3")
+endif
+
+check-venv: $(TESTS_VENV_DIR)
+
# Consolidated targets
.PHONY: check-qapi-schema check-qtest check-unit check check-clean
@@ -912,6 +937,7 @@ check-clean:
rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
rm -f tests/test-qapi-gen-timestamp
+ rm -rf $(TESTS_VENV_DIR)
clean: check-clean
diff --git a/tests/requirements.txt b/tests/requirements.txt
new file mode 100644
index 0000000000..d39f9d1576
--- /dev/null
+++ b/tests/requirements.txt
@@ -0,0 +1,3 @@
+# Add Python module requirements, one per line, to be installed
+# in the tests/venv Python virtual environment. For more info,
+# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v7 1/3] Bootstrap Python venv for tests
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 1/3] Bootstrap Python venv for tests Cleber Rosa
@ 2018-10-23 7:56 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 8+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-10-23 7:56 UTC (permalink / raw)
To: Cleber Rosa, qemu-devel
Cc: Caio Carrara, Alex Bennée, Wainer dos Santos Moschetta,
Laszlo Ersek, Fam Zheng, Eduardo Habkost, Stefan Hajnoczi,
Philippe Mathieu-Daudé
On 18/10/18 17:31, Cleber Rosa wrote:
> A number of QEMU tests are written in Python, and may benefit
> from an untainted Python venv.
>
> By using make rules, tests that depend on specific Python libs
> can set that rule as a requirement, along with rules that require
> the presence or installation of specific libraries.
>
> The tests/requirements.txt is supposed to contain the Python
> requirements that should be added to the venv created by check-venv.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
> Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> tests/Makefile.include | 26 ++++++++++++++++++++++++++
> tests/requirements.txt | 3 +++
> 2 files changed, 29 insertions(+)
> create mode 100644 tests/requirements.txt
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 7fe8578972..004a86f274 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -12,6 +12,7 @@ check-help:
> @echo " $(MAKE) check-block Run block tests"
> @echo " $(MAKE) check-tcg Run TCG tests"
> @echo " $(MAKE) check-report.html Generates an HTML test report"
> + @echo " $(MAKE) check-venv Creates a Python venv for tests"
> @echo " $(MAKE) check-clean Clean the tests"
> @echo
> @echo "Please note that HTML reports do not regenerate if the unit tests"
> @@ -899,6 +900,30 @@ check-decodetree:
> ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
> TEST, decodetree.py)
>
> +# Python venv for running tests
> +
> +.PHONY: check-venv
> +
> +TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
> +TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
> +
> +$(shell $(PYTHON) -c 'import sys; assert sys.version_info >= (3,0)' >/dev/null 2>&1)
> +ifeq ($(.SHELLSTATUS),0)
> +$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> + $(call quiet-command, \
> + $(PYTHON) -m venv --system-site-packages $@, \
> + VENV, $@)
> + $(call quiet-command, \
> + $(TESTS_VENV_DIR)/bin/python -m pip -q install -r $(TESTS_VENV_REQ), \
> + PIP, $(TESTS_VENV_REQ))
> + $(call quiet-command, touch $@)
> +else
> +$(TESTS_VENV_DIR):
> + $(error "venv directory for tests requires Python 3")
> +endif
> +
> +check-venv: $(TESTS_VENV_DIR)
> +
> # Consolidated targets
>
> .PHONY: check-qapi-schema check-qtest check-unit check check-clean
> @@ -912,6 +937,7 @@ check-clean:
> rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
> rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> rm -f tests/test-qapi-gen-timestamp
> + rm -rf $(TESTS_VENV_DIR)
>
> clean: check-clean
>
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> new file mode 100644
> index 0000000000..d39f9d1576
> --- /dev/null
> +++ b/tests/requirements.txt
> @@ -0,0 +1,3 @@
> +# Add Python module requirements, one per line, to be installed
> +# in the tests/venv Python virtual environment. For more info,
> +# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v7 2/3] Acceptance tests: add make rule for running them
2018-10-18 15:31 [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests Cleber Rosa
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 1/3] Bootstrap Python venv for tests Cleber Rosa
@ 2018-10-18 15:31 ` Cleber Rosa
2018-10-23 0:06 ` Philippe Mathieu-Daudé
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 3/3] Travis support for the acceptance tests Cleber Rosa
2018-10-26 15:16 ` [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests Eduardo Habkost
3 siblings, 1 reply; 8+ messages in thread
From: Cleber Rosa @ 2018-10-18 15:31 UTC (permalink / raw)
To: qemu-devel
Cc: Caio Carrara, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Laszlo Ersek, Fam Zheng,
Eduardo Habkost, Stefan Hajnoczi, Philippe Mathieu-Daudé,
Cleber Rosa
The acceptance (aka functional, aka Avocado-based) tests are
Python files located in "tests/acceptance" that need to be run
with the Avocado libs and test runner.
Let's provide a convenient way for QEMU developers to run them,
by making use of the tests-venv with the required setup.
Also, while the Avocado test runner will take care of creating a
location to save test results to, it was understood that it's better
if the results are kept within the build tree.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
---
docs/devel/testing.rst | 43 +++++++++++++++++++++++++++++++++++++-----
tests/Makefile.include | 21 +++++++++++++++++++--
tests/requirements.txt | 1 +
3 files changed, 58 insertions(+), 7 deletions(-)
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index fcfad87614..da20be4d27 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -543,10 +543,39 @@ Tests based on ``avocado_qemu.Test`` can easily:
- http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test
- http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
-Installation
-------------
+Running tests
+-------------
-To install Avocado and its dependencies, run:
+You can run the acceptance tests simply by executing:
+
+.. code::
+
+ make check-acceptance
+
+This involves the automatic creation of Python virtual environment
+within the build tree (at ``tests/venv``) which will have all the
+right dependencies, and will save tests results also within the
+build tree (at ``tests/results``).
+
+Note: the build environment must be using a Python 3 stack, and have
+the ``venv`` and ``pip`` packages installed. If necessary, make sure
+``configure`` is called with ``--python=`` and that those modules are
+available. On Debian and Ubuntu based systems, depending on the
+specific version, they may be on packages named ``python3-venv`` and
+``python3-pip``.
+
+The scripts installed inside the virtual environment may be used
+without an "activation". For instance, the Avocado test runner
+may be invoked by running:
+
+ .. code::
+
+ tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/
+
+Manual Installation
+-------------------
+
+To manually install Avocado and its dependencies, run:
.. code::
@@ -687,11 +716,15 @@ The exact QEMU binary to be used on QEMUMachine.
Uninstalling Avocado
--------------------
-If you've followed the installation instructions above, you can easily
-uninstall Avocado. Start by listing the packages you have installed::
+If you've followed the manual installation instructions above, you can
+easily uninstall Avocado. Start by listing the packages you have
+installed::
pip list --user
And remove any package you want with::
pip uninstall <package_name>
+
+If you've used ``make check-acceptance``, the Python virtual environment where
+Avocado is installed will be cleaned up as part of ``make check-clean``.
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 004a86f274..3e4dac2817 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -11,6 +11,7 @@ check-help:
@echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
@echo " $(MAKE) check-block Run block tests"
@echo " $(MAKE) check-tcg Run TCG tests"
+ @echo " $(MAKE) check-acceptance Run all acceptance (functional) tests"
@echo " $(MAKE) check-report.html Generates an HTML test report"
@echo " $(MAKE) check-venv Creates a Python venv for tests"
@echo " $(MAKE) check-clean Clean the tests"
@@ -902,10 +903,15 @@ check-decodetree:
# Python venv for running tests
-.PHONY: check-venv
+.PHONY: check-venv check-acceptance
TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
+TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
+# Controls the output generated by Avocado when running tests.
+# Any number of command separated loggers are accepted. For more
+# information please refer to "avocado --help".
+AVOCADO_SHOW=none
$(shell $(PYTHON) -c 'import sys; assert sys.version_info >= (3,0)' >/dev/null 2>&1)
ifeq ($(.SHELLSTATUS),0)
@@ -922,8 +928,19 @@ $(TESTS_VENV_DIR):
$(error "venv directory for tests requires Python 3")
endif
+$(TESTS_RESULTS_DIR):
+ $(call quiet-command, mkdir -p $@, \
+ MKDIR, $@)
+
check-venv: $(TESTS_VENV_DIR)
+check-acceptance: check-venv $(TESTS_RESULTS_DIR)
+ $(call quiet-command, \
+ $(TESTS_VENV_DIR)/bin/python -m avocado \
+ --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
+ --failfast=on $(SRC_PATH)/tests/acceptance, \
+ "AVOCADO", "tests/acceptance")
+
# Consolidated targets
.PHONY: check-qapi-schema check-qtest check-unit check check-clean
@@ -937,7 +954,7 @@ check-clean:
rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
rm -f tests/test-qapi-gen-timestamp
- rm -rf $(TESTS_VENV_DIR)
+ rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR)
clean: check-clean
diff --git a/tests/requirements.txt b/tests/requirements.txt
index d39f9d1576..64c6e27a94 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,3 +1,4 @@
# Add Python module requirements, one per line, to be installed
# in the tests/venv Python virtual environment. For more info,
# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
+avocado-framework==65.0
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v7 2/3] Acceptance tests: add make rule for running them
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 2/3] Acceptance tests: add make rule for running them Cleber Rosa
@ 2018-10-23 0:06 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 8+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-10-23 0:06 UTC (permalink / raw)
To: Cleber Rosa, qemu-devel
Cc: Caio Carrara, Alex Bennée, Wainer dos Santos Moschetta,
Laszlo Ersek, Fam Zheng, Eduardo Habkost, Stefan Hajnoczi,
Philippe Mathieu-Daudé
On 18/10/18 17:31, Cleber Rosa wrote:
> The acceptance (aka functional, aka Avocado-based) tests are
> Python files located in "tests/acceptance" that need to be run
> with the Avocado libs and test runner.
>
> Let's provide a convenient way for QEMU developers to run them,
> by making use of the tests-venv with the required setup.
>
> Also, while the Avocado test runner will take care of creating a
> location to save test results to, it was understood that it's better
> if the results are kept within the build tree.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
> Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> docs/devel/testing.rst | 43 +++++++++++++++++++++++++++++++++++++-----
> tests/Makefile.include | 21 +++++++++++++++++++--
> tests/requirements.txt | 1 +
> 3 files changed, 58 insertions(+), 7 deletions(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index fcfad87614..da20be4d27 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -543,10 +543,39 @@ Tests based on ``avocado_qemu.Test`` can easily:
> - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test
> - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
>
> -Installation
> -------------
> +Running tests
> +-------------
>
> -To install Avocado and its dependencies, run:
> +You can run the acceptance tests simply by executing:
> +
> +.. code::
> +
> + make check-acceptance
> +
> +This involves the automatic creation of Python virtual environment
> +within the build tree (at ``tests/venv``) which will have all the
> +right dependencies, and will save tests results also within the
> +build tree (at ``tests/results``).
> +
> +Note: the build environment must be using a Python 3 stack, and have
> +the ``venv`` and ``pip`` packages installed. If necessary, make sure
> +``configure`` is called with ``--python=`` and that those modules are
> +available. On Debian and Ubuntu based systems, depending on the
> +specific version, they may be on packages named ``python3-venv`` and
> +``python3-pip``.
> +
> +The scripts installed inside the virtual environment may be used
> +without an "activation". For instance, the Avocado test runner
> +may be invoked by running:
> +
> + .. code::
> +
> + tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/
> +
> +Manual Installation
> +-------------------
> +
> +To manually install Avocado and its dependencies, run:
>
> .. code::
>
> @@ -687,11 +716,15 @@ The exact QEMU binary to be used on QEMUMachine.
> Uninstalling Avocado
> --------------------
>
> -If you've followed the installation instructions above, you can easily
> -uninstall Avocado. Start by listing the packages you have installed::
> +If you've followed the manual installation instructions above, you can
> +easily uninstall Avocado. Start by listing the packages you have
> +installed::
>
> pip list --user
>
> And remove any package you want with::
>
> pip uninstall <package_name>
> +
> +If you've used ``make check-acceptance``, the Python virtual environment where
> +Avocado is installed will be cleaned up as part of ``make check-clean``.
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 004a86f274..3e4dac2817 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -11,6 +11,7 @@ check-help:
> @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
> @echo " $(MAKE) check-block Run block tests"
> @echo " $(MAKE) check-tcg Run TCG tests"
> + @echo " $(MAKE) check-acceptance Run all acceptance (functional) tests"
> @echo " $(MAKE) check-report.html Generates an HTML test report"
> @echo " $(MAKE) check-venv Creates a Python venv for tests"
> @echo " $(MAKE) check-clean Clean the tests"
> @@ -902,10 +903,15 @@ check-decodetree:
>
> # Python venv for running tests
>
> -.PHONY: check-venv
> +.PHONY: check-venv check-acceptance
>
> TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
> TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
> +TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
> +# Controls the output generated by Avocado when running tests.
> +# Any number of command separated loggers are accepted. For more
> +# information please refer to "avocado --help".
> +AVOCADO_SHOW=none
>
> $(shell $(PYTHON) -c 'import sys; assert sys.version_info >= (3,0)' >/dev/null 2>&1)
> ifeq ($(.SHELLSTATUS),0)
> @@ -922,8 +928,19 @@ $(TESTS_VENV_DIR):
> $(error "venv directory for tests requires Python 3")
> endif
>
> +$(TESTS_RESULTS_DIR):
> + $(call quiet-command, mkdir -p $@, \
> + MKDIR, $@)
> +
> check-venv: $(TESTS_VENV_DIR)
>
> +check-acceptance: check-venv $(TESTS_RESULTS_DIR)
> + $(call quiet-command, \
> + $(TESTS_VENV_DIR)/bin/python -m avocado \
> + --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> + --failfast=on $(SRC_PATH)/tests/acceptance, \
> + "AVOCADO", "tests/acceptance")
> +
> # Consolidated targets
>
> .PHONY: check-qapi-schema check-qtest check-unit check check-clean
> @@ -937,7 +954,7 @@ check-clean:
> rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
> rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> rm -f tests/test-qapi-gen-timestamp
> - rm -rf $(TESTS_VENV_DIR)
> + rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR)
>
> clean: check-clean
>
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index d39f9d1576..64c6e27a94 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -1,3 +1,4 @@
> # Add Python module requirements, one per line, to be installed
> # in the tests/venv Python virtual environment. For more info,
> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> +avocado-framework==65.0
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v7 3/3] Travis support for the acceptance tests
2018-10-18 15:31 [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests Cleber Rosa
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 1/3] Bootstrap Python venv for tests Cleber Rosa
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 2/3] Acceptance tests: add make rule for running them Cleber Rosa
@ 2018-10-18 15:31 ` Cleber Rosa
2018-10-18 15:50 ` Philippe Mathieu-Daudé
2018-10-26 15:16 ` [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests Eduardo Habkost
3 siblings, 1 reply; 8+ messages in thread
From: Cleber Rosa @ 2018-10-18 15:31 UTC (permalink / raw)
To: qemu-devel
Cc: Caio Carrara, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Laszlo Ersek, Fam Zheng,
Eduardo Habkost, Stefan Hajnoczi, Philippe Mathieu-Daudé,
Cleber Rosa
This enables the execution of the acceptance tests on Travis.
Because the Travis environment is based on Ubuntu Trusty, it requires
the python3-pip and python3.4-venv packages.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
---
.travis.yml | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 95be6ec59f..aa49c7b114 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -117,6 +117,14 @@ matrix:
- env: CONFIG="--target-list=x86_64-softmmu"
python:
- "3.6"
+ # Acceptance (Functional) tests
+ - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+ TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
+ addons:
+ apt:
+ packages:
+ - python3-pip
+ - python3.4-venv
# Using newer GCC with sanitizers
- addons:
apt:
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v7 3/3] Travis support for the acceptance tests
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 3/3] Travis support for the acceptance tests Cleber Rosa
@ 2018-10-18 15:50 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 8+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-10-18 15:50 UTC (permalink / raw)
To: Cleber Rosa, qemu-devel
Cc: Caio Carrara, Alex Bennée, Wainer dos Santos Moschetta,
Laszlo Ersek, Fam Zheng, Eduardo Habkost, Stefan Hajnoczi,
Philippe Mathieu-Daudé
On 18/10/2018 17:31, Cleber Rosa wrote:
> This enables the execution of the acceptance tests on Travis.
>
> Because the Travis environment is based on Ubuntu Trusty, it requires
> the python3-pip and python3.4-venv packages.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
> Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .travis.yml | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 95be6ec59f..aa49c7b114 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -117,6 +117,14 @@ matrix:
> - env: CONFIG="--target-list=x86_64-softmmu"
> python:
> - "3.6"
> + # Acceptance (Functional) tests
> + - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
> + TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
> + addons:
> + apt:
> + packages:
> + - python3-pip
> + - python3.4-venv
> # Using newer GCC with sanitizers
> - addons:
> apt:
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests
2018-10-18 15:31 [Qemu-devel] [PATCH v7 0/3] Bootstrap Python venv and acceptance/functional tests Cleber Rosa
` (2 preceding siblings ...)
2018-10-18 15:31 ` [Qemu-devel] [PATCH v7 3/3] Travis support for the acceptance tests Cleber Rosa
@ 2018-10-26 15:16 ` Eduardo Habkost
3 siblings, 0 replies; 8+ messages in thread
From: Eduardo Habkost @ 2018-10-26 15:16 UTC (permalink / raw)
To: Cleber Rosa
Cc: qemu-devel, Fam Zheng, Alex Bennée,
Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Laszlo Ersek, Stefan Hajnoczi,
Caio Carrara, Philippe Mathieu-Daudé
On Thu, Oct 18, 2018 at 11:31:31AM -0400, Cleber Rosa wrote:
> TL;DR
> =====
>
> Allow acceptance tests to be run with `make check-acceptance`.
Queued on python-next, thanks!
--
Eduardo
^ permalink raw reply [flat|nested] 8+ messages in thread