From: "Philippe Mathieu-Daudé" <philmd@redhat.com>
To: "Alex Bennée" <alex.bennee@linaro.org>,
qemu-devel@nongnu.org, "Thomas Huth" <thuth@redhat.com>,
"Brad Smith" <brad@comstyle.com>
Subject: Re: [Qemu-devel] [PATCH v2 22/23] tests: Run the iotests during "make check" again
Date: Mon, 22 Jul 2019 21:53:41 +0200 [thread overview]
Message-ID: <1e1ae24a-bed3-2acc-7727-16cfb7d877bc@redhat.com> (raw)
In-Reply-To: <20190717134335.15351-23-alex.bennee@linaro.org>
On 7/17/19 3:43 PM, Alex Bennée wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> People often forget to run the iotests before submitting patches or pull
> requests - this is likely due to the fact that we do not run the tests
> during our mandatory "make check" tests yet. Now that we've got a proper
> "auto" group of iotests that should be fine to run in every environment,
> we can enable the iotests during "make check" again by running the "auto"
> tests by default from the check-block.sh script.
>
> Some cases still need to be checked first, though: iotests need bash and
> GNU sed (otherwise they fail), and if gprof is enabled, it spoils the
> output of some test cases causing them to fail. So if we detect that one
> of the required programs is missing or that gprof is enabled, we still
> have to skip the iotests to avoid failures.
>
> And finally, since we are using check-block.sh now again, this patch also
> removes the qemu-iotests-quick.sh script since we do not need that anymore
> (and having two shell wrapper scripts around the block tests seems rather
> confusing than helpful).
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> [AJB: -makecheck to check-block.sh, move check-block to start and gate it]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20190717111947.30356-4-thuth@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/Makefile.include | 10 +++++----
> tests/check-block.sh | 44 ++++++++++++++++++++++++++++---------
> tests/qemu-iotests-quick.sh | 8 -------
> 3 files changed, 40 insertions(+), 22 deletions(-)
> delete mode 100755 tests/qemu-iotests-quick.sh
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index fd7fdb86586..6f02dfcc019 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -142,7 +142,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
> check-unit-y += tests/ptimer-test$(EXESUF)
> check-unit-y += tests/test-qapi-util$(EXESUF)
>
> -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
> +check-block-$(call land,$(CONFIG_POSIX),$(CONFIG_SOFTMMU)) += tests/check-block.sh
>
> # All QTests for now are POSIX-only, but the dependencies are
> # really in libqtest, not in the testcases themselves.
> @@ -1092,8 +1092,10 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>
> QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
>
> -.PHONY: check-tests/qemu-iotests-quick.sh
> -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
> +.PHONY: check-tests/check-block.sh
> +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) \
> + qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \
> + $(patsubst %,%/all,$(filter %-softmmu,$(TARGET_DIRS)))
> $<
>
> .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
> @@ -1167,7 +1169,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
> check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
> check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
> check-block: $(patsubst %,check-%, $(check-block-y))
> -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
> +check: check-block check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
> check-clean:
> rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
> rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> diff --git a/tests/check-block.sh b/tests/check-block.sh
> index f3d12fd602d..c8b6cec3f68 100755
> --- a/tests/check-block.sh
> +++ b/tests/check-block.sh
> @@ -1,24 +1,48 @@
> #!/bin/sh
>
> -FORMAT_LIST="raw qcow2 qed vmdk vpc"
> +# Honor the SPEED environment variable, just like we do it for the qtests.
> +if [ "$SPEED" = "slow" ]; then
> + format_list="raw qcow2"
> + group=
> +elif [ "$SPEED" = "thorough" ]; then
> + format_list="raw qcow2 qed vmdk vpc"
> + group=
> +else
> + format_list=qcow2
> + group="-g auto"
> +fi
> +
> if [ "$#" -ne 0 ]; then
> - FORMAT_LIST="$@"
> + format_list="$@"
> +fi
> +
> +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
> + echo "GPROF is enabled ==> Not running the qemu-iotests."
> + exit 0
> fi
>
> -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
> -export QEMU_IMG_PROG="$PWD/qemu-img"
> -export QEMU_IO_PROG="$PWD/qemu-io"
> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> + echo "No qemu-system binary available ==> Not running the qemu-iotests."
> + exit 0
> +fi
> +
> +if ! command -v bash >/dev/null 2>&1 ; then
> + echo "bash not available ==> Not running the qemu-iotests."
> + exit 0
> +fi
>
> -if [ ! -x $QEMU_PROG ]; then
> - echo "'make check-block' requires qemu-system-x86_64"
> - exit 1
> +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
> + if ! command -v gsed >/dev/null 2>&1; then
> + echo "GNU sed not available ==> Not running the qemu-iotests."
> + exit 0
> + fi
> fi
>
> cd tests/qemu-iotests
>
> ret=0
> -for FMT in $FORMAT_LIST ; do
> - ./check -T -nocache -$FMT || ret=1
> +for fmt in $format_list ; do
> + ./check -makecheck -$fmt $group || ret=1
> done
>
> exit $ret
> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
> deleted file mode 100755
> index 0e554bb9724..00000000000
> --- a/tests/qemu-iotests-quick.sh
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -#!/bin/sh
> -
> -cd tests/qemu-iotests
> -
> -ret=0
> -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
> -
> -exit $ret
>
Since Gerd updated the OpenBSD image, do you know if we can run vm-test
again?
I haven't check which what got merged from this previous series:
https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg07527.html
next prev parent reply other threads:[~2019-07-22 19:53 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-17 13:43 [Qemu-devel] [PATCH v2 00/23] testing/next for 4.1-rc2 (win, travis, iotests) Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 01/23] archive-source: also create a stash for submodules Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 02/23] tests/docker: add test-misc for building tools & docs Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 03/23] tests/docker: Install Sphinx in the Ubuntu images Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 04/23] tests/docker: Install Sphinx in the Fedora image Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 05/23] tests/docker: Install Ubuntu images noninteractively Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 06/23] tests/migration-test: don't spam the logs when we fail Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 07/23] tests/dockerfiles: update the win cross builds to stretch Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 08/23] shippable: re-enable the windows cross builds Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 09/23] tests/docker: Install Sphinx in the Debian images Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 10/23] tests/docker: Install the NSIS tools in the MinGW capable images Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 11/23] tests/docker: Set the correct cross-PKG_CONFIG_PATH in the MXE images Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 12/23] tests/docker: Install texinfo in the Fedora image Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 13/23] buildsys: The NSIS Windows build requires the documentation installed Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 14/23] buildsys: The NSIS Windows build requires qemu-nsis.bmp installed Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 15/23] tests/docker: Let the test-mingw test generate a NSIS installer Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 16/23] NSIS: Add missing firmware blobs Alex Bennée
2019-07-17 14:47 ` Philippe Mathieu-Daudé
2019-07-17 18:23 ` Stefan Weil
2019-07-22 20:06 ` Philippe Mathieu-Daudé
2019-07-23 5:33 ` Stefan Weil
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 17/23] hw/i386: also turn off VMMOUSE is VMPORT is disabled Alex Bennée
2019-07-17 16:02 ` Thomas Huth
2019-07-22 20:02 ` Philippe Mathieu-Daudé
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 18/23] travis: enable travis_retry for check phase Alex Bennée
2019-07-22 20:01 ` Philippe Mathieu-Daudé
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 19/23] tests/docker: invoke the DEBUG shell with --noprofile/--norc Alex Bennée
2019-07-23 7:03 ` Philippe Mathieu-Daudé
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 20/23] tests/qemu-iotests/check: Allow tests without groups Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 21/23] tests/qemu-iotests/group: Remove some more tests from the "auto" group Alex Bennée
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 22/23] tests: Run the iotests during "make check" again Alex Bennée
2019-07-22 19:53 ` Philippe Mathieu-Daudé [this message]
2019-07-23 7:20 ` [Qemu-devel] make vm-build-openbsd (was: Re: [PATCH v2 22/23] tests: Run the iotests during "make check" again) Thomas Huth
2019-07-23 8:58 ` Thomas Huth
2019-08-02 14:28 ` Gerd Hoffmann
2019-08-02 14:44 ` Thomas Huth
2019-07-23 9:30 ` [Qemu-devel] [PATCH v2 22/23] tests: Run the iotests during "make check" again Thomas Huth
2019-07-17 13:43 ` [Qemu-devel] [PATCH v2 23/23] gitlab-ci: Remove qcow2 tests that are handled by "make check" already Alex Bennée
2019-07-22 18:29 ` [Qemu-devel] [PATCH v2 00/23] testing/next for 4.1-rc2 (win, travis, iotests) Alex Bennée
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1e1ae24a-bed3-2acc-7727-16cfb7d877bc@redhat.com \
--to=philmd@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=brad@comstyle.com \
--cc=qemu-devel@nongnu.org \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).