All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Huth <thuth@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Fam Zheng" <fam@euphon.net>,
	qemu-block@nongnu.org, "Ed Maste" <emaste@freebsd.org>,
	"Kevin Wolf" <kwolf@redhat.com>, "Max Reitz" <mreitz@redhat.com>,
	"Li-Wen Hsu" <lwhsu@freebsd.org>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>
Subject: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Date: Wed, 24 Apr 2019 12:37:46 +0200	[thread overview]
Message-ID: <20190424103747.10173-6-thuth@redhat.com> (raw)
In-Reply-To: <20190424103747.10173-1-thuth@redhat.com>

People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet. Now that we've got
a new "ci" group of iotests that should be fine to run in every environ-
ment, it should be OK to enable the iotests during "make check" again.
Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
script, and only use the former "quick" group (that contains some tests
that are failing in some environments) when the user decided to run
"make check SPEED=thorough" or something similar.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include      |  2 +-
 tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 36fc73fef5..eb6a7a41e2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-block: $(patsubst %,check-%, $(check-block-y))
-check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
+check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
 check-clean:
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
index 0e554bb972..416b3fc48b 100755
--- a/tests/qemu-iotests-quick.sh
+++ b/tests/qemu-iotests-quick.sh
@@ -1,8 +1,23 @@
 #!/bin/sh
 
+# Honor the SPEED environment variable, just like we do it for the qtests.
+# The default is to run all tests that still work fine in a CI environments,
+# but if the user set SPEED=slow or SPEED=thorough, we also run all other
+# tests that are still marked as "quick"
+if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
+    group=quick
+else
+    group=ci
+fi
+
+if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+    echo "No qemu-system binary available. Skipped qemu-iotests."
+    exit 0
+fi
+
 cd tests/qemu-iotests
 
 ret=0
-TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
+TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g "$group" || ret=1
 
 exit $ret
-- 
2.21.0

WARNING: multiple messages have this Message-ID (diff)
From: Thomas Huth <thuth@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Fam Zheng" <fam@euphon.net>, "Kevin Wolf" <kwolf@redhat.com>,
	"Ed Maste" <emaste@freebsd.org>,
	qemu-block@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>,
	"Max Reitz" <mreitz@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Li-Wen Hsu" <lwhsu@freebsd.org>
Subject: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Date: Wed, 24 Apr 2019 12:37:46 +0200	[thread overview]
Message-ID: <20190424103747.10173-6-thuth@redhat.com> (raw)
Message-ID: <20190424103746.ewQgfBkUiK_vOzfZWdx6CFHjIiuulCrp5DdXldV9ATQ@z> (raw)
In-Reply-To: <20190424103747.10173-1-thuth@redhat.com>

People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet. Now that we've got
a new "ci" group of iotests that should be fine to run in every environ-
ment, it should be OK to enable the iotests during "make check" again.
Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
script, and only use the former "quick" group (that contains some tests
that are failing in some environments) when the user decided to run
"make check SPEED=thorough" or something similar.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include      |  2 +-
 tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 36fc73fef5..eb6a7a41e2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-block: $(patsubst %,check-%, $(check-block-y))
-check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
+check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
 check-clean:
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
index 0e554bb972..416b3fc48b 100755
--- a/tests/qemu-iotests-quick.sh
+++ b/tests/qemu-iotests-quick.sh
@@ -1,8 +1,23 @@
 #!/bin/sh
 
+# Honor the SPEED environment variable, just like we do it for the qtests.
+# The default is to run all tests that still work fine in a CI environments,
+# but if the user set SPEED=slow or SPEED=thorough, we also run all other
+# tests that are still marked as "quick"
+if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
+    group=quick
+else
+    group=ci
+fi
+
+if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+    echo "No qemu-system binary available. Skipped qemu-iotests."
+    exit 0
+fi
+
 cd tests/qemu-iotests
 
 ret=0
-TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
+TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g "$group" || ret=1
 
 exit $ret
-- 
2.21.0



  parent reply	other threads:[~2019-04-24 10:38 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-24 10:37 [Qemu-devel] [PATCH 0/6] tests/qemu-iotests: Run basic iotests during "make check" Thomas Huth
2019-04-24 10:37 ` Thomas Huth
2019-04-24 10:37 ` [Qemu-devel] [PATCH 1/6] tests/qemu-iotests/check: Pick a default machine if necessary Thomas Huth
2019-04-24 10:37   ` Thomas Huth
2019-04-30 21:31   ` Alex Bennée
2019-04-30 21:31     ` Alex Bennée
2019-04-24 10:37 ` [Qemu-devel] [PATCH 2/6] tests/qemu-iotests/group: Introduce a new "ci" group for CI pipelines Thomas Huth
2019-04-24 10:37   ` Thomas Huth
2019-04-24 11:25   ` Daniel P. Berrangé
2019-04-24 11:25     ` Daniel P. Berrangé
2019-04-24 12:37     ` Thomas Huth
2019-04-24 12:37       ` Thomas Huth
2019-04-24 12:50       ` Daniel P. Berrangé
2019-04-24 12:50         ` Daniel P. Berrangé
2019-04-24 13:01         ` Thomas Huth
2019-04-24 13:01           ` Thomas Huth
2019-04-24 13:12         ` Daniel P. Berrangé
2019-04-24 13:12           ` Daniel P. Berrangé
2019-04-25  7:44         ` Thomas Huth
2019-04-25  7:44           ` Thomas Huth
2019-04-24 10:37 ` [Qemu-devel] [PATCH 3/6] tests/qemu-iotests: Do not hard-code the path to bash Thomas Huth
2019-04-24 10:37   ` Thomas Huth
2019-04-24 14:14   ` Eric Blake
2019-04-24 14:14     ` Eric Blake
2019-04-24 14:42     ` Philippe Mathieu-Daudé
2019-04-24 14:42       ` Philippe Mathieu-Daudé
2019-04-24 15:23     ` Daniel P. Berrangé
2019-04-24 15:23       ` Daniel P. Berrangé
2019-04-25  7:48       ` Thomas Huth
2019-04-25  7:48         ` Thomas Huth
2019-04-30 21:33   ` Alex Bennée
2019-04-30 21:33     ` Alex Bennée
2019-04-24 10:37 ` [Qemu-devel] [PATCH 4/6] cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD Thomas Huth
2019-04-24 10:37   ` Thomas Huth
2019-04-24 21:29   ` Wainer dos Santos Moschetta
2019-04-24 21:29     ` Wainer dos Santos Moschetta
2019-04-25  6:00     ` Thomas Huth
2019-04-25  6:00       ` Thomas Huth
2019-04-26 16:43       ` Laszlo Ersek
2019-04-26 16:43         ` Laszlo Ersek
2019-05-06 10:12         ` [Qemu-devel] Update *BSD images with gnu-sed and bash (was: [PATCH 4/6] cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD) Thomas Huth
2019-05-07  8:32           ` Gerd Hoffmann
2019-05-07 10:13             ` Thomas Huth
2019-05-08  7:06           ` [Qemu-devel] Update *BSD images with gnu-sed and bash Kamil Rytarowski
2019-05-08  8:07             ` Thomas Huth
2019-05-08 19:21               ` Kamil Rytarowski
2019-04-24 10:37 ` Thomas Huth [this message]
2019-04-24 10:37   ` [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again Thomas Huth
2019-04-24 10:48   ` Daniel P. Berrangé
2019-04-24 10:48     ` Daniel P. Berrangé
2019-04-24 10:59     ` Thomas Huth
2019-04-24 10:59       ` Thomas Huth
2019-04-24 12:02       ` Vladimir Sementsov-Ogievskiy
2019-04-24 12:02         ` Vladimir Sementsov-Ogievskiy
2019-04-24 12:13         ` Daniel P. Berrangé
2019-04-24 12:13           ` Daniel P. Berrangé
2019-04-24 12:18           ` Vladimir Sementsov-Ogievskiy
2019-04-24 12:18             ` Vladimir Sementsov-Ogievskiy
2019-04-24 20:42   ` Wainer dos Santos Moschetta
2019-04-24 20:42     ` Wainer dos Santos Moschetta
2019-04-25  5:57     ` Thomas Huth
2019-04-25  5:57       ` Thomas Huth
2019-04-25 10:24       ` Thomas Huth
2019-04-25 10:24         ` Thomas Huth
2019-04-28  2:42         ` Eric Blake
2019-04-28  2:42           ` Eric Blake
2019-04-24 10:37 ` [Qemu-devel] [PATCH 6/6] tests/qemu-iotests: Remove the "_supported_os Linux" line from many tests Thomas Huth
2019-04-24 10:37   ` Thomas Huth
2019-04-27  4:51 ` [Qemu-devel] [PATCH 0/6] tests/qemu-iotests: Run basic iotests during "make check" no-reply
2019-04-27  4:51   ` no-reply
2019-04-29 12:04 ` Christian Borntraeger
2019-04-29 12:04   ` Christian Borntraeger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190424103747.10173-6-thuth@redhat.com \
    --to=thuth@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=emaste@freebsd.org \
    --cc=fam@euphon.net \
    --cc=kwolf@redhat.com \
    --cc=lwhsu@freebsd.org \
    --cc=mreitz@redhat.com \
    --cc=philmd@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.