All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v3 0/7] tests/qemu-iotests: Run basic iotests during "make check"
@ 2019-05-02  8:44 ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:44 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet.
This patch series now changes the "auto" group to only include tests
that should be safe to run in all CI environments (including FreeBSD
and macOS!). Thus these iotests can now always be run during "make
check" automatically, too.

v3:
 - Rebase to master, solved trivial conflict with new iotest 249
   (and the patch to fix the "qemu-io" output has already been merged)
 - Removed some more tests from the "auto" group that failed in
   certain environments
 - Added Reviewed-by tags

v2:
 - Use "auto" group instead of adding a new "ci" group
 - Adjusted the tests that are run automatically (after doing lots
   of CI runs on cirrus-ci.com, gitlab and travis)
 - Added patch to fix the current regression with the "qemu-io:"
   prefix in the master branch
 - Added a fix for iotest 005 on ext4 filesystems (skip it there)

Thomas Huth (7):
  tests/qemu-iotests/005: Add a sanity check for large sparse file
    support
  tests/qemu-iotests/check: Pick a default machine if necessary
  tests/qemu-iotests: Do not hard-code the path to bash
  cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD
  tests/qemu-iotests: Remove the "_supported_os Linux" line from many
    tests
  tests/qemu-iotests/group: Re-use the "auto" group for tests that can
    always run
  tests: Run the iotests during "make check" again

 .cirrus.yml                 |   4 +-
 .travis.yml                 |   1 +
 tests/Makefile.include      |   8 +-
 tests/check-block.sh        |  44 +++++++--
 tests/qemu-iotests-quick.sh |   8 --
 tests/qemu-iotests/001      |   1 -
 tests/qemu-iotests/002      |   1 -
 tests/qemu-iotests/003      |   1 -
 tests/qemu-iotests/004      |   1 -
 tests/qemu-iotests/005      |   9 ++
 tests/qemu-iotests/007      |   1 -
 tests/qemu-iotests/008      |   1 -
 tests/qemu-iotests/009      |   1 -
 tests/qemu-iotests/010      |   1 -
 tests/qemu-iotests/011      |   1 -
 tests/qemu-iotests/012      |   1 -
 tests/qemu-iotests/015      |   1 -
 tests/qemu-iotests/017      |   1 -
 tests/qemu-iotests/020      |   1 -
 tests/qemu-iotests/021      |   1 -
 tests/qemu-iotests/022      |   1 -
 tests/qemu-iotests/025      |   1 -
 tests/qemu-iotests/026      |   1 -
 tests/qemu-iotests/027      |   1 -
 tests/qemu-iotests/029      |   1 -
 tests/qemu-iotests/031      |   1 -
 tests/qemu-iotests/032      |   1 -
 tests/qemu-iotests/033      |   1 -
 tests/qemu-iotests/035      |   1 -
 tests/qemu-iotests/036      |   1 -
 tests/qemu-iotests/037      |   1 -
 tests/qemu-iotests/042      |   1 -
 tests/qemu-iotests/043      |   1 -
 tests/qemu-iotests/046      |   1 -
 tests/qemu-iotests/047      |   1 -
 tests/qemu-iotests/049      |   1 -
 tests/qemu-iotests/050      |   1 -
 tests/qemu-iotests/051      |   1 -
 tests/qemu-iotests/052      |   1 -
 tests/qemu-iotests/053      |   1 -
 tests/qemu-iotests/054      |   1 -
 tests/qemu-iotests/062      |   1 -
 tests/qemu-iotests/063      |   1 -
 tests/qemu-iotests/066      |   1 -
 tests/qemu-iotests/067      |   1 -
 tests/qemu-iotests/068      |   1 -
 tests/qemu-iotests/069      |   1 -
 tests/qemu-iotests/071      |   1 -
 tests/qemu-iotests/072      |   1 -
 tests/qemu-iotests/073      |   1 -
 tests/qemu-iotests/079      |   1 -
 tests/qemu-iotests/082      |   1 -
 tests/qemu-iotests/085      |   1 -
 tests/qemu-iotests/089      |   1 -
 tests/qemu-iotests/090      |   1 -
 tests/qemu-iotests/094      |   1 -
 tests/qemu-iotests/095      |   1 -
 tests/qemu-iotests/098      |   1 -
 tests/qemu-iotests/102      |   1 -
 tests/qemu-iotests/103      |   1 -
 tests/qemu-iotests/104      |   1 -
 tests/qemu-iotests/105      |   1 -
 tests/qemu-iotests/107      |   1 -
 tests/qemu-iotests/110      |   1 -
 tests/qemu-iotests/111      |   1 -
 tests/qemu-iotests/112      |   1 -
 tests/qemu-iotests/114      |   1 -
 tests/qemu-iotests/115      |   1 -
 tests/qemu-iotests/117      |   1 -
 tests/qemu-iotests/120      |   1 -
 tests/qemu-iotests/125      |   1 -
 tests/qemu-iotests/126      |   1 -
 tests/qemu-iotests/127      |   1 -
 tests/qemu-iotests/133      |   1 -
 tests/qemu-iotests/134      |   1 -
 tests/qemu-iotests/142      |   1 -
 tests/qemu-iotests/143      |   1 -
 tests/qemu-iotests/144      |   1 -
 tests/qemu-iotests/145      |   1 -
 tests/qemu-iotests/153      |   1 -
 tests/qemu-iotests/156      |   1 -
 tests/qemu-iotests/157      |   1 -
 tests/qemu-iotests/158      |   1 -
 tests/qemu-iotests/159      |   1 -
 tests/qemu-iotests/162      |   1 -
 tests/qemu-iotests/170      |   1 -
 tests/qemu-iotests/173      |   1 -
 tests/qemu-iotests/182      |   1 -
 tests/qemu-iotests/183      |   1 -
 tests/qemu-iotests/186      |   1 -
 tests/qemu-iotests/187      |   1 -
 tests/qemu-iotests/190      |   1 -
 tests/qemu-iotests/191      |   1 -
 tests/qemu-iotests/192      |   1 -
 tests/qemu-iotests/195      |   1 -
 tests/qemu-iotests/197      |   1 -
 tests/qemu-iotests/200      |   1 -
 tests/qemu-iotests/214      |   1 -
 tests/qemu-iotests/215      |   1 -
 tests/qemu-iotests/217      |   1 -
 tests/qemu-iotests/227      |   1 -
 tests/qemu-iotests/231      |   1 -
 tests/qemu-iotests/233      |   1 -
 tests/qemu-iotests/239      |   2 +-
 tests/qemu-iotests/240      |   3 +-
 tests/qemu-iotests/241      |   2 +-
 tests/qemu-iotests/243      |   2 +-
 tests/qemu-iotests/244      |   2 +-
 tests/qemu-iotests/247      |   1 -
 tests/qemu-iotests/check    |  13 ++-
 tests/qemu-iotests/group    | 177 +++++++++++++++++++-----------------
 111 files changed, 162 insertions(+), 211 deletions(-)
 delete mode 100755 tests/qemu-iotests-quick.sh

-- 
2.21.0

^ permalink raw reply	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 0/7] tests/qemu-iotests: Run basic iotests during "make check"
@ 2019-05-02  8:44 ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:44 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet.
This patch series now changes the "auto" group to only include tests
that should be safe to run in all CI environments (including FreeBSD
and macOS!). Thus these iotests can now always be run during "make
check" automatically, too.

v3:
 - Rebase to master, solved trivial conflict with new iotest 249
   (and the patch to fix the "qemu-io" output has already been merged)
 - Removed some more tests from the "auto" group that failed in
   certain environments
 - Added Reviewed-by tags

v2:
 - Use "auto" group instead of adding a new "ci" group
 - Adjusted the tests that are run automatically (after doing lots
   of CI runs on cirrus-ci.com, gitlab and travis)
 - Added patch to fix the current regression with the "qemu-io:"
   prefix in the master branch
 - Added a fix for iotest 005 on ext4 filesystems (skip it there)

Thomas Huth (7):
  tests/qemu-iotests/005: Add a sanity check for large sparse file
    support
  tests/qemu-iotests/check: Pick a default machine if necessary
  tests/qemu-iotests: Do not hard-code the path to bash
  cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD
  tests/qemu-iotests: Remove the "_supported_os Linux" line from many
    tests
  tests/qemu-iotests/group: Re-use the "auto" group for tests that can
    always run
  tests: Run the iotests during "make check" again

 .cirrus.yml                 |   4 +-
 .travis.yml                 |   1 +
 tests/Makefile.include      |   8 +-
 tests/check-block.sh        |  44 +++++++--
 tests/qemu-iotests-quick.sh |   8 --
 tests/qemu-iotests/001      |   1 -
 tests/qemu-iotests/002      |   1 -
 tests/qemu-iotests/003      |   1 -
 tests/qemu-iotests/004      |   1 -
 tests/qemu-iotests/005      |   9 ++
 tests/qemu-iotests/007      |   1 -
 tests/qemu-iotests/008      |   1 -
 tests/qemu-iotests/009      |   1 -
 tests/qemu-iotests/010      |   1 -
 tests/qemu-iotests/011      |   1 -
 tests/qemu-iotests/012      |   1 -
 tests/qemu-iotests/015      |   1 -
 tests/qemu-iotests/017      |   1 -
 tests/qemu-iotests/020      |   1 -
 tests/qemu-iotests/021      |   1 -
 tests/qemu-iotests/022      |   1 -
 tests/qemu-iotests/025      |   1 -
 tests/qemu-iotests/026      |   1 -
 tests/qemu-iotests/027      |   1 -
 tests/qemu-iotests/029      |   1 -
 tests/qemu-iotests/031      |   1 -
 tests/qemu-iotests/032      |   1 -
 tests/qemu-iotests/033      |   1 -
 tests/qemu-iotests/035      |   1 -
 tests/qemu-iotests/036      |   1 -
 tests/qemu-iotests/037      |   1 -
 tests/qemu-iotests/042      |   1 -
 tests/qemu-iotests/043      |   1 -
 tests/qemu-iotests/046      |   1 -
 tests/qemu-iotests/047      |   1 -
 tests/qemu-iotests/049      |   1 -
 tests/qemu-iotests/050      |   1 -
 tests/qemu-iotests/051      |   1 -
 tests/qemu-iotests/052      |   1 -
 tests/qemu-iotests/053      |   1 -
 tests/qemu-iotests/054      |   1 -
 tests/qemu-iotests/062      |   1 -
 tests/qemu-iotests/063      |   1 -
 tests/qemu-iotests/066      |   1 -
 tests/qemu-iotests/067      |   1 -
 tests/qemu-iotests/068      |   1 -
 tests/qemu-iotests/069      |   1 -
 tests/qemu-iotests/071      |   1 -
 tests/qemu-iotests/072      |   1 -
 tests/qemu-iotests/073      |   1 -
 tests/qemu-iotests/079      |   1 -
 tests/qemu-iotests/082      |   1 -
 tests/qemu-iotests/085      |   1 -
 tests/qemu-iotests/089      |   1 -
 tests/qemu-iotests/090      |   1 -
 tests/qemu-iotests/094      |   1 -
 tests/qemu-iotests/095      |   1 -
 tests/qemu-iotests/098      |   1 -
 tests/qemu-iotests/102      |   1 -
 tests/qemu-iotests/103      |   1 -
 tests/qemu-iotests/104      |   1 -
 tests/qemu-iotests/105      |   1 -
 tests/qemu-iotests/107      |   1 -
 tests/qemu-iotests/110      |   1 -
 tests/qemu-iotests/111      |   1 -
 tests/qemu-iotests/112      |   1 -
 tests/qemu-iotests/114      |   1 -
 tests/qemu-iotests/115      |   1 -
 tests/qemu-iotests/117      |   1 -
 tests/qemu-iotests/120      |   1 -
 tests/qemu-iotests/125      |   1 -
 tests/qemu-iotests/126      |   1 -
 tests/qemu-iotests/127      |   1 -
 tests/qemu-iotests/133      |   1 -
 tests/qemu-iotests/134      |   1 -
 tests/qemu-iotests/142      |   1 -
 tests/qemu-iotests/143      |   1 -
 tests/qemu-iotests/144      |   1 -
 tests/qemu-iotests/145      |   1 -
 tests/qemu-iotests/153      |   1 -
 tests/qemu-iotests/156      |   1 -
 tests/qemu-iotests/157      |   1 -
 tests/qemu-iotests/158      |   1 -
 tests/qemu-iotests/159      |   1 -
 tests/qemu-iotests/162      |   1 -
 tests/qemu-iotests/170      |   1 -
 tests/qemu-iotests/173      |   1 -
 tests/qemu-iotests/182      |   1 -
 tests/qemu-iotests/183      |   1 -
 tests/qemu-iotests/186      |   1 -
 tests/qemu-iotests/187      |   1 -
 tests/qemu-iotests/190      |   1 -
 tests/qemu-iotests/191      |   1 -
 tests/qemu-iotests/192      |   1 -
 tests/qemu-iotests/195      |   1 -
 tests/qemu-iotests/197      |   1 -
 tests/qemu-iotests/200      |   1 -
 tests/qemu-iotests/214      |   1 -
 tests/qemu-iotests/215      |   1 -
 tests/qemu-iotests/217      |   1 -
 tests/qemu-iotests/227      |   1 -
 tests/qemu-iotests/231      |   1 -
 tests/qemu-iotests/233      |   1 -
 tests/qemu-iotests/239      |   2 +-
 tests/qemu-iotests/240      |   3 +-
 tests/qemu-iotests/241      |   2 +-
 tests/qemu-iotests/243      |   2 +-
 tests/qemu-iotests/244      |   2 +-
 tests/qemu-iotests/247      |   1 -
 tests/qemu-iotests/check    |  13 ++-
 tests/qemu-iotests/group    | 177 +++++++++++++++++++-----------------
 111 files changed, 162 insertions(+), 211 deletions(-)
 delete mode 100755 tests/qemu-iotests-quick.sh

-- 
2.21.0



^ permalink raw reply	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 1/7] tests/qemu-iotests/005: Add a sanity check for large sparse file support
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

"check -raw 005" fails when running on certain filesystems - these do not
support such large sparse files. Use the same check as in test 220 to
skip the test in this case.

Suggested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/005 | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index 2fef63af88..9c7681c19b 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -55,6 +55,15 @@ if [ "$IMGPROTO" = "sheepdog" ]; then
     _notrun "image protocol $IMGPROTO does not support large image sizes"
 fi
 
+# Sanity check: For raw, we require a file system that permits the creation
+# of a HUGE (but very sparse) file. Check we can create it before continuing.
+if [ "$IMGFMT" = "raw" ]; then
+    if ! truncate --size=5T "$TEST_IMG"; then
+        _notrun "file system on $TEST_DIR does not support large enough files"
+    fi
+    rm "$TEST_IMG"
+fi
+
 echo
 echo "creating large image"
 _make_test_img 5000G
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 1/7] tests/qemu-iotests/005: Add a sanity check for large sparse file support
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

"check -raw 005" fails when running on certain filesystems - these do not
support such large sparse files. Use the same check as in test 220 to
skip the test in this case.

Suggested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/005 | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index 2fef63af88..9c7681c19b 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -55,6 +55,15 @@ if [ "$IMGPROTO" = "sheepdog" ]; then
     _notrun "image protocol $IMGPROTO does not support large image sizes"
 fi
 
+# Sanity check: For raw, we require a file system that permits the creation
+# of a HUGE (but very sparse) file. Check we can create it before continuing.
+if [ "$IMGFMT" = "raw" ]; then
+    if ! truncate --size=5T "$TEST_IMG"; then
+        _notrun "file system on $TEST_DIR does not support large enough files"
+    fi
+    rm "$TEST_IMG"
+fi
+
 echo
 echo "creating large image"
 _make_test_img 5000G
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 2/7] tests/qemu-iotests/check: Pick a default machine if necessary
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

qemu-system-arm, qemu-system-aarch64 and qemu-system-tricore do not have
a default machine, so when running the qemu-iotests with such a binary,
lots of tests are failing. Fix it by picking a default machine in the
"check" script instead.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/check | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index f9c24b6753..922c5d1d3d 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -130,7 +130,6 @@ export CACHEMODE="writeback"
 export QEMU_IO_OPTIONS=""
 export QEMU_IO_OPTIONS_NO_FMT=""
 export CACHEMODE_IS_DEFAULT=true
-export QEMU_OPTIONS="-nodefaults -machine accel=qtest"
 export VALGRIND_QEMU=
 export IMGKEYSECRET=
 export IMGOPTSSYNTAX=false
@@ -564,6 +563,18 @@ then
 fi
 export QEMU_PROG="$(type -p "$QEMU_PROG")"
 
+case "$QEMU_PROG" in
+    *qemu-system-arm|*qemu-system-aarch64)
+        export QEMU_OPTIONS="-nodefaults -machine virt,accel=qtest"
+        ;;
+    *qemu-system-tricore)
+        export QEMU_OPTIONS="-nodefaults -machine tricore_testboard,accel=qtest"
+        ;;
+    *)
+        export QEMU_OPTIONS="-nodefaults -machine accel=qtest"
+        ;;
+esac
+
 if [ -z "$QEMU_IMG_PROG" ]; then
     if [ -x "$build_iotests/qemu-img" ]; then
         export QEMU_IMG_PROG="$build_iotests/qemu-img"
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 2/7] tests/qemu-iotests/check: Pick a default machine if necessary
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

qemu-system-arm, qemu-system-aarch64 and qemu-system-tricore do not have
a default machine, so when running the qemu-iotests with such a binary,
lots of tests are failing. Fix it by picking a default machine in the
"check" script instead.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/check | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index f9c24b6753..922c5d1d3d 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -130,7 +130,6 @@ export CACHEMODE="writeback"
 export QEMU_IO_OPTIONS=""
 export QEMU_IO_OPTIONS_NO_FMT=""
 export CACHEMODE_IS_DEFAULT=true
-export QEMU_OPTIONS="-nodefaults -machine accel=qtest"
 export VALGRIND_QEMU=
 export IMGKEYSECRET=
 export IMGOPTSSYNTAX=false
@@ -564,6 +563,18 @@ then
 fi
 export QEMU_PROG="$(type -p "$QEMU_PROG")"
 
+case "$QEMU_PROG" in
+    *qemu-system-arm|*qemu-system-aarch64)
+        export QEMU_OPTIONS="-nodefaults -machine virt,accel=qtest"
+        ;;
+    *qemu-system-tricore)
+        export QEMU_OPTIONS="-nodefaults -machine tricore_testboard,accel=qtest"
+        ;;
+    *)
+        export QEMU_OPTIONS="-nodefaults -machine accel=qtest"
+        ;;
+esac
+
 if [ -z "$QEMU_IMG_PROG" ]; then
     if [ -x "$build_iotests/qemu-img" ]; then
         export QEMU_IMG_PROG="$build_iotests/qemu-img"
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 3/7] tests/qemu-iotests: Do not hard-code the path to bash
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

bash is installed in a different directory on non-Linux systems like
FreeBSD. Do not hard-code /bin/bash here so that the tests can run
there, too.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/239 | 2 +-
 tests/qemu-iotests/240 | 2 +-
 tests/qemu-iotests/241 | 2 +-
 tests/qemu-iotests/243 | 2 +-
 tests/qemu-iotests/244 | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/239 b/tests/qemu-iotests/239
index 6f085d573d..b0991ffe59 100755
--- a/tests/qemu-iotests/239
+++ b/tests/qemu-iotests/239
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test case for dmg
 #
diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240
index 65cc3b39b1..d3e663ed65 100755
--- a/tests/qemu-iotests/240
+++ b/tests/qemu-iotests/240
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test hot plugging and unplugging with iothreads
 #
diff --git a/tests/qemu-iotests/241 b/tests/qemu-iotests/241
index 017a736aab..58b64ebf41 100755
--- a/tests/qemu-iotests/241
+++ b/tests/qemu-iotests/241
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test qemu-nbd vs. unaligned images
 #
diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243
index 5838c6e89c..e563761307 100755
--- a/tests/qemu-iotests/243
+++ b/tests/qemu-iotests/243
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test qcow2 preallocation
 #
diff --git a/tests/qemu-iotests/244 b/tests/qemu-iotests/244
index d8e7122305..13978f93d2 100755
--- a/tests/qemu-iotests/244
+++ b/tests/qemu-iotests/244
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test qcow2 with external data files
 #
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 3/7] tests/qemu-iotests: Do not hard-code the path to bash
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

bash is installed in a different directory on non-Linux systems like
FreeBSD. Do not hard-code /bin/bash here so that the tests can run
there, too.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/239 | 2 +-
 tests/qemu-iotests/240 | 2 +-
 tests/qemu-iotests/241 | 2 +-
 tests/qemu-iotests/243 | 2 +-
 tests/qemu-iotests/244 | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/239 b/tests/qemu-iotests/239
index 6f085d573d..b0991ffe59 100755
--- a/tests/qemu-iotests/239
+++ b/tests/qemu-iotests/239
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test case for dmg
 #
diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240
index 65cc3b39b1..d3e663ed65 100755
--- a/tests/qemu-iotests/240
+++ b/tests/qemu-iotests/240
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test hot plugging and unplugging with iothreads
 #
diff --git a/tests/qemu-iotests/241 b/tests/qemu-iotests/241
index 017a736aab..58b64ebf41 100755
--- a/tests/qemu-iotests/241
+++ b/tests/qemu-iotests/241
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test qemu-nbd vs. unaligned images
 #
diff --git a/tests/qemu-iotests/243 b/tests/qemu-iotests/243
index 5838c6e89c..e563761307 100755
--- a/tests/qemu-iotests/243
+++ b/tests/qemu-iotests/243
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test qcow2 preallocation
 #
diff --git a/tests/qemu-iotests/244 b/tests/qemu-iotests/244
index d8e7122305..13978f93d2 100755
--- a/tests/qemu-iotests/244
+++ b/tests/qemu-iotests/244
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 #
 # Test qcow2 with external data files
 #
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 4/7] cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

We are going to enable the qemu-iotests during "make check" again,
and for running the iotests, we need bash and gnu-sed.

Reviewed-by: Li-Wen Hsu <lwhsu@freebsd.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .cirrus.yml | 4 ++--
 .travis.yml | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 47ef5bc604..8326a3a4b1 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -7,7 +7,7 @@ freebsd_12_task:
     cpu: 8
     memory: 8G
   install_script: pkg install -y
-    bison curl cyrus-sasl git glib gmake gnutls
+    bash bison curl cyrus-sasl git glib gmake gnutls gsed
     nettle perl5 pixman pkgconf png usbredir
   script:
     - mkdir build
@@ -20,7 +20,7 @@ macos_task:
   osx_instance:
     image: mojave-base
   install_script:
-    - brew install pkg-config python glib pixman make sdl2
+    - brew install pkg-config python gnu-sed glib pixman make sdl2
   script:
     - ./configure --python=/usr/local/bin/python3 || { cat config.log; exit 1; }
     - gmake -j$(sysctl -n hw.ncpu)
diff --git a/.travis.yml b/.travis.yml
index 2e06aee9d0..ba94644192 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -42,6 +42,7 @@ addons:
     packages:
       - glib
       - pixman
+      - gnu-sed
 
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 4/7] cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

We are going to enable the qemu-iotests during "make check" again,
and for running the iotests, we need bash and gnu-sed.

Reviewed-by: Li-Wen Hsu <lwhsu@freebsd.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .cirrus.yml | 4 ++--
 .travis.yml | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 47ef5bc604..8326a3a4b1 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -7,7 +7,7 @@ freebsd_12_task:
     cpu: 8
     memory: 8G
   install_script: pkg install -y
-    bison curl cyrus-sasl git glib gmake gnutls
+    bash bison curl cyrus-sasl git glib gmake gnutls gsed
     nettle perl5 pixman pkgconf png usbredir
   script:
     - mkdir build
@@ -20,7 +20,7 @@ macos_task:
   osx_instance:
     image: mojave-base
   install_script:
-    - brew install pkg-config python glib pixman make sdl2
+    - brew install pkg-config python gnu-sed glib pixman make sdl2
   script:
     - ./configure --python=/usr/local/bin/python3 || { cat config.log; exit 1; }
     - gmake -j$(sysctl -n hw.ncpu)
diff --git a/.travis.yml b/.travis.yml
index 2e06aee9d0..ba94644192 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -42,6 +42,7 @@ addons:
     packages:
       - glib
       - pixman
+      - gnu-sed
 
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 5/7] tests/qemu-iotests: Remove the "_supported_os Linux" line from many tests
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

A lot of tests run fine on FreeBSD and macOS, too - the limitation
to Linux here was likely just copied-and-pasted from other tests.
Thus remove the "_supported_os Linux" line from tests that run
successful in our CI pipelines on FreeBSD and macOS.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/001 | 1 -
 tests/qemu-iotests/002 | 1 -
 tests/qemu-iotests/003 | 1 -
 tests/qemu-iotests/004 | 1 -
 tests/qemu-iotests/007 | 1 -
 tests/qemu-iotests/008 | 1 -
 tests/qemu-iotests/009 | 1 -
 tests/qemu-iotests/010 | 1 -
 tests/qemu-iotests/011 | 1 -
 tests/qemu-iotests/012 | 1 -
 tests/qemu-iotests/015 | 1 -
 tests/qemu-iotests/017 | 1 -
 tests/qemu-iotests/020 | 1 -
 tests/qemu-iotests/021 | 1 -
 tests/qemu-iotests/022 | 1 -
 tests/qemu-iotests/025 | 1 -
 tests/qemu-iotests/026 | 1 -
 tests/qemu-iotests/027 | 1 -
 tests/qemu-iotests/029 | 1 -
 tests/qemu-iotests/031 | 1 -
 tests/qemu-iotests/032 | 1 -
 tests/qemu-iotests/033 | 1 -
 tests/qemu-iotests/035 | 1 -
 tests/qemu-iotests/036 | 1 -
 tests/qemu-iotests/037 | 1 -
 tests/qemu-iotests/042 | 1 -
 tests/qemu-iotests/043 | 1 -
 tests/qemu-iotests/046 | 1 -
 tests/qemu-iotests/047 | 1 -
 tests/qemu-iotests/049 | 1 -
 tests/qemu-iotests/050 | 1 -
 tests/qemu-iotests/051 | 1 -
 tests/qemu-iotests/052 | 1 -
 tests/qemu-iotests/053 | 1 -
 tests/qemu-iotests/054 | 1 -
 tests/qemu-iotests/062 | 1 -
 tests/qemu-iotests/063 | 1 -
 tests/qemu-iotests/066 | 1 -
 tests/qemu-iotests/067 | 1 -
 tests/qemu-iotests/068 | 1 -
 tests/qemu-iotests/069 | 1 -
 tests/qemu-iotests/071 | 1 -
 tests/qemu-iotests/072 | 1 -
 tests/qemu-iotests/073 | 1 -
 tests/qemu-iotests/079 | 1 -
 tests/qemu-iotests/082 | 1 -
 tests/qemu-iotests/085 | 1 -
 tests/qemu-iotests/089 | 1 -
 tests/qemu-iotests/090 | 1 -
 tests/qemu-iotests/094 | 1 -
 tests/qemu-iotests/095 | 1 -
 tests/qemu-iotests/098 | 1 -
 tests/qemu-iotests/102 | 1 -
 tests/qemu-iotests/103 | 1 -
 tests/qemu-iotests/104 | 1 -
 tests/qemu-iotests/105 | 1 -
 tests/qemu-iotests/107 | 1 -
 tests/qemu-iotests/110 | 1 -
 tests/qemu-iotests/111 | 1 -
 tests/qemu-iotests/112 | 1 -
 tests/qemu-iotests/114 | 1 -
 tests/qemu-iotests/115 | 1 -
 tests/qemu-iotests/117 | 1 -
 tests/qemu-iotests/120 | 1 -
 tests/qemu-iotests/125 | 1 -
 tests/qemu-iotests/126 | 1 -
 tests/qemu-iotests/127 | 1 -
 tests/qemu-iotests/133 | 1 -
 tests/qemu-iotests/134 | 1 -
 tests/qemu-iotests/142 | 1 -
 tests/qemu-iotests/143 | 1 -
 tests/qemu-iotests/144 | 1 -
 tests/qemu-iotests/145 | 1 -
 tests/qemu-iotests/153 | 1 -
 tests/qemu-iotests/156 | 1 -
 tests/qemu-iotests/157 | 1 -
 tests/qemu-iotests/158 | 1 -
 tests/qemu-iotests/159 | 1 -
 tests/qemu-iotests/162 | 1 -
 tests/qemu-iotests/170 | 1 -
 tests/qemu-iotests/173 | 1 -
 tests/qemu-iotests/182 | 1 -
 tests/qemu-iotests/183 | 1 -
 tests/qemu-iotests/186 | 1 -
 tests/qemu-iotests/187 | 1 -
 tests/qemu-iotests/190 | 1 -
 tests/qemu-iotests/191 | 1 -
 tests/qemu-iotests/192 | 1 -
 tests/qemu-iotests/195 | 1 -
 tests/qemu-iotests/197 | 1 -
 tests/qemu-iotests/200 | 1 -
 tests/qemu-iotests/214 | 1 -
 tests/qemu-iotests/215 | 1 -
 tests/qemu-iotests/217 | 1 -
 tests/qemu-iotests/227 | 1 -
 tests/qemu-iotests/231 | 1 -
 tests/qemu-iotests/233 | 1 -
 tests/qemu-iotests/240 | 1 -
 tests/qemu-iotests/247 | 1 -
 99 files changed, 99 deletions(-)

diff --git a/tests/qemu-iotests/001 b/tests/qemu-iotests/001
index 5d266e170a..d87a535c33 100755
--- a/tests/qemu-iotests/001
+++ b/tests/qemu-iotests/001
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index 7fb85084a1..fd413bce48 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index f008c57cdc..ccd3a39dfb 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 size=128M
 offset=67M
diff --git a/tests/qemu-iotests/004 b/tests/qemu-iotests/004
index 64fab3e714..d308dc4b49 100755
--- a/tests/qemu-iotests/004
+++ b/tests/qemu-iotests/004
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt raw qcow qcow2 qed vdi vmdk vhdx luks
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/007 b/tests/qemu-iotests/007
index 3ab5490db3..6abd402423 100755
--- a/tests/qemu-iotests/007
+++ b/tests/qemu-iotests/007
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # currently only qcow2 allows for consistency checks using qemu-img
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 # refcount_bits must be at least 4 so we can create ten internal snapshots
 # (1 bit supports none, 2 bits support two, 4 bits support 14)
 _unsupported_imgopts 'refcount_bits=\(1\|2\)[^0-9]'
diff --git a/tests/qemu-iotests/008 b/tests/qemu-iotests/008
index 75067e36ad..2b81b119bf 100755
--- a/tests/qemu-iotests/008
+++ b/tests/qemu-iotests/008
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index bc4b461122..51b200db1d 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index 6920408d28..48c533f632 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index b4c7e8f799..8b1fce069a 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/012 b/tests/qemu-iotests/012
index 2c3b42d9dd..12957285b3 100755
--- a/tests/qemu-iotests/012
+++ b/tests/qemu-iotests/012
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 # Remove once all tests are fixed to use TEST_IMG_FILE
 # correctly and common.rc sets it unconditionally
diff --git a/tests/qemu-iotests/015 b/tests/qemu-iotests/015
index 5a4063e4f5..eec5387f3d 100755
--- a/tests/qemu-iotests/015
+++ b/tests/qemu-iotests/015
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # actually any format that supports snapshots
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 # Internal snapshots are (currently) impossible with refcount_bits=1
 _unsupported_imgopts 'refcount_bits=1[^0-9]'
 
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index 83744f29a3..79875de454 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index 71fa753b4e..6b0ebb37d2 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
diff --git a/tests/qemu-iotests/021 b/tests/qemu-iotests/021
index f6555f3b74..f888269fd4 100755
--- a/tests/qemu-iotests/021
+++ b/tests/qemu-iotests/021
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/022 b/tests/qemu-iotests/022
index b68cd64b33..99eb08f57f 100755
--- a/tests/qemu-iotests/022
+++ b/tests/qemu-iotests/022
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format that supports snapshots
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 TEST_OFFSETS="10485760 4294967296"
 CLUSTER_SIZE="4096"
diff --git a/tests/qemu-iotests/025 b/tests/qemu-iotests/025
index d9a4ebc5e7..e05d833452 100755
--- a/tests/qemu-iotests/025
+++ b/tests/qemu-iotests/025
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt raw qcow2 qed luks
 _supported_proto file sheepdog rbd nfs
-_supported_os Linux
 
 echo "=== Creating image"
 echo
diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index ca89ad7048..e30243608b 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Currently only qcow2 supports rebasing
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 _default_cache_mode "writethrough"
 _supported_cache_modes "writethrough" "none"
 # The refcount table tests expect a certain minimum width for refcount entries
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index b7df9701f7..4cb638022a 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vmdk qcow qcow2 qed
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/029 b/tests/qemu-iotests/029
index 5f42f76cc6..94c2713132 100755
--- a/tests/qemu-iotests/029
+++ b/tests/qemu-iotests/029
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 # Internal snapshots are (currently) impossible with refcount_bits=1
 _unsupported_imgopts 'refcount_bits=1[^0-9]'
 
diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index ef92d8eee3..a3c25ec237 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 CLUSTER_SIZE=65536
 
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index a1757bb15e..23c216c549 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This works for any image format (though unlikely to segfault for raw)
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 echo
 echo === Prepare image ===
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index cfdf1ec2ba..362a48c0a0 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/035 b/tests/qemu-iotests/035
index 46aa835936..ad6fa3115a 100755
--- a/tests/qemu-iotests/035
+++ b/tests/qemu-iotests/035
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
index 1b56394129..f06ff67408 100755
--- a/tests/qemu-iotests/036
+++ b/tests/qemu-iotests/036
@@ -43,7 +43,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # Only qcow2v3 and later supports feature bits
 IMGOPTS="compat=1.1"
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index 0781bebefe..819a2a52d2 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
diff --git a/tests/qemu-iotests/042 b/tests/qemu-iotests/042
index a9a7fc3041..0e85b356b7 100755
--- a/tests/qemu-iotests/042
+++ b/tests/qemu-iotests/042
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 echo
 echo "== Creating zero size image =="
diff --git a/tests/qemu-iotests/043 b/tests/qemu-iotests/043
index 9894b154ec..67cc7e74c2 100755
--- a/tests/qemu-iotests/043
+++ b/tests/qemu-iotests/043
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046
index 95160bea4c..543355c64f 100755
--- a/tests/qemu-iotests/046
+++ b/tests/qemu-iotests/046
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 CLUSTER_SIZE=64k
 size=128M
diff --git a/tests/qemu-iotests/047 b/tests/qemu-iotests/047
index ce81fc6fa7..4528465fb0 100755
--- a/tests/qemu-iotests/047
+++ b/tests/qemu-iotests/047
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 size=128M
 
diff --git a/tests/qemu-iotests/049 b/tests/qemu-iotests/049
index bc09cd6717..c100d30ed0 100755
--- a/tests/qemu-iotests/049
+++ b/tests/qemu-iotests/049
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 filter_test_dir()
 {
diff --git a/tests/qemu-iotests/050 b/tests/qemu-iotests/050
index dd7b2c72eb..211fc00797 100755
--- a/tests/qemu-iotests/050
+++ b/tests/qemu-iotests/050
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 if test "$IMGFMT" = qcow2 && test $IMGOPTS = ""; then
   IMGOPTS=compat=1.1
diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
index 02ac960da4..a3deb1fcad 100755
--- a/tests/qemu-iotests/051
+++ b/tests/qemu-iotests/051
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # A compat=0.10 image is created in this test which does not support anything
 # other than refcount_bits=16
 _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index b3a2dc1143..6e2ecbfe21 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 # Don't do O_DIRECT on tmpfs
 _supported_cache_modes "writeback" "writethrough" "unsafe"
diff --git a/tests/qemu-iotests/053 b/tests/qemu-iotests/053
index 50c62f0f56..e82bb69881 100755
--- a/tests/qemu-iotests/053
+++ b/tests/qemu-iotests/053
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qcow
 _supported_proto file
-_supported_os Linux
 
 echo
 echo "== Creating single sector image =="
diff --git a/tests/qemu-iotests/054 b/tests/qemu-iotests/054
index 0d5e14f847..a8905b60d0 100755
--- a/tests/qemu-iotests/054
+++ b/tests/qemu-iotests/054
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 echo
 echo "creating too large image (1 EB)"
diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062
index ed7400fed2..d5f818fcce 100755
--- a/tests/qemu-iotests/062
+++ b/tests/qemu-iotests/062
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 IMGOPTS="compat=1.1"
 IMG_SIZE=64M
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index 2d5c0ce9fb..fe4892e467 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow qcow2 vmdk qed raw
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066
index f480986e35..28f8c98412 100755
--- a/tests/qemu-iotests/066
+++ b/tests/qemu-iotests/066
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 # Intentionally create an unaligned image
 IMGOPTS="compat=1.1"
diff --git a/tests/qemu-iotests/067 b/tests/qemu-iotests/067
index fda16a6b0d..926c79b37c 100755
--- a/tests/qemu-iotests/067
+++ b/tests/qemu-iotests/067
@@ -32,7 +32,6 @@ status=1	# failure is the default!
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # Because anything other than 16 would change the output of query-block
 _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
 
diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068
index 881a022107..22f5ca3ba6 100755
--- a/tests/qemu-iotests/068
+++ b/tests/qemu-iotests/068
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 IMGOPTS="compat=1.1"
 IMG_SIZE=128K
diff --git a/tests/qemu-iotests/069 b/tests/qemu-iotests/069
index 6a8e4aa22e..3974714852 100755
--- a/tests/qemu-iotests/069
+++ b/tests/qemu-iotests/069
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 IMG_SIZE=128K
diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index 7f3e5abd57..1cca9233d0 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index 6f9f247fa5..661b36da2d 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
 _supported_proto file
-_supported_os Linux
 
 IMG_SIZE=64M
 
diff --git a/tests/qemu-iotests/073 b/tests/qemu-iotests/073
index 990f90acbd..e684b1b780 100755
--- a/tests/qemu-iotests/073
+++ b/tests/qemu-iotests/073
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 CLUSTER_SIZE=64k
 size=128M
diff --git a/tests/qemu-iotests/079 b/tests/qemu-iotests/079
index 1b6594ebef..81f0c21f53 100755
--- a/tests/qemu-iotests/079
+++ b/tests/qemu-iotests/079
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 echo "=== Check option preallocation and cluster_size ==="
 echo
diff --git a/tests/qemu-iotests/082 b/tests/qemu-iotests/082
index 278511dba4..bbbdf555dc 100755
--- a/tests/qemu-iotests/082
+++ b/tests/qemu-iotests/082
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 run_qemu_img()
 {
diff --git a/tests/qemu-iotests/085 b/tests/qemu-iotests/085
index 68cb665987..d40fdab542 100755
--- a/tests/qemu-iotests/085
+++ b/tests/qemu-iotests/085
@@ -56,7 +56,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 
 # ${1}: unique identifier for the snapshot filename
diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089
index 6609954908..ad029f1f09 100755
--- a/tests/qemu-iotests/089
+++ b/tests/qemu-iotests/089
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # Because anything other than 16 would change the output of qemu_io -c info
 _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
 
diff --git a/tests/qemu-iotests/090 b/tests/qemu-iotests/090
index 193bae7d77..9f8cfbb80f 100755
--- a/tests/qemu-iotests/090
+++ b/tests/qemu-iotests/090
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 IMG_SIZE=128K
 
diff --git a/tests/qemu-iotests/094 b/tests/qemu-iotests/094
index 0bcca77261..9343e09492 100755
--- a/tests/qemu-iotests/094
+++ b/tests/qemu-iotests/094
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto nbd
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 _make_test_img 64M
diff --git a/tests/qemu-iotests/095 b/tests/qemu-iotests/095
index 18505b7181..58fe174b5e 100755
--- a/tests/qemu-iotests/095
+++ b/tests/qemu-iotests/095
@@ -44,7 +44,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size_smaller=5M
 size_larger=100M
diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098
index 461144c831..1c1d1c468f 100755
--- a/tests/qemu-iotests/098
+++ b/tests/qemu-iotests/098
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 IMGOPTS="compat=1.1"
 
diff --git a/tests/qemu-iotests/102 b/tests/qemu-iotests/102
index cedd2b25dc..749ff66b8a 100755
--- a/tests/qemu-iotests/102
+++ b/tests/qemu-iotests/102
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 IMG_SIZE=64K
 
diff --git a/tests/qemu-iotests/103 b/tests/qemu-iotests/103
index 6773e94d9f..554b9de054 100755
--- a/tests/qemu-iotests/103
+++ b/tests/qemu-iotests/103
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 # Internal snapshots are (currently) impossible with refcount_bits=1
 _unsupported_imgopts 'refcount_bits=1[^0-9]'
 
diff --git a/tests/qemu-iotests/104 b/tests/qemu-iotests/104
index 390167bad4..c70f28a9a1 100755
--- a/tests/qemu-iotests/104
+++ b/tests/qemu-iotests/104
@@ -34,7 +34,6 @@ trap "exit \$status" 0 1 2 3 15
 
 _supported_fmt raw qcow qcow2 qed vdi vmdk vhdx
 _supported_proto generic
-_supported_os Linux
 
 echo "=== Check qemu-img info output ==="
 echo
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index 3b5a596844..3346e8cb25 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 vmdk vhdx qed
 _supported_proto generic
-_supported_os Linux
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
 
diff --git a/tests/qemu-iotests/107 b/tests/qemu-iotests/107
index fcd5a24dfe..268ba27688 100755
--- a/tests/qemu-iotests/107
+++ b/tests/qemu-iotests/107
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 
 IMG_SIZE=64K
diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index fad672c1ae..4318341ac5 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 TEST_IMG_REL=$(basename "$TEST_IMG")
diff --git a/tests/qemu-iotests/111 b/tests/qemu-iotests/111
index 57395be64c..490a5bbcb5 100755
--- a/tests/qemu-iotests/111
+++ b/tests/qemu-iotests/111
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 $QEMU_IMG create -f $IMGFMT -b "$TEST_IMG.inexistent" "$TEST_IMG" 2>&1 \
diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 6d81c75a9c..706c10b600 100755
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # This test will set refcount_bits on its own which would conflict with the
 # manual setting; compat will be overridden as well
 _unsupported_imgopts refcount_bits 'compat=0.10'
diff --git a/tests/qemu-iotests/114 b/tests/qemu-iotests/114
index f36b88f3f3..f90a744fc0 100755
--- a/tests/qemu-iotests/114
+++ b/tests/qemu-iotests/114
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 
 TEST_IMG="$TEST_IMG.base" _make_test_img 64M
diff --git a/tests/qemu-iotests/115 b/tests/qemu-iotests/115
index 7ed347010f..9ed3cb6a83 100755
--- a/tests/qemu-iotests/115
+++ b/tests/qemu-iotests/115
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # This test relies on refcounts being 64 bits wide (which does not work with
 # compat=0.10)
 _unsupported_imgopts 'refcount_bits=\([^6]\|.\([^4]\|$\)\)' 'compat=0.10'
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
index 0af0f31c5a..f37b34f8b1 100755
--- a/tests/qemu-iotests/117
+++ b/tests/qemu-iotests/117
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 _make_test_img 64k
 
diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
index ca95b9276e..e9b4fbb009 100755
--- a/tests/qemu-iotests/120
+++ b/tests/qemu-iotests/120
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 _unsupported_fmt luks
 
 _make_test_img 64M
diff --git a/tests/qemu-iotests/125 b/tests/qemu-iotests/125
index 212dcd8f0d..dc4b8f5fb9 100755
--- a/tests/qemu-iotests/125
+++ b/tests/qemu-iotests/125
@@ -44,7 +44,6 @@ get_image_size_on_host()
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 if [ -z "$TEST_IMG_FILE" ]; then
     TEST_IMG_FILE=$TEST_IMG
diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
index 96dc048d59..580fae5d62 100755
--- a/tests/qemu-iotests/126
+++ b/tests/qemu-iotests/126
@@ -37,7 +37,6 @@ _supported_fmt qcow qcow2 qed vmdk
 # colons which separate a protocol prefix from the rest and colons which are
 # just part of the filename, so we cannot test protocols which require a prefix)
 _supported_proto file
-_supported_os Linux
 
 echo
 echo '=== Testing plain files ==='
diff --git a/tests/qemu-iotests/127 b/tests/qemu-iotests/127
index 3e941f74d4..b64926ab31 100755
--- a/tests/qemu-iotests/127
+++ b/tests/qemu-iotests/127
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 IMG_SIZE=64K
 
diff --git a/tests/qemu-iotests/133 b/tests/qemu-iotests/133
index 1f6056d144..6f7cacc091 100755
--- a/tests/qemu-iotests/133
+++ b/tests/qemu-iotests/133
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 TEST_IMG="$TEST_IMG.base" _make_test_img 64M
 _make_test_img -b "$TEST_IMG.base"
diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
index e9e3e84c2a..141a2eaa7e 100755
--- a/tests/qemu-iotests/134
+++ b/tests/qemu-iotests/134
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/142 b/tests/qemu-iotests/142
index d9b98cf60a..6b62271876 100755
--- a/tests/qemu-iotests/142
+++ b/tests/qemu-iotests/142
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # We test all cache modes anyway, but O_DIRECT needs to be supported
 _default_cache_mode none
diff --git a/tests/qemu-iotests/143 b/tests/qemu-iotests/143
index c223867cb3..92249ac8da 100755
--- a/tests/qemu-iotests/143
+++ b/tests/qemu-iotests/143
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 keep_stderr=y \
 _launch_qemu 2> >(_filter_nbd)
diff --git a/tests/qemu-iotests/144 b/tests/qemu-iotests/144
index 15157f33d7..011ed4f2bc 100755
--- a/tests/qemu-iotests/144
+++ b/tests/qemu-iotests/144
@@ -46,7 +46,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=512M
 
diff --git a/tests/qemu-iotests/145 b/tests/qemu-iotests/145
index 28878dc8a1..9427549651 100755
--- a/tests/qemu-iotests/145
+++ b/tests/qemu-iotests/145
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 _make_test_img 1M
 
diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
index 08ad8a6730..c969a1a16f 100755
--- a/tests/qemu-iotests/153
+++ b/tests/qemu-iotests/153
@@ -62,7 +62,6 @@ _check_ofd || _notrun "OFD lock not available"
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 _run_cmd()
 {
diff --git a/tests/qemu-iotests/156 b/tests/qemu-iotests/156
index 8d134029c6..2ffa3ca942 100755
--- a/tests/qemu-iotests/156
+++ b/tests/qemu-iotests/156
@@ -49,7 +49,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2 qed
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 # Create source disk
 TEST_IMG="$TEST_IMG.backing" _make_test_img 1M
diff --git a/tests/qemu-iotests/157 b/tests/qemu-iotests/157
index 69b25cab30..7cbac38099 100755
--- a/tests/qemu-iotests/157
+++ b/tests/qemu-iotests/157
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/158 b/tests/qemu-iotests/158
index 8c0928a7f9..ba4db6116a 100755
--- a/tests/qemu-iotests/158
+++ b/tests/qemu-iotests/158
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/159 b/tests/qemu-iotests/159
index 29066eebde..2557140ac2 100755
--- a/tests/qemu-iotests/159
+++ b/tests/qemu-iotests/159
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 _unsupported_fmt luks
 
 TEST_SIZES="5 512 1024 1999 1K 64K 1M"
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index 2e9947fd9a..4e5ed74fd5 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-_supported_os Linux
 
 test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh\( \|$\)')
 [ "$test_ssh" = "" ] && _notrun "ssh support required"
diff --git a/tests/qemu-iotests/170 b/tests/qemu-iotests/170
index 7deb7563c9..05dd6ed6c3 100755
--- a/tests/qemu-iotests/170
+++ b/tests/qemu-iotests/170
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 _unsupported_fmt luks
 
 echo
diff --git a/tests/qemu-iotests/173 b/tests/qemu-iotests/173
index 47036a5564..9e2fa2e73c 100755
--- a/tests/qemu-iotests/173
+++ b/tests/qemu-iotests/173
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto nfs
-_supported_os Linux
 
 size=100M
 
diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182
index ff3d7e7ec1..fd7627fb65 100755
--- a/tests/qemu-iotests/182
+++ b/tests/qemu-iotests/182
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=32M
 
diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183
index 93b7bd798a..fbe5a99beb 100755
--- a/tests/qemu-iotests/183
+++ b/tests/qemu-iotests/183
@@ -44,7 +44,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 raw qed quorum
 _supported_proto file
-_supported_os Linux
 
 size=64M
 _make_test_img $size
diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
index 5dd2177b89..7e7d45babc 100755
--- a/tests/qemu-iotests/186
+++ b/tests/qemu-iotests/186
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/187 b/tests/qemu-iotests/187
index a45addde09..2fcef9e2bd 100755
--- a/tests/qemu-iotests/187
+++ b/tests/qemu-iotests/187
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=64M
 _make_test_img $size
diff --git a/tests/qemu-iotests/190 b/tests/qemu-iotests/190
index e1c1d407f0..eb766ad09f 100755
--- a/tests/qemu-iotests/190
+++ b/tests/qemu-iotests/190
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # See 178 for more extensive tests across more formats
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 echo "== Huge file =="
 echo
diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191
index 1ea908ce3d..528022e8d8 100755
--- a/tests/qemu-iotests/191
+++ b/tests/qemu-iotests/191
@@ -43,7 +43,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=64M
 
diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192
index 158086f9d2..84d1ed7cfa 100755
--- a/tests/qemu-iotests/192
+++ b/tests/qemu-iotests/192
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/195 b/tests/qemu-iotests/195
index bd1b71ae5e..ef7b9a94e2 100755
--- a/tests/qemu-iotests/195
+++ b/tests/qemu-iotests/195
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index 2c664793f4..383d7d7f61 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -51,7 +51,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Test is supported for any backing file; but we force qcow2 for our wrapper.
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
 
diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200
index 1c0f8cafc2..72d431f251 100755
--- a/tests/qemu-iotests/200
+++ b/tests/qemu-iotests/200
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 BACKING_IMG="${TEST_DIR}/backing.img"
 TEST_IMG="${TEST_DIR}/test.img"
diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index c1a452ff9a..21ec8a2ad8 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -36,7 +36,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # Repairing the corrupted image requires qemu-img check to store a
 # refcount up to 3, which requires at least two refcount bits.
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index 7b063d7cfa..958c14f5a0 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -48,7 +48,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Test is supported for any backing file; but we force qcow2 for our wrapper.
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
 
diff --git a/tests/qemu-iotests/217 b/tests/qemu-iotests/217
index f5482bb669..58a78a6098 100755
--- a/tests/qemu-iotests/217
+++ b/tests/qemu-iotests/217
@@ -37,7 +37,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This test is specific to qcow2
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # This test needs clusters with at least a refcount of 2 so that
 # OFLAG_COPIED is not set.  refcount_bits=1 is therefore unsupported.
diff --git a/tests/qemu-iotests/227 b/tests/qemu-iotests/227
index 10cf144eb0..bdd727a721 100755
--- a/tests/qemu-iotests/227
+++ b/tests/qemu-iotests/227
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/231 b/tests/qemu-iotests/231
index 5b2cbab9ac..c0b053ac30 100755
--- a/tests/qemu-iotests/231
+++ b/tests/qemu-iotests/231
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto rbd
-_supported_os Linux
 
 BOGUS_CONF=${TEST_DIR}/ceph-$$.conf
 touch "${BOGUS_CONF}"
diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index b8b6c8cc4c..5855f94b9a 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -47,7 +47,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt raw qcow2
 _supported_proto file
 # If porting to non-Linux, consider using socat instead of ss in common.nbd
-_supported_os Linux
 _require_command QEMU_NBD
 
 nbd_server_set_tcp_port
diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240
index d3e663ed65..b4cf95096d 100755
--- a/tests/qemu-iotests/240
+++ b/tests/qemu-iotests/240
@@ -33,7 +33,6 @@ status=1	# failure is the default!
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests/247
index fc50eb5dc1..546a794d3d 100755
--- a/tests/qemu-iotests/247
+++ b/tests/qemu-iotests/247
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Requires backing files and .bdrv_change_backing_file support
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 size=128M
 
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 5/7] tests/qemu-iotests: Remove the "_supported_os Linux" line from many tests
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

A lot of tests run fine on FreeBSD and macOS, too - the limitation
to Linux here was likely just copied-and-pasted from other tests.
Thus remove the "_supported_os Linux" line from tests that run
successful in our CI pipelines on FreeBSD and macOS.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/001 | 1 -
 tests/qemu-iotests/002 | 1 -
 tests/qemu-iotests/003 | 1 -
 tests/qemu-iotests/004 | 1 -
 tests/qemu-iotests/007 | 1 -
 tests/qemu-iotests/008 | 1 -
 tests/qemu-iotests/009 | 1 -
 tests/qemu-iotests/010 | 1 -
 tests/qemu-iotests/011 | 1 -
 tests/qemu-iotests/012 | 1 -
 tests/qemu-iotests/015 | 1 -
 tests/qemu-iotests/017 | 1 -
 tests/qemu-iotests/020 | 1 -
 tests/qemu-iotests/021 | 1 -
 tests/qemu-iotests/022 | 1 -
 tests/qemu-iotests/025 | 1 -
 tests/qemu-iotests/026 | 1 -
 tests/qemu-iotests/027 | 1 -
 tests/qemu-iotests/029 | 1 -
 tests/qemu-iotests/031 | 1 -
 tests/qemu-iotests/032 | 1 -
 tests/qemu-iotests/033 | 1 -
 tests/qemu-iotests/035 | 1 -
 tests/qemu-iotests/036 | 1 -
 tests/qemu-iotests/037 | 1 -
 tests/qemu-iotests/042 | 1 -
 tests/qemu-iotests/043 | 1 -
 tests/qemu-iotests/046 | 1 -
 tests/qemu-iotests/047 | 1 -
 tests/qemu-iotests/049 | 1 -
 tests/qemu-iotests/050 | 1 -
 tests/qemu-iotests/051 | 1 -
 tests/qemu-iotests/052 | 1 -
 tests/qemu-iotests/053 | 1 -
 tests/qemu-iotests/054 | 1 -
 tests/qemu-iotests/062 | 1 -
 tests/qemu-iotests/063 | 1 -
 tests/qemu-iotests/066 | 1 -
 tests/qemu-iotests/067 | 1 -
 tests/qemu-iotests/068 | 1 -
 tests/qemu-iotests/069 | 1 -
 tests/qemu-iotests/071 | 1 -
 tests/qemu-iotests/072 | 1 -
 tests/qemu-iotests/073 | 1 -
 tests/qemu-iotests/079 | 1 -
 tests/qemu-iotests/082 | 1 -
 tests/qemu-iotests/085 | 1 -
 tests/qemu-iotests/089 | 1 -
 tests/qemu-iotests/090 | 1 -
 tests/qemu-iotests/094 | 1 -
 tests/qemu-iotests/095 | 1 -
 tests/qemu-iotests/098 | 1 -
 tests/qemu-iotests/102 | 1 -
 tests/qemu-iotests/103 | 1 -
 tests/qemu-iotests/104 | 1 -
 tests/qemu-iotests/105 | 1 -
 tests/qemu-iotests/107 | 1 -
 tests/qemu-iotests/110 | 1 -
 tests/qemu-iotests/111 | 1 -
 tests/qemu-iotests/112 | 1 -
 tests/qemu-iotests/114 | 1 -
 tests/qemu-iotests/115 | 1 -
 tests/qemu-iotests/117 | 1 -
 tests/qemu-iotests/120 | 1 -
 tests/qemu-iotests/125 | 1 -
 tests/qemu-iotests/126 | 1 -
 tests/qemu-iotests/127 | 1 -
 tests/qemu-iotests/133 | 1 -
 tests/qemu-iotests/134 | 1 -
 tests/qemu-iotests/142 | 1 -
 tests/qemu-iotests/143 | 1 -
 tests/qemu-iotests/144 | 1 -
 tests/qemu-iotests/145 | 1 -
 tests/qemu-iotests/153 | 1 -
 tests/qemu-iotests/156 | 1 -
 tests/qemu-iotests/157 | 1 -
 tests/qemu-iotests/158 | 1 -
 tests/qemu-iotests/159 | 1 -
 tests/qemu-iotests/162 | 1 -
 tests/qemu-iotests/170 | 1 -
 tests/qemu-iotests/173 | 1 -
 tests/qemu-iotests/182 | 1 -
 tests/qemu-iotests/183 | 1 -
 tests/qemu-iotests/186 | 1 -
 tests/qemu-iotests/187 | 1 -
 tests/qemu-iotests/190 | 1 -
 tests/qemu-iotests/191 | 1 -
 tests/qemu-iotests/192 | 1 -
 tests/qemu-iotests/195 | 1 -
 tests/qemu-iotests/197 | 1 -
 tests/qemu-iotests/200 | 1 -
 tests/qemu-iotests/214 | 1 -
 tests/qemu-iotests/215 | 1 -
 tests/qemu-iotests/217 | 1 -
 tests/qemu-iotests/227 | 1 -
 tests/qemu-iotests/231 | 1 -
 tests/qemu-iotests/233 | 1 -
 tests/qemu-iotests/240 | 1 -
 tests/qemu-iotests/247 | 1 -
 99 files changed, 99 deletions(-)

diff --git a/tests/qemu-iotests/001 b/tests/qemu-iotests/001
index 5d266e170a..d87a535c33 100755
--- a/tests/qemu-iotests/001
+++ b/tests/qemu-iotests/001
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index 7fb85084a1..fd413bce48 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index f008c57cdc..ccd3a39dfb 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 size=128M
 offset=67M
diff --git a/tests/qemu-iotests/004 b/tests/qemu-iotests/004
index 64fab3e714..d308dc4b49 100755
--- a/tests/qemu-iotests/004
+++ b/tests/qemu-iotests/004
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt raw qcow qcow2 qed vdi vmdk vhdx luks
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/007 b/tests/qemu-iotests/007
index 3ab5490db3..6abd402423 100755
--- a/tests/qemu-iotests/007
+++ b/tests/qemu-iotests/007
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # currently only qcow2 allows for consistency checks using qemu-img
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 # refcount_bits must be at least 4 so we can create ten internal snapshots
 # (1 bit supports none, 2 bits support two, 4 bits support 14)
 _unsupported_imgopts 'refcount_bits=\(1\|2\)[^0-9]'
diff --git a/tests/qemu-iotests/008 b/tests/qemu-iotests/008
index 75067e36ad..2b81b119bf 100755
--- a/tests/qemu-iotests/008
+++ b/tests/qemu-iotests/008
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index bc4b461122..51b200db1d 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index 6920408d28..48c533f632 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index b4c7e8f799..8b1fce069a 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/012 b/tests/qemu-iotests/012
index 2c3b42d9dd..12957285b3 100755
--- a/tests/qemu-iotests/012
+++ b/tests/qemu-iotests/012
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 # Remove once all tests are fixed to use TEST_IMG_FILE
 # correctly and common.rc sets it unconditionally
diff --git a/tests/qemu-iotests/015 b/tests/qemu-iotests/015
index 5a4063e4f5..eec5387f3d 100755
--- a/tests/qemu-iotests/015
+++ b/tests/qemu-iotests/015
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # actually any format that supports snapshots
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 # Internal snapshots are (currently) impossible with refcount_bits=1
 _unsupported_imgopts 'refcount_bits=1[^0-9]'
 
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index 83744f29a3..79875de454 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index 71fa753b4e..6b0ebb37d2 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
diff --git a/tests/qemu-iotests/021 b/tests/qemu-iotests/021
index f6555f3b74..f888269fd4 100755
--- a/tests/qemu-iotests/021
+++ b/tests/qemu-iotests/021
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/022 b/tests/qemu-iotests/022
index b68cd64b33..99eb08f57f 100755
--- a/tests/qemu-iotests/022
+++ b/tests/qemu-iotests/022
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format that supports snapshots
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 TEST_OFFSETS="10485760 4294967296"
 CLUSTER_SIZE="4096"
diff --git a/tests/qemu-iotests/025 b/tests/qemu-iotests/025
index d9a4ebc5e7..e05d833452 100755
--- a/tests/qemu-iotests/025
+++ b/tests/qemu-iotests/025
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt raw qcow2 qed luks
 _supported_proto file sheepdog rbd nfs
-_supported_os Linux
 
 echo "=== Creating image"
 echo
diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index ca89ad7048..e30243608b 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Currently only qcow2 supports rebasing
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 _default_cache_mode "writethrough"
 _supported_cache_modes "writethrough" "none"
 # The refcount table tests expect a certain minimum width for refcount entries
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index b7df9701f7..4cb638022a 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vmdk qcow qcow2 qed
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/029 b/tests/qemu-iotests/029
index 5f42f76cc6..94c2713132 100755
--- a/tests/qemu-iotests/029
+++ b/tests/qemu-iotests/029
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 # Internal snapshots are (currently) impossible with refcount_bits=1
 _unsupported_imgopts 'refcount_bits=1[^0-9]'
 
diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index ef92d8eee3..a3c25ec237 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 CLUSTER_SIZE=65536
 
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index a1757bb15e..23c216c549 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This works for any image format (though unlikely to segfault for raw)
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 echo
 echo === Prepare image ===
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index cfdf1ec2ba..362a48c0a0 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/035 b/tests/qemu-iotests/035
index 46aa835936..ad6fa3115a 100755
--- a/tests/qemu-iotests/035
+++ b/tests/qemu-iotests/035
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 
 size=6G
diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
index 1b56394129..f06ff67408 100755
--- a/tests/qemu-iotests/036
+++ b/tests/qemu-iotests/036
@@ -43,7 +43,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # Only qcow2v3 and later supports feature bits
 IMGOPTS="compat=1.1"
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index 0781bebefe..819a2a52d2 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
diff --git a/tests/qemu-iotests/042 b/tests/qemu-iotests/042
index a9a7fc3041..0e85b356b7 100755
--- a/tests/qemu-iotests/042
+++ b/tests/qemu-iotests/042
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 echo
 echo "== Creating zero size image =="
diff --git a/tests/qemu-iotests/043 b/tests/qemu-iotests/043
index 9894b154ec..67cc7e74c2 100755
--- a/tests/qemu-iotests/043
+++ b/tests/qemu-iotests/043
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046
index 95160bea4c..543355c64f 100755
--- a/tests/qemu-iotests/046
+++ b/tests/qemu-iotests/046
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 CLUSTER_SIZE=64k
 size=128M
diff --git a/tests/qemu-iotests/047 b/tests/qemu-iotests/047
index ce81fc6fa7..4528465fb0 100755
--- a/tests/qemu-iotests/047
+++ b/tests/qemu-iotests/047
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 size=128M
 
diff --git a/tests/qemu-iotests/049 b/tests/qemu-iotests/049
index bc09cd6717..c100d30ed0 100755
--- a/tests/qemu-iotests/049
+++ b/tests/qemu-iotests/049
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 filter_test_dir()
 {
diff --git a/tests/qemu-iotests/050 b/tests/qemu-iotests/050
index dd7b2c72eb..211fc00797 100755
--- a/tests/qemu-iotests/050
+++ b/tests/qemu-iotests/050
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 if test "$IMGFMT" = qcow2 && test $IMGOPTS = ""; then
   IMGOPTS=compat=1.1
diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
index 02ac960da4..a3deb1fcad 100755
--- a/tests/qemu-iotests/051
+++ b/tests/qemu-iotests/051
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # A compat=0.10 image is created in this test which does not support anything
 # other than refcount_bits=16
 _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index b3a2dc1143..6e2ecbfe21 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 # Don't do O_DIRECT on tmpfs
 _supported_cache_modes "writeback" "writethrough" "unsafe"
diff --git a/tests/qemu-iotests/053 b/tests/qemu-iotests/053
index 50c62f0f56..e82bb69881 100755
--- a/tests/qemu-iotests/053
+++ b/tests/qemu-iotests/053
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qcow
 _supported_proto file
-_supported_os Linux
 
 echo
 echo "== Creating single sector image =="
diff --git a/tests/qemu-iotests/054 b/tests/qemu-iotests/054
index 0d5e14f847..a8905b60d0 100755
--- a/tests/qemu-iotests/054
+++ b/tests/qemu-iotests/054
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 echo
 echo "creating too large image (1 EB)"
diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062
index ed7400fed2..d5f818fcce 100755
--- a/tests/qemu-iotests/062
+++ b/tests/qemu-iotests/062
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 IMGOPTS="compat=1.1"
 IMG_SIZE=64M
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index 2d5c0ce9fb..fe4892e467 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow qcow2 vmdk qed raw
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066
index f480986e35..28f8c98412 100755
--- a/tests/qemu-iotests/066
+++ b/tests/qemu-iotests/066
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 # Intentionally create an unaligned image
 IMGOPTS="compat=1.1"
diff --git a/tests/qemu-iotests/067 b/tests/qemu-iotests/067
index fda16a6b0d..926c79b37c 100755
--- a/tests/qemu-iotests/067
+++ b/tests/qemu-iotests/067
@@ -32,7 +32,6 @@ status=1	# failure is the default!
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # Because anything other than 16 would change the output of query-block
 _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
 
diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068
index 881a022107..22f5ca3ba6 100755
--- a/tests/qemu-iotests/068
+++ b/tests/qemu-iotests/068
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qocw2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto generic
-_supported_os Linux
 
 IMGOPTS="compat=1.1"
 IMG_SIZE=128K
diff --git a/tests/qemu-iotests/069 b/tests/qemu-iotests/069
index 6a8e4aa22e..3974714852 100755
--- a/tests/qemu-iotests/069
+++ b/tests/qemu-iotests/069
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 IMG_SIZE=128K
diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index 7f3e5abd57..1cca9233d0 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index 6f9f247fa5..661b36da2d 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
 _supported_proto file
-_supported_os Linux
 
 IMG_SIZE=64M
 
diff --git a/tests/qemu-iotests/073 b/tests/qemu-iotests/073
index 990f90acbd..e684b1b780 100755
--- a/tests/qemu-iotests/073
+++ b/tests/qemu-iotests/073
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 CLUSTER_SIZE=64k
 size=128M
diff --git a/tests/qemu-iotests/079 b/tests/qemu-iotests/079
index 1b6594ebef..81f0c21f53 100755
--- a/tests/qemu-iotests/079
+++ b/tests/qemu-iotests/079
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 echo "=== Check option preallocation and cluster_size ==="
 echo
diff --git a/tests/qemu-iotests/082 b/tests/qemu-iotests/082
index 278511dba4..bbbdf555dc 100755
--- a/tests/qemu-iotests/082
+++ b/tests/qemu-iotests/082
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 run_qemu_img()
 {
diff --git a/tests/qemu-iotests/085 b/tests/qemu-iotests/085
index 68cb665987..d40fdab542 100755
--- a/tests/qemu-iotests/085
+++ b/tests/qemu-iotests/085
@@ -56,7 +56,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 
 # ${1}: unique identifier for the snapshot filename
diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089
index 6609954908..ad029f1f09 100755
--- a/tests/qemu-iotests/089
+++ b/tests/qemu-iotests/089
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # Because anything other than 16 would change the output of qemu_io -c info
 _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
 
diff --git a/tests/qemu-iotests/090 b/tests/qemu-iotests/090
index 193bae7d77..9f8cfbb80f 100755
--- a/tests/qemu-iotests/090
+++ b/tests/qemu-iotests/090
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 IMG_SIZE=128K
 
diff --git a/tests/qemu-iotests/094 b/tests/qemu-iotests/094
index 0bcca77261..9343e09492 100755
--- a/tests/qemu-iotests/094
+++ b/tests/qemu-iotests/094
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto nbd
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 _make_test_img 64M
diff --git a/tests/qemu-iotests/095 b/tests/qemu-iotests/095
index 18505b7181..58fe174b5e 100755
--- a/tests/qemu-iotests/095
+++ b/tests/qemu-iotests/095
@@ -44,7 +44,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size_smaller=5M
 size_larger=100M
diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098
index 461144c831..1c1d1c468f 100755
--- a/tests/qemu-iotests/098
+++ b/tests/qemu-iotests/098
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 IMGOPTS="compat=1.1"
 
diff --git a/tests/qemu-iotests/102 b/tests/qemu-iotests/102
index cedd2b25dc..749ff66b8a 100755
--- a/tests/qemu-iotests/102
+++ b/tests/qemu-iotests/102
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 IMG_SIZE=64K
 
diff --git a/tests/qemu-iotests/103 b/tests/qemu-iotests/103
index 6773e94d9f..554b9de054 100755
--- a/tests/qemu-iotests/103
+++ b/tests/qemu-iotests/103
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 # Internal snapshots are (currently) impossible with refcount_bits=1
 _unsupported_imgopts 'refcount_bits=1[^0-9]'
 
diff --git a/tests/qemu-iotests/104 b/tests/qemu-iotests/104
index 390167bad4..c70f28a9a1 100755
--- a/tests/qemu-iotests/104
+++ b/tests/qemu-iotests/104
@@ -34,7 +34,6 @@ trap "exit \$status" 0 1 2 3 15
 
 _supported_fmt raw qcow qcow2 qed vdi vmdk vhdx
 _supported_proto generic
-_supported_os Linux
 
 echo "=== Check qemu-img info output ==="
 echo
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index 3b5a596844..3346e8cb25 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 vmdk vhdx qed
 _supported_proto generic
-_supported_os Linux
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
                      "subformat=twoGbMaxExtentSparse"
 
diff --git a/tests/qemu-iotests/107 b/tests/qemu-iotests/107
index fcd5a24dfe..268ba27688 100755
--- a/tests/qemu-iotests/107
+++ b/tests/qemu-iotests/107
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file nfs
-_supported_os Linux
 
 
 IMG_SIZE=64K
diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index fad672c1ae..4318341ac5 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 TEST_IMG_REL=$(basename "$TEST_IMG")
diff --git a/tests/qemu-iotests/111 b/tests/qemu-iotests/111
index 57395be64c..490a5bbcb5 100755
--- a/tests/qemu-iotests/111
+++ b/tests/qemu-iotests/111
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 
 $QEMU_IMG create -f $IMGFMT -b "$TEST_IMG.inexistent" "$TEST_IMG" 2>&1 \
diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 6d81c75a9c..706c10b600 100755
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This tests qcow2-specific low-level functionality
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # This test will set refcount_bits on its own which would conflict with the
 # manual setting; compat will be overridden as well
 _unsupported_imgopts refcount_bits 'compat=0.10'
diff --git a/tests/qemu-iotests/114 b/tests/qemu-iotests/114
index f36b88f3f3..f90a744fc0 100755
--- a/tests/qemu-iotests/114
+++ b/tests/qemu-iotests/114
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 
 TEST_IMG="$TEST_IMG.base" _make_test_img 64M
diff --git a/tests/qemu-iotests/115 b/tests/qemu-iotests/115
index 7ed347010f..9ed3cb6a83 100755
--- a/tests/qemu-iotests/115
+++ b/tests/qemu-iotests/115
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 # This test relies on refcounts being 64 bits wide (which does not work with
 # compat=0.10)
 _unsupported_imgopts 'refcount_bits=\([^6]\|.\([^4]\|$\)\)' 'compat=0.10'
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
index 0af0f31c5a..f37b34f8b1 100755
--- a/tests/qemu-iotests/117
+++ b/tests/qemu-iotests/117
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 _make_test_img 64k
 
diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
index ca95b9276e..e9b4fbb009 100755
--- a/tests/qemu-iotests/120
+++ b/tests/qemu-iotests/120
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 _unsupported_fmt luks
 
 _make_test_img 64M
diff --git a/tests/qemu-iotests/125 b/tests/qemu-iotests/125
index 212dcd8f0d..dc4b8f5fb9 100755
--- a/tests/qemu-iotests/125
+++ b/tests/qemu-iotests/125
@@ -44,7 +44,6 @@ get_image_size_on_host()
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 if [ -z "$TEST_IMG_FILE" ]; then
     TEST_IMG_FILE=$TEST_IMG
diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
index 96dc048d59..580fae5d62 100755
--- a/tests/qemu-iotests/126
+++ b/tests/qemu-iotests/126
@@ -37,7 +37,6 @@ _supported_fmt qcow qcow2 qed vmdk
 # colons which separate a protocol prefix from the rest and colons which are
 # just part of the filename, so we cannot test protocols which require a prefix)
 _supported_proto file
-_supported_os Linux
 
 echo
 echo '=== Testing plain files ==='
diff --git a/tests/qemu-iotests/127 b/tests/qemu-iotests/127
index 3e941f74d4..b64926ab31 100755
--- a/tests/qemu-iotests/127
+++ b/tests/qemu-iotests/127
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 IMG_SIZE=64K
 
diff --git a/tests/qemu-iotests/133 b/tests/qemu-iotests/133
index 1f6056d144..6f7cacc091 100755
--- a/tests/qemu-iotests/133
+++ b/tests/qemu-iotests/133
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 TEST_IMG="$TEST_IMG.base" _make_test_img 64M
 _make_test_img -b "$TEST_IMG.base"
diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
index e9e3e84c2a..141a2eaa7e 100755
--- a/tests/qemu-iotests/134
+++ b/tests/qemu-iotests/134
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/142 b/tests/qemu-iotests/142
index d9b98cf60a..6b62271876 100755
--- a/tests/qemu-iotests/142
+++ b/tests/qemu-iotests/142
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # We test all cache modes anyway, but O_DIRECT needs to be supported
 _default_cache_mode none
diff --git a/tests/qemu-iotests/143 b/tests/qemu-iotests/143
index c223867cb3..92249ac8da 100755
--- a/tests/qemu-iotests/143
+++ b/tests/qemu-iotests/143
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 keep_stderr=y \
 _launch_qemu 2> >(_filter_nbd)
diff --git a/tests/qemu-iotests/144 b/tests/qemu-iotests/144
index 15157f33d7..011ed4f2bc 100755
--- a/tests/qemu-iotests/144
+++ b/tests/qemu-iotests/144
@@ -46,7 +46,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=512M
 
diff --git a/tests/qemu-iotests/145 b/tests/qemu-iotests/145
index 28878dc8a1..9427549651 100755
--- a/tests/qemu-iotests/145
+++ b/tests/qemu-iotests/145
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 _make_test_img 1M
 
diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
index 08ad8a6730..c969a1a16f 100755
--- a/tests/qemu-iotests/153
+++ b/tests/qemu-iotests/153
@@ -62,7 +62,6 @@ _check_ofd || _notrun "OFD lock not available"
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 _run_cmd()
 {
diff --git a/tests/qemu-iotests/156 b/tests/qemu-iotests/156
index 8d134029c6..2ffa3ca942 100755
--- a/tests/qemu-iotests/156
+++ b/tests/qemu-iotests/156
@@ -49,7 +49,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2 qed
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 # Create source disk
 TEST_IMG="$TEST_IMG.backing" _make_test_img 1M
diff --git a/tests/qemu-iotests/157 b/tests/qemu-iotests/157
index 69b25cab30..7cbac38099 100755
--- a/tests/qemu-iotests/157
+++ b/tests/qemu-iotests/157
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/158 b/tests/qemu-iotests/158
index 8c0928a7f9..ba4db6116a 100755
--- a/tests/qemu-iotests/158
+++ b/tests/qemu-iotests/158
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2
 _supported_proto generic
 _unsupported_proto vxhs
-_supported_os Linux
 
 
 size=128M
diff --git a/tests/qemu-iotests/159 b/tests/qemu-iotests/159
index 29066eebde..2557140ac2 100755
--- a/tests/qemu-iotests/159
+++ b/tests/qemu-iotests/159
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 _unsupported_fmt luks
 
 TEST_SIZES="5 512 1024 1999 1K 64K 1M"
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index 2e9947fd9a..4e5ed74fd5 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-_supported_os Linux
 
 test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh\( \|$\)')
 [ "$test_ssh" = "" ] && _notrun "ssh support required"
diff --git a/tests/qemu-iotests/170 b/tests/qemu-iotests/170
index 7deb7563c9..05dd6ed6c3 100755
--- a/tests/qemu-iotests/170
+++ b/tests/qemu-iotests/170
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 _unsupported_fmt luks
 
 echo
diff --git a/tests/qemu-iotests/173 b/tests/qemu-iotests/173
index 47036a5564..9e2fa2e73c 100755
--- a/tests/qemu-iotests/173
+++ b/tests/qemu-iotests/173
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto nfs
-_supported_os Linux
 
 size=100M
 
diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182
index ff3d7e7ec1..fd7627fb65 100755
--- a/tests/qemu-iotests/182
+++ b/tests/qemu-iotests/182
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=32M
 
diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183
index 93b7bd798a..fbe5a99beb 100755
--- a/tests/qemu-iotests/183
+++ b/tests/qemu-iotests/183
@@ -44,7 +44,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 raw qed quorum
 _supported_proto file
-_supported_os Linux
 
 size=64M
 _make_test_img $size
diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
index 5dd2177b89..7e7d45babc 100755
--- a/tests/qemu-iotests/186
+++ b/tests/qemu-iotests/186
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/187 b/tests/qemu-iotests/187
index a45addde09..2fcef9e2bd 100755
--- a/tests/qemu-iotests/187
+++ b/tests/qemu-iotests/187
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=64M
 _make_test_img $size
diff --git a/tests/qemu-iotests/190 b/tests/qemu-iotests/190
index e1c1d407f0..eb766ad09f 100755
--- a/tests/qemu-iotests/190
+++ b/tests/qemu-iotests/190
@@ -41,7 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # See 178 for more extensive tests across more formats
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 echo "== Huge file =="
 echo
diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191
index 1ea908ce3d..528022e8d8 100755
--- a/tests/qemu-iotests/191
+++ b/tests/qemu-iotests/191
@@ -43,7 +43,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 size=64M
 
diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192
index 158086f9d2..84d1ed7cfa 100755
--- a/tests/qemu-iotests/192
+++ b/tests/qemu-iotests/192
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/195 b/tests/qemu-iotests/195
index bd1b71ae5e..ef7b9a94e2 100755
--- a/tests/qemu-iotests/195
+++ b/tests/qemu-iotests/195
@@ -39,7 +39,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index 2c664793f4..383d7d7f61 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -51,7 +51,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Test is supported for any backing file; but we force qcow2 for our wrapper.
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
 
diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200
index 1c0f8cafc2..72d431f251 100755
--- a/tests/qemu-iotests/200
+++ b/tests/qemu-iotests/200
@@ -42,7 +42,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 BACKING_IMG="${TEST_DIR}/backing.img"
 TEST_IMG="${TEST_DIR}/test.img"
diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index c1a452ff9a..21ec8a2ad8 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -36,7 +36,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # Repairing the corrupted image requires qemu-img check to store a
 # refcount up to 3, which requires at least two refcount bits.
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index 7b063d7cfa..958c14f5a0 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -48,7 +48,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Test is supported for any backing file; but we force qcow2 for our wrapper.
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
 
diff --git a/tests/qemu-iotests/217 b/tests/qemu-iotests/217
index f5482bb669..58a78a6098 100755
--- a/tests/qemu-iotests/217
+++ b/tests/qemu-iotests/217
@@ -37,7 +37,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This test is specific to qcow2
 _supported_fmt qcow2
 _supported_proto file
-_supported_os Linux
 
 # This test needs clusters with at least a refcount of 2 so that
 # OFLAG_COPIED is not set.  refcount_bits=1 is therefore unsupported.
diff --git a/tests/qemu-iotests/227 b/tests/qemu-iotests/227
index 10cf144eb0..bdd727a721 100755
--- a/tests/qemu-iotests/227
+++ b/tests/qemu-iotests/227
@@ -38,7 +38,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto file
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/231 b/tests/qemu-iotests/231
index 5b2cbab9ac..c0b053ac30 100755
--- a/tests/qemu-iotests/231
+++ b/tests/qemu-iotests/231
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto rbd
-_supported_os Linux
 
 BOGUS_CONF=${TEST_DIR}/ceph-$$.conf
 touch "${BOGUS_CONF}"
diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index b8b6c8cc4c..5855f94b9a 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -47,7 +47,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt raw qcow2
 _supported_proto file
 # If porting to non-Linux, consider using socat instead of ss in common.nbd
-_supported_os Linux
 _require_command QEMU_NBD
 
 nbd_server_set_tcp_port
diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240
index d3e663ed65..b4cf95096d 100755
--- a/tests/qemu-iotests/240
+++ b/tests/qemu-iotests/240
@@ -33,7 +33,6 @@ status=1	# failure is the default!
 
 _supported_fmt generic
 _supported_proto generic
-_supported_os Linux
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests/247
index fc50eb5dc1..546a794d3d 100755
--- a/tests/qemu-iotests/247
+++ b/tests/qemu-iotests/247
@@ -40,7 +40,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Requires backing files and .bdrv_change_backing_file support
 _supported_fmt qcow2 qed
 _supported_proto file
-_supported_os Linux
 
 size=128M
 
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

Currently, all tests are in the "auto" group. This is a little bit pointless.
OTOH, we need a group for the tests that we can automatically run during
"make check" each time, too. Tests in this new group are supposed to run
with every possible QEMU configuration, for example they must run with every
QEMU binary (also non-x86), without failing when an optional features is
missing (but reporting "skip" is ok), and be able to run on all kind of host
filesystems and users (i.e. also as "nobody" or "root").
So let's use the "auto" group for this class of tests now. The initial
list has been determined by running the iotests with non-x86 QEMU targets
and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
macOS and FreeBSD).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/group | 177 +++++++++++++++++++++------------------
 1 file changed, 95 insertions(+), 82 deletions(-)

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 7ac9a5ea4a..a8ca5fd782 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -1,8 +1,21 @@
 #
 # QA groups control file
 # Defines test groups
+#
+# Some notes about the groups:
+#
 # - do not start group names with a digit
 #
+# - quick : Tests in this group should finish within some few seconds.
+#
+# - img : Tests in this group can be used to excercise the qemu-img tool.
+#
+# - auto : Tests in this group are used during "make check" and should be
+#   runnable in any case. That means they should run with every QEMU binary
+#   (also non-x86), with every QEMU configuration (i.e. must not fail if
+#   an optional feature is not compiled in - but reporting a "skip" is ok),
+#   and work all kind of host filesystems and users (e.g. "nobody" or "root").
+#
 
 #
 # test-group association ... one line per test
@@ -32,11 +45,11 @@
 023 rw auto
 024 rw backing auto quick
 025 rw auto quick
-026 rw blkdbg auto
+026 rw blkdbg
 027 rw auto quick
-028 rw backing auto quick
+028 rw backing quick
 029 rw auto quick
-030 rw auto backing
+030 rw backing
 031 rw auto quick
 032 rw auto quick
 033 rw auto quick
@@ -46,35 +59,35 @@
 037 rw auto backing quick
 038 rw auto backing quick
 039 rw auto quick
-040 rw auto
-041 rw auto backing
+040 rw
+041 rw backing
 042 rw auto quick
 043 rw auto backing
-044 rw auto
-045 rw auto quick
+044 rw
+045 rw quick
 046 rw auto aio quick
 047 rw auto quick
 048 img auto quick
 049 rw auto
 050 rw auto backing quick
-051 rw auto
+051 rw
 052 rw auto backing quick
 053 rw auto quick
 054 rw auto quick
-055 rw auto
-056 rw auto backing
-057 rw auto
-058 rw auto quick
+055 rw
+056 rw backing
+057 rw
+058 rw quick
 059 rw auto quick
 060 rw auto quick
 061 rw auto
 062 rw auto quick
 063 rw auto quick
 064 rw auto quick
-065 rw auto quick
+065 rw quick
 066 rw auto quick
-067 rw auto quick
-068 rw auto quick
+067 rw quick
+068 rw quick
 069 rw auto quick
 070 rw auto quick
 071 rw auto quick
@@ -91,18 +104,18 @@
 082 rw auto quick
 083 rw auto
 084 img auto quick
-085 rw auto
+085 rw
 086 rw auto quick
-087 rw auto quick
+087 rw quick
 088 rw auto quick
 089 rw auto quick
 090 rw auto quick
 091 rw auto migration
 092 rw auto quick
-093 auto
+093 throttle
 094 rw auto quick
-095 rw auto quick
-096 rw auto quick
+095 rw quick
+096 rw quick
 097 rw auto backing
 098 rw auto backing quick
 099 rw auto quick
@@ -118,60 +131,60 @@
 109 rw auto
 110 rw auto backing quick
 111 rw auto quick
-112 rw auto
+112 rw
 113 rw auto quick
 114 rw auto quick
-115 rw auto
+115 rw
 116 rw auto quick
 117 rw auto
-118 rw auto
+118 rw
 119 rw auto quick
 120 rw auto quick
-121 rw auto
+121 rw
 122 rw auto
 123 rw auto quick
-124 rw auto backing
-125 rw auto
+124 rw backing
+125 rw
 126 rw auto backing
-127 rw auto backing quick
+127 rw backing quick
 128 rw auto quick
-129 rw auto quick
+129 rw quick
 130 rw auto quick
 131 rw auto quick
-132 rw auto quick
+132 rw quick
 133 auto quick
 134 rw auto quick
 135 rw auto
-136 rw auto
+136 rw
 137 rw auto
 138 rw auto quick
-139 rw auto quick
+139 rw quick
 140 rw auto quick
 141 rw auto quick
 142 auto
 143 auto quick
-144 rw auto quick
-145 auto quick
+144 rw quick
+145 quick
 146 auto quick
-147 auto
-148 rw auto quick
-149 rw auto sudo
+147 img
+148 rw quick
+149 rw sudo
 150 rw auto quick
-151 rw auto
-152 rw auto quick
-153 rw auto quick
+151 rw
+152 rw quick
+153 rw quick
 154 rw auto backing quick
-155 rw auto
+155 rw
 156 rw auto quick
-157 auto
+157 quick
 158 rw auto quick
 159 rw auto quick
 160 rw auto quick
 161 rw auto quick
-162 auto quick
-163 rw auto
-165 rw auto quick
-169 rw auto quick migration
+162 quick
+163 rw
+165 rw quick
+169 rw quick migration
 170 rw auto quick
 171 rw auto quick
 172 auto
@@ -180,72 +193,72 @@
 175 auto quick
 176 rw auto backing
 177 rw auto quick
-178 auto
+178 img
 179 rw auto quick
 181 rw auto migration
-182 rw auto quick
-183 rw auto migration
+182 rw quick
+183 rw migration
 184 rw auto quick
-185 rw auto
+185 rw
 186 rw auto
 187 rw auto
-188 rw auto quick
-189 rw auto
+188 rw quick
+189 rw
 190 rw auto quick
 191 rw auto
 192 rw auto quick
-194 rw auto migration quick
+194 rw migration quick
 195 rw auto quick
-196 rw auto quick migration
+196 rw quick migration
 197 rw auto quick
-198 rw auto
-199 rw auto migration
-200 rw auto
+198 rw
+199 rw migration
+200 rw
 201 rw auto migration
-202 rw auto quick
-203 rw auto migration
-204 rw auto quick
-205 rw auto quick
-206 rw auto
+202 rw quick
+203 rw migration
+204 rw quick
+205 rw quick
+206 rw
 207 rw auto
-208 rw auto quick
-209 rw auto quick
+208 rw quick
+209 rw quick
 210 rw auto
 211 rw auto quick
 212 rw auto quick
 213 rw auto quick
 214 rw auto
 215 rw auto quick
-216 rw auto quick
+216 rw quick
 217 rw auto quick
-218 rw auto quick
-219 rw auto
+218 rw quick
+219 rw
 220 rw auto
 221 rw auto quick
-222 rw auto quick
-223 rw auto quick
-224 rw auto quick
+222 rw quick
+223 rw quick
+224 rw quick
 225 rw auto quick
 226 auto quick
-227 auto quick
-228 rw auto quick
+227 quick
+228 rw quick
 229 auto quick
 231 auto quick
-232 auto quick
+232 quick
 233 auto quick
-234 auto quick migration
-235 auto quick
-236 auto quick
+234 quick migration
+235 quick
+236 quick
 237 rw auto quick
-238 auto quick
+238 quick
 239 rw auto quick
-240 auto quick
+240 quick
 241 rw auto quick
-242 rw auto quick
+242 rw quick
 243 rw auto quick
 244 rw auto quick
-245 rw auto
-246 rw auto quick
-247 rw auto quick
-248 rw auto quick
+245 rw
+246 rw quick
+247 rw quick
+248 rw quick
 249 rw auto quick
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

Currently, all tests are in the "auto" group. This is a little bit pointless.
OTOH, we need a group for the tests that we can automatically run during
"make check" each time, too. Tests in this new group are supposed to run
with every possible QEMU configuration, for example they must run with every
QEMU binary (also non-x86), without failing when an optional features is
missing (but reporting "skip" is ok), and be able to run on all kind of host
filesystems and users (i.e. also as "nobody" or "root").
So let's use the "auto" group for this class of tests now. The initial
list has been determined by running the iotests with non-x86 QEMU targets
and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
macOS and FreeBSD).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/group | 177 +++++++++++++++++++++------------------
 1 file changed, 95 insertions(+), 82 deletions(-)

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 7ac9a5ea4a..a8ca5fd782 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -1,8 +1,21 @@
 #
 # QA groups control file
 # Defines test groups
+#
+# Some notes about the groups:
+#
 # - do not start group names with a digit
 #
+# - quick : Tests in this group should finish within some few seconds.
+#
+# - img : Tests in this group can be used to excercise the qemu-img tool.
+#
+# - auto : Tests in this group are used during "make check" and should be
+#   runnable in any case. That means they should run with every QEMU binary
+#   (also non-x86), with every QEMU configuration (i.e. must not fail if
+#   an optional feature is not compiled in - but reporting a "skip" is ok),
+#   and work all kind of host filesystems and users (e.g. "nobody" or "root").
+#
 
 #
 # test-group association ... one line per test
@@ -32,11 +45,11 @@
 023 rw auto
 024 rw backing auto quick
 025 rw auto quick
-026 rw blkdbg auto
+026 rw blkdbg
 027 rw auto quick
-028 rw backing auto quick
+028 rw backing quick
 029 rw auto quick
-030 rw auto backing
+030 rw backing
 031 rw auto quick
 032 rw auto quick
 033 rw auto quick
@@ -46,35 +59,35 @@
 037 rw auto backing quick
 038 rw auto backing quick
 039 rw auto quick
-040 rw auto
-041 rw auto backing
+040 rw
+041 rw backing
 042 rw auto quick
 043 rw auto backing
-044 rw auto
-045 rw auto quick
+044 rw
+045 rw quick
 046 rw auto aio quick
 047 rw auto quick
 048 img auto quick
 049 rw auto
 050 rw auto backing quick
-051 rw auto
+051 rw
 052 rw auto backing quick
 053 rw auto quick
 054 rw auto quick
-055 rw auto
-056 rw auto backing
-057 rw auto
-058 rw auto quick
+055 rw
+056 rw backing
+057 rw
+058 rw quick
 059 rw auto quick
 060 rw auto quick
 061 rw auto
 062 rw auto quick
 063 rw auto quick
 064 rw auto quick
-065 rw auto quick
+065 rw quick
 066 rw auto quick
-067 rw auto quick
-068 rw auto quick
+067 rw quick
+068 rw quick
 069 rw auto quick
 070 rw auto quick
 071 rw auto quick
@@ -91,18 +104,18 @@
 082 rw auto quick
 083 rw auto
 084 img auto quick
-085 rw auto
+085 rw
 086 rw auto quick
-087 rw auto quick
+087 rw quick
 088 rw auto quick
 089 rw auto quick
 090 rw auto quick
 091 rw auto migration
 092 rw auto quick
-093 auto
+093 throttle
 094 rw auto quick
-095 rw auto quick
-096 rw auto quick
+095 rw quick
+096 rw quick
 097 rw auto backing
 098 rw auto backing quick
 099 rw auto quick
@@ -118,60 +131,60 @@
 109 rw auto
 110 rw auto backing quick
 111 rw auto quick
-112 rw auto
+112 rw
 113 rw auto quick
 114 rw auto quick
-115 rw auto
+115 rw
 116 rw auto quick
 117 rw auto
-118 rw auto
+118 rw
 119 rw auto quick
 120 rw auto quick
-121 rw auto
+121 rw
 122 rw auto
 123 rw auto quick
-124 rw auto backing
-125 rw auto
+124 rw backing
+125 rw
 126 rw auto backing
-127 rw auto backing quick
+127 rw backing quick
 128 rw auto quick
-129 rw auto quick
+129 rw quick
 130 rw auto quick
 131 rw auto quick
-132 rw auto quick
+132 rw quick
 133 auto quick
 134 rw auto quick
 135 rw auto
-136 rw auto
+136 rw
 137 rw auto
 138 rw auto quick
-139 rw auto quick
+139 rw quick
 140 rw auto quick
 141 rw auto quick
 142 auto
 143 auto quick
-144 rw auto quick
-145 auto quick
+144 rw quick
+145 quick
 146 auto quick
-147 auto
-148 rw auto quick
-149 rw auto sudo
+147 img
+148 rw quick
+149 rw sudo
 150 rw auto quick
-151 rw auto
-152 rw auto quick
-153 rw auto quick
+151 rw
+152 rw quick
+153 rw quick
 154 rw auto backing quick
-155 rw auto
+155 rw
 156 rw auto quick
-157 auto
+157 quick
 158 rw auto quick
 159 rw auto quick
 160 rw auto quick
 161 rw auto quick
-162 auto quick
-163 rw auto
-165 rw auto quick
-169 rw auto quick migration
+162 quick
+163 rw
+165 rw quick
+169 rw quick migration
 170 rw auto quick
 171 rw auto quick
 172 auto
@@ -180,72 +193,72 @@
 175 auto quick
 176 rw auto backing
 177 rw auto quick
-178 auto
+178 img
 179 rw auto quick
 181 rw auto migration
-182 rw auto quick
-183 rw auto migration
+182 rw quick
+183 rw migration
 184 rw auto quick
-185 rw auto
+185 rw
 186 rw auto
 187 rw auto
-188 rw auto quick
-189 rw auto
+188 rw quick
+189 rw
 190 rw auto quick
 191 rw auto
 192 rw auto quick
-194 rw auto migration quick
+194 rw migration quick
 195 rw auto quick
-196 rw auto quick migration
+196 rw quick migration
 197 rw auto quick
-198 rw auto
-199 rw auto migration
-200 rw auto
+198 rw
+199 rw migration
+200 rw
 201 rw auto migration
-202 rw auto quick
-203 rw auto migration
-204 rw auto quick
-205 rw auto quick
-206 rw auto
+202 rw quick
+203 rw migration
+204 rw quick
+205 rw quick
+206 rw
 207 rw auto
-208 rw auto quick
-209 rw auto quick
+208 rw quick
+209 rw quick
 210 rw auto
 211 rw auto quick
 212 rw auto quick
 213 rw auto quick
 214 rw auto
 215 rw auto quick
-216 rw auto quick
+216 rw quick
 217 rw auto quick
-218 rw auto quick
-219 rw auto
+218 rw quick
+219 rw
 220 rw auto
 221 rw auto quick
-222 rw auto quick
-223 rw auto quick
-224 rw auto quick
+222 rw quick
+223 rw quick
+224 rw quick
 225 rw auto quick
 226 auto quick
-227 auto quick
-228 rw auto quick
+227 quick
+228 rw quick
 229 auto quick
 231 auto quick
-232 auto quick
+232 quick
 233 auto quick
-234 auto quick migration
-235 auto quick
-236 auto quick
+234 quick migration
+235 quick
+236 quick
 237 rw auto quick
-238 auto quick
+238 quick
 239 rw auto quick
-240 auto quick
+240 quick
 241 rw auto quick
-242 rw auto quick
+242 rw quick
 243 rw auto quick
 244 rw auto quick
-245 rw auto
-246 rw auto quick
-247 rw auto quick
-248 rw auto quick
+245 rw
+246 rw quick
+247 rw quick
+248 rw quick
 249 rw auto quick
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste, Kevin Wolf,
	Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster,
	Alex Bennée

People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet. Now that we've got
a proper "auto" group of iotests that should be fine to run in every
environment, we can enable the iotests during "make check" again by
running the "auto" tests by default from the check-block.sh script.

Some cases still need to be checked first, though: iotests need bash
and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
the output of some test cases causing them to fail. So if we detect
that one of the required programs is missing or that gprof is enabled,
we still have to skip the iotests to avoid failures.

And finally, since we are using check-block.sh now again, this patch also
removes the qemu-iotests-quick.sh script since we do not need that anymore
(and having two shell wrapper scripts around the block tests seem
rather confusing than helpful).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include      |  8 +++----
 tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
 tests/qemu-iotests-quick.sh |  8 -------
 3 files changed, 38 insertions(+), 22 deletions(-)
 delete mode 100755 tests/qemu-iotests-quick.sh

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e2432d5e77..3bb7793d4a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
 check-unit-y += tests/ptimer-test$(EXESUF)
 check-unit-y += tests/test-qapi-util$(EXESUF)
 
-check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
+check-block-$(CONFIG_POSIX) += tests/check-block.sh
 
 # All QTests for now are POSIX-only, but the dependencies are
 # really in libqtest, not in the testcases themselves.
@@ -1096,8 +1096,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
 
 QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
 
-.PHONY: check-tests/qemu-iotests-quick.sh
-check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
+.PHONY: check-tests/check-block.sh
+check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
 	$<
 
 .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
@@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-block: $(patsubst %,check-%, $(check-block-y))
-check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
+check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
 check-clean:
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
diff --git a/tests/check-block.sh b/tests/check-block.sh
index f3d12fd602..3b971d6cf4 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -1,24 +1,48 @@
 #!/bin/sh
 
-FORMAT_LIST="raw qcow2 qed vmdk vpc"
+# Honor the SPEED environment variable, just like we do it for the qtests.
+if [ "$SPEED" = "slow" ]; then
+    format_list="raw qcow2"
+    group=
+elif [ "$SPEED" = "thorough" ]; then
+    format_list="raw qcow2 qed vmdk vpc"
+    group=
+else
+    format_list=qcow2
+    group="-g auto"
+fi
+
 if [ "$#" -ne 0 ]; then
-    FORMAT_LIST="$@"
+    format_list="$@"
+fi
+
+if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
+    echo "GPROF is enabled ==> Not running the qemu-iotests."
+    exit 0
 fi
 
-export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
-export QEMU_IMG_PROG="$PWD/qemu-img"
-export QEMU_IO_PROG="$PWD/qemu-io"
+if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+    echo "No qemu-system binary available ==> Not running the qemu-iotests."
+    exit 0
+fi
+
+if ! command -v bash >/dev/null 2>&1 ; then
+    echo "bash not available ==> Not running the qemu-iotests."
+    exit 0
+fi
 
-if [ ! -x $QEMU_PROG ]; then
-    echo "'make check-block' requires qemu-system-x86_64"
-    exit 1
+if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
+    if ! command -v gsed >/dev/null 2>&1; then
+        echo "GNU sed not available ==> Not running the qemu-iotests."
+        exit 0
+    fi
 fi
 
 cd tests/qemu-iotests
 
 ret=0
-for FMT in $FORMAT_LIST ; do
-    ./check -T -nocache -$FMT || ret=1
+for fmt in $format_list ; do
+    ./check -$fmt $group || ret=1
 done
 
 exit $ret
diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
deleted file mode 100755
index 0e554bb972..0000000000
--- a/tests/qemu-iotests-quick.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/sh
-
-cd tests/qemu-iotests
-
-ret=0
-TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
-
-exit $ret
-- 
2.21.0

^ permalink raw reply related	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
@ 2019-05-02  8:45   ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-02  8:45 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Max Reitz, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet. Now that we've got
a proper "auto" group of iotests that should be fine to run in every
environment, we can enable the iotests during "make check" again by
running the "auto" tests by default from the check-block.sh script.

Some cases still need to be checked first, though: iotests need bash
and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
the output of some test cases causing them to fail. So if we detect
that one of the required programs is missing or that gprof is enabled,
we still have to skip the iotests to avoid failures.

And finally, since we are using check-block.sh now again, this patch also
removes the qemu-iotests-quick.sh script since we do not need that anymore
(and having two shell wrapper scripts around the block tests seem
rather confusing than helpful).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include      |  8 +++----
 tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
 tests/qemu-iotests-quick.sh |  8 -------
 3 files changed, 38 insertions(+), 22 deletions(-)
 delete mode 100755 tests/qemu-iotests-quick.sh

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e2432d5e77..3bb7793d4a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
 check-unit-y += tests/ptimer-test$(EXESUF)
 check-unit-y += tests/test-qapi-util$(EXESUF)
 
-check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
+check-block-$(CONFIG_POSIX) += tests/check-block.sh
 
 # All QTests for now are POSIX-only, but the dependencies are
 # really in libqtest, not in the testcases themselves.
@@ -1096,8 +1096,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
 
 QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
 
-.PHONY: check-tests/qemu-iotests-quick.sh
-check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
+.PHONY: check-tests/check-block.sh
+check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
 	$<
 
 .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
@@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-block: $(patsubst %,check-%, $(check-block-y))
-check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
+check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
 check-clean:
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
diff --git a/tests/check-block.sh b/tests/check-block.sh
index f3d12fd602..3b971d6cf4 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -1,24 +1,48 @@
 #!/bin/sh
 
-FORMAT_LIST="raw qcow2 qed vmdk vpc"
+# Honor the SPEED environment variable, just like we do it for the qtests.
+if [ "$SPEED" = "slow" ]; then
+    format_list="raw qcow2"
+    group=
+elif [ "$SPEED" = "thorough" ]; then
+    format_list="raw qcow2 qed vmdk vpc"
+    group=
+else
+    format_list=qcow2
+    group="-g auto"
+fi
+
 if [ "$#" -ne 0 ]; then
-    FORMAT_LIST="$@"
+    format_list="$@"
+fi
+
+if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
+    echo "GPROF is enabled ==> Not running the qemu-iotests."
+    exit 0
 fi
 
-export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
-export QEMU_IMG_PROG="$PWD/qemu-img"
-export QEMU_IO_PROG="$PWD/qemu-io"
+if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+    echo "No qemu-system binary available ==> Not running the qemu-iotests."
+    exit 0
+fi
+
+if ! command -v bash >/dev/null 2>&1 ; then
+    echo "bash not available ==> Not running the qemu-iotests."
+    exit 0
+fi
 
-if [ ! -x $QEMU_PROG ]; then
-    echo "'make check-block' requires qemu-system-x86_64"
-    exit 1
+if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
+    if ! command -v gsed >/dev/null 2>&1; then
+        echo "GNU sed not available ==> Not running the qemu-iotests."
+        exit 0
+    fi
 fi
 
 cd tests/qemu-iotests
 
 ret=0
-for FMT in $FORMAT_LIST ; do
-    ./check -T -nocache -$FMT || ret=1
+for fmt in $format_list ; do
+    ./check -$fmt $group || ret=1
 done
 
 exit $ret
diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
deleted file mode 100755
index 0e554bb972..0000000000
--- a/tests/qemu-iotests-quick.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/sh
-
-cd tests/qemu-iotests
-
-ret=0
-TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
-
-exit $ret
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 1/7] tests/qemu-iotests/005: Add a sanity check for large sparse file support
@ 2019-05-02 17:37     ` Alex Bennée
  0 siblings, 0 replies; 41+ messages in thread
From: Alex Bennée @ 2019-05-02 17:37 UTC (permalink / raw)
  To: Thomas Huth
  Cc: qemu-devel, Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste,
	Kevin Wolf, Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster


Thomas Huth <thuth@redhat.com> writes:

> "check -raw 005" fails when running on certain filesystems - these do not
> support such large sparse files. Use the same check as in test 220 to
> skip the test in this case.
>
> Suggested-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/qemu-iotests/005 | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
> index 2fef63af88..9c7681c19b 100755
> --- a/tests/qemu-iotests/005
> +++ b/tests/qemu-iotests/005
> @@ -55,6 +55,15 @@ if [ "$IMGPROTO" = "sheepdog" ]; then
>      _notrun "image protocol $IMGPROTO does not support large image sizes"
>  fi
>
> +# Sanity check: For raw, we require a file system that permits the creation
> +# of a HUGE (but very sparse) file. Check we can create it before continuing.
> +if [ "$IMGFMT" = "raw" ]; then
> +    if ! truncate --size=5T "$TEST_IMG"; then
> +        _notrun "file system on $TEST_DIR does not support large enough files"
> +    fi
> +    rm "$TEST_IMG"
> +fi
> +
>  echo
>  echo "creating large image"
>  _make_test_img 5000G


--
Alex Bennée

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 1/7] tests/qemu-iotests/005: Add a sanity check for large sparse file support
@ 2019-05-02 17:37     ` Alex Bennée
  0 siblings, 0 replies; 41+ messages in thread
From: Alex Bennée @ 2019-05-02 17:37 UTC (permalink / raw)
  To: Thomas Huth
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, qemu-devel,
	Christophe Fergeau, Max Reitz, Wainer dos Santos Moschetta,
	Philippe Mathieu-Daudé,
	Li-Wen Hsu, Markus Armbruster


Thomas Huth <thuth@redhat.com> writes:

> "check -raw 005" fails when running on certain filesystems - these do not
> support such large sparse files. Use the same check as in test 220 to
> skip the test in this case.
>
> Suggested-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/qemu-iotests/005 | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
> index 2fef63af88..9c7681c19b 100755
> --- a/tests/qemu-iotests/005
> +++ b/tests/qemu-iotests/005
> @@ -55,6 +55,15 @@ if [ "$IMGPROTO" = "sheepdog" ]; then
>      _notrun "image protocol $IMGPROTO does not support large image sizes"
>  fi
>
> +# Sanity check: For raw, we require a file system that permits the creation
> +# of a HUGE (but very sparse) file. Check we can create it before continuing.
> +if [ "$IMGFMT" = "raw" ]; then
> +    if ! truncate --size=5T "$TEST_IMG"; then
> +        _notrun "file system on $TEST_DIR does not support large enough files"
> +    fi
> +    rm "$TEST_IMG"
> +fi
> +
>  echo
>  echo "creating large image"
>  _make_test_img 5000G


--
Alex Bennée


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
@ 2019-05-03  9:53     ` Alex Bennée
  0 siblings, 0 replies; 41+ messages in thread
From: Alex Bennée @ 2019-05-03  9:53 UTC (permalink / raw)
  To: Thomas Huth
  Cc: qemu-devel, Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste,
	Kevin Wolf, Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster


Thomas Huth <thuth@redhat.com> writes:

> People often forget to run the iotests before submitting patches or
> pull requests - this is likely due to the fact that we do not run the
> tests during our mandatory "make check" tests yet. Now that we've got
> a proper "auto" group of iotests that should be fine to run in every
> environment, we can enable the iotests during "make check" again by
> running the "auto" tests by default from the check-block.sh script.
>
> Some cases still need to be checked first, though: iotests need bash
> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
> the output of some test cases causing them to fail. So if we detect
> that one of the required programs is missing or that gprof is enabled,
> we still have to skip the iotests to avoid failures.
>
> And finally, since we are using check-block.sh now again, this patch also
> removes the qemu-iotests-quick.sh script since we do not need that anymore
> (and having two shell wrapper scripts around the block tests seem
> rather confusing than helpful).
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/Makefile.include      |  8 +++----
>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>  tests/qemu-iotests-quick.sh |  8 -------
>  3 files changed, 38 insertions(+), 22 deletions(-)
>  delete mode 100755 tests/qemu-iotests-quick.sh
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index e2432d5e77..3bb7793d4a 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
>  check-unit-y += tests/ptimer-test$(EXESUF)
>  check-unit-y += tests/test-qapi-util$(EXESUF)
>
> -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
> +check-block-$(CONFIG_POSIX) += tests/check-block.sh
>
>  # All QTests for now are POSIX-only, but the dependencies are
>  # really in libqtest, not in the testcases themselves.
> @@ -1096,8 +1096,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>
>  QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
>
> -.PHONY: check-tests/qemu-iotests-quick.sh
> -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
> +.PHONY: check-tests/check-block.sh
> +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
>  	$<
>
>  .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
> @@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  check-block: $(patsubst %,check-%, $(check-block-y))
> -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
> +check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
>  check-clean:
>  	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>  	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> diff --git a/tests/check-block.sh b/tests/check-block.sh
> index f3d12fd602..3b971d6cf4 100755
> --- a/tests/check-block.sh
> +++ b/tests/check-block.sh
> @@ -1,24 +1,48 @@
>  #!/bin/sh
>
> -FORMAT_LIST="raw qcow2 qed vmdk vpc"
> +# Honor the SPEED environment variable, just like we do it for the qtests.
> +if [ "$SPEED" = "slow" ]; then
> +    format_list="raw qcow2"
> +    group=
> +elif [ "$SPEED" = "thorough" ]; then
> +    format_list="raw qcow2 qed vmdk vpc"
> +    group=
> +else
> +    format_list=qcow2
> +    group="-g auto"
> +fi
> +
>  if [ "$#" -ne 0 ]; then
> -    FORMAT_LIST="$@"
> +    format_list="$@"
> +fi
> +
> +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
> +    echo "GPROF is enabled ==> Not running the qemu-iotests."
> +    exit 0
>  fi
>
> -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
> -export QEMU_IMG_PROG="$PWD/qemu-img"
> -export QEMU_IO_PROG="$PWD/qemu-io"
> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> +    echo "No qemu-system binary available ==> Not running the qemu-iotests."
> +    exit 0
> +fi
> +
> +if ! command -v bash >/dev/null 2>&1 ; then
> +    echo "bash not available ==> Not running the qemu-iotests."
> +    exit 0
> +fi
>
> -if [ ! -x $QEMU_PROG ]; then
> -    echo "'make check-block' requires qemu-system-x86_64"
> -    exit 1
> +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
> +    if ! command -v gsed >/dev/null 2>&1; then
> +        echo "GNU sed not available ==> Not running the qemu-iotests."
> +        exit 0
> +    fi
>  fi
>
>  cd tests/qemu-iotests
>
>  ret=0
> -for FMT in $FORMAT_LIST ; do
> -    ./check -T -nocache -$FMT || ret=1
> +for fmt in $format_list ; do
> +    ./check -$fmt $group || ret=1

This is all looking good and my only remaining objection is aesthetic.
If you run "make check -jN" you end up with the with the block test
output intermingled with the rest of the output which is now fairly
uniform.

Any chance we could hide the verbosity unless requested and have
something like:

    TEST    check-block: test xxx

emitted for each passing test? It's OK to show more in the failing case
to help with diagnosis but the passing tests should be easy on the eye
when scanning through the logs.

>  done
>
>  exit $ret
> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
> deleted file mode 100755
> index 0e554bb972..0000000000
> --- a/tests/qemu-iotests-quick.sh
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -#!/bin/sh
> -
> -cd tests/qemu-iotests
> -
> -ret=0
> -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
> -
> -exit $ret


--
Alex Bennée

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
@ 2019-05-03  9:53     ` Alex Bennée
  0 siblings, 0 replies; 41+ messages in thread
From: Alex Bennée @ 2019-05-03  9:53 UTC (permalink / raw)
  To: Thomas Huth
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, qemu-devel,
	Christophe Fergeau, Max Reitz, Wainer dos Santos Moschetta,
	Philippe Mathieu-Daudé,
	Li-Wen Hsu, Markus Armbruster


Thomas Huth <thuth@redhat.com> writes:

> People often forget to run the iotests before submitting patches or
> pull requests - this is likely due to the fact that we do not run the
> tests during our mandatory "make check" tests yet. Now that we've got
> a proper "auto" group of iotests that should be fine to run in every
> environment, we can enable the iotests during "make check" again by
> running the "auto" tests by default from the check-block.sh script.
>
> Some cases still need to be checked first, though: iotests need bash
> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
> the output of some test cases causing them to fail. So if we detect
> that one of the required programs is missing or that gprof is enabled,
> we still have to skip the iotests to avoid failures.
>
> And finally, since we are using check-block.sh now again, this patch also
> removes the qemu-iotests-quick.sh script since we do not need that anymore
> (and having two shell wrapper scripts around the block tests seem
> rather confusing than helpful).
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/Makefile.include      |  8 +++----
>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>  tests/qemu-iotests-quick.sh |  8 -------
>  3 files changed, 38 insertions(+), 22 deletions(-)
>  delete mode 100755 tests/qemu-iotests-quick.sh
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index e2432d5e77..3bb7793d4a 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
>  check-unit-y += tests/ptimer-test$(EXESUF)
>  check-unit-y += tests/test-qapi-util$(EXESUF)
>
> -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
> +check-block-$(CONFIG_POSIX) += tests/check-block.sh
>
>  # All QTests for now are POSIX-only, but the dependencies are
>  # really in libqtest, not in the testcases themselves.
> @@ -1096,8 +1096,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>
>  QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
>
> -.PHONY: check-tests/qemu-iotests-quick.sh
> -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
> +.PHONY: check-tests/check-block.sh
> +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
>  	$<
>
>  .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
> @@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  check-block: $(patsubst %,check-%, $(check-block-y))
> -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
> +check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
>  check-clean:
>  	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>  	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> diff --git a/tests/check-block.sh b/tests/check-block.sh
> index f3d12fd602..3b971d6cf4 100755
> --- a/tests/check-block.sh
> +++ b/tests/check-block.sh
> @@ -1,24 +1,48 @@
>  #!/bin/sh
>
> -FORMAT_LIST="raw qcow2 qed vmdk vpc"
> +# Honor the SPEED environment variable, just like we do it for the qtests.
> +if [ "$SPEED" = "slow" ]; then
> +    format_list="raw qcow2"
> +    group=
> +elif [ "$SPEED" = "thorough" ]; then
> +    format_list="raw qcow2 qed vmdk vpc"
> +    group=
> +else
> +    format_list=qcow2
> +    group="-g auto"
> +fi
> +
>  if [ "$#" -ne 0 ]; then
> -    FORMAT_LIST="$@"
> +    format_list="$@"
> +fi
> +
> +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
> +    echo "GPROF is enabled ==> Not running the qemu-iotests."
> +    exit 0
>  fi
>
> -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
> -export QEMU_IMG_PROG="$PWD/qemu-img"
> -export QEMU_IO_PROG="$PWD/qemu-io"
> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> +    echo "No qemu-system binary available ==> Not running the qemu-iotests."
> +    exit 0
> +fi
> +
> +if ! command -v bash >/dev/null 2>&1 ; then
> +    echo "bash not available ==> Not running the qemu-iotests."
> +    exit 0
> +fi
>
> -if [ ! -x $QEMU_PROG ]; then
> -    echo "'make check-block' requires qemu-system-x86_64"
> -    exit 1
> +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
> +    if ! command -v gsed >/dev/null 2>&1; then
> +        echo "GNU sed not available ==> Not running the qemu-iotests."
> +        exit 0
> +    fi
>  fi
>
>  cd tests/qemu-iotests
>
>  ret=0
> -for FMT in $FORMAT_LIST ; do
> -    ./check -T -nocache -$FMT || ret=1
> +for fmt in $format_list ; do
> +    ./check -$fmt $group || ret=1

This is all looking good and my only remaining objection is aesthetic.
If you run "make check -jN" you end up with the with the block test
output intermingled with the rest of the output which is now fairly
uniform.

Any chance we could hide the verbosity unless requested and have
something like:

    TEST    check-block: test xxx

emitted for each passing test? It's OK to show more in the failing case
to help with diagnosis but the passing tests should be easy on the eye
when scanning through the logs.

>  done
>
>  exit $ret
> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
> deleted file mode 100755
> index 0e554bb972..0000000000
> --- a/tests/qemu-iotests-quick.sh
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -#!/bin/sh
> -
> -cd tests/qemu-iotests
> -
> -ret=0
> -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
> -
> -exit $ret


--
Alex Bennée


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
@ 2019-05-03 10:03       ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-03 10:03 UTC (permalink / raw)
  To: Alex Bennée
  Cc: qemu-devel, Fam Zheng, qemu-block, Christophe Fergeau, Ed Maste,
	Kevin Wolf, Max Reitz, Li-Wen Hsu, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Eric Blake, Markus Armbruster

On 03/05/2019 11.53, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> People often forget to run the iotests before submitting patches or
>> pull requests - this is likely due to the fact that we do not run the
>> tests during our mandatory "make check" tests yet. Now that we've got
>> a proper "auto" group of iotests that should be fine to run in every
>> environment, we can enable the iotests during "make check" again by
>> running the "auto" tests by default from the check-block.sh script.
>>
>> Some cases still need to be checked first, though: iotests need bash
>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>> the output of some test cases causing them to fail. So if we detect
>> that one of the required programs is missing or that gprof is enabled,
>> we still have to skip the iotests to avoid failures.
>>
>> And finally, since we are using check-block.sh now again, this patch also
>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>> (and having two shell wrapper scripts around the block tests seem
>> rather confusing than helpful).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/Makefile.include      |  8 +++----
>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>  tests/qemu-iotests-quick.sh |  8 -------
>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>
>> diff --git a/tests/Makefile.include b/tests/Makefile.include
>> index e2432d5e77..3bb7793d4a 100644
>> --- a/tests/Makefile.include
>> +++ b/tests/Makefile.include
>> @@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
>>  check-unit-y += tests/ptimer-test$(EXESUF)
>>  check-unit-y += tests/test-qapi-util$(EXESUF)
>>
>> -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
>> +check-block-$(CONFIG_POSIX) += tests/check-block.sh
>>
>>  # All QTests for now are POSIX-only, but the dependencies are
>>  # really in libqtest, not in the testcases themselves.
>> @@ -1096,8 +1096,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>>
>>  QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
>>
>> -.PHONY: check-tests/qemu-iotests-quick.sh
>> -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
>> +.PHONY: check-tests/check-block.sh
>> +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
>>  	$<
>>
>>  .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
>> @@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>>  check-block: $(patsubst %,check-%, $(check-block-y))
>> -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
>> +check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
>>  check-clean:
>>  	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>>  	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
>> diff --git a/tests/check-block.sh b/tests/check-block.sh
>> index f3d12fd602..3b971d6cf4 100755
>> --- a/tests/check-block.sh
>> +++ b/tests/check-block.sh
>> @@ -1,24 +1,48 @@
>>  #!/bin/sh
>>
>> -FORMAT_LIST="raw qcow2 qed vmdk vpc"
>> +# Honor the SPEED environment variable, just like we do it for the qtests.
>> +if [ "$SPEED" = "slow" ]; then
>> +    format_list="raw qcow2"
>> +    group=
>> +elif [ "$SPEED" = "thorough" ]; then
>> +    format_list="raw qcow2 qed vmdk vpc"
>> +    group=
>> +else
>> +    format_list=qcow2
>> +    group="-g auto"
>> +fi
>> +
>>  if [ "$#" -ne 0 ]; then
>> -    FORMAT_LIST="$@"
>> +    format_list="$@"
>> +fi
>> +
>> +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
>> +    echo "GPROF is enabled ==> Not running the qemu-iotests."
>> +    exit 0
>>  fi
>>
>> -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
>> -export QEMU_IMG_PROG="$PWD/qemu-img"
>> -export QEMU_IO_PROG="$PWD/qemu-io"
>> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
>> +    echo "No qemu-system binary available ==> Not running the qemu-iotests."
>> +    exit 0
>> +fi
>> +
>> +if ! command -v bash >/dev/null 2>&1 ; then
>> +    echo "bash not available ==> Not running the qemu-iotests."
>> +    exit 0
>> +fi
>>
>> -if [ ! -x $QEMU_PROG ]; then
>> -    echo "'make check-block' requires qemu-system-x86_64"
>> -    exit 1
>> +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
>> +    if ! command -v gsed >/dev/null 2>&1; then
>> +        echo "GNU sed not available ==> Not running the qemu-iotests."
>> +        exit 0
>> +    fi
>>  fi
>>
>>  cd tests/qemu-iotests
>>
>>  ret=0
>> -for FMT in $FORMAT_LIST ; do
>> -    ./check -T -nocache -$FMT || ret=1
>> +for fmt in $format_list ; do
>> +    ./check -$fmt $group || ret=1
> 
> This is all looking good and my only remaining objection is aesthetic.
> If you run "make check -jN" you end up with the with the block test
> output intermingled with the rest of the output which is now fairly
> uniform.
> 
> Any chance we could hide the verbosity unless requested and have
> something like:
> 
>     TEST    check-block: test xxx
> 
> emitted for each passing test?
Yeah, I noticed this already, too. I was already thinking of changing
the tests/qemu-iotests/check script a little bit (maybe even teaching it
the TAP protocol?), but I felt it was too much for this series.
Considering that the iotests have been broken two times already after
the 4.1 tree has been opened, I think we should get this merged as soon
as possible to avoid at least partly future regressions. Clean up of the
test output can and will be done in a separate patch later.

 Thomas

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
@ 2019-05-03 10:03       ` Thomas Huth
  0 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-03 10:03 UTC (permalink / raw)
  To: Alex Bennée
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, qemu-devel,
	Christophe Fergeau, Max Reitz, Wainer dos Santos Moschetta,
	Philippe Mathieu-Daudé,
	Li-Wen Hsu, Markus Armbruster

On 03/05/2019 11.53, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> People often forget to run the iotests before submitting patches or
>> pull requests - this is likely due to the fact that we do not run the
>> tests during our mandatory "make check" tests yet. Now that we've got
>> a proper "auto" group of iotests that should be fine to run in every
>> environment, we can enable the iotests during "make check" again by
>> running the "auto" tests by default from the check-block.sh script.
>>
>> Some cases still need to be checked first, though: iotests need bash
>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>> the output of some test cases causing them to fail. So if we detect
>> that one of the required programs is missing or that gprof is enabled,
>> we still have to skip the iotests to avoid failures.
>>
>> And finally, since we are using check-block.sh now again, this patch also
>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>> (and having two shell wrapper scripts around the block tests seem
>> rather confusing than helpful).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/Makefile.include      |  8 +++----
>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>  tests/qemu-iotests-quick.sh |  8 -------
>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>
>> diff --git a/tests/Makefile.include b/tests/Makefile.include
>> index e2432d5e77..3bb7793d4a 100644
>> --- a/tests/Makefile.include
>> +++ b/tests/Makefile.include
>> @@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF)
>>  check-unit-y += tests/ptimer-test$(EXESUF)
>>  check-unit-y += tests/test-qapi-util$(EXESUF)
>>
>> -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
>> +check-block-$(CONFIG_POSIX) += tests/check-block.sh
>>
>>  # All QTests for now are POSIX-only, but the dependencies are
>>  # really in libqtest, not in the testcases themselves.
>> @@ -1096,8 +1096,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
>>
>>  QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF)
>>
>> -.PHONY: check-tests/qemu-iotests-quick.sh
>> -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
>> +.PHONY: check-tests/check-block.sh
>> +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y)
>>  	$<
>>
>>  .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
>> @@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>>  check-block: $(patsubst %,check-%, $(check-block-y))
>> -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
>> +check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
>>  check-clean:
>>  	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>>  	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
>> diff --git a/tests/check-block.sh b/tests/check-block.sh
>> index f3d12fd602..3b971d6cf4 100755
>> --- a/tests/check-block.sh
>> +++ b/tests/check-block.sh
>> @@ -1,24 +1,48 @@
>>  #!/bin/sh
>>
>> -FORMAT_LIST="raw qcow2 qed vmdk vpc"
>> +# Honor the SPEED environment variable, just like we do it for the qtests.
>> +if [ "$SPEED" = "slow" ]; then
>> +    format_list="raw qcow2"
>> +    group=
>> +elif [ "$SPEED" = "thorough" ]; then
>> +    format_list="raw qcow2 qed vmdk vpc"
>> +    group=
>> +else
>> +    format_list=qcow2
>> +    group="-g auto"
>> +fi
>> +
>>  if [ "$#" -ne 0 ]; then
>> -    FORMAT_LIST="$@"
>> +    format_list="$@"
>> +fi
>> +
>> +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
>> +    echo "GPROF is enabled ==> Not running the qemu-iotests."
>> +    exit 0
>>  fi
>>
>> -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64"
>> -export QEMU_IMG_PROG="$PWD/qemu-img"
>> -export QEMU_IO_PROG="$PWD/qemu-io"
>> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
>> +    echo "No qemu-system binary available ==> Not running the qemu-iotests."
>> +    exit 0
>> +fi
>> +
>> +if ! command -v bash >/dev/null 2>&1 ; then
>> +    echo "bash not available ==> Not running the qemu-iotests."
>> +    exit 0
>> +fi
>>
>> -if [ ! -x $QEMU_PROG ]; then
>> -    echo "'make check-block' requires qemu-system-x86_64"
>> -    exit 1
>> +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then
>> +    if ! command -v gsed >/dev/null 2>&1; then
>> +        echo "GNU sed not available ==> Not running the qemu-iotests."
>> +        exit 0
>> +    fi
>>  fi
>>
>>  cd tests/qemu-iotests
>>
>>  ret=0
>> -for FMT in $FORMAT_LIST ; do
>> -    ./check -T -nocache -$FMT || ret=1
>> +for fmt in $format_list ; do
>> +    ./check -$fmt $group || ret=1
> 
> This is all looking good and my only remaining objection is aesthetic.
> If you run "make check -jN" you end up with the with the block test
> output intermingled with the rest of the output which is now fairly
> uniform.
> 
> Any chance we could hide the verbosity unless requested and have
> something like:
> 
>     TEST    check-block: test xxx
> 
> emitted for each passing test?
Yeah, I noticed this already, too. I was already thinking of changing
the tests/qemu-iotests/check script a little bit (maybe even teaching it
the TAP protocol?), but I felt it was too much for this series.
Considering that the iotests have been broken two times already after
the 4.1 tree has been opened, I think we should get this merged as soon
as possible to avoid at least partly future regressions. Clean up of the
test output can and will be done in a separate patch later.

 Thomas


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-02  8:45   ` Thomas Huth
  (?)
@ 2019-05-07 13:22   ` Markus Armbruster
  2019-05-07 15:22     ` Thomas Huth
  -1 siblings, 1 reply; 41+ messages in thread
From: Markus Armbruster @ 2019-05-07 13:22 UTC (permalink / raw)
  To: Thomas Huth
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block,
	Philippe Mathieu-Daudé,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Alex Bennée, Li-Wen Hsu

Thomas Huth <thuth@redhat.com> writes:

> Currently, all tests are in the "auto" group. This is a little bit pointless.
> OTOH, we need a group for the tests that we can automatically run during
> "make check" each time, too. Tests in this new group are supposed to run
> with every possible QEMU configuration, for example they must run with every
> QEMU binary (also non-x86), without failing when an optional features is
> missing (but reporting "skip" is ok), and be able to run on all kind of host
> filesystems and users (i.e. also as "nobody" or "root").
> So let's use the "auto" group for this class of tests now. The initial
> list has been determined by running the iotests with non-x86 QEMU targets
> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
> macOS and FreeBSD).

I wonder whether we should additionally limit "make check" to "quick"
tests.  How slow are the non-quick auto tests for you?


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-07 13:22   ` Markus Armbruster
@ 2019-05-07 15:22     ` Thomas Huth
  2019-05-07 15:50       ` Eric Blake
  0 siblings, 1 reply; 41+ messages in thread
From: Thomas Huth @ 2019-05-07 15:22 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Alex Bennée,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

On 07/05/2019 15.22, Markus Armbruster wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> Currently, all tests are in the "auto" group. This is a little bit pointless.
>> OTOH, we need a group for the tests that we can automatically run during
>> "make check" each time, too. Tests in this new group are supposed to run
>> with every possible QEMU configuration, for example they must run with every
>> QEMU binary (also non-x86), without failing when an optional features is
>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>> filesystems and users (i.e. also as "nobody" or "root").
>> So let's use the "auto" group for this class of tests now. The initial
>> list has been determined by running the iotests with non-x86 QEMU targets
>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>> macOS and FreeBSD).
> 
> I wonder whether we should additionally limit "make check" to "quick"
> tests.  How slow are the non-quick auto tests for you?

I already sorted out some of the tests that run veeeery long, since the
run time on gitlab, cirrus-ci and travis is limited. "make check-block"
currently takes 3 minutes on my laptop, I think that's still ok?

When I run the tests from the auto group that are not in the quick
group, I currently get:

003 1s ...
007 2s ...
013 5s ...
014 15s ...
015 9s ...
022 1s ...
023 18s ...
043 0s ...
049 3s ...
061 4s ...
079 2s ...
080 4s ...
091 1s ...
097 2s ...
104 0s ...
117 0s ...
122 8s ...
126 1s ...
137 1s ...
142 5s ...
172 3s ...
174 0s ...
176 5s ...
181 2s ...
186 2s ...
187 0s ...
191 5s ...
201 2s ...
214 0s ...

That looks reasonable to me.

 Thomas


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-07 15:22     ` Thomas Huth
@ 2019-05-07 15:50       ` Eric Blake
  2019-05-08  5:47         ` Thomas Huth
  2019-05-08 12:46         ` Markus Armbruster
  0 siblings, 2 replies; 41+ messages in thread
From: Eric Blake @ 2019-05-07 15:50 UTC (permalink / raw)
  To: Thomas Huth, Markus Armbruster
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block,
	Philippe Mathieu-Daudé,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Alex Bennée, Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 1940 bytes --]

On 5/7/19 10:22 AM, Thomas Huth wrote:
> On 07/05/2019 15.22, Markus Armbruster wrote:
>> Thomas Huth <thuth@redhat.com> writes:
>>
>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
>>> OTOH, we need a group for the tests that we can automatically run during
>>> "make check" each time, too. Tests in this new group are supposed to run
>>> with every possible QEMU configuration, for example they must run with every
>>> QEMU binary (also non-x86), without failing when an optional features is
>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>>> filesystems and users (i.e. also as "nobody" or "root").
>>> So let's use the "auto" group for this class of tests now. The initial
>>> list has been determined by running the iotests with non-x86 QEMU targets
>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>>> macOS and FreeBSD).
>>
>> I wonder whether we should additionally limit "make check" to "quick"
>> tests.  How slow are the non-quick auto tests for you?
> 
> I already sorted out some of the tests that run veeeery long, since the
> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
> currently takes 3 minutes on my laptop, I think that's still ok?
> 
> When I run the tests from the auto group that are not in the quick
> group, I currently get:
> 

My personal threshold is about 5 seconds for quick, so:

> 003 1s ...
> 007 2s ...

Should these be moved to quick?

> 013 5s ...

this one is borderline

> 014 15s ...
> 015 9s ...

Definitely not quick, but if you think they are still okay for auto, I
can live with that.

> 022 1s ...

Another candidate for quick?

> 023 18s ...

Even longer than 14. Okay for auto?

etc.


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-07 15:50       ` Eric Blake
@ 2019-05-08  5:47         ` Thomas Huth
  2019-05-10  8:55           ` Thomas Huth
  2019-05-08 12:46         ` Markus Armbruster
  1 sibling, 1 reply; 41+ messages in thread
From: Thomas Huth @ 2019-05-08  5:47 UTC (permalink / raw)
  To: Eric Blake, Markus Armbruster
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block,
	Philippe Mathieu-Daudé,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Alex Bennée, Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 2229 bytes --]

On 07/05/2019 17.50, Eric Blake wrote:
> On 5/7/19 10:22 AM, Thomas Huth wrote:
>> On 07/05/2019 15.22, Markus Armbruster wrote:
>>> Thomas Huth <thuth@redhat.com> writes:
>>>
>>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
>>>> OTOH, we need a group for the tests that we can automatically run during
>>>> "make check" each time, too. Tests in this new group are supposed to run
>>>> with every possible QEMU configuration, for example they must run with every
>>>> QEMU binary (also non-x86), without failing when an optional features is
>>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>>>> filesystems and users (i.e. also as "nobody" or "root").
>>>> So let's use the "auto" group for this class of tests now. The initial
>>>> list has been determined by running the iotests with non-x86 QEMU targets
>>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>>>> macOS and FreeBSD).
>>>
>>> I wonder whether we should additionally limit "make check" to "quick"
>>> tests.  How slow are the non-quick auto tests for you?
>>
>> I already sorted out some of the tests that run veeeery long, since the
>> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
>> currently takes 3 minutes on my laptop, I think that's still ok?
>>
>> When I run the tests from the auto group that are not in the quick
>> group, I currently get:
>>
> 
> My personal threshold is about 5 seconds for quick, so:
> 
>> 003 1s ...
>> 007 2s ...
> 
> Should these be moved to quick?

I'll leave that decision up to the blocklayer folks ... I thought that
there might have been a different reason that these have not been put
into "quick" yet...?

>> 013 5s ...
> 
> this one is borderline
> 
>> 014 15s ...
>> 015 9s ...
> 
> Definitely not quick, but if you think they are still okay for auto, I
> can live with that.
> 
>> 022 1s ...
> 
> Another candidate for quick?
> 
>> 023 18s ...
> 
> Even longer than 14. Okay for auto?

I think I'd give it a try. If people are complaining later that "make
check" is running now way too long, we still can refine the list later.

 Thomas


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-07 15:50       ` Eric Blake
  2019-05-08  5:47         ` Thomas Huth
@ 2019-05-08 12:46         ` Markus Armbruster
  1 sibling, 0 replies; 41+ messages in thread
From: Markus Armbruster @ 2019-05-08 12:46 UTC (permalink / raw)
  To: Eric Blake
  Cc: Fam Zheng, Kevin Wolf, Thomas Huth, Ed Maste, qemu-block,
	Alex Bennée, Markus Armbruster, Christophe Fergeau,
	qemu-devel, Wainer dos Santos Moschetta, Max Reitz,
	Philippe Mathieu-Daudé,
	Li-Wen Hsu

Eric Blake <eblake@redhat.com> writes:

> On 5/7/19 10:22 AM, Thomas Huth wrote:
>> On 07/05/2019 15.22, Markus Armbruster wrote:
>>> Thomas Huth <thuth@redhat.com> writes:
>>>
>>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
>>>> OTOH, we need a group for the tests that we can automatically run during
>>>> "make check" each time, too. Tests in this new group are supposed to run
>>>> with every possible QEMU configuration, for example they must run with every
>>>> QEMU binary (also non-x86), without failing when an optional features is
>>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>>>> filesystems and users (i.e. also as "nobody" or "root").
>>>> So let's use the "auto" group for this class of tests now. The initial
>>>> list has been determined by running the iotests with non-x86 QEMU targets
>>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>>>> macOS and FreeBSD).
>>>
>>> I wonder whether we should additionally limit "make check" to "quick"
>>> tests.  How slow are the non-quick auto tests for you?
>> 
>> I already sorted out some of the tests that run veeeery long, since the
>> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
>> currently takes 3 minutes on my laptop, I think that's still ok?
>> 
>> When I run the tests from the auto group that are not in the quick
>> group, I currently get:
>> 
>
> My personal threshold is about 5 seconds for quick, so:
>
>> 003 1s ...
>> 007 2s ...
>
> Should these be moved to quick?
>
>> 013 5s ...
>
> this one is borderline
>
>> 014 15s ...
>> 015 9s ...
>
> Definitely not quick, but if you think they are still okay for auto, I
> can live with that.
>
>> 022 1s ...
>
> Another candidate for quick?
>
>> 023 18s ...
>
> Even longer than 14. Okay for auto?
>
> etc.

Kevin, Max?


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-02  8:45   ` Thomas Huth
  (?)
  (?)
@ 2019-05-09 18:08   ` Max Reitz
  2019-05-10  4:29     ` Thomas Huth
  -1 siblings, 1 reply; 41+ messages in thread
From: Max Reitz @ 2019-05-09 18:08 UTC (permalink / raw)
  To: Thomas Huth, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

On 02.05.19 10:45, Thomas Huth wrote:
> People often forget to run the iotests before submitting patches or
> pull requests - this is likely due to the fact that we do not run the
> tests during our mandatory "make check" tests yet. Now that we've got
> a proper "auto" group of iotests that should be fine to run in every
> environment, we can enable the iotests during "make check" again by
> running the "auto" tests by default from the check-block.sh script.
> 
> Some cases still need to be checked first, though: iotests need bash
> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
> the output of some test cases causing them to fail. So if we detect
> that one of the required programs is missing or that gprof is enabled,
> we still have to skip the iotests to avoid failures.
> 
> And finally, since we are using check-block.sh now again, this patch also
> removes the qemu-iotests-quick.sh script since we do not need that anymore
> (and having two shell wrapper scripts around the block tests seem
> rather confusing than helpful).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/Makefile.include      |  8 +++----
>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>  tests/qemu-iotests-quick.sh |  8 -------
>  3 files changed, 38 insertions(+), 22 deletions(-)
>  delete mode 100755 tests/qemu-iotests-quick.sh

Can I interest you in a Makefile target that explicitly excludes
check-block?  I run the iotests anyway, but I also run make check.
Running some iotests twice would be a bit pointless.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-09 18:08   ` Max Reitz
@ 2019-05-10  4:29     ` Thomas Huth
  2019-05-10 13:34       ` Max Reitz
  0 siblings, 1 reply; 41+ messages in thread
From: Thomas Huth @ 2019-05-10  4:29 UTC (permalink / raw)
  To: Max Reitz, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

On 09/05/2019 20.08, Max Reitz wrote:
> On 02.05.19 10:45, Thomas Huth wrote:
>> People often forget to run the iotests before submitting patches or
>> pull requests - this is likely due to the fact that we do not run the
>> tests during our mandatory "make check" tests yet. Now that we've got
>> a proper "auto" group of iotests that should be fine to run in every
>> environment, we can enable the iotests during "make check" again by
>> running the "auto" tests by default from the check-block.sh script.
>>
>> Some cases still need to be checked first, though: iotests need bash
>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>> the output of some test cases causing them to fail. So if we detect
>> that one of the required programs is missing or that gprof is enabled,
>> we still have to skip the iotests to avoid failures.
>>
>> And finally, since we are using check-block.sh now again, this patch also
>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>> (and having two shell wrapper scripts around the block tests seem
>> rather confusing than helpful).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/Makefile.include      |  8 +++----
>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>  tests/qemu-iotests-quick.sh |  8 -------
>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>  delete mode 100755 tests/qemu-iotests-quick.sh
> 
> Can I interest you in a Makefile target that explicitly excludes
> check-block?  I run the iotests anyway, but I also run make check.
> Running some iotests twice would be a bit pointless.

Can't you simply run

 ./check -qcow2 -x auto

instead?

 Thomas


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-08  5:47         ` Thomas Huth
@ 2019-05-10  8:55           ` Thomas Huth
  2019-05-10  9:15             ` [Qemu-devel] [PATCH v4] " Thomas Huth
                               ` (2 more replies)
  0 siblings, 3 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-10  8:55 UTC (permalink / raw)
  To: Eric Blake, Markus Armbruster
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Alex Bennée,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

On 08/05/2019 07.47, Thomas Huth wrote:
> On 07/05/2019 17.50, Eric Blake wrote:
>> On 5/7/19 10:22 AM, Thomas Huth wrote:
>>> On 07/05/2019 15.22, Markus Armbruster wrote:
>>>> Thomas Huth <thuth@redhat.com> writes:
>>>>
>>>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
>>>>> OTOH, we need a group for the tests that we can automatically run during
>>>>> "make check" each time, too. Tests in this new group are supposed to run
>>>>> with every possible QEMU configuration, for example they must run with every
>>>>> QEMU binary (also non-x86), without failing when an optional features is
>>>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>>>>> filesystems and users (i.e. also as "nobody" or "root").
>>>>> So let's use the "auto" group for this class of tests now. The initial
>>>>> list has been determined by running the iotests with non-x86 QEMU targets
>>>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>>>>> macOS and FreeBSD).
>>>>
>>>> I wonder whether we should additionally limit "make check" to "quick"
>>>> tests.  How slow are the non-quick auto tests for you?
>>>
>>> I already sorted out some of the tests that run veeeery long, since the
>>> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
>>> currently takes 3 minutes on my laptop, I think that's still ok?
>>>
>>> When I run the tests from the auto group that are not in the quick
>>> group, I currently get:
>>>
>>
>> My personal threshold is about 5 seconds for quick, so:
>>
>>> 003 1s ...
>>> 007 2s ...
>>
>> Should these be moved to quick?
> 
> I'll leave that decision up to the blocklayer folks ... I thought that
> there might have been a different reason that these have not been put
> into "quick" yet...?
> 
>>> 013 5s ...
>>
>> this one is borderline
>>
>>> 014 15s ...
>>> 015 9s ...
>>
>> Definitely not quick, but if you think they are still okay for auto, I
>> can live with that.
>>
>>> 022 1s ...
>>
>> Another candidate for quick?
>>
>>> 023 18s ...
>>
>> Even longer than 14. Okay for auto?
> 
> I think I'd give it a try. If people are complaining later that "make
> check" is running now way too long, we still can refine the list later.

Thinking about this again, "make check" now runs quite a bit longer
indeed. So I now rather tend to remove the tests that run longer than 5s
from the auto group instead... I think I'll send a v4 of this patch
where I'll remove them from the auto group.

 Thomas


^ permalink raw reply	[flat|nested] 41+ messages in thread
* [Qemu-devel] [PATCH v4] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-10  8:55           ` Thomas Huth
@ 2019-05-10  9:15             ` Thomas Huth
  2019-05-10 11:38             ` [Qemu-devel] [PATCH v3 6/7] " Markus Armbruster
  2019-05-10 14:21             ` Kevin Wolf
  2 siblings, 0 replies; 41+ messages in thread
From: Thomas Huth @ 2019-05-10  9:15 UTC (permalink / raw)
  To: qemu-devel, alex.bennee; +Cc: kwolf, armbru, qemu-block, mreitz

Currently, all tests are in the "auto" group. This is a little bit pointless.
OTOH, we need a group for the tests that we can automatically run during
"make check" each time, too. Tests in this new group are supposed to run
with every possible QEMU configuration, for example they must run with every
QEMU binary (also non-x86), without failing when an optional features is
missing (but reporting "skip" is ok), run with qcow2 and file protocol,
run reasonable fast, and be able to run on all kind of host filesystems
and users (i.e. also as "nobody" or "root").
So let's use the "auto" group for this class of tests now. The initial
list has been determined by running the iotests with non-x86 QEMU targets
and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
macOS and FreeBSD).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 v4:
 - Dropped the tests from the "auto" group that run longer than 5s
 - Dropped also the tests that do not support the qcow2 format

 Alex, could you replace the patch in your patch series, too, please?

 tests/qemu-iotests/group | 262 +++++++++++++++++++++------------------
 1 file changed, 138 insertions(+), 124 deletions(-)

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 7ac9a5ea4a..a2260d0899 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -1,8 +1,22 @@
 #
 # QA groups control file
 # Defines test groups
+#
+# Some notes about the groups:
+#
 # - do not start group names with a digit
 #
+# - quick : Tests in this group should finish within some few seconds.
+#
+# - img : Tests in this group can be used to excercise the qemu-img tool.
+#
+# - auto : Tests in this group are used during "make check" and should be
+#   runnable in any case. That means they should run with every QEMU binary
+#   (also non-x86), with every QEMU configuration (i.e. must not fail if
+#   an optional feature is not compiled in - but reporting a "skip" is ok),
+#   work with the qcow2 format and file protocol, run reasonably fast (< 10s)
+#   and work all kind of host filesystems and users (e.g. "nobody" or "root").
+#
 
 #
 # test-group association ... one line per test
@@ -20,8 +34,8 @@
 011 rw auto quick
 012 auto quick
 013 rw auto
-014 rw auto
-015 rw snapshot auto
+014 rw
+015 rw snapshot
 # 016 was removed, do not reuse
 017 rw backing auto quick
 018 rw backing auto quick
@@ -29,14 +43,14 @@
 020 rw backing auto quick
 021 io auto quick
 022 rw snapshot auto
-023 rw auto
+023 rw
 024 rw backing auto quick
 025 rw auto quick
-026 rw blkdbg auto
+026 rw blkdbg
 027 rw auto quick
-028 rw backing auto quick
+028 rw backing quick
 029 rw auto quick
-030 rw auto backing
+030 rw backing
 031 rw auto quick
 032 rw auto quick
 033 rw auto quick
@@ -46,206 +60,206 @@
 037 rw auto backing quick
 038 rw auto backing quick
 039 rw auto quick
-040 rw auto
-041 rw auto backing
+040 rw
+041 rw backing
 042 rw auto quick
 043 rw auto backing
-044 rw auto
-045 rw auto quick
+044 rw
+045 rw quick
 046 rw auto aio quick
 047 rw auto quick
 048 img auto quick
 049 rw auto
 050 rw auto backing quick
-051 rw auto
+051 rw
 052 rw auto backing quick
 053 rw auto quick
 054 rw auto quick
-055 rw auto
-056 rw auto backing
-057 rw auto
-058 rw auto quick
-059 rw auto quick
+055 rw
+056 rw backing
+057 rw
+058 rw quick
+059 rw quick
 060 rw auto quick
 061 rw auto
 062 rw auto quick
 063 rw auto quick
-064 rw auto quick
-065 rw auto quick
+064 rw quick
+065 rw quick
 066 rw auto quick
-067 rw auto quick
-068 rw auto quick
+067 rw quick
+068 rw quick
 069 rw auto quick
-070 rw auto quick
+070 rw quick
 071 rw auto quick
 072 rw auto quick
 073 rw auto quick
 074 rw auto quick
-075 rw auto quick
-076 auto
-077 rw auto quick
-078 rw auto quick
+075 rw quick
+076 io
+077 rw quick
+078 rw quick
 079 rw auto
 080 rw auto
-081 rw auto quick
+081 rw quick
 082 rw auto quick
-083 rw auto
-084 img auto quick
-085 rw auto
+083 rw
+084 img quick
+085 rw
 086 rw auto quick
-087 rw auto quick
-088 rw auto quick
+087 rw quick
+088 rw quick
 089 rw auto quick
 090 rw auto quick
 091 rw auto migration
-092 rw auto quick
-093 auto
-094 rw auto quick
-095 rw auto quick
-096 rw auto quick
+092 rw quick
+093 throttle
+094 rw quick
+095 rw quick
+096 rw quick
 097 rw auto backing
 098 rw auto backing quick
 099 rw auto quick
 # 100 was removed, do not reuse
-101 rw auto quick
+101 rw quick
 102 rw auto quick
 103 rw auto quick
 104 rw auto
 105 rw auto quick
-106 rw auto quick
+106 rw quick
 107 rw auto quick
 108 rw auto quick
-109 rw auto
+109 rw
 110 rw auto backing quick
 111 rw auto quick
-112 rw auto
-113 rw auto quick
+112 rw
+113 rw quick
 114 rw auto quick
-115 rw auto
-116 rw auto quick
+115 rw
+116 rw quick
 117 rw auto
-118 rw auto
-119 rw auto quick
+118 rw
+119 rw quick
 120 rw auto quick
-121 rw auto
+121 rw
 122 rw auto
-123 rw auto quick
-124 rw auto backing
-125 rw auto
+123 rw quick
+124 rw backing
+125 rw
 126 rw auto backing
-127 rw auto backing quick
-128 rw auto quick
-129 rw auto quick
+127 rw backing quick
+128 rw quick
+129 rw quick
 130 rw auto quick
-131 rw auto quick
-132 rw auto quick
+131 rw quick
+132 rw quick
 133 auto quick
 134 rw auto quick
-135 rw auto
-136 rw auto
+135 rw
+136 rw
 137 rw auto
 138 rw auto quick
-139 rw auto quick
+139 rw quick
 140 rw auto quick
 141 rw auto quick
 142 auto
 143 auto quick
-144 rw auto quick
-145 auto quick
-146 auto quick
-147 auto
-148 rw auto quick
-149 rw auto sudo
+144 rw quick
+145 quick
+146 quick
+147 img
+148 rw quick
+149 rw sudo
 150 rw auto quick
-151 rw auto
-152 rw auto quick
-153 rw auto quick
+151 rw
+152 rw quick
+153 rw quick
 154 rw auto backing quick
-155 rw auto
+155 rw
 156 rw auto quick
-157 auto
+157 quick
 158 rw auto quick
 159 rw auto quick
-160 rw auto quick
+160 rw quick
 161 rw auto quick
-162 auto quick
-163 rw auto
-165 rw auto quick
-169 rw auto quick migration
+162 quick
+163 rw
+165 rw quick
+169 rw quick migration
 170 rw auto quick
-171 rw auto quick
+171 rw quick
 172 auto
-173 rw auto
+173 rw
 174 auto
-175 auto quick
+175 quick
 176 rw auto backing
 177 rw auto quick
-178 auto
+178 img
 179 rw auto quick
 181 rw auto migration
-182 rw auto quick
-183 rw auto migration
+182 rw quick
+183 rw migration
 184 rw auto quick
-185 rw auto
+185 rw
 186 rw auto
 187 rw auto
-188 rw auto quick
-189 rw auto
+188 rw quick
+189 rw
 190 rw auto quick
 191 rw auto
 192 rw auto quick
-194 rw auto migration quick
+194 rw migration quick
 195 rw auto quick
-196 rw auto quick migration
+196 rw quick migration
 197 rw auto quick
-198 rw auto
-199 rw auto migration
-200 rw auto
+198 rw
+199 rw migration
+200 rw
 201 rw auto migration
-202 rw auto quick
-203 rw auto migration
-204 rw auto quick
-205 rw auto quick
-206 rw auto
-207 rw auto
-208 rw auto quick
-209 rw auto quick
-210 rw auto
-211 rw auto quick
-212 rw auto quick
-213 rw auto quick
+202 rw quick
+203 rw migration
+204 rw quick
+205 rw quick
+206 rw
+207 rw
+208 rw quick
+209 rw quick
+210 rw
+211 rw quick
+212 rw quick
+213 rw quick
 214 rw auto
 215 rw auto quick
-216 rw auto quick
+216 rw quick
 217 rw auto quick
-218 rw auto quick
-219 rw auto
+218 rw quick
+219 rw
 220 rw auto
-221 rw auto quick
-222 rw auto quick
-223 rw auto quick
-224 rw auto quick
-225 rw auto quick
+221 rw quick
+222 rw quick
+223 rw quick
+224 rw quick
+225 rw quick
 226 auto quick
-227 auto quick
-228 rw auto quick
+227 quick
+228 rw quick
 229 auto quick
-231 auto quick
-232 auto quick
+231 quick
+232 quick
 233 auto quick
-234 auto quick migration
-235 auto quick
-236 auto quick
-237 rw auto quick
-238 auto quick
-239 rw auto quick
-240 auto quick
-241 rw auto quick
-242 rw auto quick
+234 quick migration
+235 quick
+236 quick
+237 rw quick
+238 quick
+239 rw quick
+240 quick
+241 rw quick
+242 rw quick
 243 rw auto quick
 244 rw auto quick
-245 rw auto
-246 rw auto quick
-247 rw auto quick
-248 rw auto quick
+245 rw
+246 rw quick
+247 rw quick
+248 rw quick
 249 rw auto quick
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-10  8:55           ` Thomas Huth
  2019-05-10  9:15             ` [Qemu-devel] [PATCH v4] " Thomas Huth
@ 2019-05-10 11:38             ` Markus Armbruster
  2019-05-10 14:21             ` Kevin Wolf
  2 siblings, 0 replies; 41+ messages in thread
From: Markus Armbruster @ 2019-05-10 11:38 UTC (permalink / raw)
  To: Thomas Huth
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Alex Bennée,
	Philippe Mathieu-Daudé,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Li-Wen Hsu

Thomas Huth <thuth@redhat.com> writes:

> Thinking about this again, "make check" now runs quite a bit longer
> indeed. So I now rather tend to remove the tests that run longer than 5s
> from the auto group instead... I think I'll send a v4 of this patch
> where I'll remove them from the auto group.

Appreciated!

I'm all for automated testing, but "make check" is already slow enough
to make certain kinds of work painful.  Not quite slow enough to make me
renege on commitments and go on a quest to speed it up.


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-10  4:29     ` Thomas Huth
@ 2019-05-10 13:34       ` Max Reitz
  2019-05-10 13:36         ` Thomas Huth
  2019-05-10 13:38         ` Max Reitz
  0 siblings, 2 replies; 41+ messages in thread
From: Max Reitz @ 2019-05-10 13:34 UTC (permalink / raw)
  To: Thomas Huth, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 2154 bytes --]

On 10.05.19 06:29, Thomas Huth wrote:
> On 09/05/2019 20.08, Max Reitz wrote:
>> On 02.05.19 10:45, Thomas Huth wrote:
>>> People often forget to run the iotests before submitting patches or
>>> pull requests - this is likely due to the fact that we do not run the
>>> tests during our mandatory "make check" tests yet. Now that we've got
>>> a proper "auto" group of iotests that should be fine to run in every
>>> environment, we can enable the iotests during "make check" again by
>>> running the "auto" tests by default from the check-block.sh script.
>>>
>>> Some cases still need to be checked first, though: iotests need bash
>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>> the output of some test cases causing them to fail. So if we detect
>>> that one of the required programs is missing or that gprof is enabled,
>>> we still have to skip the iotests to avoid failures.
>>>
>>> And finally, since we are using check-block.sh now again, this patch also
>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>> (and having two shell wrapper scripts around the block tests seem
>>> rather confusing than helpful).
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>  tests/Makefile.include      |  8 +++----
>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>
>> Can I interest you in a Makefile target that explicitly excludes
>> check-block?  I run the iotests anyway, but I also run make check.
>> Running some iotests twice would be a bit pointless.
> 
> Can't you simply run
> 
>  ./check -qcow2 -x auto
> 
> instead?

I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.

So for which of the protocol/format combinations do I exclude the auto
group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
may that not be subject to change?

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-10 13:34       ` Max Reitz
@ 2019-05-10 13:36         ` Thomas Huth
  2019-05-10 13:47           ` Max Reitz
  2019-05-10 13:38         ` Max Reitz
  1 sibling, 1 reply; 41+ messages in thread
From: Thomas Huth @ 2019-05-10 13:36 UTC (permalink / raw)
  To: Max Reitz, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 2311 bytes --]

On 10/05/2019 15.34, Max Reitz wrote:
> On 10.05.19 06:29, Thomas Huth wrote:
>> On 09/05/2019 20.08, Max Reitz wrote:
>>> On 02.05.19 10:45, Thomas Huth wrote:
>>>> People often forget to run the iotests before submitting patches or
>>>> pull requests - this is likely due to the fact that we do not run the
>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>> a proper "auto" group of iotests that should be fine to run in every
>>>> environment, we can enable the iotests during "make check" again by
>>>> running the "auto" tests by default from the check-block.sh script.
>>>>
>>>> Some cases still need to be checked first, though: iotests need bash
>>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>>> the output of some test cases causing them to fail. So if we detect
>>>> that one of the required programs is missing or that gprof is enabled,
>>>> we still have to skip the iotests to avoid failures.
>>>>
>>>> And finally, since we are using check-block.sh now again, this patch also
>>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>>> (and having two shell wrapper scripts around the block tests seem
>>>> rather confusing than helpful).
>>>>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>>  tests/Makefile.include      |  8 +++----
>>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>>
>>> Can I interest you in a Makefile target that explicitly excludes
>>> check-block?  I run the iotests anyway, but I also run make check.
>>> Running some iotests twice would be a bit pointless.
>>
>> Can't you simply run
>>
>>  ./check -qcow2 -x auto
>>
>> instead?
> 
> I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
> qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.
> 
> So for which of the protocol/format combinations do I exclude the auto
> group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
> may that not be subject to change?

With my patch series, the auto group is only used for qcow2.

 Thomas


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-10 13:34       ` Max Reitz
  2019-05-10 13:36         ` Thomas Huth
@ 2019-05-10 13:38         ` Max Reitz
  1 sibling, 0 replies; 41+ messages in thread
From: Max Reitz @ 2019-05-10 13:38 UTC (permalink / raw)
  To: Thomas Huth, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 2300 bytes --]

On 10.05.19 15:34, Max Reitz wrote:
> On 10.05.19 06:29, Thomas Huth wrote:
>> On 09/05/2019 20.08, Max Reitz wrote:
>>> On 02.05.19 10:45, Thomas Huth wrote:
>>>> People often forget to run the iotests before submitting patches or
>>>> pull requests - this is likely due to the fact that we do not run the
>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>> a proper "auto" group of iotests that should be fine to run in every
>>>> environment, we can enable the iotests during "make check" again by
>>>> running the "auto" tests by default from the check-block.sh script.
>>>>
>>>> Some cases still need to be checked first, though: iotests need bash
>>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>>> the output of some test cases causing them to fail. So if we detect
>>>> that one of the required programs is missing or that gprof is enabled,
>>>> we still have to skip the iotests to avoid failures.
>>>>
>>>> And finally, since we are using check-block.sh now again, this patch also
>>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>>> (and having two shell wrapper scripts around the block tests seem
>>>> rather confusing than helpful).
>>>>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>>  tests/Makefile.include      |  8 +++----
>>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>>
>>> Can I interest you in a Makefile target that explicitly excludes
>>> check-block?  I run the iotests anyway, but I also run make check.
>>> Running some iotests twice would be a bit pointless.
>>
>> Can't you simply run
>>
>>  ./check -qcow2 -x auto
>>
>> instead?
> 
> I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
> qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.
> 
> So for which of the protocol/format combinations do I exclude the auto
> group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
> may that not be subject to change?

Oh, and furthermore I prefer to run the tests on tmpfs.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-10 13:36         ` Thomas Huth
@ 2019-05-10 13:47           ` Max Reitz
  2019-05-10 16:20             ` Thomas Huth
  0 siblings, 1 reply; 41+ messages in thread
From: Max Reitz @ 2019-05-10 13:47 UTC (permalink / raw)
  To: Thomas Huth, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block, Markus Armbruster,
	Christophe Fergeau, Alex Bennée,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 2751 bytes --]

On 10.05.19 15:36, Thomas Huth wrote:
> On 10/05/2019 15.34, Max Reitz wrote:
>> On 10.05.19 06:29, Thomas Huth wrote:
>>> On 09/05/2019 20.08, Max Reitz wrote:
>>>> On 02.05.19 10:45, Thomas Huth wrote:
>>>>> People often forget to run the iotests before submitting patches or
>>>>> pull requests - this is likely due to the fact that we do not run the
>>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>>> a proper "auto" group of iotests that should be fine to run in every
>>>>> environment, we can enable the iotests during "make check" again by
>>>>> running the "auto" tests by default from the check-block.sh script.
>>>>>
>>>>> Some cases still need to be checked first, though: iotests need bash
>>>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>>>> the output of some test cases causing them to fail. So if we detect
>>>>> that one of the required programs is missing or that gprof is enabled,
>>>>> we still have to skip the iotests to avoid failures.
>>>>>
>>>>> And finally, since we are using check-block.sh now again, this patch also
>>>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>>>> (and having two shell wrapper scripts around the block tests seem
>>>>> rather confusing than helpful).
>>>>>
>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>> ---
>>>>>  tests/Makefile.include      |  8 +++----
>>>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>>>
>>>> Can I interest you in a Makefile target that explicitly excludes
>>>> check-block?  I run the iotests anyway, but I also run make check.
>>>> Running some iotests twice would be a bit pointless.
>>>
>>> Can't you simply run
>>>
>>>  ./check -qcow2 -x auto
>>>
>>> instead?
>>
>> I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
>> qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.
>>
>> So for which of the protocol/format combinations do I exclude the auto
>> group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
>> may that not be subject to change?
> 
> With my patch series, the auto group is only used for qcow2.

And that is not subject to change?  Like, maybe someone wants to add nbd
in the future?

I mean, I have a test branch anyway which collects a number of patches
on top of master that make everything pass more or less reliably for me
(11 patches currently...).  I suppose I can just revert your patch on
top of that.  But that doesn’t feel right.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-10  8:55           ` Thomas Huth
  2019-05-10  9:15             ` [Qemu-devel] [PATCH v4] " Thomas Huth
  2019-05-10 11:38             ` [Qemu-devel] [PATCH v3 6/7] " Markus Armbruster
@ 2019-05-10 14:21             ` Kevin Wolf
  2019-05-10 15:29               ` Markus Armbruster
  2 siblings, 1 reply; 41+ messages in thread
From: Kevin Wolf @ 2019-05-10 14:21 UTC (permalink / raw)
  To: Thomas Huth
  Cc: Fam Zheng, Ed Maste, qemu-block, Philippe Mathieu-Daudé,
	Markus Armbruster, Christophe Fergeau, qemu-devel,
	Alex Bennée, Wainer dos Santos Moschetta, Max Reitz,
	Li-Wen Hsu

Am 10.05.2019 um 10:55 hat Thomas Huth geschrieben:
> On 08/05/2019 07.47, Thomas Huth wrote:
> > On 07/05/2019 17.50, Eric Blake wrote:
> >> On 5/7/19 10:22 AM, Thomas Huth wrote:
> >>> On 07/05/2019 15.22, Markus Armbruster wrote:
> >>>> Thomas Huth <thuth@redhat.com> writes:
> >>>>
> >>>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
> >>>>> OTOH, we need a group for the tests that we can automatically run during
> >>>>> "make check" each time, too. Tests in this new group are supposed to run
> >>>>> with every possible QEMU configuration, for example they must run with every
> >>>>> QEMU binary (also non-x86), without failing when an optional features is
> >>>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
> >>>>> filesystems and users (i.e. also as "nobody" or "root").
> >>>>> So let's use the "auto" group for this class of tests now. The initial
> >>>>> list has been determined by running the iotests with non-x86 QEMU targets
> >>>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
> >>>>> macOS and FreeBSD).
> >>>>
> >>>> I wonder whether we should additionally limit "make check" to "quick"
> >>>> tests.  How slow are the non-quick auto tests for you?
> >>>
> >>> I already sorted out some of the tests that run veeeery long, since the
> >>> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
> >>> currently takes 3 minutes on my laptop, I think that's still ok?
> >>>
> >>> When I run the tests from the auto group that are not in the quick
> >>> group, I currently get:
> >>>
> >>
> >> My personal threshold is about 5 seconds for quick, so:
> >>
> >>> 003 1s ...
> >>> 007 2s ...
> >>
> >> Should these be moved to quick?
> > 
> > I'll leave that decision up to the blocklayer folks ... I thought that
> > there might have been a different reason that these have not been put
> > into "quick" yet...?
> > 
> >>> 013 5s ...
> >>
> >> this one is borderline
> >>
> >>> 014 15s ...
> >>> 015 9s ...
> >>
> >> Definitely not quick, but if you think they are still okay for auto, I
> >> can live with that.
> >>
> >>> 022 1s ...
> >>
> >> Another candidate for quick?
> >>
> >>> 023 18s ...
> >>
> >> Even longer than 14. Okay for auto?
> > 
> > I think I'd give it a try. If people are complaining later that "make
> > check" is running now way too long, we still can refine the list later.
> 
> Thinking about this again, "make check" now runs quite a bit longer
> indeed. So I now rather tend to remove the tests that run longer than 5s
> from the auto group instead... I think I'll send a v4 of this patch
> where I'll remove them from the auto group.

I don't think time is everything. We should also consider how much
the tests contribute to basic code coverage. There is no point in
removing a test from the list because it takes 10 seconds, but if I
split it in two tests taking each 5 seconds, you would include both
halves.

For example, 030, 040 and 041 are not that quick (14/11/42 seconds,
respectively), but they are the most important tests for block jobs and
covering a lot. Sure, 42 seconds is a lot, but I'd keep 030 and 040 at
least.

Kevin


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-10 14:21             ` Kevin Wolf
@ 2019-05-10 15:29               ` Markus Armbruster
  2019-05-10 16:07                 ` Kevin Wolf
  0 siblings, 1 reply; 41+ messages in thread
From: Markus Armbruster @ 2019-05-10 15:29 UTC (permalink / raw)
  To: Kevin Wolf
  Cc: Fam Zheng, Thomas Huth, Ed Maste, qemu-block, Alex Bennée,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

Kevin Wolf <kwolf@redhat.com> writes:

> Am 10.05.2019 um 10:55 hat Thomas Huth geschrieben:
>> On 08/05/2019 07.47, Thomas Huth wrote:
>> > On 07/05/2019 17.50, Eric Blake wrote:
>> >> On 5/7/19 10:22 AM, Thomas Huth wrote:
>> >>> On 07/05/2019 15.22, Markus Armbruster wrote:
>> >>>> Thomas Huth <thuth@redhat.com> writes:
>> >>>>
>> >>>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
>> >>>>> OTOH, we need a group for the tests that we can automatically run during
>> >>>>> "make check" each time, too. Tests in this new group are supposed to run
>> >>>>> with every possible QEMU configuration, for example they must run with every
>> >>>>> QEMU binary (also non-x86), without failing when an optional features is
>> >>>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>> >>>>> filesystems and users (i.e. also as "nobody" or "root").
>> >>>>> So let's use the "auto" group for this class of tests now. The initial
>> >>>>> list has been determined by running the iotests with non-x86 QEMU targets
>> >>>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>> >>>>> macOS and FreeBSD).
>> >>>>
>> >>>> I wonder whether we should additionally limit "make check" to "quick"
>> >>>> tests.  How slow are the non-quick auto tests for you?
>> >>>
>> >>> I already sorted out some of the tests that run veeeery long, since the
>> >>> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
>> >>> currently takes 3 minutes on my laptop, I think that's still ok?
>> >>>
>> >>> When I run the tests from the auto group that are not in the quick
>> >>> group, I currently get:
>> >>>
>> >>
>> >> My personal threshold is about 5 seconds for quick, so:
>> >>
>> >>> 003 1s ...
>> >>> 007 2s ...
>> >>
>> >> Should these be moved to quick?
>> > 
>> > I'll leave that decision up to the blocklayer folks ... I thought that
>> > there might have been a different reason that these have not been put
>> > into "quick" yet...?
>> > 
>> >>> 013 5s ...
>> >>
>> >> this one is borderline
>> >>
>> >>> 014 15s ...
>> >>> 015 9s ...
>> >>
>> >> Definitely not quick, but if you think they are still okay for auto, I
>> >> can live with that.
>> >>
>> >>> 022 1s ...
>> >>
>> >> Another candidate for quick?
>> >>
>> >>> 023 18s ...
>> >>
>> >> Even longer than 14. Okay for auto?
>> > 
>> > I think I'd give it a try. If people are complaining later that "make
>> > check" is running now way too long, we still can refine the list later.
>> 
>> Thinking about this again, "make check" now runs quite a bit longer
>> indeed. So I now rather tend to remove the tests that run longer than 5s
>> from the auto group instead... I think I'll send a v4 of this patch
>> where I'll remove them from the auto group.
>
> I don't think time is everything. We should also consider how much
> the tests contribute to basic code coverage. There is no point in
> removing a test from the list because it takes 10 seconds, but if I
> split it in two tests taking each 5 seconds, you would include both
> halves.
>
> For example, 030, 040 and 041 are not that quick (14/11/42 seconds,
> respectively), but they are the most important tests for block jobs and
> covering a lot. Sure, 42 seconds is a lot, but I'd keep 030 and 040 at
> least.

Yes, we want block jobs covered.  However, 42 seconds is a lot indeed.
Can you think of ways to get a useful part of the full coverage in five
seconds or less?


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  2019-05-10 15:29               ` Markus Armbruster
@ 2019-05-10 16:07                 ` Kevin Wolf
  0 siblings, 0 replies; 41+ messages in thread
From: Kevin Wolf @ 2019-05-10 16:07 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Fam Zheng, Thomas Huth, Ed Maste, qemu-block, Alex Bennée,
	qemu-devel, Christophe Fergeau, Max Reitz,
	Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
	Li-Wen Hsu

Am 10.05.2019 um 17:29 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
> 
> > Am 10.05.2019 um 10:55 hat Thomas Huth geschrieben:
> >> On 08/05/2019 07.47, Thomas Huth wrote:
> >> > On 07/05/2019 17.50, Eric Blake wrote:
> >> >> On 5/7/19 10:22 AM, Thomas Huth wrote:
> >> >>> On 07/05/2019 15.22, Markus Armbruster wrote:
> >> >>>> Thomas Huth <thuth@redhat.com> writes:
> >> >>>>
> >> >>>>> Currently, all tests are in the "auto" group. This is a little bit pointless.
> >> >>>>> OTOH, we need a group for the tests that we can automatically run during
> >> >>>>> "make check" each time, too. Tests in this new group are supposed to run
> >> >>>>> with every possible QEMU configuration, for example they must run with every
> >> >>>>> QEMU binary (also non-x86), without failing when an optional features is
> >> >>>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
> >> >>>>> filesystems and users (i.e. also as "nobody" or "root").
> >> >>>>> So let's use the "auto" group for this class of tests now. The initial
> >> >>>>> list has been determined by running the iotests with non-x86 QEMU targets
> >> >>>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
> >> >>>>> macOS and FreeBSD).
> >> >>>>
> >> >>>> I wonder whether we should additionally limit "make check" to "quick"
> >> >>>> tests.  How slow are the non-quick auto tests for you?
> >> >>>
> >> >>> I already sorted out some of the tests that run veeeery long, since the
> >> >>> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
> >> >>> currently takes 3 minutes on my laptop, I think that's still ok?
> >> >>>
> >> >>> When I run the tests from the auto group that are not in the quick
> >> >>> group, I currently get:
> >> >>>
> >> >>
> >> >> My personal threshold is about 5 seconds for quick, so:
> >> >>
> >> >>> 003 1s ...
> >> >>> 007 2s ...
> >> >>
> >> >> Should these be moved to quick?
> >> > 
> >> > I'll leave that decision up to the blocklayer folks ... I thought that
> >> > there might have been a different reason that these have not been put
> >> > into "quick" yet...?
> >> > 
> >> >>> 013 5s ...
> >> >>
> >> >> this one is borderline
> >> >>
> >> >>> 014 15s ...
> >> >>> 015 9s ...
> >> >>
> >> >> Definitely not quick, but if you think they are still okay for auto, I
> >> >> can live with that.
> >> >>
> >> >>> 022 1s ...
> >> >>
> >> >> Another candidate for quick?
> >> >>
> >> >>> 023 18s ...
> >> >>
> >> >> Even longer than 14. Okay for auto?
> >> > 
> >> > I think I'd give it a try. If people are complaining later that "make
> >> > check" is running now way too long, we still can refine the list later.
> >> 
> >> Thinking about this again, "make check" now runs quite a bit longer
> >> indeed. So I now rather tend to remove the tests that run longer than 5s
> >> from the auto group instead... I think I'll send a v4 of this patch
> >> where I'll remove them from the auto group.
> >
> > I don't think time is everything. We should also consider how much
> > the tests contribute to basic code coverage. There is no point in
> > removing a test from the list because it takes 10 seconds, but if I
> > split it in two tests taking each 5 seconds, you would include both
> > halves.
> >
> > For example, 030, 040 and 041 are not that quick (14/11/42 seconds,
> > respectively), but they are the most important tests for block jobs and
> > covering a lot. Sure, 42 seconds is a lot, but I'd keep 030 and 040 at
> > least.
> 
> Yes, we want block jobs covered.  However, 42 seconds is a lot indeed.
> Can you think of ways to get a useful part of the full coverage in five
> seconds or less?

I'm not even sure which part of the test is taking so long. Maybe it's
only one of the 88 test functions that's taking long. Or we're
duplicating a lot of tests because we test all combinations of setups
where testing only certain combinations would give enough coverage.

Answering this will probably take some effort checking what the exact
cases tested are and how long each one takes.

Kevin


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-10 13:47           ` Max Reitz
@ 2019-05-10 16:20             ` Thomas Huth
  2019-05-10 17:40               ` Max Reitz
  0 siblings, 1 reply; 41+ messages in thread
From: Thomas Huth @ 2019-05-10 16:20 UTC (permalink / raw)
  To: Max Reitz, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block,
	Philippe Mathieu-Daudé,
	Markus Armbruster, Christophe Fergeau,
	Wainer dos Santos Moschetta, Alex Bennée, Li-Wen Hsu

On 10/05/2019 15.47, Max Reitz wrote:
> On 10.05.19 15:36, Thomas Huth wrote:
>> On 10/05/2019 15.34, Max Reitz wrote:
>>> On 10.05.19 06:29, Thomas Huth wrote:
>>>> On 09/05/2019 20.08, Max Reitz wrote:
>>>>> On 02.05.19 10:45, Thomas Huth wrote:
>>>>>> People often forget to run the iotests before submitting patches or
>>>>>> pull requests - this is likely due to the fact that we do not run the
>>>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>>>> a proper "auto" group of iotests that should be fine to run in every
>>>>>> environment, we can enable the iotests during "make check" again by
>>>>>> running the "auto" tests by default from the check-block.sh script.
>>>>>>
>>>>>> Some cases still need to be checked first, though: iotests need bash
>>>>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>>>>> the output of some test cases causing them to fail. So if we detect
>>>>>> that one of the required programs is missing or that gprof is enabled,
>>>>>> we still have to skip the iotests to avoid failures.
>>>>>>
>>>>>> And finally, since we are using check-block.sh now again, this patch also
>>>>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>>>>> (and having two shell wrapper scripts around the block tests seem
>>>>>> rather confusing than helpful).
>>>>>>
>>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>>> ---
>>>>>>  tests/Makefile.include      |  8 +++----
>>>>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>>>>
>>>>> Can I interest you in a Makefile target that explicitly excludes
>>>>> check-block?  I run the iotests anyway, but I also run make check.
>>>>> Running some iotests twice would be a bit pointless.
>>>>
>>>> Can't you simply run
>>>>
>>>>  ./check -qcow2 -x auto
>>>>
>>>> instead?
>>>
>>> I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
>>> qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.
>>>
>>> So for which of the protocol/format combinations do I exclude the auto
>>> group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
>>> may that not be subject to change?
>>
>> With my patch series, the auto group is only used for qcow2.
> 
> And that is not subject to change?  Like, maybe someone wants to add nbd
> in the future?

The current set of qcow2 auto tests takes already quite a while, so I
don't think that this will change soon.
And if the "normal" users want to run more tests, they can simply use
"make check SPEED=slow" or SPEED=thorough, so IMHO no need to extend the
quick default list right now.

> I mean, I have a test branch anyway which collects a number of patches
> on top of master that make everything pass more or less reliably for me
> (11 patches currently...).  I suppose I can just revert your patch on
> top of that.  But that doesn’t feel right.

Hmm, sure, non-upstream patches are always a bad thing. But I still
don't see why you really need an extra Makefile target for this. In the
worst case, you could also simply change your script to run something
this instead:

 make $(grep ^check: tests/Makefile.include | sed s/check-block//)

?

 Thomas


^ permalink raw reply	[flat|nested] 41+ messages in thread
* Re: [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again
  2019-05-10 16:20             ` Thomas Huth
@ 2019-05-10 17:40               ` Max Reitz
  0 siblings, 0 replies; 41+ messages in thread
From: Max Reitz @ 2019-05-10 17:40 UTC (permalink / raw)
  To: Thomas Huth, qemu-devel
  Cc: Fam Zheng, Kevin Wolf, Ed Maste, qemu-block,
	Philippe Mathieu-Daudé,
	Markus Armbruster, Christophe Fergeau,
	Wainer dos Santos Moschetta, Alex Bennée, Li-Wen Hsu

[-- Attachment #1: Type: text/plain, Size: 3623 bytes --]

On 10.05.19 18:20, Thomas Huth wrote:
> On 10/05/2019 15.47, Max Reitz wrote:
>> On 10.05.19 15:36, Thomas Huth wrote:
>>> On 10/05/2019 15.34, Max Reitz wrote:
>>>> On 10.05.19 06:29, Thomas Huth wrote:
>>>>> On 09/05/2019 20.08, Max Reitz wrote:
>>>>>> On 02.05.19 10:45, Thomas Huth wrote:
>>>>>>> People often forget to run the iotests before submitting patches or
>>>>>>> pull requests - this is likely due to the fact that we do not run the
>>>>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>>>>> a proper "auto" group of iotests that should be fine to run in every
>>>>>>> environment, we can enable the iotests during "make check" again by
>>>>>>> running the "auto" tests by default from the check-block.sh script.
>>>>>>>
>>>>>>> Some cases still need to be checked first, though: iotests need bash
>>>>>>> and GNU sed (otherwise they fail), and if gprof is enabled, it spoils
>>>>>>> the output of some test cases causing them to fail. So if we detect
>>>>>>> that one of the required programs is missing or that gprof is enabled,
>>>>>>> we still have to skip the iotests to avoid failures.
>>>>>>>
>>>>>>> And finally, since we are using check-block.sh now again, this patch also
>>>>>>> removes the qemu-iotests-quick.sh script since we do not need that anymore
>>>>>>> (and having two shell wrapper scripts around the block tests seem
>>>>>>> rather confusing than helpful).
>>>>>>>
>>>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>>>> ---
>>>>>>>  tests/Makefile.include      |  8 +++----
>>>>>>>  tests/check-block.sh        | 44 ++++++++++++++++++++++++++++---------
>>>>>>>  tests/qemu-iotests-quick.sh |  8 -------
>>>>>>>  3 files changed, 38 insertions(+), 22 deletions(-)
>>>>>>>  delete mode 100755 tests/qemu-iotests-quick.sh
>>>>>>
>>>>>> Can I interest you in a Makefile target that explicitly excludes
>>>>>> check-block?  I run the iotests anyway, but I also run make check.
>>>>>> Running some iotests twice would be a bit pointless.
>>>>>
>>>>> Can't you simply run
>>>>>
>>>>>  ./check -qcow2 -x auto
>>>>>
>>>>> instead?
>>>>
>>>> I don’t run just qcow2 tests.  I run qcow2, qcow2 with compat=0.10,
>>>> qcow2 with refcount_bits=1, raw, nbd, qed, vmdk, vhdx, ...  A lot.
>>>>
>>>> So for which of the protocol/format combinations do I exclude the auto
>>>> group?  check-block.sh says it runs raw, qcow2, qed, vmdk, and vpc.  But
>>>> may that not be subject to change?
>>>
>>> With my patch series, the auto group is only used for qcow2.
>>
>> And that is not subject to change?  Like, maybe someone wants to add nbd
>> in the future?
> 
> The current set of qcow2 auto tests takes already quite a while, so I
> don't think that this will change soon.
> And if the "normal" users want to run more tests, they can simply use
> "make check SPEED=slow" or SPEED=thorough, so IMHO no need to extend the
> quick default list right now.
> 
>> I mean, I have a test branch anyway which collects a number of patches
>> on top of master that make everything pass more or less reliably for me
>> (11 patches currently...).  I suppose I can just revert your patch on
>> top of that.  But that doesn’t feel right.
> 
> Hmm, sure, non-upstream patches are always a bad thing. But I still
> don't see why you really need an extra Makefile target for this. In the
> worst case, you could also simply change your script to run something
> this instead:
> 
>  make $(grep ^check: tests/Makefile.include | sed s/check-block//)
> 
> ?

Hm.  I didn’t think of that.  Thanks. :-)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
end of thread, other threads:[~2019-05-10 17:57 UTC | newest]

Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-02  8:44 [Qemu-devel] [PATCH v3 0/7] tests/qemu-iotests: Run basic iotests during "make check" Thomas Huth
2019-05-02  8:44 ` Thomas Huth
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 1/7] tests/qemu-iotests/005: Add a sanity check for large sparse file support Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-02 17:37   ` Alex Bennée
2019-05-02 17:37     ` Alex Bennée
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 2/7] tests/qemu-iotests/check: Pick a default machine if necessary Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 3/7] tests/qemu-iotests: Do not hard-code the path to bash Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 4/7] cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 5/7] tests/qemu-iotests: Remove the "_supported_os Linux" line from many tests Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-07 13:22   ` Markus Armbruster
2019-05-07 15:22     ` Thomas Huth
2019-05-07 15:50       ` Eric Blake
2019-05-08  5:47         ` Thomas Huth
2019-05-10  8:55           ` Thomas Huth
2019-05-10  9:15             ` [Qemu-devel] [PATCH v4] " Thomas Huth
2019-05-10 11:38             ` [Qemu-devel] [PATCH v3 6/7] " Markus Armbruster
2019-05-10 14:21             ` Kevin Wolf
2019-05-10 15:29               ` Markus Armbruster
2019-05-10 16:07                 ` Kevin Wolf
2019-05-08 12:46         ` Markus Armbruster
2019-05-02  8:45 ` [Qemu-devel] [PATCH v3 7/7] tests: Run the iotests during "make check" again Thomas Huth
2019-05-02  8:45   ` Thomas Huth
2019-05-03  9:53   ` Alex Bennée
2019-05-03  9:53     ` Alex Bennée
2019-05-03 10:03     ` Thomas Huth
2019-05-03 10:03       ` Thomas Huth
2019-05-09 18:08   ` Max Reitz
2019-05-10  4:29     ` Thomas Huth
2019-05-10 13:34       ` Max Reitz
2019-05-10 13:36         ` Thomas Huth
2019-05-10 13:47           ` Max Reitz
2019-05-10 16:20             ` Thomas Huth
2019-05-10 17:40               ` Max Reitz
2019-05-10 13:38         ` Max Reitz

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.