[PATCH 0/5] Enable more iotests during "make check-block"

[PATCH 0/5] Enable more iotests during "make check-block"

[Thomas Huth]

As discussed here:

 https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg00697.html

and here:

 https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg01388.html

it would be good to have some more valuable iotests enabled in the
"auto" group to get better iotest coverage during "make check".

And once Max' "iotests: Add and use $SOCK_DIR" patch series has been
merged, we can indeed enable these Python-based tests, too.

There is just one small downside: Since these tests require a QEMU
that features a 'virtio-blk' device, we cannot run the iotests
with binaries like qemu-system-tricore anymore. But since the iotests
were not very useful with such binaries anyway, I think it's ok now
if we skip them there.

Based-on: 20191010152457.17713-1-mreitz@redhat.com

John Snow (1):
  iotests: remove 'linux' from default supported platforms

Thomas Huth (4):
  iotests: Test 041 does not work on macOS
  iotests: Test 183 does not work on macOS and OpenBSD
  iotests: Skip "make check-block" if QEMU does not support virtio-blk
  iotests: Enable more tests in the 'auto' group to improve test
    coverage

 tests/check-block.sh          | 16 +++++++++++++++-
 tests/qemu-iotests/041        |  3 ++-
 tests/qemu-iotests/183        |  1 +
 tests/qemu-iotests/group      | 18 +++++++++---------
 tests/qemu-iotests/iotests.py | 16 +++++++++++-----
 5 files changed, 38 insertions(+), 16 deletions(-)

-- 
2.18.1

[PATCH 1/5] iotests: remove 'linux' from default supported platforms

[Thomas Huth]

From: John Snow <jsnow@redhat.com>

verify_platform will check an explicit whitelist and blacklist instead.
The default will now be assumed to be allowed to run anywhere.

For tests that do not specify their platforms explicitly, this has the effect of
enabling these tests on non-linux platforms. For tests that always specified
linux explicitly, there is no change.

For Python tests on FreeBSD at least; only seven python tests fail:
045 147 149 169 194 199 211

045 and 149 appear to be misconfigurations,
147 and 194 are the AF_UNIX path too long error,
169 and 199 are bitmap migration bugs, and
211 is a bug that shows up on Linux platforms, too.

This is at least good evidence that these tests are not Linux-only. If
they aren't suitable for other platforms, they should be disabled on a
per-platform basis as appropriate.

Therefore, let's switch these on and deal with the failures.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 tests/qemu-iotests/iotests.py | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 66090b70c1..d7b9a6f3ef 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -844,9 +844,14 @@ def verify_protocol(supported=[], unsupported=[]):
     if not_sup or (imgproto in unsupported):
         notrun('not suitable for this protocol: %s' % imgproto)
 
-def verify_platform(supported_oses=['linux']):
-    if True not in [sys.platform.startswith(x) for x in supported_oses]:
-        notrun('not suitable for this OS: %s' % sys.platform)
+def verify_platform(supported=None, unsupported=None):
+    if unsupported is not None:
+        if any((sys.platform.startswith(x) for x in unsupported)):
+            notrun('not suitable for this OS: %s' % sys.platform)
+
+    if supported is not None:
+        if not any((sys.platform.startswith(x) for x in supported)):
+            notrun('not suitable for this OS: %s' % sys.platform)
 
 def verify_cache_mode(supported_cache_modes=[]):
     if supported_cache_modes and (cachemode not in supported_cache_modes):
@@ -908,7 +913,8 @@ def execute_unittest(output, verbosity, debug):
                                     r'Ran \1 tests', output.getvalue()))
 
 def execute_test(test_function=None,
-                 supported_fmts=[], supported_oses=['linux'],
+                 supported_fmts=[],
+                 supported_platforms=None,
                  supported_cache_modes=[], unsupported_fmts=[],
                  supported_protocols=[], unsupported_protocols=[]):
     """Run either unittest or script-style tests."""
@@ -925,7 +931,7 @@ def execute_test(test_function=None,
     verbosity = 1
     verify_image_format(supported_fmts, unsupported_fmts)
     verify_protocol(supported_protocols, unsupported_protocols)
-    verify_platform(supported_oses)
+    verify_platform(supported=supported_platforms)
     verify_cache_mode(supported_cache_modes)
 
     if debug:
-- 
2.18.1

[PATCH 2/5] iotests: Test 041 does not work on macOS

[Thomas Huth]

041 works fine on Linux, FreeBSD and OpenBSD, so let's mark it as
only supported on these systems.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/041 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index 8568426311..7211f1950a 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -1123,4 +1123,5 @@ class TestOrphanedSource(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2', 'qed'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 supported_platforms=['linux', 'freebsd', 'openbsd6'])
-- 
2.18.1

[PATCH 3/5] iotests: Test 183 does not work on macOS and OpenBSD

[Thomas Huth]

When running 183 in Cirrus-CI on macOS, or with our vm-build-openbsd
target, it fails with an "Timeout waiting for return on handle 0" error.

Let's mark it as supported only on systems where the test is working
fine (i.e. Linux and FreeBSD).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/183 | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183
index bced83fae0..feeadb74c8 100755
--- a/tests/qemu-iotests/183
+++ b/tests/qemu-iotests/183
@@ -42,6 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 . ./common.qemu
 
+_supported_os Linux FreeBSD
 _supported_fmt qcow2 raw qed quorum
 _supported_proto file
 
-- 
2.18.1

[PATCH 4/5] iotests: Skip "make check-block" if QEMU does not support virtio-blk

[Thomas Huth]

The next patch is going to add some python-based tests to the "auto"
group, and these tests require virtio-blk to work properly. Running
iotests without virtio-blk likely does not make too much sense anyway,
so instead of adding a check for the availability of virtio-blk to each
and every test (which does not sound very appealing), let's rather add
a check for this at the top level in the check-block.sh script instead
(so that it is possible to run "make check" without the "check-block"
part for qemu-system-tricore for example).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/check-block.sh | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/tests/check-block.sh b/tests/check-block.sh
index 679aedec50..7582347ec2 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -26,10 +26,24 @@ if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
     exit 0
 fi
 
-if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+if [ -n "$QEMU_PROG" ]; then
+    qemu_prog="$QEMU_PROG"
+else
+    for binary in *-softmmu/qemu-system-* ; do
+        if [ -x "$binary" ]; then
+            qemu_prog="$binary"
+            break
+        fi
+    done
+fi
+if [ -z "$qemu_prog" ]; then
     echo "No qemu-system binary available ==> Not running the qemu-iotests."
     exit 0
 fi
+if ! "$qemu_prog" -M none -device help | grep virtio-blk >/dev/null 2>&1 ; then
+    echo "$qemu_prog does not support virtio-blk ==> Not running the qemu-iotests."
+    exit 0
+fi
 
 if ! command -v bash >/dev/null 2>&1 ; then
     echo "bash not available ==> Not running the qemu-iotests."
-- 
2.18.1

[PATCH 5/5] iotests: Enable more tests in the 'auto' group to improve test coverage

[Thomas Huth]

According to Kevin, tests 030, 040 and 041 are among the most valuable
tests that we have, so we should always run them if possible, even if
they take a little bit longer.

According to Max, it would be good to have a test for iothreads and
migration. 127 and 256 seem to be good candidates for iothreads. For
migration, let's enable 091, 181, 183, and 203 (which also tests
iothreads).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qemu-iotests/group | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 5805a79d9e..fa0d6143b0 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -51,7 +51,7 @@
 027 rw auto quick
 028 rw backing quick
 029 rw auto quick
-030 rw backing
+030 rw auto backing
 031 rw auto quick
 032 rw auto quick
 033 rw auto quick
@@ -61,8 +61,8 @@
 037 rw auto backing quick
 038 rw auto backing quick
 039 rw auto quick
-040 rw
-041 rw backing
+040 rw auto
+041 rw auto backing
 042 rw auto quick
 043 rw auto backing
 044 rw
@@ -112,7 +112,7 @@
 088 rw quick
 089 rw auto quick
 090 rw auto quick
-091 rw migration
+091 rw auto migration
 092 rw quick
 093 throttle
 094 rw quick
@@ -148,7 +148,7 @@
 124 rw backing
 125 rw
 126 rw auto backing
-127 rw backing quick
+127 rw auto backing quick
 128 rw quick
 129 rw quick
 130 rw auto quick
@@ -197,9 +197,9 @@
 177 rw auto quick
 178 img
 179 rw auto quick
-181 rw migration
+181 rw auto migration
 182 rw quick
-183 rw migration
+183 rw auto migration
 184 rw auto quick
 185 rw
 186 rw auto
@@ -218,7 +218,7 @@
 200 rw
 201 rw migration
 202 rw quick
-203 rw migration
+203 rw auto migration
 204 rw quick
 205 rw quick
 206 rw
@@ -270,7 +270,7 @@
 253 rw quick
 254 rw backing quick
 255 rw quick
-256 rw quick
+256 rw auto quick
 257 rw
 258 rw quick
 262 rw quick migration
-- 
2.18.1

Re: [PATCH 4/5] iotests: Skip "make check-block" if QEMU does not support virtio-blk

[Kevin Wolf]

Am 11.10.2019 um 16:50 hat Thomas Huth geschrieben:
> The next patch is going to add some python-based tests to the "auto"
> group, and these tests require virtio-blk to work properly. Running
> iotests without virtio-blk likely does not make too much sense anyway,
> so instead of adding a check for the availability of virtio-blk to each
> and every test (which does not sound very appealing), let's rather add
> a check for this at the top level in the check-block.sh script instead
> (so that it is possible to run "make check" without the "check-block"
> part for qemu-system-tricore for example).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/check-block.sh | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/check-block.sh b/tests/check-block.sh
> index 679aedec50..7582347ec2 100755
> --- a/tests/check-block.sh
> +++ b/tests/check-block.sh
> @@ -26,10 +26,24 @@ if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
>      exit 0
>  fi
>  
> -if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> +if [ -n "$QEMU_PROG" ]; then
> +    qemu_prog="$QEMU_PROG"
> +else
> +    for binary in *-softmmu/qemu-system-* ; do
> +        if [ -x "$binary" ]; then
> +            qemu_prog="$binary"
> +            break
> +        fi

Wouldn't it be better to check the availability of virtio-blk here, so
that if the current binary doesn't support it, we keep searching and
maybe pick up a different binary that supports it?

Or actually, should we work with a whitelist? We already need separate
code for s390 and x86_64 in some places to choose between -pci and -ccw,
and the presence of some virtio-blk doesn't mean that we know the
specifics of how to add a virtio-blk device for this target. This
suggests that blindly using a random binary might not be possible, but
tests may have to be adapted before the target can be whitelisted.

Kevin

> +    done
> +fi
> +if [ -z "$qemu_prog" ]; then
>      echo "No qemu-system binary available ==> Not running the qemu-iotests."
>      exit 0
>  fi
> +if ! "$qemu_prog" -M none -device help | grep virtio-blk >/dev/null 2>&1 ; then
> +    echo "$qemu_prog does not support virtio-blk ==> Not running the qemu-iotests."
> +    exit 0
> +fi
>  
>  if ! command -v bash >/dev/null 2>&1 ; then
>      echo "bash not available ==> Not running the qemu-iotests."
> -- 
> 2.18.1
> 

Re: [PATCH 4/5] iotests: Skip "make check-block" if QEMU does not support virtio-blk

[Thomas Huth]

On 14/10/2019 13.21, Kevin Wolf wrote:
> Am 11.10.2019 um 16:50 hat Thomas Huth geschrieben:
>> The next patch is going to add some python-based tests to the "auto"
>> group, and these tests require virtio-blk to work properly. Running
>> iotests without virtio-blk likely does not make too much sense anyway,
>> so instead of adding a check for the availability of virtio-blk to each
>> and every test (which does not sound very appealing), let's rather add
>> a check for this at the top level in the check-block.sh script instead
>> (so that it is possible to run "make check" without the "check-block"
>> part for qemu-system-tricore for example).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/check-block.sh | 16 +++++++++++++++-
>>  1 file changed, 15 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/check-block.sh b/tests/check-block.sh
>> index 679aedec50..7582347ec2 100755
>> --- a/tests/check-block.sh
>> +++ b/tests/check-block.sh
>> @@ -26,10 +26,24 @@ if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
>>      exit 0
>>  fi
>>  
>> -if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
>> +if [ -n "$QEMU_PROG" ]; then
>> +    qemu_prog="$QEMU_PROG"
>> +else
>> +    for binary in *-softmmu/qemu-system-* ; do
>> +        if [ -x "$binary" ]; then
>> +            qemu_prog="$binary"
>> +            break
>> +        fi
> 
> Wouldn't it be better to check the availability of virtio-blk here, so
> that if the current binary doesn't support it, we keep searching and
> maybe pick up a different binary that supports it?

That's a good idea, indeed, but then I also need to adjust the code in
the "check" script accordingly.

> Or actually, should we work with a whitelist?

I don't think that a hard-coded list will work well: Since we introduced
the Kconfig build system, it's now possible for example to also build an
qemu-system-aarch64 binary that does not contain any of the boards that
support virtio. So while virtio-blk is available by default in
qemu-system-aarch64, some builds might not contain it.

 Thomas

Re: [PATCH 4/5] iotests: Skip "make check-block" if QEMU does not support virtio-blk

[Kevin Wolf]

Am 14.10.2019 um 13:27 hat Thomas Huth geschrieben:
> On 14/10/2019 13.21, Kevin Wolf wrote:
> > Am 11.10.2019 um 16:50 hat Thomas Huth geschrieben:
> >> The next patch is going to add some python-based tests to the "auto"
> >> group, and these tests require virtio-blk to work properly. Running
> >> iotests without virtio-blk likely does not make too much sense anyway,
> >> so instead of adding a check for the availability of virtio-blk to each
> >> and every test (which does not sound very appealing), let's rather add
> >> a check for this at the top level in the check-block.sh script instead
> >> (so that it is possible to run "make check" without the "check-block"
> >> part for qemu-system-tricore for example).
> >>
> >> Signed-off-by: Thomas Huth <thuth@redhat.com>
> >> ---
> >>  tests/check-block.sh | 16 +++++++++++++++-
> >>  1 file changed, 15 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/tests/check-block.sh b/tests/check-block.sh
> >> index 679aedec50..7582347ec2 100755
> >> --- a/tests/check-block.sh
> >> +++ b/tests/check-block.sh
> >> @@ -26,10 +26,24 @@ if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
> >>      exit 0
> >>  fi
> >>  
> >> -if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> >> +if [ -n "$QEMU_PROG" ]; then
> >> +    qemu_prog="$QEMU_PROG"
> >> +else
> >> +    for binary in *-softmmu/qemu-system-* ; do
> >> +        if [ -x "$binary" ]; then
> >> +            qemu_prog="$binary"
> >> +            break
> >> +        fi
> > 
> > Wouldn't it be better to check the availability of virtio-blk here, so
> > that if the current binary doesn't support it, we keep searching and
> > maybe pick up a different binary that supports it?
> 
> That's a good idea, indeed, but then I also need to adjust the code in
> the "check" script accordingly.

I see. If this just copies the logic that qemu-iotests already uses, I
think I would be okay with taking it as is for now.

> > Or actually, should we work with a whitelist?
> 
> I don't think that a hard-coded list will work well: Since we introduced
> the Kconfig build system, it's now possible for example to also build an
> qemu-system-aarch64 binary that does not contain any of the boards that
> support virtio. So while virtio-blk is available by default in
> qemu-system-aarch64, some builds might not contain it.

Hm, good point. I'm just worried that the default config will end up
running the tests on a machine type where it works, but if you use the
wrong set of configure options, you end up with a setup where 'make
check' fails because it uses a machine type that the iotests don't
support.

Kevin

Re: [PATCH 1/5] iotests: remove 'linux' from default supported platforms

[Max Reitz]

[-- Attachment #1.1: Type: text/plain, Size: 1214 bytes --]

On 11.10.19 16:50, Thomas Huth wrote:
> From: John Snow <jsnow@redhat.com>
> 
> verify_platform will check an explicit whitelist and blacklist instead.
> The default will now be assumed to be allowed to run anywhere.
> 
> For tests that do not specify their platforms explicitly, this has the effect of
> enabling these tests on non-linux platforms. For tests that always specified
> linux explicitly, there is no change.
> 
> For Python tests on FreeBSD at least; only seven python tests fail:
> 045 147 149 169 194 199 211
> 
> 045 and 149 appear to be misconfigurations,
> 147 and 194 are the AF_UNIX path too long error,
> 169 and 199 are bitmap migration bugs, and
> 211 is a bug that shows up on Linux platforms, too.
> 
> This is at least good evidence that these tests are not Linux-only. If
> they aren't suitable for other platforms, they should be disabled on a
> per-platform basis as appropriate.
> 
> Therefore, let's switch these on and deal with the failures.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  tests/qemu-iotests/iotests.py | 16 +++++++++++-----
>  1 file changed, 11 insertions(+), 5 deletions(-)

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/5] iotests: Test 041 does not work on macOS

[Max Reitz]

[-- Attachment #1.1: Type: text/plain, Size: 1272 bytes --]

On 11.10.19 16:50, Thomas Huth wrote:
> 041 works fine on Linux, FreeBSD and OpenBSD, so let's mark it as
> only supported on these systems.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/qemu-iotests/041 | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
> index 8568426311..7211f1950a 100755
> --- a/tests/qemu-iotests/041
> +++ b/tests/qemu-iotests/041
> @@ -1123,4 +1123,5 @@ class TestOrphanedSource(iotests.QMPTestCase):
>  
>  if __name__ == '__main__':
>      iotests.main(supported_fmts=['qcow2', 'qed'],
> -                 supported_protocols=['file'])
> +                 supported_protocols=['file'],
> +                 supported_platforms=['linux', 'freebsd', 'openbsd6'])

Hmm, why not just “openbsd”, or single out macOS as unsupported?  (I
suppose the latter would require an additional patch.)

There’s also the fact that I maybe wanted to let make check skip all
tests on macOS and Windows – though that doesn’t make this patch
unnecessary, because that would only fix make check, not running the
iotests explicitly.

And I suppose maybe we even want to run the tests in macOS if it’s part
of the CI anyway?

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 4/5] iotests: Skip "make check-block" if QEMU does not support virtio-blk

[Max Reitz]

[-- Attachment #1.1: Type: text/plain, Size: 1639 bytes --]

On 11.10.19 16:50, Thomas Huth wrote:
> The next patch is going to add some python-based tests to the "auto"
> group, and these tests require virtio-blk to work properly. Running
> iotests without virtio-blk likely does not make too much sense anyway,
> so instead of adding a check for the availability of virtio-blk to each
> and every test (which does not sound very appealing), let's rather add
> a check for this at the top level in the check-block.sh script instead
> (so that it is possible to run "make check" without the "check-block"
> part for qemu-system-tricore for example).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/check-block.sh | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/check-block.sh b/tests/check-block.sh
> index 679aedec50..7582347ec2 100755
> --- a/tests/check-block.sh
> +++ b/tests/check-block.sh
> @@ -26,10 +26,24 @@ if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
>      exit 0
>  fi
>  
> -if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> +if [ -n "$QEMU_PROG" ]; then
> +    qemu_prog="$QEMU_PROG"
> +else
> +    for binary in *-softmmu/qemu-system-* ; do
> +        if [ -x "$binary" ]; then
> +            qemu_prog="$binary"
> +            break
> +        fi
> +    done
> +fi
> +if [ -z "$qemu_prog" ]; then
>      echo "No qemu-system binary available ==> Not running the qemu-iotests."
>      exit 0
>  fi
> +if ! "$qemu_prog" -M none -device help | grep virtio-blk >/dev/null 2>&1 ; then

Maybe grep -q?

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 5/5] iotests: Enable more tests in the 'auto' group to improve test coverage

[Max Reitz]

[-- Attachment #1.1: Type: text/plain, Size: 820 bytes --]

On 11.10.19 16:50, Thomas Huth wrote:
> According to Kevin, tests 030, 040 and 041 are among the most valuable
> tests that we have, so we should always run them if possible, even if
> they take a little bit longer.
> 
> According to Max, it would be good to have a test for iothreads and
> migration. 127 and 256 seem to be good candidates for iothreads. For
> migration, let's enable 091, 181, 183, and 203 (which also tests
> iothreads).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/qemu-iotests/group | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)

I’m not sure whether I’ve recently seen intermittent failures in one of
91, 181, or 183, so I’ll have to look into that again.

In the meantime:

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/5] iotests: Test 041 does not work on macOS

[Thomas Huth]

[-- Attachment #1.1: Type: text/plain, Size: 1789 bytes --]

On 18/10/2019 18.19, Max Reitz wrote:
> On 11.10.19 16:50, Thomas Huth wrote:
>> 041 works fine on Linux, FreeBSD and OpenBSD, so let's mark it as
>> only supported on these systems.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/qemu-iotests/041 | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
>> index 8568426311..7211f1950a 100755
>> --- a/tests/qemu-iotests/041
>> +++ b/tests/qemu-iotests/041
>> @@ -1123,4 +1123,5 @@ class TestOrphanedSource(iotests.QMPTestCase):
>>  
>>  if __name__ == '__main__':
>>      iotests.main(supported_fmts=['qcow2', 'qed'],
>> -                 supported_protocols=['file'])
>> +                 supported_protocols=['file'],
>> +                 supported_platforms=['linux', 'freebsd', 'openbsd6'])
> 
> Hmm, why not just “openbsd”, or single out macOS as unsupported?  (I
> suppose the latter would require an additional patch.)

Ah, right, John's patch uses "startswith(x)", so just using 'openbsd'
should work fine, too. (I initially thought that I have to use the exact
string here)

> There’s also the fact that I maybe wanted to let make check skip all
> tests on macOS and Windows – though that doesn’t make this patch
> unnecessary, because that would only fix make check, not running the
> iotests explicitly.
> 
> And I suppose maybe we even want to run the tests in macOS if it’s part
> of the CI anyway?

At least the current set of tests in the "auto" group works fine on
macOS (running on Cirrus-CI and Travis), so I think we should keep it
enabled in the CI for now (until it creates too much trouble and nobody
is able to debug it ... but we're not at that point yet).

 Thomas


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]