[PATCH 0/6] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86

[PATCH 0/6] tests/acceptance/virtio_seg_max_adjust: Restrict it to Linux/X86

[Philippe Mathieu-Daudé]

Currently our CI is mostly restricted to Linux/X86.
We want to have more, but until there we have to run these tests
manually, which is why we didn't noticed earlier this test was
failing on OSX.

Actually this test might have catched a bug, see:
https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg05119.html

This series contains the patches I used to debug the problem,
and few fixes while manually testing on heterogeneous environments.

Note that I have no idea what virtio/seg_max_adjust is used for,
it just happened to broke my Avocado queue while rebasing it on
a more recent /master.

Tested on Linux/FreeBSD/OSX, X86/Aarch64.

Regards,

Phil.

Philippe Mathieu-Daudé (6):
  tests/acceptance/virtio_seg_max_adjust: Only remove listed machines
  tests/acceptance/virtio_seg_max_adjust: Improve exception logging
  tests/acceptance/virtio_seg_max_adjust: List machine being tested
  tests/acceptance/virtio_seg_max_adjust: Only test Xen as superuser
  tests/acceptance/virtio_seg_max_adjust: Restrict to X86 architecture
  tests/acceptance/virtio_seg_max_adjust: Restrict to Linux host

 tests/acceptance/virtio_seg_max_adjust.py | 37 +++++++++++++++++++----
 1 file changed, 31 insertions(+), 6 deletions(-)

-- 
2.21.1

[PATCH 1/6] tests/acceptance/virtio_seg_max_adjust: Only remove listed machines

[Philippe Mathieu-Daudé]

Do not remove unavailable machines, this fixes:

  VirtioMaxSegSettingsCheck.test_machine_types: ERROR: list.remove(x): x not in list (0.12 s)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index 5458573138..4a417b8ef5 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -109,14 +109,15 @@ class VirtioMaxSegSettingsCheck(Test):
         return False
 
     def test_machine_types(self):
-        # collect all machine types except 'none', 'isapc', 'microvm'
+        EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
+        # collect all machine types except the ones in EXCLUDED_MACHINES
         with QEMUMachine(self.qemu_bin) as vm:
             vm.launch()
             machines = [m['name'] for m in vm.command('query-machines')]
             vm.shutdown()
-        machines.remove('none')
-        machines.remove('isapc')
-        machines.remove('microvm')
+        for m in EXCLUDED_MACHINES:
+            if m in machines:
+                machines.remove(m)
 
         for dev_type in DEV_TYPES:
             # create the list of machine types and their parameters.
-- 
2.21.1

[PATCH 2/6] tests/acceptance/virtio_seg_max_adjust: Improve exception logging

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index 4a417b8ef5..51a2dd76e8 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -77,8 +77,12 @@ class VirtioMaxSegSettingsCheck(Test):
             vm.set_machine(mt["name"])
             for s in VM_DEV_PARAMS[dev_type_name]:
                 vm.add_args(s)
-            vm.launch()
-            query_ok, props, error = self.query_virtqueue(vm, dev_type_name)
+            try:
+                vm.launch()
+                query_ok, props, error = self.query_virtqueue(vm, dev_type_name)
+            except:
+                query_ok = False
+                error = sys.exc_info()[0]
 
         if not query_ok:
             self.fail('machine type {0}: {1}'.format(mt['name'], error))
-- 
2.21.1

[PATCH 3/6] tests/acceptance/virtio_seg_max_adjust: List machine being tested

[Philippe Mathieu-Daudé]

Add logging for easier debugging of failures:

  $ avocado --show=machine run tests/acceptance/virtio_seg_max_adjust.py
   (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types:
  machine: {'name': 'pc-i440fx-2.12', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-2.0', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-q35-4.2', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-2.5', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-4.2', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  ...

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index 51a2dd76e8..f679b0eec7 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -21,6 +21,7 @@
 import sys
 import os
 import re
+import logging
 
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu.machine import QEMUMachine
@@ -73,6 +74,9 @@ class VirtioMaxSegSettingsCheck(Test):
         return query_ok, props, error
 
     def check_mt(self, mt, dev_type_name):
+        mt['device'] = dev_type_name # Only for the debug() call.
+        logger = logging.getLogger('machine')
+        logger.debug(mt)
         with QEMUMachine(self.qemu_bin) as vm:
             vm.set_machine(mt["name"])
             for s in VM_DEV_PARAMS[dev_type_name]:
-- 
2.21.1

[PATCH 4/6] tests/acceptance/virtio_seg_max_adjust: Only test Xen as superuser

[Philippe Mathieu-Daudé]

When running the test unprivileged, we get:

  $ avocado --show=app,machine run tests/acceptance/virtio_seg_max_adjust.py
  JOB ID     : b631d5d692e49b791b211d33b80730315d561d45
  JOB LOG    : job-results/job-2020-01-22T17.56-b631d5d/job.log
   (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types:
  machine: {'name': 'pc-i440fx-2.12', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'pc-i440fx-2.0', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  machine: {'name': 'xenpv', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
  FAIL: machine type xenpv: <class 'qemu.qmp.QMPConnectError'> (0.40 s)

Looking at the job.log file we find:

  xencall: error: Could not obtain handle on privileged command interface: No such file or directory
  xen be core: xen be core: can't open xen interface

Do not run this test on Xen machines if not superuser.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index f679b0eec7..ad736bcda3 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -118,6 +118,8 @@ class VirtioMaxSegSettingsCheck(Test):
 
     def test_machine_types(self):
         EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
+        if os.geteuid() != 0:
+            EXCLUDED_MACHINES += ['xenfv', 'xenpv']
         # collect all machine types except the ones in EXCLUDED_MACHINES
         with QEMUMachine(self.qemu_bin) as vm:
             vm.launch()
-- 
2.21.1

[PATCH 5/6] tests/acceptance/virtio_seg_max_adjust: Restrict to X86 architecture

[Philippe Mathieu-Daudé]

Running on mainstream KVM architectures, we get:

- Aarch64

  Timeout.

  job.log:
  -------
  No machine specified, and there is no default
  Use -machine help to list supported machines

- MIPS:

   (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: argument of type 'NoneType' is not iterable (0.14 s)

  job.log:
  -------
  Could not load MIPS bios 'mipsel_bios.bin', and no -kernel argument was specified

- PowerPC

   (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'sxxm' (0.16 s)

  job.log:
  -------
  >>> {'execute': 'query-machines'}
  <<< {'return': [{'hotpluggable-cpus': True, 'name': 'pseries-2.12-sxxm', 'numa-mem-supported': True, 'default-cpu-type': 'power8_v2.0-powerpc64-cpu', 'cpu-max': 1024, 'deprecated': False}, ...

- S390X:

   (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'virtio' (0.14 s)

  job.log:
  -------
  Traceback (most recent call last):
    File "virtio_seg_max_adjust.py", line 139, in test_machine_types
      if self.seg_max_adjust_enabled(m):
    File "virtio_seg_max_adjust.py", line 113, in seg_max_adjust_enabled
      major = int(ver[0])
  ValueError: invalid literal for int() with base 10: 'virtio'
  >>> {'execute': 'query-machines'}
  <<< {'return': [{'hotpluggable-cpus': True, 'name': 's390-ccw-virtio-4.0', 'numa-mem-supported': False, 'default-cpu-type': 'qemu-s390x-cpu', 'cpu-max': 248, 'deprecated': False}, ...

Assuming this test is only expected to run on the X86 architecture,
restrict the test to this particular architecture.

When this test is run on other architecture, the tests will be skipped.

Examples:

- running on S390X:

 (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: SKIP: Architecture 's390' unsupported

- running on Aarch64 setting the QEMU binary path:

  $ uname -m && avocado --show=app run -p qemu_bin=x86_64-softmmu/qemu-system-x86_64 tests/acceptance/virtio_seg_max_adjust.py
  aarch64
  JOB ID     : 92b7fae8868920aada0cb143f9571dffdf60931d
  JOB LOG    : job-results/job-2020-01-22T17.54-92b7fae/job.log
   (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: PASS (25.99 s)
  RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 26.13 s

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index ad736bcda3..2fc6bfcbd8 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -26,6 +26,7 @@ import logging
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu.machine import QEMUMachine
 from avocado_qemu import Test
+from avocado.core.exceptions import TestSkipError
 
 #list of machine types and virtqueue properties to test
 VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
@@ -117,12 +118,22 @@ class VirtioMaxSegSettingsCheck(Test):
         return False
 
     def test_machine_types(self):
+        """
+        :avocado: tags=arch:i386
+        :avocado: tags=arch:x86_64
+        """
         EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
         if os.geteuid() != 0:
             EXCLUDED_MACHINES += ['xenfv', 'xenpv']
         # collect all machine types except the ones in EXCLUDED_MACHINES
         with QEMUMachine(self.qemu_bin) as vm:
             vm.launch()
+            # Skip test if target is not X86
+            # TODO: Move this check to Avocado (based on the test tags)
+            target_arch = vm.command('query-target')['arch']
+            if target_arch not in ['i386', 'x86_64']:
+                errmsg = "Architecture '%s' unsupported" % target_arch
+                raise TestSkipError(errmsg)
             machines = [m['name'] for m in vm.command('query-machines')]
             vm.shutdown()
         for m in EXCLUDED_MACHINES:
-- 
2.21.1

[PATCH 6/6] tests/acceptance/virtio_seg_max_adjust: Restrict to Linux host

[Philippe Mathieu-Daudé]

This test is failing on OSX:

  TestFail: machine type pc-i440fx-2.0: <class 'TypeError'>

Looking at the job.log:

  Unexpected error in object_property_find() at qom/object.c:1201:
  qemu-system-x86_64: -device virtio-blk-pci,id=scsi0,drive=drive0: can't apply global virtio-blk-device.scsi=true: Property '.scsi' not found

Which makes sense looking at hw/block/virtio-blk.c:

  1261 static Property virtio_blk_properties[] = {
  1262     DEFINE_BLOCK_PROPERTIES(VirtIOBlock, conf.conf),
  ...
  1268 #ifdef __linux__
  1269     DEFINE_PROP_BIT64("scsi", VirtIOBlock, host_features,
  1270                       VIRTIO_BLK_F_SCSI, false),
  1271 #endif

Fix by only running this test on a Linux host:

 (38/55) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types:  SKIP: Host OS is not Linux

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/virtio_seg_max_adjust.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
index 2fc6bfcbd8..f8fd03e3b6 100755
--- a/tests/acceptance/virtio_seg_max_adjust.py
+++ b/tests/acceptance/virtio_seg_max_adjust.py
@@ -27,6 +27,7 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu.machine import QEMUMachine
 from avocado_qemu import Test
 from avocado.core.exceptions import TestSkipError
+from avocado import skipUnless
 
 #list of machine types and virtqueue properties to test
 VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
@@ -117,6 +118,8 @@ class VirtioMaxSegSettingsCheck(Test):
             return True
         return False
 
+
+    @skipUnless(sys.platform.startswith('linux'), 'Host OS is not Linux')
     def test_machine_types(self):
         """
         :avocado: tags=arch:i386
-- 
2.21.1

Re: [PATCH 1/6] tests/acceptance/virtio_seg_max_adjust: Only remove listed machines

[Cornelia Huck]

On Wed, 22 Jan 2020 23:32:42 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> Do not remove unavailable machines, this fixes:
> 
>   VirtioMaxSegSettingsCheck.test_machine_types: ERROR: list.remove(x): x not in list (0.12 s)
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/virtio_seg_max_adjust.py | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
> index 5458573138..4a417b8ef5 100755
> --- a/tests/acceptance/virtio_seg_max_adjust.py
> +++ b/tests/acceptance/virtio_seg_max_adjust.py
> @@ -109,14 +109,15 @@ class VirtioMaxSegSettingsCheck(Test):
>          return False
>  
>      def test_machine_types(self):
> -        # collect all machine types except 'none', 'isapc', 'microvm'
> +        EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']

That one seems more flexible as well.

> +        # collect all machine types except the ones in EXCLUDED_MACHINES
>          with QEMUMachine(self.qemu_bin) as vm:
>              vm.launch()
>              machines = [m['name'] for m in vm.command('query-machines')]
>              vm.shutdown()
> -        machines.remove('none')
> -        machines.remove('isapc')
> -        machines.remove('microvm')
> +        for m in EXCLUDED_MACHINES:
> +            if m in machines:
> +                machines.remove(m)
>  
>          for dev_type in DEV_TYPES:
>              # create the list of machine types and their parameters.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>

Re: [PATCH 4/6] tests/acceptance/virtio_seg_max_adjust: Only test Xen as superuser

[Cornelia Huck]

On Wed, 22 Jan 2020 23:32:45 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

$SUBJECT: s/Only test Xen as superuser/Test Xen only as superuser/ ?

> When running the test unprivileged, we get:
> 
>   $ avocado --show=app,machine run tests/acceptance/virtio_seg_max_adjust.py
>   JOB ID     : b631d5d692e49b791b211d33b80730315d561d45
>   JOB LOG    : job-results/job-2020-01-22T17.56-b631d5d/job.log
>    (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types:
>   machine: {'name': 'pc-i440fx-2.12', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
>   machine: {'name': 'pc-i440fx-2.0', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
>   machine: {'name': 'xenpv', 'seg_max_adjust': 'false', 'device': 'virtio-scsi-pci'}
>   FAIL: machine type xenpv: <class 'qemu.qmp.QMPConnectError'> (0.40 s)
> 
> Looking at the job.log file we find:
> 
>   xencall: error: Could not obtain handle on privileged command interface: No such file or directory
>   xen be core: xen be core: can't open xen interface
> 
> Do not run this test on Xen machines if not superuser.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/virtio_seg_max_adjust.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
> index f679b0eec7..ad736bcda3 100755
> --- a/tests/acceptance/virtio_seg_max_adjust.py
> +++ b/tests/acceptance/virtio_seg_max_adjust.py
> @@ -118,6 +118,8 @@ class VirtioMaxSegSettingsCheck(Test):
>  
>      def test_machine_types(self):
>          EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
> +        if os.geteuid() != 0:
> +            EXCLUDED_MACHINES += ['xenfv', 'xenpv']
>          # collect all machine types except the ones in EXCLUDED_MACHINES
>          with QEMUMachine(self.qemu_bin) as vm:
>              vm.launch()

Acked-by: Cornelia Huck <cohuck@redhat.com>

Re: [PATCH 5/6] tests/acceptance/virtio_seg_max_adjust: Restrict to X86 architecture

[Cornelia Huck]

On Wed, 22 Jan 2020 23:32:46 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> Running on mainstream KVM architectures, we get:
> 
> - Aarch64
> 
>   Timeout.
> 
>   job.log:
>   -------
>   No machine specified, and there is no default
>   Use -machine help to list supported machines

The code probably needs to be made more clever to find the machines to
run?

> 
> - MIPS:
> 
>    (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: argument of type 'NoneType' is not iterable (0.14 s)
> 
>   job.log:
>   -------
>   Could not load MIPS bios 'mipsel_bios.bin', and no -kernel argument was specified

Probably needs some hint from mips folks how this can be set up.

> 
> - PowerPC
> 
>    (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'sxxm' (0.16 s)
> 
>   job.log:
>   -------
>   >>> {'execute': 'query-machines'}  
>   <<< {'return': [{'hotpluggable-cpus': True, 'name': 'pseries-2.12-sxxm', 'numa-mem-supported': True, 'default-cpu-type': 'power8_v2.0-powerpc64-cpu', 'cpu-max': 1024, 'deprecated': False}, ...

This seems to be because the machine type parsing code cannot deal with
the format used here.

> 
> - S390X:
> 
>    (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'virtio' (0.14 s)
> 
>   job.log:
>   -------
>   Traceback (most recent call last):
>     File "virtio_seg_max_adjust.py", line 139, in test_machine_types
>       if self.seg_max_adjust_enabled(m):
>     File "virtio_seg_max_adjust.py", line 113, in seg_max_adjust_enabled
>       major = int(ver[0])
>   ValueError: invalid literal for int() with base 10: 'virtio'
>   >>> {'execute': 'query-machines'}  
>   <<< {'return': [{'hotpluggable-cpus': True, 'name': 's390-ccw-virtio-4.0', 'numa-mem-supported': False, 'default-cpu-type': 'qemu-s390x-cpu', 'cpu-max': 248, 'deprecated': False}, ...

Same here.

> 
> Assuming this test is only expected to run on the X86 architecture,
> restrict the test to this particular architecture.
> 
> When this test is run on other architecture, the tests will be skipped.
> 
> Examples:
> 
> - running on S390X:
> 
>  (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: SKIP: Architecture 's390' unsupported
> 
> - running on Aarch64 setting the QEMU binary path:
> 
>   $ uname -m && avocado --show=app run -p qemu_bin=x86_64-softmmu/qemu-system-x86_64 tests/acceptance/virtio_seg_max_adjust.py
>   aarch64
>   JOB ID     : 92b7fae8868920aada0cb143f9571dffdf60931d
>   JOB LOG    : job-results/job-2020-01-22T17.54-92b7fae/job.log
>    (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: PASS (25.99 s)
>   RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>   JOB TIME   : 26.13 s
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/virtio_seg_max_adjust.py | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
> index ad736bcda3..2fc6bfcbd8 100755
> --- a/tests/acceptance/virtio_seg_max_adjust.py
> +++ b/tests/acceptance/virtio_seg_max_adjust.py
> @@ -26,6 +26,7 @@ import logging
>  sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
>  from qemu.machine import QEMUMachine
>  from avocado_qemu import Test
> +from avocado.core.exceptions import TestSkipError
>  
>  #list of machine types and virtqueue properties to test
>  VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
> @@ -117,12 +118,22 @@ class VirtioMaxSegSettingsCheck(Test):
>          return False
>  
>      def test_machine_types(self):
> +        """
> +        :avocado: tags=arch:i386
> +        :avocado: tags=arch:x86_64
> +        """
>          EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
>          if os.geteuid() != 0:
>              EXCLUDED_MACHINES += ['xenfv', 'xenpv']
>          # collect all machine types except the ones in EXCLUDED_MACHINES
>          with QEMUMachine(self.qemu_bin) as vm:
>              vm.launch()
> +            # Skip test if target is not X86
> +            # TODO: Move this check to Avocado (based on the test tags)
> +            target_arch = vm.command('query-target')['arch']
> +            if target_arch not in ['i386', 'x86_64']:
> +                errmsg = "Architecture '%s' unsupported" % target_arch
> +                raise TestSkipError(errmsg)

I think we should rather fix the machine parsing code, and only then
exclude architectures out of the box. (Sorry, my python-fu is lacking,
or I would try it myself.)

There does not seem to be anything that is really architecture specific
in there. Just explicitly requesting the -pci versions of virtio-blk
and virtio-scsi seems wrong, though, as this looks like a generic
property, and should work on -ccw as well. (And probably also for
virtio-mmio devices.) If not, I'd like to know :)

>              machines = [m['name'] for m in vm.command('query-machines')]
>              vm.shutdown()
>          for m in EXCLUDED_MACHINES:

Re: [PATCH 6/6] tests/acceptance/virtio_seg_max_adjust: Restrict to Linux host

[Cornelia Huck]

On Wed, 22 Jan 2020 23:32:47 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> This test is failing on OSX:
> 
>   TestFail: machine type pc-i440fx-2.0: <class 'TypeError'>
> 
> Looking at the job.log:
> 
>   Unexpected error in object_property_find() at qom/object.c:1201:
>   qemu-system-x86_64: -device virtio-blk-pci,id=scsi0,drive=drive0: can't apply global virtio-blk-device.scsi=true: Property '.scsi' not found
> 
> Which makes sense looking at hw/block/virtio-blk.c:
> 
>   1261 static Property virtio_blk_properties[] = {
>   1262     DEFINE_BLOCK_PROPERTIES(VirtIOBlock, conf.conf),
>   ...
>   1268 #ifdef __linux__
>   1269     DEFINE_PROP_BIT64("scsi", VirtIOBlock, host_features,
>   1270                       VIRTIO_BLK_F_SCSI, false),
>   1271 #endif
> 
> Fix by only running this test on a Linux host:
> 
>  (38/55) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types:  SKIP: Host OS is not Linux

Does this work with the change I proposed in the other thread? If it
does, I'd prefer to send a proper fix for that. (Well, I'd prefer to
fix it regardless.)

Or is there other Linux-specific stuff in there? Does not look like it
at a glance, we were just bitten by the unrelated scsi property thing.

> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/virtio_seg_max_adjust.py | 3 +++
>  1 file changed, 3 insertions(+)

Re: [PATCH 5/6] tests/acceptance/virtio_seg_max_adjust: Restrict to X86 architecture

[Wainer dos Santos Moschetta]

On 1/23/20 9:48 AM, Cornelia Huck wrote:
> On Wed, 22 Jan 2020 23:32:46 +0100
> Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
>> Running on mainstream KVM architectures, we get:
>>
>> - Aarch64
>>
>>    Timeout.
>>
>>    job.log:
>>    -------
>>    No machine specified, and there is no default
>>    Use -machine help to list supported machines
> The code probably needs to be made more clever to find the machines to
> run?
>
>> - MIPS:
>>
>>     (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: argument of type 'NoneType' is not iterable (0.14 s)
>>
>>    job.log:
>>    -------
>>    Could not load MIPS bios 'mipsel_bios.bin', and no -kernel argument was specified
> Probably needs some hint from mips folks how this can be set up.
>
>> - PowerPC
>>
>>     (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'sxxm' (0.16 s)
>>
>>    job.log:
>>    -------
>>    >>> {'execute': 'query-machines'}
>>    <<< {'return': [{'hotpluggable-cpus': True, 'name': 'pseries-2.12-sxxm', 'numa-mem-supported': True, 'default-cpu-type': 'power8_v2.0-powerpc64-cpu', 'cpu-max': 1024, 'deprecated': False}, ...
> This seems to be because the machine type parsing code cannot deal with
> the format used here.


Indeed, looking at the comments in code the parser was meant for PC types.

Is there a way to obtain the machine type version other than parsing its 
name? If not, wouldn't be useful for management apps have that 
information returned with 'query-machines'?

- Wainer

>> - S390X:
>>
>>     (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'virtio' (0.14 s)
>>
>>    job.log:
>>    -------
>>    Traceback (most recent call last):
>>      File "virtio_seg_max_adjust.py", line 139, in test_machine_types
>>        if self.seg_max_adjust_enabled(m):
>>      File "virtio_seg_max_adjust.py", line 113, in seg_max_adjust_enabled
>>        major = int(ver[0])
>>    ValueError: invalid literal for int() with base 10: 'virtio'
>>    >>> {'execute': 'query-machines'}
>>    <<< {'return': [{'hotpluggable-cpus': True, 'name': 's390-ccw-virtio-4.0', 'numa-mem-supported': False, 'default-cpu-type': 'qemu-s390x-cpu', 'cpu-max': 248, 'deprecated': False}, ...
> Same here.
>
>> Assuming this test is only expected to run on the X86 architecture,
>> restrict the test to this particular architecture.
>>
>> When this test is run on other architecture, the tests will be skipped.
>>
>> Examples:
>>
>> - running on S390X:
>>
>>   (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: SKIP: Architecture 's390' unsupported
>>
>> - running on Aarch64 setting the QEMU binary path:
>>
>>    $ uname -m && avocado --show=app run -p qemu_bin=x86_64-softmmu/qemu-system-x86_64 tests/acceptance/virtio_seg_max_adjust.py
>>    aarch64
>>    JOB ID     : 92b7fae8868920aada0cb143f9571dffdf60931d
>>    JOB LOG    : job-results/job-2020-01-22T17.54-92b7fae/job.log
>>     (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: PASS (25.99 s)
>>    RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>    JOB TIME   : 26.13 s
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>   tests/acceptance/virtio_seg_max_adjust.py | 11 +++++++++++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
>> index ad736bcda3..2fc6bfcbd8 100755
>> --- a/tests/acceptance/virtio_seg_max_adjust.py
>> +++ b/tests/acceptance/virtio_seg_max_adjust.py
>> @@ -26,6 +26,7 @@ import logging
>>   sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
>>   from qemu.machine import QEMUMachine
>>   from avocado_qemu import Test
>> +from avocado.core.exceptions import TestSkipError
>>   
>>   #list of machine types and virtqueue properties to test
>>   VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
>> @@ -117,12 +118,22 @@ class VirtioMaxSegSettingsCheck(Test):
>>           return False
>>   
>>       def test_machine_types(self):
>> +        """
>> +        :avocado: tags=arch:i386
>> +        :avocado: tags=arch:x86_64
>> +        """
>>           EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
>>           if os.geteuid() != 0:
>>               EXCLUDED_MACHINES += ['xenfv', 'xenpv']
>>           # collect all machine types except the ones in EXCLUDED_MACHINES
>>           with QEMUMachine(self.qemu_bin) as vm:
>>               vm.launch()
>> +            # Skip test if target is not X86
>> +            # TODO: Move this check to Avocado (based on the test tags)
>> +            target_arch = vm.command('query-target')['arch']
>> +            if target_arch not in ['i386', 'x86_64']:
>> +                errmsg = "Architecture '%s' unsupported" % target_arch
>> +                raise TestSkipError(errmsg)
> I think we should rather fix the machine parsing code, and only then
> exclude architectures out of the box. (Sorry, my python-fu is lacking,
> or I would try it myself.)
>
> There does not seem to be anything that is really architecture specific
> in there. Just explicitly requesting the -pci versions of virtio-blk
> and virtio-scsi seems wrong, though, as this looks like a generic
> property, and should work on -ccw as well. (And probably also for
> virtio-mmio devices.) If not, I'd like to know :)
>
>>               machines = [m['name'] for m in vm.command('query-machines')]
>>               vm.shutdown()
>>           for m in EXCLUDED_MACHINES:
>

Re: [PATCH 1/6] tests/acceptance/virtio_seg_max_adjust: Only remove listed machines

[Wainer dos Santos Moschetta]

Hi Philippe,

That fixes one of the problems I mention in another email thread. I was 
working on a fix, so putting my hands off it. :)

Anyway, see some comments below.

On 1/22/20 8:32 PM, Philippe Mathieu-Daudé wrote:
> Do not remove unavailable machines, this fixes:
>
>    VirtioMaxSegSettingsCheck.test_machine_types: ERROR: list.remove(x): x not in list (0.12 s)
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>   tests/acceptance/virtio_seg_max_adjust.py | 9 +++++----
>   1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
> index 5458573138..4a417b8ef5 100755
> --- a/tests/acceptance/virtio_seg_max_adjust.py
> +++ b/tests/acceptance/virtio_seg_max_adjust.py
> @@ -109,14 +109,15 @@ class VirtioMaxSegSettingsCheck(Test):
>           return False
>   
>       def test_machine_types(self):
> -        # collect all machine types except 'none', 'isapc', 'microvm'
> +        EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']

I was going to suggest moving this constant declaration alongside the 
others (VIRTIO_SCSI_PROPS, VIRTIO_BLK_PROPS...). But I saw on patch 04 
that this variable can get updated, i.e. no longer it is a constant, so 
I think it could be an object attribute instead. My reasoning is: it is 
easier to figure out what to change (eventually) if it is an object 
attribute or module constant.

Also if you want to make it further flexible, you can use Avocado's 
parameters. Example:

excluded_machines = self.params.get('exclude_machines', default=['none', 
'isapc', 'microvm'])

I hope this helps.

- Wainer

> +        # collect all machine types except the ones in EXCLUDED_MACHINES
>           with QEMUMachine(self.qemu_bin) as vm:
>               vm.launch()
>               machines = [m['name'] for m in vm.command('query-machines')]
>               vm.shutdown()
> -        machines.remove('none')
> -        machines.remove('isapc')
> -        machines.remove('microvm')
> +        for m in EXCLUDED_MACHINES:
> +            if m in machines:
> +                machines.remove(m)
>   
>           for dev_type in DEV_TYPES:
>               # create the list of machine types and their parameters.

Re: [PATCH 5/6] tests/acceptance/virtio_seg_max_adjust: Restrict to X86 architecture

[Cornelia Huck]

On Thu, 23 Jan 2020 11:36:55 -0200
Wainer dos Santos Moschetta <wainersm@redhat.com> wrote:

> On 1/23/20 9:48 AM, Cornelia Huck wrote:
> > On Wed, 22 Jan 2020 23:32:46 +0100
> > Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> >  
> >> Running on mainstream KVM architectures, we get:
> >>
> >> - Aarch64
> >>
> >>    Timeout.
> >>
> >>    job.log:
> >>    -------
> >>    No machine specified, and there is no default
> >>    Use -machine help to list supported machines  
> > The code probably needs to be made more clever to find the machines to
> > run?
> >  
> >> - MIPS:
> >>
> >>     (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: argument of type 'NoneType' is not iterable (0.14 s)
> >>
> >>    job.log:
> >>    -------
> >>    Could not load MIPS bios 'mipsel_bios.bin', and no -kernel argument was specified  
> > Probably needs some hint from mips folks how this can be set up.
> >  
> >> - PowerPC
> >>
> >>     (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'sxxm' (0.16 s)
> >>
> >>    job.log:
> >>    -------  
> >>    >>> {'execute': 'query-machines'}  
> >>    <<< {'return': [{'hotpluggable-cpus': True, 'name': 'pseries-2.12-sxxm', 'numa-mem-supported': True, 'default-cpu-type': 'power8_v2.0-powerpc64-cpu', 'cpu-max': 1024, 'deprecated': False}, ...  
> > This seems to be because the machine type parsing code cannot deal with
> > the format used here.  
> 
> 
> Indeed, looking at the comments in code the parser was meant for PC types.
> 
> Is there a way to obtain the machine type version other than parsing its 
> name? If not, wouldn't be useful for management apps have that 
> information returned with 'query-machines'?

The question would be "What version?". The pseries machine type the
code chokes upon above has an additional suffix. Not all machine types
are versioned, obviously, and we don't have an uniform way to build
machine type names, either.

If I understand this test correctly, the idea is to distinguish between
"version that defaults the feature to off" and "version that defaults
the feature to on". So, the pseries machine with the extra suffix from
above and the one without would basically be the same 2.12 compatible
machine (both in the first bucket). For other tests, those two machines
might be very different, though.

tl;dr "version" might not be well-defined enough.

> 
> - Wainer
> 
> >> - S390X:
> >>
> >>     (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'virtio' (0.14 s)
> >>
> >>    job.log:
> >>    -------
> >>    Traceback (most recent call last):
> >>      File "virtio_seg_max_adjust.py", line 139, in test_machine_types
> >>        if self.seg_max_adjust_enabled(m):
> >>      File "virtio_seg_max_adjust.py", line 113, in seg_max_adjust_enabled
> >>        major = int(ver[0])
> >>    ValueError: invalid literal for int() with base 10: 'virtio'  
> >>    >>> {'execute': 'query-machines'}  
> >>    <<< {'return': [{'hotpluggable-cpus': True, 'name': 's390-ccw-virtio-4.0', 'numa-mem-supported': False, 'default-cpu-type': 'qemu-s390x-cpu', 'cpu-max': 248, 'deprecated': False}, ...  
> > Same here.