* [PATCH 0/3] Acceptance tests: make better use of machine tags
@ 2019-09-24 19:44 Cleber Rosa
2019-09-24 19:44 ` [PATCH 1/3] Acceptance test x86_cpu_model_versions: use default vm Cleber Rosa
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Cleber Rosa @ 2019-09-24 19:44 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Aleksandar Rikalo, Wainer dos Santos Moschetta,
Willian Rampazzo, Cleber Rosa, Philippe Mathieu-Daudé,
Aurelien Jarno
The "arch" tags applied to acceptance tests, besides allowing a user
to select tests for one specific target architecture, also provides
the the feature of setting the arch to be used if none was given by a
parameter.
This series does the same thing for machine tags. If a test is tagged
with "machine", its value will be set on the QEMUMachine instances
created/managed by the test. Hopefully this will eliminate boiler
plate code, and make writing tests more straightforward.
Cleber Rosa (3):
Acceptance test x86_cpu_model_versions: use default vm
Acceptance tests: introduce utility method for tags unique vals
Acceptance Tests: use avocado tags for machine type
docs/devel/testing.rst | 18 +++
tests/acceptance/avocado_qemu/__init__.py | 24 +++-
tests/acceptance/boot_linux_console.py | 15 +--
tests/acceptance/cpu_queries.py | 2 +-
tests/acceptance/linux_initrd.py | 2 +-
tests/acceptance/linux_ssh_mips_malta.py | 5 -
tests/acceptance/machine_m68k_nextcube.py | 21 +---
tests/acceptance/x86_cpu_model_versions.py | 137 ++++++++++++---------
8 files changed, 122 insertions(+), 102 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/3] Acceptance test x86_cpu_model_versions: use default vm
2019-09-24 19:44 [PATCH 0/3] Acceptance tests: make better use of machine tags Cleber Rosa
@ 2019-09-24 19:44 ` Cleber Rosa
2019-09-24 19:45 ` [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals Cleber Rosa
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Cleber Rosa @ 2019-09-24 19:44 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Aleksandar Rikalo, Wainer dos Santos Moschetta,
Willian Rampazzo, Cleber Rosa, Philippe Mathieu-Daudé,
Aurelien Jarno
The default vm provided by the test, available as self.vm, serves the
same purpose of the one obtained by self.get_vm(), but saves a line
and matches the style of other tests.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
tests/acceptance/x86_cpu_model_versions.py | 100 ++++++++++-----------
1 file changed, 46 insertions(+), 54 deletions(-)
diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/acceptance/x86_cpu_model_versions.py
index 5fc9ca4bc6..6eb977954d 100644
--- a/tests/acceptance/x86_cpu_model_versions.py
+++ b/tests/acceptance/x86_cpu_model_versions.py
@@ -25,10 +25,6 @@
import avocado_qemu
import re
-def get_cpu_prop(vm, prop):
- cpu_path = vm.command('query-cpus')[0].get('qom_path')
- return vm.command('qom-get', path=cpu_path, property=prop)
-
class X86CPUModelAliases(avocado_qemu.Test):
"""
Validation of PC CPU model versions and CPU model aliases
@@ -241,78 +237,74 @@ class CascadelakeArchCapabilities(avocado_qemu.Test):
:avocado: tags=arch:x86_64
"""
+ def get_cpu_prop(self, prop):
+ cpu_path = self.vm.command('query-cpus')[0].get('qom_path')
+ return self.vm.command('qom-get', path=cpu_path, property=prop)
+
def test_4_1(self):
# machine-type only:
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.1')
- vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
- vm.launch()
- self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.1')
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
+ self.vm.launch()
+ self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
def test_4_0(self):
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.0')
- vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
- vm.launch()
- self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.0')
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
+ self.vm.launch()
+ self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
def test_set_4_0(self):
# command line must override machine-type if CPU model is not versioned:
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.0')
- vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
- vm.launch()
- self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.0')
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
+ self.vm.launch()
+ self.assertTrue(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
def test_unset_4_1(self):
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.1')
- vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
- vm.launch()
- self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.1')
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
+ self.vm.launch()
+ self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
def test_v1_4_0(self):
# versioned CPU model overrides machine-type:
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.0')
- vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
- vm.launch()
- self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.0')
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
+ self.vm.launch()
+ self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
def test_v2_4_0(self):
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.0')
- vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
- vm.launch()
- self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
- 'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.0')
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
+ self.vm.launch()
+ self.assertTrue(self.get_cpu_prop('arch-capabilities'),
+ 'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
def test_v1_set_4_0(self):
# command line must override machine-type and versioned CPU model:
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.0')
- vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
- vm.launch()
- self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
- 'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.0')
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
+ self.vm.launch()
+ self.assertTrue(self.get_cpu_prop('arch-capabilities'),
+ 'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
def test_v2_unset_4_1(self):
- vm = self.get_vm()
- vm.add_args('-S')
- vm.set_machine('pc-i440fx-4.1')
- vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off,-arch-capabilities')
- vm.launch()
- self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
+ self.vm.add_args('-S')
+ self.vm.set_machine('pc-i440fx-4.1')
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off,-arch-capabilities')
+ self.vm.launch()
+ self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server-v2,-arch-capabilities should not have arch-capabilities')
--
2.21.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals
2019-09-24 19:44 [PATCH 0/3] Acceptance tests: make better use of machine tags Cleber Rosa
2019-09-24 19:44 ` [PATCH 1/3] Acceptance test x86_cpu_model_versions: use default vm Cleber Rosa
@ 2019-09-24 19:45 ` Cleber Rosa
2019-10-24 21:12 ` Wainer dos Santos Moschetta
2019-09-24 19:45 ` [PATCH 3/3] Acceptance Tests: use avocado tags for machine type Cleber Rosa
2019-09-25 8:03 ` [PATCH 0/3] Acceptance tests: make better use of machine tags no-reply
3 siblings, 1 reply; 9+ messages in thread
From: Cleber Rosa @ 2019-09-24 19:45 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Aleksandar Rikalo, Wainer dos Santos Moschetta,
Willian Rampazzo, Cleber Rosa, Philippe Mathieu-Daudé,
Aurelien Jarno
Currently a test can describe the target architecture binary that it
should primarily be run with, be setting a single tag value.
The same approach is expected to be done with other QEMU aspects to be
tested, for instance, the machine type and accelerator, so let's
generalize the logic into a utility method.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
tests/acceptance/avocado_qemu/__init__.py | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index bd41e0443c..02775bafcf 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -54,14 +54,21 @@ def pick_default_qemu_bin(arch=None):
class Test(avocado.Test):
+ def _get_unique_tag_val(self, tag_name):
+ """
+ Gets a tag value, if unique for a key
+ """
+ vals = self.tags.get(tag_name, [])
+ if len(vals) == 1:
+ return vals.pop()
+ return None
+
def setUp(self):
self._vms = {}
- arches = self.tags.get('arch', [])
- if len(arches) == 1:
- arch = arches.pop()
- else:
- arch = None
- self.arch = self.params.get('arch', default=arch)
+
+ self.arch = self.params.get('arch',
+ default=self._get_unique_tag_val('arch'))
+
default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
self.qemu_bin = self.params.get('qemu_bin',
default=default_qemu_bin)
--
2.21.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 3/3] Acceptance Tests: use avocado tags for machine type
2019-09-24 19:44 [PATCH 0/3] Acceptance tests: make better use of machine tags Cleber Rosa
2019-09-24 19:44 ` [PATCH 1/3] Acceptance test x86_cpu_model_versions: use default vm Cleber Rosa
2019-09-24 19:45 ` [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals Cleber Rosa
@ 2019-09-24 19:45 ` Cleber Rosa
2019-10-24 21:24 ` Wainer dos Santos Moschetta
2019-09-25 8:03 ` [PATCH 0/3] Acceptance tests: make better use of machine tags no-reply
3 siblings, 1 reply; 9+ messages in thread
From: Cleber Rosa @ 2019-09-24 19:45 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Aleksandar Rikalo, Wainer dos Santos Moschetta,
Willian Rampazzo, Cleber Rosa, Philippe Mathieu-Daudé,
Aurelien Jarno
The same way the arch tag is being used as a fallback for the arch
parameter, let's do the same for QEMU's machine and avoid some boiler
plate code.
This requires a bump in the Avocado version, as starting with 72.0,
the characters supported in tags are less strict.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
docs/devel/testing.rst | 18 ++++++++
tests/acceptance/avocado_qemu/__init__.py | 5 ++
tests/acceptance/boot_linux_console.py | 15 +-----
tests/acceptance/cpu_queries.py | 2 +-
tests/acceptance/linux_initrd.py | 2 +-
tests/acceptance/linux_ssh_mips_malta.py | 5 --
tests/acceptance/machine_m68k_nextcube.py | 21 ++-------
tests/acceptance/x86_cpu_model_versions.py | 53 ++++++++++++++++------
8 files changed, 71 insertions(+), 50 deletions(-)
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index bf75675fb0..1816ada919 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -744,6 +744,17 @@ name. If one is not given explicitly, it will either be set to
``None``, or, if the test is tagged with one (and only one)
``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
+machine
+~~~~~~~
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+The ``machine`` attribute will be set to the test parameter of the same
+name. If one is not given explicitly, it will either be set to
+``None``, or, if the test is tagged with one (and only one)
+``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
+
qemu_bin
~~~~~~~~
@@ -779,6 +790,13 @@ architecture of a kernel or disk image to boot a VM with.
This parameter has a direct relation with the ``arch`` attribute. If
not given, it will default to None.
+machine
+~~~~~~~
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+
qemu_bin
~~~~~~~~
diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 02775bafcf..fb5d6616bc 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -69,6 +69,9 @@ class Test(avocado.Test):
self.arch = self.params.get('arch',
default=self._get_unique_tag_val('arch'))
+ self.machine = self.params.get('machine',
+ default=self._get_unique_tag_val('machine'))
+
default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
self.qemu_bin = self.params.get('qemu_bin',
default=default_qemu_bin)
@@ -90,6 +93,8 @@ class Test(avocado.Test):
name = str(uuid.uuid4())
if self._vms.get(name) is None:
self._vms[name] = self._new_vm(*args)
+ if self.machine is not None:
+ self._vms[name].set_machine(self.machine)
return self._vms[name]
def tearDown(self):
diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 8a9a314ab4..3d2a53d4c8 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -82,7 +82,6 @@ class BootLinuxConsole(Test):
kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.vm.set_machine('pc')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
self.vm.add_args('-kernel', kernel_path,
@@ -105,7 +104,6 @@ class BootLinuxConsole(Test):
kernel_path = self.extract_from_deb(deb_path,
'/boot/vmlinux-2.6.32-5-4kc-malta')
- self.vm.set_machine('malta')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
self.vm.add_args('-kernel', kernel_path,
@@ -138,7 +136,6 @@ class BootLinuxConsole(Test):
kernel_path = self.extract_from_deb(deb_path,
'/boot/vmlinux-2.6.32-5-5kc-malta')
- self.vm.set_machine('malta')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
self.vm.add_args('-kernel', kernel_path,
@@ -171,7 +168,6 @@ class BootLinuxConsole(Test):
with open(initrd_path, 'wb') as f_out:
shutil.copyfileobj(f_in, f_out)
- self.vm.set_machine('malta')
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'console=ttyS0 console=tty '
@@ -197,7 +193,6 @@ class BootLinuxConsole(Test):
with open(kernel_path, 'wb') as f_out:
shutil.copyfileobj(f_in, f_out)
- self.vm.set_machine('malta')
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'mem=256m@@0x0 '
@@ -257,7 +252,6 @@ class BootLinuxConsole(Test):
kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.vm.set_machine('virt')
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
'console=ttyAMA0')
@@ -279,7 +273,6 @@ class BootLinuxConsole(Test):
kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.vm.set_machine('virt')
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
'console=ttyAMA0')
@@ -292,7 +285,7 @@ class BootLinuxConsole(Test):
def test_arm_emcraft_sf2(self):
"""
:avocado: tags=arch:arm
- :avocado: tags=machine:emcraft_sf2
+ :avocado: tags=machine:emcraft-sf2
:avocado: tags=endian:little
"""
uboot_url = ('https://raw.githubusercontent.com/'
@@ -306,7 +299,6 @@ class BootLinuxConsole(Test):
spi_hash = '85f698329d38de63aea6e884a86fbde70890a78a'
spi_path = self.fetch_asset(spi_url, asset_hash=spi_hash)
- self.vm.set_machine('emcraft-sf2')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
self.vm.add_args('-kernel', uboot_path,
@@ -319,7 +311,7 @@ class BootLinuxConsole(Test):
def test_s390x_s390_ccw_virtio(self):
"""
:avocado: tags=arch:s390x
- :avocado: tags=machine:s390_ccw_virtio
+ :avocado: tags=machine:s390-ccw-virtio
"""
kernel_url = ('https://archives.fedoraproject.org/pub/archive'
'/fedora-secondary/releases/29/Everything/s390x/os/images'
@@ -327,7 +319,6 @@ class BootLinuxConsole(Test):
kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.vm.set_machine('s390-ccw-virtio')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=sclp0'
self.vm.add_args('-nodefaults',
@@ -349,7 +340,6 @@ class BootLinuxConsole(Test):
uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
- self.vm.set_machine('clipper')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
self.vm.add_args('-vga', 'std',
@@ -370,7 +360,6 @@ class BootLinuxConsole(Test):
kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
- self.vm.set_machine('pseries')
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=hvc0'
self.vm.add_args('-kernel', kernel_path,
diff --git a/tests/acceptance/cpu_queries.py b/tests/acceptance/cpu_queries.py
index af47d2795a..293dccb89a 100644
--- a/tests/acceptance/cpu_queries.py
+++ b/tests/acceptance/cpu_queries.py
@@ -20,8 +20,8 @@ class QueryCPUModelExpansion(Test):
def test(self):
"""
:avocado: tags=arch:x86_64
+ :avocado: tags=machine:none
"""
- self.vm.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()
diff --git a/tests/acceptance/linux_initrd.py b/tests/acceptance/linux_initrd.py
index c61d9826a4..3a0ff7b098 100644
--- a/tests/acceptance/linux_initrd.py
+++ b/tests/acceptance/linux_initrd.py
@@ -20,6 +20,7 @@ class LinuxInitrd(Test):
Checks QEMU evaluates correctly the initrd file passed as -initrd option.
:avocado: tags=arch:x86_64
+ :avocado: tags=machine:pc
"""
timeout = 300
@@ -66,7 +67,6 @@ class LinuxInitrd(Test):
initrd.write(b'\0')
initrd.flush()
- self.vm.set_machine('pc')
self.vm.set_console()
kernel_command_line = 'console=ttyS0'
self.vm.add_args('-kernel', kernel_path,
diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
index 25a1df5098..e40a57de51 100644
--- a/tests/acceptance/linux_ssh_mips_malta.py
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -95,7 +95,6 @@ class LinuxSSH(Test):
image_hash = self.IMAGE_INFO[endianess]['image_hash']
image_path = self.fetch_asset(image_url, asset_hash=image_hash)
- self.vm.set_machine('malta')
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'console=ttyS0 root=/dev/sda1')
@@ -199,7 +198,6 @@ class LinuxSSH(Test):
def test_mips_malta32eb_kernel3_2_0(self):
"""
:avocado: tags=arch:mips
- :avocado: tags=machine:malta
:avocado: tags=endian:big
:avocado: tags=device:pcnet32
"""
@@ -213,7 +211,6 @@ class LinuxSSH(Test):
def test_mips_malta32el_kernel3_2_0(self):
"""
:avocado: tags=arch:mipsel
- :avocado: tags=machine:malta
:avocado: tags=endian:little
:avocado: tags=device:pcnet32
"""
@@ -227,7 +224,6 @@ class LinuxSSH(Test):
def test_mips_malta64eb_kernel3_2_0(self):
"""
:avocado: tags=arch:mips64
- :avocado: tags=machine:malta
:avocado: tags=endian:big
:avocado: tags=device:pcnet32
"""
@@ -240,7 +236,6 @@ class LinuxSSH(Test):
def test_mips_malta64el_kernel3_2_0(self):
"""
:avocado: tags=arch:mips64el
- :avocado: tags=machine:malta
:avocado: tags=endian:little
:avocado: tags=device:pcnet32
"""
diff --git a/tests/acceptance/machine_m68k_nextcube.py b/tests/acceptance/machine_m68k_nextcube.py
index fcd2c58ee7..32cf571f94 100644
--- a/tests/acceptance/machine_m68k_nextcube.py
+++ b/tests/acceptance/machine_m68k_nextcube.py
@@ -43,6 +43,11 @@ def tesseract_available(expected_version):
class NextCubeMachine(Test):
+ """
+ :avocado: tags=arch:m68k
+ :avocado: tags=machine:next-cube
+ :avocado: tags=device:framebuffer
+ """
timeout = 15
@@ -52,7 +57,6 @@ class NextCubeMachine(Test):
rom_hash = 'b3534796abae238a0111299fc406a9349f7fee24'
rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash)
- self.vm.set_machine('next-cube')
self.vm.add_args('-bios', rom_path)
self.vm.launch()
@@ -66,11 +70,6 @@ class NextCubeMachine(Test):
@skipUnless(PIL_AVAILABLE, 'Python PIL not installed')
def test_bootrom_framebuffer_size(self):
- """
- :avocado: tags=arch:m68k
- :avocado: tags=machine:next_cube
- :avocado: tags=device:framebuffer
- """
screenshot_path = os.path.join(self.workdir, "dump.png")
self.check_bootrom_framebuffer(screenshot_path)
@@ -80,11 +79,6 @@ class NextCubeMachine(Test):
@skipUnless(tesseract_available(3), 'tesseract v3 OCR tool not available')
def test_bootrom_framebuffer_ocr_with_tesseract_v3(self):
- """
- :avocado: tags=arch:m68k
- :avocado: tags=machine:next_cube
- :avocado: tags=device:framebuffer
- """
screenshot_path = os.path.join(self.workdir, "dump.png")
self.check_bootrom_framebuffer(screenshot_path)
@@ -101,11 +95,6 @@ class NextCubeMachine(Test):
# that it is still alpha-level software.
@skipUnless(tesseract_available(4), 'tesseract v4 OCR tool not available')
def test_bootrom_framebuffer_ocr_with_tesseract_v4(self):
- """
- :avocado: tags=arch:m68k
- :avocado: tags=machine:next_cube
- :avocado: tags=device:framebuffer
- """
screenshot_path = os.path.join(self.workdir, "dump.png")
self.check_bootrom_framebuffer(screenshot_path)
diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/acceptance/x86_cpu_model_versions.py
index 6eb977954d..90558d9a71 100644
--- a/tests/acceptance/x86_cpu_model_versions.py
+++ b/tests/acceptance/x86_cpu_model_versions.py
@@ -75,12 +75,15 @@ class X86CPUModelAliases(avocado_qemu.Test):
"EPYC-IBPB shouldn't be versioned")
def test_4_0_alias_compatibility(self):
- """Check if pc-*-4.0 unversioned CPU model won't be reported as aliases"""
+ """
+ Check if pc-*-4.0 unversioned CPU model won't be reported as aliases
+
+ :avocado: tags=machine:pc-i440fx-4.0
+ """
# pc-*-4.0 won't expose non-versioned CPU models as aliases
# We do this to help management software to keep compatibility
# with older QEMU versions that didn't have the versioned CPU model
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.0')
self.vm.launch()
cpus = dict((m['name'], m) for m in self.vm.command('query-cpu-definitions'))
@@ -105,9 +108,12 @@ class X86CPUModelAliases(avocado_qemu.Test):
self.assertNotIn('alias-of', c, "%s shouldn't be an alias" % (name))
def test_4_1_alias(self):
- """Check if unversioned CPU model is an alias pointing to right version"""
+ """
+ Check if unversioned CPU model is an alias pointing to right version
+
+ :avocado: tags=machine:pc-i440fx-4.1
+ """
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.1')
self.vm.launch()
cpus = dict((m['name'], m) for m in self.vm.command('query-cpu-definitions'))
@@ -207,9 +213,12 @@ class X86CPUModelAliases(avocado_qemu.Test):
self.validate_aliases(cpus)
def test_none_alias(self):
- """Check if unversioned CPU model is an alias pointing to some version"""
+ """
+ Check if unversioned CPU model is an alias pointing to some version
+
+ :avocado: tags=machine:none
+ """
self.vm.add_args('-S')
- self.vm.set_machine('none')
self.vm.launch()
cpus = dict((m['name'], m) for m in self.vm.command('query-cpu-definitions'))
@@ -242,68 +251,84 @@ class CascadelakeArchCapabilities(avocado_qemu.Test):
return self.vm.command('qom-get', path=cpu_path, property=prop)
def test_4_1(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.1
+ """
# machine-type only:
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.1')
self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
def test_4_0(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.0
+ """
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.0')
self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
def test_set_4_0(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.0
+ """
# command line must override machine-type if CPU model is not versioned:
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.0')
self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
self.vm.launch()
self.assertTrue(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
def test_unset_4_1(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.1
+ """
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.1')
self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
def test_v1_4_0(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.0
+ """
# versioned CPU model overrides machine-type:
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.0')
self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
def test_v2_4_0(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.0
+ """
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.0')
self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
self.vm.launch()
self.assertTrue(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
def test_v1_set_4_0(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.0
+ """
# command line must override machine-type and versioned CPU model:
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.0')
self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
self.vm.launch()
self.assertTrue(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
def test_v2_unset_4_1(self):
+ """
+ :avocado: tags=machine:pc-i440fx-4.1
+ """
self.vm.add_args('-S')
- self.vm.set_machine('pc-i440fx-4.1')
self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off,-arch-capabilities')
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
--
2.21.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 0/3] Acceptance tests: make better use of machine tags
2019-09-24 19:44 [PATCH 0/3] Acceptance tests: make better use of machine tags Cleber Rosa
` (2 preceding siblings ...)
2019-09-24 19:45 ` [PATCH 3/3] Acceptance Tests: use avocado tags for machine type Cleber Rosa
@ 2019-09-25 8:03 ` no-reply
3 siblings, 0 replies; 9+ messages in thread
From: no-reply @ 2019-09-25 8:03 UTC (permalink / raw)
To: crosa
Cc: ehabkost, arikalo, qemu-devel, wainersm, wrampazz, crosa, philmd,
aurelien
Patchew URL: https://patchew.org/QEMU/20190924194501.9303-1-crosa@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20190924194501.9303-1-crosa@redhat.com
Subject: [PATCH 0/3] Acceptance tests: make better use of machine tags
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Switched to a new branch 'test'
ba0bacf Acceptance Tests: use avocado tags for machine type
68ff8a9 Acceptance tests: introduce utility method for tags unique vals
ea0f62b Acceptance test x86_cpu_model_versions: use default vm
=== OUTPUT BEGIN ===
1/3 Checking commit ea0f62b13779 (Acceptance test x86_cpu_model_versions: use default vm)
ERROR: line over 90 characters
#47: FILE: tests/acceptance/x86_cpu_model_versions.py:248:
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#61: FILE: tests/acceptance/x86_cpu_model_versions.py:256:
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#76: FILE: tests/acceptance/x86_cpu_model_versions.py:265:
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
ERROR: line over 90 characters
#90: FILE: tests/acceptance/x86_cpu_model_versions.py:273:
+ self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
ERROR: line over 90 characters
#105: FILE: tests/acceptance/x86_cpu_model_versions.py:282:
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#120: FILE: tests/acceptance/x86_cpu_model_versions.py:290:
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#123: FILE: tests/acceptance/x86_cpu_model_versions.py:293:
+ 'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
ERROR: line over 90 characters
#136: FILE: tests/acceptance/x86_cpu_model_versions.py:299:
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
ERROR: line over 90 characters
#139: FILE: tests/acceptance/x86_cpu_model_versions.py:302:
+ 'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
ERROR: line over 90 characters
#150: FILE: tests/acceptance/x86_cpu_model_versions.py:307:
+ self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off,-arch-capabilities')
total: 10 errors, 0 warnings, 134 lines checked
Patch 1/3 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/3 Checking commit 68ff8a93f9cd (Acceptance tests: introduce utility method for tags unique vals)
3/3 Checking commit ba0bacf3a0d7 (Acceptance Tests: use avocado tags for machine type)
WARNING: line over 80 characters
#62: FILE: tests/acceptance/avocado_qemu/__init__.py:73:
+ default=self._get_unique_tag_val('machine'))
total: 0 errors, 1 warnings, 386 lines checked
Patch 3/3 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20190924194501.9303-1-crosa@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals
2019-09-24 19:45 ` [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals Cleber Rosa
@ 2019-10-24 21:12 ` Wainer dos Santos Moschetta
2019-10-28 23:02 ` Cleber Rosa
0 siblings, 1 reply; 9+ messages in thread
From: Wainer dos Santos Moschetta @ 2019-10-24 21:12 UTC (permalink / raw)
To: Cleber Rosa, qemu-devel
Cc: Aleksandar Rikalo, Aurelien Jarno, Philippe Mathieu-Daudé,
Eduardo Habkost, Willian Rampazzo
Hi Cleber,
On 9/24/19 4:45 PM, Cleber Rosa wrote:
> Currently a test can describe the target architecture binary that it
> should primarily be run with, be setting a single tag value.
>
> The same approach is expected to be done with other QEMU aspects to be
> tested, for instance, the machine type and accelerator, so let's
> generalize the logic into a utility method.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> tests/acceptance/avocado_qemu/__init__.py | 19 +++++++++++++------
> 1 file changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index bd41e0443c..02775bafcf 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -54,14 +54,21 @@ def pick_default_qemu_bin(arch=None):
>
>
> class Test(avocado.Test):
> + def _get_unique_tag_val(self, tag_name):
> + """
> + Gets a tag value, if unique for a key
> + """
> + vals = self.tags.get(tag_name, [])
> + if len(vals) == 1:
An small optimization:
if vals:
return vals.pop()
> + return vals.pop()
> + return None
Does it allows to express a scenario like "I want my test method to run
on x86_64 and aarch64" using tags? If so, _get_unique_tag_val logic
returns None for multi-value tags (e.g. 'tags=arch:x86_64,arch:aarch64').
Thanks,
Wainer
> +
> def setUp(self):
> self._vms = {}
> - arches = self.tags.get('arch', [])
> - if len(arches) == 1:
> - arch = arches.pop()
> - else:
> - arch = None
> - self.arch = self.params.get('arch', default=arch)
> +
> + self.arch = self.params.get('arch',
> + default=self._get_unique_tag_val('arch'))
> +
> default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
> self.qemu_bin = self.params.get('qemu_bin',
> default=default_qemu_bin)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] Acceptance Tests: use avocado tags for machine type
2019-09-24 19:45 ` [PATCH 3/3] Acceptance Tests: use avocado tags for machine type Cleber Rosa
@ 2019-10-24 21:24 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 9+ messages in thread
From: Wainer dos Santos Moschetta @ 2019-10-24 21:24 UTC (permalink / raw)
To: Cleber Rosa, qemu-devel
Cc: Aleksandar Rikalo, Aurelien Jarno, Philippe Mathieu-Daudé,
Eduardo Habkost, Willian Rampazzo
Hello,
On 9/24/19 4:45 PM, Cleber Rosa wrote:
> The same way the arch tag is being used as a fallback for the arch
> parameter, let's do the same for QEMU's machine and avoid some boiler
> plate code.
>
> This requires a bump in the Avocado version, as starting with 72.0,
> the characters supported in tags are less strict.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> docs/devel/testing.rst | 18 ++++++++
> tests/acceptance/avocado_qemu/__init__.py | 5 ++
> tests/acceptance/boot_linux_console.py | 15 +-----
> tests/acceptance/cpu_queries.py | 2 +-
> tests/acceptance/linux_initrd.py | 2 +-
> tests/acceptance/linux_ssh_mips_malta.py | 5 --
> tests/acceptance/machine_m68k_nextcube.py | 21 ++-------
> tests/acceptance/x86_cpu_model_versions.py | 53 ++++++++++++++++------
> 8 files changed, 71 insertions(+), 50 deletions(-)
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index bf75675fb0..1816ada919 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -744,6 +744,17 @@ name. If one is not given explicitly, it will either be set to
> ``None``, or, if the test is tagged with one (and only one)
> ``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
>
> +machine
> +~~~~~~~
> +
> +The machine type that will be set to all QEMUMachine instances created
> +by the test.
> +
> +The ``machine`` attribute will be set to the test parameter of the same
> +name. If one is not given explicitly, it will either be set to
> +``None``, or, if the test is tagged with one (and only one)
> +``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
> +
> qemu_bin
> ~~~~~~~~
>
> @@ -779,6 +790,13 @@ architecture of a kernel or disk image to boot a VM with.
> This parameter has a direct relation with the ``arch`` attribute. If
> not given, it will default to None.
>
> +machine
> +~~~~~~~
> +
> +The machine type that will be set to all QEMUMachine instances created
> +by the test.
> +
> +
> qemu_bin
> ~~~~~~~~
>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 02775bafcf..fb5d6616bc 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -69,6 +69,9 @@ class Test(avocado.Test):
> self.arch = self.params.get('arch',
> default=self._get_unique_tag_val('arch'))
>
> + self.machine = self.params.get('machine',
> + default=self._get_unique_tag_val('machine'))
> +
> default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
> self.qemu_bin = self.params.get('qemu_bin',
> default=default_qemu_bin)
> @@ -90,6 +93,8 @@ class Test(avocado.Test):
> name = str(uuid.uuid4())
> if self._vms.get(name) is None:
> self._vms[name] = self._new_vm(*args)
> + if self.machine is not None:
> + self._vms[name].set_machine(self.machine)
> return self._vms[name]
>
> def tearDown(self):
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 8a9a314ab4..3d2a53d4c8 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -82,7 +82,6 @@ class BootLinuxConsole(Test):
> kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
> kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>
> - self.vm.set_machine('pc')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
> self.vm.add_args('-kernel', kernel_path,
> @@ -105,7 +104,6 @@ class BootLinuxConsole(Test):
> kernel_path = self.extract_from_deb(deb_path,
> '/boot/vmlinux-2.6.32-5-4kc-malta')
>
> - self.vm.set_machine('malta')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
> self.vm.add_args('-kernel', kernel_path,
> @@ -138,7 +136,6 @@ class BootLinuxConsole(Test):
> kernel_path = self.extract_from_deb(deb_path,
> '/boot/vmlinux-2.6.32-5-5kc-malta')
>
> - self.vm.set_machine('malta')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
> self.vm.add_args('-kernel', kernel_path,
> @@ -171,7 +168,6 @@ class BootLinuxConsole(Test):
> with open(initrd_path, 'wb') as f_out:
> shutil.copyfileobj(f_in, f_out)
>
> - self.vm.set_machine('malta')
> self.vm.set_console()
> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> + 'console=ttyS0 console=tty '
> @@ -197,7 +193,6 @@ class BootLinuxConsole(Test):
> with open(kernel_path, 'wb') as f_out:
> shutil.copyfileobj(f_in, f_out)
>
> - self.vm.set_machine('malta')
> self.vm.set_console()
> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> + 'mem=256m@@0x0 '
> @@ -257,7 +252,6 @@ class BootLinuxConsole(Test):
> kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
> kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>
> - self.vm.set_machine('virt')
> self.vm.set_console()
> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> 'console=ttyAMA0')
> @@ -279,7 +273,6 @@ class BootLinuxConsole(Test):
> kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
> kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>
> - self.vm.set_machine('virt')
> self.vm.set_console()
> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> 'console=ttyAMA0')
> @@ -292,7 +285,7 @@ class BootLinuxConsole(Test):
> def test_arm_emcraft_sf2(self):
> """
> :avocado: tags=arch:arm
> - :avocado: tags=machine:emcraft_sf2
> + :avocado: tags=machine:emcraft-sf2
> :avocado: tags=endian:little
> """
> uboot_url = ('https://raw.githubusercontent.com/'
> @@ -306,7 +299,6 @@ class BootLinuxConsole(Test):
> spi_hash = '85f698329d38de63aea6e884a86fbde70890a78a'
> spi_path = self.fetch_asset(spi_url, asset_hash=spi_hash)
>
> - self.vm.set_machine('emcraft-sf2')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
> self.vm.add_args('-kernel', uboot_path,
> @@ -319,7 +311,7 @@ class BootLinuxConsole(Test):
> def test_s390x_s390_ccw_virtio(self):
> """
> :avocado: tags=arch:s390x
> - :avocado: tags=machine:s390_ccw_virtio
> + :avocado: tags=machine:s390-ccw-virtio
> """
> kernel_url = ('https://archives.fedoraproject.org/pub/archive'
> '/fedora-secondary/releases/29/Everything/s390x/os/images'
> @@ -327,7 +319,6 @@ class BootLinuxConsole(Test):
> kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
> kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>
> - self.vm.set_machine('s390-ccw-virtio')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=sclp0'
> self.vm.add_args('-nodefaults',
> @@ -349,7 +340,6 @@ class BootLinuxConsole(Test):
>
> uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
>
> - self.vm.set_machine('clipper')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
> self.vm.add_args('-vga', 'std',
> @@ -370,7 +360,6 @@ class BootLinuxConsole(Test):
> kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
> kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>
> - self.vm.set_machine('pseries')
> self.vm.set_console()
> kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=hvc0'
> self.vm.add_args('-kernel', kernel_path,
> diff --git a/tests/acceptance/cpu_queries.py b/tests/acceptance/cpu_queries.py
> index af47d2795a..293dccb89a 100644
> --- a/tests/acceptance/cpu_queries.py
> +++ b/tests/acceptance/cpu_queries.py
> @@ -20,8 +20,8 @@ class QueryCPUModelExpansion(Test):
> def test(self):
> """
> :avocado: tags=arch:x86_64
> + :avocado: tags=machine:none
> """
> - self.vm.set_machine('none')
> self.vm.add_args('-S')
> self.vm.launch()
>
> diff --git a/tests/acceptance/linux_initrd.py b/tests/acceptance/linux_initrd.py
> index c61d9826a4..3a0ff7b098 100644
> --- a/tests/acceptance/linux_initrd.py
> +++ b/tests/acceptance/linux_initrd.py
> @@ -20,6 +20,7 @@ class LinuxInitrd(Test):
> Checks QEMU evaluates correctly the initrd file passed as -initrd option.
>
> :avocado: tags=arch:x86_64
> + :avocado: tags=machine:pc
> """
>
> timeout = 300
> @@ -66,7 +67,6 @@ class LinuxInitrd(Test):
> initrd.write(b'\0')
> initrd.flush()
>
> - self.vm.set_machine('pc')
> self.vm.set_console()
> kernel_command_line = 'console=ttyS0'
> self.vm.add_args('-kernel', kernel_path,
> diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
> index 25a1df5098..e40a57de51 100644
> --- a/tests/acceptance/linux_ssh_mips_malta.py
> +++ b/tests/acceptance/linux_ssh_mips_malta.py
> @@ -95,7 +95,6 @@ class LinuxSSH(Test):
> image_hash = self.IMAGE_INFO[endianess]['image_hash']
> image_path = self.fetch_asset(image_url, asset_hash=image_hash)
>
> - self.vm.set_machine('malta')
> self.vm.set_console()
> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> + 'console=ttyS0 root=/dev/sda1')
> @@ -199,7 +198,6 @@ class LinuxSSH(Test):
> def test_mips_malta32eb_kernel3_2_0(self):
> """
> :avocado: tags=arch:mips
> - :avocado: tags=machine:malta
> :avocado: tags=endian:big
> :avocado: tags=device:pcnet32
> """
> @@ -213,7 +211,6 @@ class LinuxSSH(Test):
> def test_mips_malta32el_kernel3_2_0(self):
> """
> :avocado: tags=arch:mipsel
> - :avocado: tags=machine:malta
> :avocado: tags=endian:little
> :avocado: tags=device:pcnet32
> """
> @@ -227,7 +224,6 @@ class LinuxSSH(Test):
> def test_mips_malta64eb_kernel3_2_0(self):
> """
> :avocado: tags=arch:mips64
> - :avocado: tags=machine:malta
> :avocado: tags=endian:big
> :avocado: tags=device:pcnet32
> """
> @@ -240,7 +236,6 @@ class LinuxSSH(Test):
> def test_mips_malta64el_kernel3_2_0(self):
> """
> :avocado: tags=arch:mips64el
> - :avocado: tags=machine:malta
> :avocado: tags=endian:little
> :avocado: tags=device:pcnet32
> """
> diff --git a/tests/acceptance/machine_m68k_nextcube.py b/tests/acceptance/machine_m68k_nextcube.py
> index fcd2c58ee7..32cf571f94 100644
> --- a/tests/acceptance/machine_m68k_nextcube.py
> +++ b/tests/acceptance/machine_m68k_nextcube.py
> @@ -43,6 +43,11 @@ def tesseract_available(expected_version):
>
>
> class NextCubeMachine(Test):
> + """
> + :avocado: tags=arch:m68k
> + :avocado: tags=machine:next-cube
> + :avocado: tags=device:framebuffer
> + """
>
> timeout = 15
>
> @@ -52,7 +57,6 @@ class NextCubeMachine(Test):
> rom_hash = 'b3534796abae238a0111299fc406a9349f7fee24'
> rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash)
>
> - self.vm.set_machine('next-cube')
> self.vm.add_args('-bios', rom_path)
> self.vm.launch()
>
> @@ -66,11 +70,6 @@ class NextCubeMachine(Test):
>
> @skipUnless(PIL_AVAILABLE, 'Python PIL not installed')
> def test_bootrom_framebuffer_size(self):
> - """
> - :avocado: tags=arch:m68k
> - :avocado: tags=machine:next_cube
> - :avocado: tags=device:framebuffer
> - """
> screenshot_path = os.path.join(self.workdir, "dump.png")
> self.check_bootrom_framebuffer(screenshot_path)
>
> @@ -80,11 +79,6 @@ class NextCubeMachine(Test):
>
> @skipUnless(tesseract_available(3), 'tesseract v3 OCR tool not available')
> def test_bootrom_framebuffer_ocr_with_tesseract_v3(self):
> - """
> - :avocado: tags=arch:m68k
> - :avocado: tags=machine:next_cube
> - :avocado: tags=device:framebuffer
> - """
> screenshot_path = os.path.join(self.workdir, "dump.png")
> self.check_bootrom_framebuffer(screenshot_path)
>
> @@ -101,11 +95,6 @@ class NextCubeMachine(Test):
> # that it is still alpha-level software.
> @skipUnless(tesseract_available(4), 'tesseract v4 OCR tool not available')
> def test_bootrom_framebuffer_ocr_with_tesseract_v4(self):
> - """
> - :avocado: tags=arch:m68k
> - :avocado: tags=machine:next_cube
> - :avocado: tags=device:framebuffer
> - """
> screenshot_path = os.path.join(self.workdir, "dump.png")
> self.check_bootrom_framebuffer(screenshot_path)
>
> diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/acceptance/x86_cpu_model_versions.py
> index 6eb977954d..90558d9a71 100644
> --- a/tests/acceptance/x86_cpu_model_versions.py
> +++ b/tests/acceptance/x86_cpu_model_versions.py
> @@ -75,12 +75,15 @@ class X86CPUModelAliases(avocado_qemu.Test):
> "EPYC-IBPB shouldn't be versioned")
>
> def test_4_0_alias_compatibility(self):
> - """Check if pc-*-4.0 unversioned CPU model won't be reported as aliases"""
> + """
> + Check if pc-*-4.0 unversioned CPU model won't be reported as aliases
> +
> + :avocado: tags=machine:pc-i440fx-4.0
> + """
> # pc-*-4.0 won't expose non-versioned CPU models as aliases
> # We do this to help management software to keep compatibility
> # with older QEMU versions that didn't have the versioned CPU model
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.0')
> self.vm.launch()
> cpus = dict((m['name'], m) for m in self.vm.command('query-cpu-definitions'))
>
> @@ -105,9 +108,12 @@ class X86CPUModelAliases(avocado_qemu.Test):
> self.assertNotIn('alias-of', c, "%s shouldn't be an alias" % (name))
>
> def test_4_1_alias(self):
> - """Check if unversioned CPU model is an alias pointing to right version"""
> + """
> + Check if unversioned CPU model is an alias pointing to right version
> +
> + :avocado: tags=machine:pc-i440fx-4.1
> + """
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.1')
> self.vm.launch()
>
> cpus = dict((m['name'], m) for m in self.vm.command('query-cpu-definitions'))
> @@ -207,9 +213,12 @@ class X86CPUModelAliases(avocado_qemu.Test):
> self.validate_aliases(cpus)
>
> def test_none_alias(self):
> - """Check if unversioned CPU model is an alias pointing to some version"""
> + """
> + Check if unversioned CPU model is an alias pointing to some version
> +
> + :avocado: tags=machine:none
> + """
> self.vm.add_args('-S')
> - self.vm.set_machine('none')
> self.vm.launch()
>
> cpus = dict((m['name'], m) for m in self.vm.command('query-cpu-definitions'))
> @@ -242,68 +251,84 @@ class CascadelakeArchCapabilities(avocado_qemu.Test):
> return self.vm.command('qom-get', path=cpu_path, property=prop)
>
> def test_4_1(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.1
> + """
> # machine-type only:
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.1')
> self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
> self.vm.launch()
> self.assertFalse(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
>
> def test_4_0(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.0
> + """
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.0')
> self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
> self.vm.launch()
> self.assertFalse(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
>
> def test_set_4_0(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.0
> + """
> # command line must override machine-type if CPU model is not versioned:
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.0')
> self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
> self.vm.launch()
> self.assertTrue(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
>
> def test_unset_4_1(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.1
> + """
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.1')
> self.vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
> self.vm.launch()
> self.assertFalse(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
>
> def test_v1_4_0(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.0
> + """
> # versioned CPU model overrides machine-type:
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.0')
> self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
> self.vm.launch()
> self.assertFalse(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
>
> def test_v2_4_0(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.0
> + """
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.0')
> self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
> self.vm.launch()
> self.assertTrue(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
>
> def test_v1_set_4_0(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.0
> + """
> # command line must override machine-type and versioned CPU model:
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.0')
> self.vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
> self.vm.launch()
> self.assertTrue(self.get_cpu_prop('arch-capabilities'),
> 'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
>
> def test_v2_unset_4_1(self):
> + """
> + :avocado: tags=machine:pc-i440fx-4.1
> + """
> self.vm.add_args('-S')
> - self.vm.set_machine('pc-i440fx-4.1')
> self.vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off,-arch-capabilities')
> self.vm.launch()
> self.assertFalse(self.get_cpu_prop('arch-capabilities'),
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals
2019-10-24 21:12 ` Wainer dos Santos Moschetta
@ 2019-10-28 23:02 ` Cleber Rosa
2019-11-07 18:29 ` Wainer dos Santos Moschetta
0 siblings, 1 reply; 9+ messages in thread
From: Cleber Rosa @ 2019-10-28 23:02 UTC (permalink / raw)
To: Wainer dos Santos Moschetta
Cc: Eduardo Habkost, Aleksandar Rikalo, qemu-devel, Willian Rampazzo,
Philippe Mathieu-Daudé,
Aurelien Jarno
On Thu, Oct 24, 2019 at 06:12:25PM -0300, Wainer dos Santos Moschetta wrote:
> Hi Cleber,
>
> On 9/24/19 4:45 PM, Cleber Rosa wrote:
> > Currently a test can describe the target architecture binary that it
> > should primarily be run with, be setting a single tag value.
> >
> > The same approach is expected to be done with other QEMU aspects to be
> > tested, for instance, the machine type and accelerator, so let's
> > generalize the logic into a utility method.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> > tests/acceptance/avocado_qemu/__init__.py | 19 +++++++++++++------
> > 1 file changed, 13 insertions(+), 6 deletions(-)
> >
> > diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> > index bd41e0443c..02775bafcf 100644
> > --- a/tests/acceptance/avocado_qemu/__init__.py
> > +++ b/tests/acceptance/avocado_qemu/__init__.py
> > @@ -54,14 +54,21 @@ def pick_default_qemu_bin(arch=None):
> > class Test(avocado.Test):
> > + def _get_unique_tag_val(self, tag_name):
> > + """
> > + Gets a tag value, if unique for a key
> > + """
> > + vals = self.tags.get(tag_name, [])
> > + if len(vals) == 1:
>
>
> An small optimization:
>
> if vals:
>
> return vals.pop()
>
IIUC, this would break the idea of uniqueness that this method, for
now, has. Read on.
>
> > + return vals.pop()
> > + return None
>
> Does it allows to express a scenario like "I want my test method to run on
> x86_64 and aarch64" using tags? If so, _get_unique_tag_val logic returns
> None for multi-value tags (e.g. 'tags=arch:x86_64,arch:aarch64').
>
I thought that initially we should attempt to pick a default arch or
machine type only of len(vals) == 1. Not because what you describe
can't be done, but because I would like to go through the tests and
make sure we run them in all the given tagged arches when we allow
that.
Thanks,
- Cleber.
> Thanks,
>
> Wainer
>
> > +
> > def setUp(self):
> > self._vms = {}
> > - arches = self.tags.get('arch', [])
> > - if len(arches) == 1:
> > - arch = arches.pop()
> > - else:
> > - arch = None
> > - self.arch = self.params.get('arch', default=arch)
> > +
> > + self.arch = self.params.get('arch',
> > + default=self._get_unique_tag_val('arch'))
> > +
> > default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
> > self.qemu_bin = self.params.get('qemu_bin',
> > default=default_qemu_bin)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals
2019-10-28 23:02 ` Cleber Rosa
@ 2019-11-07 18:29 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 9+ messages in thread
From: Wainer dos Santos Moschetta @ 2019-11-07 18:29 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Aleksandar Rikalo, Willian Rampazzo,
Cleber Rosa, Philippe Mathieu-Daudé,
Aurelien Jarno
On 10/28/19 8:02 PM, Cleber Rosa wrote:
> On Thu, Oct 24, 2019 at 06:12:25PM -0300, Wainer dos Santos Moschetta wrote:
>> Hi Cleber,
>>
>> On 9/24/19 4:45 PM, Cleber Rosa wrote:
>>> Currently a test can describe the target architecture binary that it
>>> should primarily be run with, be setting a single tag value.
>>>
>>> The same approach is expected to be done with other QEMU aspects to be
>>> tested, for instance, the machine type and accelerator, so let's
>>> generalize the logic into a utility method.
>>>
>>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>>> ---
>>> tests/acceptance/avocado_qemu/__init__.py | 19 +++++++++++++------
>>> 1 file changed, 13 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
>>> index bd41e0443c..02775bafcf 100644
>>> --- a/tests/acceptance/avocado_qemu/__init__.py
>>> +++ b/tests/acceptance/avocado_qemu/__init__.py
>>> @@ -54,14 +54,21 @@ def pick_default_qemu_bin(arch=None):
>>> class Test(avocado.Test):
>>> + def _get_unique_tag_val(self, tag_name):
>>> + """
>>> + Gets a tag value, if unique for a key
>>> + """
>>> + vals = self.tags.get(tag_name, [])
>>> + if len(vals) == 1:
>>
>> An small optimization:
>>
>> if vals:
>>
>> return vals.pop()
>>
> IIUC, this would break the idea of uniqueness that this method, for
> now, has. Read on.
>
>>> + return vals.pop()
>>> + return None
>> Does it allows to express a scenario like "I want my test method to run on
>> x86_64 and aarch64" using tags? If so, _get_unique_tag_val logic returns
>> None for multi-value tags (e.g. 'tags=arch:x86_64,arch:aarch64').
>>
> I thought that initially we should attempt to pick a default arch or
> machine type only of len(vals) == 1. Not because what you describe
> can't be done, but because I would like to go through the tests and
> make sure we run them in all the given tagged arches when we allow
> that.
Ok, understood the rationale now.
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> Thanks,
> - Cleber.
>
>> Thanks,
>>
>> Wainer
>>
>>> +
>>> def setUp(self):
>>> self._vms = {}
>>> - arches = self.tags.get('arch', [])
>>> - if len(arches) == 1:
>>> - arch = arches.pop()
>>> - else:
>>> - arch = None
>>> - self.arch = self.params.get('arch', default=arch)
>>> +
>>> + self.arch = self.params.get('arch',
>>> + default=self._get_unique_tag_val('arch'))
>>> +
>>> default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
>>> self.qemu_bin = self.params.get('qemu_bin',
>>> default=default_qemu_bin)
>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-11-07 18:31 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-24 19:44 [PATCH 0/3] Acceptance tests: make better use of machine tags Cleber Rosa
2019-09-24 19:44 ` [PATCH 1/3] Acceptance test x86_cpu_model_versions: use default vm Cleber Rosa
2019-09-24 19:45 ` [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals Cleber Rosa
2019-10-24 21:12 ` Wainer dos Santos Moschetta
2019-10-28 23:02 ` Cleber Rosa
2019-11-07 18:29 ` Wainer dos Santos Moschetta
2019-09-24 19:45 ` [PATCH 3/3] Acceptance Tests: use avocado tags for machine type Cleber Rosa
2019-10-24 21:24 ` Wainer dos Santos Moschetta
2019-09-25 8:03 ` [PATCH 0/3] Acceptance tests: make better use of machine tags no-reply
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).