* [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
@ 2018-04-19 16:46 Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 1/7] AVOCADO_QEMU: Snapshot commit Philippe Mathieu-Daudé
` (9 more replies)
0 siblings, 10 replies; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis
Hi,
While previously working on a Super I/O refactor, I encountered some problems
at runtime, after building the codebase successfully and running qtests.
I had to manually start to boot different guests and check the bootlog.
I wanted to give a try at Avocado which seems designed to simplify that kind
of functional tests.
I applied Amador Pahim work following Cleber Rosa documentation from
http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03891.html,
however I had to modify few things to parse the boot console.
Since his work is not merged, I included it in this series.
The tests simply expect to find a string reported by Linux printk when a
device is detected/initialized, such "ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A"
and "i8042 KBD port at 0x60,0x64 irq 1" for the Super I/O chip, or such
"registered as PCnet/PCI II 79C970A" to confirms the PCI subsystem and network
device are correctly detected:
self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
Example of the tests output:
$ avocado run test_linux-boot-console.py -m test_linux-boot-console.py.data/parameters.yaml
JOB ID : 695094c9bbe8f6011226da7c2031c2c53e949910
JOB LOG : /home/phil/avocado/job-results/job-2018-04-19T13.36-695094c/job.log
(1/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;alpha-2582: PASS (4.76 s)
(2/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;mips-4a72: PASS (0.00 s)
(3/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;alpha-2582: PASS (0.00 s)
(4/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;mips-4a72: PASS (3.92 s)
(5/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;alpha-2582: PASS (0.00 s)
(6/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;mips-4a72: PASS (4.08 s)
RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 13.31 s
Please apologize my ugly Python, this series is just a proof-of-concept :)
I couldn't figure out how to use the @skipUnless(correct arch) decorator.
Regards,
Phil.
Amador Pahim (1):
AVOCADO_QEMU: Snapshot commit
Philippe Mathieu-Daudé (6):
avocado: Update python scripts to upstream codebase
qemu.py: Check console arch is supported before calling mktemp()
qemu.py: Avoid deprecated tempfile.mktemp()
avocado: Add an optional flag 'login' to get_console()
avocado: Add a test parsing Linux kernel booting console
avocado: Add tests booting the Malta machine
scripts/qemu.py | 50 ++-
tests/avocado/README.rst | 132 ++++++
tests/avocado/avocado_qemu/__init__.py | 0
tests/avocado/avocado_qemu/test.py | 419 ++++++++++++++++++
tests/avocado/parameters.yaml | 19 +
tests/avocado/test_info_memdev_host_nodes.py | 66 +++
tests/avocado/test_linux-boot-console.py | 180 ++++++++
.../parameters.yaml | 5 +
tests/avocado/test_nec-usb-xhci.py | 63 +++
.../test_nec-usb-xhci.py.data/parameters.yaml | 4 +
tests/avocado/test_numa_hotplug.py | 120 +++++
tests/avocado/test_ovmf_with_240_vcpus.py | 70 +++
.../parameters.yaml | 2 +
tests/avocado/variants.yaml | 62 +++
14 files changed, 1187 insertions(+), 5 deletions(-)
create mode 100644 tests/avocado/README.rst
create mode 100644 tests/avocado/avocado_qemu/__init__.py
create mode 100644 tests/avocado/avocado_qemu/test.py
create mode 100644 tests/avocado/parameters.yaml
create mode 100644 tests/avocado/test_info_memdev_host_nodes.py
create mode 100644 tests/avocado/test_linux-boot-console.py
create mode 100644 tests/avocado/test_linux-boot-console.py.data/parameters.yaml
create mode 100644 tests/avocado/test_nec-usb-xhci.py
create mode 100644 tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml
create mode 100644 tests/avocado/test_numa_hotplug.py
create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py
create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml
create mode 100644 tests/avocado/variants.yaml
--
2.17.0
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 1/7] AVOCADO_QEMU: Snapshot commit
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 2/7] avocado: Update python scripts to upstream codebase Philippe Mathieu-Daudé
` (8 subsequent siblings)
9 siblings, 0 replies; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Philippe Mathieu-Daudé,
Eduardo Habkost, Kevin Wolf, Max Reitz,
open list:Block layer core
From: Amador Pahim <apahim@redhat.com>
Applied https://patch-diff.githubusercontent.com/raw/apahim/qemu/pull/17.patch
following http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03891.html.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
scripts/qemu.py | 59 ++-
tests/avocado/README.rst | 132 ++++++
tests/avocado/avocado_qemu/__init__.py | 0
tests/avocado/avocado_qemu/test.py | 418 ++++++++++++++++++
tests/avocado/parameters.yaml | 19 +
tests/avocado/test_info_memdev_host_nodes.py | 66 +++
tests/avocado/test_nec-usb-xhci.py | 63 +++
.../test_nec-usb-xhci.py.data/parameters.yaml | 4 +
tests/avocado/test_numa_hotplug.py | 120 +++++
tests/avocado/test_ovmf_with_240_vcpus.py | 70 +++
.../parameters.yaml | 2 +
tests/avocado/variants.yaml | 62 +++
tests/qemu-iotests/iotests.py | 28 +-
13 files changed, 1019 insertions(+), 24 deletions(-)
create mode 100644 tests/avocado/README.rst
create mode 100644 tests/avocado/avocado_qemu/__init__.py
create mode 100644 tests/avocado/avocado_qemu/test.py
create mode 100644 tests/avocado/parameters.yaml
create mode 100644 tests/avocado/test_info_memdev_host_nodes.py
create mode 100644 tests/avocado/test_nec-usb-xhci.py
create mode 100644 tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml
create mode 100644 tests/avocado/test_numa_hotplug.py
create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py
create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml
create mode 100644 tests/avocado/variants.yaml
diff --git a/scripts/qemu.py b/scripts/qemu.py
index 08a3e9af5a..bd66620f45 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -55,7 +55,7 @@ class QEMUMachine(object):
def __init__(self, binary, args=None, wrapper=None, name=None,
test_dir="/var/tmp", monitor_address=None,
- socket_scm_helper=None):
+ socket_scm_helper=None, arch=None):
'''
Initialize a QEMUMachine
@@ -81,7 +81,7 @@ class QEMUMachine(object):
self._qemu_log_file = None
self._popen = None
self._binary = binary
- self._args = list(args) # Force copy args in case we modify them
+ self.args = list(args) # Force copy args in case we modify them
self._wrapper = wrapper
self._events = []
self._iolog = None
@@ -91,6 +91,10 @@ class QEMUMachine(object):
self._test_dir = test_dir
self._temp_dir = None
self._launched = False
+ if arch is None:
+ arch = binary.split('-')[-1]
+ self._arch = arch
+ self._console_address = None
# just in case logging wasn't configured by the main script:
logging.basicConfig()
@@ -105,8 +109,8 @@ class QEMUMachine(object):
# This can be used to add an unused monitor instance.
def add_monitor_telnet(self, ip, port):
args = 'tcp:%s:%d,server,nowait,telnet' % (ip, port)
- self._args.append('-monitor')
- self._args.append(args)
+ self.args.append('-monitor')
+ self.args.append(args)
def add_fd(self, fd, fdset, opaque, opts=''):
'''Pass a file descriptor to the VM'''
@@ -116,8 +120,8 @@ class QEMUMachine(object):
if opts:
options.append(opts)
- self._args.append('-add-fd')
- self._args.append(','.join(options))
+ self.args.append('-add-fd')
+ self.args.append(','.join(options))
return self
def send_fd_scm(self, fd_file_path):
@@ -179,6 +183,39 @@ class QEMUMachine(object):
'-mon', 'chardev=mon,mode=control',
'-display', 'none', '-vga', 'none']
+ def _create_console(self, console_address):
+ for item in self.args:
+ for option in ['isa-serial', 'spapr-vty', 'sclpconsole']:
+ if option in item:
+ return []
+
+ chardev = 'socket,id=console,{address},server,nowait'
+ if console_address is None:
+ console_address = tempfile.mktemp()
+ chardev = chardev.format(address='path=%s' %
+ console_address)
+ elif isinstance(console_address, tuple):
+ chardev = chardev.format(address='host=%s,port=%s' %
+ (console_address[0],
+ console_address[1]))
+ else:
+ chardev = chardev.format(address='path=%s' % console_address)
+
+ self._console_address = console_address
+
+ device = '{dev_type},chardev=console'
+ if '86' in self._arch:
+ device = device.format(dev_type='isa-serial')
+ elif 'ppc' in self._arch:
+ device = device.format(dev_type='spapr-vty')
+ elif 's390x' in self._arch:
+ device = device.format(dev_type='sclpconsole')
+ else:
+ return []
+
+ return ['-chardev', chardev,
+ '-device', device]
+
def _pre_launch(self):
self._temp_dir = tempfile.mkdtemp(dir=self._test_dir)
if self._monitor_address is not None:
@@ -206,7 +243,7 @@ class QEMUMachine(object):
shutil.rmtree(self._temp_dir)
self._temp_dir = None
- def launch(self):
+ def launch(self, console_address=None):
"""
Launch the VM and make sure we cleanup and expose the
command line/output in case of exception
@@ -218,7 +255,7 @@ class QEMUMachine(object):
self._iolog = None
self._qemu_full_args = None
try:
- self._launch()
+ self._launch(console_address)
self._launched = True
except:
self.shutdown()
@@ -230,12 +267,14 @@ class QEMUMachine(object):
LOG.debug('Output: %r', self._iolog)
raise
- def _launch(self):
+ def _launch(self, console_address):
'''Launch the VM and establish a QMP connection'''
devnull = open(os.path.devnull, 'rb')
self._pre_launch()
+ bargs = self._base_args()
+ bargs.extend(self._create_console(console_address))
self._qemu_full_args = (self._wrapper + [self._binary] +
- self._base_args() + self._args)
+ bargs + self.args)
self._popen = subprocess.Popen(self._qemu_full_args,
stdin=devnull,
stdout=self._qemu_log_file,
diff --git a/tests/avocado/README.rst b/tests/avocado/README.rst
new file mode 100644
index 0000000000..a33c4a2577
--- /dev/null
+++ b/tests/avocado/README.rst
@@ -0,0 +1,132 @@
+========================================
+ QEMU tests using the Avocado Framework
+========================================
+
+This directory hosts functional tests written using Avocado Testing
+Framework.
+
+Installation
+============
+
+To install Avocado and the dependencies needed for these tests, run::
+
+ pip install --user avocado-framework avocado-framework-plugin-varianter-yaml-to-mux aexpect
+
+Alternatively, follow the instructions on this link::
+
+ http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado
+
+Overview
+========
+
+In this directory, an ``avocado_qemu`` package is provided, containing
+the ``test`` module, which inherits from ``avocado.Test`` and provides
+a builtin and easy-to-use Qemu virtual machine. Here's a template that
+can be used as reference to start writing your own tests::
+
+ from avocado_qemu import test
+
+ class MyTest(test.QemuTest):
+ """
+ :avocado: enable
+ """
+
+ def setUp(self):
+ self.vm.args.extend(['-m', '512'])
+ self.vm.launch()
+
+ def test_01(self):
+ res = self.vm.qmp('human-monitor-command',
+ command_line='info version')
+ self.assertIn('v2.9.0', res['return'])
+
+ def tearDown(self):
+ self.vm.shutdown()
+
+To execute your test, run::
+
+ avocado run test_my_test.py
+
+To execute all tests, run::
+
+ avocado run .
+
+If you don't specify the Qemu binary to use, the ``avocado_qemu``
+package will automatically probe it. The probe will try to use the Qemu
+binary from the git tree build directory, using the same architecture as
+the local system (if the architecture is not specified). If the Qemu
+binary is not available in the git tree build directory, the next try is
+to use the system installed Qemu binary.
+
+You can define a number of optional parameters, providing them via YAML
+file using the Avocado parameters system:
+
+- ``qemu_bin``: Use a given Qemu binary, skipping the automatic
+ probe. Example: ``qemu_bin: /usr/libexec/qemu-kvm``.
+- ``qemu_dst_bin``: Use a given Qemu binary to create the destination VM
+ when the migration process takes place. If it's not provided, the same
+ binary used in the source VM will be used for the destination VM.
+ Example: ``qemu_dst_bin: /usr/libexec/qemu-kvm-binary2``.
+- ``arch``: Probe the Qemu binary from a given architecture. It has no
+ effect if ``qemu_bin`` is specified. If not provided, the binary probe
+ will use the system architecture. Example: ``arch: x86_64``
+- ``image_path``: When a test requires (usually a bootable) image, this
+ parameter is used to define where the image is located. When undefined
+ it uses ``$QEMU_ROOT/bootable_image_$arch.qcow2``. The image is added
+ to the qemu command __only__ when the test requires an image. By
+ default ``,snapshot=on`` is used, but it can be altered by
+ ``image_snapshot`` parameter.
+- ``image_user`` and ``image_pass``: When using a ``image_path``, if you
+ want to get the console from the Guest OS you have to define the Guest
+ OS credentials. Example: ``image_user: avocado`` and
+ ``image_pass: p4ssw0rd``. Both parameters have defaults to ``avocado``.
+- ``machine_type``: Use this option to define a machine type for the VM.
+ Example: ``machine_type: pc``
+- ``machine_accel``: Use this option to define a machine acceleration
+ for the VM. Example: ``machine_accel: kvm``.
+- ``machine_kvm_type``: Use this option to select the KVM type when the
+ ``accel`` is ``kvm`` and there are more than one KVM types available.
+ Example: ``machine_kvm_type: PR``
+
+Run the test with::
+
+ $ avocado run test_my_test.py -m parameters.yaml
+
+Additionally, you can use a variants file to to set different values
+for each parameter. Using the YAML tag ``!mux`` Avocado will execute the
+tests once per combination of parameters. Example::
+
+ $ cat variants.yaml
+ architecture: !mux
+ x86_64:
+ arch: x86_64
+ i386:
+ arch: i386
+
+Run it the with::
+
+ $ avocado run test_my_test.py -m variants.yaml
+
+You can use both the parameters file and the variants file in the same
+command line::
+
+ $ avocado run test_my_test.py -m parameters.yaml variants.yaml
+
+Avocado will then merge the parameters from both files and create the
+proper variants.
+
+See ``avocado run --help`` and ``man avocado`` for several other
+options, such as ``--filter-by-tags``, ``--show-job-log``,
+``--failfast``, etc.
+
+Uninstallation
+==============
+
+If you've followed the installation instructions above, you can easily
+uninstall Avocado. Start by listing the packages you have installed::
+
+ pip list --user
+
+And remove any package you want with::
+
+ pip uninstall <package_name>
diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/avocado/avocado_qemu/test.py b/tests/avocado/avocado_qemu/test.py
new file mode 100644
index 0000000000..5a08dace45
--- /dev/null
+++ b/tests/avocado/avocado_qemu/test.py
@@ -0,0 +1,418 @@
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See LICENSE for more details.
+#
+# Copyright (C) 2017 Red Hat Inc
+#
+# Authors:
+# Amador Pahim <apahim@redhat.com>
+#
+# Based on code from:
+# https://github.com/avocado-framework/avocado-virt
+
+
+"""
+Avocado Qemu Test module to extend the Avocado Test module providing
+extra features intended for Qemu testing.
+"""
+
+
+import logging
+import os
+import re
+import sys
+import tempfile
+import time
+import uuid
+
+import aexpect
+
+from avocado import Test
+from avocado.utils import network
+from avocado.utils import process
+from avocado.utils import path as utils_path
+from avocado.utils import wait
+
+QEMU_ROOT = os.path.abspath(os.path.dirname(os.path.dirname(os.path.dirname(
+ os.path.dirname(__file__)))))
+sys.path.append(os.path.join(QEMU_ROOT, 'scripts'))
+import qemu
+
+
+class QEMULoginTimeoutError(Exception):
+ """
+ If timeout expires
+ """
+
+
+class QEMULoginAuthenticationError(Exception):
+ """
+ If authentication fails
+ """
+
+
+class QEMULoginProcessTerminatedError(Exception):
+ """
+ If the client terminates during login
+ """
+
+
+class QEMULoginError(Exception):
+ """
+ If some other error occurs
+ """
+
+
+class QEMUConsoleError(Exception):
+ """
+ If some error with the console access happens
+ """
+
+
+class QEMUMigrationError(Exception):
+ """
+ If some error with the migration happens
+ """
+
+
+class QEMUCloudinitError(Exception):
+ """
+ If some error with the cloudinit happens
+ """
+
+
+def _get_qemu_bin(arch):
+ git_root = process.system_output('git rev-parse --show-toplevel',
+ ignore_status=True,
+ verbose=False)
+ qemu_binary = os.path.join(git_root,
+ "%s-softmmu" % arch,
+ "qemu-system-%s" % arch)
+ if not os.path.exists(qemu_binary):
+ qemu_binary = utils_path.find_command('qemu-system-%s' % arch)
+ return qemu_binary
+
+
+def _handle_prompts(session, username, password, prompt, timeout=60,
+ debug=False):
+ """
+ Connect to a remote host (guest) using SSH or Telnet or else.
+
+ Wait for questions and provide answers. If timeout expires while
+ waiting for output from the child (e.g. a password prompt or
+ a shell prompt) -- fail.
+
+ :param session: An Expect or ShellSession instance to operate on
+ :param username: The username to send in reply to a login prompt
+ :param password: The password to send in reply to a password prompt
+ :param prompt: The shell prompt that indicates a successful login
+ :param timeout: The maximal time duration (in seconds) to wait for each
+ step of the login procedure (i.e. the "Are you sure" prompt, the
+ password prompt, the shell prompt, etc)
+ :raise QEMULoginTimeoutError: If timeout expires
+ :raise QEMULoginAuthenticationError: If authentication fails
+ :raise QEMULoginProcessTerminatedError: If the client terminates during login
+ :raise QEMULoginError: If some other error occurs
+ :return: If connect succeed return the output text to script for further
+ debug.
+ """
+ re_kernel_message = re.compile(r"^\[\s*\d+.\d+\] ")
+
+ def get_last_nonempty_line(cont):
+ """Return last non-empty non-kernel line"""
+ nonempty_lines = [_ for _ in cont.splitlines()
+ if _.strip() and not re_kernel_message.match(_)]
+ if nonempty_lines:
+ return nonempty_lines[-1]
+ else:
+ return ""
+
+ password_prompt_count = 0
+ login_prompt_count = 0
+ last_chance = False
+ # Send enter to refresh output (in case session was attached after boot)
+ session.sendline()
+ output = ""
+ while True:
+ try:
+ match, text = session.read_until_output_matches(
+ [r"[Aa]re you sure", r"[Pp]assword:\s*",
+ # Prompt of rescue mode for Red Hat.
+ r"\(or (press|type) Control-D to continue\):\s*",
+ r"[Gg]ive.*[Ll]ogin:\s*", # Prompt of rescue mode for SUSE.
+ r"(?<![Ll]ast )[Ll]ogin:\s*", # Don't match "Last Login:"
+ r"[Cc]onnection.*closed", r"[Cc]onnection.*refused",
+ r"[Pp]lease wait", r"[Ww]arning", r"[Ee]nter.*username",
+ r"[Ee]nter.*password", r"[Cc]onnection timed out", prompt,
+ r"Escape character is.*"], get_last_nonempty_line,
+ timeout=timeout, internal_timeout=0.5)
+ output += text
+ if match == 0: # "Are you sure you want to continue connecting"
+ if debug:
+ logging.debug("Got 'Are you sure...', sending 'yes'")
+ session.sendline("yes")
+ continue
+ elif match in [1, 2, 3, 10]: # "password:"
+ if password_prompt_count == 0:
+ if debug:
+ logging.debug("Got password prompt, sending '%s'",
+ password)
+ session.sendline(password)
+ password_prompt_count += 1
+ continue
+ else:
+ raise QEMULoginAuthenticationError("Got password prompt "
+ "twice", text)
+ elif match == 4 or match == 9: # "login:"
+ if login_prompt_count == 0 and password_prompt_count == 0:
+ if debug:
+ logging.debug("Got username prompt; sending '%s'",
+ username)
+ session.sendline(username)
+ login_prompt_count += 1
+ continue
+ else:
+ if login_prompt_count > 0:
+ msg = "Got username prompt twice"
+ else:
+ msg = "Got username prompt after password prompt"
+ raise QEMULoginAuthenticationError(msg, text)
+ elif match == 5: # "Connection closed"
+ raise QEMULoginError("Client said 'connection closed'", text)
+ elif match == 6: # "Connection refused"
+ raise QEMULoginError("Client said 'connection refused'", text)
+ elif match == 11: # Connection timeout
+ raise QEMULoginError("Client said 'connection timeout'", text)
+ elif match == 7: # "Please wait"
+ if debug:
+ logging.debug("Got 'Please wait'")
+ timeout = 30
+ continue
+ elif match == 8: # "Warning added RSA"
+ if debug:
+ logging.debug("Got 'Warning added RSA to known host list")
+ continue
+ elif match == 12: # prompt
+ if debug:
+ logging.debug("Got shell prompt -- logged in")
+ break
+ elif match == 13: # console prompt
+ logging.debug("Got console prompt, send return to show login")
+ session.sendline()
+ except aexpect.ExpectTimeoutError as details:
+ # sometimes, linux kernel print some message to console
+ # the message maybe impact match login pattern, so send
+ # a empty line to avoid unexpect login timeout
+ if not last_chance:
+ time.sleep(0.5)
+ session.sendline()
+ last_chance = True
+ continue
+ else:
+ raise QEMULoginTimeoutError(details.output)
+ except aexpect.ExpectProcessTerminatedError as details:
+ raise QEMULoginProcessTerminatedError(details.status, details.output)
+
+ return output
+
+
+class _VM(qemu.QEMUMachine):
+ '''A QEMU VM'''
+
+ def __init__(self, qemu_bin=None, arch=None, qemu_dst_bin=None,
+ username=None, password=None):
+ if arch is None:
+ arch = os.uname()[4]
+ self.arch = arch
+ self.ports = network.PortTracker()
+ self.name = "qemu-%s" % str(uuid.uuid4())[:8]
+ if qemu_bin is None:
+ qemu_bin = _get_qemu_bin(arch)
+ if qemu_dst_bin is None:
+ qemu_dst_bin = qemu_bin
+ self.qemu_bin = qemu_bin
+ self.qemu_dst_bin = qemu_dst_bin
+ self.username = username
+ self.password = password
+ super(_VM, self).__init__(qemu_bin, name=self.name, arch=arch)
+ logging.getLogger('QMP').setLevel(logging.INFO)
+
+ def get_console(self, console_address=None, prompt=r"[\#\$] "):
+ """
+ :param address: Socket address, can be either a unix socket path
+ (string) or a tuple in the form (address, port)
+ for a TCP connection
+ :param prompt: The regex to identify we reached the prompt.
+ """
+
+ if not all((self.username, self.password)):
+ raise QEMULoginError('Username or password not set.')
+
+ if not self.is_running():
+ raise QEMULoginError('VM is not running.')
+
+ if console_address is None:
+ if self._console_address is None:
+ raise QEMUConsoleError("Can't determine the console address "
+ "to connect to.")
+ else:
+ console_address = self._console_address
+
+ nc_cmd = 'nc'
+ if isinstance(console_address, tuple):
+ nc_cmd += ' %s %s' % (console_address[0], console_address[1])
+ else:
+ nc_cmd += ' -U %s' % console_address
+
+ console = aexpect.ShellSession(nc_cmd)
+ try:
+ logging.info('Console: Waiting login prompt...')
+ _handle_prompts(console, self.username, self.password, prompt)
+ logging.info('Console: Ready!')
+ except:
+ console.close()
+ raise
+
+ return console
+
+ def migrate(self, console_address=None, timeout=20):
+ def migrate_complete():
+ cmd = 'info migrate'
+ res = self.qmp('human-monitor-command', command_line=cmd)
+ if 'completed' in res['return']:
+ logging.info("Migration successful")
+ return True
+ elif 'failed' in res['return']:
+ logging.error(res)
+ raise QEMUMigrationError("Migration of %s failed" % self.name)
+ return False
+
+ port = self.ports.find_free_port()
+ newvm = _VM(self.qemu_dst_bin, self._arch, username=self.username,
+ password=self.password)
+ newvm.args = self.args
+ newvm.args.extend(['-incoming', 'tcp:0:%s' % port])
+ newvm.username = self.username
+ newvm.password = self.password
+
+ newvm.launch(console_address)
+ cmd = 'migrate -d tcp:0:%s' % port
+ self.qmp('human-monitor-command', command_line=cmd)
+ mig_result = wait.wait_for(migrate_complete, timeout=timeout,
+ text='Waiting for migration to complete')
+
+ if mig_result is None:
+ raise QEMUMigrationError("Migration of %s did not complete after "
+ "%s s" % (self.name, timeout))
+
+ return newvm
+
+ def add_image(self, path, username=None, password=None, cloudinit=False,
+ snapshot=True, extra=None):
+ """
+ Adds the '-drive' command line option and its parameters to
+ the Qemu VM
+
+ :param path: Image path (i.e. /var/lib/images/guestos.qcow2)
+ :param username: The username to log into the Guest OS with
+ :param password: The password to log into the Guest OS with
+ :param cloudinit: Whether the cloudinit cdrom will be attached to
+ the image
+ :param snapshot: Whether the parameter snapshot=on will be used
+ :param extra: Extra parameters to the -drive option
+ """
+ file_option = 'file=%s' % path
+ for item in self.args:
+ if file_option in item:
+ logging.error('Image %s already present', path)
+ return
+
+ if extra is not None:
+ file_option += ',%s' % extra
+
+ if snapshot:
+ file_option += ',snapshot=on'
+
+ self.args.extend(['-drive', file_option])
+
+ if username is not None:
+ self.username = username
+
+ if password is not None:
+ self.password = password
+
+ if cloudinit:
+ self._cloudinit()
+
+ def _cloudinit(self):
+ """
+ Creates a CDROM Iso Image with the required cloudinit files
+ (meta-data and user-data) to make the initial Cloud Image
+ configuration, attaching the CDROM to the VM.
+ """
+ try:
+ geniso_bin = utils_path.find_command('genisoimage')
+ except:
+ raise QEMUCloudinitError('Command not found (genisoimage)')
+
+ data_dir = tempfile.mkdtemp()
+
+ metadata_path = os.path.join(data_dir, 'meta-data')
+ metadata_content = ("instance-id: %s\n"
+ "local-hostname: %s\n" % (self.name, self.name))
+ with open(metadata_path, 'w') as metadata_file:
+ metadata_file.write(metadata_content)
+
+ userdata_path = os.path.join(data_dir, 'user-data')
+ userdata_content = ("#cloud-config\n"
+ "password: %s\n"
+ "ssh_pwauth: True\n"
+ "chpasswd: { expire: False }\n"
+ "system_info:\n"
+ " default_user:\n"
+ " name: %s\n" %
+ (self.password, self.username))
+
+ with open(userdata_path, 'w') as userdata_file:
+ userdata_file.write(userdata_content)
+
+ iso_path = os.path.join(data_dir, 'cdrom.iso')
+ process.run("%s -output %s -volid cidata -joliet -rock %s %s" %
+ (geniso_bin, iso_path, metadata_path, userdata_path))
+
+ self.args.extend(['-cdrom', iso_path])
+
+class QemuTest(Test):
+
+ def __init__(self, methodName=None, name=None, params=None,
+ base_logdir=None, job=None, runner_queue=None):
+ super(QemuTest, self).__init__(methodName=methodName, name=name,
+ params=params, base_logdir=base_logdir,
+ job=job, runner_queue=runner_queue)
+ self.vm = _VM(qemu_bin=self.params.get('qemu_bin'),
+ arch=self.params.get('arch'),
+ qemu_dst_bin=self.params.get('qemu_dst_bin'),
+ username=self.params.get('image_user',
+ default='avocado'),
+ password=self.params.get('image_pass',
+ default='avocado'))
+
+ machine_type = self.params.get('machine_type')
+ machine_accel = self.params.get('machine_accel')
+ machine_kvm_type = self.params.get('machine_kvm_type')
+ machine = ""
+ if machine_type is not None:
+ machine += "%s," % machine_type
+ if machine_accel is not None:
+ machine += "accel=%s," % machine_accel
+ if machine_kvm_type is not None:
+ machine += "kvm-type=%s," % machine_kvm_type
+ if machine:
+ self.vm.args.extend(['-machine', machine])
diff --git a/tests/avocado/parameters.yaml b/tests/avocado/parameters.yaml
new file mode 100644
index 0000000000..03c4ed1416
--- /dev/null
+++ b/tests/avocado/parameters.yaml
@@ -0,0 +1,19 @@
+# Probe the Qemu binary from a given architecture. It has no effect if
+# 'qemu_bin' is specified. If not provided, the binary probe will use
+# the local system architecture.
+arch: null
+
+# Use a given Qemu binary, skipping the automatic probe.
+qemu_bin: null
+# Use a given Qemu binary to create the destination VM when the
+# migration process is called. If it's not provided, the same binary
+# used in the source VM will be used for the destination VM.
+qemu_dst_bin: null
+
+# Use this option to define a machine type for the VM.
+machine_type: null
+# Use this option to define a machine acceleration for the VM.
+machine_accel: null
+# Use this option to select the KVM type when the 'machine_accel' is set
+# to 'kvm' and there are more than one KVM types available.
+machine_kvm_type: null
diff --git a/tests/avocado/test_info_memdev_host_nodes.py b/tests/avocado/test_info_memdev_host_nodes.py
new file mode 100644
index 0000000000..69891b723d
--- /dev/null
+++ b/tests/avocado/test_info_memdev_host_nodes.py
@@ -0,0 +1,66 @@
+from avocado import main
+from avocado_qemu import test
+
+
+class TestInfoMemdev(test.QemuTest):
+ """
+
+ :avocado: enable
+ :avocado: tags=qmp,object_add,device_add,memdev
+ """
+
+ def setUp(self):
+ self.vm.args.extend(['-m', '4G,slots=32,maxmem=40G'])
+ self.vm.launch()
+
+ def test_hotplug_memory_default_policy(self):
+ """
+ According to the RHBZ1431939, the issue is 'host nodes'
+ returning '128'. It should return empty value when memory
+ hotplug default policy is used.
+
+ Fixed in commit d81d857f4421d205395d55200425daa6591c28a5.
+ :avocado: tags=RHBZ1431939
+ """
+
+ cmd = 'object_add memory-backend-ram,id=mem1,size=1G'
+ res = self.vm.qmp('human-monitor-command', command_line=cmd)
+ self.assertEqual('', res['return'])
+
+ cmd = 'device_add pc-dimm,id=dimm1,memdev=mem1'
+ res = self.vm.qmp('human-monitor-command', command_line=cmd)
+ self.assertEqual('', res['return'])
+
+ cmd = 'info memdev'
+ res = self.vm.qmp('human-monitor-command', command_line=cmd)
+ self.assertIn('policy: default\r', res['return'])
+ self.assertIn('host nodes: \r', res['return'])
+
+ def test_hotplug_memory_bind_policy(self):
+ """
+ According to the RHBZ1431939, the issue is 'host nodes'
+ returning '128'. It should return 0 when memory hotplug
+ bind policy is used.
+
+ Fixed in commit d81d857f4421d205395d55200425daa6591c28a5.
+ :avocado: tags=RHBZ1431939
+ """
+
+ cmd = 'object_add memory-backend-ram,id=mem1,host-nodes=0,size=2G,policy=bind'
+ res = self.vm.qmp('human-monitor-command', command_line=cmd)
+ self.assertEqual('', res['return'])
+
+ cmd = 'device_add pc-dimm,id=dimm1,memdev=mem1'
+ res = self.vm.qmp('human-monitor-command', command_line=cmd)
+ self.assertEqual('', res['return'])
+
+ cmd = 'info memdev'
+ res = self.vm.qmp('human-monitor-command', command_line=cmd)
+ self.assertIn('policy: bind\r', res['return'])
+ self.assertIn('host nodes: 0\r', res['return'])
+
+ def tearDown(self):
+ self.vm.shutdown()
+
+if __name__ == "__main__":
+ avocado.main()
diff --git a/tests/avocado/test_nec-usb-xhci.py b/tests/avocado/test_nec-usb-xhci.py
new file mode 100644
index 0000000000..c29b5ebaa1
--- /dev/null
+++ b/tests/avocado/test_nec-usb-xhci.py
@@ -0,0 +1,63 @@
+import copy
+import os
+import tempfile
+
+from avocado_qemu import test
+from avocado.utils import process
+from avocado.utils import vmimage
+
+class TestNecUsbXhci(test.QemuTest):
+ """
+ Run with:
+
+ avocado run test_nec-usb-xhci.py \
+ -m test_nec-usb-xhci.py.data/parameters.yaml
+
+ :avocado: enable
+ :avocado: tags=usbstorage
+ """
+
+ def setUp(self):
+ self.vm_dst = None
+ self.image = vmimage.get('Fedora')
+ self.vm.add_image(self.image.path, cloudinit=True, snapshot=False)
+ self.vm.args.extend(['-machine', 'accel=kvm'])
+
+ usbdevice = os.path.join(self.workdir, 'usb.img')
+ process.run('dd if=/dev/zero of=%s bs=1M count=10' % usbdevice)
+ self.vm.args.extend(['-device', 'pci-bridge,id=bridge1,chassis_nr=1'])
+ self.vm.args.extend(['-device', 'nec-usb-xhci,id=xhci1,bus=bridge1,addr=0x3'])
+ self.vm.args.extend(['-drive', 'file=%s,format=raw,id=drive_usb,if=none' % usbdevice])
+ self.vm.args.extend(['-device', 'usb-storage,drive=drive_usb,id=device_usb,bus=xhci1.0'])
+ self.vm.launch()
+
+ def test_available_after_migration(self):
+ """
+ According to the RHBZ1436616, the issue is: usb-storage device
+ under pci-bridge is unusable after migration.
+
+ Fixed in commit 243afe858b95765b98d16a1f0dd50dca262858ad.
+
+ :avocado: tags=migration,RHBZ1436616
+ """
+
+ console = self.vm.get_console()
+ console.sendline('sudo fdisk -l')
+ result = console.read_up_to_prompt()
+ console.close()
+ self.assertIn('Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors',
+ result)
+
+ self.vm_dst = self.vm.migrate()
+ console = self.vm_dst.get_console()
+ console.sendline('sudo fdisk -l')
+ result = console.read_up_to_prompt()
+ console.close()
+ self.assertIn('Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors',
+ result)
+
+ def tearDown(self):
+ self.vm.shutdown()
+ if self.vm_dst is not None:
+ self.vm_dst.shutdown()
+ os.remove(self.image.path)
diff --git a/tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml b/tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml
new file mode 100644
index 0000000000..37a4e9dc37
--- /dev/null
+++ b/tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml
@@ -0,0 +1,4 @@
+machine_accel: kvm
+image_path: /var/lib/images/fedora-25.img
+image_user: root
+image_pass: p4ssw0rd
diff --git a/tests/avocado/test_numa_hotplug.py b/tests/avocado/test_numa_hotplug.py
new file mode 100644
index 0000000000..256ec0f49f
--- /dev/null
+++ b/tests/avocado/test_numa_hotplug.py
@@ -0,0 +1,120 @@
+import re
+import time
+
+from avocado_qemu import test
+from avocado.utils import vmimage
+
+
+class TestNumaHotplug(test.QemuTest):
+ """
+ Verifies that "info numa" and "/sys/devices/system/node/" contains
+ correct values before/after inserting memory devices into default
+ and then into 13th numa node.
+
+ Associated bug trackers: RHBZ1473203
+ https://bugzilla.redhat.com/show_bug.cgi?id=1473203
+
+ Fixed in kernel commit dc421b200f91930c9c6a9586810ff8c232cf10fc.
+
+ :avocado: enable
+ :avocado: tags=RHBZ1473203,requires_linux,numa,memory,ppc64le
+ """
+
+ def setUp(self):
+ self.image = vmimage.get('Fedora')
+ self.vm.add_image(self.image.path, cloudinit=True, snapshot=False)
+
+ self.vm.args.extend(['-machine', 'accel=kvm'])
+ self.vm.args.extend(["-m", "4G,slots=208,maxmem=80G"])
+ self.vm.args.extend(["-numa", "node"] * 16)
+ self.vm.launch()
+
+ def check_mem_console(self, console, exp):
+ """
+ Verifies that memory layout is according to exp using console/ssh
+
+ :param console: session
+ :param exp: list of MemTotals per node in MB, tolerance is +-100MB
+ """
+ out = console.cmd_output_safe("echo /sys/devices/system/node/node*")
+ nodes = re.findall(r"/sys/devices/system/node/node\d+", out)
+ self.assertEqual(len(nodes), len(exp), "Number of nodes is not "
+ "%s:\n%s" % (len(exp), out))
+ for i in xrange(len(exp)):
+ out = console.cmd_output_safe("cat /sys/devices/system/node/"
+ "node%s/meminfo" % i)
+ mem = re.search(r"MemTotal:\s*(\d+) kB", out)
+ self.assertTrue(mem, "Failed to obtain node%s MemTotal:\n%s"
+ % (i, out))
+ _exp = exp[i] * 1024
+ mem = int(mem.group(1))
+ self.assertGreater(mem, _exp - 102400, "TotalMem of node%s is not "
+ "%s+-51200 kb (%s)" % (i, _exp, mem))
+ self.assertLess(mem, _exp + 102400, "TotalMem of node%s is not "
+ "%s+-51200 kb (%s)" % (i, _exp, mem))
+
+ def check_mem_monitor(self, monitor, exp):
+ """
+ Verifies that memory layout is according to exp using QMP monitor
+
+ :param console: session
+ :param exp: list of MemTotals per node in MB, tolerance is +-100MB
+ """
+ ret = monitor("human-monitor-command", command_line="info numa")
+ out = ret["return"]
+ self.assertTrue(out.startswith("%s nodes" % len(exp)), "Number of "
+ "nodes is not %s:\n%s" % (len(exp), out))
+ for i in xrange(len(exp)):
+ _exp = "node %s size: %s MB" % (i, exp[i])
+ self.assertIn(_exp, out, "%s is not in 'info numa' output, "
+ "probably wrong memory size reported:\n%s"
+ % (_exp, out))
+
+ @staticmethod
+ def _retry_until_timeout(timeout, func, *args, **kwargs):
+ """
+ Repeat the function until it returns anything ignoring AssertionError.
+ After the deadline repeate the function one more time without ignoring
+ timeout.
+ """
+ end = time.time() + timeout
+ while time.time() < end:
+ try:
+ ret = func(*args, **kwargs)
+ except AssertionError:
+ continue
+ break
+ else:
+ ret = func(*args, **kwargs)
+ return ret
+
+ def test_hotplug_mem_into_node(self):
+ console = self.vm.get_console()
+ exp = [256] * 16
+ self.check_mem_monitor(self.vm.qmp, exp)
+ self.check_mem_console(console, exp)
+ cmd = "object_add memory-backend-ram,id=mem2,size=1G"
+ res = self.vm.qmp("human-monitor-command", command_line=cmd)
+ self.assertEqual(res["return"], "")
+ cmd = "device_add pc-dimm,id=dimm2,memdev=mem2"
+ res = self.vm.qmp("human-monitor-command", command_line=cmd)
+ self.assertEqual(res["return"], "")
+ exp = [1280] + [256] * 15
+ self.check_mem_monitor(self.vm.qmp, exp)
+ # Wait up to 10s to propagate the changes
+ self._retry_until_timeout(10, self.check_mem_console, console, exp)
+ cmd = "object_add memory-backend-ram,id=mem8,size=1G"
+ res = self.vm.qmp("human-monitor-command", command_line=cmd)
+ self.assertEqual(res["return"], "")
+ cmd = "device_add pc-dimm,id=dimm8,memdev=mem8,node=13"
+ res = self.vm.qmp("human-monitor-command", command_line=cmd)
+ self.assertEqual(res["return"], "")
+ time.sleep(5)
+ exp = [1280] + [256] * 12 + [1280] + [256] * 2
+ self.check_mem_monitor(self.vm.qmp, exp)
+ # Wait up to 10s to propagate the changes
+ self._retry_until_timeout(10, self.check_mem_console, console, exp)
+ console.close()
+
+ def tearDown(self):
+ self.vm.shutdown()
diff --git a/tests/avocado/test_ovmf_with_240_vcpus.py b/tests/avocado/test_ovmf_with_240_vcpus.py
new file mode 100644
index 0000000000..da688dbc76
--- /dev/null
+++ b/tests/avocado/test_ovmf_with_240_vcpus.py
@@ -0,0 +1,70 @@
+import os
+import shutil
+import sys
+
+from avocado import main
+from avocado_qemu import test
+
+
+class TestOvmfVcpus(test.QemuTest):
+ """
+ Run with:
+
+ avocado run test_ovmf_with_240_vcpus.py \
+ -m test_ovmf_with_240_vcpus.py.data/parameters.yaml
+
+ :avocado: enable
+ :avocado: tags=ovmf
+ """
+
+ def setUp(self):
+ ovmf_code_path = self.params.get('OVMF_CODE',
+ default='/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd')
+ ovmf_vars_path = self.params.get('OVMF_VARS',
+ default='/usr/share/edk2/ovmf/OVMF_VARS.fd')
+ if not ovmf_code_path or not os.path.exists(ovmf_code_path):
+ basename = os.path.basename(__file__)
+ self.cancel('OVMF_CODE file not found. Set the correct '
+ 'path on "%s.data/parameters.yaml" and run this test '
+ 'with: "avocado run %s -m %s.data/parameters.yaml"' %
+ (basename, basename, basename))
+ if not ovmf_vars_path or not os.path.exists(ovmf_vars_path):
+ basename = os.path.basename(__file__)
+ self.cancel('OVMF_VARS file not found. Set the correct '
+ 'path on "%s.data/parameters.yaml" and run this test '
+ 'with: "avocado run %s -m %s.data/parameters.yaml"' %
+ (basename, basename, basename))
+
+ ovmf_vars_tmp = os.path.join(self.workdir,
+ os.path.basename(ovmf_vars_path))
+ if not os.path.exists(ovmf_vars_tmp):
+ shutil.copy(ovmf_vars_path, self.workdir)
+
+ self.vm.args.extend(['-drive',
+ 'file=%s,if=pflash,format=raw,readonly=on,unit=0' %
+ ovmf_code_path])
+ self.vm.args.extend(['-drive',
+ 'file=%s,if=pflash,format=raw,unit=1' %
+ ovmf_vars_tmp])
+
+ self.vm.args.extend(['-smp', '240'])
+
+ def test_run_vm(self):
+ """
+ According to the RHBZ1447027, the issue is: Guest cannot boot
+ with 240 or above vcpus when using ovmf.
+ Fixed in commit e85c0d14014514a2f0faeae5b4c23fab5b234de4.
+
+ :avocado: tags=RHBZ1447027
+ """
+
+ try:
+ self.vm.launch()
+ except Exception as details:
+ self.fail(details)
+
+ def tearDown(self):
+ self.vm.shutdown()
+
+if __name__ == "__main__":
+ avocado.main()
diff --git a/tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml b/tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml
new file mode 100644
index 0000000000..79f6da1d29
--- /dev/null
+++ b/tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml
@@ -0,0 +1,2 @@
+OVMF_CODE: /usr/share/edk2/ovmf/OVMF_CODE.secboot.fd
+OVMF_VARS: /usr/share/edk2/ovmf/OVMF_VARS.fd
diff --git a/tests/avocado/variants.yaml b/tests/avocado/variants.yaml
new file mode 100644
index 0000000000..6fc689b3ba
--- /dev/null
+++ b/tests/avocado/variants.yaml
@@ -0,0 +1,62 @@
+architecture: !mux
+ # Set the architecture of the qemu binary to execute tests
+ # with. This setting has no effect if you're using custom
+ # qemu_bin configuration.
+ x86_64:
+ arch: x86_64
+ aarch64:
+ arch: aarch64
+ alpha:
+ arch: alpha
+ arm:
+ arch: arm
+ cris:
+ arch: cris
+ i386:
+ arch: i386
+ lm32:
+ arch: lm32
+ m68k:
+ arch: m68k
+ microblazeel:
+ arch: microblazeel
+ microblaze:
+ arch: microblaze
+ mips64el:
+ arch: mips64el
+ mips64:
+ arch: mips64
+ mipsel:
+ arch: mipsel
+ mips:
+ arch: mips
+ moxie:
+ arch: moxie
+ nios2:
+ arch: nios2
+ or1k:
+ arch: or1k
+ ppc64:
+ arch: ppc64
+ ppcemb:
+ arch: ppcemb
+ ppc:
+ arch: ppc
+ s390x:
+ arch: s390x
+ sh4eb:
+ arch: sh4eb
+ sh4:
+ arch: sh4
+ sparc64:
+ arch: sparc64
+ sparc:
+ arch: sparc
+ tricore:
+ arch: tricore
+ unicore32:
+ arch: unicore32
+ xtensaeb:
+ arch: xtensaeb
+ xtensa:
+ arch: xtensa
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index b25d48a91b..a2e4f03743 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -293,18 +293,18 @@ class VM(qtest.QEMUQtestMachine):
self._num_drives = 0
def add_object(self, opts):
- self._args.append('-object')
- self._args.append(opts)
+ self.args.append('-object')
+ self.args.append(opts)
return self
def add_device(self, opts):
- self._args.append('-device')
- self._args.append(opts)
+ self.args.append('-device')
+ self.args.append(opts)
return self
def add_drive_raw(self, opts):
- self._args.append('-drive')
- self._args.append(opts)
+ self.args.append('-drive')
+ self.args.append(opts)
return self
def add_drive(self, path, opts='', interface='virtio', format=imgfmt):
@@ -322,27 +322,27 @@ class VM(qtest.QEMUQtestMachine):
if format == 'luks' and 'key-secret' not in opts:
# default luks support
- if luks_default_secret_object not in self._args:
+ if luks_default_secret_object not in self.args:
self.add_object(luks_default_secret_object)
options.append(luks_default_key_secret_opt)
- self._args.append('-drive')
- self._args.append(','.join(options))
+ self.args.append('-drive')
+ self.args.append(','.join(options))
self._num_drives += 1
return self
def add_blockdev(self, opts):
- self._args.append('-blockdev')
+ self.args.append('-blockdev')
if isinstance(opts, str):
- self._args.append(opts)
+ self.args.append(opts)
else:
- self._args.append(','.join(opts))
+ self.args.append(','.join(opts))
return self
def add_incoming(self, addr):
- self._args.append('-incoming')
- self._args.append(addr)
+ self.args.append('-incoming')
+ self.args.append(addr)
return self
def pause_drive(self, drive, event=None):
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 2/7] avocado: Update python scripts to upstream codebase
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 1/7] AVOCADO_QEMU: Snapshot commit Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-05-01 0:56 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp() Philippe Mathieu-Daudé
` (7 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Eduardo Habkost, Kevin Wolf, Max Reitz,
open list:Block layer core
QEMUMachine arguments member is called '_args'.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
scripts/qemu.py | 14 +++++++-------
tests/avocado/avocado_qemu/test.py | 12 ++++++------
tests/qemu-iotests/iotests.py | 28 ++++++++++++++--------------
3 files changed, 27 insertions(+), 27 deletions(-)
diff --git a/scripts/qemu.py b/scripts/qemu.py
index bd66620f45..0eecc44d09 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -81,7 +81,7 @@ class QEMUMachine(object):
self._qemu_log_file = None
self._popen = None
self._binary = binary
- self.args = list(args) # Force copy args in case we modify them
+ self._args = list(args) # Force copy args in case we modify them
self._wrapper = wrapper
self._events = []
self._iolog = None
@@ -109,8 +109,8 @@ class QEMUMachine(object):
# This can be used to add an unused monitor instance.
def add_monitor_telnet(self, ip, port):
args = 'tcp:%s:%d,server,nowait,telnet' % (ip, port)
- self.args.append('-monitor')
- self.args.append(args)
+ self._args.append('-monitor')
+ self._args.append(args)
def add_fd(self, fd, fdset, opaque, opts=''):
'''Pass a file descriptor to the VM'''
@@ -120,8 +120,8 @@ class QEMUMachine(object):
if opts:
options.append(opts)
- self.args.append('-add-fd')
- self.args.append(','.join(options))
+ self._args.append('-add-fd')
+ self._args.append(','.join(options))
return self
def send_fd_scm(self, fd_file_path):
@@ -184,7 +184,7 @@ class QEMUMachine(object):
'-display', 'none', '-vga', 'none']
def _create_console(self, console_address):
- for item in self.args:
+ for item in self._args:
for option in ['isa-serial', 'spapr-vty', 'sclpconsole']:
if option in item:
return []
@@ -274,7 +274,7 @@ class QEMUMachine(object):
bargs = self._base_args()
bargs.extend(self._create_console(console_address))
self._qemu_full_args = (self._wrapper + [self._binary] +
- bargs + self.args)
+ bargs + self._args)
self._popen = subprocess.Popen(self._qemu_full_args,
stdin=devnull,
stdout=self._qemu_log_file,
diff --git a/tests/avocado/avocado_qemu/test.py b/tests/avocado/avocado_qemu/test.py
index 5a08dace45..1ead917014 100644
--- a/tests/avocado/avocado_qemu/test.py
+++ b/tests/avocado/avocado_qemu/test.py
@@ -297,8 +297,8 @@ class _VM(qemu.QEMUMachine):
port = self.ports.find_free_port()
newvm = _VM(self.qemu_dst_bin, self._arch, username=self.username,
password=self.password)
- newvm.args = self.args
- newvm.args.extend(['-incoming', 'tcp:0:%s' % port])
+ newvm._args = self._args
+ newvm._args.extend(['-incoming', 'tcp:0:%s' % port])
newvm.username = self.username
newvm.password = self.password
@@ -329,7 +329,7 @@ class _VM(qemu.QEMUMachine):
:param extra: Extra parameters to the -drive option
"""
file_option = 'file=%s' % path
- for item in self.args:
+ for item in self._args:
if file_option in item:
logging.error('Image %s already present', path)
return
@@ -340,7 +340,7 @@ class _VM(qemu.QEMUMachine):
if snapshot:
file_option += ',snapshot=on'
- self.args.extend(['-drive', file_option])
+ self._args.extend(['-drive', file_option])
if username is not None:
self.username = username
@@ -387,7 +387,7 @@ class _VM(qemu.QEMUMachine):
process.run("%s -output %s -volid cidata -joliet -rock %s %s" %
(geniso_bin, iso_path, metadata_path, userdata_path))
- self.args.extend(['-cdrom', iso_path])
+ self._args.extend(['-cdrom', iso_path])
class QemuTest(Test):
@@ -415,4 +415,4 @@ class QemuTest(Test):
if machine_kvm_type is not None:
machine += "kvm-type=%s," % machine_kvm_type
if machine:
- self.vm.args.extend(['-machine', machine])
+ self.vm._args.extend(['-machine', machine])
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index a2e4f03743..b25d48a91b 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -293,18 +293,18 @@ class VM(qtest.QEMUQtestMachine):
self._num_drives = 0
def add_object(self, opts):
- self.args.append('-object')
- self.args.append(opts)
+ self._args.append('-object')
+ self._args.append(opts)
return self
def add_device(self, opts):
- self.args.append('-device')
- self.args.append(opts)
+ self._args.append('-device')
+ self._args.append(opts)
return self
def add_drive_raw(self, opts):
- self.args.append('-drive')
- self.args.append(opts)
+ self._args.append('-drive')
+ self._args.append(opts)
return self
def add_drive(self, path, opts='', interface='virtio', format=imgfmt):
@@ -322,27 +322,27 @@ class VM(qtest.QEMUQtestMachine):
if format == 'luks' and 'key-secret' not in opts:
# default luks support
- if luks_default_secret_object not in self.args:
+ if luks_default_secret_object not in self._args:
self.add_object(luks_default_secret_object)
options.append(luks_default_key_secret_opt)
- self.args.append('-drive')
- self.args.append(','.join(options))
+ self._args.append('-drive')
+ self._args.append(','.join(options))
self._num_drives += 1
return self
def add_blockdev(self, opts):
- self.args.append('-blockdev')
+ self._args.append('-blockdev')
if isinstance(opts, str):
- self.args.append(opts)
+ self._args.append(opts)
else:
- self.args.append(','.join(opts))
+ self._args.append(','.join(opts))
return self
def add_incoming(self, addr):
- self.args.append('-incoming')
- self.args.append(addr)
+ self._args.append('-incoming')
+ self._args.append(addr)
return self
def pause_drive(self, drive, event=None):
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp()
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 1/7] AVOCADO_QEMU: Snapshot commit Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 2/7] avocado: Update python scripts to upstream codebase Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-05-01 19:30 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 4/7] qemu.py: Avoid deprecated tempfile.mktemp() Philippe Mathieu-Daudé
` (6 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Eduardo Habkost
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
scripts/qemu.py | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/scripts/qemu.py b/scripts/qemu.py
index 0eecc44d09..379767b62f 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -189,6 +189,16 @@ class QEMUMachine(object):
if option in item:
return []
+ device = '{dev_type},chardev=console'
+ if '86' in self._arch:
+ device = device.format(dev_type='isa-serial')
+ elif 'ppc' in self._arch:
+ device = device.format(dev_type='spapr-vty')
+ elif 's390x' in self._arch:
+ device = device.format(dev_type='sclpconsole')
+ else:
+ return []
+
chardev = 'socket,id=console,{address},server,nowait'
if console_address is None:
console_address = tempfile.mktemp()
@@ -203,16 +213,6 @@ class QEMUMachine(object):
self._console_address = console_address
- device = '{dev_type},chardev=console'
- if '86' in self._arch:
- device = device.format(dev_type='isa-serial')
- elif 'ppc' in self._arch:
- device = device.format(dev_type='spapr-vty')
- elif 's390x' in self._arch:
- device = device.format(dev_type='sclpconsole')
- else:
- return []
-
return ['-chardev', chardev,
'-device', device]
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 4/7] qemu.py: Avoid deprecated tempfile.mktemp()
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp() Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-05-01 19:33 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 5/7] avocado: Add an optional flag 'login' to get_console() Philippe Mathieu-Daudé
` (5 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Eduardo Habkost
We already have secure temporary directory created with tempfile.mkdtemp().
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
scripts/qemu.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/scripts/qemu.py b/scripts/qemu.py
index 379767b62f..26c4b1fd3b 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -201,7 +201,8 @@ class QEMUMachine(object):
chardev = 'socket,id=console,{address},server,nowait'
if console_address is None:
- console_address = tempfile.mktemp()
+ console_address = os.path.join(self._temp_dir,
+ self._name + "-console.sock")
chardev = chardev.format(address='path=%s' %
console_address)
elif isinstance(console_address, tuple):
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 5/7] avocado: Add an optional flag 'login' to get_console()
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (3 preceding siblings ...)
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 4/7] qemu.py: Avoid deprecated tempfile.mktemp() Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-05-01 19:46 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console Philippe Mathieu-Daudé
` (4 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis
Not all consoles require users to log in :/
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
tests/avocado/avocado_qemu/test.py | 23 ++++++++++++-----------
1 file changed, 12 insertions(+), 11 deletions(-)
diff --git a/tests/avocado/avocado_qemu/test.py b/tests/avocado/avocado_qemu/test.py
index 1ead917014..ac8ee6d250 100644
--- a/tests/avocado/avocado_qemu/test.py
+++ b/tests/avocado/avocado_qemu/test.py
@@ -244,7 +244,7 @@ class _VM(qemu.QEMUMachine):
super(_VM, self).__init__(qemu_bin, name=self.name, arch=arch)
logging.getLogger('QMP').setLevel(logging.INFO)
- def get_console(self, console_address=None, prompt=r"[\#\$] "):
+ def get_console(self, console_address=None, prompt=r"[\#\$] ", login=True):
"""
:param address: Socket address, can be either a unix socket path
(string) or a tuple in the form (address, port)
@@ -252,9 +252,6 @@ class _VM(qemu.QEMUMachine):
:param prompt: The regex to identify we reached the prompt.
"""
- if not all((self.username, self.password)):
- raise QEMULoginError('Username or password not set.')
-
if not self.is_running():
raise QEMULoginError('VM is not running.')
@@ -272,13 +269,17 @@ class _VM(qemu.QEMUMachine):
nc_cmd += ' -U %s' % console_address
console = aexpect.ShellSession(nc_cmd)
- try:
- logging.info('Console: Waiting login prompt...')
- _handle_prompts(console, self.username, self.password, prompt)
- logging.info('Console: Ready!')
- except:
- console.close()
- raise
+ if login:
+ if not all((self.username, self.password)):
+ raise QEMULoginError('Username or password not set.')
+
+ try:
+ logging.info('Console: Waiting login prompt...')
+ _handle_prompts(console, self.username, self.password, prompt)
+ except:
+ console.close()
+ raise
+ logging.info('Console: Ready!')
return console
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (4 preceding siblings ...)
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 5/7] avocado: Add an optional flag 'login' to get_console() Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-04-19 19:38 ` Richard Henderson
2018-05-01 21:17 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 7/7] avocado: Add tests booting the Malta machine Philippe Mathieu-Daudé
` (3 subsequent siblings)
9 siblings, 2 replies; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Richard Henderson
Booting an Alpha DP264 machine.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
tests/avocado/test_linux-boot-console.py | 88 +++++++++++++++++++
.../parameters.yaml | 3 +
2 files changed, 91 insertions(+)
create mode 100644 tests/avocado/test_linux-boot-console.py
create mode 100644 tests/avocado/test_linux-boot-console.py.data/parameters.yaml
diff --git a/tests/avocado/test_linux-boot-console.py b/tests/avocado/test_linux-boot-console.py
new file mode 100644
index 0000000000..f54d10dd79
--- /dev/null
+++ b/tests/avocado/test_linux-boot-console.py
@@ -0,0 +1,88 @@
+# -*- coding: utf-8 -*-
+#
+# Boot a Linux kernel on the Malta board and check the serial console output
+#
+# Copyright (C) 2018 Philippe Mathieu-Daudé
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Run with:
+#
+# avocado run test_linux-boot-console.py \
+# --mux-yaml test_linux-boot-console.py.data/parameters.yaml
+# [--filter-by-tags arch_alpha ...]
+
+import os
+import tempfile
+import hashlib
+import urllib2
+import gzip
+import shutil
+
+from avocado import skipUnless
+from avocado_qemu import test
+
+# XXX Use a cleaner CacheStorage or avocado.utils.[archive|kernel]?
+def cachedfile(url, cachedir='/tmp/mycachedir', gzip_compressed=False):
+ if not os.path.isdir(cachedir):
+ os.mkdir(cachedir)
+ key = hashlib.md5(url).hexdigest()
+ fbin = cachedir + "/" + key + ".bin"
+ if not os.path.isfile(fbin):
+ with open(cachedir + "/" + key + ".url", "w") as f:
+ f.write(url + '\n')
+ with open(fbin, "wb") as f:
+ content = urllib2.urlopen(url).read()
+ f.write(content)
+ if gzip_compressed:
+ fgz = cachedir + "/" + key + ".gz"
+ shutil.move(fbin, fgz)
+ with gzip.open(fgz, 'rb') as f_in, open(fbin, 'wb') as f_out:
+ shutil.copyfileobj(f_in, f_out)
+ return fbin
+
+
+class TestAlphaClipperBoot2_6(test.QemuTest):
+ """
+ :avocado: enable
+ :avocado: tags=arch_alpha
+ """
+ ARCH = "alpha"
+
+ def kernel_url(self):
+ return 'http://archive.debian.org/debian/dists/lenny/main/installer-alpha/current/images/cdrom/vmlinuz'
+
+ def setUp(self):
+ self.console_path = tempfile.mkstemp()[1]
+ kernel_path = cachedfile(self.kernel_url(), gzip_compressed=True)
+ self.vm._args.extend(['-machine', 'clipper'])
+ self.vm._args.extend(['-m', '64'])
+ self.vm._args.extend(['-kernel', kernel_path])
+ self.vm._args.extend(['-append', '"console=ttyS0 printk.time=0"'])
+ self.vm._args.extend(['-chardev', 'socket,id=srm,server,nowait,path=' + self.console_path])
+ self.vm._args.extend(['-serial', 'chardev:srm'])
+ # This kernel crashes without VGA display
+ self.vm._args.extend(['-vga', 'std'])
+
+ def test_boot_console(self):
+ """
+ :avocado: tags=uart,printk
+ """
+ # TODO use skipUnless()
+ if self.params.get('arch') != self.ARCH:
+ return
+
+ self.vm.launch(self.console_path)
+ console = self.vm.get_console(console_address=self.console_path, login=False)
+ # no filesystem provided on purpose, wait for the Kernel panic
+ bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=30.0)[1]
+ console.close()
+ # check Super I/O
+ self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
+ self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
+ self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
+ self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
+ self.vm.shutdown()
+
+ def tearDown(self):
+ os.remove(self.console_path)
diff --git a/tests/avocado/test_linux-boot-console.py.data/parameters.yaml b/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
new file mode 100644
index 0000000000..ed8c08058c
--- /dev/null
+++ b/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
@@ -0,0 +1,3 @@
+architecture: !mux
+ alpha:
+ arch: alpha
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [Qemu-devel] [RFC PATCH 7/7] avocado: Add tests booting the Malta machine
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (5 preceding siblings ...)
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console Philippe Mathieu-Daudé
@ 2018-04-19 16:46 ` Philippe Mathieu-Daudé
2018-05-01 21:20 ` Cleber Rosa
2018-04-19 17:14 ` [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console no-reply
` (2 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-19 16:46 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Aurelien Jarno
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
tests/avocado/test_linux-boot-console.py | 92 +++++++++++++++++++
.../parameters.yaml | 2 +
2 files changed, 94 insertions(+)
diff --git a/tests/avocado/test_linux-boot-console.py b/tests/avocado/test_linux-boot-console.py
index f54d10dd79..4ae1e3bf37 100644
--- a/tests/avocado/test_linux-boot-console.py
+++ b/tests/avocado/test_linux-boot-console.py
@@ -86,3 +86,95 @@ class TestAlphaClipperBoot2_6(test.QemuTest):
def tearDown(self):
os.remove(self.console_path)
+
+
+class TestMips4kcMaltaBoot2_6(test.QemuTest):
+ """
+ :avocado: enable
+ :avocado: tags=arch_mips
+ """
+ ARCH = "mips"
+
+ def kernel_url(self):
+ return 'http://people.debian.org/~aurel32/qemu/mips/vmlinux-2.6.32-5-4kc-malta'
+
+ def setUp(self):
+ self.console_path = tempfile.mkstemp()[1]
+ kernel_path = cachedfile(self.kernel_url())
+ self.vm._args.extend(['-machine', 'malta'])
+ self.vm._args.extend(['-m', '64'])
+ self.vm._args.extend(['-kernel', kernel_path])
+ self.vm._args.extend(['-append', '"console=ttyS0 printk.time=0"'])
+ self.vm._args.extend(['-chardev', 'socket,id=uart0,server,nowait,path=' + self.console_path])
+ self.vm._args.extend(['-serial', 'chardev:uart0'])
+ self.vm._args.extend(['-nographic'])
+
+ def test_boot_console(self):
+ """
+ :avocado: tags=uart,printk
+ """
+ if self.params.get('arch') != self.ARCH:
+ return
+
+ self.vm.launch(self.console_path)
+ console = self.vm.get_console(console_address=self.console_path, login=False)
+ # no filesystem provided on purpose, wait for the Kernel panic
+ bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=6.0)[1]
+ console.close()
+ # check Super I/O
+ self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
+ self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
+ self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
+ self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
+ # check PCI (network interface)
+ self.assertIn(u'registered as PCnet/PCI II 79C970A', bootlog)
+ self.vm.shutdown()
+
+ def tearDown(self):
+ os.remove(self.console_path)
+
+# FIXME this is a copy of TestMips4kcMaltaBoot2_6 with a different the kernel url
+class TestMips4kcMaltaBoot3_2(test.QemuTest):
+ """
+ :avocado: enable
+ :avocado: tags=arch_mips
+ """
+ ARCH = "mips"
+
+ def kernel_url(self):
+ return 'http://people.debian.org/~aurel32/qemu/mips/vmlinux-3.2.0-4-4kc-malta'
+
+ def setUp(self):
+ self.console_path = tempfile.mkstemp()[1]
+ kernel_path = cachedfile(self.kernel_url())
+ self.vm._args.extend(['-machine', 'malta'])
+ self.vm._args.extend(['-m', '64'])
+ self.vm._args.extend(['-kernel', kernel_path])
+ self.vm._args.extend(['-append', '"console=ttyS0 printk.time=0"'])
+ self.vm._args.extend(['-chardev', 'socket,id=uart0,server,nowait,path=' + self.console_path])
+ self.vm._args.extend(['-serial', 'chardev:uart0'])
+ self.vm._args.extend(['-nographic'])
+
+ def test_boot_console(self):
+ """
+ :avocado: tags=uart,printk
+ """
+ if self.params.get('arch') != self.ARCH:
+ return
+
+ self.vm.launch(self.console_path)
+ console = self.vm.get_console(console_address=self.console_path, login=False)
+ # no filesystem provided on purpose, wait for the Kernel panic
+ bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=6.0)[1]
+ console.close()
+ # check Super I/O
+ self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
+ self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
+ self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
+ self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
+ # check PCI (network interface)
+ self.assertIn(u'registered as PCnet/PCI II 79C970A', bootlog)
+ self.vm.shutdown()
+
+ def tearDown(self):
+ os.remove(self.console_path)
diff --git a/tests/avocado/test_linux-boot-console.py.data/parameters.yaml b/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
index ed8c08058c..156f271a73 100644
--- a/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
+++ b/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
@@ -1,3 +1,5 @@
architecture: !mux
alpha:
arch: alpha
+ mips:
+ arch: mips
--
2.17.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (6 preceding siblings ...)
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 7/7] avocado: Add tests booting the Malta machine Philippe Mathieu-Daudé
@ 2018-04-19 17:14 ` no-reply
2018-04-20 4:17 ` Philippe Mathieu-Daudé
2018-04-30 22:49 ` Cleber Rosa
9 siblings, 0 replies; 21+ messages in thread
From: no-reply @ 2018-04-19 17:14 UTC (permalink / raw)
To: f4bug; +Cc: famz, ldoktor, crosa, apahim, stefanha, xiang.zheng
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20180419164642.9536-1-f4bug@amsat.org
Subject: [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
=== TEST SCRIPT BEGIN ===
#!/bin/bash
BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done
exit $failed
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
t [tag update] patchew/1524135103-30477-1-git-send-email-frederic.konrad@adacore.com -> patchew/1524135103-30477-1-git-send-email-frederic.konrad@adacore.com
* [new tag] patchew/20180419164642.9536-1-f4bug@amsat.org -> patchew/20180419164642.9536-1-f4bug@amsat.org
Switched to a new branch 'test'
137b1f8943 avocado: Add tests booting the Malta machine
ec4a19f56b avocado: Add a test parsing Linux kernel booting console
dc618b369b avocado: Add an optional flag 'login' to get_console()
19a3666713 qemu.py: Avoid deprecated tempfile.mktemp()
99bb4ac92b qemu.py: Check console arch is supported before calling mktemp()
163318b813 avocado: Update python scripts to upstream codebase
621512b86b AVOCADO_QEMU: Snapshot commit
=== OUTPUT BEGIN ===
Checking PATCH 1/7: AVOCADO_QEMU: Snapshot commit...
WARNING: line over 80 characters
#412: FILE: tests/avocado/avocado_qemu/test.py:121:
+ :raise QEMULoginProcessTerminatedError: If the client terminates during login
WARNING: line over 80 characters
#512: FILE: tests/avocado/avocado_qemu/test.py:221:
+ raise QEMULoginProcessTerminatedError(details.status, details.output)
WARNING: line over 80 characters
#789: FILE: tests/avocado/test_info_memdev_host_nodes.py:49:
+ cmd = 'object_add memory-backend-ram,id=mem1,host-nodes=0,size=2G,policy=bind'
WARNING: line over 80 characters
#841: FILE: tests/avocado/test_nec-usb-xhci.py:29:
+ self.vm.args.extend(['-device', 'nec-usb-xhci,id=xhci1,bus=bridge1,addr=0x3'])
ERROR: line over 90 characters
#842: FILE: tests/avocado/test_nec-usb-xhci.py:30:
+ self.vm.args.extend(['-drive', 'file=%s,format=raw,id=drive_usb,if=none' % usbdevice])
ERROR: line over 90 characters
#843: FILE: tests/avocado/test_nec-usb-xhci.py:31:
+ self.vm.args.extend(['-device', 'usb-storage,drive=drive_usb,id=device_usb,bus=xhci1.0'])
ERROR: line over 90 characters
#1039: FILE: tests/avocado/test_ovmf_with_240_vcpus.py:22:
+ default='/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd')
WARNING: line over 80 characters
#1041: FILE: tests/avocado/test_ovmf_with_240_vcpus.py:24:
+ default='/usr/share/edk2/ovmf/OVMF_VARS.fd')
total: 3 errors, 5 warnings, 1132 lines checked
Your patch has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 2/7: avocado: Update python scripts to upstream codebase...
Checking PATCH 3/7: qemu.py: Check console arch is supported before calling mktemp()...
Checking PATCH 4/7: qemu.py: Avoid deprecated tempfile.mktemp()...
Checking PATCH 5/7: avocado: Add an optional flag 'login' to get_console()...
Checking PATCH 6/7: avocado: Add a test parsing Linux kernel booting console...
ERROR: line over 90 characters
#72: FILE: tests/avocado/test_linux-boot-console.py:53:
+ return 'http://archive.debian.org/debian/dists/lenny/main/installer-alpha/current/images/cdrom/vmlinuz'
ERROR: line over 90 characters
#81: FILE: tests/avocado/test_linux-boot-console.py:62:
+ self.vm._args.extend(['-chardev', 'socket,id=srm,server,nowait,path=' + self.console_path])
WARNING: line over 80 characters
#95: FILE: tests/avocado/test_linux-boot-console.py:76:
+ console = self.vm.get_console(console_address=self.console_path, login=False)
ERROR: line over 90 characters
#97: FILE: tests/avocado/test_linux-boot-console.py:78:
+ bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=30.0)[1]
total: 3 errors, 1 warnings, 91 lines checked
Your patch has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 7/7: avocado: Add tests booting the Malta machine...
WARNING: line over 80 characters
#30: FILE: tests/avocado/test_linux-boot-console.py:99:
+ return 'http://people.debian.org/~aurel32/qemu/mips/vmlinux-2.6.32-5-4kc-malta'
ERROR: line over 90 characters
#39: FILE: tests/avocado/test_linux-boot-console.py:108:
+ self.vm._args.extend(['-chardev', 'socket,id=uart0,server,nowait,path=' + self.console_path])
WARNING: line over 80 characters
#51: FILE: tests/avocado/test_linux-boot-console.py:120:
+ console = self.vm.get_console(console_address=self.console_path, login=False)
ERROR: line over 90 characters
#53: FILE: tests/avocado/test_linux-boot-console.py:122:
+ bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=6.0)[1]
WARNING: line over 80 characters
#67: FILE: tests/avocado/test_linux-boot-console.py:136:
+# FIXME this is a copy of TestMips4kcMaltaBoot2_6 with a different the kernel url
WARNING: line over 80 characters
#76: FILE: tests/avocado/test_linux-boot-console.py:145:
+ return 'http://people.debian.org/~aurel32/qemu/mips/vmlinux-3.2.0-4-4kc-malta'
ERROR: line over 90 characters
#85: FILE: tests/avocado/test_linux-boot-console.py:154:
+ self.vm._args.extend(['-chardev', 'socket,id=uart0,server,nowait,path=' + self.console_path])
WARNING: line over 80 characters
#97: FILE: tests/avocado/test_linux-boot-console.py:166:
+ console = self.vm.get_console(console_address=self.console_path, login=False)
ERROR: line over 90 characters
#99: FILE: tests/avocado/test_linux-boot-console.py:168:
+ bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=6.0)[1]
total: 4 errors, 5 warnings, 100 lines checked
Your patch has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console Philippe Mathieu-Daudé
@ 2018-04-19 19:38 ` Richard Henderson
2018-05-01 21:17 ` Cleber Rosa
1 sibling, 0 replies; 21+ messages in thread
From: Richard Henderson @ 2018-04-19 19:38 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang
Cc: Fam Zheng, qemu-devel, Alistair Francis, Alex Bennée,
Richard Henderson
On 04/19/2018 06:46 AM, Philippe Mathieu-Daudé wrote:
> Booting an Alpha DP264 machine.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/avocado/test_linux-boot-console.py | 88 +++++++++++++++++++
> .../parameters.yaml | 3 +
> 2 files changed, 91 insertions(+)
> create mode 100644 tests/avocado/test_linux-boot-console.py
> create mode 100644 tests/avocado/test_linux-boot-console.py.data/parameters.yaml
>
> diff --git a/tests/avocado/test_linux-boot-console.py b/tests/avocado/test_linux-boot-console.py
> new file mode 100644
> index 0000000000..f54d10dd79
> --- /dev/null
> +++ b/tests/avocado/test_linux-boot-console.py
> @@ -0,0 +1,88 @@
> +# -*- coding: utf-8 -*-
> +#
> +# Boot a Linux kernel on the Malta board and check the serial console output
Not a malta. ;-)
Otherwise
Acked-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (7 preceding siblings ...)
2018-04-19 17:14 ` [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console no-reply
@ 2018-04-20 4:17 ` Philippe Mathieu-Daudé
2018-04-20 12:59 ` Philippe Mathieu-Daudé
2018-05-02 16:22 ` Cleber Rosa
2018-04-30 22:49 ` Cleber Rosa
9 siblings, 2 replies; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-20 4:17 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang, Eduardo Habkost, Laszlo Ersek
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
avocado-devel, Peter Maydell
[-- Attachment #1: Type: text/plain, Size: 4286 bytes --]
Cross-posting qemu-devel + avocado-devel.
> While previously working on a Super I/O refactor, I encountered some problems
> at runtime, after building the codebase successfully and running qtests.
> I had to manually start to boot different guests and check the bootlog.
>
> I wanted to give a try at Avocado which seems designed to simplify that kind
> of functional tests.
>
> I applied Amador Pahim work following Cleber Rosa documentation from
> http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03891.html,
> however I had to modify few things to parse the boot console.
> Since his work is not merged, I included it in this series.
>
> The tests simply expect to find a string reported by Linux printk when a
> device is detected/initialized, such "ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A"
> and "i8042 KBD port at 0x60,0x64 irq 1" for the Super I/O chip, or such
> "registered as PCnet/PCI II 79C970A" to confirms the PCI subsystem and network
> device are correctly detected:
>
>
> self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
> self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
> self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
> self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
>
> Example of the tests output:
>
> $ avocado run test_linux-boot-console.py -m test_linux-boot-console.py.data/parameters.yaml
> JOB ID : 695094c9bbe8f6011226da7c2031c2c53e949910
> JOB LOG : /home/phil/avocado/job-results/job-2018-04-19T13.36-695094c/job.log
> (1/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;alpha-2582: PASS (4.76 s)
> (2/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;mips-4a72: PASS (0.00 s)
> (3/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;alpha-2582: PASS (0.00 s)
> (4/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;mips-4a72: PASS (3.92 s)
> (5/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;alpha-2582: PASS (0.00 s)
> (6/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;mips-4a72: PASS (4.08 s)
> RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
> JOB TIME : 13.31 s
>
> Please apologize my ugly Python, this series is just a proof-of-concept :)
> I couldn't figure out how to use the @skipUnless(correct arch) decorator.
Eduardo asked me to share my first impressions after trying this
framework with QEMU.
So far it seems more designed to run tests _inside_ a qemu instance
(think user-space), eventually testing packages/scripts.
My tests are targeting the machine model itself, if the devices are
correctly instantiated and so.
The fastest approach was to check the Linux kernel bootlog, but if you
look at the TestAlphaClipperBoot console you'll see a boot "firmware" is
executed before the kernel. Casually this firmware also send information
on the console, but what if the console is not accessible?
We can use a chardev for the ioport80 POST, but some firmwares post boot
events via I2C, SPI, CAN...
To improve the testing, my idea is to use the Tracing framework.
The machine trace events would be logged in a db and avogado would
verify some of the trace events, did they occur? in the correct sequence
order? with the correct arguments? We can also check relative
timestamps, is this sequence timing fast enough?
I plan to use the gdb API to insert breakpoints and follow part
execution flow, eventually injecting (faulty) events.
The multi-arch/machines support is a bit weak yet.
I'm personally more interested in automatically testing [real world]
firmware or full machine boot process, the kind of integration testing I
can not do with qtests.
I liked the possibility to generate coredumps, or the replay function.
I also liked to be able to write a test on how a machine boots, in ~20
LOC (see TestAlphaClipperBoot2_6).
The storage API is OK to fetch a full VM image, but not a single file
like a kernel or a flash image.
Enough for my second try, good work :)
I hope this was helpful.
Regards,
Phil.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
2018-04-20 4:17 ` Philippe Mathieu-Daudé
@ 2018-04-20 12:59 ` Philippe Mathieu-Daudé
2018-05-02 16:22 ` Cleber Rosa
1 sibling, 0 replies; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2018-04-20 12:59 UTC (permalink / raw)
To: Lukáš Doktor, Cleber Rosa, Amador Pahim,
Stefan Hajnoczi, Zheng Xiang, Eduardo Habkost, Laszlo Ersek
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
avocado-devel, Peter Maydell
[-- Attachment #1: Type: text/plain, Size: 4540 bytes --]
> Cross-posting qemu-devel + avocado-devel.
>
>> While previously working on a Super I/O refactor, I encountered some problems
>> at runtime, after building the codebase successfully and running qtests.
>> I had to manually start to boot different guests and check the bootlog.
>>
>> I wanted to give a try at Avocado which seems designed to simplify that kind
>> of functional tests.
>>
>> I applied Amador Pahim work following Cleber Rosa documentation from
>> http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03891.html,
>> however I had to modify few things to parse the boot console.
>> Since his work is not merged, I included it in this series.
>>
>> The tests simply expect to find a string reported by Linux printk when a
>> device is detected/initialized, such "ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A"
>> and "i8042 KBD port at 0x60,0x64 irq 1" for the Super I/O chip, or such
>> "registered as PCnet/PCI II 79C970A" to confirms the PCI subsystem and network
>> device are correctly detected:
>>
>>
>> self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
>> self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
>> self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
>> self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
>>
>> Example of the tests output:
>>
>> $ avocado run test_linux-boot-console.py -m test_linux-boot-console.py.data/parameters.yaml
>> JOB ID : 695094c9bbe8f6011226da7c2031c2c53e949910
>> JOB LOG : /home/phil/avocado/job-results/job-2018-04-19T13.36-695094c/job.log
>> (1/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;alpha-2582: PASS (4.76 s)
>> (2/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;mips-4a72: PASS (0.00 s)
>> (3/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;alpha-2582: PASS (0.00 s)
>> (4/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;mips-4a72: PASS (3.92 s)
>> (5/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;alpha-2582: PASS (0.00 s)
>> (6/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;mips-4a72: PASS (4.08 s)
>> RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME : 13.31 s
>>
>> Please apologize my ugly Python, this series is just a proof-of-concept :)
>> I couldn't figure out how to use the @skipUnless(correct arch) decorator.
>
> Eduardo asked me to share my first impressions after trying this
> framework with QEMU.
>
> So far it seems more designed to run tests _inside_ a qemu instance
> (think user-space), eventually testing packages/scripts.
Current examples use the same pattern, which I felt a bit restricting:
[QemuTest]
.setUp: prepare VM arguments, launch the VM
.test_*: run tests in the running VM
.tearDown: shutdown the VM and cleanup dangling files
> My tests are targeting the machine model itself, if the devices are
> correctly instantiated and so.
> The fastest approach was to check the Linux kernel bootlog, but if you
> look at the TestAlphaClipperBoot console you'll see a boot "firmware" is
> executed before the kernel. Casually this firmware also send information
> on the console, but what if the console is not accessible?
> We can use a chardev for the ioport80 POST, but some firmwares post boot
> events via I2C, SPI, CAN...
>
> To improve the testing, my idea is to use the Tracing framework.
> The machine trace events would be logged in a db and avogado would
> verify some of the trace events, did they occur? in the correct sequence
> order? with the correct arguments? We can also check relative
> timestamps, is this sequence timing fast enough?
>
> I plan to use the gdb API to insert breakpoints and follow part
> execution flow, eventually injecting (faulty) events.
>
> The multi-arch/machines support is a bit weak yet.
>
> I'm personally more interested in automatically testing [real world]
> firmware or full machine boot process, the kind of integration testing I
> can not do with qtests.
>
> I liked the possibility to generate coredumps, or the replay function.
>
> I also liked to be able to write a test on how a machine boots, in ~20
> LOC (see TestAlphaClipperBoot2_6).
>
> The storage API is OK to fetch a full VM image, but not a single file
> like a kernel or a flash image
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
` (8 preceding siblings ...)
2018-04-20 4:17 ` Philippe Mathieu-Daudé
@ 2018-04-30 22:49 ` Cleber Rosa
9 siblings, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-04-30 22:49 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Stefan Hajnoczi, Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> Hi,
>
> While previously working on a Super I/O refactor, I encountered some problems
> at runtime, after building the codebase successfully and running qtests.
> I had to manually start to boot different guests and check the bootlog.
>
> I wanted to give a try at Avocado which seems designed to simplify that kind
> of functional tests.
>
> I applied Amador Pahim work following Cleber Rosa documentation from
> http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03891.html,
> however I had to modify few things to parse the boot console.
> Since his work is not merged, I included it in this series.
>
> The tests simply expect to find a string reported by Linux printk when a
> device is detected/initialized, such "ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A"
> and "i8042 KBD port at 0x60,0x64 irq 1" for the Super I/O chip, or such
> "registered as PCnet/PCI II 79C970A" to confirms the PCI subsystem and network
> device are correctly detected:
>
>
> self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
> self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
> self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
> self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
>
> Example of the tests output:
>
> $ avocado run test_linux-boot-console.py -m test_linux-boot-console.py.data/parameters.yaml
> JOB ID : 695094c9bbe8f6011226da7c2031c2c53e949910
> JOB LOG : /home/phil/avocado/job-results/job-2018-04-19T13.36-695094c/job.log
> (1/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;alpha-2582: PASS (4.76 s)
> (2/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;mips-4a72: PASS (0.00 s)
> (3/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;alpha-2582: PASS (0.00 s)
> (4/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;mips-4a72: PASS (3.92 s)
> (5/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;alpha-2582: PASS (0.00 s)
> (6/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;mips-4a72: PASS (4.08 s)
> RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
> JOB TIME : 13.31 s
>
> Please apologize my ugly Python, this series is just a proof-of-concept :)
> I couldn't figure out how to use the @skipUnless(correct arch) decorator.
>
> Regards,
>
> Phil.
>
Hi Philippe,
Thanks for this interesting RFC/PoC. As someone who has spent more time
"on the other" side of this code, this perspective is invaluable. I'll
respond to the major feature utilization points here, and then proceed
to review the individual patches.
One of the distinguishing features of Avocado is the ability to reuse
test code in different scenarios (thus reducing the amount of duplicated
code while still improving the overall coverage). I noticed you grasped
the overall usage pattern of the "YAML to Mux" plugin, by means of the
"parameter.yaml".
This is an alternative proposal for the "parameter.yaml" file:
--- "parameter.yaml" STARTS HERE
architechture: !mux
alpha:
arch: alpha
machine_type: clipper
kernel_url:
http://archive.debian.org/debian/dists/lenny/main/installer-alpha/current/images/cdrom/vmlinuz
mips:
arch: mips
machine_type: malta
check_pci: True
kernel_versions: !mux
2_6:
kernel_url:
http://people.debian.org/~aurel32/qemu/mips/vmlinux-2.6.32-5-4kc-malta
3_2:
kernel_url:
http://people.debian.org/~aurel32/qemu/mips/vmlinux-3.2.0-4-4kc-malta
--- "parameter.yaml" ENDS HERE
This file content produces the following tree (run it with "avocado -c
-t -m parameter.yaml"):
Multiplex tree representation:
┗━━ run
┗━━ architechture
╠══ alpha
║ → arch: alpha
║ → machine_type: clipper
║ → kernel_url:
http://archive.debian.org/debian/dists/lenny/main/installer-alpha/current/images/cdrom/vmlinuz
╚══ mips
┃ → check_pci: True
┃ → arch: mips
┃ → machine_type: malta
┗━━ kernel_versions
╠══ 2_6
║ → kernel_url:
http://people.debian.org/~aurel32/qemu/mips/vmlinux-2.6.32-5-4kc-malta
╚══ 3_2
→ kernel_url:
http://people.debian.org/~aurel32/qemu/mips/vmlinux-3.2.0-4-4kc-malta
And it generates the following variants (run it with "avocado -m
parameters.yaml"):
Multiplex variants (3):
Variant alpha-deaa: /run/architechture/alpha
Variant 2_6-6f08: /run/architechture/mips/kernel_versions/2_6
Variant 3_2-e556: /run/architechture/mips/kernel_versions/3_2
With these variants, it's possible to have a single code test
implementation. Here's my alternative implementation of your tests:
--- "test_linux-boot-console.py" STARTS HERE
import os
import tempfile
import gzip
from avocado_qemu import test
from avocado.utils import process
class TestBootCheckDevices(test.QemuTest):
"""
:avocado: enable
"""
def get_kernel_uncompressed(self):
kernel = self.fetch_asset(self.params.get("kernel_url"))
try:
process.run("gzip -l %s" % kernel)
except process.CmdError:
# only deal with gzip compressed kernels here
return kernel
kernel_uncompressed_path = kernel + ".uncompressed"
with gzip.open(kernel, 'rb') as kernel_gzip:
with open(kernel_uncompressed_path, 'wb') as
kernel_uncompressed:
kernel_uncompressed.write(kernel_gzip.read())
return kernel_uncompressed_path
def setUp(self):
kernel_path = self.get_kernel_uncompressed()
self.console_path = tempfile.mkstemp()[1]
self.vm._args.extend(['-m', '64'])
self.vm._args.extend(['-kernel', kernel_path])
self.vm._args.extend(['-append', '"console=ttyS0 printk.time=0"'])
self.vm._args.extend(['-chardev',
'socket,id=srm,server,nowait,path=' + self.console_path])
self.vm._args.extend(['-serial', 'chardev:srm'])
# This kernel crashes without VGA display
self.vm._args.extend(['-vga', 'std'])
self.vm.launch(self.console_path)
console = self.vm.get_console(console_address=self.console_path,
login=False)
# no filesystem provided on purpose, wait for the Kernel panic
self.bootlog = console.read_until_any_line_matches(["Kernel
panic - not syncing: VFS: Unable to mount root fs"], timeout=30.0)[1]
console.close()
def test(self):
# check Super I/O
self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A',
self.bootlog)
self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A',
self.bootlog)
self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', self.bootlog)
self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', self.bootlog)
# check PCI
if self.params.get('check_pci', default=False):
self.assertIn(u'registered as PCnet/PCI II 79C970A',
self.bootlog)
def tearDown(self):
self.vm.shutdown()
os.remove(self.console_path)
--- "test_linux-boot-console.py" ENDS HERE
A few points about this code:
1) get_kernel_uncompressed() is a prime candidate for a utility
function, even at the "avocado.utils.archive" level (a better version,
of course). It's one of those things that arise from writing more
diverse code (again, thanks for this experiment).
2) setUp() still contains a lot of boiler plate code, and can certainly
have some of that moved to "avocado_qemu". For instance, the
"avocado_qemu" code already knows how to deal with some parameters, such
as "arch" and "machine_type", so "memory", "kernel", etc could be
handled in a similar fashion.
3) It wasn't clear to me if manually creating a serial device and using
it as a console was a hard requirement of the test. The
QEMUMachine._create_console() method on the "scripts/qemu.py" file
already has some code to create many console types.
Finally, running this code + parameters (by using "avocado run -m
test_linux-boot-console.py.data/parameters.yaml --
test_linux-boot-console.py") results in:
JOB ID : bb7585de42035a062ddda539801743267aa4431a
JOB LOG :
/home/cleber/avocado/job-results/job-2018-04-30T18.46-bb7585d/job.log
(1/3) test_linux-boot-console.py:TestBootCheckDevices.test;alpha-deaa:
PASS (3.04 s)
(2/3) test_linux-boot-console.py:TestBootCheckDevices.test;2_6-6f08:
PASS (2.83 s)
(3/3) test_linux-boot-console.py:TestBootCheckDevices.test;3_2-e556:
PASS (2.86 s)
RESULTS : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
CANCEL 0
JOB TIME : 9.86 s
JOB HTML :
/home/cleber/avocado/job-results/job-2018-04-30T18.46-bb7585d/results.html
The resulting "job.log" file can be seen at:
https://paste.fedoraproject.org/paste/G4X-z5PcmMdmmIVBDKVWVg/raw
Now, unto individual commits.
- Cleber.
> Amador Pahim (1):
> AVOCADO_QEMU: Snapshot commit
>
> Philippe Mathieu-Daudé (6):
> avocado: Update python scripts to upstream codebase
> qemu.py: Check console arch is supported before calling mktemp()
> qemu.py: Avoid deprecated tempfile.mktemp()
> avocado: Add an optional flag 'login' to get_console()
> avocado: Add a test parsing Linux kernel booting console
> avocado: Add tests booting the Malta machine
>
> scripts/qemu.py | 50 ++-
> tests/avocado/README.rst | 132 ++++++
> tests/avocado/avocado_qemu/__init__.py | 0
> tests/avocado/avocado_qemu/test.py | 419 ++++++++++++++++++
> tests/avocado/parameters.yaml | 19 +
> tests/avocado/test_info_memdev_host_nodes.py | 66 +++
> tests/avocado/test_linux-boot-console.py | 180 ++++++++
> .../parameters.yaml | 5 +
> tests/avocado/test_nec-usb-xhci.py | 63 +++
> .../test_nec-usb-xhci.py.data/parameters.yaml | 4 +
> tests/avocado/test_numa_hotplug.py | 120 +++++
> tests/avocado/test_ovmf_with_240_vcpus.py | 70 +++
> .../parameters.yaml | 2 +
> tests/avocado/variants.yaml | 62 +++
> 14 files changed, 1187 insertions(+), 5 deletions(-)
> create mode 100644 tests/avocado/README.rst
> create mode 100644 tests/avocado/avocado_qemu/__init__.py
> create mode 100644 tests/avocado/avocado_qemu/test.py
> create mode 100644 tests/avocado/parameters.yaml
> create mode 100644 tests/avocado/test_info_memdev_host_nodes.py
> create mode 100644 tests/avocado/test_linux-boot-console.py
> create mode 100644 tests/avocado/test_linux-boot-console.py.data/parameters.yaml
> create mode 100644 tests/avocado/test_nec-usb-xhci.py
> create mode 100644 tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml
> create mode 100644 tests/avocado/test_numa_hotplug.py
> create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py
> create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml
> create mode 100644 tests/avocado/variants.yaml
>
--
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 2/7] avocado: Update python scripts to upstream codebase
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 2/7] avocado: Update python scripts to upstream codebase Philippe Mathieu-Daudé
@ 2018-05-01 0:56 ` Cleber Rosa
0 siblings, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 0:56 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Alex Bennée, Stefan Hajnoczi,
Zheng Xiang
Cc: Kevin Wolf, Fam Zheng, Eduardo Habkost,
open list:Block layer core, qemu-devel, Alistair Francis,
Max Reitz
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> QEMUMachine arguments member is called '_args'.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> scripts/qemu.py | 14 +++++++-------
> tests/avocado/avocado_qemu/test.py | 12 ++++++------
> tests/qemu-iotests/iotests.py | 28 ++++++++++++++--------------
> 3 files changed, 27 insertions(+), 27 deletions(-)
>
> diff --git a/scripts/qemu.py b/scripts/qemu.py
> index bd66620f45..0eecc44d09 100644
> --- a/scripts/qemu.py
> +++ b/scripts/qemu.py
> @@ -81,7 +81,7 @@ class QEMUMachine(object):
> self._qemu_log_file = None
> self._popen = None
> self._binary = binary
> - self.args = list(args) # Force copy args in case we modify them
> + self._args = list(args) # Force copy args in case we modify them
> self._wrapper = wrapper
> self._events = []
> self._iolog = None
> @@ -109,8 +109,8 @@ class QEMUMachine(object):
> # This can be used to add an unused monitor instance.
> def add_monitor_telnet(self, ip, port):
> args = 'tcp:%s:%d,server,nowait,telnet' % (ip, port)
> - self.args.append('-monitor')
> - self.args.append(args)
> + self._args.append('-monitor')
> + self._args.append(args)
>
> def add_fd(self, fd, fdset, opaque, opts=''):
> '''Pass a file descriptor to the VM'''
> @@ -120,8 +120,8 @@ class QEMUMachine(object):
> if opts:
> options.append(opts)
>
> - self.args.append('-add-fd')
> - self.args.append(','.join(options))
> + self._args.append('-add-fd')
> + self._args.append(','.join(options))
> return self
>
> def send_fd_scm(self, fd_file_path):
> @@ -184,7 +184,7 @@ class QEMUMachine(object):
> '-display', 'none', '-vga', 'none']
>
> def _create_console(self, console_address):
> - for item in self.args:
> + for item in self._args:
> for option in ['isa-serial', 'spapr-vty', 'sclpconsole']:
> if option in item:
> return []
> @@ -274,7 +274,7 @@ class QEMUMachine(object):
> bargs = self._base_args()
> bargs.extend(self._create_console(console_address))
> self._qemu_full_args = (self._wrapper + [self._binary] +
> - bargs + self.args)
> + bargs + self._args)
> self._popen = subprocess.Popen(self._qemu_full_args,
> stdin=devnull,
> stdout=self._qemu_log_file,
> diff --git a/tests/avocado/avocado_qemu/test.py b/tests/avocado/avocado_qemu/test.py
> index 5a08dace45..1ead917014 100644
> --- a/tests/avocado/avocado_qemu/test.py
> +++ b/tests/avocado/avocado_qemu/test.py
> @@ -297,8 +297,8 @@ class _VM(qemu.QEMUMachine):
> port = self.ports.find_free_port()
> newvm = _VM(self.qemu_dst_bin, self._arch, username=self.username,
> password=self.password)
> - newvm.args = self.args
> - newvm.args.extend(['-incoming', 'tcp:0:%s' % port])
> + newvm._args = self._args
> + newvm._args.extend(['-incoming', 'tcp:0:%s' % port])
> newvm.username = self.username
> newvm.password = self.password
>
> @@ -329,7 +329,7 @@ class _VM(qemu.QEMUMachine):
> :param extra: Extra parameters to the -drive option
> """
> file_option = 'file=%s' % path
> - for item in self.args:
> + for item in self._args:
> if file_option in item:
> logging.error('Image %s already present', path)
> return
> @@ -340,7 +340,7 @@ class _VM(qemu.QEMUMachine):
> if snapshot:
> file_option += ',snapshot=on'
>
> - self.args.extend(['-drive', file_option])
> + self._args.extend(['-drive', file_option])
>
> if username is not None:
> self.username = username
> @@ -387,7 +387,7 @@ class _VM(qemu.QEMUMachine):
> process.run("%s -output %s -volid cidata -joliet -rock %s %s" %
> (geniso_bin, iso_path, metadata_path, userdata_path))
>
> - self.args.extend(['-cdrom', iso_path])
> + self._args.extend(['-cdrom', iso_path])
>
> class QemuTest(Test):
>
> @@ -415,4 +415,4 @@ class QemuTest(Test):
> if machine_kvm_type is not None:
> machine += "kvm-type=%s," % machine_kvm_type
> if machine:
> - self.vm.args.extend(['-machine', machine])
> + self.vm._args.extend(['-machine', machine])
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index a2e4f03743..b25d48a91b 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -293,18 +293,18 @@ class VM(qtest.QEMUQtestMachine):
> self._num_drives = 0
>
> def add_object(self, opts):
> - self.args.append('-object')
> - self.args.append(opts)
> + self._args.append('-object')
> + self._args.append(opts)
> return self
>
> def add_device(self, opts):
> - self.args.append('-device')
> - self.args.append(opts)
> + self._args.append('-device')
> + self._args.append(opts)
> return self
>
> def add_drive_raw(self, opts):
> - self.args.append('-drive')
> - self.args.append(opts)
> + self._args.append('-drive')
> + self._args.append(opts)
> return self
>
> def add_drive(self, path, opts='', interface='virtio', format=imgfmt):
> @@ -322,27 +322,27 @@ class VM(qtest.QEMUQtestMachine):
>
> if format == 'luks' and 'key-secret' not in opts:
> # default luks support
> - if luks_default_secret_object not in self.args:
> + if luks_default_secret_object not in self._args:
> self.add_object(luks_default_secret_object)
>
> options.append(luks_default_key_secret_opt)
>
> - self.args.append('-drive')
> - self.args.append(','.join(options))
> + self._args.append('-drive')
> + self._args.append(','.join(options))
> self._num_drives += 1
> return self
>
> def add_blockdev(self, opts):
> - self.args.append('-blockdev')
> + self._args.append('-blockdev')
> if isinstance(opts, str):
> - self.args.append(opts)
> + self._args.append(opts)
> else:
> - self.args.append(','.join(opts))
> + self._args.append(','.join(opts))
> return self
>
> def add_incoming(self, addr):
> - self.args.append('-incoming')
> - self.args.append(addr)
> + self._args.append('-incoming')
> + self._args.append(addr)
> return self
>
> def pause_drive(self, drive, event=None):
>
Well, this is basically a revert of parts of this patch:
https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03446.html
There was some previous discussions on this topic, basically regarding
two conflicting points:
1) The fact that other instances of scripts.qemu.QEMUMachine (such as
the _VM instances created by avocado_qemu.tests.QemuTest) need access to
the QEMU args;
2) That attributes prefixed with (single) underscore should only be
accessed by code in the class itself.
The discussion then orbited around the usefulness (or not) of a "dumb"
wrapper for a list. For the record, I'm in favor of exposing the list
directly, until/if some extra functionality is needed.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp()
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp() Philippe Mathieu-Daudé
@ 2018-05-01 19:30 ` Cleber Rosa
2018-05-01 19:35 ` Cleber Rosa
0 siblings, 1 reply; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 19:30 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Eduardo Habkost
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> scripts/qemu.py | 20 ++++++++++----------
> 1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/scripts/qemu.py b/scripts/qemu.py
> index 0eecc44d09..379767b62f 100644
> --- a/scripts/qemu.py
> +++ b/scripts/qemu.py
> @@ -189,6 +189,16 @@ class QEMUMachine(object):
> if option in item:
> return []
>
> + device = '{dev_type},chardev=console'
> + if '86' in self._arch:
> + device = device.format(dev_type='isa-serial')
> + elif 'ppc' in self._arch:
> + device = device.format(dev_type='spapr-vty')
> + elif 's390x' in self._arch:
> + device = device.format(dev_type='sclpconsole')
> + else:
> + return []
> +
> chardev = 'socket,id=console,{address},server,nowait'
> if console_address is None:
> console_address = tempfile.mktemp()
> @@ -203,16 +213,6 @@ class QEMUMachine(object):
>
> self._console_address = console_address
>
> - device = '{dev_type},chardev=console'
> - if '86' in self._arch:
> - device = device.format(dev_type='isa-serial')
> - elif 'ppc' in self._arch:
> - device = device.format(dev_type='spapr-vty')
> - elif 's390x' in self._arch:
> - device = device.format(dev_type='sclpconsole')
> - else:
> - return []
> -
> return ['-chardev', chardev,
> '-device', device]
>
>
I understand your point here, but I found the commit message to be
misleading. You're probably referring to this snippet (from
tests/avocado/test_linux-boot-console.py):
+ def setUp(self):
+ self.console_path = tempfile.mkstemp()[1]
So I see the following points regarding this patch:
1) Function called is mkstemp(), and not mktemp(), assuming you meant
the one from the pasted snippet above.
2) The commit message should just state that it returns earlier when
architecture is not supported wrt console creation.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 4/7] qemu.py: Avoid deprecated tempfile.mktemp()
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 4/7] qemu.py: Avoid deprecated tempfile.mktemp() Philippe Mathieu-Daudé
@ 2018-05-01 19:33 ` Cleber Rosa
0 siblings, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 19:33 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Eduardo Habkost
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> We already have secure temporary directory created with tempfile.mkdtemp().
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> scripts/qemu.py | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/qemu.py b/scripts/qemu.py
> index 379767b62f..26c4b1fd3b 100644
> --- a/scripts/qemu.py
> +++ b/scripts/qemu.py
> @@ -201,7 +201,8 @@ class QEMUMachine(object):
>
> chardev = 'socket,id=console,{address},server,nowait'
> if console_address is None:
> - console_address = tempfile.mktemp()
> + console_address = os.path.join(self._temp_dir,
> + self._name + "-console.sock")
> chardev = chardev.format(address='path=%s' %
> console_address)
> elif isinstance(console_address, tuple):
>
Acked-by: Cleber Rosa <crosa@redhat.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp()
2018-05-01 19:30 ` Cleber Rosa
@ 2018-05-01 19:35 ` Cleber Rosa
0 siblings, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 19:35 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Eduardo Habkost
On 05/01/2018 03:30 PM, Cleber Rosa wrote:
>
>
> On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>> scripts/qemu.py | 20 ++++++++++----------
>> 1 file changed, 10 insertions(+), 10 deletions(-)
>>
>> diff --git a/scripts/qemu.py b/scripts/qemu.py
>> index 0eecc44d09..379767b62f 100644
>> --- a/scripts/qemu.py
>> +++ b/scripts/qemu.py
>> @@ -189,6 +189,16 @@ class QEMUMachine(object):
>> if option in item:
>> return []
>>
>> + device = '{dev_type},chardev=console'
>> + if '86' in self._arch:
>> + device = device.format(dev_type='isa-serial')
>> + elif 'ppc' in self._arch:
>> + device = device.format(dev_type='spapr-vty')
>> + elif 's390x' in self._arch:
>> + device = device.format(dev_type='sclpconsole')
>> + else:
>> + return []
>> +
>> chardev = 'socket,id=console,{address},server,nowait'
>> if console_address is None:
>> console_address = tempfile.mktemp()
>> @@ -203,16 +213,6 @@ class QEMUMachine(object):
>>
>> self._console_address = console_address
>>
>> - device = '{dev_type},chardev=console'
>> - if '86' in self._arch:
>> - device = device.format(dev_type='isa-serial')
>> - elif 'ppc' in self._arch:
>> - device = device.format(dev_type='spapr-vty')
>> - elif 's390x' in self._arch:
>> - device = device.format(dev_type='sclpconsole')
>> - else:
>> - return []
>> -
>> return ['-chardev', chardev,
>> '-device', device]
>>
>>
>
> I understand your point here, but I found the commit message to be
> misleading. You're probably referring to this snippet (from
> tests/avocado/test_linux-boot-console.py):
>
> + def setUp(self):
> + self.console_path = tempfile.mkstemp()[1]
>
> So I see the following points regarding this patch:
>
> 1) Function called is mkstemp(), and not mktemp(), assuming you meant
> the one from the pasted snippet above.
>
> 2) The commit message should just state that it returns earlier when
> architecture is not supported wrt console creation.
>
Never mind these previous comments. I was looking at a tree with the
next set of patches applied and missed the mktemp() here.
Acked-by: Cleber Rosa <crosa@redhat.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 5/7] avocado: Add an optional flag 'login' to get_console()
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 5/7] avocado: Add an optional flag 'login' to get_console() Philippe Mathieu-Daudé
@ 2018-05-01 19:46 ` Cleber Rosa
0 siblings, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 19:46 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> Not all consoles require users to log in :/
>
You're breaking a lot of assumptions here (in a good way) ;).
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/avocado/avocado_qemu/test.py | 23 ++++++++++++-----------
> 1 file changed, 12 insertions(+), 11 deletions(-)
>
> diff --git a/tests/avocado/avocado_qemu/test.py b/tests/avocado/avocado_qemu/test.py
> index 1ead917014..ac8ee6d250 100644
> --- a/tests/avocado/avocado_qemu/test.py
> +++ b/tests/avocado/avocado_qemu/test.py
> @@ -244,7 +244,7 @@ class _VM(qemu.QEMUMachine):
> super(_VM, self).__init__(qemu_bin, name=self.name, arch=arch)
> logging.getLogger('QMP').setLevel(logging.INFO)
>
> - def get_console(self, console_address=None, prompt=r"[\#\$] "):
> + def get_console(self, console_address=None, prompt=r"[\#\$] ", login=True):
> """
> :param address: Socket address, can be either a unix socket path
> (string) or a tuple in the form (address, port)
> @@ -252,9 +252,6 @@ class _VM(qemu.QEMUMachine):
> :param prompt: The regex to identify we reached the prompt.
> """
>
> - if not all((self.username, self.password)):
> - raise QEMULoginError('Username or password not set.')
> -
> if not self.is_running():
> raise QEMULoginError('VM is not running.')
>
> @@ -272,13 +269,17 @@ class _VM(qemu.QEMUMachine):
> nc_cmd += ' -U %s' % console_address
>
> console = aexpect.ShellSession(nc_cmd)
> - try:
> - logging.info('Console: Waiting login prompt...')
> - _handle_prompts(console, self.username, self.password, prompt)
> - logging.info('Console: Ready!')
> - except:
> - console.close()
> - raise
> + if login:
> + if not all((self.username, self.password)):
> + raise QEMULoginError('Username or password not set.')
> +
> + try:
> + logging.info('Console: Waiting login prompt...')
> + _handle_prompts(console, self.username, self.password, prompt)
> + except:
> + console.close()
> + raise
> + logging.info('Console: Ready!')
>
> return console
>
>
Since having docstrings for new parameters is a minor issue and not a
requirement:
Acked-by: Cleber Rosa <crosa@redhat.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console Philippe Mathieu-Daudé
2018-04-19 19:38 ` Richard Henderson
@ 2018-05-01 21:17 ` Cleber Rosa
1 sibling, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 21:17 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Richard Henderson
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> Booting an Alpha DP264 machine.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/avocado/test_linux-boot-console.py | 88 +++++++++++++++++++
> .../parameters.yaml | 3 +
> 2 files changed, 91 insertions(+)
> create mode 100644 tests/avocado/test_linux-boot-console.py
> create mode 100644 tests/avocado/test_linux-boot-console.py.data/parameters.yaml
>
> diff --git a/tests/avocado/test_linux-boot-console.py b/tests/avocado/test_linux-boot-console.py
> new file mode 100644
> index 0000000000..f54d10dd79
> --- /dev/null
> +++ b/tests/avocado/test_linux-boot-console.py
> @@ -0,0 +1,88 @@
> +# -*- coding: utf-8 -*-
> +#
> +# Boot a Linux kernel on the Malta board and check the serial console output
> +#
> +# Copyright (C) 2018 Philippe Mathieu-Daudé
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# Run with:
> +#
> +# avocado run test_linux-boot-console.py \
> +# --mux-yaml test_linux-boot-console.py.data/parameters.yaml
> +# [--filter-by-tags arch_alpha ...]
> +
> +import os
> +import tempfile
> +import hashlib
> +import urllib2
> +import gzip
> +import shutil
> +
> +from avocado import skipUnless
> +from avocado_qemu import test
> +
> +# XXX Use a cleaner CacheStorage or avocado.utils.[archive|kernel]?
> +def cachedfile(url, cachedir='/tmp/mycachedir', gzip_compressed=False):
> + if not os.path.isdir(cachedir):
> + os.mkdir(cachedir)
> + key = hashlib.md5(url).hexdigest()
> + fbin = cachedir + "/" + key + ".bin"
> + if not os.path.isfile(fbin):
> + with open(cachedir + "/" + key + ".url", "w") as f:
> + f.write(url + '\n')
> + with open(fbin, "wb") as f:
> + content = urllib2.urlopen(url).read()
> + f.write(content)
> + if gzip_compressed:
> + fgz = cachedir + "/" + key + ".gz"
> + shutil.move(fbin, fgz)
> + with gzip.open(fgz, 'rb') as f_in, open(fbin, 'wb') as f_out:
> + shutil.copyfileobj(f_in, f_out)
> + return fbin
> +
> +
As pointed on my reply to PATCH 0, Avocado has a downloading/caching
system, but the "uncompress if needed" does not exist yet. We're
tracking the implementation of such a utility function here:
https://trello.com/c/LQFOMPgV/1308-avocadoutilsarchive-add-uncompress-if-necessary
- Cleber.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 7/7] avocado: Add tests booting the Malta machine
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 7/7] avocado: Add tests booting the Malta machine Philippe Mathieu-Daudé
@ 2018-05-01 21:20 ` Cleber Rosa
0 siblings, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-01 21:20 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
Aurelien Jarno
On 04/19/2018 12:46 PM, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/avocado/test_linux-boot-console.py | 92 +++++++++++++++++++
> .../parameters.yaml | 2 +
> 2 files changed, 94 insertions(+)
>
> diff --git a/tests/avocado/test_linux-boot-console.py b/tests/avocado/test_linux-boot-console.py
> index f54d10dd79..4ae1e3bf37 100644
> --- a/tests/avocado/test_linux-boot-console.py
> +++ b/tests/avocado/test_linux-boot-console.py
> @@ -86,3 +86,95 @@ class TestAlphaClipperBoot2_6(test.QemuTest):
>
> def tearDown(self):
> os.remove(self.console_path)
> +
> +
> +class TestMips4kcMaltaBoot2_6(test.QemuTest):
> + """
> + :avocado: enable
> + :avocado: tags=arch_mips
> + """
> + ARCH = "mips"
> +
> + def kernel_url(self):
> + return 'http://people.debian.org/~aurel32/qemu/mips/vmlinux-2.6.32-5-4kc-malta'
> +
> + def setUp(self):
> + self.console_path = tempfile.mkstemp()[1]
> + kernel_path = cachedfile(self.kernel_url())
> + self.vm._args.extend(['-machine', 'malta'])
> + self.vm._args.extend(['-m', '64'])
> + self.vm._args.extend(['-kernel', kernel_path])
> + self.vm._args.extend(['-append', '"console=ttyS0 printk.time=0"'])
> + self.vm._args.extend(['-chardev', 'socket,id=uart0,server,nowait,path=' + self.console_path])
> + self.vm._args.extend(['-serial', 'chardev:uart0'])
> + self.vm._args.extend(['-nographic'])
> +
> + def test_boot_console(self):
> + """
> + :avocado: tags=uart,printk
> + """
> + if self.params.get('arch') != self.ARCH:
> + return
> +
> + self.vm.launch(self.console_path)
> + console = self.vm.get_console(console_address=self.console_path, login=False)
> + # no filesystem provided on purpose, wait for the Kernel panic
> + bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=6.0)[1]
> + console.close()
> + # check Super I/O
> + self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
> + self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
> + self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
> + self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
> + # check PCI (network interface)
> + self.assertIn(u'registered as PCnet/PCI II 79C970A', bootlog)
> + self.vm.shutdown()
> +
> + def tearDown(self):
> + os.remove(self.console_path)
> +
> +# FIXME this is a copy of TestMips4kcMaltaBoot2_6 with a different the kernel url
> +class TestMips4kcMaltaBoot3_2(test.QemuTest):
> + """
> + :avocado: enable
> + :avocado: tags=arch_mips
> + """
> + ARCH = "mips"
> +
> + def kernel_url(self):
> + return 'http://people.debian.org/~aurel32/qemu/mips/vmlinux-3.2.0-4-4kc-malta'
> +
> + def setUp(self):
> + self.console_path = tempfile.mkstemp()[1]
> + kernel_path = cachedfile(self.kernel_url())
> + self.vm._args.extend(['-machine', 'malta'])
> + self.vm._args.extend(['-m', '64'])
> + self.vm._args.extend(['-kernel', kernel_path])
> + self.vm._args.extend(['-append', '"console=ttyS0 printk.time=0"'])
> + self.vm._args.extend(['-chardev', 'socket,id=uart0,server,nowait,path=' + self.console_path])
> + self.vm._args.extend(['-serial', 'chardev:uart0'])
> + self.vm._args.extend(['-nographic'])
> +
> + def test_boot_console(self):
> + """
> + :avocado: tags=uart,printk
> + """
> + if self.params.get('arch') != self.ARCH:
> + return
> +
> + self.vm.launch(self.console_path)
> + console = self.vm.get_console(console_address=self.console_path, login=False)
> + # no filesystem provided on purpose, wait for the Kernel panic
> + bootlog = console.read_until_any_line_matches(["Kernel panic - not syncing: VFS: Unable to mount root fs"], timeout=6.0)[1]
> + console.close()
> + # check Super I/O
> + self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
> + self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
> + self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
> + self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
> + # check PCI (network interface)
> + self.assertIn(u'registered as PCnet/PCI II 79C970A', bootlog)
> + self.vm.shutdown()
> +
> + def tearDown(self):
> + os.remove(self.console_path)
> diff --git a/tests/avocado/test_linux-boot-console.py.data/parameters.yaml b/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
> index ed8c08058c..156f271a73 100644
> --- a/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
> +++ b/tests/avocado/test_linux-boot-console.py.data/parameters.yaml
> @@ -1,3 +1,5 @@
> architecture: !mux
> alpha:
> arch: alpha
> + mips:
> + arch: mips
>
The reply to PATCH 0 showed that by a bit more configuration, all of
this code could be dropped. That approach also solved the check for
architecture and canceling tests if they don't match.
I'd like to understand if this approach makes sense, or if you can see
any drawbacks.
Thanks!
- Cleber.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console
2018-04-20 4:17 ` Philippe Mathieu-Daudé
2018-04-20 12:59 ` Philippe Mathieu-Daudé
@ 2018-05-02 16:22 ` Cleber Rosa
1 sibling, 0 replies; 21+ messages in thread
From: Cleber Rosa @ 2018-05-02 16:22 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Lukáš Doktor, Amador Pahim, Stefan Hajnoczi,
Zheng Xiang, Eduardo Habkost, Laszlo Ersek
Cc: qemu-devel, Alex Bennée, Fam Zheng, Alistair Francis,
avocado-devel, Peter Maydell
On 04/20/2018 12:17 AM, Philippe Mathieu-Daudé wrote:
> Cross-posting qemu-devel + avocado-devel.
>
>> While previously working on a Super I/O refactor, I encountered some problems
>> at runtime, after building the codebase successfully and running qtests.
>> I had to manually start to boot different guests and check the bootlog.
>>
>> I wanted to give a try at Avocado which seems designed to simplify that kind
>> of functional tests.
>>
>> I applied Amador Pahim work following Cleber Rosa documentation from
>> http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03891.html,
>> however I had to modify few things to parse the boot console.
>> Since his work is not merged, I included it in this series.
>>
>> The tests simply expect to find a string reported by Linux printk when a
>> device is detected/initialized, such "ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A"
>> and "i8042 KBD port at 0x60,0x64 irq 1" for the Super I/O chip, or such
>> "registered as PCnet/PCI II 79C970A" to confirms the PCI subsystem and network
>> device are correctly detected:
>>
>>
>> self.assertIn(u'ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A', bootlog)
>> self.assertIn(u'ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A', bootlog)
>> self.assertIn(u'i8042 KBD port at 0x60,0x64 irq 1', bootlog)
>> self.assertIn(u'i8042 AUX port at 0x60,0x64 irq 12', bootlog)
>>
>> Example of the tests output:
>>
>> $ avocado run test_linux-boot-console.py -m test_linux-boot-console.py.data/parameters.yaml
>> JOB ID : 695094c9bbe8f6011226da7c2031c2c53e949910
>> JOB LOG : /home/phil/avocado/job-results/job-2018-04-19T13.36-695094c/job.log
>> (1/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;alpha-2582: PASS (4.76 s)
>> (2/6) test_linux-boot-console.py:TestAlphaClipperBoot2_6.test_boot_console;mips-4a72: PASS (0.00 s)
>> (3/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;alpha-2582: PASS (0.00 s)
>> (4/6) test_linux-boot-console.py:TestMips4kcMaltaBoot2_6.test_boot_console;mips-4a72: PASS (3.92 s)
>> (5/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;alpha-2582: PASS (0.00 s)
>> (6/6) test_linux-boot-console.py:TestMips4kcMaltaBoot3_2.test_boot_console;mips-4a72: PASS (4.08 s)
>> RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME : 13.31 s
>>
>> Please apologize my ugly Python, this series is just a proof-of-concept :)
>> I couldn't figure out how to use the @skipUnless(correct arch) decorator.
>
> Eduardo asked me to share my first impressions after trying this
> framework with QEMU.
>
> So far it seems more designed to run tests _inside_ a qemu instance
> (think user-space), eventually testing packages/scripts.
>
The first tests written indeed involved interacting with the guest OS
more often than not. I tried to show one example that did not envolve a
guest here:
https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03468.html
Just as short exercise, this is another example (sorry for the lack of
creativity):
class CpuSmp(test.QemuTest):
"""
:avocado: enable
"""
def test(self):
smp_max = int(self.params.get('smp_max', default=255))
self.vm._args.extend(['-smp', '%s' % smp_max])
self.vm.launch()
res = self.vm.qmp('query-cpus-fast')
self.assertEqual(len(res[u'return']), smp_max)
> My tests are targeting the machine model itself, if the devices are
> correctly instantiated and so.
> The fastest approach was to check the Linux kernel bootlog, but if you
> look at the TestAlphaClipperBoot console you'll see a boot "firmware" is
> executed before the kernel. Casually this firmware also send information
> on the console, but what if the console is not accessible?
> We can use a chardev for the ioport80 POST, but some firmwares post boot
> events via I2C, SPI, CAN...
>
> To improve the testing, my idea is to use the Tracing framework.
> The machine trace events would be logged in a db and avogado would
> verify some of the trace events, did they occur? in the correct sequence
> order? with the correct arguments? We can also check relative
> timestamps, is this sequence timing fast enough?
>
Right, I think I see where you're going here.
For the record, the Avocado test runner has the ability to check the
test output against a previously recorded file, and that can determine
the test status. Maybe a second implementation of the same concept can
be applied to your idea.
> I plan to use the gdb API to insert breakpoints and follow part
> execution flow, eventually injecting (faulty) events.
>
Just to be clear, do you mean the Avocado GDB API?
> The multi-arch/machines support is a bit weak yet.
>
Yes. One of the ways I believe the multi-arch/machines support can be
significantly improved is by using some level of parameter
standardization, together with tools to generate the intended test
"matrix". On a quick experiment with "pict" (an indepedent
combinatorial testing tool) on this matter, I came up with:
---
arch: x86_64, ppc64, ppcemb, ppc, aarch64, alpha, s390x
machine_type: pc, q35, pseries-2.12, virt, 40p, bamboo, g3beige, mac99,
mpc8544ds, ppce500, prep, sam460ex, clipper, s390-ccw-virtio-2.12
machine_accel: kvm, tcg
smp_max: 1, 4, 8, 15, 248, 255
IF [arch] = "x86_64" THEN
[machine_type] = "q35" OR
[machine_type] = "pc";
IF [arch] = "ppc" THEN
[machine_type] = "40p" OR
[machine_type] = "bamboo" OR
[machine_type] = "g3beige" OR
[machine_type] = "mac99" OR
[machine_type] = "mpc8544ds" OR
[machine_type] = "ppce500" OR
[machine_type] = "prep" OR
[machine_type] = "sam460ex";
IF [arch] = "alpha" THEN [machine_type] = "clipper";
IF [arch] = "ppc64" THEN [machine_type] = "pseries-2.12";
IF [arch] = "ppcemb" THEN [machine_type] = "bamboo";
IF [arch] = "aarch64" THEN [machine_type] = "virt";
IF [arch] = "s390x" THEN [machine_type] = "s390-ccw-virtio-2.12";
# On my machine, only x86_64 can do KVM
IF NOT [arch] = "x86_64" THEN [machine_accel] = "tcg";
IF [machine_type] = "q35" THEN [smp_max] = 255;
IF [machine_type] = "pc" THEN [smp_max] = 255;
IF [machine_type] = "s390-ccw-virtio-2.12" THEN [smp_max] = 248;
IF [machine_type] = "virt" THEN [smp_max] = 8;
IF [machine_type] = "mpc8544ds" THEN [smp_max] = 15;
IF [machine_type] = "clipper" THEN [smp_max] = 4;
IF [machine_type] = "40p" THEN [smp_max] = 1;
IF [machine_type] = "bamboo" THEN [smp_max] = 1;
IF [machine_type] = "g3beige" THEN [smp_max] = 1;
IF [machine_type] = "mac99" THEN [smp_max] = 1;
IF [machine_type] = "sam460ex" THEN [smp_max] = 1;
IF [machine_type] = "ppce500" THEN [smp_max] = 1;
IF [machine_type] = "prep" THEN [smp_max] = 1;
---
With this *very partial* definition of the arch/machine support in QEMU,
a single test can be run in 22 different scenarios. I've used it to run
the 4 VNC tests and the one CpuSmp test printed above, resulting in 110
different test executions:
https://cleber.fedorapeople.org/results.html
The first three parameters are already supported by avocado_qemu, and
more can follow.
And to be fully honest, "avocado_qemu" has more weaknesses than
strengthens at the moment. That's partially because it's so young (and
immature), but, can be made into our ideal grown "being".
> I'm personally more interested in automatically testing [real world]
> firmware or full machine boot process, the kind of integration testing I
> can not do with qtests.
>
> I liked the possibility to generate coredumps, or the replay function.
>
> I also liked to be able to write a test on how a machine boots, in ~20
> LOC (see TestAlphaClipperBoot2_6).
>
> The storage API is OK to fetch a full VM image, but not a single file
> like a kernel or a flash image.
>
As pointed elsewhere, fetching/caching is probably OK at this point.
But handling the (optional) gzip compression can use some TLC.
> Enough for my second try, good work :)
>
> I hope this was helpful.
>
> Regards,
>
> Phil.
>
Thanks for the review!
- Cleber.
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2018-05-02 16:22 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-19 16:46 [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 1/7] AVOCADO_QEMU: Snapshot commit Philippe Mathieu-Daudé
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 2/7] avocado: Update python scripts to upstream codebase Philippe Mathieu-Daudé
2018-05-01 0:56 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 3/7] qemu.py: Check console arch is supported before calling mktemp() Philippe Mathieu-Daudé
2018-05-01 19:30 ` Cleber Rosa
2018-05-01 19:35 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 4/7] qemu.py: Avoid deprecated tempfile.mktemp() Philippe Mathieu-Daudé
2018-05-01 19:33 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 5/7] avocado: Add an optional flag 'login' to get_console() Philippe Mathieu-Daudé
2018-05-01 19:46 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 6/7] avocado: Add a test parsing Linux kernel booting console Philippe Mathieu-Daudé
2018-04-19 19:38 ` Richard Henderson
2018-05-01 21:17 ` Cleber Rosa
2018-04-19 16:46 ` [Qemu-devel] [RFC PATCH 7/7] avocado: Add tests booting the Malta machine Philippe Mathieu-Daudé
2018-05-01 21:20 ` Cleber Rosa
2018-04-19 17:14 ` [Qemu-devel] [RFC PATCH 0/7] avocado: Add acceptance tests parsing the Linux boot console no-reply
2018-04-20 4:17 ` Philippe Mathieu-Daudé
2018-04-20 12:59 ` Philippe Mathieu-Daudé
2018-05-02 16:22 ` Cleber Rosa
2018-04-30 22:49 ` Cleber Rosa
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.