[Qemu-devel] [PATCH 0/5] Acceptance/functional tests

[Cleber Rosa <crosa@redhat.com>]

TL;DR
=====

Another version, with a minimalist approach, to the acceptance tests
infrastructure for QEMU, based on the Avocado Testing Framework.

Background
==========

The previous version, still considered an RFC, was sent to the list by
Eduardo[1] was based on the work held in Amador's branch[2].  After
reviewing in under a different light, including the experiences
done and reported by Philippe[3].

Differences from previous versions
==================================

The main difference is that this series include only the minimal
changes deemed necessary to have a starting point.  I like to think
that it's better connected to the QEMU community and project needs,
and will hopefully allow for a more organic growth.

Since this version has less features than the previous versions,
provided it's accepted, these are the next probable development tasks:

 * Provide a simple variants mechanism to allow the same tests to be
   run under different targets, machine models and devices (present on
   the previous versions as a "YAML to Mux" file with architecture
   definitions)
 * Implement QEMUMachine migration support (present on the previous
   version in the "avocado_qemu.test._VM" class)
 * Implement Guest OS image selection and download (mostly an Avocado
   feature, paired with a parameter convention and cloud-init support
   code)
 * Implement interactive support for Guest OS sessions (present on
   the previous versions, supported by the aexpect Python module)

Even though this version shares very little (if any) code with the
previous versions, the following is a list of noteworthy changes:

 * Tests directory is now "tests/acceptance" (was "tests/avocado")
 * Base test class is now "avocado_qemu.Test" (was
   "avocado_qemu.test.QemuTest")
 * Base test class is now hosted in "avocado_qemu/__init__.py" (was
   "avocado_qemu/test.py")
 * Direct use of "qemu.QEMUMachine", that is, the
   avocado_qemu.test._VM class is gone
 * avocado_qemu.Test won't search for QEMU binaries on $PATH.  To use
   QEMU binary on a custom system location it's necessary to use the
   "qemu_bin" parameter
 * Example test in README.rst is distributed as a real test
   ("test_version.py")
 * A new "Linux boot console" test, loosely modeled after Phillipe's
   use case

Commit summary
==============

Cleber Rosa (5):
  Add functional/acceptance tests infrastructure
  scripts/qemu.py: allow adding to the list of extra arguments
  Acceptance tests: add quick VNC tests
  scripts/qemu.py: introduce set_console() method
  Acceptance tests: add Linux kernel boot and console checking test

 scripts/qemu.py                             | 103 +++++++++++++++-
 scripts/test_qemu.py                        | 176 +++++++++++++++++++++++++++
 tests/acceptance/README.rst                 | 141 +++++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py   |  45 +++++++
 tests/acceptance/test_boot_linux_console.py |  37 ++++++
 tests/acceptance/test_version.py            |  13 ++
 tests/acceptance/test_vnc.py                |  50 ++++++++
 7 files changed, 564 insertions(+), 1 deletion(-)
 create mode 100644 scripts/test_qemu.py
 create mode 100644 tests/acceptance/README.rst
 create mode 100644 tests/acceptance/avocado_qemu/__init__.py
 create mode 100644 tests/acceptance/test_boot_linux_console.py
 create mode 100644 tests/acceptance/test_version.py
 create mode 100644 tests/acceptance/test_vnc.py

---

[1] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03443.html
[2] https://github.com/apahim/qemu/commits/avocado_qemu
[3] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03076.html