* [Qemu-devel] [PATCH v3 0/2] Add "boot_linux" acceptance test
@ 2019-02-06 19:53 Cleber Rosa
2019-02-06 19:53 ` [Qemu-devel] [PATCH v3 1/2] Acceptance tests: add the build directory to the system PATH Cleber Rosa
2019-02-06 19:53 ` [Qemu-devel] [PATCH v3 2/2] Add "boot_linux" acceptance test for x86_64 and pc machine type Cleber Rosa
0 siblings, 2 replies; 3+ messages in thread
From: Cleber Rosa @ 2019-02-06 19:53 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Caio Carrara, Wainer dos Santos Moschetta,
Samuel Ortiz, Philippe Mathieu-Daudé,
Cleber Rosa
Some hopefully useful pointers for reviewers:
=============================================
Git Info:
- URI: https://github.com/clebergnu/qemu/tree/sent/test_boot_linux_v3
- Remote: https://github.com/clebergnu/qemu
- Branch: sent/test_boot_linux_v3
Travis CI Info:
- Build: https://travis-ci.org/clebergnu/qemu/builds/489711644
Previous version:
- v2: https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg04318.html
- v1: http://lists.nongnu.org/archive/html/qemu-devel/2018-09/msg02530.html
Known Issues on this version:
=============================
* A recent TCG performance regression[1] affects this test in a
number of ways:
- The test execution may timeout by itself
- The generation of SSH host keys in the guest's first boot is also
affected (possibly also a timeout)
- The cloud-init "phone home" feature attempts to read the host keys
and fails, causing the test to timeout and fail
These are not observed anymore once the fix[2] is applied.
[1] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00338.html
[2] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01129.html
Changes from v2:
================
* Updated the tag to include the "arch:" key, in a similar fashion as to
the tests in the "Acceptance Tests: target architecture support":
- https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00369.html
* Renamed the test method name to test_x86_64_pc, again, similarly to the
boot_linux_console.py tests in the series mentioned before.
* Set the machine type explicitly, again similarly to the
boot_linux_console.py tests in the series mentioned before.
* Added messages after the launch of the VM, to let test runners know
the test know waits for a boot confirmation from the the guest (Eduardo).
* Updated commit message to reflect the fact that this version does
not allow for parameterization of the guest OS, version, etc.
* Dropped the RFC prefix on patch "RFC: Acceptance tests: add the
build directory to the system PATH"
* Changed the comments on "RFC: Acceptance tests: add the build
directory to the system PATH" to make it clear the addition of a
the build directory to the PATH may influence other utility code.
Changes from v1:
================
* The commit message was adjusted, removing the reference to the
avocado.utils.vmimage encoding issue on previous Avocado versions
(<= 64.0) and the fix that would (and was) included in Avocado
version 65.0.
* Effectively added pycdlib==1.6.0 to the requirements.txt file,
added on a56931eef3, and adjusted the commit message was also
to reflect that.
* Updated the default version of the guest OS, from Fedora 28 to 29.
Besides possible improvements in the (virtual) hardware coverage,
it brings a performance improvement in the order of 20% to the
test.
* Removed all direct parameters usage. Because some parameters and
its default values implemented in the test would prevent it from
running on some environments. Example: the "accel" parameter had a
default value of "kvm", which would prevent this test, that boots a
x86_64 OS, from running on a host arch different than x86_64. I
recognize that it's desirable to make tests reusable and
parameterized (that was the reason for the first version doing so),
but the mechanism to be used to define the architectures that a
given test should support is still an open issue, and has been
discussed in other threads. I'll follow up those discussions with
a proposal, and until then, removing those aspects from this test
implementation seemed to be the best option. A caveat: this test
currently adds the same tag (x86_64) and follows other assumptions
made on "boot_linux_console.py", that is, that a x86_64 target
binary will be used to run it. If a user is in an environment that
does not have a x86_64 target binary, it could filter those tests
out with: "avocado run --filter-by-tags='-x86_64' tests/acceptance".
* Removed most arguments to the QEMU command line for pretty much the
same reasons described above, and by following the general
perception that I could grasp from other discussions that QEMU
defaults should preferrably be used. This test, as well as others,
can and should be extended later to allow for different test
scenarios by passing well documented parameter values. That is,
they should respect well-known parameters such as "accel" mentioned
above, so that the same test can run with KVM or TCG.
* Changed the value of the memory argument to 1024, which based on
my experimentations and observations is the minimum amount of RAM
for the Fedora 29 cloud image to sucessfully boot on QEMU. I know
there's no such thing as a "one size fits all", specially for QEMU,
but this makes me wonder wether a x86_64 machine type shouldn't
have its default_ram_size bumped to a number practical enough to
run modern operating systems.
* Added a new patch "RFC: Acceptance tests: add the build directory
to the system PATH", which is supposed to gather feedback on how to
enable the use of built binaries, such as qemu-img, to code used by
the test code. The specific situation here is that the vmimage,
part of the avocado.utils libraries, makes use of qemu-img to create
snapshot files. Even though we could require qemu-img to be installed
as a dependency of tests, system wide, it actually goes against the
goal of testing all QEMU things from the source/build tree. This
became aparent with tests running on environments such as Travis CI,
which don't necessarily have qemu-img available elsewhere.
Cleber Rosa (2):
Acceptance tests: add the build directory to the system PATH
Add "boot_linux" acceptance test for x86_64 and pc machine type
tests/acceptance/avocado_qemu/__init__.py | 7 ++++
tests/acceptance/boot_linux.py | 51 +++++++++++++++++++++++
tests/requirements.txt | 1 +
3 files changed, 59 insertions(+)
create mode 100644 tests/acceptance/boot_linux.py
--
2.20.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [Qemu-devel] [PATCH v3 1/2] Acceptance tests: add the build directory to the system PATH
2019-02-06 19:53 [Qemu-devel] [PATCH v3 0/2] Add "boot_linux" acceptance test Cleber Rosa
@ 2019-02-06 19:53 ` Cleber Rosa
2019-02-06 19:53 ` [Qemu-devel] [PATCH v3 2/2] Add "boot_linux" acceptance test for x86_64 and pc machine type Cleber Rosa
1 sibling, 0 replies; 3+ messages in thread
From: Cleber Rosa @ 2019-02-06 19:53 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Caio Carrara, Wainer dos Santos Moschetta,
Samuel Ortiz, Philippe Mathieu-Daudé,
Cleber Rosa
So that when binaries such as qemu-img are searched for, those in the
build tree will be favored. As a clarification, SRC_ROOT_DIR is
dependent on the location from where tests are executed, so they are
equal to the build directory if one is being used.
The original motivation is that Avocado libraries such as
avocado.utils.vmimage.get() may use the matching binaries, but it may
also apply to any other binary that test code may eventually attempt
to execute.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
tests/acceptance/avocado_qemu/__init__.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 1e54fd5932..6f0075142f 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -42,6 +42,13 @@ def pick_default_qemu_bin():
class Test(avocado.Test):
def setUp(self):
+ # Some utility code uses binaries from the system's PATH. For
+ # instance, avocado.utils.vmimage.get() uses qemu-img, to
+ # create a snapshot image. This is a transparent way of
+ # making sure those utilities find and use binaries on the
+ # build tree by default.
+ os.environ['PATH'] = '%s:%s' % (SRC_ROOT_DIR, os.environ['PATH'])
+
self.vm = None
self.qemu_bin = self.params.get('qemu_bin',
default=pick_default_qemu_bin())
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [Qemu-devel] [PATCH v3 2/2] Add "boot_linux" acceptance test for x86_64 and pc machine type
2019-02-06 19:53 [Qemu-devel] [PATCH v3 0/2] Add "boot_linux" acceptance test Cleber Rosa
2019-02-06 19:53 ` [Qemu-devel] [PATCH v3 1/2] Acceptance tests: add the build directory to the system PATH Cleber Rosa
@ 2019-02-06 19:53 ` Cleber Rosa
1 sibling, 0 replies; 3+ messages in thread
From: Cleber Rosa @ 2019-02-06 19:53 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Caio Carrara, Wainer dos Santos Moschetta,
Samuel Ortiz, Philippe Mathieu-Daudé,
Cleber Rosa
This acceptance test, validates that a full blown Linux guest can
successfully boot in QEMU. In this specific case, the guest chosen is
Fedora version 29.
The method for checking the successful boot is based on "cloudinit"
and its "phone home" feature. The guest is given an ISO image
with the location of the phone home server, and the information to
post (the instance ID). Upon receiving the correct information,
from the guest, the test is considered to have PASSed.
This test is currently limited to user mode networking only, and
instructs the guest to connect to the "router" address that is hard
coded in QEMU.
To create the cloudinit ISO image that will be used to configure the
guest, the pycdlib library is also required and has been added as
requirement to the virtual environment created by "check-venv".
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
tests/acceptance/boot_linux.py | 51 ++++++++++++++++++++++++++++++++++
tests/requirements.txt | 1 +
2 files changed, 52 insertions(+)
create mode 100644 tests/acceptance/boot_linux.py
diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
new file mode 100644
index 0000000000..927fac2959
--- /dev/null
+++ b/tests/acceptance/boot_linux.py
@@ -0,0 +1,51 @@
+# Functional test that boots a complete Linux system via a cloud image
+#
+# Copyright (c) 2018-2019 Red Hat, Inc.
+#
+# Author:
+# Cleber Rosa <crosa@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+
+from avocado_qemu import Test
+
+from avocado.utils import cloudinit
+from avocado.utils import network
+from avocado.utils import vmimage
+
+
+class BootLinux(Test):
+ """
+ Boots a Linux system, checking for a successful initialization
+
+ :avocado: enable
+ """
+
+ timeout = 600
+
+ def test_x86_64_pc(self):
+ """
+ :avocado: tags=arch:x86_64
+ :avocado: tags=machine:pc
+ """
+ self.vm.set_machine('pc')
+ self.vm.add_args('-m', '1024')
+ boot = vmimage.get('fedora', arch='x86_64', version='29',
+ cache_dir=self.cache_dirs[0],
+ snapshot_dir=self.workdir)
+ self.vm.add_args('-drive', 'file=%s' % boot.path)
+
+ cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
+ phone_home_port = network.find_free_port()
+ cloudinit.iso(cloudinit_iso, self.name,
+ # QEMU's hard coded usermode router address
+ phone_home_host='10.0.2.2',
+ phone_home_port=phone_home_port)
+ self.vm.add_args('-drive', 'file=%s' % cloudinit_iso)
+
+ self.vm.launch()
+ self.log.info('VM launched, waiting for boot confirmation from guest')
+ cloudinit.wait_for_phone_home(('0.0.0.0', phone_home_port), self.name)
diff --git a/tests/requirements.txt b/tests/requirements.txt
index 64c6e27a94..fdc2d7d325 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -2,3 +2,4 @@
# in the tests/venv Python virtual environment. For more info,
# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
avocado-framework==65.0
+pycdlib==1.6.0
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-02-06 19:53 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-06 19:53 [Qemu-devel] [PATCH v3 0/2] Add "boot_linux" acceptance test Cleber Rosa
2019-02-06 19:53 ` [Qemu-devel] [PATCH v3 1/2] Acceptance tests: add the build directory to the system PATH Cleber Rosa
2019-02-06 19:53 ` [Qemu-devel] [PATCH v3 2/2] Add "boot_linux" acceptance test for x86_64 and pc machine type Cleber Rosa
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.