Re: [PATCH v7 8/8] Acceptance test: add "boot_linux" tests

From: Wainer dos Santos Moschetta <wainersm@redhat.com>
To: Cleber Rosa <crosa@redhat.com>, qemu-devel@nongnu.org
Cc: "Beraldo Leal" <bleal@redhat.com>,
	"Fabien Chouteau" <chouteau@adacore.com>,
	"KONRAD Frederic" <frederic.konrad@adacore.com>,
	qemu-ppc@nongnu.org, "Willian Rampazzo" <wrampazz@redhat.com>,
	"Hervé Poussineau" <hpoussin@reactos.org>,
	"Aleksandar Rikalo" <aleksandar.rikalo@rt-rk.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Aurelien Jarno" <aurelien@aurel32.net>,
	"Eduardo Habkost" <ehabkost@redhat.com>
Subject: Re: [PATCH v7 8/8] Acceptance test: add "boot_linux" tests
Date: Fri, 8 Nov 2019 17:42:14 -0200	[thread overview]
Message-ID: <3cffa345-c61f-397e-2292-be5d6ed946bf@redhat.com> (raw)
In-Reply-To: <20191104151323.9883-9-crosa@redhat.com>

On 11/4/19 1:13 PM, Cleber Rosa wrote:
> This acceptance test, validates that a full blown Linux guest can
> successfully boot in QEMU.  In this specific case, the guest chosen is
> Fedora version 31.
>
>   * x86_64, pc and q35 machine types, with and without kvm as an
>     accellerator
>
>   * aarch64 and virt machine type, with and without kvm as an
>     accellerator
>
>   * ppc64 and pseries machine type
>
>   * s390x and s390-ccw-virtio machine type
>
> The method for checking the successful boot is based on "cloudinit"
> and its "phone home" feature.  The guest is given an ISO image
> with the location of the phone home server, and the information to
> post (the instance ID).  Upon receiving the correct information,
> from the guest, the test is considered to have PASSed.
>
> This test is currently limited to user mode networking only, and
> instructs the guest to connect to the "router" address that is hard
> coded in QEMU.
>
> To create the cloudinit ISO image that will be used to configure the
> guest, the pycdlib library is also required and has been added as
> requirement to the virtual environment created by "check-venv".
>
> The console output is read by a separate thread, by means of the
> Avocado datadrainer utility module.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/acceptance/boot_linux.py | 175 +++++++++++++++++++++++++++++++++
>   tests/requirements.txt         |   1 +
>   2 files changed, 176 insertions(+)
>   create mode 100644 tests/acceptance/boot_linux.py
>
> diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
> new file mode 100644
> index 0000000000..882f7dc5df
> --- /dev/null
> +++ b/tests/acceptance/boot_linux.py
> @@ -0,0 +1,175 @@
> +# Functional test that boots a complete Linux system via a cloud image
> +#
> +# Copyright (c) 2018-2019 Red Hat, Inc.
> +#
> +# Author:
> +#  Cleber Rosa <crosa@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +
> +from avocado_qemu import Test, SRC_ROOT_DIR
> +
> +from qemu import kvm_available
> +
> +from avocado.utils import cloudinit
> +from avocado.utils import network
> +from avocado.utils import vmimage
> +from avocado.utils import datadrainer
> +
> +
> +KVM_NOT_AVAILABLE = "KVM accelerator does not seem to be available"
> +
> +
> +class BootLinux(Test):
> +    """
> +    Boots a Linux system, checking for a successful initialization
> +    """
> +
> +    timeout = 600
> +    chksum = None
> +
> +    def setUp(self):
> +        super(BootLinux, self).setUp()
> +        self.prepare_boot()
> +        self.vm.add_args('-smp', '2')
> +        self.vm.add_args('-m', '2048')
> +        self.vm.add_args('-drive', 'file=%s' % self.boot.path)
> +        self.prepare_cloudinit()
> +
> +    def prepare_boot(self):
> +        self.log.info('Downloading/preparing boot image')
Cosmetic: replace 'Downloading/preparing' with 'Downloading' since this 
function does only download the boot image.
> +        # Fedora 31 only provides ppc64le images
> +        image_arch = self.arch
> +        if image_arch == 'ppc64':
> +            image_arch = 'ppc64le'
> +        try:
> +            self.boot = vmimage.get(
> +                'fedora', arch=image_arch, version='31',
> +                checksum=self.chksum,
> +                algorithm='sha256',
> +                cache_dir=self.cache_dirs[0],
> +                snapshot_dir=self.workdir)
> +        except:
> +            self.cancel('Failed to download/prepare boot image')

Likewise.

> +
> +    def prepare_cloudinit(self):
> +        self.log.info('Preparing cloudinit image')
> +        try:
> +            cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
> +            self.phone_home_port = network.find_free_port()
> +            cloudinit.iso(cloudinit_iso, self.name,
> +                          username='root',
> +                          password='password',
> +                          # QEMU's hard coded usermode router address
> +                          phone_home_host='10.0.2.2',
> +                          phone_home_port=self.phone_home_port)
> +            self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
> +        except Exception:
> +            self.cancel('Failed to prepared cloudinit image')
> +
> +    def launch_and_wait(self):
> +        self.vm.set_console()
> +        self.vm.launch()
> +        console_drainer = datadrainer.LineLogger(self.vm.console_socket.fileno(),
> +                                                 logger=self.log.getChild('console'))
> +        console_drainer.start()
> +        self.log.info('VM launched, waiting for boot confirmation from guest')
> +        cloudinit.wait_for_phone_home(('0.0.0.0', self.phone_home_port), self.name)
> +
> +
> +class BootLinuxX8664(BootLinux):
> +    """
> +    :avocado: tags=arch:x86_64
> +    """
> +
> +    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
> +
> +    def test_pc(self):
> +        """
> +        :avocado: tags=machine:pc
> +        """
> +        self.launch_and_wait()
> +
> +    def test_kvm_pc(self):
> +        """
> +        :avocado: tags=machine:pc
> +        :avocado: tags=accel:kvm
> +        """
> +        if not kvm_available(self.arch):
> +            self.cancel(KVM_NOT_AVAILABLE)

kvm_available() solely checks for kvm presence on host, not if enabled 
in the target binary. That said, as a follow up we could adapt this test 
to use a strengthen approach as I proposed in patch 02 of '[PATCH 0/3] 
Acceptance tests: boot Linux with KVM test' [1] series.

[1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg627498.html

> +        self.vm.add_args("-accel", "kvm")
> +        self.launch_and_wait()
> +
> +    def test_q35(self):
> +        """
> +        :avocado: tags=machine:q35
> +        """
> +        self.launch_and_wait()
> +
> +    def test_kvm_q35(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=accel:kvm
> +        """
> +        if not kvm_available(self.arch):
> +            self.cancel(KVM_NOT_AVAILABLE)
> +        self.vm.add_args("-accel", "kvm")
> +        self.launch_and_wait()
> +
> +
> +class BootLinuxAarch64(BootLinux):
> +    """
> +    :avocado: tags=arch:aarch64
> +    :avocado: tags=machine:virt
> +    """
> +
> +    chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
> +
> +    def test_virt(self):
> +        self.vm.add_args('-cpu', 'cortex-a53')
> +        self.vm.add_args('-bios',
> +                         os.path.join(SRC_ROOT_DIR, 'pc-bios',
> +                                      'edk2-aarch64-code.fd'))
> +        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
> +        self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
> +        self.launch_and_wait()
> +
> +    def test_kvm_virt(self):
> +        """
> +        :avocado: tags=accel:kvm
> +        """
> +        if not kvm_available(self.arch):
> +            self.cancel(KVM_NOT_AVAILABLE)
> +        self.vm.add_args("-accel", "kvm")
> +        self.test_virt()
> +
> +
> +class BootLinuxPPC64(BootLinux):
> +    """
> +    :avocado: tags=arch:ppc64
> +    """
> +
> +    chksum = '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'
> +
> +    def test_pseries(self):
> +        """
> +        :avocado: tags=machine:pseries
> +        """
> +        self.launch_and_wait()
> +
> +
> +class BootLinuxS390X(BootLinux):
> +    """
> +    :avocado: tags=arch:s390x
> +    """
> +
> +    chksum = '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'
> +
> +    def test_s390_ccw_virtio(self):
> +        """
> +        :avocado: tags=machine:s390-ccw-virtio
> +        """
> +        self.launch_and_wait()
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index a2a587223a..3893361e0c 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -2,3 +2,4 @@
>   # in the tests/venv Python virtual environment. For more info,
>   # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
>   avocado-framework==72.0
> +pycdlib==1.8.0

Besides the cosmetics suggestions, this test is ready to go.

In other to write powerful tests there will be needed a mechanism to 
boot a full blown guest which can be interacted with. I see the proposed 
BootLinux class here as the first milestone towards that goal. Another 
reason to merge this as soon as possible.

Ran the tests on my x86_68 machine with RHEL 8.1, and all pass 
(BootLinuxAarch64.test_kvm_virt correctly skipped).

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>

Thanks,

Wainer