[PATCH 0/9] Avocado tests: filter out tests using TCG booting full blown distros

[PATCH 0/9] Avocado tests: filter out tests using TCG booting full blown distros

[Cleber Rosa]

It was previously reported[1] and discussed that tests booting full
blown distros and relying on TCG would take too much time to run,
especially in the environments given by GitLab CI's shared runners.

This is an implementation of a proposal to exclude those tests from
being run by default on `make check-avocado` invocations.  To make it
extra clear, all tests are still available, but those that are tagged
with "accel:tcg" and "boots:distro", are filtered out by default on
`make check-avocado`.

This is the situation of the Avocado GitLab CI jobs with and without
the changes in this PS:

                         +------------------+--- ----------------+
                         |        Now[2]    |       Before[3]    |
+------------------------+------------------+--------------------+
|          Job           |  Length |  Tests |  Length  |  Tests  |
|         Name           | (mm:ss) |   Run  |  (mm:ss) |   Run   |
+------------------------+------------------+-------------------+
|avocado-system-alpine   |  06:33      16   |   20:30       18   |
|avocado-system-debian   |  12:06      24   |   13:05       24   |
|avocado-system-centos   |  09:58      41   |   24:15       44   |
|avocado-system-fedora   |  08:50      35   |   08:59       35   |
|avocado-system-opensuse |  08:09      38   |   27:21       42   |
|avocado-system-ubuntu   |  06:52      16   |   18:52       18   |
|avocado-cfi-x86_64      |  05:43      27   |   15:07       29   |
+------------------------+------------------+--------------------+
|TOTALS                  |  58:11     197   | 2:08:09      210   |
+------------------------+------------------+--------------------+

Assuming the jobs run in parallel, the overall wait time for all the
Avocado jobs to complete is now ~12 minutes.

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg07271.html
[2] https://gitlab.com/cleber.gnu/qemu/-/pipelines/479720240
[3] https://gitlab.com/qemu-project/qemu/-/pipelines/478580581

Cleber Rosa (9):
  Avocado GitLab CI jobs: don't reset TARGETS and simplify commands
  Avocado tests: use logging namespace that is preserved in test logs
  Avocado migration test: adapt to "utils.network" API namespace change
  Avocado: bump to version 95.0
  tests/avocado/linux_ssh_mips_malta.py: add missing accel (tcg) tag
  tests/avocado/virtiofs_submounts.py: shared_dir may not exist
  Avocado tests: improve documentation on tag filtering
  Avocado tests: classify tests based on what it's booted
  Avocado tests: don't run tests with TCG that boot full blown distros

 .gitlab-ci.d/buildtest-template.yml       |  3 ++
 .gitlab-ci.d/buildtest.yml                |  9 ----
 docs/devel/testing.rst                    | 22 +++++++++
 tests/Makefile.include                    |  6 ++-
 tests/avocado/avocado_qemu/__init__.py    | 10 ++---
 tests/avocado/boot_linux.py               |  4 ++
 tests/avocado/boot_linux_console.py       | 54 +++++++++++++++++++++++
 tests/avocado/boot_xen.py                 |  3 ++
 tests/avocado/hotplug_cpu.py              |  1 +
 tests/avocado/intel_iommu.py              |  1 +
 tests/avocado/linux_initrd.py             |  5 ++-
 tests/avocado/linux_ssh_mips_malta.py     |  5 +++
 tests/avocado/machine_arm_canona1100.py   |  1 +
 tests/avocado/machine_arm_integratorcp.py |  7 ++-
 tests/avocado/machine_arm_n8x0.py         |  2 +
 tests/avocado/machine_avr6.py             |  1 +
 tests/avocado/machine_m68k_nextcube.py    |  1 +
 tests/avocado/machine_microblaze.py       |  1 +
 tests/avocado/machine_mips_fuloong2e.py   |  1 +
 tests/avocado/machine_mips_loongson3v.py  |  1 +
 tests/avocado/machine_mips_malta.py       |  6 ++-
 tests/avocado/machine_rx_gdbsim.py        |  2 +
 tests/avocado/machine_s390_ccw_virtio.py  |  4 ++
 tests/avocado/machine_sparc64_sun4u.py    |  1 +
 tests/avocado/machine_sparc_leon3.py      |  1 +
 tests/avocado/migration.py                |  4 +-
 tests/avocado/multiprocess.py             |  4 ++
 tests/avocado/ppc_405.py                  |  2 +
 tests/avocado/ppc_bamboo.py               |  2 +
 tests/avocado/ppc_mpc8544ds.py            |  1 +
 tests/avocado/ppc_prep_40p.py             |  1 +
 tests/avocado/ppc_pseries.py              |  1 +
 tests/avocado/ppc_virtex_ml507.py         |  1 +
 tests/avocado/replay_kernel.py            | 33 ++++++++++++--
 tests/avocado/replay_linux.py             |  6 +--
 tests/avocado/reverse_debugging.py        |  6 +--
 tests/avocado/smmu.py                     |  1 +
 tests/avocado/tcg_plugins.py              |  3 ++
 tests/avocado/tesseract_utils.py          |  6 +--
 tests/avocado/virtio-gpu.py               |  2 +
 tests/avocado/virtio_check_params.py      |  3 +-
 tests/avocado/virtiofs_submounts.py       |  8 ++--
 tests/requirements.txt                    |  2 +-
 43 files changed, 197 insertions(+), 41 deletions(-)

-- 
2.35.1

[PATCH 1/9] Avocado GitLab CI jobs: don't reset TARGETS and simplify commands

[Cleber Rosa]

The Avocado tests rely on the TARGETS variable, which is computed
based on the built targets.  The current set of commands on the
inherited scripts section will reset those, leaving TARGETS empty and
consequently the AVOCADO_CMDLINE_TAGS empty too.

This is causing the list of tests to have no filtering by tags, which
can be seen by the large number of CANCEL/SKIP statuses (because of
the lack of a matching qemu-system-$(ARCH) binary).

With this change, the TARGETS variable is properly computed, and so is
the AVOCADO_CMDLINE_TAGS.  This causes a reduction in the number of
tests attempted to be run on each job, and less noise on the test
results.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 3 +++
 .gitlab-ci.d/buildtest.yml          | 9 ---------
 2 files changed, 3 insertions(+), 9 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 2c7980a4f6..c038a0910f 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -64,6 +64,9 @@
         du -chs ${CI_PROJECT_DIR}/avocado-cache ;
       fi
     - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
+  script:
+    - cd build
+    - make check-avocado
   after_script:
     - cd build
     - du -chs ${CI_PROJECT_DIR}/avocado-cache
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 0aa70213fb..d0bed9c382 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -33,7 +33,6 @@ avocado-system-alpine:
       artifacts: true
   variables:
     IMAGE: alpine
-    MAKE_CHECK_ARGS: check-avocado
 
 build-system-ubuntu:
   extends: .native_build_job_template
@@ -66,7 +65,6 @@ avocado-system-ubuntu:
       artifacts: true
   variables:
     IMAGE: ubuntu2004
-    MAKE_CHECK_ARGS: check-avocado
 
 build-system-debian:
   extends: .native_build_job_template
@@ -98,7 +96,6 @@ avocado-system-debian:
       artifacts: true
   variables:
     IMAGE: debian-amd64
-    MAKE_CHECK_ARGS: check-avocado
 
 crash-test-debian:
   extends: .native_test_job_template
@@ -143,7 +140,6 @@ avocado-system-fedora:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado
 
 crash-test-fedora:
   extends: .native_test_job_template
@@ -189,7 +185,6 @@ avocado-system-centos:
       artifacts: true
   variables:
     IMAGE: centos8
-    MAKE_CHECK_ARGS: check-avocado
 
 build-system-opensuse:
   extends: .native_build_job_template
@@ -221,7 +216,6 @@ avocado-system-opensuse:
       artifacts: true
   variables:
     IMAGE: opensuse-leap
-    MAKE_CHECK_ARGS: check-avocado
 
 
 # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by
@@ -382,7 +376,6 @@ avocado-cfi-aarch64:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado
 
 build-cfi-ppc64-s390x:
   extends: .native_build_job_template
@@ -424,7 +417,6 @@ avocado-cfi-ppc64-s390x:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado
 
 build-cfi-x86_64:
   extends: .native_build_job_template
@@ -460,7 +452,6 @@ avocado-cfi-x86_64:
       artifacts: true
   variables:
     IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado
 
 tsan-build:
   extends: .native_build_job_template
-- 
2.35.1

[PATCH 2/9] Avocado tests: use logging namespace that is preserved in test logs

[Cleber Rosa]

Since Avocado 92.0[1], there's no universal preservation of logged
content via Python's "logging" APIs into the test log files.  This
changes were motivated by the fact that doing so is intrusive as it
touches on Python's root logger.

Test writers are now expected to use "avocado." as a namespace prefix
for everything that Avocado should collect/preserve, and other
prefixes for logged content that should be handled differently.

[1] - https://avocado-framework.readthedocs.io/en/94.0/releases/92_0.html#users-test-writers

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/avocado/avocado_qemu/__init__.py    | 5 ++---
 tests/avocado/linux_initrd.py             | 3 +--
 tests/avocado/machine_arm_integratorcp.py | 3 +--
 tests/avocado/machine_mips_malta.py       | 3 +--
 tests/avocado/replay_kernel.py            | 5 ++---
 tests/avocado/replay_linux.py             | 5 ++---
 tests/avocado/reverse_debugging.py        | 5 ++---
 tests/avocado/tesseract_utils.py          | 6 +++---
 tests/avocado/virtio_check_params.py      | 3 +--
 9 files changed, 15 insertions(+), 23 deletions(-)

diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
index 75063c0c30..88cec83e5c 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -8,7 +8,6 @@
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
 
-import logging
 import os
 import shutil
 import subprocess
@@ -138,7 +137,7 @@ def _console_interaction(test, success_message, failure_message,
     if vm is None:
         vm = test.vm
     console = vm.console_socket.makefile(mode='rb', encoding='utf-8')
-    console_logger = logging.getLogger('console')
+    console_logger = test.log.getChild('console')
     while True:
         if send_string:
             vm.console_socket.sendall(send_string.encode())
@@ -370,7 +369,7 @@ class LinuxSSHMixIn:
     """Contains utility methods for interacting with a guest via SSH."""
 
     def ssh_connect(self, username, credential, credential_is_key=True):
-        self.ssh_logger = logging.getLogger('ssh')
+        self.ssh_logger = self.log.getChild('ssh')
         res = self.vm.command('human-monitor-command',
                               command_line='info usernet')
         port = get_info_usernet_hostfwd_port(res)
diff --git a/tests/avocado/linux_initrd.py b/tests/avocado/linux_initrd.py
index ba02e5a563..6ebf299cd4 100644
--- a/tests/avocado/linux_initrd.py
+++ b/tests/avocado/linux_initrd.py
@@ -9,7 +9,6 @@
 # later.  See the COPYING file in the top-level directory.
 
 import os
-import logging
 import tempfile
 
 from avocado_qemu import QemuSystemTest
@@ -79,7 +78,7 @@ def test_with_2gib_file_should_work_with_linux_v4_16(self):
                              '-m', '5120')
             self.vm.launch()
             console = self.vm.console_socket.makefile()
-            console_logger = logging.getLogger('console')
+            console_logger = self.log.getChild('console')
             while True:
                 msg = console.readline()
                 console_logger.debug(msg.strip())
diff --git a/tests/avocado/machine_arm_integratorcp.py b/tests/avocado/machine_arm_integratorcp.py
index 1ffe1073ef..697ee76f6c 100644
--- a/tests/avocado/machine_arm_integratorcp.py
+++ b/tests/avocado/machine_arm_integratorcp.py
@@ -9,7 +9,6 @@
 # later.  See the COPYING file in the top-level directory.
 
 import os
-import logging
 
 from avocado import skipUnless
 from avocado_qemu import QemuSystemTest
@@ -84,7 +83,7 @@ def test_framebuffer_tux_logo(self):
         self.vm.command('human-monitor-command', command_line='stop')
         self.vm.command('human-monitor-command',
                         command_line='screendump %s' % screendump_path)
-        logger = logging.getLogger('framebuffer')
+        logger = self.log.getChild('framebuffer')
 
         cpu_count = 1
         match_threshold = 0.92
diff --git a/tests/avocado/machine_mips_malta.py b/tests/avocado/machine_mips_malta.py
index f1895d59f3..5f98ba1620 100644
--- a/tests/avocado/machine_mips_malta.py
+++ b/tests/avocado/machine_mips_malta.py
@@ -9,7 +9,6 @@
 
 import os
 import gzip
-import logging
 
 from avocado import skipUnless
 from avocado_qemu import QemuSystemTest
@@ -72,7 +71,7 @@ def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
         self.vm.command('human-monitor-command', command_line='stop')
         self.vm.command('human-monitor-command',
                         command_line='screendump %s' % screendump_path)
-        logger = logging.getLogger('framebuffer')
+        logger = self.log.getChild('framebuffer')
 
         match_threshold = 0.95
         screendump_bgr = cv2.imread(screendump_path, cv2.IMREAD_COLOR)
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
index c68a953730..40f52b3913 100644
--- a/tests/avocado/replay_kernel.py
+++ b/tests/avocado/replay_kernel.py
@@ -11,7 +11,6 @@
 import os
 import lzma
 import shutil
-import logging
 import time
 
 from avocado import skip
@@ -36,7 +35,7 @@ class ReplayKernelBase(LinuxKernelTest):
 
     def run_vm(self, kernel_path, kernel_command_line, console_pattern,
                record, shift, args, replay_path):
-        logger = logging.getLogger('replay')
+        logger = self.log.getChild('replay')
         start_time = time.time()
         vm = self.get_vm()
         vm.set_console()
@@ -74,7 +73,7 @@ def run_rr(self, kernel_path, kernel_command_line, console_pattern,
                          True, shift, args, replay_path)
         t2 = self.run_vm(kernel_path, kernel_command_line, console_pattern,
                          False, shift, args, replay_path)
-        logger = logging.getLogger('replay')
+        logger = self.log.getChild('replay')
         logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
 
 class ReplayKernelNormal(ReplayKernelBase):
diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
index 15953f9e49..b56eeccfdd 100644
--- a/tests/avocado/replay_linux.py
+++ b/tests/avocado/replay_linux.py
@@ -9,7 +9,6 @@
 # later.  See the COPYING file in the top-level directory.
 
 import os
-import logging
 import time
 
 from avocado import skipUnless
@@ -55,7 +54,7 @@ def launch_and_wait(self, record, args, shift):
             vm.add_args(*args)
         self.vm_add_disk(vm, self.boot_path, 0, self.hdd)
         self.vm_add_disk(vm, self.cloudinit_path, 1, self.cd)
-        logger = logging.getLogger('replay')
+        logger = self.log.getChild('replay')
         if record:
             logger.info('recording the execution...')
             mode = 'record'
@@ -91,7 +90,7 @@ def launch_and_wait(self, record, args, shift):
     def run_rr(self, args=None, shift=7):
         t1 = self.launch_and_wait(True, args, shift)
         t2 = self.launch_and_wait(False, args, shift)
-        logger = logging.getLogger('replay')
+        logger = self.log.getChild('replay')
         logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))
 
 @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
diff --git a/tests/avocado/reverse_debugging.py b/tests/avocado/reverse_debugging.py
index d2921e70c3..2cb39f3e59 100644
--- a/tests/avocado/reverse_debugging.py
+++ b/tests/avocado/reverse_debugging.py
@@ -8,7 +8,6 @@
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
 import os
-import logging
 
 from avocado import skipIf
 from avocado_qemu import BUILD_DIR
@@ -35,7 +34,7 @@ class ReverseDebugging(LinuxKernelTest):
     endian_is_le = True
 
     def run_vm(self, record, shift, args, replay_path, image_path, port):
-        logger = logging.getLogger('replay')
+        logger = self.log.getChild('replay')
         vm = self.get_vm()
         vm.set_console()
         if record:
@@ -95,7 +94,7 @@ def vm_get_icount(vm):
         return vm.qmp('query-replay')['return']['icount']
 
     def reverse_debugging(self, shift=7, args=None):
-        logger = logging.getLogger('replay')
+        logger = self.log.getChild('replay')
 
         # create qcow2 for snapshots
         logger.info('creating qcow2 image for VM snapshots')
diff --git a/tests/avocado/tesseract_utils.py b/tests/avocado/tesseract_utils.py
index 72cd9ab798..e0c5a6a454 100644
--- a/tests/avocado/tesseract_utils.py
+++ b/tests/avocado/tesseract_utils.py
@@ -31,8 +31,8 @@ def tesseract_available(expected_version):
 
 
 def tesseract_ocr(image_path, tesseract_args='', tesseract_version=3):
-    console_logger = logging.getLogger('tesseract')
-    console_logger.debug(image_path)
+    logger = logging.getLogger('avocado.test.tesseract')
+    logger.debug(image_path)
     if tesseract_version == 4:
         tesseract_args += ' --oem 1'
     proc = process.run("tesseract {} {} stdout".format(tesseract_args,
@@ -41,6 +41,6 @@ def tesseract_ocr(image_path, tesseract_args='', tesseract_version=3):
     for line in proc.stdout_text.split('\n'):
         sline = line.strip()
         if len(sline):
-            console_logger.debug(sline)
+            logger.debug(sline)
             lines += [sline]
     return lines
diff --git a/tests/avocado/virtio_check_params.py b/tests/avocado/virtio_check_params.py
index e869690473..fa19e9dbcb 100644
--- a/tests/avocado/virtio_check_params.py
+++ b/tests/avocado/virtio_check_params.py
@@ -20,7 +20,6 @@
 import sys
 import os
 import re
-import logging
 
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu.machine import QEMUMachine
@@ -75,7 +74,7 @@ def query_virtqueue(self, vm, dev_type_name):
 
     def check_mt(self, mt, dev_type_name):
         mt['device'] = dev_type_name # Only for the debug() call.
-        logger = logging.getLogger('machine')
+        logger = self.log.getChild('machine')
         logger.debug(mt)
         with QEMUMachine(self.qemu_bin) as vm:
             vm.set_machine(mt["name"])
-- 
2.35.1

[PATCH 3/9] Avocado migration test: adapt to "utils.network" API namespace change

[Cleber Rosa]

Since Avocado 94.0[1], the "avocado.utils.network" dropped a lot of
previously deprecated API names, having the new names into a finer
grained structure.

This simply uses the new API names for the network port utility
module.

[1] - https://avocado-framework.readthedocs.io/en/latest/releases/94_0.html#utility-apis

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/avocado/avocado_qemu/__init__.py | 5 +++--
 tests/avocado/migration.py             | 4 ++--
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
index 88cec83e5c..3f15c8a222 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -17,7 +17,8 @@
 import uuid
 
 import avocado
-from avocado.utils import cloudinit, datadrainer, network, process, ssh, vmimage
+from avocado.utils import cloudinit, datadrainer, process, ssh, vmimage
+from avocado.utils.network import ports
 from avocado.utils.path import find_command
 
 #: The QEMU build root directory.  It may also be the source directory
@@ -601,7 +602,7 @@ def prepare_cloudinit(self, ssh_pubkey=None):
         self.log.info('Preparing cloudinit image')
         try:
             cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
-            self.phone_home_port = network.find_free_port()
+            self.phone_home_port = ports.find_free_port()
             pubkey_content = None
             if ssh_pubkey:
                 with open(ssh_pubkey) as pubkey:
diff --git a/tests/avocado/migration.py b/tests/avocado/migration.py
index 584d6ef53f..4b25680c50 100644
--- a/tests/avocado/migration.py
+++ b/tests/avocado/migration.py
@@ -14,7 +14,7 @@
 from avocado_qemu import QemuSystemTest
 from avocado import skipUnless
 
-from avocado.utils import network
+from avocado.utils.network import ports
 from avocado.utils import wait
 from avocado.utils.path import find_command
 
@@ -57,7 +57,7 @@ def do_migrate(self, dest_uri, src_uri=None):
         self.assert_migration(source_vm, dest_vm)
 
     def _get_free_port(self):
-        port = network.find_free_port()
+        port = ports.find_free_port()
         if port is None:
             self.cancel('Failed to find a free port')
         return port
-- 
2.35.1

[PATCH 4/9] Avocado: bump to version 95.0

[Cleber Rosa]

Even though there have been a number of improvements (and some pretty
deep internal changes) since Avocado 88.1, only one change should
affect "make check-avocado".

With the nrunner architecture, test execution happens in parallel by
default.  But, tests may fail due to insufficient timeouts or similar
reasons when run under systems with limited or shared resources.  To
avoid breakages, especially on CI, let's keep the serial execution
until proven that it won't impact the CI jobs.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/Makefile.include | 1 +
 tests/requirements.txt | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e7153c8e91..676aa0d944 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -135,6 +135,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
             $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
 			--filter-by-tags-include-empty-key) \
             $(AVOCADO_CMDLINE_TAGS) \
+            --nrunner-max-parallel-tasks=1 \
             $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
             "AVOCADO", "tests/avocado")
 
diff --git a/tests/requirements.txt b/tests/requirements.txt
index a21b59b443..49aa0fd6f6 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,5 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==88.1
+avocado-framework==95.0
 pycdlib==1.11.0
-- 
2.35.1

[PATCH 5/9] tests/avocado/linux_ssh_mips_malta.py: add missing accel (tcg) tag

[Cleber Rosa]

Being explicit about the accelerator used on these tests is a good
thing in itself, but it will also be used in the filtering rules
applied on "make check-avocado".

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/avocado/linux_ssh_mips_malta.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
index c0f0be5ade..0179d8a6ca 100644
--- a/tests/avocado/linux_ssh_mips_malta.py
+++ b/tests/avocado/linux_ssh_mips_malta.py
@@ -23,6 +23,9 @@
 @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
 @skipUnless(ssh.SSH_CLIENT_BINARY, 'No SSH client available')
 class LinuxSSH(QemuSystemTest, LinuxSSHMixIn):
+    """
+    :avocado: tags=accel:tcg
+    """
 
     timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
 
-- 
2.35.1

[PATCH 6/9] tests/avocado/virtiofs_submounts.py: shared_dir may not exist

[Cleber Rosa]

If the test is skipped because of their conditionals, the shared_dir
attribute may not exist.

Check for its existence in the tearDown() method to avoid and
AttributeError.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/avocado/virtiofs_submounts.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/avocado/virtiofs_submounts.py b/tests/avocado/virtiofs_submounts.py
index e6dc32ffd4..d9c2c9d9ef 100644
--- a/tests/avocado/virtiofs_submounts.py
+++ b/tests/avocado/virtiofs_submounts.py
@@ -157,9 +157,10 @@ def tearDown(self):
         except:
             pass
 
-        scratch_dir = os.path.join(self.shared_dir, 'scratch')
-        self.run(('bash', self.get_data('cleanup.sh'), scratch_dir),
-                 ignore_error=True)
+        if hasattr(self, 'shared_dir'):
+            scratch_dir = os.path.join(self.shared_dir, 'scratch')
+            self.run(('bash', self.get_data('cleanup.sh'), scratch_dir),
+                     ignore_error=True)
 
     def test_pre_virtiofsd_set_up(self):
         self.set_up_shared_dir()
-- 
2.35.1

[PATCH 7/9] Avocado tests: improve documentation on tag filtering

[Cleber Rosa]

It's possible to filter based on a combination of criteria.  This adds
examples to the documentation.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 docs/devel/testing.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 92d40cdd19..f5b6e07b5c 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -936,6 +936,28 @@ in the current directory, tagged as "quick", run:
 
   avocado run -t quick .
 
+To run tests with a given value for a given tag, such as having the
+``accel`` tag set to ``kvm``, run:
+
+.. code::
+
+  avocado run -t accel:kvm .
+
+Multiple mandatory conditions can also be given.  To run only tests
+with ``arch`` set to ``x86_64`` and ``accell`` set to ``kvm``, run:
+
+.. code::
+
+  avocado run -t arch:x86_64,accel:kvm .
+
+It's also possible to exclude tests that contain a given value for a
+tag.  To list all tests that do *not* have ``arch`` set to ``x86_64``,
+run:
+
+.. code::
+
+  avocado run -t arch:-x86_64 .
+
 The ``avocado_qemu.Test`` base test class
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-- 
2.35.1

[PATCH 8/9] Avocado tests: classify tests based on what it's booted

[Cleber Rosa]

This adds some classification to the existing tests, based on the
mechanism (and a lot more loosely) on the content of the binary blob.

The proposal is to use the "boots" tag, and so far the following
values have been defined with the following meaning:

 - bios:   the "-bios" option is used to select the BIOS file to be
           loaded.  Because default bios are used in many QEMU runs,
           only tests that change the default are tagged.
 - kernel: means that the direct kernel boot mechanism (-kernel) is
           used.  Most of the time it means that a Linux kernel is
           booted, although there are occurrences of uboot usage.
 - initrd: means that an initial ram disk (-initrd) is used in
           addition to the kernel boot.
 - rootfs: means that a root filesystem is booted, in addition to a
           kernel and optionally an initrd.  This is usually done with
           a "-drive" command line option.
 - distro: means that a full blown distro image is booted, which may
           or may not include a kernel and initrd.  This is also
           usually done with a "-drive" command line option.

As with any other Avocado tags, it's possible to use them to select a
subset of tests.  For instance, if one wants to run tests that boots a
bios:

  $ avocado run -t boots:bios tests/avocado/

If one want to run tests that boots a kernel and an initrd:

  $ avocado run -t boots:kernel,boots:initrd tests/avocado/

It's possible, if deemed valuable, to further evolve this
classification into one with a clear separation between mechanism and
content.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/avocado/boot_linux.py               |  4 ++
 tests/avocado/boot_linux_console.py       | 54 +++++++++++++++++++++++
 tests/avocado/boot_xen.py                 |  3 ++
 tests/avocado/hotplug_cpu.py              |  1 +
 tests/avocado/intel_iommu.py              |  1 +
 tests/avocado/linux_initrd.py             |  2 +
 tests/avocado/linux_ssh_mips_malta.py     |  2 +
 tests/avocado/machine_arm_canona1100.py   |  1 +
 tests/avocado/machine_arm_integratorcp.py |  4 ++
 tests/avocado/machine_arm_n8x0.py         |  2 +
 tests/avocado/machine_avr6.py             |  1 +
 tests/avocado/machine_m68k_nextcube.py    |  1 +
 tests/avocado/machine_microblaze.py       |  1 +
 tests/avocado/machine_mips_fuloong2e.py   |  1 +
 tests/avocado/machine_mips_loongson3v.py  |  1 +
 tests/avocado/machine_mips_malta.py       |  3 ++
 tests/avocado/machine_rx_gdbsim.py        |  2 +
 tests/avocado/machine_s390_ccw_virtio.py  |  4 ++
 tests/avocado/machine_sparc64_sun4u.py    |  1 +
 tests/avocado/machine_sparc_leon3.py      |  1 +
 tests/avocado/multiprocess.py             |  4 ++
 tests/avocado/ppc_405.py                  |  2 +
 tests/avocado/ppc_bamboo.py               |  2 +
 tests/avocado/ppc_mpc8544ds.py            |  1 +
 tests/avocado/ppc_prep_40p.py             |  1 +
 tests/avocado/ppc_pseries.py              |  1 +
 tests/avocado/ppc_virtex_ml507.py         |  1 +
 tests/avocado/replay_kernel.py            | 28 ++++++++++++
 tests/avocado/replay_linux.py             |  1 +
 tests/avocado/reverse_debugging.py        |  1 +
 tests/avocado/smmu.py                     |  1 +
 tests/avocado/tcg_plugins.py              |  3 ++
 tests/avocado/virtio-gpu.py               |  2 +
 tests/avocado/virtiofs_submounts.py       |  1 +
 34 files changed, 139 insertions(+)

diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
index ab19146d1e..c4172f11e3 100644
--- a/tests/avocado/boot_linux.py
+++ b/tests/avocado/boot_linux.py
@@ -18,6 +18,7 @@
 class BootLinuxX8664(LinuxTest):
     """
     :avocado: tags=arch:x86_64
+    :avocado: tags=boots:distro
     """
 
     def test_pc_i440fx_tcg(self):
@@ -62,6 +63,7 @@ class BootLinuxAarch64(LinuxTest):
     :avocado: tags=arch:aarch64
     :avocado: tags=machine:virt
     :avocado: tags=machine:gic-version=2
+    :avocado: tags=boots:distro
     """
 
     def add_common_args(self):
@@ -110,6 +112,7 @@ def test_virt_kvm(self):
 class BootLinuxPPC64(LinuxTest):
     """
     :avocado: tags=arch:ppc64
+    :avocado: tags=boots:distro
     """
 
     def test_pseries_tcg(self):
@@ -125,6 +128,7 @@ def test_pseries_tcg(self):
 class BootLinuxS390X(LinuxTest):
     """
     :avocado: tags=arch:s390x
+    :avocado: tags=boots:distro
     """
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index 9c618d4809..0a8980953f 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -95,6 +95,7 @@ def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
@@ -115,6 +116,7 @@ def test_mips_malta(self):
         :avocado: tags=arch:mips
         :avocado: tags=machine:malta
         :avocado: tags=endian:big
+        :avocado: tags=boots:kernel
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20130217T032700Z/pool/main/l/linux-2.6/'
@@ -147,6 +149,7 @@ def test_mips64el_malta(self):
 
         :avocado: tags=arch:mips64el
         :avocado: tags=machine:malta
+        :avocado: tags=boots:kernel
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20130217T032700Z/pool/main/l/linux-2.6/'
@@ -169,6 +172,7 @@ def test_mips64el_fuloong2e(self):
         :avocado: tags=arch:mips64el
         :avocado: tags=machine:fuloong2e
         :avocado: tags=endian:little
+        :avocado: tags=boots:kernel
         """
         deb_url = ('http://archive.debian.org/debian/pool/main/l/linux/'
                    'linux-image-3.16.0-6-loongson-2e_3.16.56-1+deb8u1_mipsel.deb')
@@ -190,6 +194,8 @@ def test_mips_malta_cpio(self):
         :avocado: tags=arch:mips
         :avocado: tags=machine:malta
         :avocado: tags=endian:big
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20160601T041800Z/pool/main/l/linux/'
@@ -233,6 +239,8 @@ def test_mips64el_malta_5KEc_cpio(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:5KEc
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
                       'raw/9ad2df38/mips/malta/mips64el/'
@@ -292,6 +300,7 @@ def test_mips_malta32el_nanomips_4k(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:I7200
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
                       'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -305,6 +314,7 @@ def test_mips_malta32el_nanomips_16k_up(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:I7200
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
                       'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -318,6 +328,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:I7200
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
                       'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -331,6 +342,7 @@ def test_aarch64_virt(self):
         :avocado: tags=machine:virt
         :avocado: tags=accel:tcg
         :avocado: tags=cpu:cortex-a53
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
@@ -357,6 +369,8 @@ def test_aarch64_xlnx_versal_virt(self):
         :avocado: tags=device:pl011
         :avocado: tags=device:arm_gicv3
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         images_url = ('http://ports.ubuntu.com/ubuntu-ports/dists/'
                       'bionic-updates/main/installer-arm64/'
@@ -382,6 +396,7 @@ def test_arm_virt(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:virt
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
@@ -405,6 +420,8 @@ def test_arm_emcraft_sf2(self):
         :avocado: tags=endian:little
         :avocado: tags=u-boot
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:rootfs
         """
         uboot_url = ('https://raw.githubusercontent.com/'
                      'Subbaraya-Sundeep/qemu-test-binaries/'
@@ -471,6 +488,7 @@ def test_arm_raspi2_uart0(self):
         :avocado: tags=machine:raspi2b
         :avocado: tags=device:pl011
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
         """
         self.do_test_arm_raspi2(0)
 
@@ -478,6 +496,8 @@ def test_arm_raspi2_initrd(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:raspi2b
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('http://archive.raspberrypi.org/debian/'
                    'pool/main/r/raspberrypi-firmware/'
@@ -521,6 +541,8 @@ def test_arm_exynos4210_initrd(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:smdkc210
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('https://snapshot.debian.org/archive/debian/'
                    '20190928T224601Z/pool/main/l/linux/'
@@ -562,6 +584,8 @@ def test_arm_cubieboard_initrd(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:cubieboard
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('https://apt.armbian.com/pool/main/l/'
                    'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
@@ -603,6 +627,8 @@ def test_arm_cubieboard_sata(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:cubieboard
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:rootfs
         """
         deb_url = ('https://apt.armbian.com/pool/main/l/'
                    'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
@@ -648,6 +674,7 @@ def test_arm_quanta_gsj(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:quanta-gsj
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:rootfs
         """
         # 25 MiB compressed, 32 MiB uncompressed.
         image_url = (
@@ -696,6 +723,8 @@ def test_arm_quanta_gsj_initrd(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:quanta-gsj
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         initrd_url = (
                 'https://github.com/hskinnemoen/openbmc/releases/download/'
@@ -733,6 +762,7 @@ def test_arm_orangepi(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:orangepi-pc
         :avocado: tags=accel:tcg
+        :avocado: tags=boots:kernel
         """
         deb_url = ('https://apt.armbian.com/pool/main/l/'
                    'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
@@ -759,6 +789,8 @@ def test_arm_orangepi_initrd(self):
         :avocado: tags=arch:arm
         :avocado: tags=accel:tcg
         :avocado: tags=machine:orangepi-pc
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('https://apt.armbian.com/pool/main/l/'
                    'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
@@ -803,6 +835,8 @@ def test_arm_orangepi_sd(self):
         :avocado: tags=accel:tcg
         :avocado: tags=machine:orangepi-pc
         :avocado: tags=device:sd
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:rootfs
         """
         deb_url = ('https://apt.armbian.com/pool/main/l/'
                    'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
@@ -855,6 +889,7 @@ def test_arm_orangepi_bionic_20_08(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:orangepi-pc
         :avocado: tags=device:sd
+        :avocado: tags=boots:distro
         """
 
         # This test download a 275 MiB compressed image and expand it
@@ -902,6 +937,7 @@ def test_arm_orangepi_uboot_netbsd9(self):
         :avocado: tags=machine:orangepi-pc
         :avocado: tags=device:sd
         :avocado: tags=os:netbsd
+        :avocado: tags=boots:distro
         """
         # This test download a 304MB compressed image and expand it to 2GB
         deb_url = ('http://snapshot.debian.org/archive/debian/'
@@ -987,6 +1023,7 @@ def test_s390x_s390_ccw_virtio(self):
         """
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390-ccw-virtio
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive'
                       '/fedora-secondary/releases/29/Everything/s390x/os/images'
@@ -1007,6 +1044,7 @@ def test_alpha_clipper(self):
         """
         :avocado: tags=arch:alpha
         :avocado: tags=machine:clipper
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
                       'installer-alpha/20090123lenny10/images/cdrom/vmlinuz')
@@ -1028,6 +1066,7 @@ def test_m68k_q800(self):
         """
         :avocado: tags=arch:m68k
         :avocado: tags=machine:q800
+        :avocado: tags=boots:kernel
         """
         deb_url = ('https://snapshot.debian.org/archive/debian-ports'
                    '/20191021T083923Z/pool-m68k/main'
@@ -1063,6 +1102,7 @@ def test_arm_vexpressa9(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:vexpress-a9
+        :avocado: tags=boots:kernel
         """
         tar_hash = '32b7677ce8b6f1471fb0059865f451169934245b'
         self.vm.add_args('-dtb', self.workdir + '/day16/vexpress-v2p-ca9.dtb')
@@ -1072,6 +1112,7 @@ def test_arm_ast2400_palmetto_openbmc_v2_9_0(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:palmetto-bmc
+        :avocado: tags=boots:rootfs
         """
 
         image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
@@ -1086,6 +1127,7 @@ def test_arm_ast2500_romulus_openbmc_v2_9_0(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:romulus-bmc
+        :avocado: tags=boots:rootfs
         """
 
         image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
@@ -1115,6 +1157,7 @@ def test_arm_ast2600_debian(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:tacoma-bmc
+        :avocado: tags=boots:kernel
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20210302T203551Z/'
@@ -1140,6 +1183,7 @@ def test_m68k_mcf5208evb(self):
         """
         :avocado: tags=arch:m68k
         :avocado: tags=machine:mcf5208evb
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
         self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
@@ -1148,6 +1192,7 @@ def test_or1k_sim(self):
         """
         :avocado: tags=arch:or1k
         :avocado: tags=machine:or1k-sim
+        :avocado: tags=boots:kernel
         """
         tar_hash = '20334cdaf386108c530ff0badaecc955693027dd'
         self.do_test_advcal_2018('20', tar_hash, 'vmlinux')
@@ -1156,6 +1201,7 @@ def test_nios2_10m50(self):
         """
         :avocado: tags=arch:nios2
         :avocado: tags=machine:10m50-ghrd
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'e4251141726c412ac0407c5a6bceefbbff018918'
         self.do_test_advcal_2018('14', tar_hash, 'vmlinux.elf')
@@ -1165,6 +1211,7 @@ def test_ppc64_e500(self):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:ppce500
         :avocado: tags=cpu:e5500
+        :avocado: tags=boots:kernel
         """
         tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
         self.do_test_advcal_2018('19', tar_hash, 'uImage')
@@ -1194,6 +1241,7 @@ def test_ppc_powernv8(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:powernv8
+        :avocado: tags=boots:kernel
         """
         self.do_test_ppc64_powernv('P8')
 
@@ -1201,6 +1249,7 @@ def test_ppc_powernv9(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:powernv9
+        :avocado: tags=boots:kernel
         """
         self.do_test_ppc64_powernv('P9')
 
@@ -1208,6 +1257,7 @@ def test_ppc_g3beige(self):
         """
         :avocado: tags=arch:ppc
         :avocado: tags=machine:g3beige
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
         self.vm.add_args('-M', 'graphics=off')
@@ -1217,6 +1267,7 @@ def test_ppc_mac99(self):
         """
         :avocado: tags=arch:ppc
         :avocado: tags=machine:mac99
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
         self.vm.add_args('-M', 'graphics=off')
@@ -1226,6 +1277,7 @@ def test_sh4_r2d(self):
         """
         :avocado: tags=arch:sh4
         :avocado: tags=machine:r2d
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'fe06a4fd8ccbf2e27928d64472939d47829d4c7e'
         self.vm.add_args('-append', 'console=ttySC1')
@@ -1235,6 +1287,7 @@ def test_sparc_ss20(self):
         """
         :avocado: tags=arch:sparc
         :avocado: tags=machine:SS-20
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'b18550d5d61c7615d989a06edace051017726a9f'
         self.do_test_advcal_2018('11', tar_hash, 'zImage.elf')
@@ -1244,6 +1297,7 @@ def test_xtensa_lx60(self):
         :avocado: tags=arch:xtensa
         :avocado: tags=machine:lx60
         :avocado: tags=cpu:dc233c
+        :avocado: tags=boots:kernel
         """
         tar_hash = '49e88d9933742f0164b60839886c9739cb7a0d34'
         self.do_test_advcal_2018('02', tar_hash, 'santas-sleigh-ride.elf')
diff --git a/tests/avocado/boot_xen.py b/tests/avocado/boot_xen.py
index fc2faeedb5..6cc5216a57 100644
--- a/tests/avocado/boot_xen.py
+++ b/tests/avocado/boot_xen.py
@@ -61,6 +61,9 @@ def launch_xen(self, xen_path):
 
 
 class BootXen(BootXenBase):
+    """
+    :avocado: tags=boots:kernel
+    """
 
     def test_arm64_xen_411_and_dom0(self):
         """
diff --git a/tests/avocado/hotplug_cpu.py b/tests/avocado/hotplug_cpu.py
index 6374bf1b54..bb296020af 100644
--- a/tests/avocado/hotplug_cpu.py
+++ b/tests/avocado/hotplug_cpu.py
@@ -18,6 +18,7 @@ def test(self):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:q35
         :avocado: tags=accel:kvm
+        :avocado: tags=boots:distro
         """
         self.require_accelerator('kvm')
         self.vm.add_args('-accel', 'kvm')
diff --git a/tests/avocado/intel_iommu.py b/tests/avocado/intel_iommu.py
index 474d62f6bf..d308dcccc5 100644
--- a/tests/avocado/intel_iommu.py
+++ b/tests/avocado/intel_iommu.py
@@ -20,6 +20,7 @@ class IntelIOMMU(LinuxTest):
     :avocado: tags=distro_version:31
     :avocado: tags=machine:q35
     :avocado: tags=accel:kvm
+    :avocado: tags=boots:distro
     :avocado: tags=intel_iommu
     """
 
diff --git a/tests/avocado/linux_initrd.py b/tests/avocado/linux_initrd.py
index 6ebf299cd4..07582f5247 100644
--- a/tests/avocado/linux_initrd.py
+++ b/tests/avocado/linux_initrd.py
@@ -21,6 +21,8 @@ class LinuxInitrd(QemuSystemTest):
 
     :avocado: tags=arch:x86_64
     :avocado: tags=machine:pc
+    :avocado: tags=boots:kernel
+    :avocado: tags=boots:initrd
     """
 
     timeout = 300
diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
index 0179d8a6ca..2cd986c6b2 100644
--- a/tests/avocado/linux_ssh_mips_malta.py
+++ b/tests/avocado/linux_ssh_mips_malta.py
@@ -25,6 +25,8 @@
 class LinuxSSH(QemuSystemTest, LinuxSSHMixIn):
     """
     :avocado: tags=accel:tcg
+    :avocado: tags=boots:kernel
+    :avocado: tags=boots:distro
     """
 
     timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
diff --git a/tests/avocado/machine_arm_canona1100.py b/tests/avocado/machine_arm_canona1100.py
index 182a0b0513..0cfb9b873d 100644
--- a/tests/avocado/machine_arm_canona1100.py
+++ b/tests/avocado/machine_arm_canona1100.py
@@ -22,6 +22,7 @@ def test_arm_canona1100(self):
         :avocado: tags=arch:arm
         :avocado: tags=machine:canon-a1100
         :avocado: tags=device:pflash_cfi02
+        :avocado: tags=boots:bios
         """
         tar_url = ('https://www.qemu-advent-calendar.org'
                    '/2018/download/day18.tar.xz')
diff --git a/tests/avocado/machine_arm_integratorcp.py b/tests/avocado/machine_arm_integratorcp.py
index 697ee76f6c..3049aff037 100644
--- a/tests/avocado/machine_arm_integratorcp.py
+++ b/tests/avocado/machine_arm_integratorcp.py
@@ -29,6 +29,10 @@
 
 
 class IntegratorMachine(QemuSystemTest):
+    """
+    :avocado: tags=boots:kernel
+    :avocado: tags=boots:initrd
+    """
 
     timeout = 90
 
diff --git a/tests/avocado/machine_arm_n8x0.py b/tests/avocado/machine_arm_n8x0.py
index 12e9a6803b..78c65863b6 100644
--- a/tests/avocado/machine_arm_n8x0.py
+++ b/tests/avocado/machine_arm_n8x0.py
@@ -37,6 +37,7 @@ def test_n800(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:n800
+        :avocado: tags=boots:kernel
         """
         self.__do_test_n8x0()
 
@@ -45,5 +46,6 @@ def test_n810(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:n810
+        :avocado: tags=boots:kernel
         """
         self.__do_test_n8x0()
diff --git a/tests/avocado/machine_avr6.py b/tests/avocado/machine_avr6.py
index 5485db79c6..0837d4dd6b 100644
--- a/tests/avocado/machine_avr6.py
+++ b/tests/avocado/machine_avr6.py
@@ -28,6 +28,7 @@ def test_freertos(self):
         """
         :avocado: tags=arch:avr
         :avocado: tags=machine:arduino-mega-2560-v3
+        :avocado: tags=boots:bios
         """
         """
         https://github.com/seharris/qemu-avr-tests/raw/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf
diff --git a/tests/avocado/machine_m68k_nextcube.py b/tests/avocado/machine_m68k_nextcube.py
index 6790e7d9cd..26df7f89c4 100644
--- a/tests/avocado/machine_m68k_nextcube.py
+++ b/tests/avocado/machine_m68k_nextcube.py
@@ -25,6 +25,7 @@ class NextCubeMachine(QemuSystemTest):
     :avocado: tags=arch:m68k
     :avocado: tags=machine:next-cube
     :avocado: tags=device:framebuffer
+    :avocado: tags=boots:bios
     """
 
     timeout = 15
diff --git a/tests/avocado/machine_microblaze.py b/tests/avocado/machine_microblaze.py
index 4928920f96..9035c2ef3d 100644
--- a/tests/avocado/machine_microblaze.py
+++ b/tests/avocado/machine_microblaze.py
@@ -17,6 +17,7 @@ def test_microblaze_s3adsp1800(self):
         """
         :avocado: tags=arch:microblaze
         :avocado: tags=machine:petalogix-s3adsp1800
+        :avocado: tags=boots:kernel
         """
 
         tar_url = ('https://www.qemu-advent-calendar.org'
diff --git a/tests/avocado/machine_mips_fuloong2e.py b/tests/avocado/machine_mips_fuloong2e.py
index 89291f47b2..f1da0bbb75 100644
--- a/tests/avocado/machine_mips_fuloong2e.py
+++ b/tests/avocado/machine_mips_fuloong2e.py
@@ -26,6 +26,7 @@ def test_linux_kernel_isa_serial(self):
         :avocado: tags=endian:little
         :avocado: tags=device:bonito64
         :avocado: tags=device:via686b
+        :avocado: tags=boots:kernel
         """
         # Recovery system for the Yeeloong laptop
         # (enough to test the fuloong2e southbridge, accessing its ISA bus)
diff --git a/tests/avocado/machine_mips_loongson3v.py b/tests/avocado/machine_mips_loongson3v.py
index 5194cf18c9..5680fc6892 100644
--- a/tests/avocado/machine_mips_loongson3v.py
+++ b/tests/avocado/machine_mips_loongson3v.py
@@ -26,6 +26,7 @@ def test_pmon_serial_console(self):
         :avocado: tags=cpu:Loongson-3A1000
         :avocado: tags=device:liointc
         :avocado: tags=device:goldfish_rtc
+        :avocado: tags=boots:bios
         """
 
         pmon_hash = '7c8b45dd81ccfc55ff28f5aa267a41c3'
diff --git a/tests/avocado/machine_mips_malta.py b/tests/avocado/machine_mips_malta.py
index 5f98ba1620..e0b015b871 100644
--- a/tests/avocado/machine_mips_malta.py
+++ b/tests/avocado/machine_mips_malta.py
@@ -95,6 +95,7 @@ def test_mips_malta_i6400_framebuffer_logo_1core(self):
         :avocado: tags=arch:mips64el
         :avocado: tags=machine:malta
         :avocado: tags=cpu:I6400
+        :avocado: tags=boots:kernel
         """
         self.do_test_i6400_framebuffer_logo(1)
 
@@ -105,6 +106,7 @@ def test_mips_malta_i6400_framebuffer_logo_7cores(self):
         :avocado: tags=machine:malta
         :avocado: tags=cpu:I6400
         :avocado: tags=mips:smp
+        :avocado: tags=boots:kernel
         """
         self.do_test_i6400_framebuffer_logo(7)
 
@@ -115,5 +117,6 @@ def test_mips_malta_i6400_framebuffer_logo_8cores(self):
         :avocado: tags=machine:malta
         :avocado: tags=cpu:I6400
         :avocado: tags=mips:smp
+        :avocado: tags=boots:kernel
         """
         self.do_test_i6400_framebuffer_logo(8)
diff --git a/tests/avocado/machine_rx_gdbsim.py b/tests/avocado/machine_rx_gdbsim.py
index 6cd8704b01..bb3263d982 100644
--- a/tests/avocado/machine_rx_gdbsim.py
+++ b/tests/avocado/machine_rx_gdbsim.py
@@ -30,6 +30,7 @@ def test_uboot(self):
         :avocado: tags=arch:rx
         :avocado: tags=machine:gdbsim-r5f562n8
         :avocado: tags=endian:little
+        :avocado: tags=boots:bios
         """
         uboot_url = ('https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.gz')
         uboot_hash = '9b78dbd43b40b2526848c0b1ce9de02c24f4dcdb'
@@ -54,6 +55,7 @@ def test_linux_sash(self):
         :avocado: tags=arch:rx
         :avocado: tags=machine:gdbsim-r5f562n7
         :avocado: tags=endian:little
+        :avocado: tags=boots:kernel
         """
         dtb_url = ('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb')
         dtb_hash = '7b4e4e2c71905da44e86ce47adee2210b026ac18'
diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/avocado/machine_s390_ccw_virtio.py
index bd03d7160b..0e24372240 100644
--- a/tests/avocado/machine_s390_ccw_virtio.py
+++ b/tests/avocado/machine_s390_ccw_virtio.py
@@ -45,6 +45,8 @@ def test_s390x_devices(self):
         """
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390-ccw-virtio
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
 
         kernel_url = ('https://snapshot.debian.org/archive/debian/'
@@ -167,6 +169,8 @@ def test_s390x_fedora(self):
         :avocado: tags=device:virtio-gpu
         :avocado: tags=device:virtio-crypto
         :avocado: tags=device:virtio-net
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
 
         kernel_url = ('https://archives.fedoraproject.org/pub/archive'
diff --git a/tests/avocado/machine_sparc64_sun4u.py b/tests/avocado/machine_sparc64_sun4u.py
index 458165500e..18fcb92fcd 100644
--- a/tests/avocado/machine_sparc64_sun4u.py
+++ b/tests/avocado/machine_sparc64_sun4u.py
@@ -23,6 +23,7 @@ def test_sparc64_sun4u(self):
         """
         :avocado: tags=arch:sparc64
         :avocado: tags=machine:sun4u
+        :avocado: tags=boots:kernel
         """
         tar_url = ('https://www.qemu-advent-calendar.org'
                    '/2018/download/day23.tar.xz')
diff --git a/tests/avocado/machine_sparc_leon3.py b/tests/avocado/machine_sparc_leon3.py
index e61b223185..c7f941a525 100644
--- a/tests/avocado/machine_sparc_leon3.py
+++ b/tests/avocado/machine_sparc_leon3.py
@@ -22,6 +22,7 @@ def test_leon3_helenos_uimage(self):
         :avocado: tags=arch:sparc
         :avocado: tags=machine:leon3_generic
         :avocado: tags=binfmt:uimage
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('http://www.helenos.org/releases/'
                       'HelenOS-0.6.0-sparc32-leon3.bin')
diff --git a/tests/avocado/multiprocess.py b/tests/avocado/multiprocess.py
index 80a3b8f442..9415e6d00c 100644
--- a/tests/avocado/multiprocess.py
+++ b/tests/avocado/multiprocess.py
@@ -67,6 +67,8 @@ def do_test(self, kernel_url, initrd_url, kernel_command_line,
     def test_multiprocess_x86_64(self):
         """
         :avocado: tags=arch:x86_64
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/31/Everything/x86_64/os/images'
@@ -82,6 +84,8 @@ def test_multiprocess_x86_64(self):
     def test_multiprocess_aarch64(self):
         """
         :avocado: tags=arch:aarch64
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/31/Everything/aarch64/os/images'
diff --git a/tests/avocado/ppc_405.py b/tests/avocado/ppc_405.py
index a47f89b934..f767ca17c7 100644
--- a/tests/avocado/ppc_405.py
+++ b/tests/avocado/ppc_405.py
@@ -30,6 +30,7 @@ def test_ppc_taihu(self):
         :avocado: tags=arch:ppc
         :avocado: tags=machine:taihu
         :avocado: tags=cpu:405ep
+        :avocado: tags=boots:bios
         """
         self.do_test_ppc405()
 
@@ -38,5 +39,6 @@ def test_ppc_ref405ep(self):
         :avocado: tags=arch:ppc
         :avocado: tags=machine:ref405ep
         :avocado: tags=cpu:405ep
+        :avocado: tags=boots:bios
         """
         self.do_test_ppc405()
diff --git a/tests/avocado/ppc_bamboo.py b/tests/avocado/ppc_bamboo.py
index 40629e3478..7568ab2b6f 100644
--- a/tests/avocado/ppc_bamboo.py
+++ b/tests/avocado/ppc_bamboo.py
@@ -20,6 +20,8 @@ def test_ppc_bamboo(self):
         :avocado: tags=machine:bamboo
         :avocado: tags=cpu:440epb
         :avocado: tags=device:rtl8139
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         tar_url = ('http://landley.net/aboriginal/downloads/binaries/'
                    'system-image-powerpc-440fp.tar.gz')
diff --git a/tests/avocado/ppc_mpc8544ds.py b/tests/avocado/ppc_mpc8544ds.py
index 886f967b15..12efb52ea7 100644
--- a/tests/avocado/ppc_mpc8544ds.py
+++ b/tests/avocado/ppc_mpc8544ds.py
@@ -19,6 +19,7 @@ def test_ppc_mpc8544ds(self):
         """
         :avocado: tags=arch:ppc
         :avocado: tags=machine:mpc8544ds
+        :avocado: tags=boots:kernel
         """
         tar_url = ('https://www.qemu-advent-calendar.org'
                    '/2020/download/day17.tar.gz')
diff --git a/tests/avocado/ppc_prep_40p.py b/tests/avocado/ppc_prep_40p.py
index 4bd956584d..fde18ac42c 100644
--- a/tests/avocado/ppc_prep_40p.py
+++ b/tests/avocado/ppc_prep_40p.py
@@ -28,6 +28,7 @@ def test_factory_firmware_and_netbsd(self):
         :avocado: tags=machine:40p
         :avocado: tags=os:netbsd
         :avocado: tags=slowness:high
+        :avocado: tags=boots:bios
         """
         bios_url = ('http://ftpmirror.your.org/pub/misc/'
                     'ftp.software.ibm.com/rs6000/firmware/'
diff --git a/tests/avocado/ppc_pseries.py b/tests/avocado/ppc_pseries.py
index d8b04dc3ea..388dfe5487 100644
--- a/tests/avocado/ppc_pseries.py
+++ b/tests/avocado/ppc_pseries.py
@@ -19,6 +19,7 @@ def test_ppc64_pseries(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive'
                       '/fedora-secondary/releases/29/Everything/ppc64le/os'
diff --git a/tests/avocado/ppc_virtex_ml507.py b/tests/avocado/ppc_virtex_ml507.py
index a6912ee579..d9afe2cb83 100644
--- a/tests/avocado/ppc_virtex_ml507.py
+++ b/tests/avocado/ppc_virtex_ml507.py
@@ -19,6 +19,7 @@ def test_ppc_virtex_ml507(self):
         """
         :avocado: tags=arch:ppc
         :avocado: tags=machine:virtex-ml507
+        :avocado: tags=boots:kernel
         """
         tar_url = ('https://www.qemu-advent-calendar.org'
                    '/2020/download/hippo.tar.gz')
diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
index 40f52b3913..368c6b41c8 100644
--- a/tests/avocado/replay_kernel.py
+++ b/tests/avocado/replay_kernel.py
@@ -82,6 +82,7 @@ def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
@@ -99,6 +100,7 @@ def test_mips_malta(self):
         :avocado: tags=arch:mips
         :avocado: tags=machine:malta
         :avocado: tags=endian:big
+        :avocado: tags=boots:kernel
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20130217T032700Z/pool/main/l/linux-2.6/'
@@ -127,6 +129,7 @@ def test_mips64el_malta(self):
 
         :avocado: tags=arch:mips64el
         :avocado: tags=machine:malta
+        :avocado: tags=boots:kernel
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20130217T032700Z/pool/main/l/linux-2.6/'
@@ -144,6 +147,7 @@ def test_aarch64_virt(self):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         :avocado: tags=cpu:cortex-a53
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
@@ -161,6 +165,7 @@ def test_arm_virt(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:virt
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
@@ -179,6 +184,8 @@ def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:cubieboard
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('https://apt.armbian.com/pool/main/l/'
                    'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
@@ -210,6 +217,7 @@ def test_s390x_s390_ccw_virtio(self):
         """
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390-ccw-virtio
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive'
                       '/fedora-secondary/releases/29/Everything/s390x/os/images'
@@ -225,6 +233,7 @@ def test_alpha_clipper(self):
         """
         :avocado: tags=arch:alpha
         :avocado: tags=machine:clipper
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
                       'installer-alpha/20090123lenny10/images/cdrom/vmlinuz')
@@ -242,6 +251,7 @@ def test_ppc64_pseries(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive'
                       '/fedora-secondary/releases/29/Everything/ppc64le/os'
@@ -258,6 +268,7 @@ def test_m68k_q800(self):
         """
         :avocado: tags=arch:m68k
         :avocado: tags=machine:q800
+        :avocado: tags=boots:kernel
         """
         deb_url = ('https://snapshot.debian.org/archive/debian-ports'
                    '/20191021T083923Z/pool-m68k/main'
@@ -289,6 +300,7 @@ def test_arm_vexpressa9(self):
         """
         :avocado: tags=arch:arm
         :avocado: tags=machine:vexpress-a9
+        :avocado: tags=boots:kernel
         """
         tar_hash = '32b7677ce8b6f1471fb0059865f451169934245b'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -302,6 +314,7 @@ def test_m68k_mcf5208evb(self):
         """
         :avocado: tags=arch:m68k
         :avocado: tags=machine:mcf5208evb
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -314,6 +327,7 @@ def test_microblaze_s3adsp1800(self):
         """
         :avocado: tags=arch:microblaze
         :avocado: tags=machine:petalogix-s3adsp1800
+        :avocado: tags=boots:kernel
         """
         tar_hash = '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -326,6 +340,7 @@ def test_ppc64_e500(self):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:ppce500
         :avocado: tags=cpu:e5500
+        :avocado: tags=boots:kernel
         """
         tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -337,6 +352,7 @@ def test_or1k_sim(self):
         """
         :avocado: tags=arch:or1k
         :avocado: tags=machine:or1k-sim
+        :avocado: tags=boots:kernel
         """
         tar_hash = '20334cdaf386108c530ff0badaecc955693027dd'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -348,6 +364,7 @@ def test_nios2_10m50(self):
         """
         :avocado: tags=arch:nios2
         :avocado: tags=machine:10m50-ghrd
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'e4251141726c412ac0407c5a6bceefbbff018918'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -359,6 +376,7 @@ def test_ppc_g3beige(self):
         """
         :avocado: tags=arch:ppc
         :avocado: tags=machine:g3beige
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -371,6 +389,7 @@ def test_ppc_mac99(self):
         """
         :avocado: tags=arch:ppc
         :avocado: tags=machine:mac99
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -383,6 +402,7 @@ def test_sparc_ss20(self):
         """
         :avocado: tags=arch:sparc
         :avocado: tags=machine:SS-20
+        :avocado: tags=boots:kernel
         """
         tar_hash = 'b18550d5d61c7615d989a06edace051017726a9f'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -395,6 +415,7 @@ def test_xtensa_lx60(self):
         :avocado: tags=arch:xtensa
         :avocado: tags=machine:lx60
         :avocado: tags=cpu:dc233c
+        :avocado: tags=boots:kernel
         """
         tar_hash = '49e88d9933742f0164b60839886c9739cb7a0d34'
         tar_url = ('https://www.qemu-advent-calendar.org'
@@ -415,6 +436,8 @@ def test_mips_malta_cpio(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:big
         :avocado: tags=slowness:high
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         deb_url = ('http://snapshot.debian.org/archive/debian/'
                    '20160601T041800Z/pool/main/l/linux/'
@@ -446,6 +469,8 @@ def test_mips64el_malta_5KEc_cpio(self):
         :avocado: tags=endian:little
         :avocado: tags=slowness:high
         :avocado: tags=cpu:5KEc
+        :avocado: tags=boots:kernel
+        :avocado: tags=boots:initrd
         """
         kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
                       'raw/9ad2df38/mips/malta/mips64el/'
@@ -486,6 +511,7 @@ def test_mips_malta32el_nanomips_4k(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:I7200
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
                       'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -500,6 +526,7 @@ def test_mips_malta32el_nanomips_16k_up(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:I7200
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
                       'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -514,6 +541,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
         :avocado: tags=machine:malta
         :avocado: tags=endian:little
         :avocado: tags=cpu:I7200
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
                       'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
index b56eeccfdd..b2c02c07f6 100644
--- a/tests/avocado/replay_linux.py
+++ b/tests/avocado/replay_linux.py
@@ -98,6 +98,7 @@ class ReplayLinuxX8664(ReplayLinux):
     """
     :avocado: tags=arch:x86_64
     :avocado: tags=accel:tcg
+    :avocado: tags=boots:distro
     """
 
     chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
diff --git a/tests/avocado/reverse_debugging.py b/tests/avocado/reverse_debugging.py
index 2cb39f3e59..f198173bf7 100644
--- a/tests/avocado/reverse_debugging.py
+++ b/tests/avocado/reverse_debugging.py
@@ -198,6 +198,7 @@ def test_aarch64_virt(self):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         :avocado: tags=cpu:cortex-a53
+        :avocado: tags=boots:kernel
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
diff --git a/tests/avocado/smmu.py b/tests/avocado/smmu.py
index b3c4de6bf4..dcc7402beb 100644
--- a/tests/avocado/smmu.py
+++ b/tests/avocado/smmu.py
@@ -19,6 +19,7 @@ class SMMU(LinuxTest):
     :avocado: tags=cpu:host
     :avocado: tags=arch:aarch64
     :avocado: tags=machine:virt
+    :avocado: tags=boots:distro
     :avocado: tags=distro:fedora
     :avocado: tags=smmu
     """
diff --git a/tests/avocado/tcg_plugins.py b/tests/avocado/tcg_plugins.py
index 642d2e49e3..296e663152 100644
--- a/tests/avocado/tcg_plugins.py
+++ b/tests/avocado/tcg_plugins.py
@@ -69,6 +69,7 @@ def test_aarch64_virt_insn(self):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         :avocado: tags=cpu:cortex-a53
+        :avocado: tags=boots:kernel
         """
         kernel_path = self._grab_aarch64_kernel()
         kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -95,6 +96,7 @@ def test_aarch64_virt_insn_icount(self):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         :avocado: tags=cpu:cortex-a53
+        :avocado: tags=boots:kernel
         """
         kernel_path = self._grab_aarch64_kernel()
         kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -121,6 +123,7 @@ def test_aarch64_virt_mem_icount(self):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         :avocado: tags=cpu:cortex-a53
+        :avocado: tags=boots:kernel
         """
         kernel_path = self._grab_aarch64_kernel()
         kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
diff --git a/tests/avocado/virtio-gpu.py b/tests/avocado/virtio-gpu.py
index 2a249a3a2c..e91efac8ea 100644
--- a/tests/avocado/virtio-gpu.py
+++ b/tests/avocado/virtio-gpu.py
@@ -32,6 +32,8 @@ class VirtioGPUx86(QemuSystemTest):
     :avocado: tags=virtio-gpu
     :avocado: tags=arch:x86_64
     :avocado: tags=cpu:host
+    :avocado: tags=boots:kernel
+    :avocado: tags=boots:initrd
     """
 
     KERNEL_COMMAND_LINE = "printk.time=0 console=ttyS0 rdinit=/bin/bash"
diff --git a/tests/avocado/virtiofs_submounts.py b/tests/avocado/virtiofs_submounts.py
index d9c2c9d9ef..3848f351fb 100644
--- a/tests/avocado/virtiofs_submounts.py
+++ b/tests/avocado/virtiofs_submounts.py
@@ -16,6 +16,7 @@ class VirtiofsSubmountsTest(LinuxTest):
     """
     :avocado: tags=arch:x86_64
     :avocado: tags=accel:kvm
+    :avocado: tags=boots:distro
     """
 
     def run(self, args, ignore_error=False):
-- 
2.35.1

[PATCH 9/9] Avocado tests: don't run tests with TCG that boot full blown distros

[Cleber Rosa]

Tests that use TCG and boot full blown distros, such as Fedora, will
take a good time to run.  This excludes those combinations by default
on invocations of "make check-avocado".

Tests that rely on KVM instead, will continue to run.

As a reminder, one can always supply a list of tests or tags to be
used on a "make check-avocado" by setting AVOCADO_TESTS or
AVOCADO_TAGS.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/Makefile.include | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 676aa0d944..6d9cf7cbc9 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -91,8 +91,11 @@ endif
 # Any number of command separated loggers are accepted.  For more
 # information please refer to "avocado --help".
 AVOCADO_SHOW=app
+comma:=,
 ifndef AVOCADO_TAGS
-	AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
+       AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%$(comma)accel:tcg$(comma)boots:-distro, \
+						 $(filter %-softmmu,$(TARGETS)))
+       AVOCADO_CMDLINE_TAGS+=$(patsubst %-softmmu,-t arch:%$(comma)accel:kvm, \
 						 $(filter %-softmmu,$(TARGETS)))
 else
 	AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
-- 
2.35.1

Re: [PATCH 5/9] tests/avocado/linux_ssh_mips_malta.py: add missing accel (tcg) tag

[Philippe Mathieu-Daudé]

On 25/2/22 22:01, Cleber Rosa wrote:
> Being explicit about the accelerator used on these tests is a good
> thing in itself, but it will also be used in the filtering rules
> applied on "make check-avocado".
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/avocado/linux_ssh_mips_malta.py | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
> index c0f0be5ade..0179d8a6ca 100644
> --- a/tests/avocado/linux_ssh_mips_malta.py
> +++ b/tests/avocado/linux_ssh_mips_malta.py
> @@ -23,6 +23,9 @@
>   @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')

Should we remove this line then? ^^^

>   @skipUnless(ssh.SSH_CLIENT_BINARY, 'No SSH client available')
>   class LinuxSSH(QemuSystemTest, LinuxSSHMixIn):
> +    """
> +    :avocado: tags=accel:tcg
> +    """
>   
>       timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
>   

Re: [PATCH 3/9] Avocado migration test: adapt to "utils.network" API namespace change

[Philippe Mathieu-Daudé]

On 25/2/22 22:01, Cleber Rosa wrote:
> Since Avocado 94.0[1], the "avocado.utils.network" dropped a lot of
> previously deprecated API names, having the new names into a finer
> grained structure.
> 
> This simply uses the new API names for the network port utility
> module.
> 
> [1] - https://avocado-framework.readthedocs.io/en/latest/releases/94_0.html#utility-apis
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/avocado/avocado_qemu/__init__.py | 5 +++--
>   tests/avocado/migration.py             | 4 ++--
>   2 files changed, 5 insertions(+), 4 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH 2/9] Avocado tests: use logging namespace that is preserved in test logs

[Philippe Mathieu-Daudé]

On 25/2/22 22:01, Cleber Rosa wrote:
> Since Avocado 92.0[1], there's no universal preservation of logged
> content via Python's "logging" APIs into the test log files.  This
> changes were motivated by the fact that doing so is intrusive as it
> touches on Python's root logger.
> 
> Test writers are now expected to use "avocado." as a namespace prefix
> for everything that Avocado should collect/preserve, and other
> prefixes for logged content that should be handled differently.
> 
> [1] - https://avocado-framework.readthedocs.io/en/94.0/releases/92_0.html#users-test-writers
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/avocado/avocado_qemu/__init__.py    | 5 ++---
>   tests/avocado/linux_initrd.py             | 3 +--
>   tests/avocado/machine_arm_integratorcp.py | 3 +--
>   tests/avocado/machine_mips_malta.py       | 3 +--
>   tests/avocado/replay_kernel.py            | 5 ++---
>   tests/avocado/replay_linux.py             | 5 ++---
>   tests/avocado/reverse_debugging.py        | 5 ++---
>   tests/avocado/tesseract_utils.py          | 6 +++---
>   tests/avocado/virtio_check_params.py      | 3 +--
>   9 files changed, 15 insertions(+), 23 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH 4/9] Avocado: bump to version 95.0

[Thomas Huth]

On 25/02/2022 22.01, Cleber Rosa wrote:
> Even though there have been a number of improvements (and some pretty
> deep internal changes) since Avocado 88.1, only one change should
> affect "make check-avocado".
> 
> With the nrunner architecture, test execution happens in parallel by
> default.  But, tests may fail due to insufficient timeouts or similar
> reasons when run under systems with limited or shared resources.  To
> avoid breakages, especially on CI, let's keep the serial execution
> until proven that it won't impact the CI jobs.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/Makefile.include | 1 +
>   tests/requirements.txt | 2 +-
>   2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index e7153c8e91..676aa0d944 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -135,6 +135,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
>               $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
>   			--filter-by-tags-include-empty-key) \
>               $(AVOCADO_CMDLINE_TAGS) \
> +            --nrunner-max-parallel-tasks=1 \

Could you maybe change it so that it uses the "-jX" setting from "make" 
instead? ... that way the users could decide whether they want to run the 
tests in parallel or not.

I think you could get that value from the MAKEFLAGS env variable, see 
tests/check-block.sh :

JOBS=$(echo "$MAKEFLAGS" | sed -n 's/\(^\|.* \)-j\([0-9]\+\)\( .*\|$\)/-j \2/p')


  Thomas

Re: [PATCH 8/9] Avocado tests: classify tests based on what it's booted

[Pavel Dovgalyuk]

Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>

On 26.02.2022 00:01, Cleber Rosa wrote:
> This adds some classification to the existing tests, based on the
> mechanism (and a lot more loosely) on the content of the binary blob.
> 
> The proposal is to use the "boots" tag, and so far the following
> values have been defined with the following meaning:
> 
>   - bios:   the "-bios" option is used to select the BIOS file to be
>             loaded.  Because default bios are used in many QEMU runs,
>             only tests that change the default are tagged.
>   - kernel: means that the direct kernel boot mechanism (-kernel) is
>             used.  Most of the time it means that a Linux kernel is
>             booted, although there are occurrences of uboot usage.
>   - initrd: means that an initial ram disk (-initrd) is used in
>             addition to the kernel boot.
>   - rootfs: means that a root filesystem is booted, in addition to a
>             kernel and optionally an initrd.  This is usually done with
>             a "-drive" command line option.
>   - distro: means that a full blown distro image is booted, which may
>             or may not include a kernel and initrd.  This is also
>             usually done with a "-drive" command line option.
> 
> As with any other Avocado tags, it's possible to use them to select a
> subset of tests.  For instance, if one wants to run tests that boots a
> bios:
> 
>    $ avocado run -t boots:bios tests/avocado/
> 
> If one want to run tests that boots a kernel and an initrd:
> 
>    $ avocado run -t boots:kernel,boots:initrd tests/avocado/
> 
> It's possible, if deemed valuable, to further evolve this
> classification into one with a clear separation between mechanism and
> content.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/avocado/boot_linux.py               |  4 ++
>   tests/avocado/boot_linux_console.py       | 54 +++++++++++++++++++++++
>   tests/avocado/boot_xen.py                 |  3 ++
>   tests/avocado/hotplug_cpu.py              |  1 +
>   tests/avocado/intel_iommu.py              |  1 +
>   tests/avocado/linux_initrd.py             |  2 +
>   tests/avocado/linux_ssh_mips_malta.py     |  2 +
>   tests/avocado/machine_arm_canona1100.py   |  1 +
>   tests/avocado/machine_arm_integratorcp.py |  4 ++
>   tests/avocado/machine_arm_n8x0.py         |  2 +
>   tests/avocado/machine_avr6.py             |  1 +
>   tests/avocado/machine_m68k_nextcube.py    |  1 +
>   tests/avocado/machine_microblaze.py       |  1 +
>   tests/avocado/machine_mips_fuloong2e.py   |  1 +
>   tests/avocado/machine_mips_loongson3v.py  |  1 +
>   tests/avocado/machine_mips_malta.py       |  3 ++
>   tests/avocado/machine_rx_gdbsim.py        |  2 +
>   tests/avocado/machine_s390_ccw_virtio.py  |  4 ++
>   tests/avocado/machine_sparc64_sun4u.py    |  1 +
>   tests/avocado/machine_sparc_leon3.py      |  1 +
>   tests/avocado/multiprocess.py             |  4 ++
>   tests/avocado/ppc_405.py                  |  2 +
>   tests/avocado/ppc_bamboo.py               |  2 +
>   tests/avocado/ppc_mpc8544ds.py            |  1 +
>   tests/avocado/ppc_prep_40p.py             |  1 +
>   tests/avocado/ppc_pseries.py              |  1 +
>   tests/avocado/ppc_virtex_ml507.py         |  1 +
>   tests/avocado/replay_kernel.py            | 28 ++++++++++++
>   tests/avocado/replay_linux.py             |  1 +
>   tests/avocado/reverse_debugging.py        |  1 +
>   tests/avocado/smmu.py                     |  1 +
>   tests/avocado/tcg_plugins.py              |  3 ++
>   tests/avocado/virtio-gpu.py               |  2 +
>   tests/avocado/virtiofs_submounts.py       |  1 +
>   34 files changed, 139 insertions(+)
> 
> diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
> index ab19146d1e..c4172f11e3 100644
> --- a/tests/avocado/boot_linux.py
> +++ b/tests/avocado/boot_linux.py
> @@ -18,6 +18,7 @@
>   class BootLinuxX8664(LinuxTest):
>       """
>       :avocado: tags=arch:x86_64
> +    :avocado: tags=boots:distro
>       """
>   
>       def test_pc_i440fx_tcg(self):
> @@ -62,6 +63,7 @@ class BootLinuxAarch64(LinuxTest):
>       :avocado: tags=arch:aarch64
>       :avocado: tags=machine:virt
>       :avocado: tags=machine:gic-version=2
> +    :avocado: tags=boots:distro
>       """
>   
>       def add_common_args(self):
> @@ -110,6 +112,7 @@ def test_virt_kvm(self):
>   class BootLinuxPPC64(LinuxTest):
>       """
>       :avocado: tags=arch:ppc64
> +    :avocado: tags=boots:distro
>       """
>   
>       def test_pseries_tcg(self):
> @@ -125,6 +128,7 @@ def test_pseries_tcg(self):
>   class BootLinuxS390X(LinuxTest):
>       """
>       :avocado: tags=arch:s390x
> +    :avocado: tags=boots:distro
>       """
>   
>       @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
> diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
> index 9c618d4809..0a8980953f 100644
> --- a/tests/avocado/boot_linux_console.py
> +++ b/tests/avocado/boot_linux_console.py
> @@ -95,6 +95,7 @@ def test_x86_64_pc(self):
>           """
>           :avocado: tags=arch:x86_64
>           :avocado: tags=machine:pc
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> @@ -115,6 +116,7 @@ def test_mips_malta(self):
>           :avocado: tags=arch:mips
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:big
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20130217T032700Z/pool/main/l/linux-2.6/'
> @@ -147,6 +149,7 @@ def test_mips64el_malta(self):
>   
>           :avocado: tags=arch:mips64el
>           :avocado: tags=machine:malta
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20130217T032700Z/pool/main/l/linux-2.6/'
> @@ -169,6 +172,7 @@ def test_mips64el_fuloong2e(self):
>           :avocado: tags=arch:mips64el
>           :avocado: tags=machine:fuloong2e
>           :avocado: tags=endian:little
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('http://archive.debian.org/debian/pool/main/l/linux/'
>                      'linux-image-3.16.0-6-loongson-2e_3.16.56-1+deb8u1_mipsel.deb')
> @@ -190,6 +194,8 @@ def test_mips_malta_cpio(self):
>           :avocado: tags=arch:mips
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:big
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20160601T041800Z/pool/main/l/linux/'
> @@ -233,6 +239,8 @@ def test_mips64el_malta_5KEc_cpio(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:5KEc
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
>                         'raw/9ad2df38/mips/malta/mips64el/'
> @@ -292,6 +300,7 @@ def test_mips_malta32el_nanomips_4k(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:I7200
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>                         'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
> @@ -305,6 +314,7 @@ def test_mips_malta32el_nanomips_16k_up(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:I7200
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>                         'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
> @@ -318,6 +328,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:I7200
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>                         'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
> @@ -331,6 +342,7 @@ def test_aarch64_virt(self):
>           :avocado: tags=machine:virt
>           :avocado: tags=accel:tcg
>           :avocado: tags=cpu:cortex-a53
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
> @@ -357,6 +369,8 @@ def test_aarch64_xlnx_versal_virt(self):
>           :avocado: tags=device:pl011
>           :avocado: tags=device:arm_gicv3
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           images_url = ('http://ports.ubuntu.com/ubuntu-ports/dists/'
>                         'bionic-updates/main/installer-arm64/'
> @@ -382,6 +396,7 @@ def test_arm_virt(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:virt
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
> @@ -405,6 +420,8 @@ def test_arm_emcraft_sf2(self):
>           :avocado: tags=endian:little
>           :avocado: tags=u-boot
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:rootfs
>           """
>           uboot_url = ('https://raw.githubusercontent.com/'
>                        'Subbaraya-Sundeep/qemu-test-binaries/'
> @@ -471,6 +488,7 @@ def test_arm_raspi2_uart0(self):
>           :avocado: tags=machine:raspi2b
>           :avocado: tags=device:pl011
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
>           """
>           self.do_test_arm_raspi2(0)
>   
> @@ -478,6 +496,8 @@ def test_arm_raspi2_initrd(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:raspi2b
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('http://archive.raspberrypi.org/debian/'
>                      'pool/main/r/raspberrypi-firmware/'
> @@ -521,6 +541,8 @@ def test_arm_exynos4210_initrd(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:smdkc210
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('https://snapshot.debian.org/archive/debian/'
>                      '20190928T224601Z/pool/main/l/linux/'
> @@ -562,6 +584,8 @@ def test_arm_cubieboard_initrd(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:cubieboard
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('https://apt.armbian.com/pool/main/l/'
>                      'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
> @@ -603,6 +627,8 @@ def test_arm_cubieboard_sata(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:cubieboard
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:rootfs
>           """
>           deb_url = ('https://apt.armbian.com/pool/main/l/'
>                      'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
> @@ -648,6 +674,7 @@ def test_arm_quanta_gsj(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:quanta-gsj
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:rootfs
>           """
>           # 25 MiB compressed, 32 MiB uncompressed.
>           image_url = (
> @@ -696,6 +723,8 @@ def test_arm_quanta_gsj_initrd(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:quanta-gsj
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           initrd_url = (
>                   'https://github.com/hskinnemoen/openbmc/releases/download/'
> @@ -733,6 +762,7 @@ def test_arm_orangepi(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:orangepi-pc
>           :avocado: tags=accel:tcg
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('https://apt.armbian.com/pool/main/l/'
>                      'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
> @@ -759,6 +789,8 @@ def test_arm_orangepi_initrd(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=accel:tcg
>           :avocado: tags=machine:orangepi-pc
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('https://apt.armbian.com/pool/main/l/'
>                      'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
> @@ -803,6 +835,8 @@ def test_arm_orangepi_sd(self):
>           :avocado: tags=accel:tcg
>           :avocado: tags=machine:orangepi-pc
>           :avocado: tags=device:sd
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:rootfs
>           """
>           deb_url = ('https://apt.armbian.com/pool/main/l/'
>                      'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
> @@ -855,6 +889,7 @@ def test_arm_orangepi_bionic_20_08(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:orangepi-pc
>           :avocado: tags=device:sd
> +        :avocado: tags=boots:distro
>           """
>   
>           # This test download a 275 MiB compressed image and expand it
> @@ -902,6 +937,7 @@ def test_arm_orangepi_uboot_netbsd9(self):
>           :avocado: tags=machine:orangepi-pc
>           :avocado: tags=device:sd
>           :avocado: tags=os:netbsd
> +        :avocado: tags=boots:distro
>           """
>           # This test download a 304MB compressed image and expand it to 2GB
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
> @@ -987,6 +1023,7 @@ def test_s390x_s390_ccw_virtio(self):
>           """
>           :avocado: tags=arch:s390x
>           :avocado: tags=machine:s390-ccw-virtio
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive'
>                         '/fedora-secondary/releases/29/Everything/s390x/os/images'
> @@ -1007,6 +1044,7 @@ def test_alpha_clipper(self):
>           """
>           :avocado: tags=arch:alpha
>           :avocado: tags=machine:clipper
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
>                         'installer-alpha/20090123lenny10/images/cdrom/vmlinuz')
> @@ -1028,6 +1066,7 @@ def test_m68k_q800(self):
>           """
>           :avocado: tags=arch:m68k
>           :avocado: tags=machine:q800
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('https://snapshot.debian.org/archive/debian-ports'
>                      '/20191021T083923Z/pool-m68k/main'
> @@ -1063,6 +1102,7 @@ def test_arm_vexpressa9(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:vexpress-a9
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '32b7677ce8b6f1471fb0059865f451169934245b'
>           self.vm.add_args('-dtb', self.workdir + '/day16/vexpress-v2p-ca9.dtb')
> @@ -1072,6 +1112,7 @@ def test_arm_ast2400_palmetto_openbmc_v2_9_0(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:palmetto-bmc
> +        :avocado: tags=boots:rootfs
>           """
>   
>           image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
> @@ -1086,6 +1127,7 @@ def test_arm_ast2500_romulus_openbmc_v2_9_0(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:romulus-bmc
> +        :avocado: tags=boots:rootfs
>           """
>   
>           image_url = ('https://github.com/openbmc/openbmc/releases/download/2.9.0/'
> @@ -1115,6 +1157,7 @@ def test_arm_ast2600_debian(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:tacoma-bmc
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20210302T203551Z/'
> @@ -1140,6 +1183,7 @@ def test_m68k_mcf5208evb(self):
>           """
>           :avocado: tags=arch:m68k
>           :avocado: tags=machine:mcf5208evb
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>           self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
> @@ -1148,6 +1192,7 @@ def test_or1k_sim(self):
>           """
>           :avocado: tags=arch:or1k
>           :avocado: tags=machine:or1k-sim
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '20334cdaf386108c530ff0badaecc955693027dd'
>           self.do_test_advcal_2018('20', tar_hash, 'vmlinux')
> @@ -1156,6 +1201,7 @@ def test_nios2_10m50(self):
>           """
>           :avocado: tags=arch:nios2
>           :avocado: tags=machine:10m50-ghrd
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'e4251141726c412ac0407c5a6bceefbbff018918'
>           self.do_test_advcal_2018('14', tar_hash, 'vmlinux.elf')
> @@ -1165,6 +1211,7 @@ def test_ppc64_e500(self):
>           :avocado: tags=arch:ppc64
>           :avocado: tags=machine:ppce500
>           :avocado: tags=cpu:e5500
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
>           self.do_test_advcal_2018('19', tar_hash, 'uImage')
> @@ -1194,6 +1241,7 @@ def test_ppc_powernv8(self):
>           """
>           :avocado: tags=arch:ppc64
>           :avocado: tags=machine:powernv8
> +        :avocado: tags=boots:kernel
>           """
>           self.do_test_ppc64_powernv('P8')
>   
> @@ -1201,6 +1249,7 @@ def test_ppc_powernv9(self):
>           """
>           :avocado: tags=arch:ppc64
>           :avocado: tags=machine:powernv9
> +        :avocado: tags=boots:kernel
>           """
>           self.do_test_ppc64_powernv('P9')
>   
> @@ -1208,6 +1257,7 @@ def test_ppc_g3beige(self):
>           """
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:g3beige
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
>           self.vm.add_args('-M', 'graphics=off')
> @@ -1217,6 +1267,7 @@ def test_ppc_mac99(self):
>           """
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:mac99
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
>           self.vm.add_args('-M', 'graphics=off')
> @@ -1226,6 +1277,7 @@ def test_sh4_r2d(self):
>           """
>           :avocado: tags=arch:sh4
>           :avocado: tags=machine:r2d
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'fe06a4fd8ccbf2e27928d64472939d47829d4c7e'
>           self.vm.add_args('-append', 'console=ttySC1')
> @@ -1235,6 +1287,7 @@ def test_sparc_ss20(self):
>           """
>           :avocado: tags=arch:sparc
>           :avocado: tags=machine:SS-20
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'b18550d5d61c7615d989a06edace051017726a9f'
>           self.do_test_advcal_2018('11', tar_hash, 'zImage.elf')
> @@ -1244,6 +1297,7 @@ def test_xtensa_lx60(self):
>           :avocado: tags=arch:xtensa
>           :avocado: tags=machine:lx60
>           :avocado: tags=cpu:dc233c
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '49e88d9933742f0164b60839886c9739cb7a0d34'
>           self.do_test_advcal_2018('02', tar_hash, 'santas-sleigh-ride.elf')
> diff --git a/tests/avocado/boot_xen.py b/tests/avocado/boot_xen.py
> index fc2faeedb5..6cc5216a57 100644
> --- a/tests/avocado/boot_xen.py
> +++ b/tests/avocado/boot_xen.py
> @@ -61,6 +61,9 @@ def launch_xen(self, xen_path):
>   
>   
>   class BootXen(BootXenBase):
> +    """
> +    :avocado: tags=boots:kernel
> +    """
>   
>       def test_arm64_xen_411_and_dom0(self):
>           """
> diff --git a/tests/avocado/hotplug_cpu.py b/tests/avocado/hotplug_cpu.py
> index 6374bf1b54..bb296020af 100644
> --- a/tests/avocado/hotplug_cpu.py
> +++ b/tests/avocado/hotplug_cpu.py
> @@ -18,6 +18,7 @@ def test(self):
>           :avocado: tags=arch:x86_64
>           :avocado: tags=machine:q35
>           :avocado: tags=accel:kvm
> +        :avocado: tags=boots:distro
>           """
>           self.require_accelerator('kvm')
>           self.vm.add_args('-accel', 'kvm')
> diff --git a/tests/avocado/intel_iommu.py b/tests/avocado/intel_iommu.py
> index 474d62f6bf..d308dcccc5 100644
> --- a/tests/avocado/intel_iommu.py
> +++ b/tests/avocado/intel_iommu.py
> @@ -20,6 +20,7 @@ class IntelIOMMU(LinuxTest):
>       :avocado: tags=distro_version:31
>       :avocado: tags=machine:q35
>       :avocado: tags=accel:kvm
> +    :avocado: tags=boots:distro
>       :avocado: tags=intel_iommu
>       """
>   
> diff --git a/tests/avocado/linux_initrd.py b/tests/avocado/linux_initrd.py
> index 6ebf299cd4..07582f5247 100644
> --- a/tests/avocado/linux_initrd.py
> +++ b/tests/avocado/linux_initrd.py
> @@ -21,6 +21,8 @@ class LinuxInitrd(QemuSystemTest):
>   
>       :avocado: tags=arch:x86_64
>       :avocado: tags=machine:pc
> +    :avocado: tags=boots:kernel
> +    :avocado: tags=boots:initrd
>       """
>   
>       timeout = 300
> diff --git a/tests/avocado/linux_ssh_mips_malta.py b/tests/avocado/linux_ssh_mips_malta.py
> index 0179d8a6ca..2cd986c6b2 100644
> --- a/tests/avocado/linux_ssh_mips_malta.py
> +++ b/tests/avocado/linux_ssh_mips_malta.py
> @@ -25,6 +25,8 @@
>   class LinuxSSH(QemuSystemTest, LinuxSSHMixIn):
>       """
>       :avocado: tags=accel:tcg
> +    :avocado: tags=boots:kernel
> +    :avocado: tags=boots:distro
>       """
>   
>       timeout = 150 # Not for 'configure --enable-debug --enable-debug-tcg'
> diff --git a/tests/avocado/machine_arm_canona1100.py b/tests/avocado/machine_arm_canona1100.py
> index 182a0b0513..0cfb9b873d 100644
> --- a/tests/avocado/machine_arm_canona1100.py
> +++ b/tests/avocado/machine_arm_canona1100.py
> @@ -22,6 +22,7 @@ def test_arm_canona1100(self):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:canon-a1100
>           :avocado: tags=device:pflash_cfi02
> +        :avocado: tags=boots:bios
>           """
>           tar_url = ('https://www.qemu-advent-calendar.org'
>                      '/2018/download/day18.tar.xz')
> diff --git a/tests/avocado/machine_arm_integratorcp.py b/tests/avocado/machine_arm_integratorcp.py
> index 697ee76f6c..3049aff037 100644
> --- a/tests/avocado/machine_arm_integratorcp.py
> +++ b/tests/avocado/machine_arm_integratorcp.py
> @@ -29,6 +29,10 @@
>   
>   
>   class IntegratorMachine(QemuSystemTest):
> +    """
> +    :avocado: tags=boots:kernel
> +    :avocado: tags=boots:initrd
> +    """
>   
>       timeout = 90
>   
> diff --git a/tests/avocado/machine_arm_n8x0.py b/tests/avocado/machine_arm_n8x0.py
> index 12e9a6803b..78c65863b6 100644
> --- a/tests/avocado/machine_arm_n8x0.py
> +++ b/tests/avocado/machine_arm_n8x0.py
> @@ -37,6 +37,7 @@ def test_n800(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:n800
> +        :avocado: tags=boots:kernel
>           """
>           self.__do_test_n8x0()
>   
> @@ -45,5 +46,6 @@ def test_n810(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:n810
> +        :avocado: tags=boots:kernel
>           """
>           self.__do_test_n8x0()
> diff --git a/tests/avocado/machine_avr6.py b/tests/avocado/machine_avr6.py
> index 5485db79c6..0837d4dd6b 100644
> --- a/tests/avocado/machine_avr6.py
> +++ b/tests/avocado/machine_avr6.py
> @@ -28,6 +28,7 @@ def test_freertos(self):
>           """
>           :avocado: tags=arch:avr
>           :avocado: tags=machine:arduino-mega-2560-v3
> +        :avocado: tags=boots:bios
>           """
>           """
>           https://github.com/seharris/qemu-avr-tests/raw/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf
> diff --git a/tests/avocado/machine_m68k_nextcube.py b/tests/avocado/machine_m68k_nextcube.py
> index 6790e7d9cd..26df7f89c4 100644
> --- a/tests/avocado/machine_m68k_nextcube.py
> +++ b/tests/avocado/machine_m68k_nextcube.py
> @@ -25,6 +25,7 @@ class NextCubeMachine(QemuSystemTest):
>       :avocado: tags=arch:m68k
>       :avocado: tags=machine:next-cube
>       :avocado: tags=device:framebuffer
> +    :avocado: tags=boots:bios
>       """
>   
>       timeout = 15
> diff --git a/tests/avocado/machine_microblaze.py b/tests/avocado/machine_microblaze.py
> index 4928920f96..9035c2ef3d 100644
> --- a/tests/avocado/machine_microblaze.py
> +++ b/tests/avocado/machine_microblaze.py
> @@ -17,6 +17,7 @@ def test_microblaze_s3adsp1800(self):
>           """
>           :avocado: tags=arch:microblaze
>           :avocado: tags=machine:petalogix-s3adsp1800
> +        :avocado: tags=boots:kernel
>           """
>   
>           tar_url = ('https://www.qemu-advent-calendar.org'
> diff --git a/tests/avocado/machine_mips_fuloong2e.py b/tests/avocado/machine_mips_fuloong2e.py
> index 89291f47b2..f1da0bbb75 100644
> --- a/tests/avocado/machine_mips_fuloong2e.py
> +++ b/tests/avocado/machine_mips_fuloong2e.py
> @@ -26,6 +26,7 @@ def test_linux_kernel_isa_serial(self):
>           :avocado: tags=endian:little
>           :avocado: tags=device:bonito64
>           :avocado: tags=device:via686b
> +        :avocado: tags=boots:kernel
>           """
>           # Recovery system for the Yeeloong laptop
>           # (enough to test the fuloong2e southbridge, accessing its ISA bus)
> diff --git a/tests/avocado/machine_mips_loongson3v.py b/tests/avocado/machine_mips_loongson3v.py
> index 5194cf18c9..5680fc6892 100644
> --- a/tests/avocado/machine_mips_loongson3v.py
> +++ b/tests/avocado/machine_mips_loongson3v.py
> @@ -26,6 +26,7 @@ def test_pmon_serial_console(self):
>           :avocado: tags=cpu:Loongson-3A1000
>           :avocado: tags=device:liointc
>           :avocado: tags=device:goldfish_rtc
> +        :avocado: tags=boots:bios
>           """
>   
>           pmon_hash = '7c8b45dd81ccfc55ff28f5aa267a41c3'
> diff --git a/tests/avocado/machine_mips_malta.py b/tests/avocado/machine_mips_malta.py
> index 5f98ba1620..e0b015b871 100644
> --- a/tests/avocado/machine_mips_malta.py
> +++ b/tests/avocado/machine_mips_malta.py
> @@ -95,6 +95,7 @@ def test_mips_malta_i6400_framebuffer_logo_1core(self):
>           :avocado: tags=arch:mips64el
>           :avocado: tags=machine:malta
>           :avocado: tags=cpu:I6400
> +        :avocado: tags=boots:kernel
>           """
>           self.do_test_i6400_framebuffer_logo(1)
>   
> @@ -105,6 +106,7 @@ def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=cpu:I6400
>           :avocado: tags=mips:smp
> +        :avocado: tags=boots:kernel
>           """
>           self.do_test_i6400_framebuffer_logo(7)
>   
> @@ -115,5 +117,6 @@ def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=cpu:I6400
>           :avocado: tags=mips:smp
> +        :avocado: tags=boots:kernel
>           """
>           self.do_test_i6400_framebuffer_logo(8)
> diff --git a/tests/avocado/machine_rx_gdbsim.py b/tests/avocado/machine_rx_gdbsim.py
> index 6cd8704b01..bb3263d982 100644
> --- a/tests/avocado/machine_rx_gdbsim.py
> +++ b/tests/avocado/machine_rx_gdbsim.py
> @@ -30,6 +30,7 @@ def test_uboot(self):
>           :avocado: tags=arch:rx
>           :avocado: tags=machine:gdbsim-r5f562n8
>           :avocado: tags=endian:little
> +        :avocado: tags=boots:bios
>           """
>           uboot_url = ('https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.gz')
>           uboot_hash = '9b78dbd43b40b2526848c0b1ce9de02c24f4dcdb'
> @@ -54,6 +55,7 @@ def test_linux_sash(self):
>           :avocado: tags=arch:rx
>           :avocado: tags=machine:gdbsim-r5f562n7
>           :avocado: tags=endian:little
> +        :avocado: tags=boots:kernel
>           """
>           dtb_url = ('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb')
>           dtb_hash = '7b4e4e2c71905da44e86ce47adee2210b026ac18'
> diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/avocado/machine_s390_ccw_virtio.py
> index bd03d7160b..0e24372240 100644
> --- a/tests/avocado/machine_s390_ccw_virtio.py
> +++ b/tests/avocado/machine_s390_ccw_virtio.py
> @@ -45,6 +45,8 @@ def test_s390x_devices(self):
>           """
>           :avocado: tags=arch:s390x
>           :avocado: tags=machine:s390-ccw-virtio
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>   
>           kernel_url = ('https://snapshot.debian.org/archive/debian/'
> @@ -167,6 +169,8 @@ def test_s390x_fedora(self):
>           :avocado: tags=device:virtio-gpu
>           :avocado: tags=device:virtio-crypto
>           :avocado: tags=device:virtio-net
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>   
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive'
> diff --git a/tests/avocado/machine_sparc64_sun4u.py b/tests/avocado/machine_sparc64_sun4u.py
> index 458165500e..18fcb92fcd 100644
> --- a/tests/avocado/machine_sparc64_sun4u.py
> +++ b/tests/avocado/machine_sparc64_sun4u.py
> @@ -23,6 +23,7 @@ def test_sparc64_sun4u(self):
>           """
>           :avocado: tags=arch:sparc64
>           :avocado: tags=machine:sun4u
> +        :avocado: tags=boots:kernel
>           """
>           tar_url = ('https://www.qemu-advent-calendar.org'
>                      '/2018/download/day23.tar.xz')
> diff --git a/tests/avocado/machine_sparc_leon3.py b/tests/avocado/machine_sparc_leon3.py
> index e61b223185..c7f941a525 100644
> --- a/tests/avocado/machine_sparc_leon3.py
> +++ b/tests/avocado/machine_sparc_leon3.py
> @@ -22,6 +22,7 @@ def test_leon3_helenos_uimage(self):
>           :avocado: tags=arch:sparc
>           :avocado: tags=machine:leon3_generic
>           :avocado: tags=binfmt:uimage
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('http://www.helenos.org/releases/'
>                         'HelenOS-0.6.0-sparc32-leon3.bin')
> diff --git a/tests/avocado/multiprocess.py b/tests/avocado/multiprocess.py
> index 80a3b8f442..9415e6d00c 100644
> --- a/tests/avocado/multiprocess.py
> +++ b/tests/avocado/multiprocess.py
> @@ -67,6 +67,8 @@ def do_test(self, kernel_url, initrd_url, kernel_command_line,
>       def test_multiprocess_x86_64(self):
>           """
>           :avocado: tags=arch:x86_64
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/31/Everything/x86_64/os/images'
> @@ -82,6 +84,8 @@ def test_multiprocess_x86_64(self):
>       def test_multiprocess_aarch64(self):
>           """
>           :avocado: tags=arch:aarch64
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/31/Everything/aarch64/os/images'
> diff --git a/tests/avocado/ppc_405.py b/tests/avocado/ppc_405.py
> index a47f89b934..f767ca17c7 100644
> --- a/tests/avocado/ppc_405.py
> +++ b/tests/avocado/ppc_405.py
> @@ -30,6 +30,7 @@ def test_ppc_taihu(self):
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:taihu
>           :avocado: tags=cpu:405ep
> +        :avocado: tags=boots:bios
>           """
>           self.do_test_ppc405()
>   
> @@ -38,5 +39,6 @@ def test_ppc_ref405ep(self):
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:ref405ep
>           :avocado: tags=cpu:405ep
> +        :avocado: tags=boots:bios
>           """
>           self.do_test_ppc405()
> diff --git a/tests/avocado/ppc_bamboo.py b/tests/avocado/ppc_bamboo.py
> index 40629e3478..7568ab2b6f 100644
> --- a/tests/avocado/ppc_bamboo.py
> +++ b/tests/avocado/ppc_bamboo.py
> @@ -20,6 +20,8 @@ def test_ppc_bamboo(self):
>           :avocado: tags=machine:bamboo
>           :avocado: tags=cpu:440epb
>           :avocado: tags=device:rtl8139
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           tar_url = ('http://landley.net/aboriginal/downloads/binaries/'
>                      'system-image-powerpc-440fp.tar.gz')
> diff --git a/tests/avocado/ppc_mpc8544ds.py b/tests/avocado/ppc_mpc8544ds.py
> index 886f967b15..12efb52ea7 100644
> --- a/tests/avocado/ppc_mpc8544ds.py
> +++ b/tests/avocado/ppc_mpc8544ds.py
> @@ -19,6 +19,7 @@ def test_ppc_mpc8544ds(self):
>           """
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:mpc8544ds
> +        :avocado: tags=boots:kernel
>           """
>           tar_url = ('https://www.qemu-advent-calendar.org'
>                      '/2020/download/day17.tar.gz')
> diff --git a/tests/avocado/ppc_prep_40p.py b/tests/avocado/ppc_prep_40p.py
> index 4bd956584d..fde18ac42c 100644
> --- a/tests/avocado/ppc_prep_40p.py
> +++ b/tests/avocado/ppc_prep_40p.py
> @@ -28,6 +28,7 @@ def test_factory_firmware_and_netbsd(self):
>           :avocado: tags=machine:40p
>           :avocado: tags=os:netbsd
>           :avocado: tags=slowness:high
> +        :avocado: tags=boots:bios
>           """
>           bios_url = ('http://ftpmirror.your.org/pub/misc/'
>                       'ftp.software.ibm.com/rs6000/firmware/'
> diff --git a/tests/avocado/ppc_pseries.py b/tests/avocado/ppc_pseries.py
> index d8b04dc3ea..388dfe5487 100644
> --- a/tests/avocado/ppc_pseries.py
> +++ b/tests/avocado/ppc_pseries.py
> @@ -19,6 +19,7 @@ def test_ppc64_pseries(self):
>           """
>           :avocado: tags=arch:ppc64
>           :avocado: tags=machine:pseries
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive'
>                         '/fedora-secondary/releases/29/Everything/ppc64le/os'
> diff --git a/tests/avocado/ppc_virtex_ml507.py b/tests/avocado/ppc_virtex_ml507.py
> index a6912ee579..d9afe2cb83 100644
> --- a/tests/avocado/ppc_virtex_ml507.py
> +++ b/tests/avocado/ppc_virtex_ml507.py
> @@ -19,6 +19,7 @@ def test_ppc_virtex_ml507(self):
>           """
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:virtex-ml507
> +        :avocado: tags=boots:kernel
>           """
>           tar_url = ('https://www.qemu-advent-calendar.org'
>                      '/2020/download/hippo.tar.gz')
> diff --git a/tests/avocado/replay_kernel.py b/tests/avocado/replay_kernel.py
> index 40f52b3913..368c6b41c8 100644
> --- a/tests/avocado/replay_kernel.py
> +++ b/tests/avocado/replay_kernel.py
> @@ -82,6 +82,7 @@ def test_x86_64_pc(self):
>           """
>           :avocado: tags=arch:x86_64
>           :avocado: tags=machine:pc
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> @@ -99,6 +100,7 @@ def test_mips_malta(self):
>           :avocado: tags=arch:mips
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:big
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20130217T032700Z/pool/main/l/linux-2.6/'
> @@ -127,6 +129,7 @@ def test_mips64el_malta(self):
>   
>           :avocado: tags=arch:mips64el
>           :avocado: tags=machine:malta
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20130217T032700Z/pool/main/l/linux-2.6/'
> @@ -144,6 +147,7 @@ def test_aarch64_virt(self):
>           :avocado: tags=arch:aarch64
>           :avocado: tags=machine:virt
>           :avocado: tags=cpu:cortex-a53
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
> @@ -161,6 +165,7 @@ def test_arm_virt(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:virt
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
> @@ -179,6 +184,8 @@ def test_arm_cubieboard_initrd(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:cubieboard
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('https://apt.armbian.com/pool/main/l/'
>                      'linux-5.10.16-sunxi/linux-image-current-sunxi_21.02.2_armhf.deb')
> @@ -210,6 +217,7 @@ def test_s390x_s390_ccw_virtio(self):
>           """
>           :avocado: tags=arch:s390x
>           :avocado: tags=machine:s390-ccw-virtio
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive'
>                         '/fedora-secondary/releases/29/Everything/s390x/os/images'
> @@ -225,6 +233,7 @@ def test_alpha_clipper(self):
>           """
>           :avocado: tags=arch:alpha
>           :avocado: tags=machine:clipper
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('http://archive.debian.org/debian/dists/lenny/main/'
>                         'installer-alpha/20090123lenny10/images/cdrom/vmlinuz')
> @@ -242,6 +251,7 @@ def test_ppc64_pseries(self):
>           """
>           :avocado: tags=arch:ppc64
>           :avocado: tags=machine:pseries
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive'
>                         '/fedora-secondary/releases/29/Everything/ppc64le/os'
> @@ -258,6 +268,7 @@ def test_m68k_q800(self):
>           """
>           :avocado: tags=arch:m68k
>           :avocado: tags=machine:q800
> +        :avocado: tags=boots:kernel
>           """
>           deb_url = ('https://snapshot.debian.org/archive/debian-ports'
>                      '/20191021T083923Z/pool-m68k/main'
> @@ -289,6 +300,7 @@ def test_arm_vexpressa9(self):
>           """
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:vexpress-a9
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '32b7677ce8b6f1471fb0059865f451169934245b'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -302,6 +314,7 @@ def test_m68k_mcf5208evb(self):
>           """
>           :avocado: tags=arch:m68k
>           :avocado: tags=machine:mcf5208evb
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -314,6 +327,7 @@ def test_microblaze_s3adsp1800(self):
>           """
>           :avocado: tags=arch:microblaze
>           :avocado: tags=machine:petalogix-s3adsp1800
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -326,6 +340,7 @@ def test_ppc64_e500(self):
>           :avocado: tags=arch:ppc64
>           :avocado: tags=machine:ppce500
>           :avocado: tags=cpu:e5500
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -337,6 +352,7 @@ def test_or1k_sim(self):
>           """
>           :avocado: tags=arch:or1k
>           :avocado: tags=machine:or1k-sim
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '20334cdaf386108c530ff0badaecc955693027dd'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -348,6 +364,7 @@ def test_nios2_10m50(self):
>           """
>           :avocado: tags=arch:nios2
>           :avocado: tags=machine:10m50-ghrd
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'e4251141726c412ac0407c5a6bceefbbff018918'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -359,6 +376,7 @@ def test_ppc_g3beige(self):
>           """
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:g3beige
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -371,6 +389,7 @@ def test_ppc_mac99(self):
>           """
>           :avocado: tags=arch:ppc
>           :avocado: tags=machine:mac99
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'e0b872a5eb8fdc5bed19bd43ffe863900ebcedfc'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -383,6 +402,7 @@ def test_sparc_ss20(self):
>           """
>           :avocado: tags=arch:sparc
>           :avocado: tags=machine:SS-20
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = 'b18550d5d61c7615d989a06edace051017726a9f'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -395,6 +415,7 @@ def test_xtensa_lx60(self):
>           :avocado: tags=arch:xtensa
>           :avocado: tags=machine:lx60
>           :avocado: tags=cpu:dc233c
> +        :avocado: tags=boots:kernel
>           """
>           tar_hash = '49e88d9933742f0164b60839886c9739cb7a0d34'
>           tar_url = ('https://www.qemu-advent-calendar.org'
> @@ -415,6 +436,8 @@ def test_mips_malta_cpio(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:big
>           :avocado: tags=slowness:high
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           deb_url = ('http://snapshot.debian.org/archive/debian/'
>                      '20160601T041800Z/pool/main/l/linux/'
> @@ -446,6 +469,8 @@ def test_mips64el_malta_5KEc_cpio(self):
>           :avocado: tags=endian:little
>           :avocado: tags=slowness:high
>           :avocado: tags=cpu:5KEc
> +        :avocado: tags=boots:kernel
> +        :avocado: tags=boots:initrd
>           """
>           kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
>                         'raw/9ad2df38/mips/malta/mips64el/'
> @@ -486,6 +511,7 @@ def test_mips_malta32el_nanomips_4k(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:I7200
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>                         'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
> @@ -500,6 +526,7 @@ def test_mips_malta32el_nanomips_16k_up(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:I7200
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>                         'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
> @@ -514,6 +541,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
>           :avocado: tags=machine:malta
>           :avocado: tags=endian:little
>           :avocado: tags=cpu:I7200
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
>                         'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
> diff --git a/tests/avocado/replay_linux.py b/tests/avocado/replay_linux.py
> index b56eeccfdd..b2c02c07f6 100644
> --- a/tests/avocado/replay_linux.py
> +++ b/tests/avocado/replay_linux.py
> @@ -98,6 +98,7 @@ class ReplayLinuxX8664(ReplayLinux):
>       """
>       :avocado: tags=arch:x86_64
>       :avocado: tags=accel:tcg
> +    :avocado: tags=boots:distro
>       """
>   
>       chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
> diff --git a/tests/avocado/reverse_debugging.py b/tests/avocado/reverse_debugging.py
> index 2cb39f3e59..f198173bf7 100644
> --- a/tests/avocado/reverse_debugging.py
> +++ b/tests/avocado/reverse_debugging.py
> @@ -198,6 +198,7 @@ def test_aarch64_virt(self):
>           :avocado: tags=arch:aarch64
>           :avocado: tags=machine:virt
>           :avocado: tags=cpu:cortex-a53
> +        :avocado: tags=boots:kernel
>           """
>           kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                         '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
> diff --git a/tests/avocado/smmu.py b/tests/avocado/smmu.py
> index b3c4de6bf4..dcc7402beb 100644
> --- a/tests/avocado/smmu.py
> +++ b/tests/avocado/smmu.py
> @@ -19,6 +19,7 @@ class SMMU(LinuxTest):
>       :avocado: tags=cpu:host
>       :avocado: tags=arch:aarch64
>       :avocado: tags=machine:virt
> +    :avocado: tags=boots:distro
>       :avocado: tags=distro:fedora
>       :avocado: tags=smmu
>       """
> diff --git a/tests/avocado/tcg_plugins.py b/tests/avocado/tcg_plugins.py
> index 642d2e49e3..296e663152 100644
> --- a/tests/avocado/tcg_plugins.py
> +++ b/tests/avocado/tcg_plugins.py
> @@ -69,6 +69,7 @@ def test_aarch64_virt_insn(self):
>           :avocado: tags=arch:aarch64
>           :avocado: tags=machine:virt
>           :avocado: tags=cpu:cortex-a53
> +        :avocado: tags=boots:kernel
>           """
>           kernel_path = self._grab_aarch64_kernel()
>           kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> @@ -95,6 +96,7 @@ def test_aarch64_virt_insn_icount(self):
>           :avocado: tags=arch:aarch64
>           :avocado: tags=machine:virt
>           :avocado: tags=cpu:cortex-a53
> +        :avocado: tags=boots:kernel
>           """
>           kernel_path = self._grab_aarch64_kernel()
>           kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> @@ -121,6 +123,7 @@ def test_aarch64_virt_mem_icount(self):
>           :avocado: tags=arch:aarch64
>           :avocado: tags=machine:virt
>           :avocado: tags=cpu:cortex-a53
> +        :avocado: tags=boots:kernel
>           """
>           kernel_path = self._grab_aarch64_kernel()
>           kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> diff --git a/tests/avocado/virtio-gpu.py b/tests/avocado/virtio-gpu.py
> index 2a249a3a2c..e91efac8ea 100644
> --- a/tests/avocado/virtio-gpu.py
> +++ b/tests/avocado/virtio-gpu.py
> @@ -32,6 +32,8 @@ class VirtioGPUx86(QemuSystemTest):
>       :avocado: tags=virtio-gpu
>       :avocado: tags=arch:x86_64
>       :avocado: tags=cpu:host
> +    :avocado: tags=boots:kernel
> +    :avocado: tags=boots:initrd
>       """
>   
>       KERNEL_COMMAND_LINE = "printk.time=0 console=ttyS0 rdinit=/bin/bash"
> diff --git a/tests/avocado/virtiofs_submounts.py b/tests/avocado/virtiofs_submounts.py
> index d9c2c9d9ef..3848f351fb 100644
> --- a/tests/avocado/virtiofs_submounts.py
> +++ b/tests/avocado/virtiofs_submounts.py
> @@ -16,6 +16,7 @@ class VirtiofsSubmountsTest(LinuxTest):
>       """
>       :avocado: tags=arch:x86_64
>       :avocado: tags=accel:kvm
> +    :avocado: tags=boots:distro
>       """
>   
>       def run(self, args, ignore_error=False):

Re: [PATCH 1/9] Avocado GitLab CI jobs: don't reset TARGETS and simplify commands

[Daniel P. Berrangé]

On Fri, Feb 25, 2022 at 04:01:48PM -0500, Cleber Rosa wrote:
> The Avocado tests rely on the TARGETS variable, which is computed
> based on the built targets.  The current set of commands on the
> inherited scripts section will reset those, leaving TARGETS empty and
> consequently the AVOCADO_CMDLINE_TAGS empty too.
> 
> This is causing the list of tests to have no filtering by tags, which
> can be seen by the large number of CANCEL/SKIP statuses (because of
> the lack of a matching qemu-system-$(ARCH) binary).
> 
> With this change, the TARGETS variable is properly computed, and so is
> the AVOCADO_CMDLINE_TAGS.  This causes a reduction in the number of
> tests attempted to be run on each job, and less noise on the test
> results.

This description isn't making sense to me.

AFAICT, none of the avocado-system-$DISTRO  jobs in buildtest.yml
are setting the $TARGETS variable before/after this change.

> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  .gitlab-ci.d/buildtest-template.yml | 3 +++
>  .gitlab-ci.d/buildtest.yml          | 9 ---------
>  2 files changed, 3 insertions(+), 9 deletions(-)
> 
> diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> index 2c7980a4f6..c038a0910f 100644
> --- a/.gitlab-ci.d/buildtest-template.yml
> +++ b/.gitlab-ci.d/buildtest-template.yml
> @@ -64,6 +64,9 @@
>          du -chs ${CI_PROJECT_DIR}/avocado-cache ;
>        fi
>      - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
> +  script:
> +    - cd build
> +    - make check-avocado

The parent template has a 'script:' block we currently inherit 

    - scripts/git-submodule.sh update
        $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
    - cd build
    - find . -type f -exec touch {} +
    # Avoid recompiling by hiding ninja with NINJA=":"
    - make NINJA=":" $MAKE_CHECK_ARGS

so replacing this is loosing the potential git submodule update
and looses the protection against recompilation.

I'm not seeing what in this old inherited is breaking the $TARGETS
variable, not least because it was never set before/after AFAICT.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 7/9] Avocado tests: improve documentation on tag filtering

[Thomas Huth]

On 25/02/2022 22.01, Cleber Rosa wrote:
> It's possible to filter based on a combination of criteria.  This adds
> examples to the documentation.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   docs/devel/testing.rst | 22 ++++++++++++++++++++++
>   1 file changed, 22 insertions(+)
> 
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 92d40cdd19..f5b6e07b5c 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -936,6 +936,28 @@ in the current directory, tagged as "quick", run:
>   
>     avocado run -t quick .
>   
> +To run tests with a given value for a given tag, such as having the
> +``accel`` tag set to ``kvm``, run:
> +
> +.. code::
> +
> +  avocado run -t accel:kvm .
> +
> +Multiple mandatory conditions can also be given.  To run only tests
> +with ``arch`` set to ``x86_64`` and ``accell`` set to ``kvm``, run:

s/accell/accel/

> +
> +.. code::
> +
> +  avocado run -t arch:x86_64,accel:kvm .
> +
> +It's also possible to exclude tests that contain a given value for a

Just a matter of taste, I guess, but I'd prefer "It is" instead of "It's" in 
the documentation.

> +tag.  To list all tests that do *not* have ``arch`` set to ``x86_64``,
> +run:
> +
> +.. code::
> +
> +  avocado run -t arch:-x86_64 .
> +
>   The ``avocado_qemu.Test`` base test class
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

With at least "accell" fixed:
Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 2/9] Avocado tests: use logging namespace that is preserved in test logs

[Alex Bennée]

Cleber Rosa <crosa@redhat.com> writes:

> Since Avocado 92.0[1], there's no universal preservation of logged
> content via Python's "logging" APIs into the test log files.  This
> changes were motivated by the fact that doing so is intrusive as it
> touches on Python's root logger.
>
> Test writers are now expected to use "avocado." as a namespace prefix
> for everything that Avocado should collect/preserve, and other
> prefixes for logged content that should be handled differently.
>
> [1] - https://avocado-framework.readthedocs.io/en/94.0/releases/92_0.html#users-test-writers
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 3/9] Avocado migration test: adapt to "utils.network" API namespace change

[Alex Bennée]

Cleber Rosa <crosa@redhat.com> writes:

> Since Avocado 94.0[1], the "avocado.utils.network" dropped a lot of
> previously deprecated API names, having the new names into a finer
> grained structure.
>
> This simply uses the new API names for the network port utility
> module.
>
> [1] - https://avocado-framework.readthedocs.io/en/latest/releases/94_0.html#utility-apis
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 5/9] tests/avocado/linux_ssh_mips_malta.py: add missing accel (tcg) tag

[Cleber Rosa]

Philippe Mathieu-Daudé <philippe.mathieu.daude@gmail.com> writes:

> On 25/2/22 22:01, Cleber Rosa wrote:
>> Being explicit about the accelerator used on these tests is a good
>> thing in itself, but it will also be used in the filtering rules
>> applied on "make check-avocado".
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>>   tests/avocado/linux_ssh_mips_malta.py | 3 +++
>>   1 file changed, 3 insertions(+)
>> diff --git a/tests/avocado/linux_ssh_mips_malta.py
>> b/tests/avocado/linux_ssh_mips_malta.py
>> index c0f0be5ade..0179d8a6ca 100644
>> --- a/tests/avocado/linux_ssh_mips_malta.py
>> +++ b/tests/avocado/linux_ssh_mips_malta.py
>> @@ -23,6 +23,9 @@
>>   @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
>
> Should we remove this line then? ^^^
>

Yes, we definitely should.  But, I thought it should be a next step.
The reason being that manual invocations of avocado or a custom list of
tests to "make check-avocado" will still respect that at this point.

What do you think?

Cheers,
- Cleber.

Re: [PATCH 5/9] tests/avocado/linux_ssh_mips_malta.py: add missing accel (tcg) tag

[Philippe Mathieu-Daudé]

On 3/3/22 13:13, Cleber Rosa wrote:
> 
> Philippe Mathieu-Daudé <philippe.mathieu.daude@gmail.com> writes:
> 
>> On 25/2/22 22:01, Cleber Rosa wrote:
>>> Being explicit about the accelerator used on these tests is a good
>>> thing in itself, but it will also be used in the filtering rules
>>> applied on "make check-avocado".
>>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>>> ---
>>>    tests/avocado/linux_ssh_mips_malta.py | 3 +++
>>>    1 file changed, 3 insertions(+)
>>> diff --git a/tests/avocado/linux_ssh_mips_malta.py
>>> b/tests/avocado/linux_ssh_mips_malta.py
>>> index c0f0be5ade..0179d8a6ca 100644
>>> --- a/tests/avocado/linux_ssh_mips_malta.py
>>> +++ b/tests/avocado/linux_ssh_mips_malta.py
>>> @@ -23,6 +23,9 @@
>>>    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
>>
>> Should we remove this line then? ^^^
>>
> 
> Yes, we definitely should.  But, I thought it should be a next step.

OK.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

And queued to mips-next.

Re: [PATCH 6/9] tests/avocado/virtiofs_submounts.py: shared_dir may not exist

[Beraldo Leal]

On Fri, Feb 25, 2022 at 04:01:53PM -0500, Cleber Rosa wrote:
> If the test is skipped because of their conditionals, the shared_dir
> attribute may not exist.
> 
> Check for its existence in the tearDown() method to avoid and
> AttributeError.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  tests/avocado/virtiofs_submounts.py | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/avocado/virtiofs_submounts.py b/tests/avocado/virtiofs_submounts.py
> index e6dc32ffd4..d9c2c9d9ef 100644
> --- a/tests/avocado/virtiofs_submounts.py
> +++ b/tests/avocado/virtiofs_submounts.py
> @@ -157,9 +157,10 @@ def tearDown(self):
>          except:
>              pass
>  
> -        scratch_dir = os.path.join(self.shared_dir, 'scratch')
> -        self.run(('bash', self.get_data('cleanup.sh'), scratch_dir),
> -                 ignore_error=True)
> +        if hasattr(self, 'shared_dir'):
> +            scratch_dir = os.path.join(self.shared_dir, 'scratch')
> +            self.run(('bash', self.get_data('cleanup.sh'), scratch_dir),
> +                     ignore_error=True)
>  
>      def test_pre_virtiofsd_set_up(self):
>          self.set_up_shared_dir()

Reviewed-by: Beraldo Leal <bleal@redhat.com>

--
Beraldo