[PATCH v3 0/3] avocado-qemu: New SMMUv3 and intel IOMMU tests

[PATCH v3 0/3] avocado-qemu: New SMMUv3 and intel IOMMU tests

[Eric Auger]

This series adds ARM SMMU and Intel IOMMU functional
tests using Fedora cloud-init images.

ARM SMMU tests feature guests with and without RIL
(range invalidation support) using respectively fedora 33
and 31.  For each, we test the protection of virtio-net-pci
and virtio-block-pci devices. Also strict=no and passthrough
modes are tested. So there is a total of 6 tests.

Compared to v2, Intel IOMMU tests were added. Different
operating modes are tested such as strict, caching mode, pt.

The series applies on top of Cleber's series:
- [PATCH 0/3] Acceptance Tests: support choosing specific

Best Regards

Eric

The series and its dependencies can be found at:
https://github.com/eauger/qemu/tree/avocado-qemu-v3


Eric Auger (3):
  Acceptance Tests: Add default kernel params and pxeboot url to the
    KNOWN_DISTROS collection
  avocado_qemu: Add SMMUv3 tests
  avocado_qemu: Add Intel iommu tests

 tests/acceptance/avocado_qemu/__init__.py |  52 ++++++++-
 tests/acceptance/intel_iommu.py           | 115 +++++++++++++++++++
 tests/acceptance/smmu.py                  | 133 ++++++++++++++++++++++
 3 files changed, 298 insertions(+), 2 deletions(-)
 create mode 100644 tests/acceptance/intel_iommu.py
 create mode 100644 tests/acceptance/smmu.py

-- 
2.26.3

[PATCH v3 1/3] Acceptance Tests: Add default kernel params and pxeboot url to the KNOWN_DISTROS collection

[Eric Auger]

When running LinuxTests we may need to run the guest with
custom params. It is practical to store the pxeboot URL
and the default kernel params so that the
tests just need to fetch those and augment the kernel params.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v2 -> v3:
- add fed32 and fed33 checksums
---
 tests/acceptance/avocado_qemu/__init__.py | 52 ++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 2 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 81ac90bebb..8152420fa5 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -305,17 +305,59 @@ def ssh_command(self, command):
     'fedora': {
         '31': {
             'x86_64':
-            {'checksum': 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},
+            {'checksum': 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0',
+             'pxeboot_url': "https://archives.fedoraproject.org/pub/archive/fedora/"
+                            "linux/releases/31/Everything/x86_64/os/images/pxeboot/",
+             'kernel_params': "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro "
+                              "no_timer_check net.ifnames=0 "
+                              "console=tty1 console=ttyS0,115200n8"},
             'aarch64':
-            {'checksum': '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},
+            {'checksum': '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49',
+             'pxeboot_url': "https://archives.fedoraproject.org/pub/archive/fedora/"
+                            "linux/releases/31/Everything/aarch64/os/images/pxeboot/",
+             'kernel_params': "root=UUID=b6950a44-9f3c-4076-a9c2-355e8475b0a7 ro "
+                              "earlyprintk=pl011,0x9000000 ignore_loglevel "
+                              "no_timer_check printk.time=1 rd_NO_PLYMOUTH "
+                              "console=ttyAMA0 "},
             'ppc64':
             {'checksum': '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},
             's390x':
             {'checksum': '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'},
             }
+        ,
+        '32': {
+            'aarch64':
+            { 'kernel_params': "root=UUID=3df75b65-be8d-4db4-8655-14d95c0e90c5 ro "
+                              "no_timer_check net.ifnames=0 console=tty1 "
+                              "console=ttyS0,115200n8 ",
+              'checksum': 'b367755c664a2d7a26955bbfff985855adfa2ca15e908baf15b4b176d68d3967',
+              'pxeboot_url':  "https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/"
+                              "32/Server/aarch64/os/images/pxeboot/"},
+            }
+        ,
+        '33': {
+            'aarch64':
+            { 'kernel_params': "root=UUID=d20b3ffa-6397-4a63-a734-1126a0208f8a ro "
+                              "no_timer_check net.ifnames=0 console=tty1 "
+                              "console=ttyS0,115200n8 console=tty0 ",
+              'checksum': 'e7f75cdfd523fe5ac2ca9eeece68edc1a81f386a17f969c1d1c7c87031008a6b',
+              'pxeboot_url':  "https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/"
+                              "33/Server/aarch64/os/images/pxeboot/"},
+            }
         }
     }
 
+def get_known_distro_kernel_params(distro, distro_version, arch):
+    try:
+        return KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('kernel_params')
+    except AttributeError:
+        return None
+
+def get_known_distro_pxeboot_url(distro, distro_version, arch):
+    try:
+        return KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('pxeboot_url')
+    except AttributeError:
+        return None
 
 def get_known_distro_checksum(distro, distro_version, arch):
     try:
@@ -449,6 +491,12 @@ def set_up_cloudinit(self, ssh_pubkey=None):
         cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
         self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
 
+    def get_default_kernel_params(self):
+        return get_known_distro_kernel_params(self.distro, self.distro_version, self.arch)
+
+    def get_pxeboot_url(self):
+        return get_known_distro_pxeboot_url(self.distro, self.distro_version, self.arch)
+
     def launch_and_wait(self, set_up_ssh_connection=True):
         self.vm.set_console()
         self.vm.launch()
-- 
2.26.3

[PATCH v3 2/3] avocado_qemu: Add SMMUv3 tests

[Eric Auger]

Add new tests checking the good behavior of the SMMUv3 protecting
2 virtio pci devices (block and net). We check the guest boots and
we are able to install a package. Different guest configs are tested:
standard, passthrough an strict=0. This is tested with both fedora 31 and
33. The former uses a 5.3 kernel without range invalidation whereas the
latter uses a 5.8 kernel that features range invalidation.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v1 -> v2:
- removed ssh import
- combined add_command_args() and common_vm_setup()
- moved tags in class' docstring and added tags=arch:aarch64
- use self.get_default_kernel_params()
- added RIL tests with fed33 + introduce new tags
---
 tests/acceptance/smmu.py | 133 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 133 insertions(+)
 create mode 100644 tests/acceptance/smmu.py

diff --git a/tests/acceptance/smmu.py b/tests/acceptance/smmu.py
new file mode 100644
index 0000000000..bcb5416a56
--- /dev/null
+++ b/tests/acceptance/smmu.py
@@ -0,0 +1,133 @@
+# SMMUv3 Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Eric Auger <eric.auger@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+
+from avocado_qemu import LinuxTest, BUILD_DIR
+
+class SMMU(LinuxTest):
+    """
+    :avocado: tags=accel:kvm
+    :avocado: tags=cpu:host
+    :avocado: tags=arch:aarch64
+    :avocado: tags=smmu
+    """
+
+    IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
+    kernel_path = None
+    initrd_path = None
+    kernel_params = None
+
+    def set_up_boot(self):
+        path = self.download_boot()
+        self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
+                         'drive=drv0,id=virtio-disk0,bootindex=1,'
+                         'werror=stop,rerror=stop' + self.IOMMU_ADDON)
+        self.vm.add_args('-drive',
+                         'file=%s,if=none,cache=writethrough,id=drv0' % path)
+
+    def setUp(self):
+        super(SMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON)
+
+    def common_vm_setup(self, custom_kernel=None):
+        self.require_accelerator("kvm")
+        self.vm.add_args("-machine", "virt")
+        self.vm.add_args("-accel", "kvm")
+        self.vm.add_args("-cpu", "host")
+        self.vm.add_args("-smp", "8")
+        self.vm.add_args("-m", "4096")
+        self.vm.add_args("-machine", "iommu=smmuv3")
+        self.vm.add_args("-d", "guest_errors")
+        self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
+                         'edk2-aarch64-code.fd'))
+        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+        self.vm.add_args('-object',
+                         'rng-random,id=rng0,filename=/dev/urandom')
+
+        if custom_kernel is None:
+            return
+
+        kernel_url = self.get_pxeboot_url() + 'vmlinuz'
+        initrd_url = self.get_pxeboot_url() + 'initrd.img'
+        self.kernel_path = self.fetch_asset(kernel_url)
+        self.initrd_path = self.fetch_asset(initrd_url)
+
+    def run_and_check(self):
+        if self.kernel_path:
+            self.vm.add_args('-kernel', self.kernel_path,
+                             '-append', self.kernel_params,
+                             '-initrd', self.initrd_path)
+        self.launch_and_wait()
+        self.ssh_command('cat /proc/cmdline')
+        self.ssh_command('dnf -y install numactl-devel')
+
+
+    # 5.3 kernel without RIL #
+
+    def test_smmu_noril(self):
+        """
+        :avocado: tags=smmu_noril
+        :avocado: tags=smmu_noril_tests
+        :avocado: tags=distro_version:31
+        """
+        self.common_vm_setup()
+        self.run_and_check()
+
+    def test_smmu_noril_passthrough(self):
+        """
+        :avocado: tags=smmu_noril_passthrough
+        :avocado: tags=smmu_noril_tests
+        :avocado: tags=distro_version:31
+        """
+        self.common_vm_setup(True)
+        self.kernel_params = self.get_default_kernel_params() + ' iommu.passthrough=on'
+        self.run_and_check()
+
+    def test_smmu_noril_nostrict(self):
+        """
+        :avocado: tags=smmu_noril_nostrict
+        :avocado: tags=smmu_noril_tests
+        :avocado: tags=distro_version:31
+        """
+        self.common_vm_setup(True)
+        self.kernel_params = self.get_default_kernel_params() + ' iommu.strict=0'
+        self.run_and_check()
+
+    # 5.8 kernel featuring range invalidation
+    # >= v5.7 kernel
+
+    def test_smmu_ril(self):
+        """
+        :avocado: tags=smmu_ril
+        :avocado: tags=smmu_ril_tests
+        :avocado: tags=distro_version:33
+        """
+        self.common_vm_setup()
+        self.run_and_check()
+
+    def test_smmu_ril_passthrough(self):
+        """
+        :avocado: tags=smmu_ril_passthrough
+        :avocado: tags=smmu_ril_tests
+        :avocado: tags=distro_version:33
+        """
+        self.common_vm_setup(True)
+        self.kernel_params = self.get_default_kernel_params() + ' iommu.passthrough=on'
+        self.run_and_check()
+
+    def test_smmu_ril_nostrict(self):
+        """
+        :avocado: tags=smmu_ril_nostrict
+        :avocado: tags=smmu_ril_tests
+        :avocado: tags=distro_version:33
+        """
+        self.common_vm_setup(True)
+        self.kernel_params = self.get_default_kernel_params() + ' iommu.strict=0'
+        self.run_and_check()
-- 
2.26.3

[PATCH v3 3/3] avocado_qemu: Add Intel iommu tests

[Eric Auger]

Add Intel IOMMU functional tests based on fedora 31.
Different configs are checked:
- strict
- caching mode, strict
- passthrough.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
 tests/acceptance/intel_iommu.py | 115 ++++++++++++++++++++++++++++++++
 1 file changed, 115 insertions(+)
 create mode 100644 tests/acceptance/intel_iommu.py

diff --git a/tests/acceptance/intel_iommu.py b/tests/acceptance/intel_iommu.py
new file mode 100644
index 0000000000..0b68d3c572
--- /dev/null
+++ b/tests/acceptance/intel_iommu.py
@@ -0,0 +1,115 @@
+# INTEL_IOMMU Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Eric Auger <eric.auger@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+
+from avocado_qemu import LinuxTest, BUILD_DIR
+from avocado.utils import ssh
+
+class INTEL_IOMMU(LinuxTest):
+    """
+    :avocado: tags=arch:x86_64
+    :avocado: tags=distro:fedora
+    :avocado: tags=distro_version:31
+    :avocado: tags=machine:q35
+    :avocado: tags=accel:kvm
+    :avocado: tags=intel_iommu
+    """
+
+    IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
+    kernel_path = None
+    initrd_path = None
+    kernel_params = None
+
+    def set_up_boot(self):
+        path = self.download_boot()
+        self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
+                         'drive=drv0,id=virtio-disk0,bootindex=1,'
+                         'werror=stop,rerror=stop' + self.IOMMU_ADDON)
+        self.vm.add_args('-device', 'virtio-gpu-pci' + self.IOMMU_ADDON)
+        self.vm.add_args('-drive',
+                         'file=%s,if=none,cache=writethrough,id=drv0' % path)
+
+    def setUp(self):
+        super(INTEL_IOMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON)
+
+    def add_common_args(self):
+        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+        self.vm.add_args('-object',
+                         'rng-random,id=rng0,filename=/dev/urandom')
+
+    def common_vm_setup(self, custom_kernel=None):
+        self.require_accelerator("kvm")
+        self.add_common_args()
+        self.vm.add_args("-accel", "kvm")
+
+        if custom_kernel is None:
+            return
+
+        kernel_url = self.get_pxeboot_url() + 'vmlinuz'
+        initrd_url = self.get_pxeboot_url() + 'initrd.img'
+        self.kernel_path = self.fetch_asset(kernel_url)
+        self.initrd_path = self.fetch_asset(initrd_url)
+
+    def run_and_check(self):
+        if self.kernel_path:
+            self.vm.add_args('-kernel', self.kernel_path,
+                             '-append', self.kernel_params,
+                             '-initrd', self.initrd_path)
+        self.launch_and_wait()
+        self.ssh_command('cat /proc/cmdline')
+        self.ssh_command('dmesg | grep -e DMAR -e IOMMU')
+        self.ssh_command('find /sys/kernel/iommu_groups/ -type l')
+        self.ssh_command('dnf -y install numactl-devel')
+
+    def test_intel_iommu(self):
+        """
+        :avocado: tags=intel_iommu_intremap
+        """
+
+        self.common_vm_setup(True)
+        self.vm.add_args('-device', 'intel-iommu,intremap=on')
+        self.vm.add_args('-machine', 'kernel_irqchip=split')
+
+        self.kernel_params = self.get_default_kernel_params() + ' quiet intel_iommu=on'
+        self.run_and_check()
+
+    def test_intel_iommu_strict(self):
+        """
+        :avocado: tags=intel_iommu_strict
+        """
+
+        self.common_vm_setup(True)
+        self.vm.add_args('-device', 'intel-iommu,intremap=on')
+        self.vm.add_args('-machine', 'kernel_irqchip=split')
+        self.kernel_params = self.get_default_kernel_params() + ' quiet intel_iommu=on,strict'
+        self.run_and_check()
+
+    def test_intel_iommu_strict_cm(self):
+        """
+        :avocado: tags=intel_iommu_strict_cm
+        """
+
+        self.common_vm_setup(True)
+        self.vm.add_args('-device', 'intel-iommu,intremap=on,caching-mode=on')
+        self.vm.add_args('-machine', 'kernel_irqchip=split')
+        self.kernel_params = self.get_default_kernel_params() + ' quiet intel_iommu=on,strict'
+        self.run_and_check()
+
+    def test_intel_iommu_pt(self):
+        """
+        :avocado: tags=intel_iommu_pt
+        """
+
+        self.common_vm_setup(True)
+        self.vm.add_args('-device', 'intel-iommu,intremap=on')
+        self.vm.add_args('-machine', 'kernel_irqchip=split')
+        self.kernel_params = self.get_default_kernel_params() + ' quiet intel_iommu=on iommu=pt'
+        self.run_and_check()
-- 
2.26.3

Re: [PATCH v3 0/3] avocado-qemu: New SMMUv3 and intel IOMMU tests

[Philippe Mathieu-Daudé]

Cc'ing Willian

On 6/21/21 10:08 AM, Eric Auger wrote:
> This series adds ARM SMMU and Intel IOMMU functional
> tests using Fedora cloud-init images.
> 
> ARM SMMU tests feature guests with and without RIL
> (range invalidation support) using respectively fedora 33
> and 31.  For each, we test the protection of virtio-net-pci
> and virtio-block-pci devices. Also strict=no and passthrough
> modes are tested. So there is a total of 6 tests.
> 
> Compared to v2, Intel IOMMU tests were added. Different
> operating modes are tested such as strict, caching mode, pt.
> 
> The series applies on top of Cleber's series:
> - [PATCH 0/3] Acceptance Tests: support choosing specific
> 
> Best Regards
> 
> Eric
> 
> The series and its dependencies can be found at:
> https://github.com/eauger/qemu/tree/avocado-qemu-v3
> 
> 
> Eric Auger (3):
>   Acceptance Tests: Add default kernel params and pxeboot url to the
>     KNOWN_DISTROS collection
>   avocado_qemu: Add SMMUv3 tests
>   avocado_qemu: Add Intel iommu tests
> 
>  tests/acceptance/avocado_qemu/__init__.py |  52 ++++++++-
>  tests/acceptance/intel_iommu.py           | 115 +++++++++++++++++++
>  tests/acceptance/smmu.py                  | 133 ++++++++++++++++++++++
>  3 files changed, 298 insertions(+), 2 deletions(-)
>  create mode 100644 tests/acceptance/intel_iommu.py
>  create mode 100644 tests/acceptance/smmu.py
> 

Re: [PATCH v3 3/3] avocado_qemu: Add Intel iommu tests

[Peter Xu]

Hi, Eric,

On Mon, Jun 21, 2021 at 10:08:24AM +0200, Eric Auger wrote:
> Add Intel IOMMU functional tests based on fedora 31.
> Different configs are checked:
> - strict
> - caching mode, strict
> - passthrough.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>

Acked-by: Peter Xu <peterx@redhat.com>

Thanks for adding this test!

-- 
Peter Xu

Re: [PATCH v3 1/3] Acceptance Tests: Add default kernel params and pxeboot url to the KNOWN_DISTROS collection

[Wainer dos Santos Moschetta]

Hi Eric,

On 6/21/21 5:08 AM, Eric Auger wrote:
> When running LinuxTests we may need to run the guest with
> custom params. It is practical to store the pxeboot URL
> and the default kernel params so that the
> tests just need to fetch those and augment the kernel params.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v2 -> v3:
> - add fed32 and fed33 checksums
> ---
>   tests/acceptance/avocado_qemu/__init__.py | 52 ++++++++++++++++++++++-
>   1 file changed, 50 insertions(+), 2 deletions(-)
>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 81ac90bebb..8152420fa5 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -305,17 +305,59 @@ def ssh_command(self, command):
>       'fedora': {
>           '31': {
>               'x86_64':
> -            {'checksum': 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},
> +            {'checksum': 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0',
> +             'pxeboot_url': "https://archives.fedoraproject.org/pub/archive/fedora/"
> +                            "linux/releases/31/Everything/x86_64/os/images/pxeboot/",
> +             'kernel_params': "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro "
> +                              "no_timer_check net.ifnames=0 "
> +                              "console=tty1 console=ttyS0,115200n8"},
>               'aarch64':
> -            {'checksum': '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},
> +            {'checksum': '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49',
> +             'pxeboot_url': "https://archives.fedoraproject.org/pub/archive/fedora/"
> +                            "linux/releases/31/Everything/aarch64/os/images/pxeboot/",
> +             'kernel_params': "root=UUID=b6950a44-9f3c-4076-a9c2-355e8475b0a7 ro "
> +                              "earlyprintk=pl011,0x9000000 ignore_loglevel "
> +                              "no_timer_check printk.time=1 rd_NO_PLYMOUTH "
> +                              "console=ttyAMA0 "},
>               'ppc64':
>               {'checksum': '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},
>               's390x':
>               {'checksum': '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'},
>               }
> +        ,
> +        '32': {
> +            'aarch64':
> +            { 'kernel_params': "root=UUID=3df75b65-be8d-4db4-8655-14d95c0e90c5 ro "
> +                              "no_timer_check net.ifnames=0 console=tty1 "
> +                              "console=ttyS0,115200n8 ",
Nit: indentation ^
> +              'checksum': 'b367755c664a2d7a26955bbfff985855adfa2ca15e908baf15b4b176d68d3967',
> +              'pxeboot_url':  "https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/"
Maybe use dl.fedoraproject.org instead of a mirror server?
> +                              "32/Server/aarch64/os/images/pxeboot/"},
> +            }
> +        ,
> +        '33': {
> +            'aarch64':
> +            { 'kernel_params': "root=UUID=d20b3ffa-6397-4a63-a734-1126a0208f8a ro "
> +                              "no_timer_check net.ifnames=0 console=tty1 "
> +                              "console=ttyS0,115200n8 console=tty0 ",
> +              'checksum': 'e7f75cdfd523fe5ac2ca9eeece68edc1a81f386a17f969c1d1c7c87031008a6b',
> +              'pxeboot_url':  "https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/"
Likewise.
> +                              "33/Server/aarch64/os/images/pxeboot/"},
> +            }
>           }
>       }
>   
> +def get_known_distro_kernel_params(distro, distro_version, arch):
> +    try:
> +        return KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('kernel_params')
> +    except AttributeError:
> +        return None
> +
> +def get_known_distro_pxeboot_url(distro, distro_version, arch):
> +    try:
> +        return KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('pxeboot_url')
> +    except AttributeError:
> +        return None
>   
>   def get_known_distro_checksum(distro, distro_version, arch):
>       try:
> @@ -449,6 +491,12 @@ def set_up_cloudinit(self, ssh_pubkey=None):
>           cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
>           self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
>   
> +    def get_default_kernel_params(self):
> +        return get_known_distro_kernel_params(self.distro, self.distro_version, self.arch)
> +
> +    def get_pxeboot_url(self):
> +        return get_known_distro_pxeboot_url(self.distro, self.distro_version, self.arch)
> +

As the KNOWN_DISTROS grows, more loosely methods will be created in the 
avocado_qemu/__init__.py file.

I refactored the code so that KNOWN_DISTROS and related methods are 
packaged in a class. See in:

https://github.com/wainersm/qemu/tree/eauger_avocado-qemu-v3-refactor

Maybe you could incorporate that in your series? Otherwise I can send as 
a follow up series. Whatever you prefer.

Thanks!

- Wainer

>       def launch_and_wait(self, set_up_ssh_connection=True):
>           self.vm.set_console()
>           self.vm.launch()

Re: [PATCH v3 2/3] avocado_qemu: Add SMMUv3 tests

[Wainer dos Santos Moschetta]

Hi,

On 6/21/21 5:08 AM, Eric Auger wrote:
> Add new tests checking the good behavior of the SMMUv3 protecting
> 2 virtio pci devices (block and net). We check the guest boots and
> we are able to install a package. Different guest configs are tested:
> standard, passthrough an strict=0. This is tested with both fedora 31 and
> 33. The former uses a 5.3 kernel without range invalidation whereas the
> latter uses a 5.8 kernel that features range invalidation.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v1 -> v2:
> - removed ssh import
> - combined add_command_args() and common_vm_setup()
> - moved tags in class' docstring and added tags=arch:aarch64
> - use self.get_default_kernel_params()
> - added RIL tests with fed33 + introduce new tags
> ---
>   tests/acceptance/smmu.py | 133 +++++++++++++++++++++++++++++++++++++++
>   1 file changed, 133 insertions(+)
>   create mode 100644 tests/acceptance/smmu.py
>
> diff --git a/tests/acceptance/smmu.py b/tests/acceptance/smmu.py
> new file mode 100644
> index 0000000000..bcb5416a56
> --- /dev/null
> +++ b/tests/acceptance/smmu.py
> @@ -0,0 +1,133 @@
> +# SMMUv3 Functional tests
> +#
> +# Copyright (c) 2021 Red Hat, Inc.
> +#
> +# Author:
> +#  Eric Auger <eric.auger@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +
> +from avocado_qemu import LinuxTest, BUILD_DIR
> +
> +class SMMU(LinuxTest):
> +    """
> +    :avocado: tags=accel:kvm
> +    :avocado: tags=cpu:host
> +    :avocado: tags=arch:aarch64
> +    :avocado: tags=smmu
> +    """
> +
> +    IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
> +    kernel_path = None
> +    initrd_path = None
> +    kernel_params = None
> +
> +    def set_up_boot(self):
> +        path = self.download_boot()
> +        self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
> +                         'drive=drv0,id=virtio-disk0,bootindex=1,'
> +                         'werror=stop,rerror=stop' + self.IOMMU_ADDON)
> +        self.vm.add_args('-drive',
> +                         'file=%s,if=none,cache=writethrough,id=drv0' % path)
> +
> +    def setUp(self):
> +        super(SMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON)
> +
> +    def common_vm_setup(self, custom_kernel=None):
As some tests call `self.common_vm_setup(True)`, I think `custom_kernel` 
should be `True|False`.
> +        self.require_accelerator("kvm")
> +        self.vm.add_args("-machine", "virt")
FYI, if you have the "machine" tag (e.g. "tags=machine:virt") to your 
tests then avocado_qemu will set "-machine" automatically to any created VM.
> +        self.vm.add_args("-accel", "kvm")
> +        self.vm.add_args("-cpu", "host")
> +        self.vm.add_args("-smp", "8")
> +        self.vm.add_args("-m", "4096")

The `avocado_qemu.LinuxTest` provides default values for smp and memory 
which cannot be properly overwritten. The resulting command line will 
have -smp and -m twice.

I created an issue to track that improvement: 
https://gitlab.com/qemu-project/qemu/-/issues/453

> +        self.vm.add_args("-machine", "iommu=smmuv3")
> +        self.vm.add_args("-d", "guest_errors")
> +        self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
> +                         'edk2-aarch64-code.fd'))
> +        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
> +        self.vm.add_args('-object',
> +                         'rng-random,id=rng0,filename=/dev/urandom')
> +
> +        if custom_kernel is None:
> +            return
> +
> +        kernel_url = self.get_pxeboot_url() + 'vmlinuz'
> +        initrd_url = self.get_pxeboot_url() + 'initrd.img'
> +        self.kernel_path = self.fetch_asset(kernel_url)
> +        self.initrd_path = self.fetch_asset(initrd_url)
> +
> +    def run_and_check(self):
> +        if self.kernel_path:
> +            self.vm.add_args('-kernel', self.kernel_path,
> +                             '-append', self.kernel_params,
> +                             '-initrd', self.initrd_path)
> +        self.launch_and_wait()
> +        self.ssh_command('cat /proc/cmdline')
> +        self.ssh_command('dnf -y install numactl-devel')
> +
> +
> +    # 5.3 kernel without RIL #
> +
> +    def test_smmu_noril(self):
> +        """
> +        :avocado: tags=smmu_noril
> +        :avocado: tags=smmu_noril_tests
> +        :avocado: tags=distro_version:31
> +        """
> +        self.common_vm_setup()
> +        self.run_and_check()
> +
> +    def test_smmu_noril_passthrough(self):
> +        """
> +        :avocado: tags=smmu_noril_passthrough
> +        :avocado: tags=smmu_noril_tests
> +        :avocado: tags=distro_version:31

Maybe you should "distro" tag the tests (or better, tag the class), so 
that a readers of this don't need to browse the `LinuxTest` class 
looking for the distro to be used (although it may be clear to some, 
based on the distro_version)...

Thanks for contributing those tests!

- Wainer

> +        """
> +        self.common_vm_setup(True)
> +        self.kernel_params = self.get_default_kernel_params() + ' iommu.passthrough=on'
> +        self.run_and_check()
> +
> +    def test_smmu_noril_nostrict(self):
> +        """
> +        :avocado: tags=smmu_noril_nostrict
> +        :avocado: tags=smmu_noril_tests
> +        :avocado: tags=distro_version:31
> +        """
> +        self.common_vm_setup(True)
> +        self.kernel_params = self.get_default_kernel_params() + ' iommu.strict=0'
> +        self.run_and_check()
> +
> +    # 5.8 kernel featuring range invalidation
> +    # >= v5.7 kernel
> +
> +    def test_smmu_ril(self):
> +        """
> +        :avocado: tags=smmu_ril
> +        :avocado: tags=smmu_ril_tests
> +        :avocado: tags=distro_version:33
> +        """
> +        self.common_vm_setup()
> +        self.run_and_check()
> +
> +    def test_smmu_ril_passthrough(self):
> +        """
> +        :avocado: tags=smmu_ril_passthrough
> +        :avocado: tags=smmu_ril_tests
> +        :avocado: tags=distro_version:33
> +        """
> +        self.common_vm_setup(True)
> +        self.kernel_params = self.get_default_kernel_params() + ' iommu.passthrough=on'
> +        self.run_and_check()
> +
> +    def test_smmu_ril_nostrict(self):
> +        """
> +        :avocado: tags=smmu_ril_nostrict
> +        :avocado: tags=smmu_ril_tests
> +        :avocado: tags=distro_version:33
> +        """
> +        self.common_vm_setup(True)
> +        self.kernel_params = self.get_default_kernel_params() + ' iommu.strict=0'
> +        self.run_and_check()

Re: [PATCH v3 1/3] Acceptance Tests: Add default kernel params and pxeboot url to the KNOWN_DISTROS collection

[Eric Auger]

Hi Wainer,

On 6/28/21 5:06 PM, Wainer dos Santos Moschetta wrote:
> Hi Eric,
>
> On 6/21/21 5:08 AM, Eric Auger wrote:
>> When running LinuxTests we may need to run the guest with
>> custom params. It is practical to store the pxeboot URL
>> and the default kernel params so that the
>> tests just need to fetch those and augment the kernel params.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v2 -> v3:
>> - add fed32 and fed33 checksums
>> ---
>>   tests/acceptance/avocado_qemu/__init__.py | 52 ++++++++++++++++++++++-
>>   1 file changed, 50 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/acceptance/avocado_qemu/__init__.py
>> b/tests/acceptance/avocado_qemu/__init__.py
>> index 81ac90bebb..8152420fa5 100644
>> --- a/tests/acceptance/avocado_qemu/__init__.py
>> +++ b/tests/acceptance/avocado_qemu/__init__.py
>> @@ -305,17 +305,59 @@ def ssh_command(self, command):
>>       'fedora': {
>>           '31': {
>>               'x86_64':
>> -            {'checksum':
>> 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},
>> +            {'checksum':
>> 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0',
>> +             'pxeboot_url':
>> "https://archives.fedoraproject.org/pub/archive/fedora/"
>> +                           
>> "linux/releases/31/Everything/x86_64/os/images/pxeboot/",
>> +             'kernel_params':
>> "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro "
>> +                              "no_timer_check net.ifnames=0 "
>> +                              "console=tty1 console=ttyS0,115200n8"},
>>               'aarch64':
>> -            {'checksum':
>> '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},
>> +            {'checksum':
>> '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49',
>> +             'pxeboot_url':
>> "https://archives.fedoraproject.org/pub/archive/fedora/"
>> +                           
>> "linux/releases/31/Everything/aarch64/os/images/pxeboot/",
>> +             'kernel_params':
>> "root=UUID=b6950a44-9f3c-4076-a9c2-355e8475b0a7 ro "
>> +                              "earlyprintk=pl011,0x9000000
>> ignore_loglevel "
>> +                              "no_timer_check printk.time=1
>> rd_NO_PLYMOUTH "
>> +                              "console=ttyAMA0 "},
>>               'ppc64':
>>               {'checksum':
>> '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},
>>               's390x':
>>               {'checksum':
>> '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'},
>>               }
>> +        ,
>> +        '32': {
>> +            'aarch64':
>> +            { 'kernel_params':
>> "root=UUID=3df75b65-be8d-4db4-8655-14d95c0e90c5 ro "
>> +                              "no_timer_check net.ifnames=0
>> console=tty1 "
>> +                              "console=ttyS0,115200n8 ",
> Nit: indentation ^
sure
>> +              'checksum':
>> 'b367755c664a2d7a26955bbfff985855adfa2ca15e908baf15b4b176d68d3967',
>> +              'pxeboot_url': 
>> "https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/"
> Maybe use dl.fedoraproject.org instead of a mirror server?
yep
>> +                              "32/Server/aarch64/os/images/pxeboot/"},
>> +            }
>> +        ,
>> +        '33': {
>> +            'aarch64':
>> +            { 'kernel_params':
>> "root=UUID=d20b3ffa-6397-4a63-a734-1126a0208f8a ro "
>> +                              "no_timer_check net.ifnames=0
>> console=tty1 "
>> +                              "console=ttyS0,115200n8 console=tty0 ",
>> +              'checksum':
>> 'e7f75cdfd523fe5ac2ca9eeece68edc1a81f386a17f969c1d1c7c87031008a6b',
>> +              'pxeboot_url': 
>> "https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/"
> Likewise.
ok
>> +                              "33/Server/aarch64/os/images/pxeboot/"},
>> +            }
>>           }
>>       }
>>   +def get_known_distro_kernel_params(distro, distro_version, arch):
>> +    try:
>> +        return
>> KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('kernel_params')
>> +    except AttributeError:
>> +        return None
>> +
>> +def get_known_distro_pxeboot_url(distro, distro_version, arch):
>> +    try:
>> +        return
>> KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('pxeboot_url')
>> +    except AttributeError:
>> +        return None
>>     def get_known_distro_checksum(distro, distro_version, arch):
>>       try:
>> @@ -449,6 +491,12 @@ def set_up_cloudinit(self, ssh_pubkey=None):
>>           cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
>>           self.vm.add_args('-drive', 'file=%s,format=raw' %
>> cloudinit_iso)
>>   +    def get_default_kernel_params(self):
>> +        return get_known_distro_kernel_params(self.distro,
>> self.distro_version, self.arch)
>> +
>> +    def get_pxeboot_url(self):
>> +        return get_known_distro_pxeboot_url(self.distro,
>> self.distro_version, self.arch)
>> +
>
> As the KNOWN_DISTROS grows, more loosely methods will be created in
> the avocado_qemu/__init__.py file.
>
> I refactored the code so that KNOWN_DISTROS and related methods are
> packaged in a class. See in:
>
> https://github.com/wainersm/qemu/tree/eauger_avocado-qemu-v3-refactor
>
> Maybe you could incorporate that in your series? Otherwise I can send
> as a follow up series. Whatever you prefer.
OK I take your patch on top of my series then.
>
> Thanks!
>
> - Wainer
>
>>       def launch_and_wait(self, set_up_ssh_connection=True):
>>           self.vm.set_console()
>>           self.vm.launch()
>

Re: [PATCH v3 2/3] avocado_qemu: Add SMMUv3 tests

[Eric Auger]

On 6/28/21 8:27 PM, Wainer dos Santos Moschetta wrote:
> Hi,
>
> On 6/21/21 5:08 AM, Eric Auger wrote:
>> Add new tests checking the good behavior of the SMMUv3 protecting
>> 2 virtio pci devices (block and net). We check the guest boots and
>> we are able to install a package. Different guest configs are tested:
>> standard, passthrough an strict=0. This is tested with both fedora 31
>> and
>> 33. The former uses a 5.3 kernel without range invalidation whereas the
>> latter uses a 5.8 kernel that features range invalidation.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v1 -> v2:
>> - removed ssh import
>> - combined add_command_args() and common_vm_setup()
>> - moved tags in class' docstring and added tags=arch:aarch64
>> - use self.get_default_kernel_params()
>> - added RIL tests with fed33 + introduce new tags
>> ---
>>   tests/acceptance/smmu.py | 133 +++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 133 insertions(+)
>>   create mode 100644 tests/acceptance/smmu.py
>>
>> diff --git a/tests/acceptance/smmu.py b/tests/acceptance/smmu.py
>> new file mode 100644
>> index 0000000000..bcb5416a56
>> --- /dev/null
>> +++ b/tests/acceptance/smmu.py
>> @@ -0,0 +1,133 @@
>> +# SMMUv3 Functional tests
>> +#
>> +# Copyright (c) 2021 Red Hat, Inc.
>> +#
>> +# Author:
>> +#  Eric Auger <eric.auger@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import os
>> +
>> +from avocado_qemu import LinuxTest, BUILD_DIR
>> +
>> +class SMMU(LinuxTest):
>> +    """
>> +    :avocado: tags=accel:kvm
>> +    :avocado: tags=cpu:host
>> +    :avocado: tags=arch:aarch64
>> +    :avocado: tags=smmu
>> +    """
>> +
>> +    IOMMU_ADDON =
>> ',iommu_platform=on,disable-modern=off,disable-legacy=on'
>> +    kernel_path = None
>> +    initrd_path = None
>> +    kernel_params = None
>> +
>> +    def set_up_boot(self):
>> +        path = self.download_boot()
>> +        self.vm.add_args('-device',
>> 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
>> +                         'drive=drv0,id=virtio-disk0,bootindex=1,'
>> +                         'werror=stop,rerror=stop' + self.IOMMU_ADDON)
>> +        self.vm.add_args('-drive',
>> +                        
>> 'file=%s,if=none,cache=writethrough,id=drv0' % path)
>> +
>> +    def setUp(self):
>> +        super(SMMU, self).setUp(None, 'virtio-net-pci' +
>> self.IOMMU_ADDON)
>> +
>> +    def common_vm_setup(self, custom_kernel=None):
> As some tests call `self.common_vm_setup(True)`, I think
> `custom_kernel` should be `True|False`.
OK changed to False
>> +        self.require_accelerator("kvm")
>> +        self.vm.add_args("-machine", "virt")
> FYI, if you have the "machine" tag (e.g. "tags=machine:virt") to your
> tests then avocado_qemu will set "-machine" automatically to any
> created VM.
OK
>> +        self.vm.add_args("-accel", "kvm")
>> +        self.vm.add_args("-cpu", "host")
>> +        self.vm.add_args("-smp", "8")
>> +        self.vm.add_args("-m", "4096")
>
> The `avocado_qemu.LinuxTest` provides default values for smp and
> memory which cannot be properly overwritten. The resulting command
> line will have -smp and -m twice.
>
> I created an issue to track that improvement:
> https://gitlab.com/qemu-project/qemu/-/issues/453
OK removed those params and rely on defaults
>
>> +        self.vm.add_args("-machine", "iommu=smmuv3")
>> +        self.vm.add_args("-d", "guest_errors")
>> +        self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
>> +                         'edk2-aarch64-code.fd'))
>> +        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
>> +        self.vm.add_args('-object',
>> +                         'rng-random,id=rng0,filename=/dev/urandom')
>> +
>> +        if custom_kernel is None:
>> +            return
>> +
>> +        kernel_url = self.get_pxeboot_url() + 'vmlinuz'
>> +        initrd_url = self.get_pxeboot_url() + 'initrd.img'
>> +        self.kernel_path = self.fetch_asset(kernel_url)
>> +        self.initrd_path = self.fetch_asset(initrd_url)
>> +
>> +    def run_and_check(self):
>> +        if self.kernel_path:
>> +            self.vm.add_args('-kernel', self.kernel_path,
>> +                             '-append', self.kernel_params,
>> +                             '-initrd', self.initrd_path)
>> +        self.launch_and_wait()
>> +        self.ssh_command('cat /proc/cmdline')
>> +        self.ssh_command('dnf -y install numactl-devel')
>> +
>> +
>> +    # 5.3 kernel without RIL #
>> +
>> +    def test_smmu_noril(self):
>> +        """
>> +        :avocado: tags=smmu_noril
>> +        :avocado: tags=smmu_noril_tests
>> +        :avocado: tags=distro_version:31
>> +        """
>> +        self.common_vm_setup()
>> +        self.run_and_check()
>> +
>> +    def test_smmu_noril_passthrough(self):
>> +        """
>> +        :avocado: tags=smmu_noril_passthrough
>> +        :avocado: tags=smmu_noril_tests
>> +        :avocado: tags=distro_version:31
>
> Maybe you should "distro" tag the tests (or better, tag the class), so
> that a readers of this don't need to browse the `LinuxTest` class
> looking for the distro to be used (although it may be clear to some,
> based on the distro_version)...
done
>
> Thanks for contributing those tests!
you're welcome

Thanks

Eric
>
> - Wainer
>
>> +        """
>> +        self.common_vm_setup(True)
>> +        self.kernel_params = self.get_default_kernel_params() + '
>> iommu.passthrough=on'
>> +        self.run_and_check()
>> +
>> +    def test_smmu_noril_nostrict(self):
>> +        """
>> +        :avocado: tags=smmu_noril_nostrict
>> +        :avocado: tags=smmu_noril_tests
>> +        :avocado: tags=distro_version:31
>> +        """
>> +        self.common_vm_setup(True)
>> +        self.kernel_params = self.get_default_kernel_params() + '
>> iommu.strict=0'
>> +        self.run_and_check()
>> +
>> +    # 5.8 kernel featuring range invalidation
>> +    # >= v5.7 kernel
>> +
>> +    def test_smmu_ril(self):
>> +        """
>> +        :avocado: tags=smmu_ril
>> +        :avocado: tags=smmu_ril_tests
>> +        :avocado: tags=distro_version:33
>> +        """
>> +        self.common_vm_setup()
>> +        self.run_and_check()
>> +
>> +    def test_smmu_ril_passthrough(self):
>> +        """
>> +        :avocado: tags=smmu_ril_passthrough
>> +        :avocado: tags=smmu_ril_tests
>> +        :avocado: tags=distro_version:33
>> +        """
>> +        self.common_vm_setup(True)
>> +        self.kernel_params = self.get_default_kernel_params() + '
>> iommu.passthrough=on'
>> +        self.run_and_check()
>> +
>> +    def test_smmu_ril_nostrict(self):
>> +        """
>> +        :avocado: tags=smmu_ril_nostrict
>> +        :avocado: tags=smmu_ril_tests
>> +        :avocado: tags=distro_version:33
>> +        """
>> +        self.common_vm_setup(True)
>> +        self.kernel_params = self.get_default_kernel_params() + '
>> iommu.strict=0'
>> +        self.run_and_check()
>