[PULL 0/7] Python (acceptance tests) queue - 2019-09-23

[PULL 0/7] Python (acceptance tests) queue - 2019-09-23

[Cleber Rosa]

The following changes since commit 4300b7c2cd9f3f273804e8cca325842ccb93b1ad:

  Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging (2019-09-20 17:28:43 +0100)

are available in the Git repository at:

  git://github.com/clebergnu/qemu.git tags/python-next-pull-request

for you to fetch changes up to 2fe6f4d9babb957244676fdf23b9a7ca2323034b:

  Acceptance tests: use avocado.utils.ssh for SSH interaction (2019-09-20 17:13:41 -0400)

----------------------------------------------------------------
Improvements include:

 * Relaxed error code pattern on machine_m68k_nextcube.py
 * Better naming and separation of tests in x86_cpu_model_versions.py
 * New checks on guest side on linux_ssh_mips_malta.py
 * Use of avocado.utils.ssh, and thus "/usr/bin/ssh" for SSH
   interaction

A Travis-CI check for this branch can be seen at:

 * https://travis-ci.org/clebergnu/qemu/builds/587665790

----------------------------------------------------------------

Aleksandar Markovic (2):
  tests/acceptance: Refactor and improve reporting in
    linux_ssh_mips_malta.py
  tests/acceptance: Add new test cases in linux_ssh_mips_malta.py

Cleber Rosa (5):
  Acceptance test machine_m68k_nextcube.py: relax the error code pattern
  Acceptance test x86_cpu_model_versions: shutdown VMs
  Acceptance test x86_cpu_model_versions: fix mismatches between test
    and messages
  Acceptance test x86_cpu_model_versions: split into smaller tests
  Acceptance tests: use avocado.utils.ssh for SSH interaction

 tests/acceptance/linux_ssh_mips_malta.py   | 159 ++++++++++++---------
 tests/acceptance/machine_m68k_nextcube.py  |   2 +-
 tests/acceptance/x86_cpu_model_versions.py |  22 ++-
 tests/requirements.txt                     |   3 +-
 4 files changed, 110 insertions(+), 76 deletions(-)

-- 
2.21.0

[PULL 1/7] Acceptance test machine_m68k_nextcube.py: relax the error code pattern

[Cleber Rosa]

Instead of looking for a specific error, let's relax the pattern
because different errors have been seen (I'm consistenly getting 52)
and the real goal of this test is to validate the framebuffer
operation, and not to reproduce one specific error.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <huth@tuxfamily.org>
Message-Id: <20190919161400.26399-1-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/acceptance/machine_m68k_nextcube.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/acceptance/machine_m68k_nextcube.py b/tests/acceptance/machine_m68k_nextcube.py
index e09cab9f20..fcd2c58ee7 100644
--- a/tests/acceptance/machine_m68k_nextcube.py
+++ b/tests/acceptance/machine_m68k_nextcube.py
@@ -116,6 +116,6 @@ class NextCubeMachine(Test):
             if len(line):
                 console_logger.debug(line)
         self.assertIn('Testing the FPU, SCC', text)
-        self.assertIn('System test failed. Error code 51', text)
+        self.assertIn('System test failed. Error code', text)
         self.assertIn('Boot command', text)
         self.assertIn('Next>', text)
-- 
2.21.0

[PULL 2/7] Acceptance test x86_cpu_model_versions: shutdown VMs

[Cleber Rosa]

This shuts down the VMs that won't be used any longer during the
remainder of the test.

It's debatable if the very last one should also be shutdown manually,
and my opinion is that it shouldn't because that's taken care by the
immediately following tearDown().

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190828193628.7687-2-crosa@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/acceptance/x86_cpu_model_versions.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/acceptance/x86_cpu_model_versions.py
index 1c9fd6a56e..e6c76b0d4c 100644
--- a/tests/acceptance/x86_cpu_model_versions.py
+++ b/tests/acceptance/x86_cpu_model_versions.py
@@ -243,6 +243,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
+        vm.shutdown()
 
         vm = self.get_vm()
         vm.add_args('-S')
@@ -251,6 +252,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
+        vm.shutdown()
 
         # command line must override machine-type if CPU model is not versioned:
         vm = self.get_vm()
@@ -260,6 +262,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                         'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
+        vm.shutdown()
 
         vm = self.get_vm()
         vm.add_args('-S')
@@ -268,6 +271,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
+        vm.shutdown()
 
         # versioned CPU model overrides machine-type:
         vm = self.get_vm()
@@ -277,6 +281,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.1 + Cascadelake-Server-v1 should not have arch-capabilities')
+        vm.shutdown()
 
         vm = self.get_vm()
         vm.add_args('-S')
@@ -285,6 +290,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.1 + Cascadelake-Server-v1 should have arch-capabilities')
+        vm.shutdown()
 
         # command line must override machine-type and versioned CPU model:
         vm = self.get_vm()
@@ -294,6 +300,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
+        vm.shutdown()
 
         vm = self.get_vm()
         vm.add_args('-S')
-- 
2.21.0

[PULL 3/7] Acceptance test x86_cpu_model_versions: fix mismatches between test and messages

[Cleber Rosa]

This fixes a few mismatches between the test and the error messages
produced in case of failures.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190828193628.7687-3-crosa@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/acceptance/x86_cpu_model_versions.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/acceptance/x86_cpu_model_versions.py
index e6c76b0d4c..220d18f68d 100644
--- a/tests/acceptance/x86_cpu_model_versions.py
+++ b/tests/acceptance/x86_cpu_model_versions.py
@@ -280,7 +280,7 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
-                         'pc-i440fx-4.1 + Cascadelake-Server-v1 should not have arch-capabilities')
+                         'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
         vm.shutdown()
 
         vm = self.get_vm()
@@ -289,14 +289,14 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.add_args('-cpu', 'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
-                         'pc-i440fx-4.1 + Cascadelake-Server-v1 should have arch-capabilities')
+                         'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
         vm.shutdown()
 
         # command line must override machine-type and versioned CPU model:
         vm = self.get_vm()
         vm.add_args('-S')
         vm.set_machine('pc-i440fx-4.0')
-        vm.add_args('-cpu', 'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
+        vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
-- 
2.21.0

[PULL 4/7] Acceptance test x86_cpu_model_versions: split into smaller tests

[Cleber Rosa]

The justifications being automatic destruction of the vm instances
when no longer needed and more compact test naming under a common
class.

Besides those, a smaller test makes the one and only assertion rather
obvious, which suggests that we could even get rid of the more verbose
(and manual) error messages (to be decided).

Naming of the tests tries to follow the following pattern:

 test_($cpu_version)_($no_arch_capabitilies_set_or_unset)_($machine_version)

The presence of each naming component is optional, depending on
whether the test manually sets it or not.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190828193628.7687-4-crosa@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/acceptance/x86_cpu_model_versions.py | 23 ++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/acceptance/x86_cpu_model_versions.py
index 220d18f68d..5fc9ca4bc6 100644
--- a/tests/acceptance/x86_cpu_model_versions.py
+++ b/tests/acceptance/x86_cpu_model_versions.py
@@ -234,7 +234,14 @@ class X86CPUModelAliases(avocado_qemu.Test):
 
         self.validate_aliases(cpus)
 
-    def test_Cascadelake_arch_capabilities_result(self):
+
+class CascadelakeArchCapabilities(avocado_qemu.Test):
+    """
+    Validation of Cascadelake arch-capabilities
+
+    :avocado: tags=arch:x86_64
+    """
+    def test_4_1(self):
         # machine-type only:
         vm = self.get_vm()
         vm.add_args('-S')
@@ -243,8 +250,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
-        vm.shutdown()
 
+    def test_4_0(self):
         vm = self.get_vm()
         vm.add_args('-S')
         vm.set_machine('pc-i440fx-4.0')
@@ -252,8 +259,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
-        vm.shutdown()
 
+    def test_set_4_0(self):
         # command line must override machine-type if CPU model is not versioned:
         vm = self.get_vm()
         vm.add_args('-S')
@@ -262,8 +269,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                         'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
-        vm.shutdown()
 
+    def test_unset_4_1(self):
         vm = self.get_vm()
         vm.add_args('-S')
         vm.set_machine('pc-i440fx-4.1')
@@ -271,8 +278,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
-        vm.shutdown()
 
+    def test_v1_4_0(self):
         # versioned CPU model overrides machine-type:
         vm = self.get_vm()
         vm.add_args('-S')
@@ -281,8 +288,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertFalse(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
-        vm.shutdown()
 
+    def test_v2_4_0(self):
         vm = self.get_vm()
         vm.add_args('-S')
         vm.set_machine('pc-i440fx-4.0')
@@ -290,8 +297,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
-        vm.shutdown()
 
+    def test_v1_set_4_0(self):
         # command line must override machine-type and versioned CPU model:
         vm = self.get_vm()
         vm.add_args('-S')
@@ -300,8 +307,8 @@ class X86CPUModelAliases(avocado_qemu.Test):
         vm.launch()
         self.assertTrue(get_cpu_prop(vm, 'arch-capabilities'),
                          'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
-        vm.shutdown()
 
+    def test_v2_unset_4_1(self):
         vm = self.get_vm()
         vm.add_args('-S')
         vm.set_machine('pc-i440fx-4.1')
-- 
2.21.0

[PULL 5/7] tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py

[Cleber Rosa]

From: Aleksandar Markovic <amarkovic@wavecomp.com>

This patch restructures code organization around the test case
executions. At the same time, rather than outputing a cryptic message:

FAIL: True not found in [False],

the following will be reported too, if the command output does not meet
specified expectations:

'lspci -d 11ab:4620' output doesn't contain the word 'GT-64120'

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1564760158-27536-2-git-send-email-aleksandar.markovic@rt-rk.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/acceptance/linux_ssh_mips_malta.py | 36 ++++++++++++++----------
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
index 7200507a3a..7fc44f43d1 100644
--- a/tests/acceptance/linux_ssh_mips_malta.py
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -145,27 +145,33 @@ class LinuxSSH(Test):
         self.ssh_disconnect_vm()
         self.wait_for_console_pattern('Power down')
 
+    def ssh_command_output_contains(self, cmd, exp):
+        stdout, _ = self.ssh_command(cmd)
+        for line in stdout:
+            if exp in line:
+                break
+        else:
+            self.fail('"%s" output does not contain "%s"' % (cmd, exp))
+
     def run_common_commands(self):
-        stdout, stderr = self.ssh_command('lspci -d 11ab:4620')
-        self.assertIn(True, ["GT-64120" in line for line in stdout])
-
-        stdout, stderr = self.ssh_command('cat /sys/bus/i2c/devices/i2c-0/name')
-        self.assertIn(True, ["SMBus PIIX4 adapter" in line
-                             for line in stdout])
-
-        stdout, stderr = self.ssh_command('cat /proc/mtd')
-        self.assertIn(True, ["YAMON" in line
-                             for line in stdout])
-
+        self.ssh_command_output_contains(
+            'lspci -d 11ab:4620',
+            'GT-64120')
+        self.ssh_command_output_contains(
+            'cat /sys/bus/i2c/devices/i2c-0/name',
+            'SMBus PIIX4 adapter')
+        self.ssh_command_output_contains(
+            'cat /proc/mtd',
+            'YAMON')
         # Empty 'Board Config'
-        stdout, stderr = self.ssh_command('md5sum /dev/mtd2ro')
-        self.assertIn(True, ["0dfbe8aa4c20b52e1b8bf3cb6cbdf193" in line
-                             for line in stdout])
+        self.ssh_command_output_contains(
+            'md5sum /dev/mtd2ro',
+            '0dfbe8aa4c20b52e1b8bf3cb6cbdf193')
 
     def check_mips_malta(self, endianess, kernel_path, uname_m):
         self.boot_debian_wheezy_image_and_ssh_login(endianess, kernel_path)
 
-        stdout, stderr = self.ssh_command('uname -a')
+        stdout, _ = self.ssh_command('uname -a')
         self.assertIn(True, [uname_m + " GNU/Linux" in line for line in stdout])
 
         self.run_common_commands()
-- 
2.21.0

[PULL 6/7] tests/acceptance: Add new test cases in linux_ssh_mips_malta.py

[Cleber Rosa]

From: Aleksandar Markovic <amarkovic@wavecomp.com>

Add 15 new tests cases. They all rely on simple commands used for
detecting hardware configuration information.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1564760158-27536-3-git-send-email-aleksandar.markovic@rt-rk.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/acceptance/linux_ssh_mips_malta.py | 45 ++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
index 7fc44f43d1..1d334432a0 100644
--- a/tests/acceptance/linux_ssh_mips_malta.py
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -154,6 +154,51 @@ class LinuxSSH(Test):
             self.fail('"%s" output does not contain "%s"' % (cmd, exp))
 
     def run_common_commands(self):
+        self.ssh_command_output_contains(
+            'cat /proc/cpuinfo',
+            '24Kc')
+        self.ssh_command_output_contains(
+            'uname -m',
+            'mips')
+        self.ssh_command_output_contains(
+            'uname -r',
+            '3.2.0-4-4kc-malta')
+        self.ssh_command_output_contains(
+            'cat /proc/interrupts',
+            'timer')
+        self.ssh_command_output_contains(
+            'cat /proc/interrupts',
+            'i8042')
+        self.ssh_command_output_contains(
+            'cat /proc/interrupts',
+            'serial')
+        self.ssh_command_output_contains(
+            'cat /proc/interrupts',
+            'ata_piix')
+        self.ssh_command_output_contains(
+            'cat /proc/interrupts',
+            'eth0')
+        self.ssh_command_output_contains(
+            'cat /proc/interrupts',
+            'eth0')
+        self.ssh_command_output_contains(
+            'cat /proc/devices',
+            'input')
+        self.ssh_command_output_contains(
+            'cat /proc/devices',
+            'usb')
+        self.ssh_command_output_contains(
+            'cat /proc/devices',
+            'fb')
+        self.ssh_command_output_contains(
+            'cat /proc/ioports',
+            'serial')
+        self.ssh_command_output_contains(
+            'cat /proc/ioports',
+            'ata_piix')
+        self.ssh_command_output_contains(
+            'cat /proc/ioports',
+            'piix4_smbus')
         self.ssh_command_output_contains(
             'lspci -d 11ab:4620',
             'GT-64120')
-- 
2.21.0

[PULL 7/7] Acceptance tests: use avocado.utils.ssh for SSH interaction

[Cleber Rosa]

This replaces paramiko with avocado.utils.ssh module, which is based
on a (open)ssh binary, supposedly more ubiquitous.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20190919225905.10829-1-crosa@redhat.com>
[Cleber: consolidated existing skipUnless from tests to setUp]
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/acceptance/linux_ssh_mips_malta.py | 78 ++++++++----------------
 tests/requirements.txt                   |  3 +-
 2 files changed, 25 insertions(+), 56 deletions(-)

diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/acceptance/linux_ssh_mips_malta.py
index 1d334432a0..25a1df5098 100644
--- a/tests/acceptance/linux_ssh_mips_malta.py
+++ b/tests/acceptance/linux_ssh_mips_malta.py
@@ -9,13 +9,13 @@ import os
 import re
 import base64
 import logging
-import paramiko
 import time
 
 from avocado import skipUnless
 from avocado_qemu import Test
 from avocado.utils import process
 from avocado.utils import archive
+from avocado.utils import ssh
 
 
 class LinuxSSH(Test):
@@ -26,35 +26,19 @@ class LinuxSSH(Test):
     VM_IP = '127.0.0.1'
 
     IMAGE_INFO = {
-        'be': {
-            'image_url': 'https://people.debian.org/~aurel32/qemu/mips/'
-                         'debian_wheezy_mips_standard.qcow2',
-            'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5',
-            'rsa_hostkey': b'AAAAB3NzaC1yc2EAAAADAQABAAABAQCca1VitiyLAdQOld'
-                           b'zT43IOEVJZ0wHD78GJi8wDAjMiYWUzNSSn0rXGQsINHuH5'
-                           b'IlF+kBZsHinb/FtKCAyS9a8uCHhQI4SuB4QhAb0+39MlUw'
-                           b'Mm0CLkctgM2eUUZ6MQMQvDlqnue6CCkxN62EZYbaxmby7j'
-                           b'CQa1125o1HRKBvdGm2zrJWxXAfA+f1v6jHLyE8Jnu83eQ+'
-                           b'BFY25G+Vzx1PVc3zQBwJ8r0NGTRqy2//oWQP0h+bMsgeFe'
-                           b'KH/J3RJM22vg6+I4JAdBFcxnK+l781h1FuRxOn4O/Xslbg'
-                           b'go6WtB4V4TOsw2E/KfxI5IZ/icxF+swVcnvF46Hf3uQc/0'
-                           b'BBqb',
-        },
-        'le': {
-            'image_url': 'https://people.debian.org/~aurel32/qemu/mipsel/'
-                         'debian_wheezy_mipsel_standard.qcow2',
-            'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802',
-            'rsa_hostkey': b'AAAAB3NzaC1yc2EAAAADAQABAAABAQClXJlBT71HL5yKvv'
-                           b'gfC7jmxSWx5zSBCzET6CLZczwAafSIs7YKfNOy/dQTxhuk'
-                           b'yIGFUugZFoF3E9PzdhunuyvyTd56MPoNIqFbb5rGokwU5I'
-                           b'TOx3dBHZR0mClypL6MVrwe0bsiIb8GhF1zioNwcsaAZnAi'
-                           b'KfXStVDtXvn/kLLq+xLABYt48CC5KYWoFaCoICskLAY+qo'
-                           b'L+LWyAnQisj4jAH8VSaSKIImFpfkHWEXPhHcC4ZBlDKtnH'
-                           b'po9vhfCHgnfW3Pzrqmk8BI4HysqPFVmJWkJGlGUL+sGeg3'
-                           b'ZZolAYuDXGuBrw8ooPJq2v2dOH+z6dyD2q/ypmAbyPqj5C'
-                           b'rc8H',
-        },
-    }
+        'be': {'image_url': ('https://people.debian.org/~aurel32/qemu/mips/'
+                             'debian_wheezy_mips_standard.qcow2'),
+               'image_hash': '8987a63270df67345b2135a6b7a4885a35e392d5'},
+        'le': {'image_url': ('https://people.debian.org/~aurel32/qemu/mipsel/'
+                             'debian_wheezy_mipsel_standard.qcow2'),
+               'image_hash': '7866764d9de3ef536ffca24c9fb9f04ffdb45802'}
+        }
+
+
+    @skipUnless(ssh.SSH_CLIENT_BINARY, 'No SSH client available')
+    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
+    def setUp(self):
+        super(LinuxSSH, self).setUp()
 
     def wait_for_console_pattern(self, success_message,
                                  failure_message='Oops'):
@@ -78,23 +62,14 @@ class LinuxSSH(Test):
         self.log.debug("sshd listening on port:" + port)
         return port
 
-    def ssh_connect(self, username, password, rsa_hostkey_b64=None):
+    def ssh_connect(self, username, password):
         self.ssh_logger = logging.getLogger('ssh')
-        self.ssh_username = username
-        self.ssh_ps1 = '# ' if username is 'root' else '$ '
-        self.ssh_client = paramiko.SSHClient()
         port = self.get_portfwd()
-        if rsa_hostkey_b64:
-            rsa_hostkey_bin = base64.b64decode(rsa_hostkey_b64)
-            rsa_hostkey = paramiko.RSAKey(data = rsa_hostkey_bin)
-            ipport = '[%s]:%s' % (self.VM_IP, port)
-            self.ssh_logger.debug('ipport ' + ipport)
-            self.ssh_client.get_host_keys().add(ipport, 'ssh-rsa', rsa_hostkey)
+        self.ssh_session = ssh.Session(self.VM_IP, port=int(port),
+                                       user=username, password=password)
         for i in range(10):
             try:
-                self.ssh_client.connect(self.VM_IP, int(port),
-                                        username, password, banner_timeout=90)
-                self.ssh_logger.info("Entering interactive session.")
+                self.ssh_session.connect()
                 return
             except:
                 time.sleep(4)
@@ -102,15 +77,15 @@ class LinuxSSH(Test):
         self.fail("sshd timeout")
 
     def ssh_disconnect_vm(self):
-        self.ssh_client.close()
+        self.ssh_session.quit()
 
     def ssh_command(self, command, is_root=True):
-        self.ssh_logger.info(self.ssh_ps1 + command)
-        stdin, stdout, stderr = self.ssh_client.exec_command(command)
-        stdout_lines = [line.strip('\n') for line in stdout]
+        self.ssh_logger.info(command)
+        result = self.ssh_session.cmd(command)
+        stdout_lines = [line.rstrip() for line in result.stdout_text.splitlines()]
         for line in stdout_lines:
             self.ssh_logger.info(line)
-        stderr_lines = [line.strip('\n') for line in stderr]
+        stderr_lines = [line.rstrip() for line in result.stderr_text.splitlines()]
         for line in stderr_lines:
             self.ssh_logger.warning(line)
         return stdout_lines, stderr_lines
@@ -119,7 +94,6 @@ class LinuxSSH(Test):
         image_url = self.IMAGE_INFO[endianess]['image_url']
         image_hash = self.IMAGE_INFO[endianess]['image_hash']
         image_path = self.fetch_asset(image_url, asset_hash=image_hash)
-        rsa_hostkey_b64 = self.IMAGE_INFO[endianess]['rsa_hostkey']
 
         self.vm.set_machine('malta')
         self.vm.set_console()
@@ -138,7 +112,7 @@ class LinuxSSH(Test):
         self.wait_for_console_pattern(console_pattern)
         self.log.info('sshd ready')
 
-        self.ssh_connect('root', 'root', rsa_hostkey_b64=rsa_hostkey_b64)
+        self.ssh_connect('root', 'root')
 
     def shutdown_via_ssh(self):
         self.ssh_command('poweroff')
@@ -222,7 +196,6 @@ class LinuxSSH(Test):
         self.run_common_commands()
         self.shutdown_via_ssh()
 
-    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
     def test_mips_malta32eb_kernel3_2_0(self):
         """
         :avocado: tags=arch:mips
@@ -237,7 +210,6 @@ class LinuxSSH(Test):
 
         self.check_mips_malta('be', kernel_path, 'mips')
 
-    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
     def test_mips_malta32el_kernel3_2_0(self):
         """
         :avocado: tags=arch:mipsel
@@ -252,7 +224,6 @@ class LinuxSSH(Test):
 
         self.check_mips_malta('le', kernel_path, 'mips')
 
-    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
     def test_mips_malta64eb_kernel3_2_0(self):
         """
         :avocado: tags=arch:mips64
@@ -266,7 +237,6 @@ class LinuxSSH(Test):
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
         self.check_mips_malta('be', kernel_path, 'mips64')
 
-    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
     def test_mips_malta64el_kernel3_2_0(self):
         """
         :avocado: tags=arch:mips64el
diff --git a/tests/requirements.txt b/tests/requirements.txt
index bd1f7590ed..a2a587223a 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,4 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==68.0
-paramiko==2.4.2
+avocado-framework==72.0
-- 
2.21.0

Re: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190923134019.8548-1-crosa@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20190923134019.8548-1-crosa@redhat.com
Subject: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
b200b91 Acceptance tests: use avocado.utils.ssh for SSH interaction
333b8f1 tests/acceptance: Add new test cases in linux_ssh_mips_malta.py
cb9e32e tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py
c5b436a Acceptance test x86_cpu_model_versions: split into smaller tests
eb9f595 Acceptance test x86_cpu_model_versions: fix mismatches between test and messages
8bbd882 Acceptance test x86_cpu_model_versions: shutdown VMs
c5f67b5 Acceptance test machine_m68k_nextcube.py: relax the error code pattern

=== OUTPUT BEGIN ===
1/7 Checking commit c5f67b555dad (Acceptance test machine_m68k_nextcube.py: relax the error code pattern)
2/7 Checking commit 8bbd8828cf1e (Acceptance test x86_cpu_model_versions: shutdown VMs)
3/7 Checking commit eb9f595504b2 (Acceptance test x86_cpu_model_versions: fix mismatches between test and messages)
ERROR: line over 90 characters
#25: FILE: tests/acceptance/x86_cpu_model_versions.py:283:
+                         'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')

ERROR: line over 90 characters
#34: FILE: tests/acceptance/x86_cpu_model_versions.py:292:
+                         'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')

ERROR: line over 90 characters
#42: FILE: tests/acceptance/x86_cpu_model_versions.py:299:
+        vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')

total: 3 errors, 0 warnings, 24 lines checked

Patch 3/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/7 Checking commit c5b436a0666a (Acceptance test x86_cpu_model_versions: split into smaller tests)
5/7 Checking commit cb9e32eca764 (tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py)
6/7 Checking commit 333b8f13665e (tests/acceptance: Add new test cases in linux_ssh_mips_malta.py)
7/7 Checking commit b200b916e0e8 (Acceptance tests: use avocado.utils.ssh for SSH interaction)
WARNING: line over 80 characters
#126: FILE: tests/acceptance/linux_ssh_mips_malta.py:85:
+        stdout_lines = [line.rstrip() for line in result.stdout_text.splitlines()]

WARNING: line over 80 characters
#130: FILE: tests/acceptance/linux_ssh_mips_malta.py:88:
+        stderr_lines = [line.rstrip() for line in result.stderr_text.splitlines()]

total: 0 errors, 2 warnings, 158 lines checked

Patch 7/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190923134019.8548-1-crosa@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190923134019.8548-1-crosa@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20190923134019.8548-1-crosa@redhat.com
Subject: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
95d8aed Acceptance tests: use avocado.utils.ssh for SSH interaction
b5d62a1 tests/acceptance: Add new test cases in linux_ssh_mips_malta.py
41b8da1 tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py
9dbf0de Acceptance test x86_cpu_model_versions: split into smaller tests
a3bdbf3 Acceptance test x86_cpu_model_versions: fix mismatches between test and messages
4423807 Acceptance test x86_cpu_model_versions: shutdown VMs
e3d4392 Acceptance test machine_m68k_nextcube.py: relax the error code pattern

=== OUTPUT BEGIN ===
1/7 Checking commit e3d43927ffb3 (Acceptance test machine_m68k_nextcube.py: relax the error code pattern)
2/7 Checking commit 44238078fcde (Acceptance test x86_cpu_model_versions: shutdown VMs)
3/7 Checking commit a3bdbf35dd33 (Acceptance test x86_cpu_model_versions: fix mismatches between test and messages)
ERROR: line over 90 characters
#25: FILE: tests/acceptance/x86_cpu_model_versions.py:283:
+                         'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')

ERROR: line over 90 characters
#34: FILE: tests/acceptance/x86_cpu_model_versions.py:292:
+                         'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')

ERROR: line over 90 characters
#42: FILE: tests/acceptance/x86_cpu_model_versions.py:299:
+        vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')

total: 3 errors, 0 warnings, 24 lines checked

Patch 3/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/7 Checking commit 9dbf0de5f7d5 (Acceptance test x86_cpu_model_versions: split into smaller tests)
5/7 Checking commit 41b8da1921af (tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py)
6/7 Checking commit b5d62a1def54 (tests/acceptance: Add new test cases in linux_ssh_mips_malta.py)
7/7 Checking commit 95d8aed259c8 (Acceptance tests: use avocado.utils.ssh for SSH interaction)
WARNING: line over 80 characters
#126: FILE: tests/acceptance/linux_ssh_mips_malta.py:85:
+        stdout_lines = [line.rstrip() for line in result.stdout_text.splitlines()]

WARNING: line over 80 characters
#130: FILE: tests/acceptance/linux_ssh_mips_malta.py:88:
+        stderr_lines = [line.rstrip() for line in result.stderr_text.splitlines()]

total: 0 errors, 2 warnings, 158 lines checked

Patch 7/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190923134019.8548-1-crosa@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190923134019.8548-1-crosa@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20190923134019.8548-1-crosa@redhat.com
Subject: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
586193c Acceptance tests: use avocado.utils.ssh for SSH interaction
73df413 tests/acceptance: Add new test cases in linux_ssh_mips_malta.py
cfc25f3 tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py
4661118 Acceptance test x86_cpu_model_versions: split into smaller tests
2fb1036 Acceptance test x86_cpu_model_versions: fix mismatches between test and messages
f5e5d96 Acceptance test x86_cpu_model_versions: shutdown VMs
0c13726 Acceptance test machine_m68k_nextcube.py: relax the error code pattern

=== OUTPUT BEGIN ===
1/7 Checking commit 0c137262db6e (Acceptance test machine_m68k_nextcube.py: relax the error code pattern)
2/7 Checking commit f5e5d9617fe9 (Acceptance test x86_cpu_model_versions: shutdown VMs)
3/7 Checking commit 2fb10362a8ed (Acceptance test x86_cpu_model_versions: fix mismatches between test and messages)
ERROR: line over 90 characters
#25: FILE: tests/acceptance/x86_cpu_model_versions.py:283:
+                         'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')

ERROR: line over 90 characters
#34: FILE: tests/acceptance/x86_cpu_model_versions.py:292:
+                         'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')

ERROR: line over 90 characters
#42: FILE: tests/acceptance/x86_cpu_model_versions.py:299:
+        vm.add_args('-cpu', 'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')

total: 3 errors, 0 warnings, 24 lines checked

Patch 3/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/7 Checking commit 4661118c3be7 (Acceptance test x86_cpu_model_versions: split into smaller tests)
5/7 Checking commit cfc25f3d2559 (tests/acceptance: Refactor and improve reporting in linux_ssh_mips_malta.py)
6/7 Checking commit 73df413e17b2 (tests/acceptance: Add new test cases in linux_ssh_mips_malta.py)
7/7 Checking commit 586193c82d14 (Acceptance tests: use avocado.utils.ssh for SSH interaction)
WARNING: line over 80 characters
#126: FILE: tests/acceptance/linux_ssh_mips_malta.py:85:
+        stdout_lines = [line.rstrip() for line in result.stdout_text.splitlines()]

WARNING: line over 80 characters
#130: FILE: tests/acceptance/linux_ssh_mips_malta.py:88:
+        stderr_lines = [line.rstrip() for line in result.stderr_text.splitlines()]

total: 0 errors, 2 warnings, 158 lines checked

Patch 7/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190923134019.8548-1-crosa@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PULL 0/7] Python (acceptance tests) queue - 2019-09-23

[Peter Maydell]

On Mon, 23 Sep 2019 at 14:40, Cleber Rosa <crosa@redhat.com> wrote:
>
> The following changes since commit 4300b7c2cd9f3f273804e8cca325842ccb93b1ad:
>
>   Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging (2019-09-20 17:28:43 +0100)
>
> are available in the Git repository at:
>
>   git://github.com/clebergnu/qemu.git tags/python-next-pull-request
>
> for you to fetch changes up to 2fe6f4d9babb957244676fdf23b9a7ca2323034b:
>
>   Acceptance tests: use avocado.utils.ssh for SSH interaction (2019-09-20 17:13:41 -0400)
>
> ----------------------------------------------------------------
> Improvements include:
>
>  * Relaxed error code pattern on machine_m68k_nextcube.py
>  * Better naming and separation of tests in x86_cpu_model_versions.py
>  * New checks on guest side on linux_ssh_mips_malta.py
>  * Use of avocado.utils.ssh, and thus "/usr/bin/ssh" for SSH
>    interaction
>
> A Travis-CI check for this branch can be seen at:
>
>  * https://travis-ci.org/clebergnu/qemu/builds/587665790
>

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2
for any user-visible changes.

-- PMM