From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BDB0C433DB for ; Fri, 26 Mar 2021 08:08:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 72A4361A1D for ; Fri, 26 Mar 2021 08:07:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 72A4361A1D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39834 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lPhVe-0006G5-8D for qemu-devel@archiver.kernel.org; Fri, 26 Mar 2021 04:07:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57660) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lPhUK-0005DW-Hn for qemu-devel@nongnu.org; Fri, 26 Mar 2021 04:06:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:32671) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lPhUH-0004w0-3u for qemu-devel@nongnu.org; Fri, 26 Mar 2021 04:06:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616745991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LvAgeUOhw4hluahx5Egnzxh2FVg5E9kmJ3VtmQbDDjo=; b=hi/dy00UvoZMCVz9VrTKuDMY7uoHXm4MJutJIankmByckoaOk/IWbuChTjNnjYDXZPM3yV GfqC6K5HadGc6+x8WqYU46gxEcGTaMo0uqKkYAXRHQpkxxgqBLhbVyopKESzX8sNMYKYuZ XMxlI6a9mZoVXe7UfRsrblLn1sFRjN8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-390--B5pMNwpMtmbruEHLDXBMg-1; Fri, 26 Mar 2021 04:06:27 -0400 X-MC-Unique: -B5pMNwpMtmbruEHLDXBMg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B32A28018A1; Fri, 26 Mar 2021 08:06:26 +0000 (UTC) Received: from [10.36.112.13] (ovpn-112-13.ams2.redhat.com [10.36.112.13]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C694467894; Fri, 26 Mar 2021 08:06:12 +0000 (UTC) Subject: Re: [PATCH v2 00/10] Acceptance Test: introduce base class for Linux based tests To: Wainer dos Santos Moschetta , Cleber Rosa , qemu-devel@nongnu.org References: <20210323221539.3532660-1-crosa@redhat.com> <05118984-4a12-6cc5-ab84-a3a4953b535c@redhat.com> From: Auger Eric Message-ID: Date: Fri, 26 Mar 2021 09:06:10 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <05118984-4a12-6cc5-ab84-a3a4953b535c@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eric.auger@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=63.128.21.124; envelope-from=eric.auger@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Aleksandar Rikalo , Eduardo Habkost , =?UTF-8?Q?Alex_Benn=c3=a9e?= , =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= , Max Reitz , Willian Rampazzo , John Snow , Eric Auger , Willian Rampazzo , Thomas Huth , =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= , =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= , Aurelien Jarno , Beraldo Leal Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi Wainer, On 3/25/21 8:45 PM, Wainer dos Santos Moschetta wrote: > Hi, > > On 3/23/21 7:15 PM, Cleber Rosa wrote: >> This introduces a base class for tests that need to interact with a >> Linux guest.  It generalizes the "boot_linux.py" code, already been >> used by the "virtiofs_submounts.py" and also SSH related code being >> used by that and "linux_ssh_mips_malta.py". > > I ran the linux_ssh_mips_malta.py tests, they all passed: > > (11/34) > tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32eb_kernel3_2_0: > PASS (64.41 s) > (12/34) > tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32el_kernel3_2_0: > PASS (63.43 s) > (13/34) > tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64eb_kernel3_2_0: > PASS (63.76 s) > (14/34) > tests/acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64el_kernel3_2_0: > PASS (62.52 s) > > Then I tried the virtiofs_submounts.py tests, it finishes with error. > Something like that fixes it: > > diff --git a/tests/acceptance/virtiofs_submounts.py > b/tests/acceptance/virtiofs_submounts.py > index d77ee35674..21ad7d792e 100644 > --- a/tests/acceptance/virtiofs_submounts.py > +++ b/tests/acceptance/virtiofs_submounts.py > @@ -195,7 +195,7 @@ def setUp(self): > >          self.run(('ssh-keygen', '-N', '', '-t', 'ed25519', '-f', > self.ssh_key)) > > -        pubkey = open(self.ssh_key + '.pub').read() > +        pubkey = self.ssh_key + '.pub' Yes I discovered that too when developping the SMMU test. Thanks for mentionning Eric > >          super(VirtiofsSubmountsTest, self).setUp(pubkey) > > >> >> While at it, a number of fixes on hopeful improvements to those tests >> were added. >> >> Changes from v1: >> >> * Majority of v1 patches have been merged. >> >> * New patches: >>    - Acceptance Tests: make username/password configurable >>    - Acceptance Tests: set up SSH connection by default after boot for >> LinuxTest >>    - tests/acceptance/virtiofs_submounts.py: remove launch_vm() >> >> * Allowed for the configuration of the network device type (defaulting >>    to virtio-net) [Phil] >> >> * Fix module name typo (s/qemu.util/qemu.utils/) in the commit message >>    [John] >> >> * Tests based on LinuxTest will have the SSH connection already prepared >> >> Cleber Rosa (10): >>    tests/acceptance/virtiofs_submounts.py: add missing accel tag >>    tests/acceptance/virtiofs_submounts.py: evaluate string not length >>    Python: add utility function for retrieving port redirection >>    Acceptance Tests: move useful ssh methods to base class >>    Acceptance Tests: add port redirection for ssh by default >>    Acceptance Tests: make username/password configurable >>    Acceptance Tests: set up SSH connection by default after boot for >>      LinuxTest >>    tests/acceptance/virtiofs_submounts.py: remove launch_vm() >>    Acceptance Tests: add basic documentation on LinuxTest base class >>    Acceptance Tests: introduce CPU hotplug test >> >>   docs/devel/testing.rst                    | 25 ++++++++ >>   python/qemu/utils.py                      | 35 ++++++++++++ >>   tests/acceptance/avocado_qemu/__init__.py | 63 +++++++++++++++++++-- >>   tests/acceptance/hotplug_cpu.py           | 37 ++++++++++++ >>   tests/acceptance/info_usernet.py          | 29 ++++++++++ >>   tests/acceptance/linux_ssh_mips_malta.py  | 44 ++------------- >>   tests/acceptance/virtiofs_submounts.py    | 69 +++-------------------- >>   tests/vm/basevm.py                        |  7 +-- >>   8 files changed, 198 insertions(+), 111 deletions(-) >>   create mode 100644 python/qemu/utils.py >>   create mode 100644 tests/acceptance/hotplug_cpu.py >>   create mode 100644 tests/acceptance/info_usernet.py >>