qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
@ 2019-06-24 13:43 Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 01/19] tests/vm: avoid extra compressed image copy Alex Bennée
                   ` (19 more replies)
  0 siblings, 20 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell; +Cc: Alex Bennée, qemu-devel

The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:

  Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-next-240619-1

for you to fetch changes up to e0fe22c3528773fcbfd135a0ef7f6b3c5d373bb9:

  target/i386: fix feature check in hyperv-stub.c (2019-06-24 14:36:39 +0100)

----------------------------------------------------------------
Various testing fixes:

   - tests/vm updates and clean-ups
   - tests/vm serial autobuild on host
   - ensure MacOS builds do "brew update"
   - ensure we test --static user builds
   - fix hyperv compile failure

This brings my testing back to green on all CI services.

----------------------------------------------------------------
Alex Bennée (3):
      .travis.yml: default the --disable-system build to --static
      .travis.yml: force a brew update for MacOS builds
      target/i386: fix feature check in hyperv-stub.c

Cleber Rosa (4):
      tests/vm: avoid extra compressed image copy
      tests/vm: avoid image presence check and removal
      tests/vm: pin ubuntu.i386 image
      tests/vm: add source repos on ubuntu.i386

Gerd Hoffmann (11):
      tests/vm: send proxy environment variables over ssh
      tests/vm: use ssh with pty unconditionally
      tests/vm: run test builds on snapshot
      tests/vm: proper guest shutdown
      tests/vm: add vm-boot-{ssh,serial}-<guest> targets
      tests/vm: serial console support helpers
      tests/vm: openbsd autoinstall, using serial console
      tests/vm: freebsd autoinstall, using serial console
      tests/vm: netbsd autoinstall, using serial console
      tests/vm: fedora autoinstall, using serial console
      tests/vm: ubuntu.i386: apt proxy setup

Philippe Mathieu-Daudé (1):
      Makefile: Rename the 'vm-test' target as 'vm-help'

 .travis.yml               |   3 +-
 Makefile                  |   4 +-
 docs/devel/testing.rst    |   4 +-
 target/i386/hyperv-stub.c |   2 +-
 tests/vm/Makefile.include |  28 ++++++-
 tests/vm/basevm.py        | 135 ++++++++++++++++++++++++++++-----
 tests/vm/centos           |   6 +-
 tests/vm/fedora           | 189 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/vm/freebsd          | 180 ++++++++++++++++++++++++++++++++++++++++---
 tests/vm/netbsd           | 187 ++++++++++++++++++++++++++++++++++++++++++---
 tests/vm/openbsd          | 159 ++++++++++++++++++++++++++++++++++----
 tests/vm/ubuntu.i386      |  11 ++-
 12 files changed, 841 insertions(+), 67 deletions(-)
 create mode 100755 tests/vm/fedora

--
2.20.1



^ permalink raw reply	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 01/19] tests/vm: avoid extra compressed image copy
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 02/19] tests/vm: avoid image presence check and removal Alex Bennée
                   ` (18 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Cleber Rosa

From: Cleber Rosa <crosa@redhat.com>

The image copy is only really needed because xz doesn't know to
properly decompress a file not named properly.  Instead of
decompressing to stdout, and having to rely on a shell, let's just
create a link instead of copying the file.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190613130718.3763-2-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/centos b/tests/vm/centos
index 7417b50af4..b00b46a8dc 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -66,8 +66,8 @@ class CentosVM(basevm.BaseVM):
         cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp + ".xz"])
-        subprocess.check_call(["xz", "-dvf", img_tmp + ".xz"])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
         subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
         self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
         self.wait_ssh()
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index b0066017a6..5575c23a6f 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -34,8 +34,8 @@ class FreeBSDVM(basevm.BaseVM):
         img_tmp_xz = img + ".tmp.xz"
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
         if os.path.exists(img):
             os.remove(img)
         os.rename(img_tmp, img)
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 4c6624ea5e..d0508f4465 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -34,8 +34,8 @@ class NetBSDVM(basevm.BaseVM):
         img_tmp_xz = img + ".tmp.xz"
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
         if os.path.exists(img):
             os.remove(img)
         os.rename(img_tmp, img)
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 2105c01a26..87ec982489 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -36,8 +36,8 @@ class OpenBSDVM(basevm.BaseVM):
         img_tmp_xz = img + ".tmp.xz"
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
         if os.path.exists(img):
             os.remove(img)
         os.rename(img_tmp, img)
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 02/19] tests/vm: avoid image presence check and removal
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 01/19] tests/vm: avoid extra compressed image copy Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 03/19] tests/vm: pin ubuntu.i386 image Alex Bennée
                   ` (17 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Cleber Rosa

From: Cleber Rosa <crosa@redhat.com>

Python's os.rename() will silently replace an existing file,
so there's no need for the extra check and removal.

Reference: https://docs.python.org/3/library/os.html#os.rename

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190613130718.3763-3-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/centos b/tests/vm/centos
index b00b46a8dc..53976f1c4c 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -77,8 +77,6 @@ class CentosVM(basevm.BaseVM):
         self.ssh_root_check("systemctl enable docker")
         self.ssh_root("poweroff")
         self.wait()
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
         return 0
 
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 5575c23a6f..091be1a065 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -36,8 +36,6 @@ class FreeBSDVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index d0508f4465..ee9eaeab50 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -36,8 +36,6 @@ class NetBSDVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 87ec982489..28c7d25e29 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -38,8 +38,6 @@ class OpenBSDVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index a22d137e76..12867b193f 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -80,8 +80,6 @@ class UbuntuX86VM(basevm.BaseVM):
         self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
         self.ssh_root("poweroff")
         self.wait()
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
         return 0
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 03/19] tests/vm: pin ubuntu.i386 image
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 01/19] tests/vm: avoid extra compressed image copy Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 02/19] tests/vm: avoid image presence check and removal Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 04/19] tests/vm: add source repos on ubuntu.i386 Alex Bennée
                   ` (16 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Cleber Rosa

From: Cleber Rosa <crosa@redhat.com>

It's a good practice to always have the same components used in tests.
According to:

   https://cloud-images.ubuntu.com/releases/16.04/

New images are released from time to time, and the "release/"
directory points to the latest release.  Let's pin to the latest
available version, and while at it, set a hash for verification.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190613130718.3763-4-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 12867b193f..7017e6d388 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -61,7 +61,9 @@ class UbuntuX86VM(basevm.BaseVM):
         return os.path.join(cidir, "cloud-init.iso")
 
     def build_image(self, img):
-        cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
+        cimg = self._download_with_cache(
+            "https://cloud-images.ubuntu.com/releases/16.04/release-20190605/ubuntu-16.04-server-cloudimg-i386-disk1.img",
+            sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
         img_tmp = img + ".tmp"
         subprocess.check_call(["cp", "-f", cimg, img_tmp])
         subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 04/19] tests/vm: add source repos on ubuntu.i386
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (2 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 03/19] tests/vm: pin ubuntu.i386 image Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 05/19] tests/vm: send proxy environment variables over ssh Alex Bennée
                   ` (15 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Cleber Rosa

From: Cleber Rosa <crosa@redhat.com>

Possibly because of different behavior on the newly update
cloud-image, trying to run 'apt-get build-dep' results in:

   E: You must put some 'source' URIs in your sources.list

This enables all source repos (even though some are not
needed) for simplicity sake.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190613130718.3763-5-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 7017e6d388..3ea459ee20 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -77,6 +77,7 @@ class UbuntuX86VM(basevm.BaseVM):
         time.sleep(5)
         self.wait_ssh()
         # The previous update sometimes doesn't survive a reboot, so do it again
+        self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
         self.ssh_root_check("apt-get update")
         self.ssh_root_check("apt-get build-dep -y qemu")
         self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 05/19] tests/vm: send proxy environment variables over ssh
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (3 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 04/19] tests/vm: add source repos on ubuntu.i386 Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 06/19] tests/vm: use ssh with pty unconditionally Alex Bennée
                   ` (14 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

Packages are fetched via proxy that way, if configured on the host.
That might be required to pass firewalls, and it allows to route
package downloads through a caching proxy server.

Needs AcceptEnv setup in sshd_config on the guest side to work.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-2-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 4847549592..5e30bac661 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -38,6 +38,13 @@ class BaseVM(object):
     GUEST_PASS = "qemupass"
     ROOT_PASS = "qemupass"
 
+    envvars = [
+        "https_proxy",
+        "http_proxy",
+        "ftp_proxy",
+        "no_proxy",
+    ]
+
     # The script to run in the guest that builds QEMU
     BUILD_SCRIPT = ""
     # The guest name, to be overridden by subclasses
@@ -106,6 +113,8 @@ class BaseVM(object):
                    "-o", "UserKnownHostsFile=" + os.devnull,
                    "-o", "ConnectTimeout=1",
                    "-p", self.ssh_port, "-i", self._ssh_key_file]
+        for var in self.envvars:
+            ssh_cmd += ['-o', "SendEnv=%s" % var ]
         if interactive:
             ssh_cmd += ['-t']
         assert not isinstance(cmd, str)
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 06/19] tests/vm: use ssh with pty unconditionally
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (4 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 05/19] tests/vm: send proxy environment variables over ssh Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 07/19] tests/vm: run test builds on snapshot Alex Bennée
                   ` (13 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

Allways ask ssh to run with a pseudo terminal.
Not having a terminal causes problems now and then.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-3-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 5e30bac661..395eefaec9 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -107,16 +107,14 @@ class BaseVM(object):
         os.rename(fname + ".download", fname)
         return fname
 
-    def _ssh_do(self, user, cmd, check, interactive=False):
-        ssh_cmd = ["ssh", "-q",
+    def _ssh_do(self, user, cmd, check):
+        ssh_cmd = ["ssh", "-q", "-t",
                    "-o", "StrictHostKeyChecking=no",
                    "-o", "UserKnownHostsFile=" + os.devnull,
                    "-o", "ConnectTimeout=1",
                    "-p", self.ssh_port, "-i", self._ssh_key_file]
         for var in self.envvars:
             ssh_cmd += ['-o', "SendEnv=%s" % var ]
-        if interactive:
-            ssh_cmd += ['-t']
         assert not isinstance(cmd, str)
         ssh_cmd += ["%s@127.0.0.1" % user] + list(cmd)
         logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
@@ -128,9 +126,6 @@ class BaseVM(object):
     def ssh(self, *cmd):
         return self._ssh_do(self.GUEST_USER, cmd, False)
 
-    def ssh_interactive(self, *cmd):
-        return self._ssh_do(self.GUEST_USER, cmd, False, True)
-
     def ssh_root(self, *cmd):
         return self._ssh_do("root", cmd, False)
 
@@ -284,9 +279,9 @@ def main(vmcls):
         return 2
 
     if args.interactive:
-        if vm.ssh_interactive(*cmd) == 0:
+        if vm.ssh(*cmd) == 0:
             return 0
-        vm.ssh_interactive()
+        vm.ssh()
         return 3
     else:
         if vm.ssh(*cmd) != 0:
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 07/19] tests/vm: run test builds on snapshot
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (5 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 06/19] tests/vm: use ssh with pty unconditionally Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 08/19] tests/vm: proper guest shutdown Alex Bennée
                   ` (12 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Ed Maste, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

The build script doesn't shutdown the guest VMs properly,
which results in filesystem corruption and guest boot
failures sooner or later.

Use the --snapshot to run builds on a snapshot,
That way killing the VM doesn't corrupt the base image.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Acked-by: Ed Maste <emaste@freebsd.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-4-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index c59411bee0..276b870216 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -57,6 +57,7 @@ vm-build-%: $(IMAGES_DIR)/%.img
 		$(if $(V),--verbose) \
 		--image "$<" \
 		$(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \
+		--snapshot \
 		--build-qemu $(SRC_PATH) -- \
 		$(if $(TARGET_LIST),--target-list=$(TARGET_LIST)) \
 		$(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 08/19] tests/vm: proper guest shutdown
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (6 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 07/19] tests/vm: run test builds on snapshot Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 09/19] tests/vm: add vm-boot-{ssh, serial}-<guest> targets Alex Bennée
                   ` (11 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

When not running in snapshot mode ask the guest to poweroff and wait for
this to finish instead of simply quitting qemu, so the guest can flush
pending updates to disk.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-5-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 395eefaec9..f27178f3c7 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -51,6 +51,8 @@ class BaseVM(object):
     name = "#base"
     # The guest architecture, to be overridden by subclasses
     arch = "#arch"
+    # command to halt the guest, can be overridden by subclasses
+    poweroff = "poweroff"
     def __init__(self, debug=False, vcpus=None):
         self._guest = None
         self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
@@ -202,6 +204,10 @@ class BaseVM(object):
     def wait(self):
         self._guest.wait()
 
+    def graceful_shutdown(self):
+        self.ssh_root(self.poweroff)
+        self._guest.wait()
+
     def qmp(self, *args, **kwargs):
         return self._guest.qmp(*args, **kwargs)
 
@@ -278,11 +284,13 @@ def main(vmcls):
         traceback.print_exc()
         return 2
 
-    if args.interactive:
-        if vm.ssh(*cmd) == 0:
-            return 0
+    exitcode = 0
+    if vm.ssh(*cmd) != 0:
+        exitcode = 3
+    if exitcode != 0 and args.interactive:
         vm.ssh()
-        return 3
-    else:
-        if vm.ssh(*cmd) != 0:
-            return 3
+
+    if not args.snapshot:
+        vm.graceful_shutdown()
+
+    return exitcode
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 09/19] tests/vm: add vm-boot-{ssh, serial}-<guest> targets
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (7 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 08/19] tests/vm: proper guest shutdown Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 10/19] tests/vm: serial console support helpers Alex Bennée
                   ` (10 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

For testing/troubleshooting convenience.

make vm-boot-serial-<guest>
  Boot guest, with the serial console on stdio.

make vm-boot-ssh-<guest>
  Boot guest, login via ssh.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-6-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 276b870216..a61c236b8e 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -20,6 +20,10 @@ vm-test:
 	@echo "  vm-build-all                    - Build QEMU in all VMs"
 	@echo "  vm-clean-all                    - Clean up VM images"
 	@echo
+	@echo "For trouble-shooting:"
+	@echo "  vm-boot-serial-<guest>          - Boot guest, serial console on stdio"
+	@echo "  vm-boot-ssh-<guest>             - Boot guest and login via ssh"
+	@echo
 	@echo "Special variables:"
 	@echo "    BUILD_TARGET=foo		 - Override the build target"
 	@echo "    TARGET_LIST=a,b,c    	 - Override target list in builds"
@@ -63,3 +67,18 @@ vm-build-%: $(IMAGES_DIR)/%.img
 		$(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
 		"  VM-BUILD $*")
 
+vm-boot-serial-%: $(IMAGES_DIR)/%.img
+	qemu-system-x86_64 -enable-kvm -m 4G -smp 2 -nographic \
+		-drive if=none,id=vblk,cache=writeback,file="$<" \
+		-netdev user,id=vnet \
+		-device virtio-blk-pci,drive=vblk \
+		-device virtio-net-pci,netdev=vnet \
+	|| true
+
+vm-boot-ssh-%: $(IMAGES_DIR)/%.img
+	$(call quiet-command, \
+		$(SRC_PATH)/tests/vm/$* \
+		--image "$<" \
+		--interactive \
+		false, \
+		"  VM-BOOT-SSH $*") || true
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 10/19] tests/vm: serial console support helpers
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (8 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 09/19] tests/vm: add vm-boot-{ssh, serial}-<guest> targets Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 11/19] tests/vm: openbsd autoinstall, using serial console Alex Bennée
                   ` (9 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

Add a bunch of helpers to talk to the guest using the
serial console.

Also drop the hard-coded -serial parameter for the vm
so QEMUMachine.set_console() actually works.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-7-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index f27178f3c7..592a344773 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -2,10 +2,11 @@
 #
 # VM testing base class
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng <famz@redhat.com>
+#  Gerd Hoffmann <kraxel@redhat.com>
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
@@ -13,7 +14,9 @@
 
 from __future__ import print_function
 import os
+import re
 import sys
+import socket
 import logging
 import time
 import datetime
@@ -79,8 +82,7 @@ class BaseVM(object):
             "-cpu", "max",
             "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
             "-device", "virtio-net-pci,netdev=vnet",
-            "-vnc", "127.0.0.1:0,to=20",
-            "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
+            "-vnc", "127.0.0.1:0,to=20"]
         if vcpus and vcpus > 1:
             self._args += ["-smp", "%d" % vcpus]
         if kvm_available(self.arch):
@@ -162,6 +164,8 @@ class BaseVM(object):
         logging.debug("QEMU args: %s", " ".join(args))
         qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch)
         guest = QEMUMachine(binary=qemu_bin, args=args)
+        guest.set_machine('pc')
+        guest.set_console()
         try:
             guest.launch()
         except:
@@ -184,6 +188,82 @@ class BaseVM(object):
             raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \
                             usernet_info)
 
+    def console_init(self, timeout = 120):
+        vm = self._guest
+        vm.console_socket.settimeout(timeout)
+
+    def console_log(self, text):
+        for line in re.split("[\r\n]", text):
+            # filter out terminal escape sequences
+            line = re.sub("\x1b\[[0-9;?]*[a-zA-Z]", "", line)
+            line = re.sub("\x1b\([0-9;?]*[a-zA-Z]", "", line)
+            # replace unprintable chars
+            line = re.sub("\x1b", "<esc>", line)
+            line = re.sub("[\x00-\x1f]", ".", line)
+            line = re.sub("[\x80-\xff]", ".", line)
+            if line == "":
+                continue
+            # log console line
+            sys.stderr.write("con recv: %s\n" % line)
+
+    def console_wait(self, expect):
+        vm = self._guest
+        output = ""
+        while True:
+            try:
+                chars = vm.console_socket.recv(1)
+            except socket.timeout:
+                sys.stderr.write("console: *** read timeout ***\n")
+                sys.stderr.write("console: waiting for: '%s'\n" % expect)
+                sys.stderr.write("console: line buffer:\n")
+                sys.stderr.write("\n")
+                self.console_log(output.rstrip())
+                sys.stderr.write("\n")
+                raise
+            output += chars.decode("latin1")
+            if expect in output:
+                break
+            if "\r" in output or "\n" in output:
+                lines = re.split("[\r\n]", output)
+                output = lines.pop()
+                if self.debug:
+                    self.console_log("\n".join(lines))
+        if self.debug:
+            self.console_log(output)
+
+    def console_send(self, command):
+        vm = self._guest
+        if self.debug:
+            logline = re.sub("\n", "<enter>", command)
+            logline = re.sub("[\x00-\x1f]", ".", logline)
+            sys.stderr.write("con send: %s\n" % logline)
+        for char in list(command):
+            vm.console_socket.send(char.encode("utf-8"))
+            time.sleep(0.01)
+
+    def console_wait_send(self, wait, command):
+        self.console_wait(wait)
+        self.console_send(command)
+
+    def console_ssh_init(self, prompt, user, pw):
+        sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.rstrip()
+        self.console_wait_send("login:",    "%s\n" % user)
+        self.console_wait_send("Password:", "%s\n" % pw)
+        self.console_wait_send(prompt,      "mkdir .ssh\n")
+        self.console_wait_send(prompt,      sshkey_cmd)
+        self.console_wait_send(prompt,      "chmod 755 .ssh\n")
+        self.console_wait_send(prompt,      "chmod 644 .ssh/authorized_keys\n")
+
+    def console_sshd_config(self, prompt):
+        self.console_wait(prompt)
+        self.console_send("echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config\n")
+        for var in self.envvars:
+            self.console_wait(prompt)
+            self.console_send("echo 'AcceptEnv %s' >> /etc/ssh/sshd_config\n" % var)
+
+    def print_step(self, text):
+        sys.stderr.write("### %s ...\n" % text)
+
     def wait_ssh(self, seconds=300):
         starttime = datetime.datetime.now()
         endtime = starttime + datetime.timedelta(seconds=seconds)
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 11/19] tests/vm: openbsd autoinstall, using serial console
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (9 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 10/19] tests/vm: serial console support helpers Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 12/19] tests/vm: freebsd " Alex Bennée
                   ` (8 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée, Brad Smith

From: Gerd Hoffmann <kraxel@redhat.com>

Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally.  Install to disk, using the serial
console.  Create qemu user, configure ssh login.  Install packages
needed for qemu builds.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-8-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 28c7d25e29..b92c39f89a 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -2,10 +2,11 @@
 #
 # OpenBSD VM image
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng <famz@redhat.com>
+#  Gerd Hoffmann <kraxel@redhat.com>
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
@@ -13,32 +14,166 @@
 
 import os
 import sys
+import socket
 import subprocess
 import basevm
 
 class OpenBSDVM(basevm.BaseVM):
     name = "openbsd"
     arch = "x86_64"
+
+    link = "https://cdn.openbsd.org/pub/OpenBSD/6.5/amd64/install65.iso"
+    csum = "38d1f8cadd502f1c27bf05c5abde6cc505dd28f3f34f8a941048ff9a54f9f608"
+    size = "20G"
+    pkgs = [
+        # tools
+        "git",
+        "pkgconf",
+        "bzip2", "xz",
+
+        # gnu tools
+        "bash",
+        "gmake",
+        "gsed",
+        "bison",
+
+        # libs: usb
+        "libusb1",
+
+        # libs: crypto
+        "gnutls",
+
+        # libs: images
+        "jpeg",
+        "png",
+
+	# libs: ui
+        "sdl2",
+        "gtk+3",
+        "libxkbcommon",
+    ]
+
     BUILD_SCRIPT = """
         set -e;
-        rm -rf /var/tmp/qemu-test.*
-        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        rm -rf /home/qemu/qemu-test.*
+        cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+        mkdir src build; cd src;
         tar -xf /dev/rsd1c;
-        ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
-        gmake --output-sync -j{jobs} {verbose};
-        # XXX: "gmake check" seems to always hang or fail
-        #gmake --output-sync -j{jobs} check {verbose};
+        cd ../build
+        ../src/configure --cc=cc --python=python3 {configure_opts};
+        gmake --output-sync -j{jobs} {target} {verbose};
     """
+    poweroff = "halt -p"
 
     def build_image(self, img):
-        cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
-                sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
-        img_tmp_xz = img + ".tmp.xz"
+        self.print_step("Downloading install iso")
+        cimg = self._download_with_cache(self.link, sha256sum=self.csum)
         img_tmp = img + ".tmp"
-        sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+        iso = img + ".install.iso"
+
+        self.print_step("Preparing iso and disk image")
+        subprocess.check_call(["cp", "-f", cimg, iso])
+        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+                               img_tmp, self.size])
+
+        self.print_step("Booting installer")
+        self.boot(img_tmp, extra_args = [
+            "-bios", "pc-bios/bios-256k.bin",
+            "-machine", "graphics=off",
+            "-cdrom", iso
+        ])
+        self.console_init()
+        self.console_wait_send("boot>", "set tty com0\n")
+        self.console_wait_send("boot>", "\n")
+
+        # pre-install configuration
+        self.console_wait_send("(I)nstall",               "i\n")
+        self.console_wait_send("Terminal type",           "xterm\n")
+        self.console_wait_send("System hostname",         "openbsd\n")
+        self.console_wait_send("Which network interface", "vio0\n")
+        self.console_wait_send("IPv4 address",            "dhcp\n")
+        self.console_wait_send("IPv6 address",            "none\n")
+        self.console_wait_send("Which network interface", "done\n")
+        self.console_wait_send("DNS domain name",         "localnet\n")
+        self.console_wait("Password for root account")
+        self.console_send("%s\n" % self.ROOT_PASS)
+        self.console_wait("Password for root account")
+        self.console_send("%s\n" % self.ROOT_PASS)
+        self.console_wait_send("Start sshd(8)",           "yes\n")
+        self.console_wait_send("X Window System",         "\n")
+        self.console_wait_send("xenodm",                  "\n")
+        self.console_wait_send("console to com0",         "\n")
+        self.console_wait_send("Which speed",             "\n")
+
+        self.console_wait("Setup a user")
+        self.console_send("%s\n" % self.GUEST_USER)
+        self.console_wait("Full name")
+        self.console_send("%s\n" % self.GUEST_USER)
+        self.console_wait("Password")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait("Password")
+        self.console_send("%s\n" % self.GUEST_PASS)
+
+        self.console_wait_send("Allow root ssh login",    "yes\n")
+        self.console_wait_send("timezone",                "UTC\n")
+        self.console_wait_send("root disk",               "\n")
+        self.console_wait_send("(W)hole disk",            "\n")
+        self.console_wait_send("(A)uto layout",           "\n")
+        self.console_wait_send("Location of sets",        "cd0\n")
+        self.console_wait_send("Pathname to the sets",    "\n")
+        self.console_wait_send("Set name(s)",             "\n")
+        self.console_wait_send("without verification",    "yes\n")
+
+        self.print_step("Installation started now, this will take a while")
+        self.console_wait_send("Location of sets",        "done\n")
+
+        self.console_wait("successfully completed")
+        self.print_step("Installation finished, rebooting")
+        self.console_wait_send("(R)eboot",                "reboot\n")
+
+        # setup qemu user
+        prompt = "$"
+        self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+        self.console_wait_send(prompt, "exit\n")
+
+        # setup root user
+        prompt = "openbsd#"
+        self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+        self.console_sshd_config(prompt)
+
+        # setup virtio-blk #1 (tarfile)
+        self.console_wait(prompt)
+        self.console_send("echo 'chmod 666 /dev/rsd1c' >> /etc/rc.local\n")
+
+        # enable w+x for /home
+        self.console_wait(prompt)
+        self.console_send("sed -i -e '/home/s/rw,/rw,wxallowed,/' /etc/fstab\n")
+
+        # tweak datasize limit
+        self.console_wait(prompt)
+        self.console_send("sed -i -e 's/\\(datasize[^=]*\\)=[^:]*/\\1=infinity/' /etc/login.conf\n")
+
+        # use http (be proxy cache friendly)
+        self.console_wait(prompt)
+        self.console_send("sed -i -e 's/https/http/' /etc/installurl\n")
+
+        self.print_step("Configuration finished, rebooting")
+        self.console_wait_send(prompt, "reboot\n")
+        self.console_wait("login:")
+        self.wait_ssh()
+
+        self.print_step("Installing packages")
+        self.ssh_root_check("pkg_add %s\n" % " ".join(self.pkgs))
+
+        # shutdown
+        self.ssh_root(self.poweroff)
+        self.wait()
+
+        if os.path.exists(img):
+            os.remove(img)
         os.rename(img_tmp, img)
+        os.remove(iso)
+        self.print_step("All done")
 
 if __name__ == "__main__":
     sys.exit(basevm.main(OpenBSDVM))
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 12/19] tests/vm: freebsd autoinstall, using serial console
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (10 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 11/19] tests/vm: openbsd autoinstall, using serial console Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 13/19] tests/vm: netbsd " Alex Bennée
                   ` (7 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally.  Install to disk, using the serial
console.  Create qemu user, configure ssh login.  Install packages
needed for qemu builds.

Note that freebsd package downloads are delivered as non-cachable
content, so I had to configure squid with "ignore-no-store
ignore-private ignore-reload" for pkgmir.geo.freebsd.org to make the
caching actually work.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190617043858.8290-9-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 091be1a065..2a19461a90 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -2,41 +2,203 @@
 #
 # FreeBSD VM image
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng <famz@redhat.com>
+#  Gerd Hoffmann <kraxel@redhat.com>
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
 #
 
 import os
+import re
 import sys
+import time
+import socket
 import subprocess
 import basevm
 
 class FreeBSDVM(basevm.BaseVM):
     name = "freebsd"
     arch = "x86_64"
+
+    link = "https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso.xz"
+    csum = "1d40015bea89d05b8bd13e2ed80c40b522a9ec1abd8e7c8b80954fb485fb99db"
+    size = "20G"
+    pkgs = [
+        # build tools
+        "git",
+        "pkgconf",
+        "bzip2",
+
+        # gnu tools
+        "bash",
+        "gmake",
+        "gsed",
+        "flex", "bison",
+
+        # libs: crypto
+        "gnutls",
+
+        # libs: images
+        "jpeg-turbo",
+        "png",
+
+        # libs: ui
+        "sdl2",
+        "gtk3",
+        "libxkbcommon",
+
+        # libs: opengl
+        "libepoxy",
+        "mesa-libs",
+    ]
+
     BUILD_SCRIPT = """
         set -e;
-        rm -rf /var/tmp/qemu-test.*
-        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        rm -rf /home/qemu/qemu-test.*
+        cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+        mkdir src build; cd src;
         tar -xf /dev/vtbd1;
-        ./configure {configure_opts};
+        cd ../build
+        ../src/configure --python=python3.6 {configure_opts};
         gmake --output-sync -j{jobs} {target} {verbose};
     """
 
+    def console_boot_serial(self):
+        self.console_wait_send("Autoboot", "3")
+        self.console_wait_send("OK", "set console=comconsole\n")
+        self.console_wait_send("OK", "boot\n")
+
     def build_image(self, img):
-        cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz",
-                sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
-        img_tmp_xz = img + ".tmp.xz"
+        self.print_step("Downloading install iso")
+        cimg = self._download_with_cache(self.link, sha256sum=self.csum)
         img_tmp = img + ".tmp"
-        sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+        iso = img + ".install.iso"
+        iso_xz = iso + ".xz"
+
+        self.print_step("Preparing iso and disk image")
+        subprocess.check_call(["cp", "-f", cimg, iso_xz])
+        subprocess.check_call(["xz", "-dvf", iso_xz])
+        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+                               img_tmp, self.size])
+
+        self.print_step("Booting installer")
+        self.boot(img_tmp, extra_args = [
+            "-bios", "pc-bios/bios-256k.bin",
+            "-machine", "graphics=off",
+            "-cdrom", iso
+        ])
+        self.console_init()
+        self.console_boot_serial()
+        self.console_wait_send("Console type",          "xterm\n")
+
+        # pre-install configuration
+        self.console_wait_send("Welcome",               "\n")
+        self.console_wait_send("Keymap Selection",      "\n")
+        self.console_wait_send("Set Hostname",          "freebsd\n")
+        self.console_wait_send("Distribution Select",   "\n")
+        self.console_wait_send("Partitioning",          "\n")
+        self.console_wait_send("Partition",             "\n")
+        self.console_wait_send("Scheme",                "\n")
+        self.console_wait_send("Editor",                "f")
+        self.console_wait_send("Confirmation",          "c")
+
+        self.print_step("Installation started now, this will take a while")
+
+        # post-install configuration
+        self.console_wait("New Password:")
+        self.console_send("%s\n" % self.ROOT_PASS)
+        self.console_wait("Retype New Password:")
+        self.console_send("%s\n" % self.ROOT_PASS)
+
+        self.console_wait_send("Network Configuration", "\n")
+        self.console_wait_send("IPv4",                  "y")
+        self.console_wait_send("DHCP",                  "y")
+        self.console_wait_send("IPv6",                  "n")
+        self.console_wait_send("Resolver",              "\n")
+
+        self.console_wait_send("Time Zone Selector",    "a\n")
+        self.console_wait_send("Confirmation",          "y")
+        self.console_wait_send("Time & Date",           "\n")
+        self.console_wait_send("Time & Date",           "\n")
+
+        self.console_wait_send("System Configuration",  "\n")
+        self.console_wait_send("System Hardening",      "\n")
+
+        # qemu user
+        self.console_wait_send("Add User Accounts", "y")
+        self.console_wait("Username")
+        self.console_send("%s\n" % self.GUEST_USER)
+        self.console_wait("Full name")
+        self.console_send("%s\n" % self.GUEST_USER)
+        self.console_wait_send("Uid",                   "\n")
+        self.console_wait_send("Login group",           "\n")
+        self.console_wait_send("Login group",           "\n")
+        self.console_wait_send("Login class",           "\n")
+        self.console_wait_send("Shell",                 "\n")
+        self.console_wait_send("Home directory",        "\n")
+        self.console_wait_send("Home directory perm",   "\n")
+        self.console_wait_send("Use password",          "\n")
+        self.console_wait_send("Use an empty password", "\n")
+        self.console_wait_send("Use a random password", "\n")
+        self.console_wait("Enter password:")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait("Enter password again:")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait_send("Lock out",              "\n")
+        self.console_wait_send("OK",                    "yes\n")
+        self.console_wait_send("Add another user",      "no\n")
+
+        self.console_wait_send("Final Configuration",   "\n")
+        self.console_wait_send("Manual Configuration",  "\n")
+        self.console_wait_send("Complete",              "\n")
+
+        self.print_step("Installation finished, rebooting")
+        self.console_boot_serial()
+
+        # setup qemu user
+        prompt = "$"
+        self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+        self.console_wait_send(prompt, "exit\n")
+
+        # setup root user
+        prompt = "root@freebsd:~ #"
+        self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+        self.console_sshd_config(prompt)
+
+        # setup serial console
+        self.console_wait(prompt)
+        self.console_send("echo 'console=comconsole' >> /boot/loader.conf\n")
+
+        # setup boot delay
+        self.console_wait(prompt)
+        self.console_send("echo 'autoboot_delay=1' >> /boot/loader.conf\n")
+
+        # setup virtio-blk #1 (tarfile)
+        self.console_wait(prompt)
+        self.console_send("echo 'chmod 666 /dev/vtbd1' >> /etc/rc.local\n")
+
+        self.print_step("Configuration finished, rebooting")
+        self.console_wait_send(prompt, "reboot\n")
+        self.console_wait("login:")
+        self.wait_ssh()
+
+        self.print_step("Installing packages")
+        self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
+
+        # shutdown
+        self.ssh_root(self.poweroff)
+        self.console_wait("Uptime:")
+        self.wait()
+
+        if os.path.exists(img):
+            os.remove(img)
         os.rename(img_tmp, img)
+        os.remove(iso)
+        self.print_step("All done")
 
 if __name__ == "__main__":
     sys.exit(basevm.main(FreeBSDVM))
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 13/19] tests/vm: netbsd autoinstall, using serial console
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (11 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 12/19] tests/vm: freebsd " Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 14/19] tests/vm: fedora " Alex Bennée
                   ` (6 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Kamil Rytarowski, Gerd Hoffmann, Kamil Rytarowski,
	Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally.  Install to disk, using the serial
console.  Create qemu user, configure ssh login.  Install packages
needed for qemu builds.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Kamil Rytarowski <n54@gmx.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-10-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index ee9eaeab50..be59a2c1da 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -2,10 +2,11 @@
 #
 # NetBSD VM image
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng <famz@redhat.com>
+#  Gerd Hoffmann <kraxel@redhat.com>
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
@@ -13,30 +14,198 @@
 
 import os
 import sys
+import time
 import subprocess
 import basevm
 
 class NetBSDVM(basevm.BaseVM):
     name = "netbsd"
     arch = "x86_64"
+
+    link = "https://cdn.netbsd.org/pub/NetBSD/NetBSD-8.0/images/NetBSD-8.0-amd64.iso"
+    size = "20G"
+    pkgs = [
+        # tools
+        "git-base",
+        "pkgconf",
+        "xz",
+        "python37",
+
+        # gnu tools
+        "bash",
+        "gmake",
+        "gsed",
+        "flex", "bison",
+
+        # libs: crypto
+        "gnutls",
+
+        # libs: images
+        "jpeg",
+        "png",
+
+	# libs: ui
+        "SDL2",
+        "gtk3+",
+        "libxkbcommon",
+    ]
+
     BUILD_SCRIPT = """
         set -e;
-        rm -rf /var/tmp/qemu-test.*
-        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        rm -rf /home/qemu/qemu-test.*
+        cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+        mkdir src build; cd src;
         tar -xf /dev/rld1a;
-        ./configure --python=python2.7 {configure_opts};
+        cd ../build
+        ../src/configure --python=python3.7 --disable-opengl {configure_opts};
         gmake --output-sync -j{jobs} {target} {verbose};
     """
+    poweroff = "/sbin/poweroff"
 
     def build_image(self, img):
-        cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz",
-                                         sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732')
-        img_tmp_xz = img + ".tmp.xz"
+        cimg = self._download_with_cache(self.link)
         img_tmp = img + ".tmp"
-        sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+        iso = img + ".install.iso"
+
+        self.print_step("Preparing iso and disk image")
+        subprocess.check_call(["cp", "-f", cimg, iso])
+        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+                               img_tmp, self.size])
+
+        self.print_step("Booting installer")
+        self.boot(img_tmp, extra_args = [
+            "-bios", "pc-bios/bios-256k.bin",
+            "-machine", "graphics=off",
+            "-cdrom", iso
+        ])
+        self.console_init()
+        self.console_wait("Primary Bootstrap")
+
+        # serial console boot menu output doesn't work for some
+        # reason, so we have to fly blind ...
+        for char in list("5consdev com0\n"):
+            time.sleep(0.2)
+            self.console_send(char)
+            self.console_wait("")
+        self.console_wait_send("> ", "boot\n")
+
+        self.console_wait_send("Terminal type",            "xterm\n")
+        self.console_wait_send("a: Installation messages", "a\n")
+        self.console_wait_send("b: US-English",            "b\n")
+        self.console_wait_send("a: Install NetBSD",        "a\n")
+        self.console_wait("Shall we continue?")
+        self.console_wait_send("b: Yes",                   "b\n")
+
+        self.console_wait_send("a: ld0",                   "a\n")
+        self.console_wait_send("a: This is the correct",   "a\n")
+        self.console_wait_send("b: Use the entire disk",   "b\n")
+        self.console_wait("NetBSD bootcode")
+        self.console_wait_send("a: Yes",                   "a\n")
+        self.console_wait_send("b: Use existing part",     "b\n")
+        self.console_wait_send("x: Partition sizes ok",    "x\n")
+        self.console_wait_send("for your NetBSD disk",     "\n")
+        self.console_wait("Shall we continue?")
+        self.console_wait_send("b: Yes",                   "b\n")
+
+        self.console_wait_send("b: Use serial port com0",  "b\n")
+        self.console_wait_send("f: Set serial baud rate",  "f\n")
+        self.console_wait_send("a: 9600",                  "a\n")
+        self.console_wait_send("x: Exit",                  "x\n")
+
+        self.console_wait_send("a: Full installation",     "a\n")
+        self.console_wait_send("a: CD-ROM",                "a\n")
+
+        self.print_step("Installation started now, this will take a while")
+        self.console_wait_send("Hit enter to continue",    "\n")
+
+        self.console_wait_send("d: Change root password",  "d\n")
+        self.console_wait_send("a: Yes",                   "a\n")
+        self.console_wait("New password:")
+        self.console_send("%s\n" % self.ROOT_PASS)
+        self.console_wait("New password:")
+        self.console_send("%s\n" % self.ROOT_PASS)
+        self.console_wait("Retype new password:")
+        self.console_send("%s\n" % self.ROOT_PASS)
+
+        self.console_wait_send("o: Add a user",            "o\n")
+        self.console_wait("username")
+        self.console_send("%s\n" % self.GUEST_USER)
+        self.console_wait("to group wheel")
+        self.console_wait_send("a: Yes",                   "a\n")
+        self.console_wait_send("a: /bin/sh",               "a\n")
+        self.console_wait("New password:")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait("New password:")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait("Retype new password:")
+        self.console_send("%s\n" % self.GUEST_PASS)
+
+        self.console_wait_send("a: Configure network",     "a\n")
+        self.console_wait_send("a: vioif0",                "a\n")
+        self.console_wait_send("Network media type",       "\n")
+        self.console_wait("autoconfiguration")
+        self.console_wait_send("a: Yes",                   "a\n")
+        self.console_wait_send("DNS domain",               "localnet\n")
+        self.console_wait("Are they OK?")
+        self.console_wait_send("a: Yes",                   "a\n")
+        self.console_wait("installed in /etc")
+        self.console_wait_send("a: Yes",                   "a\n")
+
+        self.console_wait_send("e: Enable install",        "e\n")
+        proxy = os.environ.get("http_proxy")
+        if not proxy is None:
+            self.console_wait_send("f: Proxy",             "f\n")
+            self.console_wait("Proxy")
+            self.console_send("%s\n" % proxy)
+        self.console_wait_send("x: Install pkgin",         "x\n")
+        self.console_init(1200)
+        self.console_wait_send("Hit enter to continue", "\n")
+        self.console_init()
+
+        self.console_wait_send("g: Enable sshd",           "g\n")
+        self.console_wait_send("x: Finished conf",         "x\n")
+        self.console_wait_send("Hit enter to continue",    "\n")
+
+        self.print_step("Installation finished, rebooting")
+        self.console_wait_send("d: Reboot the computer",   "d\n")
+
+        # setup qemu user
+        prompt = "localhost$"
+        self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+        self.console_wait_send(prompt, "exit\n")
+
+        # setup root user
+        prompt = "localhost#"
+        self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+        self.console_sshd_config(prompt)
+
+        # setup virtio-blk #1 (tarfile)
+        self.console_wait(prompt)
+        self.console_send("echo 'chmod 666 /dev/rld1a' >> /etc/rc.local\n")
+
+        # turn off mprotect (conflicts with tcg)
+        self.console_wait(prompt)
+        self.console_send("echo security.pax.mprotect.enabled=0 >> /etc/sysctl.conf\n")
+
+        self.print_step("Configuration finished, rebooting")
+        self.console_wait_send(prompt, "reboot\n")
+        self.console_wait("login:")
+        self.wait_ssh()
+
+        self.print_step("Installing packages")
+        self.ssh_root_check("pkgin update\n")
+        self.ssh_root_check("pkgin -y install %s\n" % " ".join(self.pkgs))
+
+        # shutdown
+        self.ssh_root(self.poweroff)
+        self.console_wait("entering state S5")
+        self.wait()
+
+        if os.path.exists(img):
+            os.remove(img)
         os.rename(img_tmp, img)
+        os.remove(iso)
+        self.print_step("All done")
 
 if __name__ == "__main__":
     sys.exit(basevm.main(NetBSDVM))
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 14/19] tests/vm: fedora autoinstall, using serial console
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (12 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 13/19] tests/vm: netbsd " Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 15/19] tests/vm: ubuntu.i386: apt proxy setup Alex Bennée
                   ` (5 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann, Alex Bennée

From: Gerd Hoffmann <kraxel@redhat.com>

Download the install iso and prepare the image locally.  Install to
disk, using the serial console.  Create qemu user, configure ssh login.
Install packages needed for qemu builds.

Yes, we have docker images for fedora.  But for trouble-shooting it
might be helpful to have a vm too.  When vm builds fail you can use
it to figure whenever the vm setup or the guest os is the problem.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-11-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index a61c236b8e..809b80e2e5 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -2,7 +2,7 @@
 
 .PHONY: vm-build-all vm-clean-all
 
-IMAGES := ubuntu.i386 freebsd netbsd openbsd centos
+IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora
 IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
 IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
 
@@ -16,6 +16,7 @@ vm-test:
 	@echo "  vm-build-netbsd                 - Build QEMU in NetBSD VM"
 	@echo "  vm-build-openbsd                - Build QEMU in OpenBSD VM"
 	@echo "  vm-build-centos                 - Build QEMU in CentOS VM, with Docker"
+	@echo "  vm-build-fedora                 - Build QEMU in Fedora VM"
 	@echo ""
 	@echo "  vm-build-all                    - Build QEMU in all VMs"
 	@echo "  vm-clean-all                    - Clean up VM images"
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 592a344773..3d717da498 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -206,7 +206,7 @@ class BaseVM(object):
             # log console line
             sys.stderr.write("con recv: %s\n" % line)
 
-    def console_wait(self, expect):
+    def console_wait(self, expect, expectalt = None):
         vm = self._guest
         output = ""
         while True:
@@ -215,6 +215,8 @@ class BaseVM(object):
             except socket.timeout:
                 sys.stderr.write("console: *** read timeout ***\n")
                 sys.stderr.write("console: waiting for: '%s'\n" % expect)
+                if not expectalt is None:
+                    sys.stderr.write("console: waiting for: '%s' (alt)\n" % expectalt)
                 sys.stderr.write("console: line buffer:\n")
                 sys.stderr.write("\n")
                 self.console_log(output.rstrip())
@@ -223,6 +225,8 @@ class BaseVM(object):
             output += chars.decode("latin1")
             if expect in output:
                 break
+            if not expectalt is None and expectalt in output:
+                break
             if "\r" in output or "\n" in output:
                 lines = re.split("[\r\n]", output)
                 output = lines.pop()
@@ -230,6 +234,9 @@ class BaseVM(object):
                     self.console_log("\n".join(lines))
         if self.debug:
             self.console_log(output)
+        if not expectalt is None and expectalt in output:
+            return False
+        return True
 
     def console_send(self, command):
         vm = self._guest
diff --git a/tests/vm/fedora b/tests/vm/fedora
new file mode 100755
index 0000000000..e8fa5bf0d2
--- /dev/null
+++ b/tests/vm/fedora
@@ -0,0 +1,189 @@
+#!/usr/bin/env python
+#
+# Fedora VM image
+#
+# Copyright 2019 Red Hat Inc.
+#
+# Authors:
+#  Gerd Hoffmann <kraxel@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import re
+import sys
+import time
+import socket
+import subprocess
+import basevm
+
+class FedoraVM(basevm.BaseVM):
+    name = "fedora"
+    arch = "x86_64"
+
+    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+    link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
+    repo = base + "Server/x86_64/os/"
+    full = base + "Everything/x86_64/os/"
+    csum = "5e4eac4566d8c572bfb3bcf54b7d6c82006ec3c6c882a2c9235c6d3494d7b100"
+    size = "20G"
+    pkgs = [
+        # tools
+        'git-core',
+        'flex', 'bison',
+        'gcc', 'binutils', 'make',
+
+        # perl
+        'perl-Test-Harness',
+
+        # libs: usb
+        '"pkgconfig(libusb-1.0)"',
+        '"pkgconfig(libusbredirparser-0.5)"',
+
+        # libs: crypto
+        '"pkgconfig(gnutls)"',
+
+        # libs: ui
+        '"pkgconfig(sdl2)"',
+        '"pkgconfig(gtk+-3.0)"',
+        '"pkgconfig(ncursesw)"',
+
+        # libs: audio
+        '"pkgconfig(libpulse)"',
+        '"pkgconfig(alsa)"',
+    ]
+
+    BUILD_SCRIPT = """
+        set -e;
+        rm -rf /home/qemu/qemu-test.*
+        cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+        mkdir src build; cd src;
+        tar -xf /dev/vdb;
+        cd ../build
+        ../src/configure --python=python3 {configure_opts};
+        gmake --output-sync -j{jobs} {target} {verbose};
+    """
+
+    def build_image(self, img):
+        self.print_step("Downloading install iso")
+        cimg = self._download_with_cache(self.link, sha256sum=self.csum)
+        img_tmp = img + ".tmp"
+        iso = img + ".install.iso"
+
+        self.print_step("Preparing iso and disk image")
+        subprocess.check_call(["cp", "-f", cimg, iso])
+        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+                               img_tmp, self.size])
+
+        self.print_step("Booting installer")
+        self.boot(img_tmp, extra_args = [
+            "-bios", "pc-bios/bios-256k.bin",
+            "-machine", "graphics=off",
+            "-cdrom", iso
+        ])
+        self.console_init(300)
+        self.console_wait("installation process.")
+        time.sleep(0.3)
+        self.console_send("\t")
+        time.sleep(0.3)
+        self.console_send(" console=ttyS0")
+        proxy = os.environ.get("http_proxy")
+        if not proxy is None:
+            self.console_send(" proxy=%s" % proxy)
+            self.console_send(" inst.proxy=%s" % proxy)
+        self.console_send(" inst.repo=%s" % self.repo)
+        self.console_send("\n")
+
+        self.console_wait_send("2) Use text mode",         "2\n")
+
+        self.console_wait_send("5) [!] Installation Dest", "5\n")
+        self.console_wait_send("1) [x]",                   "c\n")
+        self.console_wait_send("2) [ ] Use All Space",     "2\n")
+        self.console_wait_send("2) [x] Use All Space",     "c\n")
+        self.console_wait_send("1) [ ] Standard Part",     "1\n")
+        self.console_wait_send("1) [x] Standard Part",     "c\n")
+
+        self.console_wait_send("7) [!] Root password",     "7\n")
+        self.console_wait("Password:")
+        self.console_send("%s\n" % self.ROOT_PASS)
+        self.console_wait("Password (confirm):")
+        self.console_send("%s\n" % self.ROOT_PASS)
+
+        self.console_wait_send("8) [ ] User creation",     "8\n")
+        self.console_wait_send("1) [ ] Create user",       "1\n")
+        self.console_wait_send("3) User name",             "3\n")
+        self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER)
+        self.console_wait_send("4) [ ] Use password",      "4\n")
+        self.console_wait_send("5) Password",              "5\n")
+        self.console_wait("Password:")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait("Password (confirm):")
+        self.console_send("%s\n" % self.GUEST_PASS)
+        self.console_wait_send("7) Groups",                "c\n")
+
+        while True:
+            good = self.console_wait("3) [x] Installation",
+                                     "3) [!] Installation")
+            self.console_send("r\n")
+            if good:
+                break
+            time.sleep(10)
+
+        while True:
+            good = self.console_wait("4) [x] Software",
+                                     "4) [!] Software")
+            self.console_send("r\n")
+            if good:
+                break
+            time.sleep(10)
+            self.console_send("r\n" % self.GUEST_PASS)
+
+        self.console_wait_send("'b' to begin install",     "b\n")
+
+        self.print_step("Installation started now, this will take a while")
+
+        self.console_wait_send("Installation complete",    "\n")
+        self.print_step("Installation finished, rebooting")
+
+        # setup qemu user
+        prompt = " ~]$"
+        self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+        self.console_wait_send(prompt, "exit\n")
+
+        # setup root user
+        prompt = " ~]#"
+        self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+        self.console_sshd_config(prompt)
+
+        # setup virtio-blk #1 (tarfile)
+        self.console_wait(prompt)
+        self.console_send("echo 'KERNEL==\"vdb\" MODE=\"666\"' >> %s\n" %
+                          "/etc/udev/rules.d/99-qemu.rules")
+
+        self.print_step("Configuration finished, rebooting")
+        self.console_wait_send(prompt, "reboot\n")
+        self.console_wait("login:")
+        self.wait_ssh()
+
+        self.print_step("Installing packages")
+        self.ssh_root_check("rm -vf /etc/yum.repos.d/fedora*.repo\n")
+        self.ssh_root_check("echo '[fedora]' >> /etc/yum.repos.d/qemu.repo\n")
+        self.ssh_root_check("echo 'baseurl=%s' >> /etc/yum.repos.d/qemu.repo\n" % self.full)
+        self.ssh_root_check("echo 'gpgcheck=0' >> /etc/yum.repos.d/qemu.repo\n")
+        self.ssh_root_check("dnf install -y %s\n" % " ".join(self.pkgs))
+
+        # shutdown
+        self.ssh_root(self.poweroff)
+        self.console_wait("sleep state S5")
+        self.wait()
+
+        if os.path.exists(img):
+            os.remove(img)
+        os.rename(img_tmp, img)
+        os.remove(iso)
+        self.print_step("All done")
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(FedoraVM))
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 15/19] tests/vm: ubuntu.i386: apt proxy setup
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (13 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 14/19] tests/vm: fedora " Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 16/19] .travis.yml: default the --disable-system build to --static Alex Bennée
                   ` (4 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Gerd Hoffmann

From: Gerd Hoffmann <kraxel@redhat.com>

Configure apt proxy so package downloads
can be cached and can pass firewalls.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-12-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 3ea459ee20..38f740eabf 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -51,6 +51,10 @@ class UbuntuX86VM(basevm.BaseVM):
                           "    ssh-authorized-keys:\n",
                           "    - %s\n" % basevm.SSH_PUB_KEY,
                           "locale: en_US.UTF-8\n"])
+        proxy = os.environ.get("http_proxy")
+        if not proxy is None:
+            udata.writelines(["apt:\n",
+                              "  proxy: %s" % proxy])
         udata.close()
         subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
                                "-volid", "cidata", "-joliet", "-rock",
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 16/19] .travis.yml: default the --disable-system build to --static
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (14 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 15/19] tests/vm: ubuntu.i386: apt proxy setup Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 17/19] .travis.yml: force a brew update for MacOS builds Alex Bennée
                   ` (3 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée, qemu-devel

It's fairly common to build qemu-user binaries with --static linking
so the binary can be copied around without libraries. Enable --static
in the default qemu-user build to cover this.

There are other qemu-user builds that use dynamic linking so they
should catch any problems there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/.travis.yml b/.travis.yml
index aeb9b211cd..9750dc905c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -80,7 +80,7 @@ script:
 matrix:
   include:
     - env:
-        - CONFIG="--disable-system"
+        - CONFIG="--disable-system --static"
 
 
     # we split the system builds as it takes a while to build them all
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 17/19] .travis.yml: force a brew update for MacOS builds
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (15 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 16/19] .travis.yml: default the --disable-system build to --static Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 18/19] Makefile: Rename the 'vm-test' target as 'vm-help' Alex Bennée
                   ` (2 subsequent siblings)
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée, qemu-devel

It looks like the Travis image package databases are out of date
causing the build to error with:

  Error: Your Homebrew is outdated. Please run `brew update`.
  Error: Kernel.exit

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/.travis.yml b/.travis.yml
index 9750dc905c..a891ce2485 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -43,6 +43,7 @@ addons:
       - glib
       - pixman
       - gnu-sed
+    update: true
 
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 18/19] Makefile: Rename the 'vm-test' target as 'vm-help'
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (16 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 17/19] .travis.yml: force a brew update for MacOS builds Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-06-24 13:43 ` [Qemu-devel] [PULL 19/19] target/i386: fix feature check in hyperv-stub.c Alex Bennée
  2019-07-01  9:52 ` [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Peter Maydell
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Stefano Garzarella

From: Philippe Mathieu-Daudé <philmd@redhat.com>

We already have 'make check-help', use the 'make vm-help' form
to display helps about VM testing. Keep the old target to not
bother old customs.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20190531064341.29730-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/Makefile b/Makefile
index cfb18f1525..bdeb259dfa 100644
--- a/Makefile
+++ b/Makefile
@@ -9,7 +9,7 @@ SRC_PATH=.
 UNCHECKED_GOALS := %clean TAGS cscope ctags dist \
     html info pdf txt \
     help check-help print-% \
-    docker docker-% vm-test vm-build-%
+    docker docker-% vm-help vm-test vm-build-%
 
 print-%:
 	@echo '$*=$($*)'
@@ -1148,7 +1148,7 @@ endif
 	@echo  'Test targets:'
 	@echo  '  check           - Run all tests (check-help for details)'
 	@echo  '  docker          - Help about targets running tests inside Docker containers'
-	@echo  '  vm-test         - Help about targets running tests inside VM'
+	@echo  '  vm-help         - Help about targets running tests inside VM'
 	@echo  ''
 	@echo  'Documentation targets:'
 	@echo  '  html info pdf txt'
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index da2d0fc964..68aba3926e 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -399,12 +399,12 @@ VM testing
 
 This test suite contains scripts that bootstrap various guest images that have
 necessary packages to build QEMU. The basic usage is documented in ``Makefile``
-help which is displayed with ``make vm-test``.
+help which is displayed with ``make vm-help``.
 
 Quickstart
 ----------
 
-Run ``make vm-test`` to list available make targets. Invoke a specific make
+Run ``make vm-help`` to list available make targets. Invoke a specific make
 command to run build test in an image. For example, ``make vm-build-freebsd``
 will build the source tree in the FreeBSD image. The command can be executed
 from either the source tree or the build dir; if the former, ``./configure`` is
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 809b80e2e5..3560716092 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -8,8 +8,9 @@ IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
 
 .PRECIOUS: $(IMAGE_FILES)
 
-vm-test:
-	@echo "vm-test: Test QEMU in preconfigured virtual machines"
+# 'vm-help' target was historically named 'vm-test'
+vm-help vm-test:
+	@echo "vm-help: Test QEMU in preconfigured virtual machines"
 	@echo
 	@echo "  vm-build-ubuntu.i386            - Build QEMU in ubuntu i386 VM"
 	@echo "  vm-build-freebsd                - Build QEMU in FreeBSD VM"
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* [Qemu-devel] [PULL 19/19] target/i386: fix feature check in hyperv-stub.c
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (17 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 18/19] Makefile: Rename the 'vm-test' target as 'vm-help' Alex Bennée
@ 2019-06-24 13:43 ` Alex Bennée
  2019-07-01  9:52 ` [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Peter Maydell
  19 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-06-24 13:43 UTC (permalink / raw)
  To: peter.maydell
  Cc: Eduardo Habkost, qemu-devel, Roman Kagan, Paolo Bonzini,
	Vitaly Kuznetsov, Alex Bennée, Richard Henderson

Commit 2d384d7c8 broken the build when built with:

  configure --without-default-devices --disable-user

The reason was the conversion of cpu->hyperv_synic to
cpu->hyperv_synic_kvm_only although the rest of the patch introduces a
feature checking mechanism. So I've fixed the KVM_EXIT_HYPERV_SYNIC in
hyperv-stub to do the same feature check as in the real hyperv.c

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Cc: Roman Kagan <rkagan@virtuozzo.com>

diff --git a/target/i386/hyperv-stub.c b/target/i386/hyperv-stub.c
index fe548cbae2..0028527e79 100644
--- a/target/i386/hyperv-stub.c
+++ b/target/i386/hyperv-stub.c
@@ -15,7 +15,7 @@ int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit)
 {
     switch (exit->type) {
     case KVM_EXIT_HYPERV_SYNIC:
-        if (!cpu->hyperv_synic) {
+        if (!hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC)) {
             return -1;
         }
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 26+ messages in thread

* Re: [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
  2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
                   ` (18 preceding siblings ...)
  2019-06-24 13:43 ` [Qemu-devel] [PULL 19/19] target/i386: fix feature check in hyperv-stub.c Alex Bennée
@ 2019-07-01  9:52 ` Peter Maydell
  2019-07-01 15:07   ` Alex Bennée
  2019-07-01 15:58   ` Alex Bennée
  19 siblings, 2 replies; 26+ messages in thread
From: Peter Maydell @ 2019-07-01  9:52 UTC (permalink / raw)
  To: Alex Bennée; +Cc: QEMU Developers

On Mon, 24 Jun 2019 at 14:43, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
>
>   Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-next-240619-1
>
> for you to fetch changes up to e0fe22c3528773fcbfd135a0ef7f6b3c5d373bb9:
>
>   target/i386: fix feature check in hyperv-stub.c (2019-06-24 14:36:39 +0100)
>
> ----------------------------------------------------------------
> Various testing fixes:
>
>    - tests/vm updates and clean-ups
>    - tests/vm serial autobuild on host
>    - ensure MacOS builds do "brew update"
>    - ensure we test --static user builds
>    - fix hyperv compile failure
>
> This brings my testing back to green on all CI services.

The FreeBSD tests pass OK but the new output includes
some stuff that trips my "find error/warning messages grep:
con recv: /etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
con recv: /etc/rc: WARNING: hostid: unable to figure out a UUID from
DMI data, generating a new one
pkg: Repository FreeBSD load error: access repo
file(/var/db/pkg/repo-FreeBSD.sqlite) failed: No such file or
directory
        p5-Error: 0.17027

There's also a new compile warning:
/home/qemu/qemu-test.r7G0Wv/src/migration/rdma.c:842:26: warning:
unused variable 'port_attr' [-Wunused-variable]
    struct ibv_port_attr port_attr;
                         ^
which I guess was not being flagged up by the old compiler/VM image.

OpenBSD also passes but has some output which my 'find warnings'
grep picks up:

con recv: WARNING: root is targeted by password guessing attacks,
pubkeys are safer.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
  2019-07-01  9:52 ` [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Peter Maydell
@ 2019-07-01 15:07   ` Alex Bennée
  2019-07-01 15:58   ` Alex Bennée
  1 sibling, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2019-07-01 15:07 UTC (permalink / raw)
  To: Peter Maydell; +Cc: QEMU Developers


Peter Maydell <peter.maydell@linaro.org> writes:

> On Mon, 24 Jun 2019 at 14:43, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
>>
>>   Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-testing-next-240619-1
>>
>> for you to fetch changes up to e0fe22c3528773fcbfd135a0ef7f6b3c5d373bb9:
>>
>>   target/i386: fix feature check in hyperv-stub.c (2019-06-24 14:36:39 +0100)
>>
>> ----------------------------------------------------------------
>> Various testing fixes:
>>
>>    - tests/vm updates and clean-ups
>>    - tests/vm serial autobuild on host
>>    - ensure MacOS builds do "brew update"
>>    - ensure we test --static user builds
>>    - fix hyperv compile failure
>>
>> This brings my testing back to green on all CI services.
>
> The FreeBSD tests pass OK but the new output includes
> some stuff that trips my "find error/warning messages grep:
> con recv: /etc/rc: WARNING: $hostname is not set -- see rc.conf(5).
> con recv: /etc/rc: WARNING: hostid: unable to figure out a UUID from
> DMI data, generating a new one

These are artefacts of setting up a new system - we have no hostname
until we've been through the setup. It won't occur on installed systems.

> pkg: Repository FreeBSD load error: access repo
> file(/var/db/pkg/repo-FreeBSD.sqlite) failed: No such file or
> directory
>         p5-Error: 0.17027

That seems to be a transient fault I haven't been able to replicate.

>
> There's also a new compile warning:
> /home/qemu/qemu-test.r7G0Wv/src/migration/rdma.c:842:26: warning:
> unused variable 'port_attr' [-Wunused-variable]
>     struct ibv_port_attr port_attr;

Fixed.

>                          ^
> which I guess was not being flagged up by the old compiler/VM image.
>
> OpenBSD also passes but has some output which my 'find warnings'
> grep picks up:
>
> con recv: WARNING: root is targeted by password guessing attacks,
> pubkeys are safer.

Looking at this now.

>
> thanks
> -- PMM

--
Alex Bennée


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
  2019-07-01  9:52 ` [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Peter Maydell
  2019-07-01 15:07   ` Alex Bennée
@ 2019-07-01 15:58   ` Alex Bennée
  2019-07-02 10:10     ` Philippe Mathieu-Daudé
  1 sibling, 1 reply; 26+ messages in thread
From: Alex Bennée @ 2019-07-01 15:58 UTC (permalink / raw)
  To: Peter Maydell; +Cc: QEMU Developers


Peter Maydell <peter.maydell@linaro.org> writes:

> On Mon, 24 Jun 2019 at 14:43, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
>>
>>   Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
<snip>
>
> OpenBSD also passes but has some output which my 'find warnings'
> grep picks up:
>
> con recv: WARNING: root is targeted by password guessing attacks,
> pubkeys are safer.

This is a warning the installer issues before it asks you the question:

  Allow root ssh login?

Again this will only occur on the initial setup.

Can you whitelist these warnings on your script?

--
Alex Bennée


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
  2019-07-01 15:58   ` Alex Bennée
@ 2019-07-02 10:10     ` Philippe Mathieu-Daudé
  2019-07-02 10:28       ` Alex Bennée
  0 siblings, 1 reply; 26+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-07-02 10:10 UTC (permalink / raw)
  To: Alex Bennée, Peter Maydell; +Cc: QEMU Developers

On 7/1/19 5:58 PM, Alex Bennée wrote:
> 
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
>> On Mon, 24 Jun 2019 at 14:43, Alex Bennée <alex.bennee@linaro.org> wrote:
>>>
>>> The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
>>>
>>>   Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
> <snip>
>>
>> OpenBSD also passes but has some output which my 'find warnings'
>> grep picks up:
>>
>> con recv: WARNING: root is targeted by password guessing attacks,
>> pubkeys are safer.
> 
> This is a warning the installer issues before it asks you the question:
> 
>   Allow root ssh login?
> 
> Again this will only occur on the initial setup.
> 
> Can you whitelist these warnings on your script?

Hmm if this is installer-only warning, why not filter it in
OpenBSDVM::build_image()?


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
  2019-07-02 10:10     ` Philippe Mathieu-Daudé
@ 2019-07-02 10:28       ` Alex Bennée
  2019-07-02 10:29         ` Peter Maydell
  0 siblings, 1 reply; 26+ messages in thread
From: Alex Bennée @ 2019-07-02 10:28 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: Peter Maydell, QEMU Developers


Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> On 7/1/19 5:58 PM, Alex Bennée wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>>> On Mon, 24 Jun 2019 at 14:43, Alex Bennée <alex.bennee@linaro.org> wrote:
>>>>
>>>> The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
>>>>
>>>>   Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
>> <snip>
>>>
>>> OpenBSD also passes but has some output which my 'find warnings'
>>> grep picks up:
>>>
>>> con recv: WARNING: root is targeted by password guessing attacks,
>>> pubkeys are safer.
>>
>> This is a warning the installer issues before it asks you the question:
>>
>>   Allow root ssh login?
>>
>> Again this will only occur on the initial setup.
>>
>> Can you whitelist these warnings on your script?
>
> Hmm if this is installer-only warning, why not filter it in
> OpenBSDVM::build_image()?

We could I guess. I'd prefer it if we could have the check for warnings
script in the source tree and documented so it's easier to see when we
trip up this stuff.

--
Alex Bennée


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix)
  2019-07-02 10:28       ` Alex Bennée
@ 2019-07-02 10:29         ` Peter Maydell
  0 siblings, 0 replies; 26+ messages in thread
From: Peter Maydell @ 2019-07-02 10:29 UTC (permalink / raw)
  To: Alex Bennée; +Cc: Philippe Mathieu-Daudé, QEMU Developers

On Tue, 2 Jul 2019 at 11:28, Alex Bennée <alex.bennee@linaro.org> wrote:
> We could I guess. I'd prefer it if we could have the check for warnings
> script in the source tree and documented so it's easier to see when we
> trip up this stuff.

Well, it's part of my ad-hoc testing scripts. If we can move
to using gitlab runners or whatever it'll become obsolete.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2019-07-02 10:32 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-24 13:43 [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 01/19] tests/vm: avoid extra compressed image copy Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 02/19] tests/vm: avoid image presence check and removal Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 03/19] tests/vm: pin ubuntu.i386 image Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 04/19] tests/vm: add source repos on ubuntu.i386 Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 05/19] tests/vm: send proxy environment variables over ssh Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 06/19] tests/vm: use ssh with pty unconditionally Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 07/19] tests/vm: run test builds on snapshot Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 08/19] tests/vm: proper guest shutdown Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 09/19] tests/vm: add vm-boot-{ssh, serial}-<guest> targets Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 10/19] tests/vm: serial console support helpers Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 11/19] tests/vm: openbsd autoinstall, using serial console Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 12/19] tests/vm: freebsd " Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 13/19] tests/vm: netbsd " Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 14/19] tests/vm: fedora " Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 15/19] tests/vm: ubuntu.i386: apt proxy setup Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 16/19] .travis.yml: default the --disable-system build to --static Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 17/19] .travis.yml: force a brew update for MacOS builds Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 18/19] Makefile: Rename the 'vm-test' target as 'vm-help' Alex Bennée
2019-06-24 13:43 ` [Qemu-devel] [PULL 19/19] target/i386: fix feature check in hyperv-stub.c Alex Bennée
2019-07-01  9:52 ` [Qemu-devel] [PULL 00/19] testing/next (tests/vm, Travis and hyperv build fix) Peter Maydell
2019-07-01 15:07   ` Alex Bennée
2019-07-01 15:58   ` Alex Bennée
2019-07-02 10:10     ` Philippe Mathieu-Daudé
2019-07-02 10:28       ` Alex Bennée
2019-07-02 10:29         ` Peter Maydell

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).