[Qemu-devel] [PATCH v8 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux)

[Qemu-devel] [PATCH v8 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux)

[Fam Zheng]

v8: Fix typo in code "od" => "os".
    QEMU to listen on 127.0.0.1 to be more secure.
    Update README.

v7: Add Alex's rev-by to patch 2.
    GPLv2 => GPLv2+ for all new files. [Peter]
    Add more description in file header to archive-source.sh for the intended
    usage.  [Peter]
    "output tarball" in usage. [Alex]
    Add quotes to "$1". [Peter]
    Put generated keys in separate files. [Alex]
    Use os.devnull. [Alex]
    More error info when launching QEMU fails. [Alex]

v6: Add license to new file. [Philippe]
    Change tests/.gitignore. [Philippe]

v5: Generate source tar file with a script.
    Fix tmpdir, use pwd.
    Reduce default -j to half cores.

v4: Drop unused imports and parameters. [Cleber]
    Use --exclude-vcs (still no --exclude-vcs-ignores because it's too new). [Philippe]
    Use gtar if available. [Philippe, Kamil]
    /dev/ld1a -> /dev/rld1a for netbsd. [Kamil]
    Only use '-enable-kvm' if /dev/kvm is there. [Kamil]
    Grammar fixes of README. [Stefan]
    Rename image on the server to include version and arch. [Kamil]
    Just ignore *.tmp. [Philippe]

v3: Drop RFC.
    Add Stefan's and Kamil's reviewed-bys.
    Use optparse. [Stefan]
    Drop the VGA patch. [Paolo, Stefan]
    Improve exit/exit code/doc. [Stefan]
    Drop unused line from basevm.py. [Stefan]
    Drop "--target-list" form Makefile.
    More intelligent '-j'.
    Add README. [Stefan]

v2: - Add docstring. [Stefan]
    - Call self._load_io_lod. [Stefan]
    - Use "info usernet" and dynamic ssh_port forwarding. [Stefan]
    - Add image checksum.
    - Use os.rename() and os.makedirs(). [Stefan]
    - Fix NetBSD URL. [Kamil]

Build tests in one 32 bit Linux guest and three BSD images are defined in this
series. This is a more managable way than the manually maintained virtual
machines in patchew. Also, one big advantage of ephemeral VMs over long running
guests is the reduced RAM usage of host, which makes it possible to have one
host test all these BSD variants and probably more.

The BSD guest templates are manually prepared following

https://wiki.qemu.org/Hosts/BSD

as it is not easy to automate. (The ideal approach is like the ubuntu.i386
script, which configures the guest on top of an official released image, fully
automatically.)

Need for help: "gmake check" in the added OpenBSD image fails with -ENOMEM
errors, even if I change "-m 2G" to "-m 8G" when starting VM. Ideas? And there
is a warning from ./configure about OpenBSD going to be unsupported in coming
releases, is it still the case?

Fam

Fam Zheng (13):
  gitignore: Ignore vm test images
  qemu.py: Add "wait()" method
  scripts: Add archive-source.sh
  tests: Add a test key pair
  tests: Add vm test lib
  tests: Add ubuntu.i386 image
  tests: Add FreeBSD image
  tests: Add NetBSD image
  tests: Add OpenBSD image
  Makefile: Add rules to run vm tests
  MAINTAINERS: Add tests/vm entry
  tests: Add README for vm tests
  docker: Use archive-source.py

 .gitignore                    |   1 +
 MAINTAINERS                   |   1 +
 Makefile                      |   2 +
 configure                     |   2 +-
 scripts/archive-source.sh     |  33 ++++++
 scripts/qemu.py               |   7 ++
 tests/.gitignore              |   1 +
 tests/docker/Makefile.include |  15 +--
 tests/docker/run              |   8 +-
 tests/keys/id_rsa             |  27 +++++
 tests/keys/id_rsa.pub         |   1 +
 tests/vm/Makefile.include     |  42 +++++++
 tests/vm/README               |  85 ++++++++++++++
 tests/vm/basevm.py            | 256 ++++++++++++++++++++++++++++++++++++++++++
 tests/vm/freebsd              |  42 +++++++
 tests/vm/netbsd               |  42 +++++++
 tests/vm/openbsd              |  43 +++++++
 tests/vm/ubuntu.i386          |  88 +++++++++++++++
 18 files changed, 675 insertions(+), 21 deletions(-)
 create mode 100755 scripts/archive-source.sh
 create mode 100644 tests/keys/id_rsa
 create mode 100644 tests/keys/id_rsa.pub
 create mode 100644 tests/vm/Makefile.include
 create mode 100644 tests/vm/README
 create mode 100755 tests/vm/basevm.py
 create mode 100755 tests/vm/freebsd
 create mode 100755 tests/vm/netbsd
 create mode 100755 tests/vm/openbsd
 create mode 100755 tests/vm/ubuntu.i386

-- 
2.13.5

[Qemu-devel] [PATCH v8 01/13] gitignore: Ignore vm test images

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 .gitignore       | 1 +
 tests/.gitignore | 1 +
 2 files changed, 2 insertions(+)

diff --git a/.gitignore b/.gitignore
index cf65316863..40acfcb9e2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -52,6 +52,7 @@
 /vscclient
 /vhost-user-scsi
 /fsdev/virtfs-proxy-helper
+*.tmp
 *.[1-9]
 *.a
 *.aux
diff --git a/tests/.gitignore b/tests/.gitignore
index fed0189a5a..cf6d99c91e 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -95,3 +95,4 @@ test-filter-mirror
 test-filter-redirector
 *-test
 qapi-schema/*.test.*
+vm/*.img
-- 
2.13.5

[Qemu-devel] [PATCH v8 02/13] qemu.py: Add "wait()" method

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 scripts/qemu.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/scripts/qemu.py b/scripts/qemu.py
index 8c67595ec8..774e201e8b 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -215,6 +215,13 @@ class QEMUMachine(object):
                 LOG.debug('Output: %r', self._iolog)
             raise
 
+    def wait(self):
+        '''Wait for the VM to power off'''
+        self._popen.wait()
+        self._qmp.close()
+        self._load_io_log()
+        self._post_shutdown()
+
     def shutdown(self):
         '''Terminate the VM and clean up'''
         if self.is_running():
-- 
2.13.5

[Qemu-devel] [PATCH v8 03/13] scripts: Add archive-source.sh

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 scripts/archive-source.sh | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
 create mode 100755 scripts/archive-source.sh

diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
new file mode 100755
index 0000000000..8b373e3090
--- /dev/null
+++ b/scripts/archive-source.sh
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# Author: Fam Zheng <famz@redhat.com>
+#
+# Archive source tree, including submodules. This is created for test code to
+# export the source files, in order to be built in a different enviornment,
+# such as in a docker instance or VM.
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+
+set -e
+
+if test $# -lt 1; then
+    echo "Usage: $0 <output tarball>"
+    exit 1
+fi
+
+submodules=$(git submodule foreach --recursive --quiet 'echo $name')
+
+if test -n "$submodules"; then
+    {
+        git ls-files
+        for sm in $submodules; do
+            (cd $sm; git ls-files) | sed "s:^:$sm/:"
+        done
+    } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$1".list
+else
+    git ls-files > "$1".list
+fi
+
+tar -cf "$1" -T "$1".list
+rm "$1".list
-- 
2.13.5

[Qemu-devel] [PATCH v8 04/13] tests: Add a test key pair

[Fam Zheng]

This will be used by setup test user ssh.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/keys/id_rsa     | 27 +++++++++++++++++++++++++++
 tests/keys/id_rsa.pub |  1 +
 2 files changed, 28 insertions(+)
 create mode 100644 tests/keys/id_rsa
 create mode 100644 tests/keys/id_rsa.pub

diff --git a/tests/keys/id_rsa b/tests/keys/id_rsa
new file mode 100644
index 0000000000..2933eac3db
--- /dev/null
+++ b/tests/keys/id_rsa
@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEAopAuOlmLV6LVHdFBj8/eeOwI9CqguIJPp7eAQSZvOiB4Ag/R
+coEhl/RBbrV5Yc/SmSD4PTpJO/iM10RwliNjDb4a3I8q3sykRJu9c9PI/YsH8WN9
++NH2NjKPtJIcKTu287IM5JYxyB6nDoOzILbTyJ1TDR/xH6qYEfBAyiblggdjcvhA
+RTf93QIn39F/xLypXvT1K2O9BJEsnJ8lEUvB2UXhKo/JTfSeZF8wPBeowaP9EONk
+7b+nuJOWHGg68Ji6wVi62tjwl2Szch6lxIhZBpnV7QNRKMfYHP6eIyF4pusazzZq
+Telsq6xI2ghecWLzb/MF5A+rklsGx2FNuJSAJwIDAQABAoIBAHHi4o/8VZNivz0x
+cWXn8erzKV6tUoWQvW85Lj/2RiwJvSlsnYZDkx5af1CpEE2HA/pFT8PNRqsd+MWC
+7AEy710cVsM4BYerBFYQaYxwzblaoojo88LSjVPw3h5Z0iLM8+IMVd36nwuc9dpE
+R8TecMZ1+U4Tl6BgqkK+9xToZRdPKdjS8L5MoFhGN+xY0vRbbJbGaV9Q0IHxLBkB
+rEBV7T1mUynneCHRUQlJQEwJmKpT8MH3IjsUXlG5YvnuuvcQJSNTaW2iDLxuOKp8
+cxW8+qL88zpb1D5dppoIu6rlrugN0azSq70ruFJQPc/A8GQrDKoGgRQiagxNY3u+
+vHZzXlECgYEA0dKO3gfkSxsDBb94sQwskMScqLhcKhztEa8kPxTx6Yqh+x8/scx3
+XhJyOt669P8U1v8a/2Al+s81oZzzfQSzO1Q7gEwSrgBcRMSIoRBUw9uYcy02ngb/
+j/ng3DGivfJztjjiSJwb46FHkJ2JR8mF2UisC6UMXk3NgFY/3vWQx78CgYEAxlcG
+T3hfSWSmTgKRczMJuHQOX9ULfTBIqwP5VqkkkiavzigGRirzb5lgnmuTSPTpF0LB
+XVPjR2M4q+7gzP0Dca3pocrvLEoxjwIKnCbYKnyyvnUoE9qHv4Kr+vDbgWpa2LXG
+JbLmE7tgTCIp20jOPPT4xuDvlbzQZBJ5qCQSoZkCgYEAgrotSSihlCnAOFSTXbu4
+CHp3IKe8xIBBNENq0eK61kcJpOxTQvOha3sSsJsU4JAM6+cFaxb8kseHIqonCj1j
+bhOM/uJmwQJ4el/4wGDsbxriYOBKpyq1D38gGhDS1IW6kk3erl6VAb36WJ/OaGum
+eTpN9vNeQWM4Jj2WjdNx4QECgYAwTdd6mU1TmZCrJRL5ZG+0nYc2rbMrnQvFoqUi
+BvWiJovggHzur90zy73tNzPaq9Ls2FQxf5G1vCN8NCRJqEEjeYCR59OSDMu/EXc2
+CnvQ9SevHOdS1oEDEjcCWZCMFzPi3XpRih1gptzQDe31uuiHjf3cqcGPzTlPdfRt
+D8P92QKBgC4UaBvIRwREVJsdZzpIzm224Bpe8LOmA7DeTnjlT0b3lkGiBJ36/Q0p
+VhYh/6cjX4/iuIs7gJbGon7B+YPB8scmOi3fj0+nkJAONue1mMfBNkba6qQTc6Y2
+5mEKw2/O7/JpND7ucU3OK9plcw/qnrWDgHxl0Iz95+OzUIIagxne
+-----END RSA PRIVATE KEY-----
diff --git a/tests/keys/id_rsa.pub b/tests/keys/id_rsa.pub
new file mode 100644
index 0000000000..d9888e312f
--- /dev/null
+++ b/tests/keys/id_rsa.pub
@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCikC46WYtXotUd0UGPz9547Aj0KqC4gk+nt4BBJm86IHgCD9FygSGX9EFutXlhz9KZIPg9Okk7+IzXRHCWI2MNvhrcjyrezKREm71z08j9iwfxY3340fY2Mo+0khwpO7bzsgzkljHIHqcOg7MgttPInVMNH/EfqpgR8EDKJuWCB2Ny+EBFN/3dAiff0X/EvKle9PUrY70EkSycnyURS8HZReEqj8lN9J5kXzA8F6jBo/0Q42Ttv6e4k5YcaDrwmLrBWLra2PCXZLNyHqXEiFkGmdXtA1Eox9gc/p4jIXim6xrPNmpN6WyrrEjaCF5xYvNv8wXkD6uSWwbHYU24lIAn qemu-test
-- 
2.13.5

[Qemu-devel] [PATCH v8 05/13] tests: Add vm test lib

[Fam Zheng]

This is the common code to implement a "VM test" to

  1) Download and initialize a pre-defined VM that has necessary
  dependencies to build QEMU and SSH access.

  2) Archive $SRC_PATH to a .tar file.

  3) Boot the VM, and pass the source tar file to the guest.

  4) SSH into the VM, untar the source tarball, build from the source.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/vm/basevm.py | 256 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 256 insertions(+)
 create mode 100755 tests/vm/basevm.py

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
new file mode 100755
index 0000000000..77d07b161d
--- /dev/null
+++ b/tests/vm/basevm.py
@@ -0,0 +1,256 @@
+#!/usr/bin/env python
+#
+# VM testing base class
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import logging
+import time
+import datetime
+sys.path.append(os.path.join(os.path.dirname(__file__), "..", "..", "scripts"))
+from qemu import QEMUMachine
+import subprocess
+import hashlib
+import optparse
+import atexit
+import tempfile
+import shutil
+import multiprocessing
+import traceback
+
+SSH_KEY = open(os.path.join(os.path.dirname(__file__),
+               "..", "keys", "id_rsa")).read()
+SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__),
+                   "..", "keys", "id_rsa.pub")).read()
+
+class BaseVM(object):
+    GUEST_USER = "qemu"
+    GUEST_PASS = "qemupass"
+    ROOT_PASS = "qemupass"
+
+    # The script to run in the guest that builds QEMU
+    BUILD_SCRIPT = ""
+    # The guest name, to be overridden by subclasses
+    name = "#base"
+    def __init__(self, debug=False, vcpus=None):
+        self._guest = None
+        self._tmpdir = tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")
+        atexit.register(shutil.rmtree, self._tmpdir)
+
+        self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa")
+        open(self._ssh_key_file, "w").write(SSH_KEY)
+        subprocess.check_call(["chmod", "600", self._ssh_key_file])
+
+        self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub")
+        open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY)
+
+        self.debug = debug
+        self._stderr = sys.stderr
+        self._devnull = open(os.devnull, "w")
+        if self.debug:
+            self._stdout = sys.stdout
+        else:
+            self._stdout = self._devnull
+        self._args = [ \
+            "-nodefaults", "-m", "2G",
+            "-cpu", "host",
+            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
+            "-device", "virtio-net-pci,netdev=vnet",
+            "-vnc", "127.0.0.1:0,to=20",
+            "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
+        if vcpus:
+            self._args += ["-smp", str(vcpus)]
+        if os.access("/dev/kvm", os.R_OK | os.W_OK):
+            self._args += ["-enable-kvm"]
+        else:
+            logging.info("KVM not available, not using -enable-kvm")
+        self._data_args = []
+
+    def _download_with_cache(self, url, sha256sum=None):
+        def check_sha256sum(fname):
+            if not sha256sum:
+                return True
+            checksum = subprocess.check_output(["sha256sum", fname]).split()[0]
+            return sha256sum == checksum
+
+        cache_dir = os.path.expanduser("~/.cache/qemu-vm/download")
+        if not os.path.exists(cache_dir):
+            os.makedirs(cache_dir)
+        fname = os.path.join(cache_dir, hashlib.sha1(url).hexdigest())
+        if os.path.exists(fname) and check_sha256sum(fname):
+            return fname
+        logging.debug("Downloading %s to %s...", url, fname)
+        subprocess.check_call(["wget", "-c", url, "-O", fname + ".download"],
+                              stdout=self._stdout, stderr=self._stderr)
+        os.rename(fname + ".download", fname)
+        return fname
+
+    def _ssh_do(self, user, cmd, check, interactive=False):
+        ssh_cmd = ["ssh", "-q",
+                   "-o", "StrictHostKeyChecking=no",
+                   "-o", "UserKnownHostsFile=" + os.devnull,
+                   "-o", "ConnectTimeout=1",
+                   "-p", self.ssh_port, "-i", self._ssh_key_file]
+        if interactive:
+            ssh_cmd += ['-t']
+        assert not isinstance(cmd, str)
+        ssh_cmd += ["%s@127.0.0.1" % user] + list(cmd)
+        logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
+        r = subprocess.call(ssh_cmd,
+                            stdin=sys.stdin if interactive else self._devnull,
+                            stdout=sys.stdout if interactive else self._stdout,
+                            stderr=sys.stderr if interactive else self._stderr)
+        if check and r != 0:
+            raise Exception("SSH command failed: %s" % cmd)
+        return r
+
+    def ssh(self, *cmd):
+        return self._ssh_do(self.GUEST_USER, cmd, False)
+
+    def ssh_interactive(self, *cmd):
+        return self._ssh_do(self.GUEST_USER, cmd, False, True)
+
+    def ssh_root(self, *cmd):
+        return self._ssh_do("root", cmd, False)
+
+    def ssh_check(self, *cmd):
+        self._ssh_do(self.GUEST_USER, cmd, True)
+
+    def ssh_root_check(self, *cmd):
+        self._ssh_do("root", cmd, True)
+
+    def build_image(self, img):
+        raise NotImplementedError
+
+    def add_source_dir(self, src_dir):
+        name = "data-" + hashlib.sha1(src_dir).hexdigest()[:5]
+        tarfile = os.path.join(self._tmpdir, name + ".tar")
+        logging.debug("Creating archive %s for src_dir dir: %s", tarfile, src_dir)
+        subprocess.check_call(["./scripts/archive-source.sh", tarfile],
+                              cwd=src_dir, stdin=self._devnull,
+                              stdout=self._stdout, stderr=self._stderr)
+        self._data_args += ["-drive",
+                            "file=%s,if=none,id=%s,cache=writeback,format=raw" % \
+                                    (tarfile, name),
+                            "-device",
+                            "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
+
+    def boot(self, img, extra_args=[]):
+        args = self._args + [
+            "-device", "VGA",
+            "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
+            "-device", "virtio-blk,drive=drive0,bootindex=0"]
+        args += self._data_args + extra_args
+        logging.debug("QEMU args: %s", " ".join(args))
+        qemu_bin = os.environ.get("QEMU", "qemu-system-x86_64")
+        guest = QEMUMachine(binary=qemu_bin, args=args)
+        try:
+            guest.launch()
+        except:
+            logging.error("Failed to launch QEMU, command line:")
+            logging.error(" ".join([qemu_bin] + args))
+            logging.error("Log:")
+            logging.error(guest.get_log())
+            logging.error("QEMU version >= 2.10 is required")
+            raise
+        atexit.register(self.shutdown)
+        self._guest = guest
+        usernet_info = guest.qmp("human-monitor-command",
+                                 command_line="info usernet")
+        self.ssh_port = None
+        for l in usernet_info["return"].splitlines():
+            fields = l.split()
+            if "TCP[HOST_FORWARD]" in fields and "22" in fields:
+                self.ssh_port = l.split()[3]
+        if not self.ssh_port:
+            raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \
+                            usernet_info)
+
+    def wait_ssh(self, seconds=120):
+        starttime = datetime.datetime.now()
+        guest_up = False
+        while (datetime.datetime.now() - starttime).total_seconds() < seconds:
+            if self.ssh("exit 0") == 0:
+                guest_up = True
+                break
+            time.sleep(1)
+        if not guest_up:
+            raise Exception("Timeout while waiting for guest ssh")
+
+    def shutdown(self):
+        self._guest.shutdown()
+
+    def wait(self):
+        self._guest.wait()
+
+    def qmp(self, *args, **kwargs):
+        return self._guest.qmp(*args, **kwargs)
+
+def parse_args(vm_name):
+    parser = optparse.OptionParser(description="""
+    VM test utility.  Exit codes: 0 = success, 1 = command line error, 2 = environment initialization failed, 3 = test command failed""")
+    parser.add_option("--debug", "-D", action="store_true",
+                      help="enable debug output")
+    parser.add_option("--image", "-i", default="%s.img" % vm_name,
+                      help="image file name")
+    parser.add_option("--force", "-f", action="store_true",
+                      help="force build image even if image exists")
+    parser.add_option("--jobs", type=int, default=multiprocessing.cpu_count() / 2,
+                      help="number of virtual CPUs")
+    parser.add_option("--build-image", "-b", action="store_true",
+                      help="build image")
+    parser.add_option("--build-qemu",
+                      help="build QEMU from source in guest")
+    parser.add_option("--interactive", "-I", action="store_true",
+                      help="Interactively run command")
+    parser.disable_interspersed_args()
+    return parser.parse_args()
+
+def main(vmcls):
+    try:
+        args, argv = parse_args(vmcls.name)
+        if not argv and not args.build_qemu and not args.build_image:
+            print "Nothing to do?"
+            return 1
+        if args.debug:
+            logging.getLogger().setLevel(logging.DEBUG)
+        vm = vmcls(debug=args.debug, vcpus=args.jobs)
+        if args.build_image:
+            if os.path.exists(args.image) and not args.force:
+                sys.stderr.writelines(["Image file exists: %s\n" % args.image,
+                                      "Use --force option to overwrite\n"])
+                return 1
+            return vm.build_image(args.image)
+        if args.build_qemu:
+            vm.add_source_dir(args.build_qemu)
+            cmd = [vm.BUILD_SCRIPT.format(
+                   configure_opts = " ".join(argv),
+                   jobs=args.jobs)]
+        else:
+            cmd = argv
+        vm.boot(args.image + ",snapshot=on")
+        vm.wait_ssh()
+    except Exception as e:
+        if isinstance(e, SystemExit) and e.code == 0:
+            return 0
+        sys.stderr.write("Failed to prepare guest environment\n")
+        traceback.print_exc()
+        return 2
+
+    if args.interactive:
+        if vm.ssh_interactive(*cmd) == 0:
+            return 0
+        vm.ssh_interactive()
+        return 3
+    else:
+        if vm.ssh(*cmd) != 0:
+            return 3
-- 
2.13.5

[Qemu-devel] [PATCH v8 06/13] tests: Add ubuntu.i386 image

[Fam Zheng]

This adds a 32bit guest.

The official LTS cloud image is downloaded and initialized with
cloud-init.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/vm/ubuntu.i386 | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
 create mode 100755 tests/vm/ubuntu.i386

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
new file mode 100755
index 0000000000..e70dcb89ce
--- /dev/null
+++ b/tests/vm/ubuntu.i386
@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+#
+# Ubuntu i386 image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+import time
+
+class UbuntuX86VM(basevm.BaseVM):
+    name = "ubuntu.i386"
+    BUILD_SCRIPT = """
+        set -e;
+        cd $(mktemp -d);
+        sudo chmod a+r /dev/vdb;
+        tar -xf /dev/vdb;
+        ./configure {configure_opts};
+        make -j{jobs};
+        make check;
+    """
+
+    def _gen_cloud_init_iso(self):
+        cidir = self._tmpdir
+        mdata = open(os.path.join(cidir, "meta-data"), "w")
+        mdata.writelines(["instance-id: ubuntu-vm-0\n",
+                         "local-hostname: ubuntu-guest\n"])
+        mdata.close()
+        udata = open(os.path.join(cidir, "user-data"), "w")
+        udata.writelines(["#cloud-config\n",
+                          "chpasswd:\n",
+                          "  list: |\n",
+                          "    root:%s\n" % self.ROOT_PASS,
+                          "    %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
+                          "  expire: False\n",
+                          "users:\n",
+                          "  - name: %s\n" % self.GUEST_USER,
+                          "    sudo: ALL=(ALL) NOPASSWD:ALL\n",
+                          "    ssh-authorized-keys:\n",
+                          "    - %s\n" % basevm.SSH_PUB_KEY,
+                          "  - name: root\n",
+                          "    ssh-authorized-keys:\n",
+                          "    - %s\n" % basevm.SSH_PUB_KEY])
+        udata.close()
+        subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
+                               "-volid", "cidata", "-joliet", "-rock",
+                               "user-data", "meta-data"],
+                               cwd=cidir,
+                               stdin=self._devnull, stdout=self._stdout,
+                               stderr=self._stdout)
+        return os.path.join(cidir, "cloud-init.iso")
+
+    def build_image(self, img):
+        cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
+        img_tmp = img + ".tmp"
+        subprocess.check_call(["cp", "-f", cimg, img_tmp])
+        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
+        self.wait_ssh()
+        self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
+        self.ssh_root_check("apt-get update")
+        self.ssh_root_check("apt-get install -y cloud-initramfs-growroot")
+        # Don't check the status in case the guest hang up too quickly
+        self.ssh_root("sync && reboot")
+        time.sleep(5)
+        self.wait_ssh()
+        # The previous update sometimes doesn't survive a reboot, so do it again
+        self.ssh_root_check("apt-get update")
+        self.ssh_root_check("apt-get build-dep -y qemu")
+        self.ssh_root_check("apt-get install -y libfdt-dev")
+        self.ssh_root("poweroff")
+        self.wait()
+        if os.path.exists(img):
+            os.remove(img)
+        os.rename(img_tmp, img)
+        return 0
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(UbuntuX86VM))
-- 
2.13.5

[Qemu-devel] [PATCH v8 07/13] tests: Add FreeBSD image

[Fam Zheng]

The image is prepared following instructions as in:

https://wiki.qemu.org/Hosts/BSD

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/vm/freebsd | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100755 tests/vm/freebsd

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
new file mode 100755
index 0000000000..039dad8f69
--- /dev/null
+++ b/tests/vm/freebsd
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+#
+# FreeBSD VM image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+
+class FreeBSDVM(basevm.BaseVM):
+    name = "freebsd"
+    BUILD_SCRIPT = """
+        set -e;
+        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        tar -xf /dev/vtbd1;
+        ./configure {configure_opts};
+        gmake -j{jobs};
+        gmake check;
+    """
+
+    def build_image(self, img):
+        cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz",
+                sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
+        img_tmp_xz = img + ".tmp.xz"
+        img_tmp = img + ".tmp"
+        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "-df", img_tmp_xz])
+        if os.path.exists(img):
+            os.remove(img)
+        os.rename(img_tmp, img)
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(FreeBSDVM))
-- 
2.13.5

[Qemu-devel] [PATCH v8 08/13] tests: Add NetBSD image

[Fam Zheng]

The image is prepared following instructions as in:

https://wiki.qemu.org/Hosts/BSD

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Kamil Rytarowski <n54@gmx.com>
---
 tests/vm/netbsd | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100755 tests/vm/netbsd

diff --git a/tests/vm/netbsd b/tests/vm/netbsd
new file mode 100755
index 0000000000..3972d8b45c
--- /dev/null
+++ b/tests/vm/netbsd
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+#
+# NetBSD VM image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+
+class NetBSDVM(basevm.BaseVM):
+    name = "netbsd"
+    BUILD_SCRIPT = """
+        set -e;
+        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        tar -xf /dev/rld1a;
+        ./configure --python=python2.7 {configure_opts};
+        gmake -j{jobs};
+        gmake check;
+    """
+
+    def build_image(self, img):
+        cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz",
+                                         sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732')
+        img_tmp_xz = img + ".tmp.xz"
+        img_tmp = img + ".tmp"
+        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "-df", img_tmp_xz])
+        if os.path.exists(img):
+            os.remove(img)
+        os.rename(img_tmp, img)
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(NetBSDVM))
-- 
2.13.5

[Qemu-devel] [PATCH v8 09/13] tests: Add OpenBSD image

[Fam Zheng]

The image is prepared following instructions as in:

https://wiki.qemu.org/Hosts/BSD

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/vm/openbsd | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100755 tests/vm/openbsd

diff --git a/tests/vm/openbsd b/tests/vm/openbsd
new file mode 100755
index 0000000000..6ae16d97fd
--- /dev/null
+++ b/tests/vm/openbsd
@@ -0,0 +1,43 @@
+#!/usr/bin/env python
+#
+# OpenBSD VM image
+#
+# Copyright 2017 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+
+class OpenBSDVM(basevm.BaseVM):
+    name = "openbsd"
+    BUILD_SCRIPT = """
+        set -e;
+        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        tar -xf /dev/rsd1c;
+        ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
+        gmake -j{jobs};
+        # XXX: "gmake check" seems to always hang or fail
+        #gmake check;
+    """
+
+    def build_image(self, img):
+        cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
+                sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
+        img_tmp_xz = img + ".tmp.xz"
+        img_tmp = img + ".tmp"
+        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "-df", img_tmp_xz])
+        if os.path.exists(img):
+            os.remove(img)
+        os.rename(img_tmp, img)
+
+if __name__ == "__main__":
+    sys.exit(basevm.main(OpenBSDVM))
-- 
2.13.5

[Qemu-devel] [PATCH v8 10/13] Makefile: Add rules to run vm tests

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile                  |  2 ++
 configure                 |  2 +-
 tests/vm/Makefile.include | 42 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+), 1 deletion(-)
 create mode 100644 tests/vm/Makefile.include

diff --git a/Makefile b/Makefile
index b53fc69a60..79684ab138 100644
--- a/Makefile
+++ b/Makefile
@@ -812,6 +812,7 @@ endif
 -include $(wildcard *.d tests/*.d)
 
 include $(SRC_PATH)/tests/docker/Makefile.include
+include $(SRC_PATH)/tests/vm/Makefile.include
 
 .PHONY: help
 help:
@@ -835,6 +836,7 @@ help:
 	@echo  'Test targets:'
 	@echo  '  check           - Run all tests (check-help for details)'
 	@echo  '  docker          - Help about targets running tests inside Docker containers'
+	@echo  '  vm-test         - Help about targets running tests inside VM'
 	@echo  ''
 	@echo  'Documentation targets:'
 	@echo  '  html info pdf txt'
diff --git a/configure b/configure
index 94db2d103e..bbf053e3e0 100755
--- a/configure
+++ b/configure
@@ -6505,7 +6505,7 @@ if test "$ccache_cpp2" = "yes"; then
 fi
 
 # build tree in object directory in case the source is not in the current directory
-DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests"
+DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests tests/vm"
 DIRS="$DIRS docs docs/interop fsdev"
 DIRS="$DIRS pc-bios/optionrom pc-bios/spapr-rtas pc-bios/s390-ccw"
 DIRS="$DIRS roms/seabios roms/vgabios"
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
new file mode 100644
index 0000000000..5daa2a3b73
--- /dev/null
+++ b/tests/vm/Makefile.include
@@ -0,0 +1,42 @@
+# Makefile for VM tests
+
+.PHONY: vm-build-all
+
+IMAGES := ubuntu.i386 freebsd netbsd openbsd
+IMAGE_FILES := $(patsubst %, tests/vm/%.img, $(IMAGES))
+
+.PRECIOUS: $(IMAGE_FILES)
+
+vm-test:
+	@echo "vm-test: Test QEMU in preconfigured virtual machines"
+	@echo
+	@echo "  vm-build-ubuntu.i386            - Build QEMU in ubuntu i386 VM"
+	@echo "  vm-build-freebsd                - Build QEMU in FreeBSD VM"
+	@echo "  vm-build-netbsd                 - Build QEMU in NetBSD VM"
+	@echo "  vm-build-openbsd                - Build QEMU in OpenBSD VM"
+
+vm-build-all: $(addprefix vm-build-, $(IMAGES))
+
+tests/vm/%.img: $(SRC_PATH)/tests/vm/% \
+		$(SRC_PATH)/tests/vm/basevm.py \
+		$(SRC_PATH)/tests/vm/Makefile.include
+	$(call quiet-command, \
+		$< \
+		$(if $(V)$(DEBUG), --debug) \
+		--image "$@" \
+		--force \
+		--build-image $@, \
+		"  VM-IMAGE $*")
+
+
+# Build in VM $(IMAGE)
+vm-build-%: tests/vm/%.img
+	$(call quiet-command, \
+		$(SRC_PATH)/tests/vm/$* \
+		$(if $(V)$(DEBUG), --debug) \
+		$(if $(DEBUG), --interactive) \
+		$(if $(J),--jobs $(J)) \
+		--image "$<" \
+		--build-qemu $(SRC_PATH), \
+		"  VM-BUILD $*")
+
-- 
2.13.5

[Qemu-devel] [PATCH v8 11/13] MAINTAINERS: Add tests/vm entry

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 4bd1797330..e9be7faff7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1890,6 +1890,7 @@ S: Maintained
 F: .travis.yml
 F: .shippable.yml
 F: tests/docker/
+F: tests/vm/
 W: https://travis-ci.org/qemu/qemu
 W: https://app.shippable.com/github/qemu/qemu
 W: http://patchew.org/QEMU/
-- 
2.13.5

[Qemu-devel] [PATCH v8 12/13] tests: Add README for vm tests

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/vm/README | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)
 create mode 100644 tests/vm/README

diff --git a/tests/vm/README b/tests/vm/README
new file mode 100644
index 0000000000..fa30a79ea7
--- /dev/null
+++ b/tests/vm/README
@@ -0,0 +1,85 @@
+=== VM test suite to run build in guests ===
+
+== Intro ==
+
+This test suite contains scripts that bootstrap various guest images that have
+necessary packages to build QEMU. The basic usage is documented in Makefile
+help which is displayed with "make vm-test".
+
+== Quick start ==
+
+Run "make vm-test" to list available make targets. Invoke a specific make
+command to run build test in an image. For example, "make vm-build-freebsd"
+will build the source tree in the FreeBSD image. The command can be executed
+from either the source tree or the build dir; if the former, ./configure is not
+needed. The command will then generate the test image in ./tests/vm/ under the
+working directory.
+
+== QEMU binary ==
+
+By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
+isn't one, or if it is older than 2.10, the test won't work. In this case,
+provide the QEMU binary in env var: QEMU=/path/to/qemu-2.10+.
+
+== Make jobs ==
+
+The "-j$X" option in the make command line is not propagated into the VM,
+specify "J=$X" to control the make jobs in the guest.
+
+== Debugging ==
+
+Add "DEBUG=1" and/or "V=1" to the make command to allow interactive debugging
+and verbose output. If this is not enough, see the next section.
+
+== Manual invocation ==
+
+Each guest script is an executable script with the same command line options.
+For example to work with the netbsd guest, use $QEMU_SRC/tests/vm/netbsd:
+
+    $ cd $QEMU_SRC/tests/vm
+
+    # To bootstrap the image
+    $ ./netbsd --build-image --image /var/tmp/netbsd.img
+    <...>
+
+    # To run an arbitrary command in guest (the output will not be echoed unless
+    # --debug is added)
+    $ ./netbsd --debug --image /var/tmp/netbsd.img uname -a
+
+    # To build QEMU in guest
+    $ ./netbsd --debug --image /var/tmp/netbsd.img --build-qemu $QEMU_SRC
+
+    # To get to an interactive shell
+    $ ./netbsd --interactive --image /var/tmp/netbsd.img sh
+
+== Adding new guests ==
+
+Please look at existing guest scripts for how to add new guests.
+
+Most importantly, create a subclass of BaseVM and implement build_image()
+method and define BUILD_SCRIPT, then finally call basevm.main() from the
+script's main().
+
+  - Usually in build_image(), a template image is downloaded from a predefined
+    URL. BaseVM._download_with_cache() takes care of the cache and the
+    checksum, so consider using it.
+
+  - Once the image is downloaded, users, SSH server and QEMU build deps should
+    be set up:
+
+    * Root password set to BaseVM.ROOT_PASS
+    * User BaseVM.GUEST_USER is created, and password set to BaseVM.GUEST_PASS
+    * SSH service is enabled and started on boot,
+      $QEMU_SRC/tests/keys/id_rsa.pub is added to ssh's "authorized_keys" file
+      of both root and the normal user
+    * DHCP client service is enabled and started on boot, so that it can
+      automatically configure the virtio-net-pci NIC and communicate with QEMU
+      user net (10.0.2.2)
+    * Necessary packages are installed to untar the source tarball and build
+      QEMU
+
+  - Write a proper BUILD_SCRIPT template, which should be a shell script that
+    untars a raw virtio-blk block device, which is the tarball data blob of the
+    QEMU source tree, then configure/build it. Running "make check" is also
+    recommended.
+
-- 
2.13.5

[Qemu-devel] [PATCH v8 13/13] docker: Use archive-source.py

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/Makefile.include | 15 ++-------------
 tests/docker/run              |  8 +-------
 2 files changed, 3 insertions(+), 20 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index aaab1a4208..7a027d5bd6 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -17,24 +17,13 @@ DOCKER_TOOLS := travis
 TESTS ?= %
 IMAGES ?= %
 
-# Make archive from git repo $1 to tar.gz $2
-make-archive-maybe = $(if $(wildcard $1/*), \
-	$(call quiet-command, \
-		(cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
-			git archive -1 HEAD --format=tar.gz; \
-		else \
-			git archive -1 $$(git stash create) --format=tar.gz; \
-		fi) > $2, \
-		"ARCHIVE","$(notdir $2)"))
-
 CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$)
 DOCKER_SRC_COPY := docker-src.$(CUR_TIME)
 
 $(DOCKER_SRC_COPY):
 	@mkdir $@
-	$(call make-archive-maybe, $(SRC_PATH), $@/qemu.tgz)
-	$(call make-archive-maybe, $(SRC_PATH)/dtc, $@/dtc.tgz)
-	$(call make-archive-maybe, $(SRC_PATH)/pixman, $@/pixman.tgz)
+	$(call quiet-command, $(SRC_PATH)/scripts/archive-source.sh $@/qemu.tar, \
+		"GEN", "$@/qemu.tar")
 	$(call quiet-command, cp $(SRC_PATH)/tests/docker/run $@/run, \
 		"COPY","RUNNER")
 
diff --git a/tests/docker/run b/tests/docker/run
index c1e4513bce..9eb9165f76 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -32,13 +32,7 @@ export TEST_DIR=/tmp/qemu-test
 mkdir -p $TEST_DIR/{src,build,install}
 
 # Extract the source tarballs
-tar -C $TEST_DIR/src -xzf $BASE/qemu.tgz
-for p in dtc pixman; do
-    if test -f $BASE/$p.tgz; then
-        tar -C $TEST_DIR/src/$p -xzf $BASE/$p.tgz
-        export FEATURES="$FEATURES $p"
-    fi
-done
+tar -C $TEST_DIR/src -xf $BASE/qemu.tar
 
 if test -n "$SHOW_ENV"; then
     if test -f /packages.txt; then
-- 
2.13.5

Re: [Qemu-devel] [PATCH v8 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux)

[no-reply]

Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v8 00/13] tests: Add VM based build tests (for non-x86_64 and/or non-Linux)
Message-id: 20170918024402.3265-1-famz@redhat.com
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
eada89cb49 docker: Use archive-source.py
edb516101b tests: Add README for vm tests
29ab56eed6 MAINTAINERS: Add tests/vm entry
91bbe52a91 Makefile: Add rules to run vm tests
850be050ce tests: Add OpenBSD image
793d0cde6b tests: Add NetBSD image
4fcfdb2879 tests: Add FreeBSD image
0186cba9b3 tests: Add ubuntu.i386 image
2cc2c41291 tests: Add vm test lib
c6a859d5de tests: Add a test key pair
ffc8ca2de4 scripts: Add archive-source.sh
0df34534b9 qemu.py: Add "wait()" method
7ed9b96668 gitignore: Ignore vm test images

=== OUTPUT BEGIN ===
Checking PATCH 1/13: gitignore: Ignore vm test images...
Checking PATCH 2/13: qemu.py: Add "wait()" method...
Checking PATCH 3/13: scripts: Add archive-source.sh...
Checking PATCH 4/13: tests: Add a test key pair...
Checking PATCH 5/13: tests: Add vm test lib...
WARNING: line over 80 characters
#71: FILE: tests/vm/basevm.py:46:
+        self._tmpdir = tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")

WARNING: line over 80 characters
#162: FILE: tests/vm/basevm.py:137:
+        logging.debug("Creating archive %s for src_dir dir: %s", tarfile, src_dir)

WARNING: line over 80 characters
#167: FILE: tests/vm/basevm.py:142:
+                            "file=%s,if=none,id=%s,cache=writeback,format=raw" % \

WARNING: line over 80 characters
#170: FILE: tests/vm/basevm.py:145:
+                            "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]

ERROR: line over 90 characters
#225: FILE: tests/vm/basevm.py:200:
+    VM test utility.  Exit codes: 0 = success, 1 = command line error, 2 = environment initialization failed, 3 = test command failed""")

WARNING: line over 80 characters
#232: FILE: tests/vm/basevm.py:207:
+    parser.add_option("--jobs", type=int, default=multiprocessing.cpu_count() / 2,

total: 1 errors, 5 warnings, 256 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 6/13: tests: Add ubuntu.i386 image...
Checking PATCH 7/13: tests: Add FreeBSD image...
Checking PATCH 8/13: tests: Add NetBSD image...
Checking PATCH 9/13: tests: Add OpenBSD image...
Checking PATCH 10/13: Makefile: Add rules to run vm tests...
Checking PATCH 11/13: MAINTAINERS: Add tests/vm entry...
Checking PATCH 12/13: tests: Add README for vm tests...
Checking PATCH 13/13: docker: Use archive-source.py...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org

Re: [Qemu-devel] [PATCH v8 03/13] scripts: Add archive-source.sh

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> Signed-off-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  scripts/archive-source.sh | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
>  create mode 100755 scripts/archive-source.sh
>
> diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
> new file mode 100755
> index 0000000000..8b373e3090
> --- /dev/null
> +++ b/scripts/archive-source.sh
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# Author: Fam Zheng <famz@redhat.com>
> +#
> +# Archive source tree, including submodules. This is created for test code to
> +# export the source files, in order to be built in a different enviornment,
> +# such as in a docker instance or VM.
> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +
> +set -e
> +
> +if test $# -lt 1; then
> +    echo "Usage: $0 <output tarball>"
> +    exit 1
> +fi
> +
> +submodules=$(git submodule foreach --recursive --quiet 'echo $name')
> +
> +if test -n "$submodules"; then
> +    {
> +        git ls-files
> +        for sm in $submodules; do
> +            (cd $sm; git ls-files) | sed "s:^:$sm/:"
> +        done
> +    } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$1".list
> +else
> +    git ls-files > "$1".list
> +fi
> +
> +tar -cf "$1" -T "$1".list
> +rm "$1".list


--
Alex Bennée

Re: [Qemu-devel] [PATCH v8 13/13] docker: Use archive-source.py

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> Signed-off-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/docker/Makefile.include | 15 ++-------------
>  tests/docker/run              |  8 +-------
>  2 files changed, 3 insertions(+), 20 deletions(-)
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index aaab1a4208..7a027d5bd6 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -17,24 +17,13 @@ DOCKER_TOOLS := travis
>  TESTS ?= %
>  IMAGES ?= %
>
> -# Make archive from git repo $1 to tar.gz $2
> -make-archive-maybe = $(if $(wildcard $1/*), \
> -	$(call quiet-command, \
> -		(cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
> -			git archive -1 HEAD --format=tar.gz; \
> -		else \
> -			git archive -1 $$(git stash create) --format=tar.gz; \
> -		fi) > $2, \
> -		"ARCHIVE","$(notdir $2)"))
> -
>  CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$)
>  DOCKER_SRC_COPY := docker-src.$(CUR_TIME)
>
>  $(DOCKER_SRC_COPY):
>  	@mkdir $@
> -	$(call make-archive-maybe, $(SRC_PATH), $@/qemu.tgz)
> -	$(call make-archive-maybe, $(SRC_PATH)/dtc, $@/dtc.tgz)
> -	$(call make-archive-maybe, $(SRC_PATH)/pixman, $@/pixman.tgz)
> +	$(call quiet-command, $(SRC_PATH)/scripts/archive-source.sh $@/qemu.tar, \
> +		"GEN", "$@/qemu.tar")
>  	$(call quiet-command, cp $(SRC_PATH)/tests/docker/run $@/run, \
>  		"COPY","RUNNER")
>
> diff --git a/tests/docker/run b/tests/docker/run
> index c1e4513bce..9eb9165f76 100755
> --- a/tests/docker/run
> +++ b/tests/docker/run
> @@ -32,13 +32,7 @@ export TEST_DIR=/tmp/qemu-test
>  mkdir -p $TEST_DIR/{src,build,install}
>
>  # Extract the source tarballs
> -tar -C $TEST_DIR/src -xzf $BASE/qemu.tgz
> -for p in dtc pixman; do
> -    if test -f $BASE/$p.tgz; then
> -        tar -C $TEST_DIR/src/$p -xzf $BASE/$p.tgz
> -        export FEATURES="$FEATURES $p"
> -    fi
> -done
> +tar -C $TEST_DIR/src -xf $BASE/qemu.tar
>
>  if test -n "$SHOW_ENV"; then
>      if test -f /packages.txt; then


--
Alex Bennée

Re: [Qemu-devel] [PATCH v8 12/13] tests: Add README for vm tests

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/vm/README | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 85 insertions(+)
>  create mode 100644 tests/vm/README
>
> diff --git a/tests/vm/README b/tests/vm/README
> new file mode 100644
> index 0000000000..fa30a79ea7
> --- /dev/null
> +++ b/tests/vm/README
> @@ -0,0 +1,85 @@
> +=== VM test suite to run build in guests ===
> +
> +== Intro ==
> +
> +This test suite contains scripts that bootstrap various guest images that have
> +necessary packages to build QEMU. The basic usage is documented in Makefile
> +help which is displayed with "make vm-test".
> +
> +== Quick start ==
> +
> +Run "make vm-test" to list available make targets. Invoke a specific make
> +command to run build test in an image. For example, "make vm-build-freebsd"
> +will build the source tree in the FreeBSD image. The command can be executed
> +from either the source tree or the build dir; if the former, ./configure is not
> +needed. The command will then generate the test image in ./tests/vm/ under the
> +working directory.
> +
> +== QEMU binary ==
> +
> +By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
> +isn't one, or if it is older than 2.10, the test won't work. In this case,
> +provide the QEMU binary in env var: QEMU=/path/to/qemu-2.10+.
> +
> +== Make jobs ==
> +
> +The "-j$X" option in the make command line is not propagated into the VM,
> +specify "J=$X" to control the make jobs in the guest.
> +
> +== Debugging ==
> +
> +Add "DEBUG=1" and/or "V=1" to the make command to allow interactive debugging
> +and verbose output. If this is not enough, see the next section.
> +
> +== Manual invocation ==
> +
> +Each guest script is an executable script with the same command line options.
> +For example to work with the netbsd guest, use $QEMU_SRC/tests/vm/netbsd:
> +
> +    $ cd $QEMU_SRC/tests/vm
> +
> +    # To bootstrap the image
> +    $ ./netbsd --build-image --image /var/tmp/netbsd.img
> +    <...>
> +
> +    # To run an arbitrary command in guest (the output will not be echoed unless
> +    # --debug is added)
> +    $ ./netbsd --debug --image /var/tmp/netbsd.img uname -a
> +
> +    # To build QEMU in guest
> +    $ ./netbsd --debug --image /var/tmp/netbsd.img --build-qemu
> $QEMU_SRC

This doesn't work:

10:41:38 [alex@zen:~/l/q/q/t/vm] review/fam-vmbuild-v8(+2/-2) ± ./netbsd --debug --image netbsd.img --build-qemu /home/alex/lsrc/qemu/qemu.git/
DEBUG:root:Creating archive ./vm-test-9yVAeh.tmp/data-9b5f9.tar for src_dir dir: /home/alex/lsrc/qemu/qemu.git/
./scripts/archive-source.sh: 27: ./scripts/archive-source.sh: cannot create ./vm-test-9yVAeh.tmp/data-9b5f9.tar.list: Directory nonexistent
Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/alex/lsrc/qemu/qemu.git/tests/vm/basevm.py", line 234, in main
    vm.add_source_dir(args.build_qemu)
  File "/home/alex/lsrc/qemu/qemu.git/tests/vm/basevm.py", line 140, in add_source_dir
    stdout=self._stdout, stderr=self._stderr)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['./scripts/archive-source.sh', './vm-test-9yVAeh.tmp/data-9b5f9.tar']' returned non-zero exit status 2

Either adjust the call to be from QEMU_SRC or make the call to
archive-source smarter.

> +
> +    # To get to an interactive shell
> +    $ ./netbsd --interactive --image /var/tmp/netbsd.img sh
> +
> +== Adding new guests ==
> +
> +Please look at existing guest scripts for how to add new guests.
> +
> +Most importantly, create a subclass of BaseVM and implement build_image()
> +method and define BUILD_SCRIPT, then finally call basevm.main() from the
> +script's main().
> +
> +  - Usually in build_image(), a template image is downloaded from a predefined
> +    URL. BaseVM._download_with_cache() takes care of the cache and the
> +    checksum, so consider using it.
> +
> +  - Once the image is downloaded, users, SSH server and QEMU build deps should
> +    be set up:
> +
> +    * Root password set to BaseVM.ROOT_PASS
> +    * User BaseVM.GUEST_USER is created, and password set to BaseVM.GUEST_PASS
> +    * SSH service is enabled and started on boot,
> +      $QEMU_SRC/tests/keys/id_rsa.pub is added to ssh's "authorized_keys" file
> +      of both root and the normal user
> +    * DHCP client service is enabled and started on boot, so that it can
> +      automatically configure the virtio-net-pci NIC and communicate with QEMU
> +      user net (10.0.2.2)
> +    * Necessary packages are installed to untar the source tarball and build
> +      QEMU
> +
> +  - Write a proper BUILD_SCRIPT template, which should be a shell script that
> +    untars a raw virtio-blk block device, which is the tarball data blob of the
> +    QEMU source tree, then configure/build it. Running "make check" is also
> +    recommended.
> +


--
Alex Bennée

Re: [Qemu-devel] [PATCH v8 05/13] tests: Add vm test lib

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> This is the common code to implement a "VM test" to
>
>   1) Download and initialize a pre-defined VM that has necessary
>   dependencies to build QEMU and SSH access.
>
>   2) Archive $SRC_PATH to a .tar file.
>
>   3) Boot the VM, and pass the source tar file to the guest.
>
>   4) SSH into the VM, untar the source tarball, build from the source.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/vm/basevm.py | 256 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 256 insertions(+)
>  create mode 100755 tests/vm/basevm.py
>
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> new file mode 100755
> index 0000000000..77d07b161d
> --- /dev/null
> +++ b/tests/vm/basevm.py
> @@ -0,0 +1,256 @@
> +#!/usr/bin/env python
> +#
> +# VM testing base class
> +#
> +# Copyright 2017 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +#
> +
> +import os
> +import sys
> +import logging
> +import time
> +import datetime
> +sys.path.append(os.path.join(os.path.dirname(__file__), "..", "..", "scripts"))
> +from qemu import QEMUMachine
> +import subprocess
> +import hashlib
> +import optparse
> +import atexit
> +import tempfile
> +import shutil
> +import multiprocessing
> +import traceback
> +
> +SSH_KEY = open(os.path.join(os.path.dirname(__file__),
> +               "..", "keys", "id_rsa")).read()
> +SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__),
> +                   "..", "keys", "id_rsa.pub")).read()
> +
> +class BaseVM(object):
> +    GUEST_USER = "qemu"
> +    GUEST_PASS = "qemupass"
> +    ROOT_PASS = "qemupass"
> +
> +    # The script to run in the guest that builds QEMU
> +    BUILD_SCRIPT = ""
> +    # The guest name, to be overridden by subclasses
> +    name = "#base"
> +    def __init__(self, debug=False, vcpus=None):
> +        self._guest = None
> +        self._tmpdir = tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")
> +        atexit.register(shutil.rmtree, self._tmpdir)
> +
> +        self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa")
> +        open(self._ssh_key_file, "w").write(SSH_KEY)
> +        subprocess.check_call(["chmod", "600", self._ssh_key_file])
> +
> +        self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub")
> +        open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY)
> +
> +        self.debug = debug
> +        self._stderr = sys.stderr
> +        self._devnull = open(os.devnull, "w")
> +        if self.debug:
> +            self._stdout = sys.stdout
> +        else:
> +            self._stdout = self._devnull
> +        self._args = [ \
> +            "-nodefaults", "-m", "2G",
> +            "-cpu", "host",
> +            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
> +            "-device", "virtio-net-pci,netdev=vnet",
> +            "-vnc", "127.0.0.1:0,to=20",
> +            "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
> +        if vcpus:
> +            self._args += ["-smp", str(vcpus)]
> +        if os.access("/dev/kvm", os.R_OK | os.W_OK):
> +            self._args += ["-enable-kvm"]
> +        else:
> +            logging.info("KVM not available, not using -enable-kvm")
> +        self._data_args = []
> +
> +    def _download_with_cache(self, url, sha256sum=None):
> +        def check_sha256sum(fname):
> +            if not sha256sum:
> +                return True
> +            checksum = subprocess.check_output(["sha256sum", fname]).split()[0]
> +            return sha256sum == checksum
> +
> +        cache_dir = os.path.expanduser("~/.cache/qemu-vm/download")
> +        if not os.path.exists(cache_dir):
> +            os.makedirs(cache_dir)
> +        fname = os.path.join(cache_dir, hashlib.sha1(url).hexdigest())
> +        if os.path.exists(fname) and check_sha256sum(fname):
> +            return fname
> +        logging.debug("Downloading %s to %s...", url, fname)
> +        subprocess.check_call(["wget", "-c", url, "-O", fname + ".download"],
> +                              stdout=self._stdout, stderr=self._stderr)
> +        os.rename(fname + ".download", fname)
> +        return fname
> +
> +    def _ssh_do(self, user, cmd, check, interactive=False):
> +        ssh_cmd = ["ssh", "-q",
> +                   "-o", "StrictHostKeyChecking=no",
> +                   "-o", "UserKnownHostsFile=" + os.devnull,
> +                   "-o", "ConnectTimeout=1",
> +                   "-p", self.ssh_port, "-i", self._ssh_key_file]
> +        if interactive:
> +            ssh_cmd += ['-t']
> +        assert not isinstance(cmd, str)
> +        ssh_cmd += ["%s@127.0.0.1" % user] + list(cmd)
> +        logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
> +        r = subprocess.call(ssh_cmd,
> +                            stdin=sys.stdin if interactive else self._devnull,
> +                            stdout=sys.stdout if interactive else self._stdout,
> +                            stderr=sys.stderr if interactive else self._stderr)
> +        if check and r != 0:
> +            raise Exception("SSH command failed: %s" % cmd)
> +        return r
> +
> +    def ssh(self, *cmd):
> +        return self._ssh_do(self.GUEST_USER, cmd, False)
> +
> +    def ssh_interactive(self, *cmd):
> +        return self._ssh_do(self.GUEST_USER, cmd, False, True)
> +
> +    def ssh_root(self, *cmd):
> +        return self._ssh_do("root", cmd, False)
> +
> +    def ssh_check(self, *cmd):
> +        self._ssh_do(self.GUEST_USER, cmd, True)
> +
> +    def ssh_root_check(self, *cmd):
> +        self._ssh_do("root", cmd, True)
> +
> +    def build_image(self, img):
> +        raise NotImplementedError
> +
> +    def add_source_dir(self, src_dir):
> +        name = "data-" + hashlib.sha1(src_dir).hexdigest()[:5]
> +        tarfile = os.path.join(self._tmpdir, name + ".tar")
> +        logging.debug("Creating archive %s for src_dir dir: %s", tarfile, src_dir)
> +        subprocess.check_call(["./scripts/archive-source.sh", tarfile],
> +                              cwd=src_dir, stdin=self._devnull,
> +                              stdout=self._stdout, stderr=self._stderr)
> +        self._data_args += ["-drive",
> +                            "file=%s,if=none,id=%s,cache=writeback,format=raw" % \
> +                                    (tarfile, name),
> +                            "-device",
> +                            "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
> +
> +    def boot(self, img, extra_args=[]):
> +        args = self._args + [
> +            "-device", "VGA",
> +            "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
> +            "-device", "virtio-blk,drive=drive0,bootindex=0"]
> +        args += self._data_args + extra_args
> +        logging.debug("QEMU args: %s", " ".join(args))
> +        qemu_bin = os.environ.get("QEMU", "qemu-system-x86_64")
> +        guest = QEMUMachine(binary=qemu_bin, args=args)
> +        try:
> +            guest.launch()
> +        except:
> +            logging.error("Failed to launch QEMU, command line:")
> +            logging.error(" ".join([qemu_bin] + args))
> +            logging.error("Log:")
> +            logging.error(guest.get_log())
> +            logging.error("QEMU version >= 2.10 is required")
> +            raise
> +        atexit.register(self.shutdown)
> +        self._guest = guest
> +        usernet_info = guest.qmp("human-monitor-command",
> +                                 command_line="info usernet")
> +        self.ssh_port = None
> +        for l in usernet_info["return"].splitlines():
> +            fields = l.split()
> +            if "TCP[HOST_FORWARD]" in fields and "22" in fields:
> +                self.ssh_port = l.split()[3]
> +        if not self.ssh_port:
> +            raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \
> +                            usernet_info)
> +
> +    def wait_ssh(self, seconds=120):
> +        starttime = datetime.datetime.now()
> +        guest_up = False
> +        while (datetime.datetime.now() - starttime).total_seconds() < seconds:
> +            if self.ssh("exit 0") == 0:
> +                guest_up = True
> +                break
> +            time.sleep(1)
> +        if not guest_up:
> +            raise Exception("Timeout while waiting for guest ssh")
> +
> +    def shutdown(self):
> +        self._guest.shutdown()
> +
> +    def wait(self):
> +        self._guest.wait()
> +
> +    def qmp(self, *args, **kwargs):
> +        return self._guest.qmp(*args, **kwargs)
> +
> +def parse_args(vm_name):
> +    parser = optparse.OptionParser(description="""
> +    VM test utility.  Exit codes: 0 = success, 1 = command line error, 2 = environment initialization failed, 3 = test command failed""")
> +    parser.add_option("--debug", "-D", action="store_true",
> +                      help="enable debug output")
> +    parser.add_option("--image", "-i", default="%s.img" % vm_name,
> +                      help="image file name")
> +    parser.add_option("--force", "-f", action="store_true",
> +                      help="force build image even if image exists")
> +    parser.add_option("--jobs", type=int, default=multiprocessing.cpu_count() / 2,
> +                      help="number of virtual CPUs")
> +    parser.add_option("--build-image", "-b", action="store_true",
> +                      help="build image")
> +    parser.add_option("--build-qemu",
> +                      help="build QEMU from source in guest")
> +    parser.add_option("--interactive", "-I", action="store_true",
> +                      help="Interactively run command")
> +    parser.disable_interspersed_args()
> +    return parser.parse_args()
> +
> +def main(vmcls):
> +    try:
> +        args, argv = parse_args(vmcls.name)
> +        if not argv and not args.build_qemu and not args.build_image:
> +            print "Nothing to do?"
> +            return 1
> +        if args.debug:
> +            logging.getLogger().setLevel(logging.DEBUG)
> +        vm = vmcls(debug=args.debug, vcpus=args.jobs)
> +        if args.build_image:
> +            if os.path.exists(args.image) and not args.force:
> +                sys.stderr.writelines(["Image file exists: %s\n" % args.image,
> +                                      "Use --force option to overwrite\n"])
> +                return 1
> +            return vm.build_image(args.image)
> +        if args.build_qemu:
> +            vm.add_source_dir(args.build_qemu)
> +            cmd = [vm.BUILD_SCRIPT.format(
> +                   configure_opts = " ".join(argv),
> +                   jobs=args.jobs)]
> +        else:
> +            cmd = argv
> +        vm.boot(args.image + ",snapshot=on")
> +        vm.wait_ssh()
> +    except Exception as e:
> +        if isinstance(e, SystemExit) and e.code == 0:
> +            return 0
> +        sys.stderr.write("Failed to prepare guest environment\n")
> +        traceback.print_exc()
> +        return 2
> +
> +    if args.interactive:
> +        if vm.ssh_interactive(*cmd) == 0:
> +            return 0
> +        vm.ssh_interactive()
> +        return 3
> +    else:
> +        if vm.ssh(*cmd) != 0:
> +            return 3


--
Alex Bennée

Re: [Qemu-devel] [PATCH v8 10/13] Makefile: Add rules to run vm tests

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> Signed-off-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  Makefile                  |  2 ++
>  configure                 |  2 +-
>  tests/vm/Makefile.include | 42 ++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 45 insertions(+), 1 deletion(-)
>  create mode 100644 tests/vm/Makefile.include
>
> diff --git a/Makefile b/Makefile
> index b53fc69a60..79684ab138 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -812,6 +812,7 @@ endif
>  -include $(wildcard *.d tests/*.d)
>
>  include $(SRC_PATH)/tests/docker/Makefile.include
> +include $(SRC_PATH)/tests/vm/Makefile.include
>
>  .PHONY: help
>  help:
> @@ -835,6 +836,7 @@ help:
>  	@echo  'Test targets:'
>  	@echo  '  check           - Run all tests (check-help for details)'
>  	@echo  '  docker          - Help about targets running tests inside Docker containers'
> +	@echo  '  vm-test         - Help about targets running tests inside VM'
>  	@echo  ''
>  	@echo  'Documentation targets:'
>  	@echo  '  html info pdf txt'
> diff --git a/configure b/configure
> index 94db2d103e..bbf053e3e0 100755
> --- a/configure
> +++ b/configure
> @@ -6505,7 +6505,7 @@ if test "$ccache_cpp2" = "yes"; then
>  fi
>
>  # build tree in object directory in case the source is not in the current directory
> -DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests"
> +DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests tests/vm"
>  DIRS="$DIRS docs docs/interop fsdev"
>  DIRS="$DIRS pc-bios/optionrom pc-bios/spapr-rtas pc-bios/s390-ccw"
>  DIRS="$DIRS roms/seabios roms/vgabios"
> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
> new file mode 100644
> index 0000000000..5daa2a3b73
> --- /dev/null
> +++ b/tests/vm/Makefile.include
> @@ -0,0 +1,42 @@
> +# Makefile for VM tests
> +
> +.PHONY: vm-build-all
> +
> +IMAGES := ubuntu.i386 freebsd netbsd openbsd
> +IMAGE_FILES := $(patsubst %, tests/vm/%.img, $(IMAGES))
> +
> +.PRECIOUS: $(IMAGE_FILES)
> +
> +vm-test:
> +	@echo "vm-test: Test QEMU in preconfigured virtual machines"
> +	@echo
> +	@echo "  vm-build-ubuntu.i386            - Build QEMU in ubuntu i386 VM"
> +	@echo "  vm-build-freebsd                - Build QEMU in FreeBSD VM"
> +	@echo "  vm-build-netbsd                 - Build QEMU in NetBSD VM"
> +	@echo "  vm-build-openbsd                - Build QEMU in OpenBSD VM"
> +
> +vm-build-all: $(addprefix vm-build-, $(IMAGES))
> +
> +tests/vm/%.img: $(SRC_PATH)/tests/vm/% \
> +		$(SRC_PATH)/tests/vm/basevm.py \
> +		$(SRC_PATH)/tests/vm/Makefile.include
> +	$(call quiet-command, \
> +		$< \
> +		$(if $(V)$(DEBUG), --debug) \
> +		--image "$@" \
> +		--force \
> +		--build-image $@, \
> +		"  VM-IMAGE $*")
> +
> +
> +# Build in VM $(IMAGE)
> +vm-build-%: tests/vm/%.img
> +	$(call quiet-command, \
> +		$(SRC_PATH)/tests/vm/$* \
> +		$(if $(V)$(DEBUG), --debug) \
> +		$(if $(DEBUG), --interactive) \
> +		$(if $(J),--jobs $(J)) \
> +		--image "$<" \
> +		--build-qemu $(SRC_PATH), \
> +		"  VM-BUILD $*")
> +


--
Alex Bennée

Re: [Qemu-devel] [PATCH v8 12/13] tests: Add README for vm tests

[Fam Zheng]

On Mon, 09/18 10:43, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  tests/vm/README | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 85 insertions(+)
> >  create mode 100644 tests/vm/README
> >
> > diff --git a/tests/vm/README b/tests/vm/README
> > new file mode 100644
> > index 0000000000..fa30a79ea7
> > --- /dev/null
> > +++ b/tests/vm/README
> > @@ -0,0 +1,85 @@
> > +=== VM test suite to run build in guests ===
> > +
> > +== Intro ==
> > +
> > +This test suite contains scripts that bootstrap various guest images that have
> > +necessary packages to build QEMU. The basic usage is documented in Makefile
> > +help which is displayed with "make vm-test".
> > +
> > +== Quick start ==
> > +
> > +Run "make vm-test" to list available make targets. Invoke a specific make
> > +command to run build test in an image. For example, "make vm-build-freebsd"
> > +will build the source tree in the FreeBSD image. The command can be executed
> > +from either the source tree or the build dir; if the former, ./configure is not
> > +needed. The command will then generate the test image in ./tests/vm/ under the
> > +working directory.
> > +
> > +== QEMU binary ==
> > +
> > +By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
> > +isn't one, or if it is older than 2.10, the test won't work. In this case,
> > +provide the QEMU binary in env var: QEMU=/path/to/qemu-2.10+.
> > +
> > +== Make jobs ==
> > +
> > +The "-j$X" option in the make command line is not propagated into the VM,
> > +specify "J=$X" to control the make jobs in the guest.
> > +
> > +== Debugging ==
> > +
> > +Add "DEBUG=1" and/or "V=1" to the make command to allow interactive debugging
> > +and verbose output. If this is not enough, see the next section.
> > +
> > +== Manual invocation ==
> > +
> > +Each guest script is an executable script with the same command line options.
> > +For example to work with the netbsd guest, use $QEMU_SRC/tests/vm/netbsd:
> > +
> > +    $ cd $QEMU_SRC/tests/vm
> > +
> > +    # To bootstrap the image
> > +    $ ./netbsd --build-image --image /var/tmp/netbsd.img
> > +    <...>
> > +
> > +    # To run an arbitrary command in guest (the output will not be echoed unless
> > +    # --debug is added)
> > +    $ ./netbsd --debug --image /var/tmp/netbsd.img uname -a
> > +
> > +    # To build QEMU in guest
> > +    $ ./netbsd --debug --image /var/tmp/netbsd.img --build-qemu
> > $QEMU_SRC
> 
> This doesn't work:
> 
> 10:41:38 [alex@zen:~/l/q/q/t/vm] review/fam-vmbuild-v8(+2/-2) ± ./netbsd --debug --image netbsd.img --build-qemu /home/alex/lsrc/qemu/qemu.git/
> DEBUG:root:Creating archive ./vm-test-9yVAeh.tmp/data-9b5f9.tar for src_dir dir: /home/alex/lsrc/qemu/qemu.git/
> ./scripts/archive-source.sh: 27: ./scripts/archive-source.sh: cannot create ./vm-test-9yVAeh.tmp/data-9b5f9.tar.list: Directory nonexistent
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/alex/lsrc/qemu/qemu.git/tests/vm/basevm.py", line 234, in main
>     vm.add_source_dir(args.build_qemu)
>   File "/home/alex/lsrc/qemu/qemu.git/tests/vm/basevm.py", line 140, in add_source_dir
>     stdout=self._stdout, stderr=self._stderr)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
>     raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command '['./scripts/archive-source.sh', './vm-test-9yVAeh.tmp/data-9b5f9.tar']' returned non-zero exit status 2
> 
> Either adjust the call to be from QEMU_SRC or make the call to
> archive-source smarter.

Yup, I failed to verify manual invocations since the revision which introduced
archive-source.sh.. I would use absolute path for tmpdir everywhere, by
squashing in this:

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 77d07b161d..cbaa061c72 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -43,7 +43,9 @@ class BaseVM(object):
     name = "#base"
     def __init__(self, debug=False, vcpus=None):
         self._guest = None
-        self._tmpdir = tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")
+        self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
+                                                         suffix=".tmp",
+                                                         dir="."))
         atexit.register(shutil.rmtree, self._tmpdir)

         self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa")

Fam

Re: [Qemu-devel] [PATCH v8 03/13] scripts: Add archive-source.sh

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1964 bytes --]

On 09/17/2017 09:43 PM, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  scripts/archive-source.sh | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
>  create mode 100755 scripts/archive-source.sh
> 
> diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
> new file mode 100755
> index 0000000000..8b373e3090
> --- /dev/null
> +++ b/scripts/archive-source.sh
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# Author: Fam Zheng <famz@redhat.com>
> +#
> +# Archive source tree, including submodules. This is created for test code to
> +# export the source files, in order to be built in a different enviornment,
> +# such as in a docker instance or VM.
> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +
> +set -e

Do we want to rely on 'set -e'?  The alternative would be:

> +
> +if test $# -lt 1; then
> +    echo "Usage: $0 <output tarball>"
> +    exit 1
> +fi
> +
> +submodules=$(git submodule foreach --recursive --quiet 'echo $name')
> +

...Check here that the $() ran (rather than git being missing)

> +if test -n "$submodules"; then
> +    {
> +        git ls-files

...and that this ran

> +        for sm in $submodules; do
> +            (cd $sm; git ls-files) | sed "s:^:$sm/:"
> +        done
> +    } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$1".list

...and the grep

> +else
> +    git ls-files > "$1".list

...and the git ls-files

> +fi
> +
> +tar -cf "$1" -T "$1".list

...and the tar

> +rm "$1".list
> 

I'm not opposed to the patch going in as-is; but I do like to flag
reliance on 'set -e' as it tends to NOT do what you think it does, and
ends up getting in the way more than it helps.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH v8 04/13] tests: Add a test key pair

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 962 bytes --]

On 09/17/2017 09:43 PM, Fam Zheng wrote:
> This will be used by setup test user ssh.

Shouldn't the commit message have more details about WHY we are checking
in a private key? The conversation last time included useful gems such as:

Some guests require the key to exist prior to provisioning the guest,
and reusing a pre-built key avoids consuming entropy every time the
testsuite is run.  Because the private key is well-known, care must be
taken to use the key ONLY in situations that cannot be compromised by
external network clients.

> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/keys/id_rsa     | 27 +++++++++++++++++++++++++++
>  tests/keys/id_rsa.pub |  1 +
>  2 files changed, 28 insertions(+)
>  create mode 100644 tests/keys/id_rsa
>  create mode 100644 tests/keys/id_rsa.pub
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH v8 12/13] tests: Add README for vm tests

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 725 bytes --]

On 09/17/2017 09:44 PM, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/vm/README | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 85 insertions(+)
>  create mode 100644 tests/vm/README

The README should ALSO mention that (some) of the guests created in this
manner are using a well-known public/private-key pair, and MUST NOT be
exposed to external interfaces if you are at all concerned about an
attacker being able to take over the guest VM and exploit any other qemu
bug that can then take over your host machine.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH v8 04/13] tests: Add a test key pair

[Fam Zheng]

On Mon, 09/18 10:17, Eric Blake wrote:
> On 09/17/2017 09:43 PM, Fam Zheng wrote:
> > This will be used by setup test user ssh.
> 
> Shouldn't the commit message have more details about WHY we are checking
> in a private key? The conversation last time included useful gems such as:
> 
> Some guests require the key to exist prior to provisioning the guest,
> and reusing a pre-built key avoids consuming entropy every time the
> testsuite is run.  Because the private key is well-known, care must be
> taken to use the key ONLY in situations that cannot be compromised by
> external network clients.

Yeah. Not everyone looks at the commit message, I'll add a tests/keys/README to
document that.

Fam