[Qemu-devel] [PATCH v2 00/15] tests: Introducing docker tests

[Qemu-devel] [PATCH v2 00/15] tests: Introducing docker tests

[Fam Zheng]

v2: - Fix a few coding style warnings of shellcheck and pylint.
      [Marc-André Lureau]
    - Rename make targets:
      docker-run => docker-test
      docker-build => docker-image
      docker-run-$TEST.sh@$IMAGE => docker-$TEST@$IMAGE
    - In help, document the magic makefile vars which are
      "IMAGES=", "TESTS=", "J=" and "PAUSE="
    - Rename test-basic => test-quick
    - Move docker files to tests/docker/dockerfiles/
    - Add "test-full" that builds all targets
    - Workaround broken clang for fedora 23
    - Add commit log for .gitignore changes [Alex]
    - Add "rev-by" in patch 13. [Alex]
    - Add patch 14 to make debugging easier [Alex]
    - Adopt Alex's "git archive" idea

v1: Since RFC, addressed comments from reviewers, and improved a lot of things.
    Thanks to Daniel, Eric, Paolo, Stefan, for the feedback.

This series adds a new "docker" make target family to run tests in created
docker containers.

To begin with, this can be a place to store standard env/command combinations to
build and test QEMU.

Secondly, CI usually provides "docker" capability, where we specify
standard/repeatable test environments, and run tests in them.  However, what
tests to cover is better maintained in-tree, in order to keep in sync with the
code development.

Lastly, this makes it very simple for developers to replicate such tests
themselves.

Alex Bennée (1):
  tests/docker/Makefile.include: snapshot the src for docker

Fam Zheng (14):
  tests: Add utilities for docker testing
  Makefile: Rules for docker testing
  docker: Add images
  docker: Add test runner
  docker: Add common.rc
  docker: Add quick test
  docker: Add full test
  docker: Add clang test
  docker: Add mingw test
  docker: Add travis tool
  docs: Add text for tests/docker in build-system.txt
  .gitignore: Ignore temporary dockerfile
  MAINTAINERS: Add tests/docker
  docker: Support interactive shell for debugging

 .gitignore                              |   1 +
 MAINTAINERS                             |   7 ++
 Makefile                                |   4 +-
 docs/build-system.txt                   |   5 ++
 tests/docker/Makefile.include           | 110 +++++++++++++++++++++++++++++++
 tests/docker/common.rc                  |  29 ++++++++
 tests/docker/docker.py                  | 113 ++++++++++++++++++++++++++++++++
 tests/docker/docker_build               |  42 ++++++++++++
 tests/docker/docker_clean               |  22 +++++++
 tests/docker/docker_run                 |  29 ++++++++
 tests/docker/dockerfiles/centos6.docker |   5 ++
 tests/docker/dockerfiles/fedora.docker  |   7 ++
 tests/docker/dockerfiles/ubuntu.docker  |   9 +++
 tests/docker/run                        |  34 ++++++++++
 tests/docker/test-clang                 |  28 ++++++++
 tests/docker/test-full                  |  21 ++++++
 tests/docker/test-mingw                 |  36 ++++++++++
 tests/docker/test-quick                 |  23 +++++++
 tests/docker/travis                     |  23 +++++++
 tests/docker/travis.py                  |  48 ++++++++++++++
 20 files changed, 595 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/Makefile.include
 create mode 100755 tests/docker/common.rc
 create mode 100755 tests/docker/docker.py
 create mode 100755 tests/docker/docker_build
 create mode 100755 tests/docker/docker_clean
 create mode 100755 tests/docker/docker_run
 create mode 100644 tests/docker/dockerfiles/centos6.docker
 create mode 100644 tests/docker/dockerfiles/fedora.docker
 create mode 100644 tests/docker/dockerfiles/ubuntu.docker
 create mode 100755 tests/docker/run
 create mode 100755 tests/docker/test-clang
 create mode 100755 tests/docker/test-full
 create mode 100755 tests/docker/test-mingw
 create mode 100755 tests/docker/test-quick
 create mode 100755 tests/docker/travis
 create mode 100755 tests/docker/travis.py

-- 
2.4.3

[Qemu-devel] [PATCH v2 01/15] tests: Add utilities for docker testing

[Fam Zheng]

docker_run: A wrapper for "docker run" (or "sudo -n docker run" if
necessary), which takes care of killing and removing the running
container at SIGINT.

docker_clean: A tool to tear down all the containers including inactive
ones that are started by docker_run.

docker_build: A tool to compare an image from given dockerfile and
rebuild it if they're different.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/docker.py    | 113 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/docker/docker_build |  42 +++++++++++++++++
 tests/docker/docker_clean |  22 +++++++++
 tests/docker/docker_run   |  29 ++++++++++++
 4 files changed, 206 insertions(+)
 create mode 100755 tests/docker/docker.py
 create mode 100755 tests/docker/docker_build
 create mode 100755 tests/docker/docker_clean
 create mode 100755 tests/docker/docker_run

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
new file mode 100755
index 0000000..d175a86
--- /dev/null
+++ b/tests/docker/docker.py
@@ -0,0 +1,113 @@
+#!/usr/bin/env python2 -B
+#
+# Docker controlling module
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import os
+import subprocess
+import json
+import hashlib
+import atexit
+import uuid
+
+class ContainerTerminated(Exception):
+    """ Raised if the container has already existed """
+    pass
+
+class Docker(object):
+    """ Running Docker commands """
+    def __init__(self):
+        self._command = self._guess_command()
+        self._instances = []
+        atexit.register(self._kill_instances)
+
+    def _do(self, cmd, quiet=True, **kwargs):
+        if quiet:
+            kwargs["stdout"] = subprocess.PIPE
+        return subprocess.call(self._command + cmd, **kwargs)
+
+    def _do_kill_instances(self, only_known, only_active=True):
+        cmd = ["ps", "-q"]
+        if not only_active:
+            cmd.append("-a")
+        for i in self._output(cmd).split():
+            resp = self._output(["inspect", i])
+            labels = json.loads(resp)[0]["Config"]["Labels"]
+            active = json.loads(resp)[0]["State"]["Running"]
+            if not labels:
+                continue
+            instance_uuid = labels.get("com.qemu.instance.uuid", None)
+            if not instance_uuid:
+                continue
+            if only_known and instance_uuid not in self._instances:
+                continue
+            print "Terminating", i
+            if active:
+                self._do(["kill", i])
+            self._do(["rm", i])
+
+    def clean(self):
+        self._do_kill_instances(False, False)
+        return 0
+
+    def _kill_instances(self):
+        return self._do_kill_instances(True)
+
+    def _output(self, cmd, **kwargs):
+        return subprocess.check_output(self._command + cmd,
+                                       stderr=subprocess.STDOUT,
+                                       **kwargs)
+
+    def _guess_command(self):
+        commands = [["docker"], ["sudo", "-n", "docker"]]
+        for cmd in commands:
+            if subprocess.call(cmd + ["images"],
+                               stdout=subprocess.PIPE,
+                               stderr=subprocess.PIPE) == 0:
+                return cmd
+        commands_txt = "\n".join(["  " + " ".join(x) for x in commands])
+        raise Exception("Cannot find working docker command. Tried:\n%s" % commands_txt)
+
+    def get_image_dockerfile_checksum(self, tag):
+        resp = self._output(["inspect", tag])
+        labels = json.loads(resp)[0]["Config"].get("Labels", {})
+        return labels.get("com.qemu.dockerfile-checksum", "")
+
+    def checksum(self, text):
+        return hashlib.sha1(text).hexdigest()
+
+    def build_image(self, tag, dockerfile, df, quiet=True):
+        tmp = dockerfile + "\n" + \
+              "LABEL com.qemu.dockerfile-checksum=%s" % self.checksum(dockerfile)
+        tmp_df = df + ".tmp"
+        tmp_file = open(tmp_df, "wb")
+        tmp_file.write(tmp)
+        tmp_file.close()
+        self._do(["build", "-t", tag, "-f", tmp_df, os.path.dirname(df)],
+                 quiet=quiet)
+        os.unlink(tmp_df)
+
+    def image_matches_dockerfile(self, tag, dockerfile):
+        try:
+            checksum = self.get_image_dockerfile_checksum(tag)
+        except:
+            return False
+        return checksum == self.checksum(dockerfile)
+
+    def run(self, cmd, keep, quiet):
+        label = uuid.uuid1().hex
+        if not keep:
+            self._instances.append(label)
+        ret = self._do(["run", "--label", "com.qemu.instance.uuid=" + label] + cmd, quiet=quiet)
+        if not keep:
+            self._instances.remove(label)
+        return ret
+
diff --git a/tests/docker/docker_build b/tests/docker/docker_build
new file mode 100755
index 0000000..6948e2c
--- /dev/null
+++ b/tests/docker/docker_build
@@ -0,0 +1,42 @@
+#!/usr/bin/env python2
+#
+# Compare to Dockerfile and rebuild a docker image if necessary.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import sys
+import docker
+import argparse
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("tag",
+                        help="Image Tag")
+    parser.add_argument("dockerfile",
+                        help="Dockerfile name")
+    parser.add_argument("--verbose", "-v", action="store_true",
+                        help="Print verbose information")
+    args = parser.parse_args()
+
+    dockerfile = open(args.dockerfile, "rb").read()
+    tag = args.tag
+
+    dkr = docker.Docker()
+    if dkr.image_matches_dockerfile(tag, dockerfile):
+        if args.verbose:
+            print "Image is up to date."
+        return 0
+
+    quiet = not args.verbose
+    dkr.build_image(tag, dockerfile, args.dockerfile, quiet=quiet)
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/tests/docker/docker_clean b/tests/docker/docker_clean
new file mode 100755
index 0000000..88cdba6
--- /dev/null
+++ b/tests/docker/docker_clean
@@ -0,0 +1,22 @@
+#!/usr/bin/env python2
+#
+# Clean up uselsee containers.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import sys
+import docker
+
+def main():
+    docker.Docker().clean()
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/tests/docker/docker_run b/tests/docker/docker_run
new file mode 100755
index 0000000..4c46d90
--- /dev/null
+++ b/tests/docker/docker_run
@@ -0,0 +1,29 @@
+#!/usr/bin/env python2
+#
+# Wrapper for "docker run" with automatical clean up if the execution is
+# iterrupted.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import sys
+import argparse
+import docker
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--keep", action="store_true",
+                        help="Don't remove image when the command completes")
+    parser.add_argument("--quiet", action="store_true",
+                        help="Run quietly unless an error occured")
+    args, argv = parser.parse_known_args()
+    return docker.Docker().run(argv, args.keep, quiet=args.quiet)
+
+if __name__ == "__main__":
+    sys.exit(main())
-- 
2.4.3

[Qemu-devel] [PATCH v2 02/15] Makefile: Rules for docker testing

[Fam Zheng]

This adds a group of make targets to run docker tests, all are available
in source tree without running ./configure.

The usage is shown by "make docker".

Besides the fixed ones, dynamic targets for building each image and
running each test in each image are generated automatically by make,
scanning $(SRC_PATH)/tests/docker/ files with specific patterns.

Alternative to manually list particular targets (docker-run-FOO@BAR)
set, you can control which tests/images to run by filtering variables,
TESTS= and IMAGES=, which are expressed in Makefile pattern syntax,
"foo% %bar ...". For example:

    $ make docker-run IMAGES="ubuntu fedora"

Unfortunately, it's impossible to propagate "-j $JOBS" into make in
containers, however since each combination is made a first class target
is the top Makefile, "make -j$N docker-run" still parallels the tests
coarsely.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile                      |  4 +-
 tests/docker/Makefile.include | 93 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/Makefile.include

diff --git a/Makefile b/Makefile
index f9fae3a..f104a08 100644
--- a/Makefile
+++ b/Makefile
@@ -6,7 +6,7 @@ BUILD_DIR=$(CURDIR)
 # Before including a proper config-host.mak, assume we are in the source tree
 SRC_PATH=.
 
-UNCHECKED_GOALS := %clean TAGS cscope ctags
+UNCHECKED_GOALS := %clean TAGS cscope ctags docker docker-%
 
 # All following code might depend on configuration variables
 ifneq ($(wildcard config-host.mak),)
@@ -651,3 +651,5 @@ endif
 # Include automatically generated dependency files
 # Dependencies in Makefile.objs files come from our recursive subdir rules
 -include $(wildcard *.d tests/*.d)
+
+include $(SRC_PATH)/tests/docker/Makefile.include
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
new file mode 100644
index 0000000..d8c55c3
--- /dev/null
+++ b/tests/docker/Makefile.include
@@ -0,0 +1,93 @@
+# Makefile for Docker tests
+
+$(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))
+
+.PHONY: docker docker-test docker-clean docker-image
+
+DOCKER_SUFFIX := .docker
+
+DOCKER_IMAGES := $(notdir $(basename \
+	$(wildcard $(SRC_PATH)/tests/docker/dockerfiles/*)))
+
+DOCKER_TESTS := $(notdir $(shell \
+	find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f -executable))
+
+DOCKER_TOOLS := travis
+
+TESTS ?= %
+IMAGES ?= %
+
+$(foreach i,$(DOCKER_IMAGES), \
+	$(eval docker-image: docker-image-$i) \
+	$(eval docker-image-$i: IMAGE = $i) \
+	$(eval .PHONY: docker-image-$i docker-@$i) \
+	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
+		$(eval .PHONY: docker-$t@$i) \
+		$(eval docker-$t@$i: docker-image-$i) \
+		$(eval docker-$t@$i: docker-run-$t@$i) \
+		$(eval docker-@$i: docker-$t@$i) \
+		$(eval docker-test: docker-run-$t@$i) \
+	) \
+)
+
+docker:
+	@echo 'Building QEMU and running tests inside Docker containers'
+	@echo
+	@echo 'Available targets:'
+	@echo
+	@echo '    docker:              Print this help.'
+	@echo '    docker-test:         Run all image/test combinations.'
+	@echo '    docker-clean:        Kill and remove residual docker testing containers.'
+	@echo '    docker-TEST@IMAGE:   Run "TEST" in container "IMAGE".'
+	@echo '                         Note: "TEST" is one of the listed test name,'
+	@echo '                         or a script name under $$QEMU_SRC/tests/docker/;'
+	@echo '                         "IMAGE" is one of the listed container name."'
+	@echo '    docker-@IMAGE:       Run all tests in container "IMAGE".'
+	@echo '    docker-image:        Build all images.'
+	@echo '    docker-image-IMAGE:  Build image "IMG".'
+	@echo
+	@echo 'Available container images:'
+	@echo '    $(DOCKER_IMAGES)'
+	@echo
+	@echo 'Available tests:'
+	@echo '    $(DOCKER_TESTS)'
+	@echo
+	@echo 'Available tools:'
+	@echo '    $(DOCKER_TOOLS)'
+	@echo
+	@echo 'Special variables:'
+	@echo '    IMAGES="a b c ..":   Filters which images to build or run.'
+	@echo '    TESTS="x y z .."     Filters which tests to run (for docker-test).'
+	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
+	@echo '                         (default is 1)'
+
+docker-image-%:
+	@if test -z "$(IMAGE)"; then echo "Invalid target"; exit 1; fi
+	$(if $(filter $(IMAGES),$(IMAGE)), $(call quiet-command,\
+		$(SRC_PATH)/tests/docker/docker_build qemu:$(IMAGE) \
+			$(SRC_PATH)/tests/docker/dockerfiles/$(IMAGE).docker \
+			$(if $V,-v,), "  BUILD $(IMAGE)"))
+
+
+docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
+docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
+docker-run-%:
+	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
+		then echo "Invalid target"; exit 1; \
+	fi
+	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
+		$(call quiet-command,\
+			$(SRC_PATH)/tests/docker/docker_run $(if $(KEEP),,--rm) \
+				--privileged -t --net=none \
+				-v $$(realpath $(SRC_PATH)):/var/tmp/qemu \
+				-e QEMU_SRC=/var/tmp/qemu \
+				-e V=$V -e J=$J \
+				-v /var/tmp/qemu-docker-ccache:/var/tmp/ccache \
+				-e CCACHE_DIR=/var/tmp/ccache \
+				qemu:$(IMAGE) \
+				/var/tmp/qemu/tests/docker/run \
+				/var/tmp/qemu/tests/docker/$(CMD); \
+			, "  RUN $(CMD) in $(IMAGE)")))
+
+docker-clean:
+	$(call quiet-command, $(SRC_PATH)/tests/docker/docker_clean)
-- 
2.4.3

[Qemu-devel] [PATCH v2 03/15] docker: Add images

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/dockerfiles/centos6.docker | 5 +++++
 tests/docker/dockerfiles/fedora.docker  | 7 +++++++
 tests/docker/dockerfiles/ubuntu.docker  | 9 +++++++++
 3 files changed, 21 insertions(+)
 create mode 100644 tests/docker/dockerfiles/centos6.docker
 create mode 100644 tests/docker/dockerfiles/fedora.docker
 create mode 100644 tests/docker/dockerfiles/ubuntu.docker

diff --git a/tests/docker/dockerfiles/centos6.docker b/tests/docker/dockerfiles/centos6.docker
new file mode 100644
index 0000000..7daa2fb
--- /dev/null
+++ b/tests/docker/dockerfiles/centos6.docker
@@ -0,0 +1,5 @@
+FROM centos:6
+RUN yum install -y \
+    git make gcc g++ \
+    zlib-devel glib2-devel SDL-devel pixman-devel
+
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
new file mode 100644
index 0000000..81091b7
--- /dev/null
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -0,0 +1,7 @@
+FROM fedora:23
+RUN dnf install -y \
+    ccache git \
+    glib2-devel pixman-devel zlib-devel SDL-devel \
+    gcc gcc-c++ clang make perl which bc findutils \
+    mingw{32,64}-{pixman,glib2,gmp,SDL,pkg-config,gtk2,gtk3,gnutls,nettle,libtasn1,libjpeg-turbo,libpng,curl,libssh2,bzip2}
+ENV FEATURES mingw clang
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
new file mode 100644
index 0000000..f1dc518
--- /dev/null
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -0,0 +1,9 @@
+FROM ubuntu:14.04
+RUN apt-get update
+RUN apt-get -y install \
+    libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev \
+    libseccomp-dev libgnutls-dev libssh2-1-dev  libspice-server-dev \
+    libspice-protocol-dev libnss3-dev \
+    libgtk-3-dev libvte-2.90-dev libsdl1.2-dev libpng12-dev libpixman-1-dev \
+    git make ccache python-yaml gcc clang
+ENV FEATURES clang ccache pyyaml
-- 
2.4.3

[Qemu-devel] [PATCH v2 04/15] docker: Add test runner

[Fam Zheng]

It's better to have a launcher for all tests, to make it easier to
initialize and manage the environment.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/run | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100755 tests/docker/run

diff --git a/tests/docker/run b/tests/docker/run
new file mode 100755
index 0000000..55b2737
--- /dev/null
+++ b/tests/docker/run
@@ -0,0 +1,26 @@
+#!/bin/bash -e
+#
+# Docker test runner
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+# Prepare the environment
+. /etc/profile || true
+export PATH=/usr/lib/ccache:$PATH
+
+tmp="$QEMU_SRC"
+QEMU_SRC=/var/tmp/qemu.tmp
+cp -r "$tmp" "$QEMU_SRC"
+if test -n "$J"; then
+    export MAKEFLAGS="$MAKEFLAGS -j$J"
+fi
+
+cd $QEMU_SRC/tests/docker
+"$@"
-- 
2.4.3

[Qemu-devel] [PATCH v2 05/15] docker: Add common.rc

[Fam Zheng]

"requires" checks the "FEATURE" environment for specified prerequisits,
and skip the execution of test if not found.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/common.rc | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
 create mode 100755 tests/docker/common.rc

diff --git a/tests/docker/common.rc b/tests/docker/common.rc
new file mode 100755
index 0000000..34b009e
--- /dev/null
+++ b/tests/docker/common.rc
@@ -0,0 +1,29 @@
+#!/bin/sh
+#
+# Common routines for docker test scripts.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+requires()
+{
+    for c in $@; do
+        if ! echo "$FEATURES" | grep -wq -e "$c"; then
+            echo "Prerequisite '$c' not present, skip"
+            exit 0
+        fi
+    done
+}
+
+build_qemu()
+{
+    make -C $QEMU_SRC distclean &>/dev/null
+    $QEMU_SRC/configure "$@"
+    make $MAKEFLAGS
+}
-- 
2.4.3

[Qemu-devel] [PATCH v2 06/15] docker: Add quick test

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-quick | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100755 tests/docker/test-quick

diff --git a/tests/docker/test-quick b/tests/docker/test-quick
new file mode 100755
index 0000000..ce3cc99
--- /dev/null
+++ b/tests/docker/test-quick
@@ -0,0 +1,23 @@
+#!/bin/bash -e
+#
+# Quick compiling test that everyone already does. But why not automate it?
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+mkdir build
+mkdir install
+cd build
+build_qemu \
+    --prefix="${pwd}/install" \
+    --target-list="$(echo {x86_64,i386,arm,aarch64}-{softmmu,linux-user})"
+make check $MAKEFLAGS
+make install
-- 
2.4.3

[Qemu-devel] [PATCH v2 07/15] docker: Add full test

[Fam Zheng]

This builds all available targets.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-full | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100755 tests/docker/test-full

diff --git a/tests/docker/test-full b/tests/docker/test-full
new file mode 100755
index 0000000..5708d0b
--- /dev/null
+++ b/tests/docker/test-full
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Compile all the targets.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+mkdir build
+mkdir install
+cd build
+build_qemu --prefix="${pwd}/install"
+make check $MAKEFLAGS
+make install
-- 
2.4.3

[Qemu-devel] [PATCH v2 08/15] docker: Add clang test

[Fam Zheng]

The configure options are suggested by John Snow <jsnow@redhat.com>.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-clang | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100755 tests/docker/test-clang

diff --git a/tests/docker/test-clang b/tests/docker/test-clang
new file mode 100755
index 0000000..212f481
--- /dev/null
+++ b/tests/docker/test-clang
@@ -0,0 +1,28 @@
+#!/bin/bash -e
+#
+# Compile and check with clang.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+requires clang
+
+mkdir build
+cd build
+# -fsanitize=undefined is broken on Fedora 23, skip it for now
+# See also: https://bugzilla.redhat.com/show_bug.cgi?id=1263834
+#OPTS="$OPTS --extra-cflags=-fsanitize=undefined \
+    #--extra-cflags=-fno-sanitize=float-divide-by-zero"
+build_qemu \
+    --target-list=x86_64-softmmu,aarch64-softmmu \
+    --enable-debug --cxx=clang++ --cc=clang --host-cc=clang \
+    $OPTS
+make $MAKEFLAGS check
-- 
2.4.3

[Qemu-devel] [PATCH v2 09/15] docker: Add mingw test

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-mingw | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
 create mode 100755 tests/docker/test-mingw

diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
new file mode 100755
index 0000000..25afa64
--- /dev/null
+++ b/tests/docker/test-mingw
@@ -0,0 +1,36 @@
+#!/bin/bash -e
+#
+# Cross compile QEMU with mingw toolchain on Linux.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+requires mingw
+
+mkdir build
+cd build
+for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
+    build_qemu --cross-prefix=$prefix \
+        --target-list=x86_64-softmmu,aarch64-softmmu \
+        --enable-trace-backends=simple \
+        --enable-debug \
+        --enable-gnutls \
+        --enable-nettle \
+        --enable-curl \
+        --enable-vnc \
+        --enable-bzip2 \
+        --enable-guest-agent \
+        --with-sdlabi=1.2 \
+        --with-gtkabi=2.0
+    make clean
+
+done
+
-- 
2.4.3

[Qemu-devel] [PATCH v2 10/15] docker: Add travis tool

[Fam Zheng]

The script is not named test-travis.sh so it won't run with "make
docker-run", because it can take too long.

Run it with "make docker-run-travis.sh@ubuntu".

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/travis    | 23 +++++++++++++++++++++++
 tests/docker/travis.py | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 71 insertions(+)
 create mode 100755 tests/docker/travis
 create mode 100755 tests/docker/travis.py

diff --git a/tests/docker/travis b/tests/docker/travis
new file mode 100755
index 0000000..dfea61f
--- /dev/null
+++ b/tests/docker/travis
@@ -0,0 +1,23 @@
+#!/bin/bash -e
+#
+# Mimic a travis testing matrix
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+requires pyyaml
+
+cd "$QEMU_SRC"
+
+cmdfile=/tmp/travis_cmd_list.sh
+
+./tests/docker/travis.py .travis.yml > $cmdfile
+. $cmdfile
diff --git a/tests/docker/travis.py b/tests/docker/travis.py
new file mode 100755
index 0000000..8dcc964
--- /dev/null
+++ b/tests/docker/travis.py
@@ -0,0 +1,48 @@
+#!/usr/bin/env python
+#
+# Travis YAML config parser
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import sys
+import yaml
+import itertools
+
+def load_yaml(fname):
+    return yaml.load(open(fname, "r").read())
+
+def conf_iter(conf):
+    def env_to_list(env):
+        return env if isinstance(env, list) else [env]
+    global_env = conf["env"]["global"]
+    for entry in conf["matrix"]["include"]:
+        yield {"env": global_env + env_to_list(entry["env"]),
+               "compiler": entry["compiler"]}
+    for entry in itertools.product(conf["compiler"],
+                                   conf["env"]["matrix"]):
+        yield {"env": global_env + env_to_list(entry[1]),
+               "compiler": entry[0]}
+
+def main():
+    if len(sys.argv) < 2:
+        sys.stderr.write("Usage: %s <travis-file>\n" % sys.argv[0])
+        return 1
+    conf = load_yaml(sys.argv[1])
+    for config in conf_iter(conf):
+        print "("
+        print "\n".join(config["env"])
+        print "alias cc=" + config["compiler"]
+        print "\n".join(conf["before_script"])
+        print "\n".join(conf["script"])
+        print ")"
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())
-- 
2.4.3

[Qemu-devel] [PATCH v2 11/15] docs: Add text for tests/docker in build-system.txt

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 docs/build-system.txt | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/docs/build-system.txt b/docs/build-system.txt
index 5ddddea..2af1e66 100644
--- a/docs/build-system.txt
+++ b/docs/build-system.txt
@@ -438,6 +438,11 @@ top level Makefile, so anything defined in this file will influence the
 entire build system. Care needs to be taken when writing rules for tests
 to ensure they only apply to the unit test execution / build.
 
+- tests/docker/Makefile.include
+
+Rules for Docker tests. Like tests/Makefile, this file is included
+directly by the top level Makefile, anything defined in this file will
+influence the entire build system.
 
 - po/Makefile
 
-- 
2.4.3

[Qemu-devel] [PATCH v2 12/15] .gitignore: Ignore temporary dockerfile

[Fam Zheng]

Docker build requires a "context" directory and we use the
$QEMU_SRC/tests/docker/ directory, and the temoprary dockerfile has to be in
the context.

docker_build normally cleans up this file but let's add an entry here just in
case it fails to.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 88a80ff..a335b7b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -109,3 +109,4 @@ cscope.*
 tags
 TAGS
 *~
+/tests/docker/*.docker.tmp
-- 
2.4.3

[Qemu-devel] [PATCH v2 13/15] MAINTAINERS: Add tests/docker

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 02710f8..15b8b93 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1584,3 +1584,10 @@ Build system architecture
 M: Daniel P. Berrange <berrange@redhat.com>
 S: Odd Fixes
 F: docs/build-system.txt
+
+Docker testing
+--------------
+Docker based testing framework and cases
+M: Fam Zheng <famz@redhat.com>
+S: Maintained
+F: tests/docker/
-- 
2.4.3

[Qemu-devel] [PATCH v2 14/15] docker: Support interactive shell for debugging

[Fam Zheng]

Specify "PAUSE=1" when invoking make docker-%, and a shell prompt will
show up before the test runs.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/Makefile.include | 5 ++++-
 tests/docker/run              | 8 ++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d8c55c3..aaf7d23 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -60,6 +60,8 @@ docker:
 	@echo '    TESTS="x y z .."     Filters which tests to run (for docker-test).'
 	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
 	@echo '                         (default is 1)'
+	@echo '    PAUSE=1              Stop and drop to shell in the created container'
+	@echo '                         before running the command.'
 
 docker-image-%:
 	@if test -z "$(IMAGE)"; then echo "Invalid target"; exit 1; fi
@@ -79,9 +81,10 @@ docker-run-%:
 		$(call quiet-command,\
 			$(SRC_PATH)/tests/docker/docker_run $(if $(KEEP),,--rm) \
 				--privileged -t --net=none \
+				$(if $(PAUSE),-i) \
 				-v $$(realpath $(SRC_PATH)):/var/tmp/qemu \
 				-e QEMU_SRC=/var/tmp/qemu \
-				-e V=$V -e J=$J \
+				-e V=$V -e J=$J -e PAUSE=$(PAUSE)\
 				-v /var/tmp/qemu-docker-ccache:/var/tmp/ccache \
 				-e CCACHE_DIR=/var/tmp/ccache \
 				qemu:$(IMAGE) \
diff --git a/tests/docker/run b/tests/docker/run
index 55b2737..cc8d246 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -23,4 +23,12 @@ if test -n "$J"; then
 fi
 
 cd $QEMU_SRC/tests/docker
+
+if test -n "$PAUSE"; then
+    echo "* Prepared to run command:"
+    echo "  $@"
+    echo "* Hit Ctrl-D to continue, or type 'exit 1' to quit"
+    echo
+    $SHELL
+fi
 "$@"
-- 
2.4.3

[Qemu-devel] [PATCH v2 15/15] tests/docker/Makefile.include: snapshot the src for docker

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

Instead of providing a live version of the source tree to the docker
container we snapshot it with git-archive. This ensure the tree is in a
pristine state for whatever operations the container is going to run on
them.

Uncommitted changes known to files known by the git index will be
included in the snapshot if there are any.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
[Generate an unique directory under /tmp/ - Fam Zheng]
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/Makefile.include | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index aaf7d23..7cc2130 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -2,7 +2,7 @@
 
 $(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))
 
-.PHONY: docker docker-test docker-clean docker-image
+.PHONY: docker docker-test docker-clean docker-image docker-qemu-src
 
 DOCKER_SUFFIX := .docker
 
@@ -16,6 +16,21 @@ DOCKER_TOOLS := travis
 
 TESTS ?= %
 IMAGES ?= %
+SRC_COPY := $(shell mktemp -u /tmp/qemu-src.XXXXX)
+
+.PHONY: $(SRC_COPY)
+
+$(SRC_COPY):
+	@mkdir -p $@
+	$(call quiet-command, \
+		mkdir -p $@ && (if git diff-index --quiet HEAD -- &>/dev/null; then \
+			git archive -1 HEAD --format=tar.gz; \
+		else \
+			git archive -1 $$(git stash create) --format=tar.gz; \
+		fi) | tar -xz -C $@, \
+		"  COPY SRC $@")
+
+docker-qemu-src: $(SRC_COPY)
 
 $(foreach i,$(DOCKER_IMAGES), \
 	$(eval docker-image: docker-image-$i) \
@@ -23,8 +38,7 @@ $(foreach i,$(DOCKER_IMAGES), \
 	$(eval .PHONY: docker-image-$i docker-@$i) \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
 		$(eval .PHONY: docker-$t@$i) \
-		$(eval docker-$t@$i: docker-image-$i) \
-		$(eval docker-$t@$i: docker-run-$t@$i) \
+		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
 		$(eval docker-@$i: docker-$t@$i) \
 		$(eval docker-test: docker-run-$t@$i) \
 	) \
@@ -73,7 +87,7 @@ docker-image-%:
 
 docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
 docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%:
+docker-run-%: docker-qemu-src
 	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
 		then echo "Invalid target"; exit 1; \
 	fi
-- 
2.4.3

Re: [Qemu-devel] [PATCH v2 00/15] tests: Introducing docker tests

[Fam Zheng]

On Tue, 02/16 20:39, Fam Zheng wrote:
> v2: - Fix a few coding style warnings of shellcheck and pylint.
>       [Marc-André Lureau]
>     - Rename make targets:
>       docker-run => docker-test
>       docker-build => docker-image
>       docker-run-$TEST.sh@$IMAGE => docker-$TEST@$IMAGE
>     - In help, document the magic makefile vars which are
>       "IMAGES=", "TESTS=", "J=" and "PAUSE="
>     - Rename test-basic => test-quick
>     - Move docker files to tests/docker/dockerfiles/
>     - Add "test-full" that builds all targets
>     - Workaround broken clang for fedora 23
>     - Add commit log for .gitignore changes [Alex]
>     - Add "rev-by" in patch 13. [Alex]
>     - Add patch 14 to make debugging easier [Alex]
>     - Adopt Alex's "git archive" idea

Alex, any comments on this?

Fam

Re: [Qemu-devel] [PATCH v2 00/15] tests: Introducing docker tests

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Tue, 02/16 20:39, Fam Zheng wrote:
>> v2: - Fix a few coding style warnings of shellcheck and pylint.
>>       [Marc-André Lureau]
>>     - Rename make targets:
>>       docker-run => docker-test
>>       docker-build => docker-image
>>       docker-run-$TEST.sh@$IMAGE => docker-$TEST@$IMAGE
>>     - In help, document the magic makefile vars which are
>>       "IMAGES=", "TESTS=", "J=" and "PAUSE="
>>     - Rename test-basic => test-quick
>>     - Move docker files to tests/docker/dockerfiles/
>>     - Add "test-full" that builds all targets
>>     - Workaround broken clang for fedora 23
>>     - Add commit log for .gitignore changes [Alex]
>>     - Add "rev-by" in patch 13. [Alex]
>>     - Add patch 14 to make debugging easier [Alex]
>>     - Adopt Alex's "git archive" idea
>
> Alex, any comments on this?

Yeah sorry for the delay, I'm just starting on this now.

--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 01/15] tests: Add utilities for docker testing

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> docker_run: A wrapper for "docker run" (or "sudo -n docker run" if
> necessary), which takes care of killing and removing the running
> container at SIGINT.
>
> docker_clean: A tool to tear down all the containers including inactive
> ones that are started by docker_run.
>
> docker_build: A tool to compare an image from given dockerfile and
> rebuild it if they're different.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/docker/docker.py    | 113 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/docker/docker_build |  42 +++++++++++++++++
>  tests/docker/docker_clean |  22 +++++++++
>  tests/docker/docker_run   |  29 ++++++++++++
>  4 files changed, 206 insertions(+)
>  create mode 100755 tests/docker/docker.py
>  create mode 100755 tests/docker/docker_build
>  create mode 100755 tests/docker/docker_clean
>  create mode 100755 tests/docker/docker_run
>
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> new file mode 100755
> index 0000000..d175a86
> --- /dev/null
> +++ b/tests/docker/docker.py
> @@ -0,0 +1,113 @@
> +#!/usr/bin/env python2 -B
> +#
> +# Docker controlling module
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +import os
> +import subprocess
> +import json
> +import hashlib
> +import atexit
> +import uuid
> +
> +class ContainerTerminated(Exception):
> +    """ Raised if the container has already existed """
> +    pass
> +
> +class Docker(object):
> +    """ Running Docker commands """
> +    def __init__(self):
> +        self._command = self._guess_command()
> +        self._instances = []
> +        atexit.register(self._kill_instances)
> +
> +    def _do(self, cmd, quiet=True, **kwargs):
> +        if quiet:
> +            kwargs["stdout"] = subprocess.PIPE
> +        return subprocess.call(self._command + cmd, **kwargs)
> +
> +    def _do_kill_instances(self, only_known, only_active=True):
> +        cmd = ["ps", "-q"]

Hmm ps -q barfs on my command line:

16:04 alex@zen/x86_64  [qemu.git/mttcg/multi_tcg_v8_ajb-r2] >ps -q
error: unsupported SysV option

Is there not a more portable way of doing this, even if it is a standard
library?

> +        if not only_active:
> +            cmd.append("-a")
> +        for i in self._output(cmd).split():
> +            resp = self._output(["inspect", i])
> +            labels = json.loads(resp)[0]["Config"]["Labels"]
> +            active = json.loads(resp)[0]["State"]["Running"]
> +            if not labels:
> +                continue
> +            instance_uuid = labels.get("com.qemu.instance.uuid", None)
> +            if not instance_uuid:
> +                continue
> +            if only_known and instance_uuid not in self._instances:
> +                continue
> +            print "Terminating", i
> +            if active:
> +                self._do(["kill", i])
> +            self._do(["rm", i])
> +
> +    def clean(self):
> +        self._do_kill_instances(False, False)
> +        return 0
> +
> +    def _kill_instances(self):
> +        return self._do_kill_instances(True)
> +
> +    def _output(self, cmd, **kwargs):
> +        return subprocess.check_output(self._command + cmd,
> +                                       stderr=subprocess.STDOUT,
> +                                       **kwargs)
> +
> +    def _guess_command(self):
> +        commands = [["docker"], ["sudo", "-n", "docker"]]
> +        for cmd in commands:
> +            if subprocess.call(cmd + ["images"],
> +                               stdout=subprocess.PIPE,
> +                               stderr=subprocess.PIPE) == 0:
> +                return cmd
> +        commands_txt = "\n".join(["  " + " ".join(x) for x in commands])
> +        raise Exception("Cannot find working docker command. Tried:\n%s" % commands_txt)
> +
> +    def get_image_dockerfile_checksum(self, tag):
> +        resp = self._output(["inspect", tag])
> +        labels = json.loads(resp)[0]["Config"].get("Labels", {})
> +        return labels.get("com.qemu.dockerfile-checksum", "")
> +
> +    def checksum(self, text):
> +        return hashlib.sha1(text).hexdigest()
> +
> +    def build_image(self, tag, dockerfile, df, quiet=True):
> +        tmp = dockerfile + "\n" + \
> +              "LABEL com.qemu.dockerfile-checksum=%s" % self.checksum(dockerfile)
> +        tmp_df = df + ".tmp"
> +        tmp_file = open(tmp_df, "wb")
> +        tmp_file.write(tmp)
> +        tmp_file.close()
> +        self._do(["build", "-t", tag, "-f", tmp_df, os.path.dirname(df)],
> +                 quiet=quiet)
> +        os.unlink(tmp_df)
> +
> +    def image_matches_dockerfile(self, tag, dockerfile):
> +        try:
> +            checksum = self.get_image_dockerfile_checksum(tag)
> +        except:
> +            return False
> +        return checksum == self.checksum(dockerfile)
> +
> +    def run(self, cmd, keep, quiet):
> +        label = uuid.uuid1().hex
> +        if not keep:
> +            self._instances.append(label)
> +        ret = self._do(["run", "--label", "com.qemu.instance.uuid=" + label] + cmd, quiet=quiet)
> +        if not keep:
> +            self._instances.remove(label)
> +        return ret

I think it might be useful to catch some arguments here for testing
things. It is likely to be the first script someone runs while poking
around so some help text would be useful even if it just points at the
other commands.

In fact I'm not sure why all the various commands aren't in one script
for now given this does most of the heavy lifting.

> +
> diff --git a/tests/docker/docker_build b/tests/docker/docker_build
> new file mode 100755
> index 0000000..6948e2c
> --- /dev/null
> +++ b/tests/docker/docker_build
> @@ -0,0 +1,42 @@
> +#!/usr/bin/env python2
> +#
> +# Compare to Dockerfile and rebuild a docker image if necessary.
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +import sys
> +import docker
> +import argparse
> +
> +def main():
> +    parser = argparse.ArgumentParser()
> +    parser.add_argument("tag",
> +                        help="Image Tag")
> +    parser.add_argument("dockerfile",
> +                        help="Dockerfile name")
> +    parser.add_argument("--verbose", "-v", action="store_true",
> +                        help="Print verbose information")
> +    args = parser.parse_args()
> +
> +    dockerfile = open(args.dockerfile, "rb").read()
> +    tag = args.tag
> +
> +    dkr = docker.Docker()
> +    if dkr.image_matches_dockerfile(tag, dockerfile):
> +        if args.verbose:
> +            print "Image is up to date."
> +        return 0
> +
> +    quiet = not args.verbose
> +    dkr.build_image(tag, dockerfile, args.dockerfile, quiet=quiet)
> +    return 0
> +
> +if __name__ == "__main__":
> +    sys.exit(main())
> diff --git a/tests/docker/docker_clean b/tests/docker/docker_clean
> new file mode 100755
> index 0000000..88cdba6
> --- /dev/null
> +++ b/tests/docker/docker_clean
> @@ -0,0 +1,22 @@
> +#!/usr/bin/env python2
> +#
> +# Clean up uselsee containers.
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +import sys
> +import docker
> +
> +def main():
> +    docker.Docker().clean()
> +    return 0
> +
> +if __name__ == "__main__":
> +    sys.exit(main())

Of all the scripts run if you call with --help this just does something
straight away. It should at least attempt a usage() text to prevent
accidents.

> diff --git a/tests/docker/docker_run b/tests/docker/docker_run
> new file mode 100755
> index 0000000..4c46d90
> --- /dev/null
> +++ b/tests/docker/docker_run
> @@ -0,0 +1,29 @@
> +#!/usr/bin/env python2
> +#
> +# Wrapper for "docker run" with automatical clean up if the execution is
> +# iterrupted.
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +import sys
> +import argparse
> +import docker
> +
> +def main():
> +    parser = argparse.ArgumentParser()
> +    parser.add_argument("--keep", action="store_true",
> +                        help="Don't remove image when the command completes")
> +    parser.add_argument("--quiet", action="store_true",
> +                        help="Run quietly unless an error occured")
> +    args, argv = parser.parse_known_args()
> +    return docker.Docker().run(argv, args.keep, quiet=args.quiet)
> +
> +if __name__ == "__main__":
> +    sys.exit(main())


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 02/15] Makefile: Rules for docker testing

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> This adds a group of make targets to run docker tests, all are available
> in source tree without running ./configure.
>
> The usage is shown by "make docker".
>
> Besides the fixed ones, dynamic targets for building each image and
> running each test in each image are generated automatically by make,
> scanning $(SRC_PATH)/tests/docker/ files with specific patterns.
>
> Alternative to manually list particular targets (docker-run-FOO@BAR)
> set, you can control which tests/images to run by filtering variables,
> TESTS= and IMAGES=, which are expressed in Makefile pattern syntax,
> "foo% %bar ...". For example:
>
>     $ make docker-run IMAGES="ubuntu fedora"

This doesn't seem to work for me:

16:49 alex@zen/x86_64  [qemu.git/review/docker-tests-v2] >make
docker-run IMAGES="ubuntu" V=1
/home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: overriding
commands for target `clean-timestamp'
/home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: ignoring old
commands for target `clean-timestamp'
make: *** No rule to make target `docker-run'. Stop.
`

>
> Unfortunately, it's impossible to propagate "-j $JOBS" into make in
> containers, however since each combination is made a first class target
> is the top Makefile, "make -j$N docker-run" still parallels the tests
> coarsely.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  Makefile                      |  4 +-
>  tests/docker/Makefile.include | 93 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 96 insertions(+), 1 deletion(-)
>  create mode 100644 tests/docker/Makefile.include
>
> diff --git a/Makefile b/Makefile
> index f9fae3a..f104a08 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -6,7 +6,7 @@ BUILD_DIR=$(CURDIR)
>  # Before including a proper config-host.mak, assume we are in the source tree
>  SRC_PATH=.
>
> -UNCHECKED_GOALS := %clean TAGS cscope ctags
> +UNCHECKED_GOALS := %clean TAGS cscope ctags docker docker-%
>
>  # All following code might depend on configuration variables
>  ifneq ($(wildcard config-host.mak),)
> @@ -651,3 +651,5 @@ endif
>  # Include automatically generated dependency files
>  # Dependencies in Makefile.objs files come from our recursive subdir rules
>  -include $(wildcard *.d tests/*.d)
> +
> +include $(SRC_PATH)/tests/docker/Makefile.include
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> new file mode 100644
> index 0000000..d8c55c3
> --- /dev/null
> +++ b/tests/docker/Makefile.include
> @@ -0,0 +1,93 @@
> +# Makefile for Docker tests
> +
> +$(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))
> +
> +.PHONY: docker docker-test docker-clean docker-image
> +
> +DOCKER_SUFFIX := .docker
> +
> +DOCKER_IMAGES := $(notdir $(basename \
> +	$(wildcard $(SRC_PATH)/tests/docker/dockerfiles/*)))
> +
> +DOCKER_TESTS := $(notdir $(shell \
> +	find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f -executable))
> +
> +DOCKER_TOOLS := travis
> +
> +TESTS ?= %
> +IMAGES ?= %
> +
> +$(foreach i,$(DOCKER_IMAGES), \
> +	$(eval docker-image: docker-image-$i) \
> +	$(eval docker-image-$i: IMAGE = $i) \
> +	$(eval .PHONY: docker-image-$i docker-@$i) \
> +	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
> +		$(eval .PHONY: docker-$t@$i) \
> +		$(eval docker-$t@$i: docker-image-$i) \
> +		$(eval docker-$t@$i: docker-run-$t@$i) \
> +		$(eval docker-@$i: docker-$t@$i) \
> +		$(eval docker-test: docker-run-$t@$i) \
> +	) \
> +)
> +
> +docker:
> +	@echo 'Building QEMU and running tests inside Docker containers'
> +	@echo
> +	@echo 'Available targets:'
> +	@echo
> +	@echo '    docker:              Print this help.'
> +	@echo '    docker-test:         Run all image/test combinations.'
> +	@echo '    docker-clean:        Kill and remove residual docker testing containers.'
> +	@echo '    docker-TEST@IMAGE:   Run "TEST" in container "IMAGE".'
> +	@echo '                         Note: "TEST" is one of the listed test name,'
> +	@echo '                         or a script name under $$QEMU_SRC/tests/docker/;'
> +	@echo '                         "IMAGE" is one of the listed container name."'
> +	@echo '    docker-@IMAGE:       Run all tests in container "IMAGE".'
> +	@echo '    docker-image:        Build all images.'
> +	@echo '    docker-image-IMAGE:  Build image "IMG".'
> +	@echo
> +	@echo 'Available container images:'
> +	@echo '    $(DOCKER_IMAGES)'
> +	@echo
> +	@echo 'Available tests:'
> +	@echo '    $(DOCKER_TESTS)'
> +	@echo
> +	@echo 'Available tools:'
> +	@echo '    $(DOCKER_TOOLS)'
> +	@echo
> +	@echo 'Special variables:'
> +	@echo '    IMAGES="a b c ..":   Filters which images to build or run.'
> +	@echo '    TESTS="x y z .."     Filters which tests to run (for docker-test).'
> +	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
> +	@echo '                         (default is 1)'

OK none of these test targets seem to be working for me:

  COPY SRC /tmp/qemu-src.JCUng
  RUN test-clang in centos6
Prerequisite 'clang' not present, skip
  RUN test-mingw in centos6
Prerequisite 'mingw' not present, skip
  RUN test-quick in centos6
make: *** [docker-run-test-quick@centos6] Error 2

Running with V=1 doesn't elucidate me much further:

/home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: overriding commands for target `clean-timestamp'
/home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: ignoring old commands for target `clean-timestamp'
mkdir -p /tmp/qemu-src.HtY32 && (if git diff-index --quiet HEAD -- &>/dev/null; then git archive -1 HEAD --format=tar.gz; else git archive -1 $(git stash create) --format=tar.gz; fi) | tar -xz -C /tmp/qemu-src.HtY32
/home/alex/lsrc/qemu/qemu.git/tests/docker/docker_run --rm --privileged -t --net=none  -v $(realpath /home/alex/lsrc/qemu/qemu.git):/var/tmp/qemu -e QEMU_SRC=/var/tmp/qemu -e V=1 -e J= -e PAUSE= -v /var/tmp/qemu-docker-ccache:/var/tmp/ccache -e CCACHE_DIR=/var/tmp/ccache qemu:centos6 /var/tmp/qemu/tests/docker/run /var/tmp/qemu/tests/docker/test-clang;
Prerequisite 'clang' not present, skip
/home/alex/lsrc/qemu/qemu.git/tests/docker/docker_run --rm --privileged -t --net=none  -v $(realpath /home/alex/lsrc/qemu/qemu.git):/var/tmp/qemu -e QEMU_SRC=/var/tmp/qemu -e V=1 -e J= -e PAUSE= -v /var/tmp/qemu-docker-ccache:/var/tmp/ccache -e CCACHE_DIR=/var/tmp/ccache qemu:centos6 /var/tmp/qemu/tests/docker/run /var/tmp/qemu/tests/docker/test-mingw;
Prerequisite 'mingw' not present, skip
/home/alex/lsrc/qemu/qemu.git/tests/docker/docker_run --rm --privileged -t --net=none  -v $(realpath /home/alex/lsrc/qemu/qemu.git):/var/tmp/qemu -e QEMU_SRC=/var/tmp/qemu -e V=1 -e J= -e PAUSE= -v /var/tmp/qemu-docker-ccache:/var/tmp/ccache -e CCACHE_DIR=/var/tmp/ccache qemu:centos6 /var/tmp/qemu/tests/docker/run /var/tmp/qemu/tests/docker/test-quick;
make: *** [docker-run-test-quick@centos6] Error 2

If I drop to the shell using:

make docker-test-quick@centos6 V=1 PAUSE=1

And run:

/var/tmp/qemu/tests/docker/test-quick

I get a cryptic silent fail. I think we need more verbosity (select-able
when required) to see whats going on in the depths.

> +
> +docker-image-%:
> +	@if test -z "$(IMAGE)"; then echo "Invalid target"; exit 1; fi
> +	$(if $(filter $(IMAGES),$(IMAGE)), $(call quiet-command,\
> +		$(SRC_PATH)/tests/docker/docker_build qemu:$(IMAGE) \
> +			$(SRC_PATH)/tests/docker/dockerfiles/$(IMAGE).docker \
> +			$(if $V,-v,), "  BUILD $(IMAGE)"))
> +
> +
> +docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> +docker-run-%:
> +	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> +		then echo "Invalid target"; exit 1; \
> +	fi
> +	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> +		$(call quiet-command,\
> +			$(SRC_PATH)/tests/docker/docker_run $(if $(KEEP),,--rm) \
> +				--privileged -t --net=none \
> +				-v $$(realpath $(SRC_PATH)):/var/tmp/qemu \
> +				-e QEMU_SRC=/var/tmp/qemu \
> +				-e V=$V -e J=$J \
> +				-v /var/tmp/qemu-docker-ccache:/var/tmp/ccache \
> +				-e CCACHE_DIR=/var/tmp/ccache \
> +				qemu:$(IMAGE) \
> +				/var/tmp/qemu/tests/docker/run \
> +				/var/tmp/qemu/tests/docker/$(CMD); \
> +			, "  RUN $(CMD) in $(IMAGE)")))
> +
> +docker-clean:
> +	$(call quiet-command, $(SRC_PATH)/tests/docker/docker_clean)


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 03/15] docker: Add images

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> Signed-off-by: Fam Zheng <famz@redhat.com>

I think we need to include the dtc libs in here unless we need to ship
all submodules in the tree as well.

Is there a way to get a plain install shell? PAUSE=1 on the test shells
doesn't seem to allow me to install anything.

> ---
>  tests/docker/dockerfiles/centos6.docker | 5 +++++
>  tests/docker/dockerfiles/fedora.docker  | 7 +++++++
>  tests/docker/dockerfiles/ubuntu.docker  | 9 +++++++++
>  3 files changed, 21 insertions(+)
>  create mode 100644 tests/docker/dockerfiles/centos6.docker
>  create mode 100644 tests/docker/dockerfiles/fedora.docker
>  create mode 100644 tests/docker/dockerfiles/ubuntu.docker
>
> diff --git a/tests/docker/dockerfiles/centos6.docker b/tests/docker/dockerfiles/centos6.docker
> new file mode 100644
> index 0000000..7daa2fb
> --- /dev/null
> +++ b/tests/docker/dockerfiles/centos6.docker
> @@ -0,0 +1,5 @@
> +FROM centos:6
> +RUN yum install -y \
> +    git make gcc g++ \
> +    zlib-devel glib2-devel SDL-devel pixman-devel
> +
> diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
> new file mode 100644
> index 0000000..81091b7
> --- /dev/null
> +++ b/tests/docker/dockerfiles/fedora.docker
> @@ -0,0 +1,7 @@
> +FROM fedora:23
> +RUN dnf install -y \
> +    ccache git \
> +    glib2-devel pixman-devel zlib-devel SDL-devel \
> +    gcc gcc-c++ clang make perl which bc findutils \
> +    mingw{32,64}-{pixman,glib2,gmp,SDL,pkg-config,gtk2,gtk3,gnutls,nettle,libtasn1,libjpeg-turbo,libpng,curl,libssh2,bzip2}
> +ENV FEATURES mingw clang
> diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
> new file mode 100644
> index 0000000..f1dc518
> --- /dev/null
> +++ b/tests/docker/dockerfiles/ubuntu.docker
> @@ -0,0 +1,9 @@
> +FROM ubuntu:14.04
> +RUN apt-get update
> +RUN apt-get -y install \
> +    libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev \
> +    libseccomp-dev libgnutls-dev libssh2-1-dev  libspice-server-dev \
> +    libspice-protocol-dev libnss3-dev \
> +    libgtk-3-dev libvte-2.90-dev libsdl1.2-dev libpng12-dev libpixman-1-dev \
> +    git make ccache python-yaml gcc clang
> +ENV FEATURES clang ccache pyyaml


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 00/15] tests: Introducing docker tests

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> Fam Zheng <famz@redhat.com> writes:
>
>> On Tue, 02/16 20:39, Fam Zheng wrote:
>>> v2: - Fix a few coding style warnings of shellcheck and pylint.
>>>       [Marc-André Lureau]
>>>     - Rename make targets:
>>>       docker-run => docker-test
>>>       docker-build => docker-image
>>>       docker-run-$TEST.sh@$IMAGE => docker-$TEST@$IMAGE
>>>     - In help, document the magic makefile vars which are
>>>       "IMAGES=", "TESTS=", "J=" and "PAUSE="
>>>     - Rename test-basic => test-quick
>>>     - Move docker files to tests/docker/dockerfiles/
>>>     - Add "test-full" that builds all targets
>>>     - Workaround broken clang for fedora 23
>>>     - Add commit log for .gitignore changes [Alex]
>>>     - Add "rev-by" in patch 13. [Alex]
>>>     - Add patch 14 to make debugging easier [Alex]
>>>     - Adopt Alex's "git archive" idea
>>
>> Alex, any comments on this?
>
> Yeah sorry for the delay, I'm just starting on this now.

OK I'm not sure I can go on without an easier way off seeing where it is
failing when it fails. Can we pass optional verbosity down to the final
build and tests?

--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 02/15] Makefile: Rules for docker testing

[Fam Zheng]

On Mon, 02/29 17:08, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > This adds a group of make targets to run docker tests, all are available
> > in source tree without running ./configure.
> >
> > The usage is shown by "make docker".
> >
> > Besides the fixed ones, dynamic targets for building each image and
> > running each test in each image are generated automatically by make,
> > scanning $(SRC_PATH)/tests/docker/ files with specific patterns.
> >
> > Alternative to manually list particular targets (docker-run-FOO@BAR)
> > set, you can control which tests/images to run by filtering variables,
> > TESTS= and IMAGES=, which are expressed in Makefile pattern syntax,
> > "foo% %bar ...". For example:
> >
> >     $ make docker-run IMAGES="ubuntu fedora"
> 
> This doesn't seem to work for me:
> 
> 16:49 alex@zen/x86_64  [qemu.git/review/docker-tests-v2] >make
> docker-run IMAGES="ubuntu" V=1
> /home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: overriding
> commands for target `clean-timestamp'
> /home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: ignoring old
> commands for target `clean-timestamp'
> make: *** No rule to make target `docker-run'. Stop.

The commit message is stale, as the help text, the target is now "docker-test"
which is more intuitive.

Fam

> `
> 
> >
> > Unfortunately, it's impossible to propagate "-j $JOBS" into make in
> > containers, however since each combination is made a first class target
> > is the top Makefile, "make -j$N docker-run" still parallels the tests
> > coarsely.
> >
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  Makefile                      |  4 +-
> >  tests/docker/Makefile.include | 93 +++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 96 insertions(+), 1 deletion(-)
> >  create mode 100644 tests/docker/Makefile.include
> >
> > diff --git a/Makefile b/Makefile
> > index f9fae3a..f104a08 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -6,7 +6,7 @@ BUILD_DIR=$(CURDIR)
> >  # Before including a proper config-host.mak, assume we are in the source tree
> >  SRC_PATH=.
> >
> > -UNCHECKED_GOALS := %clean TAGS cscope ctags
> > +UNCHECKED_GOALS := %clean TAGS cscope ctags docker docker-%
> >
> >  # All following code might depend on configuration variables
> >  ifneq ($(wildcard config-host.mak),)
> > @@ -651,3 +651,5 @@ endif
> >  # Include automatically generated dependency files
> >  # Dependencies in Makefile.objs files come from our recursive subdir rules
> >  -include $(wildcard *.d tests/*.d)
> > +
> > +include $(SRC_PATH)/tests/docker/Makefile.include
> > diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> > new file mode 100644
> > index 0000000..d8c55c3
> > --- /dev/null
> > +++ b/tests/docker/Makefile.include
> > @@ -0,0 +1,93 @@
> > +# Makefile for Docker tests
> > +
> > +$(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))
> > +
> > +.PHONY: docker docker-test docker-clean docker-image
> > +
> > +DOCKER_SUFFIX := .docker
> > +
> > +DOCKER_IMAGES := $(notdir $(basename \
> > +	$(wildcard $(SRC_PATH)/tests/docker/dockerfiles/*)))
> > +
> > +DOCKER_TESTS := $(notdir $(shell \
> > +	find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f -executable))
> > +
> > +DOCKER_TOOLS := travis
> > +
> > +TESTS ?= %
> > +IMAGES ?= %
> > +
> > +$(foreach i,$(DOCKER_IMAGES), \
> > +	$(eval docker-image: docker-image-$i) \
> > +	$(eval docker-image-$i: IMAGE = $i) \
> > +	$(eval .PHONY: docker-image-$i docker-@$i) \
> > +	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
> > +		$(eval .PHONY: docker-$t@$i) \
> > +		$(eval docker-$t@$i: docker-image-$i) \
> > +		$(eval docker-$t@$i: docker-run-$t@$i) \
> > +		$(eval docker-@$i: docker-$t@$i) \
> > +		$(eval docker-test: docker-run-$t@$i) \
> > +	) \
> > +)
> > +
> > +docker:
> > +	@echo 'Building QEMU and running tests inside Docker containers'
> > +	@echo
> > +	@echo 'Available targets:'
> > +	@echo
> > +	@echo '    docker:              Print this help.'
> > +	@echo '    docker-test:         Run all image/test combinations.'
> > +	@echo '    docker-clean:        Kill and remove residual docker testing containers.'
> > +	@echo '    docker-TEST@IMAGE:   Run "TEST" in container "IMAGE".'
> > +	@echo '                         Note: "TEST" is one of the listed test name,'
> > +	@echo '                         or a script name under $$QEMU_SRC/tests/docker/;'
> > +	@echo '                         "IMAGE" is one of the listed container name."'
> > +	@echo '    docker-@IMAGE:       Run all tests in container "IMAGE".'
> > +	@echo '    docker-image:        Build all images.'
> > +	@echo '    docker-image-IMAGE:  Build image "IMG".'
> > +	@echo
> > +	@echo 'Available container images:'
> > +	@echo '    $(DOCKER_IMAGES)'
> > +	@echo
> > +	@echo 'Available tests:'
> > +	@echo '    $(DOCKER_TESTS)'
> > +	@echo
> > +	@echo 'Available tools:'
> > +	@echo '    $(DOCKER_TOOLS)'
> > +	@echo
> > +	@echo 'Special variables:'
> > +	@echo '    IMAGES="a b c ..":   Filters which images to build or run.'
> > +	@echo '    TESTS="x y z .."     Filters which tests to run (for docker-test).'
> > +	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
> > +	@echo '                         (default is 1)'
> 
> OK none of these test targets seem to be working for me:
> 
>   COPY SRC /tmp/qemu-src.JCUng
>   RUN test-clang in centos6
> Prerequisite 'clang' not present, skip
>   RUN test-mingw in centos6
> Prerequisite 'mingw' not present, skip
>   RUN test-quick in centos6
> make: *** [docker-run-test-quick@centos6] Error 2
> 
> Running with V=1 doesn't elucidate me much further:
> 
> /home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: overriding commands for target `clean-timestamp'
> /home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: ignoring old commands for target `clean-timestamp'
> mkdir -p /tmp/qemu-src.HtY32 && (if git diff-index --quiet HEAD -- &>/dev/null; then git archive -1 HEAD --format=tar.gz; else git archive -1 $(git stash create) --format=tar.gz; fi) | tar -xz -C /tmp/qemu-src.HtY32
> /home/alex/lsrc/qemu/qemu.git/tests/docker/docker_run --rm --privileged -t --net=none  -v $(realpath /home/alex/lsrc/qemu/qemu.git):/var/tmp/qemu -e QEMU_SRC=/var/tmp/qemu -e V=1 -e J= -e PAUSE= -v /var/tmp/qemu-docker-ccache:/var/tmp/ccache -e CCACHE_DIR=/var/tmp/ccache qemu:centos6 /var/tmp/qemu/tests/docker/run /var/tmp/qemu/tests/docker/test-clang;
> Prerequisite 'clang' not present, skip
> /home/alex/lsrc/qemu/qemu.git/tests/docker/docker_run --rm --privileged -t --net=none  -v $(realpath /home/alex/lsrc/qemu/qemu.git):/var/tmp/qemu -e QEMU_SRC=/var/tmp/qemu -e V=1 -e J= -e PAUSE= -v /var/tmp/qemu-docker-ccache:/var/tmp/ccache -e CCACHE_DIR=/var/tmp/ccache qemu:centos6 /var/tmp/qemu/tests/docker/run /var/tmp/qemu/tests/docker/test-mingw;
> Prerequisite 'mingw' not present, skip
> /home/alex/lsrc/qemu/qemu.git/tests/docker/docker_run --rm --privileged -t --net=none  -v $(realpath /home/alex/lsrc/qemu/qemu.git):/var/tmp/qemu -e QEMU_SRC=/var/tmp/qemu -e V=1 -e J= -e PAUSE= -v /var/tmp/qemu-docker-ccache:/var/tmp/ccache -e CCACHE_DIR=/var/tmp/ccache qemu:centos6 /var/tmp/qemu/tests/docker/run /var/tmp/qemu/tests/docker/test-quick;
> make: *** [docker-run-test-quick@centos6] Error 2
> 
> If I drop to the shell using:
> 
> make docker-test-quick@centos6 V=1 PAUSE=1
> 
> And run:
> 
> /var/tmp/qemu/tests/docker/test-quick
> 
> I get a cryptic silent fail. I think we need more verbosity (select-able
> when required) to see whats going on in the depths.
> 
> > +
> > +docker-image-%:
> > +	@if test -z "$(IMAGE)"; then echo "Invalid target"; exit 1; fi
> > +	$(if $(filter $(IMAGES),$(IMAGE)), $(call quiet-command,\
> > +		$(SRC_PATH)/tests/docker/docker_build qemu:$(IMAGE) \
> > +			$(SRC_PATH)/tests/docker/dockerfiles/$(IMAGE).docker \
> > +			$(if $V,-v,), "  BUILD $(IMAGE)"))
> > +
> > +
> > +docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> > +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> > +docker-run-%:
> > +	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> > +		then echo "Invalid target"; exit 1; \
> > +	fi
> > +	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> > +		$(call quiet-command,\
> > +			$(SRC_PATH)/tests/docker/docker_run $(if $(KEEP),,--rm) \
> > +				--privileged -t --net=none \
> > +				-v $$(realpath $(SRC_PATH)):/var/tmp/qemu \
> > +				-e QEMU_SRC=/var/tmp/qemu \
> > +				-e V=$V -e J=$J \
> > +				-v /var/tmp/qemu-docker-ccache:/var/tmp/ccache \
> > +				-e CCACHE_DIR=/var/tmp/ccache \
> > +				qemu:$(IMAGE) \
> > +				/var/tmp/qemu/tests/docker/run \
> > +				/var/tmp/qemu/tests/docker/$(CMD); \
> > +			, "  RUN $(CMD) in $(IMAGE)")))
> > +
> > +docker-clean:
> > +	$(call quiet-command, $(SRC_PATH)/tests/docker/docker_clean)
> 
> 
> --
> Alex Bennée

Re: [Qemu-devel] [PATCH v2 01/15] tests: Add utilities for docker testing

[Fam Zheng]

On Mon, 02/29 16:46, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > docker_run: A wrapper for "docker run" (or "sudo -n docker run" if
> > necessary), which takes care of killing and removing the running
> > container at SIGINT.
> >
> > docker_clean: A tool to tear down all the containers including inactive
> > ones that are started by docker_run.
> >
> > docker_build: A tool to compare an image from given dockerfile and
> > rebuild it if they're different.
> >
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  tests/docker/docker.py    | 113 ++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/docker/docker_build |  42 +++++++++++++++++
> >  tests/docker/docker_clean |  22 +++++++++
> >  tests/docker/docker_run   |  29 ++++++++++++
> >  4 files changed, 206 insertions(+)
> >  create mode 100755 tests/docker/docker.py
> >  create mode 100755 tests/docker/docker_build
> >  create mode 100755 tests/docker/docker_clean
> >  create mode 100755 tests/docker/docker_run
> >
> > diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> > new file mode 100755
> > index 0000000..d175a86
> > --- /dev/null
> > +++ b/tests/docker/docker.py
> > @@ -0,0 +1,113 @@
> > +#!/usr/bin/env python2 -B
> > +#
> > +# Docker controlling module
> > +#
> > +# Copyright (c) 2016 Red Hat Inc.
> > +#
> > +# Authors:
> > +#  Fam Zheng <famz@redhat.com>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2
> > +# or (at your option) any later version. See the COPYING file in
> > +# the top-level directory.
> > +
> > +import os
> > +import subprocess
> > +import json
> > +import hashlib
> > +import atexit
> > +import uuid
> > +
> > +class ContainerTerminated(Exception):
> > +    """ Raised if the container has already existed """
> > +    pass
> > +
> > +class Docker(object):
> > +    """ Running Docker commands """
> > +    def __init__(self):
> > +        self._command = self._guess_command()
> > +        self._instances = []
> > +        atexit.register(self._kill_instances)
> > +
> > +    def _do(self, cmd, quiet=True, **kwargs):
> > +        if quiet:
> > +            kwargs["stdout"] = subprocess.PIPE
> > +        return subprocess.call(self._command + cmd, **kwargs)
> > +
> > +    def _do_kill_instances(self, only_known, only_active=True):
> > +        cmd = ["ps", "-q"]
> 
> Hmm ps -q barfs on my command line:
> 
> 16:04 alex@zen/x86_64  [qemu.git/mttcg/multi_tcg_v8_ajb-r2] >ps -q
> error: unsupported SysV option
> 
> Is there not a more portable way of doing this, even if it is a standard
> library?

Down the road this is "sudo docker ps" command. :)

> 
> > +        if not only_active:
> > +            cmd.append("-a")
> > +        for i in self._output(cmd).split():
> > +            resp = self._output(["inspect", i])
> > +            labels = json.loads(resp)[0]["Config"]["Labels"]
> > +            active = json.loads(resp)[0]["State"]["Running"]
> > +            if not labels:
> > +                continue
> > +            instance_uuid = labels.get("com.qemu.instance.uuid", None)
> > +            if not instance_uuid:
> > +                continue
> > +            if only_known and instance_uuid not in self._instances:
> > +                continue
> > +            print "Terminating", i
> > +            if active:
> > +                self._do(["kill", i])
> > +            self._do(["rm", i])
> > +
> > +    def clean(self):
> > +        self._do_kill_instances(False, False)
> > +        return 0
> > +
> > +    def _kill_instances(self):
> > +        return self._do_kill_instances(True)
> > +
> > +    def _output(self, cmd, **kwargs):
> > +        return subprocess.check_output(self._command + cmd,
> > +                                       stderr=subprocess.STDOUT,
> > +                                       **kwargs)
> > +
> > +    def _guess_command(self):
> > +        commands = [["docker"], ["sudo", "-n", "docker"]]
> > +        for cmd in commands:
> > +            if subprocess.call(cmd + ["images"],
> > +                               stdout=subprocess.PIPE,
> > +                               stderr=subprocess.PIPE) == 0:
> > +                return cmd
> > +        commands_txt = "\n".join(["  " + " ".join(x) for x in commands])
> > +        raise Exception("Cannot find working docker command. Tried:\n%s" % commands_txt)
> > +
> > +    def get_image_dockerfile_checksum(self, tag):
> > +        resp = self._output(["inspect", tag])
> > +        labels = json.loads(resp)[0]["Config"].get("Labels", {})
> > +        return labels.get("com.qemu.dockerfile-checksum", "")
> > +
> > +    def checksum(self, text):
> > +        return hashlib.sha1(text).hexdigest()
> > +
> > +    def build_image(self, tag, dockerfile, df, quiet=True):
> > +        tmp = dockerfile + "\n" + \
> > +              "LABEL com.qemu.dockerfile-checksum=%s" % self.checksum(dockerfile)
> > +        tmp_df = df + ".tmp"
> > +        tmp_file = open(tmp_df, "wb")
> > +        tmp_file.write(tmp)
> > +        tmp_file.close()
> > +        self._do(["build", "-t", tag, "-f", tmp_df, os.path.dirname(df)],
> > +                 quiet=quiet)
> > +        os.unlink(tmp_df)
> > +
> > +    def image_matches_dockerfile(self, tag, dockerfile):
> > +        try:
> > +            checksum = self.get_image_dockerfile_checksum(tag)
> > +        except:
> > +            return False
> > +        return checksum == self.checksum(dockerfile)
> > +
> > +    def run(self, cmd, keep, quiet):
> > +        label = uuid.uuid1().hex
> > +        if not keep:
> > +            self._instances.append(label)
> > +        ret = self._do(["run", "--label", "com.qemu.instance.uuid=" + label] + cmd, quiet=quiet)
> > +        if not keep:
> > +            self._instances.remove(label)
> > +        return ret
> 
> I think it might be useful to catch some arguments here for testing
> things. It is likely to be the first script someone runs while poking
> around so some help text would be useful even if it just points at the
> other commands.

Sure, I can do that.

> 
> In fact I'm not sure why all the various commands aren't in one script
> for now given this does most of the heavy lifting.

OK, I can merge them into one script.


> 
> > +
> > diff --git a/tests/docker/docker_build b/tests/docker/docker_build
> > new file mode 100755
> > index 0000000..6948e2c
> > --- /dev/null
> > +++ b/tests/docker/docker_build
> > @@ -0,0 +1,42 @@
> > +#!/usr/bin/env python2
> > +#
> > +# Compare to Dockerfile and rebuild a docker image if necessary.
> > +#
> > +# Copyright (c) 2016 Red Hat Inc.
> > +#
> > +# Authors:
> > +#  Fam Zheng <famz@redhat.com>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2
> > +# or (at your option) any later version. See the COPYING file in
> > +# the top-level directory.
> > +
> > +import sys
> > +import docker
> > +import argparse
> > +
> > +def main():
> > +    parser = argparse.ArgumentParser()
> > +    parser.add_argument("tag",
> > +                        help="Image Tag")
> > +    parser.add_argument("dockerfile",
> > +                        help="Dockerfile name")
> > +    parser.add_argument("--verbose", "-v", action="store_true",
> > +                        help="Print verbose information")
> > +    args = parser.parse_args()
> > +
> > +    dockerfile = open(args.dockerfile, "rb").read()
> > +    tag = args.tag
> > +
> > +    dkr = docker.Docker()
> > +    if dkr.image_matches_dockerfile(tag, dockerfile):
> > +        if args.verbose:
> > +            print "Image is up to date."
> > +        return 0
> > +
> > +    quiet = not args.verbose
> > +    dkr.build_image(tag, dockerfile, args.dockerfile, quiet=quiet)
> > +    return 0
> > +
> > +if __name__ == "__main__":
> > +    sys.exit(main())
> > diff --git a/tests/docker/docker_clean b/tests/docker/docker_clean
> > new file mode 100755
> > index 0000000..88cdba6
> > --- /dev/null
> > +++ b/tests/docker/docker_clean
> > @@ -0,0 +1,22 @@
> > +#!/usr/bin/env python2
> > +#
> > +# Clean up uselsee containers.
> > +#
> > +# Copyright (c) 2016 Red Hat Inc.
> > +#
> > +# Authors:
> > +#  Fam Zheng <famz@redhat.com>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2
> > +# or (at your option) any later version. See the COPYING file in
> > +# the top-level directory.
> > +
> > +import sys
> > +import docker
> > +
> > +def main():
> > +    docker.Docker().clean()
> > +    return 0
> > +
> > +if __name__ == "__main__":
> > +    sys.exit(main())
> 
> Of all the scripts run if you call with --help this just does something
> straight away. It should at least attempt a usage() text to prevent
> accidents.

Yes. Will address.

Fam

> 
> > diff --git a/tests/docker/docker_run b/tests/docker/docker_run
> > new file mode 100755
> > index 0000000..4c46d90
> > --- /dev/null
> > +++ b/tests/docker/docker_run
> > @@ -0,0 +1,29 @@
> > +#!/usr/bin/env python2
> > +#
> > +# Wrapper for "docker run" with automatical clean up if the execution is
> > +# iterrupted.
> > +#
> > +# Copyright (c) 2016 Red Hat Inc.
> > +#
> > +# Authors:
> > +#  Fam Zheng <famz@redhat.com>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2
> > +# or (at your option) any later version. See the COPYING file in
> > +# the top-level directory.
> > +
> > +import sys
> > +import argparse
> > +import docker
> > +
> > +def main():
> > +    parser = argparse.ArgumentParser()
> > +    parser.add_argument("--keep", action="store_true",
> > +                        help="Don't remove image when the command completes")
> > +    parser.add_argument("--quiet", action="store_true",
> > +                        help="Run quietly unless an error occured")
> > +    args, argv = parser.parse_known_args()
> > +    return docker.Docker().run(argv, args.keep, quiet=args.quiet)
> > +
> > +if __name__ == "__main__":
> > +    sys.exit(main())
> 
> 
> --
> Alex Bennée

Re: [Qemu-devel] [PATCH v2 03/15] docker: Add images

[Fam Zheng]

On Mon, 02/29 17:18, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> 
> I think we need to include the dtc libs in here unless we need to ship
> all submodules in the tree as well.

OK, will take a look.

> 
> Is there a way to get a plain install shell? PAUSE=1 on the test shells
> doesn't seem to allow me to install anything.

What is the error when you install things? I think it is an unmodified shell
and "yum" or "apt-get" should just work. It's just that when the container
exits the changes will be discarded.

Fam

Re: [Qemu-devel] [PATCH v2 03/15] docker: Add images

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Mon, 02/29 17:18, Alex Bennée wrote:
>>
>> Fam Zheng <famz@redhat.com> writes:
>>
>> > Signed-off-by: Fam Zheng <famz@redhat.com>
>>
>> I think we need to include the dtc libs in here unless we need to ship
>> all submodules in the tree as well.
>
> OK, will take a look.
>
>>
>> Is there a way to get a plain install shell? PAUSE=1 on the test shells
>> doesn't seem to allow me to install anything.
>
> What is the error when you install things? I think it is an unmodified shell
> and "yum" or "apt-get" should just work. It's just that when the container
> exits the changes will be discarded.

So running:

make docker-test-quick@ubuntu PAUSE=1

And I got a shell I wanted to try and get working:

root@72d09615d5b2:/var/tmp/qemu.tmp# ./configure

ERROR: DTC (libfdt) version >= 1.4.0 not present. Your options:
(1) Preferred: Install the DTC (libfdt) devel package
(2) Fetch the DTC submodule, using:
git submodule update --init dtc

root@72d09615d5b2:/var/tmp/qemu.tmp# apt-cache search libfdt
libfdt-dev - Flat Device Trees manipulation library - development
files
libfdt1 - Flat Device Trees manipulation library
root@72d09615d5b2:/var/tmp/qemu.tmp# apt-get install libfdt-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libfdt1
The following NEW packages will be installed:
libfdt-dev libfdt1
0 upgraded, 2 newly installed, 0 to remove and 10 not upgraded.
Need to get 37.8 kB of archives.
After this operation, 216 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Err http://archive.ubuntu.com/ubuntu/ trusty/main libfdt1 amd64
1.4.0+dfsg-1
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com/ubuntu/ trusty/main libfdt-dev amd64
1.4.0+dfsg-1
Could not resolve 'archive.ubuntu.com'
E: Failed to fetch
http://archive.ubuntu.com/ubuntu/pool/main/d/device-tree-compiler/libfdt1_1.4.0+dfsg-1_amd64.deb
Could not resolve 'archive.ubuntu.com'

E: Failed to fetch
http://archive.ubuntu.com/ubuntu/pool/main/d/device-tree-compiler/libfdt-dev_1.4.0+dfsg-1_amd64.deb
Could not resolve 'archive.ubuntu.com'

E: Unable to fetch some archives, maybe run apt-get update or try with
--fix-missing?
root@72d09615d5b2:/var/tmp/qemu.tmp# apt-get update
Err http://archive.ubuntu.com trusty InRelease

Err http://archive.ubuntu.com trusty-updates InRelease

Err http://archive.ubuntu.com trusty-security InRelease

Err http://archive.ubuntu.com trusty Release.gpg
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com trusty-updates Release.gpg
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com trusty-security Release.gpg
Could not resolve 'archive.ubuntu.com'
Reading package lists... Done
W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease

W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty-updates/InRelease

W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty-security/InRelease

W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg  Could not
resolve 'archive.ubuntu.com'

W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg
Could not resolve 'archive.ubuntu.com'

W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg
Could not resolve 'archive.ubuntu.com'

W: Some index files failed to download. They have been ignored, or old
ones used instead.


So it looks like networking isn't working in the test images. For
debugging problems I think we need to be able to install stuff on a
running image so we can then fix the base recipes.

>
> Fam


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 02/15] Makefile: Rules for docker testing

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Mon, 02/29 17:08, Alex Bennée wrote:
>>
>> Fam Zheng <famz@redhat.com> writes:
>>
>> > This adds a group of make targets to run docker tests, all are available
>> > in source tree without running ./configure.
>> >
>> > The usage is shown by "make docker".
>> >
>> > Besides the fixed ones, dynamic targets for building each image and
>> > running each test in each image are generated automatically by make,
>> > scanning $(SRC_PATH)/tests/docker/ files with specific patterns.
>> >
>> > Alternative to manually list particular targets (docker-run-FOO@BAR)
>> > set, you can control which tests/images to run by filtering variables,
>> > TESTS= and IMAGES=, which are expressed in Makefile pattern syntax,
>> > "foo% %bar ...". For example:
>> >
>> >     $ make docker-run IMAGES="ubuntu fedora"
>>
>> This doesn't seem to work for me:
>>
>> 16:49 alex@zen/x86_64  [qemu.git/review/docker-tests-v2] >make
>> docker-run IMAGES="ubuntu" V=1
>> /home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: overriding
>> commands for target `clean-timestamp'
>> /home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: ignoring old
>> commands for target `clean-timestamp'
>> make: *** No rule to make target `docker-run'. Stop.
>
> The commit message is stale, as the help text, the target is now "docker-test"
> which is more intuitive.
>
> Fam
>
>> `
>>
>> >
>> > Unfortunately, it's impossible to propagate "-j $JOBS" into make in
>> > containers, however since each combination is made a first class target
>> > is the top Makefile, "make -j$N docker-run" still parallels the tests
>> > coarsely.
>> >
>> > Signed-off-by: Fam Zheng <famz@redhat.com>
>> > ---
>> >  Makefile                      |  4 +-
>> >  tests/docker/Makefile.include | 93 +++++++++++++++++++++++++++++++++++++++++++
>> >  2 files changed, 96 insertions(+), 1 deletion(-)
>> >  create mode 100644 tests/docker/Makefile.include
>> >
>> > diff --git a/Makefile b/Makefile
>> > index f9fae3a..f104a08 100644
>> > --- a/Makefile
>> > +++ b/Makefile
>> > @@ -6,7 +6,7 @@ BUILD_DIR=$(CURDIR)
>> >  # Before including a proper config-host.mak, assume we are in the source tree
>> >  SRC_PATH=.
>> >
>> > -UNCHECKED_GOALS := %clean TAGS cscope ctags
>> > +UNCHECKED_GOALS := %clean TAGS cscope ctags docker docker-%
>> >
>> >  # All following code might depend on configuration variables
>> >  ifneq ($(wildcard config-host.mak),)
>> > @@ -651,3 +651,5 @@ endif
>> >  # Include automatically generated dependency files
>> >  # Dependencies in Makefile.objs files come from our recursive subdir rules
>> >  -include $(wildcard *.d tests/*.d)
>> > +
>> > +include $(SRC_PATH)/tests/docker/Makefile.include
>> > diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> > new file mode 100644
>> > index 0000000..d8c55c3
>> > --- /dev/null
>> > +++ b/tests/docker/Makefile.include
>> > @@ -0,0 +1,93 @@
>> > +# Makefile for Docker tests
>> > +
>> > +$(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))
>> > +
>> > +.PHONY: docker docker-test docker-clean docker-image
>> > +
>> > +DOCKER_SUFFIX := .docker
>> > +
>> > +DOCKER_IMAGES := $(notdir $(basename \
>> > +	$(wildcard $(SRC_PATH)/tests/docker/dockerfiles/*)))
>> > +
>> > +DOCKER_TESTS := $(notdir $(shell \
>> > +	find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f -executable))
>> > +
>> > +DOCKER_TOOLS := travis
>> > +
>> > +TESTS ?= %
>> > +IMAGES ?= %
>> > +
>> > +$(foreach i,$(DOCKER_IMAGES), \
>> > +	$(eval docker-image: docker-image-$i) \
>> > +	$(eval docker-image-$i: IMAGE = $i) \
>> > +	$(eval .PHONY: docker-image-$i docker-@$i) \
>> > +	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
>> > +		$(eval .PHONY: docker-$t@$i) \
>> > +		$(eval docker-$t@$i: docker-image-$i) \
>> > +		$(eval docker-$t@$i: docker-run-$t@$i) \
>> > +		$(eval docker-@$i: docker-$t@$i) \
>> > +		$(eval docker-test: docker-run-$t@$i) \
>> > +	) \
>> > +)

Also I think you need rules for updating the images here. I just updated
ubuntu.docker to add libfdt-dev but on running on the command line:

09:53 alex@zen/x86_64  [qemu.git/review/docker-tests-v2] >make docker-image-ubuntu V=1
/home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: overriding commands for target `clean-timestamp'
/home/alex/lsrc/qemu/qemu.git/rules.mak:178: warning: ignoring old commands for target `clean-timestamp'
make: Nothing to be done for `docker-image-ubuntu'.


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 02/15] Makefile: Rules for docker testing

[Alex Bennée]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]


Fam Zheng <famz@redhat.com> writes:

> This adds a group of make targets to run docker tests, all are available
> in source tree without running ./configure.
>
> The usage is shown by "make docker".
>
<snip>

OK I've made some tweaks which I think improve the generation and allow
for re-creation of containers when the rules change. I still need an
easy way to see the failed build when it does fail. I think this
requires "docker logs" magic.

Anyway the current state of my Makefile.include attached:


[-- Attachment #2: Current hacked up makfile.include --]
[-- Type: application/octet-stream, Size: 4017 bytes --]

# Makefile for Docker tests

$(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))

.PHONY: docker docker-test docker-clean docker-image docker-qemu-src

DOCKER_SUFFIX := .docker
DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
DOCKER_IMAGES := $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*)))
DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))

$(warning DOCKER_FILES_DIR=${DOCKER_FILES_DIR})
$(warning DOCKER_IMAGES=${DOCKER_IMAGES})
$(warning DOCKER_TARGETS=${DOCKER_TARGETS})

DOCKER_TESTS := $(notdir $(shell \
	find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f -executable))

DOCKER_TOOLS := travis

TESTS ?= %
IMAGES ?= %
SRC_COPY := $(shell mktemp -u /tmp/qemu-src.XXXXX)

.PHONY: $(SRC_COPY)

$(SRC_COPY):
	@mkdir -p $@
	$(call quiet-command, \
		mkdir -p $@ && (if git diff-index --quiet HEAD -- &>/dev/null; then \
			git archive -1 HEAD --format=tar.gz; \
		else \
			git archive -1 $$(git stash create) --format=tar.gz; \
		fi) | tar -xz -C $@, \
		"  COPY SRC $@")

docker-qemu-src: $(SRC_COPY)

.PHONY: docker-image
docker-image: ${DOCKER_TARGETS}

# General rule for building docker images
docker-image-%: $(DOCKER_FILES_DIR)/%.docker
	$(call quiet-command,\
		$(SRC_PATH)/tests/docker/docker_build qemu:$* $< \
		$(if $V,-v,), "  BUILD $(IMAGE)")

# Expand all the pre-requistes for each docker image and test combination
$(foreach i,$(DOCKER_IMAGES), \
	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
		$(eval .PHONY: docker-$t@$i) \
		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
		$(eval docker-@$i: docker-$t@$i) \
		$(eval docker-test: docker-run-$t@$i) \
	) \
)

docker:
	@echo 'Building QEMU and running tests inside Docker containers'
	@echo
	@echo 'Available targets:'
	@echo
	@echo '    docker:              Print this help.'
	@echo '    docker-test:         Run all image/test combinations.'
	@echo '    docker-clean:        Kill and remove residual docker testing containers.'
	@echo '    docker-TEST@IMAGE:   Run "TEST" in container "IMAGE".'
	@echo '                         Note: "TEST" is one of the listed test name,'
	@echo '                         or a script name under $$QEMU_SRC/tests/docker/;'
	@echo '                         "IMAGE" is one of the listed container name."'
	@echo '    docker-@IMAGE:       Run all tests in container "IMAGE".'
	@echo '    docker-image:        Build all images.'
	@echo '    docker-image-IMAGE:  Build image "IMG".'
	@echo
	@echo 'Available container images:'
	@echo '    $(DOCKER_IMAGES)'
	@echo
	@echo 'Available tests:'
	@echo '    $(DOCKER_TESTS)'
	@echo
	@echo 'Available tools:'
	@echo '    $(DOCKER_TOOLS)'
	@echo
	@echo 'Special variables:'
	@echo '    IMAGES="a b c ..":   Filters which images to build or run.'
	@echo '    TESTS="x y z .."     Filters which tests to run (for docker-test).'
	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
	@echo '                         (default is 1)'
	@echo '    PAUSE=1              Stop and drop to shell in the created container'
	@echo '                         before running the command.'

docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
docker-run-%: docker-qemu-src
	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
		then echo "Invalid target"; exit 1; \
	fi
	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
		$(call quiet-command,\
			$(SRC_PATH)/tests/docker/docker_run $(if $V,,--rm) \
				--privileged -t --net=none \
				$(if $(PAUSE),-i) \
				-v $$(realpath $(SRC_PATH)):/var/tmp/qemu \
				-e QEMU_SRC=/var/tmp/qemu \
				-e V=$V -e J=$J -e PAUSE=$(PAUSE)\
				-v /var/tmp/qemu-docker-ccache:/var/tmp/ccache \
				-e CCACHE_DIR=/var/tmp/ccache \
				qemu:$(IMAGE) \
				/var/tmp/qemu/tests/docker/run \
				/var/tmp/qemu/tests/docker/$(CMD); \
			, "  RUN $(CMD) in $(IMAGE)")))

docker-clean:
	$(call quiet-command, $(SRC_PATH)/tests/docker/docker_clean)

[-- Attachment #3: Type: text/plain, Size: 18 bytes --]



--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 15/15] tests/docker/Makefile.include: snapshot the src for docker

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> From: Alex Bennée <alex.bennee@linaro.org>
>
> Instead of providing a live version of the source tree to the docker
> container we snapshot it with git-archive. This ensure the tree is in a
> pristine state for whatever operations the container is going to run on
> them.
>
> Uncommitted changes known to files known by the git index will be
> included in the snapshot if there are any.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> [Generate an unique directory under /tmp/ - Fam Zheng]

But the docker-run-% rule never uses the source copy we just made.

> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/docker/Makefile.include | 22 ++++++++++++++++++----
>  1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index aaf7d23..7cc2130 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -2,7 +2,7 @@
>
>  $(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak))
>
> -.PHONY: docker docker-test docker-clean docker-image
> +.PHONY: docker docker-test docker-clean docker-image docker-qemu-src
>
>  DOCKER_SUFFIX := .docker
>
> @@ -16,6 +16,21 @@ DOCKER_TOOLS := travis
>
>  TESTS ?= %
>  IMAGES ?= %
> +SRC_COPY := $(shell mktemp -u /tmp/qemu-src.XXXXX)
> +
> +.PHONY: $(SRC_COPY)
> +
> +$(SRC_COPY):
> +	@mkdir -p $@
> +	$(call quiet-command, \
> +		mkdir -p $@ && (if git diff-index --quiet HEAD -- &>/dev/null; then \
> +			git archive -1 HEAD --format=tar.gz; \
> +		else \
> +			git archive -1 $$(git stash create) --format=tar.gz; \
> +		fi) | tar -xz -C $@, \
> +		"  COPY SRC $@")
> +
> +docker-qemu-src: $(SRC_COPY)
>
>  $(foreach i,$(DOCKER_IMAGES), \
>  	$(eval docker-image: docker-image-$i) \
> @@ -23,8 +38,7 @@ $(foreach i,$(DOCKER_IMAGES), \
>  	$(eval .PHONY: docker-image-$i docker-@$i) \
>  	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
>  		$(eval .PHONY: docker-$t@$i) \
> -		$(eval docker-$t@$i: docker-image-$i) \
> -		$(eval docker-$t@$i: docker-run-$t@$i) \
> +		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
>  		$(eval docker-@$i: docker-$t@$i) \
>  		$(eval docker-test: docker-run-$t@$i) \
>  	) \
> @@ -73,7 +87,7 @@ docker-image-%:
>
>  docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
>  docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> -docker-run-%:
> +docker-run-%: docker-qemu-src
>  	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
>  		then echo "Invalid target"; exit 1; \
>  	fi


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 03/15] docker: Add images

[Fam Zheng]

On Tue, 03/01 09:52, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > On Mon, 02/29 17:18, Alex Bennée wrote:
> >>
> >> Fam Zheng <famz@redhat.com> writes:
> >>
> >> > Signed-off-by: Fam Zheng <famz@redhat.com>
> >>
> >> I think we need to include the dtc libs in here unless we need to ship
> >> all submodules in the tree as well.
> >
> > OK, will take a look.
> >
> >>
> >> Is there a way to get a plain install shell? PAUSE=1 on the test shells
> >> doesn't seem to allow me to install anything.
> >
> > What is the error when you install things? I think it is an unmodified shell
> > and "yum" or "apt-get" should just work. It's just that when the container
> > exits the changes will be discarded.
> 
> So running:
> 
> make docker-test-quick@ubuntu PAUSE=1
> 
> And I got a shell I wanted to try and get working:
> 
> root@72d09615d5b2:/var/tmp/qemu.tmp# ./configure
> 
> ERROR: DTC (libfdt) version >= 1.4.0 not present. Your options:
> (1) Preferred: Install the DTC (libfdt) devel package
> (2) Fetch the DTC submodule, using:
> git submodule update --init dtc
> 
> root@72d09615d5b2:/var/tmp/qemu.tmp# apt-cache search libfdt
> libfdt-dev - Flat Device Trees manipulation library - development
> files
> libfdt1 - Flat Device Trees manipulation library
> root@72d09615d5b2:/var/tmp/qemu.tmp# apt-get install libfdt-dev
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> The following extra packages will be installed:
> libfdt1
> The following NEW packages will be installed:
> libfdt-dev libfdt1
> 0 upgraded, 2 newly installed, 0 to remove and 10 not upgraded.
> Need to get 37.8 kB of archives.
> After this operation, 216 kB of additional disk space will be used.
> Do you want to continue? [Y/n]
> Err http://archive.ubuntu.com/ubuntu/ trusty/main libfdt1 amd64
> 1.4.0+dfsg-1
> Could not resolve 'archive.ubuntu.com'
> Err http://archive.ubuntu.com/ubuntu/ trusty/main libfdt-dev amd64
> 1.4.0+dfsg-1
> Could not resolve 'archive.ubuntu.com'
> E: Failed to fetch
> http://archive.ubuntu.com/ubuntu/pool/main/d/device-tree-compiler/libfdt1_1.4.0+dfsg-1_amd64.deb
> Could not resolve 'archive.ubuntu.com'
> 
> E: Failed to fetch
> http://archive.ubuntu.com/ubuntu/pool/main/d/device-tree-compiler/libfdt-dev_1.4.0+dfsg-1_amd64.deb
> Could not resolve 'archive.ubuntu.com'
> 
> E: Unable to fetch some archives, maybe run apt-get update or try with
> --fix-missing?
> root@72d09615d5b2:/var/tmp/qemu.tmp# apt-get update
> Err http://archive.ubuntu.com trusty InRelease
> 
> Err http://archive.ubuntu.com trusty-updates InRelease
> 
> Err http://archive.ubuntu.com trusty-security InRelease
> 
> Err http://archive.ubuntu.com trusty Release.gpg
> Could not resolve 'archive.ubuntu.com'
> Err http://archive.ubuntu.com trusty-updates Release.gpg
> Could not resolve 'archive.ubuntu.com'
> Err http://archive.ubuntu.com trusty-security Release.gpg
> Could not resolve 'archive.ubuntu.com'
> Reading package lists... Done
> W: Failed to fetch
> http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease
> 
> W: Failed to fetch
> http://archive.ubuntu.com/ubuntu/dists/trusty-updates/InRelease
> 
> W: Failed to fetch
> http://archive.ubuntu.com/ubuntu/dists/trusty-security/InRelease
> 
> W: Failed to fetch
> http://archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg  Could not
> resolve 'archive.ubuntu.com'
> 
> W: Failed to fetch
> http://archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg
> Could not resolve 'archive.ubuntu.com'
> 
> W: Failed to fetch
> http://archive.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg
> Could not resolve 'archive.ubuntu.com'
> 
> W: Some index files failed to download. They have been ignored, or old
> ones used instead.
> 
> 
> So it looks like networking isn't working in the test images. For
> debugging problems I think we need to be able to install stuff on a
> running image so we can then fix the base recipes.

Oh yes, I added "-net=none" to the "docker run" command for better isolation.
So in this case we should enable it. Let's rename PAUSE=1 to DEBUG=1 and
enable network.

Fam

Re: [Qemu-devel] [PATCH v2 15/15] tests/docker/Makefile.include: snapshot the src for docker

[Fam Zheng]

On Tue, 03/01 20:16, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > From: Alex Bennée <alex.bennee@linaro.org>
> >
> > Instead of providing a live version of the source tree to the docker
> > container we snapshot it with git-archive. This ensure the tree is in a
> > pristine state for whatever operations the container is going to run on
> > them.
> >
> > Uncommitted changes known to files known by the git index will be
> > included in the snapshot if there are any.
> >
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > [Generate an unique directory under /tmp/ - Fam Zheng]
> 
> But the docker-run-% rule never uses the source copy we just made.

I failed in Makefile magic: pattern rules and dynamically generated rules
mustn't be used mixed, otherwise the former will be ignored. Will fix.

Fam

Re: [Qemu-devel] [PATCH v2 03/15] docker: Add images

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Tue, 03/01 09:52, Alex Bennée wrote:
>>
>> Fam Zheng <famz@redhat.com> writes:
>>
>> > On Mon, 02/29 17:18, Alex Bennée wrote:
>> >>
>> >> Fam Zheng <famz@redhat.com> writes:
>> >>
>> >> > Signed-off-by: Fam Zheng <famz@redhat.com>
>> >>
>> >> I think we need to include the dtc libs in here unless we need to ship
>> >> all submodules in the tree as well.
>> >
>> > OK, will take a look.
>> >
>> >>
>> >> Is there a way to get a plain install shell? PAUSE=1 on the test shells
>> >> doesn't seem to allow me to install anything.
>> >
>> > What is the error when you install things? I think it is an unmodified shell
>> > and "yum" or "apt-get" should just work. It's just that when the container
>> > exits the changes will be discarded.
>>
>> So running:
>>
>> make docker-test-quick@ubuntu PAUSE=1
>>
>> And I got a shell I wanted to try and get working:
>>
>> root@72d09615d5b2:/var/tmp/qemu.tmp# ./configure
>>
>> ERROR: DTC (libfdt) version >= 1.4.0 not present. Your options:
>> (1) Preferred: Install the DTC (libfdt) devel package
>> (2) Fetch the DTC submodule, using:
>> git submodule update --init dtc
>>
>> root@72d09615d5b2:/var/tmp/qemu.tmp# apt-cache search libfdt
>> libfdt-dev - Flat Device Trees manipulation library - development
>> files
>> libfdt1 - Flat Device Trees manipulation library
>> root@72d09615d5b2:/var/tmp/qemu.tmp# apt-get install libfdt-dev
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> The following extra packages will be installed:
>> libfdt1
>> The following NEW packages will be installed:
>> libfdt-dev libfdt1
>> 0 upgraded, 2 newly installed, 0 to remove and 10 not upgraded.
>> Need to get 37.8 kB of archives.
>> After this operation, 216 kB of additional disk space will be used.
>> Do you want to continue? [Y/n]
>> Err http://archive.ubuntu.com/ubuntu/ trusty/main libfdt1 amd64
>> 1.4.0+dfsg-1
>> Could not resolve 'archive.ubuntu.com'
>> Err http://archive.ubuntu.com/ubuntu/ trusty/main libfdt-dev amd64
>> 1.4.0+dfsg-1
>> Could not resolve 'archive.ubuntu.com'
>> E: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/pool/main/d/device-tree-compiler/libfdt1_1.4.0+dfsg-1_amd64.deb
>> Could not resolve 'archive.ubuntu.com'
>>
>> E: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/pool/main/d/device-tree-compiler/libfdt-dev_1.4.0+dfsg-1_amd64.deb
>> Could not resolve 'archive.ubuntu.com'
>>
>> E: Unable to fetch some archives, maybe run apt-get update or try with
>> --fix-missing?
>> root@72d09615d5b2:/var/tmp/qemu.tmp# apt-get update
>> Err http://archive.ubuntu.com trusty InRelease
>>
>> Err http://archive.ubuntu.com trusty-updates InRelease
>>
>> Err http://archive.ubuntu.com trusty-security InRelease
>>
>> Err http://archive.ubuntu.com trusty Release.gpg
>> Could not resolve 'archive.ubuntu.com'
>> Err http://archive.ubuntu.com trusty-updates Release.gpg
>> Could not resolve 'archive.ubuntu.com'
>> Err http://archive.ubuntu.com trusty-security Release.gpg
>> Could not resolve 'archive.ubuntu.com'
>> Reading package lists... Done
>> W: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease
>>
>> W: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/dists/trusty-updates/InRelease
>>
>> W: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/dists/trusty-security/InRelease
>>
>> W: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg  Could not
>> resolve 'archive.ubuntu.com'
>>
>> W: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg
>> Could not resolve 'archive.ubuntu.com'
>>
>> W: Failed to fetch
>> http://archive.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg
>> Could not resolve 'archive.ubuntu.com'
>>
>> W: Some index files failed to download. They have been ignored, or old
>> ones used instead.
>>
>>
>> So it looks like networking isn't working in the test images. For
>> debugging problems I think we need to be able to install stuff on a
>> running image so we can then fix the base recipes.
>
> Oh yes, I added "-net=none" to the "docker run" command for better isolation.
> So in this case we should enable it. Let's rename PAUSE=1 to DEBUG=1 and
> enable network.

Sounds good.

>
> Fam


--
Alex Bennée