[Qemu-devel] [RFC PATCH 0/4] tests: Introducing docker tests

[Qemu-devel] [RFC PATCH 0/4] tests: Introducing docker tests

[Fam Zheng]

This series adds a new "docker" make target family to run tests in created
docker containers.

To begin with, this can be a place to store standard env/command combinations to
build and test QEMU.

Secondly, CI usually provides "docker" capability (such as travis [1]), where
we define standard/repeatable test environments, and run tests in them.
However, what tests to cover is better maintained in-tree, in order to keep in
sync with the code development.

Lastly, this makes it very simple for developers to replicate such tests
themselves.

[1]: https://docs.travis-ci.com/user/docker/


Fam Zheng (4):
  tests: Introduce Docker based tests
  tests: Add clang docker test
  tests: Add mingw 32/64 cross compiling
  tests: Add travis container test case

 Makefile                     | 33 ++++++++++++++++++++++++++++++++-
 tests/docker/basic.sh        | 11 +++++++++++
 tests/docker/centos6.docker  |  5 +++++
 tests/docker/clang.sh        | 16 ++++++++++++++++
 tests/docker/fedora22.docker |  6 ++++++
 tests/docker/mingw.sh        | 14 ++++++++++++++
 tests/docker/run             | 20 ++++++++++++++++++++
 tests/docker/travis.py       | 40 ++++++++++++++++++++++++++++++++++++++++
 tests/docker/travis.sh       | 10 ++++++++++
 tests/docker/ubuntu.docker   |  9 +++++++++
 10 files changed, 163 insertions(+), 1 deletion(-)
 create mode 100755 tests/docker/basic.sh
 create mode 100644 tests/docker/centos6.docker
 create mode 100755 tests/docker/clang.sh
 create mode 100644 tests/docker/fedora22.docker
 create mode 100755 tests/docker/mingw.sh
 create mode 100755 tests/docker/run
 create mode 100755 tests/docker/travis.py
 create mode 100755 tests/docker/travis.sh
 create mode 100644 tests/docker/ubuntu.docker

-- 
2.4.3

[Qemu-devel] [RFC PATCH 1/4] tests: Introduce Docker based tests

[Fam Zheng]

A new group of targets to spawn docker instances for testing is added:

- "make docker" will spawn each available docker image and run all
  available tests under tests/docker.

- "make docker-foo" will spawn image foo and run all available tests.

- "make docker X=bar" or "make docker-foo X=bar" will limit the tests to
  only matching pattern "bar".

If the docker images (tagged like qemu:foo) don't exist, the make
command will automatically build them with foo.docker.

MAKEFLAGS is passed into the container.

Breaking down the tests/docker directory:

- *.docker in Dockerfile format define the environment in which tests
  are run.

- *.sh are testing scripts to be executed by "run".

- "run" is the entry script to run inside the container.

- Anything else are indirectly used or documents.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile                     | 33 ++++++++++++++++++++++++++++++++-
 tests/docker/basic.sh        | 11 +++++++++++
 tests/docker/centos6.docker  |  5 +++++
 tests/docker/fedora22.docker |  6 ++++++
 tests/docker/run             | 20 ++++++++++++++++++++
 tests/docker/ubuntu.docker   |  9 +++++++++
 6 files changed, 83 insertions(+), 1 deletion(-)
 create mode 100755 tests/docker/basic.sh
 create mode 100644 tests/docker/centos6.docker
 create mode 100644 tests/docker/fedora22.docker
 create mode 100755 tests/docker/run
 create mode 100644 tests/docker/ubuntu.docker

diff --git a/Makefile b/Makefile
index d0de2d4..91be64f 100644
--- a/Makefile
+++ b/Makefile
@@ -6,7 +6,7 @@ BUILD_DIR=$(CURDIR)
 # Before including a proper config-host.mak, assume we are in the source tree
 SRC_PATH=.
 
-UNCHECKED_GOALS := %clean TAGS cscope ctags
+UNCHECKED_GOALS := %clean TAGS cscope ctags docker docker-%
 
 # All following code might depend on configuration variables
 ifneq ($(wildcard config-host.mak),)
@@ -651,3 +651,34 @@ endif
 # Include automatically generated dependency files
 # Dependencies in Makefile.objs files come from our recursive subdir rules
 -include $(wildcard *.d tests/*.d)
+
+# Detect the working docker command
+DOCKER = $(shell for cmd in "docker" "sudo -n docker"; do \
+                    if $$cmd images &>/dev/null; then \
+                        echo $$cmd; \
+                    fi; done)
+
+docker: $(addprefix docker-,\
+            centos6 \
+            fedora22 \
+            ubuntu \
+        )
+docker-%: T=$(@:docker-%=%)
+docker-%:
+	@if test -z "$(DOCKER)"; then \
+		echo 'Docker not found.' \
+			'Tried "docker" and "sudo -n docker"'; \
+	fi
+	@if ! $(DOCKER) inspect qemu:$T &>/dev/null; then \
+		$(DOCKER) build -t qemu:$T -f $(SRC_PATH)/tests/docker/$T.docker \
+			$(SRC_PATH)/tests/docker; \
+	fi
+	$(DOCKER) run --privileged -t -i --rm --net=none \
+		-e IMAGE_TAG=$T \
+		-e MAKEFLAGS="$(MAKEFLAGS)" \
+		-e X="$X" \
+		-v $$(realpath $(SRC_PATH)):/var/tmp/qemu \
+		-e QEMU_SRC=/var/tmp/qemu \
+		-v /var/tmp/qemu-docker-ccache:/var/tmp/ccache \
+		-e CCACHE_DIR=/var/tmp/ccache \
+		qemu:$T sh - /var/tmp/qemu/tests/docker/run
diff --git a/tests/docker/basic.sh b/tests/docker/basic.sh
new file mode 100755
index 0000000..e51b718
--- /dev/null
+++ b/tests/docker/basic.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+cd $(mktemp -d)
+mkdir build
+mkdir install
+cd build
+$QEMU_SRC/configure --target-list=x86_64-softmmu
+    --prefix="${pwd}/install"
+make $MAKEFLAGS
+make check $MAKEFLAGS
+make install
diff --git a/tests/docker/centos6.docker b/tests/docker/centos6.docker
new file mode 100644
index 0000000..802ca79
--- /dev/null
+++ b/tests/docker/centos6.docker
@@ -0,0 +1,5 @@
+FROM centos:6
+RUN yum install -y \
+    git make gcc \
+    zlib-devel glib2-devel SDL-devel pixman-devel
+
diff --git a/tests/docker/fedora22.docker b/tests/docker/fedora22.docker
new file mode 100644
index 0000000..5fd48d6
--- /dev/null
+++ b/tests/docker/fedora22.docker
@@ -0,0 +1,6 @@
+FROM fedora:22
+RUN dnf install -y \
+    ccache git mingw{32,64}-{pixman,glib2,gmp,SDL,pkg-config} \
+    glib2-devel pixman-devel zlib-devel SDL-devel \
+    gcc g++ clang make perl which bc findutils
+
diff --git a/tests/docker/run b/tests/docker/run
new file mode 100755
index 0000000..8267766
--- /dev/null
+++ b/tests/docker/run
@@ -0,0 +1,20 @@
+#!/bin/sh
+
+set -e
+# Prepare the environment
+. /etc/profile || true
+export PATH=/usr/lib/ccache:$PATH
+
+save=$QEMU_SRC
+QEMU_SRC=/var/tmp/qemu.tmp
+cp -r $save $QEMU_SRC
+
+filter_test()
+{
+    grep $(for p in ${X:-.}; do echo " -e $p"; done)
+}
+
+for t in $(cd $QEMU_SRC/tests/docker/; ls *.sh | filter_test); do
+    echo "Running $t ..."
+    $QEMU_SRC/tests/docker/$t || break
+done
diff --git a/tests/docker/ubuntu.docker b/tests/docker/ubuntu.docker
new file mode 100644
index 0000000..004201d
--- /dev/null
+++ b/tests/docker/ubuntu.docker
@@ -0,0 +1,9 @@
+FROM ubuntu:14.04
+RUN apt-get update
+RUN apt-get -y install \
+    libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev \
+    libseccomp-dev libgnutls-dev libssh2-1-dev  libspice-server-dev \
+    libspice-protocol-dev libnss3-dev \
+    libgtk-3-dev libvte-2.90-dev libsdl1.2-dev libpng12-dev libpixman-1-dev \
+    git make ccache python-yaml gcc clang
+
-- 
2.4.3

Re: [Qemu-devel] [RFC PATCH 1/4] tests: Introduce Docker based tests

[Daniel P. Berrange]

On Wed, Feb 03, 2016 at 10:36:01PM +0800, Fam Zheng wrote:

> diff --git a/tests/docker/fedora22.docker b/tests/docker/fedora22.docker
> new file mode 100644
> index 0000000..5fd48d6
> --- /dev/null
> +++ b/tests/docker/fedora22.docker

Fedora 22 will go end of life in June. I think we should at least
start with the current Fedora 23 since that's the current preferred
Fedora release.

> @@ -0,0 +1,6 @@
> +FROM fedora:22
> +RUN dnf install -y \
> +    ccache git mingw{32,64}-{pixman,glib2,gmp,SDL,pkg-config} \
> +    glib2-devel pixman-devel zlib-devel SDL-devel \
> +    gcc g++ clang make perl which bc findutils

That's a pretty minimal set of mingw packages - many features will
end up disabled with that. Add mingw{32,64} packages for gtk2, gtk3,
gnutls, nettle, libtasn1, libjpeg-turbo, libpng, curl, libssh2,
bzip2 too, which is everything Fedora mingw is capable of providing
for QEMU


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [RFC PATCH 1/4] tests: Introduce Docker based tests

[Fam Zheng]

On Thu, 02/04 11:58, Daniel P. Berrange wrote:
> On Wed, Feb 03, 2016 at 10:36:01PM +0800, Fam Zheng wrote:
> 
> > diff --git a/tests/docker/fedora22.docker b/tests/docker/fedora22.docker
> > new file mode 100644
> > index 0000000..5fd48d6
> > --- /dev/null
> > +++ b/tests/docker/fedora22.docker
> 
> Fedora 22 will go end of life in June. I think we should at least
> start with the current Fedora 23 since that's the current preferred
> Fedora release.

Yes, will switch to 23.

> 
> > @@ -0,0 +1,6 @@
> > +FROM fedora:22
> > +RUN dnf install -y \
> > +    ccache git mingw{32,64}-{pixman,glib2,gmp,SDL,pkg-config} \
> > +    glib2-devel pixman-devel zlib-devel SDL-devel \
> > +    gcc g++ clang make perl which bc findutils
> 
> That's a pretty minimal set of mingw packages - many features will
> end up disabled with that. Add mingw{32,64} packages for gtk2, gtk3,
> gnutls, nettle, libtasn1, libjpeg-turbo, libpng, curl, libssh2,
> bzip2 too, which is everything Fedora mingw is capable of providing
> for QEMU

Okay!

Fam

[Qemu-devel] [RFC PATCH 2/4] tests: Add clang docker test

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/clang.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100755 tests/docker/clang.sh

diff --git a/tests/docker/clang.sh b/tests/docker/clang.sh
new file mode 100755
index 0000000..85fe0a5
--- /dev/null
+++ b/tests/docker/clang.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+if ! type clang clang++ &>/dev/null; then
+    echo "clang not avialable, skip"
+    exit 0
+fi
+
+
+cd $(mktemp -d)
+$QEMU_SRC/configure \
+    --cc=/usr/bin/clang --cxx=/usr/bin/clang++ --host-cc=/usr/bin/clang \
+    --extra-cflags=-fsanitize=undefined \
+    --target-list=x86_64-softmmu,aarch64-softmmu
+
+make $MAKEFLAGS
+make $MAKEFLAGS check
-- 
2.4.3

[Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Fam Zheng]

Only fedora22 has the required toolchain so it's not run elsewhere.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/mingw.sh | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100755 tests/docker/mingw.sh

diff --git a/tests/docker/mingw.sh b/tests/docker/mingw.sh
new file mode 100755
index 0000000..0f103cd
--- /dev/null
+++ b/tests/docker/mingw.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+if [ "$IMAGE_TAG" != "fedora22" ]; then
+    echo "Mingw test skipped"
+    exit 0
+fi
+
+cd $(mktemp -d)
+for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
+    $QEMU_SRC/configuire --cross-prefix=$prefix \
+        --target-list=x86_64-softmmu,aarch64-softmmu
+    make $MAKEFLAGS
+done
+
-- 
2.4.3

Re: [Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Stefan Weil]

Am 03.02.2016 um 15:36 schrieb Fam Zheng:
> Only fedora22 has the required toolchain so it's not run elsewhere.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/docker/mingw.sh | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>  create mode 100755 tests/docker/mingw.sh
> 
> diff --git a/tests/docker/mingw.sh b/tests/docker/mingw.sh
> new file mode 100755
> index 0000000..0f103cd
> --- /dev/null
> +++ b/tests/docker/mingw.sh
> @@ -0,0 +1,14 @@
> +#!/bin/bash
> +
> +if [ "$IMAGE_TAG" != "fedora22" ]; then
> +    echo "Mingw test skipped"
> +    exit 0
> +fi
> +
> +cd $(mktemp -d)
> +for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
> +    $QEMU_SRC/configuire --cross-prefix=$prefix \

configure?

> +        --target-list=x86_64-softmmu,aarch64-softmmu
> +    make $MAKEFLAGS
> +done
> +

I suggest to add a trace backend as well (my tests run with
--enable-trace-backend=stderr, so maybe you want to choose
a different one).

Another useful option for configure is --enable-debug.
The compilation will be much faster and detect nearly
the same kind of problems as the standard compilation
(use of uninitialized variables won't be detected).

Regards,
Stefan

Re: [Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Fam Zheng]

On Wed, 02/03 16:08, Stefan Weil wrote:
> Am 03.02.2016 um 15:36 schrieb Fam Zheng:
> > Only fedora22 has the required toolchain so it's not run elsewhere.
> > 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  tests/docker/mingw.sh | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> >  create mode 100755 tests/docker/mingw.sh
> > 
> > diff --git a/tests/docker/mingw.sh b/tests/docker/mingw.sh
> > new file mode 100755
> > index 0000000..0f103cd
> > --- /dev/null
> > +++ b/tests/docker/mingw.sh
> > @@ -0,0 +1,14 @@
> > +#!/bin/bash
> > +
> > +if [ "$IMAGE_TAG" != "fedora22" ]; then
> > +    echo "Mingw test skipped"
> > +    exit 0
> > +fi
> > +
> > +cd $(mktemp -d)
> > +for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
> > +    $QEMU_SRC/configuire --cross-prefix=$prefix \
> 
> configure?

Yes, how could I not notice the type? @.@

> 
> > +        --target-list=x86_64-softmmu,aarch64-softmmu
> > +    make $MAKEFLAGS
> > +done
> > +
> 
> I suggest to add a trace backend as well (my tests run with
> --enable-trace-backend=stderr, so maybe you want to choose
> a different one).

Will do.

> 
> Another useful option for configure is --enable-debug.
> The compilation will be much faster and detect nearly
> the same kind of problems as the standard compilation
> (use of uninitialized variables won't be detected).

OK.

Thanks for taking a look!

Fam

> 
> Regards,
> Stefan
> 

Re: [Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Paolo Bonzini]

On 04/02/2016 03:29, Fam Zheng wrote:
>> > I suggest to add a trace backend as well (my tests run with
>> > --enable-trace-backend=stderr, so maybe you want to choose
>> > a different one).
> Will do.
> 

Note that the stderr (now renamed to "log") is now the default.

Paolo

Re: [Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

On 02/03/2016 07:36 AM, Fam Zheng wrote:
> Only fedora22 has the required toolchain so it's not run elsewhere.

Not fedora 23 or rawhide? In another year, when F22 is obsolete, this
will be dead code unless you meant for it to be >= F22.

> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/docker/mingw.sh | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>  create mode 100755 tests/docker/mingw.sh
> 
> diff --git a/tests/docker/mingw.sh b/tests/docker/mingw.sh
> new file mode 100755
> index 0000000..0f103cd
> --- /dev/null
> +++ b/tests/docker/mingw.sh
> @@ -0,0 +1,14 @@
> +#!/bin/bash
> +
> +if [ "$IMAGE_TAG" != "fedora22" ]; then
> +    echo "Mingw test skipped"
> +    exit 0
> +fi
> +
> +cd $(mktemp -d)
> +for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
> +    $QEMU_SRC/configuire --cross-prefix=$prefix \
> +        --target-list=x86_64-softmmu,aarch64-softmmu
> +    make $MAKEFLAGS
> +done
> +
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Fam Zheng]

On Wed, 02/03 09:14, Eric Blake wrote:
> On 02/03/2016 07:36 AM, Fam Zheng wrote:
> > Only fedora22 has the required toolchain so it's not run elsewhere.
> 
> Not fedora 23 or rawhide? In another year, when F22 is obsolete, this
> will be dead code unless you meant for it to be >= F22.

Good point, will address that. Thanks!

Fam

> 
> > 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > ---
> >  tests/docker/mingw.sh | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> >  create mode 100755 tests/docker/mingw.sh
> > 
> > diff --git a/tests/docker/mingw.sh b/tests/docker/mingw.sh
> > new file mode 100755
> > index 0000000..0f103cd
> > --- /dev/null
> > +++ b/tests/docker/mingw.sh
> > @@ -0,0 +1,14 @@
> > +#!/bin/bash
> > +
> > +if [ "$IMAGE_TAG" != "fedora22" ]; then
> > +    echo "Mingw test skipped"
> > +    exit 0
> > +fi
> > +
> > +cd $(mktemp -d)
> > +for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
> > +    $QEMU_SRC/configuire --cross-prefix=$prefix \
> > +        --target-list=x86_64-softmmu,aarch64-softmmu
> > +    make $MAKEFLAGS
> > +done
> > +
> > 
> 
> -- 
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

Re: [Qemu-devel] [RFC PATCH 3/4] tests: Add mingw 32/64 cross compiling

[Daniel P. Berrange]

On Wed, Feb 03, 2016 at 10:36:03PM +0800, Fam Zheng wrote:
> Only fedora22 has the required toolchain so it's not run elsewhere.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  tests/docker/mingw.sh | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>  create mode 100755 tests/docker/mingw.sh
> 
> diff --git a/tests/docker/mingw.sh b/tests/docker/mingw.sh
> new file mode 100755
> index 0000000..0f103cd
> --- /dev/null
> +++ b/tests/docker/mingw.sh
> @@ -0,0 +1,14 @@
> +#!/bin/bash
> +
> +if [ "$IMAGE_TAG" != "fedora22" ]; then
> +    echo "Mingw test skipped"
> +    exit 0
> +fi
> +
> +cd $(mktemp -d)
> +for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
> +    $QEMU_SRC/configuire --cross-prefix=$prefix \
> +        --target-list=x86_64-softmmu,aarch64-softmmu
> +    make $MAKEFLAGS
> +done

Automated build tests should really pass explicit '--enable-xxx'
flags for every feature we expect to be present. This means that
if someone breaks feature detection in QEMU, we are more likely
to see a build failure, rather than the feature being silently
disabled for months on end.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

[Qemu-devel] [RFC PATCH 4/4] tests: Add travis container test case

[Fam Zheng]

.travis.yml is parsed by an auxiliary python script, to generate the
command list out of the matrix.

TODO: respect "compiler".

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/travis.py | 40 ++++++++++++++++++++++++++++++++++++++++
 tests/docker/travis.sh | 10 ++++++++++
 2 files changed, 50 insertions(+)
 create mode 100755 tests/docker/travis.py
 create mode 100755 tests/docker/travis.sh

diff --git a/tests/docker/travis.py b/tests/docker/travis.py
new file mode 100755
index 0000000..7fd5e4b
--- /dev/null
+++ b/tests/docker/travis.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python2
+import sys
+import yaml
+import json
+import itertools
+
+def load_yaml(fname):
+    return yaml.load(open(sys.argv[1], "r").read())
+
+def conf_iter(conf):
+    def env_list(e):
+        return e if type(e) is list else [e]
+    global_env = conf["env"]["global"]
+    for c in conf["matrix"]["include"]:
+        a = { "env": global_env + env_list(c["env"]),
+              "compiler": c["compiler"]
+            }
+        yield a
+    for c in itertools.product(conf["compiler"],
+                               conf["env"]["matrix"]):
+        a = { "env": global_env + env_list(c[1]),
+              "compiler": c[0]
+            }
+        yield a
+
+def main():
+    if len(sys.argv) < 2:
+        sys.stderr.write("Usage: %s <travis-file>\n" % sys.argv[0])
+        return 1
+    conf = load_yaml(sys.argv[1])
+    for config in conf_iter(conf):
+        print "("
+        print "\n".join(config["env"])
+        print "\n".join(conf["before_script"])
+        print "\n".join(conf["script"])
+        print ")"
+    return 0
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/tests/docker/travis.sh b/tests/docker/travis.sh
new file mode 100755
index 0000000..e7e1f20
--- /dev/null
+++ b/tests/docker/travis.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+if ! python -c 'import yaml' &>/dev/null; then
+    echo "PyYAML not found, skip"
+    exit 0
+fi
+
+cd $(mktemp -d)
+$QEMU_SRC/tests/docker/travis.py $QEMU_SRC/.travis.yml > travis_command_list
+. travis_command_list
-- 
2.4.3

Re: [Qemu-devel] [RFC PATCH 0/4] tests: Introducing docker tests

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

On Wed, Feb 03, 2016 at 10:36:00PM +0800, Fam Zheng wrote:
> This series adds a new "docker" make target family to run tests in created
> docker containers.
> 
> To begin with, this can be a place to store standard env/command combinations to
> build and test QEMU.
> 
> Secondly, CI usually provides "docker" capability (such as travis [1]), where
> we define standard/repeatable test environments, and run tests in them.
> However, what tests to cover is better maintained in-tree, in order to keep in
> sync with the code development.
> 
> Lastly, this makes it very simple for developers to replicate such tests
> themselves.
> 
> [1]: https://docs.travis-ci.com/user/docker/
> 
> 
> Fam Zheng (4):
>   tests: Introduce Docker based tests
>   tests: Add clang docker test
>   tests: Add mingw 32/64 cross compiling
>   tests: Add travis container test case

Nice, having standard build/test environments will make it easier to
reproduce issues.

I sent several pull requests to Peter Maydell last year that ended up
failing in his build environments.  Although some of those issues were
non-Linux and therefore not captured by your Docker environments,
capturing common Linux environments is a step forward.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [RFC PATCH 0/4] tests: Introducing docker tests

[Fam Zheng]

On Wed, 02/03 15:24, Stefan Hajnoczi wrote:
> On Wed, Feb 03, 2016 at 10:36:00PM +0800, Fam Zheng wrote:
> > This series adds a new "docker" make target family to run tests in created
> > docker containers.
> > 
> > To begin with, this can be a place to store standard env/command combinations to
> > build and test QEMU.
> > 
> > Secondly, CI usually provides "docker" capability (such as travis [1]), where
> > we define standard/repeatable test environments, and run tests in them.
> > However, what tests to cover is better maintained in-tree, in order to keep in
> > sync with the code development.
> > 
> > Lastly, this makes it very simple for developers to replicate such tests
> > themselves.
> > 
> > [1]: https://docs.travis-ci.com/user/docker/
> > 
> > 
> > Fam Zheng (4):
> >   tests: Introduce Docker based tests
> >   tests: Add clang docker test
> >   tests: Add mingw 32/64 cross compiling
> >   tests: Add travis container test case
> 
> Nice, having standard build/test environments will make it easier to
> reproduce issues.
> 
> I sent several pull requests to Peter Maydell last year that ended up
> failing in his build environments.  Although some of those issues were
> non-Linux and therefore not captured by your Docker environments,
> capturing common Linux environments is a step forward.

Non-Linux environments can be sorted out in this framework by pulling
appropriate 3rd party images from docker hub, assuming there isn't too much
copyright hassle for Mac OSX and Windows stuff. This is theoretically possible
because with "--privileged -v /dev/kvm:/dev/kvm": we can spawn a kvm guest
inside the container.  Even without kvm, TCG may also be useful for non-x86
build env.

This series also covers mingw cross compiling which might catch some windows
problems. I'm not sure how different is it from natively building QEMU on
Windows with MinGW. Stefan Weil?

Fam

Re: [Qemu-devel] [RFC PATCH 0/4] tests: Introducing docker tests

[Paolo Bonzini]

On 04/02/2016 03:43, Fam Zheng wrote:
> Non-Linux environments can be sorted out in this framework by pulling
> appropriate 3rd party images from docker hub, assuming there isn't too much
> copyright hassle for Mac OSX and Windows stuff. This is theoretically possible
> because with "--privileged -v /dev/kvm:/dev/kvm": we can spawn a kvm guest
> inside the container.  Even without kvm, TCG may also be useful for non-x86
> build env.
> 
> This series also covers mingw cross compiling which might catch some windows
> problems.

And especially 32-bit problems.  Failing on mingw but not arm32 is rare
overall.

Paolo