[Qemu-devel] [PULL v2 0/4] Docker patches

[Qemu-devel] [PULL v2 0/4] Docker patches

[Fam Zheng]

The following changes since commit 6aa5a3679449cdf0b6fe5a6829b22e642ded57fd:

  Merge remote-tracking branch 'remotes/kraxel/tags/pull-ui-20161013-1' into staging (2016-10-13 14:27:58 +0100)

are available in the git repository at:

  git@github.com:famz/qemu tags/for-upstream

for you to fetch changes up to e86c9a64f455018fb04d631e14c5f926e36c69fb:

  tests/docker/Makefile.include: add a generic docker-run target (2016-10-17 10:05:48 +0800)

----------------------------------------------------------------

v2: Fix $(MAKE) in patch 4. [Paolo]

----------------------------------------------------------------

Alex Bennée (4):
  tests/docker: add travis dockerfile
  tests/docker: test-build script
  tests/docker: make test-mingw honour TARGET_LIST
  tests/docker/Makefile.include: add a generic docker-run target

 tests/docker/Makefile.include          | 61 +++++++++++++++++++++-------------
 tests/docker/dockerfiles/travis.docker |  6 ++++
 tests/docker/test-build                | 20 +++++++++++
 tests/docker/test-mingw                |  3 +-
 4 files changed, 66 insertions(+), 24 deletions(-)
 create mode 100644 tests/docker/dockerfiles/travis.docker
 create mode 100755 tests/docker/test-build

-- 
2.7.4

[Qemu-devel] [PULL v2 1/4] tests/docker: add travis dockerfile

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

This target grabs the latest Travis containers from their repository at
quay.io and then installs QEMU's build dependencies. With this it is
possible to run on broadly the same setup as they have on travis-ci.org.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20161011161625.9070-2-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/dockerfiles/travis.docker | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 tests/docker/dockerfiles/travis.docker

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
new file mode 100644
index 0000000..e4983ae
--- /dev/null
+++ b/tests/docker/dockerfiles/travis.docker
@@ -0,0 +1,6 @@
+FROM quay.io/travisci/travis-ruby
+RUN apt-get update
+RUN apt-get -y build-dep qemu
+RUN apt-get -y build-dep device-tree-compiler
+RUN apt-get -y install python2.7 dh-autoreconf
+ENV FEATURES pyyaml
-- 
2.7.4

[Qemu-devel] [PULL v2 2/4] tests/docker: test-build script

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

Much like test-quick but only builds. This is useful for some of the
build targets like ThreadSanitizer that don't yet pass "make check".

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20161011161625.9070-3-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-build | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100755 tests/docker/test-build

diff --git a/tests/docker/test-build b/tests/docker/test-build
new file mode 100755
index 0000000..031a7d9
--- /dev/null
+++ b/tests/docker/test-build
@@ -0,0 +1,20 @@
+#!/bin/bash -e
+#
+# Quick compile test without the make check step of test-quick.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
-- 
2.7.4

[Qemu-devel] [PULL v2 3/4] tests/docker: make test-mingw honour TARGET_LIST

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

The other builders honour this variable, so should the mingw build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20161011161625.9070-4-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-mingw | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
index 3396876..2adadcb 100755
--- a/tests/docker/test-mingw
+++ b/tests/docker/test-mingw
@@ -16,9 +16,10 @@
 requires mingw dtc
 
 cd "$BUILD_DIR"
+DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 
 for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
-    TARGET_LIST=x86_64-softmmu,aarch64-softmmu \
+    TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
         build_qemu --cross-prefix=$prefix \
         --enable-trace-backends=simple \
         --enable-debug \
-- 
2.7.4

[Qemu-devel] [PULL v2 4/4] tests/docker/Makefile.include: add a generic docker-run target

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

This re-factors the docker makefile to include a docker-run target which
can be controlled entirely from environment variables specified on the
make command line. This allows us to run against any given docker image
we may have in our repository, for example:

    make docker-run TEST="test-quick" IMAGE="debian:arm64" \
         EXECUTABLE=./aarch64-linux-user/qemu-aarch64

The existing docker-foo@bar targets still work but the inline
verification has been dropped because we already don't hit that due to
other pattern rules in rules.mak.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20161011161625.9070-5-alex.bennee@linaro.org>
Message-Id: <20161011161625.9070-6-alex.bennee@linaro.org>
[Squash in the verification removal patch. - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/Makefile.include | 61 +++++++++++++++++++++++++++----------------
 1 file changed, 38 insertions(+), 23 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index b44daab..3f15d5a 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -78,6 +78,7 @@ docker:
 	@echo '                         "IMAGE" is one of the listed container name."'
 	@echo '    docker-image:        Build all images.'
 	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
+	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
 	@echo
 	@echo 'Available container images:'
 	@echo '    $(DOCKER_IMAGES)'
@@ -101,31 +102,45 @@ docker:
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
+# This rule if for directly running against an arbitrary docker target.
+# It is called by the expanded docker targets (e.g. make
+# docker-test-foo@bar) which will do additional verification.
+#
+# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
+#
+docker-run: docker-qemu-src
+	@mkdir -p "$(DOCKER_CCACHE_DIR)"
+	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
+		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
+	fi
+	$(if $(EXECUTABLE),						\
+		$(call quiet-command,					\
+			$(SRC_PATH)/tests/docker/docker.py update 	\
+			$(IMAGE) $(EXECUTABLE),				\
+			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
+	$(call quiet-command,						\
+		$(SRC_PATH)/tests/docker/docker.py run 			\
+			-t 						\
+			$(if $V,,--rm) 					\
+			$(if $(DEBUG),-i,--net=none) 			\
+			-e TARGET_LIST=$(TARGET_LIST) 			\
+			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
+			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
+			-e SHOW_ENV=$(SHOW_ENV) 			\
+			-e CCACHE_DIR=/var/tmp/ccache 			\
+			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
+			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
+			$(IMAGE) 					\
+			/var/tmp/qemu/run 				\
+			$(TEST), "  RUN $(TEST) in ${IMAGE}")
+
+# Run targets:
+#
+# Of the form docker-TEST-FOO@IMAGE-BAR which will then be expanded into a call to "make docker-run"
 docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
 docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%: docker-qemu-src
-	@mkdir -p "$(DOCKER_CCACHE_DIR)"
-	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
-		then echo "Invalid target"; exit 1; \
-	fi
-	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
-		$(call quiet-command,\
-			if $(SRC_PATH)/tests/docker/docker.py images | \
-				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
-				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
-				-t \
-				$(if $(DEBUG),-i,--net=none) \
-				-e TARGET_LIST=$(TARGET_LIST) \
-				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
-				-e V=$V -e J=$J -e DEBUG=$(DEBUG) -e SHOW_ENV=$(SHOW_ENV)\
-				-e CCACHE_DIR=/var/tmp/ccache \
-				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
-				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
-				qemu:$(IMAGE) \
-				/var/tmp/qemu/run \
-				$(CMD); \
-			fi \
-			,"RUN","$(CMD) in $(IMAGE)")))
+docker-run-%:
+	@$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
 
 docker-clean:
 	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
-- 
2.7.4

Re: [Qemu-devel] [PULL v2 0/4] Docker patches

[Peter Maydell]

On 17 October 2016 at 03:10, Fam Zheng <famz@redhat.com> wrote:
> The following changes since commit 6aa5a3679449cdf0b6fe5a6829b22e642ded57fd:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/pull-ui-20161013-1' into staging (2016-10-13 14:27:58 +0100)
>
> are available in the git repository at:
>
>   git@github.com:famz/qemu tags/for-upstream
>
> for you to fetch changes up to e86c9a64f455018fb04d631e14c5f926e36c69fb:
>
>   tests/docker/Makefile.include: add a generic docker-run target (2016-10-17 10:05:48 +0800)
>
> ----------------------------------------------------------------
>
> v2: Fix $(MAKE) in patch 4. [Paolo]
>
> ----------------------------------------------------------------

Applied, thanks.

-- PMM