[Qemu-devel] [PATCH 0/3] new docker targets

[Qemu-devel] [PATCH 0/3] new docker targets

[Alex Bennée]

Hi Fam,

A few patches which include the re-factoring of Makfile.include with
your review comments addressed (I hope ;-). I've also added a new
docker target for Travis which essentially just pulls their quay.io
hosted images as described here (although without the su - travis):

    https://docs.travis-ci.com/user/common-build-problems/#Troubleshooting-Locally-in-a-Docker-Image

I've also included a straight build test although arguably it might be
better to come up with a way to suppress "make check" on the existing
test-quick test.

For reference I now use the direct run to cross-build by doing:

    make docker-run TEST="test-build" J=9 IMAGE="debian:arm64" \
        EXECUTABLE=./aarch64-linux-user/qemu-aarch64  \
        TARGET_LIST=arm-linux-user,arm-softmmu,aarch64-linux-user,aarch64-softmmu

Alex Bennée (3):
  tests/docker: add travis dockerfile
  tests/docker: test-build script
  tests/docker/Makefile.include: add a generic docker-run target

 tests/docker/Makefile.include          | 88 +++++++++++++++++++++++++---------
 tests/docker/dockerfiles/travis.docker |  6 +++
 tests/docker/test-build                | 18 +++++++
 3 files changed, 89 insertions(+), 23 deletions(-)
 create mode 100644 tests/docker/dockerfiles/travis.docker
 create mode 100755 tests/docker/test-build

-- 
2.9.3

[Qemu-devel] [PATCH 1/3] tests/docker: add travis dockerfile

[Alex Bennée]

This target grabs the latest Travis containers from their repository at
quay.io and then installs QEMU's build dependencies. With this it is
possible to run on broadly the same setup as they have on travis-ci.org.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/travis.docker | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 tests/docker/dockerfiles/travis.docker

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
new file mode 100644
index 0000000..e4983ae
--- /dev/null
+++ b/tests/docker/dockerfiles/travis.docker
@@ -0,0 +1,6 @@
+FROM quay.io/travisci/travis-ruby
+RUN apt-get update
+RUN apt-get -y build-dep qemu
+RUN apt-get -y build-dep device-tree-compiler
+RUN apt-get -y install python2.7 dh-autoreconf
+ENV FEATURES pyyaml
-- 
2.9.3

[Qemu-devel] [PATCH 2/3] tests/docker: test-build script

[Alex Bennée]

Much like test-quick but only builds. This is useful for some of the
build targets like ThreadSanitizer that don't yet pass "make check".

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/test-build | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100755 tests/docker/test-build

diff --git a/tests/docker/test-build b/tests/docker/test-build
new file mode 100755
index 0000000..d237ead
--- /dev/null
+++ b/tests/docker/test-build
@@ -0,0 +1,18 @@
+#!/bin/bash -e
+#
+# Quick compiling test that everyone already does. But why not automate it?
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
-- 
2.9.3

[Qemu-devel] [PATCH 3/3] tests/docker/Makefile.include: add a generic docker-run target

[Alex Bennée]

This re-factors the docker makefile to include a docker-run target which
can be controlled entirely from environment variables specified on the
make command line. This allows us to run against any given docker image
we may have in our repository, for example:

    make docker-run TEST="test-quick" IMAGE="debian:arm64" \
         EXECUTABLE=./aarch64-linux-user/qemu-aarch64

The existing docker-foo@bar targets still work but the inline
verification has been shunted into other target prerequisites before a
sub-make is invoked for the docker-run target.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
NB: I dropped the awk magic that verifies the image exists before
running. I couldn't get the thing to work in my shell so wasn't quite
sure what it was doing.

v2
 - fix spelling on arbitrary
 - document docker-run in help
 - add note on docker.py update
 - reduce noise on verifying other images
 - revert back to using filter
---
 tests/docker/Makefile.include | 88 ++++++++++++++++++++++++++++++++-----------
 1 file changed, 65 insertions(+), 23 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 19d4cc7..5f71293 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -78,6 +78,7 @@ docker:
 	@echo '                         "IMAGE" is one of the listed container name."'
 	@echo '    docker-image:        Build all images.'
 	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
+	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
 	@echo
 	@echo 'Available container images:'
 	@echo '    $(DOCKER_IMAGES)'
@@ -101,31 +102,72 @@ docker:
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
-docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
-docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%: docker-qemu-src
+# This rule if for directly running against an arbitrary docker target.
+# It is called by the expanded docker targets (e.g. make
+# docker-test-foo@bar) which will do additional verification.
+#
+# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
+#
+docker-run: docker-qemu-src
 	@mkdir -p "$(DOCKER_CCACHE_DIR)"
-	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
-		then echo "Invalid target"; exit 1; \
+	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
+		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
+	fi
+	$(if $(EXECUTABLE),						\
+		$(call quiet-command,					\
+			$(SRC_PATH)/tests/docker/docker.py update 	\
+			$(IMAGE) $(EXECUTABLE),				\
+			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
+	$(call quiet-command,						\
+		$(SRC_PATH)/tests/docker/docker.py run 			\
+			-t 						\
+			$(if $V,,--rm) 					\
+			$(if $(DEBUG),-i,--net=none) 			\
+			-e TARGET_LIST=$(TARGET_LIST) 			\
+			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
+			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
+			-e CCACHE_DIR=/var/tmp/ccache 			\
+			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
+			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
+			$(IMAGE) 					\
+			/var/tmp/qemu/run 				\
+			$(TEST), "  RUN $(TEST) in ${IMAGE}")
+
+#
+# Verification targets
+#
+# These targets help verify the test (CMD) and docker tag (IMAGE) are
+# part of the built in set of tests and images. You can still call the
+# docker-run target directly for testsing against arbitary images.
+#
+
+docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
+docker-verify-image-%:
+	@if test -z "$(IMAGE)";			 	\
+		then echo "Invalid image"; exit 1; 	\
 	fi
-	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
-		$(call quiet-command,\
-			if $(SRC_PATH)/tests/docker/docker.py images | \
-				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
-				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
-				-t \
-				$(if $(DEBUG),-i,--net=none) \
-				-e TARGET_LIST=$(TARGET_LIST) \
-				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
-				-e V=$V -e J=$J -e DEBUG=$(DEBUG)\
-				-e CCACHE_DIR=/var/tmp/ccache \
-				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
-				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
-				qemu:$(IMAGE) \
-				/var/tmp/qemu/run \
-				$(CMD); \
-			fi \
-			, "  RUN $(CMD) in $(IMAGE)")))
+	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
+		echo "$(IMAGE) is not a known image"; exit 1)	\
+
+docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
+docker-verify-test-%:
+	@if test -z "$(CMD)"; 				\
+		then echo "Invalid test"; exit 1; 	\
+	fi
+	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
+		echo "$(CMD) is not a known test"; exit 1)
+
+# Run targets
+#
+# This will take a target such as docker-test-foo@bar and verify that:
+#  - the test test-foo is a known test
+#  - the image bar is a known image
+#
+# It will then call the docker-run
+docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
+docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
+docker-run-%: docker-verify-image-% docker-verify-test-%
+	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
 
 docker-clean:
 	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
-- 
2.9.3

Re: [Qemu-devel] [PATCH 1/3] tests/docker: add travis dockerfile

[Fam Zheng]

On Thu, 09/22 14:57, Alex Bennée wrote:
> This target grabs the latest Travis containers from their repository at
> quay.io and then installs QEMU's build dependencies. With this it is
> possible to run on broadly the same setup as they have on travis-ci.org.

This is nice, Thanks!

> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/dockerfiles/travis.docker | 6 ++++++
>  1 file changed, 6 insertions(+)
>  create mode 100644 tests/docker/dockerfiles/travis.docker
> 
> diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
> new file mode 100644
> index 0000000..e4983ae
> --- /dev/null
> +++ b/tests/docker/dockerfiles/travis.docker
> @@ -0,0 +1,6 @@
> +FROM quay.io/travisci/travis-ruby
> +RUN apt-get update
> +RUN apt-get -y build-dep qemu
> +RUN apt-get -y build-dep device-tree-compiler
> +RUN apt-get -y install python2.7 dh-autoreconf
> +ENV FEATURES pyyaml

Other images now have a /packages.txt listing installed packages,
that can be dumped with SHOW_ENV=1:

https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04925.html

In this image the list is not explicit. While the file is not mandatory, is it
worth to do something like

    RUN apt-rdepends --build-depends qemu > /packges.txt

?

Fam

Re: [Qemu-devel] [PATCH 2/3] tests/docker: test-build script

[Fam Zheng]

On Thu, 09/22 14:57, Alex Bennée wrote:
> Much like test-quick but only builds. This is useful for some of the
> build targets like ThreadSanitizer that don't yet pass "make check".
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/test-build | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
>  create mode 100755 tests/docker/test-build
> 
> diff --git a/tests/docker/test-build b/tests/docker/test-build
> new file mode 100755
> index 0000000..d237ead
> --- /dev/null
> +++ b/tests/docker/test-build
> @@ -0,0 +1,18 @@
> +#!/bin/bash -e
> +#
> +# Quick compiling test that everyone already does. But why not automate it?

Could you update the header to reflect the reason in the commit message?

> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"

As seen in the other series I pointed out in reply to patch 1, this has been
expanded in test-quick.

> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
> +build_qemu
> -- 
> 2.9.3
> 

Fam

Re: [Qemu-devel] [PATCH 3/3] tests/docker/Makefile.include: add a generic docker-run target

[Fam Zheng]

On Thu, 09/22 14:57, Alex Bennée wrote:
> This re-factors the docker makefile to include a docker-run target which
> can be controlled entirely from environment variables specified on the
> make command line. This allows us to run against any given docker image
> we may have in our repository, for example:
> 
>     make docker-run TEST="test-quick" IMAGE="debian:arm64" \
>          EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> 
> The existing docker-foo@bar targets still work but the inline
> verification has been shunted into other target prerequisites before a
> sub-make is invoked for the docker-run target.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

You probably need to rebase as a pull req is on the way that updates this file
and causes conflict.

> 
> ---
> NB: I dropped the awk magic that verifies the image exists before
> running. I couldn't get the thing to work in my shell so wasn't quite
> sure what it was doing.

Does this mean that "make docker-test" will unconditionally run
debian-bootstrap and fail if binfmt is not setup properly?

Fam

> 
> v2
>  - fix spelling on arbitrary
>  - document docker-run in help
>  - add note on docker.py update
>  - reduce noise on verifying other images
>  - revert back to using filter
> ---
>  tests/docker/Makefile.include | 88 ++++++++++++++++++++++++++++++++-----------
>  1 file changed, 65 insertions(+), 23 deletions(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 19d4cc7..5f71293 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -78,6 +78,7 @@ docker:
>  	@echo '                         "IMAGE" is one of the listed container name."'
>  	@echo '    docker-image:        Build all images.'
>  	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
> +	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
>  	@echo
>  	@echo 'Available container images:'
>  	@echo '    $(DOCKER_IMAGES)'
> @@ -101,31 +102,72 @@ docker:
>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>  
> -docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> -docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> -docker-run-%: docker-qemu-src
> +# This rule if for directly running against an arbitrary docker target.
> +# It is called by the expanded docker targets (e.g. make
> +# docker-test-foo@bar) which will do additional verification.
> +#
> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> +#
> +docker-run: docker-qemu-src
>  	@mkdir -p "$(DOCKER_CCACHE_DIR)"
> -	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> -		then echo "Invalid target"; exit 1; \
> +	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
> +		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
> +	fi
> +	$(if $(EXECUTABLE),						\
> +		$(call quiet-command,					\
> +			$(SRC_PATH)/tests/docker/docker.py update 	\
> +			$(IMAGE) $(EXECUTABLE),				\
> +			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
> +	$(call quiet-command,						\
> +		$(SRC_PATH)/tests/docker/docker.py run 			\
> +			-t 						\
> +			$(if $V,,--rm) 					\
> +			$(if $(DEBUG),-i,--net=none) 			\
> +			-e TARGET_LIST=$(TARGET_LIST) 			\
> +			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
> +			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
> +			-e CCACHE_DIR=/var/tmp/ccache 			\
> +			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> +			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
> +			$(IMAGE) 					\
> +			/var/tmp/qemu/run 				\
> +			$(TEST), "  RUN $(TEST) in ${IMAGE}")
> +
> +#
> +# Verification targets
> +#
> +# These targets help verify the test (CMD) and docker tag (IMAGE) are
> +# part of the built in set of tests and images. You can still call the
> +# docker-run target directly for testsing against arbitary images.
> +#
> +
> +docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
> +docker-verify-image-%:
> +	@if test -z "$(IMAGE)";			 	\
> +		then echo "Invalid image"; exit 1; 	\
>  	fi
> -	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> -		$(call quiet-command,\
> -			if $(SRC_PATH)/tests/docker/docker.py images | \
> -				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
> -				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
> -				-t \
> -				$(if $(DEBUG),-i,--net=none) \
> -				-e TARGET_LIST=$(TARGET_LIST) \
> -				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
> -				-e V=$V -e J=$J -e DEBUG=$(DEBUG)\
> -				-e CCACHE_DIR=/var/tmp/ccache \
> -				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> -				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
> -				qemu:$(IMAGE) \
> -				/var/tmp/qemu/run \
> -				$(CMD); \
> -			fi \
> -			, "  RUN $(CMD) in $(IMAGE)")))
> +	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
> +		echo "$(IMAGE) is not a known image"; exit 1)	\
> +
> +docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
> +docker-verify-test-%:
> +	@if test -z "$(CMD)"; 				\
> +		then echo "Invalid test"; exit 1; 	\
> +	fi
> +	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
> +		echo "$(CMD) is not a known test"; exit 1)
> +
> +# Run targets
> +#
> +# This will take a target such as docker-test-foo@bar and verify that:
> +#  - the test test-foo is a known test
> +#  - the image bar is a known image
> +#
> +# It will then call the docker-run
> +docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> +docker-run-%: docker-verify-image-% docker-verify-test-%
> +	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
>  
>  docker-clean:
>  	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
> -- 
> 2.9.3
>