From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38346) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bmcKD-00028C-CU for qemu-devel@nongnu.org; Wed, 21 Sep 2016 03:52:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bmcK7-0001ST-9d for qemu-devel@nongnu.org; Wed, 21 Sep 2016 03:52:12 -0400 Received: from mail-wm0-f44.google.com ([74.125.82.44]:35972) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bmcK6-0001MB-FL for qemu-devel@nongnu.org; Wed, 21 Sep 2016 03:52:07 -0400 Received: by mail-wm0-f44.google.com with SMTP id w84so176914252wmg.1 for ; Wed, 21 Sep 2016 00:51:44 -0700 (PDT) References: <20160920135616.2215-1-alex.bennee@linaro.org> <20160921040901.GC20653@lemon> From: Alex =?utf-8?Q?Benn=C3=A9e?= In-reply-to: <20160921040901.GC20653@lemon> Date: Wed, 21 Sep 2016 08:50:42 +0100 Message-ID: <87lgylbsj1.fsf@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [RFC] test/docker/Makefile.include: add a generic docker-run target List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Fam Zheng Cc: qemu-devel@nongnu.org Fam Zheng writes: > On Tue, 09/20 14:56, Alex Bennée wrote: >> This re-factors the docker makefile to include a docker-run target which >> can be controlled entirely from environment variables specified on the >> make command line. This allows us to run against any given docker image >> we may have in our repository, for example: >> >> make docker-run TEST="test-quick" IMAGE="debian:arm64" \ >> EXECUTABLE=./aarch64-linux-user/qemu-aarch64 >> >> The existing docker-foo@bar targets still work but the inline >> verification has been shunted into other target prerequisites before a >> sub-make is invoked for the docker-run target. > > Hi Alex, > > I understand sometimes one can have specialized images, but still: is it > possible to convert them to Dockerfile and include in the tree? > > Or, is this for testing/debugging purpose? A bit of both. In this particular use case I'm using a debootstrap image while updating the binfmt_misc executable. Currently there is a 1->N relationship for debootstrap as we can bootstrap multiple architectures in different images. By splitting the docker-run from the expansions we give ourselves a little more flexibility for running stuff. But I think it's also useful for testing/debugging. I wrote this up as I was trying to debug a Travis build failure with gcc-6 so I was generating lots of test images and wanting to build against those. I would also like to add a travis Dockerfile at some point but at the moment what exactly goes into one of those is a little opaque to me. > >> >> Signed-off-by: Alex Bennée >> >> --- >> NB: I dropped the awk magic that verifies the image exists before >> running. I couldn't get the thing to work in my shell so wasn't quite >> sure what it was doing. > > It was to allow "make docker-test" to skip debian-bootstrap image if it is not > there (e.g. when qemu-user not available). Ahh ok. I got a little confused as the docker images command can filter things based on tag so maybe we can come up with a cleaner test? > > I'm not much too concerned about that though, since most of the time we will > use docker-FOO@BAR, for specific combinations, instead of docker-test for a > blanket coverage. What does patchew use? > >> --- >> tests/docker/Makefile.include | 82 ++++++++++++++++++++++++++++++++----------- >> 1 file changed, 62 insertions(+), 20 deletions(-) >> >> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include >> index 19d4cc7..ee47085 100644 >> --- a/tests/docker/Makefile.include >> +++ b/tests/docker/Makefile.include >> @@ -101,31 +101,73 @@ docker: >> @echo ' NOCACHE=1 Ignore cache when build images.' >> @echo ' EXECUTABLE= Include executable in image.' >> >> -docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/') >> -docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/') >> -docker-run-%: docker-qemu-src >> +# This rule if for directly running against an arbitary docker target. >> +# It is called by the expanded docker targets (e.g. make >> +# docker-test-foo@bar) which will do additional verification. >> +# >> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64 >> +# >> +docker-run: docker-qemu-src > > This target should probably be documented in "make docker". OK > >> @mkdir -p "$(DOCKER_CCACHE_DIR)" >> - @if test -z "$(IMAGE)" || test -z "$(CMD)"; \ >> - then echo "Invalid target"; exit 1; \ >> + @if test -z "$(IMAGE)" || test -z "$(TEST)"; \ >> + then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \ >> fi >> - $(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \ >> - $(call quiet-command,\ >> - if $(SRC_PATH)/tests/docker/docker.py images | \ >> - awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \ >> - $(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \ >> - -t \ >> - $(if $(DEBUG),-i,--net=none) \ >> - -e TARGET_LIST=$(TARGET_LIST) \ >> + $(if $(EXECUTABLE), \ >> + $(call quiet-command, \ >> + $(SRC_PATH)/tests/docker/docker.py update \ >> + $(IMAGE) $(EXECUTABLE))) >> + $(call quiet-command, \ >> + $(SRC_PATH)/tests/docker/docker.py run \ >> + -t \ >> + $(if $V,,--rm) \ >> + $(if $(DEBUG),-i,--net=none) \ >> + -e TARGET_LIST=$(TARGET_LIST) \ >> -e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \ >> - -e V=$V -e J=$J -e DEBUG=$(DEBUG)\ >> - -e CCACHE_DIR=/var/tmp/ccache \ >> + -e V=$V -e J=$J -e DEBUG=$(DEBUG) \ >> + -e CCACHE_DIR=/var/tmp/ccache \ >> -v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \ >> -v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \ >> - qemu:$(IMAGE) \ >> - /var/tmp/qemu/run \ >> - $(CMD); \ >> - fi \ >> - , " RUN $(CMD) in $(IMAGE)"))) >> + $(IMAGE) \ >> + /var/tmp/qemu/run \ >> + $(TEST), " RUN $(TEST) in ${IMAGE}") >> + >> +# >> +# Verification targets >> +# >> +# These targets help verify the test (CMD) and docker tag (IMAGE) are >> +# part of the built in set of tests and images. You can still call the >> +# docker-run target directly for testsing against arbitary images. > > s/arbitary/arbitrary/ > > One more of this below. OK. > >> +# >> + >> +docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/') >> +docker-verify-image-%: >> + @if test -z "$(IMAGE)"; \ >> + then echo "Invalid image"; exit 1; \ >> + fi >> + $(if $(findstring $(IMAGE), $(DOCKER_IMAGES)) \ > > This seems inaccurate. what if it is in a form of "$(findstring abc, > abcde)"? I'll see if I can replace it with the original filter test. > >> + , @echo "Verified $(IMAGE)" \ >> + , @echo "$(IMAGE) is not a known image"; exit 1) >> + >> +docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/') >> +docker-verify-test-%: >> + @if test -z "$(CMD)"; \ >> + then echo "Invalid test"; exit 1; \ >> + fi >> + $(if $(findstring $(CMD), $(DOCKER_TESTS)) \ > > The same question to findstring as above. > >> + , @echo "Verified $(CMD)" \ > > No need to echo anything in case of verified, or at least use quiet-command > please. Sure. This was mostly convincing me I got my make-fu right ;-) > >> + , @echo "$(CMD) is not a known test"; exit 1) >> + >> +# Run targets >> +# >> +# This will take a target such as docker-test-foo@bar and verify that: >> +# - the test test-foo is a known test >> +# - the image bar is a known image >> +# >> +# It will then call the docker-run >> +docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/') >> +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/') >> +docker-run-%: docker-verify-image-% docker-verify-test-% >> + @make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE) >> >> docker-clean: >> $(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean) >> -- >> 2.9.3 >> > > Fam -- Alex Bennée