From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51934) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aVJZs-0002XB-3g for qemu-devel@nongnu.org; Mon, 15 Feb 2016 08:52:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aVJZn-0005YO-If for qemu-devel@nongnu.org; Mon, 15 Feb 2016 08:52:36 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55263) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aVJZn-0005Y3-Aj for qemu-devel@nongnu.org; Mon, 15 Feb 2016 08:52:31 -0500 Date: Mon, 15 Feb 2016 21:52:28 +0800 From: Fam Zheng Message-ID: <20160215135228.GC18222@ad.usersys.redhat.com> References: <1454664263-25969-1-git-send-email-famz@redhat.com> <1454664263-25969-3-git-send-email-famz@redhat.com> <87bn7inv7v.fsf@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <87bn7inv7v.fsf@linaro.org> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH 02/12] Makefile: Rules for docker testing List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex =?iso-8859-1?Q?Benn=E9e?= Cc: kwolf@redhat.com, peter.maydell@linaro.org, sw@weilnetz.de, qemu-devel@nongnu.org, stefanha@redhat.com, Paolo Bonzini , jsnow@redhat.com, david@gibson.dropbear.id.au On Mon, 02/15 10:06, Alex Benn=E9e wrote: >=20 > Fam Zheng writes: >=20 > > This adds a group of make targets to run docker tests, all are availa= ble > > in source tree without running ./configure. > > > > The usage is shown by "make docker". > > > > Besides the fixed ones, dynamic targets for building each image and > > running each test in each image are generated automatically by make, > > scanning $(SRC_PATH)/tests/docker/ files with specific patterns. > > > > Alternative to manually list particular targets (docker-run-FOO@BAR) > > set, you can control which tests/images to run by filtering variables= , > > TESTS=3D and IMAGES=3D, which are expressed in Makefile pattern synta= x, > > "foo% %bar ...". For example: > > > > $ make docker-run IMAGES=3D"ubuntu fedora" >=20 > This could do with repeating in the docs so that you don't need to > search commits for example invocations. Okay. >=20 > > > > Unfortunately, it's impossible to propagate "-j $JOBS" into make in > > containers, however since each combination is made a first class targ= et > > is the top Makefile, "make -j$N docker-run" still parallels the tests > > coarsely. > > > > Signed-off-by: Fam Zheng > > --- > > Makefile | 4 +- > > tests/docker/Makefile.include | 88 +++++++++++++++++++++++++++++++++= ++++++++++ > > 2 files changed, 91 insertions(+), 1 deletion(-) > > create mode 100644 tests/docker/Makefile.include > > > > diff --git a/Makefile b/Makefile > > index d0de2d4..7da70f4 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -6,7 +6,7 @@ BUILD_DIR=3D$(CURDIR) > > # Before including a proper config-host.mak, assume we are in the so= urce tree > > SRC_PATH=3D. > > > > -UNCHECKED_GOALS :=3D %clean TAGS cscope ctags > > +UNCHECKED_GOALS :=3D %clean TAGS cscope ctags docker docker-% > > > > # All following code might depend on configuration variables > > ifneq ($(wildcard config-host.mak),) > > @@ -651,3 +651,5 @@ endif > > # Include automatically generated dependency files > > # Dependencies in Makefile.objs files come from our recursive subdir= rules > > -include $(wildcard *.d tests/*.d) > > + > > +include $(SRC_PATH)/tests/docker/Makefile.include > > diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.in= clude > > new file mode 100644 > > index 0000000..ca84c35 > > --- /dev/null > > +++ b/tests/docker/Makefile.include > > @@ -0,0 +1,88 @@ > > +# Makefile for Docker tests > > + > > +$(if $(quiet-command),,$(eval include $(SRC_PATH)/rules.mak)) > > + > > +.PHONY: docker docker-build docker-run docker-clean > > + > > +DOCKER_SUFFIX =3D .docker > > + > > +DOCKER_IMAGES :=3D $(patsubst %$(DOCKER_SUFFIX),%, $(shell \ > > + ls $(SRC_PATH)/tests/docker/ | grep '$(DOCKER_SUFFIX)$$')) > > + > > +DOCKER_SCRIPTS :=3D $(shell ls $(SRC_PATH)/tests/docker/ | grep '\.s= h$$') > > +DOCKER_TESTS :=3D $(filter test-%, $(DOCKER_SCRIPTS)) > > +DOCKER_TOOLS :=3D $(filter-out test-%, $(DOCKER_SCRIPTS)) > > + > > +TESTS ?=3D % > > +IMAGES ?=3D % > > + > > +$(foreach i,$(DOCKER_IMAGES), \ > > + $(eval docker-build: docker-build-$i) \ > > + $(foreach t,$(DOCKER_SCRIPTS), \ > > + $(eval docker-build-$i docker-run-$t@$i: IMAGE=3D$i) \ > > + $(eval docker-run-$t@$i: SCRIPT=3D$t) \ > > + $(eval docker-run-$t@$i: docker-build-$i) \ > > + $(if $(filter test-%,$t), \ > > + $(eval docker-run: docker-run-$t@$i) \ > > + $(eval docker-run-$t@: docker-run-$t@$i) \ > > + $(eval docker-run-@$i: docker-run-$t@$i)) \ > > +)) > > + > > +docker: > > + @echo 'Building QEMU and running tests or tools inside Docker conta= iners' > > + @echo > > + @echo 'Available targets:' > > + @echo > > + @echo ' docker: Print this help.' > > + @echo ' docker-run: Run all image/test combinations.' > > + @echo ' You can override the test cases to = run by providing' > > + @echo ' TESTS=3D"foo bar" in the make comma= nd, and the image set' > > + @echo ' by providing IMAGES=3D"baz qux".' > > + @echo ' docker-clean: Kill and remove residual docker tes= ting containers.' > > + @echo ' docker-build: Build all images.' > > + @echo ' docker-build-IMG: Build image "IMG".' > > + @echo ' "IMG" is one of the listed image na= me."' > > + @echo ' docker-run-FOO@BAR: Run "FOO" in container "BAR".' > > + @echo ' "FOO" must be one of the listed tes= t/tool name."' > > + @echo ' "BAR" must be one of the listed ima= ge name."' > > + @echo ' docker-run-FOO@: Run "FOO" in all containers.' > > + @echo ' "FOO" must be one of the listed tes= t name."' > > + @echo ' docker-run-@BAR: Run all tests in container "BAR".' > > + @echo ' "BAR" must be one of the listed ima= ge name."' > > + @echo > > + @echo 'Available images:' > > + @echo ' $(DOCKER_IMAGES)' > > + @echo > > + @echo 'Available tests:' > > + @echo ' $(DOCKER_TESTS)' > > + @echo > > + @echo 'Available tools:' > > + @echo ' $(DOCKER_TOOLS)' >=20 > I commend the addition of help ;-) >=20 > Perhaps replace FOO and BAR with TEST and IMAGE to cement the actual > rather than the abstract in the readers mind. OK! >=20 > Also I expected this to work: >=20 > $ make docker-run-basic@fedora > Invalid target > make: *** [docker-run-basic@fedora] Error 1 >=20 > The help gives: >=20 > Available images: > centos6 fedora ubuntu >=20 > Available tests: > test-basic.sh test-clang.sh test-mingw.sh >=20 > Available tools: > travis.sh >=20 > The test and .sh seem superfluous to the user wanting to invoke things. > We know they are tests and the implementation detail of the shell shoul= d > be irrelevant to the user. I want to distinguish tests from usual tools by prefix (test-). Tools shouldn't be invoked by "make docker-run". I'll remove the .sh suffixes, and "s/docker-run-FOO/docker-FOO/". Then yo= u just: $ make docker-test-basic@fedora $ make docker-travis@fedora Thanks, Fam