From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45372)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <famz@redhat.com>) id 1bmYqK-0007gt-2h
	for qemu-devel@nongnu.org; Wed, 21 Sep 2016 00:09:09 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <famz@redhat.com>) id 1bmYqG-0006FH-OB
	for qemu-devel@nongnu.org; Wed, 21 Sep 2016 00:09:07 -0400
Received: from mx1.redhat.com ([209.132.183.28]:45056)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <famz@redhat.com>) id 1bmYqG-0006Ez-EX
	for qemu-devel@nongnu.org; Wed, 21 Sep 2016 00:09:04 -0400
Date: Wed, 21 Sep 2016 12:09:01 +0800
From: Fam Zheng <famz@redhat.com>
Message-ID: <20160921040901.GC20653@lemon>
References: <20160920135616.2215-1-alex.bennee@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <20160920135616.2215-1-alex.bennee@linaro.org>
Content-Transfer-Encoding: quoted-printable
Subject: Re: [Qemu-devel] [RFC] test/docker/Makefile.include: add a generic
 docker-run target
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel/>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Alex =?iso-8859-1?Q?Benn=E9e?= <alex.bennee@linaro.org>
Cc: qemu-devel@nongnu.org

On Tue, 09/20 14:56, Alex Benn=E9e wrote:
> This re-factors the docker makefile to include a docker-run target whic=
h
> can be controlled entirely from environment variables specified on the
> make command line. This allows us to run against any given docker image
> we may have in our repository, for example:
>=20
>     make docker-run TEST=3D"test-quick" IMAGE=3D"debian:arm64" \
>          EXECUTABLE=3D./aarch64-linux-user/qemu-aarch64
>=20
> The existing docker-foo@bar targets still work but the inline
> verification has been shunted into other target prerequisites before a
> sub-make is invoked for the docker-run target.

Hi Alex,

I understand sometimes one can have specialized images, but still: is it
possible to convert them to Dockerfile and include in the tree?

Or, is this for testing/debugging purpose?

>=20
> Signed-off-by: Alex Benn=E9e <alex.bennee@linaro.org>
>=20
> ---
> NB: I dropped the awk magic that verifies the image exists before
> running. I couldn't get the thing to work in my shell so wasn't quite
> sure what it was doing.

It was to allow "make docker-test" to skip debian-bootstrap image if it i=
s not
there (e.g. when qemu-user not available).

I'm not much too concerned about that though, since most of the time we w=
ill
use docker-FOO@BAR, for specific combinations, instead of docker-test for=
 a
blanket coverage.

> ---
>  tests/docker/Makefile.include | 82 ++++++++++++++++++++++++++++++++---=
--------
>  1 file changed, 62 insertions(+), 20 deletions(-)
>=20
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.incl=
ude
> index 19d4cc7..ee47085 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -101,31 +101,73 @@ docker:
>  	@echo '    NOCACHE=3D1            Ignore cache when build images.'
>  	@echo '    EXECUTABLE=3D<path>    Include executable in image.'
> =20
> -docker-run-%: CMD =3D $(shell echo '$@' | sed -e 's/docker-run-\([^@]*=
\)@\(.*\)/\1/')
> -docker-run-%: IMAGE =3D $(shell echo '$@' | sed -e 's/docker-run-\([^@=
]*\)@\(.*\)/\2/')
> -docker-run-%: docker-qemu-src
> +# This rule if for directly running against an arbitary docker target.
> +# It is called by the expanded docker targets (e.g. make
> +# docker-test-foo@bar) which will do additional verification.
> +#
> +# For example: make docker-run TEST=3D"test-quick" IMAGE=3D"debian:arm=
64" EXECUTABLE=3D./aarch64-linux-user/qemu-aarch64
> +#
> +docker-run: docker-qemu-src

This target should probably be documented in "make docker".

>  	@mkdir -p "$(DOCKER_CCACHE_DIR)"
> -	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> -		then echo "Invalid target"; exit 1; \
> +	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
> +		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
>  	fi
> -	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> -		$(call quiet-command,\
> -			if $(SRC_PATH)/tests/docker/docker.py images | \
> -				awk '$$1=3D=3D"qemu" && $$2=3D=3D"$(IMAGE)"{found=3D1} END{exit(!f=
ound)}'; then \
> -				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
> -				-t \
> -				$(if $(DEBUG),-i,--net=3Dnone) \
> -				-e TARGET_LIST=3D$(TARGET_LIST) \
> +	$(if $(EXECUTABLE),						\
> +		$(call quiet-command,					\
> +			$(SRC_PATH)/tests/docker/docker.py update 	\
> +			$(IMAGE) $(EXECUTABLE)))
> +	$(call quiet-command,						\
> +		$(SRC_PATH)/tests/docker/docker.py run 			\
> +				-t 					\
> +				$(if $V,,--rm) 				\
> +				$(if $(DEBUG),-i,--net=3Dnone) 		\
> +				-e TARGET_LIST=3D$(TARGET_LIST) 		\
>  				-e EXTRA_CONFIGURE_OPTS=3D$(EXTRA_CONFIGURE_OPTS) \
> -				-e V=3D$V -e J=3D$J -e DEBUG=3D$(DEBUG)\
> -				-e CCACHE_DIR=3D/var/tmp/ccache \
> +				-e V=3D$V -e J=3D$J -e DEBUG=3D$(DEBUG)	\
> +				-e CCACHE_DIR=3D/var/tmp/ccache 		\
>  				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
>  				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
> -				qemu:$(IMAGE) \
> -				/var/tmp/qemu/run \
> -				$(CMD); \
> -			fi \
> -			, "  RUN $(CMD) in $(IMAGE)")))
> +				$(IMAGE) 				\
> +				/var/tmp/qemu/run 			\
> +				$(TEST), "  RUN $(TEST) in ${IMAGE}")
> +
> +#
> +# Verification targets
> +#
> +# These targets help verify the test (CMD) and docker tag (IMAGE) are
> +# part of the built in set of tests and images. You can still call the
> +# docker-run target directly for testsing against arbitary images.

s/arbitary/arbitrary/

One more of this below.

> +#
> +
> +docker-verify-image-%: IMAGE =3D $(shell echo '$@' | sed -e 's/docker-=
verify-image-\([^@]*\)@\(.*\)/\2/')
> +docker-verify-image-%:
> +	@if test -z "$(IMAGE)";			 	\
> +		then echo "Invalid image"; exit 1; 	\
> +	fi
> +	$(if $(findstring $(IMAGE), $(DOCKER_IMAGES))	\

This seems inaccurate. what if it is in a form of "$(findstring abc, abcd=
e)"?

> +		, @echo "Verified $(IMAGE)"		\
> +		, @echo "$(IMAGE) is not a known image"; exit 1)
> +
> +docker-verify-test-%: CMD =3D $(shell echo '$@' | sed -e 's/docker-ver=
ify-test-\([^@]*\)@\(.*\)/\1/')
> +docker-verify-test-%:
> +	@if test -z "$(CMD)"; 				\
> +		then echo "Invalid test"; exit 1; 	\
> +	fi
> +	$(if $(findstring $(CMD), $(DOCKER_TESTS))	\

The same question to findstring as above.

> +		, @echo "Verified $(CMD)"		\

No need to echo anything in case of verified, or at least use quiet-comma=
nd
please.

> +		, @echo "$(CMD) is not a known test"; exit 1)
> +
> +# Run targets
> +#
> +# This will take a target such as docker-test-foo@bar and verify that:
> +#  - the test test-foo is a known test
> +#  - the image bar is a known image
> +#
> +# It will then call the docker-run
> +docker-run-%: CMD =3D $(shell echo '$@' | sed -e 's/docker-run-\([^@]*=
\)@\(.*\)/\1/')
> +docker-run-%: IMAGE =3D $(shell echo '$@' | sed -e 's/docker-run-\([^@=
]*\)@\(.*\)/\2/')
> +docker-run-%: docker-verify-image-% docker-verify-test-%
> +	@make docker-run TEST=3D$(CMD) IMAGE=3Dqemu:$(IMAGE)
> =20
>  docker-clean:
>  	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
> --=20
> 2.9.3
>=20

Fam