All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Alex Bennée" <alex.bennee@linaro.org>
To: cota@braap.org, famz@redhat.com, berrange@redhat.com,
	f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com,
	aurelien@aurel32.net, agraf@suse.de, peter.maydell@linaro.org
Cc: qemu-devel@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>
Subject: [Qemu-devel] [PATCH v7 48/54] tests: add top-level make dependency for docker builds
Date: Fri, 15 Jun 2018 20:46:59 +0100	[thread overview]
Message-ID: <20180615194705.28019-49-alex.bennee@linaro.org> (raw)
In-Reply-To: <20180615194705.28019-1-alex.bennee@linaro.org>

One problem with satisfying your docker dependencies in a sub-make it
you might end up trying to satisfy the dependency multiple times. This
is especially a problem with debian-sid based cross compilers and CI
setups. We solve this by doing a docker build pass at the top level
before any sub-makes are called.

We still need to satisfy dependencies in the Makefile.target call so
people can run tests from individual target directories. We introduce
a new Makefile.probe which gets called for each PROBE_TARGET and
allows us to build up the list. It does require multiply including
config-target.mak which shouldn't cause any issues as it shouldn't
define anything that clashes with config-host.mak. However we undefine
a few key variables each time around.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v7
  - remove $(info) spam
  - just add the explicit docker dependency to each build target
  - include build-tcg-prereqs for CI tasks
---
 tests/Makefile.include   | 11 +++++++++++
 tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++
 2 files changed, 42 insertions(+)
 create mode 100644 tests/tcg/Makefile.probe

diff --git a/tests/Makefile.include b/tests/Makefile.include
index c8faa01a33..2dd0700a88 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -938,6 +938,14 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS))
 CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS))
 RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS))
 
+# Probe for the Docker Builds needed for each build
+DOCKER_BUILD_TARGETS:=
+$(foreach PROBE_TARGET,$(TARGET_LIST), 				\
+	$(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) 	\
+	$(if $(DOCKER_PREREQ), 					\
+		$(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ))  \
+		$(eval build-tcg-tests-$(PROBE_TARGET): $(DOCKER_PREREQ))))
+
 build-tcg-tests-%:
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,)
 
@@ -947,6 +955,9 @@ run-tcg-tests-%: build-tcg-tests-%
 clean-tcg-tests-%:
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" clean-guest-tests,)
 
+.PHONY: build-tcg-prereqs
+build-tcg-prereqs: $(DOCKER_BUILD_TARGETS)
+
 .PHONY: build-tcg
 build-tcg: $(BUILD_TCG_TARGET_RULES)
 
diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe
new file mode 100644
index 0000000000..15c0412657
--- /dev/null
+++ b/tests/tcg/Makefile.probe
@@ -0,0 +1,31 @@
+# -*- Mode: makefile -*-
+#
+# TCG Compiler Probe
+#
+# This Makefile fragement is included multiple times in the main make
+# script to probe for available compilers. This is used to build up a
+# selection of required docker targets before we invoke a sub-make for
+# each target.
+
+# First we need the target makefile which tells us the target architecture
+-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak
+
+# Then we load up the target architecture makefiles which tell us
+# about the compilers
+CROSS_CC_GUEST:=
+DOCKER_IMAGE:=
+DOCKER_PREREQ:=
+
+-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include
+-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include
+
+ifndef CROSS_CC_GUEST
+ifneq ($(DOCKER_IMAGE),)
+DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE)
+endif
+endif
+
+# Clean-up
+# undefine TARGET_NAME
+# undefine TARGET_BASE_ARCH
+# undefine TARGET_ABI_DIR
-- 
2.17.1

  parent reply	other threads:[~2018-06-15 19:55 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-15 19:46 [Qemu-devel] [PATCH v7 00/54] fix building of tests/tcg - last chance to review! Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 01/54] configure: add support for --cross-cc-FOO Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 02/54] configure: move i386_cc to cross_cc_i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 03/54] configure: allow user to specify --cross-cc-cflags-foo= Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 04/54] configure: set cross_cc_FOO for host compiler Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 05/54] docker: Add "cc" subcommand Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 06/54] docker: extend "cc" command to accept compiler Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 07/54] docker: allow "cc" command to run in user context Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 08/54] docker: Makefile.include introduce DOCKER_SCRIPT Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 09/54] tests/tcg: move architecture independent tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 10/54] tests/tcg/multiarch: Build fix for linux-test Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 11/54] tests/tcg/multiarch: enable additional linux-test tests Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 12/54] tests/tcg/multiarch: don't hard code paths/ports for linux-test Alex Bennée
2018-06-16 19:46   ` Thomas Huth
2018-06-18 10:56     ` Alex Bennée
2018-06-18 11:08       ` Daniel P. Berrangé
2018-06-18 11:09         ` Thomas Huth
2018-06-18 11:14           ` Daniel P. Berrangé
2018-06-18 12:04         ` Alex Bennée
2018-06-18 12:40           ` Thomas Huth
2018-06-18 11:08       ` Thomas Huth
2018-06-16 22:24   ` Philippe Mathieu-Daudé
2018-06-17  9:18     ` Alex Bennée
2018-06-18 15:18       ` Philippe Mathieu-Daudé
2018-06-18 15:23         ` Philippe Mathieu-Daudé
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 13/54] tests/tcg/multiarch: move most output to stdout Alex Bennée
2018-06-16 19:48   ` Thomas Huth
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 14/54] tests/tcg: move i386 specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 15/54] tests/tcg: enable building for i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 16/54] tests/tcg/i386: Build fix for hello-i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 17/54] tests/tcg/i386: fix test-i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 18/54] tests/tcg/i386: add runner for test-i386-fprem Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 19/54] tests/tcg/x86_64: add Makefile.target Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 20/54] tests/tcg/i386/test-i386: use modern vector_size attributes Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 21/54] tests/tcg/i386/test-i386: fix printf format Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 22/54] tests/tcg: move ARM specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 23/54] tests/tcg: enable building for ARM Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 24/54] tests/tcg/arm: fix up test-arm-iwmmxt test Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 25/54] tests/tcg: enable building for AArch64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 26/54] tests/tcg/arm: add fcvt test cases for AArch32/64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 27/54] tests/tcg: move MIPS specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 28/54] tests/tcg: enable building for MIPS Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 29/54] tests/docker/Makefile.include: fix mipsel-cross dependancy Alex Bennée
2018-06-16 22:27   ` Philippe Mathieu-Daudé
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 30/54] tests/tcg/mips: include common mips hello-mips Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 31/54] tests/tcg: enable building for s390x Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 32/54] tests/tcg: enable building for ppc64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 33/54] tests/tcg: enable building for Alpha Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 34/54] tests/tcg/alpha: add Alpha specific tests Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 35/54] tests/tcg: enable building for HPPA Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 36/54] tests/tcg: enable building for m68k Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 37/54] tests/tcg: enable building for sh4 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 38/54] tests/tcg: enable building for sparc64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 39/54] tests/tcg: enable building for mips64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 40/54] tests/tcg: enable building for RISCV64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 41/54] docker: move debian-powerpc-cross to sid based build Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 42/54] tests/tcg: enable building for PowerPC Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 43/54] tests/tcg/Makefile: update to be called from Makefile.target Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 44/54] Makefile.target: add (clean-/build-)guest-tests targets Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 45/54] tests/Makefile.include: add [build|clean|check]-tcg targets Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 46/54] tests/tcg: add run, diff, and skip helper macros Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 47/54] tests/tcg: override runners for broken tests Alex Bennée
2018-06-15 19:46 ` Alex Bennée [this message]
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 49/54] docker: docker.py don't conflate checksums for extra_files Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 50/54] docker: docker.py add check sub-command Alex Bennée
2018-06-18 15:06   ` Philippe Mathieu-Daudé
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 51/54] tests/Makefile: call sub-makes with SKIP_DOCKER_BUILD=1 Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 52/54] docker: docker.py adding age check command Alex Bennée
2018-06-18 15:07   ` Philippe Mathieu-Daudé
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 53/54] tests/docker/Makefile.include: only force SID to NOCACHE if old Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 54/54] .travis.yml: add check-tcg test Alex Bennée
2018-06-15 22:29 ` [Qemu-devel] [PATCH v7 00/54] fix building of tests/tcg - last chance to review! no-reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180615194705.28019-49-alex.bennee@linaro.org \
    --to=alex.bennee@linaro.org \
    --cc=agraf@suse.de \
    --cc=aurelien@aurel32.net \
    --cc=balrogg@gmail.com \
    --cc=berrange@redhat.com \
    --cc=cota@braap.org \
    --cc=f4bug@amsat.org \
    --cc=famz@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=richard.henderson@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.