[PATCH 00/15] tests/docker and tests/tcg cleanup and diet

[PATCH 00/15] tests/docker and tests/tcg cleanup and diet

[Paolo Bonzini]

The first seven patches simplify and clean up a bit the
tests/docker/Makefile.include file.

The rest adjust tests/Makefile.include to invoke the tests/tcg
Makefiles directly, without going through Makefile.qemu and
Makefile.prereqs.  All information is gathered at configure time,
and the config*.mak files for each TCG test target includes
the normal Makefile variables (AC, CC, LD, etc.).

This is also a first step towards moving the cross-compilation
infrastructure from tests/tcg to all of QEMU, so that it can be
used to build firmware binaries.

Paolo

Paolo Bonzini (15):
  tests/docker: remove dead code
  tests/docker: remove test targets
  tests/docker: remove dead variable
  tests/docker: remove unnecessary default definitions
  tests/docker: inline variable definitions or move close to use
  tests/docker: remove unnecessary filtering of $(DOCKER_IMAGES)
  tests/docker: simplify docker-TEST@IMAGE targets
  tests/tcg: add compiler tests when using containers
  tests/tcg: remove CONFIG_LINUX_USER from config-target.mak
  tests/tcg: remove CONFIG_USER_ONLY from config-target.mak
  tests/tcg: prepare Makefile.prereqs at configure time
  tests/tcg: list test targets in Makefile.prereqs
  tests/tcg: invoke Makefile.target directly from QEMU's makefile
  tests/docker: remove SKIP_DOCKER_BUILD
  tests/tcg: fix non-static build

 tests/Makefile.include                    |  63 +++++------
 tests/docker/Makefile.include             |  89 ++++------------
 tests/tcg/Makefile.prereqs                |  18 ----
 tests/tcg/Makefile.qemu                   | 121 ----------------------
 tests/tcg/Makefile.target                 |  11 +-
 tests/tcg/aarch64/Makefile.softmmu-target |   2 +-
 tests/tcg/aarch64/Makefile.target         |  10 +-
 tests/tcg/configure.sh                    |  72 ++++++++-----
 tests/tcg/i386/Makefile.target            |   2 +-
 tests/tcg/multiarch/Makefile.target       |   2 +-
 tests/tcg/ppc64/Makefile.target           |   4 +-
 tests/tcg/ppc64le/Makefile.target         |   4 +-
 tests/tcg/x86_64/Makefile.target          |   2 +-
 13 files changed, 116 insertions(+), 284 deletions(-)
 delete mode 100644 tests/tcg/Makefile.prereqs
 delete mode 100644 tests/tcg/Makefile.qemu

-- 
2.31.1

[PATCH 01/15] tests/docker: remove dead code

[Paolo Bonzini]

debian-powerpc-user-cross was the only linux-user powered cross builder
and it was removed in commit 80394ccf21 ("tests/docker: remove
debian-powerpc-user-cross", 2019-09-26). Remove all the infrastructure
around it since it is now unused.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e495b163a0..a6a5a20949 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -229,12 +229,6 @@ DOCKER_PARTIAL_IMAGES += debian-tricore-cross
 DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
 DOCKER_PARTIAL_IMAGES += fedora-cris-cross
 
-# Rules for building linux-user powered images
-#
-# These are slower than using native cross compiler setups but can
-# work around issues with poorly working multi-arch systems and broken
-# packages.
-
 # Expand all the pre-requistes for each docker image and test combination
 $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES) $(DOCKER_VIRTUAL_IMAGES),$(DOCKER_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS), \
@@ -266,11 +260,6 @@ docker:
 	@echo
 	@echo 'Available container images:'
 	@echo '    $(DOCKER_IMAGES)'
-ifneq ($(DOCKER_USER_IMAGES),)
-	@echo
-	@echo 'Available linux-user images (docker-binfmt-image-debian-%):'
-	@echo '    $(DOCKER_USER_IMAGES)'
-endif
 	@echo
 	@echo 'Available tests:'
 	@echo '    $(DOCKER_TESTS)'
-- 
2.31.1

[PATCH 02/15] tests/docker: remove test targets

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 18 ------------------
 1 file changed, 18 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index a6a5a20949..8248cfdb4f 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -99,24 +99,6 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
 			"CHECK", "debian-$* exists"))
 
-# These are test targets
-USER_TCG_TARGETS=$(patsubst %-linux-user,qemu-%,$(filter %-linux-user,$(TARGET_DIRS)))
-EXEC_COPY_TESTS=$(patsubst %,docker-exec-copy-test-%, $(USER_TCG_TARGETS))
-
-$(EXEC_COPY_TESTS): docker-exec-copy-test-%: $(DOCKER_FILES_DIR)/empty.docker
-	$(call quiet-command,							\
-		$(DOCKER_SCRIPT) build -t qemu/exec-copy-test-$* -f $< 		\
-			$(if $V,,--quiet) --no-cache 				\
-			--include-executable=$*					\
-			--skip-binfmt,						\
-			"TEST","copy $* to container")
-	$(call quiet-command,							\
-		$(DOCKER_SCRIPT) run qemu/exec-copy-test-$* 			\
-			/$* -version > tests/docker-exec-copy-test-$*.out,	\
-			"TEST","check $* works in container")
-
-docker-exec-copy-test: $(EXEC_COPY_TESTS)
-
 endif
 
 # Enforce dependencies for composite images
-- 
2.31.1

[PATCH 03/15] tests/docker: remove dead variable

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 8248cfdb4f..372517f113 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -13,7 +13,6 @@ HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
 TESTS ?= %
 IMAGES ?= %
 
-DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian10 debian11
-- 
2.31.1

[PATCH 04/15] tests/docker: remove unnecessary default definitions

[Paolo Bonzini]

The definition of DOCKER_IMAGES and DOCKER_TESTS copes already with an
empty value of $(IMAGES) and $(TESTS), no need to force them to "%" if
undefined.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 372517f113..80bb188620 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -8,11 +8,6 @@ COMMA := ,
 
 HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
 
-# These variables can be set by the user to limit the set of docker
-# images and tests to a more restricted subset
-TESTS ?= %
-IMAGES ?= %
-
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian10 debian11
-- 
2.31.1

[PATCH 05/15] tests/docker: inline variable definitions or move close to use

[Paolo Bonzini]

Variables that are defined with ":=" are handled imperatively, so moving
them closer to the first use ensures that all the assignments prior to
the first use are taken into account.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 80bb188620..f4bbba47a5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -9,26 +9,12 @@ COMMA := ,
 HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
 
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
-# we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian10 debian11
-# we don't directly build virtual images (they are used to build other images)
-DOCKER_VIRTUAL_IMAGES := debian-bootstrap debian-toolchain empty
-__IMAGES := $(sort $(filter-out $(DOCKER_VIRTUAL_IMAGES), $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
-DOCKER_IMAGES := $(if $(IMAGES), $(filter $(IMAGES), $(__IMAGES)), $(__IMAGES))
-DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
-# Use a global constant ccache directory to speed up repetitive builds
-DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
 ifeq ($(HOST_ARCH),x86_64)
 DOCKER_DEFAULT_REGISTRY := registry.gitlab.com/qemu-project/qemu
 endif
 DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTRY))
 
-__TESTS := $(notdir $(shell \
-		find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f))
-DOCKER_TESTS := $(if $(TESTS), $(filter $(TESTS), $(__TESTS)), $(__TESTS))
-
 ENGINE := auto
-
 DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --engine $(ENGINE)
 
 CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$)
@@ -47,8 +33,6 @@ $(DOCKER_SRC_COPY):
 
 docker-qemu-src: $(DOCKER_SRC_COPY)
 
-docker-image: ${DOCKER_TARGETS}
-
 # General rule for building docker images. If we are a sub-make
 # invoked with SKIP_DOCKER_BUILD we still check the image is up to date
 # though
@@ -96,6 +80,8 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 endif
 
 # Enforce dependencies for composite images
+# we don't run tests on intermediate images (used as base by another image)
+DOCKER_PARTIAL_IMAGES := debian10 debian11
 ifeq ($(HOST_ARCH),x86_64)
 docker-image-debian-amd64: docker-image-debian10
 DOCKER_PARTIAL_IMAGES += debian-amd64-cross
@@ -205,6 +191,15 @@ DOCKER_PARTIAL_IMAGES += debian-tricore-cross
 DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
 DOCKER_PARTIAL_IMAGES += fedora-cris-cross
 
+# images that are only used to build other images
+DOCKER_VIRTUAL_IMAGES := debian-bootstrap debian-toolchain empty
+
+__IMAGES := $(sort $(filter-out $(DOCKER_VIRTUAL_IMAGES), $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
+DOCKER_IMAGES := $(if $(IMAGES), $(filter $(IMAGES), $(__IMAGES)), $(__IMAGES))
+
+__TESTS := $(notdir $(shell find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f))
+DOCKER_TESTS := $(if $(TESTS), $(filter $(TESTS), $(__TESTS)), $(__TESTS))
+
 # Expand all the pre-requistes for each docker image and test combination
 $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES) $(DOCKER_VIRTUAL_IMAGES),$(DOCKER_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS), \
@@ -263,6 +258,9 @@ docker:
 
 docker-help: docker
 
+# Use a global constant ccache directory to speed up repetitive builds
+DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
+
 # This rule if for directly running against an arbitrary docker target.
 # It is called by the expanded docker targets (e.g. make
 # docker-test-foo@bar) which will do additional verification.
@@ -308,5 +306,7 @@ docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/
 docker-run-%:
 	@$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu/$(IMAGE)
 
+docker-image: ${DOCKER_IMAGES:%=docker-image-%}
+
 docker-clean:
 	$(call quiet-command, $(DOCKER_SCRIPT) clean)
-- 
2.31.1

[PATCH 06/15] tests/docker: remove unnecessary filtering of $(DOCKER_IMAGES)

[Paolo Bonzini]

Now that DOCKER_IMAGES is only defined after DOCKER_VIRTUAL_IMAGES is
complete, there is no need to re-filter DOCKER_IMAGES against it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index f4bbba47a5..5cadd1b53e 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -201,7 +201,7 @@ __TESTS := $(notdir $(shell find $(SRC_PATH)/tests/docker/ -name 'test-*' -type
 DOCKER_TESTS := $(if $(TESTS), $(filter $(TESTS), $(__TESTS)), $(__TESTS))
 
 # Expand all the pre-requistes for each docker image and test combination
-$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES) $(DOCKER_VIRTUAL_IMAGES),$(DOCKER_IMAGES)), \
+$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS), \
 		$(eval .PHONY: docker-$t@$i) \
 		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
-- 
2.31.1

[PATCH 07/15] tests/docker: simplify docker-TEST@IMAGE targets

[Paolo Bonzini]

No need to go through the shell when we already have the test and images at
the point where the targets are declared.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 5cadd1b53e..e1bea20451 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -204,7 +204,9 @@ DOCKER_TESTS := $(if $(TESTS), $(filter $(TESTS), $(__TESTS)), $(__TESTS))
 $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS), \
 		$(eval .PHONY: docker-$t@$i) \
-		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
+		$(eval docker-$t@$i: TEST=$t) \
+		$(eval docker-$t@$i: IMAGE=qemu/$i) \
+		$(eval docker-$t@$i: docker-image-$i; @$(MAKE) docker-run) \
 	) \
 	$(foreach t,$(DOCKER_TESTS), \
 		$(eval docker-all-tests: docker-$t@$i) \
@@ -298,14 +300,6 @@ docker-run: docker-qemu-src
 	$(call quiet-command, rm -r $(DOCKER_SRC_COPY), \
 		"  CLEANUP $(DOCKER_SRC_COPY)")
 
-# Run targets:
-#
-# Of the form docker-TEST-FOO@IMAGE-BAR which will then be expanded into a call to "make docker-run"
-docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
-docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%:
-	@$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu/$(IMAGE)
-
 docker-image: ${DOCKER_IMAGES:%=docker-image-%}
 
 docker-clean:
-- 
2.31.1

[PATCH 08/15] tests/tcg: add compiler test variables when using containers

[Paolo Bonzini]

Even for container-based cross compilation use $(CROSS_CC_HAS_*) variables.
This makes the TCG test makefiles oblivious of whether the compiler is
invoked through a container or not.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/tcg/aarch64/Makefile.softmmu-target |  2 +-
 tests/tcg/aarch64/Makefile.target         | 10 +++++-----
 tests/tcg/configure.sh                    | 16 ++++++++++++++++
 tests/tcg/i386/Makefile.target            |  2 +-
 tests/tcg/ppc64/Makefile.target           |  4 ++--
 tests/tcg/ppc64le/Makefile.target         |  4 ++--
 6 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-target
index a7286ac295..f6fcd4829e 100644
--- a/tests/tcg/aarch64/Makefile.softmmu-target
+++ b/tests/tcg/aarch64/Makefile.softmmu-target
@@ -64,7 +64,7 @@ run-memory-replay: memory-replay run-memory-record
 
 EXTRA_RUNS+=run-memory-replay
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_3),)
+ifneq ($(CROSS_CC_HAS_ARMV8_3),)
 pauth-3: CFLAGS += -march=armv8.3-a
 else
 pauth-3:
diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target
index f7121cb4d8..6ad0ad49f9 100644
--- a/tests/tcg/aarch64/Makefile.target
+++ b/tests/tcg/aarch64/Makefile.target
@@ -18,7 +18,7 @@ run-fcvt: fcvt
 	$(call diff-out,$<,$(AARCH64_SRC)/fcvt.ref)
 
 # Pauth Tests
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_3),)
+ifneq ($(CROSS_CC_HAS_ARMV8_3),)
 AARCH64_TESTS += pauth-1 pauth-2 pauth-4 pauth-5
 pauth-%: CFLAGS += -march=armv8.3-a
 run-pauth-%: QEMU_OPTS += -cpu max
@@ -27,7 +27,7 @@ endif
 
 # BTI Tests
 # bti-1 tests the elf notes, so we require special compiler support.
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_BTI),)
+ifneq ($(CROSS_CC_HAS_ARMV8_BTI),)
 AARCH64_TESTS += bti-1
 bti-1: CFLAGS += -mbranch-protection=standard
 bti-1: LDFLAGS += -nostdlib
@@ -36,12 +36,12 @@ endif
 AARCH64_TESTS += bti-2
 
 # MTE Tests
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_MTE),)
+ifneq ($(CROSS_CC_HAS_ARMV8_MTE),)
 AARCH64_TESTS += mte-1 mte-2 mte-3 mte-4 mte-5 mte-6 mte-7
 mte-%: CFLAGS += -march=armv8.5-a+memtag
 endif
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_SVE),)
+ifneq ($(CROSS_CC_HAS_SVE),)
 # System Registers Tests
 AARCH64_TESTS += sysregs
 sysregs: CFLAGS+=-march=armv8.1-a+sve
@@ -88,7 +88,7 @@ EXTRA_RUNS += run-gdbstub-sysregs run-gdbstub-sve-ioctls
 endif
 endif
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_SVE2),)
+ifneq ($(CROSS_CC_HAS_SVE2),)
 AARCH64_TESTS += test-826
 test-826: CFLAGS+=-march=armv8.1-a+sve2
 endif
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 84f928f7f8..0162b2f6c4 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -350,5 +350,21 @@ for target in $target_list; do
               fi
           fi
       done
+      case $target in
+          aarch64-*)
+              echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
+              ;;
+          ppc*)
+              echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
+              echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
+              ;;
+          i386-linux-user)
+              echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
+              ;;
+      esac
   fi
 done
diff --git a/tests/tcg/i386/Makefile.target b/tests/tcg/i386/Makefile.target
index e1c0310be6..bd73c96d0d 100644
--- a/tests/tcg/i386/Makefile.target
+++ b/tests/tcg/i386/Makefile.target
@@ -30,7 +30,7 @@ hello-i386: LDFLAGS+=-nostdlib
 # test-386 includes a couple of additional objects that need to be
 # linked together, we also need a no-pie capable compiler due to the
 # non-pic calls into 16-bit mode
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_I386_NOPIE),)
+ifneq ($(CROSS_CC_HAS_I386_NOPIE),)
 test-i386: CFLAGS += -fno-pie
 
 test-i386: test-i386.c test-i386-code16.S test-i386-vm86.S test-i386.h test-i386-shift.h test-i386-muldiv.h
diff --git a/tests/tcg/ppc64/Makefile.target b/tests/tcg/ppc64/Makefile.target
index 8197c288a7..babd209573 100644
--- a/tests/tcg/ppc64/Makefile.target
+++ b/tests/tcg/ppc64/Makefile.target
@@ -5,14 +5,14 @@
 VPATH += $(SRC_PATH)/tests/tcg/ppc64
 VPATH += $(SRC_PATH)/tests/tcg/ppc64le
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_POWER8_VECTOR),)
+ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
 PPC64_TESTS=bcdsub non_signalling_xscv
 endif
 $(PPC64_TESTS): CFLAGS += -mpower8-vector
 
 PPC64_TESTS += mtfsf
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_POWER10),)
+ifneq ($(CROSS_CC_HAS_POWER10),)
 PPC64_TESTS += byte_reverse sha512-vector
 endif
 byte_reverse: CFLAGS += -mcpu=power10
diff --git a/tests/tcg/ppc64le/Makefile.target b/tests/tcg/ppc64le/Makefile.target
index 9624bb1e9c..5b0eb5e870 100644
--- a/tests/tcg/ppc64le/Makefile.target
+++ b/tests/tcg/ppc64le/Makefile.target
@@ -4,12 +4,12 @@
 
 VPATH += $(SRC_PATH)/tests/tcg/ppc64le
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_POWER8_VECTOR),)
+ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
 PPC64LE_TESTS=bcdsub non_signalling_xscv
 endif
 $(PPC64LE_TESTS): CFLAGS += -mpower8-vector
 
-ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_POWER10),)
+ifneq ($(CROSS_CC_HAS_POWER10),)
 PPC64LE_TESTS += byte_reverse sha512-vector
 endif
 byte_reverse: CFLAGS += -mcpu=power10
-- 
2.31.1

[PATCH 09/15] tests/tcg: remove CONFIG_LINUX_USER from config-target.mak

[Paolo Bonzini]

Just check the target name instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/tcg/configure.sh              | 2 --
 tests/tcg/multiarch/Makefile.target | 2 +-
 tests/tcg/x86_64/Makefile.target    | 2 +-
 3 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 0162b2f6c4..b09956c14d 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -227,12 +227,10 @@ for target in $target_list; do
   case $target in
     *-linux-user)
       echo "CONFIG_USER_ONLY=y" >> $config_target_mak
-      echo "CONFIG_LINUX_USER=y" >> $config_target_mak
       echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
       ;;
     *-bsd-user)
       echo "CONFIG_USER_ONLY=y" >> $config_target_mak
-      echo "CONFIG_BSD_USER=y" >> $config_target_mak
       echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
       ;;
     *-softmmu)
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index dec401e67f..6bba523729 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -10,7 +10,7 @@ MULTIARCH_SRC=$(SRC_PATH)/tests/tcg/multiarch
 # Set search path for all sources
 VPATH 	       += $(MULTIARCH_SRC)
 MULTIARCH_SRCS =  $(notdir $(wildcard $(MULTIARCH_SRC)/*.c))
-ifneq ($(CONFIG_LINUX_USER),)
+ifeq ($(filter %-linux-user, $(TARGET)),$(TARGET))
 VPATH 	       += $(MULTIARCH_SRC)/linux
 MULTIARCH_SRCS += $(notdir $(wildcard $(MULTIARCH_SRC)/linux/*.c))
 endif
diff --git a/tests/tcg/x86_64/Makefile.target b/tests/tcg/x86_64/Makefile.target
index 17cf168f0a..f9fcd31caf 100644
--- a/tests/tcg/x86_64/Makefile.target
+++ b/tests/tcg/x86_64/Makefile.target
@@ -8,7 +8,7 @@
 
 include $(SRC_PATH)/tests/tcg/i386/Makefile.target
 
-ifneq ($(CONFIG_LINUX_USER),)
+ifeq ($(filter %-linux-user, $(TARGET)),$(TARGET))
 X86_64_TESTS += vsyscall
 TESTS=$(MULTIARCH_TESTS) $(X86_64_TESTS) test-x86_64
 else
-- 
2.31.1

[PATCH 10/15] tests/tcg: remove CONFIG_USER_ONLY from config-target.mak

[Paolo Bonzini]

Just check the target name instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/tcg/Makefile.target |  8 ++++----
 tests/tcg/configure.sh    | 12 +++---------
 2 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target
index ae8004c76e..083b09c742 100644
--- a/tests/tcg/Makefile.target
+++ b/tests/tcg/Makefile.target
@@ -32,7 +32,7 @@
 all:
 -include ../../../config-host.mak
 -include ../config-$(TARGET).mak
-ifeq ($(CONFIG_USER_ONLY),y)
+ifeq ($(filter %-softmmu, $(TARGET)),)
 -include $(SRC_PATH)/configs/targets/$(TARGET)/default.mak
 endif
 
@@ -42,7 +42,7 @@ COMMA := ,
 quiet-command = $(if $(V),$1,$(if $(2),@printf "  %-7s %s\n" $2 $3 && $1, @$1))
 
 # $1 = test name, $2 = cmd, $3 = desc
-ifdef CONFIG_USER_ONLY
+ifeq ($(filter %-softmmu, $(TARGET)),)
 run-test = $(call quiet-command, timeout --foreground $(TIMEOUT) $2 > $1.out, \
 	"TEST",$3)
 else
@@ -89,7 +89,7 @@ QEMU_OPTS=
 #   90s    with --enable-tcg-interpreter
 TIMEOUT=90
 
-ifdef CONFIG_USER_ONLY
+ifeq ($(filter %-softmmu, $(TARGET)),)
 # The order we include is important. We include multiarch first and
 # then the target. If there are common tests shared between
 # sub-targets (e.g. ARM & AArch64) then it is up to
@@ -151,7 +151,7 @@ extract-plugin = $(wordlist 2, 2, $(subst -with-, ,$1))
 
 RUN_TESTS+=$(EXTRA_RUNS)
 
-ifdef CONFIG_USER_ONLY
+ifeq ($(filter %-softmmu, $(TARGET)),)
 run-%: %
 	$(call run-test, $<, $(QEMU) $(QEMU_OPTS) $<, "$< on $(TARGET_NAME)")
 
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index b09956c14d..a17db8ce64 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -225,18 +225,12 @@ for target in $target_list; do
   echo "TARGET_NAME=$arch" >> $config_target_mak
   echo "target=$target" >> $config_target_mak
   case $target in
-    *-linux-user)
-      echo "CONFIG_USER_ONLY=y" >> $config_target_mak
-      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
-      ;;
-    *-bsd-user)
-      echo "CONFIG_USER_ONLY=y" >> $config_target_mak
-      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
-      ;;
     *-softmmu)
-      echo "CONFIG_SOFTMMU=y" >> $config_target_mak
       echo "QEMU=$PWD/qemu-system-$arch" >> $config_target_mak
       ;;
+    *)
+      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
+      ;;
   esac
 
   eval "target_compiler_cflags=\${cross_cc_cflags_$arch}"
-- 
2.31.1

[PATCH 11/15] tests/tcg: prepare Makefile.prereqs at configure time

[Paolo Bonzini]

List the dependencies of the build-tcg-tests-* and run-tcg-tests-*
targets in a Makefile fragment, without going through Makefile.prereqs's
"parsing" of config-*.mak.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include     |  9 ++++++---
 tests/tcg/Makefile.prereqs | 18 ------------------
 tests/tcg/configure.sh     | 11 +++++++++--
 3 files changed, 15 insertions(+), 23 deletions(-)
 delete mode 100644 tests/tcg/Makefile.prereqs

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 05c534ea56..b5d0d6bc98 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -39,14 +39,17 @@ SPEED = quick
 # Build up our target list from the filtered list of ninja targets
 TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
 
+-include tests/tcg/Makefile.prereqs
+config-host.mak: $(SRC_PATH)/tests/tcg/configure.sh
+tests/tcg/Makefile.prereqs: config-host.mak
+
 # Per guest TCG tests
 BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(TARGETS))
 CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(TARGETS))
 RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(TARGETS))
 
-# Probe for the Docker Builds needed for each build
-$(foreach PROBE_TARGET,$(TARGET_DIRS), 				\
-	$(eval -include $(SRC_PATH)/tests/tcg/Makefile.prereqs))
+$(foreach TARGET,$(TARGETS), \
+        $(eval $(BUILD_DIR)/tests/tcg/config-$(TARGET).mak: config-host.mak))
 
 $(BUILD_TCG_TARGET_RULES): build-tcg-tests-%: $(if $(CONFIG_PLUGIN),test-plugins)
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) \
diff --git a/tests/tcg/Makefile.prereqs b/tests/tcg/Makefile.prereqs
deleted file mode 100644
index 9a29604a83..0000000000
--- a/tests/tcg/Makefile.prereqs
+++ /dev/null
@@ -1,18 +0,0 @@
-# -*- Mode: makefile -*-
-#
-# TCG Compiler Probe
-#
-# This Makefile fragment is included multiple times in the main make
-# script to probe for available compilers. This is used to build up a
-# selection of required docker targets before we invoke a sub-make for
-# each target.
-
-DOCKER_IMAGE:=
-
--include $(BUILD_DIR)/tests/tcg/config-$(PROBE_TARGET).mak
-
-ifneq ($(DOCKER_IMAGE),)
-build-tcg-tests-$(PROBE_TARGET): docker-image-$(DOCKER_IMAGE)
-endif
-$(BUILD_DIR)/tests/tcg/config_$(PROBE_TARGET).mak: config-host.mak
-config-host.mak: $(SRC_PATH)/tests/tcg/configure.sh
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index a17db8ce64..991b905354 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -80,6 +80,8 @@ fi
 : ${cross_as_tricore="tricore-as"}
 : ${cross_ld_tricore="tricore-ld"}
 
+makefile=tests/tcg/Makefile.prereqs
+: > $makefile
 for target in $target_list; do
   arch=${target%%-*}
 
@@ -226,14 +228,17 @@ for target in $target_list; do
   echo "target=$target" >> $config_target_mak
   case $target in
     *-softmmu)
-      echo "QEMU=$PWD/qemu-system-$arch" >> $config_target_mak
+      qemu="qemu-system-$arch"
       ;;
     *)
-      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
+      qemu="qemu-$arch"
       ;;
   esac
 
+  echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
+
   eval "target_compiler_cflags=\${cross_cc_cflags_$arch}"
+  echo "QEMU=$PWD/$qemu" >> $config_target_mak
   echo "CROSS_CC_GUEST_CFLAGS=$target_compiler_cflags" >> $config_target_mak
 
   got_cross_cc=no
@@ -329,6 +334,7 @@ for target in $target_list; do
           test -n "$container_image"; then
       for host in $container_hosts; do
           if test "$host" = "$cpu"; then
+              echo "build-tcg-tests-$target: docker-image-$container_image" >> $makefile
               echo "DOCKER_IMAGE=$container_image" >> $config_target_mak
               echo "DOCKER_CROSS_CC_GUEST=$container_cross_cc" >> \
                    $config_target_mak
@@ -340,6 +346,7 @@ for target in $target_list; do
                   echo "DOCKER_CROSS_LD_GUEST=$container_cross_ld" >> \
                       $config_target_mak
               fi
+              break
           fi
       done
       case $target in
-- 
2.31.1

[PATCH 12/15] tests/tcg: list test targets in Makefile.prereqs

[Paolo Bonzini]

Omit the rules altogether for targets that do not have a compiler.
Makefile.qemu now is only invoked if the tests are actually built/run.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include  | 14 +++++++-------
 tests/tcg/Makefile.qemu | 11 -----------
 tests/tcg/configure.sh  | 14 ++++++++++----
 3 files changed, 17 insertions(+), 22 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index b5d0d6bc98..091ca8513f 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -36,19 +36,16 @@ export SRC_PATH
 
 SPEED = quick
 
-# Build up our target list from the filtered list of ninja targets
-TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
-
 -include tests/tcg/Makefile.prereqs
 config-host.mak: $(SRC_PATH)/tests/tcg/configure.sh
 tests/tcg/Makefile.prereqs: config-host.mak
 
 # Per guest TCG tests
-BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(TARGETS))
-CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(TARGETS))
-RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(TARGETS))
+BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(TCG_TESTS_TARGETS))
+CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(TCG_TESTS_TARGETS))
+RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(TCG_TESTS_TARGETS))
 
-$(foreach TARGET,$(TARGETS), \
+$(foreach TARGET,$(TCG_TESTS_TARGETS), \
         $(eval $(BUILD_DIR)/tests/tcg/config-$(TARGET).mak: config-host.mak))
 
 $(BUILD_TCG_TARGET_RULES): build-tcg-tests-%: $(if $(CONFIG_PLUGIN),test-plugins)
@@ -84,6 +81,9 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES)
 
 .PHONY: check-venv check-avocado check-acceptance check-acceptance-deprecated-warning
 
+# Build up our target list from the filtered list of ninja targets
+TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
+
 TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
 TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
 TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
diff --git a/tests/tcg/Makefile.qemu b/tests/tcg/Makefile.qemu
index 84c8543878..cda5e5a33e 100644
--- a/tests/tcg/Makefile.qemu
+++ b/tests/tcg/Makefile.qemu
@@ -95,7 +95,6 @@ all:
 
 .PHONY: guest-tests
 
-ifneq ($(GUEST_BUILD),)
 guest-tests: $(GUEST_BUILD)
 
 run-guest-tests: guest-tests
@@ -105,16 +104,6 @@ run-guest-tests: guest-tests
 	 		SRC_PATH="$(SRC_PATH)" SPEED=$(SPEED) run), \
 	"RUN", "tests for $(TARGET_NAME)")
 
-else
-guest-tests:
-	$(call quiet-command, true, "BUILD", \
-		"$(TARGET) guest-tests SKIPPED")
-
-run-guest-tests:
-	$(call quiet-command, true, "RUN", \
-		"tests for $(TARGET) SKIPPED")
-endif
-
 # It doesn't matter if these don't exits
 .PHONY: clean-guest-tests
 clean-guest-tests:
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 991b905354..05e1ac507b 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -82,6 +82,8 @@ fi
 
 makefile=tests/tcg/Makefile.prereqs
 : > $makefile
+
+tcg_tests_targets=
 for target in $target_list; do
   arch=${target%%-*}
 
@@ -228,6 +230,7 @@ for target in $target_list; do
   echo "target=$target" >> $config_target_mak
   case $target in
     *-softmmu)
+      test -f $source_path/tests/tcg/$arch/Makefile.softmmu-target || continue
       qemu="qemu-system-$arch"
       ;;
     *)
@@ -235,11 +238,7 @@ for target in $target_list; do
       ;;
   esac
 
-  echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
-
   eval "target_compiler_cflags=\${cross_cc_cflags_$arch}"
-  echo "QEMU=$PWD/$qemu" >> $config_target_mak
-  echo "CROSS_CC_GUEST_CFLAGS=$target_compiler_cflags" >> $config_target_mak
 
   got_cross_cc=no
 
@@ -365,5 +364,12 @@ for target in $target_list; do
               echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
               ;;
       esac
+  else
+      continue
   fi
+  echo "QEMU=$PWD/$qemu" >> $config_target_mak
+  echo "CROSS_CC_GUEST_CFLAGS=$target_compiler_cflags" >> $config_target_mak
+  echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
+  tcg_tests_targets="$tcg_tests_targets $target"
 done
+echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile
-- 
2.31.1

[PATCH 13/15] tests/tcg: invoke Makefile.target directly from QEMU's makefile

[Paolo Bonzini]

Build the "docker.py cc" invocation directly in tests/tcg/configure.sh, and
remove the Makefile.qemu wrapper around Makefile.target.  The config-*.mak
files now include the actual variables used when building the tests, rather
than the CROSS_* variables that Makefile.qemu used to "translate".

This is a first step towards generalizing the cross-compilation infrastructure
so that it can be used for firmware as well.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include    |  40 +++++++-------
 tests/tcg/Makefile.qemu   | 110 --------------------------------------
 tests/tcg/Makefile.target |   3 +-
 tests/tcg/configure.sh    |  28 +++++-----
 4 files changed, 35 insertions(+), 146 deletions(-)
 delete mode 100644 tests/tcg/Makefile.qemu

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 091ca8513f..ec84b2ebc0 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -48,25 +48,27 @@ RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(TCG_TESTS_TARGETS))
 $(foreach TARGET,$(TCG_TESTS_TARGETS), \
         $(eval $(BUILD_DIR)/tests/tcg/config-$(TARGET).mak: config-host.mak))
 
-$(BUILD_TCG_TARGET_RULES): build-tcg-tests-%: $(if $(CONFIG_PLUGIN),test-plugins)
-	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) \
-		-f $(SRC_PATH)/tests/tcg/Makefile.qemu \
-		SRC_PATH=$(SRC_PATH) \
-	       	V="$(V)" TARGET="$*" guest-tests, \
-		"BUILD", "TCG tests for $*")
-
-$(RUN_TCG_TARGET_RULES): run-tcg-tests-%: build-tcg-tests-% all
-	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) \
-		-f $(SRC_PATH)/tests/tcg/Makefile.qemu \
-		SRC_PATH=$(SRC_PATH) SPEED="$(SPEED)" \
-		V="$(V)" TARGET="$*" run-guest-tests, \
-		"RUN", "TCG tests for $*")
-
-$(CLEAN_TCG_TARGET_RULES): clean-tcg-tests-%:
-	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) \
-		-f $(SRC_PATH)/tests/tcg/Makefile.qemu \
-		SRC_PATH=$(SRC_PATH) TARGET="$*" clean-guest-tests, \
-		"CLEAN", "TCG tests for $*")
+.PHONY: $(TCG_TESTS_TARGETS:%=build-tcg-tests-%)
+$(TCG_TESTS_TARGETS:%=build-tcg-tests-%): build-tcg-tests-%: $(BUILD_DIR)/tests/tcg/config-%.mak
+	$(call quiet-command, \
+            $(MAKE) -C tests/tcg/$* -f ../Makefile.target $(SUBDIR_MAKEFLAGS) \
+                        DOCKER_SCRIPT="$(DOCKER_SCRIPT)" \
+                        TARGET="$*" SRC_PATH="$(SRC_PATH)", \
+        "BUILD","$* guest-tests")
+
+.PHONY: $(TCG_TESTS_TARGETS:%=run-tcg-tests-%)
+$(TCG_TESTS_TARGETS:%=run-tcg-tests-%): run-tcg-tests-%: build-tcg-tests-% $(if $(CONFIG_PLUGIN),test-plugins)
+	$(call quiet-command, \
+           $(MAKE) -C tests/tcg/$* -f ../Makefile.target $(SUBDIR_MAKEFLAGS) \
+                        TARGET="$*" SRC_PATH="$(SRC_PATH)" SPEED=$(SPEED) run, \
+        "RUN", "$* guest-tests")
+
+.PHONY: $(TCG_TESTS_TARGETS:%=clean-tcg-tests-%)
+$(TCG_TESTS_TARGETS:%=clean-tcg-tests-%): clean-tcg-tests-%:
+	$(call quiet-command, \
+           $(MAKE) -C tests/tcg/$* -f ../Makefile.target $(SUBDIR_MAKEFLAGS) \
+                        TARGET="$*" SRC_PATH="$(SRC_PATH)" clean, \
+        "CLEAN", "$* guest-tests")
 
 .PHONY: build-tcg
 build-tcg: $(BUILD_TCG_TARGET_RULES)
diff --git a/tests/tcg/Makefile.qemu b/tests/tcg/Makefile.qemu
deleted file mode 100644
index cda5e5a33e..0000000000
--- a/tests/tcg/Makefile.qemu
+++ /dev/null
@@ -1,110 +0,0 @@
-# -*- Mode: makefile -*-
-#
-# TCG tests (per-target rules)
-#
-# This Makefile fragment is included from the build-tcg target, once
-# for each target we build. We have two options for compiling, either
-# using a configured guest compiler or calling one of our docker images
-# to do it for us.
-#
-
-# The configure script fills in extra information about
-# useful docker images or alternative compiler flags.
-
-# Usage: $(call quiet-command,command and args,"NAME","args to print")
-# This will run "command and args", and either:
-#  if V=1 just print the whole command and args
-#  otherwise print the 'quiet' output in the format "  NAME     args to print"
-# NAME should be a short name of the command, 7 letters or fewer.
-# If called with only a single argument, will print nothing in quiet mode.
-quiet-command-run = $(if $(V),,$(if $2,printf "  %-7s %s\n" $2 $3 && ))$1
-quiet-@ = $(if $(V),,@)
-quiet-command = $(quiet-@)$(call quiet-command-run,$1,$2,$3)
-
-CROSS_CC_GUEST:=
-CROSS_AS_GUEST:=
-CROSS_LD_GUEST:=
-DOCKER_IMAGE:=
-
--include tests/tcg/config-$(TARGET).mak
-
-GUEST_BUILD=
-TCG_MAKE=../Makefile.target
-
-# We also need the Docker make rules to depend on
-SKIP_DOCKER_BUILD=1
-include $(SRC_PATH)/tests/docker/Makefile.include
-
-# Support installed Cross Compilers
-
-ifdef CROSS_CC_GUEST
-
-.PHONY: cross-build-guest-tests
-cross-build-guest-tests:
-	$(call quiet-command, \
-	   (mkdir -p tests/tcg/$(TARGET) && cd tests/tcg/$(TARGET) && \
-	    $(MAKE) -f $(TCG_MAKE) TARGET="$(TARGET)" CC="$(CROSS_CC_GUEST)" \
-			$(if $(CROSS_AS_GUEST),AS="$(CROSS_AS_GUEST)") \
-			$(if $(CROSS_LD_GUEST),LD="$(CROSS_LD_GUEST)") \
-			SRC_PATH="$(SRC_PATH)" BUILD_STATIC=$(CROSS_CC_GUEST_STATIC) \
-			EXTRA_CFLAGS="$(CROSS_CC_GUEST_CFLAGS)"), \
-	"BUILD","$(TARGET) guest-tests with $(CROSS_CC_GUEST)")
-
-GUEST_BUILD=cross-build-guest-tests
-
-endif
-
-# Support building with Docker
-
-ifneq ($(DOCKER_IMAGE),)
-
-DOCKER_COMPILE_CMD="$(DOCKER_SCRIPT) cc \
-		--cc $(DOCKER_CROSS_CC_GUEST) \
-		-i qemu/$(DOCKER_IMAGE) \
-		-s $(SRC_PATH) -- "
-
-DOCKER_AS_CMD=$(if $(DOCKER_CROSS_AS_GUEST),"$(DOCKER_SCRIPT) cc \
-		--cc $(DOCKER_CROSS_AS_GUEST) \
-		-i qemu/$(DOCKER_IMAGE) \
-		-s $(SRC_PATH) -- ")
-
-DOCKER_LD_CMD=$(if $(DOCKER_CROSS_LD_GUEST),"$(DOCKER_SCRIPT) cc \
-		--cc $(DOCKER_CROSS_LD_GUEST) \
-		-i qemu/$(DOCKER_IMAGE) \
-		-s $(SRC_PATH) -- ")
-
-
-.PHONY: docker-build-guest-tests
-docker-build-guest-tests: docker-image-$(DOCKER_IMAGE)
-	$(call quiet-command, \
-	  (mkdir -p tests/tcg/$(TARGET) && cd tests/tcg/$(TARGET) && \
-	   $(MAKE) -f $(TCG_MAKE) TARGET="$(TARGET)" CC=$(DOCKER_COMPILE_CMD) \
-			$(if $(DOCKER_AS_CMD),AS=$(DOCKER_AS_CMD)) \
-			$(if $(DOCKER_LD_CMD),LD=$(DOCKER_LD_CMD)) \
-			SRC_PATH="$(SRC_PATH)" BUILD_STATIC=y \
-			EXTRA_CFLAGS="$(CROSS_CC_GUEST_CFLAGS)"), \
-	"BUILD","$(TARGET) guest-tests with docker qemu/$(DOCKER_IMAGE)")
-
-GUEST_BUILD=docker-build-guest-tests
-
-endif
-
-# Final targets
-all:
-	@echo "Do not invoke this Makefile directly"; exit 1
-
-.PHONY: guest-tests
-
-guest-tests: $(GUEST_BUILD)
-
-run-guest-tests: guest-tests
-	$(call quiet-command, \
-	(cd tests/tcg/$(TARGET) && \
-	 $(MAKE) -f $(TCG_MAKE) TARGET="$(TARGET)" \
-	 		SRC_PATH="$(SRC_PATH)" SPEED=$(SPEED) run), \
-	"RUN", "tests for $(TARGET_NAME)")
-
-# It doesn't matter if these don't exits
-.PHONY: clean-guest-tests
-clean-guest-tests:
-	rm -rf tests/tcg/$(TARGET)
diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target
index 083b09c742..24543cc09f 100644
--- a/tests/tcg/Makefile.target
+++ b/tests/tcg/Makefile.target
@@ -185,4 +185,5 @@ gdb-%: %
 .PHONY: run
 run: $(RUN_TESTS)
 
-# There is no clean target, the calling make just rm's the tests build dir
+clean:
+	rm -f $(TESTS) *.o
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 05e1ac507b..5c95adfd0d 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -227,7 +227,6 @@ for target in $target_list; do
 
   echo "# Automatically generated by configure - do not modify" > $config_target_mak
   echo "TARGET_NAME=$arch" >> $config_target_mak
-  echo "target=$target" >> $config_target_mak
   case $target in
     *-softmmu)
       test -f $source_path/tests/tcg/$arch/Makefile.softmmu-target || continue
@@ -257,13 +256,13 @@ for target in $target_list; do
                   if do_compiler "$target_compiler" $target_compiler_cflags \
                                  -o $TMPE $TMPC ; then
                       got_cross_cc=yes
-                      echo "CROSS_CC_GUEST_STATIC=y" >> $config_target_mak
-                      echo "CROSS_CC_GUEST=$target_compiler" >> $config_target_mak
+                      echo "BUILD_STATIC=y" >> $config_target_mak
+                      echo "CC=$target_compiler" >> $config_target_mak
                   fi
               else
                   got_cross_cc=yes
-                  echo "CROSS_CC_GUEST_STATIC=y" >> $config_target_mak
-                  echo "CROSS_CC_GUEST=$target_compiler" >> $config_target_mak
+                  echo "BUILD_STATIC=y" >> $config_target_mak
+                  echo "CC=$target_compiler" >> $config_target_mak
               fi
           fi
       fi
@@ -274,9 +273,8 @@ for target in $target_list; do
       if has $target_as && has $target_ld; then
           case $target in
               tricore-softmmu)
-                  echo "CROSS_CC_GUEST=$target_as" >> $config_target_mak
-                  echo "CROSS_AS_GUEST=$target_as" >> $config_target_mak
-                  echo "CROSS_LD_GUEST=$target_ld" >> $config_target_mak
+                  echo "AS=$target_as" >> $config_target_mak
+                  echo "LD=$target_ld" >> $config_target_mak
                   got_cross_cc=yes
                   ;;
           esac
@@ -334,16 +332,13 @@ for target in $target_list; do
       for host in $container_hosts; do
           if test "$host" = "$cpu"; then
               echo "build-tcg-tests-$target: docker-image-$container_image" >> $makefile
-              echo "DOCKER_IMAGE=$container_image" >> $config_target_mak
-              echo "DOCKER_CROSS_CC_GUEST=$container_cross_cc" >> \
-                   $config_target_mak
+              echo "BUILD_STATIC=y" >> $config_target_mak
+              echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --" >> $config_target_mak
               if test -n "$container_cross_as"; then
-                  echo "DOCKER_CROSS_AS_GUEST=$container_cross_as" >> \
-                      $config_target_mak
+                  echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --" >> $config_target_mak
               fi
               if test -n "$container_cross_ld"; then
-                  echo "DOCKER_CROSS_LD_GUEST=$container_cross_ld" >> \
-                      $config_target_mak
+                  echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --" >> $config_target_mak
               fi
               break
           fi
@@ -367,8 +362,9 @@ for target in $target_list; do
   else
       continue
   fi
+  mkdir -p tests/tcg/$target
   echo "QEMU=$PWD/$qemu" >> $config_target_mak
-  echo "CROSS_CC_GUEST_CFLAGS=$target_compiler_cflags" >> $config_target_mak
+  echo "EXTRA_CFLAGS=$target_compiler_cflags" >> $config_target_mak
   echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
   tcg_tests_targets="$tcg_tests_targets $target"
 done
-- 
2.31.1

[PATCH 14/15] tests/docker: remove SKIP_DOCKER_BUILD

[Paolo Bonzini]

It is now unused.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/docker/Makefile.include | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e1bea20451..1b10585f28 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -33,15 +33,7 @@ $(DOCKER_SRC_COPY):
 
 docker-qemu-src: $(DOCKER_SRC_COPY)
 
-# General rule for building docker images. If we are a sub-make
-# invoked with SKIP_DOCKER_BUILD we still check the image is up to date
-# though
-ifdef SKIP_DOCKER_BUILD
-docker-image-%: $(DOCKER_FILES_DIR)/%.docker
-	$(call quiet-command, \
-		$(DOCKER_SCRIPT) check --quiet qemu/$* $<, \
-		"CHECK", "$*")
-else
+# General rule for building docker images.
 docker-image-%: $(DOCKER_FILES_DIR)/%.docker
 	$(call quiet-command,\
 		$(DOCKER_SCRIPT) build -t qemu/$* -f $< \
@@ -77,8 +69,6 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
 			"CHECK", "debian-$* exists"))
 
-endif
-
 # Enforce dependencies for composite images
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian10 debian11
-- 
2.31.1

[PATCH 15/15] tests/tcg: fix non-static build

[Paolo Bonzini]

If linking with -static fails at configure time, -static should not be used
at build time either.  Do not include BUILD_STATIC in $config_target_mak.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/tcg/configure.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 5c95adfd0d..f195d2d873 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -256,7 +256,6 @@ for target in $target_list; do
                   if do_compiler "$target_compiler" $target_compiler_cflags \
                                  -o $TMPE $TMPC ; then
                       got_cross_cc=yes
-                      echo "BUILD_STATIC=y" >> $config_target_mak
                       echo "CC=$target_compiler" >> $config_target_mak
                   fi
               else
-- 
2.31.1

Re: [PATCH 00/15] tests/docker and tests/tcg cleanup and diet

[Richard Henderson]

On 3/28/22 08:02, Paolo Bonzini wrote:
> This is also a first step towards moving the cross-compilation
> infrastructure from tests/tcg to all of QEMU, so that it can be
> used to build firmware binaries.

Yay!

However, the tricore special cases broke:

/home/rth/qemu/src/tests/docker/docker.py --engine auto build -t qemu/debian-tricore-cross 
-f /home/rth/qemu/src/tests/docker/dockerfiles/debian-tricore-cross.docker   --registry 
registry.gitlab.com/qemu-project/qemu --add-current-user
Image is up to date.
make -C tests/tcg/tricore-softmmu -f ../Makefile.target 
DOCKER_SCRIPT="/home/rth/qemu/src/tests/docker/docker.py --engine auto" 
TARGET="tricore-softmmu" SRC_PATH="/home/rth/qemu/src"
make[1]: Entering directory '/home/rth/qemu/bld/tests/tcg/tricore-softmmu'
cc -E -o test_abs.pS /home/rth/qemu/src/tests/tcg/tricore/test_abs.S
/home/rth/qemu/src/tests/docker/docker.py --engine auto cc --cc  -i 
qemu/debian-tricore-cross -s /home/rth/qemu/src --  -o test_abs.o test_abs.pS
usage: docker.py <subcommand> ... cc [-h] [--quiet] --image IMAGE [--cc CC] [--source-path 
[PATHS [PATHS ...]]]
docker.py <subcommand> ... cc: error: argument --cc: expected one argument
make[1]: *** [/home/rth/qemu/src/tests/tcg/tricore/Makefile.softmmu-target:23: test_abs.o] 
Error 2


r~

Re: [PATCH 01/15] tests/docker: remove dead code

[Alex Bennée]

Paolo Bonzini <pbonzini@redhat.com> writes:

> debian-powerpc-user-cross was the only linux-user powered cross builder
> and it was removed in commit 80394ccf21 ("tests/docker: remove
> debian-powerpc-user-cross", 2019-09-26). Remove all the infrastructure
> around it since it is now unused.

It doesn't remove it all - we still have the
docker-binfmt-image-debian-% rule (which I'd like to keep). Anyway:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 02/15] tests/docker: remove test targets

[Alex Bennée]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  tests/docker/Makefile.include | 18 ------------------
>  1 file changed, 18 deletions(-)
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index a6a5a20949..8248cfdb4f 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -99,24 +99,6 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
>  			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
>  			"CHECK", "debian-$* exists"))
>  
> -# These are test targets
> -USER_TCG_TARGETS=$(patsubst %-linux-user,qemu-%,$(filter %-linux-user,$(TARGET_DIRS)))
> -EXEC_COPY_TESTS=$(patsubst %,docker-exec-copy-test-%, $(USER_TCG_TARGETS))
> -
> -$(EXEC_COPY_TESTS): docker-exec-copy-test-%:
> $(DOCKER_FILES_DIR)/empty.docker

Should probably clean-up the empty.docker while you are at it. It's a
niche command but I wonder how we would copy new tests now?

> -	$(call quiet-command,							\
> -		$(DOCKER_SCRIPT) build -t qemu/exec-copy-test-$* -f $< 		\
> -			$(if $V,,--quiet) --no-cache 				\
> -			--include-executable=$*					\
> -			--skip-binfmt,						\
> -			"TEST","copy $* to container")
> -	$(call quiet-command,							\
> -		$(DOCKER_SCRIPT) run qemu/exec-copy-test-$* 			\
> -			/$* -version > tests/docker-exec-copy-test-$*.out,	\
> -			"TEST","check $* works in container")
> -
> -docker-exec-copy-test: $(EXEC_COPY_TESTS)
> -
>  endif
>  
>  # Enforce dependencies for composite images


-- 
Alex Bennée

Re: [PATCH 03/15] tests/docker: remove dead variable

[Alex Bennée]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 08/15] tests/tcg: add compiler test variables when using containers

[Alex Bennée]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Even for container-based cross compilation use $(CROSS_CC_HAS_*) variables.
> This makes the TCG test makefiles oblivious of whether the compiler is
> invoked through a container or not.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 00/15] tests/docker and tests/tcg cleanup and diet

[Paolo Bonzini]

On 3/28/22 17:53, Richard Henderson wrote:
> 
>> This is also a first step towards moving the cross-compilation
>> infrastructure from tests/tcg to all of QEMU, so that it can be
>> used to build firmware binaries.
> 
> Yay!
> 
> However, the tricore special cases broke:
Silly pasto:

diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index f195d2d873..c0022b47a6 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -334,7 +334,7 @@ for target in $target_list; do
                echo "BUILD_STATIC=y" >> $config_target_mak
                echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --" >> $config_target_mak
                if test -n "$container_cross_as"; then
-                  echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --" >> $config_target_mak
+                  echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --" >> $config_target_mak
                fi
                if test -n "$container_cross_ld"; then
                    echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --" >> $config_target_mak

Paolo

Re: [PATCH 01/15] tests/docker: remove dead code

[Paolo Bonzini]

On 3/28/22 18:18, Alex Bennée wrote:
> 
>> debian-powerpc-user-cross was the only linux-user powered cross builder
>> and it was removed in commit 80394ccf21 ("tests/docker: remove
>> debian-powerpc-user-cross", 2019-09-26). Remove all the infrastructure
>> around it since it is now unused.
> It doesn't remove it all - we still have the
> docker-binfmt-image-debian-% rule (which I'd like to keep). Anyway:
> 
> Reviewed-by: Alex Bennée<alex.bennee@linaro.org>

Yeah, I wasn't sure about that one.

Paolo

Re: [PATCH 02/15] tests/docker: remove test targets

[Paolo Bonzini]

On 3/28/22 18:44, Alex Bennée wrote:
> 
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>   tests/docker/Makefile.include | 18 ------------------
>>   1 file changed, 18 deletions(-)
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index a6a5a20949..8248cfdb4f 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -99,24 +99,6 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
>>   			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
>>   			"CHECK", "debian-$* exists"))
>>   
>> -# These are test targets
>> -USER_TCG_TARGETS=$(patsubst %-linux-user,qemu-%,$(filter %-linux-user,$(TARGET_DIRS)))
>> -EXEC_COPY_TESTS=$(patsubst %,docker-exec-copy-test-%, $(USER_TCG_TARGETS))
>> -
>> -$(EXEC_COPY_TESTS): docker-exec-copy-test-%:
>> $(DOCKER_FILES_DIR)/empty.docker
> 
> Should probably clean-up the empty.docker while you are at it. It's a
> niche command but I wonder how we would copy new tests now?

You can always document them as an alternative to this patch. :)

Paolo

Re: [PATCH 07/15] tests/docker: simplify docker-TEST@IMAGE targets

[Philippe Mathieu-Daudé]

On 28/3/22 16:02, Paolo Bonzini wrote:
> No need to go through the shell when we already have the test and images at
> the point where the targets are declared.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>   tests/docker/Makefile.include | 12 +++---------
>   1 file changed, 3 insertions(+), 9 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH 09/15] tests/tcg: remove CONFIG_LINUX_USER from config-target.mak

[Philippe Mathieu-Daudé]

On 28/3/22 16:02, Paolo Bonzini wrote:
> Just check the target name instead.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>   tests/tcg/configure.sh              | 2 --
>   tests/tcg/multiarch/Makefile.target | 2 +-
>   tests/tcg/x86_64/Makefile.target    | 2 +-
>   3 files changed, 2 insertions(+), 4 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH 10/15] tests/tcg: remove CONFIG_USER_ONLY from config-target.mak

[Philippe Mathieu-Daudé]

On 28/3/22 16:02, Paolo Bonzini wrote:
> Just check the target name instead.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>   tests/tcg/Makefile.target |  8 ++++----
>   tests/tcg/configure.sh    | 12 +++---------
>   2 files changed, 7 insertions(+), 13 deletions(-)

> diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
> index b09956c14d..a17db8ce64 100755
> --- a/tests/tcg/configure.sh
> +++ b/tests/tcg/configure.sh
> @@ -225,18 +225,12 @@ for target in $target_list; do
>     echo "TARGET_NAME=$arch" >> $config_target_mak
>     echo "target=$target" >> $config_target_mak
>     case $target in
> -    *-linux-user)
> -      echo "CONFIG_USER_ONLY=y" >> $config_target_mak
> -      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
> -      ;;
> -    *-bsd-user)
> -      echo "CONFIG_USER_ONLY=y" >> $config_target_mak
> -      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
> -      ;;
>       *-softmmu)
> -      echo "CONFIG_SOFTMMU=y" >> $config_target_mak
>         echo "QEMU=$PWD/qemu-system-$arch" >> $config_target_mak
>         ;;
> +    *)

Can we restrict to:

        *-user)

just in case?

> +      echo "QEMU=$PWD/qemu-$arch" >> $config_target_mak
> +      ;;
>     esac

With '*-user':
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH 04/15] tests/docker: remove unnecessary default definitions

[Alex Bennée]

Paolo Bonzini <pbonzini@redhat.com> writes:

> The definition of DOCKER_IMAGES and DOCKER_TESTS copes already with an
> empty value of $(IMAGES) and $(TESTS), no need to force them to "%" if
> undefined.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée