[Qemu-devel] [PATCH v1 0/2] docker DEF_TARGET_LIST cleanup

[Qemu-devel] [PATCH v1 0/2] docker DEF_TARGET_LIST cleanup

[Alex Bennée]

Hi,

While reviewing Markus' header cleanups I realised my advice to use:

  make docker-test-build

to run through all the various builds would miss some of the cases
that shippable catches. This is because the default target list is
hard-coded into the tests. This series allows it to be over-ridden by
each docker image by setting the DEF_TARGET_LIST env variable. The user
still has full control of the target list by calling:

  make docker-test-build TARGET_LIST=foo


Alex Bennée (2):
  tests/docker: move DEF_TARGET_LIST setting to common.rc
  tests/docker: set DEF_TARGET_LIST for some containers

 tests/docker/common.rc                                | 4 ++++
 tests/docker/dockerfiles/debian-arm64-cross.docker    | 1 +
 tests/docker/dockerfiles/debian-armel-cross.docker    | 1 +
 tests/docker/dockerfiles/debian-armhf-cross.docker    | 1 +
 tests/docker/dockerfiles/debian-mips-cross.docker     | 1 +
 tests/docker/dockerfiles/debian-mips64el-cross.docker | 1 +
 tests/docker/dockerfiles/debian-ppc64el-cross.docker  | 1 +
 tests/docker/dockerfiles/debian-s390x-cross.docker    | 1 +
 tests/docker/test-build                               | 1 -
 tests/docker/test-mingw                               | 1 -
 tests/docker/test-quick                               | 1 -
 11 files changed, 11 insertions(+), 3 deletions(-)

-- 
2.20.1

[Qemu-devel] [PATCH v1 1/2] tests/docker: move DEF_TARGET_LIST setting to common.rc

[Alex Bennée]

We might as well not repeat ourselves. At the same time allow it to be
overridden which we will use later from docker targets.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/common.rc  | 4 ++++
 tests/docker/test-build | 1 -
 tests/docker/test-mingw | 1 -
 tests/docker/test-quick | 1 -
 4 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/docker/common.rc b/tests/docker/common.rc
index 4011561587a..512202b0a19 100755
--- a/tests/docker/common.rc
+++ b/tests/docker/common.rc
@@ -11,6 +11,10 @@
 # or (at your option) any later version. See the COPYING file in
 # the top-level directory.
 
+# This might be set by ENV of a docker container... it is always
+# overriden by TARGET_LIST if the user sets it.
+DEF_TARGET_LIST=${DEF_TARGET_LIST:-"x86_64-softmmu,aarch64-softmmu"}
+
 requires()
 {
     for c in $@; do
diff --git a/tests/docker/test-build b/tests/docker/test-build
index 22766cfacc0..2b2a7832f15 100755
--- a/tests/docker/test-build
+++ b/tests/docker/test-build
@@ -15,7 +15,6 @@
 
 cd "$BUILD_DIR"
 
-DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
 build_qemu
 install_qemu
diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
index fdb1c2c879d..c30eb654eb7 100755
--- a/tests/docker/test-mingw
+++ b/tests/docker/test-mingw
@@ -16,7 +16,6 @@
 requires mingw dtc
 
 cd "$BUILD_DIR"
-DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 
 for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
     TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
diff --git a/tests/docker/test-quick b/tests/docker/test-quick
index eee59c55fba..feee44b2476 100755
--- a/tests/docker/test-quick
+++ b/tests/docker/test-quick
@@ -15,7 +15,6 @@
 
 cd "$BUILD_DIR"
 
-DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
 build_qemu
 check_qemu
-- 
2.20.1

[Qemu-devel] [PATCH v1 2/2] tests/docker: set DEF_TARGET_LIST for some containers

[Alex Bennée]

You can assume the failures most people are interested in are the
cross-compile failures that are specific to the cross compile target.
Set DEF_TARGET_LIST based on what we use for shippable, the user can
always override by calling with TARGET_LIST set.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/debian-arm64-cross.docker    | 1 +
 tests/docker/dockerfiles/debian-armel-cross.docker    | 1 +
 tests/docker/dockerfiles/debian-armhf-cross.docker    | 1 +
 tests/docker/dockerfiles/debian-mips-cross.docker     | 1 +
 tests/docker/dockerfiles/debian-mips64el-cross.docker | 1 +
 tests/docker/dockerfiles/debian-ppc64el-cross.docker  | 1 +
 tests/docker/dockerfiles/debian-s390x-cross.docker    | 1 +
 7 files changed, 7 insertions(+)

diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker
index 7c2cc93daf8..6b59ef0843a 100644
--- a/tests/docker/dockerfiles/debian-arm64-cross.docker
+++ b/tests/docker/dockerfiles/debian-arm64-cross.docker
@@ -17,6 +17,7 @@ RUN apt update && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=aarch64-linux-gnu-
+ENV DEF_TARGET_LIST aarch64-softmmu,aarch64-linux-user
 
 RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/docker/dockerfiles/debian-armel-cross.docker
index d866fe5d75b..363a162a253 100644
--- a/tests/docker/dockerfiles/debian-armel-cross.docker
+++ b/tests/docker/dockerfiles/debian-armel-cross.docker
@@ -15,6 +15,7 @@ RUN dpkg --add-architecture armel && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabi-
+ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
 
 RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
index 2b8627673c3..d5365881e79 100644
--- a/tests/docker/dockerfiles/debian-armhf-cross.docker
+++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
@@ -17,6 +17,7 @@ RUN apt update && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
+ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
 
 RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
diff --git a/tests/docker/dockerfiles/debian-mips-cross.docker b/tests/docker/dockerfiles/debian-mips-cross.docker
index 3c4d6f9ec1f..a54efa7253a 100644
--- a/tests/docker/dockerfiles/debian-mips-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips-cross.docker
@@ -20,6 +20,7 @@ RUN apt update && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mips-linux-gnu-
+ENV DEF_TARGET_LIST mips-softmmu,mipsel-linux-user
 
 # Install extra libraries to increase code coverage
 RUN apt update && \
diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker
index 34b0b828957..a1481e16e2c 100644
--- a/tests/docker/dockerfiles/debian-mips64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker
@@ -21,6 +21,7 @@ RUN apt update && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mips64el-linux-gnuabi64-
+ENV DEF_TARGET_LIST mips64el-softmmu,mips64el-linux-user
 
 # Install extra libraries to increase code coverage
 RUN apt update && \
diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
index fc056d7bc35..f10883c8188 100644
--- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
@@ -16,6 +16,7 @@ RUN apt update && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=powerpc64le-linux-gnu-
+ENV DEF_TARGET_LIST ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user
 
 # Install extra libraries to increase code coverage
 RUN apt update && \
diff --git a/tests/docker/dockerfiles/debian-s390x-cross.docker b/tests/docker/dockerfiles/debian-s390x-cross.docker
index 6732f9ec789..efa346007d6 100644
--- a/tests/docker/dockerfiles/debian-s390x-cross.docker
+++ b/tests/docker/dockerfiles/debian-s390x-cross.docker
@@ -21,6 +21,7 @@ RUN apt update && \
 
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=s390x-linux-gnu-
+ENV DEF_TARGET_LIST s390x-softmmu,s390x-linux-user
 
 # Install extra libraries to increase code coverage
 RUN apt update && \
-- 
2.20.1

Re: [Qemu-devel] [PATCH v1 1/2] tests/docker: move DEF_TARGET_LIST setting to common.rc

[Philippe Mathieu-Daudé]

On 8/9/19 5:50 PM, Alex Bennée wrote:
> We might as well not repeat ourselves. At the same time allow it to be
> overridden which we will use later from docker targets.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

> ---
>  tests/docker/common.rc  | 4 ++++
>  tests/docker/test-build | 1 -
>  tests/docker/test-mingw | 1 -
>  tests/docker/test-quick | 1 -
>  4 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/docker/common.rc b/tests/docker/common.rc
> index 4011561587a..512202b0a19 100755
> --- a/tests/docker/common.rc
> +++ b/tests/docker/common.rc
> @@ -11,6 +11,10 @@
>  # or (at your option) any later version. See the COPYING file in
>  # the top-level directory.
>  
> +# This might be set by ENV of a docker container... it is always
> +# overriden by TARGET_LIST if the user sets it.
> +DEF_TARGET_LIST=${DEF_TARGET_LIST:-"x86_64-softmmu,aarch64-softmmu"}
> +
>  requires()
>  {
>      for c in $@; do
> diff --git a/tests/docker/test-build b/tests/docker/test-build
> index 22766cfacc0..2b2a7832f15 100755
> --- a/tests/docker/test-build
> +++ b/tests/docker/test-build
> @@ -15,7 +15,6 @@
>  
>  cd "$BUILD_DIR"
>  
> -DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
>  TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>  build_qemu
>  install_qemu
> diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
> index fdb1c2c879d..c30eb654eb7 100755
> --- a/tests/docker/test-mingw
> +++ b/tests/docker/test-mingw
> @@ -16,7 +16,6 @@
>  requires mingw dtc
>  
>  cd "$BUILD_DIR"
> -DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
>  
>  for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
>      TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
> diff --git a/tests/docker/test-quick b/tests/docker/test-quick
> index eee59c55fba..feee44b2476 100755
> --- a/tests/docker/test-quick
> +++ b/tests/docker/test-quick
> @@ -15,7 +15,6 @@
>  
>  cd "$BUILD_DIR"
>  
> -DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
>  TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>  build_qemu
>  check_qemu
> 

Re: [Qemu-devel] [PATCH v1 2/2] tests/docker: set DEF_TARGET_LIST for some containers

[Philippe Mathieu-Daudé]

Hi Alex,

On 8/9/19 5:50 PM, Alex Bennée wrote:
> You can assume the failures most people are interested in are the
> cross-compile failures that are specific to the cross compile target.
> Set DEF_TARGET_LIST based on what we use for shippable, the user can
> always override by calling with TARGET_LIST set.

Some thoughts about this commit description:

I always wondered why .shippable.yml had this IMAGE/TARGET_LIST
relation, because while using TCG I'm always interested in the opposite,
compile for targets different than my host (except the endianess case,
where compiling for the opposite endianess might also be useful).

Now I can understand than when using KVM, one is interested in compiling
the same target than the host. However in the Docker cross-compilation
case we mostly can not test for KVM on the host...

Time for a host/os/target/feature/... CI matrix list on the wiki? :)