[Qemu-devel] [PATCH v4 0/4] Docker cross-compile targets and user build support

[Qemu-devel] [PATCH v4 0/4] Docker cross-compile targets and user build support

[Alex Bennée]

Hi Fam,

Here is the latest iteration of the docker changes incorporating your
suggestions:

  - -u behaves safely if user already added
  - cross containers specify QEMU_CONFIGURE_OPTS
  - merged docker and build automation in MAINTAINERS

Regards,

Alex Bennée (4):
  tests/docker: add basic user mapping support
  new: debian docker targets for cross-compiling
  .shippable.yml: new CI provider
  MAINTAINERS: merge Build and test automation with Docker tests

 .shippable.yml                                     | 21 ++++++++++++++++++
 MAINTAINERS                                        | 13 ++++++-----
 tests/docker/Makefile.include                      |  6 ++++++
 tests/docker/common.rc                             |  2 +-
 tests/docker/docker.py                             | 16 ++++++++++++--
 tests/docker/dockerfiles/debian-arm64-cross.docker | 15 +++++++++++++
 tests/docker/dockerfiles/debian-armhf-cross.docker | 15 +++++++++++++
 tests/docker/dockerfiles/debian.docker             | 25 ++++++++++++++++++++++
 8 files changed, 103 insertions(+), 10 deletions(-)
 create mode 100644 .shippable.yml
 create mode 100644 tests/docker/dockerfiles/debian-arm64-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian-armhf-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian.docker

-- 
2.11.0

[Qemu-devel] [PATCH v4 1/4] tests/docker: add basic user mapping support

[Alex Bennée]

Currently all docker builds are done by exporting a tarball to the
docker container and running the build as the containers root user.
Other use cases are possible however and it is possible to map a part
of users file-system to the container. This is useful for example for
doing cross-builds of arbitrary source trees. For this to work
smoothly the container needs to have a user created that maps cleanly
to the host system.

This adds a -u option to the docker script so that:

  DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
    -u --include-executable=arm-linux-user/qemu-arm \
    debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker

Will build a container that can then be run like:

  docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
    --user=alex:alex -w /src/ debian:armhf \
    sh -c "make clean && ./configure -s && make"

All docker containers built will add the current user unless
explicitly disabled by specifying NOUSER when invoking the Makefile:

  make docker-image-debian-armhf-cross NOUSER=1

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>

---
v2
  - write the useradd directly
  - change long option to --add-current-user
v3
  - images -> image's
  - add r-b
  - add USER to Makefile
v4
  - s/USER/NOUSER/ and default to on
  - fix the add-user code to skip if user already setup (for chained images)
---
 tests/docker/Makefile.include |  2 ++
 tests/docker/docker.py        | 16 ++++++++++++++--
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 3f15d5aea8..4778b27ca8 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
 	$(call quiet-command,\
 		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
 		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
+		$(if $(NOUSER),,--add-current-user) \
 		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
 		"BUILD","$*")
 
@@ -99,6 +100,7 @@ docker:
 	@echo '                         (default is 1)'
 	@echo '    DEBUG=1              Stop and drop to shell in the created container'
 	@echo '                         before running the command.'
+	@echo '    NOUSER=1		Disable adding current user to containers passwd.'
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 37d83199e7..d277a2268f 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -25,6 +25,7 @@ import signal
 from tarfile import TarFile, TarInfo
 from StringIO import StringIO
 from shutil import copy, rmtree
+from pwd import getpwuid
 
 
 DEVNULL = open(os.devnull, 'wb')
@@ -149,13 +150,21 @@ class Docker(object):
         labels = json.loads(resp)[0]["Config"].get("Labels", {})
         return labels.get("com.qemu.dockerfile-checksum", "")
 
-    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
+    def build_image(self, tag, docker_dir, dockerfile,
+                    quiet=True, user=False, argv=None):
         if argv == None:
             argv = []
 
         tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
         tmp_df.write(dockerfile)
 
+        if user:
+            uid = os.getuid()
+            uname = getpwuid(uid).pw_name
+            tmp_df.write("\n")
+            tmp_df.write("RUN id %s || useradd -u %d -U %s" %
+                         (uname, uid, uname))
+
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
                      _text_checksum(dockerfile))
@@ -225,6 +234,9 @@ class BuildCommand(SubCommand):
                             help="""Specify a binary that will be copied to the
                             container together with all its dependent
                             libraries""")
+        parser.add_argument("--add-current-user", "-u", dest="user",
+                            action="store_true",
+                            help="Add the current user to image's passwd")
         parser.add_argument("tag",
                             help="Image Tag")
         parser.add_argument("dockerfile",
@@ -261,7 +273,7 @@ class BuildCommand(SubCommand):
                                        docker_dir)
 
             dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, argv=argv)
+                            quiet=args.quiet, user=args.user, argv=argv)
 
             rmtree(docker_dir)
 
-- 
2.11.0

[Qemu-devel] [PATCH v4 2/4] new: debian docker targets for cross-compiling

[Alex Bennée]

This provides a basic Debian install with access to the emdebian cross
compilers. The debian-armhf-cross and debian-arm64-cross targets build
on the basic Debian image to allow cross compiling to those targets.

A new environment variable (QEMU_CONFIGURE_OPTS) is set as part of the
docker container and passed to the build to specify the
--cross-prefix. The user still calls the build in the usual way, for
example:

  make docker-test-build@debian-arm64-cross \
    TARGET_LIST="aarch64-softmmu,aarch64-linux-user"

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - add clang (keep shippable happy)
  - rm adduser code (done direct now)
  - add aptitude (useful for debugging package clashes)
v3
  - split into debian, debian-armhf-cross and debian-aarch64-cross
v4
  - Add QEMU_CONFIGURE_OPTS
---
 tests/docker/Makefile.include                      |  4 ++++
 tests/docker/common.rc                             |  2 +-
 tests/docker/dockerfiles/debian-arm64-cross.docker | 15 +++++++++++++
 tests/docker/dockerfiles/debian-armhf-cross.docker | 15 +++++++++++++
 tests/docker/dockerfiles/debian.docker             | 25 ++++++++++++++++++++++
 5 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/dockerfiles/debian-arm64-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian-armhf-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 4778b27ca8..84bdcc944a 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -54,6 +54,10 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
 		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
 		"BUILD","$*")
 
+# Enforce dependancies for composite images
+docker-image-debian-armhf-cross: docker-image-debian
+docker-image-debian-arm64-cross: docker-image-debian
+
 # Expand all the pre-requistes for each docker image and test combination
 $(foreach i,$(DOCKER_IMAGES), \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
diff --git a/tests/docker/common.rc b/tests/docker/common.rc
index 21657e87c6..6865689bb5 100755
--- a/tests/docker/common.rc
+++ b/tests/docker/common.rc
@@ -29,7 +29,7 @@ build_qemu()
     config_opts="--enable-werror \
                  ${TARGET_LIST:+--target-list=${TARGET_LIST}} \
                  --prefix=$PWD/install \
-                 $EXTRA_CONFIGURE_OPTS \
+                 $QEMU_CONFIGURE_OPTS $EXTRA_CONFIGURE_OPTS \
                  $@"
     echo "Configure options:"
     echo $config_opts
diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker
new file mode 100644
index 0000000000..ce90da943f
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-arm64-cross.docker
@@ -0,0 +1,15 @@
+#
+# Docker arm64 cross-compiler target
+#
+# This docker target builds on the base debian image.
+#
+FROM qemu:debian
+
+# Add the foreign architecture we want and install dependacies
+RUN dpkg --add-architecture arm64
+RUN apt update
+RUN apt install -yy crossbuild-essential-arm64
+RUN apt-get build-dep -yy -a arm64 qemu
+
+# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV QEMU_CONFIGURE_OPTS --cross-prefix=aarch64-linux-gnu-
diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
new file mode 100644
index 0000000000..e0f81a556d
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
@@ -0,0 +1,15 @@
+#
+# Docker armhf cross-compiler target
+#
+# This docker target builds on the base debian image.
+#
+FROM qemu:debian
+
+# Add the foreign architecture we want and install dependacies
+RUN dpkg --add-architecture armhf
+RUN apt update
+RUN apt install -yy crossbuild-essential-armhf
+RUN apt-get build-dep -yy -a armhf qemu
+
+# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
diff --git a/tests/docker/dockerfiles/debian.docker b/tests/docker/dockerfiles/debian.docker
new file mode 100644
index 0000000000..52bd79938e
--- /dev/null
+++ b/tests/docker/dockerfiles/debian.docker
@@ -0,0 +1,25 @@
+#
+# Docker multiarch cross-compiler target
+#
+# This docker target is builds on Debian and Emdebian's cross compiler targets
+# to build distro with a selection of cross compilers for building test binaries.
+#
+# On its own you can't build much but the docker-foo-cross targets
+# build on top of the base debian image.
+#
+FROM debian:stable-slim
+
+# Setup some basic tools we need
+RUN apt update
+RUN apt install -yy curl aptitude
+
+# Setup Emdebian
+RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >> /etc/apt/sources.list
+RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
+
+# Install common build utilities
+RUN apt update
+RUN apt install -yy build-essential clang
-- 
2.11.0

[Qemu-devel] [PATCH v4 3/4] .shippable.yml: new CI provider

[Alex Bennée]

Ostensibly Shippable offers a similar set of services as Travis.
However they are focused on Docker container based work-flows so we
can use our existing containers to run a few extra builds - in this
case a bunch of cross-compiled targets on a Debian multiarch system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v3
  - reduce matrix to armhf/arm64 which currently work
  - use the make docker-image-* build stanzas
  - add TARGET_LIST to each build
---
 .shippable.yml | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 .shippable.yml

diff --git a/.shippable.yml b/.shippable.yml
new file mode 100644
index 0000000000..4de8daf341
--- /dev/null
+++ b/.shippable.yml
@@ -0,0 +1,21 @@
+language: c
+env:
+  matrix:
+    - IMAGE=debian-armhf-cross
+      TARGET_LIST=arm-softmmu,arm-linux-user
+    - IMAGE=debian-arm64-cross
+      TARGET_LIST=aarch64-softmmu,aarch64-linux-user
+    - IMAGE=centos6
+      TARGET_LIST=i386-softmmu,x86_64-softmmu
+build:
+  pre_ci:
+    - make docker-image-${IMAGE}
+  pre_ci_boot:
+    image_name: qemu
+    image_tag: ${IMAGE}
+    pull: false
+    options: "-e HOME=/root"
+  ci:
+    - unset CC
+    - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
+    - make -j2
-- 
2.11.0

[Qemu-devel] [PATCH v4 4/4] MAINTAINERS: merge Build and test automation with Docker tests

[Alex Bennée]

The docker framework is really just another piece in the build
automation puzzle so lets merge it together. For added bonus I've also
included the Travis and Patchew status links. The Shippable links will
be added later once mainline tests have been configured and setup.

Suggested-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 MAINTAINERS | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7afbadaa15..4e4a3fb83c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1802,9 +1802,14 @@ F: docs/block-replication.txt
 Build and test automation
 -------------------------
 M: Alex Bennée <alex.bennee@linaro.org>
+M: Fam Zheng <famz@redhat.com>
 L: qemu-devel@nongnu.org
-S: Supported
+S: Maintained
 F: .travis.yml
+F: .shippable.yml
+F: tests/docker/
+W: https://travis-ci.org/qemu/qemu
+W: http://patchew.org/QEMU/
 
 Documentation
 -------------
@@ -1813,9 +1818,3 @@ M: Daniel P. Berrange <berrange@redhat.com>
 S: Odd Fixes
 F: docs/build-system.txt
 
-Docker testing
---------------
-Docker based testing framework and cases
-M: Fam Zheng <famz@redhat.com>
-S: Maintained
-F: tests/docker/
-- 
2.11.0

Re: [Qemu-devel] [PATCH v4 1/4] tests/docker: add basic user mapping support

[Fam Zheng]

On Thu, 02/16 12:34, Alex Bennée wrote:
> Currently all docker builds are done by exporting a tarball to the
> docker container and running the build as the containers root user.
> Other use cases are possible however and it is possible to map a part
> of users file-system to the container. This is useful for example for
> doing cross-builds of arbitrary source trees. For this to work
> smoothly the container needs to have a user created that maps cleanly
> to the host system.
> 
> This adds a -u option to the docker script so that:
> 
>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
>     -u --include-executable=arm-linux-user/qemu-arm \
>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
> 
> Will build a container that can then be run like:
> 
>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
>     --user=alex:alex -w /src/ debian:armhf \
>     sh -c "make clean && ./configure -s && make"
> 
> All docker containers built will add the current user unless
> explicitly disabled by specifying NOUSER when invoking the Makefile:
> 
>   make docker-image-debian-armhf-cross NOUSER=1
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Fam Zheng <famz@redhat.com>
> 
> ---
> v2
>   - write the useradd directly
>   - change long option to --add-current-user
> v3
>   - images -> image's
>   - add r-b
>   - add USER to Makefile
> v4
>   - s/USER/NOUSER/ and default to on
>   - fix the add-user code to skip if user already setup (for chained images)
> ---
>  tests/docker/Makefile.include |  2 ++
>  tests/docker/docker.py        | 16 ++++++++++++++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 3f15d5aea8..4778b27ca8 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>  	$(call quiet-command,\
>  		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
>  		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
> +		$(if $(NOUSER),,--add-current-user) \
>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>  		"BUILD","$*")
>  
> @@ -99,6 +100,7 @@ docker:
>  	@echo '                         (default is 1)'
>  	@echo '    DEBUG=1              Stop and drop to shell in the created container'
>  	@echo '                         before running the command.'
> +	@echo '    NOUSER=1		Disable adding current user to containers passwd.'

Please expand tabs in the middle into whitespaces.

>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>  
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index 37d83199e7..d277a2268f 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -25,6 +25,7 @@ import signal
>  from tarfile import TarFile, TarInfo
>  from StringIO import StringIO
>  from shutil import copy, rmtree
> +from pwd import getpwuid
>  
>  
>  DEVNULL = open(os.devnull, 'wb')
> @@ -149,13 +150,21 @@ class Docker(object):
>          labels = json.loads(resp)[0]["Config"].get("Labels", {})
>          return labels.get("com.qemu.dockerfile-checksum", "")
>  
> -    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
> +    def build_image(self, tag, docker_dir, dockerfile,
> +                    quiet=True, user=False, argv=None):
>          if argv == None:
>              argv = []
>  
>          tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
>          tmp_df.write(dockerfile)
>  
> +        if user:
> +            uid = os.getuid()
> +            uname = getpwuid(uid).pw_name
> +            tmp_df.write("\n")
> +            tmp_df.write("RUN id %s || useradd -u %d -U %s" %
> +                         (uname, uid, uname))

Please "2>/dev/null" the id command.

> +
>          tmp_df.write("\n")
>          tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
>                       _text_checksum(dockerfile))
> @@ -225,6 +234,9 @@ class BuildCommand(SubCommand):
>                              help="""Specify a binary that will be copied to the
>                              container together with all its dependent
>                              libraries""")
> +        parser.add_argument("--add-current-user", "-u", dest="user",
> +                            action="store_true",
> +                            help="Add the current user to image's passwd")
>          parser.add_argument("tag",
>                              help="Image Tag")
>          parser.add_argument("dockerfile",
> @@ -261,7 +273,7 @@ class BuildCommand(SubCommand):
>                                         docker_dir)
>  
>              dkr.build_image(tag, docker_dir, dockerfile,
> -                            quiet=args.quiet, argv=argv)
> +                            quiet=args.quiet, user=args.user, argv=argv)
>  
>              rmtree(docker_dir)
>  
> -- 
> 2.11.0
> 

Fam

Re: [Qemu-devel] [PATCH v4 2/4] new: debian docker targets for cross-compiling

[Fam Zheng]

On Thu, 02/16 12:34, Alex Bennée wrote:
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 4778b27ca8..84bdcc944a 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -54,6 +54,10 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>  		"BUILD","$*")
>  
> +# Enforce dependancies for composite images
> +docker-image-debian-armhf-cross: docker-image-debian
> +docker-image-debian-arm64-cross: docker-image-debian
> +

This is an interesting way to reuse parts, thanks!

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [PATCH v4 4/4] MAINTAINERS: merge Build and test automation with Docker tests

[Fam Zheng]

On Thu, 02/16 12:34, Alex Bennée wrote:
> The docker framework is really just another piece in the build
> automation puzzle so lets merge it together. For added bonus I've also
> included the Travis and Patchew status links. The Shippable links will
> be added later once mainline tests have been configured and setup.
> 
> Suggested-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  MAINTAINERS | 13 ++++++-------
>  1 file changed, 6 insertions(+), 7 deletions(-)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 7afbadaa15..4e4a3fb83c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1802,9 +1802,14 @@ F: docs/block-replication.txt
>  Build and test automation
>  -------------------------
>  M: Alex Bennée <alex.bennee@linaro.org>
> +M: Fam Zheng <famz@redhat.com>
>  L: qemu-devel@nongnu.org
> -S: Supported
> +S: Maintained

Yes:

> Supported:	Someone is actually paid to look after this.
> Maintained:	Someone actually looks after it.

While I'm paid to work on QEMU upstream, docker testing is not stricly my job
responsibility. So as for patchew. That's probably why I put 'maintained'
before.

>  F: .travis.yml
> +F: .shippable.yml

This line could be pulled into patch 3.

> +F: tests/docker/
> +W: https://travis-ci.org/qemu/qemu
> +W: http://patchew.org/QEMU/

Thanks for the link!

If you move the 'shippable.yml' line:

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [PATCH v4 3/4] .shippable.yml: new CI provider

[Fam Zheng]

On Thu, 02/16 12:34, Alex Bennée wrote:
> Ostensibly Shippable offers a similar set of services as Travis.
> However they are focused on Docker container based work-flows so we
> can use our existing containers to run a few extra builds - in this
> case a bunch of cross-compiled targets on a Debian multiarch system.

Out of curiosity, what are the forms of build report from shippable service?

> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v3
>   - reduce matrix to armhf/arm64 which currently work
>   - use the make docker-image-* build stanzas
>   - add TARGET_LIST to each build
> ---
>  .shippable.yml | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>  create mode 100644 .shippable.yml
> 
> diff --git a/.shippable.yml b/.shippable.yml
> new file mode 100644
> index 0000000000..4de8daf341
> --- /dev/null
> +++ b/.shippable.yml
> @@ -0,0 +1,21 @@
> +language: c
> +env:
> +  matrix:
> +    - IMAGE=debian-armhf-cross
> +      TARGET_LIST=arm-softmmu,arm-linux-user
> +    - IMAGE=debian-arm64-cross
> +      TARGET_LIST=aarch64-softmmu,aarch64-linux-user
> +    - IMAGE=centos6
> +      TARGET_LIST=i386-softmmu,x86_64-softmmu
> +build:
> +  pre_ci:
> +    - make docker-image-${IMAGE}
> +  pre_ci_boot:
> +    image_name: qemu
> +    image_tag: ${IMAGE}
> +    pull: false
> +    options: "-e HOME=/root"
> +  ci:
> +    - unset CC
> +    - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
> +    - make -j2

It's a good surprise that our docker infrastructure fits in the shippable
framework pretty nicely. :)

Looks good to me without cross checking the shippable API:

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [PATCH v4 0/4] Docker cross-compile targets and user build support

[Fam Zheng]

On Thu, 02/16 12:34, Alex Bennée wrote:
> Hi Fam,
> 
> Here is the latest iteration of the docker changes incorporating your
> suggestions:
> 
>   - -u behaves safely if user already added
>   - cross containers specify QEMU_CONFIGURE_OPTS
>   - merged docker and build automation in MAINTAINERS

Thanks for iterating.  I suggested a few minor changes, after which I think it's
time to merge!

Fam

Re: [Qemu-devel] [PATCH v4 3/4] .shippable.yml: new CI provider

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Thu, 02/16 12:34, Alex Bennée wrote:
>> Ostensibly Shippable offers a similar set of services as Travis.
>> However they are focused on Docker container based work-flows so we
>> can use our existing containers to run a few extra builds - in this
>> case a bunch of cross-compiled targets on a Debian multiarch system.
>
> Out of curiosity, what are the forms of build report from shippable
> service?

It sends out emails and there is the web status page:

https://app.shippable.com/runs/58a596c4266ed80f003a9385/summary

There is also support for build badges a.la Travis.

>
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>
>> ---
>> v3
>>   - reduce matrix to armhf/arm64 which currently work
>>   - use the make docker-image-* build stanzas
>>   - add TARGET_LIST to each build
>> ---
>>  .shippable.yml | 21 +++++++++++++++++++++
>>  1 file changed, 21 insertions(+)
>>  create mode 100644 .shippable.yml
>>
>> diff --git a/.shippable.yml b/.shippable.yml
>> new file mode 100644
>> index 0000000000..4de8daf341
>> --- /dev/null
>> +++ b/.shippable.yml
>> @@ -0,0 +1,21 @@
>> +language: c
>> +env:
>> +  matrix:
>> +    - IMAGE=debian-armhf-cross
>> +      TARGET_LIST=arm-softmmu,arm-linux-user
>> +    - IMAGE=debian-arm64-cross
>> +      TARGET_LIST=aarch64-softmmu,aarch64-linux-user
>> +    - IMAGE=centos6
>> +      TARGET_LIST=i386-softmmu,x86_64-softmmu
>> +build:
>> +  pre_ci:
>> +    - make docker-image-${IMAGE}
>> +  pre_ci_boot:
>> +    image_name: qemu
>> +    image_tag: ${IMAGE}
>> +    pull: false
>> +    options: "-e HOME=/root"
>> +  ci:
>> +    - unset CC
>> +    - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
>> +    - make -j2
>
> It's a good surprise that our docker infrastructure fits in the shippable
> framework pretty nicely. :)
>
> Looks good to me without cross checking the shippable API:
>
> Reviewed-by: Fam Zheng <famz@redhat.com>


--
Alex Bennée

Re: [Qemu-devel] [PATCH v4 2/4] new: debian docker targets for cross-compiling

[Philippe Mathieu-Daudé]

On 02/16/2017 09:34 AM, Alex Bennée wrote:
> This provides a basic Debian install with access to the emdebian cross
> compilers. The debian-armhf-cross and debian-arm64-cross targets build
> on the basic Debian image to allow cross compiling to those targets.
>
> A new environment variable (QEMU_CONFIGURE_OPTS) is set as part of the
> docker container and passed to the build to specify the
> --cross-prefix. The user still calls the build in the usual way, for
> example:
>
>   make docker-test-build@debian-arm64-cross \
>     TARGET_LIST="aarch64-softmmu,aarch64-linux-user"
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
> v2
>   - add clang (keep shippable happy)
>   - rm adduser code (done direct now)
>   - add aptitude (useful for debugging package clashes)
> v3
>   - split into debian, debian-armhf-cross and debian-aarch64-cross
> v4
>   - Add QEMU_CONFIGURE_OPTS
> ---
>  tests/docker/Makefile.include                      |  4 ++++
>  tests/docker/common.rc                             |  2 +-
>  tests/docker/dockerfiles/debian-arm64-cross.docker | 15 +++++++++++++
>  tests/docker/dockerfiles/debian-armhf-cross.docker | 15 +++++++++++++
>  tests/docker/dockerfiles/debian.docker             | 25 ++++++++++++++++++++++
>  5 files changed, 60 insertions(+), 1 deletion(-)
>  create mode 100644 tests/docker/dockerfiles/debian-arm64-cross.docker
>  create mode 100644 tests/docker/dockerfiles/debian-armhf-cross.docker
>  create mode 100644 tests/docker/dockerfiles/debian.docker
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 4778b27ca8..84bdcc944a 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -54,6 +54,10 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>  		"BUILD","$*")
>
> +# Enforce dependancies for composite images
> +docker-image-debian-armhf-cross: docker-image-debian
> +docker-image-debian-arm64-cross: docker-image-debian
> +
>  # Expand all the pre-requistes for each docker image and test combination
>  $(foreach i,$(DOCKER_IMAGES), \
>  	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
> diff --git a/tests/docker/common.rc b/tests/docker/common.rc
> index 21657e87c6..6865689bb5 100755
> --- a/tests/docker/common.rc
> +++ b/tests/docker/common.rc
> @@ -29,7 +29,7 @@ build_qemu()
>      config_opts="--enable-werror \
>                   ${TARGET_LIST:+--target-list=${TARGET_LIST}} \
>                   --prefix=$PWD/install \
> -                 $EXTRA_CONFIGURE_OPTS \
> +                 $QEMU_CONFIGURE_OPTS $EXTRA_CONFIGURE_OPTS \
>                   $@"
>      echo "Configure options:"
>      echo $config_opts
> diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker
> new file mode 100644
> index 0000000000..ce90da943f
> --- /dev/null
> +++ b/tests/docker/dockerfiles/debian-arm64-cross.docker
> @@ -0,0 +1,15 @@
> +#
> +# Docker arm64 cross-compiler target
> +#
> +# This docker target builds on the base debian image.
> +#
> +FROM qemu:debian
> +
> +# Add the foreign architecture we want and install dependacies

typo "dependencies"

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> +RUN dpkg --add-architecture arm64
> +RUN apt update
> +RUN apt install -yy crossbuild-essential-arm64
> +RUN apt-get build-dep -yy -a arm64 qemu
> +
> +# Specify the cross prefix for this image (see tests/docker/common.rc)
> +ENV QEMU_CONFIGURE_OPTS --cross-prefix=aarch64-linux-gnu-
> diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
> new file mode 100644
> index 0000000000..e0f81a556d
> --- /dev/null
> +++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
> @@ -0,0 +1,15 @@
> +#
> +# Docker armhf cross-compiler target
> +#
> +# This docker target builds on the base debian image.
> +#
> +FROM qemu:debian
> +
> +# Add the foreign architecture we want and install dependacies
> +RUN dpkg --add-architecture armhf
> +RUN apt update
> +RUN apt install -yy crossbuild-essential-armhf
> +RUN apt-get build-dep -yy -a armhf qemu
> +
> +# Specify the cross prefix for this image (see tests/docker/common.rc)
> +ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
> diff --git a/tests/docker/dockerfiles/debian.docker b/tests/docker/dockerfiles/debian.docker
> new file mode 100644
> index 0000000000..52bd79938e
> --- /dev/null
> +++ b/tests/docker/dockerfiles/debian.docker
> @@ -0,0 +1,25 @@
> +#
> +# Docker multiarch cross-compiler target
> +#
> +# This docker target is builds on Debian and Emdebian's cross compiler targets
> +# to build distro with a selection of cross compilers for building test binaries.
> +#
> +# On its own you can't build much but the docker-foo-cross targets
> +# build on top of the base debian image.
> +#
> +FROM debian:stable-slim
> +
> +# Setup some basic tools we need
> +RUN apt update
> +RUN apt install -yy curl aptitude
> +
> +# Setup Emdebian
> +RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >> /etc/apt/sources.list
> +RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
> +
> +# Duplicate deb line as deb-src
> +RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
> +
> +# Install common build utilities
> +RUN apt update
> +RUN apt install -yy build-essential clang
>

Re: [Qemu-devel] [PATCH v4 1/4] tests/docker: add basic user mapping support

[Philippe Mathieu-Daudé]

Hi Alex,

I first tried "make docker-image-debian-armhf-cross NOUSER=1 V=1" which 
worked fine, then "make docker-image-debian-armhf-cross NOUSER=0" but 
got a "Image is up to date." I thought I should have to remove the image 
manually so I typed "docker rmi qemu:debian-armhf-cross" and tried again 
but still no change in my 
tests/docker/dockerfiles/debian-armhf-cross.docker adding my username, 
then I reviewed your change in the Makefile and got it, setting NOUSER 
to '0' has the same behavior, it's just set. To build the image with my 
user I had to _not_ use the flag or use it unset "NOUSER=" and it worked 
like charm.


On 02/16/2017 09:34 AM, Alex Bennée wrote:
> Currently all docker builds are done by exporting a tarball to the
> docker container and running the build as the containers root user.
> Other use cases are possible however and it is possible to map a part
> of users file-system to the container. This is useful for example for
> doing cross-builds of arbitrary source trees. For this to work
> smoothly the container needs to have a user created that maps cleanly
> to the host system.
>
> This adds a -u option to the docker script so that:
>
>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
>     -u --include-executable=arm-linux-user/qemu-arm \
>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
>
> Will build a container that can then be run like:
>
>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
>     --user=alex:alex -w /src/ debian:armhf \
>     sh -c "make clean && ./configure -s && make"
>
> All docker containers built will add the current user unless
> explicitly disabled by specifying NOUSER when invoking the Makefile:
>
>   make docker-image-debian-armhf-cross NOUSER=1
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Fam Zheng <famz@redhat.com>

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> ---
> v2
>   - write the useradd directly
>   - change long option to --add-current-user
> v3
>   - images -> image's
>   - add r-b
>   - add USER to Makefile
> v4
>   - s/USER/NOUSER/ and default to on
>   - fix the add-user code to skip if user already setup (for chained images)
> ---
>  tests/docker/Makefile.include |  2 ++
>  tests/docker/docker.py        | 16 ++++++++++++++--
>  2 files changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 3f15d5aea8..4778b27ca8 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>  	$(call quiet-command,\
>  		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
>  		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
> +		$(if $(NOUSER),,--add-current-user) \
>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>  		"BUILD","$*")
>
> @@ -99,6 +100,7 @@ docker:
>  	@echo '                         (default is 1)'
>  	@echo '    DEBUG=1              Stop and drop to shell in the created container'
>  	@echo '                         before running the command.'
> +	@echo '    NOUSER=1		Disable adding current user to containers passwd.'
>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index 37d83199e7..d277a2268f 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -25,6 +25,7 @@ import signal
>  from tarfile import TarFile, TarInfo
>  from StringIO import StringIO
>  from shutil import copy, rmtree
> +from pwd import getpwuid
>
>
>  DEVNULL = open(os.devnull, 'wb')
> @@ -149,13 +150,21 @@ class Docker(object):
>          labels = json.loads(resp)[0]["Config"].get("Labels", {})
>          return labels.get("com.qemu.dockerfile-checksum", "")
>
> -    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
> +    def build_image(self, tag, docker_dir, dockerfile,
> +                    quiet=True, user=False, argv=None):
>          if argv == None:
>              argv = []
>
>          tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
>          tmp_df.write(dockerfile)
>
> +        if user:
> +            uid = os.getuid()
> +            uname = getpwuid(uid).pw_name
> +            tmp_df.write("\n")
> +            tmp_df.write("RUN id %s || useradd -u %d -U %s" %
> +                         (uname, uid, uname))
> +
>          tmp_df.write("\n")
>          tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
>                       _text_checksum(dockerfile))
> @@ -225,6 +234,9 @@ class BuildCommand(SubCommand):
>                              help="""Specify a binary that will be copied to the
>                              container together with all its dependent
>                              libraries""")
> +        parser.add_argument("--add-current-user", "-u", dest="user",
> +                            action="store_true",
> +                            help="Add the current user to image's passwd")
>          parser.add_argument("tag",
>                              help="Image Tag")
>          parser.add_argument("dockerfile",
> @@ -261,7 +273,7 @@ class BuildCommand(SubCommand):
>                                         docker_dir)
>
>              dkr.build_image(tag, docker_dir, dockerfile,
> -                            quiet=args.quiet, argv=argv)
> +                            quiet=args.quiet, user=args.user, argv=argv)
>
>              rmtree(docker_dir)
>
>

Re: [Qemu-devel] [PATCH v4 1/4] tests/docker: add basic user mapping support

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Hi Alex,
>
> I first tried "make docker-image-debian-armhf-cross NOUSER=1 V=1"
> which worked fine, then "make docker-image-debian-armhf-cross
> NOUSER=0" but got a "Image is up to date." I thought I should have to
> remove the image manually so I typed "docker rmi
> qemu:debian-armhf-cross" and tried again but still no change in my
> tests/docker/dockerfiles/debian-armhf-cross.docker adding my username,
> then I reviewed your change in the Makefile and got it, setting NOUSER
> to '0' has the same behavior, it's just set. To build the image with
> my user I had to _not_ use the flag or use it unset "NOUSER=" and it
> worked like charm.

Ahh yes. I followed the same pattern as NOCACHE which does the same
thing. I've made it clearer:

	@echo '    NOUSER               Define to disable adding current user to containers passwd.'

>
>
> On 02/16/2017 09:34 AM, Alex Bennée wrote:
>> Currently all docker builds are done by exporting a tarball to the
>> docker container and running the build as the containers root user.
>> Other use cases are possible however and it is possible to map a part
>> of users file-system to the container. This is useful for example for
>> doing cross-builds of arbitrary source trees. For this to work
>> smoothly the container needs to have a user created that maps cleanly
>> to the host system.
>>
>> This adds a -u option to the docker script so that:
>>
>>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
>>     -u --include-executable=arm-linux-user/qemu-arm \
>>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
>>
>> Will build a container that can then be run like:
>>
>>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
>>     --user=alex:alex -w /src/ debian:armhf \
>>     sh -c "make clean && ./configure -s && make"
>>
>> All docker containers built will add the current user unless
>> explicitly disabled by specifying NOUSER when invoking the Makefile:
>>
>>   make docker-image-debian-armhf-cross NOUSER=1
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Reviewed-by: Fam Zheng <famz@redhat.com>
>
> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Thanks

>
>> ---
>> v2
>>   - write the useradd directly
>>   - change long option to --add-current-user
>> v3
>>   - images -> image's
>>   - add r-b
>>   - add USER to Makefile
>> v4
>>   - s/USER/NOUSER/ and default to on
>>   - fix the add-user code to skip if user already setup (for chained images)
>> ---
>>  tests/docker/Makefile.include |  2 ++
>>  tests/docker/docker.py        | 16 ++++++++++++++--
>>  2 files changed, 16 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 3f15d5aea8..4778b27ca8 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>>  	$(call quiet-command,\
>>  		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
>>  		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
>> +		$(if $(NOUSER),,--add-current-user) \
>>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>>  		"BUILD","$*")
>>
>> @@ -99,6 +100,7 @@ docker:
>>  	@echo '                         (default is 1)'
>>  	@echo '    DEBUG=1              Stop and drop to shell in the created container'
>>  	@echo '                         before running the command.'
>> +	@echo '    NOUSER=1		Disable adding current user to containers passwd.'
>>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>>
>> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
>> index 37d83199e7..d277a2268f 100755
>> --- a/tests/docker/docker.py
>> +++ b/tests/docker/docker.py
>> @@ -25,6 +25,7 @@ import signal
>>  from tarfile import TarFile, TarInfo
>>  from StringIO import StringIO
>>  from shutil import copy, rmtree
>> +from pwd import getpwuid
>>
>>
>>  DEVNULL = open(os.devnull, 'wb')
>> @@ -149,13 +150,21 @@ class Docker(object):
>>          labels = json.loads(resp)[0]["Config"].get("Labels", {})
>>          return labels.get("com.qemu.dockerfile-checksum", "")
>>
>> -    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
>> +    def build_image(self, tag, docker_dir, dockerfile,
>> +                    quiet=True, user=False, argv=None):
>>          if argv == None:
>>              argv = []
>>
>>          tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
>>          tmp_df.write(dockerfile)
>>
>> +        if user:
>> +            uid = os.getuid()
>> +            uname = getpwuid(uid).pw_name
>> +            tmp_df.write("\n")
>> +            tmp_df.write("RUN id %s || useradd -u %d -U %s" %
>> +                         (uname, uid, uname))
>> +
>>          tmp_df.write("\n")
>>          tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
>>                       _text_checksum(dockerfile))
>> @@ -225,6 +234,9 @@ class BuildCommand(SubCommand):
>>                              help="""Specify a binary that will be copied to the
>>                              container together with all its dependent
>>                              libraries""")
>> +        parser.add_argument("--add-current-user", "-u", dest="user",
>> +                            action="store_true",
>> +                            help="Add the current user to image's passwd")
>>          parser.add_argument("tag",
>>                              help="Image Tag")
>>          parser.add_argument("dockerfile",
>> @@ -261,7 +273,7 @@ class BuildCommand(SubCommand):
>>                                         docker_dir)
>>
>>              dkr.build_image(tag, docker_dir, dockerfile,
>> -                            quiet=args.quiet, argv=argv)
>> +                            quiet=args.quiet, user=args.user, argv=argv)
>>
>>              rmtree(docker_dir)
>>
>>


-- 
Alex Bennée

Re: [Qemu-devel] [PATCH v4 2/4] new: debian docker targets for cross-compiling

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 02/16/2017 09:34 AM, Alex Bennée wrote:
>> This provides a basic Debian install with access to the emdebian cross
>> compilers. The debian-armhf-cross and debian-arm64-cross targets build
>> on the basic Debian image to allow cross compiling to those targets.
>>
>> A new environment variable (QEMU_CONFIGURE_OPTS) is set as part of the
>> docker container and passed to the build to specify the
>> --cross-prefix. The user still calls the build in the usual way, for
>> example:
>>
>>   make docker-test-build@debian-arm64-cross \
>>     TARGET_LIST="aarch64-softmmu,aarch64-linux-user"
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>
>> ---
>> v2
>>   - add clang (keep shippable happy)
>>   - rm adduser code (done direct now)
>>   - add aptitude (useful for debugging package clashes)
>> v3
>>   - split into debian, debian-armhf-cross and debian-aarch64-cross
>> v4
>>   - Add QEMU_CONFIGURE_OPTS
>> ---
>>  tests/docker/Makefile.include                      |  4 ++++
>>  tests/docker/common.rc                             |  2 +-
>>  tests/docker/dockerfiles/debian-arm64-cross.docker | 15 +++++++++++++
>>  tests/docker/dockerfiles/debian-armhf-cross.docker | 15 +++++++++++++
>>  tests/docker/dockerfiles/debian.docker             | 25 ++++++++++++++++++++++
>>  5 files changed, 60 insertions(+), 1 deletion(-)
>>  create mode 100644 tests/docker/dockerfiles/debian-arm64-cross.docker
>>  create mode 100644 tests/docker/dockerfiles/debian-armhf-cross.docker
>>  create mode 100644 tests/docker/dockerfiles/debian.docker
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 4778b27ca8..84bdcc944a 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -54,6 +54,10 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>>  		"BUILD","$*")
>>
>> +# Enforce dependancies for composite images
>> +docker-image-debian-armhf-cross: docker-image-debian
>> +docker-image-debian-arm64-cross: docker-image-debian
>> +
>>  # Expand all the pre-requistes for each docker image and test combination
>>  $(foreach i,$(DOCKER_IMAGES), \
>>  	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
>> diff --git a/tests/docker/common.rc b/tests/docker/common.rc
>> index 21657e87c6..6865689bb5 100755
>> --- a/tests/docker/common.rc
>> +++ b/tests/docker/common.rc
>> @@ -29,7 +29,7 @@ build_qemu()
>>      config_opts="--enable-werror \
>>                   ${TARGET_LIST:+--target-list=${TARGET_LIST}} \
>>                   --prefix=$PWD/install \
>> -                 $EXTRA_CONFIGURE_OPTS \
>> +                 $QEMU_CONFIGURE_OPTS $EXTRA_CONFIGURE_OPTS \
>>                   $@"
>>      echo "Configure options:"
>>      echo $config_opts
>> diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker
>> new file mode 100644
>> index 0000000000..ce90da943f
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/debian-arm64-cross.docker
>> @@ -0,0 +1,15 @@
>> +#
>> +# Docker arm64 cross-compiler target
>> +#
>> +# This docker target builds on the base debian image.
>> +#
>> +FROM qemu:debian
>> +
>> +# Add the foreign architecture we want and install dependacies
>
> typo "dependencies"

Fixed, thanks.

>
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
>> +RUN dpkg --add-architecture arm64
>> +RUN apt update
>> +RUN apt install -yy crossbuild-essential-arm64
>> +RUN apt-get build-dep -yy -a arm64 qemu
>> +
>> +# Specify the cross prefix for this image (see tests/docker/common.rc)
>> +ENV QEMU_CONFIGURE_OPTS --cross-prefix=aarch64-linux-gnu-
>> diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
>> new file mode 100644
>> index 0000000000..e0f81a556d
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
>> @@ -0,0 +1,15 @@
>> +#
>> +# Docker armhf cross-compiler target
>> +#
>> +# This docker target builds on the base debian image.
>> +#
>> +FROM qemu:debian
>> +
>> +# Add the foreign architecture we want and install dependacies
>> +RUN dpkg --add-architecture armhf
>> +RUN apt update
>> +RUN apt install -yy crossbuild-essential-armhf
>> +RUN apt-get build-dep -yy -a armhf qemu
>> +
>> +# Specify the cross prefix for this image (see tests/docker/common.rc)
>> +ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
>> diff --git a/tests/docker/dockerfiles/debian.docker b/tests/docker/dockerfiles/debian.docker
>> new file mode 100644
>> index 0000000000..52bd79938e
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/debian.docker
>> @@ -0,0 +1,25 @@
>> +#
>> +# Docker multiarch cross-compiler target
>> +#
>> +# This docker target is builds on Debian and Emdebian's cross compiler targets
>> +# to build distro with a selection of cross compilers for building test binaries.
>> +#
>> +# On its own you can't build much but the docker-foo-cross targets
>> +# build on top of the base debian image.
>> +#
>> +FROM debian:stable-slim
>> +
>> +# Setup some basic tools we need
>> +RUN apt update
>> +RUN apt install -yy curl aptitude
>> +
>> +# Setup Emdebian
>> +RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >> /etc/apt/sources.list
>> +RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
>> +
>> +# Duplicate deb line as deb-src
>> +RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
>> +
>> +# Install common build utilities
>> +RUN apt update
>> +RUN apt install -yy build-essential clang
>>


-- 
Alex Bennée