[Qemu-devel] [PATCH v2 1/7] .travis.yml: don't specify CONFIG twice

[Qemu-devel] [PATCH v2 1/7] .travis.yml: don't specify CONFIG twice

[Alex Bennée]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index d83e2d493b..3dc1e6439d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -104,8 +104,7 @@ matrix:
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
     # Trusty build with latest stable clang
-    - env: CONFIG=""
-      sudo: required
+    - sudo: required
       addons:
       dist: trusty
       language: generic
-- 
2.11.0

[Qemu-devel] [PATCH v2 2/7] .travis.yml: split VM based builds

[Alex Bennée]

The Trusty based builds run a little slower than the main container
based ones. This is also true for the latest version of Clang. The
builds are getting very close (and occasionally run over) the 50 minute
timeout. Rather than partitioning by target I just split them into
linux-user and system builds.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 38 ++++++++++++++++++++++++++++++++++----
 1 file changed, 34 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 3dc1e6439d..9008a796f0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -92,8 +92,19 @@ matrix:
     - env: CONFIG=""
       os: osx
       compiler: clang
-    # Plain Trusty Build
-    - env: CONFIG=""
+    # Plain Trusty System Build
+    - env: CONFIG="--disable-linux-user"
+      sudo: required
+      addons:
+      dist: trusty
+      compiler: gcc
+      before_install:
+        - sudo apt-get update -qq
+        - sudo apt-get build-dep -qq qemu
+        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
+        - git submodule update --init --recursive
+    # Plain Trusty Linux User Build
+    - env: CONFIG="--disable-system"
       sudo: required
       addons:
       dist: trusty
@@ -103,7 +114,26 @@ matrix:
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
-    # Trusty build with latest stable clang
+    # Trusty System build with latest stable clang
+    - sudo: required
+      addons:
+      dist: trusty
+      language: generic
+      compiler: none
+      env:
+        - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
+        - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
+      before_install:
+        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - sudo apt-get update -qq
+        - sudo apt-get install -qq -y clang-3.9
+        - sudo apt-get build-dep -qq qemu
+        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
+        - git submodule update --init --recursive
+      before_script:
+        - ./configure ${CONFIG} || cat config.log
+    # Trusty Linux User build with latest stable clang
     - sudo: required
       addons:
       dist: trusty
@@ -111,7 +141,7 @@ matrix:
       compiler: none
       env:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
-        - CONFIG="--cc=clang-3.9 --cxx=clang++-3.9"
+        - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
         - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
         - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-- 
2.11.0

[Qemu-devel] [PATCH v2 3/7] .travis.yml: DEBUG MacOSX build

[Alex Bennée]

Currently the MacOSX build is failing. Dump some information to help
with figuring out what has gone wrong.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index 9008a796f0..a9ce215f42 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -92,6 +92,7 @@ matrix:
     - env: CONFIG=""
       os: osx
       compiler: clang
+      script: make -j3 || cat qemu-options.texi && false
     # Plain Trusty System Build
     - env: CONFIG="--disable-linux-user"
       sudo: required
-- 
2.11.0

[Qemu-devel] [PATCH v2 4/7] tests/docker: add basic user mapping support

[Alex Bennée]

Currently all docker builds are done by exporting a tarball to the
docker container and running the build as the containers root user.
Other use cases are possible however and it is possible to map a part
of users file-system to the container. This is useful for example for
doing cross-builds of arbitrary source trees. For this to work
smoothly the container needs to have a user created that maps cleanly
to the host system.

This adds a -u option to the docker script so that:

  DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
    -u --include-executable=arm-linux-user/qemu-arm \
    debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker

Will build a container that can then be run like:

  docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
    --user=alex:alex -w /src/ debian:armhf \
    sh -c "make clean && ./configure -s && make"

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - write the useradd directly
  - change long option to --add-current-user
---
 tests/docker/docker.py | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 37d83199e7..b85739a9e2 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -25,6 +25,7 @@ import signal
 from tarfile import TarFile, TarInfo
 from StringIO import StringIO
 from shutil import copy, rmtree
+from pwd import getpwuid
 
 
 DEVNULL = open(os.devnull, 'wb')
@@ -149,13 +150,20 @@ class Docker(object):
         labels = json.loads(resp)[0]["Config"].get("Labels", {})
         return labels.get("com.qemu.dockerfile-checksum", "")
 
-    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
+    def build_image(self, tag, docker_dir, dockerfile,
+                    quiet=True, user=False, argv=None):
         if argv == None:
             argv = []
 
         tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
         tmp_df.write(dockerfile)
 
+        if user:
+            uid = os.getuid()
+            uname = getpwuid(uid).pw_name
+            tmp_df.write("\n")
+            tmp_df.write("RUN useradd -u %d -U %s" % (uid, uname))
+
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
                      _text_checksum(dockerfile))
@@ -225,6 +233,9 @@ class BuildCommand(SubCommand):
                             help="""Specify a binary that will be copied to the
                             container together with all its dependent
                             libraries""")
+        parser.add_argument("--add-current-user", "-u", dest="user",
+                            action="store_true",
+                            help="Add the current user to images passwd")
         parser.add_argument("tag",
                             help="Image Tag")
         parser.add_argument("dockerfile",
@@ -261,7 +272,7 @@ class BuildCommand(SubCommand):
                                        docker_dir)
 
             dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, argv=argv)
+                            quiet=args.quiet, user=args.user, argv=argv)
 
             rmtree(docker_dir)
 
-- 
2.11.0

[Qemu-devel] [PATCH v2 5/7] new debian-multiarch-cross.docker: a cross-compiling docker guest

[Alex Bennée]

This provides a basic Debian install with access to the emdebian cross
compilers for armhf, arm64, mipsel and ppc64el. On its own it provides
the basic cross build essentials for building any of those
architectures. More complex cross-compilations would need to install
library dependencies on top of this image.

An example use:

  docker run --rm -it \
    -v /home/alex/lsrc/qemu/qemu.git/:/home/alex/lsrc/qemu/qemu.git/ \
    --user=alex:alex \
    -w /home/alex/lsrc/qemu/qemu.git/arm-linux-user/tests \
    -e CROSS_CC="arm-linux-gnueabihf-gcc" \
    debian:cross make -f ../../tests/tcg/arm/Makefile

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - add clang (keep shippable happy)
  - rm adduser code (done direct now)
  - add aptitude (useful for debugging package clashes)
---
 .../dockerfiles/debian-multiarch-cross.docker      | 45 ++++++++++++++++++++++
 1 file changed, 45 insertions(+)
 create mode 100644 tests/docker/dockerfiles/debian-multiarch-cross.docker

diff --git a/tests/docker/dockerfiles/debian-multiarch-cross.docker b/tests/docker/dockerfiles/debian-multiarch-cross.docker
new file mode 100644
index 0000000000..749fdf07e1
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-multiarch-cross.docker
@@ -0,0 +1,45 @@
+#
+# Docker multiarch cross-compiler target
+#
+# This docker target is builds on Debian and Emdebian's cross compiler targets
+# to build distro with a selection of cross compilers for building test binaries.
+#
+# This does not include the build dependancies for QEMU as there are
+# so many there are unresolvable clashes.
+#
+FROM debian:stable-slim
+
+# Setup some basic tools we need
+RUN apt update
+RUN apt install -yy curl aptitude
+
+# Setup Emdebian
+RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >> /etc/apt/sources.list
+RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
+
+# Add the foriegn architectures we want
+RUN dpkg --add-architecture armhf
+RUN dpkg --add-architecture arm64
+RUN dpkg --add-architecture mipsel
+RUN dpkg --add-architecture ppc64el
+
+#FIXME: Currently these cause clashes due to multi-arch being an ongoig process
+#RUN dpkg --add-architecture mips
+#RUN dpkg --add-architecture powerpc
+
+# Final update
+RUN apt update
+
+# Install the crossbuild essential packages
+# FIXME: crossbuild-essential-powerpc has a clash with '/lib/ld.so.1'
+RUN apt install -yy clang crossbuild-essential-arm64 crossbuild-essential-armhf crossbuild-essential-mipsel crossbuild-essential-ppc64el
+
+# Before you can actually build something you need to install its build dependancies.
+# This is best done in a child container as architecture specific -dev packages have a habit of clashing with
+# each other because not everything is fully complient with the multiarch specififcation.
+#
+# To install the QEMU build dependacnies you need to run:
+# RUN apt-get build-dep -yy -a armhf qemu
-- 
2.11.0

[Qemu-devel] [PATCH v2 6/7] .shippable.yml: new CI provider

[Alex Bennée]

Ostensibly Shippable offers a similar set of services as Travis.
However they are focused on Container based work-flows so we can use
our existing containers to run a few extra builds - in this case a
bunch of cross-compiled targets on a Debian multiarch system.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .shippable.yml | 24 ++++++++++++++++++++++++
 MAINTAINERS    |  1 +
 2 files changed, 25 insertions(+)
 create mode 100644 .shippable.yml

diff --git a/.shippable.yml b/.shippable.yml
new file mode 100644
index 0000000000..10496acb7c
--- /dev/null
+++ b/.shippable.yml
@@ -0,0 +1,24 @@
+language: c
+env:
+  matrix:
+    - CROSS_ARCH=armhf   CROSS_PREFIX=arm-linux-gnueabihf-
+    - CROSS_ARCH=arm64   CROSS_PREFIX=aarch64-linux-gnu-
+    - CROSS_ARCH=mipsel  CROSS_PREFIX=mipsel-linux-gnu-
+    - CROSS_ARCH=ppc64el CROSS_PREFIX=powerpc64le-linux-gnu-
+build:
+  pre_ci:
+    - mkdir -p docker-build
+    - cp ./tests/docker/dockerfiles/debian-multiarch-cross.docker docker-build/Dockerfile
+    - docker build -t debian-multiarch-cross:base docker-build
+    - echo -e "FROM debian-multiarch-cross:base\nRUN apt-get build-dep -a ${CROSS_ARCH} -qq qemu" | docker build -t debian-multiarch-cross:${CROSS_ARCH} -
+  pre_ci_boot:
+    image_name: debian-multiarch-cross
+    image_tag: ${CROSS_ARCH}
+    pull: false
+    options: "-e HOME=/root"
+  ci:
+    - env
+    - apt-get build-dep -a ${CROSS_ARCH} -qq qemu
+    - unset CC
+    - ./configure --cross-prefix=${CROSS_PREFIX}
+    - make -j2
diff --git a/MAINTAINERS b/MAINTAINERS
index ad1000415d..5e7d1a7548 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1784,6 +1784,7 @@ M: Alex Bennée <alex.bennee@linaro.org>
 L: qemu-devel@nongnu.org
 S: Supported
 F: .travis.yml
+F: .shippable.yml
 
 Documentation
 -------------
-- 
2.11.0

[Qemu-devel] [PATCH v2 7/7] qemu-options: remove stray space

[Alex Bennée]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 qemu-options.hx | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index 80df52651a..55c68cecbb 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1705,7 +1705,7 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
     "                old way to initialize a host network interface\n"
     "                (use the -netdev option if possible instead)\n", QEMU_ARCH_ALL)
 STEXI
-@item -net nic[,vlan=@var{n}][,macaddr=@var{mac}][,model=@var{type}] [,name=@var{name}][,addr=@var{addr}][,vectors=@var{v}]
+@item -net nic[,vlan=@var{n}][,macaddr=@var{mac}][,model=@var{type}][,name=@var{name}][,addr=@var{addr}][,vectors=@var{v}]
 @findex -net
 Create a new Network Interface Card and connect it to VLAN @var{n} (@var{n}
 = 0 is the default). The NIC is an e1000 by default on the PC
-- 
2.11.0

Re: [Qemu-devel] [PATCH v2 4/7] tests/docker: add basic user mapping support

[Fam Zheng]

On Tue, 01/24 14:33, Alex Bennée wrote:
> Currently all docker builds are done by exporting a tarball to the
> docker container and running the build as the containers root user.
> Other use cases are possible however and it is possible to map a part
> of users file-system to the container. This is useful for example for
> doing cross-builds of arbitrary source trees. For this to work
> smoothly the container needs to have a user created that maps cleanly
> to the host system.
> 
> This adds a -u option to the docker script so that:
> 
>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
>     -u --include-executable=arm-linux-user/qemu-arm \
>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
> 
> Will build a container that can then be run like:
> 
>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
>     --user=alex:alex -w /src/ debian:armhf \
>     sh -c "make clean && ./configure -s && make"
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v2
>   - write the useradd directly
>   - change long option to --add-current-user
> ---
>  tests/docker/docker.py | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index 37d83199e7..b85739a9e2 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -25,6 +25,7 @@ import signal
>  from tarfile import TarFile, TarInfo
>  from StringIO import StringIO
>  from shutil import copy, rmtree
> +from pwd import getpwuid
>  
>  
>  DEVNULL = open(os.devnull, 'wb')
> @@ -149,13 +150,20 @@ class Docker(object):
>          labels = json.loads(resp)[0]["Config"].get("Labels", {})
>          return labels.get("com.qemu.dockerfile-checksum", "")
>  
> -    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
> +    def build_image(self, tag, docker_dir, dockerfile,
> +                    quiet=True, user=False, argv=None):
>          if argv == None:
>              argv = []
>  
>          tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
>          tmp_df.write(dockerfile)
>  
> +        if user:
> +            uid = os.getuid()
> +            uname = getpwuid(uid).pw_name
> +            tmp_df.write("\n")
> +            tmp_df.write("RUN useradd -u %d -U %s" % (uid, uname))
> +
>          tmp_df.write("\n")
>          tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
>                       _text_checksum(dockerfile))
> @@ -225,6 +233,9 @@ class BuildCommand(SubCommand):
>                              help="""Specify a binary that will be copied to the
>                              container together with all its dependent
>                              libraries""")
> +        parser.add_argument("--add-current-user", "-u", dest="user",
> +                            action="store_true",
> +                            help="Add the current user to images passwd")

s/images/image's/ ?

otherwise looks good. If you fix that, you can add

Reviewed-by: Fam Zheng <famz@redhat.com>

>          parser.add_argument("tag",
>                              help="Image Tag")
>          parser.add_argument("dockerfile",
> @@ -261,7 +272,7 @@ class BuildCommand(SubCommand):
>                                         docker_dir)
>  
>              dkr.build_image(tag, docker_dir, dockerfile,
> -                            quiet=args.quiet, argv=argv)
> +                            quiet=args.quiet, user=args.user, argv=argv)
>  
>              rmtree(docker_dir)
>  
> -- 
> 2.11.0
> 
> 

Re: [Qemu-devel] [PATCH v2 5/7] new debian-multiarch-cross.docker: a cross-compiling docker guest

[Fam Zheng]

On Tue, 01/24 14:33, Alex Bennée wrote:
> This provides a basic Debian install with access to the emdebian cross
> compilers for armhf, arm64, mipsel and ppc64el. On its own it provides
> the basic cross build essentials for building any of those
> architectures. More complex cross-compilations would need to install
> library dependencies on top of this image.
> 
> An example use:
> 
>   docker run --rm -it \
>     -v /home/alex/lsrc/qemu/qemu.git/:/home/alex/lsrc/qemu/qemu.git/ \
>     --user=alex:alex \
>     -w /home/alex/lsrc/qemu/qemu.git/arm-linux-user/tests \
>     -e CROSS_CC="arm-linux-gnueabihf-gcc" \
>     debian:cross make -f ../../tests/tcg/arm/Makefile
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v2
>   - add clang (keep shippable happy)
>   - rm adduser code (done direct now)
>   - add aptitude (useful for debugging package clashes)
> ---
>  .../dockerfiles/debian-multiarch-cross.docker      | 45 ++++++++++++++++++++++
>  1 file changed, 45 insertions(+)
>  create mode 100644 tests/docker/dockerfiles/debian-multiarch-cross.docker
> 
> diff --git a/tests/docker/dockerfiles/debian-multiarch-cross.docker b/tests/docker/dockerfiles/debian-multiarch-cross.docker
> new file mode 100644
> index 0000000000..749fdf07e1
> --- /dev/null
> +++ b/tests/docker/dockerfiles/debian-multiarch-cross.docker
> @@ -0,0 +1,45 @@
> +#
> +# Docker multiarch cross-compiler target
> +#
> +# This docker target is builds on Debian and Emdebian's cross compiler targets
> +# to build distro with a selection of cross compilers for building test binaries.
> +#
> +# This does not include the build dependancies for QEMU as there are
> +# so many there are unresolvable clashes.
> +#
> +FROM debian:stable-slim
> +
> +# Setup some basic tools we need
> +RUN apt update
> +RUN apt install -yy curl aptitude
> +
> +# Setup Emdebian
> +RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >> /etc/apt/sources.list
> +RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
> +
> +# Duplicate deb line as deb-src
> +RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
> +
> +# Add the foriegn architectures we want
> +RUN dpkg --add-architecture armhf
> +RUN dpkg --add-architecture arm64
> +RUN dpkg --add-architecture mipsel
> +RUN dpkg --add-architecture ppc64el
> +
> +#FIXME: Currently these cause clashes due to multi-arch being an ongoig process

"ongoing"?

> +#RUN dpkg --add-architecture mips
> +#RUN dpkg --add-architecture powerpc
> +
> +# Final update
> +RUN apt update
> +
> +# Install the crossbuild essential packages
> +# FIXME: crossbuild-essential-powerpc has a clash with '/lib/ld.so.1'
> +RUN apt install -yy clang crossbuild-essential-arm64 crossbuild-essential-armhf crossbuild-essential-mipsel crossbuild-essential-ppc64el
> +
> +# Before you can actually build something you need to install its build dependancies.
> +# This is best done in a child container as architecture specific -dev packages have a habit of clashing with
> +# each other because not everything is fully complient with the multiarch specififcation.

"compliant"?

> +#
> +# To install the QEMU build dependacnies you need to run:
> +# RUN apt-get build-dep -yy -a armhf qemu
> -- 
> 2.11.0
> 

Does this image work with our tests, such as "make
docker-test-quick@debian-multiarch-cross"? If not let's put it into a different
folder okay?

Fam

Re: [Qemu-devel] [PATCH v2 7/7] qemu-options: remove stray space

[Fam Zheng]

On Tue, 01/24 14:33, Alex Bennée wrote:
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  qemu-options.hx | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 80df52651a..55c68cecbb 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -1705,7 +1705,7 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
>      "                old way to initialize a host network interface\n"
>      "                (use the -netdev option if possible instead)\n", QEMU_ARCH_ALL)
>  STEXI
> -@item -net nic[,vlan=@var{n}][,macaddr=@var{mac}][,model=@var{type}] [,name=@var{name}][,addr=@var{addr}][,vectors=@var{v}]
> +@item -net nic[,vlan=@var{n}][,macaddr=@var{mac}][,model=@var{type}][,name=@var{name}][,addr=@var{addr}][,vectors=@var{v}]
>  @findex -net
>  Create a new Network Interface Card and connect it to VLAN @var{n} (@var{n}
>  = 0 is the default). The NIC is an e1000 by default on the PC
> -- 
> 2.11.0
> 
> 

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [PATCH v2 3/7] .travis.yml: DEBUG MacOSX build

[Fam Zheng]

On Tue, 01/24 14:33, Alex Bennée wrote:
> Currently the MacOSX build is failing. Dump some information to help
> with figuring out what has gone wrong.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .travis.yml | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 9008a796f0..a9ce215f42 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -92,6 +92,7 @@ matrix:
>      - env: CONFIG=""
>        os: osx
>        compiler: clang
> +      script: make -j3 || cat qemu-options.texi && false
>      # Plain Trusty System Build
>      - env: CONFIG="--disable-linux-user"
>        sudo: required
> -- 
> 2.11.0
> 
> 

Why qemu-options.texi? This looks really ad-hocery, not sure it's useful for the
mainline.

Fam

Re: [Qemu-devel] [PATCH v2 3/7] .travis.yml: DEBUG MacOSX build

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Tue, 01/24 14:33, Alex Bennée wrote:
>> Currently the MacOSX build is failing. Dump some information to help
>> with figuring out what has gone wrong.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  .travis.yml | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 9008a796f0..a9ce215f42 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -92,6 +92,7 @@ matrix:
>>      - env: CONFIG=""
>>        os: osx
>>        compiler: clang
>> +      script: make -j3 || cat qemu-options.texi && false
>>      # Plain Trusty System Build
>>      - env: CONFIG="--disable-linux-user"
>>        sudo: required
>> --
>> 2.11.0
>>
>>
>
> Why qemu-options.texi? This looks really ad-hocery, not sure it's useful for the
> mainline.

This is what was failing - I wasn't intending for this to get pulled, it
was just an example for people wanting to debug the current (now fixed?)
problem.

>
> Fam


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 5/7] new debian-multiarch-cross.docker: a cross-compiling docker guest

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Tue, 01/24 14:33, Alex Bennée wrote:
>> This provides a basic Debian install with access to the emdebian cross
>> compilers for armhf, arm64, mipsel and ppc64el. On its own it provides
>> the basic cross build essentials for building any of those
>> architectures. More complex cross-compilations would need to install
>> library dependencies on top of this image.
>>
>> An example use:
>>
>>   docker run --rm -it \
>>     -v /home/alex/lsrc/qemu/qemu.git/:/home/alex/lsrc/qemu/qemu.git/ \
>>     --user=alex:alex \
>>     -w /home/alex/lsrc/qemu/qemu.git/arm-linux-user/tests \
>>     -e CROSS_CC="arm-linux-gnueabihf-gcc" \
>>     debian:cross make -f ../../tests/tcg/arm/Makefile
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>
>> ---
>> v2
>>   - add clang (keep shippable happy)
>>   - rm adduser code (done direct now)
>>   - add aptitude (useful for debugging package clashes)
>> ---
>>  .../dockerfiles/debian-multiarch-cross.docker      | 45 ++++++++++++++++++++++
>>  1 file changed, 45 insertions(+)
>>  create mode 100644 tests/docker/dockerfiles/debian-multiarch-cross.docker
>>
>> diff --git a/tests/docker/dockerfiles/debian-multiarch-cross.docker b/tests/docker/dockerfiles/debian-multiarch-cross.docker
>> new file mode 100644
>> index 0000000000..749fdf07e1
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/debian-multiarch-cross.docker
>> @@ -0,0 +1,45 @@
>> +#
>> +# Docker multiarch cross-compiler target
>> +#
>> +# This docker target is builds on Debian and Emdebian's cross compiler targets
>> +# to build distro with a selection of cross compilers for building test binaries.
>> +#
>> +# This does not include the build dependancies for QEMU as there are
>> +# so many there are unresolvable clashes.
>> +#
>> +FROM debian:stable-slim
>> +
>> +# Setup some basic tools we need
>> +RUN apt update
>> +RUN apt install -yy curl aptitude
>> +
>> +# Setup Emdebian
>> +RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >> /etc/apt/sources.list
>> +RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
>> +
>> +# Duplicate deb line as deb-src
>> +RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
>> +
>> +# Add the foriegn architectures we want
>> +RUN dpkg --add-architecture armhf
>> +RUN dpkg --add-architecture arm64
>> +RUN dpkg --add-architecture mipsel
>> +RUN dpkg --add-architecture ppc64el
>> +
>> +#FIXME: Currently these cause clashes due to multi-arch being an ongoig process
>
> "ongoing"?
>
>> +#RUN dpkg --add-architecture mips
>> +#RUN dpkg --add-architecture powerpc
>> +
>> +# Final update
>> +RUN apt update
>> +
>> +# Install the crossbuild essential packages
>> +# FIXME: crossbuild-essential-powerpc has a clash with '/lib/ld.so.1'
>> +RUN apt install -yy clang crossbuild-essential-arm64 crossbuild-essential-armhf crossbuild-essential-mipsel crossbuild-essential-ppc64el
>> +
>> +# Before you can actually build something you need to install its build dependancies.
>> +# This is best done in a child container as architecture specific -dev packages have a habit of clashing with
>> +# each other because not everything is fully complient with the multiarch specififcation.
>
> "compliant"?
>
>> +#
>> +# To install the QEMU build dependacnies you need to run:
>> +# RUN apt-get build-dep -yy -a armhf qemu
>> --
>> 2.11.0
>>
>
> Does this image work with our tests, such as "make
> docker-test-quick@debian-multiarch-cross"? If not let's put it into a different
> folder okay?

There is no particular reason why it couldn't be included in the docker
targets. We can either expand the existing build.sh to include a
dependency step or add a new build script for use here.

--
Alex Bennée