[Qemu-devel] [PULL 0/6] Travis updates (3rd pull attempt)

[Qemu-devel] [PULL 0/6] Travis updates (3rd pull attempt)

[Alex Bennée]

The following changes since commit ee8e8f92a730afc17ab8be6e86df6b9a23b8ebc6:

  Merge remote-tracking branch 'remotes/amit-migration/tags/migration-for-2.6-2' into staging (2016-02-05 14:20:46 +0000)

are available in the git repository at:

  https://github.com/stsquad/qemu.git tags/pull-build-test-20160209

for you to fetch changes up to b9e02c061b74cf99fa4865381f85559bcdb09103:

  MAINTAINERS: Add .travis.yml (2016-02-08 18:50:31 +0000)

----------------------------------------------------------------
This is the third attempt for this pull request.

Since the v4 was posted:
 - fixed merge conflict with ed7f5f1d8db06fc31352a5ef4f54985e630c575a
 - added cleaner separation line to MAINTAINERS at Fam's request
 - skip "make check" for --enable-trace-backends=simple (see 41fc57e44ed)

----------------------------------------------------------------
Alex Bennée (6):
      .travis.yml: migrate to container builds
      .travis.yml: run make check for all matrix targets
      .travis.yml: enable each of the co-routine backends
      .travis.yml: enable ccache for the builds
      .travis.yml: reduce the test matrix a little
      MAINTAINERS: Add .travis.yml

 .travis.yml | 111 ++++++++++++++++++++++++++++++++++++++----------------------
 MAINTAINERS |   6 ++++
 2 files changed, 76 insertions(+), 41 deletions(-)


-- 
2.7.0

[Qemu-devel] [PULL 1/6] .travis.yml: migrate to container builds

[Alex Bennée]

This moves the Travis tests from the legacy VM infrastructure (which
only seems to run 5-6 jobs at once) to the new container based approach.

The principle difference is there is no sudo in the containers so all
packages are installed using the apt add-on. This means one of the build
combinations can be dropped as it was only for checking the build with
additional packages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
 .travis.yml | 44 ++++++++++++++++++++++++++++++++------------
 1 file changed, 32 insertions(+), 12 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 0428aed..1a7db11 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,9 +1,37 @@
+sudo: false
 language: c
 python:
   - "2.4"
 compiler:
   - gcc
   - clang
+addons:
+  apt:
+    packages:
+      - libaio-dev
+      - libattr1-dev
+      - libbrlapi-dev
+      - libcap-ng-dev
+      - libgnutls-dev
+      - libgtk-3-dev
+      - libiscsi-dev
+      - liblttng-ust-dev
+      - libncurses5-dev
+      - libnss3-dev
+      - libpixman-1-dev
+      - libpng12-dev
+      - librados-dev
+      - libsdl1.2-dev
+      - libseccomp-dev
+      - libspice-protocol-dev
+      - libspice-server-dev
+      - libssh2-1-dev
+      - liburcu-dev
+      - libusb-1.0-0-dev
+      - libvte-2.90-dev
+      - sparse
+      - uuid-dev
+
 notifications:
   irc:
     channels:
@@ -14,11 +42,6 @@ env:
   global:
     - TEST_CMD=""
     - EXTRA_CONFIG=""
-    # Development packages, EXTRA_PKGS saved for additional builds
-    - CORE_PKGS="libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev"
-    - NET_PKGS="libseccomp-dev libgnutls-dev libssh2-1-dev  libspice-server-dev libspice-protocol-dev libnss3-dev"
-    - GUI_PKGS="libgtk-3-dev libvte-2.90-dev libsdl1.2-dev libpng12-dev libpixman-1-dev"
-    - EXTRA_PKGS=""
   matrix:
     # Group major targets together with their linux-user counterparts
     - TARGETS=alpha-softmmu,alpha-linux-user
@@ -43,8 +66,6 @@ git:
 before_install:
   - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
   - git submodule update --init --recursive
-  - sudo apt-get update -qq
-  - sudo apt-get install -qq ${CORE_PKGS} ${NET_PKGS} ${GUI_PKGS} ${EXTRA_PKGS}
 before_script:
   - ./configure --target-list=${TARGETS} --enable-debug-tcg ${EXTRA_CONFIG}
 script:
@@ -54,7 +75,7 @@ matrix:
   include:
     # Make check target (we only do this once)
     - env:
-        - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,unicore32-linux-user,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu
+        - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu
           TEST_CMD="make check"
       compiler: gcc
     # Debug related options
@@ -64,16 +85,16 @@ matrix:
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter"
       compiler: gcc
-    # All the extra -dev packages
+    # Disable a few of the optional features
     - env: TARGETS=i386-softmmu,x86_64-softmmu
-           EXTRA_PKGS="libaio-dev libcap-ng-dev libattr1-dev libbrlapi-dev uuid-dev libusb-1.0.0-dev"
+           EXTRA_CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb"
       compiler: gcc
     # Currently configure doesn't force --disable-pie
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-gprof --enable-gcov --disable-pie"
       compiler: gcc
+    # Sparse
     - env: TARGETS=i386-softmmu,x86_64-softmmu
-           EXTRA_PKGS="sparse"
            EXTRA_CONFIG="--enable-sparse"
       compiler: gcc
     # All the trace backends (apart from dtrace)
@@ -87,7 +108,6 @@ matrix:
            EXTRA_CONFIG="--enable-trace-backends=ftrace"
       compiler: gcc
     - env: TARGETS=i386-softmmu,x86_64-softmmu
-          EXTRA_PKGS="liblttng-ust-dev liburcu-dev"
           EXTRA_CONFIG="--enable-trace-backends=ust"
       compiler: gcc
     - env: TARGETS=i386-softmmu,x86_64-softmmu
-- 
2.7.0

[Qemu-devel] [PULL 2/6] .travis.yml: run make check for all matrix targets

[Alex Bennée]

We only ran make check once before it used to be an unreliable target.
It was only a stop gap measure and we should be able to revert it now.
This also stops us needing a large all-MMU build.

We disable "make check" for a couple of the extra config targets which
are currently broken.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Tested-by: David Gibson <david@gibson.dropbear.id.au>
---
 .travis.yml | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 1a7db11..60d8957 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -40,7 +40,7 @@ notifications:
     on_failure: always
 env:
   global:
-    - TEST_CMD=""
+    - TEST_CMD="make check"
     - EXTRA_CONFIG=""
   matrix:
     # Group major targets together with their linux-user counterparts
@@ -73,17 +73,14 @@ script:
 matrix:
   # We manually include a number of additional build for non-standard bits
   include:
-    # Make check target (we only do this once)
-    - env:
-        - TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu
-          TEST_CMD="make check"
-      compiler: gcc
     # Debug related options
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-debug"
       compiler: gcc
+    # We currently disable "make check"
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter"
+           TEST_CMD=""
       compiler: gcc
     # Disable a few of the optional features
     - env: TARGETS=i386-softmmu,x86_64-softmmu
@@ -101,14 +98,20 @@ matrix:
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=log"
       compiler: gcc
+    # We currently disable "make check" (until 41fc57e44ed regression fixed)
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=simple"
+           TEST_CMD=""
       compiler: gcc
+    # We currently disable "make check"
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=ftrace"
+           TEST_CMD=""
       compiler: gcc
+    # We currently disable "make check"
     - env: TARGETS=i386-softmmu,x86_64-softmmu
-          EXTRA_CONFIG="--enable-trace-backends=ust"
+           EXTRA_CONFIG="--enable-trace-backends=ust"
+           TEST_CMD=""
       compiler: gcc
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-modules"
-- 
2.7.0

[Qemu-devel] [PULL 3/6] .travis.yml: enable each of the co-routine backends

[Alex Bennée]

We disable "make check" for the gthread backend as it is broken.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Tested-by: David Gibson <david@gibson.dropbear.id.au>
---
 .travis.yml | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 60d8957..92f47ab 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -94,6 +94,10 @@ matrix:
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-sparse"
       compiler: gcc
+    # Modules
+    - env: TARGETS=i386-softmmu,x86_64-softmmu
+           EXTRA_CONFIG="--enable-modules"
+      compiler: gcc
     # All the trace backends (apart from dtrace)
     - env: TARGETS=i386-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=log"
@@ -113,6 +117,15 @@ matrix:
            EXTRA_CONFIG="--enable-trace-backends=ust"
            TEST_CMD=""
       compiler: gcc
+    # All the co-routine backends (apart from windows)
+    # We currently disable "make check"
     - env: TARGETS=i386-softmmu,x86_64-softmmu
-           EXTRA_CONFIG="--enable-modules"
+           EXTRA_CONFIG="--with-coroutine=gthread"
+           TEST_CMD=""
+      compiler: gcc
+    - env: TARGETS=i386-softmmu,x86_64-softmmu
+           EXTRA_CONFIG="--with-coroutine=ucontext"
+      compiler: gcc
+    - env: TARGETS=i386-softmmu,x86_64-softmmu
+           EXTRA_CONFIG="--with-coroutine=sigaltstack"
       compiler: gcc
-- 
2.7.0

[Qemu-devel] [PULL 4/6] .travis.yml: enable ccache for the builds

[Alex Bennée]

Travis support ccache on a cache-per-branch basis. Given not much of the
build changes between pushes as well as the duplication in each build it
seems worthwhile enabling this.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Tested-by: David Gibson <david@gibson.dropbear.id.au>
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index 92f47ab..31d062f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,6 +5,7 @@ python:
 compiler:
   - gcc
   - clang
+cache: ccache
 addons:
   apt:
     packages:
-- 
2.7.0

[Qemu-devel] [PULL 5/6] .travis.yml: reduce the test matrix a little

[Alex Bennée]

As we are now running "make check" on more of the matrix it is worth
making more of an effort to reduce the overall load on Travis. I've done
a few things:

  - Combining a number of the targets
  - Building one target for each ancillary build

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Tested-by: David Gibson <david@gibson.dropbear.id.au>
---
 .travis.yml | 42 +++++++++++++++++-------------------------
 1 file changed, 17 insertions(+), 25 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 31d062f..9e5873b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -45,20 +45,12 @@ env:
     - EXTRA_CONFIG=""
   matrix:
     # Group major targets together with their linux-user counterparts
-    - TARGETS=alpha-softmmu,alpha-linux-user
+    - TARGETS=alpha-softmmu,alpha-linux-user,cris-softmmu,cris-linux-user,m68k-softmmu,m68k-linux-user,microblaze-softmmu,microblazeel-softmmu,microblaze-linux-user,microblazeel-linux-user
     - TARGETS=arm-softmmu,arm-linux-user,armeb-linux-user,aarch64-softmmu,aarch64-linux-user
-    - TARGETS=cris-softmmu,cris-linux-user
     - TARGETS=i386-softmmu,i386-linux-user,x86_64-softmmu,x86_64-linux-user
-    - TARGETS=m68k-softmmu,m68k-linux-user
-    - TARGETS=microblaze-softmmu,microblazeel-softmmu,microblaze-linux-user,microblazeel-linux-user
-    - TARGETS=mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu
-    - TARGETS=mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,mipsn32-linux-user,mipsn32el-linux-user
-    - TARGETS=or32-softmmu,or32-linux-user
-    - TARGETS=ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,ppc-linux-user,ppc64-linux-user,ppc64abi32-linux-user,ppc64le-linux-user
-    - TARGETS=s390x-softmmu,s390x-linux-user
-    - TARGETS=sh4-softmmu,sh4eb-softmmu,sh4-linux-user sh4eb-linux-user
-    - TARGETS=sparc-softmmu,sparc64-softmmu,sparc-linux-user,sparc32plus-linux-user,sparc64-linux-user
-    - TARGETS=unicore32-softmmu,unicore32-linux-user
+    - TARGETS=mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,mipsn32-linux-user,mipsn32el-linux-user
+    - TARGETS=or32-softmmu,or32-linux-user,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,ppc-linux-user,ppc64-linux-user,ppc64abi32-linux-user,ppc64le-linux-user
+    - TARGETS=s390x-softmmu,s390x-linux-user,sh4-softmmu,sh4eb-softmmu,sh4-linux-user,sh4eb-linux-user,sparc-softmmu,sparc64-softmmu,sparc-linux-user,sparc32plus-linux-user,sparc64-linux-user,unicore32-softmmu,unicore32-linux-user
     # Group remaining softmmu only targets into one build
     - TARGETS=lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu
 git:
@@ -75,58 +67,58 @@ matrix:
   # We manually include a number of additional build for non-standard bits
   include:
     # Debug related options
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--enable-debug"
       compiler: gcc
     # We currently disable "make check"
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=alpha-softmmu
            EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter"
            TEST_CMD=""
       compiler: gcc
     # Disable a few of the optional features
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb"
       compiler: gcc
     # Currently configure doesn't force --disable-pie
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--enable-gprof --enable-gcov --disable-pie"
       compiler: gcc
     # Sparse
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--enable-sparse"
       compiler: gcc
     # Modules
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=arm-softmmu,x86_64-softmmu
            EXTRA_CONFIG="--enable-modules"
       compiler: gcc
     # All the trace backends (apart from dtrace)
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=i386-softmmu
            EXTRA_CONFIG="--enable-trace-backends=log"
       compiler: gcc
     # We currently disable "make check" (until 41fc57e44ed regression fixed)
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=simple"
            TEST_CMD=""
       compiler: gcc
     # We currently disable "make check"
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=ftrace"
            TEST_CMD=""
       compiler: gcc
     # We currently disable "make check"
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--enable-trace-backends=ust"
            TEST_CMD=""
       compiler: gcc
     # All the co-routine backends (apart from windows)
     # We currently disable "make check"
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--with-coroutine=gthread"
            TEST_CMD=""
       compiler: gcc
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--with-coroutine=ucontext"
       compiler: gcc
-    - env: TARGETS=i386-softmmu,x86_64-softmmu
+    - env: TARGETS=x86_64-softmmu
            EXTRA_CONFIG="--with-coroutine=sigaltstack"
       compiler: gcc
-- 
2.7.0

[Qemu-devel] [PULL 6/6] MAINTAINERS: Add .travis.yml

[Alex Bennée]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
 MAINTAINERS | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index b6ed87a..2d6ee17 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1566,6 +1566,12 @@ L: qemu-block@nongnu.org
 S: Supported
 F: tests/image-fuzzer/
 
+Build and test automation
+-------------------------
+M: Alex Bennée <alex.bennee@linaro.org>
+L: qemu-devel@nongnu.org
+S: Supported
+F: .travis.yml
 
 Documentation
 -------------
-- 
2.7.0

Re: [Qemu-devel] [PULL 0/6] Travis updates (3rd pull attempt)

[Peter Maydell]

On 9 February 2016 at 12:36, Alex Bennée <alex.bennee@linaro.org> wrote:
> The following changes since commit ee8e8f92a730afc17ab8be6e86df6b9a23b8ebc6:
>
>   Merge remote-tracking branch 'remotes/amit-migration/tags/migration-for-2.6-2' into staging (2016-02-05 14:20:46 +0000)
>
> are available in the git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-build-test-20160209
>
> for you to fetch changes up to b9e02c061b74cf99fa4865381f85559bcdb09103:
>
>   MAINTAINERS: Add .travis.yml (2016-02-08 18:50:31 +0000)
>
> ----------------------------------------------------------------
> This is the third attempt for this pull request.
>
> Since the v4 was posted:
>  - fixed merge conflict with ed7f5f1d8db06fc31352a5ef4f54985e630c575a
>  - added cleaner separation line to MAINTAINERS at Fam's request
>  - skip "make check" for --enable-trace-backends=simple (see 41fc57e44ed)
>
> ----------------------------------------------------------------
> Alex Bennée (6):
>       .travis.yml: migrate to container builds
>       .travis.yml: run make check for all matrix targets
>       .travis.yml: enable each of the co-routine backends
>       .travis.yml: enable ccache for the builds
>       .travis.yml: reduce the test matrix a little
>       MAINTAINERS: Add .travis.yml

Applied, thanks.

-- PMM