All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH 00/11] travis: try to reduce failures
@ 2017-06-16 16:13 Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 01/11] travis: build using all available cpus Philippe Mathieu-Daudé
                   ` (10 more replies)
  0 siblings, 11 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

These patches try to improve our Travis CI usage (quite a few failures the last
days). This series include Peter's "Automate coverity scan uploads via Travis"
patches (https://www.mail-archive.com/qemu-devel@nongnu.org/msg457443.html).

Patch 1 is a no-op as of today, but if eventually Shippable improve their
hardware the builds will profit of it directly.

Patch 2 allow to check how many time is spent building the codebase versus
running the tests.

Patch 4/5 speed up a bit Travis building many test objects in parallel (using
all available cores) previous to run the tests sequentially (as before) so the
tests output don't get mixed.

Patch 6 use 'travis_retry' feature to avoid aborting a job on network failure
retrying 3 times (default).

Then are Peter patches improving Coverity.

Regards,

Phil.

Peter Maydell (3):
  travis: install more library dependencies
  scripts/run-coverity-scan: Script to run Coverity Scan build
  travis: Add config to do a Coverity Scan upload

Philippe Mathieu-Daudé (8):
  travis: build using all available cpus
  travis: split building/testing to have finer elapsed time
  travis: use yes/no variable to enable/disable tests
  tests: add rule to compile many objects used by tests
  travis: build tests objects in parallel, then run tests sequentially
  travis: retry if llvm.org timeouts
  travis: install more library dependencies
  MAINTAINERS: self-appoint me as reviewer in build/test automation

 .travis.yml               |  82 ++++++++++++++++++----
 MAINTAINERS               |   1 +
 scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/Makefile.include    |   8 ++-
 4 files changed, 244 insertions(+), 17 deletions(-)
 create mode 100755 scripts/run-coverity-scan

-- 
2.11.0

^ permalink raw reply	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 01/11] travis: build using all available cpus
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-19 10:48   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 02/11] travis: split building/testing to have finer elapsed time Philippe Mathieu-Daudé
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

no improvement as of today, but if Travis release their limit on the opensource
plan or upgrade their hardware the builds will get some benefit.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 27a2d9cfb3..d21a2a3602 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -45,6 +45,7 @@ notifications:
     on_failure: always
 env:
   global:
+    - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
     - TEST_CMD="make check"
   matrix:
     - CONFIG=""
@@ -64,7 +65,7 @@ before_install:
 before_script:
   - ./configure ${CONFIG}
 script:
-  - make -j3 && ${TEST_CMD}
+  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
 matrix:
   include:
     # Test with CLang for compile portability
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 02/11] travis: split building/testing to have finer elapsed time
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 01/11] travis: build using all available cpus Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-19 10:49   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests Philippe Mathieu-Daudé
                   ` (8 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

to check each step elapsed time on the travis output report.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index d21a2a3602..b2c69fdd59 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -65,7 +65,8 @@ before_install:
 before_script:
   - ./configure ${CONFIG}
 script:
-  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
+  - make -j${PARALLEL_JOBS}
+  - test -z "${TEST_CMD}" || ${TEST_CMD}
 matrix:
   include:
     # Test with CLang for compile portability
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 01/11] travis: build using all available cpus Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 02/11] travis: split building/testing to have finer elapsed time Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-17  8:06   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 04/11] tests: add rule to compile many objects used by tests Philippe Mathieu-Daudé
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

having the command to run in the variable make scripts harder to manage.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---

using 0/1 or no/yes is pretty much the same.

 .travis.yml | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index b2c69fdd59..c9ac741afc 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -46,7 +46,7 @@ notifications:
 env:
   global:
     - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
-    - TEST_CMD="make check"
+    - RUN_TESTS="yes"
   matrix:
     - CONFIG=""
     - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
@@ -66,7 +66,7 @@ before_script:
   - ./configure ${CONFIG}
 script:
   - make -j${PARALLEL_JOBS}
-  - test -z "${TEST_CMD}" || ${TEST_CMD}
+  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
 matrix:
   include:
     # Test with CLang for compile portability
@@ -77,16 +77,16 @@ matrix:
       compiler: gcc
     # We manually include builds which we disable "make check" for
     - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG="--enable-trace-backends=simple"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG="--enable-trace-backends=ftrace"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG="--enable-trace-backends=ust"
-           TEST_CMD=""
+           RUN_TESTS="no"
       compiler: gcc
     - env: CONFIG=""
       os: osx
@@ -191,6 +191,6 @@ matrix:
       env:
         - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
         - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
-        - TEST_CMD=""
+        - RUN_TESTS="no"
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 04/11] tests: add rule to compile many objects used by tests
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (2 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-19 10:56   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially Philippe Mathieu-Daudé
                   ` (6 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

all those objects can get compiled simultaneously

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/Makefile.include | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index f42f3dfa72..3773f9d8d2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -804,6 +804,7 @@ check-help:
 	@echo " make check-qapi-schema    Run QAPI schema tests"
 	@echo " make check-block          Run block tests"
 	@echo " make check-report.html    Generates an HTML test report"
+	@echo " make check-compile        Compile objects used by tests"
 	@echo " make check-clean          Clean the tests"
 	@echo
 	@echo "Please note that HTML reports do not regenerate if the unit tests"
@@ -887,7 +888,7 @@ check-tests/qapi-schema/doc-good.texi: tests/qapi-schema/doc-good.test.texi
 
 # Consolidated targets
 
-.PHONY: check-qapi-schema check-qtest check-unit check check-clean
+.PHONY: check-qapi-schema check-qtest check-unit check check-clean check-compile
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-unit: $(patsubst %,check-%, $(check-unit-y))
@@ -897,7 +898,10 @@ check-clean:
 	$(MAKE) -C tests/tcg clean
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
-
+check-compile: $(check-qtest-generic-y) $(check-unit-y) $(test-util-obj-y) $(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y) $(libqos-pc-obj-y) $(chardev-obj-y) $(QEMU_IOTESTS_HELPERS-y)
+ifeq ($(CONFIG_SOFTMMU),y)
+check-compile: $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)))
+endif
 clean: check-clean
 
 # Build the help program automatically
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (3 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 04/11] tests: add rule to compile many objects used by tests Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-19 10:58   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 06/11] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
                   ` (5 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

tests are run sequentially to avoid mixed results output.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index c9ac741afc..dbbb11617d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -66,7 +66,7 @@ before_script:
   - ./configure ${CONFIG}
 script:
   - make -j${PARALLEL_JOBS}
-  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
+  - if [ "${RUN_TESTS}" == "yes" ]; then make -j${PARALLEL_JOBS} check-compile && make check; fi
 matrix:
   include:
     # Test with CLang for compile portability
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 06/11] travis: retry if llvm.org timeouts
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (4 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-19 11:00   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 07/11] travis: install more library dependencies Philippe Mathieu-Daudé
                   ` (4 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857

    $ sudo apt-get update -qq
    W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg  Connection failed
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    The command "sudo apt-get update -qq" failed and exited with 100 during .
    Your build has been stopped.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index dbbb11617d..69a960b714 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -123,10 +123,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
@@ -142,10 +142,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 07/11] travis: install more library dependencies
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (5 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 06/11] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 08/11] " Philippe Mathieu-Daudé
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

From: Peter Maydell <peter.maydell@linaro.org>

Update the travis list of library packages to install so that
our build tests cover more of our code base.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 69a960b714..4889c192c2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,28 +11,37 @@ addons:
       # Build dependencies
       - libaio-dev
       - libattr1-dev
+      - libbluetooth-dev
       - libbrlapi-dev
+      - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
       - libgtk-3-dev
       - libiscsi-dev
       - liblttng-ust-dev
+      - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
       - librados-dev
+      - librdmacm-dev
       - libsdl1.2-dev
       - libseccomp-dev
+      - libsnappy-dev
       - libspice-protocol-dev
       - libspice-server-dev
       - libssh2-1-dev
       - liburcu-dev
       - libusb-1.0-0-dev
+      - libvde-dev
       - libvte-2.90-dev
+      - libxen-dev
+      - nettle-dev
       - sparse
       - uuid-dev
+      - xfslibs-dev
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
 # to prevent IRC notifications from forks. This was created using:
@@ -164,28 +173,37 @@ matrix:
             # Build dependencies
             - libaio-dev
             - libattr1-dev
+            - libbluetooth-dev
             - libbrlapi-dev
+            - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
             - libgtk-3-dev
             - libiscsi-dev
             - liblttng-ust-dev
+            - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
             - librados-dev
+            - librdmacm-dev
             - libsdl1.2-dev
             - libseccomp-dev
+            - libsnappy-dev
             - libspice-protocol-dev
             - libspice-server-dev
             - libssh2-1-dev
             - liburcu-dev
             - libusb-1.0-0-dev
+            - libvde-dev
             - libvte-2.90-dev
+            - libxen-dev
+            - nettle-dev
             - sparse
             - uuid-dev
+            - xfslibs-dev
       language: generic
       compiler: none
       env:
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 08/11] travis: install more library dependencies
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (6 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 07/11] travis: install more library dependencies Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-19 11:01   ` Alex Bennée
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 09/11] scripts/run-coverity-scan: Script to run Coverity Scan build Philippe Mathieu-Daudé
                   ` (2 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

so more codebase is built

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 4889c192c2..4c0f7f444e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -9,10 +9,12 @@ addons:
   apt:
     packages:
       # Build dependencies
+      - glusterfs-common
       - libaio-dev
       - libattr1-dev
       - libbluetooth-dev
       - libbrlapi-dev
+      - libbz2-dev
       - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
@@ -22,6 +24,8 @@ addons:
       - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
+      - libncursesw5-dev
+      - libnfs-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
@@ -171,10 +175,12 @@ matrix:
             - gcc-5
             - g++-5
             # Build dependencies
+            - glusterfs-common
             - libaio-dev
             - libattr1-dev
             - libbluetooth-dev
             - libbrlapi-dev
+            - libbz2-dev
             - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
@@ -184,6 +190,8 @@ matrix:
             - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
+            - libncursesw5-dev
+            - libnfs-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 09/11] scripts/run-coverity-scan: Script to run Coverity Scan build
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (7 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 08/11] " Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 10/11] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 11/11] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé
  10 siblings, 0 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

From: Peter Maydell <peter.maydell@linaro.org>

Add a new script to automate the process of running the Coverity
Scan build tools and uploading the resulting tarball to the
website. This is primarily intended to be driven from Travis,
but it can be run locally (if you are a maintainer of the
QEMU project on the Coverity Scan website and have the secret
upload token).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 170 insertions(+)
 create mode 100755 scripts/run-coverity-scan

diff --git a/scripts/run-coverity-scan b/scripts/run-coverity-scan
new file mode 100755
index 0000000000..e6d5fc58d8
--- /dev/null
+++ b/scripts/run-coverity-scan
@@ -0,0 +1,170 @@
+#!/bin/sh -e
+
+# Upload a created tarball to Coverity Scan, as per
+# https://scan.coverity.com/projects/qemu/builds/new
+
+# This work is licensed under the terms of the GNU GPL version 2,
+# or (at your option) any later version.
+# See the COPYING file in the top-level directory.
+#
+# Copyright (c) 2017 Linaro Limited
+# Written by Peter Maydell
+
+# Note that this script will automatically download and
+# run the (closed-source) coverity build tools, so don't
+# use it if you don't trust them!
+
+# This script assumes that you're running it from a QEMU source
+# tree, and that tree is a fresh clean one, because we do an in-tree
+# build. (This is necessary so that the filenames that the Coverity
+# Scan server sees are relative paths that match up with the component
+# regular expressions it uses; an out-of-tree build won't work for this.)
+# The host machine should have as many of QEMU's dependencies
+# installed as possible, for maximum coverity coverage.
+
+# You need to pass the following environment variables to the script:
+#  COVERITY_TOKEN -- this is the secret 8 digit hex string which lets
+#                    you upload to Coverity Scan. If you're a maintainer
+#                    in Coverity then the web UI will tell you this.
+#  COVERITY_EMAIL -- the email address to use for uploads
+
+# and optionally
+#  COVERITY_DRYRUN -- set to not actually do the upload
+#  COVERITY_BUILD_CMD -- make command (defaults to 'make -j8')
+#  COVERITY_TOOL_BASE -- set to directory to put coverity tools
+#                        (defaults to /tmp/coverity-tools)
+
+# The primary purpose of this script is to be run as part of
+# a Travis build, but it is possible to run it manually locally.
+
+if [ -z "$COVERITY_TOKEN" ]; then
+    echo "COVERITY_TOKEN environment variable not set"
+    exit 1
+fi
+
+if [ -z "$COVERITY_EMAIL" ]; then
+    echo "COVERITY_EMAIL environment variable not set"
+    exit 1
+fi
+
+if [ -z "$COVERITY_BUILD_CMD" ]; then
+    echo "COVERITY_BUILD_CMD: using default 'make -j8'"
+    COVERITY_BUILD_CMD="make -j8"
+fi
+
+if [ -z "$COVERITY_TOOL_BASE" ]; then
+    echo "COVERITY_TOOL_BASE: using default /tmp/coverity-tools"
+    COVERITY_TOOL_BASE=/tmp/coverity-tools
+fi
+
+PROJTOKEN="$COVERITY_TOKEN"
+PROJNAME=QEMU
+TARBALL=cov-int.tar.xz
+SRCDIR="$(pwd)"
+
+echo "Checking this is a QEMU source tree..."
+if ! [ -e VERSION ]; then
+    echo "Not in a QEMU source tree?"
+    exit 1
+fi
+
+echo "Checking upload permissions..."
+
+if ! up_perm="$(wget https://scan.coverity.com/api/upload_permitted --post-data "token=$PROJTOKEN&project=$PROJNAME" -q -O -)"; then
+    echo "Coverity Scan API access denied: bad token?"
+    exit 1
+fi
+
+# Really up_perm is a JSON response with either
+# {upload_permitted:true} or {next_upload_permitted_at:<date>}
+# We do some hacky string parsing instead of properly parsing it.
+case "$up_perm" in
+    *upload_permitted*true*)
+        echo "Coverity Scan: upload permitted"
+        ;;
+    *next_upload_permitted_at*)
+        if [ -z "$COVERITY_DRYRUN" ]; then
+            echo "Coverity Scan: upload quota reached; stopping here"
+            # Exit success as this isn't a build error.
+            exit 0
+        else
+            echo "Coverity Scan: upload quota reached, continuing dry run"
+        fi
+        ;;
+    *)
+        echo "Coverity Scan upload check: unexpected result $up_perm"
+        exit 1
+        ;;
+esac
+
+mkdir -p "$COVERITY_TOOL_BASE"
+cd "$COVERITY_TOOL_BASE"
+
+echo "Checking for new version of coverity build tools..."
+wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME&md5=1" -O coverity_tool.md5.new
+
+if ! cmp -s coverity_tool.md5 coverity_tool.md5.new; then
+    # out of date md5 or no md5: download new build tool
+    # blow away the old build tool
+    echo "Downloading coverity build tools..."
+    rm -rf coverity_tool coverity_tool.tgz
+    wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME" -O coverity_tool.tgz
+    if ! (cat coverity_tool.md5.new; echo "  coverity_tool.tgz") | md5sum -c --status; then
+        echo "Downloaded tarball didn't match md5sum!"
+        exit 1
+    fi
+    # extract the new one, keeping it corralled in a 'coverity_tool' directory
+    echo "Unpacking coverity build tools..."
+    mkdir -p coverity_tool
+    cd coverity_tool
+    tar xf ../coverity_tool.tgz
+    cd ..
+    mv coverity_tool.md5.new coverity_tool.md5
+fi
+
+rm -f coverity_tool.md5.new
+
+TOOLBIN="$(echo $(pwd)/coverity_tool/cov-analysis-*/bin)"
+
+if ! test -x "$TOOLBIN/cov-build"; then
+    echo "Couldn't find cov-build in the coverity build-tool directory??"
+    exit 1
+fi
+
+export PATH="$TOOLBIN:$PATH"
+
+cd "$SRCDIR"
+
+echo "Doing make distclean..."
+make distclean
+
+echo "Configuring..."
+./configure --audio-drv-list=oss,alsa,sdl,pa --disable-werror
+
+echo "Making libqemustub.a..."
+make libqemustub.a
+
+echo "Running cov-build..."
+rm -rf cov-int
+mkdir cov-int
+cov-build --dir cov-int $COVERITY_BUILD_CMD
+
+echo "Creating results tarball..."
+tar cvf - cov-int | xz > "$TARBALL"
+
+echo "Uploading results tarball..."
+
+VERSION="$(git describe --always HEAD)"
+DESCRIPTION="$(git rev-parse HEAD)"
+
+if ! [ -z "$COVERITY_DRYRUN" ]; then
+    echo "Dry run only, not uploading $TARBALL"
+    exit 0
+fi
+
+curl --form token="$PROJTOKEN" --form email="$COVERITY_EMAIL" \
+     --form file=@"$TARBALL" --form version="$VERSION" \
+     --form description="$DESCRIPTION" \
+     https://scan.coverity.com/builds?project="$PROJNAME"
+
+echo "Done."
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 10/11] travis: Add config to do a Coverity Scan upload
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (8 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 09/11] scripts/run-coverity-scan: Script to run Coverity Scan build Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 11/11] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé
  10 siblings, 0 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

From: Peter Maydell <peter.maydell@linaro.org>

Add config to travis to do a Coverity Scan build and upload, using
the new run-coverity-scan script.

There is an official integration between Travis and Coverity Scan:
 https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/addons/coverity_scan.rb
which slurps values out of the .travis.yml and downloads a build
script from Coverity which does the bulk of the work:
 https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh

However we choose to roll our own since this seems less
confusing and also allows us to include debug features
(notably the ability to do a "dry run" test which doesn't
actually upload anything).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 4c0f7f444e..fc91e2fd28 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -220,3 +220,27 @@ matrix:
         - RUN_TESTS="no"
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
+    # Build and upload to Coverity Scan.
+    # We do not impose any rate limiting here, but instead rely on the
+    # limiting done by the coverity servers, which for a project of QEMU's
+    # size means one build a day. The run-coverity-scan script will exit
+    # early if the limiter does not permit a new upload, so the effect will
+    # be that the first build (only) in each 24 hour period will be scanned.
+    # If we needed to apply a limit at the Travis end, the simplest approach
+    # would be to run the scan only if the branch was 'coverity-scan', and
+    # use a cron job to push master to the 'coverity-scan' branch periodically.
+    # We run on the trusty Travis hosts so that there's a wider set of
+    # dependencies satisfied to improve coverage.
+    - dist: trusty
+      env:
+        - COVERITY=1
+        - COVERITY_BUILD_CMD="make -j3"
+        - COVERITY_EMAIL=peter.maydell@linaro.org
+        # This 'secure' setting sets COVERITY_TOKEN=<secret token>
+        # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
+        - secure: "D3E6E5bacui53fYBQrx0wQr8ZTvo6VIBPKfg0QHj2uwa6OPFkUlcMr/EHWvdbZNAa4Q1bv1vhlED5OPRfPmQYzxQNT4SAxDZeuZnikgIymfqQXNOjKw4kRUDO9P42QanyFd+EAu2JDVClAeJPgBpa/ns4CNrGDK+Q3coGndCP8o="
+      before_script:
+        - if [ "$TRAVIS_PULL_REQUEST" = "true" ]; then echo "Skipping Coverity (pullreq)"; exit 0; fi
+        - if [ "$TRAVIS_BRANCH" != "master" ]; then echo "Skipping Coverity (wrong branch)"; exit 0; fi
+      script:
+        - ./scripts/run-coverity-scan
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 11/11] MAINTAINERS: self-appoint me as reviewer in build/test automation
  2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
                   ` (9 preceding siblings ...)
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 10/11] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
@ 2017-06-16 16:13 ` Philippe Mathieu-Daudé
  10 siblings, 0 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-16 16:13 UTC (permalink / raw)
  To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
  Cc: Philippe Mathieu-Daudé

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 120788d8fb..e62cbc439a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1858,6 +1858,7 @@ Build and test automation
 -------------------------
 M: Alex Bennée <alex.bennee@linaro.org>
 M: Fam Zheng <famz@redhat.com>
+R: Philippe Mathieu-Daudé <f4bug@amsat.org>
 L: qemu-devel@nongnu.org
 S: Maintained
 F: .travis.yml
-- 
2.11.0

^ permalink raw reply related	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests Philippe Mathieu-Daudé
@ 2017-06-17  8:06   ` Alex Bennée
  2017-06-20 17:08     ` Philippe Mathieu-Daudé
  0 siblings, 1 reply; 22+ messages in thread
From: Alex Bennée @ 2017-06-17  8:06 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> having the command to run in the variable make scripts harder to
> manage.

I'm not sure this adds much over the TEST_CMD not set == don't run we
had before.

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>
> using 0/1 or no/yes is pretty much the same.
>
>  .travis.yml | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index b2c69fdd59..c9ac741afc 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -46,7 +46,7 @@ notifications:
>  env:
>    global:
>      - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
> -    - TEST_CMD="make check"
> +    - RUN_TESTS="yes"
>    matrix:
>      - CONFIG=""
>      - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
> @@ -66,7 +66,7 @@ before_script:
>    - ./configure ${CONFIG}
>  script:
>    - make -j${PARALLEL_JOBS}
> -  - test -z "${TEST_CMD}" || ${TEST_CMD}
> +  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
>  matrix:
>    include:
>      # Test with CLang for compile portability
> @@ -77,16 +77,16 @@ matrix:
>        compiler: gcc
>      # We manually include builds which we disable "make check" for
>      - env: CONFIG="--enable-debug --enable-tcg-interpreter"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG="--enable-trace-backends=simple"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG="--enable-trace-backends=ftrace"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG="--enable-trace-backends=ust"
> -           TEST_CMD=""
> +           RUN_TESTS="no"
>        compiler: gcc
>      - env: CONFIG=""
>        os: osx
> @@ -191,6 +191,6 @@ matrix:
>        env:
>          - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
>          - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
> -        - TEST_CMD=""
> +        - RUN_TESTS="no"
>        before_script:
>          - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log


--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 01/11] travis: build using all available cpus
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 01/11] travis: build using all available cpus Philippe Mathieu-Daudé
@ 2017-06-19 10:48   ` Alex Bennée
  0 siblings, 0 replies; 22+ messages in thread
From: Alex Bennée @ 2017-06-19 10:48 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> no improvement as of today, but if Travis release their limit on the opensource
> plan or upgrade their hardware the builds will get some benefit.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 27a2d9cfb3..d21a2a3602 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -45,6 +45,7 @@ notifications:
>      on_failure: always
>  env:
>    global:
> +    - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
>      - TEST_CMD="make check"
>    matrix:
>      - CONFIG=""
> @@ -64,7 +65,7 @@ before_install:
>  before_script:
>    - ./configure ${CONFIG}
>  script:
> -  - make -j3 && ${TEST_CMD}
> +  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 02/11] travis: split building/testing to have finer elapsed time
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 02/11] travis: split building/testing to have finer elapsed time Philippe Mathieu-Daudé
@ 2017-06-19 10:49   ` Alex Bennée
  0 siblings, 0 replies; 22+ messages in thread
From: Alex Bennée @ 2017-06-19 10:49 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> to check each step elapsed time on the travis output report.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>


> ---
>  .travis.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index d21a2a3602..b2c69fdd59 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -65,7 +65,8 @@ before_install:
>  before_script:
>    - ./configure ${CONFIG}
>  script:
> -  - make -j${PARALLEL_JOBS} && ${TEST_CMD}
> +  - make -j${PARALLEL_JOBS}
> +  - test -z "${TEST_CMD}" || ${TEST_CMD}
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 04/11] tests: add rule to compile many objects used by tests
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 04/11] tests: add rule to compile many objects used by tests Philippe Mathieu-Daudé
@ 2017-06-19 10:56   ` Alex Bennée
  0 siblings, 0 replies; 22+ messages in thread
From: Alex Bennée @ 2017-06-19 10:56 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> all those objects can get compiled simultaneously
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/Makefile.include | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index f42f3dfa72..3773f9d8d2 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -804,6 +804,7 @@ check-help:
>  	@echo " make check-qapi-schema    Run QAPI schema tests"
>  	@echo " make check-block          Run block tests"
>  	@echo " make check-report.html    Generates an HTML test report"
> +	@echo " make check-compile        Compile objects used by tests"
>  	@echo " make check-clean          Clean the tests"
>  	@echo
>  	@echo "Please note that HTML reports do not regenerate if the unit tests"
> @@ -887,7 +888,7 @@ check-tests/qapi-schema/doc-good.texi: tests/qapi-schema/doc-good.test.texi
>
>  # Consolidated targets
>
> -.PHONY: check-qapi-schema check-qtest check-unit check check-clean
> +.PHONY: check-qapi-schema check-qtest check-unit check check-clean check-compile
>  check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  check-unit: $(patsubst %,check-%, $(check-unit-y))
> @@ -897,7 +898,10 @@ check-clean:
>  	$(MAKE) -C tests/tcg clean
>  	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>  	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> -
> +check-compile: $(check-qtest-generic-y) $(check-unit-y) $(test-util-obj-y) $(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y) $(libqos-pc-obj-y) $(chardev-obj-y) $(QEMU_IOTESTS_HELPERS-y)
> +ifeq ($(CONFIG_SOFTMMU),y)
> +check-compile: $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)))
> +endif
>  clean: check-clean
>
>  # Build the help program automatically

Something has gone wrong here:

  11:54 alex@zen taken:111, git:review/travis-speedup, [/home/alex/lsrc/qemu/qemu.git]> make check-compile
          CHK version_gen.h
    LINK    tests/ac97-test
  tests/ac97-test.o: In function `main':
  /home/alex/lsrc/qemu/qemu.git/tests/ac97-test.c:23: undefined reference to `qtest_add_func'
  tests/ac97-test.o: In function `qtest_start':
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:521: undefined reference to `qtest_init'
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:521: undefined reference to `global_qtest'
  tests/ac97-test.o: In function `qtest_end':
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:532: undefined reference to `global_qtest'
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:532: undefined reference to `qtest_quit'
  /home/alex/lsrc/qemu/qemu.git/tests/libqtest.h:533: undefined reference to `global_qtest'
  collect2: error: ld returned 1 exit status
  /home/alex/lsrc/qemu/qemu.git/rules.mak:121: recipe for target 'tests/ac97-test' failed
  make: *** [tests/ac97-test] Error 1

Yet "make check" completes without issue.

Also I would expect the check: target to have check-compile as one of
its pre-requisites otherwise this target will bit rot.

--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially Philippe Mathieu-Daudé
@ 2017-06-19 10:58   ` Alex Bennée
  2017-06-19 11:12     ` Peter Maydell
  0 siblings, 1 reply; 22+ messages in thread
From: Alex Bennée @ 2017-06-19 10:58 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> tests are run sequentially to avoid mixed results output.

I'm not so sure we are worried about mixed results output. I don't think
anyone can compare the entire log and they tend to scroll to the end
where the error will be.

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c9ac741afc..dbbb11617d 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -66,7 +66,7 @@ before_script:
>    - ./configure ${CONFIG}
>  script:
>    - make -j${PARALLEL_JOBS}
> -  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
> +  - if [ "${RUN_TESTS}" == "yes" ]; then make -j${PARALLEL_JOBS} check-compile && make check; fi
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 06/11] travis: retry if llvm.org timeouts
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 06/11] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
@ 2017-06-19 11:00   ` Alex Bennée
  0 siblings, 0 replies; 22+ messages in thread
From: Alex Bennée @ 2017-06-19 11:00 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857
>
>     $ sudo apt-get update -qq
>     W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg  Connection failed
>     E: Some index files failed to download. They have been ignored, or old ones used instead.
>     The command "sudo apt-get update -qq" failed and exited with 100 during .
>     Your build has been stopped.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

This is good - I suspect there are other places we could use travis_retry.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index dbbb11617d..69a960b714 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -123,10 +123,10 @@ matrix:
>          - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
>          - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
>        before_install:
> -        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> -        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> -        - sudo apt-get update -qq
> -        - sudo apt-get install -qq -y clang-3.9
> +        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> +        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> +        - travis_retry sudo apt-get update -qq
> +        - travis_retry sudo apt-get install -qq -y clang-3.9
>          - sudo apt-get build-dep -qq qemu
>          - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive
> @@ -142,10 +142,10 @@ matrix:
>          - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
>          - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
>        before_install:
> -        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> -        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> -        - sudo apt-get update -qq
> -        - sudo apt-get install -qq -y clang-3.9
> +        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
> +        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
> +        - travis_retry sudo apt-get update -qq
> +        - travis_retry sudo apt-get install -qq -y clang-3.9
>          - sudo apt-get build-dep -qq qemu
>          - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive


--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 08/11] travis: install more library dependencies
  2017-06-16 16:13 ` [Qemu-devel] [PATCH 08/11] " Philippe Mathieu-Daudé
@ 2017-06-19 11:01   ` Alex Bennée
  0 siblings, 0 replies; 22+ messages in thread
From: Alex Bennée @ 2017-06-19 11:01 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> so more codebase is built
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Acked-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 4889c192c2..4c0f7f444e 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -9,10 +9,12 @@ addons:
>    apt:
>      packages:
>        # Build dependencies
> +      - glusterfs-common
>        - libaio-dev
>        - libattr1-dev
>        - libbluetooth-dev
>        - libbrlapi-dev
> +      - libbz2-dev
>        - libcap-dev
>        - libcap-ng-dev
>        - libgnutls-dev
> @@ -22,6 +24,8 @@ addons:
>        - liblzo2-dev
>        - libnfs-dev
>        - libncurses5-dev
> +      - libncursesw5-dev
> +      - libnfs-dev
>        - libnss3-dev
>        - libpixman-1-dev
>        - libpng12-dev
> @@ -171,10 +175,12 @@ matrix:
>              - gcc-5
>              - g++-5
>              # Build dependencies
> +            - glusterfs-common
>              - libaio-dev
>              - libattr1-dev
>              - libbluetooth-dev
>              - libbrlapi-dev
> +            - libbz2-dev
>              - libcap-dev
>              - libcap-ng-dev
>              - libgnutls-dev
> @@ -184,6 +190,8 @@ matrix:
>              - liblzo2-dev
>              - libnfs-dev
>              - libncurses5-dev
> +            - libncursesw5-dev
> +            - libnfs-dev
>              - libnss3-dev
>              - libpixman-1-dev
>              - libpng12-dev


--
Alex Bennée

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially
  2017-06-19 10:58   ` Alex Bennée
@ 2017-06-19 11:12     ` Peter Maydell
  2017-06-23 18:22       ` Philippe Mathieu-Daudé
  0 siblings, 1 reply; 22+ messages in thread
From: Peter Maydell @ 2017-06-19 11:12 UTC (permalink / raw)
  To: Alex Bennée; +Cc: Philippe Mathieu-Daudé, QEMU Developers, Fam Zheng

On 19 June 2017 at 11:58, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> tests are run sequentially to avoid mixed results output.
>
> I'm not so sure we are worried about mixed results output. I don't think
> anyone can compare the entire log and they tend to scroll to the end
> where the error will be.

Indeed, and make's --output-sync flag should be enough to keep
individual test output distinct.

What is perhaps a problem is that I'm not sure our tests all support
being run in parallel with each other without tripping each other up by
using the same temporary file / TCP port / etc at once. To the
extent that this doesn't work it's a bug, but we should start
by doing a solid soak test of the test suite to make sure -j<bignum>
works reliably.

thanks
-- PMM

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests
  2017-06-17  8:06   ` Alex Bennée
@ 2017-06-20 17:08     ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-20 17:08 UTC (permalink / raw)
  To: Alex Bennée; +Cc: qemu-devel, Fam Zheng, Peter Maydell

On 06/17/2017 05:06 AM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> having the command to run in the variable make scripts harder to
>> manage.
>
> I'm not sure this adds much over the TEST_CMD not set == don't run we
> had before.

I don't like it neither but had trouble expanding commands in $TEST_CMD. 
Anyway I drop this approach in the next version of this series.

>
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>
>> using 0/1 or no/yes is pretty much the same.
>>
>>  .travis.yml | 14 +++++++-------
>>  1 file changed, 7 insertions(+), 7 deletions(-)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index b2c69fdd59..c9ac741afc 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -46,7 +46,7 @@ notifications:
>>  env:
>>    global:
>>      - PARALLEL_JOBS="$(($(getconf _NPROCESSORS_ONLN) + 1))"
>> -    - TEST_CMD="make check"
>> +    - RUN_TESTS="yes"
>>    matrix:
>>      - CONFIG=""
>>      - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
>> @@ -66,7 +66,7 @@ before_script:
>>    - ./configure ${CONFIG}
>>  script:
>>    - make -j${PARALLEL_JOBS}
>> -  - test -z "${TEST_CMD}" || ${TEST_CMD}
>> +  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
>>  matrix:
>>    include:
>>      # Test with CLang for compile portability
>> @@ -77,16 +77,16 @@ matrix:
>>        compiler: gcc
>>      # We manually include builds which we disable "make check" for
>>      - env: CONFIG="--enable-debug --enable-tcg-interpreter"
>> -           TEST_CMD=""
>> +           RUN_TESTS="no"
>>        compiler: gcc
>>      - env: CONFIG="--enable-trace-backends=simple"
>> -           TEST_CMD=""
>> +           RUN_TESTS="no"
>>        compiler: gcc
>>      - env: CONFIG="--enable-trace-backends=ftrace"
>> -           TEST_CMD=""
>> +           RUN_TESTS="no"
>>        compiler: gcc
>>      - env: CONFIG="--enable-trace-backends=ust"
>> -           TEST_CMD=""
>> +           RUN_TESTS="no"
>>        compiler: gcc
>>      - env: CONFIG=""
>>        os: osx
>> @@ -191,6 +191,6 @@ matrix:
>>        env:
>>          - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
>>          - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
>> -        - TEST_CMD=""
>> +        - RUN_TESTS="no"
>>        before_script:
>>          - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
>
>
> --
> Alex Bennée
>

^ permalink raw reply	[flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially
  2017-06-19 11:12     ` Peter Maydell
@ 2017-06-23 18:22       ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 22+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-23 18:22 UTC (permalink / raw)
  To: Peter Maydell, Alex Bennée; +Cc: QEMU Developers, Fam Zheng

On 06/19/2017 08:12 AM, Peter Maydell wrote:

> What is perhaps a problem is that I'm not sure our tests all support
> being run in parallel with each other without tripping each other up by
> using the same temporary file / TCP port / etc at once. To the
> extent that this doesn't work it's a bug, but we should start
> by doing a solid soak test of the test suite to make sure -j<bignum>
> works reliably.

I might have caught one:

$ gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

$ export MAKEFLAGS="-j ${PARALLEL_JOBS}"

$ echo "Using ${PARALLEL_JOBS} parallel jobs"
Using 3 parallel jobs

$ ${TEST_CMD}
[...]
   GTESTER check-qtest-unicore32
   GTESTER check-qtest-x86_64
   GTESTER check-qtest-xtensa
   GTESTER check-qtest-xtensaeb
   GTESTER tests/test-qobject-output-visitor
   GTESTER tests/test-clone-visitor
   GTESTER tests/test-qobject-input-visitor
   GTESTER tests/test-qmp-commands
   GTESTER tests/test-string-input-visitor
   GTESTER tests/test-string-output-visitor
   GTESTER tests/test-qmp-event
   GTESTER tests/test-opts-visitor
   GTESTER tests/test-qht-par
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
**
ERROR:tests/vhost-user-test.c:807:test_connect_fail: child process 
(/x86_64/vhost-user/connect-fail/subprocess [2642]) failed unexpectedly
qemu-system-x86_64: Failed to set msg fds.
qemu-system-x86_64: vhost VQ 0 ring restore failed: -1: Resource 
temporarily unavailable (11)
qemu-system-x86_64: Failed to set msg fds.
qemu-system-x86_64: vhost VQ 1 ring restore failed: -1: Resource 
temporarily unavailable (11)
GTester: last random seed: R02S8c0c7c003144320339c3c86cfa72ba62
make: *** [check-qtest-x86_64] Error 1

(build: https://travis-ci.org/philmd/qemu/jobs/246199793)

^ permalink raw reply	[flat|nested] 22+ messages in thread
end of thread, other threads:[~2017-06-23 18:22 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-16 16:13 [Qemu-devel] [PATCH 00/11] travis: try to reduce failures Philippe Mathieu-Daudé
2017-06-16 16:13 ` [Qemu-devel] [PATCH 01/11] travis: build using all available cpus Philippe Mathieu-Daudé
2017-06-19 10:48   ` Alex Bennée
2017-06-16 16:13 ` [Qemu-devel] [PATCH 02/11] travis: split building/testing to have finer elapsed time Philippe Mathieu-Daudé
2017-06-19 10:49   ` Alex Bennée
2017-06-16 16:13 ` [Qemu-devel] [PATCH 03/11] travis: use yes/no variable to enable/disable tests Philippe Mathieu-Daudé
2017-06-17  8:06   ` Alex Bennée
2017-06-20 17:08     ` Philippe Mathieu-Daudé
2017-06-16 16:13 ` [Qemu-devel] [PATCH 04/11] tests: add rule to compile many objects used by tests Philippe Mathieu-Daudé
2017-06-19 10:56   ` Alex Bennée
2017-06-16 16:13 ` [Qemu-devel] [PATCH 05/11] travis: build tests objects in parallel, then run tests sequentially Philippe Mathieu-Daudé
2017-06-19 10:58   ` Alex Bennée
2017-06-19 11:12     ` Peter Maydell
2017-06-23 18:22       ` Philippe Mathieu-Daudé
2017-06-16 16:13 ` [Qemu-devel] [PATCH 06/11] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
2017-06-19 11:00   ` Alex Bennée
2017-06-16 16:13 ` [Qemu-devel] [PATCH 07/11] travis: install more library dependencies Philippe Mathieu-Daudé
2017-06-16 16:13 ` [Qemu-devel] [PATCH 08/11] " Philippe Mathieu-Daudé
2017-06-19 11:01   ` Alex Bennée
2017-06-16 16:13 ` [Qemu-devel] [PATCH 09/11] scripts/run-coverity-scan: Script to run Coverity Scan build Philippe Mathieu-Daudé
2017-06-16 16:13 ` [Qemu-devel] [PATCH 10/11] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
2017-06-16 16:13 ` [Qemu-devel] [PATCH 11/11] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.