* [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures
@ 2017-06-22 3:32 Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
` (20 more replies)
0 siblings, 21 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
These patches try to improve our Travis CI usage (quite a few failures the last
days).
This series include Peter's "Automate coverity scan uploads via Travis"
patches (here numbered 3 to 6). See
https://www.mail-archive.com/qemu-devel@nongnu.org/msg457443.html
Patch 7 is expected to enter /master as of today.
Travis warned few months ago:
"On Wednesday, June 21st 2017, we are going to update all our Ubuntu Trusty
14.04 images."
Patches 2, 10 and 15 use 'travis_retry' feature to avoid aborting a job on
network failure, retrying 3 times (default).
Patches 8, 19 are upgrades allowing to use recent VMs/toolchains.
Patch 12 speed up builds using multicore parallelism, caring about not overkill
Travis hardware.
Patches 9, 11, 13, 17 speed up builds improving cache usage.
Patches 18, 21 silent warnings.
Patch 14 gives hint when ./configure fails.
Patch 16 split slow gprof/gcov job.
Finally patch 1 add a missing rule to build system-qemu binary before running
qtests.
time comparison:
----------------
current /master https://travis-ci.org/qemu/qemu/builds/245061137:
Ran for 2 hrs 4 min 17 sec
Total time 7 hrs 34 min 22 sec
this seris with 3 more jobs https://travis-ci.org/philmd/qemu/builds/245593509:
Ran for 1 hr 26 min 35 sec
Total time 6 hrs 32 min 49 sec
Regards,
Phil.
Peter Maydell (3):
travis: install more library dependencies
scripts/run-coverity-scan: Script to run Coverity Scan build
travis: Add config to do a Coverity Scan upload
Philippe Mathieu-Daudé (18):
tests: add missing dependency to build QTEST_QEMU_BINARY
travis: retry if llvm.org timeouts
travis: install more library dependencies
travis: update sudo-enabled Trusty images
travis: use gcc-6 sanitizers
travis: enable multiple caching features
travis: increase S3 cache timeout
travis: cache git submodules
travis: build using all available cores
travis: improve ccache use
travis: dump config.log if ./configure script fails
travis: retry when git submodules initialization fails
travis: split the gprof/gcov job
travis/osx: don't update Homebrew cache
travis/osx: silent texinfo warnings
travis/osx: build using more Xcode versions
MAINTAINERS: self-appoint me as reviewer in build/test automation
ui/vnc: silent unuseful OSX clang warning
.travis.yml | 135 +++++++++++++++++++++++++++++-------
MAINTAINERS | 1 +
scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
tests/Makefile.include | 2 +-
ui/Makefile.objs | 8 +++
5 files changed, 290 insertions(+), 26 deletions(-)
create mode 100755 scripts/run-coverity-scan
--
2.11.0
^ permalink raw reply [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 9:55 ` Alex Bennée
` (2 more replies)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 02/21] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
` (19 subsequent siblings)
20 siblings, 3 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell,
John Snow, Paolo Bonzini, Stefan Hajnoczi, Markus Armbruster
Cc: Philippe Mathieu-Daudé
This allow a one liner from fresh repository clone, i.e.:
./configure && make -j check-qtest-aarch64
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
tests/Makefile.include | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 77da9b7f4b..f7c8f3eddb 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -823,7 +823,7 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
# gtester tests, possibly with verbose output
.PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
-$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
+$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-softmmu $(check-qtest-y)
$(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
$(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
QTEST_QEMU_IMG=qemu-img$(EXESUF) \
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 02/21] travis: retry if llvm.org timeouts
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 03/21] travis: install more library dependencies Philippe Mathieu-Daudé
` (18 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857
$ sudo apt-get update -qq
W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg Connection failed
E: Some index files failed to download. They have been ignored, or old ones used instead.
The command "sudo apt-get update -qq" failed and exited with 100 during .
Your build has been stopped.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
.travis.yml | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 27a2d9cfb3..f78300ee99 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -121,10 +121,10 @@ matrix:
- COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
- CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
before_install:
- - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
- - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
- - sudo apt-get update -qq
- - sudo apt-get install -qq -y clang-3.9
+ - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+ - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+ - travis_retry sudo apt-get update -qq
+ - travis_retry sudo apt-get install -qq -y clang-3.9
- sudo apt-get build-dep -qq qemu
- wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
@@ -140,10 +140,10 @@ matrix:
- COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
- CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
before_install:
- - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
- - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
- - sudo apt-get update -qq
- - sudo apt-get install -qq -y clang-3.9
+ - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+ - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+ - travis_retry sudo apt-get update -qq
+ - travis_retry sudo apt-get install -qq -y clang-3.9
- sudo apt-get build-dep -qq qemu
- wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 03/21] travis: install more library dependencies
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 02/21] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 04/21] " Philippe Mathieu-Daudé
` (17 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
From: Peter Maydell <peter.maydell@linaro.org>
Update the travis list of library packages to install so that
our build tests cover more of our code base.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index f78300ee99..53d8e79bf5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,28 +11,37 @@ addons:
# Build dependencies
- libaio-dev
- libattr1-dev
+ - libbluetooth-dev
- libbrlapi-dev
+ - libcap-dev
- libcap-ng-dev
- libgnutls-dev
- libgtk-3-dev
- libiscsi-dev
- liblttng-ust-dev
+ - liblzo2-dev
- libnfs-dev
- libncurses5-dev
- libnss3-dev
- libpixman-1-dev
- libpng12-dev
- librados-dev
+ - librdmacm-dev
- libsdl1.2-dev
- libseccomp-dev
+ - libsnappy-dev
- libspice-protocol-dev
- libspice-server-dev
- libssh2-1-dev
- liburcu-dev
- libusb-1.0-0-dev
+ - libvde-dev
- libvte-2.90-dev
+ - libxen-dev
+ - nettle-dev
- sparse
- uuid-dev
+ - xfslibs-dev
# The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
# to prevent IRC notifications from forks. This was created using:
@@ -162,28 +171,37 @@ matrix:
# Build dependencies
- libaio-dev
- libattr1-dev
+ - libbluetooth-dev
- libbrlapi-dev
+ - libcap-dev
- libcap-ng-dev
- libgnutls-dev
- libgtk-3-dev
- libiscsi-dev
- liblttng-ust-dev
+ - liblzo2-dev
- libnfs-dev
- libncurses5-dev
- libnss3-dev
- libpixman-1-dev
- libpng12-dev
- librados-dev
+ - librdmacm-dev
- libsdl1.2-dev
- libseccomp-dev
+ - libsnappy-dev
- libspice-protocol-dev
- libspice-server-dev
- libssh2-1-dev
- liburcu-dev
- libusb-1.0-0-dev
+ - libvde-dev
- libvte-2.90-dev
+ - libxen-dev
+ - nettle-dev
- sparse
- uuid-dev
+ - xfslibs-dev
language: generic
compiler: none
env:
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 04/21] travis: install more library dependencies
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 03/21] travis: install more library dependencies Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 05/21] scripts/run-coverity-scan: Script to run Coverity Scan build Philippe Mathieu-Daudé
` (16 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
so more codebase is built
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
---
.travis.yml | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 53d8e79bf5..0220f7472e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -9,10 +9,12 @@ addons:
apt:
packages:
# Build dependencies
+ - glusterfs-common
- libaio-dev
- libattr1-dev
- libbluetooth-dev
- libbrlapi-dev
+ - libbz2-dev
- libcap-dev
- libcap-ng-dev
- libgnutls-dev
@@ -22,6 +24,8 @@ addons:
- liblzo2-dev
- libnfs-dev
- libncurses5-dev
+ - libncursesw5-dev
+ - libnfs-dev
- libnss3-dev
- libpixman-1-dev
- libpng12-dev
@@ -169,10 +173,12 @@ matrix:
- gcc-5
- g++-5
# Build dependencies
+ - glusterfs-common
- libaio-dev
- libattr1-dev
- libbluetooth-dev
- libbrlapi-dev
+ - libbz2-dev
- libcap-dev
- libcap-ng-dev
- libgnutls-dev
@@ -182,6 +188,8 @@ matrix:
- liblzo2-dev
- libnfs-dev
- libncurses5-dev
+ - libncursesw5-dev
+ - libnfs-dev
- libnss3-dev
- libpixman-1-dev
- libpng12-dev
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 05/21] scripts/run-coverity-scan: Script to run Coverity Scan build
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (3 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 04/21] " Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
` (15 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
From: Peter Maydell <peter.maydell@linaro.org>
Add a new script to automate the process of running the Coverity
Scan build tools and uploading the resulting tarball to the
website. This is primarily intended to be driven from Travis,
but it can be run locally (if you are a maintainer of the
QEMU project on the Coverity Scan website and have the secret
upload token).
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 170 insertions(+)
create mode 100755 scripts/run-coverity-scan
diff --git a/scripts/run-coverity-scan b/scripts/run-coverity-scan
new file mode 100755
index 0000000000..e6d5fc58d8
--- /dev/null
+++ b/scripts/run-coverity-scan
@@ -0,0 +1,170 @@
+#!/bin/sh -e
+
+# Upload a created tarball to Coverity Scan, as per
+# https://scan.coverity.com/projects/qemu/builds/new
+
+# This work is licensed under the terms of the GNU GPL version 2,
+# or (at your option) any later version.
+# See the COPYING file in the top-level directory.
+#
+# Copyright (c) 2017 Linaro Limited
+# Written by Peter Maydell
+
+# Note that this script will automatically download and
+# run the (closed-source) coverity build tools, so don't
+# use it if you don't trust them!
+
+# This script assumes that you're running it from a QEMU source
+# tree, and that tree is a fresh clean one, because we do an in-tree
+# build. (This is necessary so that the filenames that the Coverity
+# Scan server sees are relative paths that match up with the component
+# regular expressions it uses; an out-of-tree build won't work for this.)
+# The host machine should have as many of QEMU's dependencies
+# installed as possible, for maximum coverity coverage.
+
+# You need to pass the following environment variables to the script:
+# COVERITY_TOKEN -- this is the secret 8 digit hex string which lets
+# you upload to Coverity Scan. If you're a maintainer
+# in Coverity then the web UI will tell you this.
+# COVERITY_EMAIL -- the email address to use for uploads
+
+# and optionally
+# COVERITY_DRYRUN -- set to not actually do the upload
+# COVERITY_BUILD_CMD -- make command (defaults to 'make -j8')
+# COVERITY_TOOL_BASE -- set to directory to put coverity tools
+# (defaults to /tmp/coverity-tools)
+
+# The primary purpose of this script is to be run as part of
+# a Travis build, but it is possible to run it manually locally.
+
+if [ -z "$COVERITY_TOKEN" ]; then
+ echo "COVERITY_TOKEN environment variable not set"
+ exit 1
+fi
+
+if [ -z "$COVERITY_EMAIL" ]; then
+ echo "COVERITY_EMAIL environment variable not set"
+ exit 1
+fi
+
+if [ -z "$COVERITY_BUILD_CMD" ]; then
+ echo "COVERITY_BUILD_CMD: using default 'make -j8'"
+ COVERITY_BUILD_CMD="make -j8"
+fi
+
+if [ -z "$COVERITY_TOOL_BASE" ]; then
+ echo "COVERITY_TOOL_BASE: using default /tmp/coverity-tools"
+ COVERITY_TOOL_BASE=/tmp/coverity-tools
+fi
+
+PROJTOKEN="$COVERITY_TOKEN"
+PROJNAME=QEMU
+TARBALL=cov-int.tar.xz
+SRCDIR="$(pwd)"
+
+echo "Checking this is a QEMU source tree..."
+if ! [ -e VERSION ]; then
+ echo "Not in a QEMU source tree?"
+ exit 1
+fi
+
+echo "Checking upload permissions..."
+
+if ! up_perm="$(wget https://scan.coverity.com/api/upload_permitted --post-data "token=$PROJTOKEN&project=$PROJNAME" -q -O -)"; then
+ echo "Coverity Scan API access denied: bad token?"
+ exit 1
+fi
+
+# Really up_perm is a JSON response with either
+# {upload_permitted:true} or {next_upload_permitted_at:<date>}
+# We do some hacky string parsing instead of properly parsing it.
+case "$up_perm" in
+ *upload_permitted*true*)
+ echo "Coverity Scan: upload permitted"
+ ;;
+ *next_upload_permitted_at*)
+ if [ -z "$COVERITY_DRYRUN" ]; then
+ echo "Coverity Scan: upload quota reached; stopping here"
+ # Exit success as this isn't a build error.
+ exit 0
+ else
+ echo "Coverity Scan: upload quota reached, continuing dry run"
+ fi
+ ;;
+ *)
+ echo "Coverity Scan upload check: unexpected result $up_perm"
+ exit 1
+ ;;
+esac
+
+mkdir -p "$COVERITY_TOOL_BASE"
+cd "$COVERITY_TOOL_BASE"
+
+echo "Checking for new version of coverity build tools..."
+wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME&md5=1" -O coverity_tool.md5.new
+
+if ! cmp -s coverity_tool.md5 coverity_tool.md5.new; then
+ # out of date md5 or no md5: download new build tool
+ # blow away the old build tool
+ echo "Downloading coverity build tools..."
+ rm -rf coverity_tool coverity_tool.tgz
+ wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME" -O coverity_tool.tgz
+ if ! (cat coverity_tool.md5.new; echo " coverity_tool.tgz") | md5sum -c --status; then
+ echo "Downloaded tarball didn't match md5sum!"
+ exit 1
+ fi
+ # extract the new one, keeping it corralled in a 'coverity_tool' directory
+ echo "Unpacking coverity build tools..."
+ mkdir -p coverity_tool
+ cd coverity_tool
+ tar xf ../coverity_tool.tgz
+ cd ..
+ mv coverity_tool.md5.new coverity_tool.md5
+fi
+
+rm -f coverity_tool.md5.new
+
+TOOLBIN="$(echo $(pwd)/coverity_tool/cov-analysis-*/bin)"
+
+if ! test -x "$TOOLBIN/cov-build"; then
+ echo "Couldn't find cov-build in the coverity build-tool directory??"
+ exit 1
+fi
+
+export PATH="$TOOLBIN:$PATH"
+
+cd "$SRCDIR"
+
+echo "Doing make distclean..."
+make distclean
+
+echo "Configuring..."
+./configure --audio-drv-list=oss,alsa,sdl,pa --disable-werror
+
+echo "Making libqemustub.a..."
+make libqemustub.a
+
+echo "Running cov-build..."
+rm -rf cov-int
+mkdir cov-int
+cov-build --dir cov-int $COVERITY_BUILD_CMD
+
+echo "Creating results tarball..."
+tar cvf - cov-int | xz > "$TARBALL"
+
+echo "Uploading results tarball..."
+
+VERSION="$(git describe --always HEAD)"
+DESCRIPTION="$(git rev-parse HEAD)"
+
+if ! [ -z "$COVERITY_DRYRUN" ]; then
+ echo "Dry run only, not uploading $TARBALL"
+ exit 0
+fi
+
+curl --form token="$PROJTOKEN" --form email="$COVERITY_EMAIL" \
+ --form file=@"$TARBALL" --form version="$VERSION" \
+ --form description="$DESCRIPTION" \
+ https://scan.coverity.com/builds?project="$PROJNAME"
+
+echo "Done."
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (4 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 05/21] scripts/run-coverity-scan: Script to run Coverity Scan build Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 9:56 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images Philippe Mathieu-Daudé
` (14 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
From: Peter Maydell <peter.maydell@linaro.org>
Add config to travis to do a Coverity Scan build and upload, using
the new run-coverity-scan script.
There is an official integration between Travis and Coverity Scan:
https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/addons/coverity_scan.rb
which slurps values out of the .travis.yml and downloads a build
script from Coverity which does the bulk of the work:
https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh
However we choose to roll our own since this seems less
confusing and also allows us to include debug features
(notably the ability to do a "dry run" test which doesn't
actually upload anything).
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 0220f7472e..29c9ef72a4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -218,3 +218,27 @@ matrix:
- TEST_CMD=""
before_script:
- ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
+ # Build and upload to Coverity Scan.
+ # We do not impose any rate limiting here, but instead rely on the
+ # limiting done by the coverity servers, which for a project of QEMU's
+ # size means one build a day. The run-coverity-scan script will exit
+ # early if the limiter does not permit a new upload, so the effect will
+ # be that the first build (only) in each 24 hour period will be scanned.
+ # If we needed to apply a limit at the Travis end, the simplest approach
+ # would be to run the scan only if the branch was 'coverity-scan', and
+ # use a cron job to push master to the 'coverity-scan' branch periodically.
+ # We run on the trusty Travis hosts so that there's a wider set of
+ # dependencies satisfied to improve coverage.
+ - dist: trusty
+ env:
+ - COVERITY=1
+ - COVERITY_BUILD_CMD="make -j3"
+ - COVERITY_EMAIL=peter.maydell@linaro.org
+ # This 'secure' setting sets COVERITY_TOKEN=<secret token>
+ # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
+ - secure: "D3E6E5bacui53fYBQrx0wQr8ZTvo6VIBPKfg0QHj2uwa6OPFkUlcMr/EHWvdbZNAa4Q1bv1vhlED5OPRfPmQYzxQNT4SAxDZeuZnikgIymfqQXNOjKw4kRUDO9P42QanyFd+EAu2JDVClAeJPgBpa/ns4CNrGDK+Q3coGndCP8o="
+ before_script:
+ - if [ "$TRAVIS_PULL_REQUEST" = "true" ]; then echo "Skipping Coverity (pullreq)"; exit 0; fi
+ - if [ "$TRAVIS_BRANCH" != "master" ]; then echo "Skipping Coverity (wrong branch)"; exit 0; fi
+ script:
+ - ./scripts/run-coverity-scan
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (5 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 9:58 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 08/21] travis: use gcc-6 sanitizers Philippe Mathieu-Daudé
` (13 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 29c9ef72a4..e60a5cd8e6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -128,6 +128,7 @@ matrix:
- sudo: required
addons:
dist: trusty
+ group: edge
language: generic
compiler: none
env:
@@ -147,6 +148,7 @@ matrix:
- sudo: required
addons:
dist: trusty
+ group: edge
language: generic
compiler: none
env:
@@ -230,6 +232,7 @@ matrix:
# We run on the trusty Travis hosts so that there's a wider set of
# dependencies satisfied to improve coverage.
- dist: trusty
+ group: edge
env:
- COVERITY=1
- COVERITY_BUILD_CMD="make -j3"
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 08/21] travis: use gcc-6 sanitizers
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (6 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features Philippe Mathieu-Daudé
` (12 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index e60a5cd8e6..61394b4764 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -172,8 +172,8 @@ matrix:
- ubuntu-toolchain-r-test
packages:
# Extra toolchains
- - gcc-5
- - g++-5
+ - gcc-6
+ - g++-6
# Build dependencies
- glusterfs-common
- libaio-dev
@@ -215,8 +215,8 @@ matrix:
language: generic
compiler: none
env:
- - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
- - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
+ - COMPILER_NAME=gcc CXX=g++-6 CC=gcc-6
+ - CONFIG="--disable-pie --disable-linux-user"
- TEST_CMD=""
before_script:
- ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (7 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 08/21] travis: use gcc-6 sanitizers Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:16 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout Philippe Mathieu-Daudé
` (11 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
(this eases git workflow)
see https://docs.travis-ci.com/user/caching#enabling-multiple-caching-features
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 61394b4764..be28e0642c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -4,7 +4,8 @@ python:
- "2.4"
compiler:
- gcc
-cache: ccache
+cache:
+ ccache: true
addons:
apt:
packages:
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (8 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:17 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules Philippe Mathieu-Daudé
` (10 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.travis.yml b/.travis.yml
index be28e0642c..62b9dfd2ae 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,6 +6,7 @@ compiler:
- gcc
cache:
ccache: true
+ timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
addons:
apt:
packages:
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (9 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:17 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores Philippe Mathieu-Daudé
` (9 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Travis caching uses the branch name to store packed cache, so each new branch
will trigger a cache miss and will clone all submodules. Subsequent builds will
benefit from the cache.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 62b9dfd2ae..777914f759 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -7,6 +7,8 @@ compiler:
cache:
ccache: true
timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
+ directories:
+ - $HOME/qemu/.git/modules
addons:
apt:
packages:
@@ -74,7 +76,6 @@ git:
before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
- - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
before_script:
- ./configure ${CONFIG}
@@ -113,7 +114,6 @@ matrix:
before_install:
- sudo apt-get update -qq
- sudo apt-get build-dep -qq qemu
- - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
# Plain Trusty Linux User Build
- env: CONFIG="--disable-system"
@@ -124,7 +124,6 @@ matrix:
before_install:
- sudo apt-get update -qq
- sudo apt-get build-dep -qq qemu
- - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
# Trusty System build with latest stable clang
- sudo: required
@@ -142,7 +141,6 @@ matrix:
- travis_retry sudo apt-get update -qq
- travis_retry sudo apt-get install -qq -y clang-3.9
- sudo apt-get build-dep -qq qemu
- - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
before_script:
- ./configure ${CONFIG} || cat config.log
@@ -162,7 +160,6 @@ matrix:
- travis_retry sudo apt-get update -qq
- travis_retry sudo apt-get install -qq -y clang-3.9
- sudo apt-get build-dep -qq qemu
- - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
- git submodule update --init --recursive
before_script:
- ./configure ${CONFIG} || cat config.log
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (10 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:21 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use Philippe Mathieu-Daudé
` (8 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Few improvement as of today, but if Travis release their limit on the opensource
plan or upgrade their hardware, new builds will get some benefit.
Restrict to as most 4 cores to respect Travis Open Source plan.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 777914f759..a7512f3ab0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -62,7 +62,14 @@ notifications:
on_failure: always
env:
global:
- - TEST_CMD="make check"
+ # jobs
+ # detects cores on the hosting hardware, rather than the VM (https://docs.travis-ci.com/user/languages/c#hints)
+ - VM_CORES="4" HW_CORES="$(getconf _NPROCESSORS_ONLN)"
+ # be kind with Travis free plan, restrict to at most 4 cores
+ - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
+ - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
+ # tests
+ - TEST_CMD="make -j${PARALLEL_JOBS} check"
matrix:
- CONFIG=""
- CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
@@ -78,9 +85,11 @@ before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
- git submodule update --init --recursive
before_script:
+ - echo "Using ${PARALLEL_JOBS} parallel jobs"
- ./configure ${CONFIG}
script:
- - make -j3 && ${TEST_CMD}
+ - make -j${PARALLEL_JOBS}
+ - ${TEST_CMD}
matrix:
include:
# Test with CLang for compile portability
@@ -234,7 +243,7 @@ matrix:
group: edge
env:
- COVERITY=1
- - COVERITY_BUILD_CMD="make -j3"
+ - COVERITY_BUILD_CMD="make -j${PARALLEL_JOBS}"
- COVERITY_EMAIL=peter.maydell@linaro.org
# This 'secure' setting sets COVERITY_TOKEN=<secret token>
# and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (11 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:22 ` Alex Bennée
2017-06-22 10:57 ` Paolo Bonzini
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails Philippe Mathieu-Daudé
` (7 subsequent siblings)
20 siblings, 2 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index a7512f3ab0..96ddc16cfd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -68,6 +68,8 @@ env:
# be kind with Travis free plan, restrict to at most 4 cores
- PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
- HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
+ # ccache
+ - CCACHE_SLOPPINESS="file_macro,time_macros"
# tests
- TEST_CMD="make -j${PARALLEL_JOBS} check"
matrix:
@@ -82,14 +84,21 @@ git:
submodules: false
before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
- - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
+ - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
- git submodule update --init --recursive
before_script:
+ # https://docs.travis-ci.com/user/caching#ccache-on-os-x
+ - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
+ - if [ "$CC" == "clang" ]; then export CCACHE_CPP2=1 ; fi
+ - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
- echo "Using ${PARALLEL_JOBS} parallel jobs"
+ - export HOST_CC="${CC}"
- ./configure ${CONFIG}
script:
- - make -j${PARALLEL_JOBS}
+ - make -j${PARALLEL_JOBS} && ccache -s
- ${TEST_CMD}
+before_cache:
+ - ccache -s
matrix:
include:
# Test with CLang for compile portability
@@ -112,6 +121,7 @@ matrix:
TEST_CMD=""
compiler: gcc
- env: CONFIG=""
+ CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
os: osx
compiler: clang
# Plain Trusty System Build
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (12 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:23 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails Philippe Mathieu-Daudé
` (6 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 96ddc16cfd..bac5bea744 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -93,7 +93,7 @@ before_script:
- if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
- echo "Using ${PARALLEL_JOBS} parallel jobs"
- export HOST_CC="${CC}"
- - ./configure ${CONFIG}
+ - ./configure ${CONFIG} || cat config.log
script:
- make -j${PARALLEL_JOBS} && ccache -s
- ${TEST_CMD}
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (13 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:24 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job Philippe Mathieu-Daudé
` (5 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
example of failure: https://travis-ci.org/philmd/qemu/jobs/245612939
$ git submodule update --init --recursive
[...]
Submodule 'pixman' (git://anongit.freedesktop.org/pixman) registered for path 'pixman'
Cloning into 'pixman'...
fatal: unable to connect to anongit.freedesktop.org:
anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out
anongit.freedesktop.org[1: 2610:10:20:722:a800:ff:fe24:61cf]: errno=Network is unreachable
Clone of 'git://anongit.freedesktop.org/pixman' into submodule path 'pixman' failed
The command "git submodule update --init --recursive" failed and exited with 1 during .
Your build has been stopped.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index bac5bea744..34391722a2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -85,7 +85,7 @@ git:
before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
- - git submodule update --init --recursive
+ - travis_retry git submodule update --init --recursive
before_script:
# https://docs.travis-ci.com/user/caching#ccache-on-os-x
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (14 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 10:24 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 17/21] travis/osx: don't update Homebrew cache Philippe Mathieu-Daudé
` (4 subsequent siblings)
20 siblings, 1 reply; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
gcov generates lot of output and often trigger job limit timeout
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 34391722a2..4bb288a192 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -105,7 +105,9 @@ matrix:
- env: CONFIG=""
compiler: clang
# gprof/gcov are GCC features
- - env: CONFIG="--enable-gprof --enable-gcov --disable-pie"
+ - env: CONFIG="--enable-gprof --disable-pie"
+ compiler: gcc
+ - env: CONFIG="--enable-gcov --disable-pie"
compiler: gcc
# We manually include builds which we disable "make check" for
- env: CONFIG="--enable-debug --enable-tcg-interpreter"
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 17/21] travis/osx: don't update Homebrew cache
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (15 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 18/21] travis/osx: silent texinfo warnings Philippe Mathieu-Daudé
` (3 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
installed VMs come already updated, and brew is smart enough to upgrade some
dependencies are missing.
see https://docs.travis-ci.com/user/osx-ci-environment#homebrew
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 1 -
1 file changed, 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 4bb288a192..3de3798121 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -83,7 +83,6 @@ git:
# we want to do this ourselves
submodules: false
before_install:
- - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
- travis_retry git submodule update --init --recursive
before_script:
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 18/21] travis/osx: silent texinfo warnings
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (16 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 17/21] travis/osx: don't update Homebrew cache Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 19/21] travis/osx: build using more Xcode versions Philippe Mathieu-Daudé
` (2 subsequent siblings)
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
$ make info
GEN qemu-doc.html
qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
GEN qemu-doc.txt
qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 3de3798121..d007fdc9fb 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -84,6 +84,8 @@ git:
submodules: false
before_install:
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
+ # silent texinfo warnings (https://github.com/xiaohanyu/oh-my-emacs/blob/c664894e2f1c1cb0f95a9f2da88d41b00f190856/core/ome-basic.org#homebrew)
+ - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install texinfo && brew link texinfo --force ; fi
- travis_retry git submodule update --init --recursive
before_script:
# https://docs.travis-ci.com/user/caching#ccache-on-os-x
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 19/21] travis/osx: build using more Xcode versions
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (17 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 18/21] travis/osx: silent texinfo warnings Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 20/21] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning Philippe Mathieu-Daudé
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
currently default builder is based on Xcode 7.3.1:
OS X 10.11 (darwin14.5.0)
Apple LLVM version 7.3.0
add the oldest available Xcode (6.4):
OS X 10.10 (darwin14.5.0)
Apple LLVM 6.1.0 (based on LLVM 3.6.0svn)
and the newer available Xcode (8.3.3):
OS X 10.12 (darwin16.6.0)
Apple LLVM version 8.1.0
see https://docs.travis-ci.com/user/osx-ci-environment#os-x-version
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
.travis.yml | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index d007fdc9fb..880a057c78 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -126,6 +126,17 @@ matrix:
- env: CONFIG=""
CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
os: osx
+ osx_image: xcode6.4
+ compiler: clang
+ - env: CONFIG=""
+ CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
+ os: osx
+ osx_image: xcode7.3
+ compiler: clang
+ - env: CONFIG=""
+ CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
+ os: osx
+ osx_image: xcode8.3
compiler: clang
# Plain Trusty System Build
- env: CONFIG="--disable-linux-user"
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [PATCH v2 20/21] MAINTAINERS: self-appoint me as reviewer in build/test automation
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (18 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 19/21] travis/osx: build using more Xcode versions Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning Philippe Mathieu-Daudé
20 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
Cc: Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 120788d8fb..e62cbc439a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1858,6 +1858,7 @@ Build and test automation
-------------------------
M: Alex Bennée <alex.bennee@linaro.org>
M: Fam Zheng <famz@redhat.com>
+R: Philippe Mathieu-Daudé <f4bug@amsat.org>
L: qemu-devel@nongnu.org
S: Maintained
F: .travis.yml
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
` (19 preceding siblings ...)
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 20/21] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé
@ 2017-06-22 3:32 ` Philippe Mathieu-Daudé
2017-06-22 7:28 ` Peter Maydell
2017-06-22 7:33 ` Gerd Hoffmann
20 siblings, 2 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-22 3:32 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell, Gerd Hoffmann
Cc: Philippe Mathieu-Daudé
ui/vnc.c:3945:20: warning: 'sasl_server_init' is deprecated: first deprecated in OS X 10.11 [-Wdeprecated-declarations]
if ((saslErr = sasl_server_init(NULL, "qemu")) != SASL_OK) {
^
/usr/include/sasl/sasl.h:1016:17: note: 'sasl_server_init' has been explicitly marked deprecated here
LIBSASL_API int sasl_server_init(const sasl_callback_t *callbacks,
^
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
ui/Makefile.objs | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 3369451285..08fb573a48 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
vnc-obj-y += vnc-ws.o
vnc-obj-y += vnc-jobs.o
+ifeq ($(CONFIG_VNC_SASL),y)
+# silent OSX SASL warnings (from https://stackoverflow.com/a/7406994):
+# because OpenSSL doesn’t offer API compatibility between versions, [...] Apple
+# can't provide security updates without breaking existing apps, so is migrating
+# from OpenSSL to Common Crypto.
+vnc.o-cflags := -Wno-deprecated-declarations
+vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
+endif
common-obj-y += keymaps.o console.o cursor.o qemu-pixman.o
common-obj-y += input.o input-keymap.o input-legacy.o
--
2.11.0
^ permalink raw reply related [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning
2017-06-22 3:32 ` [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning Philippe Mathieu-Daudé
@ 2017-06-22 7:28 ` Peter Maydell
2017-06-22 7:33 ` Gerd Hoffmann
1 sibling, 0 replies; 46+ messages in thread
From: Peter Maydell @ 2017-06-22 7:28 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: QEMU Developers, Alex Bennée, Fam Zheng, Gerd Hoffmann
On 22 June 2017 at 04:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> ui/vnc.c:3945:20: warning: 'sasl_server_init' is deprecated: first deprecated in OS X 10.11 [-Wdeprecated-declarations]
> if ((saslErr = sasl_server_init(NULL, "qemu")) != SASL_OK) {
> ^
> /usr/include/sasl/sasl.h:1016:17: note: 'sasl_server_init' has been explicitly marked deprecated here
> LIBSASL_API int sasl_server_init(const sasl_callback_t *callbacks,
> ^
Mmm, I've been vaguely wondering what to do about those warnings
for a while. The difficulty is that applying -Wno-deprecated-declarations
silences all deprecation warnings, not just the SASL ones, so we
lose a bit of warning coverage. Maybe that's just a hit we have
to take, though.
thanks
-- PMM
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning
2017-06-22 3:32 ` [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning Philippe Mathieu-Daudé
2017-06-22 7:28 ` Peter Maydell
@ 2017-06-22 7:33 ` Gerd Hoffmann
2017-06-22 7:35 ` Peter Maydell
1 sibling, 1 reply; 46+ messages in thread
From: Gerd Hoffmann @ 2017-06-22 7:33 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
> index 3369451285..08fb573a48 100644
> --- a/ui/Makefile.objs
> +++ b/ui/Makefile.objs
> @@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
> vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
> vnc-obj-y += vnc-ws.o
> vnc-obj-y += vnc-jobs.o
> +ifeq ($(CONFIG_VNC_SASL),y)
> +# silent OSX SASL warnings (from https://stackoverflow.com/a/7406994
> ):
> +# because OpenSSL doesn’t offer API compatibility between versions,
> [...] Apple
> +# can't provide security updates without breaking existing apps, so
> is migrating
> +# from OpenSSL to Common Crypto.
> +vnc.o-cflags := -Wno-deprecated-declarations
> +vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
> +endif
Hmm, does clang understand "#pragma GCC diagnostic" ?
If so, then this can be done in in the code not the makefiles, and also
selectively for osx only.
See include/ui/gtk.h for an example.
cheers,
Gerd
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning
2017-06-22 7:33 ` Gerd Hoffmann
@ 2017-06-22 7:35 ` Peter Maydell
2017-06-22 13:50 ` Gerd Hoffmann
0 siblings, 1 reply; 46+ messages in thread
From: Peter Maydell @ 2017-06-22 7:35 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: Philippe Mathieu-Daudé,
QEMU Developers, Alex Bennée, Fam Zheng
On 22 June 2017 at 08:33, Gerd Hoffmann <kraxel@redhat.com> wrote:
> diff --git a/ui/Makefile.objs b/ui/Makefile.objs
>> index 3369451285..08fb573a48 100644
>> --- a/ui/Makefile.objs
>> +++ b/ui/Makefile.objs
>> @@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
>> vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
>> vnc-obj-y += vnc-ws.o
>> vnc-obj-y += vnc-jobs.o
>> +ifeq ($(CONFIG_VNC_SASL),y)
>> +# silent OSX SASL warnings (from https://stackoverflow.com/a/7406994
>> ):
>> +# because OpenSSL doesn’t offer API compatibility between versions,
>> [...] Apple
>> +# can't provide security updates without breaking existing apps, so
>> is migrating
>> +# from OpenSSL to Common Crypto.
>> +vnc.o-cflags := -Wno-deprecated-declarations
>> +vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
>> +endif
>
> Hmm, does clang understand "#pragma GCC diagnostic" ?
>
> If so, then this can be done in in the code not the makefiles, and also
> selectively for osx only.
It does have some diagnostic pragma support. The awkward part is
that it has to be in force at the point where the deprecated
function is used, not where it's declared. So you can't just wrap
the #include of the ssl header in pragmas, you'd have to either
do it at every callsite or else over the whole .c file.
thanks
-- PMM
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
@ 2017-06-22 9:55 ` Alex Bennée
2017-06-23 18:43 ` Philippe Mathieu-Daudé
2017-06-22 13:00 ` Stefan Hajnoczi
2017-06-23 19:27 ` John Snow
2 siblings, 1 reply; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 9:55 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Fam Zheng, Peter Maydell, John Snow, Paolo Bonzini,
Stefan Hajnoczi, Markus Armbruster
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> This allow a one liner from fresh repository clone, i.e.:
>
> ./configure && make -j check-qtest-aarch64
TIL -j doesn't need a number and GNU Make has a -l option ;-)
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/Makefile.include | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 77da9b7f4b..f7c8f3eddb 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -823,7 +823,7 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
> # gtester tests, possibly with verbose output
>
> .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
> -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
> +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-softmmu $(check-qtest-y)
> $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
> $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
> QTEST_QEMU_IMG=qemu-img$(EXESUF) \
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
@ 2017-06-22 9:56 ` Alex Bennée
2017-06-23 13:32 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 9:56 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> From: Peter Maydell <peter.maydell@linaro.org>
>
> Add config to travis to do a Coverity Scan build and upload, using
> the new run-coverity-scan script.
>
> There is an official integration between Travis and Coverity Scan:
> https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/addons/coverity_scan.rb
> which slurps values out of the .travis.yml and downloads a build
> script from Coverity which does the bulk of the work:
> https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh
>
> However we choose to roll our own since this seems less
> confusing and also allows us to include debug features
> (notably the ability to do a "dry run" test which doesn't
> actually upload anything).
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> .travis.yml | 24 ++++++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 0220f7472e..29c9ef72a4 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -218,3 +218,27 @@ matrix:
> - TEST_CMD=""
> before_script:
> - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
> + # Build and upload to Coverity Scan.
> + # We do not impose any rate limiting here, but instead rely on the
> + # limiting done by the coverity servers, which for a project of QEMU's
> + # size means one build a day. The run-coverity-scan script will exit
> + # early if the limiter does not permit a new upload, so the effect will
> + # be that the first build (only) in each 24 hour period will be scanned.
> + # If we needed to apply a limit at the Travis end, the simplest approach
> + # would be to run the scan only if the branch was 'coverity-scan', and
> + # use a cron job to push master to the 'coverity-scan' branch periodically.
> + # We run on the trusty Travis hosts so that there's a wider set of
> + # dependencies satisfied to improve coverage.
> + - dist: trusty
> + env:
> + - COVERITY=1
> + - COVERITY_BUILD_CMD="make -j3"
> + - COVERITY_EMAIL=peter.maydell@linaro.org
> + # This 'secure' setting sets COVERITY_TOKEN=<secret token>
> + # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
> + - secure: "D3E6E5bacui53fYBQrx0wQr8ZTvo6VIBPKfg0QHj2uwa6OPFkUlcMr/EHWvdbZNAa4Q1bv1vhlED5OPRfPmQYzxQNT4SAxDZeuZnikgIymfqQXNOjKw4kRUDO9P42QanyFd+EAu2JDVClAeJPgBpa/ns4CNrGDK+Q3coGndCP8o="
> + before_script:
> + - if [ "$TRAVIS_PULL_REQUEST" = "true" ]; then echo "Skipping Coverity (pullreq)"; exit 0; fi
> + - if [ "$TRAVIS_BRANCH" != "master" ]; then echo "Skipping
> Coverity (wrong branch)"; exit 0; fi
I think this is waiting on a fix I mention when reviewing Peter's
original patches.
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images Philippe Mathieu-Daudé
@ 2017-06-22 9:58 ` Alex Bennée
2017-06-23 13:35 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 9:58 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
The way I read that blog post is group: edge is for testing things still
work but you probably don't want to merge that into your master branch.
Otherwise we'll automatically be running the bleeding edge.
> ---
> .travis.yml | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 29c9ef72a4..e60a5cd8e6 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -128,6 +128,7 @@ matrix:
> - sudo: required
> addons:
> dist: trusty
> + group: edge
> language: generic
> compiler: none
> env:
> @@ -147,6 +148,7 @@ matrix:
> - sudo: required
> addons:
> dist: trusty
> + group: edge
> language: generic
> compiler: none
> env:
> @@ -230,6 +232,7 @@ matrix:
> # We run on the trusty Travis hosts so that there's a wider set of
> # dependencies satisfied to improve coverage.
> - dist: trusty
> + group: edge
> env:
> - COVERITY=1
> - COVERITY_BUILD_CMD="make -j3"
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features Philippe Mathieu-Daudé
@ 2017-06-22 10:16 ` Alex Bennée
0 siblings, 0 replies; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:16 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> (this eases git workflow)
>
> see
> https://docs.travis-ci.com/user/caching#enabling-multiple-caching-features
I wonder if we could replace our submodule pre-seed hack with a native
Travis version? And now I see you do that in a later patch ;-)
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 61394b4764..be28e0642c 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -4,7 +4,8 @@ python:
> - "2.4"
> compiler:
> - gcc
> -cache: ccache
> +cache:
> + ccache: true
> addons:
> apt:
> packages:
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout Philippe Mathieu-Daudé
@ 2017-06-22 10:17 ` Alex Bennée
0 siblings, 0 replies; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:17 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index be28e0642c..62b9dfd2ae 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -6,6 +6,7 @@ compiler:
> - gcc
> cache:
> ccache: true
> + timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
> addons:
> apt:
> packages:
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules Philippe Mathieu-Daudé
@ 2017-06-22 10:17 ` Alex Bennée
2017-06-22 13:51 ` Alex Bennée
0 siblings, 1 reply; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:17 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> Travis caching uses the branch name to store packed cache, so each new branch
> will trigger a cache miss and will clone all submodules. Subsequent builds will
> benefit from the cache.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
\o/
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 7 ++-----
> 1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 62b9dfd2ae..777914f759 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -7,6 +7,8 @@ compiler:
> cache:
> ccache: true
> timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
> + directories:
> + - $HOME/qemu/.git/modules
> addons:
> apt:
> packages:
> @@ -74,7 +76,6 @@ git:
> before_install:
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
> - git submodule update --init --recursive
> before_script:
> - ./configure ${CONFIG}
> @@ -113,7 +114,6 @@ matrix:
> before_install:
> - sudo apt-get update -qq
> - sudo apt-get build-dep -qq qemu
> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
> - git submodule update --init --recursive
> # Plain Trusty Linux User Build
> - env: CONFIG="--disable-system"
> @@ -124,7 +124,6 @@ matrix:
> before_install:
> - sudo apt-get update -qq
> - sudo apt-get build-dep -qq qemu
> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
> - git submodule update --init --recursive
> # Trusty System build with latest stable clang
> - sudo: required
> @@ -142,7 +141,6 @@ matrix:
> - travis_retry sudo apt-get update -qq
> - travis_retry sudo apt-get install -qq -y clang-3.9
> - sudo apt-get build-dep -qq qemu
> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
> - git submodule update --init --recursive
> before_script:
> - ./configure ${CONFIG} || cat config.log
> @@ -162,7 +160,6 @@ matrix:
> - travis_retry sudo apt-get update -qq
> - travis_retry sudo apt-get install -qq -y clang-3.9
> - sudo apt-get build-dep -qq qemu
> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
> - git submodule update --init --recursive
> before_script:
> - ./configure ${CONFIG} || cat config.log
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores Philippe Mathieu-Daudé
@ 2017-06-22 10:21 ` Alex Bennée
0 siblings, 0 replies; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:21 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> Few improvement as of today, but if Travis release their limit on the opensource
> plan or upgrade their hardware, new builds will get some benefit.
>
> Restrict to as most 4 cores to respect Travis Open Source plan.
You need to mention you are also enabling a parallel make check here.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> .travis.yml | 15 ++++++++++++---
> 1 file changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 777914f759..a7512f3ab0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -62,7 +62,14 @@ notifications:
> on_failure: always
> env:
> global:
> - - TEST_CMD="make check"
> + # jobs
> + # detects cores on the hosting hardware, rather than the VM (https://docs.travis-ci.com/user/languages/c#hints)
> + - VM_CORES="4" HW_CORES="$(getconf _NPROCESSORS_ONLN)"
> + # be kind with Travis free plan, restrict to at most 4 cores
> + - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
> + - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> + # tests
> + - TEST_CMD="make -j${PARALLEL_JOBS} check"
> matrix:
> - CONFIG=""
> - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
> @@ -78,9 +85,11 @@ before_install:
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
> - git submodule update --init --recursive
> before_script:
> + - echo "Using ${PARALLEL_JOBS} parallel jobs"
> - ./configure ${CONFIG}
> script:
> - - make -j3 && ${TEST_CMD}
> + - make -j${PARALLEL_JOBS}
> + - ${TEST_CMD}
> matrix:
> include:
> # Test with CLang for compile portability
> @@ -234,7 +243,7 @@ matrix:
> group: edge
> env:
> - COVERITY=1
> - - COVERITY_BUILD_CMD="make -j3"
> + - COVERITY_BUILD_CMD="make -j${PARALLEL_JOBS}"
> - COVERITY_EMAIL=peter.maydell@linaro.org
> # This 'secure' setting sets COVERITY_TOKEN=<secret token>
> # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use Philippe Mathieu-Daudé
@ 2017-06-22 10:22 ` Alex Bennée
2017-06-23 13:41 ` Philippe Mathieu-Daudé
2017-06-22 10:57 ` Paolo Bonzini
1 sibling, 1 reply; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:22 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Can you expand the commit message to breifly explain what all these
extra settings change. What is CCACHE_SLOPINESS and ccache -s doing?
> ---
> .travis.yml | 14 ++++++++++++--
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index a7512f3ab0..96ddc16cfd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -68,6 +68,8 @@ env:
> # be kind with Travis free plan, restrict to at most 4 cores
> - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
> - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> + # ccache
> + - CCACHE_SLOPPINESS="file_macro,time_macros"
> # tests
> - TEST_CMD="make -j${PARALLEL_JOBS} check"
> matrix:
> @@ -82,14 +84,21 @@ git:
> submodules: false
> before_install:
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
> - - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
> + - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
> - git submodule update --init --recursive
> before_script:
> + # https://docs.travis-ci.com/user/caching#ccache-on-os-x
> + - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
> + - if [ "$CC" == "clang" ]; then export CCACHE_CPP2=1 ; fi
> + - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
> - echo "Using ${PARALLEL_JOBS} parallel jobs"
> + - export HOST_CC="${CC}"
> - ./configure ${CONFIG}
> script:
> - - make -j${PARALLEL_JOBS}
> + - make -j${PARALLEL_JOBS} && ccache -s
> - ${TEST_CMD}
> +before_cache:
> + - ccache -s
> matrix:
> include:
> # Test with CLang for compile portability
> @@ -112,6 +121,7 @@ matrix:
> TEST_CMD=""
> compiler: gcc
> - env: CONFIG=""
> + CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
> os: osx
> compiler: clang
> # Plain Trusty System Build
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails Philippe Mathieu-Daudé
@ 2017-06-22 10:23 ` Alex Bennée
2017-06-23 13:45 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:23 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> .travis.yml | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 96ddc16cfd..bac5bea744 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -93,7 +93,7 @@ before_script:
> - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
> - echo "Using ${PARALLEL_JOBS} parallel jobs"
> - export HOST_CC="${CC}"
> - - ./configure ${CONFIG}
> + - ./configure ${CONFIG} || cat config.log
Won't the explicit || mean the job will keep trying even if config is
broken? Should it be || cat config.log && false?
> script:
> - make -j${PARALLEL_JOBS} && ccache -s
> - ${TEST_CMD}
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails Philippe Mathieu-Daudé
@ 2017-06-22 10:24 ` Alex Bennée
0 siblings, 0 replies; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:24 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> example of failure: https://travis-ci.org/philmd/qemu/jobs/245612939
>
> $ git submodule update --init --recursive
> [...]
> Submodule 'pixman' (git://anongit.freedesktop.org/pixman) registered for path 'pixman'
> Cloning into 'pixman'...
> fatal: unable to connect to anongit.freedesktop.org:
> anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out
> anongit.freedesktop.org[1: 2610:10:20:722:a800:ff:fe24:61cf]: errno=Network is unreachable
> Clone of 'git://anongit.freedesktop.org/pixman' into submodule path 'pixman' failed
> The command "git submodule update --init --recursive" failed and exited with 1 during .
> Your build has been stopped.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index bac5bea744..34391722a2 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -85,7 +85,7 @@ git:
> before_install:
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
> - - git submodule update --init --recursive
> + - travis_retry git submodule update --init --recursive
> before_script:
> # https://docs.travis-ci.com/user/caching#ccache-on-os-x
> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job Philippe Mathieu-Daudé
@ 2017-06-22 10:24 ` Alex Bennée
0 siblings, 0 replies; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 10:24 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> gcov generates lot of output and often trigger job limit timeout
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 34391722a2..4bb288a192 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -105,7 +105,9 @@ matrix:
> - env: CONFIG=""
> compiler: clang
> # gprof/gcov are GCC features
> - - env: CONFIG="--enable-gprof --enable-gcov --disable-pie"
> + - env: CONFIG="--enable-gprof --disable-pie"
> + compiler: gcc
> + - env: CONFIG="--enable-gcov --disable-pie"
> compiler: gcc
> # We manually include builds which we disable "make check" for
> - env: CONFIG="--enable-debug --enable-tcg-interpreter"
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use Philippe Mathieu-Daudé
2017-06-22 10:22 ` Alex Bennée
@ 2017-06-22 10:57 ` Paolo Bonzini
1 sibling, 0 replies; 46+ messages in thread
From: Paolo Bonzini @ 2017-06-22 10:57 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell
On 22/06/2017 05:32, Philippe Mathieu-Daudé wrote:
> diff --git a/.travis.yml b/.travis.yml
> index a7512f3ab0..96ddc16cfd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -68,6 +68,8 @@ env:
> # be kind with Travis free plan, restrict to at most 4 cores
> - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
> - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> + # ccache
> + - CCACHE_SLOPPINESS="file_macro,time_macros"
Does this actually have any effect? We should not be using time macros.
include_file_ctime/mtime makes sense though.
Paolo
> # tests
> - TEST_CMD="make -j${PARALLEL_JOBS} check"
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
2017-06-22 9:55 ` Alex Bennée
@ 2017-06-22 13:00 ` Stefan Hajnoczi
2017-06-23 19:27 ` John Snow
2 siblings, 0 replies; 46+ messages in thread
From: Stefan Hajnoczi @ 2017-06-22 13:00 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell,
John Snow, Paolo Bonzini, Markus Armbruster
[-- Attachment #1: Type: text/plain, Size: 394 bytes --]
On Thu, Jun 22, 2017 at 12:32:11AM -0300, Philippe Mathieu-Daudé wrote:
> This allow a one liner from fresh repository clone, i.e.:
>
> ./configure && make -j check-qtest-aarch64
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/Makefile.include | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning
2017-06-22 7:35 ` Peter Maydell
@ 2017-06-22 13:50 ` Gerd Hoffmann
0 siblings, 0 replies; 46+ messages in thread
From: Gerd Hoffmann @ 2017-06-22 13:50 UTC (permalink / raw)
To: Peter Maydell
Cc: Philippe Mathieu-Daudé,
QEMU Developers, Alex Bennée, Fam Zheng
On Thu, 2017-06-22 at 08:35 +0100, Peter Maydell wrote:
> On 22 June 2017 at 08:33, Gerd Hoffmann <kraxel@redhat.com> wrote:
> > diff --git a/ui/Makefile.objs b/ui/Makefile.objs
> > > index 3369451285..08fb573a48 100644
> > > --- a/ui/Makefile.objs
> > > +++ b/ui/Makefile.objs
> > > @@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
> > > vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
> > > vnc-obj-y += vnc-ws.o
> > > vnc-obj-y += vnc-jobs.o
> > > +ifeq ($(CONFIG_VNC_SASL),y)
> > > +# silent OSX SASL warnings (from https://stackoverflow.com/a/740
> > > 6994
> > > ):
> > > +# because OpenSSL doesn’t offer API compatibility between
> > > versions,
> > > [...] Apple
> > > +# can't provide security updates without breaking existing apps,
> > > so
> > > is migrating
> > > +# from OpenSSL to Common Crypto.
> > > +vnc.o-cflags := -Wno-deprecated-declarations
> > > +vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
> > > +endif
> >
> > Hmm, does clang understand "#pragma GCC diagnostic" ?
> >
> > If so, then this can be done in in the code not the makefiles, and
> > also
> > selectively for osx only.
>
> It does have some diagnostic pragma support. The awkward part is
> that it has to be in force at the point where the deprecated
> function is used, not where it's declared. So you can't just wrap
> the #include of the ssl header in pragmas, you'd have to either
> do it at every callsite or else over the whole .c file.
Do it on the whole file on osx only is still better because we continue
to get the warnings on non-osx then, which should be enough to avoid
the code bitroting unnoticed.
cheers,
Gerd
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules
2017-06-22 10:17 ` Alex Bennée
@ 2017-06-22 13:51 ` Alex Bennée
0 siblings, 0 replies; 46+ messages in thread
From: Alex Bennée @ 2017-06-22 13:51 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel, Fam Zheng, Peter Maydell
Alex Bennée <alex.bennee@linaro.org> writes:
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> Travis caching uses the branch name to store packed cache, so each new branch
>> will trigger a cache miss and will clone all submodules. Subsequent builds will
>> benefit from the cache.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> \o/
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Hmmm looking deeper it doesn't look as though we are benefiting from the
caching here. The submodule checkout is around 60-70s with some jobs
coming down to 30-40s. The old system seems to be ~20s for the
wget/checkout step (assuming the wget is cached).
>
>
>> ---
>> .travis.yml | 7 ++-----
>> 1 file changed, 2 insertions(+), 5 deletions(-)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 62b9dfd2ae..777914f759 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -7,6 +7,8 @@ compiler:
>> cache:
>> ccache: true
>> timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
>> + directories:
>> + - $HOME/qemu/.git/modules
>> addons:
>> apt:
>> packages:
>> @@ -74,7 +76,6 @@ git:
>> before_install:
>> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
>> - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
>> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>> - git submodule update --init --recursive
>> before_script:
>> - ./configure ${CONFIG}
>> @@ -113,7 +114,6 @@ matrix:
>> before_install:
>> - sudo apt-get update -qq
>> - sudo apt-get build-dep -qq qemu
>> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>> - git submodule update --init --recursive
>> # Plain Trusty Linux User Build
>> - env: CONFIG="--disable-system"
>> @@ -124,7 +124,6 @@ matrix:
>> before_install:
>> - sudo apt-get update -qq
>> - sudo apt-get build-dep -qq qemu
>> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>> - git submodule update --init --recursive
>> # Trusty System build with latest stable clang
>> - sudo: required
>> @@ -142,7 +141,6 @@ matrix:
>> - travis_retry sudo apt-get update -qq
>> - travis_retry sudo apt-get install -qq -y clang-3.9
>> - sudo apt-get build-dep -qq qemu
>> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>> - git submodule update --init --recursive
>> before_script:
>> - ./configure ${CONFIG} || cat config.log
>> @@ -162,7 +160,6 @@ matrix:
>> - travis_retry sudo apt-get update -qq
>> - travis_retry sudo apt-get install -qq -y clang-3.9
>> - sudo apt-get build-dep -qq qemu
>> - - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>> - git submodule update --init --recursive
>> before_script:
>> - ./configure ${CONFIG} || cat config.log
--
Alex Bennée
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload
2017-06-22 9:56 ` Alex Bennée
@ 2017-06-23 13:32 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-23 13:32 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel, Fam Zheng, Peter Maydell
On 06/22/2017 06:56 AM, Alex Bennée wrote:[...]
> I think this is waiting on a fix I mention when reviewing Peter's
> original patches.
Ok! I'll wait or drop the Coverity part.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images
2017-06-22 9:58 ` Alex Bennée
@ 2017-06-23 13:35 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-23 13:35 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel, Fam Zheng, Peter Maydell
On 06/22/2017 06:58 AM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> The way I read that blog post is group: edge is for testing things still
> work but you probably don't want to merge that into your master branch.
> Otherwise we'll automatically be running the bleeding edge.
Ok I misunderstood this part, good you noticed.
Travis team posted a new entry after which is clearer:
https://blog.travis-ci.com/2017-06-21-trusty-updates-2017-Q2-launch
I'm now using "group: deprecated-2017Q2" instead of "edge".
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use
2017-06-22 10:22 ` Alex Bennée
@ 2017-06-23 13:41 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-23 13:41 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel, Fam Zheng, Peter Maydell
On 06/22/2017 07:22 AM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> Can you expand the commit message to breifly explain what all these
> extra settings change. What is CCACHE_SLOPINESS and ccache -s doing?
I'm pretty sure I wrote something here ... probably lost using git Fu :(
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails
2017-06-22 10:23 ` Alex Bennée
@ 2017-06-23 13:45 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-23 13:45 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel, Fam Zheng, Peter Maydell
On 06/22/2017 07:23 AM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
[...]
>> - - ./configure ${CONFIG}
>> + - ./configure ${CONFIG} || cat config.log
>
> Won't the explicit || mean the job will keep trying even if config is
> broken? Should it be || cat config.log && false?
Yes, you right. I actually copied it from the "Trusty build with latest
stable clang" job from commit ae1a772c :p
I'll see to fix other occurrences.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY
2017-06-22 9:55 ` Alex Bennée
@ 2017-06-23 18:43 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 46+ messages in thread
From: Philippe Mathieu-Daudé @ 2017-06-23 18:43 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, Fam Zheng, Peter Maydell, John Snow, Paolo Bonzini,
Stefan Hajnoczi, Markus Armbruster
On 06/22/2017 06:55 AM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> This allow a one liner from fresh repository clone, i.e.:
>>
>> ./configure && make -j check-qtest-aarch64
>
> TIL -j doesn't need a number and GNU Make has a -l option ;-)
Also since 2.8 (2016) git fetch has -j option, but Travis is using a
release version 1.8 (from the git-core PPA)
on my debian/unstable git version 2.13.1 I can do:
$ git submodule update --init --recursive --depth 1 -j ${PARALLEL_JOBS}
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
2017-06-22 9:55 ` Alex Bennée
2017-06-22 13:00 ` Stefan Hajnoczi
@ 2017-06-23 19:27 ` John Snow
2 siblings, 0 replies; 46+ messages in thread
From: John Snow @ 2017-06-23 19:27 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
qemu-devel, Alex Bennée, Fam Zheng, Peter Maydell,
Paolo Bonzini, Stefan Hajnoczi, Markus Armbruster
On 06/21/2017 11:32 PM, Philippe Mathieu-Daudé wrote:
> This allow a one liner from fresh repository clone, i.e.:
>
> ./configure && make -j check-qtest-aarch64
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> tests/Makefile.include | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 77da9b7f4b..f7c8f3eddb 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -823,7 +823,7 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
> # gtester tests, possibly with verbose output
>
> .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
> -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
> +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-softmmu $(check-qtest-y)
> $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
> $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
> QTEST_QEMU_IMG=qemu-img$(EXESUF) \
>
I'm on board, it's nice to have proper dependencies.
Reviewed-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 46+ messages in thread
end of thread, other threads:[~2017-06-23 19:27 UTC | newest]
Thread overview: 46+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-22 3:32 [Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY Philippe Mathieu-Daudé
2017-06-22 9:55 ` Alex Bennée
2017-06-23 18:43 ` Philippe Mathieu-Daudé
2017-06-22 13:00 ` Stefan Hajnoczi
2017-06-23 19:27 ` John Snow
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 02/21] travis: retry if llvm.org timeouts Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 03/21] travis: install more library dependencies Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 04/21] " Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 05/21] scripts/run-coverity-scan: Script to run Coverity Scan build Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload Philippe Mathieu-Daudé
2017-06-22 9:56 ` Alex Bennée
2017-06-23 13:32 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images Philippe Mathieu-Daudé
2017-06-22 9:58 ` Alex Bennée
2017-06-23 13:35 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 08/21] travis: use gcc-6 sanitizers Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features Philippe Mathieu-Daudé
2017-06-22 10:16 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout Philippe Mathieu-Daudé
2017-06-22 10:17 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules Philippe Mathieu-Daudé
2017-06-22 10:17 ` Alex Bennée
2017-06-22 13:51 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores Philippe Mathieu-Daudé
2017-06-22 10:21 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use Philippe Mathieu-Daudé
2017-06-22 10:22 ` Alex Bennée
2017-06-23 13:41 ` Philippe Mathieu-Daudé
2017-06-22 10:57 ` Paolo Bonzini
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails Philippe Mathieu-Daudé
2017-06-22 10:23 ` Alex Bennée
2017-06-23 13:45 ` Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails Philippe Mathieu-Daudé
2017-06-22 10:24 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job Philippe Mathieu-Daudé
2017-06-22 10:24 ` Alex Bennée
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 17/21] travis/osx: don't update Homebrew cache Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 18/21] travis/osx: silent texinfo warnings Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 19/21] travis/osx: build using more Xcode versions Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [PATCH v2 20/21] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé
2017-06-22 3:32 ` [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning Philippe Mathieu-Daudé
2017-06-22 7:28 ` Peter Maydell
2017-06-22 7:33 ` Gerd Hoffmann
2017-06-22 7:35 ` Peter Maydell
2017-06-22 13:50 ` Gerd Hoffmann
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.