[Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures

[Qemu-devel] [PATCH v2 00/21] travis: speedup to reduce failures

[Philippe Mathieu-Daudé]

These patches try to improve our Travis CI usage (quite a few failures the last
days).

This series include Peter's "Automate coverity scan uploads via Travis"
patches (here numbered 3 to 6). See
https://www.mail-archive.com/qemu-devel@nongnu.org/msg457443.html

Patch 7 is expected to enter /master as of today.
Travis warned few months ago:
  "On Wednesday, June 21st 2017, we are going to update all our Ubuntu Trusty
  14.04 images."

Patches 2, 10 and 15 use 'travis_retry' feature to avoid aborting a job on
network failure, retrying 3 times (default).

Patches 8, 19 are upgrades allowing to use recent VMs/toolchains.

Patch 12 speed up builds using multicore parallelism, caring about not overkill
Travis hardware.

Patches 9, 11, 13, 17 speed up builds improving cache usage.

Patches 18, 21 silent warnings.

Patch 14 gives hint when ./configure fails.

Patch 16 split slow gprof/gcov job.

Finally patch 1 add a missing rule to build system-qemu binary before running
qtests.

time comparison:
----------------

current /master https://travis-ci.org/qemu/qemu/builds/245061137:
 Ran for 2 hrs 4 min 17 sec
 Total time 7 hrs 34 min 22 sec

this seris with 3 more jobs https://travis-ci.org/philmd/qemu/builds/245593509:
 Ran for 1 hr 26 min 35 sec
 Total time 6 hrs 32 min 49 sec

Regards,

Phil.

Peter Maydell (3):
  travis: install more library dependencies
  scripts/run-coverity-scan: Script to run Coverity Scan build
  travis: Add config to do a Coverity Scan upload

Philippe Mathieu-Daudé (18):
  tests: add missing dependency to build QTEST_QEMU_BINARY
  travis: retry if llvm.org timeouts
  travis: install more library dependencies
  travis: update sudo-enabled Trusty images
  travis: use gcc-6 sanitizers
  travis: enable multiple caching features
  travis: increase S3 cache timeout
  travis: cache git submodules
  travis: build using all available cores
  travis: improve ccache use
  travis: dump config.log if ./configure script fails
  travis: retry when git submodules initialization fails
  travis: split the gprof/gcov job
  travis/osx: don't update Homebrew cache
  travis/osx: silent texinfo warnings
  travis/osx: build using more Xcode versions
  MAINTAINERS: self-appoint me as reviewer in build/test automation
  ui/vnc: silent unuseful OSX clang warning

 .travis.yml               | 135 +++++++++++++++++++++++++++++-------
 MAINTAINERS               |   1 +
 scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/Makefile.include    |   2 +-
 ui/Makefile.objs          |   8 +++
 5 files changed, 290 insertions(+), 26 deletions(-)
 create mode 100755 scripts/run-coverity-scan

-- 
2.11.0

[Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY

[Philippe Mathieu-Daudé]

This allow a one liner from fresh repository clone, i.e.:

  ./configure && make -j check-qtest-aarch64

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 77da9b7f4b..f7c8f3eddb 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -823,7 +823,7 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
 # gtester tests, possibly with verbose output
 
 .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
-$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
+$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-softmmu $(check-qtest-y)
 	$(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
 	$(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
 		QTEST_QEMU_IMG=qemu-img$(EXESUF) \
-- 
2.11.0

[Qemu-devel] [PATCH v2 02/21] travis: retry if llvm.org timeouts

[Philippe Mathieu-Daudé]

example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857

    $ sudo apt-get update -qq
    W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg  Connection failed
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    The command "sudo apt-get update -qq" failed and exited with 100 during .
    Your build has been stopped.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 27a2d9cfb3..f78300ee99 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -121,10 +121,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
@@ -140,10 +140,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
-- 
2.11.0

[Qemu-devel] [PATCH v2 03/21] travis: install more library dependencies

[Philippe Mathieu-Daudé]

From: Peter Maydell <peter.maydell@linaro.org>

Update the travis list of library packages to install so that
our build tests cover more of our code base.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index f78300ee99..53d8e79bf5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,28 +11,37 @@ addons:
       # Build dependencies
       - libaio-dev
       - libattr1-dev
+      - libbluetooth-dev
       - libbrlapi-dev
+      - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
       - libgtk-3-dev
       - libiscsi-dev
       - liblttng-ust-dev
+      - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
       - librados-dev
+      - librdmacm-dev
       - libsdl1.2-dev
       - libseccomp-dev
+      - libsnappy-dev
       - libspice-protocol-dev
       - libspice-server-dev
       - libssh2-1-dev
       - liburcu-dev
       - libusb-1.0-0-dev
+      - libvde-dev
       - libvte-2.90-dev
+      - libxen-dev
+      - nettle-dev
       - sparse
       - uuid-dev
+      - xfslibs-dev
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
 # to prevent IRC notifications from forks. This was created using:
@@ -162,28 +171,37 @@ matrix:
             # Build dependencies
             - libaio-dev
             - libattr1-dev
+            - libbluetooth-dev
             - libbrlapi-dev
+            - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
             - libgtk-3-dev
             - libiscsi-dev
             - liblttng-ust-dev
+            - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
             - librados-dev
+            - librdmacm-dev
             - libsdl1.2-dev
             - libseccomp-dev
+            - libsnappy-dev
             - libspice-protocol-dev
             - libspice-server-dev
             - libssh2-1-dev
             - liburcu-dev
             - libusb-1.0-0-dev
+            - libvde-dev
             - libvte-2.90-dev
+            - libxen-dev
+            - nettle-dev
             - sparse
             - uuid-dev
+            - xfslibs-dev
       language: generic
       compiler: none
       env:
-- 
2.11.0

[Qemu-devel] [PATCH v2 04/21] travis: install more library dependencies

[Philippe Mathieu-Daudé]

so more codebase is built

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 53d8e79bf5..0220f7472e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -9,10 +9,12 @@ addons:
   apt:
     packages:
       # Build dependencies
+      - glusterfs-common
       - libaio-dev
       - libattr1-dev
       - libbluetooth-dev
       - libbrlapi-dev
+      - libbz2-dev
       - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
@@ -22,6 +24,8 @@ addons:
       - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
+      - libncursesw5-dev
+      - libnfs-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
@@ -169,10 +173,12 @@ matrix:
             - gcc-5
             - g++-5
             # Build dependencies
+            - glusterfs-common
             - libaio-dev
             - libattr1-dev
             - libbluetooth-dev
             - libbrlapi-dev
+            - libbz2-dev
             - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
@@ -182,6 +188,8 @@ matrix:
             - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
+            - libncursesw5-dev
+            - libnfs-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
-- 
2.11.0

[Qemu-devel] [PATCH v2 05/21] scripts/run-coverity-scan: Script to run Coverity Scan build

[Philippe Mathieu-Daudé]

From: Peter Maydell <peter.maydell@linaro.org>

Add a new script to automate the process of running the Coverity
Scan build tools and uploading the resulting tarball to the
website. This is primarily intended to be driven from Travis,
but it can be run locally (if you are a maintainer of the
QEMU project on the Coverity Scan website and have the secret
upload token).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 scripts/run-coverity-scan | 170 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 170 insertions(+)
 create mode 100755 scripts/run-coverity-scan

diff --git a/scripts/run-coverity-scan b/scripts/run-coverity-scan
new file mode 100755
index 0000000000..e6d5fc58d8
--- /dev/null
+++ b/scripts/run-coverity-scan
@@ -0,0 +1,170 @@
+#!/bin/sh -e
+
+# Upload a created tarball to Coverity Scan, as per
+# https://scan.coverity.com/projects/qemu/builds/new
+
+# This work is licensed under the terms of the GNU GPL version 2,
+# or (at your option) any later version.
+# See the COPYING file in the top-level directory.
+#
+# Copyright (c) 2017 Linaro Limited
+# Written by Peter Maydell
+
+# Note that this script will automatically download and
+# run the (closed-source) coverity build tools, so don't
+# use it if you don't trust them!
+
+# This script assumes that you're running it from a QEMU source
+# tree, and that tree is a fresh clean one, because we do an in-tree
+# build. (This is necessary so that the filenames that the Coverity
+# Scan server sees are relative paths that match up with the component
+# regular expressions it uses; an out-of-tree build won't work for this.)
+# The host machine should have as many of QEMU's dependencies
+# installed as possible, for maximum coverity coverage.
+
+# You need to pass the following environment variables to the script:
+#  COVERITY_TOKEN -- this is the secret 8 digit hex string which lets
+#                    you upload to Coverity Scan. If you're a maintainer
+#                    in Coverity then the web UI will tell you this.
+#  COVERITY_EMAIL -- the email address to use for uploads
+
+# and optionally
+#  COVERITY_DRYRUN -- set to not actually do the upload
+#  COVERITY_BUILD_CMD -- make command (defaults to 'make -j8')
+#  COVERITY_TOOL_BASE -- set to directory to put coverity tools
+#                        (defaults to /tmp/coverity-tools)
+
+# The primary purpose of this script is to be run as part of
+# a Travis build, but it is possible to run it manually locally.
+
+if [ -z "$COVERITY_TOKEN" ]; then
+    echo "COVERITY_TOKEN environment variable not set"
+    exit 1
+fi
+
+if [ -z "$COVERITY_EMAIL" ]; then
+    echo "COVERITY_EMAIL environment variable not set"
+    exit 1
+fi
+
+if [ -z "$COVERITY_BUILD_CMD" ]; then
+    echo "COVERITY_BUILD_CMD: using default 'make -j8'"
+    COVERITY_BUILD_CMD="make -j8"
+fi
+
+if [ -z "$COVERITY_TOOL_BASE" ]; then
+    echo "COVERITY_TOOL_BASE: using default /tmp/coverity-tools"
+    COVERITY_TOOL_BASE=/tmp/coverity-tools
+fi
+
+PROJTOKEN="$COVERITY_TOKEN"
+PROJNAME=QEMU
+TARBALL=cov-int.tar.xz
+SRCDIR="$(pwd)"
+
+echo "Checking this is a QEMU source tree..."
+if ! [ -e VERSION ]; then
+    echo "Not in a QEMU source tree?"
+    exit 1
+fi
+
+echo "Checking upload permissions..."
+
+if ! up_perm="$(wget https://scan.coverity.com/api/upload_permitted --post-data "token=$PROJTOKEN&project=$PROJNAME" -q -O -)"; then
+    echo "Coverity Scan API access denied: bad token?"
+    exit 1
+fi
+
+# Really up_perm is a JSON response with either
+# {upload_permitted:true} or {next_upload_permitted_at:<date>}
+# We do some hacky string parsing instead of properly parsing it.
+case "$up_perm" in
+    *upload_permitted*true*)
+        echo "Coverity Scan: upload permitted"
+        ;;
+    *next_upload_permitted_at*)
+        if [ -z "$COVERITY_DRYRUN" ]; then
+            echo "Coverity Scan: upload quota reached; stopping here"
+            # Exit success as this isn't a build error.
+            exit 0
+        else
+            echo "Coverity Scan: upload quota reached, continuing dry run"
+        fi
+        ;;
+    *)
+        echo "Coverity Scan upload check: unexpected result $up_perm"
+        exit 1
+        ;;
+esac
+
+mkdir -p "$COVERITY_TOOL_BASE"
+cd "$COVERITY_TOOL_BASE"
+
+echo "Checking for new version of coverity build tools..."
+wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME&md5=1" -O coverity_tool.md5.new
+
+if ! cmp -s coverity_tool.md5 coverity_tool.md5.new; then
+    # out of date md5 or no md5: download new build tool
+    # blow away the old build tool
+    echo "Downloading coverity build tools..."
+    rm -rf coverity_tool coverity_tool.tgz
+    wget https://scan.coverity.com/download/linux64 --post-data "token=$PROJTOKEN&project=$PROJNAME" -O coverity_tool.tgz
+    if ! (cat coverity_tool.md5.new; echo "  coverity_tool.tgz") | md5sum -c --status; then
+        echo "Downloaded tarball didn't match md5sum!"
+        exit 1
+    fi
+    # extract the new one, keeping it corralled in a 'coverity_tool' directory
+    echo "Unpacking coverity build tools..."
+    mkdir -p coverity_tool
+    cd coverity_tool
+    tar xf ../coverity_tool.tgz
+    cd ..
+    mv coverity_tool.md5.new coverity_tool.md5
+fi
+
+rm -f coverity_tool.md5.new
+
+TOOLBIN="$(echo $(pwd)/coverity_tool/cov-analysis-*/bin)"
+
+if ! test -x "$TOOLBIN/cov-build"; then
+    echo "Couldn't find cov-build in the coverity build-tool directory??"
+    exit 1
+fi
+
+export PATH="$TOOLBIN:$PATH"
+
+cd "$SRCDIR"
+
+echo "Doing make distclean..."
+make distclean
+
+echo "Configuring..."
+./configure --audio-drv-list=oss,alsa,sdl,pa --disable-werror
+
+echo "Making libqemustub.a..."
+make libqemustub.a
+
+echo "Running cov-build..."
+rm -rf cov-int
+mkdir cov-int
+cov-build --dir cov-int $COVERITY_BUILD_CMD
+
+echo "Creating results tarball..."
+tar cvf - cov-int | xz > "$TARBALL"
+
+echo "Uploading results tarball..."
+
+VERSION="$(git describe --always HEAD)"
+DESCRIPTION="$(git rev-parse HEAD)"
+
+if ! [ -z "$COVERITY_DRYRUN" ]; then
+    echo "Dry run only, not uploading $TARBALL"
+    exit 0
+fi
+
+curl --form token="$PROJTOKEN" --form email="$COVERITY_EMAIL" \
+     --form file=@"$TARBALL" --form version="$VERSION" \
+     --form description="$DESCRIPTION" \
+     https://scan.coverity.com/builds?project="$PROJNAME"
+
+echo "Done."
-- 
2.11.0

[Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload

[Philippe Mathieu-Daudé]

From: Peter Maydell <peter.maydell@linaro.org>

Add config to travis to do a Coverity Scan build and upload, using
the new run-coverity-scan script.

There is an official integration between Travis and Coverity Scan:
 https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/addons/coverity_scan.rb
which slurps values out of the .travis.yml and downloads a build
script from Coverity which does the bulk of the work:
 https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh

However we choose to roll our own since this seems less
confusing and also allows us to include debug features
(notably the ability to do a "dry run" test which doesn't
actually upload anything).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 0220f7472e..29c9ef72a4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -218,3 +218,27 @@ matrix:
         - TEST_CMD=""
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
+    # Build and upload to Coverity Scan.
+    # We do not impose any rate limiting here, but instead rely on the
+    # limiting done by the coverity servers, which for a project of QEMU's
+    # size means one build a day. The run-coverity-scan script will exit
+    # early if the limiter does not permit a new upload, so the effect will
+    # be that the first build (only) in each 24 hour period will be scanned.
+    # If we needed to apply a limit at the Travis end, the simplest approach
+    # would be to run the scan only if the branch was 'coverity-scan', and
+    # use a cron job to push master to the 'coverity-scan' branch periodically.
+    # We run on the trusty Travis hosts so that there's a wider set of
+    # dependencies satisfied to improve coverage.
+    - dist: trusty
+      env:
+        - COVERITY=1
+        - COVERITY_BUILD_CMD="make -j3"
+        - COVERITY_EMAIL=peter.maydell@linaro.org
+        # This 'secure' setting sets COVERITY_TOKEN=<secret token>
+        # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
+        - secure: "D3E6E5bacui53fYBQrx0wQr8ZTvo6VIBPKfg0QHj2uwa6OPFkUlcMr/EHWvdbZNAa4Q1bv1vhlED5OPRfPmQYzxQNT4SAxDZeuZnikgIymfqQXNOjKw4kRUDO9P42QanyFd+EAu2JDVClAeJPgBpa/ns4CNrGDK+Q3coGndCP8o="
+      before_script:
+        - if [ "$TRAVIS_PULL_REQUEST" = "true" ]; then echo "Skipping Coverity (pullreq)"; exit 0; fi
+        - if [ "$TRAVIS_BRANCH" != "master" ]; then echo "Skipping Coverity (wrong branch)"; exit 0; fi
+      script:
+        - ./scripts/run-coverity-scan
-- 
2.11.0

[Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images

[Philippe Mathieu-Daudé]

see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 29c9ef72a4..e60a5cd8e6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -128,6 +128,7 @@ matrix:
     - sudo: required
       addons:
       dist: trusty
+      group: edge
       language: generic
       compiler: none
       env:
@@ -147,6 +148,7 @@ matrix:
     - sudo: required
       addons:
       dist: trusty
+      group: edge
       language: generic
       compiler: none
       env:
@@ -230,6 +232,7 @@ matrix:
     # We run on the trusty Travis hosts so that there's a wider set of
     # dependencies satisfied to improve coverage.
     - dist: trusty
+      group: edge
       env:
         - COVERITY=1
         - COVERITY_BUILD_CMD="make -j3"
-- 
2.11.0

[Qemu-devel] [PATCH v2 08/21] travis: use gcc-6 sanitizers

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index e60a5cd8e6..61394b4764 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -172,8 +172,8 @@ matrix:
             - ubuntu-toolchain-r-test
           packages:
             # Extra toolchains
-            - gcc-5
-            - g++-5
+            - gcc-6
+            - g++-6
             # Build dependencies
             - glusterfs-common
             - libaio-dev
@@ -215,8 +215,8 @@ matrix:
       language: generic
       compiler: none
       env:
-        - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
-        - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
+        - COMPILER_NAME=gcc CXX=g++-6 CC=gcc-6
+        - CONFIG="--disable-pie --disable-linux-user"
         - TEST_CMD=""
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
-- 
2.11.0

[Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features

[Philippe Mathieu-Daudé]

(this eases git workflow)

see https://docs.travis-ci.com/user/caching#enabling-multiple-caching-features

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 61394b4764..be28e0642c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -4,7 +4,8 @@ python:
   - "2.4"
 compiler:
   - gcc
-cache: ccache
+cache:
+  ccache: true
 addons:
   apt:
     packages:
-- 
2.11.0

[Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index be28e0642c..62b9dfd2ae 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,6 +6,7 @@ compiler:
   - gcc
 cache:
   ccache: true
+  timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
 addons:
   apt:
     packages:
-- 
2.11.0

[Qemu-devel] [PATCH v2 11/21] travis: cache git submodules

[Philippe Mathieu-Daudé]

Travis caching uses the branch name to store packed cache, so each new branch
will trigger a cache miss and will clone all submodules. Subsequent builds will
benefit from the cache.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 62b9dfd2ae..777914f759 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -7,6 +7,8 @@ compiler:
 cache:
   ccache: true
   timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
+  directories:
+  - $HOME/qemu/.git/modules
 addons:
   apt:
     packages:
@@ -74,7 +76,6 @@ git:
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
-  - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
   - git submodule update --init --recursive
 before_script:
   - ./configure ${CONFIG}
@@ -113,7 +114,6 @@ matrix:
       before_install:
         - sudo apt-get update -qq
         - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
     # Plain Trusty Linux User Build
     - env: CONFIG="--disable-system"
@@ -124,7 +124,6 @@ matrix:
       before_install:
         - sudo apt-get update -qq
         - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
     # Trusty System build with latest stable clang
     - sudo: required
@@ -142,7 +141,6 @@ matrix:
         - travis_retry sudo apt-get update -qq
         - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
       before_script:
         - ./configure ${CONFIG} || cat config.log
@@ -162,7 +160,6 @@ matrix:
         - travis_retry sudo apt-get update -qq
         - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
       before_script:
         - ./configure ${CONFIG} || cat config.log
-- 
2.11.0

[Qemu-devel] [PATCH v2 12/21] travis: build using all available cores

[Philippe Mathieu-Daudé]

Few improvement as of today, but if Travis release their limit on the opensource
plan or upgrade their hardware, new builds will get some benefit.

Restrict to as most 4 cores to respect Travis Open Source plan.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 777914f759..a7512f3ab0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -62,7 +62,14 @@ notifications:
     on_failure: always
 env:
   global:
-    - TEST_CMD="make check"
+    # jobs
+      # detects cores on the hosting hardware, rather than the VM (https://docs.travis-ci.com/user/languages/c#hints)
+    - VM_CORES="4" HW_CORES="$(getconf _NPROCESSORS_ONLN)"
+      # be kind with Travis free plan, restrict to at most 4 cores
+    - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
+    - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
+    # tests
+    - TEST_CMD="make -j${PARALLEL_JOBS} check"
   matrix:
     - CONFIG=""
     - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
@@ -78,9 +85,11 @@ before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
   - git submodule update --init --recursive
 before_script:
+  - echo "Using ${PARALLEL_JOBS} parallel jobs"
   - ./configure ${CONFIG}
 script:
-  - make -j3 && ${TEST_CMD}
+  - make -j${PARALLEL_JOBS}
+  - ${TEST_CMD}
 matrix:
   include:
     # Test with CLang for compile portability
@@ -234,7 +243,7 @@ matrix:
       group: edge
       env:
         - COVERITY=1
-        - COVERITY_BUILD_CMD="make -j3"
+        - COVERITY_BUILD_CMD="make -j${PARALLEL_JOBS}"
         - COVERITY_EMAIL=peter.maydell@linaro.org
         # This 'secure' setting sets COVERITY_TOKEN=<secret token>
         # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
-- 
2.11.0

[Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index a7512f3ab0..96ddc16cfd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -68,6 +68,8 @@ env:
       # be kind with Travis free plan, restrict to at most 4 cores
     - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
     - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
+    # ccache
+    - CCACHE_SLOPPINESS="file_macro,time_macros"
     # tests
     - TEST_CMD="make -j${PARALLEL_JOBS} check"
   matrix:
@@ -82,14 +84,21 @@ git:
   submodules: false
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
+  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
   - git submodule update --init --recursive
 before_script:
+  # https://docs.travis-ci.com/user/caching#ccache-on-os-x
+  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
+  - if [ "$CC" == "clang" ]; then export CCACHE_CPP2=1 ; fi
+  - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
   - echo "Using ${PARALLEL_JOBS} parallel jobs"
+  - export HOST_CC="${CC}"
   - ./configure ${CONFIG}
 script:
-  - make -j${PARALLEL_JOBS}
+  - make -j${PARALLEL_JOBS} && ccache -s
   - ${TEST_CMD}
+before_cache:
+  - ccache -s
 matrix:
   include:
     # Test with CLang for compile portability
@@ -112,6 +121,7 @@ matrix:
            TEST_CMD=""
       compiler: gcc
     - env: CONFIG=""
+           CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
       os: osx
       compiler: clang
     # Plain Trusty System Build
-- 
2.11.0

[Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 96ddc16cfd..bac5bea744 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -93,7 +93,7 @@ before_script:
   - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
   - echo "Using ${PARALLEL_JOBS} parallel jobs"
   - export HOST_CC="${CC}"
-  - ./configure ${CONFIG}
+  - ./configure ${CONFIG} || cat config.log
 script:
   - make -j${PARALLEL_JOBS} && ccache -s
   - ${TEST_CMD}
-- 
2.11.0

[Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails

[Philippe Mathieu-Daudé]

example of failure: https://travis-ci.org/philmd/qemu/jobs/245612939

  $ git submodule update --init --recursive
  [...]
  Submodule 'pixman' (git://anongit.freedesktop.org/pixman) registered for path 'pixman'
  Cloning into 'pixman'...
  fatal: unable to connect to anongit.freedesktop.org:
  anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out
  anongit.freedesktop.org[1: 2610:10:20:722:a800:ff:fe24:61cf]: errno=Network is unreachable
  Clone of 'git://anongit.freedesktop.org/pixman' into submodule path 'pixman' failed
  The command "git submodule update --init --recursive" failed and exited with 1 during .
  Your build has been stopped.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index bac5bea744..34391722a2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -85,7 +85,7 @@ git:
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
-  - git submodule update --init --recursive
+  - travis_retry git submodule update --init --recursive
 before_script:
   # https://docs.travis-ci.com/user/caching#ccache-on-os-x
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
-- 
2.11.0

[Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job

[Philippe Mathieu-Daudé]

gcov generates lot of output and often trigger job limit timeout

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 34391722a2..4bb288a192 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -105,7 +105,9 @@ matrix:
     - env: CONFIG=""
       compiler: clang
     # gprof/gcov are GCC features
-    - env: CONFIG="--enable-gprof --enable-gcov --disable-pie"
+    - env: CONFIG="--enable-gprof --disable-pie"
+      compiler: gcc
+    - env: CONFIG="--enable-gcov --disable-pie"
       compiler: gcc
     # We manually include builds which we disable "make check" for
     - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-- 
2.11.0

[Qemu-devel] [PATCH v2 17/21] travis/osx: don't update Homebrew cache

[Philippe Mathieu-Daudé]

installed VMs come already updated, and brew is smart enough to upgrade some
dependencies are missing.

see https://docs.travis-ci.com/user/osx-ci-environment#homebrew

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 4bb288a192..3de3798121 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -83,7 +83,6 @@ git:
   # we want to do this ourselves
   submodules: false
 before_install:
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
   - travis_retry git submodule update --init --recursive
 before_script:
-- 
2.11.0

[Qemu-devel] [PATCH v2 18/21] travis/osx: silent texinfo warnings

[Philippe Mathieu-Daudé]

  $ make info
    GEN     qemu-doc.html
  qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
    GEN     qemu-doc.txt
  qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 3de3798121..d007fdc9fb 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -84,6 +84,8 @@ git:
   submodules: false
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
+  # silent texinfo warnings (https://github.com/xiaohanyu/oh-my-emacs/blob/c664894e2f1c1cb0f95a9f2da88d41b00f190856/core/ome-basic.org#homebrew)
+  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install texinfo && brew link texinfo --force ; fi
   - travis_retry git submodule update --init --recursive
 before_script:
   # https://docs.travis-ci.com/user/caching#ccache-on-os-x
-- 
2.11.0

[Qemu-devel] [PATCH v2 19/21] travis/osx: build using more Xcode versions

[Philippe Mathieu-Daudé]

currently default builder is based on Xcode 7.3.1:
    OS X 10.11 (darwin14.5.0)
    Apple LLVM version 7.3.0

add the oldest available Xcode (6.4):
    OS X 10.10 (darwin14.5.0)
    Apple LLVM 6.1.0 (based on LLVM 3.6.0svn)

and the newer available Xcode (8.3.3):
    OS X 10.12 (darwin16.6.0)
    Apple LLVM version 8.1.0

see https://docs.travis-ci.com/user/osx-ci-environment#os-x-version

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index d007fdc9fb..880a057c78 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -126,6 +126,17 @@ matrix:
     - env: CONFIG=""
            CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
       os: osx
+      osx_image: xcode6.4
+      compiler: clang
+    - env: CONFIG=""
+           CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
+      os: osx
+      osx_image: xcode7.3
+      compiler: clang
+    - env: CONFIG=""
+           CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
+      os: osx
+      osx_image: xcode8.3
       compiler: clang
     # Plain Trusty System Build
     - env: CONFIG="--disable-linux-user"
-- 
2.11.0

[Qemu-devel] [PATCH v2 20/21] MAINTAINERS: self-appoint me as reviewer in build/test automation

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 120788d8fb..e62cbc439a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1858,6 +1858,7 @@ Build and test automation
 -------------------------
 M: Alex Bennée <alex.bennee@linaro.org>
 M: Fam Zheng <famz@redhat.com>
+R: Philippe Mathieu-Daudé <f4bug@amsat.org>
 L: qemu-devel@nongnu.org
 S: Maintained
 F: .travis.yml
-- 
2.11.0

[Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning

[Philippe Mathieu-Daudé]

    ui/vnc.c:3945:20: warning: 'sasl_server_init' is deprecated: first deprecated in OS X 10.11 [-Wdeprecated-declarations]
        if ((saslErr = sasl_server_init(NULL, "qemu")) != SASL_OK) {
                       ^
    /usr/include/sasl/sasl.h:1016:17: note: 'sasl_server_init' has been explicitly marked deprecated here
    LIBSASL_API int sasl_server_init(const sasl_callback_t *callbacks,
                    ^

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 ui/Makefile.objs | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 3369451285..08fb573a48 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
 vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
 vnc-obj-y += vnc-ws.o
 vnc-obj-y += vnc-jobs.o
+ifeq ($(CONFIG_VNC_SASL),y)
+# silent OSX SASL warnings (from https://stackoverflow.com/a/7406994):
+# because OpenSSL doesn’t offer API compatibility between versions, [...] Apple
+# can't provide security updates without breaking existing apps, so is migrating
+# from OpenSSL to Common Crypto.
+vnc.o-cflags := -Wno-deprecated-declarations
+vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
+endif
 
 common-obj-y += keymaps.o console.o cursor.o qemu-pixman.o
 common-obj-y += input.o input-keymap.o input-legacy.o
-- 
2.11.0

Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning

[Peter Maydell]

On 22 June 2017 at 04:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>     ui/vnc.c:3945:20: warning: 'sasl_server_init' is deprecated: first deprecated in OS X 10.11 [-Wdeprecated-declarations]
>         if ((saslErr = sasl_server_init(NULL, "qemu")) != SASL_OK) {
>                        ^
>     /usr/include/sasl/sasl.h:1016:17: note: 'sasl_server_init' has been explicitly marked deprecated here
>     LIBSASL_API int sasl_server_init(const sasl_callback_t *callbacks,
>                     ^

Mmm, I've been vaguely wondering what to do about those warnings
for a while. The difficulty is that applying -Wno-deprecated-declarations
silences all deprecation warnings, not just the SASL ones, so we
lose a bit of warning coverage. Maybe that's just a hit we have
to take, though.

thanks
-- PMM

Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning

[Gerd Hoffmann]

diff --git a/ui/Makefile.objs b/ui/Makefile.objs
> index 3369451285..08fb573a48 100644
> --- a/ui/Makefile.objs
> +++ b/ui/Makefile.objs
> @@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
>  vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
>  vnc-obj-y += vnc-ws.o
>  vnc-obj-y += vnc-jobs.o
> +ifeq ($(CONFIG_VNC_SASL),y)
> +# silent OSX SASL warnings (from https://stackoverflow.com/a/7406994
> ):
> +# because OpenSSL doesn’t offer API compatibility between versions,
> [...] Apple
> +# can't provide security updates without breaking existing apps, so
> is migrating
> +# from OpenSSL to Common Crypto.
> +vnc.o-cflags := -Wno-deprecated-declarations
> +vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
> +endif

Hmm, does clang understand "#pragma GCC diagnostic" ?

If so, then this can be done in in the code not the makefiles, and also
selectively for osx only.

See include/ui/gtk.h for an example.

cheers,
  Gerd

Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning

[Peter Maydell]

On 22 June 2017 at 08:33, Gerd Hoffmann <kraxel@redhat.com> wrote:
> diff --git a/ui/Makefile.objs b/ui/Makefile.objs
>> index 3369451285..08fb573a48 100644
>> --- a/ui/Makefile.objs
>> +++ b/ui/Makefile.objs
>> @@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
>>  vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
>>  vnc-obj-y += vnc-ws.o
>>  vnc-obj-y += vnc-jobs.o
>> +ifeq ($(CONFIG_VNC_SASL),y)
>> +# silent OSX SASL warnings (from https://stackoverflow.com/a/7406994
>> ):
>> +# because OpenSSL doesn’t offer API compatibility between versions,
>> [...] Apple
>> +# can't provide security updates without breaking existing apps, so
>> is migrating
>> +# from OpenSSL to Common Crypto.
>> +vnc.o-cflags := -Wno-deprecated-declarations
>> +vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
>> +endif
>
> Hmm, does clang understand "#pragma GCC diagnostic" ?
>
> If so, then this can be done in in the code not the makefiles, and also
> selectively for osx only.

It does have some diagnostic pragma support. The awkward part is
that it has to  be in force at the point where the deprecated
function is used, not where it's declared. So you can't just wrap
the #include of the ssl header in pragmas, you'd have to either
do it at every callsite or else over the whole .c file.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> This allow a one liner from fresh repository clone, i.e.:
>
>   ./configure && make -j check-qtest-aarch64

TIL -j doesn't need a number and GNU Make has a -l option ;-)

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 77da9b7f4b..f7c8f3eddb 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -823,7 +823,7 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
>  # gtester tests, possibly with verbose output
>
>  .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
> -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
> +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-softmmu $(check-qtest-y)
>  	$(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
>  	$(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
>  		QTEST_QEMU_IMG=qemu-img$(EXESUF) \


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> From: Peter Maydell <peter.maydell@linaro.org>
>
> Add config to travis to do a Coverity Scan build and upload, using
> the new run-coverity-scan script.
>
> There is an official integration between Travis and Coverity Scan:
>  https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/addons/coverity_scan.rb
> which slurps values out of the .travis.yml and downloads a build
> script from Coverity which does the bulk of the work:
>  https://scan.coverity.com/scripts/travisci_build_coverity_scan.sh
>
> However we choose to roll our own since this seems less
> confusing and also allows us to include debug features
> (notably the ability to do a "dry run" test which doesn't
> actually upload anything).
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 0220f7472e..29c9ef72a4 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -218,3 +218,27 @@ matrix:
>          - TEST_CMD=""
>        before_script:
>          - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
> +    # Build and upload to Coverity Scan.
> +    # We do not impose any rate limiting here, but instead rely on the
> +    # limiting done by the coverity servers, which for a project of QEMU's
> +    # size means one build a day. The run-coverity-scan script will exit
> +    # early if the limiter does not permit a new upload, so the effect will
> +    # be that the first build (only) in each 24 hour period will be scanned.
> +    # If we needed to apply a limit at the Travis end, the simplest approach
> +    # would be to run the scan only if the branch was 'coverity-scan', and
> +    # use a cron job to push master to the 'coverity-scan' branch periodically.
> +    # We run on the trusty Travis hosts so that there's a wider set of
> +    # dependencies satisfied to improve coverage.
> +    - dist: trusty
> +      env:
> +        - COVERITY=1
> +        - COVERITY_BUILD_CMD="make -j3"
> +        - COVERITY_EMAIL=peter.maydell@linaro.org
> +        # This 'secure' setting sets COVERITY_TOKEN=<secret token>
> +        # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
> +        - secure: "D3E6E5bacui53fYBQrx0wQr8ZTvo6VIBPKfg0QHj2uwa6OPFkUlcMr/EHWvdbZNAa4Q1bv1vhlED5OPRfPmQYzxQNT4SAxDZeuZnikgIymfqQXNOjKw4kRUDO9P42QanyFd+EAu2JDVClAeJPgBpa/ns4CNrGDK+Q3coGndCP8o="
> +      before_script:
> +        - if [ "$TRAVIS_PULL_REQUEST" = "true" ]; then echo "Skipping Coverity (pullreq)"; exit 0; fi
> +        - if [ "$TRAVIS_BRANCH" != "master" ]; then echo "Skipping
> Coverity (wrong branch)"; exit 0; fi

I think this is waiting on a fix I mention when reviewing Peter's
original patches.


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

The way I read that blog post is group: edge is for testing things still
work but you probably don't want to merge that into your master branch.
Otherwise we'll automatically be running the bleeding edge.

> ---
>  .travis.yml | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 29c9ef72a4..e60a5cd8e6 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -128,6 +128,7 @@ matrix:
>      - sudo: required
>        addons:
>        dist: trusty
> +      group: edge
>        language: generic
>        compiler: none
>        env:
> @@ -147,6 +148,7 @@ matrix:
>      - sudo: required
>        addons:
>        dist: trusty
> +      group: edge
>        language: generic
>        compiler: none
>        env:
> @@ -230,6 +232,7 @@ matrix:
>      # We run on the trusty Travis hosts so that there's a wider set of
>      # dependencies satisfied to improve coverage.
>      - dist: trusty
> +      group: edge
>        env:
>          - COVERITY=1
>          - COVERITY_BUILD_CMD="make -j3"


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 09/21] travis: enable multiple caching features

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> (this eases git workflow)
>
> see
> https://docs.travis-ci.com/user/caching#enabling-multiple-caching-features

I wonder if we could replace our submodule pre-seed hack with a native
Travis version? And now I see you do that in a later patch ;-)

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 61394b4764..be28e0642c 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -4,7 +4,8 @@ python:
>    - "2.4"
>  compiler:
>    - gcc
> -cache: ccache
> +cache:
> +  ccache: true
>  addons:
>    apt:
>      packages:


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 10/21] travis: increase S3 cache timeout

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index be28e0642c..62b9dfd2ae 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -6,6 +6,7 @@ compiler:
>    - gcc
>  cache:
>    ccache: true
> +  timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
>  addons:
>    apt:
>      packages:


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Travis caching uses the branch name to store packed cache, so each new branch
> will trigger a cache miss and will clone all submodules. Subsequent builds will
> benefit from the cache.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

\o/

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>


> ---
>  .travis.yml | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 62b9dfd2ae..777914f759 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -7,6 +7,8 @@ compiler:
>  cache:
>    ccache: true
>    timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
> +  directories:
> +  - $HOME/qemu/.git/modules
>  addons:
>    apt:
>      packages:
> @@ -74,7 +76,6 @@ git:
>  before_install:
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
> -  - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>    - git submodule update --init --recursive
>  before_script:
>    - ./configure ${CONFIG}
> @@ -113,7 +114,6 @@ matrix:
>        before_install:
>          - sudo apt-get update -qq
>          - sudo apt-get build-dep -qq qemu
> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive
>      # Plain Trusty Linux User Build
>      - env: CONFIG="--disable-system"
> @@ -124,7 +124,6 @@ matrix:
>        before_install:
>          - sudo apt-get update -qq
>          - sudo apt-get build-dep -qq qemu
> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive
>      # Trusty System build with latest stable clang
>      - sudo: required
> @@ -142,7 +141,6 @@ matrix:
>          - travis_retry sudo apt-get update -qq
>          - travis_retry sudo apt-get install -qq -y clang-3.9
>          - sudo apt-get build-dep -qq qemu
> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive
>        before_script:
>          - ./configure ${CONFIG} || cat config.log
> @@ -162,7 +160,6 @@ matrix:
>          - travis_retry sudo apt-get update -qq
>          - travis_retry sudo apt-get install -qq -y clang-3.9
>          - sudo apt-get build-dep -qq qemu
> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - git submodule update --init --recursive
>        before_script:
>          - ./configure ${CONFIG} || cat config.log


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Few improvement as of today, but if Travis release their limit on the opensource
> plan or upgrade their hardware, new builds will get some benefit.
>
> Restrict to as most 4 cores to respect Travis Open Source plan.

You need to mention you are also enabling a parallel make check here.

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 15 ++++++++++++---
>  1 file changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 777914f759..a7512f3ab0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -62,7 +62,14 @@ notifications:
>      on_failure: always
>  env:
>    global:
> -    - TEST_CMD="make check"
> +    # jobs
> +      # detects cores on the hosting hardware, rather than the VM (https://docs.travis-ci.com/user/languages/c#hints)
> +    - VM_CORES="4" HW_CORES="$(getconf _NPROCESSORS_ONLN)"
> +      # be kind with Travis free plan, restrict to at most 4 cores
> +    - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
> +    - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> +    # tests
> +    - TEST_CMD="make -j${PARALLEL_JOBS} check"
>    matrix:
>      - CONFIG=""
>      - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
> @@ -78,9 +85,11 @@ before_install:
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
>    - git submodule update --init --recursive
>  before_script:
> +  - echo "Using ${PARALLEL_JOBS} parallel jobs"
>    - ./configure ${CONFIG}
>  script:
> -  - make -j3 && ${TEST_CMD}
> +  - make -j${PARALLEL_JOBS}
> +  - ${TEST_CMD}
>  matrix:
>    include:
>      # Test with CLang for compile portability
> @@ -234,7 +243,7 @@ matrix:
>        group: edge
>        env:
>          - COVERITY=1
> -        - COVERITY_BUILD_CMD="make -j3"
> +        - COVERITY_BUILD_CMD="make -j${PARALLEL_JOBS}"
>          - COVERITY_EMAIL=peter.maydell@linaro.org
>          # This 'secure' setting sets COVERITY_TOKEN=<secret token>
>          # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Can you expand the commit message to breifly explain what all these
extra settings change. What is CCACHE_SLOPINESS and ccache -s doing?

> ---
>  .travis.yml | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index a7512f3ab0..96ddc16cfd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -68,6 +68,8 @@ env:
>        # be kind with Travis free plan, restrict to at most 4 cores
>      - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
>      - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> +    # ccache
> +    - CCACHE_SLOPPINESS="file_macro,time_macros"
>      # tests
>      - TEST_CMD="make -j${PARALLEL_JOBS} check"
>    matrix:
> @@ -82,14 +84,21 @@ git:
>    submodules: false
>  before_install:
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
> -  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
> +  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
>    - git submodule update --init --recursive
>  before_script:
> +  # https://docs.travis-ci.com/user/caching#ccache-on-os-x
> +  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
> +  - if [ "$CC" == "clang" ]; then export CCACHE_CPP2=1 ; fi
> +  - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
>    - echo "Using ${PARALLEL_JOBS} parallel jobs"
> +  - export HOST_CC="${CC}"
>    - ./configure ${CONFIG}
>  script:
> -  - make -j${PARALLEL_JOBS}
> +  - make -j${PARALLEL_JOBS} && ccache -s
>    - ${TEST_CMD}
> +before_cache:
> +  - ccache -s
>  matrix:
>    include:
>      # Test with CLang for compile portability
> @@ -112,6 +121,7 @@ matrix:
>             TEST_CMD=""
>        compiler: gcc
>      - env: CONFIG=""
> +           CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
>        os: osx
>        compiler: clang
>      # Plain Trusty System Build


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 96ddc16cfd..bac5bea744 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -93,7 +93,7 @@ before_script:
>    - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
>    - echo "Using ${PARALLEL_JOBS} parallel jobs"
>    - export HOST_CC="${CC}"
> -  - ./configure ${CONFIG}
> +  - ./configure ${CONFIG} || cat config.log

Won't the explicit || mean the job will keep trying even if config is
broken? Should it be || cat config.log && false?

>  script:
>    - make -j${PARALLEL_JOBS} && ccache -s
>    - ${TEST_CMD}


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 15/21] travis: retry when git submodules initialization fails

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> example of failure: https://travis-ci.org/philmd/qemu/jobs/245612939
>
>   $ git submodule update --init --recursive
>   [...]
>   Submodule 'pixman' (git://anongit.freedesktop.org/pixman) registered for path 'pixman'
>   Cloning into 'pixman'...
>   fatal: unable to connect to anongit.freedesktop.org:
>   anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out
>   anongit.freedesktop.org[1: 2610:10:20:722:a800:ff:fe24:61cf]: errno=Network is unreachable
>   Clone of 'git://anongit.freedesktop.org/pixman' into submodule path 'pixman' failed
>   The command "git submodule update --init --recursive" failed and exited with 1 during .
>   Your build has been stopped.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index bac5bea744..34391722a2 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -85,7 +85,7 @@ git:
>  before_install:
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
> -  - git submodule update --init --recursive
> +  - travis_retry git submodule update --init --recursive
>  before_script:
>    # https://docs.travis-ci.com/user/caching#ccache-on-os-x
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 16/21] travis: split the gprof/gcov job

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> gcov generates lot of output and often trigger job limit timeout
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 34391722a2..4bb288a192 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -105,7 +105,9 @@ matrix:
>      - env: CONFIG=""
>        compiler: clang
>      # gprof/gcov are GCC features
> -    - env: CONFIG="--enable-gprof --enable-gcov --disable-pie"
> +    - env: CONFIG="--enable-gprof --disable-pie"
> +      compiler: gcc
> +    - env: CONFIG="--enable-gcov --disable-pie"
>        compiler: gcc
>      # We manually include builds which we disable "make check" for
>      - env: CONFIG="--enable-debug --enable-tcg-interpreter"


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

[Paolo Bonzini]

On 22/06/2017 05:32, Philippe Mathieu-Daudé wrote:
> diff --git a/.travis.yml b/.travis.yml
> index a7512f3ab0..96ddc16cfd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -68,6 +68,8 @@ env:
>        # be kind with Travis free plan, restrict to at most 4 cores
>      - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
>      - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> +    # ccache
> +    - CCACHE_SLOPPINESS="file_macro,time_macros"

Does this actually have any effect?  We should not be using time macros.

include_file_ctime/mtime makes sense though.

Paolo

>      # tests
>      - TEST_CMD="make -j${PARALLEL_JOBS} check"

Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 394 bytes --]

On Thu, Jun 22, 2017 at 12:32:11AM -0300, Philippe Mathieu-Daudé wrote:
> This allow a one liner from fresh repository clone, i.e.:
> 
>   ./configure && make -j check-qtest-aarch64
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [RFC PATCH v2 21/21] ui/vnc: silent unuseful OSX clang warning

[Gerd Hoffmann]

On Thu, 2017-06-22 at 08:35 +0100, Peter Maydell wrote:
> On 22 June 2017 at 08:33, Gerd Hoffmann <kraxel@redhat.com> wrote:
> > diff --git a/ui/Makefile.objs b/ui/Makefile.objs
> > > index 3369451285..08fb573a48 100644
> > > --- a/ui/Makefile.objs
> > > +++ b/ui/Makefile.objs
> > > @@ -6,6 +6,14 @@ vnc-obj-y += vnc-auth-vencrypt.o
> > >  vnc-obj-$(CONFIG_VNC_SASL) += vnc-auth-sasl.o
> > >  vnc-obj-y += vnc-ws.o
> > >  vnc-obj-y += vnc-jobs.o
> > > +ifeq ($(CONFIG_VNC_SASL),y)
> > > +# silent OSX SASL warnings (from https://stackoverflow.com/a/740
> > > 6994
> > > ):
> > > +# because OpenSSL doesn’t offer API compatibility between
> > > versions,
> > > [...] Apple
> > > +# can't provide security updates without breaking existing apps,
> > > so
> > > is migrating
> > > +# from OpenSSL to Common Crypto.
> > > +vnc.o-cflags := -Wno-deprecated-declarations
> > > +vnc-auth-sasl.o-cflags := -Wno-deprecated-declarations
> > > +endif
> > 
> > Hmm, does clang understand "#pragma GCC diagnostic" ?
> > 
> > If so, then this can be done in in the code not the makefiles, and
> > also
> > selectively for osx only.
> 
> It does have some diagnostic pragma support. The awkward part is
> that it has to  be in force at the point where the deprecated
> function is used, not where it's declared. So you can't just wrap
> the #include of the ssl header in pragmas, you'd have to either
> do it at every callsite or else over the whole .c file.

Do it on the whole file on osx only is still better because we continue
to get the warnings on non-osx then, which should be enough to avoid
the code bitroting unnoticed.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH v2 11/21] travis: cache git submodules

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> Travis caching uses the branch name to store packed cache, so each new branch
>> will trigger a cache miss and will clone all submodules. Subsequent builds will
>> benefit from the cache.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> \o/
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

Hmmm looking deeper it doesn't look as though we are benefiting from the
caching here. The submodule checkout is around 60-70s with some jobs
coming down to 30-40s. The old system seems to be ~20s for the
wget/checkout step (assuming the wget is cached).

>
>
>> ---
>>  .travis.yml | 7 ++-----
>>  1 file changed, 2 insertions(+), 5 deletions(-)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 62b9dfd2ae..777914f759 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -7,6 +7,8 @@ compiler:
>>  cache:
>>    ccache: true
>>    timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
>> +  directories:
>> +  - $HOME/qemu/.git/modules
>>  addons:
>>    apt:
>>      packages:
>> @@ -74,7 +76,6 @@ git:
>>  before_install:
>>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
>>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
>> -  - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>>    - git submodule update --init --recursive
>>  before_script:
>>    - ./configure ${CONFIG}
>> @@ -113,7 +114,6 @@ matrix:
>>        before_install:
>>          - sudo apt-get update -qq
>>          - sudo apt-get build-dep -qq qemu
>> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>>          - git submodule update --init --recursive
>>      # Plain Trusty Linux User Build
>>      - env: CONFIG="--disable-system"
>> @@ -124,7 +124,6 @@ matrix:
>>        before_install:
>>          - sudo apt-get update -qq
>>          - sudo apt-get build-dep -qq qemu
>> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>>          - git submodule update --init --recursive
>>      # Trusty System build with latest stable clang
>>      - sudo: required
>> @@ -142,7 +141,6 @@ matrix:
>>          - travis_retry sudo apt-get update -qq
>>          - travis_retry sudo apt-get install -qq -y clang-3.9
>>          - sudo apt-get build-dep -qq qemu
>> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>>          - git submodule update --init --recursive
>>        before_script:
>>          - ./configure ${CONFIG} || cat config.log
>> @@ -162,7 +160,6 @@ matrix:
>>          - travis_retry sudo apt-get update -qq
>>          - travis_retry sudo apt-get install -qq -y clang-3.9
>>          - sudo apt-get build-dep -qq qemu
>> -        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>>          - git submodule update --init --recursive
>>        before_script:
>>          - ./configure ${CONFIG} || cat config.log


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 06/21] travis: Add config to do a Coverity Scan upload

[Philippe Mathieu-Daudé]

On 06/22/2017 06:56 AM, Alex Bennée wrote:[...]
> I think this is waiting on a fix I mention when reviewing Peter's
> original patches.

Ok! I'll wait or drop the Coverity part.

Re: [Qemu-devel] [PATCH v2 07/21] travis: update sudo-enabled Trusty images

[Philippe Mathieu-Daudé]

On 06/22/2017 06:58 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> The way I read that blog post is group: edge is for testing things still
> work but you probably don't want to merge that into your master branch.
> Otherwise we'll automatically be running the bleeding edge.

Ok I misunderstood this part, good you noticed.

Travis team posted a new entry after which is clearer:
https://blog.travis-ci.com/2017-06-21-trusty-updates-2017-Q2-launch

I'm now using "group: deprecated-2017Q2" instead of "edge".

Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

[Philippe Mathieu-Daudé]

On 06/22/2017 07:22 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> Can you expand the commit message to breifly explain what all these
> extra settings change. What is CCACHE_SLOPINESS and ccache -s doing?

I'm pretty sure I wrote something here ... probably lost using git Fu :(

Re: [Qemu-devel] [PATCH v2 14/21] travis: dump config.log if ./configure script fails

[Philippe Mathieu-Daudé]

On 06/22/2017 07:23 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
[...]
>> -  - ./configure ${CONFIG}
>> +  - ./configure ${CONFIG} || cat config.log
> 
> Won't the explicit || mean the job will keep trying even if config is
> broken? Should it be || cat config.log && false?

Yes, you right. I actually copied it from the "Trusty build with latest 
stable clang" job from commit ae1a772c :p

I'll see to fix other occurrences.

Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY

[Philippe Mathieu-Daudé]

On 06/22/2017 06:55 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> This allow a one liner from fresh repository clone, i.e.:
>>
>>    ./configure && make -j check-qtest-aarch64
> 
> TIL -j doesn't need a number and GNU Make has a -l option ;-)

Also since 2.8 (2016) git fetch has -j option, but Travis is using a 
release version 1.8 (from the git-core PPA)

on my debian/unstable git version 2.13.1 I can do:

  $ git submodule update --init --recursive --depth 1 -j ${PARALLEL_JOBS}

Re: [Qemu-devel] [PATCH v2 01/21] tests: add missing dependency to build QTEST_QEMU_BINARY

[John Snow]

On 06/21/2017 11:32 PM, Philippe Mathieu-Daudé wrote:
> This allow a one liner from fresh repository clone, i.e.:
> 
>   ./configure && make -j check-qtest-aarch64
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 77da9b7f4b..f7c8f3eddb 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -823,7 +823,7 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
>  # gtester tests, possibly with verbose output
>  
>  .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
> -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
> +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: subdir-%-softmmu $(check-qtest-y)
>  	$(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
>  	$(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
>  		QTEST_QEMU_IMG=qemu-img$(EXESUF) \
> 

I'm on board, it's nice to have proper dependencies.

Reviewed-by: John Snow <jsnow@redhat.com>