[Qemu-devel] [PATCH for 4.1? v1 0/7] testing/next (docker, win-cross)

[Qemu-devel] [PATCH for 4.1? v1 0/7] testing/next (docker, win-cross)

[Alex Bennée]

Hi,

This is my current queue for testing/next. I think they are all worth
it for 4.1 but it would be nice to get the Windows cross builds up and
running again.

The following patches need review
 patch 0001/tests docker add test misc for building tools doc.patch
 patch 0005/tests migration test don t spam the logs when we .patch
 patch 0006/tests dockerfiles update the win cross builds to .patch
 patch 0007/shippable re enable the windows cross builds.patch

Alex Bennée (4):
  tests/docker: add test-misc for building tools & docs
  tests/migration-test: don't spam the logs when we fail
  tests/dockerfiles: update the win cross builds to stretch
  shippable: re-enable the windows cross builds

Philippe Mathieu-Daudé (3):
  tests/docker: Install Sphinx in the Ubuntu images
  tests/docker: Install Sphinx in the Fedora image
  tests/docker: Install Ubuntu images noninteractively

 .shippable.yml                                |  9 ++++----
 tests/docker/Makefile.include                 |  6 ++---
 .../dockerfiles/debian-win32-cross.docker     |  4 ++--
 .../dockerfiles/debian-win64-cross.docker     |  4 ++--
 ...{debian8-mxe.docker => debian9-mxe.docker} | 11 ++++++----
 tests/docker/dockerfiles/fedora.docker        |  1 +
 tests/docker/dockerfiles/ubuntu.docker        |  3 ++-
 tests/docker/dockerfiles/ubuntu1804.docker    |  3 ++-
 tests/docker/test-misc                        | 22 +++++++++++++++++++
 tests/migration-test.c                        | 19 ++++++++++------
 10 files changed, 57 insertions(+), 25 deletions(-)
 rename tests/docker/dockerfiles/{debian8-mxe.docker => debian9-mxe.docker} (56%)
 create mode 100755 tests/docker/test-misc

-- 
2.20.1

[Qemu-devel] [PATCH v1 1/7] tests/docker: add test-misc for building tools & docs

[Alex Bennée]

Add yet another test type so we cna quickly exercise the miscellaneous
build products of the build system under various docer configurations.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/test-misc | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100755 tests/docker/test-misc

diff --git a/tests/docker/test-misc b/tests/docker/test-misc
new file mode 100755
index 00000000000..d480afedca7
--- /dev/null
+++ b/tests/docker/test-misc
@@ -0,0 +1,22 @@
+#!/bin/bash -e
+#
+# Build the miscellaneous components
+#
+# Copyright (c) 2019 Linaro Ltd.
+#
+# Authors:
+#  Alex Bennée <alex.bennee@linaro.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+# build everything else but QEMU
+configure_qemu --disable-user --disable-system --enable-docs --enable-tools
+build_qemu
+install_qemu
-- 
2.20.1

[Qemu-devel] [PATCH v1 2/7] tests/docker: Install Sphinx in the Ubuntu images

[Alex Bennée]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190711120609.12773-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/ubuntu.docker     | 1 +
 tests/docker/dockerfiles/ubuntu1804.docker | 1 +
 2 files changed, 2 insertions(+)

diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index d3b72209c85..2500ec84b6f 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -61,6 +61,7 @@ ENV PACKAGES flex bison \
     libxen-dev \
     make \
     python-yaml \
+    python3-sphinx \
     sparse \
     texinfo \
     xfslibs-dev
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index 9d80b115009..0bb8088658d 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -48,6 +48,7 @@ ENV PACKAGES flex bison \
     libxen-dev \
     make \
     python-yaml \
+    python3-sphinx \
     sparse \
     texinfo \
     xfslibs-dev
-- 
2.20.1

[Qemu-devel] [PATCH v1 3/7] tests/docker: Install Sphinx in the Fedora image

[Alex Bennée]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190711102710.2263-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/fedora.docker | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 619d1b5656d..e6d39e14cb1 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -76,6 +76,7 @@ ENV PACKAGES \
     perl-Test-Harness \
     pixman-devel \
     python3 \
+    python3-sphinx \
     PyYAML \
     rdma-core-devel \
     SDL2-devel \
-- 
2.20.1

[Qemu-devel] [PATCH v1 4/7] tests/docker: Install Ubuntu images noninteractively

[Alex Bennée]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

We correctly use the DEBIAN_FRONTEND environment variable on
the Debian images, but forgot the Ubuntu ones are based on it.

Since building docker images is not interactive, we need to
inform the APT tools about it using the DEBIAN_FRONTEND
environment variable (we already use it on our Debian images).

This fixes:

  $ make docker-image-ubuntu V=1
  [...]
  Setting up tzdata (2019b-0ubuntu0.19.04) ...
  debconf: unable to initialize frontend: Dialog
  debconf: (TERM is not set, so the dialog frontend is not usable.)
  debconf: falling back to frontend: Readline
  Configuring tzdata
  ------------------

  Please select the geographic area in which you live. Subsequent configuration
  questions will narrow this down by presenting a list of cities, representing
  the time zones in which they are located.

    1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
    2. America     5. Arctic     8. Europe    11. SystemV
    3. Antarctica  6. Asia       9. Indian    12. US
  Geographic area: 12
  [HANG]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190711124805.26476-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/ubuntu.docker     | 2 +-
 tests/docker/dockerfiles/ubuntu1804.docker | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index 2500ec84b6f..a4f601395c8 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -66,6 +66,6 @@ ENV PACKAGES flex bison \
     texinfo \
     xfslibs-dev
 RUN apt-get update && \
-    apt-get -y install $PACKAGES
+    DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
 RUN dpkg -l $PACKAGES | sort > /packages.txt
 ENV FEATURES clang pyyaml sdl2
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index 0bb8088658d..44bbf0f77ae 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -53,6 +53,6 @@ ENV PACKAGES flex bison \
     texinfo \
     xfslibs-dev
 RUN apt-get update && \
-    apt-get -y install $PACKAGES
+    DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
 RUN dpkg -l $PACKAGES | sort > /packages.txt
 ENV FEATURES clang pyyaml sdl2
-- 
2.20.1

[Qemu-devel] [PATCH v1 5/7] tests/migration-test: don't spam the logs when we fail

[Alex Bennée]

Quite often the information about which test failed is hidden by the
wall of repeated failures for each page. Stop outputting the error
after 10 bad pages and just summarise the total damage at the end.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/migration-test.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index b6434628e1c..ce041f80c2a 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -308,7 +308,7 @@ static void check_guests_ram(QTestState *who)
     uint8_t first_byte;
     uint8_t last_byte;
     bool hit_edge = false;
-    bool bad = false;
+    int bad = 0;
 
     qtest_memread(who, start_address, &first_byte, 1);
     last_byte = first_byte;
@@ -327,15 +327,20 @@ static void check_guests_ram(QTestState *who)
                 hit_edge = true;
                 last_byte = b;
             } else {
-                fprintf(stderr, "Memory content inconsistency at %x"
-                                " first_byte = %x last_byte = %x current = %x"
-                                " hit_edge = %x\n",
-                                address, first_byte, last_byte, b, hit_edge);
-                bad = true;
+                bad++;
+                if (bad <= 10) {
+                    fprintf(stderr, "Memory content inconsistency at %x"
+                            " first_byte = %x last_byte = %x current = %x"
+                            " hit_edge = %x\n",
+                            address, first_byte, last_byte, b, hit_edge);
+                }
             }
         }
     }
-    g_assert_false(bad);
+    if (bad >= 10) {
+        fprintf(stderr, "and in another %d pages", bad);
+    }
+    g_assert(bad == 0);
 }
 
 static void cleanup(const char *filename)
-- 
2.20.1

[Qemu-devel] [PATCH v1 6/7] tests/dockerfiles: update the win cross builds to stretch

[Alex Bennée]

While fixing up pkg.mxe.cc they move the URLs around a bit and dropped
Jessie support in favour of Stretch. We also need to update the keys
used to verify the packages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include                         |  6 +++---
 tests/docker/dockerfiles/debian-win32-cross.docker    |  4 ++--
 tests/docker/dockerfiles/debian-win64-cross.docker    |  4 ++--
 .../{debian8-mxe.docker => debian9-mxe.docker}        | 11 +++++++----
 4 files changed, 14 insertions(+), 11 deletions(-)
 rename tests/docker/dockerfiles/{debian8-mxe.docker => debian9-mxe.docker} (56%)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index aaf5396b85d..dbd58e548c1 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -85,7 +85,7 @@ endif
 
 # Enforce dependencies for composite images
 docker-image-debian: docker-image-debian9
-docker-image-debian8-mxe: docker-image-debian8
+docker-image-debian9-mxe: docker-image-debian9
 docker-image-debian-amd64: docker-image-debian9
 docker-image-debian-armel-cross: docker-image-debian9
 docker-image-debian-armhf-cross: docker-image-debian9
@@ -96,8 +96,8 @@ docker-image-debian-mipsel-cross: docker-image-debian9
 docker-image-debian-mips64el-cross: docker-image-debian9
 docker-image-debian-ppc64el-cross: docker-image-debian9
 docker-image-debian-s390x-cross: docker-image-debian9
-docker-image-debian-win32-cross: docker-image-debian8-mxe
-docker-image-debian-win64-cross: docker-image-debian8-mxe
+docker-image-debian-win32-cross: docker-image-debian9-mxe
+docker-image-debian-win64-cross: docker-image-debian9-mxe
 
 docker-image-debian-alpha-cross: docker-image-debian-sid
 docker-image-debian-hppa-cross: docker-image-debian-sid
diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker
index 0a4970c0683..c787e432454 100644
--- a/tests/docker/dockerfiles/debian-win32-cross.docker
+++ b/tests/docker/dockerfiles/debian-win32-cross.docker
@@ -1,9 +1,9 @@
 #
 # Docker mingw32 cross-compiler target
 #
-# This docker target builds on the debian Jessie MXE base image.
+# This docker target builds on the debian Stretch MXE base image.
 #
-FROM qemu:debian8-mxe
+FROM qemu:debian9-mxe
 
 MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
 
diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker
index b27985b1b1f..a7068ed6ac6 100644
--- a/tests/docker/dockerfiles/debian-win64-cross.docker
+++ b/tests/docker/dockerfiles/debian-win64-cross.docker
@@ -1,9 +1,9 @@
 #
 # Docker mingw64 cross-compiler target
 #
-# This docker target builds on the debian Jessie MXE base image.
+# This docker target builds on the debian Stretch MXE base image.
 #
-FROM qemu:debian8-mxe
+FROM qemu:debian9-mxe
 
 MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
 
diff --git a/tests/docker/dockerfiles/debian8-mxe.docker b/tests/docker/dockerfiles/debian9-mxe.docker
similarity index 56%
rename from tests/docker/dockerfiles/debian8-mxe.docker
rename to tests/docker/dockerfiles/debian9-mxe.docker
index 2df4cc8c5c9..5bc8a6d5c36 100644
--- a/tests/docker/dockerfiles/debian8-mxe.docker
+++ b/tests/docker/dockerfiles/debian9-mxe.docker
@@ -1,15 +1,18 @@
 #
 # Docker mingw cross-compiler target
 #
-# This docker target builds on the debian Jessie base image.
+# This docker target builds on the debian Stretch base image.
 #
-FROM qemu:debian8
+FROM qemu:debian9
 
 MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
 
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt install -y --no-install-recommends gnupg dirmngr
+
 # Add the foreign architecture we want and install dependencies
-RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D43A795B73B16ABE9643FE1AFD8FFF16DB45C6AB && \
-    echo "deb http://pkg.mxe.cc/repos/apt/debian jessie main" > /etc/apt/sources.list.d/mxeapt.list
+RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C6BF758A33A3A276 && \
+    echo "deb http://pkg.mxe.cc/repos/apt stretch main" > /etc/apt/sources.list.d/mxeapt.list
 RUN apt-get update
 RUN DEBIAN_FRONTEND=noninteractive eatmydata \
     apt-get install -y --no-install-recommends \
-- 
2.20.1

[Qemu-devel] [PATCH v1 7/7] shippable: re-enable the windows cross builds

[Alex Bennée]

The pkg.mxe.cc repo has been restored.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .shippable.yml | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/.shippable.yml b/.shippable.yml
index f2ffef21d11..f74a3de3ffd 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -7,11 +7,10 @@ env:
   matrix:
     - IMAGE=debian-amd64
       TARGET_LIST=x86_64-softmmu,x86_64-linux-user
-    # currently disabled as the mxe.cc repos are down
-    # - IMAGE=debian-win32-cross
-    #   TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
-    # - IMAGE=debian-win64-cross
-    #   TARGET_LIST=aarch64-softmmu,sparc64-softmmu,x86_64-softmmu
+    - IMAGE=debian-win32-cross
+      TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
+    - IMAGE=debian-win64-cross
+      TARGET_LIST=aarch64-softmmu,sparc64-softmmu,x86_64-softmmu
     - IMAGE=debian-armel-cross
       TARGET_LIST=arm-softmmu,arm-linux-user,armeb-linux-user
     - IMAGE=debian-armhf-cross
-- 
2.20.1

Re: [Qemu-devel] [PATCH v1 5/7] tests/migration-test: don't spam the logs when we fail

[Laurent Vivier]

On 12/07/2019 13:18, Alex Bennée wrote:
> Quite often the information about which test failed is hidden by the
> wall of repeated failures for each page. Stop outputting the error
> after 10 bad pages and just summarise the total damage at the end.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/migration-test.c | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index b6434628e1c..ce041f80c2a 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -308,7 +308,7 @@ static void check_guests_ram(QTestState *who)
>      uint8_t first_byte;
>      uint8_t last_byte;
>      bool hit_edge = false;
> -    bool bad = false;
> +    int bad = 0;
>  
>      qtest_memread(who, start_address, &first_byte, 1);
>      last_byte = first_byte;
> @@ -327,15 +327,20 @@ static void check_guests_ram(QTestState *who)
>                  hit_edge = true;
>                  last_byte = b;
>              } else {
> -                fprintf(stderr, "Memory content inconsistency at %x"
> -                                " first_byte = %x last_byte = %x current = %x"
> -                                " hit_edge = %x\n",
> -                                address, first_byte, last_byte, b, hit_edge);
> -                bad = true;
> +                bad++;
> +                if (bad <= 10) {
> +                    fprintf(stderr, "Memory content inconsistency at %x"
> +                            " first_byte = %x last_byte = %x current = %x"
> +                            " hit_edge = %x\n",
> +                            address, first_byte, last_byte, b, hit_edge);
> +                }
>              }
>          }
>      }
> -    g_assert_false(bad);
> +    if (bad >= 10) {
> +        fprintf(stderr, "and in another %d pages", bad);

"bad - 10" as you have already displayed 10 errors.

Thanks,
Laurent

Re: [Qemu-devel] [PATCH v1 5/7] tests/migration-test: don't spam the logs when we fail

[Thomas Huth]

On 12/07/2019 13.18, Alex Bennée wrote:
> Quite often the information about which test failed is hidden by the
> wall of repeated failures for each page. Stop outputting the error
> after 10 bad pages and just summarise the total damage at the end.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/migration-test.c | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index b6434628e1c..ce041f80c2a 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -308,7 +308,7 @@ static void check_guests_ram(QTestState *who)
>      uint8_t first_byte;
>      uint8_t last_byte;
>      bool hit_edge = false;
> -    bool bad = false;
> +    int bad = 0;
>  
>      qtest_memread(who, start_address, &first_byte, 1);
>      last_byte = first_byte;
> @@ -327,15 +327,20 @@ static void check_guests_ram(QTestState *who)
>                  hit_edge = true;
>                  last_byte = b;
>              } else {
> -                fprintf(stderr, "Memory content inconsistency at %x"
> -                                " first_byte = %x last_byte = %x current = %x"
> -                                " hit_edge = %x\n",
> -                                address, first_byte, last_byte, b, hit_edge);
> -                bad = true;
> +                bad++;
> +                if (bad <= 10) {
> +                    fprintf(stderr, "Memory content inconsistency at %x"
> +                            " first_byte = %x last_byte = %x current = %x"
> +                            " hit_edge = %x\n",
> +                            address, first_byte, last_byte, b, hit_edge);
> +                }
>              }
>          }
>      }
> -    g_assert_false(bad);
> +    if (bad >= 10) {
> +        fprintf(stderr, "and in another %d pages", bad);
> +    }
> +    g_assert(bad == 0);
>  }
>  
>  static void cleanup(const char *filename)

Good idea.

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [Qemu-devel] [PATCH v1 5/7] tests/migration-test: don't spam the logs when we fail

[Dr. David Alan Gilbert]

* Alex Bennée (alex.bennee@linaro.org) wrote:
> Quite often the information about which test failed is hidden by the
> wall of repeated failures for each page. Stop outputting the error
> after 10 bad pages and just summarise the total damage at the end.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Yep, occasionally you do find you do want the full set to see what's
going on, but this is true most of the time.

With 10 you should be able to see if it's a single page that's wrong or
the flip gone wrong.

> ---
>  tests/migration-test.c | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index b6434628e1c..ce041f80c2a 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -308,7 +308,7 @@ static void check_guests_ram(QTestState *who)
>      uint8_t first_byte;
>      uint8_t last_byte;
>      bool hit_edge = false;
> -    bool bad = false;
> +    int bad = 0;
>  
>      qtest_memread(who, start_address, &first_byte, 1);
>      last_byte = first_byte;
> @@ -327,15 +327,20 @@ static void check_guests_ram(QTestState *who)
>                  hit_edge = true;
>                  last_byte = b;
>              } else {
> -                fprintf(stderr, "Memory content inconsistency at %x"
> -                                " first_byte = %x last_byte = %x current = %x"
> -                                " hit_edge = %x\n",
> -                                address, first_byte, last_byte, b, hit_edge);
> -                bad = true;
> +                bad++;
> +                if (bad <= 10) {
> +                    fprintf(stderr, "Memory content inconsistency at %x"
> +                            " first_byte = %x last_byte = %x current = %x"
> +                            " hit_edge = %x\n",
> +                            address, first_byte, last_byte, b, hit_edge);
> +                }
>              }
>          }
>      }
> -    g_assert_false(bad);
> +    if (bad >= 10) {
> +        fprintf(stderr, "and in another %d pages", bad);
> +    }

as Laurent says, the 'another' would need -10 but other than that:


Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> +    g_assert(bad == 0);
>  }
>  
>  static void cleanup(const char *filename)
> -- 
> 2.20.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v1 5/7] tests/migration-test: don't spam the logs when we fail

[Alex Bennée]

Laurent Vivier <lvivier@redhat.com> writes:

> On 12/07/2019 13:18, Alex Bennée wrote:
>> Quite often the information about which test failed is hidden by the
>> wall of repeated failures for each page. Stop outputting the error
>> after 10 bad pages and just summarise the total damage at the end.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  tests/migration-test.c | 19 ++++++++++++-------
>>  1 file changed, 12 insertions(+), 7 deletions(-)
>>
>> diff --git a/tests/migration-test.c b/tests/migration-test.c
>> index b6434628e1c..ce041f80c2a 100644
>> --- a/tests/migration-test.c
>> +++ b/tests/migration-test.c
>> @@ -308,7 +308,7 @@ static void check_guests_ram(QTestState *who)
>>      uint8_t first_byte;
>>      uint8_t last_byte;
>>      bool hit_edge = false;
>> -    bool bad = false;
>> +    int bad = 0;
>>
>>      qtest_memread(who, start_address, &first_byte, 1);
>>      last_byte = first_byte;
>> @@ -327,15 +327,20 @@ static void check_guests_ram(QTestState *who)
>>                  hit_edge = true;
>>                  last_byte = b;
>>              } else {
>> -                fprintf(stderr, "Memory content inconsistency at %x"
>> -                                " first_byte = %x last_byte = %x current = %x"
>> -                                " hit_edge = %x\n",
>> -                                address, first_byte, last_byte, b, hit_edge);
>> -                bad = true;
>> +                bad++;
>> +                if (bad <= 10) {
>> +                    fprintf(stderr, "Memory content inconsistency at %x"
>> +                            " first_byte = %x last_byte = %x current = %x"
>> +                            " hit_edge = %x\n",
>> +                            address, first_byte, last_byte, b, hit_edge);
>> +                }
>>              }
>>          }
>>      }
>> -    g_assert_false(bad);
>> +    if (bad >= 10) {
>> +        fprintf(stderr, "and in another %d pages", bad);
>
> "bad - 10" as you have already displayed 10 errors.

Will do.

>
> Thanks,
> Laurent


--
Alex Bennée

Re: [Qemu-devel] [PATCH v1 5/7] tests/migration-test: don't spam the logs when we fail

[Laurent Vivier]

On 12/07/2019 15:35, Alex Bennée wrote:
> 
> Laurent Vivier <lvivier@redhat.com> writes:
> 
>> On 12/07/2019 13:18, Alex Bennée wrote:
>>> Quite often the information about which test failed is hidden by the
>>> wall of repeated failures for each page. Stop outputting the error
>>> after 10 bad pages and just summarise the total damage at the end.
>>>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> ---
>>>  tests/migration-test.c | 19 ++++++++++++-------
>>>  1 file changed, 12 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/tests/migration-test.c b/tests/migration-test.c
>>> index b6434628e1c..ce041f80c2a 100644
>>> --- a/tests/migration-test.c
>>> +++ b/tests/migration-test.c
>>> @@ -308,7 +308,7 @@ static void check_guests_ram(QTestState *who)
>>>      uint8_t first_byte;
>>>      uint8_t last_byte;
>>>      bool hit_edge = false;
>>> -    bool bad = false;
>>> +    int bad = 0;
>>>
>>>      qtest_memread(who, start_address, &first_byte, 1);
>>>      last_byte = first_byte;
>>> @@ -327,15 +327,20 @@ static void check_guests_ram(QTestState *who)
>>>                  hit_edge = true;
>>>                  last_byte = b;
>>>              } else {
>>> -                fprintf(stderr, "Memory content inconsistency at %x"
>>> -                                " first_byte = %x last_byte = %x current = %x"
>>> -                                " hit_edge = %x\n",
>>> -                                address, first_byte, last_byte, b, hit_edge);
>>> -                bad = true;
>>> +                bad++;
>>> +                if (bad <= 10) {
>>> +                    fprintf(stderr, "Memory content inconsistency at %x"
>>> +                            " first_byte = %x last_byte = %x current = %x"
>>> +                            " hit_edge = %x\n",
>>> +                            address, first_byte, last_byte, b, hit_edge);
>>> +                }
>>>              }
>>>          }
>>>      }
>>> -    g_assert_false(bad);
>>> +    if (bad >= 10) {
>>> +        fprintf(stderr, "and in another %d pages", bad);
>>
>> "bad - 10" as you have already displayed 10 errors.
> 
> Will do.

You can add my:

Reviewed-by: Laurent Vivier <lvivier@redhat.com>

Thanks,
Laurent

Re: [Qemu-devel] [PATCH v1 6/7] tests/dockerfiles: update the win cross builds to stretch

[Philippe Mathieu-Daudé]

On 7/12/19 1:18 PM, Alex Bennée wrote:
> While fixing up pkg.mxe.cc they move the URLs around a bit and dropped
> Jessie support in favour of Stretch. We also need to update the keys
> used to verify the packages.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include                         |  6 +++---
>  tests/docker/dockerfiles/debian-win32-cross.docker    |  4 ++--
>  tests/docker/dockerfiles/debian-win64-cross.docker    |  4 ++--
>  .../{debian8-mxe.docker => debian9-mxe.docker}        | 11 +++++++----
>  4 files changed, 14 insertions(+), 11 deletions(-)
>  rename tests/docker/dockerfiles/{debian8-mxe.docker => debian9-mxe.docker} (56%)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index aaf5396b85d..dbd58e548c1 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -85,7 +85,7 @@ endif
>  
>  # Enforce dependencies for composite images
>  docker-image-debian: docker-image-debian9
> -docker-image-debian8-mxe: docker-image-debian8
> +docker-image-debian9-mxe: docker-image-debian9
>  docker-image-debian-amd64: docker-image-debian9
>  docker-image-debian-armel-cross: docker-image-debian9
>  docker-image-debian-armhf-cross: docker-image-debian9
> @@ -96,8 +96,8 @@ docker-image-debian-mipsel-cross: docker-image-debian9
>  docker-image-debian-mips64el-cross: docker-image-debian9
>  docker-image-debian-ppc64el-cross: docker-image-debian9
>  docker-image-debian-s390x-cross: docker-image-debian9
> -docker-image-debian-win32-cross: docker-image-debian8-mxe
> -docker-image-debian-win64-cross: docker-image-debian8-mxe
> +docker-image-debian-win32-cross: docker-image-debian9-mxe
> +docker-image-debian-win64-cross: docker-image-debian9-mxe
>  
>  docker-image-debian-alpha-cross: docker-image-debian-sid
>  docker-image-debian-hppa-cross: docker-image-debian-sid
> diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker
> index 0a4970c0683..c787e432454 100644
> --- a/tests/docker/dockerfiles/debian-win32-cross.docker
> +++ b/tests/docker/dockerfiles/debian-win32-cross.docker
> @@ -1,9 +1,9 @@
>  #
>  # Docker mingw32 cross-compiler target
>  #
> -# This docker target builds on the debian Jessie MXE base image.
> +# This docker target builds on the debian Stretch MXE base image.
>  #
> -FROM qemu:debian8-mxe
> +FROM qemu:debian9-mxe
>  
>  MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
>  
> diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker
> index b27985b1b1f..a7068ed6ac6 100644
> --- a/tests/docker/dockerfiles/debian-win64-cross.docker
> +++ b/tests/docker/dockerfiles/debian-win64-cross.docker
> @@ -1,9 +1,9 @@
>  #
>  # Docker mingw64 cross-compiler target
>  #
> -# This docker target builds on the debian Jessie MXE base image.
> +# This docker target builds on the debian Stretch MXE base image.
>  #
> -FROM qemu:debian8-mxe
> +FROM qemu:debian9-mxe
>  
>  MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
>  
> diff --git a/tests/docker/dockerfiles/debian8-mxe.docker b/tests/docker/dockerfiles/debian9-mxe.docker
> similarity index 56%
> rename from tests/docker/dockerfiles/debian8-mxe.docker
> rename to tests/docker/dockerfiles/debian9-mxe.docker
> index 2df4cc8c5c9..5bc8a6d5c36 100644
> --- a/tests/docker/dockerfiles/debian8-mxe.docker
> +++ b/tests/docker/dockerfiles/debian9-mxe.docker
> @@ -1,15 +1,18 @@
>  #
>  # Docker mingw cross-compiler target
>  #
> -# This docker target builds on the debian Jessie base image.
> +# This docker target builds on the debian Stretch base image.
>  #
> -FROM qemu:debian8
> +FROM qemu:debian9
>  
>  MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
>  
> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +    apt install -y --no-install-recommends gnupg dirmngr
> +
>  # Add the foreign architecture we want and install dependencies
> -RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D43A795B73B16ABE9643FE1AFD8FFF16DB45C6AB && \
> -    echo "deb http://pkg.mxe.cc/repos/apt/debian jessie main" > /etc/apt/sources.list.d/mxeapt.list
> +RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C6BF758A33A3A276 && \
> +    echo "deb http://pkg.mxe.cc/repos/apt stretch main" > /etc/apt/sources.list.d/mxeapt.list
>  RUN apt-get update
>  RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>      apt-get install -y --no-install-recommends \
> 

Thank you Alex!

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [Qemu-devel] [PATCH v1 7/7] shippable: re-enable the windows cross builds

[Philippe Mathieu-Daudé]

On 7/12/19 1:18 PM, Alex Bennée wrote:
> The pkg.mxe.cc repo has been restored.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .shippable.yml | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/.shippable.yml b/.shippable.yml
> index f2ffef21d11..f74a3de3ffd 100644
> --- a/.shippable.yml
> +++ b/.shippable.yml
> @@ -7,11 +7,10 @@ env:
>    matrix:
>      - IMAGE=debian-amd64
>        TARGET_LIST=x86_64-softmmu,x86_64-linux-user
> -    # currently disabled as the mxe.cc repos are down
> -    # - IMAGE=debian-win32-cross
> -    #   TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
> -    # - IMAGE=debian-win64-cross
> -    #   TARGET_LIST=aarch64-softmmu,sparc64-softmmu,x86_64-softmmu
> +    - IMAGE=debian-win32-cross
> +      TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
> +    - IMAGE=debian-win64-cross
> +      TARGET_LIST=aarch64-softmmu,sparc64-softmmu,x86_64-softmmu
>      - IMAGE=debian-armel-cross
>        TARGET_LIST=arm-softmmu,arm-linux-user,armeb-linux-user
>      - IMAGE=debian-armhf-cross
> 

Builds now take ~25min longer. The win32 build is definitively helpful
and should catch most of the errors the win64 could find. I suggest we
build the docker image but not cross-compile on win64. Anyway this can
be improved later.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [Qemu-devel] [PATCH v1 1/7] tests/docker: add test-misc for building tools & docs

[Philippe Mathieu-Daudé]

On 7/12/19 1:18 PM, Alex Bennée wrote:
> Add yet another test type so we cna quickly exercise the miscellaneous

"cna" -> "can"

> build products of the build system under various docer configurations.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/test-misc | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>  create mode 100755 tests/docker/test-misc
> 
> diff --git a/tests/docker/test-misc b/tests/docker/test-misc
> new file mode 100755
> index 00000000000..d480afedca7
> --- /dev/null
> +++ b/tests/docker/test-misc
> @@ -0,0 +1,22 @@
> +#!/bin/bash -e
> +#
> +# Build the miscellaneous components
> +#
> +# Copyright (c) 2019 Linaro Ltd.
> +#
> +# Authors:
> +#  Alex Bennée <alex.bennee@linaro.org>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +. common.rc
> +
> +cd "$BUILD_DIR"
> +
> +# build everything else but QEMU
> +configure_qemu --disable-user --disable-system --enable-docs --enable-tools
> +build_qemu
> +install_qemu
> 

I got:

$ make docker-test-misc@debian-win32-cross
  BUILD   debian9
  GEN
/home/phil/source/qemu/docker-src.2019-07-13-17.03.48.3829/qemu.tar
  BUILD   debian9-mxe
usage: git archive [<options>] <tree-ish> [<path>...]
   or: git archive --list
   or: git archive --remote <repo> [--exec <cmd>] [<options>] <tree-ish>
[<path>...]
   or: git archive --remote <repo> [--exec <cmd>] --list

    --format <fmt>        archive format
    --prefix <prefix>     prepend prefix to each pathname in the archive
    -o, --output <file>   write the archive to this file
    --worktree-attributes
                          read .gitattributes in working directory
    -v, --verbose         report archived files on stderr
    -0                    store only
    -1                    compress faster
    -9                    compress better

    -l, --list            list supported archive formats

    --remote <repo>       retrieve the archive from remote repository <repo>
    --exec <command>      path to the remote git-upload-archive command

failed to archive qemu
make[1]: *** [tests/docker/Makefile.include:31:
qemu/docker-src.2019-07-13-17.03.48.3829] Error 1

And remembered some script/archive patch on the list:
https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg02073.html

With it amended this works correctly, so:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [Qemu-devel] [PATCH v1 1/7] tests/docker: add test-misc for building tools & docs

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> On 7/12/19 1:18 PM, Alex Bennée wrote:
>> Add yet another test type so we cna quickly exercise the miscellaneous
>
> "cna" -> "can"
>
>> build products of the build system under various docer configurations.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  tests/docker/test-misc | 22 ++++++++++++++++++++++
>>  1 file changed, 22 insertions(+)
>>  create mode 100755 tests/docker/test-misc
>>
>> diff --git a/tests/docker/test-misc b/tests/docker/test-misc
>> new file mode 100755
>> index 00000000000..d480afedca7
>> --- /dev/null
>> +++ b/tests/docker/test-misc
>> @@ -0,0 +1,22 @@
>> +#!/bin/bash -e
>> +#
>> +# Build the miscellaneous components
>> +#
>> +# Copyright (c) 2019 Linaro Ltd.
>> +#
>> +# Authors:
>> +#  Alex Bennée <alex.bennee@linaro.org>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
>> +# See the COPYING file in the top-level directory.
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +. common.rc
>> +
>> +cd "$BUILD_DIR"
>> +
>> +# build everything else but QEMU
>> +configure_qemu --disable-user --disable-system --enable-docs --enable-tools
>> +build_qemu
>> +install_qemu
>>
>
> I got:
>
> $ make docker-test-misc@debian-win32-cross
>   BUILD   debian9
>   GEN
> /home/phil/source/qemu/docker-src.2019-07-13-17.03.48.3829/qemu.tar
>   BUILD   debian9-mxe
> usage: git archive [<options>] <tree-ish> [<path>...]
>    or: git archive --list
>    or: git archive --remote <repo> [--exec <cmd>] [<options>] <tree-ish>
> [<path>...]
>    or: git archive --remote <repo> [--exec <cmd>] --list
>
>     --format <fmt>        archive format
>     --prefix <prefix>     prepend prefix to each pathname in the archive
>     -o, --output <file>   write the archive to this file
>     --worktree-attributes
>                           read .gitattributes in working directory
>     -v, --verbose         report archived files on stderr
>     -0                    store only
>     -1                    compress faster
>     -9                    compress better
>
>     -l, --list            list supported archive formats
>
>     --remote <repo>       retrieve the archive from remote repository <repo>
>     --exec <command>      path to the remote git-upload-archive command
>
> failed to archive qemu
> make[1]: *** [tests/docker/Makefile.include:31:
> qemu/docker-src.2019-07-13-17.03.48.3829] Error 1

Hmm I haven't been able to get this... does it depend on having
uncommited changes in a subproject?

>
> And remembered some script/archive patch on the list:
> https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg02073.html

I'll look at snarfing that.

>
> With it amended this works correctly, so:
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>


--
Alex Bennée