[PATCH 0/5] travis.yml improvements: Update libraries, build with arm64

[PATCH 0/5] travis.yml improvements: Update libraries, build with arm64

[Thomas Huth]

Update / add some libraries that we use in the Travis builds, and
enable compiling on the aarch64 host, too.

Thomas Huth (5):
  travis.yml: Add libvdeplug-dev to compile-test net/vde.c
  travis.yml: Use libsdl2 instead of libsdl1.2, and install
    libsdl2-image
  travis.yml: Use newer version of libgnutls and libpng
  travis.yml: Fix the ccache lines
  travis.yml: Compile on arm64, too

 .travis.yml | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

-- 
2.18.1

[PATCH 1/5] travis.yml: Add libvdeplug-dev to compile-test net/vde.c

[Thomas Huth]

This library is needed to compile the VDE network backend.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index d0b9e099b9..63328387f6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -46,6 +46,7 @@ addons:
       - libssh-dev
       - liburcu-dev
       - libusb-1.0-0-dev
+      - libvdeplug-dev
       - libvte-2.91-dev
       - sparse
       - uuid-dev
-- 
2.18.1

[PATCH 2/5] travis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image

[Thomas Huth]

We've removed support for SDL 1.2 quite a while ago already, so let's
use SDL 2 now in Travis to get test coverage for SDL again.
And while we're at it, also add libsdl2-image-dev which can be used
by QEMU nowadays, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 63328387f6..e21b9ad666 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -39,7 +39,8 @@ addons:
       - libpixman-1-dev
       - libpng12-dev
       - librados-dev
-      - libsdl1.2-dev
+      - libsdl2-dev
+      - libsdl2-image-dev
       - libseccomp-dev
       - libspice-protocol-dev
       - libspice-server-dev
@@ -308,7 +309,8 @@ matrix:
             - libpixman-1-dev
             - libpng12-dev
             - librados-dev
-            - libsdl1.2-dev
+            - libsdl2-dev
+            - libsdl2-image-dev
             - libseccomp-dev
             - libspice-protocol-dev
             - libspice-server-dev
-- 
2.18.1

[PATCH 3/5] travis.yml: Use newer version of libgnutls and libpng

[Thomas Huth]

libgnutls-dev and libpng12-dev are not available in newer versions
of Ubuntu anymore, so installing these packages fails e.g. in the
new arm64 containers on Travis. Let's use newer versions of these
packages by default instead. (The old versions still get tested in
the "gcc-9" build).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index e21b9ad666..616e59867a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -29,7 +29,7 @@ addons:
       - libcap-dev
       - libcap-ng-dev
       - libgcc-4.8-dev
-      - libgnutls-dev
+      - libgnutls28-dev
       - libgtk-3-dev
       - libiscsi-dev
       - liblttng-ust-dev
@@ -37,7 +37,7 @@ addons:
       - libnfs-dev
       - libnss3-dev
       - libpixman-1-dev
-      - libpng12-dev
+      - libpng-dev
       - librados-dev
       - libsdl2-dev
       - libsdl2-image-dev
-- 
2.18.1

[PATCH 4/5] travis.yml: Fix the ccache lines

[Thomas Huth]

The "command -v ccache && ccache ..." likely were supposed to test
the availability of ccache before running the program. But this
shell construct causes Travis to abort if ccache is not available.
Use an if-statement instead to fix this problem.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 616e59867a..0c88e8757b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -91,13 +91,13 @@ git:
 
 before_script:
   - if [ "$TRAVIS_OS_NAME" == "osx" ] ; then export PATH="/usr/local/opt/ccache/libexec:$PATH" ; fi
-  - command -v ccache && ccache --zero-stats
+  - if command -v ccache ; then ccache --zero-stats ; fi
   - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
   - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; }
 script:
   - make -j3 && travis_retry ${TEST_CMD}
 after_script:
-  - command -v ccache && ccache --show-stats
+  - if command -v ccache ; then ccache --show-stats ; fi
 
 
 matrix:
-- 
2.18.1

[PATCH 5/5] travis.yml: Compile on arm64, too

[Thomas Huth]

Travis now features an arm64 build host, so let's check compilation
there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have
  to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests
  can't be used and the hd-geo-test fails (thus we can't compile
  x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders,
  so we better limit the target list to some few architectures.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 0c88e8757b..357ca02890 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -342,3 +342,9 @@ matrix:
         - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
         - TEST_CMD="make -j3 check-tcg V=1"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
+        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
+      arch: arm64
-- 
2.18.1

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

[Philippe Mathieu-Daudé]

On 10/9/19 7:07 PM, Thomas Huth wrote:
> The "command -v ccache && ccache ..." likely were supposed to test
> the availability of ccache before running the program. But this
> shell construct causes Travis to abort if ccache is not available.

Oops.

Why can't you install ccache if these are Ubuntu systems?
It is even more wanted if the arm64 machine are slow...

> Use an if-statement instead to fix this problem.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   .travis.yml | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 616e59867a..0c88e8757b 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -91,13 +91,13 @@ git:
>   
>   before_script:
>     - if [ "$TRAVIS_OS_NAME" == "osx" ] ; then export PATH="/usr/local/opt/ccache/libexec:$PATH" ; fi
> -  - command -v ccache && ccache --zero-stats
> +  - if command -v ccache ; then ccache --zero-stats ; fi
>     - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
>     - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; }
>   script:
>     - make -j3 && travis_retry ${TEST_CMD}
>   after_script:
> -  - command -v ccache && ccache --show-stats
> +  - if command -v ccache ; then ccache --show-stats ; fi
>   
>   
>   matrix:
> 

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Philippe Mathieu-Daudé]

On 10/9/19 7:07 PM, Thomas Huth wrote:
> Travis now features an arm64 build host, so let's check compilation
> there, too.
> 
> Unfortunately, there are some quirks:
> - block/ssh.c does not compile properly in this environment, so we have
>    to use --disable-libssh until that problem is fixed.
> - test-util-filemonitor fails, so we can not run the unit tests there
> - The file system size seems to be very limited, so the iotest tests
>    can't be used and the hd-geo-test fails (thus we can't compile
>    x86_64-softmmu here and run "check-qtest" afterwards)
> - Compiling seems to be quite a bit slower than on the x86 builders,
>    so we better limit the target list to some few architectures.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   .travis.yml | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 0c88e8757b..357ca02890 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -342,3 +342,9 @@ matrix:
>           - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
>           - TEST_CMD="make -j3 check-tcg V=1"
>           - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
> +
> +    # Non-x86 builds:
> +    - env:
> +        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"

Is aarch64 on aarch64 interesting?

Do you know if we can use KVM there?

> +        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
> +      arch: arm64
> 

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Thomas Huth]

On 09/10/2019 21.06, Philippe Mathieu-Daudé wrote:
> On 10/9/19 7:07 PM, Thomas Huth wrote:
>> Travis now features an arm64 build host, so let's check compilation
>> there, too.
>>
>> Unfortunately, there are some quirks:
>> - block/ssh.c does not compile properly in this environment, so we have
>>    to use --disable-libssh until that problem is fixed.
>> - test-util-filemonitor fails, so we can not run the unit tests there
>> - The file system size seems to be very limited, so the iotest tests
>>    can't be used and the hd-geo-test fails (thus we can't compile
>>    x86_64-softmmu here and run "check-qtest" afterwards)
>> - Compiling seems to be quite a bit slower than on the x86 builders,
>>    so we better limit the target list to some few architectures.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>   .travis.yml | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 0c88e8757b..357ca02890 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -342,3 +342,9 @@ matrix:
>>           -
>> CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
>>
>>           - TEST_CMD="make -j3 check-tcg V=1"
>>           - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
>> +
>> +    # Non-x86 builds:
>> +    - env:
>> +        - CONFIG="--disable-libssh
>> --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
>>
> 
> Is aarch64 on aarch64 interesting?
> 
> Do you know if we can use KVM there?

I don't think that KVM is usable there, but at least this should give us
some compile coverage for target/arm/kvm.c which we don't get otherwise.

 Thomas

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

[Thomas Huth]

On 09/10/2019 21.04, Philippe Mathieu-Daudé wrote:
> On 10/9/19 7:07 PM, Thomas Huth wrote:
>> The "command -v ccache && ccache ..." likely were supposed to test
>> the availability of ccache before running the program. But this
>> shell construct causes Travis to abort if ccache is not available.
> 
> Oops.
> 
> Why can't you install ccache if these are Ubuntu systems?
> It is even more wanted if the arm64 machine are slow...

I just tried to add "ccache" to the list of packages that should be
installed, but I don't see a difference in the runtime.

First run with ccache enabled:

 https://travis-ci.com/huth/qemu/jobs/244117945

Second run where I'd expect a speedup:

 https://travis-ci.com/huth/qemu/jobs/244124599

The statistics at the end say that it had only 1 cache hit. Any ideas
what might be wrong here?

 Thomas

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 09/10/2019 21.06, Philippe Mathieu-Daudé wrote:
>> On 10/9/19 7:07 PM, Thomas Huth wrote:
>>> Travis now features an arm64 build host, so let's check compilation
>>> there, too.
>>>
>>> Unfortunately, there are some quirks:
>>> - block/ssh.c does not compile properly in this environment, so we have
>>>    to use --disable-libssh until that problem is fixed.
>>> - test-util-filemonitor fails, so we can not run the unit tests there
>>> - The file system size seems to be very limited, so the iotest tests
>>>    can't be used and the hd-geo-test fails (thus we can't compile
>>>    x86_64-softmmu here and run "check-qtest" afterwards)
>>> - Compiling seems to be quite a bit slower than on the x86 builders,
>>>    so we better limit the target list to some few architectures.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>   .travis.yml | 6 ++++++
>>>   1 file changed, 6 insertions(+)
>>>
>>> diff --git a/.travis.yml b/.travis.yml
>>> index 0c88e8757b..357ca02890 100644
>>> --- a/.travis.yml
>>> +++ b/.travis.yml
>>> @@ -342,3 +342,9 @@ matrix:
>>>           -
>>> CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
>>>
>>>           - TEST_CMD="make -j3 check-tcg V=1"
>>>           - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
>>> +
>>> +    # Non-x86 builds:
>>> +    - env:
>>> +        - CONFIG="--disable-libssh
>>> --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
>>>
>>
>> Is aarch64 on aarch64 interesting?
>>
>> Do you know if we can use KVM there?
>
> I don't think that KVM is usable there, but at least this should give us
> some compile coverage for target/arm/kvm.c which we don't get
> otherwise.

We get compile coverage for !x86 kvm from the cross compilers on
shippable. I'm more interested in running tests on !x86.

>
>  Thomas


--
Alex Bennée

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Daniel P. Berrangé]

On Wed, Oct 09, 2019 at 07:07:01PM +0200, Thomas Huth wrote:
> Travis now features an arm64 build host, so let's check compilation
> there, too.
> 
> Unfortunately, there are some quirks:
> - block/ssh.c does not compile properly in this environment, so we have
>   to use --disable-libssh until that problem is fixed.
> - test-util-filemonitor fails, so we can not run the unit tests there

Do you have any more info on this failure ?

Running tests with FILEMONITOR_DEBUG=1 env variable set will make
it print more info


Skipping unit tests seems to throw away the main benefit of using
the travis arm64 hosts, as it is the tests which add value over
our cross-compiled arm builds.

> - The file system size seems to be very limited, so the iotest tests
>   can't be used and the hd-geo-test fails (thus we can't compile
>   x86_64-softmmu here and run "check-qtest" afterwards)
> - Compiling seems to be quite a bit slower than on the x86 builders,
>   so we better limit the target list to some few architectures.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .travis.yml | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 0c88e8757b..357ca02890 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -342,3 +342,9 @@ matrix:
>          - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
>          - TEST_CMD="make -j3 check-tcg V=1"
>          - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
> +
> +    # Non-x86 builds:
> +    - env:
> +        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
> +        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
> +      arch: arm64
> -- 
> 2.18.1
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

[Philippe Mathieu-Daudé]

On 10/10/19 11:38 AM, Thomas Huth wrote:
> On 09/10/2019 21.04, Philippe Mathieu-Daudé wrote:
>> On 10/9/19 7:07 PM, Thomas Huth wrote:
>>> The "command -v ccache && ccache ..." likely were supposed to test
>>> the availability of ccache before running the program. But this
>>> shell construct causes Travis to abort if ccache is not available.
>>
>> Oops.
>>
>> Why can't you install ccache if these are Ubuntu systems?
>> It is even more wanted if the arm64 machine are slow...
> 
> I just tried to add "ccache" to the list of packages that should be
> installed, but I don't see a difference in the runtime.
> 
> First run with ccache enabled:
> 
>   https://travis-ci.com/huth/qemu/jobs/244117945
> 
> Second run where I'd expect a speedup:
> 
>   https://travis-ci.com/huth/qemu/jobs/244124599
> 
> The statistics at the end say that it had only 1 cache hit. Any ideas
> what might be wrong here?

Looking there and your following commit 
(https://github.com/huth/qemu/commit/eaf80e7851) I see you already 
figured this out :)

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

[Thomas Huth]

On 10/10/2019 13.16, Philippe Mathieu-Daudé wrote:
> On 10/10/19 11:38 AM, Thomas Huth wrote:
>> On 09/10/2019 21.04, Philippe Mathieu-Daudé wrote:
>>> On 10/9/19 7:07 PM, Thomas Huth wrote:
>>>> The "command -v ccache && ccache ..." likely were supposed to test
>>>> the availability of ccache before running the program. But this
>>>> shell construct causes Travis to abort if ccache is not available.
>>>
>>> Oops.
>>>
>>> Why can't you install ccache if these are Ubuntu systems?
>>> It is even more wanted if the arm64 machine are slow...
>>
>> I just tried to add "ccache" to the list of packages that should be
>> installed, but I don't see a difference in the runtime.
>>
>> First run with ccache enabled:
>>
>>   https://travis-ci.com/huth/qemu/jobs/244117945
>>
>> Second run where I'd expect a speedup:
>>
>>   https://travis-ci.com/huth/qemu/jobs/244124599
>>
>> The statistics at the end say that it had only 1 cache hit. Any ideas
>> what might be wrong here?
> 
> Looking there and your following commit
> (https://github.com/huth/qemu/commit/eaf80e7851) I see you already
> figured this out :)

No, that was just a try, but it did not change anything:

 https://travis-ci.com/huth/qemu/jobs/244137697#L5813

But I also noticed that in the arm64 builds, the cache information is
missing:

 https://travis-ci.com/huth/qemu/jobs/244137697#L1844

... so I assume that ccache support needs to be supported in the image
that is provided by Travis, and you can not simply install it
afterwards. So this is likely just a quirk that hopefully will be fixed
by Travis later (arm64 is still marked as "alpha" there if I've got that
right).

 Thomas

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

[Philippe Mathieu-Daudé]

On 10/10/19 1:29 PM, Thomas Huth wrote:
> On 10/10/2019 13.16, Philippe Mathieu-Daudé wrote:
>> On 10/10/19 11:38 AM, Thomas Huth wrote:
>>> On 09/10/2019 21.04, Philippe Mathieu-Daudé wrote:
>>>> On 10/9/19 7:07 PM, Thomas Huth wrote:
>>>>> The "command -v ccache && ccache ..." likely were supposed to test
>>>>> the availability of ccache before running the program. But this
>>>>> shell construct causes Travis to abort if ccache is not available.
>>>>
>>>> Oops.
>>>>
>>>> Why can't you install ccache if these are Ubuntu systems?
>>>> It is even more wanted if the arm64 machine are slow...
>>>
>>> I just tried to add "ccache" to the list of packages that should be
>>> installed, but I don't see a difference in the runtime.
>>>
>>> First run with ccache enabled:
>>>
>>>    https://travis-ci.com/huth/qemu/jobs/244117945
>>>
>>> Second run where I'd expect a speedup:
>>>
>>>    https://travis-ci.com/huth/qemu/jobs/244124599
>>>
>>> The statistics at the end say that it had only 1 cache hit. Any ideas
>>> what might be wrong here?
>>
>> Looking there and your following commit
>> (https://github.com/huth/qemu/commit/eaf80e7851) I see you already
>> figured this out :)
> 
> No, that was just a try, but it did not change anything:
> 
>   https://travis-ci.com/huth/qemu/jobs/244137697#L5813

Oh I checked the x86 build then...

https://travis-ci.com/huth/qemu/jobs/244137696#L11299

> But I also noticed that in the arm64 builds, the cache information is
> missing:
> 
>   https://travis-ci.com/huth/qemu/jobs/244137697#L1844
> 
> ... so I assume that ccache support needs to be supported in the image
> that is provided by Travis, and you can not simply install it
> afterwards. So this is likely just a quirk that hopefully will be fixed
> by Travis later (arm64 is still marked as "alpha" there if I've got that
> right).

Ah, you might be correct.

If so, your job is now ready to use the feature once they enable it :)

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Thomas Huth]

On 10/10/2019 12.58, Daniel P. Berrangé wrote:
> On Wed, Oct 09, 2019 at 07:07:01PM +0200, Thomas Huth wrote:
>> Travis now features an arm64 build host, so let's check compilation
>> there, too.
>>
>> Unfortunately, there are some quirks:
>> - block/ssh.c does not compile properly in this environment, so we have
>>   to use --disable-libssh until that problem is fixed.
>> - test-util-filemonitor fails, so we can not run the unit tests there
> 
> Do you have any more info on this failure ?
> 
> Running tests with FILEMONITOR_DEBUG=1 env variable set will make
> it print more info

Here you go:

https://travis-ci.com/huth/qemu/jobs/244168910#L5775

Quoting:

"
Add watch /tmp/test-util-filemonitor-PH0B9Z (null)
Watch ID 100000000
Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt
Watch ID 100000001
Add watch /tmp/test-util-filemonitor-PH0B9Z two.txt
Watch ID 100000002
Create /tmp/test-util-filemonitor-PH0B9Z/one.txt
Event id=100000000 event=0 file=one.txt
Queue event id 100000000 event 0 file one.txt
Queue event id 100000001 event 0 file one.txt
Event id=100000001 event=0 file=one.txt
Create /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 0 file two.txt
Queue event id 100000002 event 0 file two.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Create /tmp/test-util-filemonitor-PH0B9Z/three.txt
Queue event id 100000000 event 0 file three.txt
Event id=100000000 event=0 file=three.txt
Unlink /tmp/test-util-filemonitor-PH0B9Z/three.txt
Queue event id 100000000 event 2 file three.txt
Event id=100000000 event=2 file=three.txt
Rename /tmp/test-util-filemonitor-PH0B9Z/one.txt ->
/tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 2 file one.txt
Queue event id 100000001 event 2 file one.txt
Queue event id 100000000 event 0 file two.txt
Queue event id 100000002 event 0 file two.txt
Event id=100000000 event=2 file=one.txt
Event id=100000001 event=2 file=one.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Append /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 1 file two.txt
Queue event id 100000002 event 1 file two.txt
Event id=100000000 event=1 file=two.txt
Event id=100000002 event=1 file=two.txt
Touch /tmp/test-util-filemonitor-PH0B9Z/two.txt
Event id=100000000 event=3 file=two.txt
Queue event id 100000000 event 3 file two.txt
Queue event id 100000002 event 3 file two.txt
Event id=100000002 event=3 file=two.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 100000001
Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt
Watch ID 100000003
Create /tmp/test-util-filemonitor-PH0B9Z/one.txt
Queue event id 100000000 event 0 file one.txt
Event id=100000000 event=0 file=one.txt
Queue event id 100000003 event 0 file one.txt
Event id=100000003 event=0 file=one.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 100000003
Unlink /tmp/test-util-filemonitor-PH0B9Z/one.txt
Queue event id 100000000 event 2 file one.txt
Event id=100000000 event=2 file=one.txt
Mkdir /tmp/test-util-filemonitor-PH0B9Z/fish
Queue event id 100000000 event 0 file fish
Event id=100000000 event=0 file=fish
Add watch /tmp/test-util-filemonitor-PH0B9Z fish/
Watch ID 200000000
Add watch /tmp/test-util-filemonitor-PH0B9Z fish/one.txt
Watch ID 200000001
Create /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt
Queue event id 200000000 event 0 file one.txt
Queue event id 200000001 event 0 file one.txt
Event id=200000000 event=0 file=one.txt
Event id=200000001 event=0 file=one.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 200000001
Rename /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt ->
/tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 200000000 event 2 file one.txt
Event id=200000000 event=2 file=one.txt
Queue event id 100000000 event 0 file two.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Queue event id 100000002 event 0 file two.txt
Rmdir /tmp/test-util-filemonitor-PH0B9Z/fish
Queue event id 100000000 event 2 file fish
Event id=200000000 event=4 file=
Missing event watch id 200000000 event 4 file
**
ERROR:tests/test-util-filemonitor.c:690:test_file_monitor_events:
assertion failed: (err == 0)
ERROR - Bail out!
ERROR:tests/test-util-filemonitor.c:690:test_file_monitor_events:
assertion failed: (err == 0)
Aborted
/home/travis/build/huth/qemu/tests/Makefile.include:904: recipe for
target 'check-unit' failed
make: *** [check-unit] Error 1
"

 Thomas

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Daniel P. Berrangé]

On Thu, Oct 10, 2019 at 01:57:11PM +0200, Thomas Huth wrote:
> On 10/10/2019 12.58, Daniel P. Berrangé wrote:
> > On Wed, Oct 09, 2019 at 07:07:01PM +0200, Thomas Huth wrote:
> >> Travis now features an arm64 build host, so let's check compilation
> >> there, too.
> >>
> >> Unfortunately, there are some quirks:
> >> - block/ssh.c does not compile properly in this environment, so we have
> >>   to use --disable-libssh until that problem is fixed.
> >> - test-util-filemonitor fails, so we can not run the unit tests there
> > 
> > Do you have any more info on this failure ?
> > 
> > Running tests with FILEMONITOR_DEBUG=1 env variable set will make
> > it print more info
> 
> Here you go:
> 
> https://travis-ci.com/huth/qemu/jobs/244168910#L5775
> 
> Quoting:
> 
> "
> Add watch /tmp/test-util-filemonitor-PH0B9Z (null)
> Watch ID 100000000
> Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt
> Watch ID 100000001
> Add watch /tmp/test-util-filemonitor-PH0B9Z two.txt
> Watch ID 100000002
> Create /tmp/test-util-filemonitor-PH0B9Z/one.txt
> Event id=100000000 event=0 file=one.txt
> Queue event id 100000000 event 0 file one.txt
> Queue event id 100000001 event 0 file one.txt
> Event id=100000001 event=0 file=one.txt
> Create /tmp/test-util-filemonitor-PH0B9Z/two.txt
> Queue event id 100000000 event 0 file two.txt
> Queue event id 100000002 event 0 file two.txt
> Event id=100000000 event=0 file=two.txt
> Event id=100000002 event=0 file=two.txt
> Create /tmp/test-util-filemonitor-PH0B9Z/three.txt
> Queue event id 100000000 event 0 file three.txt
> Event id=100000000 event=0 file=three.txt
> Unlink /tmp/test-util-filemonitor-PH0B9Z/three.txt
> Queue event id 100000000 event 2 file three.txt
> Event id=100000000 event=2 file=three.txt
> Rename /tmp/test-util-filemonitor-PH0B9Z/one.txt ->
> /tmp/test-util-filemonitor-PH0B9Z/two.txt
> Queue event id 100000000 event 2 file one.txt
> Queue event id 100000001 event 2 file one.txt
> Queue event id 100000000 event 0 file two.txt
> Queue event id 100000002 event 0 file two.txt
> Event id=100000000 event=2 file=one.txt
> Event id=100000001 event=2 file=one.txt
> Event id=100000000 event=0 file=two.txt
> Event id=100000002 event=0 file=two.txt
> Append /tmp/test-util-filemonitor-PH0B9Z/two.txt
> Queue event id 100000000 event 1 file two.txt
> Queue event id 100000002 event 1 file two.txt
> Event id=100000000 event=1 file=two.txt
> Event id=100000002 event=1 file=two.txt
> Touch /tmp/test-util-filemonitor-PH0B9Z/two.txt
> Event id=100000000 event=3 file=two.txt
> Queue event id 100000000 event 3 file two.txt
> Queue event id 100000002 event 3 file two.txt
> Event id=100000002 event=3 file=two.txt
> Del watch /tmp/test-util-filemonitor-PH0B9Z 100000001
> Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt
> Watch ID 100000003
> Create /tmp/test-util-filemonitor-PH0B9Z/one.txt
> Queue event id 100000000 event 0 file one.txt
> Event id=100000000 event=0 file=one.txt
> Queue event id 100000003 event 0 file one.txt
> Event id=100000003 event=0 file=one.txt
> Del watch /tmp/test-util-filemonitor-PH0B9Z 100000003
> Unlink /tmp/test-util-filemonitor-PH0B9Z/one.txt
> Queue event id 100000000 event 2 file one.txt
> Event id=100000000 event=2 file=one.txt
> Mkdir /tmp/test-util-filemonitor-PH0B9Z/fish
> Queue event id 100000000 event 0 file fish
> Event id=100000000 event=0 file=fish
> Add watch /tmp/test-util-filemonitor-PH0B9Z fish/
> Watch ID 200000000
> Add watch /tmp/test-util-filemonitor-PH0B9Z fish/one.txt
> Watch ID 200000001
> Create /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt
> Queue event id 200000000 event 0 file one.txt
> Queue event id 200000001 event 0 file one.txt
> Event id=200000000 event=0 file=one.txt
> Event id=200000001 event=0 file=one.txt
> Del watch /tmp/test-util-filemonitor-PH0B9Z 200000001
> Rename /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt ->
> /tmp/test-util-filemonitor-PH0B9Z/two.txt
> Queue event id 200000000 event 2 file one.txt
> Event id=200000000 event=2 file=one.txt
> Queue event id 100000000 event 0 file two.txt
> Event id=100000000 event=0 file=two.txt
> Event id=100000002 event=0 file=two.txt
> Queue event id 100000002 event 0 file two.txt
> Rmdir /tmp/test-util-filemonitor-PH0B9Z/fish
> Queue event id 100000000 event 2 file fish
> Event id=200000000 event=4 file=
> Missing event watch id 200000000 event 4 file
> **

This is really strange behaviour.  I've done some further test builds
on travis with various extra debug info & AFAICT, inotify is not
working correctly on the host - it is never emitting one of the events
that it should. Could be some wierd bug related to the filesystem
overlay setup used for the container env in travis I guess.

I'm not sure what todo here. I can't see an obvious trait that I can
use to identify a broken setup, in order to auto-skip the test. Might
just need to build without the inotify support somehow.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 0/5] travis.yml improvements: Update libraries, build with arm64

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> Update / add some libraries that we use in the Travis builds, and
> enable compiling on the aarch64 host, too.

Queued 1-3 to testing/next and I'll have a play with 4/5 but probably
won't include them in the final PR unless they seem solid.

>
> Thomas Huth (5):
>   travis.yml: Add libvdeplug-dev to compile-test net/vde.c
>   travis.yml: Use libsdl2 instead of libsdl1.2, and install
>     libsdl2-image
>   travis.yml: Use newer version of libgnutls and libpng
>   travis.yml: Fix the ccache lines
>   travis.yml: Compile on arm64, too
>
>  .travis.yml | 21 +++++++++++++++------
>  1 file changed, 15 insertions(+), 6 deletions(-)


--
Alex Bennée

Re: [PATCH 0/5] travis.yml improvements: Update libraries, build with arm64

[Thomas Huth]

On 16/10/2019 19.56, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> Update / add some libraries that we use in the Travis builds, and
>> enable compiling on the aarch64 host, too.
> 
> Queued 1-3 to testing/next and I'll have a play with 4/5 but probably
> won't include them in the final PR unless they seem solid.

Thanks! I think you could also pick patch 4 since the current line with
"command -v ccache && ccache ..." is definitely nonsense (it currently
just works since we always compile with ccache). We should either apply
my patch 4, or if we say that we can not run without ccache, we should
remove the "command -v ccache &&" part instead.

 Thomas

Re: [PATCH 0/5] travis.yml improvements: Update libraries, build with arm64

[Philippe Mathieu-Daudé]

On 10/21/19 3:53 PM, Thomas Huth wrote:
> On 16/10/2019 19.56, Alex Bennée wrote:
>>
>> Thomas Huth <thuth@redhat.com> writes:
>>
>>> Update / add some libraries that we use in the Travis builds, and
>>> enable compiling on the aarch64 host, too.
>>
>> Queued 1-3 to testing/next and I'll have a play with 4/5 but probably
>> won't include them in the final PR unless they seem solid.
> 
> Thanks! I think you could also pick patch 4 since the current line with
> "command -v ccache && ccache ..." is definitely nonsense (it currently
> just works since we always compile with ccache). We should either apply
> my patch 4, or if we say that we can not run without ccache, we should
> remove the "command -v ccache &&" part instead.

Yes, pick also #4 please :)

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> Travis now features an arm64 build host, so let's check compilation
> there, too.
>
> Unfortunately, there are some quirks:
> - block/ssh.c does not compile properly in this environment, so we have
>   to use --disable-libssh until that problem is fixed.
> - test-util-filemonitor fails, so we can not run the unit tests there
> - The file system size seems to be very limited, so the iotest tests
>   can't be used and the hd-geo-test fails (thus we can't compile
>   x86_64-softmmu here and run "check-qtest" afterwards)
> - Compiling seems to be quite a bit slower than on the x86 builders,
>   so we better limit the target list to some few architectures.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .travis.yml | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 0c88e8757b..357ca02890 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -342,3 +342,9 @@ matrix:
>          - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
>          - TEST_CMD="make -j3 check-tcg V=1"
>          - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
> +
> +    # Non-x86 builds:
> +    - env:
> +        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
> +        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
> +      arch: arm64


Hmm this keeps fail for me now but it looks like missing deps:

Reading state information...

E: Unable to locate package libspice-server-dev
apt-get.diagnostics

apt-get install failed

The command "sudo -E apt-get -yq --no-install-suggests --no-install-recommends $(travis_apt_get_options) install libaio-dev libattr1-dev libbrlapi-dev libcap-dev libcap-ng-dev libgcc-4.8-dev libgnutls28-dev libgtk-3-dev libiscsi-dev liblttng-ust-dev libncurses5-dev libnfs-dev libnss3-dev libpixman-1-dev libpng-dev librados-dev libsdl2-dev libsdl2-image-dev libseccomp-dev libspice-protocol-dev libspice-server-dev libssh-dev liburcu-dev libusb-1.0-0-dev libvdeplug-dev libvte-2.91-dev sparse uuid-dev gcovr" failed and exited with 100 during .

--
Alex Bennée

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

[Thomas Huth]

On 22/10/2019 12.41, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> Travis now features an arm64 build host, so let's check compilation
>> there, too.
>>
>> Unfortunately, there are some quirks:
>> - block/ssh.c does not compile properly in this environment, so we have
>>   to use --disable-libssh until that problem is fixed.
>> - test-util-filemonitor fails, so we can not run the unit tests there
>> - The file system size seems to be very limited, so the iotest tests
>>   can't be used and the hd-geo-test fails (thus we can't compile
>>   x86_64-softmmu here and run "check-qtest" afterwards)
>> - Compiling seems to be quite a bit slower than on the x86 builders,
>>   so we better limit the target list to some few architectures.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  .travis.yml | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 0c88e8757b..357ca02890 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -342,3 +342,9 @@ matrix:
>>          - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
>>          - TEST_CMD="make -j3 check-tcg V=1"
>>          - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
>> +
>> +    # Non-x86 builds:
>> +    - env:
>> +        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
>> +        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
>> +      arch: arm64
> 
> 
> Hmm this keeps fail for me now but it looks like missing deps:
> 
> Reading state information...
> 
> E: Unable to locate package libspice-server-dev

It used to work two weeks before:

 https://travis-ci.com/huth/qemu/jobs/244124599#L484

I guess there was an update to a different version and they removed the
package from arm64 now...?

Seems like this arm64 builder is still a moving target (it has been
announced as alpha software IIRC), so I think we should not include this
patch yet.

 Thomas