[PATCH v2 0/2] Run tcg tests with tci on Travis

[PATCH v2 0/2] Run tcg tests with tci on Travis

[Thomas Huth]

It's now possible to run some TCG-based tests with our Tiny Code
Generator Interpreter (TCI), too. These two patches enable the
testing on Travis.

Alex Bennée (1):
  configure: allow disable of cross compilation containers

Thomas Huth (1):
  travis.yml: Run tcg tests with tci

 .travis.yml            | 7 ++++---
 configure              | 8 +++++++-
 tests/tcg/configure.sh | 6 ++++--
 3 files changed, 15 insertions(+), 6 deletions(-)

-- 
2.18.1

[PATCH v2 1/2] configure: allow disable of cross compilation containers

[Thomas Huth]

From: Alex Bennée <alex.bennee@linaro.org>

Our docker infrastructure isn't quite as multiarch as we would wish so
lets allow the user to disable it if they want. This will allow us to
use still run check-tcg on non-x86 CI setups.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure              | 8 +++++++-
 tests/tcg/configure.sh | 6 ++++--
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 6099be1d84..fe6d0971f1 100755
--- a/configure
+++ b/configure
@@ -302,6 +302,7 @@ audio_win_int=""
 libs_qga=""
 debug_info="yes"
 stack_protector=""
+use_containers="yes"
 
 if test -e "$source_path/.git"
 then
@@ -1539,6 +1540,10 @@ for opt do
   ;;
   --disable-plugins) plugins="no"
   ;;
+  --enable-containers) use_containers="yes"
+  ;;
+  --disable-containers) use_containers="no"
+  ;;
   *)
       echo "ERROR: unknown option $opt"
       echo "Try '$0 --help' for more information"
@@ -1722,6 +1727,7 @@ Advanced options (experts only):
                            track the maximum stack usage of stacks created by qemu_alloc_stack
   --enable-plugins
                            enable plugins via shared library loading
+  --disable-containers     don't use containers for cross-building
 
 Optional features, enabled with --enable-FEATURE and
 disabled with --disable-FEATURE, default is enabled if available:
@@ -8039,7 +8045,7 @@ done
 (for i in $cross_cc_vars; do
   export $i
 done
-export target_list source_path
+export target_list source_path use_containers
 $source_path/tests/tcg/configure.sh)
 
 # temporary config to build submodules
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 6c4a471aea..210e68396f 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -36,8 +36,10 @@ TMPC="${TMPDIR1}/qemu-conf.c"
 TMPE="${TMPDIR1}/qemu-conf.exe"
 
 container="no"
-if has "docker" || has "podman"; then
-  container=$($python $source_path/tests/docker/docker.py probe)
+if test $use_containers = "yes"; then
+    if has "docker" || has "podman"; then
+        container=$($python $source_path/tests/docker/docker.py probe)
+    fi
 fi
 
 # cross compilers defaults, can be overridden with --cross-cc-ARCH
-- 
2.18.1

[PATCH v2 2/2] travis.yml: Run tcg tests with tci

[Thomas Huth]

So far we only have compile coverage for tci. But since commit
2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
for INDEX_op_ld16u_i64") has been included now, we can also run the
"tcg" and "qtest" tests with tci, so let's enable them in Travis now.
Since we don't gain much additional test coverage by compiling all
targets, and TCI is broken e.g. with the Sparc targets, we also limit
the target list to a reasonable subset now (which should still get
us test coverage by tests/boot-serial-test for example).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index c09b6a0014..de7559e777 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end
-- 
2.18.1

Re: [PATCH v2 0/2] Run tcg tests with tci on Travis

[Philippe Mathieu-Daudé]

On 11/28/19 4:35 PM, Thomas Huth wrote:
> It's now possible to run some TCG-based tests with our Tiny Code
> Generator Interpreter (TCI), too. These two patches enable the
> testing on Travis.
> 
> Alex Bennée (1):
>    configure: allow disable of cross compilation containers
> 
> Thomas Huth (1):
>    travis.yml: Run tcg tests with tci
> 
>   .travis.yml            | 7 ++++---
>   configure              | 8 +++++++-
>   tests/tcg/configure.sh | 6 ++++--
>   3 files changed, 15 insertions(+), 6 deletions(-)

Good idea to add/use '--disable-containers'.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH v2 1/2] configure: allow disable of cross compilation containers

[Stefan Weil]

Am 28.11.19 um 16:35 schrieb Thomas Huth:

> From: Alex Bennée <alex.bennee@linaro.org>
>
> Our docker infrastructure isn't quite as multiarch as we would wish so
> lets allow the user to disable it if they want. This will allow us to


s/lets/let's/ ?

Otherwise fine, thank you.

Reviewed-by: Stefan Weil <sw@weilnetz.de>

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

[Stefan Weil]

Am 28.11.19 um 16:35 schrieb Thomas Huth:

> So far we only have compile coverage for tci. But since commit
> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
> for INDEX_op_ld16u_i64") has been included now, we can also run the
> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
> Since we don't gain much additional test coverage by compiling all
> targets, and TCI is broken e.g. with the Sparc targets, we also limit


As far as I know it is broken with Sparc hosts (not Sparc targets).

I tested without limiting the target list on an x86_64 host, and the
tests passed.


> the target list to a reasonable subset now (which should still get
> us test coverage by tests/boot-serial-test for example).
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .travis.yml | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c09b6a0014..de7559e777 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -215,10 +215,11 @@ matrix:
>          - TEST_CMD=""
>  
>  
> -    # We manually include builds which we disable "make check" for
> +    # Check the TCG interpreter (TCI)
>      - env:
> -        - CONFIG="--enable-debug --enable-tcg-interpreter"
> -        - TEST_CMD=""
> +        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers


You could also --disable-kvm. It should not be needed, and disabling it
might need less build resources.


> +            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
> +        - TEST_CMD="make check-qtest check-tcg V=1"
>  
>  
>      # We don't need to exercise every backend with every front-end


Thank you for adding these tests.

Tested-by: Stefan Weil <sw@weilnetz.de>

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

[Stefan Weil]

Am 28.11.19 um 22:06 schrieb Stefan Weil:

> Am 28.11.19 um 16:35 schrieb Thomas Huth:
>
>> So far we only have compile coverage for tci. But since commit
>> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
>> for INDEX_op_ld16u_i64") has been included now, we can also run the
>> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
>> Since we don't gain much additional test coverage by compiling all
>> targets, and TCI is broken e.g. with the Sparc targets, we also limit
>
> As far as I know it is broken with Sparc hosts (not Sparc targets).
>
> I tested without limiting the target list on an x86_64 host, and the
> tests passed.


Sorry, I have to correct myself: check-qtest-sparc64 fails. I'll examine
that.

Stefan

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

[Thomas Huth]

On 28/11/2019 22.06, Stefan Weil wrote:
> Am 28.11.19 um 16:35 schrieb Thomas Huth:
> 
>> So far we only have compile coverage for tci. But since commit
>> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
>> for INDEX_op_ld16u_i64") has been included now, we can also run the
>> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
>> Since we don't gain much additional test coverage by compiling all
>> targets, and TCI is broken e.g. with the Sparc targets, we also limit
> 
> 
> As far as I know it is broken with Sparc hosts (not Sparc targets).

It was definitely hanging with sparc64-softmmu:

 https://travis-ci.com/huth/qemu/jobs/261335163

... but since you've mentioned that it should work with the 32-bit
sparc-softmmu, I'll give it another try with sparc-softmmu.

>> the target list to a reasonable subset now (which should still get
>> us test coverage by tests/boot-serial-test for example).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  .travis.yml | 7 ++++---
>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index c09b6a0014..de7559e777 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -215,10 +215,11 @@ matrix:
>>          - TEST_CMD=""
>>  
>>  
>> -    # We manually include builds which we disable "make check" for
>> +    # Check the TCG interpreter (TCI)
>>      - env:
>> -        - CONFIG="--enable-debug --enable-tcg-interpreter"
>> -        - TEST_CMD=""
>> +        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers
> 
> 
> You could also --disable-kvm. It should not be needed, and disabling it
> might need less build resources.

Good idea. KVM is not usable by default in Travis, so we should not
accidentally use it for the tests that specify "accel=kvm:tcg", but in
case that changes with a future version of the environment, we should
maybe be prepared.

>> +            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
>> +        - TEST_CMD="make check-qtest check-tcg V=1"
>>  
>>  
>>      # We don't need to exercise every backend with every front-end
> 
> 
> Thank you for adding these tests.
> 
> Tested-by: Stefan Weil <sw@weilnetz.de>

 Thanks,
  Thomas

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 28/11/2019 22.06, Stefan Weil wrote:
>> Am 28.11.19 um 16:35 schrieb Thomas Huth:
>> 
>>> So far we only have compile coverage for tci. But since commit
>>> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
>>> for INDEX_op_ld16u_i64") has been included now, we can also run the
>>> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
>>> Since we don't gain much additional test coverage by compiling all
>>> targets, and TCI is broken e.g. with the Sparc targets, we also limit
>> 
>> 
>> As far as I know it is broken with Sparc hosts (not Sparc targets).
>
> It was definitely hanging with sparc64-softmmu:
>
>  https://travis-ci.com/huth/qemu/jobs/261335163
>
> ... but since you've mentioned that it should work with the 32-bit
> sparc-softmmu, I'll give it another try with sparc-softmmu.
>
>>> the target list to a reasonable subset now (which should still get
>>> us test coverage by tests/boot-serial-test for example).
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>  .travis.yml | 7 ++++---
>>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/.travis.yml b/.travis.yml
>>> index c09b6a0014..de7559e777 100644
>>> --- a/.travis.yml
>>> +++ b/.travis.yml
>>> @@ -215,10 +215,11 @@ matrix:
>>>          - TEST_CMD=""
>>>  
>>>  
>>> -    # We manually include builds which we disable "make check" for
>>> +    # Check the TCG interpreter (TCI)
>>>      - env:
>>> -        - CONFIG="--enable-debug --enable-tcg-interpreter"
>>> -        - TEST_CMD=""
>>> +        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers
>> 
>> 
>> You could also --disable-kvm. It should not be needed, and disabling it
>> might need less build resources.
>
> Good idea. KVM is not usable by default in Travis, so we should not
> accidentally use it for the tests that specify "accel=kvm:tcg", but in
> case that changes with a future version of the environment, we should
> maybe be prepared.

Makes sense, I'll wait for v3 before applying to me tree.

>
>>> +            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
>>> +        - TEST_CMD="make check-qtest check-tcg V=1"
>>>  
>>>  
>>>      # We don't need to exercise every backend with every front-end
>> 
>> 
>> Thank you for adding these tests.
>> 
>> Tested-by: Stefan Weil <sw@weilnetz.de>
>
>  Thanks,
>   Thomas


-- 
Alex Bennée

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

[Thomas Huth]

On 28/11/2019 22.33, Stefan Weil wrote:
> Am 28.11.19 um 22:06 schrieb Stefan Weil:
> 
>> Am 28.11.19 um 16:35 schrieb Thomas Huth:
>>
>>> So far we only have compile coverage for tci. But since commit
>>> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
>>> for INDEX_op_ld16u_i64") has been included now, we can also run the
>>> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
>>> Since we don't gain much additional test coverage by compiling all
>>> targets, and TCI is broken e.g. with the Sparc targets, we also limit
>>
>> As far as I know it is broken with Sparc hosts (not Sparc targets).
>>
>> I tested without limiting the target list on an x86_64 host, and the
>> tests passed.
> 
> 
> Sorry, I have to correct myself: check-qtest-sparc64 fails. I'll examine
> that.

The 32-bit sparc target fails, too - when doing the prom-env-test:

https://travis-ci.com/huth/qemu/jobs/262572937#L7690

 Thomas

[PATCH v3] travis.yml: Run tcg tests with tci

[Thomas Huth]

So far we only have compile coverage for tci. But since commit
2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
for INDEX_op_ld16u_i64") has been included now, we can also run the
"tcg" and "qtest" tests with tci, so let's enable them in Travis now.
Since we don't gain much additional test coverage by compiling all
targets, and TCI is broken e.g. with the Sparc targets, we also limit
the target list to a reasonable subset now (which should still get us
test coverage by tests/boot-serial-test for example).

Tested-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 v3:
 - Add --disable-kvm option since we're only interested in TCG here

 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 445b0646c1..d73e2fb744 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm --disable-containers
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end
-- 
2.18.1

Re: [PATCH v3] travis.yml: Run tcg tests with tci

[Richard Henderson]

On 12/4/19 12:31 AM, Thomas Huth wrote:
> -    # We manually include builds which we disable "make check" for
> +    # Check the TCG interpreter (TCI)
>      - env:
> -        - CONFIG="--enable-debug --enable-tcg-interpreter"
> -        - TEST_CMD=""
> +        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm

While we're changing things, the interpreter will go much faster with
optimization enabled.  We can change this to --enable-debug-tcg, which leaves
the asserts enabled, but compiles with -O2.


r~

Re: [PATCH v3] travis.yml: Run tcg tests with tci

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> So far we only have compile coverage for tci. But since commit
> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
> for INDEX_op_ld16u_i64") has been included now, we can also run the
> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
> Since we don't gain much additional test coverage by compiling all
> targets, and TCI is broken e.g. with the Sparc targets, we also limit
> the target list to a reasonable subset now (which should still get us
> test coverage by tests/boot-serial-test for example).

Queued to testing/next, thanks.
>
> Tested-by: Stefan Weil <sw@weilnetz.de>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  v3:
>  - Add --disable-kvm option since we're only interested in TCG here
>
>  .travis.yml | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 445b0646c1..d73e2fb744 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -215,10 +215,11 @@ matrix:
>          - TEST_CMD=""
>  
>  
> -    # We manually include builds which we disable "make check" for
> +    # Check the TCG interpreter (TCI)
>      - env:
> -        - CONFIG="--enable-debug --enable-tcg-interpreter"
> -        - TEST_CMD=""
> +        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm --disable-containers
> +            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
> +        - TEST_CMD="make check-qtest check-tcg V=1"
>  
>  
>      # We don't need to exercise every backend with every front-end


-- 
Alex Bennée

Re: [PATCH v3] travis.yml: Run tcg tests with tci

[Thomas Huth]

On 04/12/2019 14.48, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> So far we only have compile coverage for tci. But since commit
>> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
>> for INDEX_op_ld16u_i64") has been included now, we can also run the
>> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
>> Since we don't gain much additional test coverage by compiling all
>> targets, and TCI is broken e.g. with the Sparc targets, we also limit
>> the target list to a reasonable subset now (which should still get us
>> test coverage by tests/boot-serial-test for example).
> 
> Queued to testing/next, thanks.

Thanks! But could you maybe s/--enable-debug/--enable-debug-tcg/ as
Richard just suggested in his mail? Or want me to send a v4?

 Thomas

Re: [PATCH v3] travis.yml: Run tcg tests with tci

[Alex Bennée]

Richard Henderson <richard.henderson@linaro.org> writes:

> On 12/4/19 12:31 AM, Thomas Huth wrote:
>> -    # We manually include builds which we disable "make check" for
>> +    # Check the TCG interpreter (TCI)
>>      - env:
>> -        - CONFIG="--enable-debug --enable-tcg-interpreter"
>> -        - TEST_CMD=""
>> +        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm
>
> While we're changing things, the interpreter will go much faster with
> optimization enabled.  We can change this to --enable-debug-tcg, which leaves
> the asserts enabled, but compiles with -O2.

I've amended the commit in my tree - no need to resend.
>
>
> r~


-- 
Alex Bennée

Re: [PATCH v3] travis.yml: Run tcg tests with tci

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 04/12/2019 14.48, Alex Bennée wrote:
>> 
>> Thomas Huth <thuth@redhat.com> writes:
>> 
>>> So far we only have compile coverage for tci. But since commit
>>> 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation
>>> for INDEX_op_ld16u_i64") has been included now, we can also run the
>>> "tcg" and "qtest" tests with tci, so let's enable them in Travis now.
>>> Since we don't gain much additional test coverage by compiling all
>>> targets, and TCI is broken e.g. with the Sparc targets, we also limit
>>> the target list to a reasonable subset now (which should still get us
>>> test coverage by tests/boot-serial-test for example).
>> 
>> Queued to testing/next, thanks.
>
> Thanks! But could you maybe s/--enable-debug/--enable-debug-tcg/ as
> Richard just suggested in his mail? Or want me to send a v4?

I fixed it in my tree, no need to send v4

>
>  Thomas


-- 
Alex Bennée