[PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Philippe Mathieu-Daudé]

Travis-CI seems to have enforced memory limit on containers,
and the 'GCC check-tcg' job started to fail on AArch64 [*]:

  [2041/3679] Compiling C++ object libcommon.fa.p/disas_nanomips.cpp.o
  FAILED: libcommon.fa.p/disas_nanomips.cpp.o
  {standard input}: Assembler messages:
  {standard input}:577781: Warning: end of file not at end of a line; newline inserted
  {standard input}:577882: Error: unknown pseudo-op: `.lvl35769'
  {standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
  c++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.

Until we have a replacement for this job on Gitlab-CI, disable
compilation of C++ files by forcing the c++ compiler to /bin/false
so Meson build system can not detect it:

  $ ../configure --cxx=/bin/false

  Compilation
                       C compiler: cc
                  Host C compiler: cc
                     C++ compiler: NO

[*] https://travis-ci.org/github/qemu/qemu/jobs/757819402#L3754

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
v3: Aarch -> AArch
v2: Link to first line with error, describe Meson

Supersedes: <20210206200537.2249362-1-f4bug@amsat.org>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 5f1dea873ec..b4b2d66fa4b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -261,7 +261,7 @@ jobs:
           - genisoimage
       env:
         - TEST_CMD="make check check-tcg V=1"
-        - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS}"
+        - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS} --cxx=/bin/false"
         - UNRELIABLE=true
 
     - name: "[ppc64] GCC check-tcg"
-- 
2.26.2

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Wainer dos Santos Moschetta]

On 2/7/21 9:12 AM, Philippe Mathieu-Daudé wrote:
> Travis-CI seems to have enforced memory limit on containers,
> and the 'GCC check-tcg' job started to fail on AArch64 [*]:
>
>    [2041/3679] Compiling C++ object libcommon.fa.p/disas_nanomips.cpp.o
>    FAILED: libcommon.fa.p/disas_nanomips.cpp.o
>    {standard input}: Assembler messages:
>    {standard input}:577781: Warning: end of file not at end of a line; newline inserted
>    {standard input}:577882: Error: unknown pseudo-op: `.lvl35769'
>    {standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
>    c++: fatal error: Killed signal terminated program cc1plus
>    compilation terminated.
>
> Until we have a replacement for this job on Gitlab-CI, disable
> compilation of C++ files by forcing the c++ compiler to /bin/false
> so Meson build system can not detect it:
>
>    $ ../configure --cxx=/bin/false
>
>    Compilation
>                         C compiler: cc
>                    Host C compiler: cc
>                       C++ compiler: NO
>
> [*] https://travis-ci.org/github/qemu/qemu/jobs/757819402#L3754
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> v3: Aarch -> AArch
> v2: Link to first line with error, describe Meson
>
> Supersedes: <20210206200537.2249362-1-f4bug@amsat.org>
> ---
>   .travis.yml | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)


Either disable then migrate to gitlab or going straight to the later, 
works for me.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>

>
> diff --git a/.travis.yml b/.travis.yml
> index 5f1dea873ec..b4b2d66fa4b 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -261,7 +261,7 @@ jobs:
>             - genisoimage
>         env:
>           - TEST_CMD="make check check-tcg V=1"
> -        - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS}"
> +        - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS} --cxx=/bin/false"
>           - UNRELIABLE=true
>   
>       - name: "[ppc64] GCC check-tcg"

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Philippe Mathieu-Daudé]

On Tue, Feb 9, 2021 at 1:53 PM Wainer dos Santos Moschetta
<wainersm@redhat.com> wrote:
> On 2/7/21 9:12 AM, Philippe Mathieu-Daudé wrote:
> > Travis-CI seems to have enforced memory limit on containers,
> > and the 'GCC check-tcg' job started to fail on AArch64 [*]:
> >
> >    [2041/3679] Compiling C++ object libcommon.fa.p/disas_nanomips.cpp.o
> >    FAILED: libcommon.fa.p/disas_nanomips.cpp.o
> >    {standard input}: Assembler messages:
> >    {standard input}:577781: Warning: end of file not at end of a line; newline inserted
> >    {standard input}:577882: Error: unknown pseudo-op: `.lvl35769'
> >    {standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
> >    c++: fatal error: Killed signal terminated program cc1plus
> >    compilation terminated.
> >
> > Until we have a replacement for this job on Gitlab-CI, disable
> > compilation of C++ files by forcing the c++ compiler to /bin/false
> > so Meson build system can not detect it:
> >
> >    $ ../configure --cxx=/bin/false
> >
> >    Compilation
> >                         C compiler: cc
> >                    Host C compiler: cc
> >                       C++ compiler: NO
> >
> > [*] https://travis-ci.org/github/qemu/qemu/jobs/757819402#L3754
> >
> > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > ---
> > v3: Aarch -> AArch
> > v2: Link to first line with error, describe Meson
> >
> > Supersedes: <20210206200537.2249362-1-f4bug@amsat.org>
> > ---
> >   .travis.yml | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
>
>
> Either disable then migrate to gitlab or going straight to the later,
> works for me.

Migration of this job is pending of Cleber's possibility to add an AArch64
runner to Gitlab-CI, right? Then we need someone to support and maintain
the hardware... I don't think anybody volunteered.
Meanwhile we get a replacement I agree with Daniel [1] we should stop
spending time on Travis-CI and simply remove it, along with AArch64 and
s390x coverage.

[1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg780608.html

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Peter Maydell]

On Tue, 9 Feb 2021 at 13:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> Migration of this job is pending of Cleber's possibility to add an AArch64
> runner to Gitlab-CI, right? Then we need someone to support and maintain
> the hardware... I don't think anybody volunteered.

We have the hardware already. Effectively Alex is maintaining it...

thanks
-- PMM

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Philippe Mathieu-Daudé]

On 2/9/21 2:41 PM, Peter Maydell wrote:
> On Tue, 9 Feb 2021 at 13:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>> Migration of this job is pending of Cleber's possibility to add an AArch64
>> runner to Gitlab-CI, right? Then we need someone to support and maintain
>> the hardware... I don't think anybody volunteered.
> 
> We have the hardware already. Effectively Alex is maintaining it...

I missed to read if Alex volunteered for this task but am certainly
happy if he is :) Although this should be documented somewhere (who
to contact if the AArch64 runner starts to fail?).

Assuming Cleber's runner script is merged and working on the AArch64
runner, then we need to figure how contributors can use it.
Assuming this runner will be registered under the QEMU organization
namespace in Gitlab, then contributors would have to open a Merge
Request to trigger the CI jobs (similarly to when you push to the
/staging branch). Then we would cancel the MR. We can ask contributors
to cancel their MR once testing done.

Midway could be having maintainers opening such MR?

I have no idea, just asking questions to see if other have ideas or
see the big picture here.

Regards,

Phil.

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Thomas Huth]

On 09/02/2021 15.12, Philippe Mathieu-Daudé wrote:
> On 2/9/21 2:41 PM, Peter Maydell wrote:
>> On Tue, 9 Feb 2021 at 13:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>>> Migration of this job is pending of Cleber's possibility to add an AArch64
>>> runner to Gitlab-CI, right? Then we need someone to support and maintain
>>> the hardware... I don't think anybody volunteered.
>>
>> We have the hardware already. Effectively Alex is maintaining it...
> 
> I missed to read if Alex volunteered for this task but am certainly
> happy if he is :) Although this should be documented somewhere (who
> to contact if the AArch64 runner starts to fail?).
> 
> Assuming Cleber's runner script is merged and working on the AArch64
> runner, then we need to figure how contributors can use it.
> Assuming this runner will be registered under the QEMU organization
> namespace in Gitlab, then contributors would have to open a Merge
> Request to trigger the CI jobs (similarly to when you push to the
> /staging branch). Then we would cancel the MR. We can ask contributors
> to cancel their MR once testing done.

I'm not sure, but if I got that right, opening a MR will only trigger a CI 
run on side of the requester, not of the target project? For example, when I 
look at:

  https://gitlab.com/libvirt/libvirt/-/merge_requests/65

the pipeline that is shown there ran on side of the requester, not on side 
of the libvirt project?

  Thomas

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Daniel P. Berrangé]

On Tue, Feb 09, 2021 at 03:12:30PM +0100, Philippe Mathieu-Daudé wrote:
> On 2/9/21 2:41 PM, Peter Maydell wrote:
> > On Tue, 9 Feb 2021 at 13:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> >> Migration of this job is pending of Cleber's possibility to add an AArch64
> >> runner to Gitlab-CI, right? Then we need someone to support and maintain
> >> the hardware... I don't think anybody volunteered.
> > 
> > We have the hardware already. Effectively Alex is maintaining it...
> 
> I missed to read if Alex volunteered for this task but am certainly
> happy if he is :) Although this should be documented somewhere (who
> to contact if the AArch64 runner starts to fail?).
> 
> Assuming Cleber's runner script is merged and working on the AArch64
> runner, then we need to figure how contributors can use it.
> Assuming this runner will be registered under the QEMU organization
> namespace in Gitlab, then contributors would have to open a Merge
> Request to trigger the CI jobs (similarly to when you push to the
> /staging branch). Then we would cancel the MR. We can ask contributors
> to cancel their MR once testing done.
> 
> Midway could be having maintainers opening such MR?
> 
> I have no idea, just asking questions to see if other have ideas or
> see the big picture here.

To start with, I'd say these custom runners are exclusively for the
staging branch merge testing done by Peter. If the jobs fail he will
just report them to the maintainer's pull request in the normal way
that's been done historically.


For a second phase, I think we should consider whether we really
still need / want subsystem maintainers to be sending pull requests
via email at all.

Many subsystem maintainers are already using GitLab for doing CI testing
before sending a pull request. Then they send email to the list, which
then has to be applied to a staging branch and pushed back to GitLab.
The loop through email is pretty crazy here.

It would be a lower overhead workflow if the subsystem maintainers just
opened a Merge request directly against the main repo in gitlab. This
will trigger CI, and if it passes, then it merely needs someone to
hit "merge". If it needs rebasing due to conflicts, gitlab can immediately
tell the maintainer which reduces the turn around time.


As for individual contributors using merge requests, that is of course
a technical possibility, but that would be quite a radical change in
QEMU's contribution philosphy, so I think it is wise to ignore that
to avoid getting distracted in the short term.

Focus on getting better use of GitLab and CI for subsystem maintainers
as our primary near term goal, to reduce the burden on the person doing
merges to master.

I made a suggestion for this here but no one replied...

   https://lists.gnu.org/archive/html/qemu-devel/2020-11/msg04897.html

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Daniel P. Berrangé]

On Tue, Feb 09, 2021 at 03:16:46PM +0100, Thomas Huth wrote:
> On 09/02/2021 15.12, Philippe Mathieu-Daudé wrote:
> > On 2/9/21 2:41 PM, Peter Maydell wrote:
> > > On Tue, 9 Feb 2021 at 13:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> > > > Migration of this job is pending of Cleber's possibility to add an AArch64
> > > > runner to Gitlab-CI, right? Then we need someone to support and maintain
> > > > the hardware... I don't think anybody volunteered.
> > > 
> > > We have the hardware already. Effectively Alex is maintaining it...
> > 
> > I missed to read if Alex volunteered for this task but am certainly
> > happy if he is :) Although this should be documented somewhere (who
> > to contact if the AArch64 runner starts to fail?).
> > 
> > Assuming Cleber's runner script is merged and working on the AArch64
> > runner, then we need to figure how contributors can use it.
> > Assuming this runner will be registered under the QEMU organization
> > namespace in Gitlab, then contributors would have to open a Merge
> > Request to trigger the CI jobs (similarly to when you push to the
> > /staging branch). Then we would cancel the MR. We can ask contributors
> > to cancel their MR once testing done.
> 
> I'm not sure, but if I got that right, opening a MR will only trigger a CI
> run on side of the requester, not of the target project? For example, when I
> look at:
> 
>  https://gitlab.com/libvirt/libvirt/-/merge_requests/65
> 
> the pipeline that is shown there ran on side of the requester, not on side
> of the libvirt project?

I'll have to search out the details, but there was an enhancement to
gitlab late last year that makes it possible for merge requests to
run CI in the context of the upstram project not the fork. It is
mentioned somewhere in the gitlab release notes.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 2/9/21 2:41 PM, Peter Maydell wrote:
>> On Tue, 9 Feb 2021 at 13:32, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>>> Migration of this job is pending of Cleber's possibility to add an AArch64
>>> runner to Gitlab-CI, right? Then we need someone to support and maintain
>>> the hardware... I don't think anybody volunteered.
>> 
>> We have the hardware already. Effectively Alex is maintaining it...
>
> I missed to read if Alex volunteered for this task but am certainly
> happy if he is :) Although this should be documented somewhere (who
> to contact if the AArch64 runner starts to fail?).

I'm listed on:

  https://wiki.qemu.org/AdminContacts

and have the root keys. I had given Cleber access to get the runners up
and running which I thought they were:

  3291365 ?        Ssl    8:41 /usr/bin/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --user gitlab-runner

I'm not sure what else is needed on the HW side, it's just waiting for
the gitlab bits to be plumbed in.

I should mention the HW is due to be upgraded in the next few weeks but
I don't expect the upgrade to affect the way we plan to use the machine.

> Assuming Cleber's runner script is merged and working on the AArch64
> runner, then we need to figure how contributors can use it.
> Assuming this runner will be registered under the QEMU organization
> namespace in Gitlab, then contributors would have to open a Merge
> Request to trigger the CI jobs (similarly to when you push to the
> /staging branch). Then we would cancel the MR. We can ask contributors
> to cancel their MR once testing done.
>
> Midway could be having maintainers opening such MR?
>
> I have no idea, just asking questions to see if other have ideas or
> see the big picture here.
>
> Regards,
>
> Phil.


-- 
Alex Bennée

Re: [PATCH v3] travis-ci: Disable C++ optional objects on AArch64 container

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Travis-CI seems to have enforced memory limit on containers,
> and the 'GCC check-tcg' job started to fail on AArch64 [*]:
>
>   [2041/3679] Compiling C++ object libcommon.fa.p/disas_nanomips.cpp.o
>   FAILED: libcommon.fa.p/disas_nanomips.cpp.o
>   {standard input}: Assembler messages:
>   {standard input}:577781: Warning: end of file not at end of a line; newline inserted
>   {standard input}:577882: Error: unknown pseudo-op: `.lvl35769'
>   {standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
>   c++: fatal error: Killed signal terminated program cc1plus
>   compilation terminated.
>
> Until we have a replacement for this job on Gitlab-CI, disable
> compilation of C++ files by forcing the c++ compiler to /bin/false
> so Meson build system can not detect it:
>
>   $ ../configure --cxx=/bin/false
>
>   Compilation
>                        C compiler: cc
>                   Host C compiler: cc
>                      C++ compiler: NO
>
> [*] https://travis-ci.org/github/qemu/qemu/jobs/757819402#L3754
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Queued to testing/next, thanks.
-- 
Alex Bennée