[PATCH 0/2] gitlab: avoid timeouts in windows cross builds

[PATCH 0/2] gitlab: avoid timeouts in windows cross builds

[Daniel P. Berrangé]

The win64 cross build is quite often hitting the job timeout, despite
having an elevated timeout of 80 minutes. A typical time is 75-78
minutes in my tests, which leaves little headroom.

I'm not inclined to increase the timeout beyond 80 minutes as this is
already unreasonably long.

Interestingly the win32 job doesn't timeout in the same way. In local
tests I've found the mingw32 gcc is as much as 50% faster than
mingw64 gcc in building QEMU. This explains at least why we only see
win64 gitlab builds timeout normally.

None the less, this series thus cuts down on the amount of stuff built
in both win32 and win64 jobs, as well as cross build jobs more
generally.

With this applied:

 - win32 job runs in 40 minutes instead of 55 minutes
 - win64 job runs in 50 minutes instead of 75 minutes
 - other cross build jobs are a few minutes faster

this should give us plenty of headroom to avoid hitting the job
timeouts.

Daniel P. Berrangé (2):
  gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds
  gitlab: skip many more targets in windows cross builds

 .gitlab-ci.d/crossbuild-template.yml | 3 ++-
 .gitlab-ci.d/crossbuilds.yml         | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

-- 
2.31.1

[PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Daniel P. Berrangé]

We need to cut down compile time by excluding more targets. Both these
targets still have their 64-bit variant enabled, so the loss of coverage
is mitigated to some degree.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/crossbuild-template.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 7d3ad00a1e..cfb576b54c 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -9,7 +9,8 @@
       ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-user --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
-          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
+          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
+          sparc-softmmu xtensa-softmmu"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
     - if grep -q "EXESUF=.exe" config-host.mak;
       then make installer;
-- 
2.31.1

[PATCH 2/2] gitlab: skip many more targets in windows cross builds

[Daniel P. Berrangé]

The windows cross builds still take way too long in gitlab CI, so need
more targets to be skipped. We don't want to hurt coverage of other
cross builds more though, so we let jobs fine tune with a new env
variale $CROSS_SKIP_TARGETS.

We take the set of targets that are considered relatively niche or
very old architectures, and skip approx half of them in win32 builds
and the other half of them in win64.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/crossbuild-template.yml | 2 +-
 .gitlab-ci.d/crossbuilds.yml         | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index cfb576b54c..10d22dcf6c 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -10,7 +10,7 @@
         --disable-user --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
           mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
-          sparc-softmmu xtensa-softmmu"
+          sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
     - if grep -q "EXESUF=.exe" config-host.mak;
       then make installer;
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 4ff3aa3cfc..a5f9dbcbeb 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -160,6 +160,7 @@ cross-win32-system:
     job: win32-fedora-cross-container
   variables:
     IMAGE: fedora-win32-cross
+    CROSS_SKIP_TARGETS: or1k-softmmu rx-softmmu sh4eb-softmmu sparc64-softmmu tricore-softmmu xtensaeb-softmmu
   artifacts:
     paths:
       - build/qemu-setup*.exe
@@ -170,6 +171,7 @@ cross-win64-system:
     job: win64-fedora-cross-container
   variables:
     IMAGE: fedora-win64-cross
+    CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu microblazeel-softmmu mips64el-softmmu nios2-softmmu
   artifacts:
     paths:
       - build/qemu-setup*.exe
-- 
2.31.1

Re: [PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Willian Rampazzo]

On Tue, Aug 10, 2021 at 11:07 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> We need to cut down compile time by excluding more targets. Both these
> targets still have their 64-bit variant enabled, so the loss of coverage
> is mitigated to some degree.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/crossbuild-template.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

Re: [PATCH 2/2] gitlab: skip many more targets in windows cross builds

[Willian Rampazzo]

On Tue, Aug 10, 2021 at 11:07 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> The windows cross builds still take way too long in gitlab CI, so need
> more targets to be skipped. We don't want to hurt coverage of other
> cross builds more though, so we let jobs fine tune with a new env
> variale $CROSS_SKIP_TARGETS.

s/variale/variable

>
> We take the set of targets that are considered relatively niche or
> very old architectures, and skip approx half of them in win32 builds
> and the other half of them in win64.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/crossbuild-template.yml | 2 +-
>  .gitlab-ci.d/crossbuilds.yml         | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

Re: [PATCH 0/2] gitlab: avoid timeouts in windows cross builds

[Thomas Huth]

On 10/08/2021 16.06, Daniel P. Berrangé wrote:
> The win64 cross build is quite often hitting the job timeout, despite
> having an elevated timeout of 80 minutes. A typical time is 75-78
> minutes in my tests, which leaves little headroom.
> 
> I'm not inclined to increase the timeout beyond 80 minutes as this is
> already unreasonably long.
> 
> Interestingly the win32 job doesn't timeout in the same way. In local
> tests I've found the mingw32 gcc is as much as 50% faster than
> mingw64 gcc in building QEMU. This explains at least why we only see
> win64 gitlab builds timeout normally.

I've had similar experiences in the past, and once even already sent a patch:

 
https://patchwork.ozlabs.org/project/qemu-devel/patch/20210414081907.871437-4-thuth@redhat.com/

But your apprach is looking way nicer, indeed!

Queued to my testing-next branch now:

  https://gitlab.com/thuth/qemu/-/commits/testing-next/

Thanks,
  Thomas

Re: [PATCH 2/2] gitlab: skip many more targets in windows cross builds

[Philippe Mathieu-Daudé]

On 8/10/21 4:06 PM, Daniel P. Berrangé wrote:
> The windows cross builds still take way too long in gitlab CI, so need
> more targets to be skipped. We don't want to hurt coverage of other
> cross builds more though, so we let jobs fine tune with a new env
> variale $CROSS_SKIP_TARGETS.
> 
> We take the set of targets that are considered relatively niche or
> very old architectures, and skip approx half of them in win32 builds
> and the other half of them in win64.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/crossbuild-template.yml | 2 +-
>  .gitlab-ci.d/crossbuilds.yml         | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
> index cfb576b54c..10d22dcf6c 100644
> --- a/.gitlab-ci.d/crossbuild-template.yml
> +++ b/.gitlab-ci.d/crossbuild-template.yml
> @@ -10,7 +10,7 @@
>          --disable-user --target-list-exclude="arm-softmmu cris-softmmu
>            i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
>            mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
> -          sparc-softmmu xtensa-softmmu"
> +          sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
>      - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
>      - if grep -q "EXESUF=.exe" config-host.mak;
>        then make installer;
> diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
> index 4ff3aa3cfc..a5f9dbcbeb 100644
> --- a/.gitlab-ci.d/crossbuilds.yml
> +++ b/.gitlab-ci.d/crossbuilds.yml
> @@ -160,6 +160,7 @@ cross-win32-system:
>      job: win32-fedora-cross-container
>    variables:
>      IMAGE: fedora-win32-cross
> +    CROSS_SKIP_TARGETS: or1k-softmmu rx-softmmu sh4eb-softmmu sparc64-softmmu tricore-softmmu xtensaeb-softmmu
>    artifacts:
>      paths:
>        - build/qemu-setup*.exe
> @@ -170,6 +171,7 @@ cross-win64-system:
>      job: win64-fedora-cross-container
>    variables:
>      IMAGE: fedora-win64-cross
> +    CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu microblazeel-softmmu mips64el-softmmu nios2-softmmu

It looks you inverted the lists. We expect more Win64 users, and these
are the targets of interest. I'd keep here (skipping in Win32):

alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu microblazeel-softmmu
mips64el-softmmu sparc64-softmmu

And skip (keep them in win32):

or1k-softmmu rx-softmmu sh4eb-softmmu nios2-softmmu tricore-softmmu
xtensaeb-softmmu

>    artifacts:
>      paths:
>        - build/qemu-setup*.exe
> 

Re: [PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Philippe Mathieu-Daudé]

+Mark/Artyom

On 8/10/21 4:06 PM, Daniel P. Berrangé wrote:
> We need to cut down compile time by excluding more targets. Both these
> targets still have their 64-bit variant enabled, so the loss of coverage
> is mitigated to some degree.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/crossbuild-template.yml | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
> index 7d3ad00a1e..cfb576b54c 100644
> --- a/.gitlab-ci.d/crossbuild-template.yml
> +++ b/.gitlab-ci.d/crossbuild-template.yml
> @@ -9,7 +9,8 @@
>        ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
>          --disable-user --target-list-exclude="arm-softmmu cris-softmmu
>            i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
> -          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
> +          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
> +          sparc-softmmu xtensa-softmmu"

I disagree for sparc, as there are some differences (sparc64 is not
a superset of it). However I'm not against this patch.

>      - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
>      - if grep -q "EXESUF=.exe" config-host.mak;
>        then make installer;
> 

Re: [PATCH 2/2] gitlab: skip many more targets in windows cross builds

[Daniel P. Berrangé]

On Wed, Aug 11, 2021 at 08:20:37AM +0200, Philippe Mathieu-Daudé wrote:
> On 8/10/21 4:06 PM, Daniel P. Berrangé wrote:
> > The windows cross builds still take way too long in gitlab CI, so need
> > more targets to be skipped. We don't want to hurt coverage of other
> > cross builds more though, so we let jobs fine tune with a new env
> > variale $CROSS_SKIP_TARGETS.
> > 
> > We take the set of targets that are considered relatively niche or
> > very old architectures, and skip approx half of them in win32 builds
> > and the other half of them in win64.
> > 
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >  .gitlab-ci.d/crossbuild-template.yml | 2 +-
> >  .gitlab-ci.d/crossbuilds.yml         | 2 ++
> >  2 files changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
> > index cfb576b54c..10d22dcf6c 100644
> > --- a/.gitlab-ci.d/crossbuild-template.yml
> > +++ b/.gitlab-ci.d/crossbuild-template.yml
> > @@ -10,7 +10,7 @@
> >          --disable-user --target-list-exclude="arm-softmmu cris-softmmu
> >            i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
> >            mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
> > -          sparc-softmmu xtensa-softmmu"
> > +          sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
> >      - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
> >      - if grep -q "EXESUF=.exe" config-host.mak;
> >        then make installer;
> > diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
> > index 4ff3aa3cfc..a5f9dbcbeb 100644
> > --- a/.gitlab-ci.d/crossbuilds.yml
> > +++ b/.gitlab-ci.d/crossbuilds.yml
> > @@ -160,6 +160,7 @@ cross-win32-system:
> >      job: win32-fedora-cross-container
> >    variables:
> >      IMAGE: fedora-win32-cross
> > +    CROSS_SKIP_TARGETS: or1k-softmmu rx-softmmu sh4eb-softmmu sparc64-softmmu tricore-softmmu xtensaeb-softmmu
> >    artifacts:
> >      paths:
> >        - build/qemu-setup*.exe
> > @@ -170,6 +171,7 @@ cross-win64-system:
> >      job: win64-fedora-cross-container
> >    variables:
> >      IMAGE: fedora-win64-cross
> > +    CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu microblazeel-softmmu mips64el-softmmu nios2-softmmu
> 
> It looks you inverted the lists. We expect more Win64 users, and these
> are the targets of interest. I'd keep here (skipping in Win32):

I wouldn't say inverted, because the split was entirely arbitrary. While
32-bit vs 64-bit could conceivably show a difference, in practice those
differences are likely to already be caught by non-windows builds. These
jobs are most important for catching UNIX vs Windows differences, so in
that view whether a target is built for win32 vs win64 doesn't matter.

> 
> alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu microblazeel-softmmu
> mips64el-softmmu sparc64-softmmu
> 
> And skip (keep them in win32):
> 
> or1k-softmmu rx-softmmu sh4eb-softmmu nios2-softmmu tricore-softmmu
> xtensaeb-softmmu

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Mark Cave-Ayland]

On 10/08/2021 15:06, Daniel P. Berrangé wrote:

> We need to cut down compile time by excluding more targets. Both these
> targets still have their 64-bit variant enabled, so the loss of coverage
> is mitigated to some degree.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   .gitlab-ci.d/crossbuild-template.yml | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
> index 7d3ad00a1e..cfb576b54c 100644
> --- a/.gitlab-ci.d/crossbuild-template.yml
> +++ b/.gitlab-ci.d/crossbuild-template.yml
> @@ -9,7 +9,8 @@
>         ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
>           --disable-user --target-list-exclude="arm-softmmu cris-softmmu
>             i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
> -          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
> +          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
> +          sparc-softmmu xtensa-softmmu"
>       - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
>       - if grep -q "EXESUF=.exe" config-host.mak;
>         then make installer;

I'd prefer to keep sparc-softmmu if possible, simply because my everyday platform is 
Linux and so having a cross-build for Windows will catch things that I may miss on a 
day-to-day basis. Is sparc-softmmu currently enabled as part of the native MINGW64 build?

If I go to my Gitlab QEMU fork Settings -> CI/CD -> Variables there is an option to 
set variables that can be used in job scripts. Perhaps this could be used so that I 
can configure my personal QEMU fork to always run sparc-softmmu builds when preparing 
PRs even if they aren't enabled for everyone by default? At least this would then 
allow me to spot any breakage before sending a final PR to Peter.


ATB,

Mark.

Re: [PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Daniel P. Berrangé]

On Wed, Aug 18, 2021 at 10:15:47AM +0100, Mark Cave-Ayland wrote:
> On 10/08/2021 15:06, Daniel P. Berrangé wrote:
> 
> > We need to cut down compile time by excluding more targets. Both these
> > targets still have their 64-bit variant enabled, so the loss of coverage
> > is mitigated to some degree.
> > 
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >   .gitlab-ci.d/crossbuild-template.yml | 3 ++-
> >   1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
> > index 7d3ad00a1e..cfb576b54c 100644
> > --- a/.gitlab-ci.d/crossbuild-template.yml
> > +++ b/.gitlab-ci.d/crossbuild-template.yml
> > @@ -9,7 +9,8 @@
> >         ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
> >           --disable-user --target-list-exclude="arm-softmmu cris-softmmu
> >             i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
> > -          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
> > +          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
> > +          sparc-softmmu xtensa-softmmu"
> >       - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
> >       - if grep -q "EXESUF=.exe" config-host.mak;
> >         then make installer;
> 
> I'd prefer to keep sparc-softmmu if possible, simply because my everyday
> platform is Linux and so having a cross-build for Windows will catch things
> that I may miss on a day-to-day basis. Is sparc-softmmu currently enabled as
> part of the native MINGW64 build?

Note the builds still include  'sparc64-softmmu', so we're only loosing
cover in places where it diverges fromthe 64-bit build, but this will
sstil get coverage in native builds.

> If I go to my Gitlab QEMU fork Settings -> CI/CD -> Variables there is an
> option to set variables that can be used in job scripts. Perhaps this could
> be used so that I can configure my personal QEMU fork to always run
> sparc-softmmu builds when preparing PRs even if they aren't enabled for
> everyone by default? At least this would then allow me to spot any breakage
> before sending a final PR to Peter.

Separately from this I'm doing some work to make things more configurable

  https://lists.gnu.org/archive/html/qemu-devel/2021-08/msg02102.html

but not on this level of granularity.

Rather than globally excluding from all cross builds, I think we ought to
split it up more equitably acrss the different builds in some way.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Mark Cave-Ayland]

On 18/08/2021 10:29, Daniel P. Berrangé wrote:

> On Wed, Aug 18, 2021 at 10:15:47AM +0100, Mark Cave-Ayland wrote:
>> On 10/08/2021 15:06, Daniel P. Berrangé wrote:
>>
>>> We need to cut down compile time by excluding more targets. Both these
>>> targets still have their 64-bit variant enabled, so the loss of coverage
>>> is mitigated to some degree.
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>    .gitlab-ci.d/crossbuild-template.yml | 3 ++-
>>>    1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
>>> index 7d3ad00a1e..cfb576b54c 100644
>>> --- a/.gitlab-ci.d/crossbuild-template.yml
>>> +++ b/.gitlab-ci.d/crossbuild-template.yml
>>> @@ -9,7 +9,8 @@
>>>          ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
>>>            --disable-user --target-list-exclude="arm-softmmu cris-softmmu
>>>              i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
>>> -          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
>>> +          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
>>> +          sparc-softmmu xtensa-softmmu"
>>>        - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
>>>        - if grep -q "EXESUF=.exe" config-host.mak;
>>>          then make installer;
>>
>> I'd prefer to keep sparc-softmmu if possible, simply because my everyday
>> platform is Linux and so having a cross-build for Windows will catch things
>> that I may miss on a day-to-day basis. Is sparc-softmmu currently enabled as
>> part of the native MINGW64 build?
> 
> Note the builds still include  'sparc64-softmmu', so we're only loosing
> cover in places where it diverges fromthe 64-bit build, but this will
> sstil get coverage in native builds.

Okay I see. From my perspective sparc-softmmu covers more corner cases simply because 
everything is wired up as cMMIO peripherals on a custum bus compared with 
sparc64-softmmu which is a PCI machine and so there is already a lot of overlap 
there. However...

>> If I go to my Gitlab QEMU fork Settings -> CI/CD -> Variables there is an
>> option to set variables that can be used in job scripts. Perhaps this could
>> be used so that I can configure my personal QEMU fork to always run
>> sparc-softmmu builds when preparing PRs even if they aren't enabled for
>> everyone by default? At least this would then allow me to spot any breakage
>> before sending a final PR to Peter.
> 
> Separately from this I'm doing some work to make things more configurable
> 
>    https://lists.gnu.org/archive/html/qemu-devel/2021-08/msg02102.html
> 
> but not on this level of granularity.
> 
> Rather than globally excluding from all cross builds, I think we ought to
> split it up more equitably acrss the different builds in some way.

 From my perspective as long as I have something in Gitlab that I can push and then 
get a green CI build that covers both sparc platforms on Linux/Windows then I'm not 
too concerned about the implementation details. As long as the combination is covered 
in a job somewhere in your latest set of build changes then that's okay with me.


ATB,

Mark.

Re: [PATCH 1/2] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds

[Philippe Mathieu-Daudé]

On 8/18/21 11:45 AM, Mark Cave-Ayland wrote:
> On 18/08/2021 10:29, Daniel P. Berrangé wrote:
>> On Wed, Aug 18, 2021 at 10:15:47AM +0100, Mark Cave-Ayland wrote:
>>> On 10/08/2021 15:06, Daniel P. Berrangé wrote:
>>>
>>>> We need to cut down compile time by excluding more targets. Both these
>>>> targets still have their 64-bit variant enabled, so the loss of
>>>> coverage
>>>> is mitigated to some degree.
>>>>
>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>>> ---
>>>>    .gitlab-ci.d/crossbuild-template.yml | 3 ++-
>>>>    1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/.gitlab-ci.d/crossbuild-template.yml
>>>> b/.gitlab-ci.d/crossbuild-template.yml
>>>> index 7d3ad00a1e..cfb576b54c 100644
>>>> --- a/.gitlab-ci.d/crossbuild-template.yml
>>>> +++ b/.gitlab-ci.d/crossbuild-template.yml
>>>> @@ -9,7 +9,8 @@
>>>>          ../configure --enable-werror --disable-docs
>>>> $QEMU_CONFIGURE_OPTS
>>>>            --disable-user --target-list-exclude="arm-softmmu
>>>> cris-softmmu
>>>>              i386-softmmu microblaze-softmmu mips-softmmu
>>>> mipsel-softmmu
>>>> -          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
>>>> +          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
>>>> +          sparc-softmmu xtensa-softmmu"
>>>>        - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
>>>>        - if grep -q "EXESUF=.exe" config-host.mak;
>>>>          then make installer;
>>>
>>> I'd prefer to keep sparc-softmmu if possible, simply because my everyday
>>> platform is Linux and so having a cross-build for Windows will catch
>>> things
>>> that I may miss on a day-to-day basis. Is sparc-softmmu currently
>>> enabled as
>>> part of the native MINGW64 build?
>>
>> Note the builds still include  'sparc64-softmmu', so we're only loosing
>> cover in places where it diverges fromthe 64-bit build, but this will
>> sstil get coverage in native builds.
> 
> Okay I see. From my perspective sparc-softmmu covers more corner cases
> simply because everything is wired up as cMMIO peripherals on a custum
> bus compared with sparc64-softmmu which is a PCI machine and so there is
> already a lot of overlap there. However...
> 
>>> If I go to my Gitlab QEMU fork Settings -> CI/CD -> Variables there
>>> is an
>>> option to set variables that can be used in job scripts. Perhaps this
>>> could
>>> be used so that I can configure my personal QEMU fork to always run
>>> sparc-softmmu builds when preparing PRs even if they aren't enabled for
>>> everyone by default? At least this would then allow me to spot any
>>> breakage
>>> before sending a final PR to Peter.
>>
>> Separately from this I'm doing some work to make things more configurable
>>
>>    https://lists.gnu.org/archive/html/qemu-devel/2021-08/msg02102.html
>>
>> but not on this level of granularity.
>>
>> Rather than globally excluding from all cross builds, I think we ought to
>> split it up more equitably acrss the different builds in some way.
> 
> From my perspective as long as I have something in Gitlab that I can
> push and then get a green CI build that covers both sparc platforms on
> Linux/Windows then I'm not too concerned about the implementation
> details. As long as the combination is covered in a job somewhere in
> your latest set of build changes then that's okay with me.

Your use case seems similar to mine: I overload the current mainstream
pipeline with extra jobs. See:
https://docs.gitlab.com/ee/ci/pipelines/settings.html#specify-a-custom-cicd-configuration-file

My namespace CI points at a custom config in some branch:
https://gitlab.com/philmd/qemu/raw/my_gci_config/.gitlab-ci.d/philmd.yml

This files simply contains:

include:
  # Keep mainstream config ...
  - local: '/.gitlab-ci.d/qemu-project.yml'
  # ... but include my extra jobs.
  -
'https://gitlab.com/philmd/qemu/raw/my_gci_config/.gitlab-ci.d/nvme_wip.yml'
  -
'https://gitlab.com/philmd/qemu/raw/my_gci_config/.gitlab-ci.d/mips_slow.yml'
  -
'https://gitlab.com/philmd/qemu/raw/my_gci_config/.gitlab-ci.d/philmd_runners.yml'