* gitlab-ci: Only build /staging branch? @ 2021-03-24 18:01 Philippe Mathieu-Daudé 2021-03-24 18:33 ` Philippe Mathieu-Daudé 2021-03-24 23:11 ` Cleber Rosa 0 siblings, 2 replies; 11+ messages in thread From: Philippe Mathieu-Daudé @ 2021-03-24 18:01 UTC (permalink / raw) To: Paolo Bonzini, Alex Bennée, Thomas Huth Cc: qemu-devel, Willian Rampazzo, Cleber Rosa Hi, Peter's current workflow is push to /staging and if his testing succeeds, he pushes the same commit as /master. IMO there is no point in building /master branch, as it has already been built earlier as /staging. Regards, Phil. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-24 18:01 gitlab-ci: Only build /staging branch? Philippe Mathieu-Daudé @ 2021-03-24 18:33 ` Philippe Mathieu-Daudé 2021-03-24 21:58 ` Philippe Mathieu-Daudé 2021-03-24 23:11 ` Cleber Rosa 1 sibling, 1 reply; 11+ messages in thread From: Philippe Mathieu-Daudé @ 2021-03-24 18:33 UTC (permalink / raw) To: Paolo Bonzini, Alex Bennée, Thomas Huth Cc: qemu-devel, Willian Rampazzo, Cleber Rosa On 3/24/21 7:01 PM, Philippe Mathieu-Daudé wrote: > Hi, > > Peter's current workflow is push to /staging and if his > testing succeeds, he pushes the same commit as /master. > > IMO there is no point in building /master branch, as it > has already been built earlier as /staging. Also this might be the reason why the /staging "deploy" job fails with "input/output error". ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-24 18:33 ` Philippe Mathieu-Daudé @ 2021-03-24 21:58 ` Philippe Mathieu-Daudé 2021-03-25 5:43 ` Thomas Huth 0 siblings, 1 reply; 11+ messages in thread From: Philippe Mathieu-Daudé @ 2021-03-24 21:58 UTC (permalink / raw) To: Paolo Bonzini, Alex Bennée, Thomas Huth Cc: qemu-devel, Willian Rampazzo, Cleber Rosa On 3/24/21 7:33 PM, Philippe Mathieu-Daudé wrote: > On 3/24/21 7:01 PM, Philippe Mathieu-Daudé wrote: >> Hi, >> >> Peter's current workflow is push to /staging and if his >> testing succeeds, he pushes the same commit as /master. >> >> IMO there is no point in building /master branch, as it >> has already been built earlier as /staging. Similarly with tags. Although we don't tag often. Suggestion: - staging: always build - tags: on-demand build (manual) - master: no build Thinking about reproducible artifacts, we could have a tag job to produce the archive with submodules, but someone still have to sign it. Thoughts? ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-24 21:58 ` Philippe Mathieu-Daudé @ 2021-03-25 5:43 ` Thomas Huth 2021-03-25 9:29 ` Philippe Mathieu-Daudé 0 siblings, 1 reply; 11+ messages in thread From: Thomas Huth @ 2021-03-25 5:43 UTC (permalink / raw) To: Philippe Mathieu-Daudé, Paolo Bonzini, Alex Bennée Cc: qemu-devel, Willian Rampazzo, Cleber Rosa On 24/03/2021 22.58, Philippe Mathieu-Daudé wrote: > On 3/24/21 7:33 PM, Philippe Mathieu-Daudé wrote: >> On 3/24/21 7:01 PM, Philippe Mathieu-Daudé wrote: >>> Hi, >>> >>> Peter's current workflow is push to /staging and if his >>> testing succeeds, he pushes the same commit as /master. >>> >>> IMO there is no point in building /master branch, as it >>> has already been built earlier as /staging. > > Similarly with tags. Although we don't tag often. Tags are used for pull-requests. So I think we should run the whole CI for tags, to make it clear that a pull-request always includes code that builds fine. Thomas ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-25 5:43 ` Thomas Huth @ 2021-03-25 9:29 ` Philippe Mathieu-Daudé 2021-03-25 9:50 ` Philippe Mathieu-Daudé 2021-03-25 10:34 ` Peter Maydell 0 siblings, 2 replies; 11+ messages in thread From: Philippe Mathieu-Daudé @ 2021-03-25 9:29 UTC (permalink / raw) To: Thomas Huth, Alex Bennée Cc: Paolo Bonzini, qemu-devel, Willian Rampazzo, Cleber Rosa On 3/25/21 6:43 AM, Thomas Huth wrote: > On 24/03/2021 22.58, Philippe Mathieu-Daudé wrote: >> On 3/24/21 7:33 PM, Philippe Mathieu-Daudé wrote: >>> On 3/24/21 7:01 PM, Philippe Mathieu-Daudé wrote: >>>> Hi, >>>> >>>> Peter's current workflow is push to /staging and if his >>>> testing succeeds, he pushes the same commit as /master. >>>> >>>> IMO there is no point in building /master branch, as it >>>> has already been built earlier as /staging. >> >> Similarly with tags. Although we don't tag often. > > Tags are used for pull-requests. So I think we should run the whole CI > for tags, to make it clear that a pull-request always includes code that > builds fine. Sorry the context was not clear :/ This is only relevant for the qemu-project/qemu gitlab namespace. v6.0 is at the door and I was wondering what is missing to have the CI used as a gate. - Stefan/Paolo moved the main repository location. - Alex made yet another effort to get the CI pipeline green again. - IIRC Peter said waiting 2h after pushing /staging is too long. Currently worst case it takes ~2h25 between one /staging and the next one, simply because /master is rebuilt in the middle. If we remove /master we have ~1h15 per /staging pipeline. - I don't remember what is missing from Cleber script, maybe we can use it as it without waiting for a respin? We have never been that close, but we are not there yet... Regards, Phil. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-25 9:29 ` Philippe Mathieu-Daudé @ 2021-03-25 9:50 ` Philippe Mathieu-Daudé 2021-03-25 10:34 ` Peter Maydell 1 sibling, 0 replies; 11+ messages in thread From: Philippe Mathieu-Daudé @ 2021-03-25 9:50 UTC (permalink / raw) To: Thomas Huth, Alex Bennée Cc: Paolo Bonzini, qemu-devel, Willian Rampazzo, Cleber Rosa On 3/25/21 10:29 AM, Philippe Mathieu-Daudé wrote: > On 3/25/21 6:43 AM, Thomas Huth wrote: >> On 24/03/2021 22.58, Philippe Mathieu-Daudé wrote: >>> On 3/24/21 7:33 PM, Philippe Mathieu-Daudé wrote: >>>> On 3/24/21 7:01 PM, Philippe Mathieu-Daudé wrote: >>>>> Hi, >>>>> >>>>> Peter's current workflow is push to /staging and if his >>>>> testing succeeds, he pushes the same commit as /master. >>>>> >>>>> IMO there is no point in building /master branch, as it >>>>> has already been built earlier as /staging. >>> >>> Similarly with tags. Although we don't tag often. >> >> Tags are used for pull-requests. So I think we should run the whole CI >> for tags, to make it clear that a pull-request always includes code that >> builds fine. > > Sorry the context was not clear :/ > > This is only relevant for the qemu-project/qemu gitlab namespace. > > v6.0 is at the door and I was wondering what is missing to have the > CI used as a gate. > > - Stefan/Paolo moved the main repository location. > > - Alex made yet another effort to get the CI pipeline green again. > > - IIRC Peter said waiting 2h after pushing /staging is too long. > Currently worst case it takes ~2h25 between one /staging and the > next one, simply because /master is rebuilt in the middle. If we > remove /master we have ~1h15 per /staging pipeline. > > - I don't remember what is missing from Cleber script, maybe we can > use it as it without waiting for a respin? As someone else is caring about this, please disregard this thread and its questions/suggestions. Regards, Phil. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-25 9:29 ` Philippe Mathieu-Daudé 2021-03-25 9:50 ` Philippe Mathieu-Daudé @ 2021-03-25 10:34 ` Peter Maydell 2021-03-25 11:05 ` Paolo Bonzini 1 sibling, 1 reply; 11+ messages in thread From: Peter Maydell @ 2021-03-25 10:34 UTC (permalink / raw) To: Philippe Mathieu-Daudé Cc: Thomas Huth, qemu-devel, Willian Rampazzo, Cleber Rosa, Paolo Bonzini, Alex Bennée On Thu, 25 Mar 2021 at 09:33, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote: > v6.0 is at the door and I was wondering what is missing to have the > CI used as a gate. It needs to be faster. Mostly I do check the gitlab CI pipeline status, but in the run-up to getting rc0 out I stopped waiting for the gitlab CI job to finish, because I was continually finding that I kicked off a run, my local build-tests would complete within an hour or so, and the gitlab CI jobs were still pending, barely started, etc. Turnaround on testing a merge must be 90 minutes or less, especially during release periods, because there are always a huge number of merges that arrive for me to test in the last couple of days before freeze. thanks -- PMM ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-25 10:34 ` Peter Maydell @ 2021-03-25 11:05 ` Paolo Bonzini 2021-03-25 11:06 ` Peter Maydell 2021-03-25 11:14 ` Daniel P. Berrangé 0 siblings, 2 replies; 11+ messages in thread From: Paolo Bonzini @ 2021-03-25 11:05 UTC (permalink / raw) To: Peter Maydell, Philippe Mathieu-Daudé Cc: Thomas Huth, Alex Bennée, qemu-devel, Willian Rampazzo, Cleber Rosa On 25/03/21 11:34, Peter Maydell wrote: > It needs to be faster. Mostly I do check the gitlab CI pipeline > status, but in the run-up to getting rc0 out I stopped waiting > for the gitlab CI job to finish, because I was continually finding > that I kicked off a run, my local build-tests would complete within > an hour or so, and the gitlab CI jobs were still pending, barely > started, etc. Turnaround on testing a merge must be 90 minutes or > less, especially during release periods, because there are always > a huge number of merges that arrive for me to test in the last > couple of days before freeze. Perhaps we could script it so that if the pipeline passes the merge to master is done automatically. Paolo ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-25 11:05 ` Paolo Bonzini @ 2021-03-25 11:06 ` Peter Maydell 2021-03-25 11:14 ` Daniel P. Berrangé 1 sibling, 0 replies; 11+ messages in thread From: Peter Maydell @ 2021-03-25 11:06 UTC (permalink / raw) To: Paolo Bonzini Cc: Thomas Huth, Philippe Mathieu-Daudé, qemu-devel, Willian Rampazzo, Cleber Rosa, Alex Bennée On Thu, 25 Mar 2021 at 11:05, Paolo Bonzini <pbonzini@redhat.com> wrote: > > On 25/03/21 11:34, Peter Maydell wrote: > > It needs to be faster. Mostly I do check the gitlab CI pipeline > > status, but in the run-up to getting rc0 out I stopped waiting > > for the gitlab CI job to finish, because I was continually finding > > that I kicked off a run, my local build-tests would complete within > > an hour or so, and the gitlab CI jobs were still pending, barely > > started, etc. Turnaround on testing a merge must be 90 minutes or > > less, especially during release periods, because there are always > > a huge number of merges that arrive for me to test in the last > > couple of days before freeze. > > Perhaps we could script it so that if the pipeline passes the merge to > master is done automatically. That would be nice eventually, but we can't do it until the gitlab CI is the *only* gating criterion. thanks -- PMM ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-25 11:05 ` Paolo Bonzini 2021-03-25 11:06 ` Peter Maydell @ 2021-03-25 11:14 ` Daniel P. Berrangé 1 sibling, 0 replies; 11+ messages in thread From: Daniel P. Berrangé @ 2021-03-25 11:14 UTC (permalink / raw) To: Paolo Bonzini Cc: Peter Maydell, Thomas Huth, Philippe Mathieu-Daudé, qemu-devel, Willian Rampazzo, Cleber Rosa, Alex Bennée On Thu, Mar 25, 2021 at 12:05:32PM +0100, Paolo Bonzini wrote: > On 25/03/21 11:34, Peter Maydell wrote: > > It needs to be faster. Mostly I do check the gitlab CI pipeline > > status, but in the run-up to getting rc0 out I stopped waiting > > for the gitlab CI job to finish, because I was continually finding > > that I kicked off a run, my local build-tests would complete within > > an hour or so, and the gitlab CI jobs were still pending, barely > > started, etc. Turnaround on testing a merge must be 90 minutes or > > less, especially during release periods, because there are always > > a huge number of merges that arrive for me to test in the last > > couple of days before freeze. > > Perhaps we could script it so that if the pipeline passes the merge to > master is done automatically. No need to script it, that functionality already exists in GitLab. Push to the staging branch, and open a merge request for applying staging -> master, and enable "merge when pipeline succeeds". You can actually do this all in one command https://docs.gitlab.com/ee/user/project/push_options.html git push \ -o merge_request.create \ -o merge_request.target=master \ -o merge_request.merge_when_pipeline_succeeds \ origin staging The gitlab-ci.yml file could then be configured so that pipeline jobs are associated with a merge request, rather than push event. This will avoid the pipeline being re-run on master after the merge. If you enable "merge trains" option in the repo, then you can even push to multiple branches concurrently, and gitlab will serialize the CI pipelines from each merge request in turn, (assuming no conflicts between then). Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: gitlab-ci: Only build /staging branch? 2021-03-24 18:01 gitlab-ci: Only build /staging branch? Philippe Mathieu-Daudé 2021-03-24 18:33 ` Philippe Mathieu-Daudé @ 2021-03-24 23:11 ` Cleber Rosa 1 sibling, 0 replies; 11+ messages in thread From: Cleber Rosa @ 2021-03-24 23:11 UTC (permalink / raw) To: Philippe Mathieu-Daudé Cc: Paolo Bonzini, Thomas Huth, Alex Bennée, qemu-devel, Willian Rampazzo [-- Attachment #1: Type: text/plain, Size: 836 bytes --] On Wed, Mar 24, 2021 at 07:01:12PM +0100, Philippe Mathieu-Daudé wrote: > Hi, > > Peter's current workflow is push to /staging and if his > testing succeeds, he pushes the same commit as /master. > > IMO there is no point in building /master branch, as it > has already been built earlier as /staging. > Hi, I agree, I currently see no point in running the entire pipeline again. But we need to watch out for the job conditional rules. For instance, both of these pipelines are about the exact same commit: https://gitlab.com/qemu-project/qemu/-/pipelines/275925246 https://gitlab.com/qemu-project/qemu/-/pipelines/275925289 But one has 113, while the other has 119 jobs. Also, we may want to check if tags also trigger new (unnecessary) pipelines. > Regards, > > Phil. > Best, - Cleber [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-03-25 11:15 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-24 18:01 gitlab-ci: Only build /staging branch? Philippe Mathieu-Daudé 2021-03-24 18:33 ` Philippe Mathieu-Daudé 2021-03-24 21:58 ` Philippe Mathieu-Daudé 2021-03-25 5:43 ` Thomas Huth 2021-03-25 9:29 ` Philippe Mathieu-Daudé 2021-03-25 9:50 ` Philippe Mathieu-Daudé 2021-03-25 10:34 ` Peter Maydell 2021-03-25 11:05 ` Paolo Bonzini 2021-03-25 11:06 ` Peter Maydell 2021-03-25 11:14 ` Daniel P. Berrangé 2021-03-24 23:11 ` Cleber Rosa
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).