[PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

[PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

[Philippe Mathieu-Daudé]

While we are interested in building docker images in different
branches, it only makes sense to push 'latest' to the registry
when this is the project default branch (usually 'master').

Else when pushing different branches concurrently we might have
inconsistent image state between branches.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 .gitlab-ci.d/containers.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index 90fac85ce46..52a915f4141 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -17,7 +17,7 @@
           -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
           -r $CI_REGISTRY_IMAGE
     - docker tag "qemu/$NAME" "$TAG"
-    - docker push "$TAG"
+    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push "$TAG"
   after_script:
     - docker logout
   rules:
-- 
2.26.2

Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

[Thomas Huth]

On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
> While we are interested in building docker images in different
> branches, it only makes sense to push 'latest' to the registry
> when this is the project default branch (usually 'master').
> 
> Else when pushing different branches concurrently we might have
> inconsistent image state between branches.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>   .gitlab-ci.d/containers.yml | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
> index 90fac85ce46..52a915f4141 100644
> --- a/.gitlab-ci.d/containers.yml
> +++ b/.gitlab-ci.d/containers.yml
> @@ -17,7 +17,7 @@
>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>             -r $CI_REGISTRY_IMAGE
>       - docker tag "qemu/$NAME" "$TAG"
> -    - docker push "$TAG"
> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push "$TAG"

So does that mean that the following stages in the CI (i.e. build, test) are 
only always (i.e. also for the non-master branches) going to use containers 
that have been build on the master branch?

  Thomas

Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

[Philippe Mathieu-Daudé]

On 2/16/21 7:55 AM, Thomas Huth wrote:
> On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
>> While we are interested in building docker images in different
>> branches, it only makes sense to push 'latest' to the registry
>> when this is the project default branch (usually 'master').
>>
>> Else when pushing different branches concurrently we might have
>> inconsistent image state between branches.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>   .gitlab-ci.d/containers.yml | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
>> index 90fac85ce46..52a915f4141 100644
>> --- a/.gitlab-ci.d/containers.yml
>> +++ b/.gitlab-ci.d/containers.yml
>> @@ -17,7 +17,7 @@
>>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>>             -r $CI_REGISTRY_IMAGE
>>       - docker tag "qemu/$NAME" "$TAG"
>> -    - docker push "$TAG"
>> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push
>> "$TAG"
> 
> So does that mean that the following stages in the CI (i.e. build, test)
> are only always (i.e. also for the non-master branches) going to use
> containers that have been build on the master branch?

Hmm good point. Should we use "$CI_COMMIT_BRANCH" instead of "latest"?

Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

[Daniel P. Berrangé]

On Tue, Feb 16, 2021 at 08:05:09AM +0100, Philippe Mathieu-Daudé wrote:
> On 2/16/21 7:55 AM, Thomas Huth wrote:
> > On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
> >> While we are interested in building docker images in different
> >> branches, it only makes sense to push 'latest' to the registry
> >> when this is the project default branch (usually 'master').
> >>
> >> Else when pushing different branches concurrently we might have
> >> inconsistent image state between branches.
> >>
> >> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> >> ---
> >>   .gitlab-ci.d/containers.yml | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
> >> index 90fac85ce46..52a915f4141 100644
> >> --- a/.gitlab-ci.d/containers.yml
> >> +++ b/.gitlab-ci.d/containers.yml
> >> @@ -17,7 +17,7 @@
> >>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
> >>             -r $CI_REGISTRY_IMAGE
> >>       - docker tag "qemu/$NAME" "$TAG"
> >> -    - docker push "$TAG"
> >> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push
> >> "$TAG"
> > 
> > So does that mean that the following stages in the CI (i.e. build, test)
> > are only always (i.e. also for the non-master branches) going to use
> > containers that have been build on the master branch?
> 
> Hmm good point. Should we use "$CI_COMMIT_BRANCH" instead of "latest"?

This doesn't work because git branch names are not guaranteed to be
valid docker tag names. Please see my thread here:

  https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg03535.html

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

[Philippe Mathieu-Daudé]

On 2/16/21 10:52 AM, Daniel P. Berrangé wrote:
> On Tue, Feb 16, 2021 at 08:05:09AM +0100, Philippe Mathieu-Daudé wrote:
>> On 2/16/21 7:55 AM, Thomas Huth wrote:
>>> On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
>>>> While we are interested in building docker images in different
>>>> branches, it only makes sense to push 'latest' to the registry
>>>> when this is the project default branch (usually 'master').
>>>>
>>>> Else when pushing different branches concurrently we might have
>>>> inconsistent image state between branches.
>>>>
>>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>>>> ---
>>>>   .gitlab-ci.d/containers.yml | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
>>>> index 90fac85ce46..52a915f4141 100644
>>>> --- a/.gitlab-ci.d/containers.yml
>>>> +++ b/.gitlab-ci.d/containers.yml
>>>> @@ -17,7 +17,7 @@
>>>>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>>>>             -r $CI_REGISTRY_IMAGE
>>>>       - docker tag "qemu/$NAME" "$TAG"
>>>> -    - docker push "$TAG"
>>>> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push
>>>> "$TAG"
>>>
>>> So does that mean that the following stages in the CI (i.e. build, test)
>>> are only always (i.e. also for the non-master branches) going to use
>>> containers that have been build on the master branch?
>>
>> Hmm good point. Should we use "$CI_COMMIT_BRANCH" instead of "latest"?
> 
> This doesn't work because git branch names are not guaranteed to be
> valid docker tag names. Please see my thread here:
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg03535.html

Indeed this thread describes the problem I am having. I'll see if there
are follow up on your thread.

Another way of fixing this is to find a way to only allow 1 pipeline at
a time. I haven't tried it yet because I don't like the idea of making
CI slower, but this is still better than debugging inconsistent
pipelines when Docker images are rebuilt.

Regards,

Phil.