[PATCH 1/2] ci: make get-binary-toolchain run per build

[PATCH 1/2] ci: make get-binary-toolchain run per build

[Jon Mason]

In a distributed, non-homogeneous CI setup, the binary-toolchain setup
script might not run on the machine that needs the toolchains.  Make
this per-build and it will always be there, at the expense of running on
builds that might not need it (though it still should be fast).

Signed-off-by: Jon Mason <jon.mason@arm.com>
---
 .gitlab-ci.yml           | 9 ++-------
 ci/get-binary-toolchains | 2 +-
 2 files changed, 3 insertions(+), 8 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 697603b..ab254f6 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -24,6 +24,8 @@ stages:
     - echo DL_DIR = $DL_DIR
     - rm -rf $KAS_WORK_DIR
     - mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR
+    # Must do this here, as it's the only way to make sure the toolhchain is installed on the same builder
+    - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR
 
 # Generalised fragment to do a Kas build
 .build:
@@ -43,13 +45,6 @@ update-repos:
   script:
   - flock --verbose --timeout 60 $KAS_REPO_REF_DIR ./ci/update-repos
 
-get-binary-toolchains:
-  extends: .setup
-  stage: prep
-  script:
-    - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR
-
-
 #
 # Bootstrap stage, bootstrap and machine coverage
 #
diff --git a/ci/get-binary-toolchains b/ci/get-binary-toolchains
index 6fa7ff4..fae3b01 100755
--- a/ci/get-binary-toolchains
+++ b/ci/get-binary-toolchains
@@ -7,7 +7,7 @@ VER="10.2-2020.11"
 DOWNLOAD_DIR=$1
 TOOLCHAIN_DIR=$2
 
-# These should be already created by .bitlab-ci.yml, but do here if run outside of that env
+# These should be already created by .gitlab-ci.yml, but do here if run outside of that env
 mkdir -p $DOWNLOAD_DIR $TOOLCHAIN_DIR
 
 if [ $HOST_ARCH = "aarch64" ]; then
-- 
2.17.1

[PATCH 2/2] ci: update binary gcc version to 10.3

[Jon Mason]

Signed-off-by: Jon Mason <jon.mason@arm.com>
---
 ci/get-binary-toolchains | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/get-binary-toolchains b/ci/get-binary-toolchains
index fae3b01..d1072e2 100755
--- a/ci/get-binary-toolchains
+++ b/ci/get-binary-toolchains
@@ -2,7 +2,7 @@
 set -u
 
 HOST_ARCH=$(uname -m)
-VER="10.2-2020.11"
+VER="10.3-2021.07"
 
 DOWNLOAD_DIR=$1
 TOOLCHAIN_DIR=$2
-- 
2.17.1

Re: [meta-arm] [PATCH 1/2] ci: make get-binary-toolchain run per build

[Sumit Garg]

On Mon, 13 Sept 2021 at 23:14, Jon Mason <jon.mason@arm.com> wrote:
>
> In a distributed, non-homogeneous CI setup, the binary-toolchain setup
> script might not run on the machine that needs the toolchains.  Make
> this per-build and it will always be there, at the expense of running on
> builds that might not need it (though it still should be fast).
>
> Signed-off-by: Jon Mason <jon.mason@arm.com>
> ---
>  .gitlab-ci.yml           | 9 ++-------
>  ci/get-binary-toolchains | 2 +-
>  2 files changed, 3 insertions(+), 8 deletions(-)
>

Thanks John for the updates.

Acked-by: Sumit Garg <sumit.garg@linaro.org>

-Sumit

> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 697603b..ab254f6 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -24,6 +24,8 @@ stages:
>      - echo DL_DIR = $DL_DIR
>      - rm -rf $KAS_WORK_DIR
>      - mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR
> +    # Must do this here, as it's the only way to make sure the toolhchain is installed on the same builder
> +    - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR
>
>  # Generalised fragment to do a Kas build
>  .build:
> @@ -43,13 +45,6 @@ update-repos:
>    script:
>    - flock --verbose --timeout 60 $KAS_REPO_REF_DIR ./ci/update-repos
>
> -get-binary-toolchains:
> -  extends: .setup
> -  stage: prep
> -  script:
> -    - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR
> -
> -
>  #
>  # Bootstrap stage, bootstrap and machine coverage
>  #
> diff --git a/ci/get-binary-toolchains b/ci/get-binary-toolchains
> index 6fa7ff4..fae3b01 100755
> --- a/ci/get-binary-toolchains
> +++ b/ci/get-binary-toolchains
> @@ -7,7 +7,7 @@ VER="10.2-2020.11"
>  DOWNLOAD_DIR=$1
>  TOOLCHAIN_DIR=$2
>
> -# These should be already created by .bitlab-ci.yml, but do here if run outside of that env
> +# These should be already created by .gitlab-ci.yml, but do here if run outside of that env
>  mkdir -p $DOWNLOAD_DIR $TOOLCHAIN_DIR
>
>  if [ $HOST_ARCH = "aarch64" ]; then
> --
> 2.17.1
>
>
> 
>

Re: [meta-arm] [PATCH 1/2] ci: make get-binary-toolchain run per build

[Jon Mason]

On Tue, Sep 14, 2021 at 1:23 AM Sumit Garg <sumit.garg@linaro.org> wrote:
>
> On Mon, 13 Sept 2021 at 23:14, Jon Mason <jon.mason@arm.com> wrote:
> >
> > In a distributed, non-homogeneous CI setup, the binary-toolchain setup
> > script might not run on the machine that needs the toolchains.  Make
> > this per-build and it will always be there, at the expense of running on
> > builds that might not need it (though it still should be fast).
> >
> > Signed-off-by: Jon Mason <jon.mason@arm.com>
> > ---
> >  .gitlab-ci.yml           | 9 ++-------
> >  ci/get-binary-toolchains | 2 +-
> >  2 files changed, 3 insertions(+), 8 deletions(-)
> >
>
> Thanks John for the updates.
>
> Acked-by: Sumit Garg <sumit.garg@linaro.org>

Unfortunately, I found a bug when running against multiple branches
simultaneously.  v2 coming shortly :)

Thanks,
Jon

>
> -Sumit
>
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index 697603b..ab254f6 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -24,6 +24,8 @@ stages:
> >      - echo DL_DIR = $DL_DIR
> >      - rm -rf $KAS_WORK_DIR
> >      - mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR
> > +    # Must do this here, as it's the only way to make sure the toolhchain is installed on the same builder
> > +    - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR
> >
> >  # Generalised fragment to do a Kas build
> >  .build:
> > @@ -43,13 +45,6 @@ update-repos:
> >    script:
> >    - flock --verbose --timeout 60 $KAS_REPO_REF_DIR ./ci/update-repos
> >
> > -get-binary-toolchains:
> > -  extends: .setup
> > -  stage: prep
> > -  script:
> > -    - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR
> > -
> > -
> >  #
> >  # Bootstrap stage, bootstrap and machine coverage
> >  #
> > diff --git a/ci/get-binary-toolchains b/ci/get-binary-toolchains
> > index 6fa7ff4..fae3b01 100755
> > --- a/ci/get-binary-toolchains
> > +++ b/ci/get-binary-toolchains
> > @@ -7,7 +7,7 @@ VER="10.2-2020.11"
> >  DOWNLOAD_DIR=$1
> >  TOOLCHAIN_DIR=$2
> >
> > -# These should be already created by .bitlab-ci.yml, but do here if run outside of that env
> > +# These should be already created by .gitlab-ci.yml, but do here if run outside of that env
> >  mkdir -p $DOWNLOAD_DIR $TOOLCHAIN_DIR
> >
> >  if [ $HOST_ARCH = "aarch64" ]; then
> > --
> > 2.17.1
> >
> >
> >
> >
>
> 
>