[XEN PATCH v2 0/2] automation: avoid unnecessary analyses

[XEN PATCH v2 0/2] automation: avoid unnecessary analyses

[Simone Ballarin]

This series aims to reduce the analyses performed by the ECLAIR
runner by avoiding some branches already excluded by other jobs
and requiring analyses on people/.* to be manually triggered.

---
Changes in v2:
- move some changes from 1/2 to 2/2.
- remove useless except clause in .yocto-test
- avoid ECLAIR jobs if the WTOKEN variable is not defined

Simone Ballarin (2):
  automation/eclair: avoid unintentional ECLAIR analysis
  automation: avoid pipelines on specific branches

 .gitlab-ci.yml                    |  6 ++++++
 automation/gitlab-ci/analyze.yaml | 22 +++++++++++++++++-----
 automation/gitlab-ci/build.yaml   | 11 -----------
 automation/gitlab-ci/test.yaml    |  5 -----
 automation/scripts/eclair         |  5 -----
 5 files changed, 23 insertions(+), 26 deletions(-)

-- 
2.34.1

[XEN PATCH v2 1/2] automation/eclair: avoid unintentional ECLAIR analysis

[Simone Ballarin]

With this patch, ECLAIR jobs will need to be manually
started for "people/.*" pipelines and will not be triggered
if the WTOKEN variable is missing.

This avoids occupying the runner on analyzes that might
not be used by developers.

If developers want to analyze their own repositories
they need to launch them from GitLab.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

---
Changes in v2:
- avoid ECLAIR jobs if the WTOKEN variable is not defined.
---
 automation/gitlab-ci/analyze.yaml | 22 +++++++++++++++++-----
 automation/scripts/eclair         |  5 -----
 2 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
index 4aa4abe2ee..bd9a68de31 100644
--- a/automation/gitlab-ci/analyze.yaml
+++ b/automation/gitlab-ci/analyze.yaml
@@ -18,28 +18,40 @@
       - '*.log'
     reports:
       codequality: gl-code-quality-report.json
+  rules:
+    - if: $WTOKEN == null
+      when: never
+    - when: always
   needs: []
 
-eclair-x86_64:
+.eclair-analysis:triggered:
   extends: .eclair-analysis
+  allow_failure: true
+  rules:
+    - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
+      when: manual
+    - !reference [.eclair-analysis, rules]
+
+eclair-x86_64:
+  extends: .eclair-analysis:triggered
   variables:
     LOGFILE: "eclair-x86_64.log"
     VARIANT: "X86_64"
     RULESET: "Set1"
-  allow_failure: true
 
 eclair-ARM64:
-  extends: .eclair-analysis
+  extends: .eclair-analysis:triggered
   variables:
     LOGFILE: "eclair-ARM64.log"
     VARIANT: "ARM64"
     RULESET: "Set1"
-  allow_failure: true
 
 .eclair-analysis:on-schedule:
   extends: .eclair-analysis
   rules:
-    - if: $CI_PIPELINE_SOURCE == "schedule"
+    - if: $CI_PIPELINE_SOURCE != "schedule"
+      when: never
+    - !reference [.eclair-analysis, rules]
 
 eclair-x86_64-Set1:on-schedule:
   extends: .eclair-analysis:on-schedule
diff --git a/automation/scripts/eclair b/automation/scripts/eclair
index 813a56eb6a..14e47a6f97 100755
--- a/automation/scripts/eclair
+++ b/automation/scripts/eclair
@@ -4,11 +4,6 @@ ECLAIR_ANALYSIS_DIR=automation/eclair_analysis
 ECLAIR_DIR="${ECLAIR_ANALYSIS_DIR}/ECLAIR"
 ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}")
 
-if [ -z "${WTOKEN:-}" ]; then
-    echo "Failure: the WTOKEN variable is not defined." >&2
-    exit 1
-fi
-
 "${ECLAIR_ANALYSIS_DIR}/prepare.sh" "${VARIANT}"
 
 ex=0
-- 
2.34.1

[XEN PATCH v2 2/2] automation: avoid pipelines on specific branches

[Simone Ballarin]

This patch avoids the execution of pipelines in the
following branches:
- master
- smoke
- coverirty-tested/.*
- stable-.*

The job-level exclusions have been removed as they are
pointless with this new workspace-level exclusion.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

---
Changes in v2:
- remove useless except clause in .yocto-test.
---
 .gitlab-ci.yml                  |  6 ++++++
 automation/gitlab-ci/build.yaml | 11 -----------
 automation/gitlab-ci/test.yaml  |  5 -----
 3 files changed, 6 insertions(+), 16 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ee5430b8b7..ef4484e09a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,3 +1,9 @@
+workflow:
+  rules:
+    - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/
+      when: never
+    - when: always
+
 stages:
   - analyze
   - build
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 1a4a5e490d..b633facff4 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -12,11 +12,6 @@
       - '*/*.log'
     when: always
   needs: []
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
 
 .gcc-tmpl:
   variables: &gcc
@@ -214,11 +209,6 @@
 .yocto-test:
   stage: build
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
   script:
     - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
   variables:
@@ -269,7 +259,6 @@
 .test-jobs-artifact-common:
   stage: build
   needs: []
-  except: !reference [.test-jobs-common, except]
 
 # Arm test artifacts
 
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 810631bc46..8737f367c8 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -1,11 +1,6 @@
 .test-jobs-common:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
-  except:
-    - master
-    - smoke
-    - /^coverity-tested\/.*/
-    - /^stable-.*/
 
 .arm64-test-needs: &arm64-test-needs
   - alpine-3.18-arm64-rootfs-export
-- 
2.34.1

Re: [XEN PATCH v2 1/2] automation/eclair: avoid unintentional ECLAIR analysis

[Stefano Stabellini]

On Mon, 21 Aug 2023, Simone Ballarin wrote:
> With this patch, ECLAIR jobs will need to be manually
> started for "people/.*" pipelines and will not be triggered
> if the WTOKEN variable is missing.
> 
> This avoids occupying the runner on analyzes that might
> not be used by developers.
> 
> If developers want to analyze their own repositories
> they need to launch them from GitLab.
> 
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

Great job!

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v2:
> - avoid ECLAIR jobs if the WTOKEN variable is not defined.
> ---
>  automation/gitlab-ci/analyze.yaml | 22 +++++++++++++++++-----
>  automation/scripts/eclair         |  5 -----
>  2 files changed, 17 insertions(+), 10 deletions(-)
> 
> diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml
> index 4aa4abe2ee..bd9a68de31 100644
> --- a/automation/gitlab-ci/analyze.yaml
> +++ b/automation/gitlab-ci/analyze.yaml
> @@ -18,28 +18,40 @@
>        - '*.log'
>      reports:
>        codequality: gl-code-quality-report.json
> +  rules:
> +    - if: $WTOKEN == null
> +      when: never
> +    - when: always
>    needs: []
>  
> -eclair-x86_64:
> +.eclair-analysis:triggered:
>    extends: .eclair-analysis
> +  allow_failure: true
> +  rules:
> +    - if: $WTOKEN && $CI_PROJECT_PATH =~ /^xen-project\/people\/.*$/
> +      when: manual
> +    - !reference [.eclair-analysis, rules]
> +
> +eclair-x86_64:
> +  extends: .eclair-analysis:triggered
>    variables:
>      LOGFILE: "eclair-x86_64.log"
>      VARIANT: "X86_64"
>      RULESET: "Set1"
> -  allow_failure: true
>  
>  eclair-ARM64:
> -  extends: .eclair-analysis
> +  extends: .eclair-analysis:triggered
>    variables:
>      LOGFILE: "eclair-ARM64.log"
>      VARIANT: "ARM64"
>      RULESET: "Set1"
> -  allow_failure: true
>  
>  .eclair-analysis:on-schedule:
>    extends: .eclair-analysis
>    rules:
> -    - if: $CI_PIPELINE_SOURCE == "schedule"
> +    - if: $CI_PIPELINE_SOURCE != "schedule"
> +      when: never
> +    - !reference [.eclair-analysis, rules]
>  
>  eclair-x86_64-Set1:on-schedule:
>    extends: .eclair-analysis:on-schedule
> diff --git a/automation/scripts/eclair b/automation/scripts/eclair
> index 813a56eb6a..14e47a6f97 100755
> --- a/automation/scripts/eclair
> +++ b/automation/scripts/eclair
> @@ -4,11 +4,6 @@ ECLAIR_ANALYSIS_DIR=automation/eclair_analysis
>  ECLAIR_DIR="${ECLAIR_ANALYSIS_DIR}/ECLAIR"
>  ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}")
>  
> -if [ -z "${WTOKEN:-}" ]; then
> -    echo "Failure: the WTOKEN variable is not defined." >&2
> -    exit 1
> -fi
> -
>  "${ECLAIR_ANALYSIS_DIR}/prepare.sh" "${VARIANT}"
>  
>  ex=0
> -- 
> 2.34.1
> 

Re: [XEN PATCH v2 2/2] automation: avoid pipelines on specific branches

[Stefano Stabellini]

On Mon, 21 Aug 2023, Simone Ballarin wrote:
> This patch avoids the execution of pipelines in the
> following branches:
> - master
> - smoke
> - coverirty-tested/.*
> - stable-.*
> 
> The job-level exclusions have been removed as they are
> pointless with this new workspace-level exclusion.
> 
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v2:
> - remove useless except clause in .yocto-test.
> ---
>  .gitlab-ci.yml                  |  6 ++++++
>  automation/gitlab-ci/build.yaml | 11 -----------
>  automation/gitlab-ci/test.yaml  |  5 -----
>  3 files changed, 6 insertions(+), 16 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index ee5430b8b7..ef4484e09a 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,3 +1,9 @@
> +workflow:
> +  rules:
> +    - if: $CI_COMMIT_BRANCH =~ /^(master|smoke|^coverity-tested\/.*|stable-.*)$/
> +      when: never
> +    - when: always
> +
>  stages:
>    - analyze
>    - build
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 1a4a5e490d..b633facff4 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -12,11 +12,6 @@
>        - '*/*.log'
>      when: always
>    needs: []
> -  except:
> -    - master
> -    - smoke
> -    - /^coverity-tested\/.*/
> -    - /^stable-.*/
>  
>  .gcc-tmpl:
>    variables: &gcc
> @@ -214,11 +209,6 @@
>  .yocto-test:
>    stage: build
>    image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> -  except:
> -    - master
> -    - smoke
> -    - /^coverity-tested\/.*/
> -    - /^stable-.*/
>    script:
>      - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
>    variables:
> @@ -269,7 +259,6 @@
>  .test-jobs-artifact-common:
>    stage: build
>    needs: []
> -  except: !reference [.test-jobs-common, except]
>  
>  # Arm test artifacts
>  
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 810631bc46..8737f367c8 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -1,11 +1,6 @@
>  .test-jobs-common:
>    stage: test
>    image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> -  except:
> -    - master
> -    - smoke
> -    - /^coverity-tested\/.*/
> -    - /^stable-.*/
>  
>  .arm64-test-needs: &arm64-test-needs
>    - alpine-3.18-arm64-rootfs-export
> -- 
> 2.34.1
>