[xenomai-images][PATCH v2 0/4] Incorporate Comments and fix AWS upload

[xenomai-images][PATCH v2 0/4] Incorporate Comments and fix AWS upload

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

 Incorporate the comments from patchset[1] set by uploading a test-report.xml
 to gitlab artifacts[2] and replace sed commands with envsubst[3].


Changes V2:
 - use gitlab reports to display a test summary,  e.g. [4]
 - simplify if dtb exists check

[1]: https://xenomai.org/pipermail/xenomai/2021-July/045823.html
[2]: https://xenomai.org/pipermail/xenomai/2021-July/045833.html
[3]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html
[4]: https://gitlab.com/Quirin.Gy/xenomai-images-test/-/pipelines/336323070/test_report

Quirin Gylstorff (4):
  scripts/deploy_to_aws: Correct upload of dtb
  scripts/run-lava-tests: upload lava test report
  scripts/run-lava-tests: replace sed with envsubst
  scripts/run-lava-tests: parse test-report for errors

 ci/gitlab-ci-base.yml     |  6 ++++++
 scripts/deploy_to_aws.sh  |  2 +-
 scripts/run-lava-tests.sh | 23 +++++++++++++++--------
 3 files changed, 22 insertions(+), 9 deletions(-)

-- 
2.20.1

[xenomai-images][PATCH v2 1/4] scripts/deploy_to_aws: Correct upload of dtb

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

If a dtb exists it was not uploaded correctly.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 scripts/deploy_to_aws.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/deploy_to_aws.sh b/scripts/deploy_to_aws.sh
index 814aec7..5a5b29a 100755
--- a/scripts/deploy_to_aws.sh
+++ b/scripts/deploy_to_aws.sh
@@ -40,7 +40,7 @@ fi
 image=$(ls "${images_dir}/${target}/${isar_base_name}".*.gz)
 aws s3 cp "${aws_args}" "${image}" "${deploy_dir}/$(basename "${image}")"
 # DTB
-dtb="$(ls "${images_dir}/${target}/*.dtb" 2> /dev/null)"
+dtb=$(ls "${images_dir}/${target}/"*.dtb 2> /dev/null || true )
 if [ -n "${dtb}" ] ; then
     aws s3 cp "${aws_args}" "${dtb}" "${deploy_dir}/$(basename "${dtb}")"
 fi
-- 
2.20.1

[xenomai-images][PATCH v2 2/4] scripts/run-lava-tests: upload lava test report

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

upload a test report to gitlab to increase tracing of build
errors.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 ci/gitlab-ci-base.yml     | 6 ++++++
 scripts/run-lava-tests.sh | 9 +++++++++
 2 files changed, 15 insertions(+)

diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
index 96b9c2c..7f245e3 100644
--- a/ci/gitlab-ci-base.yml
+++ b/ci/gitlab-ci-base.yml
@@ -57,6 +57,12 @@ default:
     - scripts/run-lava-tests.sh ${TARGET}
   variables:
     BUILD_JOB_NAME: "build-${BUILD_IDENTIFIER}:${TARGET}${TARGET_EXTENSION}"
+  artifacts:
+    when: always
+    reports:
+      junit:
+        - report.xml
+    expire_in: 1 week
   only:
     variables:
       - $LAVA_TESTS_ENABLED == "true"
diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index 72138c3..e398afd 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -74,8 +74,17 @@ echo "submit lava job"
 test_id=$(lavacli jobs submit "${template}")
 lavacli jobs logs "${test_id}"
 lavacli results "${test_id}"
+
+# add test report download
+api="${lava_master_uri}/api/v0.2/jobs/${test_id}/junit"
+auth="?token=${LAVA_MASTER_TOKEN}"
+url="${api}/${auth}"
+
+curl --silent "${url}" -o report.xml
+
 # change return code to generate a error in gitlab-ci if a test is failed
 number_of_fails="$(lavacli results "${test_id}" | grep -c fail )"
+
 if [ -n "${ssh_pid}" ];then
    kill "${ssh_pid}"
 fi
-- 
2.20.1

[xenomai-images][PATCH v2 3/4] scripts/run-lava-tests: replace sed with envsubst

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This patch incorporates a comment from the mailing list[1].

It simplifies the script.

[1]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 scripts/run-lava-tests.sh | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index e398afd..44e02fc 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -50,24 +50,22 @@ echo "connect to lava server: ${lava_master_uri}"
 
 # connect to lava master
 lavacli identities add --token "${LAVA_MASTER_TOKEN}" --uri "${lava_master_uri}" --username "${LAVA_MASTER_ACCOUNT}" default
+
 #generate lava job description from template
 if [ -n "${USE_GITLAB_ARTIFACTS}" ]; then
     DEPLOY_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/jobs/${BUILD_JOB_ID}/artifacts/build/tmp/deploy/images/${TARGET}"
 else
     DEPLOY_URL="${artifact_url}/${CI_PIPELINE_ID}/${DEPLOY_DIR_EXTENSION}"
 fi
-
+export DEPLOY_URL
 echo "Deploy artifacts from '${artifact_url}'"
 
 job_template_path="${JOB_TEMPLATE_PATH:-tests/jobs/xenomai}"
 tmp_dir=$(mktemp -d)
 template="${tmp_dir}/job_${TARGET}_${CI_PIPELINE_ID}.yml"
-cp "${job_template_path}-${TARGET}.yml" "${template}"
-sed -i "s|\${TARGET}|${TARGET}|g" "$template"
-sed -i "s|\${BUILD_ARCH}|${BUILD_ARCH}|g" "$template"
-sed -i "s|\${DEPLOY_URL}|${DEPLOY_URL}|g" "$template"
-sed -i "s|\${ISAR_IMAGE}|${ISAR_IMAGE}|g" "$template"
-sed -i "s|\${ISAR_DISTRIBUTION}|${ISAR_DISTRIBUTION}|g" "$template"
+
+cat "${job_template_path}-${TARGET}.yml" | envsubst > "$template"
+
 
 echo "submit lava job"
 
-- 
2.20.1

[xenomai-images][PATCH v2 4/4] scripts/run-lava-tests: parse test-report for errors

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

The total number of errors is easier collected by parsing
the LAVA API output as it already contains the information
requested. The lavacli output does not contain the sum of errors.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 scripts/run-lava-tests.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index 44e02fc..7972279 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -81,7 +81,7 @@ url="${api}/${auth}"
 curl --silent "${url}" -o report.xml
 
 # change return code to generate a error in gitlab-ci if a test is failed
-number_of_fails="$(lavacli results "${test_id}" | grep -c fail )"
+number_of_fails="$(grep -o "errors=\"[0-9]*\"" report.xml | head -1  | cut -d\" -f2 )"
 
 if [ -n "${ssh_pid}" ];then
    kill "${ssh_pid}"
-- 
2.20.1

Re: [xenomai-images][PATCH v2 0/4] Incorporate Comments and fix AWS upload

[Bezdeka, Florian]

On Wed, 2021-07-14 at 09:41 +0200, Q. Gylstorff via Xenomai wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
>  Incorporate the comments from patchset[1] set by uploading a test-report.xml
>  to gitlab artifacts[2] and replace sed commands with envsubst[3].
> 
> 
> Changes V2:
>  - use gitlab reports to display a test summary,  e.g. [4]
>  - simplify if dtb exists check
> 
> [1]: https://xenomai.org/pipermail/xenomai/2021-July/045823.html
> [2]: https://xenomai.org/pipermail/xenomai/2021-July/045833.html
> [3]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html
> [4]: https://gitlab.com/Quirin.Gy/xenomai-images-test/-/pipelines/336323070/test_report

[4] looks like I would expect it. Thanks!

> 
> Quirin Gylstorff (4):
>   scripts/deploy_to_aws: Correct upload of dtb
>   scripts/run-lava-tests: upload lava test report
>   scripts/run-lava-tests: replace sed with envsubst
>   scripts/run-lava-tests: parse test-report for errors
> 
>  ci/gitlab-ci-base.yml     |  6 ++++++
>  scripts/deploy_to_aws.sh  |  2 +-
>  scripts/run-lava-tests.sh | 23 +++++++++++++++--------
>  3 files changed, 22 insertions(+), 9 deletions(-)
> 

Re: [xenomai-images][PATCH v2 0/4] Incorporate Comments and fix AWS upload

[Jan Kiszka]

On 14.07.21 09:41, Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
>  Incorporate the comments from patchset[1] set by uploading a test-report.xml
>  to gitlab artifacts[2] and replace sed commands with envsubst[3].
> 
> 
> Changes V2:
>  - use gitlab reports to display a test summary,  e.g. [4]
>  - simplify if dtb exists check
> 
> [1]: https://xenomai.org/pipermail/xenomai/2021-July/045823.html
> [2]: https://xenomai.org/pipermail/xenomai/2021-July/045833.html
> [3]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html
> [4]: https://gitlab.com/Quirin.Gy/xenomai-images-test/-/pipelines/336323070/test_report
> 
> Quirin Gylstorff (4):
>   scripts/deploy_to_aws: Correct upload of dtb
>   scripts/run-lava-tests: upload lava test report
>   scripts/run-lava-tests: replace sed with envsubst
>   scripts/run-lava-tests: parse test-report for errors
> 
>  ci/gitlab-ci-base.yml     |  6 ++++++
>  scripts/deploy_to_aws.sh  |  2 +-
>  scripts/run-lava-tests.sh | 23 +++++++++++++++--------
>  3 files changed, 22 insertions(+), 9 deletions(-)
> 

Thanks, all applied to next. Let's see if we can get the new lab setup
started...

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux