[xenomai-images][PATCH 0/4] Incorporate Comments and fix AWS upload

[xenomai-images][PATCH 0/4] Incorporate Comments and fix AWS upload

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

 Incorporate the comments from patchset[1] set by uploading a test-report.xml
 to gitlab artifacts[2] and replace sed commands with envsubst[3].

[1]: https://xenomai.org/pipermail/xenomai/2021-July/045823.html
[2]: https://xenomai.org/pipermail/xenomai/2021-July/045833.html
[3]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html

Quirin Gylstorff (4):
  scripts/deploy_to_aws: Correct upload of dtb
  scripts/run-lava-tests: upload lava test report
  scripts/run-lava-tests: replace sed with envsubst
  scripts/run-lava-tests: parse test-report for errors

 ci/gitlab-ci-base.yml     |  5 +++++
 scripts/deploy_to_aws.sh  |  4 ++--
 scripts/run-lava-tests.sh | 26 ++++++++++++++++++--------
 3 files changed, 25 insertions(+), 10 deletions(-)

-- 
2.20.1

[xenomai-images][PATCH 1/4] scripts/deploy_to_aws: Correct upload of dtb

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

If a dtb exists it was not uploaded correctly.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 scripts/deploy_to_aws.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/deploy_to_aws.sh b/scripts/deploy_to_aws.sh
index 814aec7..02f7aeb 100755
--- a/scripts/deploy_to_aws.sh
+++ b/scripts/deploy_to_aws.sh
@@ -40,7 +40,7 @@ fi
 image=$(ls "${images_dir}/${target}/${isar_base_name}".*.gz)
 aws s3 cp "${aws_args}" "${image}" "${deploy_dir}/$(basename "${image}")"
 # DTB
-dtb="$(ls "${images_dir}/${target}/*.dtb" 2> /dev/null)"
-if [ -n "${dtb}" ] ; then
+if ls "${images_dir}/${target}/"*.dtb ; then
+    dtb=$(ls "${images_dir}/${target}/"*.dtb )
     aws s3 cp "${aws_args}" "${dtb}" "${deploy_dir}/$(basename "${dtb}")"
 fi
-- 
2.20.1

[xenomai-images][PATCH 2/4] scripts/run-lava-tests: upload lava test report

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

upload a test report to gitlab to increase tracing of build
errors.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 ci/gitlab-ci-base.yml     |  5 +++++
 scripts/run-lava-tests.sh | 12 ++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
index 96b9c2c..e14cef5 100644
--- a/ci/gitlab-ci-base.yml
+++ b/ci/gitlab-ci-base.yml
@@ -57,6 +57,11 @@ default:
     - scripts/run-lava-tests.sh ${TARGET}
   variables:
     BUILD_JOB_NAME: "build-${BUILD_IDENTIFIER}:${TARGET}${TARGET_EXTENSION}"
+  artifacts:
+    when: always
+    paths:
+    - test-report.xml
+    expire_in: 1 week
   only:
     variables:
       - $LAVA_TESTS_ENABLED == "true"
diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index 72138c3..a69bd87 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -9,6 +9,7 @@
 #
 # SPDX-License-Identifier: MIT
 #
+
 TARGET=$1
 if [ -z "${TARGET}" ]; then
     echo "no target was given"
@@ -72,10 +73,21 @@ sed -i "s|\${ISAR_DISTRIBUTION}|${ISAR_DISTRIBUTION}|g" "$template"
 echo "submit lava job"
 
 test_id=$(lavacli jobs submit "${template}")
+
+echo "get logs from  lava job ${test_id}"
 lavacli jobs logs "${test_id}"
 lavacli results "${test_id}"
+
+# add test report download
+api="${lava_master_uri}/api/v0.2/jobs/${test_id}/junit"
+auth="?token=${LAVA_MASTER_TOKEN}"
+url="${api}/${auth}"
+
+curl --silent "${url}" -o test-report.xml
+
 # change return code to generate a error in gitlab-ci if a test is failed
 number_of_fails="$(lavacli results "${test_id}" | grep -c fail )"
+
 if [ -n "${ssh_pid}" ];then
    kill "${ssh_pid}"
 fi
-- 
2.20.1

[xenomai-images][PATCH 3/4] scripts/run-lava-tests: replace sed with envsubst

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

This patch incorporates a comment from the mailing list[1].

It simplifies the script.

[1]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 scripts/run-lava-tests.sh | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index a69bd87..c787160 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -51,24 +51,22 @@ echo "connect to lava server: ${lava_master_uri}"
 
 # connect to lava master
 lavacli identities add --token "${LAVA_MASTER_TOKEN}" --uri "${lava_master_uri}" --username "${LAVA_MASTER_ACCOUNT}" default
+
 #generate lava job description from template
 if [ -n "${USE_GITLAB_ARTIFACTS}" ]; then
     DEPLOY_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/jobs/${BUILD_JOB_ID}/artifacts/build/tmp/deploy/images/${TARGET}"
 else
     DEPLOY_URL="${artifact_url}/${CI_PIPELINE_ID}/${DEPLOY_DIR_EXTENSION}"
 fi
-
+export DEPLOY_URL
 echo "Deploy artifacts from '${artifact_url}'"
 
 job_template_path="${JOB_TEMPLATE_PATH:-tests/jobs/xenomai}"
 tmp_dir=$(mktemp -d)
 template="${tmp_dir}/job_${TARGET}_${CI_PIPELINE_ID}.yml"
-cp "${job_template_path}-${TARGET}.yml" "${template}"
-sed -i "s|\${TARGET}|${TARGET}|g" "$template"
-sed -i "s|\${BUILD_ARCH}|${BUILD_ARCH}|g" "$template"
-sed -i "s|\${DEPLOY_URL}|${DEPLOY_URL}|g" "$template"
-sed -i "s|\${ISAR_IMAGE}|${ISAR_IMAGE}|g" "$template"
-sed -i "s|\${ISAR_DISTRIBUTION}|${ISAR_DISTRIBUTION}|g" "$template"
+
+cat "${job_template_path}-${TARGET}.yml" | envsubst > "$template"
+
 
 echo "submit lava job"
 
-- 
2.20.1

[xenomai-images][PATCH 4/4] scripts/run-lava-tests: parse test-report for errors

[Q. Gylstorff]

From: Quirin Gylstorff <quirin.gylstorff@siemens.com>

The total number of errors is easier collected by parsing
the LAVA API output as it already contains the information
requested. The lavacli output does not contain the sum of errors.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
 scripts/run-lava-tests.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index c787160..0c39340 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -84,7 +84,7 @@ url="${api}/${auth}"
 curl --silent "${url}" -o test-report.xml
 
 # change return code to generate a error in gitlab-ci if a test is failed
-number_of_fails="$(lavacli results "${test_id}" | grep -c fail )"
+number_of_fails="$(grep -o "errors=\"[0-9]*\"" test-report.xml | head -1  | cut -d\" -f2 )"
 
 if [ -n "${ssh_pid}" ];then
    kill "${ssh_pid}"
-- 
2.20.1

Re: [xenomai-images][PATCH 0/4] Incorporate Comments and fix AWS upload

[Bezdeka, Florian]

On Tue, 2021-07-13 at 16:51 +0200, Q. Gylstorff via Xenomai wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
>  Incorporate the comments from patchset[1] set by uploading a test-report.xml
>  to gitlab artifacts[2] and replace sed commands with envsubst[3].
> 
> [1]: https://xenomai.org/pipermail/xenomai/2021-July/045823.html
> [2]: https://xenomai.org/pipermail/xenomai/2021-July/045833.html
> [3]: https://xenomai.org/pipermail/xenomai/2021-July/045830.html
> 

Thanks for taking care. I had that on my list, but that is even better
;-)

> Quirin Gylstorff (4):
>   scripts/deploy_to_aws: Correct upload of dtb
>   scripts/run-lava-tests: upload lava test report
>   scripts/run-lava-tests: replace sed with envsubst
>   scripts/run-lava-tests: parse test-report for errors
> 
>  ci/gitlab-ci-base.yml     |  5 +++++
>  scripts/deploy_to_aws.sh  |  4 ++--
>  scripts/run-lava-tests.sh | 26 ++++++++++++++++++--------
>  3 files changed, 25 insertions(+), 10 deletions(-)
> 

Re: [xenomai-images][PATCH 2/4] scripts/run-lava-tests: upload lava test report

[Bezdeka, Florian]

On Tue, 2021-07-13 at 16:51 +0200, Q. Gylstorff via Xenomai wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> upload a test report to gitlab to increase tracing of build
> errors.
> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> ---
>  ci/gitlab-ci-base.yml     |  5 +++++
>  scripts/run-lava-tests.sh | 12 ++++++++++++
>  2 files changed, 17 insertions(+)
> 
> diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
> index 96b9c2c..e14cef5 100644
> --- a/ci/gitlab-ci-base.yml
> +++ b/ci/gitlab-ci-base.yml
> @@ -57,6 +57,11 @@ default:
>      - scripts/run-lava-tests.sh ${TARGET}
>    variables:
>      BUILD_JOB_NAME: "build-${BUILD_IDENTIFIER}:${TARGET}${TARGET_EXTENSION}"
> +  artifacts:
> +    when: always
> +    paths:
> +    - test-report.xml

Marking this file as junit test report should simplify the test result
integration in gitlab, but I'm not 100% sure this is still necessary.

artifacts:
  reports:
    junit: test-report.xml

^^ Something like that should do it.

> +    expire_in: 1 week
>    only:
>      variables:
>        - $LAVA_TESTS_ENABLED == "true"
> diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
> index 72138c3..a69bd87 100755
> --- a/scripts/run-lava-tests.sh
> +++ b/scripts/run-lava-tests.sh
> @@ -9,6 +9,7 @@
>  #
>  # SPDX-License-Identifier: MIT
>  #
> +
>  TARGET=$1
>  if [ -z "${TARGET}" ]; then
>      echo "no target was given"
> @@ -72,10 +73,21 @@ sed -i "s|\${ISAR_DISTRIBUTION}|${ISAR_DISTRIBUTION}|g" "$template"
>  echo "submit lava job"
>  
> 
> 
> 
>  test_id=$(lavacli jobs submit "${template}")
> +
> +echo "get logs from  lava job ${test_id}"
>  lavacli jobs logs "${test_id}"
>  lavacli results "${test_id}"
> +
> +# add test report download
> +api="${lava_master_uri}/api/v0.2/jobs/${test_id}/junit"
> +auth="?token=${LAVA_MASTER_TOKEN}"
> +url="${api}/${auth}"
> +
> +curl --silent "${url}" -o test-report.xml
> +
>  # change return code to generate a error in gitlab-ci if a test is failed
>  number_of_fails="$(lavacli results "${test_id}" | grep -c fail )"
> +
>  if [ -n "${ssh_pid}" ];then
>     kill "${ssh_pid}"
>  fi

Re: [xenomai-images][PATCH 1/4] scripts/deploy_to_aws: Correct upload of dtb

[Jan Kiszka]

On 13.07.21 16:51, Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> If a dtb exists it was not uploaded correctly.
> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> ---
>  scripts/deploy_to_aws.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/deploy_to_aws.sh b/scripts/deploy_to_aws.sh
> index 814aec7..02f7aeb 100755
> --- a/scripts/deploy_to_aws.sh
> +++ b/scripts/deploy_to_aws.sh
> @@ -40,7 +40,7 @@ fi
>  image=$(ls "${images_dir}/${target}/${isar_base_name}".*.gz)
>  aws s3 cp "${aws_args}" "${image}" "${deploy_dir}/$(basename "${image}")"
>  # DTB
> -dtb="$(ls "${images_dir}/${target}/*.dtb" 2> /dev/null)"

Wasn't that quoting the actual issue? The old pattern of setting dtb
first was better than running ls twice.

Jan

> -if [ -n "${dtb}" ] ; then
> +if ls "${images_dir}/${target}/"*.dtb ; then
> +    dtb=$(ls "${images_dir}/${target}/"*.dtb )
>      aws s3 cp "${aws_args}" "${dtb}" "${deploy_dir}/$(basename "${dtb}")"
>  fi
> 

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [xenomai-images][PATCH 2/4] scripts/run-lava-tests: upload lava test report

[Jan Kiszka]

On 13.07.21 16:58, Bezdeka, Florian (T RDA IOT SES-DE) wrote:
> On Tue, 2021-07-13 at 16:51 +0200, Q. Gylstorff via Xenomai wrote:
>> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>>
>> upload a test report to gitlab to increase tracing of build
>> errors.
>>
>> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
>> ---
>>  ci/gitlab-ci-base.yml     |  5 +++++
>>  scripts/run-lava-tests.sh | 12 ++++++++++++
>>  2 files changed, 17 insertions(+)
>>
>> diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
>> index 96b9c2c..e14cef5 100644
>> --- a/ci/gitlab-ci-base.yml
>> +++ b/ci/gitlab-ci-base.yml
>> @@ -57,6 +57,11 @@ default:
>>      - scripts/run-lava-tests.sh ${TARGET}
>>    variables:
>>      BUILD_JOB_NAME: "build-${BUILD_IDENTIFIER}:${TARGET}${TARGET_EXTENSION}"
>> +  artifacts:
>> +    when: always
>> +    paths:
>> +    - test-report.xml
> 
> Marking this file as junit test report should simplify the test result
> integration in gitlab, but I'm not 100% sure this is still necessary.
> 
> artifacts:
>   reports:
>     junit: test-report.xml
> 
> ^^ Something like that should do it.
> 

That's what the docu suggests as well:
https://docs.gitlab.com/ee/ci/unit_test_reports.html

Jan

>> +    expire_in: 1 week
>>    only:
>>      variables:
>>        - $LAVA_TESTS_ENABLED == "true"
>> diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
>> index 72138c3..a69bd87 100755
>> --- a/scripts/run-lava-tests.sh
>> +++ b/scripts/run-lava-tests.sh
>> @@ -9,6 +9,7 @@
>>  #
>>  # SPDX-License-Identifier: MIT
>>  #
>> +
>>  TARGET=$1
>>  if [ -z "${TARGET}" ]; then
>>      echo "no target was given"
>> @@ -72,10 +73,21 @@ sed -i "s|\${ISAR_DISTRIBUTION}|${ISAR_DISTRIBUTION}|g" "$template"
>>  echo "submit lava job"
>>
>>
>>
>>
>>  test_id=$(lavacli jobs submit "${template}")
>> +
>> +echo "get logs from  lava job ${test_id}"
>>  lavacli jobs logs "${test_id}"
>>  lavacli results "${test_id}"
>> +
>> +# add test report download
>> +api="${lava_master_uri}/api/v0.2/jobs/${test_id}/junit"
>> +auth="?token=${LAVA_MASTER_TOKEN}"
>> +url="${api}/${auth}"
>> +
>> +curl --silent "${url}" -o test-report.xml
>> +
>>  # change return code to generate a error in gitlab-ci if a test is failed
>>  number_of_fails="$(lavacli results "${test_id}" | grep -c fail )"
>> +
>>  if [ -n "${ssh_pid}" ];then
>>     kill "${ssh_pid}"
>>  fi
> 

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux