[PATCH 0/3] gitlab-ci: Refactor show logs code and archive logs

[PATCH 0/3] gitlab-ci: Refactor show logs code and archive logs

[Wainer dos Santos Moschetta]

Since the Travis CI time, it has carried (commit 67892c9537d9ac500934) a python
code to print the logs of acceptance tests in case of fail. Recently Avocado
released a new plug-in (called "testlogs") which allow us to configure the
framework to show the logs of tests which finished with a given status. For
example, print the logs of tests which FAIL or CANCEL. This series is about
replacing the old implementation by that native Avocado feature, on the
GitLab CI.

I took that opportunity to also introduce a change in the configuration
to archive the tests results files on GitLab CI for 2 days, regarless the
job failed or not. If the logs are important enough to retain them, it is
possible to do so by clicking on a 'Keep' button on UI; or you can simply
download it to your workstation.

Here is an example of job that succeed:
https://gitlab.com/wainersm/qemu/-/jobs/905873563

And its results files can be found at:
https://gitlab.com/wainersm/qemu/-/jobs/905873563/artifacts/browse/build/tests/results/latest/test-results/

Then an example of job that failed (I changed a test, forcing its fail):
https://gitlab.com/wainersm/qemu/-/jobs/906089498

[1] https://github.com/avocado-framework/avocado/issues/4266

Wainer dos Santos Moschetta (3):
  tests/acceptance: Bump avocado requirements to 83.0
  gitlab-ci: Refactor code that show logs of failed acceptances
  gitlab-ci: Archive logs of acceptance tests

 .gitlab-ci.yml         | 7 ++++++-
 tests/requirements.txt | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)

-- 
2.28.0

[PATCH 1/3] tests/acceptance: Bump avocado requirements to 83.0

[Wainer dos Santos Moschetta]

To use Avocado's testlogs plug-in on CI it is required to use
its 83.0 or greater version.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
---
 tests/requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/requirements.txt b/tests/requirements.txt
index a1c631fa59..62e8ffd28c 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,5 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==81.0
+avocado-framework==83.0
 pycdlib==1.11.0
-- 
2.28.0

Re: [PATCH 1/3] tests/acceptance: Bump avocado requirements to 83.0

[Willian Rampazzo]

On Fri, Dec 11, 2020 at 3:38 PM Wainer dos Santos Moschetta
<wainersm@redhat.com> wrote:
>
> To use Avocado's testlogs plug-in on CI it is required to use
> its 83.0 or greater version.
>
> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
>  tests/requirements.txt | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

[PATCH 2/3] gitlab-ci: Refactor code that show logs of failed acceptances

[Wainer dos Santos Moschetta]

Replace the code (python) on after_script of the acceptance jobs that
is currently used to show the logs of failed tests. Instead it is used
the Avocado's testlogs plug-in which works likewise.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
---
 .gitlab-ci.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 98bff03b47..85aa20ffde 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -58,13 +58,14 @@ include:
     - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
     - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
            >> ~/.config/avocado/avocado.conf
+    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL"]'
+           >> ~/.config/avocado/avocado.conf
     - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
         du -chs ${CI_PROJECT_DIR}/avocado-cache ;
       fi
     - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
   after_script:
     - cd build
-    - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP", "CANCEL")]' | xargs cat
     - du -chs ${CI_PROJECT_DIR}/avocado-cache
 
 build-system-ubuntu:
-- 
2.28.0

Re: [PATCH 2/3] gitlab-ci: Refactor code that show logs of failed acceptances

[Willian Rampazzo]

On 12/11/20 3:38 PM, Wainer dos Santos Moschetta wrote:
> Replace the code (python) on after_script of the acceptance jobs that
> is currently used to show the logs of failed tests. Instead it is used
> the Avocado's testlogs plug-in which works likewise.
> 
> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
>   .gitlab-ci.yml | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

[PATCH 3/3] gitlab-ci: Archive logs of acceptance tests

[Wainer dos Santos Moschetta]

Keep the logs of acceptance tests for two days on GitLab. If you want
to make it available for more time, click on the 'Keep' button on
the Job page at web UI.

By default GitLab will archive artifacts only if the job succeed.
Instead let's keep it on both success and failure, so it gives the
opportunity to the developer/maintainer to check the error logs
as well as to the logs of CANCEL tests (not shown on the job logs).

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 85aa20ffde..bf3df843e2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -49,8 +49,12 @@ include:
       - ${CI_PROJECT_DIR}/avocado-cache
     policy: pull-push
   artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+    when: always
+    expire_in: 2 days
     paths:
       - build/tests/results/latest/results.xml
+      - build/tests/results/latest/test-results
     reports:
       junit: build/tests/results/latest/results.xml
   before_script:
-- 
2.28.0

Re: [PATCH 3/3] gitlab-ci: Archive logs of acceptance tests

[Thomas Huth]

On 11/12/2020 19.38, Wainer dos Santos Moschetta wrote:
> Keep the logs of acceptance tests for two days on GitLab. If you want
> to make it available for more time, click on the 'Keep' button on
> the Job page at web UI.
> 
> By default GitLab will archive artifacts only if the job succeed.
> Instead let's keep it on both success and failure, so it gives the
> opportunity to the developer/maintainer to check the error logs
> as well as to the logs of CANCEL tests (not shown on the job logs).
> 
> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
>  .gitlab-ci.yml | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 85aa20ffde..bf3df843e2 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -49,8 +49,12 @@ include:
>        - ${CI_PROJECT_DIR}/avocado-cache
>      policy: pull-push
>    artifacts:
> +    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
> +    when: always
> +    expire_in: 2 days
>      paths:
>        - build/tests/results/latest/results.xml
> +      - build/tests/results/latest/test-results
>      reports:
>        junit: build/tests/results/latest/results.xml
>    before_script:
> 

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 3/3] gitlab-ci: Archive logs of acceptance tests

[Willian Rampazzo]

On Fri, Dec 11, 2020 at 3:38 PM Wainer dos Santos Moschetta
<wainersm@redhat.com> wrote:
>
> Keep the logs of acceptance tests for two days on GitLab. If you want
> to make it available for more time, click on the 'Keep' button on
> the Job page at web UI.
>
> By default GitLab will archive artifacts only if the job succeed.
> Instead let's keep it on both success and failure, so it gives the
> opportunity to the developer/maintainer to check the error logs
> as well as to the logs of CANCEL tests (not shown on the job logs).
>
> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
>  .gitlab-ci.yml | 4 ++++
>  1 file changed, 4 insertions(+)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>