Re: [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook

[Cleber Rosa <crosa@redhat.com>]

On Wed, Jun 9, 2021 at 1:46 PM Willian Rampazzo <wrampazz@redhat.com> wrote:
>
> On Tue, Jun 8, 2021 at 12:14 AM Cleber Rosa <crosa@redhat.com> wrote:
> >
> > To have the jobs dispatched to custom runners, gitlab-runner must
> > be installed, active as a service and properly configured.  The
> > variables file and playbook introduced here should help with those
> > steps.
> >
> > The playbook introduced here covers the Linux distributions and
> > has been primarily tested on OS/machines that the QEMU project
> > has available to act as runners, namely:
> >
> >  * Ubuntu 20.04 on aarch64
> >  * Ubuntu 18.04 on s390x
> >
> > But, it should work on all other Linux distributions.  Earlier
> > versions were tested on FreeBSD too, so chances of success are
> > high.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> >  docs/devel/ci.rst                  | 57 ++++++++++++++++++++++++++++
> >  scripts/ci/setup/.gitignore        |  1 +
> >  scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++
> >  scripts/ci/setup/vars.yml.template | 12 ++++++
> >  4 files changed, 131 insertions(+)
> >  create mode 100644 scripts/ci/setup/.gitignore
> >  create mode 100644 scripts/ci/setup/gitlab-runner.yml
> >  create mode 100644 scripts/ci/setup/vars.yml.template
> >
> > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> > index 35c6b5e269..bbd89e54d7 100644
> > --- a/docs/devel/ci.rst
> > +++ b/docs/devel/ci.rst
> > @@ -56,3 +56,60 @@ To run the playbook, execute::
> >
> >    cd scripts/ci/setup
> >    ansible-playbook -i inventory build-environment.yml
> > +
> > +gitlab-runner setup and registration
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +The gitlab-runner agent needs to be installed on each machine that
> > +will run jobs.  The association between a machine and a GitLab project
> > +happens with a registration token.  To find the registration token for
> > +your repository/project, navigate on GitLab's web UI to:
> > +
> > + * Settings (the gears like icon), then
>
> * Settings (the gears like icon in the end of the left menu), then
>

ACK.  What about:

" * Settings (the gears-like icon at the bottom of the left hand side
   vertical toolbar), then"

> I took some time to find it as it was hidden at the end of the scrolling.
>
> > + * CI/CD, then
> > + * Runners, and click on the "Expand" button, then
> > + * Under "Set up a specific Runner manually", look for the value under
> > +   "Use the following registration token during setup"
>
> For me, it shows: "And this registration token:"
>

Right, it must have changed.  Updating it on v7.

> > +
> > +Copy the ``scripts/ci/setup/vars.yml.template`` file to
> > +``scripts/ci/setup/vars.yml``.  Then, set the
> > +``gitlab_runner_registration_token`` variable to the value obtained
> > +earlier.
> > +
> > +.. note:: gitlab-runner is not available from the standard location
> > +          for all OS and architectures combinations.  For some systems,
> > +          a custom build may be necessary.  Some builds are avaiable
>
> If you keep this block (see comment from Wainer), s/avaiable/available/
>

I'd rather remove it, but thanks for spotting the typo.

> > +          at https://cleber.fedorapeople.org/gitlab-runner/ and this
> > +          URI may be used as a value on ``vars.yml``
> > +
> > +To run the playbook, execute::
> > +
> > +  cd scripts/ci/setup
> > +  ansible-playbook -i inventory gitlab-runner.yml
> > +
> > +Following the registration, it's necessary to configure the runner tags,
> > +and optionally other configurations on the GitLab UI.  Navigate to:
> > +
> > + * Settings (the gears like icon), then
> > + * CI/CD, then
> > + * Runners, and click on the "Expand" button, then
> > + * "Runners activated for this project", then
> > + * Click on the "Edit" icon (next to the "Lock" Icon)
> > +
> > +Under tags, add values matching the jobs a runner should run.  For a
> > +Ubuntu 20.04 aarch64 system, the tags should be set as::
> > +
> > +  ubuntu_20.04,aarch64
> > +
> > +Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
> > +would contain::
> > +
> > +  ubuntu-20.04-aarch64-all:
> > +   tags:
> > +   - ubuntu_20.04
> > +   - aarch64
> > +
> > +It's also recommended to:
> > +
> > + * increase the "Maximum job timeout" to something like ``2h``
> > + * give it a better Description
> > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
> > new file mode 100644
> > index 0000000000..f112d05dd0
> > --- /dev/null
> > +++ b/scripts/ci/setup/.gitignore
> > @@ -0,0 +1 @@
> > +vars.yml
> > \ No newline at end of file
> > diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml
> > new file mode 100644
> > index 0000000000..98dab92bb5
> > --- /dev/null
> > +++ b/scripts/ci/setup/gitlab-runner.yml
> > @@ -0,0 +1,61 @@
> > +---
> > +- name: Installation of gitlab-runner
> > +  hosts: all
> > +  vars_files:
> > +    - vars.yml
> > +  tasks:
> > +    - debug:
> > +        msg: 'Checking for a valid GitLab registration token'
> > +      failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'"
> > +
> > +    - name: Create a group for the gitlab-runner service
> > +      group:
> > +        name: gitlab-runner
> > +
> > +    - name: Create a user for the gitlab-runner service
> > +      user:
> > +        user: gitlab-runner
> > +        group: gitlab-runner
> > +        comment: GitLab Runner
> > +        home: /home/gitlab-runner
> > +        shell: /bin/bash
> > +
> > +    - name: Remove the .bash_logout file when on Ubuntu systems
> > +      file:
> > +        path: /home/gitlab-runner/.bash_logout
> > +        state: absent
> > +      when: "ansible_facts['distribution'] == 'Ubuntu'"
> > +
> > +    - name: Set the Operating System for gitlab-runner
> > +      set_fact:
> > +        gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}"
> > +    - debug:
> > +        msg: gitlab-runner OS is {{ gitlab_runner_os }}
> > +
> > +    - name: Set the architecture for gitlab-runner
> > +      set_fact:
> > +        gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}"
> > +    - debug:
> > +        msg: gitlab-runner arch is {{ gitlab_runner_arch }}
> > +
> > +    - name: Download the matching gitlab-runner
> > +      get_url:
> > +        dest: /usr/local/bin/gitlab-runner
> > +        url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version  }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}"
> > +        owner: gitlab-runner
> > +        group: gitlab-runner
> > +        mode: u=rwx,g=rwx,o=rx
> > +
> > +    - name: Register the gitlab-runner
> > +      command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
> > +
> > +    - name: Install the gitlab-runner service using its own functionality
> > +      command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner
> > +      register: gitlab_runner_install_service_result
> > +      failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr"
> > +
> > +    - name: Enable the gitlab-runner service
> > +      service:
> > +        name: gitlab-runner
> > +        state: started
> > +        enabled: yes
> > diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template
> > new file mode 100644
> > index 0000000000..e48089761f
> > --- /dev/null
> > +++ b/scripts/ci/setup/vars.yml.template
> > @@ -0,0 +1,12 @@
> > +# The version of the gitlab-runner to use
> > +gitlab_runner_version: 13.12.0
> > +# The URL of the gitlab server to use, usually https://gitlab.com unless you're
> > +# using a private GitLab instance
> > +gitlab_runner_server_url: https://gitlab.com
> > +# A mapping of the ansible to gitlab architecture nomenclature
> > +ansible_to_gitlab_arch:
> > +  x86_64: amd64
> > +  aarch64: arm64
> > +  s390x: s390x
> > +# A unique token made available by GitLab to your project for registering runners
> > +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
> > --
> > 2.25.4
> >
>
> With my suggestions and Wainer's:
>
> Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> Tested-by: Willian Rampazzo <willianr@redhat.com>
>