From: Wainer dos Santos Moschetta <wainersm@redhat.com>
To: "Cleber Rosa" <crosa@redhat.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Peter Maydell" <peter.maydell@linaro.org>,
qemu-devel@nongnu.org
Cc: "Fam Zheng" <fam@euphon.net>, "Thomas Huth" <thuth@redhat.com>,
"Daniel P . Berrangé" <berrange@redhat.com>,
"Eduardo Habkost" <ehabkost@redhat.com>,
"Erik Skultety" <eskultet@redhat.com>,
"Stefan Hajnoczi" <stefanha@gmail.com>,
"Andrea Bolognani" <abologna@redhat.com>,
"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
"Willian Rampazzo" <willianr@redhat.com>,
"Willian Rampazzo" <wrampazz@redhat.com>,
"Philippe Mathieu-Daudé" <philmd@redhat.com>,
"Beraldo Leal" <bleal@redhat.com>
Subject: Re: [PATCH v6 2/4] Jobs based on custom runners: build environment docs and playbook
Date: Tue, 8 Jun 2021 15:48:27 -0300 [thread overview]
Message-ID: <eee53288-6f8e-1d33-68a1-83a5f78316d1@redhat.com> (raw)
In-Reply-To: <20210608031425.833536-3-crosa@redhat.com>
Hi,
On 6/8/21 12:14 AM, Cleber Rosa wrote:
> To run basic jobs on custom runners, the environment needs to be
> properly set up. The most common requirement is having the right
> packages installed.
>
> The playbook introduced here covers the QEMU's project s390x and
> aarch64 machines. At the time this is being proposed, those machines
> have already had this playbook applied to them.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> docs/devel/ci.rst | 30 ++++++++
> scripts/ci/setup/build-environment.yml | 98 ++++++++++++++++++++++++++
> scripts/ci/setup/inventory.template | 1 +
> 3 files changed, 129 insertions(+)
> create mode 100644 scripts/ci/setup/build-environment.yml
> create mode 100644 scripts/ci/setup/inventory.template
>
> diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> index 585b7bf4b8..35c6b5e269 100644
> --- a/docs/devel/ci.rst
> +++ b/docs/devel/ci.rst
> @@ -26,3 +26,33 @@ gitlab-runner, is called a "custom runner".
> The GitLab CI jobs definition for the custom runners are located under::
>
> .gitlab-ci.d/custom-runners.yml
> +
> +Machine Setup Howto
> +-------------------
> +
> +For all Linux based systems, the setup can be mostly automated by the
> +execution of two Ansible playbooks. Create an ``inventory`` file
> +under ``scripts/ci/setup``, such as this::
Missing to mention the template file.
> +
> + fully.qualified.domain
> + other.machine.hostname
> +
> +You may need to set some variables in the inventory file itself. One
> +very common need is to tell Ansible to use a Python 3 interpreter on
> +those hosts. This would look like::
> +
> + fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
> + other.machine.hostname ansible_python_interpreter=/usr/bin/python3
> +
> +Build environment
> +~~~~~~~~~~~~~~~~~
> +
> +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
> +set up machines with the environment needed to perform builds and run
> +QEMU tests. It covers a number of different Linux distributions and
> +FreeBSD.
> +
> +To run the playbook, execute::
> +
> + cd scripts/ci/setup
> + ansible-playbook -i inventory build-environment.yml
> diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
> new file mode 100644
> index 0000000000..664f2f0519
> --- /dev/null
> +++ b/scripts/ci/setup/build-environment.yml
> @@ -0,0 +1,98 @@
> +---
> +- name: Installation of basic packages to build QEMU
> + hosts: all
You will need to "become: yes" if the login user is not privileged, right?
Or mention on the documentation how the user should configure the
connection for privileged login.
About privilege escalation with Ansible:
https://docs.ansible.com/ansible/latest/user_guide/become.html
> + tasks:
Just a tip: you can put all those task under a block
(https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html)
so check if "ansible_facts['distribution'] == 'Ubuntu'" only once.
I reviewed the remain of the playbook; it looks good to me.
> + - name: Update apt cache
> + apt:
> + update_cache: yes
> + when:
> + - ansible_facts['distribution'] == 'Ubuntu'
> +
> + - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
> + package:
> + name:
> + # Originally from tests/docker/dockerfiles/ubuntu1804.docker
> + - ccache
> + - gcc
> + - gettext
> + - git
> + - glusterfs-common
> + - libaio-dev
> + - libattr1-dev
> + - libbrlapi-dev
> + - libbz2-dev
> + - libcacard-dev
> + - libcap-ng-dev
> + - libcurl4-gnutls-dev
> + - libdrm-dev
> + - libepoxy-dev
> + - libfdt-dev
> + - libgbm-dev
> + - libgtk-3-dev
> + - libibverbs-dev
> + - libiscsi-dev
> + - libjemalloc-dev
> + - libjpeg-turbo8-dev
> + - liblzo2-dev
> + - libncurses5-dev
> + - libncursesw5-dev
> + - libnfs-dev
> + - libnss3-dev
> + - libnuma-dev
> + - libpixman-1-dev
> + - librados-dev
> + - librbd-dev
> + - librdmacm-dev
> + - libsasl2-dev
> + - libsdl2-dev
> + - libseccomp-dev
> + - libsnappy-dev
> + - libspice-protocol-dev
> + - libssh-dev
> + - libusb-1.0-0-dev
> + - libusbredirhost-dev
> + - libvdeplug-dev
> + - libvte-2.91-dev
> + - libzstd-dev
> + - make
> + - python3-yaml
> + - python3-sphinx
> + - python3-sphinx-rtd-theme
> + - ninja-build
> + - sparse
> + - xfslibs-dev
> + state: present
> + when:
> + - ansible_facts['distribution'] == 'Ubuntu'
> +
> + - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
> + package:
> + name:
> + - libspice-server-dev
> + - libxen-dev
> + state: present
> + when:
> + - ansible_facts['distribution'] == 'Ubuntu'
> + - ansible_facts['architecture'] != 's390x'
> +
> + - name: Install basic packages to build QEMU on Ubuntu 18.04
> + package:
> + name:
> + # Originally from tests/docker/dockerfiles/ubuntu1804.docker
> + - clang
> + when:
> + - ansible_facts['distribution'] == 'Ubuntu'
> + - ansible_facts['distribution_version'] == '18.04'
> +
> + - name: Install basic packages to build QEMU on Ubuntu 20.04
> + package:
> + name:
> + # Originally from tests/docker/dockerfiles/ubuntu2004.docker
> + - clang-10
> + - genisoimage
> + - liblttng-ust-dev
> + - libslirp-dev
> + - netcat-openbsd
> + when:
> + - ansible_facts['distribution'] == 'Ubuntu'
> + - ansible_facts['distribution_version'] == '20.04'
> diff --git a/scripts/ci/setup/inventory.template b/scripts/ci/setup/inventory.template
> new file mode 100644
> index 0000000000..2fbb50c4a8
> --- /dev/null
> +++ b/scripts/ci/setup/inventory.template
> @@ -0,0 +1 @@
> +localhost
next prev parent reply other threads:[~2021-06-08 18:51 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-08 3:14 [PATCH v6 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
2021-06-08 3:14 ` [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
2021-06-08 18:29 ` Wainer dos Santos Moschetta
2021-06-09 13:24 ` Alex Bennée
2021-06-09 14:22 ` Thomas Huth
2021-06-09 14:24 ` Willian Rampazzo
2021-06-08 3:14 ` [PATCH v6 2/4] Jobs based on custom runners: build environment docs and playbook Cleber Rosa
2021-06-08 18:48 ` Wainer dos Santos Moschetta [this message]
2021-06-09 16:13 ` Willian Rampazzo
2021-06-29 15:23 ` Cleber Rosa
2021-06-29 15:06 ` Cleber Rosa
2021-06-09 13:31 ` Alex Bennée
2021-06-09 14:21 ` Cleber Rosa Junior
2021-06-09 15:26 ` Alex Bennée
2021-06-09 17:09 ` Cleber Rosa Junior
2021-06-11 10:40 ` Alex Bennée
2021-06-28 23:07 ` Cleber Rosa
2021-06-09 17:16 ` Willian Rampazzo
2021-06-10 8:13 ` Erik Skultety
2021-06-29 23:35 ` Cleber Rosa
2021-06-29 23:30 ` Cleber Rosa
2021-06-08 3:14 ` [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Cleber Rosa
2021-06-08 19:04 ` Wainer dos Santos Moschetta
2021-06-29 23:51 ` Cleber Rosa
2021-06-09 17:46 ` Willian Rampazzo
2021-06-30 0:04 ` Cleber Rosa
2021-06-10 6:23 ` Thomas Huth
2021-06-30 0:18 ` Cleber Rosa
2021-06-08 3:14 ` [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Cleber Rosa
2021-06-08 6:29 ` Philippe Mathieu-Daudé
2021-06-08 13:36 ` Cleber Rosa Junior
2021-06-08 19:07 ` Wainer dos Santos Moschetta
2021-06-09 15:09 ` Stefan Hajnoczi
2021-06-30 0:47 ` Cleber Rosa
2021-06-09 14:54 ` Stefan Hajnoczi
2021-06-30 0:40 ` Cleber Rosa
2021-06-11 11:00 ` Alex Bennée
2021-06-30 1:08 ` Cleber Rosa
2021-06-30 14:24 ` Willian Rampazzo
2021-06-09 14:22 ` Stefan Hajnoczi
2021-06-08 18:27 ` Wainer dos Santos Moschetta
2021-06-09 15:53 ` Alex Bennée
2021-06-30 0:30 ` Cleber Rosa
2021-06-09 18:56 ` Willian Rampazzo
2021-06-10 6:18 ` Thomas Huth
2021-06-30 1:02 ` Cleber Rosa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=eee53288-6f8e-1d33-68a1-83a5f78316d1@redhat.com \
--to=wainersm@redhat.com \
--cc=abologna@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=berrange@redhat.com \
--cc=bleal@redhat.com \
--cc=crosa@redhat.com \
--cc=ehabkost@redhat.com \
--cc=eskultet@redhat.com \
--cc=f4bug@amsat.org \
--cc=fam@euphon.net \
--cc=peter.maydell@linaro.org \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@gmail.com \
--cc=thuth@redhat.com \
--cc=willianr@redhat.com \
--cc=wrampazz@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).