All of lore.kernel.org
 help / color / mirror / Atom feed
From: Cleber Rosa <crosa@redhat.com>
To: "Alex Bennée" <alex.bennee@linaro.org>,
	"Peter Maydell" <peter.maydell@linaro.org>,
	qemu-devel@nongnu.org
Cc: "Fam Zheng" <fam@euphon.net>, "Thomas Huth" <thuth@redhat.com>,
	"Daniel P . Berrangé" <berrange@redhat.com>,
	"Eduardo Habkost" <ehabkost@redhat.com>,
	"Erik Skultety" <eskultet@redhat.com>,
	"Stefan Hajnoczi" <stefanha@gmail.com>,
	"Andrea Bolognani" <abologna@redhat.com>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Willian Rampazzo" <willianr@redhat.com>,
	"Willian Rampazzo" <wrampazz@redhat.com>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Beraldo Leal" <bleal@redhat.com>
Subject: [PATCH v6 2/4] Jobs based on custom runners: build environment docs and playbook
Date: Mon,  7 Jun 2021 23:14:23 -0400	[thread overview]
Message-ID: <20210608031425.833536-3-crosa@redhat.com> (raw)
In-Reply-To: <20210608031425.833536-1-crosa@redhat.com>

To run basic jobs on custom runners, the environment needs to be
properly set up.  The most common requirement is having the right
packages installed.

The playbook introduced here covers the QEMU's project s390x and
aarch64 machines.  At the time this is being proposed, those machines
have already had this playbook applied to them.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 docs/devel/ci.rst                      | 30 ++++++++
 scripts/ci/setup/build-environment.yml | 98 ++++++++++++++++++++++++++
 scripts/ci/setup/inventory.template    |  1 +
 3 files changed, 129 insertions(+)
 create mode 100644 scripts/ci/setup/build-environment.yml
 create mode 100644 scripts/ci/setup/inventory.template

diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 585b7bf4b8..35c6b5e269 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -26,3 +26,33 @@ gitlab-runner, is called a "custom runner".
 The GitLab CI jobs definition for the custom runners are located under::
 
   .gitlab-ci.d/custom-runners.yml
+
+Machine Setup Howto
+-------------------
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks.  Create an ``inventory`` file
+under ``scripts/ci/setup``, such as this::
+
+  fully.qualified.domain
+  other.machine.hostname
+
+You may need to set some variables in the inventory file itself.  One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts.  This would look like::
+
+  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~~~~~~~~~~~~~~~~~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests.  It covers a number of different Linux distributions and
+FreeBSD.
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory build-environment.yml
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
new file mode 100644
index 0000000000..664f2f0519
--- /dev/null
+++ b/scripts/ci/setup/build-environment.yml
@@ -0,0 +1,98 @@
+---
+- name: Installation of basic packages to build QEMU
+  hosts: all
+  tasks:
+    - name: Update apt cache
+      apt:
+        update_cache: yes
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+
+    - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+      package:
+        name:
+        # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+          - ccache
+          - gcc
+          - gettext
+          - git
+          - glusterfs-common
+          - libaio-dev
+          - libattr1-dev
+          - libbrlapi-dev
+          - libbz2-dev
+          - libcacard-dev
+          - libcap-ng-dev
+          - libcurl4-gnutls-dev
+          - libdrm-dev
+          - libepoxy-dev
+          - libfdt-dev
+          - libgbm-dev
+          - libgtk-3-dev
+          - libibverbs-dev
+          - libiscsi-dev
+          - libjemalloc-dev
+          - libjpeg-turbo8-dev
+          - liblzo2-dev
+          - libncurses5-dev
+          - libncursesw5-dev
+          - libnfs-dev
+          - libnss3-dev
+          - libnuma-dev
+          - libpixman-1-dev
+          - librados-dev
+          - librbd-dev
+          - librdmacm-dev
+          - libsasl2-dev
+          - libsdl2-dev
+          - libseccomp-dev
+          - libsnappy-dev
+          - libspice-protocol-dev
+          - libssh-dev
+          - libusb-1.0-0-dev
+          - libusbredirhost-dev
+          - libvdeplug-dev
+          - libvte-2.91-dev
+          - libzstd-dev
+          - make
+          - python3-yaml
+          - python3-sphinx
+          - python3-sphinx-rtd-theme
+          - ninja-build
+          - sparse
+          - xfslibs-dev
+        state: present
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+
+    - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
+      package:
+        name:
+          - libspice-server-dev
+          - libxen-dev
+        state: present
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+        - ansible_facts['architecture'] != 's390x'
+
+    - name: Install basic packages to build QEMU on Ubuntu 18.04
+      package:
+        name:
+        # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+          - clang
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+        - ansible_facts['distribution_version'] == '18.04'
+
+    - name: Install basic packages to build QEMU on Ubuntu 20.04
+      package:
+        name:
+        # Originally from tests/docker/dockerfiles/ubuntu2004.docker
+          - clang-10
+          - genisoimage
+          - liblttng-ust-dev
+          - libslirp-dev
+          - netcat-openbsd
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+        - ansible_facts['distribution_version'] == '20.04'
diff --git a/scripts/ci/setup/inventory.template b/scripts/ci/setup/inventory.template
new file mode 100644
index 0000000000..2fbb50c4a8
--- /dev/null
+++ b/scripts/ci/setup/inventory.template
@@ -0,0 +1 @@
+localhost
-- 
2.25.4



  parent reply	other threads:[~2021-06-08  3:15 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-08  3:14 [PATCH v6 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
2021-06-08  3:14 ` [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
2021-06-08 18:29   ` Wainer dos Santos Moschetta
2021-06-09 13:24   ` Alex Bennée
2021-06-09 14:22   ` Thomas Huth
2021-06-09 14:24   ` Willian Rampazzo
2021-06-08  3:14 ` Cleber Rosa [this message]
2021-06-08 18:48   ` [PATCH v6 2/4] Jobs based on custom runners: build environment docs and playbook Wainer dos Santos Moschetta
2021-06-09 16:13     ` Willian Rampazzo
2021-06-29 15:23       ` Cleber Rosa
2021-06-29 15:06     ` Cleber Rosa
2021-06-09 13:31   ` Alex Bennée
2021-06-09 14:21     ` Cleber Rosa Junior
2021-06-09 15:26       ` Alex Bennée
2021-06-09 17:09         ` Cleber Rosa Junior
2021-06-11 10:40           ` Alex Bennée
2021-06-28 23:07             ` Cleber Rosa
2021-06-09 17:16   ` Willian Rampazzo
2021-06-10  8:13     ` Erik Skultety
2021-06-29 23:35       ` Cleber Rosa
2021-06-29 23:30     ` Cleber Rosa
2021-06-08  3:14 ` [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Cleber Rosa
2021-06-08 19:04   ` Wainer dos Santos Moschetta
2021-06-29 23:51     ` Cleber Rosa
2021-06-09 17:46   ` Willian Rampazzo
2021-06-30  0:04     ` Cleber Rosa
2021-06-10  6:23   ` Thomas Huth
2021-06-30  0:18     ` Cleber Rosa
2021-06-08  3:14 ` [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Cleber Rosa
2021-06-08  6:29   ` Philippe Mathieu-Daudé
2021-06-08 13:36     ` Cleber Rosa Junior
2021-06-08 19:07       ` Wainer dos Santos Moschetta
2021-06-09 15:09         ` Stefan Hajnoczi
2021-06-30  0:47           ` Cleber Rosa
2021-06-09 14:54       ` Stefan Hajnoczi
2021-06-30  0:40         ` Cleber Rosa
2021-06-11 11:00       ` Alex Bennée
2021-06-30  1:08         ` Cleber Rosa
2021-06-30 14:24           ` Willian Rampazzo
2021-06-09 14:22     ` Stefan Hajnoczi
2021-06-08 18:27   ` Wainer dos Santos Moschetta
2021-06-09 15:53     ` Alex Bennée
2021-06-30  0:30     ` Cleber Rosa
2021-06-09 18:56   ` Willian Rampazzo
2021-06-10  6:18   ` Thomas Huth
2021-06-30  1:02     ` Cleber Rosa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210608031425.833536-3-crosa@redhat.com \
    --to=crosa@redhat.com \
    --cc=abologna@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=berrange@redhat.com \
    --cc=bleal@redhat.com \
    --cc=ehabkost@redhat.com \
    --cc=eskultet@redhat.com \
    --cc=f4bug@amsat.org \
    --cc=fam@euphon.net \
    --cc=peter.maydell@linaro.org \
    --cc=philmd@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@gmail.com \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=willianr@redhat.com \
    --cc=wrampazz@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.