* [PATCH v3 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI)
@ 2020-10-14 5:21 Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
` (3 more replies)
0 siblings, 4 replies; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 5:21 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Peter Maydell
Cc: Thomas Huth, Daniel P . Berrangé,
Eduardo Habkost, Erik Skultety, Stefan Hajnoczi,
Andrea Bolognani, Wainer dos Santos Moschetta, Willian Rampazzo,
Cleber Rosa, Philippe Mathieu-Daudé,
Beraldo Leal
TL;DR: this should allow the QEMU maintainer to push to the staging
branch, and have custom jobs running on the project's aarch64 and
s390x machines. Simple usage looks like:
git push remote staging
./scripts/ci/gitlab-pipeline-status --verbose --wait
Long version:
The idea about a public facing Gating CI for QEMU was summarized in an
RFC[1]. Since then, it was decided that a simpler version should be
attempted first.
At this point, there are two specific runners (an aarch64 and an s390)
registered with GitLab, at https://gitlab.com/qemu-project, currently
setup to the "qemu" repository.
Changes from v2:
- The overall idea of "Gating CI" has been re-worded "custom runners",
given that the other jobs running on shared runners are also
considered gating (Daniel)
- Fixed wording and typos on the documentation, including:
* update -> up to date (Erik)
* a different set of CI jobs -> different CI jobs (Erik)
* Pull requests will only be merged -> code will only be merged (Stefan)
* Setup -> set up (Stefan)
* them -> they (Stefan)
* the -> where the (Stefan)
* dropped "in the near future" (Stefan)
- Changed comment on "build-environment.yml" regarding the origin of
the package list (Stefan)
- Removed inclusion of "vars.yml" from "build-environment.yml", given that
no external variable is used there
- Updated package list in "build-environment.yml" from current
dockerfiles
- Tested "build-environment" on Fedora 31 and 32 (in addition to Fedora 30),
and noted that it's possible to use it on those distros
- Moved CI documentation from "testing.rst" to its own file (Phillipe)
- Split "GitLab Gating CI: initial set of jobs, documentation and scripts"
into (Phillipe):
1) Basic documentation and configuration (gitlab-ci.yml) placeholder
2) Playbooks for setting up a build environment
3) Playbooks for setting up gitlab-runner
4) Actual GitLab CI jobs configuration
- Set custom jobs to be on the "build" stage, given that they combine
build and test.
- Set custom jobs to not depend on any other job, so they can start
right away.
- Set rules for starting jobs so that all pushing to any branch that
start with name "staging". This allows the project maintainer to
use the "push to staging" workflow, while also allowing others to
generate similar jobs. If this project has configured custom
runners, the jobs will run, if not, the pipeline will be marked as
"stuck".
- Changed "scripts" on custom jobs to follow the now common pattern
(on other jobs) of creating a "build" directory.
Changes from v1:
- Added jobs that require specific GitLab runners already available
(Ubuntu 20.04 on aarch64, and Ubuntu 18.04 on s390x)
- Removed jobs that require specific GitLab runners not yet available
(Fedora 30, FreeBSD 12.1)
- Updated documentation
- Added copyright and license to new scripts
- Moved script to from "contrib" to "scripts/ci/"
- Moved setup playbooks form "contrib" to "scripts/ci/setup"
- Moved "gating.yml" to ".gitlab-ci.d" directory
- Removed "staging" only branch restriction on jobs defined in
".gitlab-ci.yml", assumes that the additional jobs on the staging
branch running on the freely available gitlab shared runner are
positive
- Dropped patches 1-3 (already merged)
- Simplified amount of version specifity on Ubuntu, from 18.04.3 to
simply 18.04 (assumes no diverse minor levels will be used or
specific runners)
Changes from the RFC patches[2] accompanying the RFC document:
- Moved gating job definitions to .gitlab-ci-gating.yml
- Added info on "--disable-libssh" build option requirement
(https://bugs.launchpad.net/qemu/+bug/1838763) to Ubuntu 18.04 jobs
- Added info on "--disable-glusterfs" build option requirement
(there's no static version of those libs in distro supplied
packages) to one
- Dropped ubuntu-18.04.3-x86_64-notools job definition, because it
doesn't fall into the general scope of gating job described by PMM
(and it did not run any test)
- Added w32 and w64 cross builds based on Fedora 30
- Added a FreeBSD based job that builds all targets and runs `make
check`
- Added "-j`nproc`" and "-j`sysctl -n hw.ncpu`" options to make as a
simple but effective way of speeding up the builds and tests by
using a number of make jobs matching the number of CPUs
- Because the Ansible playbooks reference the content on Dockerfiles,
some fixes to some Dockerfiles caught in the process were included
- New patch with script to check or wait on a pipeline execution
[1] - https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg00231.html
[2] - https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg00154.html
Cleber Rosa (4):
Jobs based on custom runners: documentation and configuration
placeholder
Jobs based on custom runners: build environment docs and playbook
Jobs based on custom runners: docs and gitlab-runner setup playbook
Jobs based on custom runners: add job definitions for QEMU's machines
.gitlab-ci.d/custom-runners.yml | 206 +++++++++++++++++++++++
.gitlab-ci.yml | 1 +
docs/devel/ci.rst | 149 +++++++++++++++++
docs/devel/index.rst | 1 +
scripts/ci/setup/.gitignore | 1 +
scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
scripts/ci/setup/gitlab-runner.yml | 72 ++++++++
scripts/ci/setup/inventory | 2 +
scripts/ci/setup/vars.yml.template | 13 ++
9 files changed, 665 insertions(+)
create mode 100644 .gitlab-ci.d/custom-runners.yml
create mode 100644 docs/devel/ci.rst
create mode 100644 scripts/ci/setup/.gitignore
create mode 100644 scripts/ci/setup/build-environment.yml
create mode 100644 scripts/ci/setup/gitlab-runner.yml
create mode 100644 scripts/ci/setup/inventory
create mode 100644 scripts/ci/setup/vars.yml.template
--
2.25.4
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder
2020-10-14 5:21 [PATCH v3 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
@ 2020-10-14 5:21 ` Cleber Rosa
2020-10-14 17:28 ` Daniel P. Berrangé
2020-10-14 5:21 ` [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook Cleber Rosa
` (2 subsequent siblings)
3 siblings, 1 reply; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 5:21 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Peter Maydell
Cc: Thomas Huth, Daniel P . Berrangé,
Eduardo Habkost, Erik Skultety, Stefan Hajnoczi,
Andrea Bolognani, Wainer dos Santos Moschetta, Willian Rampazzo,
Cleber Rosa, Philippe Mathieu-Daudé,
Beraldo Leal
As described in the included documentation, the "custom runner" jobs
extend the GitLab CI jobs already in place.
Those jobs are intended to run on hardware and/or Operating Systems
not provided by GitLab's shared runners.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
.gitlab-ci.d/custom-runners.yml | 14 +++++++++
.gitlab-ci.yml | 1 +
docs/devel/ci.rst | 54 +++++++++++++++++++++++++++++++++
docs/devel/index.rst | 1 +
4 files changed, 70 insertions(+)
create mode 100644 .gitlab-ci.d/custom-runners.yml
create mode 100644 docs/devel/ci.rst
diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
new file mode 100644
index 0000000000..3004da2bda
--- /dev/null
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -0,0 +1,14 @@
+# The CI jobs defined here require GitLab runners installed and
+# registered on machines that match their operating system names,
+# versions and architectures. This is in contrast to the other CI
+# jobs that are intended to run on GitLab's "shared" runners.
+
+# Different than the default approach on "shared" runners, based on
+# containers, the custom runners have no such *requirement*, as those
+# jobs should be capable of running on operating systems with no
+# compatible container implementation, or no support from
+# gitlab-runner. To avoid problems that gitlab-runner can cause while
+# reusing the GIT repository, let's enable the recursive submodule
+# strategy.
+variables:
+ GIT_SUBMODULE_STRATEGY: recursive
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 8ffd415ca5..b33c433fd7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -18,6 +18,7 @@ include:
- local: '/.gitlab-ci.d/opensbi.yml'
- local: '/.gitlab-ci.d/containers.yml'
- local: '/.gitlab-ci.d/crossbuilds.yml'
+ - local: '/.gitlab-ci.d/custom-runners.yml'
.native_build_job_template: &native_build_job_definition
stage: build
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
new file mode 100644
index 0000000000..41a4bbddad
--- /dev/null
+++ b/docs/devel/ci.rst
@@ -0,0 +1,54 @@
+==
+CI
+==
+
+QEMU has configurations enabled for a number of different CI services.
+The most up to date information about them and their status can be
+found at::
+
+ https://wiki.qemu.org/Testing/CI
+
+Jobs on Custom Runners
+======================
+
+Besides the jobs run under the various CI systems listed before, there
+are a number additional jobs that will run before an actual merge.
+These use the same GitLab CI's service/framework already used for all
+other GitLab based CI jobs, but rely on additional systems, not the
+ones provided by GitLab as "shared runners".
+
+The architecture of GitLab's CI service allows different machines to
+be set up with GitLab's "agent", called gitlab-runner, which will take
+care of running jobs created by events such as a push to a branch.
+Here, the combination of a machine, properly configured with GitLab's
+gitlab-runner, is called a "custom runner" here.
+
+The GitLab CI jobs definition for the custom runners are located under::
+
+ .gitlab-ci.d/custom-runners.yml
+
+Current Jobs
+------------
+
+The current CI jobs based on custom runners have the primary goal of
+catching and preventing regressions on a wider number of host systems
+than the ones provided by GitLab's shared runners.
+
+Also, the mechanics of reliability, capacity and overall maintanance
+of the machines provided by the QEMU project itself for those jobs
+will be evaluated.
+
+Future Plans and Jobs
+---------------------
+
+Once the CI Jobs based on custom runners have been proved mature with
+the initial set of jobs run on machines from the QEMU project, other
+members in the community should be able provide their own machine
+configuration documentation/scripts, and accompanying job definitions.
+
+As a general rule, those newly added contributed jobs should run as
+"non-gating", until their reliability is verified.
+
+The precise minimum requirements and exact rules for machine
+configuration documentation/scripts, and the success rate of jobs are
+still to be defined.
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 77baae5c77..2fdd36e751 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -21,6 +21,7 @@ Contents:
atomics
stable-process
testing
+ ci
qtest
decodetree
secure-coding-practices
--
2.25.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook
2020-10-14 5:21 [PATCH v3 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
@ 2020-10-14 5:21 ` Cleber Rosa
2020-10-14 17:30 ` Daniel P. Berrangé
2020-10-14 5:21 ` [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Cleber Rosa
3 siblings, 1 reply; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 5:21 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Peter Maydell
Cc: Thomas Huth, Daniel P . Berrangé,
Eduardo Habkost, Erik Skultety, Stefan Hajnoczi,
Andrea Bolognani, Wainer dos Santos Moschetta, Willian Rampazzo,
Cleber Rosa, Philippe Mathieu-Daudé,
Beraldo Leal
To run basic jobs on custom runners, the environment needs to be
properly set up. The most common requirement is having the right
packages installed.
The playbook introduced here covers a number of different Linux
distributions and FreeBSD, and are intended to provide a reproducible
environment.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
docs/devel/ci.rst | 32 ++++
scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
scripts/ci/setup/inventory | 2 +
3 files changed, 254 insertions(+)
create mode 100644 scripts/ci/setup/build-environment.yml
create mode 100644 scripts/ci/setup/inventory
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 41a4bbddad..208b5e399b 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as
The precise minimum requirements and exact rules for machine
configuration documentation/scripts, and the success rate of jobs are
still to be defined.
+
+Machine Setup Howto
+-------------------
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks. Start by adding your machines to
+the ``inventory`` file under ``scripts/ci/setup``, such as this::
+
+ [local]
+ fully.qualified.domain
+ other.machine.hostname
+
+You may need to set some variables in the inventory file itself. One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts. This would look like::
+
+ [local]
+ fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+ other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~~~~~~~~~~~~~~~~~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests. It covers a number of different Linux distributions and
+FreeBSD.
+
+To run the playbook, execute::
+
+ cd scripts/ci/setup
+ ansible-playbook -i inventory build-environment.yml
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
new file mode 100644
index 0000000000..0e8894bca9
--- /dev/null
+++ b/scripts/ci/setup/build-environment.yml
@@ -0,0 +1,220 @@
+---
+- name: Installation of basic packages to build QEMU
+ hosts: all
+ tasks:
+ - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+ apt:
+ update_cache: yes
+ # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+ pkg:
+ - ccache
+ - clang
+ - gcc
+ - gettext
+ - git
+ - glusterfs-common
+ - libaio-dev
+ - libattr1-dev
+ - libbrlapi-dev
+ - libbz2-dev
+ - libcacard-dev
+ - libcap-ng-dev
+ - libcurl4-gnutls-dev
+ - libdrm-dev
+ - libepoxy-dev
+ - libfdt-dev
+ - libgbm-dev
+ - libgtk-3-dev
+ - libibverbs-dev
+ - libiscsi-dev
+ - libjemalloc-dev
+ - libjpeg-turbo8-dev
+ - liblzo2-dev
+ - libncurses5-dev
+ - libncursesw5-dev
+ - libnfs-dev
+ - libnss3-dev
+ - libnuma-dev
+ - libpixman-1-dev
+ - librados-dev
+ - librbd-dev
+ - librdmacm-dev
+ - libsasl2-dev
+ - libsdl2-dev
+ - libseccomp-dev
+ - libsnappy-dev
+ - libspice-protocol-dev
+ - libssh-dev
+ - libusb-1.0-0-dev
+ - libusbredirhost-dev
+ - libvdeplug-dev
+ - libvte-2.91-dev
+ - libzstd-dev
+ - make
+ - python3-yaml
+ - python3-sphinx
+ - sparse
+ - xfslibs-dev
+ state: present
+ when: "ansible_facts['distribution'] == 'Ubuntu'"
+
+ - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
+ apt:
+ update_cache: yes
+ pkg:
+ - libspice-server-dev
+ - libxen-dev
+ state: present
+ when:
+ - "ansible_facts['distribution'] == 'Ubuntu'"
+ - "ansible_facts['architecture'] != 's390x'"
+
+ - name: Install basic packages to build QEMU on FreeBSD 12.x
+ pkgng:
+ # This matches the packages on .cirrus.yml under the freebsd_12_task
+ name: bash,bison,curl,cyrus-sasl,git,glib,gmake,gnutls,gsed,nettle,perl5,pixman,pkgconf,png,usbredir
+ state: present
+ when: "ansible_facts['os_family'] == 'FreeBSD'"
+
+ - name: Install basic packages to build QEMU on Fedora 30, 31 and 32
+ dnf:
+ # Originally from tests/docker/dockerfiles/fedora.docker
+ name:
+ - SDL2-devel
+ - bc
+ - brlapi-devel
+ - bzip2
+ - bzip2-devel
+ - ccache
+ - clang
+ - cyrus-sasl-devel
+ - dbus-daemon
+ - device-mapper-multipath-devel
+ - diffutils
+ - findutils
+ - gcc
+ - gcc-c++
+ - genisoimage
+ - gettext
+ - git
+ - glib2-devel
+ - glusterfs-api-devel
+ - gnutls-devel
+ - gtk3-devel
+ - hostname
+ - libaio-devel
+ - libasan
+ - libattr-devel
+ - libblockdev-mpath-devel
+ - libcap-ng-devel
+ - libcurl-devel
+ - libepoxy-devel
+ - libfdt-devel
+ - libiscsi-devel
+ - libjpeg-devel
+ - libpmem-devel
+ - libpng-devel
+ - librbd-devel
+ - libseccomp-devel
+ - libssh-devel
+ - libubsan
+ - libudev-devel
+ - libusbx-devel
+ - libxml2-devel
+ - libzstd-devel
+ - llvm
+ - lzo-devel
+ - make
+ - mingw32-SDL2
+ - mingw32-bzip2
+ - mingw32-curl
+ - mingw32-glib2
+ - mingw32-gmp
+ - mingw32-gnutls
+ - mingw32-gtk3
+ - mingw32-libjpeg-turbo
+ - mingw32-libpng
+ - mingw32-libtasn1
+ - mingw32-nettle
+ - mingw32-nsis
+ - mingw32-pixman
+ - mingw32-pkg-config
+ - mingw64-SDL2
+ - mingw64-bzip2
+ - mingw64-curl
+ - mingw64-glib2
+ - mingw64-gmp
+ - mingw64-gnutls
+ - mingw64-gtk3
+ - mingw64-libjpeg-turbo
+ - mingw64-libpng
+ - mingw64-libtasn1
+ - mingw64-nettle
+ - mingw64-pixman
+ - mingw64-pkg-config
+ - ncurses-devel
+ - nettle-devel
+ - nss-devel
+ - numactl-devel
+ - perl
+ - perl-Test-Harness
+ - pixman-devel
+ - python3
+ - python3-PyYAML
+ - python3-numpy
+ - python3-opencv
+ - python3-pillow
+ - python3-pip
+ - python3-sphinx
+ - python3-virtualenv
+ - rdma-core-devel
+ - snappy-devel
+ - sparse
+ - spice-server-devel
+ - systemd-devel
+ - systemtap-sdt-devel
+ - tar
+ - tesseract
+ - tesseract-langpack-eng
+ - usbredir-devel
+ - virglrenderer-devel
+ - vte291-devel
+ - which
+ - xen-devel
+ - zlib-devel
+ state: present
+ when: "ansible_facts['distribution'] == 'Fedora'"
+
+ - name: Install basic packages to build QEMU on CentOS 8
+ dnf:
+ # Originally from tests/docker/dockerfiles/centos8.docker
+ name:
+ - SDL-devel
+ - bzip2
+ - bzip2-devel
+ - dbus-daemon
+ - gcc
+ - gcc-c++
+ - genisoimage
+ - gettext
+ - git
+ - glib2-devel
+ - libaio-devel
+ - libepoxy-devel
+ - libgcrypt-devel
+ - lzo-devel
+ - make
+ - mesa-libEGL-devel
+ - nettle-devel
+ - perl-Test-Harness
+ - pixman-devel
+ - python36
+ - rdma-core-devel
+ - spice-glib-devel
+ - spice-server
+ - tar
+ - zlib-devel
+ state: present
+ when:
+ - "ansible_facts['distribution'] == 'CentOS'"
+ - "ansible_facts['distribution_major_version'] == '8'"
diff --git a/scripts/ci/setup/inventory b/scripts/ci/setup/inventory
new file mode 100644
index 0000000000..8bb7ba6b33
--- /dev/null
+++ b/scripts/ci/setup/inventory
@@ -0,0 +1,2 @@
+[local]
+localhost
--
2.25.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
2020-10-14 5:21 [PATCH v3 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook Cleber Rosa
@ 2020-10-14 5:21 ` Cleber Rosa
2020-10-14 17:31 ` Daniel P. Berrangé
2020-10-14 5:21 ` [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Cleber Rosa
3 siblings, 1 reply; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 5:21 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Peter Maydell
Cc: Thomas Huth, Daniel P . Berrangé,
Eduardo Habkost, Erik Skultety, Stefan Hajnoczi,
Andrea Bolognani, Wainer dos Santos Moschetta, Willian Rampazzo,
Cleber Rosa, Philippe Mathieu-Daudé,
Beraldo Leal
To have the jobs dispatched to custom runners, gitlab-runner must
be installed, active as a service and properly configured. The
variables file and playbook introduced here should help with those
steps.
The playbook introduced here covers a number of different Linux
distributions and FreeBSD, and are intended to provide a reproducible
environment.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
docs/devel/ci.rst | 63 ++++++++++++++++++++++++++
scripts/ci/setup/.gitignore | 1 +
scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++
scripts/ci/setup/vars.yml.template | 13 ++++++
4 files changed, 149 insertions(+)
create mode 100644 scripts/ci/setup/.gitignore
create mode 100644 scripts/ci/setup/gitlab-runner.yml
create mode 100644 scripts/ci/setup/vars.yml.template
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 208b5e399b..a234a5e24c 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -84,3 +84,66 @@ To run the playbook, execute::
cd scripts/ci/setup
ansible-playbook -i inventory build-environment.yml
+
+gitlab-runner setup and registration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The gitlab-runner agent needs to be installed on each machine that
+will run jobs. The association between a machine and a GitLab project
+happens with a registration token. To find the registration token for
+your repository/project, navigate on GitLab's web UI to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * Under "Set up a specific Runner manually", look for the value under
+ "Use the following registration token during setup"
+
+Copy the ``scripts/ci/setup/vars.yml.template`` file to
+``scripts/ci/setup/vars.yml``. Then, set the
+``gitlab_runner_registration_token`` variable to the value obtained
+earlier.
+
+.. note:: gitlab-runner is not available from the standard location
+ for all OS and architectures combinations. For some systems,
+ a custom build may be necessary. Some builds are avaiable
+ at https://cleber.fedorapeople.org/gitlab-runner/ and this
+ URI may be used as a value on ``vars.yml``
+
+To run the playbook, execute::
+
+ cd scripts/ci/setup
+ ansible-playbook -i inventory gitlab-runner.yml
+
+.. note:: there are currently limitations to gitlab-runner itself when
+ setting up a service under FreeBSD systems. You will need to
+ perform steps 4 to 10 manually, as described at
+ https://docs.gitlab.com/runner/install/freebsd.html
+
+Following the registration, it's necessary to configure the runner tags,
+and optionally other configurations on the GitLab UI. Navigate to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * "Runners activated for this project", then
+ * Click on the "Edit" icon (next to the "Lock" Icon)
+
+Under tags, add values matching the jobs a runner should run. For a
+FreeBSD 12.1 x86_64 system, the tags should be set as::
+
+ freebsd12.1,x86_64
+
+Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
+would contain::
+
+ freebsd-12.1-x86_64-all:
+ tags:
+ - freebsd_12.1
+ - x86_64
+
+It's also recommended to:
+
+ * increase the "Maximum job timeout" to something like ``2h``
+ * uncheck the "Run untagged jobs" check box
+ * give it a better Description
diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
new file mode 100644
index 0000000000..f112d05dd0
--- /dev/null
+++ b/scripts/ci/setup/.gitignore
@@ -0,0 +1 @@
+vars.yml
\ No newline at end of file
diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml
new file mode 100644
index 0000000000..c2f52dad10
--- /dev/null
+++ b/scripts/ci/setup/gitlab-runner.yml
@@ -0,0 +1,72 @@
+---
+- name: Installation of gitlab-runner
+ hosts: all
+ vars_files:
+ - vars.yml
+ tasks:
+ - debug:
+ msg: 'Checking for a valid GitLab registration token'
+ failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'"
+
+ - name: Checks the availability of official gitlab-runner builds in the archive
+ uri:
+ url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-linux-386
+ method: HEAD
+ status_code:
+ - 200
+ - 403
+ register: gitlab_runner_available_archive
+
+ - name: Update base url
+ set_fact:
+ gitlab_runner_base_url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-
+ when: gitlab_runner_available_archive.status == 200
+ - debug:
+ msg: Base gitlab-runner url is {{ gitlab_runner_base_url }}
+
+ - name: Set OS name (FreeBSD)
+ set_fact:
+ gitlab_runner_os: freebsd
+ when: "ansible_facts['system'] == 'FreeBSD'"
+
+ - name: Create a group for the gitlab-runner service
+ group:
+ name: gitlab-runner
+
+ - name: Create a user for the gitlab-runner service
+ user:
+ user: gitlab-runner
+ group: gitlab-runner
+ comment: GitLab Runner
+ home: /home/gitlab-runner
+ shell: /bin/bash
+
+ - name: Remove the .bash_logout file when on Ubuntu systems
+ file:
+ path: /home/gitlab-runner/.bash_logout
+ state: absent
+ when: "ansible_facts['distribution'] == 'Ubuntu'"
+
+ - name: Downloads the matching gitlab-runner
+ get_url:
+ dest: /usr/local/bin/gitlab-runner
+ url: "{{ gitlab_runner_base_url }}{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}"
+ owner: gitlab-runner
+ group: gitlab-runner
+ mode: u=rwx,g=rwx,o=rx
+
+ - name: Register the gitlab-runner
+ command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
+
+ - name: Install the gitlab-runner service using its own functionality
+ command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner
+ register: gitlab_runner_install_service_result
+ failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr"
+ when: "ansible_facts['os_family'] != 'FreeBSD'"
+
+ - name: Enable the gitlab-runner service
+ service:
+ name: gitlab-runner
+ state: started
+ enabled: yes
+ when: "ansible_facts['os_family'] != 'FreeBSD'"
diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template
new file mode 100644
index 0000000000..621435d030
--- /dev/null
+++ b/scripts/ci/setup/vars.yml.template
@@ -0,0 +1,13 @@
+# The version of the gitlab-runner to use
+gitlab_runner_version: 13.1.1
+# The base location of gitlab-runner binaries, this will be suffixed by $OS-$ARCH
+gitlab_runner_base_url: https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-
+# The URL of the gitlab server to use, usually https://gitlab.com unless you're
+# using a private GitLab instance
+gitlab_runner_server_url: https://gitlab.com
+# Defaults to linux, checks can be used to change this
+gitlab_runner_os: linux
+# Defaults to amd64 (x86_64), checks can be used to change this
+gitlab_runner_arch: amd64
+# A unique token made available by GitLab to your project for registering runners
+gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
--
2.25.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 5:21 [PATCH v3 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
` (2 preceding siblings ...)
2020-10-14 5:21 ` [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Cleber Rosa
@ 2020-10-14 5:21 ` Cleber Rosa
2020-10-14 17:46 ` Daniel P. Berrangé
3 siblings, 1 reply; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 5:21 UTC (permalink / raw)
To: qemu-devel, Alex Bennée, Peter Maydell
Cc: Thomas Huth, Daniel P . Berrangé,
Eduardo Habkost, Erik Skultety, Stefan Hajnoczi,
Andrea Bolognani, Wainer dos Santos Moschetta, Willian Rampazzo,
Cleber Rosa, Philippe Mathieu-Daudé,
Beraldo Leal
The QEMU project has two machines (aarch64 and s390) that can be used
for jobs that do build and run tests. This introduces those jobs,
which are a mapping of custom scripts used for the same purpose.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
.gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
1 file changed, 192 insertions(+)
diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 3004da2bda..5b51d1b336 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -12,3 +12,195 @@
# strategy.
variables:
GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-18.04-s390x-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-18.04-s390x-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-20.04-aarch64-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-20.04-aarch64-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-20.04-aarch64-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
--
2.25.4
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder
2020-10-14 5:21 ` [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
@ 2020-10-14 17:28 ` Daniel P. Berrangé
0 siblings, 0 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-14 17:28 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
On Wed, Oct 14, 2020 at 01:21:37AM -0400, Cleber Rosa wrote:
> As described in the included documentation, the "custom runner" jobs
> extend the GitLab CI jobs already in place.
>
> Those jobs are intended to run on hardware and/or Operating Systems
> not provided by GitLab's shared runners.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> .gitlab-ci.d/custom-runners.yml | 14 +++++++++
> .gitlab-ci.yml | 1 +
> docs/devel/ci.rst | 54 +++++++++++++++++++++++++++++++++
> docs/devel/index.rst | 1 +
> 4 files changed, 70 insertions(+)
> create mode 100644 .gitlab-ci.d/custom-runners.yml
> create mode 100644 docs/devel/ci.rst
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook
2020-10-14 5:21 ` [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook Cleber Rosa
@ 2020-10-14 17:30 ` Daniel P. Berrangé
2020-10-14 18:59 ` Cleber Rosa
0 siblings, 1 reply; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-14 17:30 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
On Wed, Oct 14, 2020 at 01:21:38AM -0400, Cleber Rosa wrote:
> To run basic jobs on custom runners, the environment needs to be
> properly set up. The most common requirement is having the right
> packages installed.
>
> The playbook introduced here covers a number of different Linux
> distributions and FreeBSD, and are intended to provide a reproducible
> environment.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> docs/devel/ci.rst | 32 ++++
> scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
> scripts/ci/setup/inventory | 2 +
> 3 files changed, 254 insertions(+)
> create mode 100644 scripts/ci/setup/build-environment.yml
> create mode 100644 scripts/ci/setup/inventory
>
> diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> index 41a4bbddad..208b5e399b 100644
> --- a/docs/devel/ci.rst
> +++ b/docs/devel/ci.rst
> @@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as
> The precise minimum requirements and exact rules for machine
> configuration documentation/scripts, and the success rate of jobs are
> still to be defined.
> +
> +Machine Setup Howto
> +-------------------
> +
> +For all Linux based systems, the setup can be mostly automated by the
> +execution of two Ansible playbooks. Start by adding your machines to
> +the ``inventory`` file under ``scripts/ci/setup``, such as this::
> +
> + [local]
> + fully.qualified.domain
> + other.machine.hostname
> +
> +You may need to set some variables in the inventory file itself. One
> +very common need is to tell Ansible to use a Python 3 interpreter on
> +those hosts. This would look like::
> +
> + [local]
> + fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
> + other.machine.hostname ansible_python_interpreter=/usr/bin/python3
> +
> +Build environment
> +~~~~~~~~~~~~~~~~~
> +
> +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
> +set up machines with the environment needed to perform builds and run
> +QEMU tests. It covers a number of different Linux distributions and
> +FreeBSD.
> +
> +To run the playbook, execute::
> +
> + cd scripts/ci/setup
> + ansible-playbook -i inventory build-environment.yml
> diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
> new file mode 100644
> index 0000000000..0e8894bca9
> --- /dev/null
> +++ b/scripts/ci/setup/build-environment.yml
> @@ -0,0 +1,220 @@
> +---
> +- name: Installation of basic packages to build QEMU
> + hosts: all
> + tasks:
> + - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
> + apt:
> + update_cache: yes
> + # Originally from tests/docker/dockerfiles/ubuntu1804.docker
> + pkg:
> + - ccache
> + - clang
> + - gcc
> + - gettext
> + - git
> + - glusterfs-common
> + - libaio-dev
> + - libattr1-dev
> + - libbrlapi-dev
> + - libbz2-dev
> + - libcacard-dev
> + - libcap-ng-dev
> + - libcurl4-gnutls-dev
> + - libdrm-dev
> + - libepoxy-dev
> + - libfdt-dev
> + - libgbm-dev
> + - libgtk-3-dev
> + - libibverbs-dev
> + - libiscsi-dev
> + - libjemalloc-dev
> + - libjpeg-turbo8-dev
> + - liblzo2-dev
> + - libncurses5-dev
> + - libncursesw5-dev
> + - libnfs-dev
> + - libnss3-dev
> + - libnuma-dev
> + - libpixman-1-dev
> + - librados-dev
> + - librbd-dev
> + - librdmacm-dev
> + - libsasl2-dev
> + - libsdl2-dev
> + - libseccomp-dev
> + - libsnappy-dev
> + - libspice-protocol-dev
> + - libssh-dev
> + - libusb-1.0-0-dev
> + - libusbredirhost-dev
> + - libvdeplug-dev
> + - libvte-2.91-dev
> + - libzstd-dev
> + - make
> + - python3-yaml
> + - python3-sphinx
> + - sparse
> + - xfslibs-dev
This needs updating to add meson, and with Paolo's series today you
might as well go ahead and add ninja-build immediately too
https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html
Same for all the other distro package lists.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
2020-10-14 5:21 ` [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Cleber Rosa
@ 2020-10-14 17:31 ` Daniel P. Berrangé
0 siblings, 0 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-14 17:31 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
On Wed, Oct 14, 2020 at 01:21:39AM -0400, Cleber Rosa wrote:
> To have the jobs dispatched to custom runners, gitlab-runner must
> be installed, active as a service and properly configured. The
> variables file and playbook introduced here should help with those
> steps.
>
> The playbook introduced here covers a number of different Linux
> distributions and FreeBSD, and are intended to provide a reproducible
> environment.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> docs/devel/ci.rst | 63 ++++++++++++++++++++++++++
> scripts/ci/setup/.gitignore | 1 +
> scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++
> scripts/ci/setup/vars.yml.template | 13 ++++++
> 4 files changed, 149 insertions(+)
> create mode 100644 scripts/ci/setup/.gitignore
> create mode 100644 scripts/ci/setup/gitlab-runner.yml
> create mode 100644 scripts/ci/setup/vars.yml.template
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 5:21 ` [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Cleber Rosa
@ 2020-10-14 17:46 ` Daniel P. Berrangé
2020-10-14 21:13 ` Cleber Rosa
0 siblings, 1 reply; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-14 17:46 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> The QEMU project has two machines (aarch64 and s390) that can be used
> for jobs that do build and run tests. This introduces those jobs,
> which are a mapping of custom scripts used for the same purpose.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
> 1 file changed, 192 insertions(+)
>
> diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> index 3004da2bda..5b51d1b336 100644
> --- a/.gitlab-ci.d/custom-runners.yml
> +++ b/.gitlab-ci.d/custom-runners.yml
> @@ -12,3 +12,195 @@
> # strategy.
> variables:
> GIT_SUBMODULE_STRATEGY: recursive
> +
> +# All ubuntu-18.04 jobs should run successfully in an environment
> +# setup by the scripts/ci/setup/build-environment.yml task
> +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> +ubuntu-18.04-s390x-all-linux-static:
> + needs: []
> + stage: build
> + tags:
> + - ubuntu_18.04
> + - s390x
> + rules:
> + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
IIRC, in the previous v2 (or was it v1) we discussed changing this
so that users who provide their own runners, don't have to always
use the "staging" branch name.
IIUC, the key thing is that we don't want the job running on the
"master" or "stable-*" branches in the primary QEMU git. So could
check
$CI_PROJECT_NAMESPACE == 'qemu-project'
&&
$CI_COMMIT_BRANCH !~ '^master$'
&&
$CI_COMMIT_BRANCH !~ '^stable-$'
which would let it work on users forks no matter what branch names
they use
What happens to the job if the user doesn't have runners ? Is it
simply skipped, or does the pipeline stall and get marked as failed ?
If the jobs aren't auto-skiped, we would need to add an env variable
(
$CI_PROJECT_NAMESPACE == 'qemu-project'
&&
$CI_COMMIT_BRANCH !~ '^master$'
&&
$CI_COMMIT_BRANCH !~ '^stable-$'
)
||
$QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
in the web UI for their fork
That all said, I don't mind if you postpone this rules change to a
followup patch.
> + script:
> + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
This bug links to
https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
which is marked as fixed. So I'm thinking we can drop the --disable-libssh
arg from all these jobs
> + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> + - mkdir build
> + - cd build
> + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> + - make --output-sync -j`nproc`
> + - make --output-sync -j`nproc` check V=1
> + - make --output-sync -j`nproc` check-tcg V=1
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook
2020-10-14 17:30 ` Daniel P. Berrangé
@ 2020-10-14 18:59 ` Cleber Rosa
2020-10-14 19:19 ` Cleber Rosa
0 siblings, 1 reply; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 18:59 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Stefan Hajnoczi, qemu-devel, Wainer dos Santos Moschetta,
Andrea Bolognani, Alex Bennée, Willian Rampazzo,
Philippe Mathieu-Daudé,
Beraldo Leal
[-- Attachment #1: Type: text/plain, Size: 5147 bytes --]
On Wed, Oct 14, 2020 at 06:30:09PM +0100, Daniel P. Berrangé wrote:
> On Wed, Oct 14, 2020 at 01:21:38AM -0400, Cleber Rosa wrote:
> > To run basic jobs on custom runners, the environment needs to be
> > properly set up. The most common requirement is having the right
> > packages installed.
> >
> > The playbook introduced here covers a number of different Linux
> > distributions and FreeBSD, and are intended to provide a reproducible
> > environment.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> > docs/devel/ci.rst | 32 ++++
> > scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
> > scripts/ci/setup/inventory | 2 +
> > 3 files changed, 254 insertions(+)
> > create mode 100644 scripts/ci/setup/build-environment.yml
> > create mode 100644 scripts/ci/setup/inventory
> >
> > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> > index 41a4bbddad..208b5e399b 100644
> > --- a/docs/devel/ci.rst
> > +++ b/docs/devel/ci.rst
> > @@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as
> > The precise minimum requirements and exact rules for machine
> > configuration documentation/scripts, and the success rate of jobs are
> > still to be defined.
> > +
> > +Machine Setup Howto
> > +-------------------
> > +
> > +For all Linux based systems, the setup can be mostly automated by the
> > +execution of two Ansible playbooks. Start by adding your machines to
> > +the ``inventory`` file under ``scripts/ci/setup``, such as this::
> > +
> > + [local]
> > + fully.qualified.domain
> > + other.machine.hostname
> > +
> > +You may need to set some variables in the inventory file itself. One
> > +very common need is to tell Ansible to use a Python 3 interpreter on
> > +those hosts. This would look like::
> > +
> > + [local]
> > + fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
> > + other.machine.hostname ansible_python_interpreter=/usr/bin/python3
> > +
> > +Build environment
> > +~~~~~~~~~~~~~~~~~
> > +
> > +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
> > +set up machines with the environment needed to perform builds and run
> > +QEMU tests. It covers a number of different Linux distributions and
> > +FreeBSD.
> > +
> > +To run the playbook, execute::
> > +
> > + cd scripts/ci/setup
> > + ansible-playbook -i inventory build-environment.yml
> > diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
> > new file mode 100644
> > index 0000000000..0e8894bca9
> > --- /dev/null
> > +++ b/scripts/ci/setup/build-environment.yml
> > @@ -0,0 +1,220 @@
> > +---
> > +- name: Installation of basic packages to build QEMU
> > + hosts: all
> > + tasks:
> > + - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
> > + apt:
> > + update_cache: yes
> > + # Originally from tests/docker/dockerfiles/ubuntu1804.docker
> > + pkg:
> > + - ccache
> > + - clang
> > + - gcc
> > + - gettext
> > + - git
> > + - glusterfs-common
> > + - libaio-dev
> > + - libattr1-dev
> > + - libbrlapi-dev
> > + - libbz2-dev
> > + - libcacard-dev
> > + - libcap-ng-dev
> > + - libcurl4-gnutls-dev
> > + - libdrm-dev
> > + - libepoxy-dev
> > + - libfdt-dev
> > + - libgbm-dev
> > + - libgtk-3-dev
> > + - libibverbs-dev
> > + - libiscsi-dev
> > + - libjemalloc-dev
> > + - libjpeg-turbo8-dev
> > + - liblzo2-dev
> > + - libncurses5-dev
> > + - libncursesw5-dev
> > + - libnfs-dev
> > + - libnss3-dev
> > + - libnuma-dev
> > + - libpixman-1-dev
> > + - librados-dev
> > + - librbd-dev
> > + - librdmacm-dev
> > + - libsasl2-dev
> > + - libsdl2-dev
> > + - libseccomp-dev
> > + - libsnappy-dev
> > + - libspice-protocol-dev
> > + - libssh-dev
> > + - libusb-1.0-0-dev
> > + - libusbredirhost-dev
> > + - libvdeplug-dev
> > + - libvte-2.91-dev
> > + - libzstd-dev
> > + - make
> > + - python3-yaml
> > + - python3-sphinx
> > + - sparse
> > + - xfslibs-dev
>
> This needs updating to add meson, and with Paolo's series today you
> might as well go ahead and add ninja-build immediately too
>
> https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html
>
> Same for all the other distro package lists.
>
>
Good point, will do.
Thanks!
- Cleber.
> Regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
>
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook
2020-10-14 18:59 ` Cleber Rosa
@ 2020-10-14 19:19 ` Cleber Rosa
2020-10-15 8:29 ` Daniel P. Berrangé
0 siblings, 1 reply; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 19:19 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Stefan Hajnoczi, qemu-devel, Wainer dos Santos Moschetta,
Andrea Bolognani, Alex Bennée, Willian Rampazzo,
Philippe Mathieu-Daudé,
Beraldo Leal
[-- Attachment #1: Type: text/plain, Size: 745 bytes --]
On Wed, Oct 14, 2020 at 02:59:58PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 06:30:09PM +0100, Daniel P. Berrangé wrote:
> >
> > This needs updating to add meson, and with Paolo's series today you
> > might as well go ahead and add ninja-build immediately too
> >
I replied too quickly, but allow me to get this right: meson is *not*
included in the dockerfiles (and other similar configurations), and
all setups I found rely on the submodule. Are suggesting to add meson
and diverge from the dockerfiles?
> > https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html
> >
^ I'll add meson according to this, of course.
Thanks,
- Cleber.
> > Same for all the other distro package lists.
> >
> >
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 17:46 ` Daniel P. Berrangé
@ 2020-10-14 21:13 ` Cleber Rosa
2020-10-14 23:24 ` Cleber Rosa
2020-10-15 8:34 ` Daniel P. Berrangé
0 siblings, 2 replies; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 21:13 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
[-- Attachment #1: Type: text/plain, Size: 4753 bytes --]
On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> > The QEMU project has two machines (aarch64 and s390) that can be used
> > for jobs that do build and run tests. This introduces those jobs,
> > which are a mapping of custom scripts used for the same purpose.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> > .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
> > 1 file changed, 192 insertions(+)
> >
> > diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> > index 3004da2bda..5b51d1b336 100644
> > --- a/.gitlab-ci.d/custom-runners.yml
> > +++ b/.gitlab-ci.d/custom-runners.yml
> > @@ -12,3 +12,195 @@
> > # strategy.
> > variables:
> > GIT_SUBMODULE_STRATEGY: recursive
> > +
> > +# All ubuntu-18.04 jobs should run successfully in an environment
> > +# setup by the scripts/ci/setup/build-environment.yml task
> > +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> > +ubuntu-18.04-s390x-all-linux-static:
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>
> IIRC, in the previous v2 (or was it v1) we discussed changing this
> so that users who provide their own runners, don't have to always
> use the "staging" branch name.
>
Right, and what I got from that is that users can use a *prefix* as a
flag to indicate that they want the extra set of jobs.
> IIUC, the key thing is that we don't want the job running on the
> "master" or "stable-*" branches in the primary QEMU git. So could
> check
>
> $CI_PROJECT_NAMESPACE == 'qemu-project'
> &&
> $CI_COMMIT_BRANCH !~ '^master$'
> &&
> $CI_COMMIT_BRANCH !~ '^stable-$'
>
> which would let it work on users forks no matter what branch names
> they use
>
> What happens to the job if the user doesn't have runners ? Is it
> simply skipped, or does the pipeline stall and get marked as failed ?
>
The pipeline gets "stuck" (literaly, that's the status name it gets).
That's the main issue that made me believe that opting *in* (by using
a common branch name prefix) was the simpler solution.
> If the jobs aren't auto-skiped, we would need to add an env variable
>
> (
> $CI_PROJECT_NAMESPACE == 'qemu-project'
> &&
> $CI_COMMIT_BRANCH !~ '^master$'
> &&
> $CI_COMMIT_BRANCH !~ '^stable-$'
> )
> ||
> $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
>
> and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
> in the web UI for their fork
>
We can do that, but I think it's more than we need. The odds that a
user will have all of the same runners, and will be able to run all
the extra jobs, are very very low IMO. Right from the start, very few
people have an s390 machine running Ubuntu 18.04.
So, I believe that whenever a user pushes to a branch such as
"staging-topic-foo", he will have to deal with some of the extra jobs
(such as canceling the ones that will never run) anyway. Having to
deal with those on every single push, or alternatively having to turn
on/off $QEMU_ENABLE_CUSTOM_RUNNERS doesn't the best experience to me.
The "staging" prefix convention (with a better prefix name now or in
the future?) seems to result in the best experience to me.
> That all said, I don't mind if you postpone this rules change to a
> followup patch.
>
OK, let me know if you agree with my explanation above, or if you
really want to see a followup patch.
> > + script:
> > + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
>
> This bug links to
>
> https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
>
> which is marked as fixed. So I'm thinking we can drop the --disable-libssh
> arg from all these jobs
>
OK, I'll double check that and, if the fix is comfirmed beyond the bug
tracker, I'll update it.
Thanks!
- Cleber.
> > + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
> > + - mkdir build
> > + - cd build
> > + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
> > + - make --output-sync -j`nproc`
> > + - make --output-sync -j`nproc` check V=1
> > + - make --output-sync -j`nproc` check-tcg V=1
>
>
>
> Regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 21:13 ` Cleber Rosa
@ 2020-10-14 23:24 ` Cleber Rosa
2020-10-15 6:09 ` Thomas Huth
2020-10-15 8:32 ` Daniel P. Berrangé
2020-10-15 8:34 ` Daniel P. Berrangé
1 sibling, 2 replies; 18+ messages in thread
From: Cleber Rosa @ 2020-10-14 23:24 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
[-- Attachment #1: Type: text/plain, Size: 926 bytes --]
On Wed, Oct 14, 2020 at 05:14:01PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> >
> > This bug links to
> >
> > https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
> >
> > which is marked as fixed. So I'm thinking we can drop the --disable-libssh
> > arg from all these jobs
> >
>
> OK, I'll double check that and, if the fix is comfirmed beyond the bug
> tracker, I'll update it.
>
I was still able to reproduce the build failure with the latest packages.
The last update on the bug, is very misleading, but there's a hint of
its outcome:
---
Martin Wimpress (flexiondotorg) on 2019-12-17
tags: added: rls-bb-notfixing
tags: removed: bionic
no longer affects: libssh (Ubuntu Bionic)
---
So, *notfixing* means it no longer affects that package on that
distro? Right...
Anyway, keeping this AS IS.
Regards,
- Cleber.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 23:24 ` Cleber Rosa
@ 2020-10-15 6:09 ` Thomas Huth
2020-10-15 8:32 ` Daniel P. Berrangé
1 sibling, 0 replies; 18+ messages in thread
From: Thomas Huth @ 2020-10-15 6:09 UTC (permalink / raw)
To: Cleber Rosa, Daniel P. Berrangé, Peter Maydell
Cc: Eduardo Habkost, Erik Skultety, Alex Bennée, qemu-devel,
Wainer dos Santos Moschetta, Andrea Bolognani, Willian Rampazzo,
Stefan Hajnoczi, Philippe Mathieu-Daudé,
Beraldo Leal
On 15/10/2020 01.24, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 05:14:01PM -0400, Cleber Rosa wrote:
>> On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
>>>
>>> This bug links to
>>>
>>> https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
>>>
>>> which is marked as fixed. So I'm thinking we can drop the --disable-libssh
>>> arg from all these jobs
>>>
>>
>> OK, I'll double check that and, if the fix is comfirmed beyond the bug
>> tracker, I'll update it.
>>
>
> I was still able to reproduce the build failure with the latest packages.
>
> The last update on the bug, is very misleading, but there's a hint of
> its outcome:
>
> ---
> Martin Wimpress (flexiondotorg) on 2019-12-17
> tags: added: rls-bb-notfixing
> tags: removed: bionic
> no longer affects: libssh (Ubuntu Bionic)
> ---
>
> So, *notfixing* means it no longer affects that package on that
> distro? Right...
>
> Anyway, keeping this AS IS.
I can confirm that the libssh bug is still there in Ubuntu 18.04 ... when I
was updating our .travis.yml some weeks ago to stop using Xenial, I also hit
the bug again on Bionic.
Maybe rather a question for Peter, but what about updating the runners to
20.04 (Focal) instead? The libssh bug is gone there.
Thomas
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook
2020-10-14 19:19 ` Cleber Rosa
@ 2020-10-15 8:29 ` Daniel P. Berrangé
2020-10-19 1:43 ` Cleber Rosa
0 siblings, 1 reply; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-15 8:29 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Stefan Hajnoczi, qemu-devel, Wainer dos Santos Moschetta,
Andrea Bolognani, Alex Bennée, Willian Rampazzo,
Philippe Mathieu-Daudé,
Beraldo Leal
On Wed, Oct 14, 2020 at 03:19:47PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 02:59:58PM -0400, Cleber Rosa wrote:
> > On Wed, Oct 14, 2020 at 06:30:09PM +0100, Daniel P. Berrangé wrote:
> > >
> > > This needs updating to add meson, and with Paolo's series today you
> > > might as well go ahead and add ninja-build immediately too
> > >
>
> I replied too quickly, but allow me to get this right: meson is *not*
> included in the dockerfiles (and other similar configurations), and
> all setups I found rely on the submodule. Are suggesting to add meson
> and diverge from the dockerfiles?
Doh, right, I forgot that we use the submodule for now, since we need
such a new meson. So ignore this...
> > > https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html
> > >
>
> ^ I'll add meson according to this, of course.
Just ninja is needed
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 23:24 ` Cleber Rosa
2020-10-15 6:09 ` Thomas Huth
@ 2020-10-15 8:32 ` Daniel P. Berrangé
1 sibling, 0 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-15 8:32 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Stefan Hajnoczi, Alex Bennée,
Beraldo Leal
On Wed, Oct 14, 2020 at 07:24:30PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 05:14:01PM -0400, Cleber Rosa wrote:
> > On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> > >
> > > This bug links to
> > >
> > > https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514
> > >
> > > which is marked as fixed. So I'm thinking we can drop the --disable-libssh
> > > arg from all these jobs
> > >
> >
> > OK, I'll double check that and, if the fix is comfirmed beyond the bug
> > tracker, I'll update it.
> >
>
> I was still able to reproduce the build failure with the latest packages.
>
> The last update on the bug, is very misleading, but there's a hint of
> its outcome:
>
> ---
> Martin Wimpress (flexiondotorg) on 2019-12-17
> tags: added: rls-bb-notfixing
> tags: removed: bionic
> no longer affects: libssh (Ubuntu Bionic)
> ---
>
> So, *notfixing* means it no longer affects that package on that
> distro? Right...
Urgh, launchpad is such a confusing bug tracker :-(
> Anyway, keeping this AS IS.
Yep
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
2020-10-14 21:13 ` Cleber Rosa
2020-10-14 23:24 ` Cleber Rosa
@ 2020-10-15 8:34 ` Daniel P. Berrangé
1 sibling, 0 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2020-10-15 8:34 UTC (permalink / raw)
To: Cleber Rosa
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Stefan Hajnoczi, qemu-devel, Wainer dos Santos Moschetta,
Andrea Bolognani, Alex Bennée, Willian Rampazzo,
Philippe Mathieu-Daudé,
Beraldo Leal
On Wed, Oct 14, 2020 at 05:13:56PM -0400, Cleber Rosa wrote:
> On Wed, Oct 14, 2020 at 06:46:55PM +0100, Daniel P. Berrangé wrote:
> > On Wed, Oct 14, 2020 at 01:21:40AM -0400, Cleber Rosa wrote:
> > > The QEMU project has two machines (aarch64 and s390) that can be used
> > > for jobs that do build and run tests. This introduces those jobs,
> > > which are a mapping of custom scripts used for the same purpose.
> > >
> > > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > > ---
> > > .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
> > > 1 file changed, 192 insertions(+)
> > >
> > > diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> > > index 3004da2bda..5b51d1b336 100644
> > > --- a/.gitlab-ci.d/custom-runners.yml
> > > +++ b/.gitlab-ci.d/custom-runners.yml
> > > @@ -12,3 +12,195 @@
> > > # strategy.
> > > variables:
> > > GIT_SUBMODULE_STRATEGY: recursive
> > > +
> > > +# All ubuntu-18.04 jobs should run successfully in an environment
> > > +# setup by the scripts/ci/setup/build-environment.yml task
> > > +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> > > +ubuntu-18.04-s390x-all-linux-static:
> > > + needs: []
> > > + stage: build
> > > + tags:
> > > + - ubuntu_18.04
> > > + - s390x
> > > + rules:
> > > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> >
> > IIRC, in the previous v2 (or was it v1) we discussed changing this
> > so that users who provide their own runners, don't have to always
> > use the "staging" branch name.
> >
>
> Right, and what I got from that is that users can use a *prefix* as a
> flag to indicate that they want the extra set of jobs.
>
> > IIUC, the key thing is that we don't want the job running on the
> > "master" or "stable-*" branches in the primary QEMU git. So could
> > check
> >
> > $CI_PROJECT_NAMESPACE == 'qemu-project'
> > &&
> > $CI_COMMIT_BRANCH !~ '^master$'
> > &&
> > $CI_COMMIT_BRANCH !~ '^stable-$'
> >
> > which would let it work on users forks no matter what branch names
> > they use
> >
> > What happens to the job if the user doesn't have runners ? Is it
> > simply skipped, or does the pipeline stall and get marked as failed ?
> >
>
> The pipeline gets "stuck" (literaly, that's the status name it gets).
> That's the main issue that made me believe that opting *in* (by using
> a common branch name prefix) was the simpler solution.
Hmm, that's very annoying behaviour.
> > If the jobs aren't auto-skiped, we would need to add an env variable
> >
> > (
> > $CI_PROJECT_NAMESPACE == 'qemu-project'
> > &&
> > $CI_COMMIT_BRANCH !~ '^master$'
> > &&
> > $CI_COMMIT_BRANCH !~ '^stable-$'
> > )
> > ||
> > $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
> >
> > and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable
> > in the web UI for their fork
> >
>
> We can do that, but I think it's more than we need. The odds that a
> user will have all of the same runners, and will be able to run all
> the extra jobs, are very very low IMO. Right from the start, very few
> people have an s390 machine running Ubuntu 18.04.
>
> So, I believe that whenever a user pushes to a branch such as
> "staging-topic-foo", he will have to deal with some of the extra jobs
> (such as canceling the ones that will never run) anyway. Having to
> deal with those on every single push, or alternatively having to turn
> on/off $QEMU_ENABLE_CUSTOM_RUNNERS doesn't the best experience to me.
>
> The "staging" prefix convention (with a better prefix name now or in
> the future?) seems to result in the best experience to me.
Well "staging" prefix wasn';t appealing to me since none of the branches
I work on have such a name prefix.
> > That all said, I don't mind if you postpone this rules change to a
> > followup patch.
> >
>
> OK, let me know if you agree with my explanation above, or if you
> really want to see a followup patch.
Just ignore it for now. I'll do more thinking to see if I can figure
out a more attractive solution.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook
2020-10-15 8:29 ` Daniel P. Berrangé
@ 2020-10-19 1:43 ` Cleber Rosa
0 siblings, 0 replies; 18+ messages in thread
From: Cleber Rosa @ 2020-10-19 1:43 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Peter Maydell, Thomas Huth, Eduardo Habkost, Erik Skultety,
Stefan Hajnoczi, Philippe Mathieu-Daudé,
qemu-devel, Wainer dos Santos Moschetta, Andrea Bolognani,
Willian Rampazzo, Alex Bennée, Beraldo Leal
[-- Attachment #1: Type: text/plain, Size: 1055 bytes --]
On Thu, Oct 15, 2020 at 09:29:40AM +0100, Daniel P. Berrangé wrote:
> On Wed, Oct 14, 2020 at 03:19:47PM -0400, Cleber Rosa wrote:
> > On Wed, Oct 14, 2020 at 02:59:58PM -0400, Cleber Rosa wrote:
> > > On Wed, Oct 14, 2020 at 06:30:09PM +0100, Daniel P. Berrangé wrote:
> > > >
> > > > This needs updating to add meson, and with Paolo's series today you
> > > > might as well go ahead and add ninja-build immediately too
> > > >
> >
> > I replied too quickly, but allow me to get this right: meson is *not*
> > included in the dockerfiles (and other similar configurations), and
> > all setups I found rely on the submodule. Are suggesting to add meson
> > and diverge from the dockerfiles?
>
> Doh, right, I forgot that we use the submodule for now, since we need
> such a new meson. So ignore this...
>
> > > > https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html
> > > >
> >
> > ^ I'll add meson according to this, of course.
>
> Just ninja is needed
>
Right, I meant ninja there!
Thanks,
- Cleber.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2020-10-19 1:52 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-14 5:21 [PATCH v3 0/4] GitLab Custom Runners and Jobs (was: QEMU Gating CI) Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder Cleber Rosa
2020-10-14 17:28 ` Daniel P. Berrangé
2020-10-14 5:21 ` [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook Cleber Rosa
2020-10-14 17:30 ` Daniel P. Berrangé
2020-10-14 18:59 ` Cleber Rosa
2020-10-14 19:19 ` Cleber Rosa
2020-10-15 8:29 ` Daniel P. Berrangé
2020-10-19 1:43 ` Cleber Rosa
2020-10-14 5:21 ` [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook Cleber Rosa
2020-10-14 17:31 ` Daniel P. Berrangé
2020-10-14 5:21 ` [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines Cleber Rosa
2020-10-14 17:46 ` Daniel P. Berrangé
2020-10-14 21:13 ` Cleber Rosa
2020-10-14 23:24 ` Cleber Rosa
2020-10-15 6:09 ` Thomas Huth
2020-10-15 8:32 ` Daniel P. Berrangé
2020-10-15 8:34 ` Daniel P. Berrangé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).