All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 1/8] ci: add README and makefile for containers
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
@ 2018-03-15 18:17 ` Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 2/8] ci: add Dockerfile for CentOS 7.2 Doug Goldstein
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Add a basic README explaining the containers and how people can use them
to locally test with if they see an error in CI and want to reproduce it
locally. Added a makefile to help with building and pushing the
containers to the container registry.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/Makefile  | 17 +++++++++++++++++
 automation/build/README.md | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+)
 create mode 100644 automation/build/Makefile
 create mode 100644 automation/build/README.md

diff --git a/automation/build/Makefile b/automation/build/Makefile
new file mode 100644
index 0000000..773b160
--- /dev/null
+++ b/automation/build/Makefile
@@ -0,0 +1,17 @@
+
+# the base of where these containers will appear
+REGISTRY := registry.gitlab.com/xen-project/xen
+
+help:
+	@echo "Builds containers for building Xen based on different distros"
+	@echo "To build one run 'make DISTRO/VERSION'. Available containers:"
+	@$(foreach file,$(sort $(subst .dockerfile,,$(wildcard */*.dockerfile))), \
+		echo ${file} ; \
+	)
+	@echo "To push container builds, set the env var PUSH"
+
+%: %.dockerfile ## Builds containers
+	docker build -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
+	@if [ ! -z $${PUSH+x} ]; then \
+		docker push $(REGISTRY)/$(@D):$(@F); \
+	fi
diff --git a/automation/build/README.md b/automation/build/README.md
new file mode 100644
index 0000000..0206d57
--- /dev/null
+++ b/automation/build/README.md
@@ -0,0 +1,34 @@
+Docker Containers
+=================
+
+These Docker containers should make it possible to build Xen in
+any of the available environments on any system that supports
+running Docker. They are organized by distro and tagged with
+the version of that distro. They are available from the GitLab
+Container Registry under the Xen project at:
+
+registry.gitlab.com/xen-project/xen/DISTRO:VERSION
+
+To see the list of available containers run `make` in this
+directory. You will have to replace the `/` with a `:` to use
+them.
+
+Building Xen
+------------
+
+From the top level of the source tree it should be possible to
+run the following:
+
+docker run --rm -it -v $(PWD):/build -u $(id -u) -e CC=gcc $(CONTAINER) make
+
+There are other modifications that can be made but this will run
+the `make` command inside the specified container. It will use your
+currently checked out source tree to build with, ensure that file
+permissions remain consistent and clean up after itself.
+
+Building a container
+--------------------
+
+There is a makefile to make this process easier. You should be
+able to run `make DISTRO/VERSION` to have Docker build the container
+for you.
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 2/8] ci: add Dockerfile for CentOS 7.2
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 1/8] ci: add README and makefile for containers Doug Goldstein
@ 2018-03-15 18:17 ` Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 3/8] ci: add Dockerfile for Ubuntu 14.04 Doug Goldstein
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Added a Dockerfile which captures all the necessary dependencies to
build Xen on a CentOS 7.2 system.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/centos/7.2.dockerfile  | 41 ++++++++++++++++++++++++++-
 automation/build/centos/CentOS-7.2.repo | 35 ++++++++++++++++++++++-
 2 files changed, 76 insertions(+)
 create mode 100644 automation/build/centos/7.2.dockerfile
 create mode 100644 automation/build/centos/CentOS-7.2.repo

diff --git a/automation/build/centos/7.2.dockerfile b/automation/build/centos/7.2.dockerfile
new file mode 100644
index 0000000..a246781
--- /dev/null
+++ b/automation/build/centos/7.2.dockerfile
@@ -0,0 +1,41 @@
+FROM centos:7.2.1511
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+# ensure we only get bits from the vault for
+# the version we want
+COPY CentOS-7.2.repo /etc/yum.repos.d/CentOS-Base.repo
+
+RUN mkdir /build
+WORKDIR /build
+
+# work around https://github.com/moby/moby/issues/10180
+# and install Xen depends
+RUN rpm --rebuilddb && \
+    yum -y install \
+        yum-plugin-ovl \
+        gcc \
+        gcc-c++ \
+        ncurses-devel \
+        zlib-devel \
+        openssl-devel \
+        python-devel \
+        libuuid-devel \
+        pkgconfig \
+        gettext \
+        flex \
+        bison \
+        libaio-devel \
+        glib2-devel \
+        yajl-devel \
+        pixman-devel \
+        glibc-devel \
+        glibc-devel.i686 \
+        make \
+        binutils \
+        git \
+        wget \
+        acpica-tools \
+        python-markdown \
+        patch \
+    && yum clean all
diff --git a/automation/build/centos/CentOS-7.2.repo b/automation/build/centos/CentOS-7.2.repo
new file mode 100644
index 0000000..4da27fa
--- /dev/null
+++ b/automation/build/centos/CentOS-7.2.repo
@@ -0,0 +1,35 @@
+# CentOS-Base.repo
+#
+# This is a replacement file that pins things to just use CentOS 7.2
+# from the CentOS Vault.
+#
+
+[base]
+name=CentOS-7.2.1511 - Base
+baseurl=http://vault.centos.org/7.2.1511/os/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#released updates 
+[updates]
+name=CentOS-7.2.1511 - Updates
+baseurl=http://vault.centos.org/7.2.1511/updates/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#additional packages that may be useful
+[extras]
+name=CentOS-7.2.1511 - Extras
+baseurl=http://vault.centos.org/7.2.1511/extras/$basearch/
+gpgcheck=1
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
+#additional packages that extend functionality of existing packages
+[centosplus]
+name=CentOS-7.2.1511 - Plus
+baseurl=http://vault.centos.org/7.2.1511/centosplus/$basearch/
+gpgcheck=1
+gpgcheck=1
+enabled=0
+gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
+
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 3/8] ci: add Dockerfile for Ubuntu 14.04
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 1/8] ci: add README and makefile for containers Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 2/8] ci: add Dockerfile for CentOS 7.2 Doug Goldstein
@ 2018-03-15 18:17 ` Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 4/8] ci: add Dockerfile for Ubuntu 16.04 Doug Goldstein
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Added a Dockerfile which captures all the necessary dependencies to
build Xen on a Ubuntu 14.04 system.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/ubuntu/trusty.dockerfile | 47 ++++++++++++++++++++++++-
 1 file changed, 47 insertions(+)
 create mode 100644 automation/build/ubuntu/trusty.dockerfile

diff --git a/automation/build/ubuntu/trusty.dockerfile b/automation/build/ubuntu/trusty.dockerfile
new file mode 100644
index 0000000..cc75087
--- /dev/null
+++ b/automation/build/ubuntu/trusty.dockerfile
@@ -0,0 +1,47 @@
+FROM ubuntu:14.04
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python2.7-dev \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        gettext \
+        acpica-tools \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        libc6-dev-i386 \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 4/8] ci: add Dockerfile for Ubuntu 16.04
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (2 preceding siblings ...)
  2018-03-15 18:17 ` [PATCH v2 3/8] ci: add Dockerfile for Ubuntu 14.04 Doug Goldstein
@ 2018-03-15 18:17 ` Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 5/8] ci: add Dockerfile for Debian jessie Doug Goldstein
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Added a Dockerfile which captures all the necessary dependencies to
build Xen on a Ubuntu 16.04 system.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/ubuntu/xenial.dockerfile | 47 ++++++++++++++++++++++++-
 1 file changed, 47 insertions(+)
 create mode 100644 automation/build/ubuntu/xenial.dockerfile

diff --git a/automation/build/ubuntu/xenial.dockerfile b/automation/build/ubuntu/xenial.dockerfile
new file mode 100644
index 0000000..aa551c1
--- /dev/null
+++ b/automation/build/ubuntu/xenial.dockerfile
@@ -0,0 +1,47 @@
+FROM ubuntu:16.04
+LABEL maintainer.name="The Xen Project " \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python2.7-dev \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        gettext \
+        acpica-tools \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        libc6-dev-i386 \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 5/8] ci: add Dockerfile for Debian jessie
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (3 preceding siblings ...)
  2018-03-15 18:17 ` [PATCH v2 4/8] ci: add Dockerfile for Ubuntu 16.04 Doug Goldstein
@ 2018-03-15 18:17 ` Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 6/8] ci: add Dockerfile for Debian stretch Doug Goldstein
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Added a Dockerfile which captures all the necessary dependencies to
build Xen on a Debian jessie system.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/debian/jessie.dockerfile | 47 ++++++++++++++++++++++++-
 1 file changed, 47 insertions(+)
 create mode 100644 automation/build/debian/jessie.dockerfile

diff --git a/automation/build/debian/jessie.dockerfile b/automation/build/debian/jessie.dockerfile
new file mode 100644
index 0000000..9bb1bdf
--- /dev/null
+++ b/automation/build/debian/jessie.dockerfile
@@ -0,0 +1,47 @@
+FROM debian:jessie
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python2.7-dev \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        gettext \
+        acpica-tools \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        libc6-dev-i386 \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 6/8] ci: add Dockerfile for Debian stretch
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (4 preceding siblings ...)
  2018-03-15 18:17 ` [PATCH v2 5/8] ci: add Dockerfile for Debian jessie Doug Goldstein
@ 2018-03-15 18:17 ` Doug Goldstein
  2018-03-15 18:18 ` [PATCH v2 7/8] ci: add cfg to use GitLab CI to build Doug Goldstein
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Added a Dockerfile which captures all the necessary dependencies to
build Xen on a Debian stretch system.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/debian/stretch.dockerfile | 47 +++++++++++++++++++++++-
 1 file changed, 47 insertions(+)
 create mode 100644 automation/build/debian/stretch.dockerfile

diff --git a/automation/build/debian/stretch.dockerfile b/automation/build/debian/stretch.dockerfile
new file mode 100644
index 0000000..f068457
--- /dev/null
+++ b/automation/build/debian/stretch.dockerfile
@@ -0,0 +1,47 @@
+FROM debian:stretch
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python2.7-dev \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        gettext \
+        acpica-tools \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        libc6-dev-i386 \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 7/8] ci: add cfg to use GitLab CI to build
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (5 preceding siblings ...)
  2018-03-15 18:17 ` [PATCH v2 6/8] ci: add Dockerfile for Debian stretch Doug Goldstein
@ 2018-03-15 18:18 ` Doug Goldstein
  2018-03-15 18:18 ` [PATCH v2 8/8] ci: add new bits to MAINTAINERS combine with Travis Doug Goldstein
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:18 UTC (permalink / raw)
  To: xen-devel; +Cc: Doug Goldstein

Added a GitLab CI config which has a lot more flexibility to allow us to
test a lot more distro configurations than Travis can and even build
test on FreeBSD.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 .gitlab-ci.yml | 164 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 164 insertions(+)
 create mode 100644 .gitlab-ci.yml

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 0000000..1d0c6c4
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,164 @@
+stages:
+  - build
+
+centos-7-2:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/centos:7.2
+  variables:
+    CC: gcc
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+centos-7-2-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/centos:7.2
+  variables:
+    CC: gcc
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+debian-jessie-clang:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:jessie
+  variables:
+    CC: clang
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+debian-jessie-clang-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:jessie
+  variables:
+    CC: clang
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+debian-jessie-gcc:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:jessie
+  variables:
+    CC: gcc
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+debian-jessie-gcc-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:jessie
+  variables:
+    CC: gcc
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+debian-stretch-clang:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:stretch
+  variables:
+    CC: clang
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+debian-stretch-clang-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:stretch
+  variables:
+    CC: clang
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+debian-stretch-gcc:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:stretch
+  variables:
+    CC: gcc
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+debian-stretch-gcc-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/debian:stretch
+  variables:
+    CC: gcc
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+ubuntu-trusty-clang:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:trusty
+  variables:
+    CC: clang
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+ubuntu-trusty-clang-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:trusty
+  variables:
+    CC: clang
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+ubuntu-trusty-gcc:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:trusty
+  variables:
+    CC: gcc
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+ubuntu-trusty-gcc-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:trusty
+  variables:
+    CC: gcc
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+ubuntu-xenial-clang:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:xenial
+  variables:
+    CC: clang
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+ubuntu-xenial-clang-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:xenial
+  variables:
+    CC: clang
+    debug: y
+  script:
+    - ./scripts/travis-build
+
+ubuntu-xenial-gcc:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:xenial
+  variables:
+    CC: gcc
+    debug: n
+  script:
+    - ./scripts/travis-build
+
+ubuntu-xenial-gcc-debug:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/ubuntu:xenial
+  variables:
+    CC: gcc
+    debug: y
+  script:
+    - ./scripts/travis-build
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 8/8] ci: add new bits to MAINTAINERS combine with Travis
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (6 preceding siblings ...)
  2018-03-15 18:18 ` [PATCH v2 7/8] ci: add cfg to use GitLab CI to build Doug Goldstein
@ 2018-03-15 18:18 ` Doug Goldstein
  2018-03-15 18:51 ` [PATCH v2 0/8] Using GitLab CI for build testing Konrad Rzeszutek Wilk
  2018-03-16  1:55 ` John Thomson
  9 siblings, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:18 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Wei Liu, George Dunlap, Andrew Cooper,
	Doug Goldstein, Tim Deegan, Julien Grall, Jan Beulich,
	Ian Jackson

Created a new section just called 'CI' since this is adding GitLab CI
and still leaving the old Travis CI files around. This consolidates the
two sections and adds the new files as well as adding another Travis
file that was missing.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
---
 MAINTAINERS | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index a5b3e95..81ec312 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -181,6 +181,16 @@ BLKTAP2
 S:	Orphaned
 F:	tools/blktap2/
 
+CI
+M:	Doug Goldstein <cardoe@cardoe.com>
+W:	https://gitlab.com/xen-project/xen
+W:	https://travis-ci.org/xen-project/xen
+S:	Supported
+F:	.gitlab-ci.yml
+F:	.travis.yml
+F:	automation/
+F:	scripts/travis-build
+
 CPU POOLS
 M:	Juergen Gross <jgross@suse.com>
 M:	Dario Faggioli <dfaggioli@suse.com>
@@ -384,12 +394,6 @@ F:	xen/common/tmem*
 F:	xen/include/xen/tmem*
 F:	docs/misc/tmem*
 
-TRAVIS CI
-M:	Doug Goldstein <cardoe@cardoe.com>
-W:	https://travis-ci.org/xen-project/xen
-S:	Supported
-F:	.travis.yml
-
 UNMODIFIED LINUX PV DRIVERS
 M:	Jan Beulich <jbeulich@suse.com>
 S:	Obsolete
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v2 0/8] Using GitLab CI for build testing
@ 2018-03-15 18:21 Doug Goldstein
  2018-03-15 18:17 ` [PATCH v2 1/8] ci: add README and makefile for containers Doug Goldstein
                   ` (9 more replies)
  0 siblings, 10 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-15 18:21 UTC (permalink / raw)
  To: xen-devel
  Cc: Stefano Stabellini, Wei Liu, George Dunlap, Ian Jackson,
	Tim Deegan, Julien Grall, Jan Beulich, Andrew Cooper,
	Doug Goldstein

Really early work on switching over to using GitLab CI over
Travis CI. GitLab is a competitor to GitHub with some advantages
such as an integrated CI system with a lot more flexibility
and control. It additionally is fully open sourced unlike GitHub
and Travis CI. We can even run an instance if that is preferred
over using the hosted instance.

This change uses GitLab CI's ability to use Docker based runners
for running tests. With GitHub we also use a Docker based runner
but we are limited to one Docker container that is then morphed
a number of different ways. With this approach we can specify
different Docker containers for every run (or use the same). By
using different Docker containers we can build environments that
match systems where Xen can and should build. Using this
approach we should be able to cutdown on the number of surpise
build failures encountered by users.

Worth noting another advantage is that builders can be VMs or even
physical hosts as well. So we can have a FreeBSD VM that can be a
build environment.

Further more the above link is to a GitLab pipeline, pipelines are
made of stages which are further composed of jobs. Currently the
example uses one stage called build and all the different distros
are different jobs.  But there's a lot of flexibility as to what
can be done here. There can be stages that check code style or other
pre-flight checks that people may be interested. There can be stages
that happen after the build stage as well such as some simple tests
(e.g. I use it to run the just built xen.gz with an initramfs only
dom0 that contains a small Alpine Linux VM that spits out a string
to an HTTP endpoint which decides that Xen build is good enough to
allow it to be merged into our testing branch).

Overall there are a lot more possibilities than what I've put together
so far. The coverage is not yet the same as Travis CI due to missing
arm builds.

Example run: https://gitlab.com/cardoe/xen/pipelines/18968075
---
change since v1:
- added makefile to help build containers
- moved readme as the first patch
- moved from "cardoe" tenant to "xen-project"
- added debian:stretch
- added clang builds
- added debug y/n builds
- added maintainer info

Doug Goldstein (8):
  ci: add README and makefile for containers
  ci: add Dockerfile for CentOS 7.2
  ci: add Dockerfile for Ubuntu 14.04
  ci: add Dockerfile for Ubuntu 16.04
  ci: add Dockerfile for Debian jessie
  ci: add Dockerfile for Debian stretch
  ci: add cfg to use GitLab CI to build
  ci: add new bits to MAINTAINERS combine with Travis

 .gitlab-ci.yml                             | 164 ++++++++++++++++++++++-
 MAINTAINERS                                |  16 +-
 automation/build/Makefile                  |  17 ++-
 automation/build/README.md                 |  34 +++++-
 automation/build/centos/7.2.dockerfile     |  41 ++++++-
 automation/build/centos/CentOS-7.2.repo    |  35 +++++-
 automation/build/debian/jessie.dockerfile  |  47 ++++++-
 automation/build/debian/stretch.dockerfile |  47 ++++++-
 automation/build/ubuntu/trusty.dockerfile  |  47 ++++++-
 automation/build/ubuntu/xenial.dockerfile  |  47 ++++++-
 10 files changed, 489 insertions(+), 6 deletions(-)
 create mode 100644 .gitlab-ci.yml
 create mode 100644 automation/build/Makefile
 create mode 100644 automation/build/README.md
 create mode 100644 automation/build/centos/7.2.dockerfile
 create mode 100644 automation/build/centos/CentOS-7.2.repo
 create mode 100644 automation/build/debian/jessie.dockerfile
 create mode 100644 automation/build/debian/stretch.dockerfile
 create mode 100644 automation/build/ubuntu/trusty.dockerfile
 create mode 100644 automation/build/ubuntu/xenial.dockerfile

base-commit: c0e34ba78aaec2e2b1aa2a810bc7b3ee1fdfcf3f
-- 
git-series 0.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v2 0/8] Using GitLab CI for build testing
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (7 preceding siblings ...)
  2018-03-15 18:18 ` [PATCH v2 8/8] ci: add new bits to MAINTAINERS combine with Travis Doug Goldstein
@ 2018-03-15 18:51 ` Konrad Rzeszutek Wilk
  2018-03-16  1:55 ` John Thomson
  9 siblings, 0 replies; 13+ messages in thread
From: Konrad Rzeszutek Wilk @ 2018-03-15 18:51 UTC (permalink / raw)
  To: Doug Goldstein
  Cc: Stefano Stabellini, Wei Liu, George Dunlap, Ian Jackson,
	Tim Deegan, xen-devel, Julien Grall, Jan Beulich, Andrew Cooper

On Thu, Mar 15, 2018 at 01:21:08PM -0500, Doug Goldstein wrote:
> Really early work on switching over to using GitLab CI over
> Travis CI. GitLab is a competitor to GitHub with some advantages
> such as an integrated CI system with a lot more flexibility
> and control. It additionally is fully open sourced unlike GitHub
> and Travis CI. We can even run an instance if that is preferred
> over using the hosted instance.
> 
> This change uses GitLab CI's ability to use Docker based runners
> for running tests. With GitHub we also use a Docker based runner
> but we are limited to one Docker container that is then morphed
> a number of different ways. With this approach we can specify
> different Docker containers for every run (or use the same). By
> using different Docker containers we can build environments that
> match systems where Xen can and should build. Using this
> approach we should be able to cutdown on the number of surpise
> build failures encountered by users.
> 
> Worth noting another advantage is that builders can be VMs or even
> physical hosts as well. So we can have a FreeBSD VM that can be a
> build environment.
> 
> Further more the above link is to a GitLab pipeline, pipelines are
> made of stages which are further composed of jobs. Currently the
> example uses one stage called build and all the different distros
> are different jobs.  But there's a lot of flexibility as to what
> can be done here. There can be stages that check code style or other
> pre-flight checks that people may be interested. There can be stages
> that happen after the build stage as well such as some simple tests
> (e.g. I use it to run the just built xen.gz with an initramfs only
> dom0 that contains a small Alpine Linux VM that spits out a string
> to an HTTP endpoint which decides that Xen build is good enough to
> allow it to be merged into our testing branch).
> 
> Overall there are a lot more possibilities than what I've put together
> so far. The coverage is not yet the same as Travis CI due to missing
> arm builds.

I took a peek at your patches and I am not an expert of GitLab but it 
all looks fine to me. You can add

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

On them if you would like.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v2 0/8] Using GitLab CI for build testing
  2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
                   ` (8 preceding siblings ...)
  2018-03-15 18:51 ` [PATCH v2 0/8] Using GitLab CI for build testing Konrad Rzeszutek Wilk
@ 2018-03-16  1:55 ` John Thomson
  2018-03-17 17:03   ` Doug Goldstein
  2018-03-18  0:18   ` Doug Goldstein
  9 siblings, 2 replies; 13+ messages in thread
From: John Thomson @ 2018-03-16  1:55 UTC (permalink / raw)
  To: Doug Goldstein, xen-devel
  Cc: Stefano Stabellini, Wei Liu, George Dunlap, Tim Deegan,
	Ian Jackson, Julien Grall, Jan Beulich, Andrew Cooper

Hi,

I have some suggestions / queries.
I package Xen using GitLab CI for my use: https://gitlab.com/archlinux-packages-johnth/xen/pipelines
My examples here are just mock-ups and not tested.

On Fri, 16 Mar 2018, at 04:21, Doug Goldstein wrote:
> Example run: https://gitlab.com/cardoe/xen/pipelines/18968075

- You probably want to set "XEN_TARGET_ARCH". The example build misses tools.
job:
  variables:
    ...
    XEN_TARGET_ARCH: x86_64


- Is it worth storing the resultant built files as artifacts?
  This way they can be downloaded or tested later.

artifacts:
  paths:
    - 'dist'

> Currently the example uses one stage called build and all the different distros are different jobs.  

- Is is worth caching and artifacting the build (tools) dependencies as a job,
  so that each CI job does not hit the dependency Xen repos and websites for the same data,
  and you have an easily accessible pre-build state?
  This may need an empty job for collect stage where not wanted? I am unsure if you can skip a first stage.

stages:
  - collect
  - build

collect:tools:
  stage: collect
  script:
    - Clone all tools dependency repos & downloads
    - Move files into place, and set repo URLs (example; QEMU_UPSTREAM_URL) to the local copies.
  artifacts:
    untracked: true
  cache:
    untracked: true

build-job-example:
  ...
  dependencies:
    - collect:tools

- My build log exceeds the GitLab CI Job limit. You may not hit this limit, then you can ignore this.
  It looks like this, at the end of the GitLab CI log, but mid-way through the job log:
  ...
  "Job's log exceeded limit of 4194304 bytes."
  To work around this, I log to a file, then store this file as an artifact.
  These can then be downloaded later.
  This could be something like:

artifacts:
  paths:
    - '*.log'
  when: always

job:
  ...
  script:
    - ./scripts/travis-build 2>&1 | tee build.log


Cheers,
-- 
  John Thomson

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v2 0/8] Using GitLab CI for build testing
  2018-03-16  1:55 ` John Thomson
@ 2018-03-17 17:03   ` Doug Goldstein
  2018-03-18  0:18   ` Doug Goldstein
  1 sibling, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-17 17:03 UTC (permalink / raw)
  To: John Thomson, xen-devel
  Cc: Stefano Stabellini, Wei Liu, George Dunlap, Tim Deegan,
	Ian Jackson, Julien Grall, Jan Beulich, Andrew Cooper

On 3/15/18 8:55 PM, John Thomson wrote:
> Hi,
> 
> I have some suggestions / queries.
> I package Xen using GitLab CI for my use: https://gitlab.com/archlinux-packages-johnth/xen/pipelines
> My examples here are just mock-ups and not tested.
> 
> On Fri, 16 Mar 2018, at 04:21, Doug Goldstein wrote:
>> Example run: https://gitlab.com/cardoe/xen/pipelines/18968075
> 
> - You probably want to set "XEN_TARGET_ARCH". The example build misses tools.
> job:
>   variables:
>     ...
>     XEN_TARGET_ARCH: x86_64

Agreed. I've added for a v3.

> 
> 
> - Is it worth storing the resultant built files as artifacts?
>   This way they can be downloaded or tested later.
> 
> artifacts:
>   paths:
>     - 'dist'

I agree as well. I was likely going to keep them for a week only to
start with. Internally I use the artifacts to do some basic boot tests
with Xen.

> 
>> Currently the example uses one stage called build and all the different distros are different jobs.  
> 
> - Is is worth caching and artifacting the build (tools) dependencies as a job,
>   so that each CI job does not hit the dependency Xen repos and websites for the same data,
>   and you have an easily accessible pre-build state?
>   This may need an empty job for collect stage where not wanted? I am unsure if you can skip a first stage.
> 
> stages:
>   - collect
>   - build
> 
> collect:tools:
>   stage: collect
>   script:
>     - Clone all tools dependency repos & downloads
>     - Move files into place, and set repo URLs (example; QEMU_UPSTREAM_URL) to the local copies.
>   artifacts:
>     untracked: true
>   cache:
>     untracked: true
> 
> build-job-example:
>   ...
>   dependencies:
>     - collect:tools

Good suggestion. We'll also be able to ensure all the builds used the
same revs in case they start at different times.


> 
> - My build log exceeds the GitLab CI Job limit. You may not hit this limit, then you can ignore this.
>   It looks like this, at the end of the GitLab CI log, but mid-way through the job log:
>   ...
>   "Job's log exceeded limit of 4194304 bytes."
>   To work around this, I log to a file, then store this file as an artifact.
>   These can then be downloaded later.
>   This could be something like:
> 
> artifacts:
>   paths:
>     - '*.log'
>   when: always
> 
> job:
>   ...
>   script:
>     - ./scripts/travis-build 2>&1 | tee build.log
> 
> 
> Cheers,
> 

Good to know. Thanks!

I appreciate the feedback and I'll incorporate your suggestions.
-- 
Doug Goldstein

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v2 0/8] Using GitLab CI for build testing
  2018-03-16  1:55 ` John Thomson
  2018-03-17 17:03   ` Doug Goldstein
@ 2018-03-18  0:18   ` Doug Goldstein
  1 sibling, 0 replies; 13+ messages in thread
From: Doug Goldstein @ 2018-03-18  0:18 UTC (permalink / raw)
  To: John Thomson, xen-devel
  Cc: Stefano Stabellini, Wei Liu, George Dunlap, Tim Deegan,
	Ian Jackson, Julien Grall, Jan Beulich, Andrew Cooper


[-- Attachment #1.1.1: Type: text/plain, Size: 494 bytes --]

On 3/15/18 8:55 PM, John Thomson wrote:
> Hi,
> 
> I have some suggestions / queries.
> I package Xen using GitLab CI for my use: https://gitlab.com/archlinux-packages-johnth/xen/pipelines
> My examples here are just mock-ups and not tested.
> 
> On Fri, 16 Mar 2018, at 04:21, Doug Goldstein wrote:
>> Example run: https://gitlab.com/cardoe/xen/pipelines/18968075
> 

John,

If you want to contribute an ArchLinux container I'd appreciate that as
well.

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2018-03-18  0:18 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-15 18:21 [PATCH v2 0/8] Using GitLab CI for build testing Doug Goldstein
2018-03-15 18:17 ` [PATCH v2 1/8] ci: add README and makefile for containers Doug Goldstein
2018-03-15 18:17 ` [PATCH v2 2/8] ci: add Dockerfile for CentOS 7.2 Doug Goldstein
2018-03-15 18:17 ` [PATCH v2 3/8] ci: add Dockerfile for Ubuntu 14.04 Doug Goldstein
2018-03-15 18:17 ` [PATCH v2 4/8] ci: add Dockerfile for Ubuntu 16.04 Doug Goldstein
2018-03-15 18:17 ` [PATCH v2 5/8] ci: add Dockerfile for Debian jessie Doug Goldstein
2018-03-15 18:17 ` [PATCH v2 6/8] ci: add Dockerfile for Debian stretch Doug Goldstein
2018-03-15 18:18 ` [PATCH v2 7/8] ci: add cfg to use GitLab CI to build Doug Goldstein
2018-03-15 18:18 ` [PATCH v2 8/8] ci: add new bits to MAINTAINERS combine with Travis Doug Goldstein
2018-03-15 18:51 ` [PATCH v2 0/8] Using GitLab CI for build testing Konrad Rzeszutek Wilk
2018-03-16  1:55 ` John Thomson
2018-03-17 17:03   ` Doug Goldstein
2018-03-18  0:18   ` Doug Goldstein

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.