[stage1-xen PATCH v1 00/10] Add Fedora support

[stage1-xen PATCH v1 00/10] Add Fedora support

[Rajiv Ranganath]

Hi Stefanao,

I've made changes based on RFC feedback. Kindly please review.

Changes in V1:

- build/fedora: Add `run` and `components/*` scripts

Introduce scripts to build individual components (xen, rkt, qemu) both
within a container or directly on the host. It also has constants that
can be used to customize the build. `run` script now uses `component/*`
scripts.

- build/fedora: Add `xen-unstable-runit/*` scripts

Removed usage of `pipefile`, `pushd`, `popd`. Also added `set -e` and
switched to `/bin/bash`.

- build/fedora: Add `RUNNING_STAGE1_XEN.md`
- BUILDING.md: Add Fedora instructions

Updated documentation to highlight both container build and manual
build.

Here is a temporary repo [1] with v1 series.

Best,
Rajiv

[1]: https://github.com/lambda-linux-fedora/stage1-xen/tree/wip1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 01/10] .gitignore: Add

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 .gitignore |    2 ++
 1 file changed, 2 insertions(+)
 create mode 100644 .gitignore

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..873f8f6
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,2 @@
+# build/fedora
+build/fedora/binutils-*.tar


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 02/10] build/fedora: Add `buildroot-README.md`

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build/fedora/buildroot-README.md |   50 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 build/fedora/buildroot-README.md

diff --git a/build/fedora/buildroot-README.md b/build/fedora/buildroot-README.md
new file mode 100644
index 0000000..0efb150
--- /dev/null
+++ b/build/fedora/buildroot-README.md
@@ -0,0 +1,50 @@
+## stage1-xen Fedora Buildroot
+
+stage1-xen build artifacts for Fedora is built in two phases. In the first phase
+a docker container is prepared with all the build dependencies. We refer to it
+as `stage1-xen-fedora-buildroot`. In the next phase we execute the `run` script
+that uses `stage1-xen-fedora-buildroot` and to produce the build artifacts.
+
+### Building `stage1-xen-fedora-buildroot`
+
+`stage1-xen-fedora-buildroot` has a external dependency
+on [`binutils`](https://github.com/lambda-linux-fedora/binutils) package that is
+compiled with `i386pe` support. You can download the pre-built RPMs
+from [here](https://drive.google.com/open?id=0B_tTbuxmuRzIR05wQ3E1eWVyaGs).
+Please download `binutils-2.26.1-1.1.fc25.tar`.
+
+To build docker image
+
+```
+cd stage1-xen/build/fedora
+
+docker build -f buildroot-Dockerfile -t stage1-xen-fedora-buildroot .
+```
+
+### Running `stage1-xen-fedora-buildroot`
+
+```
+cd stage1-xen
+
+docker run --rm \
+  -v `pwd`:/root/gopath/src/github.com/rkt/stage1-xen \
+  -v /tmp:/tmp \
+  -t -i stage1-xen-fedora-buildroot \
+  /sbin/my_init -- /root/bin/run
+```
+
+The generated build artifacts are in `/tmp` directory.
+
+To debug build issues -
+
+```
+cd stage1-xen
+
+docker run --rm \
+  -v `pwd`:/root/gopath/src/github.com/rkt/stage1-xen \
+  -v /tmp:/tmp \
+  -t -i stage1-xen-fedora-buildroot \
+  /sbin/my_init -- /bin/bash
+```
+
+Also see section on `ipdb` in `buildroot-Dockerfile`.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 03/10] build/fedora: Add `buildroot-Dockerfile`

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build/fedora/buildroot-Dockerfile |  113 +++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)
 create mode 100644 build/fedora/buildroot-Dockerfile

diff --git a/build/fedora/buildroot-Dockerfile b/build/fedora/buildroot-Dockerfile
new file mode 100644
index 0000000..971560e
--- /dev/null
+++ b/build/fedora/buildroot-Dockerfile
@@ -0,0 +1,113 @@
+# tarballs checksum
+# -----------------
+# 974b3091232d781c4fc410ccca98fb62ba9febe9e6a988e348804483c4f66742  binutils-2.26.1-1.1.fc25.tar
+
+FROM lambdalinuxfedora/baseimage-fedora
+
+CMD ["/sbin/my_init"]
+
+COPY [ \
+  "./binutils-2.26.1-1.1.fc25.tar", \
+  \
+  "./components/*", \
+  "./run", \
+  "/tmp/docker-build/" \
+]
+
+RUN \
+  # dnf
+  echo "Running dnf update..." && \
+  dnf update -y && \
+  dnf install -y less && \
+  dnf install -y sudo && \
+  \
+  # circleci container requirements
+  # https://circleci.com/docs/2.0/custom-images/#adding-required-and-custom-tools-or-files
+  dnf install -y git && \
+  dnf install -y openssh-clients && \
+  dnf install -y tar && \
+  dnf install -y gzip && \
+  dnf install -y ca-certificates && \
+  \
+  # install `binutils`
+  pushd /tmp/docker-build && \
+    # verify checksum
+    echo "974b3091232d781c4fc410ccca98fb62ba9febe9e6a988e348804483c4f66742  binutils-2.26.1-1.1.fc25.tar" | sha256sum -c - && \
+    tar xvf binutils-2.26.1-1.1.fc25.tar && \
+    dnf install -y ./binutils/2.26.1/1.1.fc25/x86_64/binutils-2.26.1-1.1.fc25.x86_64.rpm && \
+  popd && \
+  \
+  dnf install -y @buildsys-build && \
+  \
+  # Having `ipdb` around is useful when debugging `run` script. Uncomment this
+  # section as required
+  # dnf install -y python2-devel && \
+  # dnf install -y python-pip && \
+  # su -l root -c "pip2 install --user ipdb==0.8 ipython==5.3.0" && \
+  \
+  # Note: xen and qemu has some duplicate package dependencies. We are
+  # explicitly calling out dependencies for xen and qemu
+  #
+  # xen build dependencies
+  dnf install -y bridge-utils && \
+  dnf install -y gettext && \
+  dnf install -y glib2-devel && \
+  dnf install -y glibc-devel.i686 && \
+  dnf install -y grub2 && \
+  dnf install -y iasl && \
+  dnf install -y libaio-devel && \
+  dnf install -y libuuid-devel && \
+  dnf install -y ncurses-devel && \
+  dnf install -y openssl-devel && \
+  dnf install -y pixman-devel && \
+  dnf install -y python2-devel && \
+  dnf install -y wget && \
+  dnf install -y yajl-devel && \
+  \
+  # qemu build dependencies
+  dnf install -y glib2-devel && \
+  dnf install -y libaio-devel && \
+  dnf install -y libattr-devel && \
+  dnf install -y libcap-devel && \
+  dnf install -y libcap-ng-devel && \
+  dnf install -y pixman-devel && \
+  dnf install -y zlib-devel && \
+  \
+  # rkt build dependencies
+  dnf install -y autoconf && \
+  dnf install -y automake && \
+  dnf install -y git && \
+  dnf install -y glibc-static && \
+  dnf install -y gnupg && \
+  dnf install -y golang && \
+  dnf install -y libacl-devel && \
+  dnf install -y squashfs-tools && \
+  dnf install -y systemd-devel && \
+  dnf install -y wget && \
+  \
+  # stage1-xen build dependencies
+  dnf install -y bc && \
+  dnf install -y busybox && \
+  dnf install -y glide && \
+  dnf install -y golang && \
+  dnf install -y jq && \
+  dnf install -y libacl-devel && \
+  dnf install -y wget && \
+  \
+  # copy `run` file and `components/{qemu,rkt,xen}`
+  su -l root -c "mkdir /root/bin" && \
+  su -l root -c "cp /tmp/docker-build/run /root/bin" && \
+  su -l root -c "mkdir /root/bin/components" && \
+  su -l root -c "cp /tmp/docker-build/qemu /root/bin/components" && \
+  su -l root -c "cp /tmp/docker-build/rkt /root/bin/components" && \
+  su -l root -c "cp /tmp/docker-build/xen /root/bin/components" && \
+  \
+  # create `stage1-xen` directory
+  mkdir -p /root/gopath/src/github.com/rkt/stage1-xen && \
+  \
+  # cleanup
+  rm -rf /tmp/docker-build && \
+  dnf clean all && \
+  rm -rf /var/cache/dnf/* && \
+  rm -rf /tmp/* && \
+  rm -rf /var/tmp/*


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build/fedora/components/qemu |   50 ++++++++++++++++++++++++++++++++++++
 build/fedora/components/rkt  |   58 ++++++++++++++++++++++++++++++++++++++++++
 build/fedora/components/xen  |   46 +++++++++++++++++++++++++++++++++
 build/fedora/run             |   56 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 210 insertions(+)
 create mode 100755 build/fedora/components/qemu
 create mode 100755 build/fedora/components/rkt
 create mode 100755 build/fedora/components/xen
 create mode 100755 build/fedora/run

diff --git a/build/fedora/components/qemu b/build/fedora/components/qemu
new file mode 100755
index 0000000..6c89e2c
--- /dev/null
+++ b/build/fedora/components/qemu
@@ -0,0 +1,50 @@
+#!/usr/bin/python2
+
+import shlex
+import subprocess
+import sys
+import os
+
+# Modify this if you would like to install Qemu elsewhere on your filesystem or
+# a different version of Qemu
+QEMU_PREFIX = '/opt/qemu-unstable'
+QEMU_BRANCH = 'master'
+
+# This should correspond to your Xen install prefix
+XEN_PREFIX = '/opt/xen-unstable'
+
+
+# helper function to capture stdout from a long running process
+def subprocess_stdout(cmd, cwd, env):
+    p = subprocess.Popen(
+        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
+    while p.poll() is None:
+        l = p.stdout.readline()
+        sys.stdout.write(l)
+    if p.returncode != 0:
+        sys.exit(1)
+
+env = os.environ.copy()
+
+# build and install qemu
+print "Cloning qemu..."
+cmd = "git clone --branch %(branch)s git://git.qemu.org/qemu.git" % {
+    'branch': QEMU_BRANCH
+}
+subprocess.check_output(shlex.split(cmd), cwd='/root')
+
+steps = [
+    "./configure --prefix=%(qemu_prefix)s --enable-xen --target-list=i386-softmmu --extra-cflags=\"-I%(xen_prefix)s/include\" --extra-ldflags=\"-L%(xen_prefix)s/lib -Wl,-rpath,%(xen_prefix)s/lib\" --disable-kvm --enable-virtfs --enable-linux-aio"
+    % {
+        'qemu_prefix': QEMU_PREFIX,
+        'xen_prefix': XEN_PREFIX
+    }, 'make', 'make install'
+]
+for cmd in steps:
+    cwd = '/root/qemu'
+    subprocess_stdout(cmd, cwd, env)
+
+cmd = "cp i386-softmmu/qemu-system-i386 %(xen_prefix)s/lib/xen/bin/qemu-system-i386" % {
+    'xen_prefix': XEN_PREFIX
+}
+subprocess.check_output(shlex.split(cmd), cwd='/root/qemu')
diff --git a/build/fedora/components/rkt b/build/fedora/components/rkt
new file mode 100755
index 0000000..edfdd1c
--- /dev/null
+++ b/build/fedora/components/rkt
@@ -0,0 +1,58 @@
+#!/usr/bin/python2
+
+import shlex
+import subprocess
+import sys
+import os
+
+# `rkt` is installed in the same prefix as `stage1-xen`. Modify this if you
+# would like to install rkt elsewhere on your filesystem.
+STAGE1_XEN_PREFIX = '/opt/stage1-xen'
+RKT_PREFIX = STAGE1_XEN_PREFIX
+RKT_BRANCH = 'master'
+
+# Adjust this according to what RKT_BRANCH generates
+RKT_BUILD_VER = 'rkt-1.28.1+git'
+
+
+# helper function to capture stdout from a long running process
+def subprocess_stdout(cmd, cwd, env):
+    p = subprocess.Popen(
+        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
+    while p.poll() is None:
+        l = p.stdout.readline()
+        sys.stdout.write(l)
+    if p.returncode != 0:
+        sys.exi(1)
+
+
+env = os.environ.copy()
+
+# build rkt
+print "Cloning rkt..."
+cmd = "git clone --branch %(branch)s https://github.com/rkt/rkt.git" % {
+    'branch': RKT_BRANCH
+}
+subprocess.check_output(shlex.split(cmd), cwd='/root')
+
+steps = [
+    './autogen.sh', './configure --disable-tpm --with-stage1-flavors=coreos',
+    'make'
+]
+for cmd in steps:
+    cwd = '/root/rkt'
+    subprocess_stdout(cmd, cwd, env)
+
+# install rkt build artifacts to RKT_PREFIX
+steps = [
+    "mkdir -p %(prefix)s/bin" % {
+        'prefix': RKT_PREFIX
+    },
+    "cp /root/rkt/build-%(build_ver)s/target/bin/rkt %(prefix)s/bin/rkt" % {
+        'build_ver': RKT_BUILD_VER,
+        'prefix': RKT_PREFIX
+    }
+]
+for cmd in steps:
+    cwd = '/root/rkt'
+    subprocess_stdout(cmd, cwd, env)
diff --git a/build/fedora/components/xen b/build/fedora/components/xen
new file mode 100755
index 0000000..95da9a6
--- /dev/null
+++ b/build/fedora/components/xen
@@ -0,0 +1,46 @@
+#!/usr/bin/python2
+
+import shlex
+import subprocess
+import sys
+import os
+
+# Modify this if you would like to install Xen elsewhere on your filesystem or
+# a different version of Xen
+XEN_PREFIX = '/opt/xen-unstable'
+XEN_BRANCH = 'master'
+
+
+# helper function to capture stdout from a long running process
+def subprocess_stdout(cmd, cwd, env):
+    p = subprocess.Popen(
+        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
+    while p.poll() is None:
+        l = p.stdout.readline()
+        sys.stdout.write(l)
+    if p.returncode != 0:
+        sys.exit(1)
+
+
+env = os.environ.copy()
+
+# build and install xen
+print "Cloning xen..."
+cmd = "git clone --branch %(branch)s git://xenbits.xen.org/xen.git" % {
+    'branch': XEN_BRANCH
+}
+subprocess.check_output(shlex.split(cmd), cwd='/root')
+
+steps = [
+    "./configure --prefix=%(prefix)s --with-system-qemu=%(prefix)s/lib/xen/bin/qemu-system-i386 --disable-stubdom --disable-qemu-traditional --disable-rombios --sysconfdir=%(prefix)s/etc --enable-rpath --disable-systemd"
+    % {
+        'prefix': XEN_PREFIX
+    }, 'make',
+    "make install BOOT_DIR=%(prefix)s/boot DEBUG_DIR=%(prefix)s/lib/debug EFI_DIR=%(prefix)s/boot/efi/EFI/xen"
+    % {
+        'prefix': XEN_PREFIX
+    }
+]
+for cmd in steps:
+    cwd = '/root/xen'
+    subprocess_stdout(cmd, cwd, env)
diff --git a/build/fedora/run b/build/fedora/run
new file mode 100755
index 0000000..6cb6417
--- /dev/null
+++ b/build/fedora/run
@@ -0,0 +1,56 @@
+#!/usr/bin/python2
+
+import shlex
+import subprocess
+import sys
+import os
+
+# This scripts calls out to `xen`, `qemu` and `rkt` scripts in the
+# `components/` directory within a container. It is expected that components
+# directory is present at the same directory level as run script.
+STAGE1_XEN_COMPONENTS = ['xen', 'qemu', 'rkt']
+
+
+# helper function to capture stdout from a long running process
+def subprocess_stdout(cmd, cwd, env):
+    p = subprocess.Popen(
+        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
+    while p.poll() is None:
+        l = p.stdout.readline()
+        sys.stdout.write(l)
+    if p.returncode != 0:
+        sys.exit(1)
+
+
+env = os.environ.copy()
+
+dirname = os.path.dirname(os.path.realpath(__file__))
+steps = [os.path.join(dirname, 'components', x) for x in STAGE1_XEN_COMPONENTS]
+for cmd in steps:
+    cwd = '/root'
+    subprocess_stdout(cmd, cwd, env)
+
+# build stage1-xen
+env['GOPATH'] = '/root/gopath'
+cwd = '/root/gopath/src/github.com/rkt/stage1-xen'
+cmd = 'bash build.sh'
+subprocess_stdout(cmd, cwd, env)
+
+# install build artifacts to `/opt/stage1-xen/aci` and create a tarball
+steps = [
+    'mkdir -p /opt/stage1-xen/aci',
+    'cp /root/gopath/src/github.com/rkt/stage1-xen/stage1-xen.aci /opt/stage1-xen/aci/stage1-xen.aci',
+    'cp /root/gopath/src/github.com/rkt/stage1-xen/build/fedora/source_path.sh /opt/stage1-xen/bin/source_path.sh',
+    'cp -r /root/gopath/src/github.com/rkt/stage1-xen/build/fedora/xen-unstable-runit /opt/xen-unstable-runit'
+]
+for cmd in steps:
+    cwd = '/root'
+    subprocess_stdout(cmd, cwd, env)
+
+cwd = '/opt'
+cmd = 'tar zcvf /root/stage1-xen-build.tar.gz qemu-unstable/ stage1-xen/ xen-unstable/ xen-unstable-runit/'
+subprocess_stdout(cmd, cwd, env)
+
+cwd = '/root'
+cmd = 'mv /root/stage1-xen-build.tar.gz /tmp'
+subprocess_stdout(cmd, cwd, env)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 05/10] build/fedora: Add `source_path.sh`

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build/fedora/source_path.sh |    3 +++
 1 file changed, 3 insertions(+)
 create mode 100755 build/fedora/source_path.sh

diff --git a/build/fedora/source_path.sh b/build/fedora/source_path.sh
new file mode 100755
index 0000000..57ac7ed
--- /dev/null
+++ b/build/fedora/source_path.sh
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+export PATH=/opt/xen-unstable/sbin:/opt/xen-unstable/bin:/opt/stage1-xen/bin:/opt/qemu-unstable/bin:$PATH


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build/fedora/xen-unstable-runit/setup.sh           |   18 ++++++++++++++++
 build/fedora/xen-unstable-runit/teardown.sh        |   18 ++++++++++++++++
 .../xen-init-dom0-disk-backend/run                 |   11 ++++++++++
 build/fedora/xen-unstable-runit/xen-init-dom0/run  |    9 ++++++++
 build/fedora/xen-unstable-runit/xenconsoled/run    |   13 +++++++++++
 build/fedora/xen-unstable-runit/xenstored/run      |   23 ++++++++++++++++++++
 6 files changed, 92 insertions(+)
 create mode 100755 build/fedora/xen-unstable-runit/setup.sh
 create mode 100755 build/fedora/xen-unstable-runit/teardown.sh
 create mode 100755 build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
 create mode 100755 build/fedora/xen-unstable-runit/xen-init-dom0/run
 create mode 100755 build/fedora/xen-unstable-runit/xenconsoled/run
 create mode 100755 build/fedora/xen-unstable-runit/xenstored/run

diff --git a/build/fedora/xen-unstable-runit/setup.sh b/build/fedora/xen-unstable-runit/setup.sh
new file mode 100755
index 0000000..b5adf8c
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/setup.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -e
+
+# runit RPM creates `/etc/service` directory
+if [ ! -d "/etc/service" ]; then
+    echo "/etc/service directory not found. Please install runit RPM."
+    exit 1
+fi
+
+runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend xenstored"
+
+for service in $runit_services; do
+    ln -sf /opt/xen-unstable-runit/$service /etc/service/$service
+done
+
+echo "Successfully created symlinks in /etc/service directory."
+exit 0
diff --git a/build/fedora/xen-unstable-runit/teardown.sh b/build/fedora/xen-unstable-runit/teardown.sh
new file mode 100755
index 0000000..d333807
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/teardown.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -e
+
+# runit RPM creates `/etc/service` directory
+if [ ! -d "/etc/service" ]; then
+    echo "/etc/service directory not found."
+    exit 1
+fi
+
+runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend xenstored"
+
+for service in $runit_services; do
+    rm -f /etc/service/$service
+done
+
+echo "Successfully deleted symlinks in /etc/service directory."
+exit 0
diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
new file mode 100755
index 0000000..6315d48
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -e
+
+sv check xenstored >/dev/null || exit 1
+sv check xenconsoled >/dev/null || exit 1
+
+# In case of failure, allow user to run teardown script
+sleep 5s
+
+exec /opt/xen-unstable/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -monitor /dev/null -serial /dev/null -parallel /dev/null -nodefaults -no-user-config
diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0/run b/build/fedora/xen-unstable-runit/xen-init-dom0/run
new file mode 100755
index 0000000..193ba19
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xen-init-dom0/run
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -e
+
+sv check xenstored >/dev/null || exit 1
+
+/opt/xen-unstable/lib/xen/bin/xen-init-dom0
+
+exec chpst -b xen-init-dom0 runit-pause
diff --git a/build/fedora/xen-unstable-runit/xenconsoled/run b/build/fedora/xen-unstable-runit/xenconsoled/run
new file mode 100755
index 0000000..b5b7a9f
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xenconsoled/run
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+set -e
+
+sv check xen-init-dom0 >/dev/null || exit 1
+
+[ ! -d /var/log/xen/console ] && mkdir -p /var/log/xen/console
+
+# In case of failure, allow user to run teardown script
+sleep 5s
+
+# --log=[none|guest|hv|all]
+exec /opt/xen-unstable/sbin/xenconsoled -i --log=none
diff --git a/build/fedora/xen-unstable-runit/xenstored/run b/build/fedora/xen-unstable-runit/xenstored/run
new file mode 100755
index 0000000..beb2a5f
--- /dev/null
+++ b/build/fedora/xen-unstable-runit/xenstored/run
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+set -e
+
+[ ! -d /var/run/xen ] && mkdir -p /var/run/xen
+[ ! -d /var/run/xenstored ] && mkdir -p /var/run/xenstored
+[ ! -d /var/log/xen ] && mkdir -p /var/log/xen
+[ ! -d /var/lib/xen ] && mkdir -p /var/lib/xen
+[ ! -d /var/lib/xen/dump ] && mkdir -p /var/lib/xen/dump
+[ ! -d /var/lib/xen/xenpaging ] && mkdir -p /var/lib/xen/paging
+[ ! -d /var/lib/xenstored ] && mkdir -p /var/lib/xenstored
+modprobe -q xen-evtchn || exit 1
+modprobe -q xen-gntdev || exit 1
+modprobe -q xen-gntalloc || exit 1
+mountpoint -q /proc/xen || mount -t xenfs xenfs /proc/xen
+mountpoint -q /var/lib/xenstored || mount -t tmpfs xenstored /var/lib/xenstored
+grep -q "control_d" /proc/xen/capabilities || exit 1
+
+# In case of failure, allow user to run teardown script
+sleep 5s
+
+# Use `--trace-file /var/log/xen/xenstored-trace.log` for logs
+exec /opt/xen-unstable/sbin/xenstored --no-fork


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 07/10] .circleci/config.yml: Add

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 .circleci/config.yml |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 .circleci/config.yml

diff --git a/.circleci/config.yml b/.circleci/config.yml
new file mode 100644
index 0000000..93315b4
--- /dev/null
+++ b/.circleci/config.yml
@@ -0,0 +1,21 @@
+version: 2
+jobs:
+  build:
+    working_directory: /root
+    docker:
+      - image: lambdalinuxfedora/stage1-xen-fedora-buildroot:1708260126
+        command: /sbin/my_init
+    steps:
+      - run:
+          # We create `stage1-xen` directory in Dockerfile for local dev
+          # environment. Removing it here so CircleCI checkout step can work
+          # correctly
+          name: Removing stage1-xen directory from GOPATH...
+          command: |
+            rm -rf /root/gopath/src/github.com/rkt/stage1-xen
+      - checkout:
+          path: /root/gopath/src/github.com/rkt/stage1-xen
+      - run:
+          name: Starting run...
+          command: |
+            /root/bin/run


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 08/10] README.md: Add CircleCI badge

[Rajiv Ranganath]

From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 README.md |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/README.md b/README.md
index 9ea6adf..e1cd40c 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,7 @@
 # stage1-xen - A Xen based stage1 for CoreOS rkt
 
+[![Build Status](https://circleci.com/gh/rkt/stage1-xen/tree/master.svg?style=shield&circle-token=:circle-token)](https://circleci.com/gh/rkt/stage1-xen/tree/master)
+
 ## Goal
 
 CoreOS rkt is a modular container engine with [three stages of execution](https://coreos.com/rkt/docs/latest/devel/stage1-implementors-guide.html). Stage1 is responsible for creating the execution environment for the contained applications.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 09/10] build/fedora: Add `RUNNING_STAGE1_XEN.md`

[Rajiv Ranganath]

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build/fedora/RUNNING_STAGE1_XEN.md |  391 ++++++++++++++++++++++++++++++++++++
 1 file changed, 391 insertions(+)
 create mode 100644 build/fedora/RUNNING_STAGE1_XEN.md

diff --git a/build/fedora/RUNNING_STAGE1_XEN.md b/build/fedora/RUNNING_STAGE1_XEN.md
new file mode 100644
index 0000000..867b8b4
--- /dev/null
+++ b/build/fedora/RUNNING_STAGE1_XEN.md
@@ -0,0 +1,391 @@
+# Running Stage1 Xen on Fedora
+
+This document outlines the steps to get started with stage1-xen on Fedora. They are &ndash;
+
+ * [Preparing your machine and installing minimal Fedora](#preparing_your_machine_and_installing_minimal_fedora)
+ * [Booting into Xen](#booting_into_xen)
+ * [Launching Xen services](#launching_xen_services)
+ * [Setting up Xen networking](#setting_up_xen_networking)
+ * [Running stage1-xen](#running_stage1-xen)
+
+<a name="preparing_your_machine_and_installing_minimal_fedora"></a>
+## Preparing your machine and installing minimal Fedora
+
+On x86 platform there are two ways to start an operating system or a hypervisor. They are &ndash;
+
+ * Legacy BIOS Mode
+ * EFI Mode
+
+Latest operating systems and hypervisors including Fedora and Xen has support for EFI mode. If you are unfamiliar with EFI we recommend checking out this [article](http://www.rodsbooks.com/efi-bootloaders/principles.html).
+
+By default, most BIOS now boot using EFI Mode. In your BIOS menu, there might be an option to toggle _Legacy BIOS Mode_. Do not toggle that option.
+
+### Enable VT-x and VT-d
+
+Please ensure that you have enabled VT-x and if available VT-d.
+
+### Disable Secure Boot
+
+As we will be booting a custom build of Xen, we need to disable secure boot. You will find an option in your BIOS menu to disable secure boot.
+
+### Installing minimal Fedora
+
+The default Fedora installation installs packages that we do not require when running Xen. We recommend doing a minimal Fedora as follows.
+
+ 1. Download Net Install image
+
+ 2. Prepare a USB drive
+
+ 3. Do a minimal Fedora Install
+
+You can download the Fedora net install image [here](https://alt.fedoraproject.org/). You can select either the Fedora Server or Fedora Workstation image, it doesn't really matter.
+
+After downloading the net install images, please copy the raw image onto a USB drive. Please see [this](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Installation_Guide/sect-making-usb-media.html) link on how to prepare USB drive.
+
+EFI BIOS comes with a _BIOS Boot Menu_ using which you can select the device to boot from. Insert the USB drive, then go into your BIOS Boot Menu and boot using the USB drive. This should start the Fedora Network Installer.
+
+In the Fedora Installer, there is a section for under _SOFTWARE_ called _SOFTWARE SELECTION_. In this section please **select** either _Minimal Install_ or _Basic Desktop_, **without** any add-ons. 
+
+**Note:** If there is existing data on the hard disk, please ensure that _INSTALLATION DESTINATION_ under _SYSTEM_ section is appropriately configured. 
+
+Then click on _Begin Installation_ to complete the installation.
+
+Once the installation is complete, please disable SELinux by editing `/etc/selinux/config`.
+
+You now have a minimal Fedora Installation, which is good for working with Xen.
+
+<a name="booting_into_xen"></a>
+## Booting into Xen
+
+Build and install Xen and stage1-xen. Please see [BUILDING.md](/BUILDING.md#build_fedora).
+
+If you followed the container build with Docker, then copy over `stage1-xen-build.tar.gz`. Extract `stage1-xen-build.tar.gz` into `/opt` directory.
+
+```shell
+[root@localhost ~]# tar zxvf stage1-xen-build.tar.gz -C /opt
+
+[root@localhost ~]# ls /opt
+qemu-unstable  stage1-xen  xen-unstable  xen-unstable-runit
+```
+
+This will extract all the build artifacts into `/opt` directory.
+
+Next we will create a BIOS Boot Menu entry to boot `xen-4.10-unstable.efi`. This will start Xen hypervisor. Xen will then start Fedora as Dom-0 guest.
+
+On Fedora, EFI system partition (ESP) is usually mounted at `/boot/efi`. This is a `vfat` partition. You can check if EFI system partition is mounted as follows &ndash;
+
+```shell
+[root@localhost ~]# mount | grep '\/boot\/efi'
+/dev/sda1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
+```
+
+Create a directory for Xen under `/boot/efi/EFI` and copy over `xen-4.10-unstable.efi`.
+
+```shell
+[root@localhost ~]# mkdir -p /boot/efi/EFI/xen
+[root@localhost ~]# cp /opt/xen-unstable/boot/efi/EFI/xen/xen-4.10-unstable.efi /boot/efi/EFI/xen/
+```
+
+Inspect `/boot/efi/EFI/fedora/grub.cfg`. Under section `### BEGIN /etc/grub.d/10_linux ###` you will find `menuentry` for Fedora kernel and initrd. Look for `linuxefi` and `initrdefi`. Copy over the `vmlinuz` and `initramfs` files that you want to use for your Dom-0 into `/boot/efi/EFI/xen` directory.
+
+```shell
+[root@localhost ~]# cp /boot/vmlinuz-A.B.C-D.fcXX.x86_64 /boot/efi/EFI/xen/
+
+[root@localhost ~]# cp /boot/initramfs-A.B.C-D.fcXX.x86_64.img /boot/efi/EFI/xen/
+```
+
+Now in `/boot/efi/EFI/xen/` you should have the following files.
+
+```shell
+[root@localhost ~]# ls /boot/efi/EFI/xen/
+initramfs-A.B.C-D.fcXX.x86_64.img  vmlinuz-A.B.C-D.fcXX.x86_64  xen-4.10-unstable.efi
+```
+
+Next create a file `xen-4.10-unstable.cfg` in `/boot/efi/EFI/xen/`. This is the [configuration file](https://xenbits.xen.org/docs/unstable/misc/efi.html) that Xen EFI loader will use to load Dom-0 kernel and initrd.
+
+Following are contents of `xen-4.10-unstable.cfg`
+
+```
+[global]
+default=fedora-A.B.C-D.fc25
+
+[fedora-A.B.C-D.fc25]
+options=console=vga,com1 com1=115200,8n1 iommu=verbose ucode=scan flask=disabled conring_size=2097152 loglvl=all autoballoon=0 dom0_mem=4096M,max:4096M
+kernel=vmlinuz-A.B.C-D.fc25.x86_64 root=UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ro rhgb console=hvc0 console=tty0
+ramdisk=initramfs-A.B.C-D.fc25.x86_64.img
+```
+
+You can find the boot parameters for `kernel=` from `linuxefi` entry in `/boot/efi/EFI/fedora/grub.cfg` Adjust `dom0_mem` appropriately leaving sufficient room for dom-U guests.
+
+We can now use `efibootmgr` to create a boot entry for Xen. If this the first time you are using `efibootmgr` please checkout the man pages by doing `man efibootmgr`.
+
+Use `efibootmgr -v` to list all the EFI boot entires.
+
+```shell
+[root@localhost ~]# efibootmgr -v
+BootCurrent: 0002
+Timeout: 2 seconds
+BootOrder: ...
+
+[...]
+
+Boot0001* Xen   HD(1,GPT,7d511991-1c25-4e33-900b-1d61d7752f19,0x800,0x82000)/File(\EFI\xen\xen-4.10-unstable.efi)
+Boot0002* Fedora        HD(1,GPT,7d511991-1c25-4e33-900b-1d61d7752f19,0x800,0x82000)/File(\EFI\fedora\shim.efi)
+
+[...]
+```
+
+In the above example there is already an entry for Xen with a boot number of `1`. Fedora is at boot number `2`. Your entires would look different. You won't have the Xen entry as yet! We are showing you an example where the Xen boot entry has already been created.
+
+Let us now create a boot entry for Xen. First we need to identify the disk and the partition number for EFI system partition. In most cases it is at `/dev/sda1`. You can identify this by doing &ndash;
+
+```shell
+[root@localhost ~]# df /boot/efi
+Filesystem     1K-blocks  Used Available Use% Mounted on
+/dev/sda1         262128 63019    199109  25% /boot/efi
+
+[root@localhost ~]# sgdisk -p /dev/sda
+Disk /dev/sda: 976773168 sectors, 465.8 GiB
+Logical sector size: 512 bytes
+
+[...]
+
+Number  Start (sector)    End (sector)  Size       Code  Name
+   1            2048          534527   260.0 MiB   EF00  EFI System Partition
+```
+
+You can now create boot entry for Xen using the following command. Adjust `/dev/sda` and `-p 1`, according to where your EFI system partition is located.
+
+```shell
+[root@localhost ~]# efibootmgr -c -w -L Xen -d /dev/sda -p 1 -l '\EFI\xen\xen-4.10-unstable.efi'
+BootCurrent: ...
+Timeout: 2 seconds
+BootOrder: 0001,0002,0000,0010,0011,0012,0013,0017,0018,0019,001A,001B,001C
+
+[...]
+
+Boot0002* Fedora
+
+[...]
+
+Boot0001* Xen
+```
+
+The output indicates that a boot entry for Xen is created with a boot number of `1`. 
+
+We will now show you how to delete an existing boot entry.
+
+**Note:** Be careful when deleting boot entires that you have not created. Do not delete Fedora or any entry unless you really know what you are doing. You have been warned!
+
+```shell
+[root@localhost ~]# efibootmgr -b <boot_num> -B
+
+[root@localhost ~]# efibootmgr -b 1 -B
+BootCurrent: ...
+BootOrder: ...
+
+[...]
+
+Boot0002* Fedora
+Boot0010  Setup
+
+[...]
+```
+
+Once we have created a boot entry we can now boot into Xen. Restart machine and from the BIOS boot menu select **Xen**. You'll see Xen starting followed by Linux.
+
+After booting into Linux, you can see if have successfully booted Xen by checking out `dmesg`.
+
+```shell
+[root@localhost ~]# dmesg | grep [Xx]en
+[    0.000000] Xen: [mem 0x0000000000000000-0x0000000000057fff] usable
+
+[...]
+
+[    0.000000] Hypervisor detected: Xen
+[    0.000000] Setting APIC routing to Xen PV.
+[    0.000000] Booting paravirtualized kernel on Xen
+[    0.000000] Xen version: 4.10-unstable (preserve-AD)
+[    0.001000] Xen: using vcpuop timer interface
+[    0.001000] installing Xen timer for CPU 0
+```
+
+If you don't see Xen mentioned in your `dmesg`, then please check the previous steps.
+
+<a name="launching_xen_services"></a>
+## Launching Xen services
+
+In Dom-0, we need to launch services required by Xen. If you followed the manual build, please make sure that xencommons init script has been started at boot.
+
+For container build you can use [`runit`](http://smarden.org/runit/) process supervisor. You can download and install `runit` RPMs for Fedora from [here](https://drive.google.com/open?id=0B_tTbuxmuRzIR05wQ3E1eWVyaGs).
+
+```shell
+(ensure correct checksum on the downloaded binary)
+[root@localhost ~]# echo "10cc62ffc040c49efa0dd85cbacd70c0712a7c10c58717a376610b786bc49d19  runit-2.1.2-1.1.fc25.tar" | sha256sum -c -
+runit-2.1.2-1.1.fc25.tar: OK
+
+[root@localhost ~]# tar xvf runit-2.1.2-1.1.fc25.tar
+
+[root@localhost ~]# dnf install -y ./runit/2.1.2/1.1.fc25/x86_64/runit-2.1.2-1.1.fc25.x86_64.rpm
+
+[root@localhost ~]# pgrep -af runsvdir
+1679 runsvdir -P -H /etc/service log: ..........................................................
+```
+
+In `/opt/xen-unstable-runit` we provide two scripts to manage Xen services. 
+
+ * `setup.sh`
+ * `teardown.sh`
+
+`setup.sh` is used to setup Xen services. If you are going to be running Fedora directly without Xen, please use `teardown.sh` prior to shutting down Domain-0. This will disable launching Xen services under Fedora without Xen.
+
+Run `setup.sh`
+
+```shell
+[root@localhost ~]# /opt/xen-unstable-runit/setup.sh
+Successfully created symlinks in /etc/service directory.
+```
+
+You can verify Xen services are running correctly by doing the following &ndash;
+
+```shell
+[root@localhost ~]# ls /etc/service | xargs -L 1 -I {} sv status {}
+run: xenconsoled: (pid 29673) 115s
+run: xen-init-dom0: (pid 29672) 115s
+run: xen-init-dom0-disk-backend: (pid 29675) 115s
+run: xenstored: (pid 29674) 115s
+
+[root@localhost ~]# source /opt/stage1-xen/bin/source_path.sh
+
+[root@localhost ~]# xl info
+host                   : localhost.localdomain
+release                : 4.11.12-200.fc25.x86_64
+version                : #1 SMP Fri Jul 21 16:41:43 UTC 2017
+machine                : x86_64
+
+[...]
+
+cc_compile_domain      : [unknown]
+cc_compile_date        : Fri Aug 18 06:32:55 UTC 2017
+build_id               : 4a65e1ae96407a8dd47f318db4bdf7d3
+xend_config_format     : 4
+
+[root@localhost ~]# xl list
+Name                                        ID   Mem VCPUs      State   Time(s)
+Domain-0                                     0  4096     4     r-----     121.2
+```
+
+<a name="setting_up_xen_networking"></a>
+## Setting up Xen networking
+
+There are multiple ways to do networking on Xen. Two common configurations are [bridging](https://wiki.xenproject.org/wiki/Xen_Networking#Bridging) and [NAT](https://wiki.xenproject.org/wiki/Xen_Networking#Network_Address_Translation). Bridging is the default and most simple configuration to setup. However wireless device drivers are unable to do bridging. To overcome this limitation, we setup an internal  bridge and then use NAT to send packets externally. This setup works for both wired and wireless devices.
+
+```shell
+[root@localhost ~]# ip link show
+1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
+    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
+    link/ether c8:5b:76:71:40:c8 brd ff:ff:ff:ff:ff:f
+3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000
+    link/ether e4:a7:a0:93:9f:13 brd ff:ff:ff:ff:ff:f
+```
+
+We have two devices `enp0s31f6` which is a wired ethernet device and `wlp4s0` which is a wireless ethernet device. We will use `wlp4s0` in the following example. However similar approach would also work for `enp0s31f6` device. You can also adjust the private network 10.1.1.0/24 to a non-overlapping private subnet. 
+
+```shell
+[root@localhost ~]# brctl show
+bridge name     bridge id               STP enabled     interfaces
+
+[root@localhost ~]# ip link add xenbr0 type bridge
+
+[root@localhost ~]# ip addr add 10.1.1.1/24 dev xenbr0
+
+[root@localhost ~]# ip link set xenbr0 up
+
+[root@localhost ~]# modprobe dummy
+
+[root@localhost ~]# ip link set dummy0 up
+
+[root@localhost ~]# brctl addif xenbr0 dummy0
+
+[root@localhost ~]# brctl show
+bridge name     bridge id               STP enabled     interfaces
+xenbr0          8000.d21b5c4113b7       no              dummy0
+
+[root@localhost ~]# iptables -I FORWARD -j ACCEPT
+
+[root@localhost ~]# iptables -t nat -I POSTROUTING --out-interface wlp4s0 -j MASQUERADE
+
+[root@localhost ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
+```
+
+With this configuration we can launch Dom-U Xen guests using the following configuration setting.
+
+```
+# Network configuration
+vif = ['bridge=xenbr0']
+```
+
+Then from within the guest, we will need to setup `eth0` interface with a static IP address in the range of 10.1.1.0/24 and gateway as 10.1.1.1.
+
+<a name="running_stage1-xen"></a>
+## Running stage1-xen
+
+Once we have Xen setup, it is fairly straightforward to run stage1-xen.
+
+If you followed manual build, then please ensure that you have `xl` and `rkt` in your path.
+
+For container build, we provide a script to source all the required binaries from Xen, QEMU and rkt into our path.
+
+```shell
+[root@localhost ~]# source /opt/stage1-xen/bin/source_path.sh
+```
+
+We can now download images using `rkt` and run them under stage1-xenbits
+
+```shell
+[root@localhost ~]# rkt --insecure-options=image fetch docker://alpine
+Downloading sha256:88286f41530 [=============================] 1.99 MB / 1.99 MB
+sha512-f84f971f8e01284f4ad0c3cf3efaa770
+
+[root@localhost ~]# rkt run sha512-f84f971f8e01284f4ad0c3cf3efaa770 \
+                      --interactive --insecure-options=image \
+                      --stage1-path=/opt/stage1-xen/aci/stage1-xen.aci
+```
+
+Within the container, we can see we are running as a Xen PV guest, and using 9pfs
+
+```shell
+/ # dmesg | grep [Xx]en
+[    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
+[    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
+[    0.000000] Xen: [mem 0x0000000000100000-0x000000003fffffff] usable
+[    0.000000] Hypervisor detected: Xen
+[    0.000000] Booting paravirtualized kernel on Xen
+[    0.000000] Xen version: 4.10-unstable (preserve-AD)
+[    0.000000] xen:events: Using FIFO-based ABI
+
+[...]
+
+[    1.605990] Initialising Xen transport for 9pfs
+```
+
+From Domain-0, we can run `rkt` and `xl` to get the details of the container.
+
+```shell
+[root@localhost ~]# rkt list
+UUID            APP     IMAGE NAME                                      STATE   CREATED       STARTED          NETWORKS
+222083ec        alpine  registry-1.docker.io/library/alpine:latest      running 4 minutes ago 4 minutes ago    default:ip4=172.16.28.15
+
+[root@localhost ~]# xl list
+Name                                        ID   Mem VCPUs      State   Time(s)
+Domain-0                                     0  4093     4     r-----    1056.9
+222083ec-d6da-4347-b261-0a733bae6802         1  1024     2     -b----       2.2
+
+[root@localhost ~]# rkt stop 222083ec
+"222083ec-d6da-4347-b261-0a733bae6802"
+
+[root@localhost ~]# xl list
+Name                                        ID   Mem VCPUs      State   Time(s)
+Domain-0                                     0  4093     4     r-----    1058.5
+```


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1 10/10] BUILDING.md: Add Fedora instructions

[Rajiv Ranganath]

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 BUILDING.md |   96 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 91 insertions(+), 5 deletions(-)

diff --git a/BUILDING.md b/BUILDING.md
index 3ef5311..946c799 100644
--- a/BUILDING.md
+++ b/BUILDING.md
@@ -1,7 +1,13 @@
 # Build
-stage1-xen requires new Xen and QEMU versions at the time of writing. You are unlikely to find them already packaged with your distro. This document describes how to build and install the latest Xen and QEMU from scratch. In addition, given that CoreOS rkt is also missing from reasonably new distros such as Ubuntu Xenial Xerus, I added instructions on how to build that too. The document includes the dependencies needed for the build based on Ubuntu Xenial Xerus.
+stage1-xen requires new Xen and QEMU versions at the time of writing. You are unlikely to find them already packaged with your distro. This document describes how to build and install the latest Xen, QEMU and rkt from scratch for Ubuntu Xenial Xerus and Fedora. Differently from documentation for Ubuntu, the documentation for Fedora uses a Docker container for the build. There is also support for building on host on Fedora.
 
-## Building Xen
+ * [Ubuntu Xenial Xerus](#build_ubuntu)
+ * [Fedora](#build_fedora)
+
+<a name="build_ubuntu"></a>
+## Ubuntu Xenial Xerus
+
+### Building Xen
 ```
 apt-get install git build-essential python-dev gettext uuid-dev libncurses5-dev libyajl-dev libaio-dev pkg-config libglib2.0-dev libssl-dev libpixman-1-dev bridge-utils wget libfdt-dev bin86 bcc liblzma-dev iasl libc6-dev-i386
 
@@ -17,7 +23,7 @@ reboot
 Make sure to select Xen at boot, or edit /boot/grub/grub.cfg to make it the default, changing "set default="0" to point to the appropriate entry below (the one booting xen.gz), which could be entry number "4" for example.
 
 
-## Building QEMU
+### Building QEMU
 ```
 apt-get install libglib2.0-dev libpixman-1-dev libcap-dev libattr1-dev
 
@@ -54,7 +60,7 @@ make install
 cp i386-softmmu/qemu-system-i386 /usr/lib/xen/bin/
 ```
 
-## Building CoreOS rkt
+### Building CoreOS rkt
 ```
 apt-get install golang automake libacl1-dev libsystemd-dev
 ./configure --disable-tpm --with-stage1-flavors=coreos
@@ -62,7 +68,7 @@ make
 cp build-rkt-1.26.0+git/target/bin/rkt /usr/sbin
 ```
 
-## Building stage1-xen
+### Building stage1-xen
 ```
 apt-get install busybox-static jq
 
@@ -72,3 +78,83 @@ export GOPATH=/path/to/gopath
 bash build.sh
 cp stage1-xen.aci /home/username
 ```
+
+<a name="build_fedora"></a>
+## Fedora
+
+On Fedora there are two ways to build stage1-xen artifacts.
+
+ * [Container Build](#build_fedora_container_build)
+ * [Manual Build](#build_fedora_manual_build)
+
+<a name="build_fedora_container_build"></a>
+### Container Build
+
+We can build stage1-xen artifacts (Xen, QEMU and rkt) automatically in a docker container as follows &ndash;
+
+```
+cd stage1-xen
+
+docker pull lambdalinuxfedora/stage1-xen-fedora-buildroot
+
+docker run --rm \
+  -v `pwd`:/root/gopath/src/github.com/rkt/stage1-xen \
+  -v /tmp:/tmp \
+  -t -i lambdalinuxfedora/stage1-xen-fedora-buildroot \
+  /sbin/my_init -- /root/bin/run
+```
+
+Once `docker run` completes, the build artifact `stage1-xen-build.tar.gz` is generated in `/tmp` directory. Please see [RUNNING_STAGE1_XEN.md](build/fedora/RUNNING_STAGE1_XEN.md) for details on how to setup Fedora for running stage1-xen.
+
+<a name="build_fedora_manual_build"></a>
+### Manual Build
+
+It is also possible to manually build stage1-xen components on a Fedora host. 
+
+Please ensure that you have all the dependencies installed. The dependencies for Xen, QEMU, rkt and stage1-xen is documented in [buildroot-Dockerfile](build/fedora/buildroot-Dockerfile). You will also need to install [`binutils`](https://github.com/lambda-linux-fedora/binutils) package that is compiled with `i386pe` support. You can download the pre-built RPMs from [here](https://drive.google.com/open?id=0B_tTbuxmuRzIR05wQ3E1eWVyaGs).
+
+Install `binutils` package.
+
+```
+tar xvf binutils-2.26.1-1.1.fc25.tar
+
+dnf install -y ./binutils/2.26.1/1.1.fc25/x86_64/binutils-2.26.1-1.1.fc25.x86_64.rpm
+```
+
+You can verify `i386pe` support in `binutils` by doing the following.
+
+```
+[root@localhost]# ld -V
+GNU ld version 2.26.1-1.1.fc25  Supported emulations:
+   elf_x86_64
+   elf32_x86_64
+   elf_i386
+   elf_iamcu
+   i386linux
+   elf_l1om
+   elf_k1om
+   i386pep
+   i386pe
+```
+
+You should see the lines `i386pep` and `i386pe` in the output.
+
+Next you can build Xen, Qemu and rkt using the following scripts &ndash;
+
+ * [`build/fedora/components/xen`](build/fedora/components/xen)
+ * [`build/fedora/components/qemu`](build/fedora/components/qemu)
+ * [`build/fedora/components/rkt`](build/fedora/components/rkt)
+
+Please review the scripts and adjust the paths according to your requirements.
+
+Once the dependencies are installed, you can build stage1-xen
+
+```
+git clone https://github.com/rkt/stage1-xen.git
+cd stage1-xen
+export GOPATH=/path/to/gopath
+bash build.sh
+cp stage1-xen.aci /home/username
+```
+
+Please see [RUNNING_STAGE1_XEN.md](build/fedora/RUNNING_STAGE1_XEN.md) for details on how to run rkt with stage1-xen.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

The series is much better now thank you. One question: why did you write
your own init scripts rather than reusing xencommons (with the caveat
that you would have to add make sure to source_path.sh before running
xencommons)?  Does it have something to do with systemd?


> ---
>  build/fedora/xen-unstable-runit/setup.sh           |   18 ++++++++++++++++
>  build/fedora/xen-unstable-runit/teardown.sh        |   18 ++++++++++++++++
>  .../xen-init-dom0-disk-backend/run                 |   11 ++++++++++
>  build/fedora/xen-unstable-runit/xen-init-dom0/run  |    9 ++++++++
>  build/fedora/xen-unstable-runit/xenconsoled/run    |   13 +++++++++++
>  build/fedora/xen-unstable-runit/xenstored/run      |   23 ++++++++++++++++++++
>  6 files changed, 92 insertions(+)
>  create mode 100755 build/fedora/xen-unstable-runit/setup.sh
>  create mode 100755 build/fedora/xen-unstable-runit/teardown.sh
>  create mode 100755 build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
>  create mode 100755 build/fedora/xen-unstable-runit/xen-init-dom0/run
>  create mode 100755 build/fedora/xen-unstable-runit/xenconsoled/run
>  create mode 100755 build/fedora/xen-unstable-runit/xenstored/run
> 
> diff --git a/build/fedora/xen-unstable-runit/setup.sh b/build/fedora/xen-unstable-runit/setup.sh
> new file mode 100755
> index 0000000..b5adf8c
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/setup.sh
> @@ -0,0 +1,18 @@
> +#!/bin/bash
> +
> +set -e
> +
> +# runit RPM creates `/etc/service` directory
> +if [ ! -d "/etc/service" ]; then
> +    echo "/etc/service directory not found. Please install runit RPM."
> +    exit 1
> +fi
> +
> +runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend xenstored"
> +
> +for service in $runit_services; do
> +    ln -sf /opt/xen-unstable-runit/$service /etc/service/$service
> +done
> +
> +echo "Successfully created symlinks in /etc/service directory."
> +exit 0
> diff --git a/build/fedora/xen-unstable-runit/teardown.sh b/build/fedora/xen-unstable-runit/teardown.sh
> new file mode 100755
> index 0000000..d333807
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/teardown.sh
> @@ -0,0 +1,18 @@
> +#!/bin/bash
> +
> +set -e
> +
> +# runit RPM creates `/etc/service` directory
> +if [ ! -d "/etc/service" ]; then
> +    echo "/etc/service directory not found."
> +    exit 1
> +fi
> +
> +runit_services="xenconsoled xen-init-dom0 xen-init-dom0-disk-backend xenstored"
> +
> +for service in $runit_services; do
> +    rm -f /etc/service/$service
> +done
> +
> +echo "Successfully deleted symlinks in /etc/service directory."
> +exit 0
> diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
> new file mode 100755
> index 0000000..6315d48
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xen-init-dom0-disk-backend/run
> @@ -0,0 +1,11 @@
> +#!/bin/bash
> +
> +set -e
> +
> +sv check xenstored >/dev/null || exit 1
> +sv check xenconsoled >/dev/null || exit 1
> +
> +# In case of failure, allow user to run teardown script
> +sleep 5s
> +
> +exec /opt/xen-unstable/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -monitor /dev/null -serial /dev/null -parallel /dev/null -nodefaults -no-user-config
> diff --git a/build/fedora/xen-unstable-runit/xen-init-dom0/run b/build/fedora/xen-unstable-runit/xen-init-dom0/run
> new file mode 100755
> index 0000000..193ba19
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xen-init-dom0/run
> @@ -0,0 +1,9 @@
> +#!/bin/bash
> +
> +set -e
> +
> +sv check xenstored >/dev/null || exit 1
> +
> +/opt/xen-unstable/lib/xen/bin/xen-init-dom0
> +
> +exec chpst -b xen-init-dom0 runit-pause
> diff --git a/build/fedora/xen-unstable-runit/xenconsoled/run b/build/fedora/xen-unstable-runit/xenconsoled/run
> new file mode 100755
> index 0000000..b5b7a9f
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xenconsoled/run
> @@ -0,0 +1,13 @@
> +#!/bin/bash
> +
> +set -e
> +
> +sv check xen-init-dom0 >/dev/null || exit 1
> +
> +[ ! -d /var/log/xen/console ] && mkdir -p /var/log/xen/console
> +
> +# In case of failure, allow user to run teardown script
> +sleep 5s
> +
> +# --log=[none|guest|hv|all]
> +exec /opt/xen-unstable/sbin/xenconsoled -i --log=none
> diff --git a/build/fedora/xen-unstable-runit/xenstored/run b/build/fedora/xen-unstable-runit/xenstored/run
> new file mode 100755
> index 0000000..beb2a5f
> --- /dev/null
> +++ b/build/fedora/xen-unstable-runit/xenstored/run
> @@ -0,0 +1,23 @@
> +#!/bin/bash
> +
> +set -e
> +
> +[ ! -d /var/run/xen ] && mkdir -p /var/run/xen
> +[ ! -d /var/run/xenstored ] && mkdir -p /var/run/xenstored
> +[ ! -d /var/log/xen ] && mkdir -p /var/log/xen
> +[ ! -d /var/lib/xen ] && mkdir -p /var/lib/xen
> +[ ! -d /var/lib/xen/dump ] && mkdir -p /var/lib/xen/dump
> +[ ! -d /var/lib/xen/xenpaging ] && mkdir -p /var/lib/xen/paging
> +[ ! -d /var/lib/xenstored ] && mkdir -p /var/lib/xenstored
> +modprobe -q xen-evtchn || exit 1
> +modprobe -q xen-gntdev || exit 1
> +modprobe -q xen-gntalloc || exit 1
> +mountpoint -q /proc/xen || mount -t xenfs xenfs /proc/xen
> +mountpoint -q /var/lib/xenstored || mount -t tmpfs xenstored /var/lib/xenstored
> +grep -q "control_d" /proc/xen/capabilities || exit 1
> +
> +# In case of failure, allow user to run teardown script
> +sleep 5s
> +
> +# Use `--trace-file /var/log/xen/xenstored-trace.log` for logs
> +exec /opt/xen-unstable/sbin/xenstored --no-fork
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 01/10] .gitignore: Add

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
>  .gitignore |    2 ++
>  1 file changed, 2 insertions(+)
>  create mode 100644 .gitignore
> 
> diff --git a/.gitignore b/.gitignore
> new file mode 100644
> index 0000000..873f8f6
> --- /dev/null
> +++ b/.gitignore
> @@ -0,0 +1,2 @@
> +# build/fedora
> +build/fedora/binutils-*.tar
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 02/10] build/fedora: Add `buildroot-README.md`

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
>  build/fedora/buildroot-README.md |   50 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 build/fedora/buildroot-README.md
> 
> diff --git a/build/fedora/buildroot-README.md b/build/fedora/buildroot-README.md
> new file mode 100644
> index 0000000..0efb150
> --- /dev/null
> +++ b/build/fedora/buildroot-README.md
> @@ -0,0 +1,50 @@
> +## stage1-xen Fedora Buildroot
> +
> +stage1-xen build artifacts for Fedora is built in two phases. In the first phase
> +a docker container is prepared with all the build dependencies. We refer to it
> +as `stage1-xen-fedora-buildroot`. In the next phase we execute the `run` script
> +that uses `stage1-xen-fedora-buildroot` and to produce the build artifacts.
> +
> +### Building `stage1-xen-fedora-buildroot`
> +
> +`stage1-xen-fedora-buildroot` has a external dependency
> +on [`binutils`](https://github.com/lambda-linux-fedora/binutils) package that is
> +compiled with `i386pe` support. You can download the pre-built RPMs
> +from [here](https://drive.google.com/open?id=0B_tTbuxmuRzIR05wQ3E1eWVyaGs).
> +Please download `binutils-2.26.1-1.1.fc25.tar`.
> +
> +To build docker image
> +
> +```
> +cd stage1-xen/build/fedora
> +
> +docker build -f buildroot-Dockerfile -t stage1-xen-fedora-buildroot .
> +```
> +
> +### Running `stage1-xen-fedora-buildroot`
> +
> +```
> +cd stage1-xen
> +
> +docker run --rm \
> +  -v `pwd`:/root/gopath/src/github.com/rkt/stage1-xen \
> +  -v /tmp:/tmp \
> +  -t -i stage1-xen-fedora-buildroot \
> +  /sbin/my_init -- /root/bin/run
> +```
> +
> +The generated build artifacts are in `/tmp` directory.
> +
> +To debug build issues -
> +
> +```
> +cd stage1-xen
> +
> +docker run --rm \
> +  -v `pwd`:/root/gopath/src/github.com/rkt/stage1-xen \
> +  -v /tmp:/tmp \
> +  -t -i stage1-xen-fedora-buildroot \
> +  /sbin/my_init -- /bin/bash
> +```
> +
> +Also see section on `ipdb` in `buildroot-Dockerfile`.
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 03/10] build/fedora: Add `buildroot-Dockerfile`

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  build/fedora/buildroot-Dockerfile |  113 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 113 insertions(+)
>  create mode 100644 build/fedora/buildroot-Dockerfile
> 
> diff --git a/build/fedora/buildroot-Dockerfile b/build/fedora/buildroot-Dockerfile
> new file mode 100644
> index 0000000..971560e
> --- /dev/null
> +++ b/build/fedora/buildroot-Dockerfile
> @@ -0,0 +1,113 @@
> +# tarballs checksum
> +# -----------------
> +# 974b3091232d781c4fc410ccca98fb62ba9febe9e6a988e348804483c4f66742  binutils-2.26.1-1.1.fc25.tar
> +
> +FROM lambdalinuxfedora/baseimage-fedora
> +
> +CMD ["/sbin/my_init"]
> +
> +COPY [ \
> +  "./binutils-2.26.1-1.1.fc25.tar", \
> +  \
> +  "./components/*", \
> +  "./run", \
> +  "/tmp/docker-build/" \
> +]
> +
> +RUN \
> +  # dnf
> +  echo "Running dnf update..." && \
> +  dnf update -y && \
> +  dnf install -y less && \
> +  dnf install -y sudo && \
> +  \
> +  # circleci container requirements
> +  # https://circleci.com/docs/2.0/custom-images/#adding-required-and-custom-tools-or-files
> +  dnf install -y git && \
> +  dnf install -y openssh-clients && \
> +  dnf install -y tar && \
> +  dnf install -y gzip && \
> +  dnf install -y ca-certificates && \
> +  \
> +  # install `binutils`
> +  pushd /tmp/docker-build && \
> +    # verify checksum
> +    echo "974b3091232d781c4fc410ccca98fb62ba9febe9e6a988e348804483c4f66742  binutils-2.26.1-1.1.fc25.tar" | sha256sum -c - && \
> +    tar xvf binutils-2.26.1-1.1.fc25.tar && \
> +    dnf install -y ./binutils/2.26.1/1.1.fc25/x86_64/binutils-2.26.1-1.1.fc25.x86_64.rpm && \
> +  popd && \
> +  \
> +  dnf install -y @buildsys-build && \
> +  \
> +  # Having `ipdb` around is useful when debugging `run` script. Uncomment this
> +  # section as required
> +  # dnf install -y python2-devel && \
> +  # dnf install -y python-pip && \
> +  # su -l root -c "pip2 install --user ipdb==0.8 ipython==5.3.0" && \
> +  \
> +  # Note: xen and qemu has some duplicate package dependencies. We are
> +  # explicitly calling out dependencies for xen and qemu
> +  #
> +  # xen build dependencies
> +  dnf install -y bridge-utils && \
> +  dnf install -y gettext && \
> +  dnf install -y glib2-devel && \
> +  dnf install -y glibc-devel.i686 && \
> +  dnf install -y grub2 && \
> +  dnf install -y iasl && \
> +  dnf install -y libaio-devel && \
> +  dnf install -y libuuid-devel && \
> +  dnf install -y ncurses-devel && \
> +  dnf install -y openssl-devel && \
> +  dnf install -y pixman-devel && \
> +  dnf install -y python2-devel && \
> +  dnf install -y wget && \
> +  dnf install -y yajl-devel && \
> +  \
> +  # qemu build dependencies
> +  dnf install -y glib2-devel && \
> +  dnf install -y libaio-devel && \
> +  dnf install -y libattr-devel && \
> +  dnf install -y libcap-devel && \
> +  dnf install -y libcap-ng-devel && \
> +  dnf install -y pixman-devel && \
> +  dnf install -y zlib-devel && \
> +  \
> +  # rkt build dependencies
> +  dnf install -y autoconf && \
> +  dnf install -y automake && \
> +  dnf install -y git && \
> +  dnf install -y glibc-static && \
> +  dnf install -y gnupg && \
> +  dnf install -y golang && \
> +  dnf install -y libacl-devel && \
> +  dnf install -y squashfs-tools && \
> +  dnf install -y systemd-devel && \
> +  dnf install -y wget && \
> +  \
> +  # stage1-xen build dependencies
> +  dnf install -y bc && \
> +  dnf install -y busybox && \
> +  dnf install -y glide && \
> +  dnf install -y golang && \
> +  dnf install -y jq && \
> +  dnf install -y libacl-devel && \
> +  dnf install -y wget && \
> +  \
> +  # copy `run` file and `components/{qemu,rkt,xen}`
> +  su -l root -c "mkdir /root/bin" && \
> +  su -l root -c "cp /tmp/docker-build/run /root/bin" && \
> +  su -l root -c "mkdir /root/bin/components" && \
> +  su -l root -c "cp /tmp/docker-build/qemu /root/bin/components" && \
> +  su -l root -c "cp /tmp/docker-build/rkt /root/bin/components" && \
> +  su -l root -c "cp /tmp/docker-build/xen /root/bin/components" && \
> +  \
> +  # create `stage1-xen` directory
> +  mkdir -p /root/gopath/src/github.com/rkt/stage1-xen && \
> +  \
> +  # cleanup
> +  rm -rf /tmp/docker-build && \
> +  dnf clean all && \
> +  rm -rf /var/cache/dnf/* && \
> +  rm -rf /tmp/* && \
> +  rm -rf /var/tmp/*
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 08/10] README.md: Add CircleCI badge

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  README.md |    2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/README.md b/README.md
> index 9ea6adf..e1cd40c 100644
> --- a/README.md
> +++ b/README.md
> @@ -1,5 +1,7 @@
>  # stage1-xen - A Xen based stage1 for CoreOS rkt
>  
> +[![Build Status](https://circleci.com/gh/rkt/stage1-xen/tree/master.svg?style=shield&circle-token=:circle-token)](https://circleci.com/gh/rkt/stage1-xen/tree/master)
> +
>  ## Goal
>  
>  CoreOS rkt is a modular container engine with [three stages of execution](https://coreos.com/rkt/docs/latest/devel/stage1-implementors-guide.html). Stage1 is responsible for creating the execution environment for the contained applications.
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
> ---
>  build/fedora/components/qemu |   50 ++++++++++++++++++++++++++++++++++++
>  build/fedora/components/rkt  |   58 ++++++++++++++++++++++++++++++++++++++++++
>  build/fedora/components/xen  |   46 +++++++++++++++++++++++++++++++++
>  build/fedora/run             |   56 +++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 210 insertions(+)
>  create mode 100755 build/fedora/components/qemu
>  create mode 100755 build/fedora/components/rkt
>  create mode 100755 build/fedora/components/xen
>  create mode 100755 build/fedora/run
> 
> diff --git a/build/fedora/components/qemu b/build/fedora/components/qemu
> new file mode 100755
> index 0000000..6c89e2c
> --- /dev/null
> +++ b/build/fedora/components/qemu
> @@ -0,0 +1,50 @@
> +#!/usr/bin/python2
> +
> +import shlex
> +import subprocess
> +import sys
> +import os
> +
> +# Modify this if you would like to install Qemu elsewhere on your filesystem or
> +# a different version of Qemu
> +QEMU_PREFIX = '/opt/qemu-unstable'
> +QEMU_BRANCH = 'master'

I am not sure we want to checkout always the latest QEMU. It is a
running target. It makes sense to use one of the latest releases
instead, such as v2.10.0?


> +# This should correspond to your Xen install prefix
> +XEN_PREFIX = '/opt/xen-unstable'
> +
> +
> +# helper function to capture stdout from a long running process
> +def subprocess_stdout(cmd, cwd, env):
> +    p = subprocess.Popen(
> +        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
> +    while p.poll() is None:
> +        l = p.stdout.readline()
> +        sys.stdout.write(l)
> +    if p.returncode != 0:
> +        sys.exit(1)

Is this the same as
  #!/bin/bash
  set -e
?

Please add a few words in the commit message about the benefit of this
approach of writing scripts.


> +env = os.environ.copy()
> +
> +# build and install qemu
> +print "Cloning qemu..."
> +cmd = "git clone --branch %(branch)s git://git.qemu.org/qemu.git" % {
> +    'branch': QEMU_BRANCH
> +}
> +subprocess.check_output(shlex.split(cmd), cwd='/root')
> +
> +steps = [
> +    "./configure --prefix=%(qemu_prefix)s --enable-xen --target-list=i386-softmmu --extra-cflags=\"-I%(xen_prefix)s/include\" --extra-ldflags=\"-L%(xen_prefix)s/lib -Wl,-rpath,%(xen_prefix)s/lib\" --disable-kvm --enable-virtfs --enable-linux-aio"
> +    % {
> +        'qemu_prefix': QEMU_PREFIX,
> +        'xen_prefix': XEN_PREFIX
> +    }, 'make', 'make install'
> +]
> +for cmd in steps:
> +    cwd = '/root/qemu'
> +    subprocess_stdout(cmd, cwd, env)
> +
> +cmd = "cp i386-softmmu/qemu-system-i386 %(xen_prefix)s/lib/xen/bin/qemu-system-i386" % {
> +    'xen_prefix': XEN_PREFIX
> +}
> +subprocess.check_output(shlex.split(cmd), cwd='/root/qemu')
> diff --git a/build/fedora/components/rkt b/build/fedora/components/rkt
> new file mode 100755
> index 0000000..edfdd1c
> --- /dev/null
> +++ b/build/fedora/components/rkt
> @@ -0,0 +1,58 @@
> +#!/usr/bin/python2
> +
> +import shlex
> +import subprocess
> +import sys
> +import os
> +
> +# `rkt` is installed in the same prefix as `stage1-xen`. Modify this if you
> +# would like to install rkt elsewhere on your filesystem.
> +STAGE1_XEN_PREFIX = '/opt/stage1-xen'
> +RKT_PREFIX = STAGE1_XEN_PREFIX
> +RKT_BRANCH = 'master'
> +
> +# Adjust this according to what RKT_BRANCH generates
> +RKT_BUILD_VER = 'rkt-1.28.1+git'

I think it would be best if git-checked out the tag (v1.28.1) so that we
are sure there are no version mismatches. In fact, I would remove
RKT_BRANCH and just use a single variable to specify the version to
clone and build.


> +# helper function to capture stdout from a long running process
> +def subprocess_stdout(cmd, cwd, env):
> +    p = subprocess.Popen(
> +        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
> +    while p.poll() is None:
> +        l = p.stdout.readline()
> +        sys.stdout.write(l)
> +    if p.returncode != 0:
> +        sys.exi(1)
> +
> +
> +env = os.environ.copy()
> +
> +# build rkt
> +print "Cloning rkt..."
> +cmd = "git clone --branch %(branch)s https://github.com/rkt/rkt.git" % {
> +    'branch': RKT_BRANCH
> +}
> +subprocess.check_output(shlex.split(cmd), cwd='/root')
> +
> +steps = [
> +    './autogen.sh', './configure --disable-tpm --with-stage1-flavors=coreos',
> +    'make'
> +]
> +for cmd in steps:
> +    cwd = '/root/rkt'
> +    subprocess_stdout(cmd, cwd, env)
> +
> +# install rkt build artifacts to RKT_PREFIX
> +steps = [
> +    "mkdir -p %(prefix)s/bin" % {
> +        'prefix': RKT_PREFIX
> +    },
> +    "cp /root/rkt/build-%(build_ver)s/target/bin/rkt %(prefix)s/bin/rkt" % {
> +        'build_ver': RKT_BUILD_VER,
> +        'prefix': RKT_PREFIX
> +    }
> +]
> +for cmd in steps:
> +    cwd = '/root/rkt'
> +    subprocess_stdout(cmd, cwd, env)
> diff --git a/build/fedora/components/xen b/build/fedora/components/xen
> new file mode 100755
> index 0000000..95da9a6
> --- /dev/null
> +++ b/build/fedora/components/xen
> @@ -0,0 +1,46 @@
> +#!/usr/bin/python2
> +
> +import shlex
> +import subprocess
> +import sys
> +import os
> +
> +# Modify this if you would like to install Xen elsewhere on your filesystem or
> +# a different version of Xen
> +XEN_PREFIX = '/opt/xen-unstable'
> +XEN_BRANCH = 'master'

Same here, I think that for a stage1-xen CI-loop it's best to build and
test again a stable target. Thus, instead of master, I would use a tag
like RELEASE-4.9.0.


> +# helper function to capture stdout from a long running process
> +def subprocess_stdout(cmd, cwd, env):
> +    p = subprocess.Popen(
> +        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
> +    while p.poll() is None:
> +        l = p.stdout.readline()
> +        sys.stdout.write(l)
> +    if p.returncode != 0:
> +        sys.exit(1)
> +
> +
> +env = os.environ.copy()
> +
> +# build and install xen
> +print "Cloning xen..."
> +cmd = "git clone --branch %(branch)s git://xenbits.xen.org/xen.git" % {
> +    'branch': XEN_BRANCH
> +}
> +subprocess.check_output(shlex.split(cmd), cwd='/root')
> +
> +steps = [
> +    "./configure --prefix=%(prefix)s --with-system-qemu=%(prefix)s/lib/xen/bin/qemu-system-i386 --disable-stubdom --disable-qemu-traditional --disable-rombios --sysconfdir=%(prefix)s/etc --enable-rpath --disable-systemd"
> +    % {
> +        'prefix': XEN_PREFIX
> +    }, 'make',
> +    "make install BOOT_DIR=%(prefix)s/boot DEBUG_DIR=%(prefix)s/lib/debug EFI_DIR=%(prefix)s/boot/efi/EFI/xen"
> +    % {
> +        'prefix': XEN_PREFIX
> +    }
> +]
> +for cmd in steps:
> +    cwd = '/root/xen'
> +    subprocess_stdout(cmd, cwd, env)
> diff --git a/build/fedora/run b/build/fedora/run
> new file mode 100755
> index 0000000..6cb6417
> --- /dev/null
> +++ b/build/fedora/run
> @@ -0,0 +1,56 @@
> +#!/usr/bin/python2
> +
> +import shlex
> +import subprocess
> +import sys
> +import os
> +
> +# This scripts calls out to `xen`, `qemu` and `rkt` scripts in the
> +# `components/` directory within a container. It is expected that components
> +# directory is present at the same directory level as run script.
> +STAGE1_XEN_COMPONENTS = ['xen', 'qemu', 'rkt']
> +
> +
> +# helper function to capture stdout from a long running process
> +def subprocess_stdout(cmd, cwd, env):
> +    p = subprocess.Popen(
> +        shlex.split(cmd), cwd=cwd, env=env, stdout=subprocess.PIPE)
> +    while p.poll() is None:
> +        l = p.stdout.readline()
> +        sys.stdout.write(l)
> +    if p.returncode != 0:
> +        sys.exit(1)
> +
> +
> +env = os.environ.copy()
> +
> +dirname = os.path.dirname(os.path.realpath(__file__))
> +steps = [os.path.join(dirname, 'components', x) for x in STAGE1_XEN_COMPONENTS]
> +for cmd in steps:
> +    cwd = '/root'
> +    subprocess_stdout(cmd, cwd, env)
> +
> +# build stage1-xen
> +env['GOPATH'] = '/root/gopath'
> +cwd = '/root/gopath/src/github.com/rkt/stage1-xen'
> +cmd = 'bash build.sh'
> +subprocess_stdout(cmd, cwd, env)
> +
> +# install build artifacts to `/opt/stage1-xen/aci` and create a tarball
> +steps = [
> +    'mkdir -p /opt/stage1-xen/aci',
> +    'cp /root/gopath/src/github.com/rkt/stage1-xen/stage1-xen.aci /opt/stage1-xen/aci/stage1-xen.aci',
> +    'cp /root/gopath/src/github.com/rkt/stage1-xen/build/fedora/source_path.sh /opt/stage1-xen/bin/source_path.sh',
> +    'cp -r /root/gopath/src/github.com/rkt/stage1-xen/build/fedora/xen-unstable-runit /opt/xen-unstable-runit'
> +]
> +for cmd in steps:
> +    cwd = '/root'
> +    subprocess_stdout(cmd, cwd, env)
> +
> +cwd = '/opt'
> +cmd = 'tar zcvf /root/stage1-xen-build.tar.gz qemu-unstable/ stage1-xen/ xen-unstable/ xen-unstable-runit/'
> +subprocess_stdout(cmd, cwd, env)
> +
> +cwd = '/root'
> +cmd = 'mv /root/stage1-xen-build.tar.gz /tmp'
> +subprocess_stdout(cmd, cwd, env)
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 07/10] .circleci/config.yml: Add

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> From: Rajiv M Ranganath <rajiv.ranganath@atihita.com>
> 
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  .circleci/config.yml |   21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>  create mode 100644 .circleci/config.yml
> 
> diff --git a/.circleci/config.yml b/.circleci/config.yml
> new file mode 100644
> index 0000000..93315b4
> --- /dev/null
> +++ b/.circleci/config.yml
> @@ -0,0 +1,21 @@
> +version: 2
> +jobs:
> +  build:
> +    working_directory: /root
> +    docker:
> +      - image: lambdalinuxfedora/stage1-xen-fedora-buildroot:1708260126
> +        command: /sbin/my_init
> +    steps:
> +      - run:
> +          # We create `stage1-xen` directory in Dockerfile for local dev
> +          # environment. Removing it here so CircleCI checkout step can work
> +          # correctly
> +          name: Removing stage1-xen directory from GOPATH...
> +          command: |
> +            rm -rf /root/gopath/src/github.com/rkt/stage1-xen
> +      - checkout:
> +          path: /root/gopath/src/github.com/rkt/stage1-xen
> +      - run:
> +          name: Starting run...
> +          command: |
> +            /root/bin/run
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 10/10] BUILDING.md: Add Fedora instructions

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  BUILDING.md |   96 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 91 insertions(+), 5 deletions(-)
> 
> diff --git a/BUILDING.md b/BUILDING.md
> index 3ef5311..946c799 100644
> --- a/BUILDING.md
> +++ b/BUILDING.md
> @@ -1,7 +1,13 @@
>  # Build
> -stage1-xen requires new Xen and QEMU versions at the time of writing. You are unlikely to find them already packaged with your distro. This document describes how to build and install the latest Xen and QEMU from scratch. In addition, given that CoreOS rkt is also missing from reasonably new distros such as Ubuntu Xenial Xerus, I added instructions on how to build that too. The document includes the dependencies needed for the build based on Ubuntu Xenial Xerus.
> +stage1-xen requires new Xen and QEMU versions at the time of writing. You are unlikely to find them already packaged with your distro. This document describes how to build and install the latest Xen, QEMU and rkt from scratch for Ubuntu Xenial Xerus and Fedora. Differently from documentation for Ubuntu, the documentation for Fedora uses a Docker container for the build. There is also support for building on host on Fedora.
>  
> -## Building Xen
> + * [Ubuntu Xenial Xerus](#build_ubuntu)
> + * [Fedora](#build_fedora)
> +
> +<a name="build_ubuntu"></a>
> +## Ubuntu Xenial Xerus
> +
> +### Building Xen
>  ```
>  apt-get install git build-essential python-dev gettext uuid-dev libncurses5-dev libyajl-dev libaio-dev pkg-config libglib2.0-dev libssl-dev libpixman-1-dev bridge-utils wget libfdt-dev bin86 bcc liblzma-dev iasl libc6-dev-i386
>  
> @@ -17,7 +23,7 @@ reboot
>  Make sure to select Xen at boot, or edit /boot/grub/grub.cfg to make it the default, changing "set default="0" to point to the appropriate entry below (the one booting xen.gz), which could be entry number "4" for example.
>  
>  
> -## Building QEMU
> +### Building QEMU
>  ```
>  apt-get install libglib2.0-dev libpixman-1-dev libcap-dev libattr1-dev
>  
> @@ -54,7 +60,7 @@ make install
>  cp i386-softmmu/qemu-system-i386 /usr/lib/xen/bin/
>  ```
>  
> -## Building CoreOS rkt
> +### Building CoreOS rkt
>  ```
>  apt-get install golang automake libacl1-dev libsystemd-dev
>  ./configure --disable-tpm --with-stage1-flavors=coreos
> @@ -62,7 +68,7 @@ make
>  cp build-rkt-1.26.0+git/target/bin/rkt /usr/sbin
>  ```
>  
> -## Building stage1-xen
> +### Building stage1-xen
>  ```
>  apt-get install busybox-static jq
>  
> @@ -72,3 +78,83 @@ export GOPATH=/path/to/gopath
>  bash build.sh
>  cp stage1-xen.aci /home/username
>  ```
> +
> +<a name="build_fedora"></a>
> +## Fedora
> +
> +On Fedora there are two ways to build stage1-xen artifacts.
> +
> + * [Container Build](#build_fedora_container_build)
> + * [Manual Build](#build_fedora_manual_build)
> +
> +<a name="build_fedora_container_build"></a>
> +### Container Build
> +
> +We can build stage1-xen artifacts (Xen, QEMU and rkt) automatically in a docker container as follows &ndash;
> +
> +```
> +cd stage1-xen
> +
> +docker pull lambdalinuxfedora/stage1-xen-fedora-buildroot
> +
> +docker run --rm \
> +  -v `pwd`:/root/gopath/src/github.com/rkt/stage1-xen \
> +  -v /tmp:/tmp \
> +  -t -i lambdalinuxfedora/stage1-xen-fedora-buildroot \
> +  /sbin/my_init -- /root/bin/run
> +```
> +
> +Once `docker run` completes, the build artifact `stage1-xen-build.tar.gz` is generated in `/tmp` directory. Please see [RUNNING_STAGE1_XEN.md](build/fedora/RUNNING_STAGE1_XEN.md) for details on how to setup Fedora for running stage1-xen.
> +
> +<a name="build_fedora_manual_build"></a>
> +### Manual Build
> +
> +It is also possible to manually build stage1-xen components on a Fedora host. 
> +
> +Please ensure that you have all the dependencies installed. The dependencies for Xen, QEMU, rkt and stage1-xen is documented in [buildroot-Dockerfile](build/fedora/buildroot-Dockerfile). You will also need to install [`binutils`](https://github.com/lambda-linux-fedora/binutils) package that is compiled with `i386pe` support. You can download the pre-built RPMs from [here](https://drive.google.com/open?id=0B_tTbuxmuRzIR05wQ3E1eWVyaGs).
> +
> +Install `binutils` package.
> +
> +```
> +tar xvf binutils-2.26.1-1.1.fc25.tar
> +
> +dnf install -y ./binutils/2.26.1/1.1.fc25/x86_64/binutils-2.26.1-1.1.fc25.x86_64.rpm
> +```
> +
> +You can verify `i386pe` support in `binutils` by doing the following.
> +
> +```
> +[root@localhost]# ld -V
> +GNU ld version 2.26.1-1.1.fc25  Supported emulations:
> +   elf_x86_64
> +   elf32_x86_64
> +   elf_i386
> +   elf_iamcu
> +   i386linux
> +   elf_l1om
> +   elf_k1om
> +   i386pep
> +   i386pe
> +```
> +
> +You should see the lines `i386pep` and `i386pe` in the output.
> +
> +Next you can build Xen, Qemu and rkt using the following scripts &ndash;
> +
> + * [`build/fedora/components/xen`](build/fedora/components/xen)
> + * [`build/fedora/components/qemu`](build/fedora/components/qemu)
> + * [`build/fedora/components/rkt`](build/fedora/components/rkt)
> +
> +Please review the scripts and adjust the paths according to your requirements.
> +
> +Once the dependencies are installed, you can build stage1-xen
> +
> +```
> +git clone https://github.com/rkt/stage1-xen.git
> +cd stage1-xen
> +export GOPATH=/path/to/gopath
> +bash build.sh
> +cp stage1-xen.aci /home/username
> +```
> +
> +Please see [RUNNING_STAGE1_XEN.md](build/fedora/RUNNING_STAGE1_XEN.md) for details on how to run rkt with stage1-xen.
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 09/10] build/fedora: Add `RUNNING_STAGE1_XEN.md`

[Stefano Stabellini]

On Sun, 27 Aug 2017, Rajiv Ranganath wrote:
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>

This is very detailed, good stuff. I have one question below:


> +<a name="booting_into_xen"></a>
> +## Booting into Xen
> +
> +Build and install Xen and stage1-xen. Please see [BUILDING.md](/BUILDING.md#build_fedora).
> +
> +If you followed the container build with Docker, then copy over `stage1-xen-build.tar.gz`. Extract `stage1-xen-build.tar.gz` into `/opt` directory.
> +
> +```shell
> +[root@localhost ~]# tar zxvf stage1-xen-build.tar.gz -C /opt
> +
> +[root@localhost ~]# ls /opt
> +qemu-unstable  stage1-xen  xen-unstable  xen-unstable-runit
> +```
> +
> +This will extract all the build artifacts into `/opt` directory.

Is there a reason to keep all the binaries under /opt? I mean, at this
point, we could do something like

  cp -ar /opt/xen-unstable/* /

and do the same for the other components.

Do we keep them under /opt for ease of management, so that the next time
we do a build, we can easily test with a different Xen version? Or is
there another reason?


> +Next we will create a BIOS Boot Menu entry to boot `xen-4.10-unstable.efi`. This will start Xen hypervisor. Xen will then start Fedora as Dom-0 guest.
> +
> +On Fedora, EFI system partition (ESP) is usually mounted at `/boot/efi`. This is a `vfat` partition. You can check if EFI system partition is mounted as follows &ndash;
> +
> +```shell
> +[root@localhost ~]# mount | grep '\/boot\/efi'
> +/dev/sda1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
> +```
> +
> +Create a directory for Xen under `/boot/efi/EFI` and copy over `xen-4.10-unstable.efi`.
> +
> +```shell
> +[root@localhost ~]# mkdir -p /boot/efi/EFI/xen
> +[root@localhost ~]# cp /opt/xen-unstable/boot/efi/EFI/xen/xen-4.10-unstable.efi /boot/efi/EFI/xen/
> +```
> +
> +Inspect `/boot/efi/EFI/fedora/grub.cfg`. Under section `### BEGIN /etc/grub.d/10_linux ###` you will find `menuentry` for Fedora kernel and initrd. Look for `linuxefi` and `initrdefi`. Copy over the `vmlinuz` and `initramfs` files that you want to use for your Dom-0 into `/boot/efi/EFI/xen` directory.
> +
> +```shell
> +[root@localhost ~]# cp /boot/vmlinuz-A.B.C-D.fcXX.x86_64 /boot/efi/EFI/xen/
> +
> +[root@localhost ~]# cp /boot/initramfs-A.B.C-D.fcXX.x86_64.img /boot/efi/EFI/xen/
> +```
> +
> +Now in `/boot/efi/EFI/xen/` you should have the following files.
> +
> +```shell
> +[root@localhost ~]# ls /boot/efi/EFI/xen/
> +initramfs-A.B.C-D.fcXX.x86_64.img  vmlinuz-A.B.C-D.fcXX.x86_64  xen-4.10-unstable.efi
> +```
> +
> +Next create a file `xen-4.10-unstable.cfg` in `/boot/efi/EFI/xen/`. This is the [configuration file](https://xenbits.xen.org/docs/unstable/misc/efi.html) that Xen EFI loader will use to load Dom-0 kernel and initrd.
> +
> +Following are contents of `xen-4.10-unstable.cfg`
> +
> +```
> +[global]
> +default=fedora-A.B.C-D.fc25
> +
> +[fedora-A.B.C-D.fc25]
> +options=console=vga,com1 com1=115200,8n1 iommu=verbose ucode=scan flask=disabled conring_size=2097152 loglvl=all autoballoon=0 dom0_mem=4096M,max:4096M
> +kernel=vmlinuz-A.B.C-D.fc25.x86_64 root=UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ro rhgb console=hvc0 console=tty0
> +ramdisk=initramfs-A.B.C-D.fc25.x86_64.img
> +```
> +
> +You can find the boot parameters for `kernel=` from `linuxefi` entry in `/boot/efi/EFI/fedora/grub.cfg` Adjust `dom0_mem` appropriately leaving sufficient room for dom-U guests.
> +
> +We can now use `efibootmgr` to create a boot entry for Xen. If this the first time you are using `efibootmgr` please checkout the man pages by doing `man efibootmgr`.
> +
> +Use `efibootmgr -v` to list all the EFI boot entires.
> +
> +```shell
> +[root@localhost ~]# efibootmgr -v
> +BootCurrent: 0002
> +Timeout: 2 seconds
> +BootOrder: ...
> +
> +[...]
> +
> +Boot0001* Xen   HD(1,GPT,7d511991-1c25-4e33-900b-1d61d7752f19,0x800,0x82000)/File(\EFI\xen\xen-4.10-unstable.efi)
> +Boot0002* Fedora        HD(1,GPT,7d511991-1c25-4e33-900b-1d61d7752f19,0x800,0x82000)/File(\EFI\fedora\shim.efi)
> +
> +[...]
> +```
> +
> +In the above example there is already an entry for Xen with a boot number of `1`. Fedora is at boot number `2`. Your entires would look different. You won't have the Xen entry as yet! We are showing you an example where the Xen boot entry has already been created.
> +
> +Let us now create a boot entry for Xen. First we need to identify the disk and the partition number for EFI system partition. In most cases it is at `/dev/sda1`. You can identify this by doing &ndash;
> +
> +```shell
> +[root@localhost ~]# df /boot/efi
> +Filesystem     1K-blocks  Used Available Use% Mounted on
> +/dev/sda1         262128 63019    199109  25% /boot/efi
> +
> +[root@localhost ~]# sgdisk -p /dev/sda
> +Disk /dev/sda: 976773168 sectors, 465.8 GiB
> +Logical sector size: 512 bytes
> +
> +[...]
> +
> +Number  Start (sector)    End (sector)  Size       Code  Name
> +   1            2048          534527   260.0 MiB   EF00  EFI System Partition
> +```
> +
> +You can now create boot entry for Xen using the following command. Adjust `/dev/sda` and `-p 1`, according to where your EFI system partition is located.
> +
> +```shell
> +[root@localhost ~]# efibootmgr -c -w -L Xen -d /dev/sda -p 1 -l '\EFI\xen\xen-4.10-unstable.efi'
> +BootCurrent: ...
> +Timeout: 2 seconds
> +BootOrder: 0001,0002,0000,0010,0011,0012,0013,0017,0018,0019,001A,001B,001C
> +
> +[...]
> +
> +Boot0002* Fedora
> +
> +[...]
> +
> +Boot0001* Xen
> +```
> +
> +The output indicates that a boot entry for Xen is created with a boot number of `1`. 
> +
> +We will now show you how to delete an existing boot entry.
> +
> +**Note:** Be careful when deleting boot entires that you have not created. Do not delete Fedora or any entry unless you really know what you are doing. You have been warned!
> +
> +```shell
> +[root@localhost ~]# efibootmgr -b <boot_num> -B
> +
> +[root@localhost ~]# efibootmgr -b 1 -B
> +BootCurrent: ...
> +BootOrder: ...
> +
> +[...]
> +
> +Boot0002* Fedora
> +Boot0010  Setup
> +
> +[...]
> +```
> +
> +Once we have created a boot entry we can now boot into Xen. Restart machine and from the BIOS boot menu select **Xen**. You'll see Xen starting followed by Linux.
> +
> +After booting into Linux, you can see if have successfully booted Xen by checking out `dmesg`.
> +
> +```shell
> +[root@localhost ~]# dmesg | grep [Xx]en
> +[    0.000000] Xen: [mem 0x0000000000000000-0x0000000000057fff] usable
> +
> +[...]
> +
> +[    0.000000] Hypervisor detected: Xen
> +[    0.000000] Setting APIC routing to Xen PV.
> +[    0.000000] Booting paravirtualized kernel on Xen
> +[    0.000000] Xen version: 4.10-unstable (preserve-AD)
> +[    0.001000] Xen: using vcpuop timer interface
> +[    0.001000] installing Xen timer for CPU 0
> +```
> +
> +If you don't see Xen mentioned in your `dmesg`, then please check the previous steps.
> +
> +<a name="launching_xen_services"></a>
> +## Launching Xen services
> +
> +In Dom-0, we need to launch services required by Xen. If you followed the manual build, please make sure that xencommons init script has been started at boot.
> +
> +For container build you can use [`runit`](http://smarden.org/runit/) process supervisor. You can download and install `runit` RPMs for Fedora from [here](https://drive.google.com/open?id=0B_tTbuxmuRzIR05wQ3E1eWVyaGs).
> +
> +```shell
> +(ensure correct checksum on the downloaded binary)
> +[root@localhost ~]# echo "10cc62ffc040c49efa0dd85cbacd70c0712a7c10c58717a376610b786bc49d19  runit-2.1.2-1.1.fc25.tar" | sha256sum -c -
> +runit-2.1.2-1.1.fc25.tar: OK
> +
> +[root@localhost ~]# tar xvf runit-2.1.2-1.1.fc25.tar
> +
> +[root@localhost ~]# dnf install -y ./runit/2.1.2/1.1.fc25/x86_64/runit-2.1.2-1.1.fc25.x86_64.rpm
> +
> +[root@localhost ~]# pgrep -af runsvdir
> +1679 runsvdir -P -H /etc/service log: ..........................................................
> +```
> +
> +In `/opt/xen-unstable-runit` we provide two scripts to manage Xen services. 
> +
> + * `setup.sh`
> + * `teardown.sh`
> +
> +`setup.sh` is used to setup Xen services. If you are going to be running Fedora directly without Xen, please use `teardown.sh` prior to shutting down Domain-0. This will disable launching Xen services under Fedora without Xen.
> +
> +Run `setup.sh`
> +
> +```shell
> +[root@localhost ~]# /opt/xen-unstable-runit/setup.sh
> +Successfully created symlinks in /etc/service directory.
> +```
> +
> +You can verify Xen services are running correctly by doing the following &ndash;
> +
> +```shell
> +[root@localhost ~]# ls /etc/service | xargs -L 1 -I {} sv status {}
> +run: xenconsoled: (pid 29673) 115s
> +run: xen-init-dom0: (pid 29672) 115s
> +run: xen-init-dom0-disk-backend: (pid 29675) 115s
> +run: xenstored: (pid 29674) 115s
> +
> +[root@localhost ~]# source /opt/stage1-xen/bin/source_path.sh
> +
> +[root@localhost ~]# xl info
> +host                   : localhost.localdomain
> +release                : 4.11.12-200.fc25.x86_64
> +version                : #1 SMP Fri Jul 21 16:41:43 UTC 2017
> +machine                : x86_64
> +
> +[...]
> +
> +cc_compile_domain      : [unknown]
> +cc_compile_date        : Fri Aug 18 06:32:55 UTC 2017
> +build_id               : 4a65e1ae96407a8dd47f318db4bdf7d3
> +xend_config_format     : 4
> +
> +[root@localhost ~]# xl list
> +Name                                        ID   Mem VCPUs      State   Time(s)
> +Domain-0                                     0  4096     4     r-----     121.2
> +```
> +
> +<a name="setting_up_xen_networking"></a>
> +## Setting up Xen networking
> +
> +There are multiple ways to do networking on Xen. Two common configurations are [bridging](https://wiki.xenproject.org/wiki/Xen_Networking#Bridging) and [NAT](https://wiki.xenproject.org/wiki/Xen_Networking#Network_Address_Translation). Bridging is the default and most simple configuration to setup. However wireless device drivers are unable to do bridging. To overcome this limitation, we setup an internal  bridge and then use NAT to send packets externally. This setup works for both wired and wireless devices.
> +
> +```shell
> +[root@localhost ~]# ip link show
> +1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
> +    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> +2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
> +    link/ether c8:5b:76:71:40:c8 brd ff:ff:ff:ff:ff:f
> +3: wlp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000
> +    link/ether e4:a7:a0:93:9f:13 brd ff:ff:ff:ff:ff:f
> +```
> +
> +We have two devices `enp0s31f6` which is a wired ethernet device and `wlp4s0` which is a wireless ethernet device. We will use `wlp4s0` in the following example. However similar approach would also work for `enp0s31f6` device. You can also adjust the private network 10.1.1.0/24 to a non-overlapping private subnet. 
> +
> +```shell
> +[root@localhost ~]# brctl show
> +bridge name     bridge id               STP enabled     interfaces
> +
> +[root@localhost ~]# ip link add xenbr0 type bridge
> +
> +[root@localhost ~]# ip addr add 10.1.1.1/24 dev xenbr0
> +
> +[root@localhost ~]# ip link set xenbr0 up
> +
> +[root@localhost ~]# modprobe dummy
> +
> +[root@localhost ~]# ip link set dummy0 up
> +
> +[root@localhost ~]# brctl addif xenbr0 dummy0
> +
> +[root@localhost ~]# brctl show
> +bridge name     bridge id               STP enabled     interfaces
> +xenbr0          8000.d21b5c4113b7       no              dummy0
> +
> +[root@localhost ~]# iptables -I FORWARD -j ACCEPT
> +
> +[root@localhost ~]# iptables -t nat -I POSTROUTING --out-interface wlp4s0 -j MASQUERADE
> +
> +[root@localhost ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
> +```
> +
> +With this configuration we can launch Dom-U Xen guests using the following configuration setting.
> +
> +```
> +# Network configuration
> +vif = ['bridge=xenbr0']
> +```
> +
> +Then from within the guest, we will need to setup `eth0` interface with a static IP address in the range of 10.1.1.0/24 and gateway as 10.1.1.1.
> +
> +<a name="running_stage1-xen"></a>
> +## Running stage1-xen
> +
> +Once we have Xen setup, it is fairly straightforward to run stage1-xen.
> +
> +If you followed manual build, then please ensure that you have `xl` and `rkt` in your path.
> +
> +For container build, we provide a script to source all the required binaries from Xen, QEMU and rkt into our path.
> +
> +```shell
> +[root@localhost ~]# source /opt/stage1-xen/bin/source_path.sh
> +```
> +
> +We can now download images using `rkt` and run them under stage1-xenbits
> +
> +```shell
> +[root@localhost ~]# rkt --insecure-options=image fetch docker://alpine
> +Downloading sha256:88286f41530 [=============================] 1.99 MB / 1.99 MB
> +sha512-f84f971f8e01284f4ad0c3cf3efaa770
> +
> +[root@localhost ~]# rkt run sha512-f84f971f8e01284f4ad0c3cf3efaa770 \
> +                      --interactive --insecure-options=image \
> +                      --stage1-path=/opt/stage1-xen/aci/stage1-xen.aci
> +```
> +
> +Within the container, we can see we are running as a Xen PV guest, and using 9pfs
> +
> +```shell
> +/ # dmesg | grep [Xx]en
> +[    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
> +[    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
> +[    0.000000] Xen: [mem 0x0000000000100000-0x000000003fffffff] usable
> +[    0.000000] Hypervisor detected: Xen
> +[    0.000000] Booting paravirtualized kernel on Xen
> +[    0.000000] Xen version: 4.10-unstable (preserve-AD)
> +[    0.000000] xen:events: Using FIFO-based ABI
> +
> +[...]
> +
> +[    1.605990] Initialising Xen transport for 9pfs
> +```
> +
> +From Domain-0, we can run `rkt` and `xl` to get the details of the container.
> +
> +```shell
> +[root@localhost ~]# rkt list
> +UUID            APP     IMAGE NAME                                      STATE   CREATED       STARTED          NETWORKS
> +222083ec        alpine  registry-1.docker.io/library/alpine:latest      running 4 minutes ago 4 minutes ago    default:ip4=172.16.28.15
> +
> +[root@localhost ~]# xl list
> +Name                                        ID   Mem VCPUs      State   Time(s)
> +Domain-0                                     0  4093     4     r-----    1056.9
> +222083ec-d6da-4347-b261-0a733bae6802         1  1024     2     -b----       2.2
> +
> +[root@localhost ~]# rkt stop 222083ec
> +"222083ec-d6da-4347-b261-0a733bae6802"
> +
> +[root@localhost ~]# xl list
> +Name                                        ID   Mem VCPUs      State   Time(s)
> +Domain-0                                     0  4093     4     r-----    1058.5
> +```
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Rajiv Ranganath]

On Thu, Sep 07 2017 at 12:10:21 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:

[...]

> The series is much better now thank you. One question: why did you write
> your own init scripts rather than reusing xencommons (with the caveat
> that you would have to add make sure to source_path.sh before running
> xencommons)?  Does it have something to do with systemd?

There are a few related reasons for this.

1. Using runit lets us abstract out our dependency on systemd and
upstart. We can use the same abstraction in containers [1], virtual
machines and on bare metal.

2. In Linux distributions, there is tight coupling between package
management system (rpm/deb), init systems (upstart/systemd), and service
daemons.

With containers, if the expectation is that most service daemons and
apps would be containerized, and managed by a node agent then a natural
question to ask would be what should be the role of init systems like
systemd?

By using runit (on systemd, upstart and within containers), we defer
answering this question. 

3. One of the use cases that we want to support is to have different
versions of xen co-exist on the same filesystem. Then a higher level
tool can do rolling updates and if required rollbacks.

While it is possible to accomplish this on existing init systems,
depending on how xen is packaged and deployed, it might involve using
distro package and repository management tools.

With runit, we can use regular docker tools, which is much more friendly
for mainstream developers and CI systems. We also abstract over init
systems, which is a desirable property to have.

4. I looked into xencommons script and systemd unit files when creating
runit scripts. Our runit scripts is straightforward translation of how
one would start xen manually.

Perhaps the only part of the script that might need some explanation is
in `xen-init-dom0/run`.

```
exec chpst -b xen-init-dom0 runit-pause
```

This is a pattern used to build equivalent of "oneshot" service in
systemd. It was developed in Ignite (a Arch Linux project before they
switched to systemd) and later co-opted by Void Linux [2].

I am not sure if I answered your question. Sometimes I feel, maybe we
should just let questions around init systems be like one of those
"unanswered questions" in theology. :-) [3]

Best,
Rajiv

[1] https://github.com/lambda-linux/baseimage-amzn#adding_additional_daemons

[2] https://github.com/voidlinux/void-runit/commit/7aecf46ec589a5bc49ae2392137bcd0e7468dd08

[3] https://en.wikipedia.org/wiki/The_unanswered_questions

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Rajiv Ranganath]

On Thu, Sep 07 2017 at 12:29:54 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:

[...]

>> +QEMU_BRANCH = 'master'
>
> I am not sure we want to checkout always the latest QEMU. It is a
> running target. It makes sense to use one of the latest releases
> instead, such as v2.10.0?
>

[...]

I feel once we have an understanding around what stable xen container
experience for our users should be, it makes a lot of sense to support
two stable versions (on a rolling basis) along with unstable/devel
versions of xen, qemu and rkt.

I am hoping we can include the following before adding support for
stable version.

1. Kernel - PV Calls backend support will be in 4.14, which is few
months away.

2. PVHv2 - xl and PVHv2 support is inflight for 4.10. I would like to
see xen container users start off with PVHv2 and using PV Calls
networking. Therefore I am a bit hesitant adding support for Xen 4.9.

3. Multiboot2 - One of the reasons why I documented using EFI is because
I could not get multiboot2 to work. It looks like the fix for it is on
its way. I anticipate using multiboot2 would be easier for users.

4. Rkt - Support for Kubernetes CRI and OCI image format will be of
importance to our users. Rkt is working on it but I'm not sure of their
progress. There are other projects that are also incubating in CNCF -
cri-o and cri-containerd.

PV Calls networking is new to me, and I wanted to do some prototyping to
understand how it would integrate with the rest of the container
ecosystem it after landing this series.

By adding support for xen-4.9, qemu-2.10 or rkt-1.28.1 I feel we should
not set some kind stability or backward compatibility expectations
around stage1-xen as yet.

My preference would be to keep things on master (albeit deliberately)
till we can figure out a good xen container experience for our users.

Please let me know what you think.

>> +    if p.returncode != 0:
>> +        sys.exit(1)
>
> Is this the same as
>   #!/bin/bash
>   set -e
> ?

That's right. 

> Please add a few words in the commit message about the benefit of this
> approach of writing scripts.
>

I'll update the commit message in the next version of the series.

Best,
Rajiv


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 09/10] build/fedora: Add `RUNNING_STAGE1_XEN.md`

[Rajiv Ranganath]

On Thu, Sep 07 2017 at 12:44:16 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:

[...]

>> +[root@localhost ~]# ls /opt
>> +qemu-unstable  stage1-xen  xen-unstable  xen-unstable-runit
>> +```
>> +
>> +This will extract all the build artifacts into `/opt` directory.
>
> Is there a reason to keep all the binaries under /opt? I mean, at this
> point, we could do something like
>
>   cp -ar /opt/xen-unstable/* /
>
> and do the same for the other components.

Yes, we can do that, but I feel its a good idea. :-)

Outside of specific paths (such as /var or /etc), its better to let RPM
manage files in the / hierarchy. That way rpm -qf can return sensible
results when we need to login and debug issues.

> Do we keep them under /opt for ease of management, so that the next time
> we do a build, we can easily test with a different Xen version? Or is
> there another reason?

That's correct. Keeping things isolated in /opt lets us test different
versions of xen during development. In production we can use the same
approach to support multiple versions of xen and do rolling updates or
rollbacks.

Btw, I should point out that this is not something new. NixOS has been
using the approach of building packages in separate filesystem hierarchy
for a while now.

We are just selectively adopting their ideas.

Best,
Rajiv

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 09/10] build/fedora: Add `RUNNING_STAGE1_XEN.md`

[Rajiv Ranganath]

On Sat, Sep 09 2017 at 07:40:17 AM, Rajiv Ranganath <rajiv.ranganath@atihita.com> wrote:

[...]

>>
>> and do the same for the other components.
>
> Yes, we can do that, but I feel its a good idea. :-)
                             ^^^^^^^^^^^^^^^^^^^^

I meant to say _not_ a good idea! Sorry for the typo!

Best,
Rajiv

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Stefano Stabellini]

On Sat, 9 Sep 2017, Rajiv Ranganath wrote:
> On Thu, Sep 07 2017 at 12:29:54 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> [...]
> 
> >> +QEMU_BRANCH = 'master'
> >
> > I am not sure we want to checkout always the latest QEMU. It is a
> > running target. It makes sense to use one of the latest releases
> > instead, such as v2.10.0?
> >
> 
> [...]
> 
> I feel once we have an understanding around what stable xen container
> experience for our users should be, it makes a lot of sense to support
> two stable versions (on a rolling basis) along with unstable/devel
> versions of xen, qemu and rkt.

Yes, I think that would be ideal too.


> I am hoping we can include the following before adding support for
> stable version.
> 
> 1. Kernel - PV Calls backend support will be in 4.14, which is few
> months away.
> 
> 2. PVHv2 - xl and PVHv2 support is inflight for 4.10. I would like to
> see xen container users start off with PVHv2 and using PV Calls
> networking. Therefore I am a bit hesitant adding support for Xen 4.9.

Yes, that would fantastic. Fortunately, from the stage1-xen code point
of view, there is very little difference between PVHv2 and PV. Switching
from one to the other should be a matter of adding one line to the xl
config file.

Regarding statements of support, see below.


> 3. Multiboot2 - One of the reasons why I documented using EFI is because
> I could not get multiboot2 to work. It looks like the fix for it is on
> its way. I anticipate using multiboot2 would be easier for users.

That's for the host right? I didn't have that problem, but maybe because
I am not using Fedora.


> 4. Rkt - Support for Kubernetes CRI and OCI image format will be of
> importance to our users. Rkt is working on it but I'm not sure of their
> progress. There are other projects that are also incubating in CNCF -
> cri-o and cri-containerd.
> 
> PV Calls networking is new to me, and I wanted to do some prototyping to
> understand how it would integrate with the rest of the container
> ecosystem it after landing this series.
> 
> By adding support for xen-4.9, qemu-2.10 or rkt-1.28.1 I feel we should
> not set some kind stability or backward compatibility expectations
> around stage1-xen as yet.

I agree we should not set any kind of backward compatibility
expectations yet. See below.


> My preference would be to keep things on master (albeit deliberately)
> till we can figure out a good xen container experience for our users.
> 
> Please let me know what you think.

You have a good point. I think we should be clear about the stability of
the project and the backward compatibility in the README. We should
openly say that it is still a "preview" and there is no "support" or
"compatibility" yet.

Choosing Xen 4.9 should not be seen as a statement of support. I think
we should choose the Xen version based only on the technical merits.

In the long term it would be great to support multiple stable versions
and a development version of Xen. As of now, I think it makes sense to
have an "add-hoc approach": I would use Xen 4.9 just because it is the
best choice at the moment. Then, I would update to other versions when
it makes sense, manually. I don't think that building against a changing
target ("master") is a good idea, because we might end up stumbling
across confusing and time-consuming bugs that have nothing to do with
stage1-xen. However, we could pick a random commit on the Xen tree if
that's convenient for us, because at this stage there is no support
really. For example, PVCalls will require some tools changes in Xen.
Once they are upstream, we'll want to update the Xen version to the
latest with PVCalls support.

Does it make sense?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 09/10] build/fedora: Add `RUNNING_STAGE1_XEN.md`

[Stefano Stabellini]

On Sat, 9 Sep 2017, Rajiv Ranganath wrote:
> On Thu, Sep 07 2017 at 12:44:16 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> [...]
> 
> >> +[root@localhost ~]# ls /opt
> >> +qemu-unstable  stage1-xen  xen-unstable  xen-unstable-runit
> >> +```
> >> +
> >> +This will extract all the build artifacts into `/opt` directory.
> >
> > Is there a reason to keep all the binaries under /opt? I mean, at this
> > point, we could do something like
> >
> >   cp -ar /opt/xen-unstable/* /
> >
> > and do the same for the other components.
> 
> Yes, we can do that, but I feel its a good idea. :-)
> 
> Outside of specific paths (such as /var or /etc), its better to let RPM
> manage files in the / hierarchy. That way rpm -qf can return sensible
> results when we need to login and debug issues.
> 
> > Do we keep them under /opt for ease of management, so that the next time
> > we do a build, we can easily test with a different Xen version? Or is
> > there another reason?
> 
> That's correct. Keeping things isolated in /opt lets us test different
> versions of xen during development. In production we can use the same
> approach to support multiple versions of xen and do rolling updates or
> rollbacks.
> 
> Btw, I should point out that this is not something new. NixOS has been
> using the approach of building packages in separate filesystem hierarchy
> for a while now.
> 
> We are just selectively adopting their ideas.

That's OK for me. Please add a few words on this in the commit message.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Stefano Stabellini]

On Sat, 9 Sep 2017, Rajiv Ranganath wrote:
> On Thu, Sep 07 2017 at 12:10:21 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> [...]
> 
> > The series is much better now thank you. One question: why did you write
> > your own init scripts rather than reusing xencommons (with the caveat
> > that you would have to add make sure to source_path.sh before running
> > xencommons)?  Does it have something to do with systemd?
> 
> There are a few related reasons for this.
> 
> 1. Using runit lets us abstract out our dependency on systemd and
> upstart. We can use the same abstraction in containers [1], virtual
> machines and on bare metal.
> 
> 2. In Linux distributions, there is tight coupling between package
> management system (rpm/deb), init systems (upstart/systemd), and service
> daemons.
> 
> With containers, if the expectation is that most service daemons and
> apps would be containerized, and managed by a node agent then a natural
> question to ask would be what should be the role of init systems like
> systemd?
> 
> By using runit (on systemd, upstart and within containers), we defer
> answering this question. 
> 
> 3. One of the use cases that we want to support is to have different
> versions of xen co-exist on the same filesystem. Then a higher level
> tool can do rolling updates and if required rollbacks.
> 
> While it is possible to accomplish this on existing init systems,
> depending on how xen is packaged and deployed, it might involve using
> distro package and repository management tools.
> 
> With runit, we can use regular docker tools, which is much more friendly
> for mainstream developers and CI systems. We also abstract over init
> systems, which is a desirable property to have.
> 
> 4. I looked into xencommons script and systemd unit files when creating
> runit scripts. Our runit scripts is straightforward translation of how
> one would start xen manually.
> 
> Perhaps the only part of the script that might need some explanation is
> in `xen-init-dom0/run`.
> 
> ```
> exec chpst -b xen-init-dom0 runit-pause
> ```
> 
> This is a pattern used to build equivalent of "oneshot" service in
> systemd. It was developed in Ignite (a Arch Linux project before they
> switched to systemd) and later co-opted by Void Linux [2].
> 
> I am not sure if I answered your question. Sometimes I feel, maybe we
> should just let questions around init systems be like one of those
> "unanswered questions" in theology. :-) [3]
> 
> Best,
> Rajiv
> 
> [1] https://github.com/lambda-linux/baseimage-amzn#adding_additional_daemons
> 
> [2] https://github.com/voidlinux/void-runit/commit/7aecf46ec589a5bc49ae2392137bcd0e7468dd08

Thank you for the pointers. I have no opinions on the format of the init
scripts. runit is fine by me in that respect, and I understand the
advantages you pointed out.

My only concern is about diverging from the upstream Xen codebase. I
think the runit scripts should call xencommons underneath. If xencommons
cannot cope with being called from runit, we could make changes to
xencommon in xen.git to make it so.

Otherwise, we will end up in a situation such as:
- xen.git changes xencommons
- we don't notice
- we upgrade Xen version
- stage1-xen doesn't work anymore

If we used xencommons underneath we would avoid this, and it looks like
xencommons could be made to work well with runit.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Andrew Cooper]

On 11/09/2017 21:20, Stefano Stabellini wrote:
> On Sat, 9 Sep 2017, Rajiv Ranganath wrote:
>> On Thu, Sep 07 2017 at 12:10:21 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
>>
>> [...]
>>
>>> The series is much better now thank you. One question: why did you write
>>> your own init scripts rather than reusing xencommons (with the caveat
>>> that you would have to add make sure to source_path.sh before running
>>> xencommons)?  Does it have something to do with systemd?
>> There are a few related reasons for this.
>>
>> 1. Using runit lets us abstract out our dependency on systemd and
>> upstart. We can use the same abstraction in containers [1], virtual
>> machines and on bare metal.
>>
>> 2. In Linux distributions, there is tight coupling between package
>> management system (rpm/deb), init systems (upstart/systemd), and service
>> daemons.
>>
>> With containers, if the expectation is that most service daemons and
>> apps would be containerized, and managed by a node agent then a natural
>> question to ask would be what should be the role of init systems like
>> systemd?
>>
>> By using runit (on systemd, upstart and within containers), we defer
>> answering this question. 
>>
>> 3. One of the use cases that we want to support is to have different
>> versions of xen co-exist on the same filesystem. Then a higher level
>> tool can do rolling updates and if required rollbacks.
>>
>> While it is possible to accomplish this on existing init systems,
>> depending on how xen is packaged and deployed, it might involve using
>> distro package and repository management tools.
>>
>> With runit, we can use regular docker tools, which is much more friendly
>> for mainstream developers and CI systems. We also abstract over init
>> systems, which is a desirable property to have.
>>
>> 4. I looked into xencommons script and systemd unit files when creating
>> runit scripts. Our runit scripts is straightforward translation of how
>> one would start xen manually.
>>
>> Perhaps the only part of the script that might need some explanation is
>> in `xen-init-dom0/run`.
>>
>> ```
>> exec chpst -b xen-init-dom0 runit-pause
>> ```
>>
>> This is a pattern used to build equivalent of "oneshot" service in
>> systemd. It was developed in Ignite (a Arch Linux project before they
>> switched to systemd) and later co-opted by Void Linux [2].
>>
>> I am not sure if I answered your question. Sometimes I feel, maybe we
>> should just let questions around init systems be like one of those
>> "unanswered questions" in theology. :-) [3]
>>
>> Best,
>> Rajiv
>>
>> [1] https://github.com/lambda-linux/baseimage-amzn#adding_additional_daemons
>>
>> [2] https://github.com/voidlinux/void-runit/commit/7aecf46ec589a5bc49ae2392137bcd0e7468dd08
> Thank you for the pointers. I have no opinions on the format of the init
> scripts. runit is fine by me in that respect, and I understand the
> advantages you pointed out.
>
> My only concern is about diverging from the upstream Xen codebase. I
> think the runit scripts should call xencommons underneath. If xencommons
> cannot cope with being called from runit, we could make changes to
> xencommon in xen.git to make it so.
>
> Otherwise, we will end up in a situation such as:
> - xen.git changes xencommons
> - we don't notice
> - we upgrade Xen version
> - stage1-xen doesn't work anymore
>
> If we used xencommons underneath we would avoid this, and it looks like
> xencommons could be made to work well with runit.

If possible, upstream Xen should be made to be compatible with runit
(this would be the ideal case).  If not, upstream Xen should contain
different styles of these files, which are selected between by a
./configure option (this is suboptimal, but better than locally
forking).  This offers the greatest chance that updates to one don't
cause the other to be stale.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 06/10] build/fedora: Add `xen-unstable-runit/*` scripts

[Rajiv Ranganath]

On Tue, Sep 12 2017 at 04:38:19 AM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 11/09/2017 21:20, Stefano Stabellini wrote:

[...]

>> My only concern is about diverging from the upstream Xen codebase. I
>> think the runit scripts should call xencommons underneath. If xencommons
>> cannot cope with being called from runit, we could make changes to
>> xencommon in xen.git to make it so.
>>
>> Otherwise, we will end up in a situation such as:
>> - xen.git changes xencommons
>> - we don't notice
>> - we upgrade Xen version
>> - stage1-xen doesn't work anymore
>>
>> If we used xencommons underneath we would avoid this, and it looks like
>> xencommons could be made to work well with runit.
>
> If possible, upstream Xen should be made to be compatible with runit
> (this would be the ideal case). If not, upstream Xen should contain
> different styles of these files, which are selected between by a
> ./configure option (this is suboptimal, but better than locally
> forking). This offers the greatest chance that updates to one don't
> cause the other to be stale.

I agree that it would be beneficial to have upstream Xen support for
runit. However, runit is packaged differently in every distro.

We work around this issue by packaging our own version of runit [1].
Fedora does not include runit in its repositories. That helps because we
don't have to worry about conflicting with distro packaged runit.

One option to consider is for xen project to package its own version of
runit for major distros (we will have one for Fedora in stage1-xen), and
use that as the basis for runit support.

Since stage1-xen is still under development, maybe we can use runit in
stage1-xen as a testing ground. If things work out well, we can then see
how best to integrate with xencommons or add a configure option.

By then we will also know if there is broader community interest in
having runit support in xen.

As to changes to xencommons breaking stage1-xen, as we get closer to
stable release, we probably will have integration tests to catch this
and many other things! :-)

Best,
Rajiv

[1]: https://github.com/lambda-linux-fedora/runit/tree/ver-2.1.2-1.1.fc25

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Rajiv Ranganath]

On Tue, Sep 12 2017 at 01:36:04 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:

[...]

> Fortunately, from the stage1-xen code point of view, there is very
> little difference between PVHv2 and PV. Switching from one to the
> other should be a matter of adding one line to the xl config file.

There is a related use-case here that I think will be important to
users.

In stage1-xen we are packaging a Dom-U kernel. When this kernel crashes
we would want to capture its crash log. Depending on the nature of the
issue, users can then work with their own kernel team, vendor (who is
open to supporting LTS kernels) or upstream.

We might also want to consider supporting two LTS kernel versions on a
rolling basis. Users can then use something like labels [1] or
annotations [2] to toggle the kernel version. That way if their
containers start crashing under a newer Dom-U kernel, they can roll back
to a working kernel.

[...]

>> 3. Multiboot2 - One of the reasons why I documented using EFI is because
>> I could not get multiboot2 to work. It looks like the fix for it is on
>> its way. I anticipate using multiboot2 would be easier for users.
>
> That's for the host right? I didn't have that problem, but maybe because
> I am not using Fedora.

That's correct! I ran into this issue on Fedora host.

[...]

> You have a good point. I think we should be clear about the stability
> of the project and the backward compatibility in the README. We should
> openly say that it is still a "preview" and there is no "support" or
> "compatibility" yet.

Sounds good. I'll update README to reflect this.

> Choosing Xen 4.9 should not be seen as a statement of support. I think
> we should choose the Xen version based only on the technical merits.
>
> In the long term it would be great to support multiple stable versions
> and a development version of Xen. As of now, I think it makes sense to
> have an "add-hoc approach": I would use Xen 4.9 just because it is the
> best choice at the moment. Then, I would update to other versions when
> it makes sense, manually. I don't think that building against a changing
> target ("master") is a good idea, because we might end up stumbling
> across confusing and time-consuming bugs that have nothing to do with
> stage1-xen. However, we could pick a random commit on the Xen tree if
> that's convenient for us, because at this stage there is no support
> really. For example, PVCalls will require some tools changes in Xen.
> Once they are upstream, we'll want to update the Xen version to the
> latest with PVCalls support.
>
> Does it make sense?

Yes, it does. I'll switch to xen-4.9, qemu-2.10 and rkt-1.28 in the next
version of the patchset.

Best,
Rajiv

[1] https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
[2] https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1 04/10] build/fedora: Add `run` and `components/*` scripts

[Stefano Stabellini]

On Wed, 13 Sep 2017, Rajiv Ranganath wrote:
> On Tue, Sep 12 2017 at 01:36:04 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> [...]
> 
> > Fortunately, from the stage1-xen code point of view, there is very
> > little difference between PVHv2 and PV. Switching from one to the
> > other should be a matter of adding one line to the xl config file.
> 
> There is a related use-case here that I think will be important to
> users.
> 
> In stage1-xen we are packaging a Dom-U kernel. When this kernel crashes
> we would want to capture its crash log. Depending on the nature of the
> issue, users can then work with their own kernel team, vendor (who is
> open to supporting LTS kernels) or upstream.
> 
> We might also want to consider supporting two LTS kernel versions on a
> rolling basis. Users can then use something like labels [1] or
> annotations [2] to toggle the kernel version. That way if their
> containers start crashing under a newer Dom-U kernel, they can roll back
> to a working kernel.
> 
> [...]

Yes, I agree. I think it makes sense to allow users to change the DomU
kernel version, at least at build time. At runtime it would be useful
too. One day we might even support kernels other than Linux.



> > You have a good point. I think we should be clear about the stability
> > of the project and the backward compatibility in the README. We should
> > openly say that it is still a "preview" and there is no "support" or
> > "compatibility" yet.
> 
> Sounds good. I'll update README to reflect this.
>
> > Choosing Xen 4.9 should not be seen as a statement of support. I think
> > we should choose the Xen version based only on the technical merits.
> >
> > In the long term it would be great to support multiple stable versions
> > and a development version of Xen. As of now, I think it makes sense to
> > have an "add-hoc approach": I would use Xen 4.9 just because it is the
> > best choice at the moment. Then, I would update to other versions when
> > it makes sense, manually. I don't think that building against a changing
> > target ("master") is a good idea, because we might end up stumbling
> > across confusing and time-consuming bugs that have nothing to do with
> > stage1-xen. However, we could pick a random commit on the Xen tree if
> > that's convenient for us, because at this stage there is no support
> > really. For example, PVCalls will require some tools changes in Xen.
> > Once they are upstream, we'll want to update the Xen version to the
> > latest with PVCalls support.
> >
> > Does it make sense?
> 
> Yes, it does. I'll switch to xen-4.9, qemu-2.10 and rkt-1.28 in the next
> version of the patchset.

Thanks!


> Best,
> Rajiv
> 
> [1] https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
> [2] https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel