[PATCH 0/3] Automation: improve openSUSE containers + podman

[PATCH 0/3] Automation: improve openSUSE containers + podman

[Dario Faggioli]

Hello,

This short series contains some improvements for building Xen in
openSUSE containers. In fact, the build dependencies inside the
Tumbleweed container are updated and more handy helpers are added, in
containerize, for referring to both Leap and Tumbleweed containers.

In addition to that, in patch 3, the containerize script is enhanced so
that it is now possible to use podman, instead of docker. Rootless mode
for podman also works (provided the system is properly configured) which,
IMO, is rather nice.

Docker of course continue to work, and is kept as the default.

Regards
---
Dario Faggioli (3):
      automation: update openSUSE Tumbleweed building dependencies
      automation: openSUSE distro names helpers for containerize.
      automation: implement (rootless) podman support in containerize

 automation/build/README.md                         |   10 ++++++++++
 .../build/suse/opensuse-tumbleweed.dockerfile      |    2 ++
 automation/scripts/containerize                    |   19 +++++++++++++++----
 3 files changed, 27 insertions(+), 4 deletions(-)
--
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

[PATCH 1/3] automation: update openSUSE Tumbleweed building dependencies

[Dario Faggioli]

We need python3 (and the respective -devel package), these days.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
This patch was submitted already, but not as part of this series.

Anyway, changes from v1:
* add python3 instead of replacing python2 with it.
---
I think the tumbleweed image in our registry needs to be updated.
---
 .../build/suse/opensuse-tumbleweed.dockerfile      |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/build/suse/opensuse-tumbleweed.dockerfile b/automation/build/suse/opensuse-tumbleweed.dockerfile
index 2676a87c85..084cce0921 100644
--- a/automation/build/suse/opensuse-tumbleweed.dockerfile
+++ b/automation/build/suse/opensuse-tumbleweed.dockerfile
@@ -56,6 +56,8 @@ RUN zypper install -y --no-recommends \
         pkg-config \
         python \
         python-devel \
+        python3 \
+        python3-devel \
         systemd-devel \
         tar \
         transfig \

[PATCH 2/3] automation: openSUSE distro names helpers for containerize.

[Dario Faggioli]

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
---
 automation/scripts/containerize |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index fbc4bc22d6..eb805bf96c 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -24,6 +24,8 @@ case "_${CONTAINER}" in
     _stretch|_) CONTAINER="${BASE}/debian:stretch" ;;
     _trusty) CONTAINER="${BASE}/ubuntu:trusty" ;;
     _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;;
+    _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;;
+    _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/suse:opensuse-tumbleweed" ;;
 esac
 
 # Use this variable to control whether root should be used

[PATCH 3/3] automation: implement (rootless) podman support in containerize

[Dario Faggioli]

Right now only docker is supported, when using the containerize script
for building inside containers. Enable podman as well.

Note that podman can be use in rootless mode too, but for that to work
the files /etc/subuid and /etc/subgid must be properly configured.

For instance:

dario@localhost> cat /etc/subuid
dario:100000:65536

dario@localhost:> cat /etc/subgid
dario:100000:65536

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
---
 automation/build/README.md      |   10 ++++++++++
 automation/scripts/containerize |   17 +++++++++++++----
 2 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/automation/build/README.md b/automation/build/README.md
index 8cda2b65a5..e1fb3124de 100644
--- a/automation/build/README.md
+++ b/automation/build/README.md
@@ -34,6 +34,16 @@ the default shell.
 There are several environment variables which the containerize script
 understands.
 
+- DOCKED_CMD: Whether to use docker or podman for running the containers.
+  podman can be used as a regular user (rootless podman), but for that
+  to work, /etc/subuid and /etc/subgid needs to containe the proper
+  entries, for such user.
+  docker is the default, for running with podman, do:
+
+  ```
+  DOCKER_CMD=podman ./automation/scripts/containerize make
+  ```
+
 - CONTAINER: This overrides the container to use. For CentOS 7.2, use:
 
   ```
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index eb805bf96c..04b9fc7ba4 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -1,5 +1,14 @@
 #!/bin/bash
 
+#
+# DOCKER_CMD should be either `docker` or `podman`.
+#
+# if using (rootless) podman, remember to set /etc/subuid
+# and /etc/subgid.
+#
+docker_cmd=${DOCKER_CMD:-"docker"}
+[ "$DOCKER_CMD" = "podman" ] && userns_podman="--userns=keep-id"
+
 einfo() {
     echo "$*" >&2
 }
@@ -31,7 +40,7 @@ esac
 # Use this variable to control whether root should be used
 case "_${CONTAINER_UID0}" in
     _1)   userarg= ;;
-    _0|_) userarg="-u $(id -u)" ;;
+    _0|_) userarg="-u $(id -u) $userns_podman" ;;
 esac
 
 # Save the commands for future use
@@ -49,8 +58,8 @@ tty -s && termint=t
 #
 if [[ "_${CONTAINER_NO_PULL}" != "_1" ]]; then
     einfo "*** Ensuring ${CONTAINER} is up to date"
-    docker pull ${CONTAINER} > /dev/null ||     \
-        die "Failed to update docker container"
+    ${docker_cmd} pull ${CONTAINER} > /dev/null ||     \
+        die "Failed to update container"
 fi
 
 if hash greadlink > /dev/null 2>&1; then
@@ -82,7 +91,7 @@ fi
 
 # Kick off Docker
 einfo "*** Launching container ..."
-exec docker run \
+exec ${docker_cmd} run \
     ${userarg} \
     ${SSH_AUTH_SOCK:+-e SSH_AUTH_SOCK="/tmp/ssh-agent/${SSH_AUTH_NAME}"} \
     -v "${CONTAINER_PATH}":/build:rw \

Re: [PATCH 0/3] Automation: improve openSUSE containers + podman

[Dario Faggioli]

[-- Attachment #1: Type: text/plain, Size: 990 bytes --]

On Thu, 2020-04-30 at 20:27 +0200, Dario Faggioli wrote:
> Hello,
> 
> This short series contains some improvements for building Xen in
> openSUSE containers. In fact, the build dependencies inside the
> Tumbleweed container are updated and more handy helpers are added, in
> containerize, for referring to both Leap and Tumbleweed containers.
> 
> In addition to that, in patch 3, the containerize script is enhanced
> so
> that it is now possible to use podman, instead of docker. Rootless
> mode
> for podman also works (provided the system is properly configured)
> which,
> IMO, is rather nice.
> 
> Docker of course continue to work, and is kept as the default.
> 
Ping?

Thanks and Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 0/3] Automation: improve openSUSE containers + podman

[Dario Faggioli]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

On Thu, 2020-05-21 at 09:43 +0200, Dario Faggioli wrote:
> On Thu, 2020-04-30 at 20:27 +0200, Dario Faggioli wrote:
> > Hello,
> > 
> > This short series contains some improvements for building Xen in
> > openSUSE containers. In fact, the build dependencies inside the
> > Tumbleweed container are updated and more handy helpers are added,
> > in
> > containerize, for referring to both Leap and Tumbleweed containers.
> > 
> > In addition to that, in patch 3, the containerize script is
> > enhanced
> > so
> > that it is now possible to use podman, instead of docker. Rootless
> > mode
> > for podman also works (provided the system is properly configured)
> > which,
> > IMO, is rather nice.
> > 
> > Docker of course continue to work, and is kept as the default.
> > 
> Ping?
>
Ping^2? :-D

Adding Wei. get-maintainers.pl seems told me I should no Cc him, so I
dind't, but I've seen automation/ stuff Acked-by him recently, so...

Thanks and Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 0/3] Automation: improve openSUSE containers + podman

[Wei Liu]

On Fri, May 29, 2020 at 12:20:25PM +0200, Dario Faggioli wrote:
> On Thu, 2020-05-21 at 09:43 +0200, Dario Faggioli wrote:
> > On Thu, 2020-04-30 at 20:27 +0200, Dario Faggioli wrote:
> > > Hello,
> > > 
> > > This short series contains some improvements for building Xen in
> > > openSUSE containers. In fact, the build dependencies inside the
> > > Tumbleweed container are updated and more handy helpers are added,
> > > in
> > > containerize, for referring to both Leap and Tumbleweed containers.
> > > 
> > > In addition to that, in patch 3, the containerize script is
> > > enhanced
> > > so
> > > that it is now possible to use podman, instead of docker. Rootless
> > > mode
> > > for podman also works (provided the system is properly configured)
> > > which,
> > > IMO, is rather nice.
> > > 
> > > Docker of course continue to work, and is kept as the default.
> > > 
> > Ping?
> >
> Ping^2? :-D
> 
> Adding Wei. get-maintainers.pl seems told me I should no Cc him, so I
> dind't, but I've seen automation/ stuff Acked-by him recently, so...

I think these are good improvements, so in Doug's absence:

Acked-by: Wei Liu <wl@xen.org>

You can already push to the container registries right?

Cc Paul. Gitlab CI is not gating pushes. I think there is very low risk
involved in committing this series during freeze.


> 
> Thanks and Regards
> -- 
> Dario Faggioli, Ph.D
> http://about.me/dario.faggioli
> Virtualization Software Engineer
> SUSE Labs, SUSE https://www.suse.com/
> -------------------------------------------------------------------
> <<This happens because _I_ choose it to happen!>> (Raistlin Majere)
> 

RE: [PATCH 0/3] Automation: improve openSUSE containers + podman

[Paul Durrant]

> -----Original Message-----
> From: Wei Liu <wl@xen.org>
> Sent: 02 June 2020 10:07
> To: Dario Faggioli <dfaggioli@suse.com>
> Cc: xen-devel@lists.xenproject.org; Andrew Cooper <andrew.cooper3@citrix.com>; Doug Goldstein
> <cardoe@cardoe.com>; Wei Liu <wl@xen.org>; Paul Durrant <paul@xen.org>
> Subject: Re: [PATCH 0/3] Automation: improve openSUSE containers + podman
> 
> On Fri, May 29, 2020 at 12:20:25PM +0200, Dario Faggioli wrote:
> > On Thu, 2020-05-21 at 09:43 +0200, Dario Faggioli wrote:
> > > On Thu, 2020-04-30 at 20:27 +0200, Dario Faggioli wrote:
> > > > Hello,
> > > >
> > > > This short series contains some improvements for building Xen in
> > > > openSUSE containers. In fact, the build dependencies inside the
> > > > Tumbleweed container are updated and more handy helpers are added,
> > > > in
> > > > containerize, for referring to both Leap and Tumbleweed containers.
> > > >
> > > > In addition to that, in patch 3, the containerize script is
> > > > enhanced
> > > > so
> > > > that it is now possible to use podman, instead of docker. Rootless
> > > > mode
> > > > for podman also works (provided the system is properly configured)
> > > > which,
> > > > IMO, is rather nice.
> > > >
> > > > Docker of course continue to work, and is kept as the default.
> > > >
> > > Ping?
> > >
> > Ping^2? :-D
> >
> > Adding Wei. get-maintainers.pl seems told me I should no Cc him, so I
> > dind't, but I've seen automation/ stuff Acked-by him recently, so...
> 
> I think these are good improvements, so in Doug's absence:
> 
> Acked-by: Wei Liu <wl@xen.org>
> 
> You can already push to the container registries right?
> 
> Cc Paul. Gitlab CI is not gating pushes. I think there is very low risk
> involved in committing this series during freeze.
> 

I'll trust your judegment :-)

Release-acked-by: Paul Durrant <paul@xen.org>

> 
> >
> > Thanks and Regards
> > --
> > Dario Faggioli, Ph.D
> > http://about.me/dario.faggioli
> > Virtualization Software Engineer
> > SUSE Labs, SUSE https://www.suse.com/
> > -------------------------------------------------------------------
> > <<This happens because _I_ choose it to happen!>> (Raistlin Majere)
> >
>