[PATCH 0/7] testing/next: docker.py removal and kaniko updates

[PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Alex Bennée]

This series attempts to remove our dependence on the docker.py script
and build things directly with the appropriate tool. I've been
noodling around with how we build images on gitlab to see if they can
cache better because the normal case should be we don't need to
rebuild everything if the upstream distro hasn't updated its package
list.

Anyway what do people think?

Alex Bennée (7):
  configure: expose the direct container command
  tests/dockerfiles: unify debian-toolchain references
  tests/lcitool: append user setting stanza to dockerfiles
  tests/docker: add USER stanzas to non-lci images
  tests/docker: use direct RUNC call to build containers
  tests/docker: use direct RUNC call to run test jobs
  tests/gitlab: use kaniko to build images

 configure                                     |  3 +++
 .gitlab-ci.d/cirrus/freebsd-12.vars           |  5 ++++
 .gitlab-ci.d/cirrus/freebsd-13.vars           |  5 ++++
 .gitlab-ci.d/cirrus/macos-12.vars             |  5 ++++
 .gitlab-ci.d/container-template.yml           | 23 +++++++---------
 tests/docker/Makefile.include                 | 27 +++++++++++--------
 tests/docker/dockerfiles/alpine.docker        |  5 ++++
 tests/docker/dockerfiles/centos8.docker       |  5 ++++
 .../dockerfiles/debian-all-test-cross.docker  |  5 ++++
 .../dockerfiles/debian-alpha-cross.docker     |  5 ++++
 .../dockerfiles/debian-amd64-cross.docker     |  5 ++++
 tests/docker/dockerfiles/debian-amd64.docker  |  5 ++++
 .../dockerfiles/debian-arm64-cross.docker     |  5 ++++
 .../dockerfiles/debian-armel-cross.docker     |  5 ++++
 .../dockerfiles/debian-armhf-cross.docker     |  5 ++++
 .../dockerfiles/debian-hexagon-cross.docker   |  5 ++++
 .../dockerfiles/debian-hppa-cross.docker      |  5 ++++
 .../dockerfiles/debian-loongarch-cross.docker |  5 ++++
 .../dockerfiles/debian-m68k-cross.docker      |  5 ++++
 .../dockerfiles/debian-mips-cross.docker      |  5 ++++
 .../dockerfiles/debian-mips64-cross.docker    |  5 ++++
 .../dockerfiles/debian-mips64el-cross.docker  |  5 ++++
 .../dockerfiles/debian-mipsel-cross.docker    |  5 ++++
 tests/docker/dockerfiles/debian-native.docker |  5 ++++
 .../debian-powerpc-test-cross.docker          |  6 ++++-
 .../dockerfiles/debian-ppc64el-cross.docker   |  5 ++++
 .../dockerfiles/debian-riscv64-cross.docker   |  5 ++++
 .../debian-riscv64-test-cross.docker          |  5 ++++
 .../dockerfiles/debian-s390x-cross.docker     |  5 ++++
 .../dockerfiles/debian-sh4-cross.docker       |  5 ++++
 .../dockerfiles/debian-sparc64-cross.docker   |  5 ++++
 .../dockerfiles/debian-toolchain.docker       |  9 +++++--
 .../dockerfiles/debian-tricore-cross.docker   |  5 ++++
 .../dockerfiles/debian-xtensa-cross.docker    |  5 ++++
 .../dockerfiles/fedora-cris-cross.docker      |  5 ++++
 .../dockerfiles/fedora-i386-cross.docker      |  5 ++++
 .../dockerfiles/fedora-win32-cross.docker     |  5 ++++
 .../dockerfiles/fedora-win64-cross.docker     |  5 ++++
 tests/docker/dockerfiles/fedora.docker        |  5 ++++
 tests/docker/dockerfiles/opensuse-leap.docker |  5 ++++
 tests/docker/dockerfiles/python.docker        |  5 ++++
 tests/docker/dockerfiles/ubuntu2004.docker    |  5 ++++
 tests/docker/dockerfiles/ubuntu2204.docker    |  5 ++++
 tests/lcitool/refresh                         | 11 +++++++-
 44 files changed, 240 insertions(+), 29 deletions(-)

-- 
2.39.1

[PATCH 1/7] configure: expose the direct container command

[Alex Bennée]

In the process of migrating away from using docker.py to build our
containers we need to expose the command to the build environment. The
script is still a useful way to probe which command works though.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/configure b/configure
index cf6db3d551..b6a1cebad9 100755
--- a/configure
+++ b/configure
@@ -1744,6 +1744,7 @@ fi
 # functions to probe cross compilers
 
 container="no"
+runc=""
 if test $use_containers = "yes" && (has "docker" || has "podman"); then
     case $($python "$source_path"/tests/docker/docker.py probe) in
         *docker) container=docker ;;
@@ -1752,6 +1753,7 @@ if test $use_containers = "yes" && (has "docker" || has "podman"); then
     esac
     if test "$container" != "no"; then
         docker_py="$python $source_path/tests/docker/docker.py --engine $container"
+        runc=$($python "$source_path"/tests/docker/docker.py probe)
     fi
 fi
 
@@ -2351,6 +2353,7 @@ fi
 
 if test "$container" != no; then
     echo "ENGINE=$container" >> $config_host_mak
+    echo "RUNC=$runc" >> $config_host_mak
 fi
 echo "ROMS=$roms" >> $config_host_mak
 echo "MAKE=$make" >> $config_host_mak
-- 
2.39.1

[PATCH 2/7] tests/dockerfiles: unify debian-toolchain references

[Alex Bennée]

We use the debian release number elsewhere so fix it for consistency
along with the broken comment.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/debian-toolchain.docker | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/docker/dockerfiles/debian-toolchain.docker b/tests/docker/dockerfiles/debian-toolchain.docker
index 6c73408b34..dc9545857f 100644
--- a/tests/docker/dockerfiles/debian-toolchain.docker
+++ b/tests/docker/dockerfiles/debian-toolchain.docker
@@ -30,7 +30,7 @@ ADD build-toolchain.sh /root/build-toolchain.sh
 RUN cd /root && ./build-toolchain.sh
 
 # Throw away the extra toolchain build deps, the downloaded source,
-# and the build trees by restoring the original debian10 image,
+# and the build trees by restoring the original image,
 # then copying the built toolchain from stage 0.
-FROM docker.io/library/debian:bullseye-slim
+FROM docker.io/library/debian:11-slim
 COPY --from=0 /usr/local /usr/local
-- 
2.39.1

[PATCH 3/7] tests/lcitool: append user setting stanza to dockerfiles

[Alex Bennée]

For the cross-compilation use-case it is important to add the host
user to the dockerfile so we can map them to the docker environment
when cross-building files.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.d/cirrus/freebsd-12.vars                   |  5 +++++
 .gitlab-ci.d/cirrus/freebsd-13.vars                   |  5 +++++
 .gitlab-ci.d/cirrus/macos-12.vars                     |  5 +++++
 tests/docker/dockerfiles/alpine.docker                |  5 +++++
 tests/docker/dockerfiles/centos8.docker               |  5 +++++
 tests/docker/dockerfiles/debian-amd64-cross.docker    |  5 +++++
 tests/docker/dockerfiles/debian-amd64.docker          |  5 +++++
 tests/docker/dockerfiles/debian-arm64-cross.docker    |  5 +++++
 tests/docker/dockerfiles/debian-armel-cross.docker    |  5 +++++
 tests/docker/dockerfiles/debian-armhf-cross.docker    |  5 +++++
 tests/docker/dockerfiles/debian-mips64el-cross.docker |  5 +++++
 tests/docker/dockerfiles/debian-mipsel-cross.docker   |  5 +++++
 tests/docker/dockerfiles/debian-ppc64el-cross.docker  |  5 +++++
 tests/docker/dockerfiles/debian-s390x-cross.docker    |  5 +++++
 tests/docker/dockerfiles/fedora-win32-cross.docker    |  5 +++++
 tests/docker/dockerfiles/fedora-win64-cross.docker    |  5 +++++
 tests/docker/dockerfiles/fedora.docker                |  5 +++++
 tests/docker/dockerfiles/opensuse-leap.docker         |  5 +++++
 tests/docker/dockerfiles/ubuntu2004.docker            |  5 +++++
 tests/docker/dockerfiles/ubuntu2204.docker            |  5 +++++
 tests/lcitool/refresh                                 | 11 ++++++++++-
 21 files changed, 110 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/cirrus/freebsd-12.vars b/.gitlab-ci.d/cirrus/freebsd-12.vars
index 44d8a2a511..0bff53be44 100644
--- a/.gitlab-ci.d/cirrus/freebsd-12.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-12.vars
@@ -14,3 +14,8 @@ PIP3='/usr/local/bin/pip-3.8'
 PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja opencv pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 zstd'
 PYPI_PKGS=''
 PYTHON='/usr/local/bin/python3'
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/.gitlab-ci.d/cirrus/freebsd-13.vars b/.gitlab-ci.d/cirrus/freebsd-13.vars
index 7622c849b2..235d08a5ee 100644
--- a/.gitlab-ci.d/cirrus/freebsd-13.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-13.vars
@@ -14,3 +14,8 @@ PIP3='/usr/local/bin/pip-3.8'
 PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja opencv pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 zstd'
 PYPI_PKGS=''
 PYTHON='/usr/local/bin/python3'
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-12.vars
index da6aa6469b..599e210707 100644
--- a/.gitlab-ci.d/cirrus/macos-12.vars
+++ b/.gitlab-ci.d/cirrus/macos-12.vars
@@ -14,3 +14,8 @@ PIP3='/opt/homebrew/bin/pip3'
 PKGS='bash bc bison bzip2 capstone ccache cmocka ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo json-c libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson ncurses nettle ninja pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy socat sparse spice-protocol tesseract usbredir vde vte3 zlib zstd'
 PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme'
 PYTHON='/opt/homebrew/bin/python3'
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index 56cf14e553..7b82dec8e5 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -124,3 +124,8 @@ ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker
index 1291ae1b04..2cca33e730 100644
--- a/tests/docker/dockerfiles/centos8.docker
+++ b/tests/docker/dockerfiles/centos8.docker
@@ -134,3 +134,8 @@ ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-amd64-cross.docker b/tests/docker/dockerfiles/debian-amd64-cross.docker
index 856db95100..50a2b34c62 100644
--- a/tests/docker/dockerfiles/debian-amd64-cross.docker
+++ b/tests/docker/dockerfiles/debian-amd64-cross.docker
@@ -169,3 +169,8 @@ ENV ABI "x86_64-linux-gnu"
 ENV MESON_OPTS "--cross-file=x86_64-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=x86_64-linux-gnu-
 ENV DEF_TARGET_LIST x86_64-softmmu,x86_64-linux-user,i386-softmmu,i386-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index e3dba71ad5..d77e3d169a 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -155,3 +155,8 @@ RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
 RUN cd /usr/src/netmap && git checkout v11.3
 RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
 ENV QEMU_CONFIGURE_OPTS --enable-netmap
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker
index b00e9e9bcf..5288435da1 100644
--- a/tests/docker/dockerfiles/debian-arm64-cross.docker
+++ b/tests/docker/dockerfiles/debian-arm64-cross.docker
@@ -168,3 +168,8 @@ ENV ABI "aarch64-linux-gnu"
 ENV MESON_OPTS "--cross-file=aarch64-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=aarch64-linux-gnu-
 ENV DEF_TARGET_LIST aarch64-softmmu,aarch64-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/docker/dockerfiles/debian-armel-cross.docker
index fb1129f256..95ce5bbfd0 100644
--- a/tests/docker/dockerfiles/debian-armel-cross.docker
+++ b/tests/docker/dockerfiles/debian-armel-cross.docker
@@ -167,3 +167,8 @@ ENV ABI "arm-linux-gnueabi"
 ENV MESON_OPTS "--cross-file=arm-linux-gnueabi"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabi-
 ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
index 7a2b864a38..20f6074d5e 100644
--- a/tests/docker/dockerfiles/debian-armhf-cross.docker
+++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
@@ -168,3 +168,8 @@ ENV ABI "arm-linux-gnueabihf"
 ENV MESON_OPTS "--cross-file=arm-linux-gnueabihf"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
 ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker
index 5a3340e964..15f8568cb1 100644
--- a/tests/docker/dockerfiles/debian-mips64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker
@@ -165,3 +165,8 @@ ENV ABI "mips64el-linux-gnuabi64"
 ENV MESON_OPTS "--cross-file=mips64el-linux-gnuabi64"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mips64el-linux-gnuabi64-
 ENV DEF_TARGET_LIST mips64el-softmmu,mips64el-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/docker/dockerfiles/debian-mipsel-cross.docker
index 422fdebe8f..cc6a44dbe7 100644
--- a/tests/docker/dockerfiles/debian-mipsel-cross.docker
+++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker
@@ -165,3 +165,8 @@ ENV ABI "mipsel-linux-gnu"
 ENV MESON_OPTS "--cross-file=mipsel-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mipsel-linux-gnu-
 ENV DEF_TARGET_LIST mipsel-softmmu,mipsel-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
index 78d7ae6211..7ff1e44b88 100644
--- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
@@ -167,3 +167,8 @@ ENV ABI "powerpc64le-linux-gnu"
 ENV MESON_OPTS "--cross-file=powerpc64le-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=powerpc64le-linux-gnu-
 ENV DEF_TARGET_LIST ppc64-softmmu,ppc64-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/debian-s390x-cross.docker b/tests/docker/dockerfiles/debian-s390x-cross.docker
index d06ea3605a..d084f34cb1 100644
--- a/tests/docker/dockerfiles/debian-s390x-cross.docker
+++ b/tests/docker/dockerfiles/debian-s390x-cross.docker
@@ -166,3 +166,8 @@ ENV ABI "s390x-linux-gnu"
 ENV MESON_OPTS "--cross-file=s390x-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=s390x-linux-gnu-
 ENV DEF_TARGET_LIST s390x-softmmu,s390x-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/fedora-win32-cross.docker b/tests/docker/dockerfiles/fedora-win32-cross.docker
index 21ed1c6081..87d91a6759 100644
--- a/tests/docker/dockerfiles/fedora-win32-cross.docker
+++ b/tests/docker/dockerfiles/fedora-win32-cross.docker
@@ -100,3 +100,8 @@ ENV ABI "i686-w64-mingw32"
 ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw32.meson"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=i686-w64-mingw32-
 ENV DEF_TARGET_LIST i386-softmmu
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/fedora-win64-cross.docker b/tests/docker/dockerfiles/fedora-win64-cross.docker
index 95d30e7936..877766ec6b 100644
--- a/tests/docker/dockerfiles/fedora-win64-cross.docker
+++ b/tests/docker/dockerfiles/fedora-win64-cross.docker
@@ -100,3 +100,8 @@ ENV ABI "x86_64-w64-mingw32"
 ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw64.meson"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=x86_64-w64-mingw32-
 ENV DEF_TARGET_LIST x86_64-softmmu
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 8e06d080b8..4a7a986dfd 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -146,3 +146,8 @@ ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
index 568c1c979f..f5f8d4714d 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -137,3 +137,8 @@ ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 75233064de..4d257b922b 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -146,3 +146,8 @@ ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/docker/dockerfiles/ubuntu2204.docker b/tests/docker/dockerfiles/ubuntu2204.docker
index 30b9e56793..41c2d2878e 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
+++ b/tests/docker/dockerfiles/ubuntu2204.docker
@@ -145,3 +145,8 @@ ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
 ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
\ No newline at end of file
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index cc9e34ac87..88bf33fb74 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -40,6 +40,15 @@ def atomic_write(filename, content):
         tmp.unlink()
         raise
 
+# Optional user setting, this will always be the last thing added
+# so maximise the number of layers that are cached
+add_user_mapping = [
+    "# As a final step configure the user (if env is defined)",
+    "ARG USER",
+    "ARG UID",
+    "RUN if [ \"${USER}\" ]; then \\",
+    "  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi"
+]
 
 def generate(filename, cmd, trailer):
     print("Generate %s" % filename)
@@ -51,9 +60,9 @@ def generate(filename, cmd, trailer):
     content = lcitool.stdout.decode("utf8")
     if trailer is not None:
         content += trailer
+    content += "\n".join(add_user_mapping)
     atomic_write(filename, content)
 
-
 def generate_dockerfile(host, target, cross=None, trailer=None):
     filename = Path(src_dir, "tests", "docker", "dockerfiles", host + ".docker")
     cmd = lcitool_cmd + ["dockerfile"]
-- 
2.39.1

[PATCH 4/7] tests/docker: add USER stanzas to non-lci images

[Alex Bennée]

These are flat but not generated by lcitool so we need to manually
update them with the `useradd` stanza.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/debian-all-test-cross.docker     | 5 +++++
 tests/docker/dockerfiles/debian-alpha-cross.docker        | 5 +++++
 tests/docker/dockerfiles/debian-hexagon-cross.docker      | 5 +++++
 tests/docker/dockerfiles/debian-hppa-cross.docker         | 5 +++++
 tests/docker/dockerfiles/debian-loongarch-cross.docker    | 5 +++++
 tests/docker/dockerfiles/debian-m68k-cross.docker         | 5 +++++
 tests/docker/dockerfiles/debian-mips-cross.docker         | 5 +++++
 tests/docker/dockerfiles/debian-mips64-cross.docker       | 5 +++++
 tests/docker/dockerfiles/debian-native.docker             | 5 +++++
 tests/docker/dockerfiles/debian-powerpc-test-cross.docker | 6 +++++-
 tests/docker/dockerfiles/debian-riscv64-cross.docker      | 5 +++++
 tests/docker/dockerfiles/debian-riscv64-test-cross.docker | 5 +++++
 tests/docker/dockerfiles/debian-sh4-cross.docker          | 5 +++++
 tests/docker/dockerfiles/debian-sparc64-cross.docker      | 5 +++++
 tests/docker/dockerfiles/debian-toolchain.docker          | 5 +++++
 tests/docker/dockerfiles/debian-tricore-cross.docker      | 5 +++++
 tests/docker/dockerfiles/debian-xtensa-cross.docker       | 5 +++++
 tests/docker/dockerfiles/fedora-cris-cross.docker         | 5 +++++
 tests/docker/dockerfiles/fedora-i386-cross.docker         | 5 +++++
 tests/docker/dockerfiles/python.docker                    | 5 +++++
 20 files changed, 100 insertions(+), 1 deletion(-)

diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
index 8dc5e1b5de..981e9bdc7b 100644
--- a/tests/docker/dockerfiles/debian-all-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
@@ -61,3 +61,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
 
 ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs --disable-tools
 ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-alpha-cross.docker b/tests/docker/dockerfiles/debian-alpha-cross.docker
index 4eeb43c78a..7fa7bf1bde 100644
--- a/tests/docker/dockerfiles/debian-alpha-cross.docker
+++ b/tests/docker/dockerfiles/debian-alpha-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-alpha-linux-gnu \
         libc6.1-dev-alpha-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker
index 8a0d748343..5308ccb8fe 100644
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -33,3 +33,8 @@ ENV TOOLCHAIN_URL https://codelinaro.jfrog.io/artifactory/codelinaro-toolchain-f
 
 RUN curl -#SL "$TOOLCHAIN_URL" | tar -xJC "$TOOLCHAIN_INSTALL"
 ENV PATH $PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-hppa-cross.docker b/tests/docker/dockerfiles/debian-hppa-cross.docker
index af1c8403d8..dd47ffdfa4 100644
--- a/tests/docker/dockerfiles/debian-hppa-cross.docker
+++ b/tests/docker/dockerfiles/debian-hppa-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-hppa-linux-gnu \
         libc6-dev-hppa-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-loongarch-cross.docker b/tests/docker/dockerfiles/debian-loongarch-cross.docker
index a8e8e98909..9d957547b5 100644
--- a/tests/docker/dockerfiles/debian-loongarch-cross.docker
+++ b/tests/docker/dockerfiles/debian-loongarch-cross.docker
@@ -25,3 +25,8 @@ RUN curl -#SL https://github.com/loongson/build-tools/releases/download/2022.05.
 
 ENV PATH $PATH:/opt/cross-tools/bin
 ENV LD_LIBRARY_PATH /opt/cross-tools/lib:/opt/cross-tools/loongarch64-unknown-linux-gnu/lib:$LD_LIBRARY_PATH
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-m68k-cross.docker b/tests/docker/dockerfiles/debian-m68k-cross.docker
index dded71c5d2..25dd1c1e68 100644
--- a/tests/docker/dockerfiles/debian-m68k-cross.docker
+++ b/tests/docker/dockerfiles/debian-m68k-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-m68k-linux-gnu \
         libc6-dev-m68k-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-mips-cross.docker b/tests/docker/dockerfiles/debian-mips-cross.docker
index 7b55f0f3b2..2cbc568ed1 100644
--- a/tests/docker/dockerfiles/debian-mips-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
             gcc-mips-linux-gnu \
             libc6-dev-mips-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-mips64-cross.docker b/tests/docker/dockerfiles/debian-mips64-cross.docker
index afcff9726f..ba965cf564 100644
--- a/tests/docker/dockerfiles/debian-mips64-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-mips64-linux-gnuabi64 \
         libc6-dev-mips64-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-native.docker b/tests/docker/dockerfiles/debian-native.docker
index 8dd033097c..abac7d7cd7 100644
--- a/tests/docker/dockerfiles/debian-native.docker
+++ b/tests/docker/dockerfiles/debian-native.docker
@@ -47,3 +47,8 @@ RUN apt update && \
 
 ENV QEMU_CONFIGURE_OPTS $QEMU_CONFIGURE_OPTS
 ENV DEF_TARGET_LIST "none"
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-powerpc-test-cross.docker b/tests/docker/dockerfiles/debian-powerpc-test-cross.docker
index d6b2909cc4..23779413d3 100644
--- a/tests/docker/dockerfiles/debian-powerpc-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-powerpc-test-cross.docker
@@ -16,4 +16,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
         libc6-dev-ppc64-cross \
         gcc-10-powerpc64le-linux-gnu \
         libc6-dev-ppc64el-cross
-
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
index 3daf93968a..803afb9573 100644
--- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
@@ -50,3 +50,8 @@ RUN apt update && \
 # Specify the cross prefix for this image (see tests/docker/common.rc)
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu-
 ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-riscv64-test-cross.docker b/tests/docker/dockerfiles/debian-riscv64-test-cross.docker
index e5f83a5aeb..6e631295bc 100644
--- a/tests/docker/dockerfiles/debian-riscv64-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-test-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-riscv64-linux-gnu \
         libc6-dev-riscv64-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-sh4-cross.docker b/tests/docker/dockerfiles/debian-sh4-cross.docker
index d48ed9065f..6bd8171d33 100644
--- a/tests/docker/dockerfiles/debian-sh4-cross.docker
+++ b/tests/docker/dockerfiles/debian-sh4-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-sh4-linux-gnu \
         libc6-dev-sh4-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-sparc64-cross.docker b/tests/docker/dockerfiles/debian-sparc64-cross.docker
index 8d3d306bc1..1ef735f223 100644
--- a/tests/docker/dockerfiles/debian-sparc64-cross.docker
+++ b/tests/docker/dockerfiles/debian-sparc64-cross.docker
@@ -12,3 +12,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
     eatmydata apt-get install --no-install-recommends -y \
         gcc-sparc64-linux-gnu \
         libc6-dev-sparc64-cross
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-toolchain.docker b/tests/docker/dockerfiles/debian-toolchain.docker
index dc9545857f..687a97fec4 100644
--- a/tests/docker/dockerfiles/debian-toolchain.docker
+++ b/tests/docker/dockerfiles/debian-toolchain.docker
@@ -34,3 +34,8 @@ RUN cd /root && ./build-toolchain.sh
 # then copying the built toolchain from stage 0.
 FROM docker.io/library/debian:11-slim
 COPY --from=0 /usr/local /usr/local
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-tricore-cross.docker b/tests/docker/dockerfiles/debian-tricore-cross.docker
index 82e4576485..cfd2faf9a8 100644
--- a/tests/docker/dockerfiles/debian-tricore-cross.docker
+++ b/tests/docker/dockerfiles/debian-tricore-cross.docker
@@ -41,3 +41,8 @@ RUN curl -#SL https://github.com/bkoppelmann/package_940/releases/download/trico
 # This image can only build a very minimal QEMU as well as the tests
 ENV DEF_TARGET_LIST tricore-softmmu
 ENV QEMU_CONFIGURE_OPTS --disable-user --disable-tools --disable-fdt
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/debian-xtensa-cross.docker b/tests/docker/dockerfiles/debian-xtensa-cross.docker
index 2f11b3b7bc..082b50da19 100644
--- a/tests/docker/dockerfiles/debian-xtensa-cross.docker
+++ b/tests/docker/dockerfiles/debian-xtensa-cross.docker
@@ -27,3 +27,8 @@ RUN for cpu in $CPU_LIST; do \
     done
 
 ENV PATH $PATH:/opt/$TOOLCHAIN_RELEASE/xtensa-dc232b-elf/bin:/opt/$TOOLCHAIN_RELEASE/xtensa-dc233c-elf/bin:/opt/$TOOLCHAIN_RELEASE/xtensa-de233_fpu-elf/bin:/opt/$TOOLCHAIN_RELEASE/xtensa-dsp3400-elf/bin
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/fedora-cris-cross.docker b/tests/docker/dockerfiles/fedora-cris-cross.docker
index 91c373fdd3..f2899af410 100644
--- a/tests/docker/dockerfiles/fedora-cris-cross.docker
+++ b/tests/docker/dockerfiles/fedora-cris-cross.docker
@@ -6,3 +6,8 @@ FROM registry.fedoraproject.org/fedora:33
 ENV PACKAGES gcc-cris-linux-gnu
 RUN dnf install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/fedora-i386-cross.docker b/tests/docker/dockerfiles/fedora-i386-cross.docker
index f58b64dc3e..14c1fb2c93 100644
--- a/tests/docker/dockerfiles/fedora-i386-cross.docker
+++ b/tests/docker/dockerfiles/fedora-i386-cross.docker
@@ -32,3 +32,8 @@ ENV PKG_CONFIG_LIBDIR /usr/lib/pkgconfig
 
 RUN dnf update -y && dnf install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker
index 56d88417df..708ebbed75 100644
--- a/tests/docker/dockerfiles/python.docker
+++ b/tests/docker/dockerfiles/python.docker
@@ -16,3 +16,8 @@ ENV PACKAGES \
 
 RUN dnf install -y $PACKAGES
 RUN rpm -q $PACKAGES | sort > /packages.txt
+# As a final step configure the user (if env is defined)
+ARG USER
+ARG UID
+RUN if [ "${USER}" ]; then \
+  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
-- 
2.39.1

[PATCH 5/7] tests/docker: use direct RUNC call to build containers

[Alex Bennée]

We don't really need stuff from docker.py to do the build as we have
everything we need with a direct call. We do rely on the dockerfiles
being able to tweak the UID/name mapping as the last step.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index bfb0dcac21..9e73ff5cf3 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -7,6 +7,8 @@ SPACE := $(NULL) #
 COMMA := ,
 
 HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
+USER = $(if $(NOUSER),,$(shell id -un))
+UID = $(if $(NOUSER),,$(shell id -u))
 
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 ifeq ($(HOST_ARCH),x86_64)
@@ -14,6 +16,7 @@ DOCKER_DEFAULT_REGISTRY := registry.gitlab.com/qemu-project/qemu
 endif
 DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTRY))
 
+RUNC ?= docker
 ENGINE ?= auto
 DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --engine $(ENGINE)
 
@@ -35,15 +38,16 @@ docker-qemu-src: $(DOCKER_SRC_COPY)
 
 # General rule for building docker images.
 docker-image-%: $(DOCKER_FILES_DIR)/%.docker
-	$(call quiet-command,\
-		$(DOCKER_SCRIPT) build -t qemu/$* -f $< \
-		$(if $V,,--quiet) \
-		$(if $(NOCACHE),--no-cache, \
-			$(if $(DOCKER_REGISTRY),--registry $(DOCKER_REGISTRY))) \
-		$(if $(NOUSER),,--add-current-user) \
-		$(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\
-		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
-		"BUILD","$*")
+	  $(call quiet-command,			\
+		$(RUNC) build				\
+		$(if $V,,--quiet)			\
+		$(if $(NOCACHE),--no-cache,		\
+			$(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \
+		$(if $(NOUSER),,			\
+			--build-arg USER=$(USER)	\
+			--build-arg UID=$(UID))	\
+		-t qemu/$* - < $<, 			\
+		"BUILD", $1)
 
 # Special rule for debootstraped binfmt linux-user images
 docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
-- 
2.39.1

[PATCH 6/7] tests/docker: use direct RUNC call to run test jobs

[Alex Bennée]

If we build them without the script we can certainly run them without
it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 9e73ff5cf3..cee1b34703 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -223,8 +223,9 @@ docker-run: docker-qemu-src
 			$(IMAGE) --executable $(EXECUTABLE),		\
 			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
 	$(call quiet-command,						\
-		$(DOCKER_SCRIPT) run 					\
-			$(if $(NOUSER),,--run-as-current-user) 		\
+		$(RUNC) run 						\
+			--rm						\
+			$(if $(NOUSER),,-u $(UID)) 			\
 			--security-opt seccomp=unconfined		\
 			$(if $(DEBUG),-ti,)				\
 			$(if $(NETWORK),$(if $(subst $(NETWORK),,1),--net=$(NETWORK)),--net=none) \
-- 
2.39.1

[PATCH 7/7] tests/gitlab: use kaniko to build images

[Alex Bennée]

Apparently the docker-in-docker approach has some flaws including
needing privileged mode to run and being quite slow. An alternative
approach is to use Google's kaniko tool. It also works across
different gitlab executors.

Following the gitlab example code we drop all the direct docker calls
and usage of the script and make a direct call to kaniko and hope the
images are cacheable by others.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.d/container-template.yml | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml
index c434b9c8f3..c2d7950df8 100644
--- a/.gitlab-ci.d/container-template.yml
+++ b/.gitlab-ci.d/container-template.yml
@@ -1,22 +1,17 @@
 .container_job_template:
   extends: .base_job_template
-  image: docker:stable
+  image:
+    name: gcr.io/kaniko-project/executor:v1.9.0-debug
+    entrypoint: [""]
   stage: containers
-  services:
-    - docker:dind
   before_script:
     - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
-    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
-    - apk add python3
-    - docker info
-    - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
+    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
   script:
     - echo "TAG:$TAG"
     - echo "COMMON_TAG:$COMMON_TAG"
-    - ./tests/docker/docker.py --engine docker build
-          -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
-          -r $CI_REGISTRY/qemu-project/qemu
-    - docker tag "qemu/$NAME" "$TAG"
-    - docker push "$TAG"
-  after_script:
-    - docker logout
+    - /kaniko/executor
+          --reproducible
+          --context "${CI_PROJECT_DIR}"
+          --dockerfile "${CI_PROJECT_DIR}/tests/docker/dockerfiles/$NAME.docker"
+          --destination "${TAG}"
-- 
2.39.1

Re: [PATCH 2/7] tests/dockerfiles: unify debian-toolchain references

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:52PM +0000, Alex Bennée wrote:
> We use the debian release number elsewhere so fix it for consistency
> along with the broken comment.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/dockerfiles/debian-toolchain.docker | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 1/7] configure: expose the direct container command

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:51PM +0000, Alex Bennée wrote:
> In the process of migrating away from using docker.py to build our
> containers we need to expose the command to the build environment. The
> script is still a useful way to probe which command works though.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  configure | 3 +++
>  1 file changed, 3 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 3/7] tests/lcitool: append user setting stanza to dockerfiles

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:53PM +0000, Alex Bennée wrote:
> For the cross-compilation use-case it is important to add the host
> user to the dockerfile so we can map them to the docker environment
> when cross-building files.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .gitlab-ci.d/cirrus/freebsd-12.vars                   |  5 +++++
>  .gitlab-ci.d/cirrus/freebsd-13.vars                   |  5 +++++
>  .gitlab-ci.d/cirrus/macos-12.vars                     |  5 +++++
>  tests/docker/dockerfiles/alpine.docker                |  5 +++++
>  tests/docker/dockerfiles/centos8.docker               |  5 +++++
>  tests/docker/dockerfiles/debian-amd64-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/debian-amd64.docker          |  5 +++++
>  tests/docker/dockerfiles/debian-arm64-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/debian-armel-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/debian-armhf-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/debian-mips64el-cross.docker |  5 +++++
>  tests/docker/dockerfiles/debian-mipsel-cross.docker   |  5 +++++
>  tests/docker/dockerfiles/debian-ppc64el-cross.docker  |  5 +++++
>  tests/docker/dockerfiles/debian-s390x-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/fedora-win32-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/fedora-win64-cross.docker    |  5 +++++
>  tests/docker/dockerfiles/fedora.docker                |  5 +++++
>  tests/docker/dockerfiles/opensuse-leap.docker         |  5 +++++
>  tests/docker/dockerfiles/ubuntu2004.docker            |  5 +++++
>  tests/docker/dockerfiles/ubuntu2204.docker            |  5 +++++
>  tests/lcitool/refresh                                 | 11 ++++++++++-
>  21 files changed, 110 insertions(+), 1 deletion(-)


> 
> diff --git a/.gitlab-ci.d/cirrus/freebsd-12.vars b/.gitlab-ci.d/cirrus/freebsd-12.vars
> index 44d8a2a511..0bff53be44 100644
> --- a/.gitlab-ci.d/cirrus/freebsd-12.vars
> +++ b/.gitlab-ci.d/cirrus/freebsd-12.vars
> @@ -14,3 +14,8 @@ PIP3='/usr/local/bin/pip-3.8'
>  PKGS='alsa-lib bash bison bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage cmocka ctags curl cyrus-sasl dbus diffutils dtc flex fusefs-libs3 gettext git glib gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs libslirp libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja opencv pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy sndio socat spice-protocol tesseract usbredir virglrenderer vte3 zstd'
>  PYPI_PKGS=''
>  PYTHON='/usr/local/bin/python3'
> +# As a final step configure the user (if env is defined)
> +ARG USER
> +ARG UID
> +RUN if [ "${USER}" ]; then \
> +  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
> \ No newline at end of file

The cirrus/*.vars files aren't dockerfiles, they're shell
scripts, so don't modify them


> diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
> index 56cf14e553..7b82dec8e5 100644
> --- a/tests/docker/dockerfiles/alpine.docker
> +++ b/tests/docker/dockerfiles/alpine.docker
> @@ -124,3 +124,8 @@ ENV LANG "en_US.UTF-8"
>  ENV MAKE "/usr/bin/make"
>  ENV NINJA "/usr/bin/ninja"
>  ENV PYTHON "/usr/bin/python3"
> +# As a final step configure the user (if env is defined)
> +ARG USER
> +ARG UID
> +RUN if [ "${USER}" ]; then \
> +  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
> \ No newline at end of file

Add a trailing newline to these perhaps ?


> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
> index cc9e34ac87..88bf33fb74 100755
> --- a/tests/lcitool/refresh
> +++ b/tests/lcitool/refresh
> @@ -40,6 +40,15 @@ def atomic_write(filename, content):
>          tmp.unlink()
>          raise
>  
> +# Optional user setting, this will always be the last thing added
> +# so maximise the number of layers that are cached
> +add_user_mapping = [
> +    "# As a final step configure the user (if env is defined)",
> +    "ARG USER",
> +    "ARG UID",
> +    "RUN if [ \"${USER}\" ]; then \\",
> +    "  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi"
> +]
>  
>  def generate(filename, cmd, trailer):
>      print("Generate %s" % filename)
> @@ -51,9 +60,9 @@ def generate(filename, cmd, trailer):
>      content = lcitool.stdout.decode("utf8")
>      if trailer is not None:
>          content += trailer
> +    content += "\n".join(add_user_mapping)

  if filename.endswith(".docker"):
       content += "\n".join(add_user_mapping)

to avoid splattering the cirrus vars files


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 4/7] tests/docker: add USER stanzas to non-lci images

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:54PM +0000, Alex Bennée wrote:
> These are flat but not generated by lcitool so we need to manually
> update them with the `useradd` stanza.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/dockerfiles/debian-all-test-cross.docker     | 5 +++++
>  tests/docker/dockerfiles/debian-alpha-cross.docker        | 5 +++++
>  tests/docker/dockerfiles/debian-hexagon-cross.docker      | 5 +++++
>  tests/docker/dockerfiles/debian-hppa-cross.docker         | 5 +++++
>  tests/docker/dockerfiles/debian-loongarch-cross.docker    | 5 +++++
>  tests/docker/dockerfiles/debian-m68k-cross.docker         | 5 +++++
>  tests/docker/dockerfiles/debian-mips-cross.docker         | 5 +++++
>  tests/docker/dockerfiles/debian-mips64-cross.docker       | 5 +++++
>  tests/docker/dockerfiles/debian-native.docker             | 5 +++++
>  tests/docker/dockerfiles/debian-powerpc-test-cross.docker | 6 +++++-
>  tests/docker/dockerfiles/debian-riscv64-cross.docker      | 5 +++++
>  tests/docker/dockerfiles/debian-riscv64-test-cross.docker | 5 +++++
>  tests/docker/dockerfiles/debian-sh4-cross.docker          | 5 +++++
>  tests/docker/dockerfiles/debian-sparc64-cross.docker      | 5 +++++
>  tests/docker/dockerfiles/debian-toolchain.docker          | 5 +++++
>  tests/docker/dockerfiles/debian-tricore-cross.docker      | 5 +++++
>  tests/docker/dockerfiles/debian-xtensa-cross.docker       | 5 +++++
>  tests/docker/dockerfiles/fedora-cris-cross.docker         | 5 +++++
>  tests/docker/dockerfiles/fedora-i386-cross.docker         | 5 +++++

Unrelated to this, but I mentioned in another thread, that we
should really drop the fedora-i386 and tell lcitool to generate
a debian-i386 docker file one day. There's no compelling reason
for it to be an exception and use Fedora instead of Debian.

>  tests/docker/dockerfiles/python.docker                    | 5 +++++
>  20 files changed, 100 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 5/7] tests/docker: use direct RUNC call to build containers

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:55PM +0000, Alex Bennée wrote:
> We don't really need stuff from docker.py to do the build as we have
> everything we need with a direct call. We do rely on the dockerfiles
> being able to tweak the UID/name mapping as the last step.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include | 22 +++++++++++++---------
>  1 file changed, 13 insertions(+), 9 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 6/7] tests/docker: use direct RUNC call to run test jobs

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:56PM +0000, Alex Bennée wrote:
> If we build them without the script we can certainly run them without
> it.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 7/7] tests/gitlab: use kaniko to build images

[Daniel P. Berrangé]

On Fri, Feb 24, 2023 at 06:08:57PM +0000, Alex Bennée wrote:
> Apparently the docker-in-docker approach has some flaws including
> needing privileged mode to run and being quite slow. An alternative
> approach is to use Google's kaniko tool. It also works across
> different gitlab executors.

Interesting, I've not come across this tool before.

> Following the gitlab example code we drop all the direct docker calls
> and usage of the script and make a direct call to kaniko and hope the
> images are cacheable by others.



> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .gitlab-ci.d/container-template.yml | 23 +++++++++--------------
>  1 file changed, 9 insertions(+), 14 deletions(-)
> 
> diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml
> index c434b9c8f3..c2d7950df8 100644
> --- a/.gitlab-ci.d/container-template.yml
> +++ b/.gitlab-ci.d/container-template.yml
> @@ -1,22 +1,17 @@
>  .container_job_template:
>    extends: .base_job_template
> -  image: docker:stable
> +  image:
> +    name: gcr.io/kaniko-project/executor:v1.9.0-debug
> +    entrypoint: [""]
>    stage: containers
> -  services:
> -    - docker:dind
>    before_script:
>      - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
> -    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
> -    - apk add python3
> -    - docker info
> -    - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
> +    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
>    script:
>      - echo "TAG:$TAG"
>      - echo "COMMON_TAG:$COMMON_TAG"
> -    - ./tests/docker/docker.py --engine docker build
> -          -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
> -          -r $CI_REGISTRY/qemu-project/qemu
> -    - docker tag "qemu/$NAME" "$TAG"
> -    - docker push "$TAG"
> -  after_script:
> -    - docker logout
> +    - /kaniko/executor
> +          --reproducible
> +          --context "${CI_PROJECT_DIR}"
> +          --dockerfile "${CI_PROJECT_DIR}/tests/docker/dockerfiles/$NAME.docker"
> +          --destination "${TAG}"

AFAICT from reading the docs, this does not allow for caching of layers
from the previous built image, so will always do a from scratch build.
This feels like it would kill any performance benefits over docker.

https://github.com/GoogleContainerTools/kaniko#caching

  "Users can opt into caching by setting the --cache=true flag. A
   remote repository for storing cached layers can be provided via
   the --cache-repo flag. If this flag isn't provided, a cached
   repo will be inferred from the --destination provided."

So it would sound like we need --cache=true adding to this command,
and in theory it should prime its cache from ${TAG}, but i'm not
entirely confident in that without testing. Also does not seem
like its psosible to have the dual cache COMMON_TAG + TAG, which
is particularly important for forks, since COMMON_TAG is more
likely to be up2date.

We could live with the latter restriction if we *always* used
COMMON_TAG as the cache, because 99% of the time forks are
going to just be using dockerfiles that match upstream. Only
those few contributors who have a branch that's changing
the dockerfiles would benefit from pointing the cache at
their fork.

So we could achieve that I believe using

     - /kaniko/executor
           --reproducible
           --context "${CI_PROJECT_DIR}"
	   --cache=true
	   --cache-repo "${COMMON_TAG}"
           --dockerfile "${CI_PROJECT_DIR}/tests/docker/dockerfiles/$NAME.docker"
           --destination "${TAG}"

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 2/7] tests/dockerfiles: unify debian-toolchain references

[Philippe Mathieu-Daudé]

On 24/2/23 19:08, Alex Bennée wrote:
> We use the debian release number elsewhere so fix it for consistency
> along with the broken comment.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/docker/dockerfiles/debian-toolchain.docker | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 4/7] tests/docker: add USER stanzas to non-lci images

[Philippe Mathieu-Daudé]

On 24/2/23 19:08, Alex Bennée wrote:
> These are flat but not generated by lcitool so we need to manually
> update them with the `useradd` stanza.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/docker/dockerfiles/debian-all-test-cross.docker     | 5 +++++
>   tests/docker/dockerfiles/debian-alpha-cross.docker        | 5 +++++
>   tests/docker/dockerfiles/debian-hexagon-cross.docker      | 5 +++++
>   tests/docker/dockerfiles/debian-hppa-cross.docker         | 5 +++++
>   tests/docker/dockerfiles/debian-loongarch-cross.docker    | 5 +++++
>   tests/docker/dockerfiles/debian-m68k-cross.docker         | 5 +++++
>   tests/docker/dockerfiles/debian-mips-cross.docker         | 5 +++++
>   tests/docker/dockerfiles/debian-mips64-cross.docker       | 5 +++++
>   tests/docker/dockerfiles/debian-native.docker             | 5 +++++
>   tests/docker/dockerfiles/debian-powerpc-test-cross.docker | 6 +++++-
>   tests/docker/dockerfiles/debian-riscv64-cross.docker      | 5 +++++
>   tests/docker/dockerfiles/debian-riscv64-test-cross.docker | 5 +++++
>   tests/docker/dockerfiles/debian-sh4-cross.docker          | 5 +++++
>   tests/docker/dockerfiles/debian-sparc64-cross.docker      | 5 +++++
>   tests/docker/dockerfiles/debian-toolchain.docker          | 5 +++++
>   tests/docker/dockerfiles/debian-tricore-cross.docker      | 5 +++++
>   tests/docker/dockerfiles/debian-xtensa-cross.docker       | 5 +++++
>   tests/docker/dockerfiles/fedora-cris-cross.docker         | 5 +++++
>   tests/docker/dockerfiles/fedora-i386-cross.docker         | 5 +++++
>   tests/docker/dockerfiles/python.docker                    | 5 +++++
>   20 files changed, 100 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
> index 8dc5e1b5de..981e9bdc7b 100644
> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
> @@ -61,3 +61,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>   
>   ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs --disable-tools
>   ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user
> +# As a final step configure the user (if env is defined)
> +ARG USER
> +ARG UID
> +RUN if [ "${USER}" ]; then \
> +  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi

Is that intended for local image building?

Personally I only use the image built by gitlab and mount the containers
with -u $UID -v $HOME/.ccache -v $HOME/source/qemu. Would that still
keep working, or do I have to map from gitlab user to mine?

Re: [PATCH 4/7] tests/docker: add USER stanzas to non-lci images

[Daniel P. Berrangé]

On Tue, Feb 28, 2023 at 12:43:01PM +0100, Philippe Mathieu-Daudé wrote:
> On 24/2/23 19:08, Alex Bennée wrote:
> > These are flat but not generated by lcitool so we need to manually
> > update them with the `useradd` stanza.
> > 
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > ---
> >   tests/docker/dockerfiles/debian-all-test-cross.docker     | 5 +++++
> >   tests/docker/dockerfiles/debian-alpha-cross.docker        | 5 +++++
> >   tests/docker/dockerfiles/debian-hexagon-cross.docker      | 5 +++++
> >   tests/docker/dockerfiles/debian-hppa-cross.docker         | 5 +++++
> >   tests/docker/dockerfiles/debian-loongarch-cross.docker    | 5 +++++
> >   tests/docker/dockerfiles/debian-m68k-cross.docker         | 5 +++++
> >   tests/docker/dockerfiles/debian-mips-cross.docker         | 5 +++++
> >   tests/docker/dockerfiles/debian-mips64-cross.docker       | 5 +++++
> >   tests/docker/dockerfiles/debian-native.docker             | 5 +++++
> >   tests/docker/dockerfiles/debian-powerpc-test-cross.docker | 6 +++++-
> >   tests/docker/dockerfiles/debian-riscv64-cross.docker      | 5 +++++
> >   tests/docker/dockerfiles/debian-riscv64-test-cross.docker | 5 +++++
> >   tests/docker/dockerfiles/debian-sh4-cross.docker          | 5 +++++
> >   tests/docker/dockerfiles/debian-sparc64-cross.docker      | 5 +++++
> >   tests/docker/dockerfiles/debian-toolchain.docker          | 5 +++++
> >   tests/docker/dockerfiles/debian-tricore-cross.docker      | 5 +++++
> >   tests/docker/dockerfiles/debian-xtensa-cross.docker       | 5 +++++
> >   tests/docker/dockerfiles/fedora-cris-cross.docker         | 5 +++++
> >   tests/docker/dockerfiles/fedora-i386-cross.docker         | 5 +++++
> >   tests/docker/dockerfiles/python.docker                    | 5 +++++
> >   20 files changed, 100 insertions(+), 1 deletion(-)
> > 
> > diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
> > index 8dc5e1b5de..981e9bdc7b 100644
> > --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
> > +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
> > @@ -61,3 +61,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >   ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs --disable-tools
> >   ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user
> > +# As a final step configure the user (if env is defined)
> > +ARG USER
> > +ARG UID
> > +RUN if [ "${USER}" ]; then \
> > +  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
> 
> Is that intended for local image building?
> 
> Personally I only use the image built by gitlab and mount the containers
> with -u $UID -v $HOME/.ccache -v $HOME/source/qemu. Would that still
> keep working, or do I have to map from gitlab user to mine?

The images fetched from gitlab already have this present, as the
current docker.py  adds it to all our containers. So this should
essentially be a no functional change.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 1/7] configure: expose the direct container command

[Philippe Mathieu-Daudé]

On 24/2/23 19:08, Alex Bennée wrote:
> In the process of migrating away from using docker.py to build our
> containers we need to expose the command to the build environment. The
> script is still a useful way to probe which command works though.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   configure | 3 +++
>   1 file changed, 3 insertions(+)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 5/7] tests/docker: use direct RUNC call to build containers

[Philippe Mathieu-Daudé]

On 24/2/23 19:08, Alex Bennée wrote:
> We don't really need stuff from docker.py to do the build as we have
> everything we need with a direct call. We do rely on the dockerfiles
> being able to tweak the UID/name mapping as the last step.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/docker/Makefile.include | 22 +++++++++++++---------
>   1 file changed, 13 insertions(+), 9 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Philippe Mathieu-Daudé]

On 24/2/23 19:08, Alex Bennée wrote:
> This series attempts to remove our dependence on the docker.py script
> and build things directly with the appropriate tool. I've been
> noodling around with how we build images on gitlab to see if they can
> cache better because the normal case should be we don't need to
> rebuild everything if the upstream distro hasn't updated its package
> list.
> 
> Anyway what do people think?

Removing dind limitation is interesting.

Unrelated, can we tag registry.gitlab.com/qemu-project's
docker images along with releases?

Re: [PATCH 4/7] tests/docker: add USER stanzas to non-lci images

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> On 24/2/23 19:08, Alex Bennée wrote:
>> These are flat but not generated by lcitool so we need to manually
>> update them with the `useradd` stanza.
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>   tests/docker/dockerfiles/debian-all-test-cross.docker     | 5 +++++
>>   tests/docker/dockerfiles/debian-alpha-cross.docker        | 5 +++++
>>   tests/docker/dockerfiles/debian-hexagon-cross.docker      | 5 +++++
>>   tests/docker/dockerfiles/debian-hppa-cross.docker         | 5 +++++
>>   tests/docker/dockerfiles/debian-loongarch-cross.docker    | 5 +++++
>>   tests/docker/dockerfiles/debian-m68k-cross.docker         | 5 +++++
>>   tests/docker/dockerfiles/debian-mips-cross.docker         | 5 +++++
>>   tests/docker/dockerfiles/debian-mips64-cross.docker       | 5 +++++
>>   tests/docker/dockerfiles/debian-native.docker             | 5 +++++
>>   tests/docker/dockerfiles/debian-powerpc-test-cross.docker | 6 +++++-
>>   tests/docker/dockerfiles/debian-riscv64-cross.docker      | 5 +++++
>>   tests/docker/dockerfiles/debian-riscv64-test-cross.docker | 5 +++++
>>   tests/docker/dockerfiles/debian-sh4-cross.docker          | 5 +++++
>>   tests/docker/dockerfiles/debian-sparc64-cross.docker      | 5 +++++
>>   tests/docker/dockerfiles/debian-toolchain.docker          | 5 +++++
>>   tests/docker/dockerfiles/debian-tricore-cross.docker      | 5 +++++
>>   tests/docker/dockerfiles/debian-xtensa-cross.docker       | 5 +++++
>>   tests/docker/dockerfiles/fedora-cris-cross.docker         | 5 +++++
>>   tests/docker/dockerfiles/fedora-i386-cross.docker         | 5 +++++
>>   tests/docker/dockerfiles/python.docker                    | 5 +++++
>>   20 files changed, 100 insertions(+), 1 deletion(-)
>> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker
>> b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> index 8dc5e1b5de..981e9bdc7b 100644
>> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
>> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> @@ -61,3 +61,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>     ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs
>> --disable-tools
>>   ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user
>> +# As a final step configure the user (if env is defined)
>> +ARG USER
>> +ARG UID
>> +RUN if [ "${USER}" ]; then \
>> +  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
>
> Is that intended for local image building?

For building tests. When we run the compilers inside the docker
container we want to ensure the final binary is visible and accessible
outside the container.

> Personally I only use the image built by gitlab and mount the containers
> with -u $UID -v $HOME/.ccache -v $HOME/source/qemu. Would that still
> keep working, or do I have to map from gitlab user to mine?

Its only added on for local builds so in theory you should cache all the
layers apart from the last one.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 4/7] tests/docker: add USER stanzas to non-lci images

[Philippe Mathieu-Daudé]

On 28/2/23 13:18, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
> 
>> On 24/2/23 19:08, Alex Bennée wrote:
>>> These are flat but not generated by lcitool so we need to manually
>>> update them with the `useradd` stanza.
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> ---
>>>    tests/docker/dockerfiles/debian-all-test-cross.docker     | 5 +++++
>>>    tests/docker/dockerfiles/debian-alpha-cross.docker        | 5 +++++
>>>    tests/docker/dockerfiles/debian-hexagon-cross.docker      | 5 +++++
>>>    tests/docker/dockerfiles/debian-hppa-cross.docker         | 5 +++++
>>>    tests/docker/dockerfiles/debian-loongarch-cross.docker    | 5 +++++
>>>    tests/docker/dockerfiles/debian-m68k-cross.docker         | 5 +++++
>>>    tests/docker/dockerfiles/debian-mips-cross.docker         | 5 +++++
>>>    tests/docker/dockerfiles/debian-mips64-cross.docker       | 5 +++++
>>>    tests/docker/dockerfiles/debian-native.docker             | 5 +++++
>>>    tests/docker/dockerfiles/debian-powerpc-test-cross.docker | 6 +++++-
>>>    tests/docker/dockerfiles/debian-riscv64-cross.docker      | 5 +++++
>>>    tests/docker/dockerfiles/debian-riscv64-test-cross.docker | 5 +++++
>>>    tests/docker/dockerfiles/debian-sh4-cross.docker          | 5 +++++
>>>    tests/docker/dockerfiles/debian-sparc64-cross.docker      | 5 +++++
>>>    tests/docker/dockerfiles/debian-toolchain.docker          | 5 +++++
>>>    tests/docker/dockerfiles/debian-tricore-cross.docker      | 5 +++++
>>>    tests/docker/dockerfiles/debian-xtensa-cross.docker       | 5 +++++
>>>    tests/docker/dockerfiles/fedora-cris-cross.docker         | 5 +++++
>>>    tests/docker/dockerfiles/fedora-i386-cross.docker         | 5 +++++
>>>    tests/docker/dockerfiles/python.docker                    | 5 +++++
>>>    20 files changed, 100 insertions(+), 1 deletion(-)
>>> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker
>>> b/tests/docker/dockerfiles/debian-all-test-cross.docker
>>> index 8dc5e1b5de..981e9bdc7b 100644
>>> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
>>> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
>>> @@ -61,3 +61,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>>      ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs
>>> --disable-tools
>>>    ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user
>>> +# As a final step configure the user (if env is defined)
>>> +ARG USER
>>> +ARG UID
>>> +RUN if [ "${USER}" ]; then \
>>> +  id ${USER} 2>/dev/null || useradd -u ${UID} -U ${USER}; fi
>>
>> Is that intended for local image building?
> 
> For building tests. When we run the compilers inside the docker
> container we want to ensure the final binary is visible and accessible
> outside the container.
> 
>> Personally I only use the image built by gitlab and mount the containers
>> with -u $UID -v $HOME/.ccache -v $HOME/source/qemu. Would that still
>> keep working, or do I have to map from gitlab user to mine?
> 
> Its only added on for local builds so in theory you should cache all the
> layers apart from the last one.

OK.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Daniel P. Berrangé]

On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote:
> On 24/2/23 19:08, Alex Bennée wrote:
> > This series attempts to remove our dependence on the docker.py script
> > and build things directly with the appropriate tool. I've been
> > noodling around with how we build images on gitlab to see if they can
> > cache better because the normal case should be we don't need to
> > rebuild everything if the upstream distro hasn't updated its package
> > list.
> > 
> > Anyway what do people think?
> 
> Removing dind limitation is interesting.
> 
> Unrelated, can we tag registry.gitlab.com/qemu-project's
> docker images along with releases?

Can you elaborate on this ?

We're only using the images for CI purposes and they must always reflect
the current state of git master. We're using a fixed docker tag 'latest',
as that avoids the container registry growing arbitrarily large.

Our CI rules should prevent the pipelines running on stable branches,
so we shouldn't need container tags for stable.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Philippe Mathieu-Daudé]

On 28/2/23 13:58, Daniel P. Berrangé wrote:
> On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote:
>> On 24/2/23 19:08, Alex Bennée wrote:
>>> This series attempts to remove our dependence on the docker.py script
>>> and build things directly with the appropriate tool. I've been
>>> noodling around with how we build images on gitlab to see if they can
>>> cache better because the normal case should be we don't need to
>>> rebuild everything if the upstream distro hasn't updated its package
>>> list.
>>>
>>> Anyway what do people think?
>>
>> Removing dind limitation is interesting.
>>
>> Unrelated, can we tag registry.gitlab.com/qemu-project's
>> docker images along with releases?
> 
> Can you elaborate on this ?
> 
> We're only using the images for CI purposes and they must always reflect
> the current state of git master. We're using a fixed docker tag 'latest',
> as that avoids the container registry growing arbitrarily large.
> 
> Our CI rules should prevent the pipelines running on stable branches,
> so we shouldn't need container tags for stable.

I'm not suggesting keeping jobs to build, but doing a snapshot of the
released containers.

I.e. when we release 8.0, we should tag qemu/fedora:v8.0 and never touch
it again. This is useful when bisecting pre-v8, but also to build pre-v8
and do performance comparison. One shouldn't have to upgrade such
container (in particular when package mirror disappear), since it
already contains all we need.

Personally I'd like to keep generic and problematic ones:
- qemu/fedora
- qemu/fedora-win64-cross
- qemu/debian-xtensa-cross

Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Daniel P. Berrangé]

On Tue, Feb 28, 2023 at 02:29:12PM +0100, Philippe Mathieu-Daudé wrote:
> On 28/2/23 13:58, Daniel P. Berrangé wrote:
> > On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote:
> > > On 24/2/23 19:08, Alex Bennée wrote:
> > > > This series attempts to remove our dependence on the docker.py script
> > > > and build things directly with the appropriate tool. I've been
> > > > noodling around with how we build images on gitlab to see if they can
> > > > cache better because the normal case should be we don't need to
> > > > rebuild everything if the upstream distro hasn't updated its package
> > > > list.
> > > > 
> > > > Anyway what do people think?
> > > 
> > > Removing dind limitation is interesting.
> > > 
> > > Unrelated, can we tag registry.gitlab.com/qemu-project's
> > > docker images along with releases?
> > 
> > Can you elaborate on this ?
> > 
> > We're only using the images for CI purposes and they must always reflect
> > the current state of git master. We're using a fixed docker tag 'latest',
> > as that avoids the container registry growing arbitrarily large.
> > 
> > Our CI rules should prevent the pipelines running on stable branches,
> > so we shouldn't need container tags for stable.
> 
> I'm not suggesting keeping jobs to build, but doing a snapshot of the
> released containers.
> 
> I.e. when we release 8.0, we should tag qemu/fedora:v8.0 and never touch
> it again. This is useful when bisecting pre-v8, but also to build pre-v8
> and do performance comparison. One shouldn't have to upgrade such
> container (in particular when package mirror disappear), since it
> already contains all we need.

The main risk with this is the impact on our storage quota. With the
OSS Program membership IIUC we get Ultimate level features which
is 250 GB of storage, across git repos, pipeline cache/artifacts/logs,
container registry.

Currently they have no way to enforce that since their accounting of
usage is not accurate enough. They're working on fixing that so at
somepoint we'll be subject to the 250 GB limit.

What I don't know is how much storage we're currently using across
the /qemu-project namespace, and what extra is implied by taking
a snapshot of our container registry 3 times a year. I'm expecting
it to probably be in the high 10's of GB though for the container
registry.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Tue, Feb 28, 2023 at 02:29:12PM +0100, Philippe Mathieu-Daudé wrote:
>> On 28/2/23 13:58, Daniel P. Berrangé wrote:
>> > On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote:
>> > > On 24/2/23 19:08, Alex Bennée wrote:
>> > > > This series attempts to remove our dependence on the docker.py script
>> > > > and build things directly with the appropriate tool. I've been
>> > > > noodling around with how we build images on gitlab to see if they can
>> > > > cache better because the normal case should be we don't need to
>> > > > rebuild everything if the upstream distro hasn't updated its package
>> > > > list.
>> > > > 
>> > > > Anyway what do people think?
>> > > 
>> > > Removing dind limitation is interesting.
>> > > 
>> > > Unrelated, can we tag registry.gitlab.com/qemu-project's
>> > > docker images along with releases?
>> > 
>> > Can you elaborate on this ?
>> > 
>> > We're only using the images for CI purposes and they must always reflect
>> > the current state of git master. We're using a fixed docker tag 'latest',
>> > as that avoids the container registry growing arbitrarily large.
>> > 
>> > Our CI rules should prevent the pipelines running on stable branches,
>> > so we shouldn't need container tags for stable.
>> 
>> I'm not suggesting keeping jobs to build, but doing a snapshot of the
>> released containers.
>> 
>> I.e. when we release 8.0, we should tag qemu/fedora:v8.0 and never touch
>> it again. This is useful when bisecting pre-v8, but also to build pre-v8
>> and do performance comparison. One shouldn't have to upgrade such
>> container (in particular when package mirror disappear), since it
>> already contains all we need.
>
> The main risk with this is the impact on our storage quota. With the
> OSS Program membership IIUC we get Ultimate level features which
> is 250 GB of storage, across git repos, pipeline cache/artifacts/logs,
> container registry.
>
> Currently they have no way to enforce that since their accounting of
> usage is not accurate enough. They're working on fixing that so at
> somepoint we'll be subject to the 250 GB limit.
>
> What I don't know is how much storage we're currently using across
> the /qemu-project namespace, and what extra is implied by taking
> a snapshot of our container registry 3 times a year. I'm expecting
> it to probably be in the high 10's of GB though for the container
> registry.

Currently we are using:

86.6 Gb of artefacts
28.5 Gb of containers
220 Mb of file uploads
24.8Mb of git repo

We could probably cut down a lot of usage of artefacts by either not
building full fat ones with debug symbols and passing between layers or
tweaking the build system to prevent re-building of object files if the
final binary is present in the file system.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Philippe Mathieu-Daudé]

On 28/2/23 14:57, Alex Bennée wrote:
> 
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
>> On Tue, Feb 28, 2023 at 02:29:12PM +0100, Philippe Mathieu-Daudé wrote:
>>> On 28/2/23 13:58, Daniel P. Berrangé wrote:
>>>> On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote:
>>>>> On 24/2/23 19:08, Alex Bennée wrote:
>>>>>> This series attempts to remove our dependence on the docker.py script
>>>>>> and build things directly with the appropriate tool. I've been
>>>>>> noodling around with how we build images on gitlab to see if they can
>>>>>> cache better because the normal case should be we don't need to
>>>>>> rebuild everything if the upstream distro hasn't updated its package
>>>>>> list.
>>>>>>
>>>>>> Anyway what do people think?
>>>>>
>>>>> Removing dind limitation is interesting.
>>>>>
>>>>> Unrelated, can we tag registry.gitlab.com/qemu-project's
>>>>> docker images along with releases?
>>>>
>>>> Can you elaborate on this ?
>>>>
>>>> We're only using the images for CI purposes and they must always reflect
>>>> the current state of git master. We're using a fixed docker tag 'latest',
>>>> as that avoids the container registry growing arbitrarily large.
>>>>
>>>> Our CI rules should prevent the pipelines running on stable branches,
>>>> so we shouldn't need container tags for stable.
>>>
>>> I'm not suggesting keeping jobs to build, but doing a snapshot of the
>>> released containers.
>>>
>>> I.e. when we release 8.0, we should tag qemu/fedora:v8.0 and never touch
>>> it again. This is useful when bisecting pre-v8, but also to build pre-v8
>>> and do performance comparison. One shouldn't have to upgrade such
>>> container (in particular when package mirror disappear), since it
>>> already contains all we need.
>>
>> The main risk with this is the impact on our storage quota. With the
>> OSS Program membership IIUC we get Ultimate level features which
>> is 250 GB of storage, across git repos, pipeline cache/artifacts/logs,
>> container registry.
>>
>> Currently they have no way to enforce that since their accounting of
>> usage is not accurate enough. They're working on fixing that so at
>> somepoint we'll be subject to the 250 GB limit.
>>
>> What I don't know is how much storage we're currently using across
>> the /qemu-project namespace, and what extra is implied by taking
>> a snapshot of our container registry 3 times a year. I'm expecting
>> it to probably be in the high 10's of GB though for the container
>> registry.

Maybe we can keep (a subset of) the stable release container images
and store them on a registry outside of gitlab. That would be nice,
but if I'm the only one interested and nobody felt this was useful
before, no need to spend more energy on this topic.

> Currently we are using:
> 
> 86.6 Gb of artefacts

^ transient

> 28.5 Gb of containers

^ constant

> 220 Mb of file uploads
> 24.8Mb of git repo
> 
> We could probably cut down a lot of usage of artefacts by either not
> building full fat ones with debug symbols and passing between layers or
> tweaking the build system to prevent re-building of object files if the
> final binary is present in the file system.
>