* [PATCH v5 01/18] ui: avoid compiler warnings from unused clipboard info variable
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 02/18] spice: Update QXLInterface for spice >= 0.15.0 Daniel P. Berrangé
` (17 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Richard Henderson,
Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
With latest clang 13.0.0 we get
../ui/clipboard.c:47:34: error: variable 'old' set but not used [-Werror,-Wunused-but-set-variable]
g_autoptr(QemuClipboardInfo) old = NULL;
^
The compiler can't tell that we only declared this variable in
order to get the side effect of free'ing it when out of scope.
This pattern is a little dubious for a use of g_autoptr, so
rewrite the code to avoid it.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
ui/clipboard.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/ui/clipboard.c b/ui/clipboard.c
index d7b008d62a..7672058e84 100644
--- a/ui/clipboard.c
+++ b/ui/clipboard.c
@@ -44,12 +44,11 @@ void qemu_clipboard_peer_release(QemuClipboardPeer *peer,
void qemu_clipboard_update(QemuClipboardInfo *info)
{
- g_autoptr(QemuClipboardInfo) old = NULL;
assert(info->selection < QEMU_CLIPBOARD_SELECTION__COUNT);
notifier_list_notify(&clipboard_notifiers, info);
- old = cbinfo[info->selection];
+ qemu_clipboard_info_unref(cbinfo[info->selection]);
cbinfo[info->selection] = qemu_clipboard_info_ref(info);
}
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 02/18] spice: Update QXLInterface for spice >= 0.15.0
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 01/18] ui: avoid compiler warnings from unused clipboard info variable Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 03/18] meson: require liburing >= 0.3 Daniel P. Berrangé
` (16 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P . Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, John Snow,
Gerd Hoffmann, Philippe Mathieu-Daudé
From: John Snow <jsnow@redhat.com>
spice updated the spelling (and arguments) of "attache_worker" in
0.15.0. Update QEMU to match, preventing -Wdeprecated-declarations
compilations from reporting build errors.
See also:
https://gitlab.freedesktop.org/spice/spice/-/commit/974692bda1e77af92b71ed43b022439448492cb9
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
hw/display/qxl.c | 14 +++++++++++++-
include/ui/qemu-spice.h | 6 ++++++
ui/spice-display.c | 11 +++++++++++
3 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index 29c80b4289..1da6703e44 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -517,13 +517,20 @@ static int qxl_track_command(PCIQXLDevice *qxl, struct QXLCommandExt *ext)
/* spice display interface callbacks */
-static void interface_attach_worker(QXLInstance *sin, QXLWorker *qxl_worker)
+static void interface_attached_worker(QXLInstance *sin)
{
PCIQXLDevice *qxl = container_of(sin, PCIQXLDevice, ssd.qxl);
trace_qxl_interface_attach_worker(qxl->id);
}
+#if !(SPICE_HAS_ATTACHED_WORKER)
+static void interface_attach_worker(QXLInstance *sin, QXLWorker *qxl_worker)
+{
+ interface_attached_worker(sin);
+}
+#endif
+
static void interface_set_compression_level(QXLInstance *sin, int level)
{
PCIQXLDevice *qxl = container_of(sin, PCIQXLDevice, ssd.qxl);
@@ -1131,7 +1138,12 @@ static const QXLInterface qxl_interface = {
.base.major_version = SPICE_INTERFACE_QXL_MAJOR,
.base.minor_version = SPICE_INTERFACE_QXL_MINOR,
+#if SPICE_HAS_ATTACHED_WORKER
+ .attached_worker = interface_attached_worker,
+#else
.attache_worker = interface_attach_worker,
+#endif
+
.set_compression_level = interface_set_compression_level,
#if SPICE_NEEDS_SET_MM_TIME
.set_mm_time = interface_set_mm_time,
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index 71ecd6cfd1..21fe195e18 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -40,6 +40,12 @@ int qemu_spice_migrate_info(const char *hostname, int port, int tls_port,
#define SPICE_NEEDS_SET_MM_TIME 0
#endif
+#if defined(SPICE_SERVER_VERSION) && (SPICE_SERVER_VERSION >= 0x000f00)
+#define SPICE_HAS_ATTACHED_WORKER 1
+#else
+#define SPICE_HAS_ATTACHED_WORKER 0
+#endif
+
#else /* CONFIG_SPICE */
#include "qemu/error-report.h"
diff --git a/ui/spice-display.c b/ui/spice-display.c
index f59c69882d..1a60cebb7d 100644
--- a/ui/spice-display.c
+++ b/ui/spice-display.c
@@ -500,10 +500,17 @@ void qemu_spice_display_refresh(SimpleSpiceDisplay *ssd)
/* spice display interface callbacks */
+#if SPICE_HAS_ATTACHED_WORKER
+static void interface_attached_worker(QXLInstance *sin)
+{
+ /* nothing to do */
+}
+#else
static void interface_attach_worker(QXLInstance *sin, QXLWorker *qxl_worker)
{
/* nothing to do */
}
+#endif
static void interface_set_compression_level(QXLInstance *sin, int level)
{
@@ -702,7 +709,11 @@ static const QXLInterface dpy_interface = {
.base.major_version = SPICE_INTERFACE_QXL_MAJOR,
.base.minor_version = SPICE_INTERFACE_QXL_MINOR,
+#if SPICE_HAS_ATTACHED_WORKER
+ .attached_worker = interface_attached_worker,
+#else
.attache_worker = interface_attach_worker,
+#endif
.set_compression_level = interface_set_compression_level,
#if SPICE_NEEDS_SET_MM_TIME
.set_mm_time = interface_set_mm_time,
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 03/18] meson: require liburing >= 0.3
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 01/18] ui: avoid compiler warnings from unused clipboard info variable Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 02/18] spice: Update QXLInterface for spice >= 0.15.0 Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 04/18] ui: avoid warnings about directdb on Alpine / musl libc Daniel P. Berrangé
` (15 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
openSUSE Leap 15.2 ships with liburing == 0.2 against which QEMU fails
to build.
../util/fdmon-io_uring.c: In function ‘fdmon_io_uring_need_wait’:
../util/fdmon-io_uring.c:305:9: error: implicit declaration of function ‘io_uring_sq_ready’; did you mean ‘io_uring_cq_ready’? [-Werror=implicit-function-declaration]
if (io_uring_sq_ready(&ctx->fdmon_io_uring)) {
^~~~~~~~~~~~~~~~~
io_uring_cq_ready
This method was introduced in liburing 0.3, so set that as a minimum
requirement.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
meson.build | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meson.build b/meson.build
index 96de1a6ef9..3b77d7f5b6 100644
--- a/meson.build
+++ b/meson.build
@@ -425,7 +425,8 @@ if not get_option('linux_aio').auto() or have_block
endif
linux_io_uring = not_found
if not get_option('linux_io_uring').auto() or have_block
- linux_io_uring = dependency('liburing', required: get_option('linux_io_uring'),
+ linux_io_uring = dependency('liburing', version: '>=0.3',
+ required: get_option('linux_io_uring'),
method: 'pkg-config', kwargs: static_kwargs)
endif
libxml2 = not_found
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 04/18] ui: avoid warnings about directdb on Alpine / musl libc
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (2 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 03/18] meson: require liburing >= 0.3 Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 05/18] ci: explicitly skip I/O tests on alpine Daniel P. Berrangé
` (14 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
On Alpine, SDL is built with directfb support and this triggers warnings
during QEMU build
In file included from /usr/include/directfb/direct/thread.h:38,
from /usr/include/directfb/direct/debug.h:43,
from /usr/include/directfb/direct/interface.h:36,
from /usr/include/directfb/directfb.h:49,
from /usr/include/SDL2/SDL_syswm.h:80,
from /builds/berrange/qemu/include/ui/sdl2.h:8,
from ../ui/sdl2-gl.c:31:
/usr/include/directfb/direct/os/waitqueue.h:41:25: error: redundant redeclaration of 'direct_waitqueue_init' [-Werror=redundant-decls]
41 | DirectResult DIRECT_API direct_waitqueue_init ( DirectWaitQueue *queue );
| ^~~~~~~~~~~~~~~~~~~~~
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
include/ui/sdl2.h | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/include/ui/sdl2.h b/include/ui/sdl2.h
index f85c117a78..0330978df8 100644
--- a/include/ui/sdl2.h
+++ b/include/ui/sdl2.h
@@ -5,7 +5,18 @@
#undef WIN32_LEAN_AND_MEAN
#include <SDL.h>
+
+/* with Alpine / muslc SDL headers pull in directfb headers
+ * which in turn trigger warning about redundant decls for
+ * direct_waitqueue_deinit.
+ */
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wredundant-decls"
+
#include <SDL_syswm.h>
+
+#pragma GCC diagnostic pop
+
#ifdef CONFIG_SDL_IMAGE
# include <SDL_image.h>
#endif
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 05/18] ci: explicitly skip I/O tests on alpine
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (3 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 04/18] ui: avoid warnings about directdb on Alpine / musl libc Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 06/18] tests/docker: switch fedora image to release 35 Daniel P. Berrangé
` (13 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
The block I/O tests don't work on Alpine because their alternative libc
impl emits different strings for errnos, which breaks the expected
output matching. e.g.
=== IO: pattern 102
wrote 512/512 bytes at offset 512
512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
+qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error
4
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0
Currently the I/O tests are skipped as a side effect of the Alpine image
containing a minimal busybox 'sed' binary, rather than GNU sed. This is
a fragile assumption that will be invalidated when the dockerfile is
changed to be autogenerated from a standardized package list that
includes GNU sed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
.gitlab-ci.d/buildtest.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 71d0f407ad..e1fe37e563 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -24,7 +24,7 @@ check-system-alpine:
artifacts: true
variables:
IMAGE: alpine
- MAKE_CHECK_ARGS: check
+ MAKE_CHECK_ARGS: check-unit check-qtest
avocado-system-alpine:
extends: .avocado_test_job_template
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 06/18] tests/docker: switch fedora image to release 35
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (4 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 05/18] ci: explicitly skip I/O tests on alpine Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 07/18] tests: integrate lcitool for generating build env manifests Daniel P. Berrangé
` (12 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
The Fedora 33 release is shortly end of life. Switch to the newest
Fedora 35 to maximise lifespan until we need to update again.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/fedora.docker | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index c6fd7e1113..855aefaac5 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -1,4 +1,4 @@
-FROM registry.fedoraproject.org/fedora:33
+FROM registry.fedoraproject.org/fedora:35
# Please keep this list sorted alphabetically
ENV PACKAGES \
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 07/18] tests: integrate lcitool for generating build env manifests
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (5 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 06/18] tests/docker: switch fedora image to release 35 Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 08/18] tests/docker: auto-generate centos8.docker with lcitool Daniel P. Berrangé
` (11 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This introduces
https://gitlab.com/libvirt/libvirt-ci
as a git submodule at tests/lcitool/libvirt-ci
The 'lcitool' program within this submodule will be used to
automatically generate build environment manifests from a definition
of requirements in tests/lcitool/projects/qemu.yml
It will ultimately be capable of generating
- Dockerfiles
- Package lists for installation in VMs
- Variables for configuring Cirrus CI environments
When a new build pre-requisite is needed for QEMU, if this package
is not currently known to libvirt-ci, it must first be added to the
'mappings.yml' file in the above git repo.
Then the submodule can be updated and the build pre-requisite added
to the tests/lcitool/projects/qemu.yml file. Now all the build env
manifests can be re-generated using 'make lcitool-refresh'
This ensures that when a new build pre-requisite is introduced, it
is added to all the different OS containers, VMs and Cirrus CI
environments consistently.
It also facilitates the addition of containers targetting new distros
or updating existing containers to new versions of the same distro,
where packages might have been renamed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
.gitmodules | 3 +
Makefile | 2 +
docs/devel/testing.rst | 104 ++++++++++++++++++++++++++++-
tests/lcitool/Makefile.include | 17 +++++
tests/lcitool/libvirt-ci | 1 +
tests/lcitool/projects/qemu.yml | 115 ++++++++++++++++++++++++++++++++
tests/lcitool/refresh | 67 +++++++++++++++++++
7 files changed, 306 insertions(+), 3 deletions(-)
create mode 100644 tests/lcitool/Makefile.include
create mode 160000 tests/lcitool/libvirt-ci
create mode 100644 tests/lcitool/projects/qemu.yml
create mode 100755 tests/lcitool/refresh
diff --git a/.gitmodules b/.gitmodules
index 08b1b48a09..84425d87e2 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -64,3 +64,6 @@
[submodule "roms/vbootrom"]
path = roms/vbootrom
url = https://gitlab.com/qemu-project/vbootrom.git
+[submodule "tests/lcitool/libvirt-ci"]
+ path = tests/lcitool/libvirt-ci
+ url = http://gitlab.com/libvirt/libvirt-ci
diff --git a/Makefile b/Makefile
index 74c5b46d38..aec4728240 100644
--- a/Makefile
+++ b/Makefile
@@ -287,6 +287,7 @@ cscope:
# Needed by "meson install"
export DESTDIR
+include $(SRC_PATH)/tests/lcitool/Makefile.include
include $(SRC_PATH)/tests/docker/Makefile.include
include $(SRC_PATH)/tests/vm/Makefile.include
@@ -316,6 +317,7 @@ endif
@echo 'Test targets:'
$(call print-help,check,Run all tests (check-help for details))
$(call print-help,bench,Run all benchmarks)
+ $(call print-help,lcitool-help,Help about targets for managing build environment manifests)
$(call print-help,docker-help,Help about targets running tests inside containers)
$(call print-help,vm-help,Help about targets running tests inside VM)
@echo ''
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 755343c7dd..d744b5909c 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -382,14 +382,112 @@ Along with many other images, the ``centos8`` image is defined in a Dockerfile
in ``tests/docker/dockerfiles/``, called ``centos8.docker``. ``make docker-help``
command will list all the available images.
-To add a new image, simply create a new ``.docker`` file under the
-``tests/docker/dockerfiles/`` directory.
-
A ``.pre`` script can be added beside the ``.docker`` file, which will be
executed before building the image under the build context directory. This is
mainly used to do necessary host side setup. One such setup is ``binfmt_misc``,
for example, to make qemu-user powered cross build containers work.
+Most of the existing Dockerfiles were written by hand, simply by creating a
+a new ``.docker`` file under the ``tests/docker/dockerfiles/`` directory.
+This has led to an inconsistent set of packages being present across the
+different containers.
+
+Thus going forward, QEMU is aiming to automatically generate the Dockerfiles
+using the ``lcitool`` program provided by the ``libvirt-ci`` project:
+
+ https://gitlab.com/libvirt/libvirt-ci
+
+In that project, there is a ``mappings.yml`` file defining the distro native
+package names for a wide variety of third party projects. This is processed
+in combination with a project defined list of build pre-requisites to determine
+the list of native packages to install on each distribution. This can be used
+to generate dockerfiles, VM package lists and Cirrus CI variables needed to
+setup build environments across OS distributions with a consistent set of
+packages present.
+
+When preparing a patch series that adds a new build pre-requisite to QEMU,
+updates to various lcitool data files may be required.
+
+
+Adding new build pre-requisites
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In the simple case where the pre-requisite is already known to ``libvirt-ci``
+the following steps are needed
+
+ * Edit ``tests/lcitool/projects/qemu.yml`` and add the pre-requisite
+
+ * Run ``make lcitool-refresh`` to re-generate all relevant build environment
+ manifests
+
+In some cases ``libvirt-ci`` will not know about the build pre-requisite and
+thus some extra preparation steps will be required first
+
+ * Fork the ``libvirt-ci`` project on gitlab
+
+ * Edit the ``mappings.yml`` change to add an entry for the new build
+ prerequisite, listing its native package name on as many OS distros
+ as practical.
+
+ * Commit the ``mappings.yml`` change and submit a merge request to
+ the ``libvirt-ci`` project, noting in the description that this
+ is a new build pre-requisite desired for use with QEMU
+
+ * CI pipeline will run to validate that the changes to ``mappings.yml``
+ are correct, by attempting to install the newly listed package on
+ all OS distributions supported by ``libvirt-ci``.
+
+ * Once the merge request is accepted, go back to QEMU and update
+ the ``libvirt-ci`` submodule to point to a commit that contains
+ the ``mappings.yml`` update.
+
+
+Adding new OS distros
+^^^^^^^^^^^^^^^^^^^^^
+
+In some cases ``libvirt-ci`` will not know about the OS distro that is
+desired to be tested. Before adding a new OS distro, discuss the proposed
+addition:
+
+ * Send a mail to qemu-devel, copying people listed in the
+ MAINTAINERS file for ``Build and test automation``.
+
+ There are limited CI compute resources available to QEMU, so the
+ cost/benefit tradeoff of adding new OS distros needs to be considered.
+
+ * File an issue at https://gitlab.com/libvirt/libvirt-ci/-/issues
+ pointing to the qemu-devel mail thread in the archives.
+
+ This alerts other people who might be interested in the work
+ to avoid duplication, as well as to get feedback from libvirt-ci
+ maintainers on any tips to ease the addition
+
+Assuming there is agreement to add a new OS distro then
+
+ * Fork the ``libvirt-ci`` project on gitlab
+
+ * Add metadata under ``guests/lcitool/lcitool/ansible/group_vars/``
+ for the new OS distro. There might be code changes required if
+ the OS distro uses a package format not currently known. The
+ ``libvirt-ci`` maintainers can advise on this when the issue
+ is file.
+
+ * Edit the ``mappings.yml`` change to update all the existing package
+ entries, providing details of the new OS distro
+
+ * Commit the ``mappings.yml`` change and submit a merge request to
+ the ``libvirt-ci`` project, noting in the description that this
+ is a new build pre-requisite desired for use with QEMU
+
+ * CI pipeline will run to validate that the changes to ``mappings.yml``
+ are correct, by attempting to install the newly listed package on
+ all OS distributions supported by ``libvirt-ci``.
+
+ * Once the merge request is accepted, go back to QEMU and update
+ the ``libvirt-ci`` submodule to point to a commit that contains
+ the ``mappings.yml`` update.
+
+
Tests
~~~~~
diff --git a/tests/lcitool/Makefile.include b/tests/lcitool/Makefile.include
new file mode 100644
index 0000000000..cff7c0b814
--- /dev/null
+++ b/tests/lcitool/Makefile.include
@@ -0,0 +1,17 @@
+
+LCITOOL_REFRESH = $(SRC_PATH)/tests/lcitool/refresh
+
+lcitool:
+ @echo 'Manage build environment manifests'
+ @echo
+ @echo 'Available targets:'
+ @echo
+ @echo ' lcitool: Print this help.'
+ @echo ' lcitool-refresh: Re-generate all build environment manifests.'
+ @echo
+
+lcitool-help: lcitool
+
+lcitool-refresh:
+ $(call quiet-command, git submodule update --init $(SRC_PATH)/tests/lcitool/libvirt-ci)
+ $(call quiet-command, $(LCITOOL_REFRESH))
diff --git a/tests/lcitool/libvirt-ci b/tests/lcitool/libvirt-ci
new file mode 160000
index 0000000000..29cec2153b
--- /dev/null
+++ b/tests/lcitool/libvirt-ci
@@ -0,0 +1 @@
+Subproject commit 29cec2153b9a4dbb2e66f1cbc9866a4eff519cfd
diff --git a/tests/lcitool/projects/qemu.yml b/tests/lcitool/projects/qemu.yml
new file mode 100644
index 0000000000..2e2271510e
--- /dev/null
+++ b/tests/lcitool/projects/qemu.yml
@@ -0,0 +1,115 @@
+---
+packages:
+ - alsa
+ - bash
+ - bc
+ - brlapi
+ - bzip2
+ - bzip2-libs
+ - capstone
+ - ccache
+ - clang
+ - column
+ - ctags
+ - cyrus-sasl
+ - daxctl
+ - dbus-daemon
+ - device-mapper-multipath
+ - diffutils
+ - dtrace
+ - findutils
+ - g++
+ - gcc
+ - gcovr
+ - gettext
+ - genisoimage
+ - glib2
+ - glib2-static
+ - glibc-static
+ - glusterfs
+ - gnutls
+ - gtk3
+ - hostname
+ - libaio
+ - libattr
+ - libasan
+ - libbpf
+ - libcacard
+ - libcap-ng
+ - libcurl
+ - libdrm
+ - libepoxy
+ - libfdt
+ - libffi
+ - libgcrypt
+ - libibverbs
+ - libiscsi
+ - libjemalloc
+ - libjpeg
+ - libnfs
+ - libnuma
+ - libpmem
+ - libpng
+ - librbd
+ - librdmacm
+ - libseccomp
+ - libselinux
+ - libslirp
+ - libssh
+ - libtasn1
+ - libubsan
+ - libudev
+ - liburing
+ - libusbx
+ - libvdeplug
+ - libxml2
+ - libzstd
+ - llvm
+ - lttng-ust
+ - lzo
+ - netcat
+ - nettle
+ - ninja
+ - nsis
+ - make
+ - mesa-libgbm
+ - meson
+ - ncursesw
+ - pam
+ - pcre-static
+ - perl
+ - perl-Test-Harness
+ - pixman
+ - pkg-config
+ - pulseaudio
+ - python3
+ - python3-PyYAML
+ - python3-numpy
+ - python3-opencv
+ - python3-pillow
+ - python3-pip
+ - python3-sphinx
+ - python3-sphinx-rtd-theme
+ - python3-virtualenv
+ - rpm2cpio
+ - sdl2
+ - sdl2-image
+ - sed
+ - snappy
+ - sparse
+ - spice-protocol
+ - spice-server
+ - ssh-client
+ - systemd
+ - tar
+ - tesseract
+ - tesseract-eng
+ - texinfo
+ - usbredir
+ - virglrenderer
+ - vte
+ - which
+ - xen
+ - xfsprogs
+ - zlib
+ - zlib-static
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
new file mode 100755
index 0000000000..b47e25f64b
--- /dev/null
+++ b/tests/lcitool/refresh
@@ -0,0 +1,67 @@
+#!/usr/bin/python3
+#
+# Re-generate container recipes
+#
+# This script uses the "lcitool" available from
+#
+# https://gitlab.com/libvirt/libvirt-ci
+#
+# Copyright (c) 2020 Red Hat Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import sys
+import os
+import subprocess
+
+from pathlib import Path
+
+if len(sys.argv) != 1:
+ print("syntax: %s" % sys.argv[0], file=sys.stderr)
+ sys.exit(1)
+
+self_dir = Path(__file__).parent
+src_dir = self_dir.parent.parent
+dockerfiles_dir = Path(src_dir, "tests", "docker", "dockerfiles")
+
+lcitool_path = Path(self_dir, "libvirt-ci", "lcitool")
+
+lcitool_cmd = [lcitool_path, "--data-dir", self_dir]
+
+def atomic_write(filename, content):
+ tmp = filename.with_suffix(filename.suffix + ".tmp")
+ try:
+ with tmp.open("w") as fp:
+ print(content, file=fp, end="")
+ tmp.rename(filename)
+ except Exception as ex:
+ tmp.unlink()
+ raise
+
+def generate(filename, cmd, trailer):
+ print("Generate %s" % filename)
+ lcitool=subprocess.run(cmd, capture_output=True)
+
+ if lcitool.returncode != 0:
+ raise Exception("Failed to generate %s: %s" % (filename, lcitool.stderr))
+
+ content = lcitool.stdout.decode("utf8")
+ if trailer is not None:
+ content += trailer
+ atomic_write(filename, content)
+
+def generate_dockerfile(host, target, cross=None, trailer=None):
+ filename = Path(src_dir, "tests", "docker", "dockerfiles", host + ".docker")
+ cmd = lcitool_cmd + ["dockerfile"]
+ if cross is not None:
+ cmd.extend(["--cross", cross])
+ cmd.extend([target, "qemu"])
+ generate(filename, cmd, trailer)
+
+try:
+ sys.exit(0)
+except Exception as ex:
+ print(str(ex), file=sys.stderr)
+ sys.exit(1)
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 08/18] tests/docker: auto-generate centos8.docker with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (6 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 07/18] tests: integrate lcitool for generating build env manifests Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 09/18] tests/docker: auto-generate fedora.docker " Daniel P. Berrangé
` (10 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This commit is best examined using the "-b" option to diff.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/centos8.docker | 243 +++++++++++++-----------
tests/lcitool/refresh | 2 +
2 files changed, 135 insertions(+), 110 deletions(-)
diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker
index 7f135f8e8c..3c62b62a99 100644
--- a/tests/docker/dockerfiles/centos8.docker
+++ b/tests/docker/dockerfiles/centos8.docker
@@ -1,112 +1,135 @@
-FROM docker.io/centos:8
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool dockerfile centos-8 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
-RUN dnf -y update
-ENV PACKAGES \
- SDL2-devel \
- alsa-lib-devel \
- bc \
- brlapi-devel \
- bzip2 \
- bzip2-devel \
- ca-certificates \
- capstone-devel \
- ccache \
- clang \
- ctags \
- cyrus-sasl-devel \
- daxctl-devel \
- dbus-daemon \
- device-mapper-multipath-devel \
- diffutils \
- findutils \
- gcc \
- gcc-c++ \
- genisoimage \
- gettext \
- git \
- glib2-devel \
- glibc-langpack-en \
- glibc-static \
- glusterfs-api-devel \
- gnutls-devel \
- gtk3-devel \
- hostname \
- jemalloc-devel \
- libaio-devel \
- libasan \
- libattr-devel \
- libbpf-devel \
- libcacard-devel \
- libcap-ng-devel \
- libcurl-devel \
- libdrm-devel \
- libepoxy-devel \
- libfdt-devel \
- libffi-devel \
- libgcrypt-devel \
- libiscsi-devel \
- libjpeg-devel \
- libnfs-devel \
- libpmem-devel \
- libpng-devel \
- librbd-devel \
- libseccomp-devel \
- libselinux-devel \
- libslirp-devel \
- libssh-devel \
- libtasn1-devel \
- libubsan \
- libudev-devel \
- libusbx-devel \
- libxml2-devel \
- libzstd-devel \
- llvm \
- lzo-devel \
- make \
- mesa-libgbm-devel \
- ncurses-devel \
- nettle-devel \
- ninja-build \
- nmap-ncat \
- numactl-devel \
- openssh-clients \
- pam-devel \
- perl \
- perl-Test-Harness \
- pixman-devel \
- pkgconfig \
- pulseaudio-libs-devel \
- python3 \
- python3-PyYAML \
- python3-numpy \
- python3-pillow \
- python3-pip \
- python3-setuptools \
- python3-sphinx \
- python3-sphinx_rtd_theme \
- python3-virtualenv \
- python3-wheel \
- rdma-core-devel \
- rpm \
- sed \
- snappy-devel \
- spice-protocol \
- spice-server-devel \
- systemd-devel \
- systemtap-sdt-devel \
- tar \
- texinfo \
- usbredir-devel \
- util-linux \
- virglrenderer-devel \
- vte291-devel \
- which \
- xfsprogs-devel \
- zlib-devel
+FROM docker.io/library/centos:8
-RUN dnf install -y dnf-plugins-core && \
- dnf config-manager --set-enabled powertools && \
- dnf install -y centos-release-advanced-virtualization && \
- dnf install -y epel-release && \
- dnf install -y $PACKAGES
-RUN rpm -q $PACKAGES | sort > /packages.txt
+RUN dnf update -y && \
+ dnf install 'dnf-command(config-manager)' -y && \
+ dnf config-manager --set-enabled -y powertools && \
+ dnf install -y centos-release-advanced-virtualization && \
+ dnf install -y epel-release && \
+ dnf install -y \
+ SDL2-devel \
+ alsa-lib-devel \
+ bash \
+ bc \
+ brlapi-devel \
+ bzip2 \
+ bzip2-devel \
+ ca-certificates \
+ capstone-devel \
+ ccache \
+ clang \
+ ctags \
+ cyrus-sasl-devel \
+ daxctl-devel \
+ dbus-daemon \
+ device-mapper-multipath-devel \
+ diffutils \
+ findutils \
+ gcc \
+ gcc-c++ \
+ genisoimage \
+ gettext \
+ git \
+ glib2-devel \
+ glib2-static \
+ glibc-langpack-en \
+ glibc-static \
+ glusterfs-api-devel \
+ gnutls-devel \
+ gtk3-devel \
+ hostname \
+ jemalloc-devel \
+ libaio-devel \
+ libasan \
+ libattr-devel \
+ libbpf-devel \
+ libcacard-devel \
+ libcap-ng-devel \
+ libcurl-devel \
+ libdrm-devel \
+ libepoxy-devel \
+ libfdt-devel \
+ libffi-devel \
+ libgcrypt-devel \
+ libiscsi-devel \
+ libjpeg-devel \
+ libnfs-devel \
+ libpmem-devel \
+ libpng-devel \
+ librbd-devel \
+ libseccomp-devel \
+ libselinux-devel \
+ libslirp-devel \
+ libssh-devel \
+ libtasn1-devel \
+ libubsan \
+ libudev-devel \
+ liburing-devel \
+ libusbx-devel \
+ libxml2-devel \
+ libzstd-devel \
+ llvm \
+ lttng-ust-devel \
+ lzo-devel \
+ make \
+ mesa-libgbm-devel \
+ meson \
+ ncurses-devel \
+ nettle-devel \
+ ninja-build \
+ nmap-ncat \
+ numactl-devel \
+ openssh-clients \
+ pam-devel \
+ pcre-static \
+ perl \
+ perl-Test-Harness \
+ pixman-devel \
+ pkgconfig \
+ pulseaudio-libs-devel \
+ python3 \
+ python3-PyYAML \
+ python3-numpy \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx_rtd_theme \
+ python3-virtualenv \
+ rdma-core-devel \
+ rpm \
+ sed \
+ snappy-devel \
+ spice-protocol \
+ spice-server-devel \
+ systemd-devel \
+ systemtap-sdt-devel \
+ tar \
+ texinfo \
+ usbredir-devel \
+ util-linux \
+ virglrenderer-devel \
+ vte291-devel \
+ which \
+ xfsprogs-devel \
+ zlib-devel \
+ zlib-static && \
+ dnf autoremove -y && \
+ dnf clean all -y && \
+ rpm -qa | sort > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index b47e25f64b..824b1c0ef2 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -61,6 +61,8 @@ def generate_dockerfile(host, target, cross=None, trailer=None):
generate(filename, cmd, trailer)
try:
+ generate_dockerfile("centos8", "centos-8")
+
sys.exit(0)
except Exception as ex:
print(str(ex), file=sys.stderr)
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 09/18] tests/docker: auto-generate fedora.docker with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (7 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 08/18] tests/docker: auto-generate centos8.docker with lcitool Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 10/18] tests/docker: auto-generate ubuntu1804.docker " Daniel P. Berrangé
` (9 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This commit is best examined using the "-b" option to diff.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/fedora.docker | 260 ++++++++++++++-----------
tests/lcitool/refresh | 1 +
2 files changed, 146 insertions(+), 115 deletions(-)
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 855aefaac5..6784878b56 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -1,118 +1,148 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool dockerfile fedora-35 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
FROM registry.fedoraproject.org/fedora:35
-# Please keep this list sorted alphabetically
-ENV PACKAGES \
- SDL2-devel \
- SDL2_image-devel \
- alsa-lib-devel \
- bc \
- brlapi-devel \
- bzip2 \
- bzip2-devel \
- ca-certificates \
- capstone-devel \
- ccache \
- clang \
- ctags \
- cyrus-sasl-devel \
- daxctl-devel \
- dbus-daemon \
- device-mapper-multipath-devel \
- diffutils \
- findutils \
- gcc \
- gcc-c++ \
- gcovr \
- genisoimage \
- gettext \
- git \
- glib2-devel \
- glibc-langpack-en \
- glibc-static \
- glusterfs-api-devel \
- gnutls-devel \
- gtk3-devel \
- hostname \
- jemalloc-devel \
- libaio-devel \
- libasan \
- libattr-devel \
- libbpf-devel \
- libcacard-devel \
- libcap-ng-devel \
- libcurl-devel \
- libdrm-devel \
- libepoxy-devel \
- libfdt-devel \
- libffi-devel \
- libgcrypt-devel \
- libiscsi-devel \
- libjpeg-devel \
- libnfs-devel \
- libpmem-devel \
- libpng-devel \
- librbd-devel \
- libseccomp-devel \
- libselinux-devel \
- libslirp-devel \
- libssh-devel \
- libtasn1-devel \
- libubsan \
- libudev-devel \
- liburing-devel \
- libusbx-devel \
- libxml2-devel \
- libzstd-devel \
- llvm \
- lttng-ust-devel \
- lzo-devel \
- make \
- mesa-libgbm-devel \
- meson \
- ncurses-devel \
- nettle-devel \
- ninja-build \
- nmap-ncat \
- numactl-devel \
- openssh-clients \
- pam-devel \
- perl-Test-Harness \
- perl-base \
- pixman-devel \
- pkgconfig \
- pulseaudio-libs-devel \
- python3 \
- python3-PyYAML \
- python3-numpy \
- python3-opencv \
- python3-pillow \
- python3-pip \
- python3-sphinx \
- python3-sphinx_rtd_theme \
- python3-virtualenv \
- rdma-core-devel \
- rpm \
- sed \
- snappy-devel \
- sparse \
- spice-protocol \
- spice-server-devel \
- systemd-devel \
- systemtap-sdt-devel \
- tar \
- tesseract \
- tesseract-langpack-eng \
- texinfo \
- usbredir-devel \
- util-linux \
- virglrenderer-devel \
- vte291-devel \
- which \
- xen-devel \
- xfsprogs-devel \
- zlib-devel
-ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
+RUN dnf install -y nosync && \
+ echo -e '#!/bin/sh\n\
+if test -d /usr/lib64\n\
+then\n\
+ export LD_PRELOAD=/usr/lib64/nosync/nosync.so\n\
+else\n\
+ export LD_PRELOAD=/usr/lib/nosync/nosync.so\n\
+fi\n\
+exec "$@"' > /usr/bin/nosync && \
+ chmod +x /usr/bin/nosync && \
+ nosync dnf update -y && \
+ nosync dnf install -y \
+ SDL2-devel \
+ SDL2_image-devel \
+ alsa-lib-devel \
+ bash \
+ bc \
+ brlapi-devel \
+ bzip2 \
+ bzip2-devel \
+ ca-certificates \
+ capstone-devel \
+ ccache \
+ clang \
+ ctags \
+ cyrus-sasl-devel \
+ daxctl-devel \
+ dbus-daemon \
+ device-mapper-multipath-devel \
+ diffutils \
+ findutils \
+ gcc \
+ gcc-c++ \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ glib2-devel \
+ glib2-static \
+ glibc-langpack-en \
+ glibc-static \
+ glusterfs-api-devel \
+ gnutls-devel \
+ gtk3-devel \
+ hostname \
+ jemalloc-devel \
+ libaio-devel \
+ libasan \
+ libattr-devel \
+ libbpf-devel \
+ libcacard-devel \
+ libcap-ng-devel \
+ libcurl-devel \
+ libdrm-devel \
+ libepoxy-devel \
+ libfdt-devel \
+ libffi-devel \
+ libgcrypt-devel \
+ libiscsi-devel \
+ libjpeg-devel \
+ libnfs-devel \
+ libpmem-devel \
+ libpng-devel \
+ librbd-devel \
+ libseccomp-devel \
+ libselinux-devel \
+ libslirp-devel \
+ libssh-devel \
+ libtasn1-devel \
+ libubsan \
+ libudev-devel \
+ liburing-devel \
+ libusbx-devel \
+ libxml2-devel \
+ libzstd-devel \
+ llvm \
+ lttng-ust-devel \
+ lzo-devel \
+ make \
+ mesa-libgbm-devel \
+ meson \
+ ncurses-devel \
+ nettle-devel \
+ ninja-build \
+ nmap-ncat \
+ numactl-devel \
+ openssh-clients \
+ pam-devel \
+ pcre-static \
+ perl-Test-Harness \
+ perl-base \
+ pixman-devel \
+ pkgconfig \
+ pulseaudio-libs-devel \
+ python3 \
+ python3-PyYAML \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx_rtd_theme \
+ python3-virtualenv \
+ rdma-core-devel \
+ rpm \
+ sed \
+ snappy-devel \
+ sparse \
+ spice-protocol \
+ spice-server-devel \
+ systemd-devel \
+ systemtap-sdt-devel \
+ tar \
+ tesseract \
+ tesseract-langpack-eng \
+ texinfo \
+ usbredir-devel \
+ util-linux \
+ virglrenderer-devel \
+ vte291-devel \
+ which \
+ xen-devel \
+ xfsprogs-devel \
+ zlib-devel \
+ zlib-static && \
+ nosync dnf autoremove -y && \
+ nosync dnf clean all -y && \
+ rpm -qa | sort > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
-RUN dnf install -y $PACKAGES
-RUN rpm -q $PACKAGES | sort > /packages.txt
-ENV PATH $PATH:/usr/libexec/python3-sphinx/
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 824b1c0ef2..d79d2b8c06 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -62,6 +62,7 @@ def generate_dockerfile(host, target, cross=None, trailer=None):
try:
generate_dockerfile("centos8", "centos-8")
+ generate_dockerfile("fedora", "fedora-35")
sys.exit(0)
except Exception as ex:
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 10/18] tests/docker: auto-generate ubuntu1804.docker with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (8 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 09/18] tests/docker: auto-generate fedora.docker " Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 11/18] tests/docker: auto-generate ubuntu2004.docker " Daniel P. Berrangé
` (8 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This commit is best examined using the "-b" option to diff.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/ubuntu1804.docker | 255 ++++++++++++---------
tests/lcitool/refresh | 7 +
2 files changed, 149 insertions(+), 113 deletions(-)
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index 450fd06d0d..0ffa3c4d4b 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -1,117 +1,146 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool dockerfile ubuntu-1804 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
FROM docker.io/library/ubuntu:18.04
-ENV PACKAGES \
- bc \
- bsdmainutils \
- bzip2 \
- ca-certificates \
- ccache \
- clang \
- dbus \
- debianutils \
- diffutils \
- exuberant-ctags \
- findutils \
- g++ \
- gcc \
- gcovr \
- genisoimage \
- gettext \
- git \
- glusterfs-common \
- hostname \
- libaio-dev \
- libasan5 \
- libasound2-dev \
- libattr1-dev \
- libbrlapi-dev \
- libbz2-dev \
- libc6-dev \
- libcacard-dev \
- libcap-ng-dev \
- libcapstone-dev \
- libcurl4-gnutls-dev \
- libdaxctl-dev \
- libdrm-dev \
- libepoxy-dev \
- libfdt-dev \
- libffi-dev \
- libgbm-dev \
- libgcrypt20-dev \
- libglib2.0-dev \
- libgnutls28-dev \
- libgtk-3-dev \
- libibverbs-dev \
- libiscsi-dev \
- libjemalloc-dev \
- libjpeg-turbo8-dev \
- liblttng-ust-dev \
- liblzo2-dev \
- libncursesw5-dev \
- libnfs-dev \
- libnuma-dev \
- libpam0g-dev \
- libpixman-1-dev \
- libpmem-dev \
- libpng-dev \
- libpulse-dev \
- librbd-dev \
- librdmacm-dev \
- libsasl2-dev \
- libsdl2-dev \
- libsdl2-image-dev \
- libseccomp-dev \
- libselinux-dev \
- libsnappy-dev \
- libspice-protocol-dev \
- libspice-server-dev \
- libssh-dev \
- libsystemd-dev \
- libtasn1-6-dev \
- libtest-harness-perl \
- libubsan1 \
- libudev-dev \
- libusb-1.0-0-dev \
- libusbredirhost-dev \
- libvdeplug-dev \
- libvirglrenderer-dev \
- libvte-2.91-dev \
- libxen-dev \
- libxml2-dev \
- libzstd-dev \
- llvm \
- locales \
- make \
- multipath-tools \
- netcat-openbsd \
- nettle-dev \
- ninja-build \
- openssh-client \
- perl-base \
- pkgconf \
- python3 \
- python3-numpy \
- python3-opencv \
- python3-pillow \
- python3-pip \
- python3-setuptools \
- python3-sphinx \
- python3-sphinx-rtd-theme \
- python3-venv \
- python3-wheel \
- python3-yaml \
- rpm2cpio \
- sed \
- sparse \
- systemtap-sdt-dev \
- tar \
- tesseract-ocr \
- tesseract-ocr-eng \
- texinfo \
- xfslibs-dev \
- zlib1g-dev
-RUN apt-get update && \
- DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
-RUN dpkg -l $PACKAGES | sort > /packages.txt
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdmainutils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ clang \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ g++ \
+ gcc \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ glusterfs-common \
+ hostname \
+ libaio-dev \
+ libasan5 \
+ libasound2-dev \
+ libattr1-dev \
+ libbrlapi-dev \
+ libbz2-dev \
+ libc6-dev \
+ libcacard-dev \
+ libcap-ng-dev \
+ libcapstone-dev \
+ libcurl4-gnutls-dev \
+ libdaxctl-dev \
+ libdrm-dev \
+ libepoxy-dev \
+ libfdt-dev \
+ libffi-dev \
+ libgbm-dev \
+ libgcrypt20-dev \
+ libglib2.0-dev \
+ libgnutls28-dev \
+ libgtk-3-dev \
+ libibverbs-dev \
+ libiscsi-dev \
+ libjemalloc-dev \
+ libjpeg-turbo8-dev \
+ liblttng-ust-dev \
+ liblzo2-dev \
+ libncursesw5-dev \
+ libnfs-dev \
+ libnuma-dev \
+ libpam0g-dev \
+ libpcre2-dev \
+ libpixman-1-dev \
+ libpmem-dev \
+ libpng-dev \
+ libpulse-dev \
+ librbd-dev \
+ librdmacm-dev \
+ libsasl2-dev \
+ libsdl2-dev \
+ libsdl2-image-dev \
+ libseccomp-dev \
+ libselinux1-dev \
+ libsnappy-dev \
+ libspice-protocol-dev \
+ libspice-server-dev \
+ libssh-dev \
+ libsystemd-dev \
+ libtasn1-6-dev \
+ libtest-harness-perl \
+ libubsan1 \
+ libudev-dev \
+ libusb-1.0-0-dev \
+ libusbredirhost-dev \
+ libvdeplug-dev \
+ libvirglrenderer-dev \
+ libvte-2.91-dev \
+ libxen-dev \
+ libxml2-dev \
+ libzstd-dev \
+ llvm \
+ locales \
+ make \
+ multipath-tools \
+ netcat-openbsd \
+ nettle-dev \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-setuptools \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-wheel \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ systemtap-sdt-dev \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo \
+ xfslibs-dev \
+ zlib1g-dev && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
+
+RUN pip3 install \
+ meson==0.56.0
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
# https://bugs.launchpad.net/qemu/+bug/1838763
ENV QEMU_CONFIGURE_OPTS --disable-libssh
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index d79d2b8c06..1e30674d67 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -60,9 +60,16 @@ def generate_dockerfile(host, target, cross=None, trailer=None):
cmd.extend([target, "qemu"])
generate(filename, cmd, trailer)
+ubuntu1804_skipssh = [
+ "# https://bugs.launchpad.net/qemu/+bug/1838763\n",
+ "ENV QEMU_CONFIGURE_OPTS --disable-libssh\n"
+]
+
try:
generate_dockerfile("centos8", "centos-8")
generate_dockerfile("fedora", "fedora-35")
+ generate_dockerfile("ubuntu1804", "ubuntu-1804",
+ trailer="".join(ubuntu1804_skipssh))
sys.exit(0)
except Exception as ex:
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 11/18] tests/docker: auto-generate ubuntu2004.docker with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (9 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 10/18] tests/docker: auto-generate ubuntu1804.docker " Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 12/18] tests/docker: auto-generate opensuse-leap.docker " Daniel P. Berrangé
` (7 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This commit is best examined using the "-b" option to diff.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/ubuntu2004.docker | 257 ++++++++++++---------
tests/lcitool/refresh | 9 +-
2 files changed, 151 insertions(+), 115 deletions(-)
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 15a026be09..40402b91fe 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -1,119 +1,148 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool dockerfile ubuntu-2004 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
FROM docker.io/library/ubuntu:20.04
-ENV PACKAGES \
- bc \
- bsdmainutils \
- bzip2 \
- ca-certificates \
- ccache \
- clang \
- dbus \
- debianutils \
- diffutils \
- exuberant-ctags \
- findutils \
- g++ \
- gcc \
- gcovr \
- genisoimage \
- gettext \
- git \
- hostname \
- libaio-dev \
- libasan5 \
- libasound2-dev \
- libattr1-dev \
- libbrlapi-dev \
- libbz2-dev \
- libc6-dev \
- libcacard-dev \
- libcap-ng-dev \
- libcapstone-dev \
- libcurl4-gnutls-dev \
- libdaxctl-dev \
- libdrm-dev \
- libepoxy-dev \
- libfdt-dev \
- libffi-dev \
- libgbm-dev \
- libgcrypt20-dev \
- libglib2.0-dev \
- libglusterfs-dev \
- libgnutls28-dev \
- libgtk-3-dev \
- libibverbs-dev \
- libiscsi-dev \
- libjemalloc-dev \
- libjpeg-turbo8-dev \
- liblttng-ust-dev \
- liblzo2-dev \
- libncursesw5-dev \
- libnfs-dev \
- libnuma-dev \
- libpam0g-dev \
- libpixman-1-dev \
- libpmem-dev \
- libpng-dev \
- libpulse-dev \
- librbd-dev \
- librdmacm-dev \
- libsasl2-dev \
- libsdl2-dev \
- libsdl2-image-dev \
- libseccomp-dev \
- libselinux-dev \
- libslirp-dev \
- libsnappy-dev \
- libspice-protocol-dev \
- libspice-server-dev \
- libssh-dev \
- libsystemd-dev \
- libtasn1-6-dev \
- libtest-harness-perl \
- libubsan1 \
- libudev-dev \
- libusb-1.0-0-dev \
- libusbredirhost-dev \
- libvdeplug-dev \
- libvirglrenderer-dev \
- libvte-2.91-dev \
- libxen-dev \
- libxml2-dev \
- libzstd-dev \
- llvm \
- locales \
- make \
- multipath-tools \
- ncat \
- nettle-dev \
- ninja-build \
- openssh-client \
- perl-base \
- pkgconf \
- python3 \
- python3-numpy \
- python3-opencv \
- python3-pillow \
- python3-pip \
- python3-setuptools \
- python3-sphinx \
- python3-sphinx-rtd-theme \
- python3-venv \
- python3-wheel \
- python3-yaml \
- rpm2cpio \
- sed \
- sparse \
- systemtap-sdt-dev \
- tar \
- tesseract-ocr \
- tesseract-ocr-eng \
- texinfo \
- xfslibs-dev \
- zlib1g-dev
-RUN apt-get update && \
- DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
-RUN dpkg -l $PACKAGES | sort > /packages.txt
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdmainutils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ clang \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ g++ \
+ gcc \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libaio-dev \
+ libasan5 \
+ libasound2-dev \
+ libattr1-dev \
+ libbrlapi-dev \
+ libbz2-dev \
+ libc6-dev \
+ libcacard-dev \
+ libcap-ng-dev \
+ libcapstone-dev \
+ libcurl4-gnutls-dev \
+ libdaxctl-dev \
+ libdrm-dev \
+ libepoxy-dev \
+ libfdt-dev \
+ libffi-dev \
+ libgbm-dev \
+ libgcrypt20-dev \
+ libglib2.0-dev \
+ libglusterfs-dev \
+ libgnutls28-dev \
+ libgtk-3-dev \
+ libibverbs-dev \
+ libiscsi-dev \
+ libjemalloc-dev \
+ libjpeg-turbo8-dev \
+ liblttng-ust-dev \
+ liblzo2-dev \
+ libncursesw5-dev \
+ libnfs-dev \
+ libnuma-dev \
+ libpam0g-dev \
+ libpcre2-dev \
+ libpixman-1-dev \
+ libpmem-dev \
+ libpng-dev \
+ libpulse-dev \
+ librbd-dev \
+ librdmacm-dev \
+ libsasl2-dev \
+ libsdl2-dev \
+ libsdl2-image-dev \
+ libseccomp-dev \
+ libselinux1-dev \
+ libslirp-dev \
+ libsnappy-dev \
+ libspice-protocol-dev \
+ libspice-server-dev \
+ libssh-dev \
+ libsystemd-dev \
+ libtasn1-6-dev \
+ libtest-harness-perl \
+ libubsan1 \
+ libudev-dev \
+ libusb-1.0-0-dev \
+ libusbredirhost-dev \
+ libvdeplug-dev \
+ libvirglrenderer-dev \
+ libvte-2.91-dev \
+ libxen-dev \
+ libxml2-dev \
+ libzstd-dev \
+ llvm \
+ locales \
+ make \
+ multipath-tools \
+ ncat \
+ nettle-dev \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-setuptools \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-wheel \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ systemtap-sdt-dev \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo \
+ xfslibs-dev \
+ zlib1g-dev && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
+
+RUN pip3 install \
+ meson==0.56.0
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
# Apply patch https://reviews.llvm.org/D75820
# This is required for TSan in clang-10 to compile with QEMU.
RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 1e30674d67..310bad1315 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -65,12 +65,19 @@ ubuntu1804_skipssh = [
"ENV QEMU_CONFIGURE_OPTS --disable-libssh\n"
]
+ubuntu2004_tsanhack = [
+ "# Apply patch https://reviews.llvm.org/D75820\n",
+ "# This is required for TSan in clang-10 to compile with QEMU.\n",
+ "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n"
+]
+
try:
generate_dockerfile("centos8", "centos-8")
generate_dockerfile("fedora", "fedora-35")
generate_dockerfile("ubuntu1804", "ubuntu-1804",
trailer="".join(ubuntu1804_skipssh))
-
+ generate_dockerfile("ubuntu2004", "ubuntu-2004",
+ trailer="".join(ubuntu2004_tsanhack))
sys.exit(0)
except Exception as ex:
print(str(ex), file=sys.stderr)
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 12/18] tests/docker: auto-generate opensuse-leap.docker with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (10 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 11/18] tests/docker: auto-generate ubuntu2004.docker " Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 13/18] tests/docker: remove ubuntu.docker container Daniel P. Berrangé
` (6 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This commit is best examined using the "-b" option to diff.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/opensuse-leap.docker | 245 ++++++++++--------
tests/lcitool/refresh | 1 +
2 files changed, 135 insertions(+), 111 deletions(-)
diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
index 3bbdb67f4f..5510bdf19c 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -1,114 +1,137 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool dockerfile opensuse-leap-152 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
FROM registry.opensuse.org/opensuse/leap:15.2
-# Please keep this list sorted alphabetically
-ENV PACKAGES \
- Mesa-devel \
- alsa-lib-devel \
- bc \
- brlapi-devel \
- bzip2 \
- ca-certificates \
- ccache \
- clang \
- ctags \
- cyrus-sasl-devel \
- dbus-1 \
- diffutils \
- findutils \
- gcc \
- gcc-c++ \
- gcovr \
- gettext-runtime \
- git \
- glib2-devel \
- glibc-locale \
- glibc-static \
- glusterfs-devel \
- gtk3-devel \
- hostname \
- jemalloc-devel \
- libSDL2-devel \
- libSDL2_image-devel \
- libaio-devel \
- libasan6 \
- libattr-devel \
- libbpf-devel \
- libbz2-devel \
- libcacard-devel \
- libcap-ng-devel \
- libcurl-devel \
- libdrm-devel \
- libepoxy-devel \
- libfdt-devel \
- libffi-devel \
- libgcrypt-devel \
- libgnutls-devel \
- libiscsi-devel \
- libjpeg8-devel \
- libndctl-devel \
- libnettle-devel \
- libnfs-devel \
- libnuma-devel \
- libpixman-1-0-devel \
- libpmem-devel \
- libpng16-devel \
- libpulse-devel \
- librbd-devel \
- libseccomp-devel \
- libselinux-devel \
- libspice-server-devel \
- libssh-devel \
- libtasn1-devel \
- libubsan1 \
- libudev-devel \
- libusb-1_0-devel \
- libxml2-devel \
- libzstd-devel \
- llvm \
- lttng-ust-devel \
- lzo-devel \
- make \
- mkisofs \
- ncat \
- ncurses-devel \
- ninja \
- openssh \
- pam-devel \
- perl-Test-Harness \
- perl-base \
- pkgconfig \
- python3-Pillow \
- python3-PyYAML \
- python3-Sphinx \
- python3-base \
- python3-numpy \
- python3-opencv \
- python3-pip \
- python3-setuptools \
- python3-sphinx_rtd_theme \
- python3-virtualenv \
- python3-wheel \
- rdma-core-devel \
- rpm \
- sed \
- snappy-devel \
- sparse \
- spice-protocol-devel \
- systemd-devel \
- systemtap-sdt-devel \
- tar \
- tesseract-ocr \
- tesseract-ocr-traineddata-english \
- texinfo \
- usbredir-devel \
- util-linux \
- virglrenderer-devel \
- vte-devel \
- which \
- xen-devel \
- xfsprogs-devel \
- zlib-devel
-ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3.6
+RUN zypper update -y && \
+ zypper install -y \
+ Mesa-devel \
+ alsa-lib-devel \
+ bash \
+ bc \
+ brlapi-devel \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ clang \
+ ctags \
+ cyrus-sasl-devel \
+ dbus-1 \
+ diffutils \
+ findutils \
+ gcc \
+ gcc-c++ \
+ gcovr \
+ gettext-runtime \
+ git \
+ glib2-devel \
+ glibc-locale \
+ glibc-static \
+ glusterfs-devel \
+ gtk3-devel \
+ hostname \
+ jemalloc-devel \
+ libSDL2-devel \
+ libSDL2_image-devel \
+ libaio-devel \
+ libasan6 \
+ libattr-devel \
+ libbpf-devel \
+ libbz2-devel \
+ libcacard-devel \
+ libcap-ng-devel \
+ libcurl-devel \
+ libdrm-devel \
+ libepoxy-devel \
+ libfdt-devel \
+ libffi-devel \
+ libgcrypt-devel \
+ libgnutls-devel \
+ libiscsi-devel \
+ libjpeg8-devel \
+ libndctl-devel \
+ libnettle-devel \
+ libnfs-devel \
+ libnuma-devel \
+ libpixman-1-0-devel \
+ libpmem-devel \
+ libpng16-devel \
+ libpulse-devel \
+ librbd-devel \
+ libseccomp-devel \
+ libselinux-devel \
+ libspice-server-devel \
+ libssh-devel \
+ libtasn1-devel \
+ libubsan1 \
+ libudev-devel \
+ liburing-devel \
+ libusb-1_0-devel \
+ libxml2-devel \
+ libzstd-devel \
+ llvm \
+ lttng-ust-devel \
+ lzo-devel \
+ make \
+ mkisofs \
+ ncat \
+ ncurses-devel \
+ ninja \
+ openssh \
+ pam-devel \
+ pcre-devel-static \
+ perl-Test-Harness \
+ perl-base \
+ pkgconfig \
+ python3-Pillow \
+ python3-PyYAML \
+ python3-Sphinx \
+ python3-base \
+ python3-numpy \
+ python3-opencv \
+ python3-pip \
+ python3-setuptools \
+ python3-sphinx_rtd_theme \
+ python3-virtualenv \
+ python3-wheel \
+ rdma-core-devel \
+ rpm \
+ sed \
+ snappy-devel \
+ sparse \
+ spice-protocol-devel \
+ systemd-devel \
+ systemtap-sdt-devel \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-traineddata-english \
+ texinfo \
+ usbredir-devel \
+ util-linux \
+ virglrenderer-devel \
+ vte-devel \
+ which \
+ xen-devel \
+ xfsprogs-devel \
+ zlib-devel \
+ zlib-devel-static && \
+ zypper clean --all && \
+ rpm -qa | sort > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
+
+RUN pip3 install \
+ meson==0.56.0
-RUN zypper update -y && zypper --non-interactive install -y $PACKAGES
-RUN rpm -q $PACKAGES | sort > /packages.txt
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 310bad1315..b8a69cee59 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -78,6 +78,7 @@ try:
trailer="".join(ubuntu1804_skipssh))
generate_dockerfile("ubuntu2004", "ubuntu-2004",
trailer="".join(ubuntu2004_tsanhack))
+ generate_dockerfile("opensuse-leap", "opensuse-leap-152")
sys.exit(0)
except Exception as ex:
print(str(ex), file=sys.stderr)
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 13/18] tests/docker: remove ubuntu.docker container
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (11 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 12/18] tests/docker: auto-generate opensuse-leap.docker " Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 14/18] .gitlab-ci.d/cirrus: auto-generate variables with lcitool Daniel P. Berrangé
` (5 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This duplicates the ubuntu2004 container but with an inconsistent set of
packages.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
.gitlab-ci.d/containers.yml | 5 --
tests/docker/dockerfiles/ubuntu.docker | 71 --------------------------
2 files changed, 76 deletions(-)
delete mode 100644 tests/docker/dockerfiles/ubuntu.docker
diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index cd06d3f5f4..b9b675fdcb 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -29,11 +29,6 @@ amd64-ubuntu2004-container:
variables:
NAME: ubuntu2004
-amd64-ubuntu-container:
- extends: .container_job_template
- variables:
- NAME: ubuntu
-
amd64-opensuse-leap-container:
extends: .container_job_template
variables:
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
deleted file mode 100644
index f0e0180d21..0000000000
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ /dev/null
@@ -1,71 +0,0 @@
-#
-# Latest Ubuntu Release
-#
-# Useful for testing against relatively bleeding edge libraries and
-# compilers. We also have seperate recipe for the most recent LTS
-# release.
-#
-# When updating use the full tag not :latest otherwise the build
-# system won't pick up that it has changed.
-#
-
-FROM docker.io/library/ubuntu:20.04
-ENV PACKAGES \
- ccache \
- clang \
- dbus \
- gcc \
- gettext \
- git \
- glusterfs-common \
- libaio-dev \
- libattr1-dev \
- libbrlapi-dev \
- libbz2-dev \
- libcacard-dev \
- libcap-ng-dev \
- libcurl4-gnutls-dev \
- libdrm-dev \
- libepoxy-dev \
- libfdt-dev \
- libffi-dev \
- libgbm-dev \
- libgnutls28-dev \
- libgtk-3-dev \
- libibverbs-dev \
- libiscsi-dev \
- libjemalloc-dev \
- libjpeg-turbo8-dev \
- liblzo2-dev \
- libncurses5-dev \
- libncursesw5-dev \
- libnfs-dev \
- libnuma-dev \
- libpixman-1-dev \
- libpng-dev \
- librados-dev \
- librbd-dev \
- librdmacm-dev \
- libsasl2-dev \
- libsdl2-dev \
- libseccomp-dev \
- libsnappy-dev \
- libspice-protocol-dev \
- libspice-server-dev \
- libssh-dev \
- libusb-1.0-0-dev \
- libusbredirhost-dev \
- libvdeplug-dev \
- libvte-2.91-dev \
- libxen-dev \
- libzstd-dev \
- make \
- ninja-build \
- python3-yaml \
- python3-sphinx \
- python3-sphinx-rtd-theme \
- sparse \
- xfslibs-dev
-RUN apt-get update && \
- DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
-RUN dpkg -l $PACKAGES | sort > /packages.txt
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 14/18] .gitlab-ci.d/cirrus: auto-generate variables with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (12 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 13/18] tests/docker: remove ubuntu.docker container Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 15/18] tests/docker: updates to alpine package list Daniel P. Berrangé
` (4 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
The current Cirrus CI variables files were previously generated by using
lcitool. This change wires them up to the refresh script to make that
link explicit.
This changes the package list because libvirt-ci now knows about the
mapping for dtc on FreeBSD and macOS platforms.
The variables are also now emit in sorted order for stability across
runs.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
.gitlab-ci.d/cirrus/freebsd-12.vars | 11 +++++++----
.gitlab-ci.d/cirrus/freebsd-13.vars | 11 +++++++----
.gitlab-ci.d/cirrus/macos-11.vars | 11 ++++++-----
tests/lcitool/refresh | 10 ++++++++++
4 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/.gitlab-ci.d/cirrus/freebsd-12.vars b/.gitlab-ci.d/cirrus/freebsd-12.vars
index 2099b21354..9c52266811 100644
--- a/.gitlab-ci.d/cirrus/freebsd-12.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-12.vars
@@ -2,12 +2,15 @@
#
# $ lcitool variables freebsd-12 qemu
#
-# https://gitlab.com/libvirt/libvirt-ci/-/commit/c7e275ab27ac0dcd09da290817b9adeea1fd1eb1
+# https://gitlab.com/libvirt/libvirt-ci
-PACKAGING_COMMAND='pkg'
CCACHE='/usr/local/bin/ccache'
+CPAN_PKGS=''
+CROSS_PKGS=''
MAKE='/usr/local/bin/gmake'
NINJA='/usr/local/bin/ninja'
-PYTHON='/usr/local/bin/python3'
+PACKAGING_COMMAND='pkg'
PIP3='/usr/local/bin/pip-3.8'
-PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage ctags curl cyrus-sasl dbus diffutils gettext git glib gmake gnutls gsed gtk3 libepoxy libffi libgcrypt libjpeg-turbo libnfs libspice-server libssh libtasn1 libxml2 llvm lttng-ust lzo2 meson ncurses nettle ninja opencv p5-Test-Harness perl5 pixman pkgconf png py38-numpy py38-pillow py38-pip py38-sphinx py38-sphinx_rtd_theme py38-virtualenv py38-yaml python3 rpm2cpio sdl2 sdl2_image snappy spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
+PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage ctags curl cyrus-sasl dbus diffutils dtc gettext git glib gmake gnutls gsed gtk3 libepoxy libffi libgcrypt libjpeg-turbo libnfs libspice-server libssh libtasn1 libxml2 llvm lttng-ust lzo2 meson ncurses nettle ninja opencv p5-Test-Harness perl5 pixman pkgconf png py38-numpy py38-pillow py38-pip py38-sphinx py38-sphinx_rtd_theme py38-virtualenv py38-yaml python3 rpm2cpio sdl2 sdl2_image snappy spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
+PYPI_PKGS=''
+PYTHON='/usr/local/bin/python3'
diff --git a/.gitlab-ci.d/cirrus/freebsd-13.vars b/.gitlab-ci.d/cirrus/freebsd-13.vars
index 323fe806d5..7b44dba324 100644
--- a/.gitlab-ci.d/cirrus/freebsd-13.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-13.vars
@@ -2,12 +2,15 @@
#
# $ lcitool variables freebsd-13 qemu
#
-# https://gitlab.com/libvirt/libvirt-ci/-/commit/c7e275ab27ac0dcd09da290817b9adeea1fd1eb1
+# https://gitlab.com/libvirt/libvirt-ci
-PACKAGING_COMMAND='pkg'
CCACHE='/usr/local/bin/ccache'
+CPAN_PKGS=''
+CROSS_PKGS=''
MAKE='/usr/local/bin/gmake'
NINJA='/usr/local/bin/ninja'
-PYTHON='/usr/local/bin/python3'
+PACKAGING_COMMAND='pkg'
PIP3='/usr/local/bin/pip-3.8'
-PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage ctags curl cyrus-sasl dbus diffutils gettext git glib gmake gnutls gsed gtk3 libepoxy libffi libgcrypt libjpeg-turbo libnfs libspice-server libssh libtasn1 libxml2 llvm lttng-ust lzo2 meson ncurses nettle ninja opencv p5-Test-Harness perl5 pixman pkgconf png py38-numpy py38-pillow py38-pip py38-sphinx py38-sphinx_rtd_theme py38-virtualenv py38-yaml python3 rpm2cpio sdl2 sdl2_image snappy spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
+PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage ctags curl cyrus-sasl dbus diffutils dtc gettext git glib gmake gnutls gsed gtk3 libepoxy libffi libgcrypt libjpeg-turbo libnfs libspice-server libssh libtasn1 libxml2 llvm lttng-ust lzo2 meson ncurses nettle ninja opencv p5-Test-Harness perl5 pixman pkgconf png py38-numpy py38-pillow py38-pip py38-sphinx py38-sphinx_rtd_theme py38-virtualenv py38-yaml python3 rpm2cpio sdl2 sdl2_image snappy spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
+PYPI_PKGS=''
+PYTHON='/usr/local/bin/python3'
diff --git a/.gitlab-ci.d/cirrus/macos-11.vars b/.gitlab-ci.d/cirrus/macos-11.vars
index cbec8a44a3..613d1373c2 100644
--- a/.gitlab-ci.d/cirrus/macos-11.vars
+++ b/.gitlab-ci.d/cirrus/macos-11.vars
@@ -2,14 +2,15 @@
#
# $ lcitool variables macos-11 qemu
#
-# https://gitlab.com/libvirt/libvirt-ci/-/commit/c7e275ab27ac0dcd09da290817b9adeea1fd1eb1
+# https://gitlab.com/libvirt/libvirt-ci
-PACKAGING_COMMAND='brew'
CCACHE='/usr/local/bin/ccache'
+CPAN_PKGS='Test::Harness'
+CROSS_PKGS=''
MAKE='/usr/local/bin/gmake'
NINJA='/usr/local/bin/ninja'
-PYTHON='/usr/local/bin/python3'
+PACKAGING_COMMAND='brew'
PIP3='/usr/local/bin/pip3'
-PKGS='bash bc bzip2 capstone ccache cpanminus ctags curl dbus diffutils gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb libxml2 llvm lzo make meson ncurses nettle ninja perl pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy sparse spice-protocol tesseract texinfo usbredir vde vte3 zlib zstd'
+PKGS='bash bc bzip2 capstone ccache cpanminus ctags curl dbus diffutils dtc gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb libxml2 llvm lzo make meson ncurses nettle ninja perl pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy sparse spice-protocol tesseract texinfo usbredir vde vte3 zlib zstd'
PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme virtualenv'
-CPAN_PKGS='Test::Harness'
+PYTHON='/usr/local/bin/python3'
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index b8a69cee59..b8cf0a7386 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -60,6 +60,11 @@ def generate_dockerfile(host, target, cross=None, trailer=None):
cmd.extend([target, "qemu"])
generate(filename, cmd, trailer)
+def generate_cirrus(target, trailer=None):
+ filename = Path(src_dir, ".gitlab-ci.d", "cirrus", target + ".vars")
+ cmd = [lcitool_path, "variables", target, "qemu"]
+ generate(filename, cmd, trailer)
+
ubuntu1804_skipssh = [
"# https://bugs.launchpad.net/qemu/+bug/1838763\n",
"ENV QEMU_CONFIGURE_OPTS --disable-libssh\n"
@@ -79,6 +84,11 @@ try:
generate_dockerfile("ubuntu2004", "ubuntu-2004",
trailer="".join(ubuntu2004_tsanhack))
generate_dockerfile("opensuse-leap", "opensuse-leap-152")
+
+ generate_cirrus("freebsd-12")
+ generate_cirrus("freebsd-13")
+ generate_cirrus("macos-11")
+
sys.exit(0)
except Exception as ex:
print(str(ex), file=sys.stderr)
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 15/18] tests/docker: updates to alpine package list
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (13 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 14/18] .gitlab-ci.d/cirrus: auto-generate variables with lcitool Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 16/18] tests/docker: fix sorting of alpine image package lists Daniel P. Berrangé
` (3 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
Cleanup the package lists by removing some entries that we don't need to
directly reference
binutils: implied by the compiler toolchain
coreutils: not required by QEMU build
mesa-egl mesa-gbm: implied by mesa-dev
ninja: alias for samurai package
shadow: not required by QEMU build
util-linux-dev: not directly required by QEMU build
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/alpine.docker | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index 7e6997e301..5a1808726e 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -8,9 +8,7 @@ RUN apk upgrade
ENV PACKAGES \
alsa-lib-dev \
bash \
- binutils \
ccache \
- coreutils \
curl-dev \
g++ \
gcc \
@@ -33,22 +31,18 @@ ENV PACKAGES \
lzo-dev \
make \
mesa-dev \
- mesa-egl \
- mesa-gbm \
meson \
ncurses-dev \
- ninja \
perl \
pulseaudio-dev \
python3 \
py3-sphinx \
py3-sphinx_rtd_theme \
- shadow \
+ samurai \
snappy-dev \
spice-dev \
texinfo \
usbredir-dev \
- util-linux-dev \
vde2-dev \
virglrenderer-dev \
vte3-dev \
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 16/18] tests/docker: fix sorting of alpine image package lists
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (14 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 15/18] tests/docker: updates to alpine package list Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 17/18] tests/docker: fully expand the alpine package list Daniel P. Berrangé
` (2 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
"python" sorts alphabetically after "py3-xxxx"
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/alpine.docker | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index 5a1808726e..ca4b3b58d2 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -35,9 +35,9 @@ ENV PACKAGES \
ncurses-dev \
perl \
pulseaudio-dev \
- python3 \
py3-sphinx \
py3-sphinx_rtd_theme \
+ python3 \
samurai \
snappy-dev \
spice-dev \
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 17/18] tests/docker: fully expand the alpine package list
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (15 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 16/18] tests/docker: fix sorting of alpine image package lists Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2021-12-15 14:19 ` [PATCH v5 18/18] tests/docker: auto-generate alpine.docker with lcitool Daniel P. Berrangé
2022-01-05 9:33 ` [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Alex Bennée
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
Add many extra alpine packages to cover the various optional QEMU build
dependencies pulled in by other dockerfiles.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/alpine.docker | 58 +++++++++++++++++++++++++-
1 file changed, 57 insertions(+), 1 deletion(-)
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index ca4b3b58d2..0ac30c8014 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -7,11 +7,29 @@ RUN apk upgrade
# Please keep this list sorted alphabetically
ENV PACKAGES \
alsa-lib-dev \
+ attr-dev \
bash \
+ bc \
+ bzip2 \
+ bzip2-dev \
+ ca-certificates \
+ capstone-dev \
ccache \
+ cdrkit \
+ ceph-dev \
+ clang \
+ ctags \
curl-dev \
+ cyrus-sasl-dev \
+ dbus \
+ diffutils \
+ dtc-dev \
+ eudev-dev \
+ findutils \
g++ \
gcc \
+ gcovr \
+ gettext \
git \
glib-dev \
glib-static \
@@ -20,34 +38,72 @@ ENV PACKAGES \
libaio-dev \
libbpf-dev \
libcap-ng-dev \
+ libdrm-dev \
+ libepoxy-dev \
libffi-dev \
+ libgcrypt-dev \
libjpeg-turbo-dev \
libnfs-dev \
libpng-dev \
libseccomp-dev \
+ libselinux-dev \
+ libslirp-dev \
libssh-dev \
+ libtasn1-dev \
+ liburing-dev \
libusb-dev \
libxml2-dev \
+ linux-pam-dev \
+ llvm11 \
+ lttng-ust-dev \
lzo-dev \
make \
mesa-dev \
meson \
+ multipath-tools \
ncurses-dev \
+ ndctl-dev \
+ net-tools \
+ nettle-dev \
+ nmap-ncat \
+ numactl-dev \
+ openssh-client \
+ pcre-dev \
perl \
+ perl-test-harness \
+ pixman-dev \
+ pkgconf \
pulseaudio-dev \
+ py3-numpy \
+ py3-pillow \
+ py3-pip \
py3-sphinx \
py3-sphinx_rtd_theme \
+ py3-virtualenv \
+ py3-yaml \
python3 \
+ rpm2cpio \
samurai \
+ sdl2-dev \
+ sdl2_image-dev \
+ sed \
snappy-dev \
+ sparse \
spice-dev \
+ spice-protocol \
+ tar \
+ tesseract-ocr \
texinfo \
usbredir-dev \
+ util-linux \
vde2-dev \
virglrenderer-dev \
vte3-dev \
+ which \
+ xen-dev \
xfsprogs-dev \
zlib-dev \
- zlib-static
+ zlib-static \
+ zstd-dev
RUN apk add $PACKAGES
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v5 18/18] tests/docker: auto-generate alpine.docker with lcitool
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (16 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 17/18] tests/docker: fully expand the alpine package list Daniel P. Berrangé
@ 2021-12-15 14:19 ` Daniel P. Berrangé
2022-01-05 9:33 ` [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Alex Bennée
18 siblings, 0 replies; 20+ messages in thread
From: Daniel P. Berrangé @ 2021-12-15 14:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Beraldo Leal, Alex Bennée, Philippe Mathieu-Daudé,
Wainer dos Santos Moschetta, Willian Rampazzo, Gerd Hoffmann,
Philippe Mathieu-Daudé
This commit is best examined using the "-b" option to diff.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/docker/dockerfiles/alpine.docker | 225 +++++++++++++------------
tests/lcitool/refresh | 1 +
2 files changed, 120 insertions(+), 106 deletions(-)
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index 0ac30c8014..97c7a88d1f 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -1,109 +1,122 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool dockerfile alpine-edge qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
-FROM alpine:edge
+FROM docker.io/library/alpine:edge
-RUN apk update
-RUN apk upgrade
+RUN apk update && \
+ apk upgrade && \
+ apk add \
+ alsa-lib-dev \
+ attr-dev \
+ bash \
+ bc \
+ bzip2 \
+ bzip2-dev \
+ ca-certificates \
+ capstone-dev \
+ ccache \
+ cdrkit \
+ ceph-dev \
+ clang \
+ ctags \
+ curl-dev \
+ cyrus-sasl-dev \
+ dbus \
+ diffutils \
+ dtc-dev \
+ eudev-dev \
+ findutils \
+ g++ \
+ gcc \
+ gcovr \
+ gettext \
+ git \
+ glib-dev \
+ glib-static \
+ gnutls-dev \
+ gtk+3.0-dev \
+ libaio-dev \
+ libbpf-dev \
+ libcap-ng-dev \
+ libdrm-dev \
+ libepoxy-dev \
+ libffi-dev \
+ libgcrypt-dev \
+ libjpeg-turbo-dev \
+ libnfs-dev \
+ libpng-dev \
+ libseccomp-dev \
+ libselinux-dev \
+ libslirp-dev \
+ libssh-dev \
+ libtasn1-dev \
+ liburing-dev \
+ libusb-dev \
+ libxml2-dev \
+ linux-pam-dev \
+ llvm11 \
+ lttng-ust-dev \
+ lzo-dev \
+ make \
+ mesa-dev \
+ meson \
+ multipath-tools \
+ ncurses-dev \
+ ndctl-dev \
+ net-tools \
+ nettle-dev \
+ nmap-ncat \
+ numactl-dev \
+ openssh-client \
+ pcre-dev \
+ perl \
+ perl-test-harness \
+ pixman-dev \
+ pkgconf \
+ pulseaudio-dev \
+ py3-numpy \
+ py3-pillow \
+ py3-pip \
+ py3-sphinx \
+ py3-sphinx_rtd_theme \
+ py3-virtualenv \
+ py3-yaml \
+ python3 \
+ rpm2cpio \
+ samurai \
+ sdl2-dev \
+ sdl2_image-dev \
+ sed \
+ snappy-dev \
+ sparse \
+ spice-dev \
+ spice-protocol \
+ tar \
+ tesseract-ocr \
+ texinfo \
+ usbredir-dev \
+ util-linux \
+ vde2-dev \
+ virglrenderer-dev \
+ vte3-dev \
+ which \
+ xen-dev \
+ xfsprogs-dev \
+ zlib-dev \
+ zlib-static \
+ zstd-dev && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
-# Please keep this list sorted alphabetically
-ENV PACKAGES \
- alsa-lib-dev \
- attr-dev \
- bash \
- bc \
- bzip2 \
- bzip2-dev \
- ca-certificates \
- capstone-dev \
- ccache \
- cdrkit \
- ceph-dev \
- clang \
- ctags \
- curl-dev \
- cyrus-sasl-dev \
- dbus \
- diffutils \
- dtc-dev \
- eudev-dev \
- findutils \
- g++ \
- gcc \
- gcovr \
- gettext \
- git \
- glib-dev \
- glib-static \
- gnutls-dev \
- gtk+3.0-dev \
- libaio-dev \
- libbpf-dev \
- libcap-ng-dev \
- libdrm-dev \
- libepoxy-dev \
- libffi-dev \
- libgcrypt-dev \
- libjpeg-turbo-dev \
- libnfs-dev \
- libpng-dev \
- libseccomp-dev \
- libselinux-dev \
- libslirp-dev \
- libssh-dev \
- libtasn1-dev \
- liburing-dev \
- libusb-dev \
- libxml2-dev \
- linux-pam-dev \
- llvm11 \
- lttng-ust-dev \
- lzo-dev \
- make \
- mesa-dev \
- meson \
- multipath-tools \
- ncurses-dev \
- ndctl-dev \
- net-tools \
- nettle-dev \
- nmap-ncat \
- numactl-dev \
- openssh-client \
- pcre-dev \
- perl \
- perl-test-harness \
- pixman-dev \
- pkgconf \
- pulseaudio-dev \
- py3-numpy \
- py3-pillow \
- py3-pip \
- py3-sphinx \
- py3-sphinx_rtd_theme \
- py3-virtualenv \
- py3-yaml \
- python3 \
- rpm2cpio \
- samurai \
- sdl2-dev \
- sdl2_image-dev \
- sed \
- snappy-dev \
- sparse \
- spice-dev \
- spice-protocol \
- tar \
- tesseract-ocr \
- texinfo \
- usbredir-dev \
- util-linux \
- vde2-dev \
- virglrenderer-dev \
- vte3-dev \
- which \
- xen-dev \
- xfsprogs-dev \
- zlib-dev \
- zlib-static \
- zstd-dev
-
-RUN apk add $PACKAGES
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index b8cf0a7386..033120e223 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -84,6 +84,7 @@ try:
generate_dockerfile("ubuntu2004", "ubuntu-2004",
trailer="".join(ubuntu2004_tsanhack))
generate_dockerfile("opensuse-leap", "opensuse-leap-152")
+ generate_dockerfile("alpine", "alpine-edge")
generate_cirrus("freebsd-12")
generate_cirrus("freebsd-13")
--
2.33.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles
2021-12-15 14:19 [PATCH v5 00/18] tests/docker: start using libvirt-ci's "lcitool" for dockerfiles Daniel P. Berrangé
` (17 preceding siblings ...)
2021-12-15 14:19 ` [PATCH v5 18/18] tests/docker: auto-generate alpine.docker with lcitool Daniel P. Berrangé
@ 2022-01-05 9:33 ` Alex Bennée
18 siblings, 0 replies; 20+ messages in thread
From: Alex Bennée @ 2022-01-05 9:33 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Fam Zheng, Peter Maydell, Thomas Huth, Beraldo Leal, qemu-devel,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Gerd Hoffmann, Philippe Mathieu-Daudé
Daniel P. Berrangé <berrange@redhat.com> writes:
> Currently the tests/docker/dockerfiles/*Dockerfile recipes are all hand
> written by contributors. There is a common design pattern, but the set
> of packages listed for installation leaves alot to be desired
Queued to testing/next, thanks.
--
Alex Bennée
^ permalink raw reply [flat|nested] 20+ messages in thread